2024-04-24T23:13:46.194 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T23:13:46.195 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --verbose --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/multimds_misc} --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190 -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/orig.config.yaml 2024-04-24T23:13:46.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:13:46.348 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/multimds_misc} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672190' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Scrub error on inode sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDx0szPYw7CaWRErs8DjFuW2U76iopXRUDhkqyZZ7sYbcyS3wmGy2qfHQDxmOSx0Mm9fjrjtsbVV6ZUjFMw84tc= smithi029.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIV/eIkpXapl3uvodBPAHmFEw4QeknvGd8am2syjhcSxhSYh2ns0Vuh56XaaDVVXjFaFeFy0PMGWhGlFnExSSsQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_multimds_misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T23:13:46.348 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T23:13:46.349 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T23:13:46.349 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T23:13:46.350 INFO:teuthology.task.internal:Checking packages... 2024-04-24T23:13:46.375 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T23:13:46.375 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:13:46.375 INFO:teuthology.packaging:ref: None 2024-04-24T23:13:46.375 INFO:teuthology.packaging:tag: None 2024-04-24T23:13:46.375 INFO:teuthology.packaging:branch: squid 2024-04-24T23:13:46.375 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:13:46.376 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T23:13:46.688 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T23:13:46.690 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T23:13:46.697 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T23:13:46.697 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T23:13:46.752 INFO:teuthology.task.internal:Saving configuration 2024-04-24T23:13:46.769 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T23:13:46.775 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:13:46.808 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:08:31.231109', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDx0szPYw7CaWRErs8DjFuW2U76iopXRUDhkqyZZ7sYbcyS3wmGy2qfHQDxmOSx0Mm9fjrjtsbVV6ZUjFMw84tc='} 2024-04-24T23:13:46.842 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:08:31.229510', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIV/eIkpXapl3uvodBPAHmFEw4QeknvGd8am2syjhcSxhSYh2ns0Vuh56XaaDVVXjFaFeFy0PMGWhGlFnExSSsQ='} 2024-04-24T23:13:46.842 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T23:13:46.849 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-24T23:13:46.849 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-24T23:13:46.849 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T23:13:46.936 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f16478770d0>, signals=[15]) 2024-04-24T23:13:46.937 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T23:13:46.944 INFO:teuthology.task.internal:Opening connections... 2024-04-24T23:13:46.945 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:13:46.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:13:47.032 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:13:47.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:13:47.122 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T23:13:47.128 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-04-24T23:13:47.147 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-04-24T23:13:47.147 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-04-24T23:13:47.206 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-04-24T23:13:47.206 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-04-24T23:13:47.206 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-04-24T23:13:47.206 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:13:47.206 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-04-24T23:13:47.206 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:13:47.206 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:13:47.207 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-04-24T23:13:47.207 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:13:47.207 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:13:47.207 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:13:47.207 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:13:47.207 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:13:47.207 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:13:47.207 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-04-24T23:13:47.245 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2024-04-24T23:13:47.264 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2024-04-24T23:13:47.264 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2024-04-24T23:13:47.321 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="9" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="9" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:13:47.322 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:13:47.323 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2024-04-24T23:13:47.353 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T23:13:47.364 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T23:13:47.371 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T23:13:47.371 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:13:47.374 DEBUG:teuthology.orchestra.run.smithi029:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:13:47.391 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T23:13:47.398 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T23:13:47.398 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:13:47.431 DEBUG:teuthology.orchestra.run.smithi029:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:13:47.457 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T23:13:47.528 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T23:13:47.543 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T23:13:47.543 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-24T23:13:47.543 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-24T23:13:47.543 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:13:47.544 DEBUG:teuthology.orchestra.run.smithi029:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:13:47.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:13:47.560 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-04-24T23:13:47.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:13:47.561 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-04-24T23:13:47.618 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:13:47.618 INFO:teuthology.task.kernel:Running kernel on smithi026: 5.14.0-437.el9.x86_64 2024-04-24T23:13:47.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum install -y kernel 2024-04-24T23:13:47.619 INFO:teuthology.orchestra.run.smithi029.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:13:47.620 INFO:teuthology.task.kernel:Running kernel on smithi029: 5.14.0-437.el9.x86_64 2024-04-24T23:13:47.620 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum install -y kernel 2024-04-24T23:13:50.843 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-24T23:13:52.530 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.0 MB 00:03 2024-04-24T23:13:53.239 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-24T23:13:58.033 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-04-24T23:13:59.492 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-24T23:14:00.605 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-24T23:14:05.923 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-24T23:14:07.880 INFO:teuthology.orchestra.run.smithi029.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:14:07.880 INFO:teuthology.orchestra.run.smithi029.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:14:07.905 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-04-24T23:14:08.001 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-04-24T23:14:08.002 INFO:teuthology.orchestra.run.smithi029.stdout:Nothing to do. 2024-04-24T23:14:08.002 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-04-24T23:14:08.172 DEBUG:teuthology.orchestra.run.smithi029:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:14:08.696 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:14:05 PM UTC. 2024-04-24T23:14:08.835 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-04-24T23:14:08.861 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-04-24T23:14:08.861 INFO:teuthology.orchestra.run.smithi029.stdout: Package Architecture Version Repository Size 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stdout:Reinstalling: 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 255 k 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 0 2024-04-24T23:14:08.862 INFO:teuthology.orchestra.run.smithi029.stderr:Operation aborted. 2024-04-24T23:14:08.923 INFO:teuthology.orchestra.run.smithi029.stdout:Is this ok [y/N]: 2024-04-24T23:14:08.924 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum reinstall -y kernel || true 2024-04-24T23:14:09.440 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:00:04 ago on Wed 24 Apr 2024 11:14:05 PM UTC. 2024-04-24T23:14:09.561 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-04-24T23:14:09.562 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-04-24T23:14:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: Package Architecture Version Repository Size 2024-04-24T23:14:09.562 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-04-24T23:14:09.562 INFO:teuthology.orchestra.run.smithi029.stdout:Reinstalling: 2024-04-24T23:14:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:14:09.562 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:14:09.563 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-04-24T23:14:09.563 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2024-04-24T23:14:09.563 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:14:09.563 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 255 k 2024-04-24T23:14:09.563 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 0 2024-04-24T23:14:09.563 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-04-24T23:14:09.852 INFO:teuthology.orchestra.run.smithi029.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-24T23:14:09.853 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:14:09.853 INFO:teuthology.orchestra.run.smithi029.stdout:Total 880 kB/s | 255 kB 00:00 2024-04-24T23:14:09.871 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-04-24T23:14:10.074 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-04-24T23:14:10.074 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-04-24T23:14:10.079 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-04-24T23:14:10.080 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-04-24T23:14:10.253 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-04-24T23:14:10.362 INFO:teuthology.orchestra.run.smithi029.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:14:10.363 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:14:11.530 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:14:11.530 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:14:12.389 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:14:12.389 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:14:12.389 INFO:teuthology.orchestra.run.smithi029.stdout:Reinstalled: 2024-04-24T23:14:12.389 INFO:teuthology.orchestra.run.smithi029.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:14:12.390 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:14:12.390 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-04-24T23:14:12.623 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:14:12.672 INFO:teuthology.orchestra.run.smithi029.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:14:12.672 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:14:12.672 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:14:12.672 DEBUG:teuthology.parallel:result is None 2024-04-24T23:14:12.805 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.2 MB 00:01 2024-04-24T23:14:14.539 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-24T23:14:16.090 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-24T23:14:21.374 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-24T23:14:23.330 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:14:23.330 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:14:23.420 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-24T23:14:23.421 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-04-24T23:14:23.422 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-04-24T23:14:23.522 DEBUG:teuthology.orchestra.run.smithi026:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:14:24.042 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:14:21 PM UTC. 2024-04-24T23:14:24.165 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-24T23:14:24.177 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-24T23:14:24.177 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-04-24T23:14:24.177 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-24T23:14:24.177 INFO:teuthology.orchestra.run.smithi026.stdout:Reinstalling: 2024-04-24T23:14:24.177 INFO:teuthology.orchestra.run.smithi026.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:14:24.178 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:14:24.178 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-04-24T23:14:24.178 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-24T23:14:24.178 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:14:24.178 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 255 k 2024-04-24T23:14:24.178 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 0 2024-04-24T23:14:24.178 INFO:teuthology.orchestra.run.smithi026.stderr:Operation aborted. 2024-04-24T23:14:24.233 INFO:teuthology.orchestra.run.smithi026.stdout:Is this ok [y/N]: 2024-04-24T23:14:24.234 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum reinstall -y kernel || true 2024-04-24T23:14:24.746 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:14:21 PM UTC. 2024-04-24T23:14:24.867 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-24T23:14:24.868 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-24T23:14:24.868 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-04-24T23:14:24.868 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-24T23:14:24.868 INFO:teuthology.orchestra.run.smithi026.stdout:Reinstalling: 2024-04-24T23:14:24.869 INFO:teuthology.orchestra.run.smithi026.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:14:24.869 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:14:24.869 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-04-24T23:14:24.869 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-24T23:14:24.869 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:14:24.869 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 255 k 2024-04-24T23:14:24.869 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 0 2024-04-24T23:14:24.869 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-04-24T23:14:25.392 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.0 MB/s | 255 kB 00:00 2024-04-24T23:14:25.392 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:14:25.392 INFO:teuthology.orchestra.run.smithi026.stdout:Total 487 kB/s | 255 kB 00:00 2024-04-24T23:14:25.412 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-04-24T23:14:25.611 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-04-24T23:14:25.611 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-04-24T23:14:25.616 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-04-24T23:14:25.616 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-04-24T23:14:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-04-24T23:14:25.835 INFO:teuthology.orchestra.run.smithi026.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:14:25.835 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:14:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:14:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:14:27.236 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:14:27.237 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:14:27.237 INFO:teuthology.orchestra.run.smithi026.stdout:Reinstalled: 2024-04-24T23:14:27.237 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:14:27.237 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:14:27.237 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-04-24T23:14:27.433 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:14:27.481 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:14:27.481 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:14:27.482 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:14:27.482 DEBUG:teuthology.parallel:result is None 2024-04-24T23:14:27.482 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T23:14:27.490 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T23:14:27.491 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:14:27.493 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:14:27.511 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T23:14:27.518 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T23:14:27.528 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T23:14:27.529 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:14:27.548 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:14:27.585 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T23:14:27.591 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T23:14:27.592 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:14:27.609 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:14:27.653 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:14:27.662 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:14:27.669 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:14:27.682 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:14:27.684 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T23:14:27.691 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T23:14:27.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:14:27.694 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:14:27.738 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T23:14:27.751 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T23:14:27.752 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:14:27.758 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:14:27.794 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:14:27.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:14:27.900 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:14:27.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:14:28.021 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:14:28.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:14:28.085 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:14:28.139 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:14:28.181 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:14:28.273 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:14:28.310 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:14:28.310 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:14:28.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-04-24T23:14:28.380 DEBUG:teuthology.orchestra.run.smithi029:> sudo service rsyslog restart 2024-04-24T23:14:28.413 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:14:28.460 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:14:28.895 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T23:14:28.904 INFO:teuthology.task.internal:Starting timer... 2024-04-24T23:14:28.904 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T23:14:28.914 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T23:14:28.923 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-24T23:14:28.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-04-24T23:14:28.998 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-04-24T23:14:29.000 DEBUG:teuthology.orchestra.run.smithi029:> sudo service auditd rotate 2024-04-24T23:14:29.086 INFO:teuthology.orchestra.run.smithi029.stdout:Rotating logs: 2024-04-24T23:14:29.088 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T23:14:29.088 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-04-24T23:14:29.110 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-04-24T23:14:29.110 DEBUG:teuthology.orchestra.run.smithi029:> /usr/sbin/getenforce 2024-04-24T23:14:29.134 INFO:teuthology.orchestra.run.smithi029.stdout:Permissive 2024-04-24T23:14:29.134 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi029.front.sepia.ceph.com': 'permissive'} 2024-04-24T23:14:29.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:14:29.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:14:29.175 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:14:29.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:14:29.202 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T23:14:29.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:14:29.240 DEBUG:teuthology.orchestra.run.smithi029:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:14:29.271 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T23:14:29.282 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T23:14:29.288 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T23:14:29.441 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T23:14:29.456 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T23:14:29.458 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi029.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T23:18:04.046 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi029.front.sepia.ceph.com')] 2024-04-24T23:18:04.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-04-24T23:18:04.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:18:04.144 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-04-24T23:18:04.222 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-04-24T23:18:04.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-04-24T23:18:04.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:18:04.310 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-04-24T23:18:04.391 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-04-24T23:18:04.391 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T23:18:04.402 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T23:18:04.402 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:18:04.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:18:04.405 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:18:04.405 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:18:04.433 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:18:04.447 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:18:04.462 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:18:04.471 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-04-24T23:18:04.477 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:18:04.482 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-04-24T23:18:04.496 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:18:04.502 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: ntpd: command not found 2024-04-24T23:18:04.509 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:18:04.513 INFO:teuthology.orchestra.run.smithi029.stdout:506 Cannot talk to daemon 2024-04-24T23:18:04.528 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:18:04.542 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:18:04.564 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:18:04.567 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:18:04.567 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-04-24T23:18:04.567 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:18:04.567 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:18:04.567 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:18:04.567 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:18:04.597 INFO:teuthology.orchestra.run.smithi029.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:18:04.600 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:18:04.600 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-04-24T23:18:04.600 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:18:04.600 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:18:04.600 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:18:04.600 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:18:04.601 INFO:teuthology.run_tasks:Running task install... 2024-04-24T23:18:04.610 DEBUG:teuthology.task.install:project ceph 2024-04-24T23:18:04.610 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T23:18:04.610 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T23:18:04.610 INFO:teuthology.task.install:Using flavor: default 2024-04-24T23:18:04.616 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T23:18:04.616 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T23:18:04.617 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:18:04.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:18:04.619 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:18:04.619 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:18:04.865 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:18:04.866 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:18:04.880 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:18:04.880 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:18:05.048 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:18:05.048 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:18:05.048 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:18:05.057 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:18:05.057 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:18:05.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:18:05.075 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:18:05.075 DEBUG:teuthology.orchestra.run.smithi029:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:18:05.084 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:18:05.084 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:18:05.143 DEBUG:teuthology.orchestra.run.smithi029:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:18:05.149 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:18:05.219 DEBUG:teuthology.orchestra.run.smithi029:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:18:05.224 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:18:05.284 INFO:teuthology.orchestra.run.smithi029.stdout:check_obsoletes = 1 2024-04-24T23:18:05.286 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum clean all 2024-04-24T23:18:05.288 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-04-24T23:18:05.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-04-24T23:18:05.521 INFO:teuthology.orchestra.run.smithi029.stdout:48 files removed 2024-04-24T23:18:05.526 INFO:teuthology.orchestra.run.smithi026.stdout:48 files removed 2024-04-24T23:18:05.549 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:18:05.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:18:06.240 INFO:teuthology.orchestra.run.smithi029.stdout:ceph packages for x86_64 199 kB/s | 83 kB 00:00 2024-04-24T23:18:06.291 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 179 kB/s | 83 kB 00:00 2024-04-24T23:18:06.666 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-24T23:18:06.675 INFO:teuthology.orchestra.run.smithi029.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-04-24T23:18:07.035 INFO:teuthology.orchestra.run.smithi029.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-04-24T23:18:07.100 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-24T23:18:07.725 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-24T23:18:08.778 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-04-24T23:18:09.826 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-24T23:18:12.188 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 19 MB 00:02 2024-04-24T23:18:13.919 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-24T23:18:15.529 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-24T23:18:16.509 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.2 MB 00:00 2024-04-24T23:18:16.946 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-24T23:18:18.019 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-04-24T23:18:19.550 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-24T23:18:22.271 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T23:18:24.456 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:18:24.457 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:18:24.467 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:18:24.468 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:18:24.469 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:18:24.470 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:18:24.472 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:18:24.474 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:18:24.475 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:18:24.540 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-24T23:18:24.549 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-04-24T23:18:24.549 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-04-24T23:18:24.549 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-04-24T23:18:24.549 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:18:24.550 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:18:24.551 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:18:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:18:24.553 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:18:24.554 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:18:24.555 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:18:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:18:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:18:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:18:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-04-24T23:18:24.560 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-04-24T23:18:24.561 INFO:teuthology.orchestra.run.smithi026.stdout:Install 148 Packages 2024-04-24T23:18:24.561 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 4 Packages 2024-04-24T23:18:24.561 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:18:24.561 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 220 M 2024-04-24T23:18:24.561 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-04-24T23:18:24.806 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T23:18:25.356 INFO:teuthology.orchestra.run.smithi026.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-24T23:18:25.673 INFO:teuthology.orchestra.run.smithi026.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 2.7 MB/s | 890 kB 00:00 2024-04-24T23:18:25.748 INFO:teuthology.orchestra.run.smithi026.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.9 MB/s | 145 kB 00:00 2024-04-24T23:18:25.857 INFO:teuthology.orchestra.run.smithi026.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 7.2 MB/s | 5.4 MB 00:00 2024-04-24T23:18:25.998 INFO:teuthology.orchestra.run.smithi026.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-24T23:18:26.257 INFO:teuthology.orchestra.run.smithi026.stdout:(6/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 4.6 MB/s | 2.4 MB 00:00 2024-04-24T23:18:26.365 INFO:teuthology.orchestra.run.smithi026.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-24T23:18:27.028 INFO:teuthology.orchestra.run.smithi029.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:18:27.028 INFO:teuthology.orchestra.run.smithi029.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:18:27.040 INFO:teuthology.orchestra.run.smithi029.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:18:27.041 INFO:teuthology.orchestra.run.smithi029.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:18:27.042 INFO:teuthology.orchestra.run.smithi029.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:18:27.043 INFO:teuthology.orchestra.run.smithi029.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:18:27.045 INFO:teuthology.orchestra.run.smithi029.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:18:27.047 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:18:27.047 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:18:27.113 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-04-24T23:18:27.122 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-04-24T23:18:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-04-24T23:18:27.122 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-04-24T23:18:27.122 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-04-24T23:18:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:18:27.122 INFO:teuthology.orchestra.run.smithi029.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:18:27.123 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:18:27.124 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:18:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:18:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:18:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:18:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:18:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:18:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:18:27.125 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrading: 2024-04-24T23:18:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:18:27.125 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:18:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:18:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:18:27.126 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-04-24T23:18:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:18:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:18:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:18:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:18:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:18:27.126 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:18:27.127 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:18:27.128 INFO:teuthology.orchestra.run.smithi029.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:18:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:18:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:18:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:18:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:18:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:18:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:18:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:18:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:18:27.129 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:18:27.130 INFO:teuthology.orchestra.run.smithi029.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:18:27.131 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:18:27.132 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:18:27.133 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:18:27.134 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:18:27.135 INFO:teuthology.orchestra.run.smithi029.stdout:Installing weak dependencies: 2024-04-24T23:18:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:18:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:18:27.136 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-04-24T23:18:27.136 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-04-24T23:18:27.136 INFO:teuthology.orchestra.run.smithi029.stdout:Install 148 Packages 2024-04-24T23:18:27.136 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrade 4 Packages 2024-04-24T23:18:27.136 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:18:27.137 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 220 M 2024-04-24T23:18:27.137 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-04-24T23:18:27.141 INFO:teuthology.orchestra.run.smithi026.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 13 MB/s | 10 MB 00:00 2024-04-24T23:18:27.216 INFO:teuthology.orchestra.run.smithi026.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 337 kB/s | 25 kB 00:00 2024-04-24T23:18:28.224 INFO:teuthology.orchestra.run.smithi026.stdout:(10/152): ceph-common-19.0.0-2445.gd2c45d97.el9 6.7 MB/s | 21 MB 00:03 2024-04-24T23:18:28.294 INFO:teuthology.orchestra.run.smithi029.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-24T23:18:28.308 INFO:teuthology.orchestra.run.smithi026.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.6 MB/s | 222 kB 00:00 2024-04-24T23:18:28.384 INFO:teuthology.orchestra.run.smithi026.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 360 kB/s | 27 kB 00:00 2024-04-24T23:18:28.467 INFO:teuthology.orchestra.run.smithi026.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 382 kB/s | 32 kB 00:00 2024-04-24T23:18:28.651 INFO:teuthology.orchestra.run.smithi026.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 4.0 MB/s | 745 kB 00:00 2024-04-24T23:18:28.751 INFO:teuthology.orchestra.run.smithi026.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 163 kB 00:00 2024-04-24T23:18:28.960 INFO:teuthology.orchestra.run.smithi026.stdout:(16/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 6.2 MB/s | 17 MB 00:02 2024-04-24T23:18:28.976 INFO:teuthology.orchestra.run.smithi026.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 563 kB/s | 127 kB 00:00 2024-04-24T23:18:29.035 INFO:teuthology.orchestra.run.smithi026.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 6.5 MB/s | 501 kB 00:00 2024-04-24T23:18:29.103 INFO:teuthology.orchestra.run.smithi029.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 890 kB 00:00 2024-04-24T23:18:29.144 INFO:teuthology.orchestra.run.smithi026.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 417 kB/s | 45 kB 00:00 2024-04-24T23:18:29.260 INFO:teuthology.orchestra.run.smithi026.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.1 MB/s | 133 kB 00:00 2024-04-24T23:18:29.369 INFO:teuthology.orchestra.run.smithi026.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.5 MB/s | 162 kB 00:00 2024-04-24T23:18:29.378 INFO:teuthology.orchestra.run.smithi029.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 526 kB/s | 145 kB 00:00 2024-04-24T23:18:29.478 INFO:teuthology.orchestra.run.smithi026.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.9 MB/s | 321 kB 00:00 2024-04-24T23:18:29.595 INFO:teuthology.orchestra.run.smithi026.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.5 MB/s | 299 kB 00:00 2024-04-24T23:18:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:(24/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 7.7 MB/s | 5.3 MB 00:00 2024-04-24T23:18:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 989 kB/s | 99 kB 00:00 2024-04-24T23:18:29.746 INFO:teuthology.orchestra.run.smithi026.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-24T23:18:29.838 INFO:teuthology.orchestra.run.smithi026.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-24T23:18:29.930 INFO:teuthology.orchestra.run.smithi026.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 347 kB/s | 32 kB 00:00 2024-04-24T23:18:30.030 INFO:teuthology.orchestra.run.smithi026.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.4 MB/s | 141 kB 00:00 2024-04-24T23:18:30.429 INFO:teuthology.orchestra.run.smithi029.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-04-24T23:18:30.696 INFO:teuthology.orchestra.run.smithi029.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-24T23:18:30.772 INFO:teuthology.orchestra.run.smithi029.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-04-24T23:18:30.839 INFO:teuthology.orchestra.run.smithi026.stdout:(30/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 13 MB/s | 48 MB 00:03 2024-04-24T23:18:30.889 INFO:teuthology.orchestra.run.smithi026.stdout:(31/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 2.6 MB/s | 3.1 MB 00:01 2024-04-24T23:18:30.973 INFO:teuthology.orchestra.run.smithi026.stdout:(32/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.7 MB/s | 3.5 MB 00:00 2024-04-24T23:18:31.081 INFO:teuthology.orchestra.run.smithi026.stdout:(33/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.3 MB/s | 249 kB 00:00 2024-04-24T23:18:31.106 INFO:teuthology.orchestra.run.smithi026.stdout:(34/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 370 kB/s | 49 kB 00:00 2024-04-24T23:18:31.223 INFO:teuthology.orchestra.run.smithi026.stdout:(35/152): ceph-prometheus-alerts-19.0.0-2445.gd 108 kB/s | 15 kB 00:00 2024-04-24T23:18:31.265 INFO:teuthology.orchestra.run.smithi026.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 1.6 MB/s | 264 kB 00:00 2024-04-24T23:18:31.357 INFO:teuthology.orchestra.run.smithi026.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 199 kB/s | 18 kB 00:00 2024-04-24T23:18:31.389 INFO:teuthology.orchestra.run.smithi029.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 6.8 MB/s | 4.7 MB 00:00 2024-04-24T23:18:31.391 INFO:teuthology.orchestra.run.smithi026.stdout:(38/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.5 MB/s | 762 kB 00:00 2024-04-24T23:18:31.574 INFO:teuthology.orchestra.run.smithi026.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-24T23:18:31.791 INFO:teuthology.orchestra.run.smithi026.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 185 kB/s | 80 kB 00:00 2024-04-24T23:18:32.133 INFO:teuthology.orchestra.run.smithi026.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-24T23:18:32.158 INFO:teuthology.orchestra.run.smithi026.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 690 kB/s | 253 kB 00:00 2024-04-24T23:18:32.217 INFO:teuthology.orchestra.run.smithi026.stdout:(43/152): libquadmath-11.4.1-3.el9.x86_64.rpm 304 kB/s | 195 kB 00:00 2024-04-24T23:18:32.267 INFO:teuthology.orchestra.run.smithi026.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-04-24T23:18:32.309 INFO:teuthology.orchestra.run.smithi026.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-24T23:18:32.343 INFO:teuthology.orchestra.run.smithi026.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-04-24T23:18:32.377 INFO:teuthology.orchestra.run.smithi026.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-24T23:18:32.397 INFO:teuthology.orchestra.run.smithi029.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 10 MB/s | 10 MB 00:01 2024-04-24T23:18:32.418 INFO:teuthology.orchestra.run.smithi026.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-24T23:18:32.452 INFO:teuthology.orchestra.run.smithi026.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-04-24T23:18:32.519 INFO:teuthology.orchestra.run.smithi026.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-04-24T23:18:32.552 INFO:teuthology.orchestra.run.smithi026.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-04-24T23:18:32.661 INFO:teuthology.orchestra.run.smithi026.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-04-24T23:18:32.695 INFO:teuthology.orchestra.run.smithi026.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-04-24T23:18:32.698 INFO:teuthology.orchestra.run.smithi029.stdout:(9/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 8.7 MB/s | 17 MB 00:01 2024-04-24T23:18:32.723 INFO:teuthology.orchestra.run.smithi029.stdout:(10/152): ceph-selinux-19.0.0-2445.gd2c45d97.el 77 kB/s | 25 kB 00:00 2024-04-24T23:18:32.792 INFO:teuthology.orchestra.run.smithi029.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 3.2 MB/s | 222 kB 00:00 2024-04-24T23:18:32.803 INFO:teuthology.orchestra.run.smithi026.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-04-24T23:18:32.828 INFO:teuthology.orchestra.run.smithi026.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 18 kB 00:00 2024-04-24T23:18:32.868 INFO:teuthology.orchestra.run.smithi029.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 360 kB/s | 27 kB 00:00 2024-04-24T23:18:32.904 INFO:teuthology.orchestra.run.smithi026.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-04-24T23:18:32.937 INFO:teuthology.orchestra.run.smithi026.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-24T23:18:32.943 INFO:teuthology.orchestra.run.smithi029.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 422 kB/s | 32 kB 00:00 2024-04-24T23:18:33.004 INFO:teuthology.orchestra.run.smithi026.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-24T23:18:33.046 INFO:teuthology.orchestra.run.smithi026.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-24T23:18:33.110 INFO:teuthology.orchestra.run.smithi029.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 4.4 MB/s | 745 kB 00:00 2024-04-24T23:18:33.147 INFO:teuthology.orchestra.run.smithi026.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-04-24T23:18:33.203 INFO:teuthology.orchestra.run.smithi029.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.7 MB/s | 163 kB 00:00 2024-04-24T23:18:33.247 INFO:teuthology.orchestra.run.smithi026.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 475 kB/s | 47 kB 00:00 2024-04-24T23:18:33.286 INFO:teuthology.orchestra.run.smithi029.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.5 MB/s | 127 kB 00:00 2024-04-24T23:18:33.355 INFO:teuthology.orchestra.run.smithi026.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-24T23:18:33.395 INFO:teuthology.orchestra.run.smithi029.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 4.5 MB/s | 501 kB 00:00 2024-04-24T23:18:33.431 INFO:teuthology.orchestra.run.smithi026.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 1.3 MB/s | 582 kB 00:00 2024-04-24T23:18:33.464 INFO:teuthology.orchestra.run.smithi026.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-24T23:18:33.531 INFO:teuthology.orchestra.run.smithi026.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-04-24T23:18:34.079 INFO:teuthology.orchestra.run.smithi029.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 7.8 MB/s | 5.3 MB 00:00 2024-04-24T23:18:34.155 INFO:teuthology.orchestra.run.smithi029.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 600 kB/s | 45 kB 00:00 2024-04-24T23:18:34.239 INFO:teuthology.orchestra.run.smithi029.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.6 MB/s | 133 kB 00:00 2024-04-24T23:18:34.331 INFO:teuthology.orchestra.run.smithi029.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.7 MB/s | 162 kB 00:00 2024-04-24T23:18:34.423 INFO:teuthology.orchestra.run.smithi029.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.4 MB/s | 321 kB 00:00 2024-04-24T23:18:34.524 INFO:teuthology.orchestra.run.smithi029.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.9 MB/s | 299 kB 00:00 2024-04-24T23:18:34.616 INFO:teuthology.orchestra.run.smithi029.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.1 MB/s | 99 kB 00:00 2024-04-24T23:18:34.725 INFO:teuthology.orchestra.run.smithi029.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 785 kB/s | 85 kB 00:00 2024-04-24T23:18:34.739 INFO:teuthology.orchestra.run.smithi026.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.4 MB/s | 3.0 MB 00:02 2024-04-24T23:18:35.836 INFO:teuthology.orchestra.run.smithi029.stdout:(26/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 15 MB/s | 48 MB 00:03 2024-04-24T23:18:35.920 INFO:teuthology.orchestra.run.smithi029.stdout:(27/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 2.6 MB/s | 3.1 MB 00:01 2024-04-24T23:18:35.923 INFO:teuthology.orchestra.run.smithi026.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 4.4 MB 00:02 2024-04-24T23:18:35.945 INFO:teuthology.orchestra.run.smithi029.stdout:(28/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.5 MB/s | 171 kB 00:00 2024-04-24T23:18:35.979 INFO:teuthology.orchestra.run.smithi029.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 539 kB/s | 32 kB 00:00 2024-04-24T23:18:36.013 INFO:teuthology.orchestra.run.smithi029.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.0 MB/s | 141 kB 00:00 2024-04-24T23:18:36.365 INFO:teuthology.orchestra.run.smithi026.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.6 MB/s | 4.7 MB 00:02 2024-04-24T23:18:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 1.9 MB/s | 911 kB 00:00 2024-04-24T23:18:36.597 INFO:teuthology.orchestra.run.smithi029.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-04-24T23:18:36.623 INFO:teuthology.orchestra.run.smithi026.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 516 kB/s | 133 kB 00:00 2024-04-24T23:18:36.689 INFO:teuthology.orchestra.run.smithi029.stdout:(32/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 5.0 MB/s | 3.5 MB 00:00 2024-04-24T23:18:36.723 INFO:teuthology.orchestra.run.smithi029.stdout:(33/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.9 MB/s | 249 kB 00:00 2024-04-24T23:18:36.749 INFO:teuthology.orchestra.run.smithi029.stdout:(34/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 838 kB/s | 49 kB 00:00 2024-04-24T23:18:36.783 INFO:teuthology.orchestra.run.smithi029.stdout:(35/152): ceph-prometheus-alerts-19.0.0-2445.gd 258 kB/s | 15 kB 00:00 2024-04-24T23:18:36.825 INFO:teuthology.orchestra.run.smithi029.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.4 MB/s | 264 kB 00:00 2024-04-24T23:18:36.875 INFO:teuthology.orchestra.run.smithi029.stdout:(37/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 8.1 MB/s | 762 kB 00:00 2024-04-24T23:18:36.901 INFO:teuthology.orchestra.run.smithi029.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 241 kB/s | 18 kB 00:00 2024-04-24T23:18:36.907 INFO:teuthology.orchestra.run.smithi026.stdout:(71/152): postgresql-13.14-1.el9.x86_64.rpm 750 kB/s | 1.6 MB 00:02 2024-04-24T23:18:37.011 INFO:teuthology.orchestra.run.smithi029.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 592 kB/s | 80 kB 00:00 2024-04-24T23:18:37.087 INFO:teuthology.orchestra.run.smithi029.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-24T23:18:37.099 INFO:teuthology.orchestra.run.smithi026.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 456 kB/s | 87 kB 00:00 2024-04-24T23:18:37.121 INFO:teuthology.orchestra.run.smithi029.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-24T23:18:37.155 INFO:teuthology.orchestra.run.smithi029.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-04-24T23:18:37.222 INFO:teuthology.orchestra.run.smithi029.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-24T23:18:37.247 INFO:teuthology.orchestra.run.smithi029.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-24T23:18:37.273 INFO:teuthology.orchestra.run.smithi029.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-24T23:18:37.299 INFO:teuthology.orchestra.run.smithi029.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-24T23:18:37.332 INFO:teuthology.orchestra.run.smithi029.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-24T23:18:37.483 INFO:teuthology.orchestra.run.smithi029.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-04-24T23:18:37.525 INFO:teuthology.orchestra.run.smithi029.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-04-24T23:18:37.893 INFO:teuthology.orchestra.run.smithi029.stdout:(50/152): ceph-common-19.0.0-2445.gd2c45d97.el9 2.1 MB/s | 21 MB 00:09 2024-04-24T23:18:37.926 INFO:teuthology.orchestra.run.smithi029.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 241 kB/s | 107 kB 00:00 2024-04-24T23:18:37.952 INFO:teuthology.orchestra.run.smithi029.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 50 kB/s | 21 kB 00:00 2024-04-24T23:18:37.994 INFO:teuthology.orchestra.run.smithi029.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 787 kB/s | 33 kB 00:00 2024-04-24T23:18:38.036 INFO:teuthology.orchestra.run.smithi029.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 434 kB/s | 18 kB 00:00 2024-04-24T23:18:38.066 INFO:teuthology.orchestra.run.smithi026.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 716 kB/s | 1.0 MB 00:01 2024-04-24T23:18:38.113 INFO:teuthology.orchestra.run.smithi029.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-24T23:18:38.138 INFO:teuthology.orchestra.run.smithi029.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 109 kB/s | 11 kB 00:00 2024-04-24T23:18:38.172 INFO:teuthology.orchestra.run.smithi029.stdout:(57/152): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-04-24T23:18:38.206 INFO:teuthology.orchestra.run.smithi029.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 672 kB/s | 45 kB 00:00 2024-04-24T23:18:38.232 INFO:teuthology.orchestra.run.smithi029.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-24T23:18:38.265 INFO:teuthology.orchestra.run.smithi029.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.6 MB/s | 582 kB 00:00 2024-04-24T23:18:38.299 INFO:teuthology.orchestra.run.smithi029.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 662 kB 00:00 2024-04-24T23:18:38.325 INFO:teuthology.orchestra.run.smithi029.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 274 kB/s | 25 kB 00:00 2024-04-24T23:18:38.325 INFO:teuthology.orchestra.run.smithi026.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 3.0 MB/s | 5.8 MB 00:01 2024-04-24T23:18:38.358 INFO:teuthology.orchestra.run.smithi029.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 511 kB/s | 47 kB 00:00 2024-04-24T23:18:38.384 INFO:teuthology.orchestra.run.smithi029.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-24T23:18:38.410 INFO:teuthology.orchestra.run.smithi029.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 695 kB/s | 35 kB 00:00 2024-04-24T23:18:38.433 INFO:teuthology.orchestra.run.smithi026.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-24T23:18:38.458 INFO:teuthology.orchestra.run.smithi026.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 623 kB/s | 244 kB 00:00 2024-04-24T23:18:38.468 INFO:teuthology.orchestra.run.smithi029.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-24T23:18:38.542 INFO:teuthology.orchestra.run.smithi026.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-04-24T23:18:38.567 INFO:teuthology.orchestra.run.smithi026.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-04-24T23:18:38.594 INFO:teuthology.orchestra.run.smithi029.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-04-24T23:18:38.651 INFO:teuthology.orchestra.run.smithi026.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-04-24T23:18:38.677 INFO:teuthology.orchestra.run.smithi026.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-04-24T23:18:38.695 INFO:teuthology.orchestra.run.smithi029.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-24T23:18:38.745 INFO:teuthology.orchestra.run.smithi029.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-04-24T23:18:38.770 INFO:teuthology.orchestra.run.smithi029.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-24T23:18:38.806 INFO:teuthology.orchestra.run.smithi029.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 4.2 MB/s | 911 kB 00:00 2024-04-24T23:18:38.857 INFO:teuthology.orchestra.run.smithi029.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-24T23:18:38.882 INFO:teuthology.orchestra.run.smithi029.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-04-24T23:18:38.933 INFO:teuthology.orchestra.run.smithi029.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-24T23:18:39.019 INFO:teuthology.orchestra.run.smithi026.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-04-24T23:18:39.050 INFO:teuthology.orchestra.run.smithi029.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-04-24T23:18:39.075 INFO:teuthology.orchestra.run.smithi029.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-24T23:18:39.101 INFO:teuthology.orchestra.run.smithi029.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 507 kB/s | 26 kB 00:00 2024-04-24T23:18:39.127 INFO:teuthology.orchestra.run.smithi029.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 378 kB/s | 19 kB 00:00 2024-04-24T23:18:39.160 INFO:teuthology.orchestra.run.smithi029.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-24T23:18:39.185 INFO:teuthology.orchestra.run.smithi029.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 590 kB/s | 35 kB 00:00 2024-04-24T23:18:39.303 INFO:teuthology.orchestra.run.smithi029.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-24T23:18:39.337 INFO:teuthology.orchestra.run.smithi029.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-24T23:18:39.352 INFO:teuthology.orchestra.run.smithi026.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.3 MB/s | 5.1 MB 00:00 2024-04-24T23:18:39.371 INFO:teuthology.orchestra.run.smithi029.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-24T23:18:39.377 INFO:teuthology.orchestra.run.smithi026.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 620 kB/s | 222 kB 00:00 2024-04-24T23:18:39.396 INFO:teuthology.orchestra.run.smithi029.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T23:18:39.421 INFO:teuthology.orchestra.run.smithi029.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 838 kB/s | 42 kB 00:00 2024-04-24T23:18:39.452 INFO:teuthology.orchestra.run.smithi026.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 771 kB/s | 77 kB 00:00 2024-04-24T23:18:39.455 INFO:teuthology.orchestra.run.smithi029.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-24T23:18:39.478 INFO:teuthology.orchestra.run.smithi026.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-04-24T23:18:39.491 INFO:teuthology.orchestra.run.smithi029.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-24T23:18:39.517 INFO:teuthology.orchestra.run.smithi029.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.5 MB/s | 279 kB 00:00 2024-04-24T23:18:39.542 INFO:teuthology.orchestra.run.smithi029.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T23:18:39.561 INFO:teuthology.orchestra.run.smithi026.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-04-24T23:18:39.567 INFO:teuthology.orchestra.run.smithi029.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T23:18:39.610 INFO:teuthology.orchestra.run.smithi029.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-24T23:18:39.686 INFO:teuthology.orchestra.run.smithi026.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-04-24T23:18:39.718 INFO:teuthology.orchestra.run.smithi029.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.1 MB/s | 5.1 MB 00:00 2024-04-24T23:18:39.744 INFO:teuthology.orchestra.run.smithi029.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 329 kB/s | 44 kB 00:00 2024-04-24T23:18:39.770 INFO:teuthology.orchestra.run.smithi029.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-24T23:18:39.787 INFO:teuthology.orchestra.run.smithi026.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 514 kB/s | 159 kB 00:00 2024-04-24T23:18:39.795 INFO:teuthology.orchestra.run.smithi029.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 709 kB/s | 36 kB 00:00 2024-04-24T23:18:39.812 INFO:teuthology.orchestra.run.smithi026.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 407 kB/s | 51 kB 00:00 2024-04-24T23:18:39.821 INFO:teuthology.orchestra.run.smithi029.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 937 kB/s | 47 kB 00:00 2024-04-24T23:18:39.854 INFO:teuthology.orchestra.run.smithi029.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-24T23:18:39.880 INFO:teuthology.orchestra.run.smithi029.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 980 kB/s | 58 kB 00:00 2024-04-24T23:18:39.887 INFO:teuthology.orchestra.run.smithi026.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-04-24T23:18:39.906 INFO:teuthology.orchestra.run.smithi029.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T23:18:39.931 INFO:teuthology.orchestra.run.smithi029.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-24T23:18:39.957 INFO:teuthology.orchestra.run.smithi029.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 883 kB/s | 45 kB 00:00 2024-04-24T23:18:39.988 INFO:teuthology.orchestra.run.smithi026.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-04-24T23:18:40.017 INFO:teuthology.orchestra.run.smithi029.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 260 kB/s | 22 kB 00:00 2024-04-24T23:18:40.096 INFO:teuthology.orchestra.run.smithi026.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-04-24T23:18:40.326 INFO:teuthology.orchestra.run.smithi029.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-24T23:18:40.410 INFO:teuthology.orchestra.run.smithi029.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-24T23:18:40.438 INFO:teuthology.orchestra.run.smithi026.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 646 kB/s | 221 kB 00:00 2024-04-24T23:18:40.547 INFO:teuthology.orchestra.run.smithi026.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-04-24T23:18:40.664 INFO:teuthology.orchestra.run.smithi026.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-04-24T23:18:40.677 INFO:teuthology.orchestra.run.smithi029.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-24T23:18:40.836 INFO:teuthology.orchestra.run.smithi029.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 670 kB/s | 548 kB 00:00 2024-04-24T23:18:40.870 INFO:teuthology.orchestra.run.smithi029.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 569 kB/s | 309 kB 00:00 2024-04-24T23:18:41.095 INFO:teuthology.orchestra.run.smithi029.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-04-24T23:18:41.189 INFO:teuthology.orchestra.run.smithi026.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-24T23:18:41.213 INFO:teuthology.orchestra.run.smithi029.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-24T23:18:41.214 INFO:teuthology.orchestra.run.smithi026.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 554 kB/s | 305 kB 00:00 2024-04-24T23:18:41.289 INFO:teuthology.orchestra.run.smithi026.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 579 kB/s | 58 kB 00:00 2024-04-24T23:18:41.390 INFO:teuthology.orchestra.run.smithi026.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 637 kB/s | 64 kB 00:00 2024-04-24T23:18:41.415 INFO:teuthology.orchestra.run.smithi026.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 467 kB/s | 94 kB 00:00 2024-04-24T23:18:41.446 INFO:teuthology.orchestra.run.smithi029.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 192 kB/s | 67 kB 00:00 2024-04-24T23:18:41.490 INFO:teuthology.orchestra.run.smithi026.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 446 kB/s | 45 kB 00:00 2024-04-24T23:18:41.566 INFO:teuthology.orchestra.run.smithi026.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-04-24T23:18:41.647 INFO:teuthology.orchestra.run.smithi029.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2024-04-24T23:18:41.681 INFO:teuthology.orchestra.run.smithi029.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 323 kB/s | 151 kB 00:00 2024-04-24T23:18:41.816 INFO:teuthology.orchestra.run.smithi026.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-04-24T23:18:41.850 INFO:teuthology.orchestra.run.smithi026.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-04-24T23:18:41.875 INFO:teuthology.orchestra.run.smithi026.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-04-24T23:18:41.967 INFO:teuthology.orchestra.run.smithi026.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-04-24T23:18:42.025 INFO:teuthology.orchestra.run.smithi026.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-04-24T23:18:42.031 INFO:teuthology.orchestra.run.smithi029.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-04-24T23:18:42.065 INFO:teuthology.orchestra.run.smithi029.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 838 kB 00:00 2024-04-24T23:18:42.084 INFO:teuthology.orchestra.run.smithi026.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-04-24T23:18:42.142 INFO:teuthology.orchestra.run.smithi026.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-24T23:18:42.226 INFO:teuthology.orchestra.run.smithi026.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-24T23:18:42.249 INFO:teuthology.orchestra.run.smithi029.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 911 kB/s | 548 kB 00:00 2024-04-24T23:18:42.285 INFO:teuthology.orchestra.run.smithi026.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-24T23:18:42.300 INFO:teuthology.orchestra.run.smithi029.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 120 kB/s | 32 kB 00:00 2024-04-24T23:18:42.333 INFO:teuthology.orchestra.run.smithi029.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-24T23:18:42.354 INFO:teuthology.orchestra.run.smithi026.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-04-24T23:18:42.419 INFO:teuthology.orchestra.run.smithi026.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-04-24T23:18:42.435 INFO:teuthology.orchestra.run.smithi026.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 521 kB/s | 43 kB 00:00 2024-04-24T23:18:42.469 INFO:teuthology.orchestra.run.smithi026.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-04-24T23:18:42.486 INFO:teuthology.orchestra.run.smithi026.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-24T23:18:42.528 INFO:teuthology.orchestra.run.smithi026.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-24T23:18:42.553 INFO:teuthology.orchestra.run.smithi026.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-24T23:18:42.587 INFO:teuthology.orchestra.run.smithi026.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-24T23:18:42.604 INFO:teuthology.orchestra.run.smithi026.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 701 kB/s | 35 kB 00:00 2024-04-24T23:18:42.646 INFO:teuthology.orchestra.run.smithi026.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-24T23:18:42.677 INFO:teuthology.orchestra.run.smithi029.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 402 kB/s | 172 kB 00:00 2024-04-24T23:18:42.704 INFO:teuthology.orchestra.run.smithi026.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-24T23:18:42.805 INFO:teuthology.orchestra.run.smithi026.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-24T23:18:42.811 INFO:teuthology.orchestra.run.smithi029.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 702 kB/s | 358 kB 00:00 2024-04-24T23:18:42.837 INFO:teuthology.orchestra.run.smithi029.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 603 kB/s | 303 kB 00:00 2024-04-24T23:18:42.855 INFO:teuthology.orchestra.run.smithi026.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-04-24T23:18:42.881 INFO:teuthology.orchestra.run.smithi026.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-24T23:18:42.914 INFO:teuthology.orchestra.run.smithi026.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-24T23:18:42.939 INFO:teuthology.orchestra.run.smithi026.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-24T23:18:42.945 INFO:teuthology.orchestra.run.smithi029.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-24T23:18:42.973 INFO:teuthology.orchestra.run.smithi026.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-04-24T23:18:43.032 INFO:teuthology.orchestra.run.smithi026.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-04-24T23:18:43.065 INFO:teuthology.orchestra.run.smithi026.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-24T23:18:43.093 INFO:teuthology.orchestra.run.smithi026.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-24T23:18:43.118 INFO:teuthology.orchestra.run.smithi026.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-24T23:18:43.151 INFO:teuthology.orchestra.run.smithi026.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-24T23:18:43.177 INFO:teuthology.orchestra.run.smithi026.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-24T23:18:43.210 INFO:teuthology.orchestra.run.smithi026.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-24T23:18:43.235 INFO:teuthology.orchestra.run.smithi026.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-04-24T23:18:43.269 INFO:teuthology.orchestra.run.smithi026.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-04-24T23:18:43.311 INFO:teuthology.orchestra.run.smithi026.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-24T23:18:43.321 INFO:teuthology.orchestra.run.smithi029.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 403 kB/s | 206 kB 00:00 2024-04-24T23:18:43.336 INFO:teuthology.orchestra.run.smithi026.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-04-24T23:18:43.370 INFO:teuthology.orchestra.run.smithi026.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-04-24T23:18:43.380 INFO:teuthology.orchestra.run.smithi029.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 333 kB/s | 144 kB 00:00 2024-04-24T23:18:43.395 INFO:teuthology.orchestra.run.smithi026.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 874 kB/s | 51 kB 00:00 2024-04-24T23:18:43.428 INFO:teuthology.orchestra.run.smithi026.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-24T23:18:43.454 INFO:teuthology.orchestra.run.smithi026.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-24T23:18:43.496 INFO:teuthology.orchestra.run.smithi026.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-04-24T23:18:43.505 INFO:teuthology.orchestra.run.smithi029.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-04-24T23:18:43.521 INFO:teuthology.orchestra.run.smithi026.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-24T23:18:43.555 INFO:teuthology.orchestra.run.smithi026.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-24T23:18:43.639 INFO:teuthology.orchestra.run.smithi029.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-04-24T23:18:43.664 INFO:teuthology.orchestra.run.smithi026.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-24T23:18:43.665 INFO:teuthology.orchestra.run.smithi029.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 62 kB/s | 18 kB 00:00 2024-04-24T23:18:43.774 INFO:teuthology.orchestra.run.smithi029.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 86 kB/s | 23 kB 00:00 2024-04-24T23:18:43.907 INFO:teuthology.orchestra.run.smithi029.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 72 kB/s | 19 kB 00:00 2024-04-24T23:18:43.958 INFO:teuthology.orchestra.run.smithi029.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 67 kB/s | 20 kB 00:00 2024-04-24T23:18:43.972 INFO:teuthology.orchestra.run.smithi026.stdout:(148/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-24T23:18:44.147 INFO:teuthology.orchestra.run.smithi026.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 5.7 MB/s | 3.4 MB 00:00 2024-04-24T23:18:44.250 INFO:teuthology.orchestra.run.smithi029.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-24T23:18:44.300 INFO:teuthology.orchestra.run.smithi029.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 230 kB/s | 79 kB 00:00 2024-04-24T23:18:44.459 INFO:teuthology.orchestra.run.smithi029.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-24T23:18:44.613 INFO:teuthology.orchestra.run.smithi029.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 160 kB/s | 58 kB 00:00 2024-04-24T23:18:44.729 INFO:teuthology.orchestra.run.smithi029.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-04-24T23:18:44.798 INFO:teuthology.orchestra.run.smithi026.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 1.3 MB 00:00 2024-04-24T23:18:44.815 INFO:teuthology.orchestra.run.smithi029.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 532 kB/s | 272 kB 00:00 2024-04-24T23:18:44.839 INFO:teuthology.orchestra.run.smithi026.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-04-24T23:18:44.963 INFO:teuthology.orchestra.run.smithi029.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 256 kB/s | 90 kB 00:00 2024-04-24T23:18:44.997 INFO:teuthology.orchestra.run.smithi029.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 105 kB/s | 28 kB 00:00 2024-04-24T23:18:45.081 INFO:teuthology.orchestra.run.smithi029.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-04-24T23:18:45.340 INFO:teuthology.orchestra.run.smithi026.stdout:(152/152): python3-babel-2.9.1-2.el9.noarch.rpm 741 kB/s | 6.0 MB 00:08 2024-04-24T23:18:45.342 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:18:45.343 INFO:teuthology.orchestra.run.smithi026.stdout:Total 11 MB/s | 220 MB 00:20 2024-04-24T23:18:45.348 INFO:teuthology.orchestra.run.smithi029.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-04-24T23:18:45.373 INFO:teuthology.orchestra.run.smithi029.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 157 kB/s | 59 kB 00:00 2024-04-24T23:18:45.407 INFO:teuthology.orchestra.run.smithi029.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 425 kB/s | 188 kB 00:00 2024-04-24T23:18:45.691 INFO:teuthology.orchestra.run.smithi029.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-04-24T23:18:45.750 INFO:teuthology.orchestra.run.smithi029.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-04-24T23:18:45.884 INFO:teuthology.orchestra.run.smithi029.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 450 kB/s | 230 kB 00:00 2024-04-24T23:18:46.026 INFO:teuthology.orchestra.run.smithi029.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 72 kB/s | 20 kB 00:00 2024-04-24T23:18:46.285 INFO:teuthology.orchestra.run.smithi029.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 720 kB/s | 427 kB 00:00 2024-04-24T23:18:46.310 INFO:teuthology.orchestra.run.smithi029.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-04-24T23:18:46.471 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-04-24T23:18:46.629 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-04-24T23:18:46.629 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-04-24T23:18:46.994 INFO:teuthology.orchestra.run.smithi029.stdout:(148/152): librados2-19.0.0-2445.gd2c45d97.el9. 4.8 MB/s | 3.4 MB 00:00 2024-04-24T23:18:47.055 INFO:teuthology.orchestra.run.smithi029.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-04-24T23:18:47.130 INFO:teuthology.orchestra.run.smithi029.stdout:(150/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-04-24T23:18:47.305 INFO:teuthology.orchestra.run.smithi029.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 1.3 MB 00:00 2024-04-24T23:18:47.422 INFO:teuthology.orchestra.run.smithi029.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 2.4 MB 00:00 2024-04-24T23:18:47.425 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:18:47.426 INFO:teuthology.orchestra.run.smithi029.stdout:Total 11 MB/s | 220 MB 00:20 2024-04-24T23:18:48.508 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-04-24T23:18:48.509 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-04-24T23:18:48.560 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-04-24T23:18:48.721 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-04-24T23:18:48.721 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-04-24T23:18:50.600 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-04-24T23:18:50.600 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-04-24T23:18:54.653 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-04-24T23:18:54.777 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:18:55.025 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:18:55.080 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:18:55.120 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:18:55.153 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:18:55.198 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:18:55.544 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:18:55.653 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:18:55.769 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:18:55.796 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:18:55.872 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:18:55.901 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:18:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:18:56.106 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:18:56.134 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:18:56.239 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:18:56.268 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:18:56.331 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:18:56.352 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:18:56.371 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:18:56.593 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:18:56.777 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:18:56.888 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:18:56.984 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:18:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:18:57.053 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:18:57.076 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-04-24T23:18:57.129 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:18:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:18:57.187 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:18:57.221 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:18:57.243 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:18:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:18:57.289 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:18:57.297 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:18:57.341 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:18:57.369 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:18:57.375 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:18:57.420 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:18:57.421 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:18:57.450 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:18:57.494 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:18:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:18:57.604 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:18:57.673 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:18:57.701 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:18:57.775 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:18:57.777 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:18:57.825 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:18:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:18:57.970 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:18:58.040 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:18:58.096 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:18:58.123 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:18:58.153 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:18:58.217 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:18:58.262 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:18:58.420 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:18:58.479 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:18:58.508 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:18:58.614 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:18:58.643 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:18:58.707 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:18:58.708 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:18:58.729 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:18:58.755 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:18:58.799 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:18:58.834 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:18:58.877 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:18:58.915 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:18:58.958 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:18:58.965 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:18:58.999 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:18:59.111 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:18:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:18:59.193 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:18:59.216 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:18:59.249 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:18:59.250 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:18:59.294 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:18:59.332 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:18:59.369 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:18:59.374 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:18:59.405 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:18:59.420 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:18:59.453 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:18:59.465 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:18:59.487 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:18:59.498 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:18:59.531 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:18:59.535 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:18:59.573 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:18:59.616 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:18:59.668 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:18:59.707 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:18:59.717 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:18:59.754 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:18:59.762 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:18:59.792 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:18:59.834 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:18:59.840 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:18:59.873 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:18:59.917 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:18:59.932 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:18:59.967 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:19:00.007 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:19:00.040 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:19:00.065 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:19:00.096 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:19:00.143 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:19:00.282 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:19:00.287 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:19:00.314 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:19:00.337 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:19:00.342 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:19:00.377 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:19:00.416 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:19:00.435 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:19:00.456 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:19:00.485 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:19:00.520 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:19:00.553 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:19:00.580 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:19:00.595 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:19:00.629 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:19:00.636 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:19:00.718 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:19:00.768 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:19:00.799 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:19:00.847 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:19:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:19:01.018 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:19:01.193 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:19:01.287 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:19:01.322 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:19:01.366 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:19:01.405 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:19:01.455 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:19:01.657 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:19:01.729 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:19:01.786 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:19:01.786 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:19:01.868 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:19:01.897 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:19:01.906 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:19:01.930 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:19:01.945 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:19:02.001 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:19:02.033 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:19:02.038 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:19:02.089 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:19:02.090 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:19:02.141 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:19:02.178 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:19:02.216 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:19:02.295 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:19:02.351 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:19:02.389 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:19:02.399 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:19:02.430 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:19:02.440 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:19:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:19:02.539 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:19:02.683 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:19:02.734 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:19:02.768 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:19:02.801 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:19:02.889 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:19:03.031 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:19:03.073 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:19:03.110 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:19:03.145 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:19:03.187 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:19:03.190 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:19:03.244 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:19:03.274 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:19:03.298 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:19:03.309 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:19:03.343 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:19:03.386 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:19:03.420 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:19:03.511 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:19:03.570 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:19:03.610 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:19:03.659 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:19:03.770 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:19:03.832 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:19:04.598 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:19:04.602 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:19:04.658 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:19:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:19:04.692 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:19:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:19:04.753 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:19:04.796 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:19:04.855 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:19:04.945 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:19:04.990 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:19:05.072 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:19:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:19:05.177 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:19:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:19:05.205 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:19:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:19:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:19:05.315 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:19:05.762 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:19:05.793 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:19:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:19:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:19:05.984 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:19:06.116 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:19:07.432 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:19:07.543 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:19:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:19:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:19:07.935 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:19:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:19:08.088 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:19:08.118 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:19:08.164 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:19:08.221 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:19:08.257 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:19:08.663 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:19:08.692 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:19:08.764 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:19:08.795 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:19:08.825 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:19:08.878 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:19:08.970 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:19:08.971 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:19:08.971 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:19:08.971 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:09.004 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:19:11.245 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:19:11.283 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:19:11.855 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:19:11.906 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:19:12.006 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:19:12.006 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:19:12.006 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:19:12.007 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:12.044 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:19:20.819 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:19:20.819 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-04-24T23:19:20.819 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-04-24T23:19:20.820 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-04-24T23:19:20.820 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-04-24T23:19:20.820 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-04-24T23:19:20.820 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-04-24T23:19:20.820 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-04-24T23:19:20.820 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:20.999 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:19:21.014 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:19:21.015 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:21.015 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:19:21.015 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:22.544 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:19:22.570 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:19:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:19:22.571 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:19:22.571 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:19:22.571 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:22.811 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:19:22.835 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:19:22.835 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:22.835 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:19:22.835 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:19:22.835 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:19:22.835 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:19:23.327 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:19:23.327 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:23.328 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:19:23.328 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:19:23.328 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:19:23.328 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:23.443 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:19:23.500 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:19:23.564 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:19:23.761 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:19:23.813 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:19:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:19:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:19:24.135 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:19:24.171 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:19:24.203 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:19:24.230 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:19:24.700 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:19:24.729 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:19:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:19:24.825 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:19:24.860 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:19:25.944 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:19:25.978 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:19:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:19:27.012 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:19:27.120 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:19:27.291 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:19:27.323 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:19:27.347 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:19:27.347 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:27.347 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:19:27.347 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:19:27.347 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:19:27.347 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:19:27.409 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:19:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:19:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:19:27.797 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:19:27.797 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /sys 2024-04-24T23:19:27.797 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /proc 2024-04-24T23:19:27.797 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /mnt 2024-04-24T23:19:27.797 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /var/tmp 2024-04-24T23:19:27.797 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /home 2024-04-24T23:19:27.797 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /root 2024-04-24T23:19:27.798 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /tmp 2024-04-24T23:19:27.798 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:27.980 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:19:28.001 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:19:28.002 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:28.002 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:19:28.002 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:28.508 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:19:28.534 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:19:28.558 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:19:28.558 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:28.558 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:19:28.558 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:19:28.558 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:19:28.558 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:28.601 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:19:28.624 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:19:28.624 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:28.624 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:19:28.624 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:28.668 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:19:28.692 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:19:28.692 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:28.692 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:19:28.692 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:19:28.692 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:19:28.692 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:19:29.126 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:19:29.127 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:29.127 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:19:29.127 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:19:29.127 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:19:29.127 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:29.536 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:19:29.564 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:19:29.565 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:29.565 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:19:29.565 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:19:29.565 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:19:29.565 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:29.804 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:19:29.829 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:19:29.829 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:29.829 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:19:29.829 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:19:29.829 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:19:29.829 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:30.298 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:19:30.322 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:19:30.323 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:30.323 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:19:30.323 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:19:30.323 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:19:30.323 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:30.439 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:19:30.498 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:19:30.562 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:19:30.766 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:19:30.823 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:19:30.905 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:19:31.002 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:19:31.155 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:19:31.205 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:19:31.243 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:19:31.271 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:19:31.752 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:19:31.784 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:19:31.824 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:19:31.889 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:19:31.924 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:19:33.017 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:19:33.062 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:19:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:19:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:19:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:19:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:19:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:19:34.080 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:19:34.113 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:19:34.222 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:19:34.223 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:19:34.268 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:19:34.298 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:19:34.402 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:19:34.433 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:19:34.458 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:19:34.459 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:34.459 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:19:34.459 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:19:34.459 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:19:34.459 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:34.510 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:19:34.521 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:19:34.640 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:19:34.695 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:19:34.831 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:19:34.871 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:19:34.891 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:19:35.643 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:19:35.674 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:19:35.699 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:19:35.699 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:35.699 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:19:35.699 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:19:35.699 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:19:35.699 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:35.742 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:19:35.766 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:19:35.766 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:35.766 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:19:35.766 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:35.788 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:19:35.810 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:19:35.834 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:19:35.834 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:35.835 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:19:35.835 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:19:35.835 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:19:35.835 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:35.875 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:19:35.926 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:19:36.011 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:19:36.046 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:19:36.046 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:19:36.130 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:19:36.162 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:19:36.163 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:19:36.163 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:19:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:19:36.186 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:19:36.186 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:19:36.186 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:19:36.186 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:19:36.186 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:36.254 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:19:36.324 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:19:40.971 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:19:41.015 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:19:41.057 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:19:41.099 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:19:41.133 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:19:41.328 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:19:41.393 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:19:41.423 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:19:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:19:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:19:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:19:41.997 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:19:41.998 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:19:41.999 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:19:42.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:19:42.001 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:19:42.002 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:19:42.003 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:19:42.004 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:19:42.005 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:19:42.006 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:19:42.007 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:19:42.086 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:19:42.147 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:19:42.168 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:19:43.074 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:19:43.190 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:19:43.233 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:19:43.328 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:19:43.366 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:19:43.366 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:19:43.468 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:19:43.511 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:19:43.511 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:19:43.608 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:19:43.689 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:19:44.076 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:19:44.076 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:44.076 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-04-24T23:19:44.076 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.076 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.076 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:19:44.076 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:19:44.076 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-04-24T23:19:44.076 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.077 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.078 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.079 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:19:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:19:44.081 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:19:44.082 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:19:44.083 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:19:44.084 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:19:44.085 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:44.086 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:19:44.087 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:19:44.087 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:19:44.087 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:19:44.087 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:19:44.087 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:19:44.087 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:19:44.087 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:19:44.087 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-04-24T23:19:44.471 DEBUG:teuthology.parallel:result is None 2024-04-24T23:19:54.359 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:19:54.359 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:19:54.360 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:19:54.361 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:19:54.362 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:19:54.362 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:19:54.362 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:19:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:19:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:19:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:19:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:19:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:19:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:19:54.374 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:19:54.375 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:19:54.376 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:19:54.377 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:19:54.378 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:19:54.379 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:19:54.380 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:19:54.381 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:19:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:19:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:19:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:19:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:19:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:19:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:19:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:19:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:19:54.382 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:19:54.383 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:19:54.383 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:19:54.384 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:19:54.385 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:19:54.385 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:19:54.385 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:19:56.621 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:19:56.621 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:56.621 INFO:teuthology.orchestra.run.smithi029.stdout:Upgraded: 2024-04-24T23:19:56.621 INFO:teuthology.orchestra.run.smithi029.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.621 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.621 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:19:56.621 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.622 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.623 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:19:56.624 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:19:56.625 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:19:56.626 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.627 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:19:56.628 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:19:56.629 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:19:56.630 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:19:56.631 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:19:56.632 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:19:56.633 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:19:56.633 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-04-24T23:19:57.130 DEBUG:teuthology.parallel:result is None 2024-04-24T23:19:57.131 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:19:57.287 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:19:57.314 INFO:teuthology.orchestra.run.smithi026.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:19:57.315 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:19:57.315 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:19:57.316 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:19:57.548 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:19:57.569 INFO:teuthology.orchestra.run.smithi029.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:19:57.570 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:19:57.570 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:19:57.571 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T23:19:57.572 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:19:57.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:19:57.604 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:19:57.604 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:19:57.640 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T23:19:57.640 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:19:57.640 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:19:57.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:19:57.740 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:19:57.740 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:19:57.765 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:19:57.829 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T23:19:57.829 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:19:57.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:19:57.857 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:19:57.923 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:19:57.923 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:19:57.948 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:19:58.015 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T23:19:58.015 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:19:58.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:19:58.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:19:58.105 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:19:58.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:19:58.129 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:19:58.196 INFO:teuthology.run_tasks:Running task ceph... 2024-04-24T23:19:58.284 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-24T23:19:58.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /var/log/ceph 2024-04-24T23:19:58.287 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 777 /var/log/ceph 2024-04-24T23:19:58.313 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-24T23:19:58.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T23:19:58.355 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T23:19:58.379 INFO:tasks.ceph:Creating extra log directories... 2024-04-24T23:19:58.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T23:19:58.420 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T23:19:58.451 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-24T23:19:58.451 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-24T23:19:58.451 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-24T23:19:58.452 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:19:58.452 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T23:19:58.499 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T23:19:58.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T23:19:58.642 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:19:58.642 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T23:19:58.657 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T23:19:58.782 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T23:19:58.812 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-24T23:19:58.812 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Scrub error on inode'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-24T23:19:58.812 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/multimds_misc}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672190', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Scrub error on inode'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi026.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDx0szPYw7CaWRErs8DjFuW2U76iopXRUDhkqyZZ7sYbcyS3wmGy2qfHQDxmOSx0Mm9fjrjtsbVV6ZUjFMw84tc=', 'smithi029.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIV/eIkpXapl3uvodBPAHmFEw4QeknvGd8am2syjhcSxhSYh2ns0Vuh56XaaDVVXjFaFeFy0PMGWhGlFnExSSsQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Scrub error on inode'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_multimds_misc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-24T23:19:58.812 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T23:19:58.814 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T23:19:58.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T23:19:58.870 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T23:19:58.925 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:19:58.925 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:19:58.947 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:19:58.947 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-04-24T23:19:59.001 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:19:59.001 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:19:59.001 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-24T23:19:59.002 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:19:59.002 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:19:59.002 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-24 23:19:36.703460625 +0000 2024-04-24T23:19:59.002 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-24 23:17:43.455304886 +0000 2024-04-24T23:19:59.002 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-24 23:17:43.455304886 +0000 2024-04-24T23:19:59.002 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-24 23:17:43.455304886 +0000 2024-04-24T23:19:59.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:19:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-24T23:19:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-24T23:19:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000103408 s, 5.0 MB/s 2024-04-24T23:19:59.065 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:19:59.121 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-04-24T23:19:59.177 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:19:59.177 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:19:59.177 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T23:19:59.177 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:19:59.177 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:19:59.177 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-24 23:19:36.703460625 +0000 2024-04-24T23:19:59.177 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-24 23:17:43.794299591 +0000 2024-04-24T23:19:59.177 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-24 23:17:43.794299591 +0000 2024-04-24T23:19:59.177 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-24 23:17:43.794299591 +0000 2024-04-24T23:19:59.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:19:59.239 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-24T23:19:59.240 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-24T23:19:59.240 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 9.541e-05 s, 5.4 MB/s 2024-04-24T23:19:59.241 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:19:59.297 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-04-24T23:19:59.352 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:19:59.353 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:19:59.353 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-24T23:19:59.353 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:19:59.353 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:19:59.353 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-24 23:19:36.703460625 +0000 2024-04-24T23:19:59.353 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-24 23:17:44.129294359 +0000 2024-04-24T23:19:59.353 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-24 23:17:44.129294359 +0000 2024-04-24T23:19:59.353 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-24 23:17:44.129294359 +0000 2024-04-24T23:19:59.353 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:19:59.415 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-24T23:19:59.415 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-24T23:19:59.415 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000101541 s, 5.0 MB/s 2024-04-24T23:19:59.416 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:19:59.472 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-04-24T23:19:59.528 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:19:59.528 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:19:59.528 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-24T23:19:59.528 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:19:59.528 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:19:59.528 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-24 23:19:36.704460608 +0000 2024-04-24T23:19:59.528 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-24 23:17:44.474288970 +0000 2024-04-24T23:19:59.528 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-24 23:17:44.474288970 +0000 2024-04-24T23:19:59.529 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-24 23:17:44.474288970 +0000 2024-04-24T23:19:59.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:19:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-24T23:19:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-24T23:19:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 9.5559e-05 s, 5.4 MB/s 2024-04-24T23:19:59.592 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:19:59.649 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:19:59.649 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:19:59.649 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:19:59.666 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:19:59.667 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_1 2024-04-24T23:19:59.723 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:19:59.723 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:19:59.723 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-24T23:19:59.723 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:19:59.723 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:19:59.723 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-04-24 23:19:44.082314865 +0000 2024-04-24T23:19:59.723 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-04-24 23:17:45.596009601 +0000 2024-04-24T23:19:59.723 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-04-24 23:17:45.596009601 +0000 2024-04-24T23:19:59.723 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-04-24 23:17:45.596009601 +0000 2024-04-24T23:19:59.724 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:19:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-24T23:19:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-24T23:19:59.787 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 9.8347e-05 s, 5.2 MB/s 2024-04-24T23:19:59.788 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:19:59.846 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_2 2024-04-24T23:19:59.901 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:19:59.902 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:19:59.902 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T23:19:59.902 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:19:59.902 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:19:59.902 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-04-24 23:19:44.083314849 +0000 2024-04-24T23:19:59.902 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-04-24 23:17:45.940003942 +0000 2024-04-24T23:19:59.902 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-04-24 23:17:45.940003942 +0000 2024-04-24T23:19:59.902 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-04-24 23:17:45.940003942 +0000 2024-04-24T23:19:59.902 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:19:59.966 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-24T23:19:59.966 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-24T23:19:59.966 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000113616 s, 4.5 MB/s 2024-04-24T23:19:59.967 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:20:00.024 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_3 2024-04-24T23:20:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:20:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:20:00.080 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T23:20:00.080 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:20:00.080 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:20:00.080 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-04-24 23:19:44.083314849 +0000 2024-04-24T23:20:00.080 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-04-24 23:17:46.288998199 +0000 2024-04-24T23:20:00.080 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-04-24 23:17:46.288998199 +0000 2024-04-24T23:20:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-04-24 23:17:46.288998199 +0000 2024-04-24T23:20:00.080 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:20:00.144 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-24T23:20:00.144 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-24T23:20:00.144 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 9.6945e-05 s, 5.3 MB/s 2024-04-24T23:20:00.145 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:20:00.202 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_4 2024-04-24T23:20:00.257 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:20:00.257 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:20:00.257 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-24T23:20:00.257 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:20:00.257 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:20:00.257 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-04-24 23:19:44.084314833 +0000 2024-04-24T23:20:00.257 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-04-24 23:17:46.633992523 +0000 2024-04-24T23:20:00.258 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-04-24 23:17:46.633992523 +0000 2024-04-24T23:20:00.258 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-04-24 23:17:46.633992523 +0000 2024-04-24T23:20:00.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:20:00.321 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-04-24T23:20:00.322 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-04-24T23:20:00.322 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 9.5695e-05 s, 5.4 MB/s 2024-04-24T23:20:00.323 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:20:00.379 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:20:00.379 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T23:20:00.380 INFO:tasks.ceph:Generating config... 2024-04-24T23:20:00.381 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-24T23:20:00.381 INFO:tasks.ceph:[client] debug client = 20 2024-04-24T23:20:00.381 INFO:tasks.ceph:[client] debug ms = 1 2024-04-24T23:20:00.381 INFO:tasks.ceph:[client] pid file = 2024-04-24T23:20:00.381 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-24T23:20:00.382 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-24T23:20:00.382 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-24T23:20:00.382 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-24T23:20:00.383 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-24T23:20:00.383 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-24T23:20:00.383 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-24T23:20:00.383 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-24T23:20:00.383 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-24T23:20:00.383 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-24T23:20:00.383 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-24T23:20:00.383 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-24T23:20:00.383 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-24T23:20:00.383 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-24T23:20:00.383 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-24T23:20:00.383 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-24T23:20:00.383 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-24T23:20:00.383 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-24T23:20:00.383 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-24T23:20:00.383 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-24T23:20:00.384 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-24T23:20:00.384 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-24T23:20:00.384 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-24T23:20:00.384 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-24T23:20:00.384 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-24T23:20:00.384 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-24T23:20:00.384 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-24T23:20:00.384 INFO:tasks.ceph:Setting up mon.a... 2024-04-24T23:20:00.385 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-24T23:20:00.442 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.keyring 2024-04-24T23:20:00.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-24T23:20:00.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T23:20:00.514 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.26'), ('mon.b', '172.21.15.29'), ('mon.c', '[v2:172.21.15.29:3301,v1:172.21.15.29:6790]')] 2024-04-24T23:20:00.515 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.26,172.21.15.29,[v2:172.21.15.29:3301,v1:172.21.15.29:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-24T23:20:00.516 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:00.516 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T23:20:00.571 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.26 --add b 172.21.15.29 --addv c '[v2:172.21.15.29:3301,v1:172.21.15.29:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:00.658 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:00.658 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: generated fsid 5a872e4b-97fd-4255-95a5-3ea24d1048cf 2024-04-24T23:20:00.658 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = quincy 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 0 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:fsid 5a872e4b-97fd-4255-95a5-3ea24d1048cf 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:last_changed 2024-04-24T23:20:00.657578+0000 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-04-24T23:20:00.657578+0000 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:min_mon_release 17 (quincy) 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:election_strategy: 1 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.b 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:2: [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] mon.c 2024-04-24T23:20:00.659 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-24T23:20:00.660 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T23:20:00.676 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5a872e4b-97fd-4255-95a5-3ea24d1048cf... 2024-04-24T23:20:00.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T23:20:00.718 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T23:20:00.757 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-04-24T23:20:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: chdir = "" 2024-04-24T23:20:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T23:20:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: auth supported = cephx 2024-04-24T23:20:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: filestore xattr use omap = true 2024-04-24T23:20:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: mon clock drift allowed = 1.000 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: osd crush chooseleaf type = 0 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: auth debug = true 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on old message = true 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on bug = true 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: mon pg warn max object skew = 0 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default size = 2 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow primary affinity = true 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow pg remap = true 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on legacy crush tunables = false 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on crush straw calc version zero = false 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on no sortbitwise = false 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on osd down out interval zero = false 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on too few osds = false 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: mon_allow_pool_size_one = true 2024-04-24T23:20:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: osd default data pool replay window = 5 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: mon allow pool delete = true 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: mon cluster log file level = debug 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: debug asserts on shutdown = true 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: mon health detail to clog = false 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: mon host = "172.21.15.26,172.21.15.29,[v2:172.21.15.29:3301,v1:172.21.15.29:6790]" 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on skipped message = False 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 5a872e4b-97fd-4255-95a5-3ea24d1048cf 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout:[osd] 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: osd journal size = 100 2024-04-24T23:20:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub load threshold = 5.0 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub max interval = 600 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock profile = high_recovery_ops 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd recover clone overlap = true 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd recovery max chunk = 1048576 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug shutdown = true 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug op order = true 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify stray on activate = true 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug trim objects = true 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd open classes on start = true 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug pg log writeout = true 2024-04-24T23:20:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: osd deep scrub update digest min age = 30 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: osd map max advance = 10 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: journal zero on create = true 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: filestore ondisk finisher threads = 3 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: filestore apply finisher threads = 3 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: bdev debug aio = true 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug misdirected ops = true 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: bdev async discard = True 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: bdev enable discard = True 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore allocator = bitmap 2024-04-24T23:20:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore block size = 96636764160 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore fsck on mount = True 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluefs = 1/20 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluestore = 1/20 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: debug osd = 20 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: debug rocksdb = 4/10 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd full ratio = 0.9 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd nearfull ratio = 0.8 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: osd failsafe full ratio = 0.95 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: osd objectstore = bluestore 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr] 2024-04-24T23:20:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: debug mgr = 20 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: mgr/telemetry/nag = false 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout:[mon] 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: debug paxos = 20 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-04-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi026.stdout: mon data avail warn = 5 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: mon mgr mkfs grace = 240 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd reporter subtree level = osd 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd prime pg temp = true 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T23:20:00.765 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: # 1m isn't quite enough 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: mon_down_mkfs_grace = 2m 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_filestore_osds = false 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: mon down mkfs grace = 300 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: mon op complaint time = 120 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout:[client] 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: rgw cache enabled = true 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable ops log = true 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable usage log = true 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T23:20:00.766 INFO:teuthology.orchestra.run.smithi026.stdout: client mount timeout = 600 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = "" 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.a] 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.b] 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.c] 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds] 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds = 20 2024-04-24T23:20:00.767 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds balancer = 20 2024-04-24T23:20:00.768 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-24T23:20:00.768 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug frag = True 2024-04-24T23:20:00.768 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug scatterstat = True 2024-04-24T23:20:00.768 INFO:teuthology.orchestra.run.smithi026.stdout: mds op complaint time = 180 2024-04-24T23:20:00.768 INFO:teuthology.orchestra.run.smithi026.stdout: mds verify scatter = True 2024-04-24T23:20:00.768 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-04-24T23:20:00.768 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-04-24T23:20:00.768 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout:[global] 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: chdir = "" 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: auth supported = cephx 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: filestore xattr use omap = true 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: mon clock drift allowed = 1.000 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: osd crush chooseleaf type = 0 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: auth debug = true 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.769 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on old message = true 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on bug = true 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: mon pg warn max object skew = 0 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default size = 2 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow primary affinity = true 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow pg remap = true 2024-04-24T23:20:00.770 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on legacy crush tunables = false 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on crush straw calc version zero = false 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on no sortbitwise = false 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on osd down out interval zero = false 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on too few osds = false 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: mon_allow_pool_size_one = true 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: osd default data pool replay window = 5 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: mon allow pool delete = true 2024-04-24T23:20:00.771 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: mon cluster log file level = debug 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: debug asserts on shutdown = true 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: mon health detail to clog = false 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: mon host = "172.21.15.26,172.21.15.29,[v2:172.21.15.29:3301,v1:172.21.15.29:6790]" 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on skipped message = False 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: fsid = 5a872e4b-97fd-4255-95a5-3ea24d1048cf 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout:[osd] 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: osd journal size = 100 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub load threshold = 5.0 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub max interval = 600 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: osd mclock profile = high_recovery_ops 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.772 INFO:teuthology.orchestra.run.smithi029.stdout: osd recover clone overlap = true 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: osd recovery max chunk = 1048576 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug shutdown = true 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug op order = true 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug verify stray on activate = true 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug trim objects = true 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: osd open classes on start = true 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug pg log writeout = true 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: osd deep scrub update digest min age = 30 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: osd map max advance = 10 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.773 INFO:teuthology.orchestra.run.smithi029.stdout: journal zero on create = true 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: filestore ondisk finisher threads = 3 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: filestore apply finisher threads = 3 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: bdev debug aio = true 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug misdirected ops = true 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: bdev async discard = True 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: bdev enable discard = True 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore allocator = bitmap 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore block size = 96636764160 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore fsck on mount = True 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluefs = 1/20 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluestore = 1/20 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-24T23:20:00.774 INFO:teuthology.orchestra.run.smithi029.stdout: debug osd = 20 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: debug rocksdb = 4/10 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd full ratio = 0.9 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd nearfull ratio = 0.8 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: osd failsafe full ratio = 0.95 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: osd objectstore = bluestore 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: osd op complaint time = 180 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr] 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: debug mgr = 20 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:20:00.775 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: mgr/telemetry/nag = false 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: debug client = 20 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout:[mon] 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: debug paxos = 20 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: mon data avail warn = 5 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: mon mgr mkfs grace = 240 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd reporter subtree level = osd 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd prime pg temp = true 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:20:00.776 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: # 1m isn't quite enough 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: mon_down_mkfs_grace = 2m 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_filestore_osds = false 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: mon down mkfs grace = 300 2024-04-24T23:20:00.777 INFO:teuthology.orchestra.run.smithi029.stdout: mon op complaint time = 120 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout:[client] 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: rgw cache enabled = true 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable ops log = true 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable usage log = true 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: client mount timeout = 600 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: debug client = 20 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: pid file = "" 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: rados mon op timeout = 900 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout: rados osd op timeout = 900 2024-04-24T23:20:00.778 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.a] 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.b] 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.c] 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.a] 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.c] 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.b] 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.d] 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout:[mds] 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: debug mds = 20 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: debug mds balancer = 20 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: mds debug frag = True 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: mds debug scatterstat = True 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: mds op complaint time = 180 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: mds verify scatter = True 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: osd op complaint time = 180 2024-04-24T23:20:00.779 INFO:teuthology.orchestra.run.smithi029.stdout: rados mon op timeout = 900 2024-04-24T23:20:00.780 INFO:teuthology.orchestra.run.smithi029.stdout: rados osd op timeout = 900 2024-04-24T23:20:00.781 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-24T23:20:00.781 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-24T23:20:00.874 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-24T23:20:00.874 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:00.875 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-24T23:20:00.892 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:00.892 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-24T23:20:00.947 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:20:00.947 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:00.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T23:20:00.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T23:20:01.020 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:01.020 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:01.075 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:20:01.075 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:01.076 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T23:20:01.076 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T23:20:01.110 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:01.110 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:01.166 INFO:tasks.ceph:Setting up mon nodes... 2024-04-24T23:20:01.166 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-24T23:20:01.166 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T23:20:01.216 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T23:20:01.218 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T23:20:01.289 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T23:20:01.291 INFO:tasks.ceph:Setting up mds nodes... 2024-04-24T23:20:01.292 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T23:20:01.341 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T23:20:01.344 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-24T23:20:01.368 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T23:20:01.458 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T23:20:01.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-24T23:20:01.485 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T23:20:01.535 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T23:20:01.537 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-24T23:20:01.561 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T23:20:01.650 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T23:20:01.653 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-24T23:20:01.677 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-24T23:20:01.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T23:20:01.718 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-24T23:20:01.730 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-24T23:20:01.810 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-24T23:20:01.823 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T23:20:01.863 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-24T23:20:01.875 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-24T23:20:01.955 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-24T23:20:01.967 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-24T23:20:01.967 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T23:20:01.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-24T23:20:01.991 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:20:01.992 INFO:tasks.ceph:role: osd.0 2024-04-24T23:20:01.992 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:20:01.992 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T23:20:02.055 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:20:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:20:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:20:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:20:02.056 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:20:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:20:02.056 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:20:02.056 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:20:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:20:02.056 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:20:02.091 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-04-24T23:20:02.093 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-04-24T23:20:02.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-24T23:20:02.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-24T23:20:02.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-24T23:20:02.293 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:20:02.293 INFO:tasks.ceph:role: osd.1 2024-04-24T23:20:02.294 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:20:02.294 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T23:20:02.357 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:20:02.358 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:20:02.358 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:20:02.358 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:20:02.358 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:20:02.358 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:20:02.358 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:20:02.358 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:20:02.358 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:20:02.358 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:20:02.390 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-04-24T23:20:02.391 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-04-24T23:20:02.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-24T23:20:02.459 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-24T23:20:02.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-24T23:20:02.592 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:20:02.592 INFO:tasks.ceph:role: osd.2 2024-04-24T23:20:02.592 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:20:02.593 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T23:20:02.656 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:20:02.656 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:20:02.656 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:20:02.657 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:20:02.657 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:20:02.657 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:20:02.657 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:20:02.657 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:20:02.657 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:20:02.657 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:20:02.688 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-04-24T23:20:02.690 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-04-24T23:20:02.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-24T23:20:02.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-24T23:20:02.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-24T23:20:02.891 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:20:02.892 INFO:tasks.ceph:role: osd.3 2024-04-24T23:20:02.892 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:20:02.892 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T23:20:02.955 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:20:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:20:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:20:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:20:02.956 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:20:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:20:02.956 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:20:02.956 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:20:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:20:02.956 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:20:02.988 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-04-24T23:20:02.990 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-04-24T23:20:02.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-24T23:20:03.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-24T23:20:03.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:03.216+0000 7f115281e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-24T23:20:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:03.216+0000 7f115281e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-24T23:20:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:03.216+0000 7f115281e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T23:20:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:03.216+0000 7f115281e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T23:20:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:03.216+0000 7f115281e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T23:20:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:03.217+0000 7f115281e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-24T23:20:06.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T23:20:06.095 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:06.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:06.183+0000 7f48d69d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-24T23:20:06.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:06.183+0000 7f48d69d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-24T23:20:06.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:06.183+0000 7f48d69d6640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T23:20:06.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:06.183+0000 7f48d69d6640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T23:20:06.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:06.183+0000 7f48d69d6640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T23:20:06.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:06.183+0000 7f48d69d6640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-24T23:20:09.032 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T23:20:09.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:09.150+0000 7f7132a21640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-24T23:20:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:09.151+0000 7f7132a21640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-24T23:20:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:09.151+0000 7f7132a21640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T23:20:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:09.151+0000 7f7132a21640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T23:20:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:09.151+0000 7f7132a21640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T23:20:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:09.151+0000 7f7132a21640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-24T23:20:11.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T23:20:12.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:12.119+0000 7f3de6e18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-24T23:20:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:12.119+0000 7f3de6e18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-24T23:20:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:12.119+0000 7f3de6e18640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T23:20:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:12.119+0000 7f3de6e18640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T23:20:12.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:12.119+0000 7f3de6e18640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T23:20:12.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:12.119+0000 7f3de6e18640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-24T23:20:14.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T23:20:14.997 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-24T23:20:15.027 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:20:15.027 INFO:tasks.ceph:role: osd.4 2024-04-24T23:20:15.027 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:20:15.027 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:20:15.096 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:20:15.128 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-04-24T23:20:15.130 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-04-24T23:20:15.131 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-24T23:20:15.202 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-24T23:20:15.274 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-24T23:20:15.340 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:20:15.340 INFO:tasks.ceph:role: osd.5 2024-04-24T23:20:15.340 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:20:15.340 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:20:15.405 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:20:15.437 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-04-24T23:20:15.439 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-04-24T23:20:15.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-24T23:20:15.507 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-24T23:20:15.576 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-24T23:20:15.640 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:20:15.641 INFO:tasks.ceph:role: osd.6 2024-04-24T23:20:15.641 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:20:15.641 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:20:15.704 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:20:15.736 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-04-24T23:20:15.738 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-04-24T23:20:15.738 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-24T23:20:15.806 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-24T23:20:15.875 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-24T23:20:15.939 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:20:15.939 INFO:tasks.ceph:role: osd.7 2024-04-24T23:20:15.939 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:20:15.939 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T23:20:16.003 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:20:16.003 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:20:16.003 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:20:16.003 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:20:16.004 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:20:16.004 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:20:16.004 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:20:16.004 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:20:16.004 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:20:16.004 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:20:16.035 INFO:teuthology.orchestra.run.smithi029.stdout:Discarding blocks...Done. 2024-04-24T23:20:16.037 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-04-24T23:20:16.037 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-24T23:20:16.105 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-24T23:20:16.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:16.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:16.261+0000 7f74fb00d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-24T23:20:16.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:16.261+0000 7f74fb00d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-24T23:20:16.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:16.261+0000 7f74fb00d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T23:20:16.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:16.261+0000 7f74fb00d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T23:20:16.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:16.261+0000 7f74fb00d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T23:20:16.264 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:16.261+0000 7f74fb00d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-24T23:20:19.116 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T23:20:19.146 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:19.233+0000 7f19e7e1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-24T23:20:19.236 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:19.233+0000 7f19e7e1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-24T23:20:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:19.233+0000 7f19e7e1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T23:20:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:19.233+0000 7f19e7e1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T23:20:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:19.233+0000 7f19e7e1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T23:20:19.237 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:19.233+0000 7f19e7e1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-24T23:20:22.093 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T23:20:22.122 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:22.209+0000 7fc229414640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-24T23:20:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:22.209+0000 7fc229414640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-24T23:20:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:22.209+0000 7fc229414640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T23:20:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:22.209+0000 7fc229414640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T23:20:22.212 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:22.209+0000 7fc229414640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T23:20:22.213 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:22.210+0000 7fc229414640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-24T23:20:25.064 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-24T23:20:25.093 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:25.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:25.180+0000 7fd731014640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-24T23:20:25.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:25.180+0000 7fd731014640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-24T23:20:25.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:25.180+0000 7fd731014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T23:20:25.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:25.180+0000 7fd731014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T23:20:25.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:25.180+0000 7fd731014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T23:20:25.183 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:25.180+0000 7fd731014640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-24T23:20:28.034 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-24T23:20:28.064 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-24T23:20:28.065 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:28.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-24T23:20:28.094 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:28.095 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-24T23:20:28.164 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:28.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-24T23:20:28.231 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:28.231 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-24T23:20:28.311 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:28.311 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-24T23:20:28.386 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:28.386 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-24T23:20:28.453 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:28.454 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-24T23:20:28.523 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:28.524 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-24T23:20:28.554 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:28.554 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-24T23:20:28.624 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:28.624 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-24T23:20:28.692 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:28.692 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-24T23:20:28.759 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:28.759 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-24T23:20:28.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:20:28.816 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:20:28.825 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:28.826 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-24T23:20:28.928 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:28.928 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-24T23:20:28.993 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:28.993 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-24T23:20:29.010 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:29.010 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-24T23:20:29.066 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:29.066 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-24T23:20:29.081 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:29.082 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-24T23:20:29.139 INFO:tasks.ceph:Adding keys to all mons... 2024-04-24T23:20:29.139 DEBUG:teuthology.orchestra.run.smithi026:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T23:20:29.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.y] 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmqYXWDBAAnuAn2h4Gb9qSFBoAZX73qQ== 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmg5ZUFBAAkapM9La3f+wSyLczFkm/0w== 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmStpLGxAAz7molt8venundBs+CcFSbg== 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.0] 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCjkylmQt74DBAAZLk9JTH3qTKRouUawqNURw== 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.1] 2024-04-24T23:20:29.183 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCmkylmi6IJCxAAWBnNl2y0XIUW/YoAIt6VIQ== 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.2] 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCpkylmRcgOCRAA79pdcH0OlB2VROLSRE4dcw== 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.3] 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCskylm8SopBxAArvUph95+zD5/0LmNM8WPIA== 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.x] 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmq+8yERAAMKjK+5f2Qr35ral8L2qJMw== 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmPS3cHxAASacBQN+95Ds6N5668Je+DA== 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylml0TBJhAAXQkjExbw1EXc90XlcAEBBw== 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.4] 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCwkylmbHipDxAAVUI9OpiMFiM/HFo03fEzSA== 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.5] 2024-04-24T23:20:29.184 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCzkylmDm4CDhAA1T/X+ywXEUCL96OGyi6S9Q== 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.6] 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC2kylmwWyVDBAAJsFFLltURqVqGLNjK9OCJQ== 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.7] 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC5kylm/nLXChAA6kk0UI2hchkN2lhPO1sXHw== 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ== 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout:[client.2] 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmjnJGMBAAvwAmiHrkGkDFTeL/WZbdWw== 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout:[client.1] 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylm/BVuMxAAAdEb6dd3Q5eMc+I3oszOyg== 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout:[client.3] 2024-04-24T23:20:29.185 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmPh3qOBAAa4cbQeF96mdpgNVb/l3xPg== 2024-04-24T23:20:29.202 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr.y] 2024-04-24T23:20:29.202 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylmqYXWDBAAnuAn2h4Gb9qSFBoAZX73qQ== 2024-04-24T23:20:29.202 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.a] 2024-04-24T23:20:29.202 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylmg5ZUFBAAkapM9La3f+wSyLczFkm/0w== 2024-04-24T23:20:29.202 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.c] 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylmStpLGxAAz7molt8venundBs+CcFSbg== 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.0] 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQCjkylmQt74DBAAZLk9JTH3qTKRouUawqNURw== 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.1] 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQCmkylmi6IJCxAAWBnNl2y0XIUW/YoAIt6VIQ== 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.2] 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQCpkylmRcgOCRAA79pdcH0OlB2VROLSRE4dcw== 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.3] 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQCskylm8SopBxAArvUph95+zD5/0LmNM8WPIA== 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr.x] 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylmq+8yERAAMKjK+5f2Qr35ral8L2qJMw== 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.b] 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylmPS3cHxAASacBQN+95Ds6N5668Je+DA== 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout:[mds.d] 2024-04-24T23:20:29.203 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylml0TBJhAAXQkjExbw1EXc90XlcAEBBw== 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.4] 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQCwkylmbHipDxAAVUI9OpiMFiM/HFo03fEzSA== 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.5] 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQCzkylmDm4CDhAA1T/X+ywXEUCL96OGyi6S9Q== 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.6] 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQC2kylmwWyVDBAAJsFFLltURqVqGLNjK9OCJQ== 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.7] 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQC5kylm/nLXChAA6kk0UI2hchkN2lhPO1sXHw== 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ== 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout:[client.2] 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylmjnJGMBAAvwAmiHrkGkDFTeL/WZbdWw== 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout:[client.1] 2024-04-24T23:20:29.204 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylm/BVuMxAAAdEb6dd3Q5eMc+I3oszOyg== 2024-04-24T23:20:29.205 INFO:teuthology.orchestra.run.smithi029.stdout:[client.3] 2024-04-24T23:20:29.205 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQChkylmPh3qOBAAa4cbQeF96mdpgNVb/l3xPg== 2024-04-24T23:20:29.206 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:20:29.229 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:20:29.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:20:29.299 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:20:29.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:20:29.354 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:20:29.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.409 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.459 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.462 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.514 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.567 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:20:29.619 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:20:29.668 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:20:29.670 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:20:29.723 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:20:29.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:20:29.779 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.781 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.833 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.885 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.938 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:20:29.988 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:20:29.991 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:20:30.044 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:20:30.046 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:20:30.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:20:30.101 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:20:30.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:20:30.158 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:20:30.210 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-24T23:20:30.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-24T23:20:30.235 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T23:20:30.329 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-24T23:20:30.354 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-24T23:20:30.379 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T23:20:30.474 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-24T23:20:30.498 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-24T23:20:30.562 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T23:20:30.656 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-24T23:20:30.682 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:30.685 DEBUG:teuthology.orchestra.run.smithi029:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:20:30.737 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-24T23:20:30.737 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T23:20:30.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-24T23:20:30.741 INFO:tasks.ceph.mon.a:Started 2024-04-24T23:20:30.741 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T23:20:30.741 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-24T23:20:30.780 INFO:tasks.ceph.mon.b:Started 2024-04-24T23:20:30.780 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T23:20:30.780 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T23:20:30.782 INFO:tasks.ceph.mon.c:Started 2024-04-24T23:20:30.782 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-24T23:20:30.782 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-24T23:20:30.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-24T23:20:30.784 INFO:tasks.ceph.mgr.y:Started 2024-04-24T23:20:30.785 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-24T23:20:30.785 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-24T23:20:30.786 INFO:tasks.ceph.mgr.x:Started 2024-04-24T23:20:30.787 INFO:tasks.ceph:Setting crush tunables to default 2024-04-24T23:20:30.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph osd crush tunables default 2024-04-24T23:20:30.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.885+0000 7fb4e891b640 1 Processor -- start 2024-04-24T23:20:30.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.885+0000 7fb4e891b640 1 -- start start 2024-04-24T23:20:30.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e891b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb4e415e310 0x7fb4e4162830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:30.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e891b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 0x7fb4e406cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e891b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4e415f700 con 0x7fb4e406d180 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e891b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e40b3740 con 0x7fb4e415e310 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e891b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e40d0310 con 0x7fb4e406c440 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb4e406d180 0x7fb4e40ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44280/0 (socket says 172.21.15.26:44280) 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e2d9d640 1 -- 172.21.15.26:0/4264590263 learned_addr learned my addr 172.21.15.26:0/4264590263 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e259c640 1 --2- 172.21.15.26:0/4264590263 >> v2:172.21.15.29:3300/0 conn(0x7fb4e415e310 0x7fb4e4162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4dbfff640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 0x7fb4e406cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e259c640 1 -- 172.21.15.26:0/4264590263 >> v2:172.21.15.29:3300/0 conn(0x7fb4e415e310 msgr2=0x7fb4e4162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4dbfff640 1 -- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 msgr2=0x7fb4e406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e259c640 1 -- 172.21.15.26:0/4264590263 >> v2:172.21.15.29:3300/0 conn(0x7fb4e415e310 msgr2=0x7fb4e4162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:20:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4dbfff640 1 -- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 msgr2=0x7fb4e406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:20:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4dbfff640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 0x7fb4e406cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:20:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e259c640 1 --2- 172.21.15.26:0/4264590263 >> v2:172.21.15.29:3300/0 conn(0x7fb4e415e310 0x7fb4e4162830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:20:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4dbfff640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 0x7fb4e406cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:20:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:30.886+0000 7fb4e259c640 1 --2- 172.21.15.26:0/4264590263 >> v2:172.21.15.29:3300/0 conn(0x7fb4e415e310 0x7fb4e4162830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:20:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508887610 0 0) 0x7fb4e415f700 con 0x7fb4e406d180 2024-04-24T23:20:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4d0003520 con 0x7fb4e406d180 2024-04-24T23:20:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3408525137 0 0) 0x7fb4d0003520 con 0x7fb4e406d180 2024-04-24T23:20:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 msgr2=0x7fb4e406cc40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:20:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 0x7fb4e406cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 >> v2:172.21.15.29:3300/0 conn(0x7fb4e415e310 msgr2=0x7fb4e4162830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:20:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 --2- 172.21.15.26:0/4264590263 >> v2:172.21.15.29:3300/0 conn(0x7fb4e415e310 0x7fb4e4162830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e40d05f0 con 0x7fb4e406d180 2024-04-24T23:20:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fb4dc0019f0 con 0x7fb4e406d180 2024-04-24T23:20:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 >> v1:172.21.15.26:6789/0 conn(0x7fb4e406d180 legacy=0x7fb4e40ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 0x7fb4d00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0003fe0 0x7fb4d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e40b3740 con 0x7fb4d0004d40 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e40d0310 con 0x7fb4d0003fe0 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d000bac0 con 0x7fb4e406c440 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e2d9d640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4dbfff640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0003fe0 0x7fb4d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.040+0000 7fb4e259c640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 0x7fb4d00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.041+0000 7fb4e2d9d640 1 -- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 msgr2=0x7fb4d00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.041+0000 7fb4e2d9d640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e406c440 0x7fb4d00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.041+0000 7fb4e2d9d640 1 -- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0003fe0 msgr2=0x7fb4d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.041+0000 7fb4e2d9d640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d0003fe0 0x7fb4d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.041+0000 7fb4e2d9d640 1 -- 172.21.15.26:0/4264590263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d0003520 con 0x7fb4d0004d40 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.041+0000 7fb4e2d9d640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4d00051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00a8a0 tx=0x7fb4d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab1d20335e0e9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.041+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00a4c0 con 0x7fb4d0004d40 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.041+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4dc02f070 con 0x7fb4d0004d40 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.041+0000 7fb4e1d9b640 1 -- 172.21.15.26:0/4264590263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc02bd70 con 0x7fb4d0004d40 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.042+0000 7fb4e891b640 1 -- 172.21.15.26:0/4264590263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 msgr2=0x7fb4d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.042+0000 7fb4e891b640 1 --2- 172.21.15.26:0/4264590263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4d00051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00a8a0 tx=0x7fb4d0005830 comp rx=0 tx=0).stop 2024-04-24T23:20:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.042+0000 7fb4e2d9d640 1 -- 172.21.15.26:0/4264590263 reap_dead start 2024-04-24T23:20:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.042+0000 7fb4e891b640 1 -- 172.21.15.26:0/4264590263 shutdown_connections 2024-04-24T23:20:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.042+0000 7fb4e891b640 1 -- 172.21.15.26:0/4264590263 >> 172.21.15.26:0/4264590263 conn(0x7fb4e40d1dd0 msgr2=0x7fb4e40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.042+0000 7fb4e891b640 1 -- 172.21.15.26:0/4264590263 shutdown_connections 2024-04-24T23:20:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.042+0000 7fb4e891b640 1 -- 172.21.15.26:0/4264590263 wait complete. 2024-04-24T23:20:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.044+0000 7fb4e891b640 1 Processor -- start 2024-04-24T23:20:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.044+0000 7fb4e891b640 1 -- start start 2024-04-24T23:20:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.044+0000 7fb4e891b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4e4151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.044+0000 7fb4e891b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4e41522d0 0x7fb4e41e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.044+0000 7fb4e891b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e41e9230 0x7fb4e41e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.044+0000 7fb4e891b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e40d0e90 con 0x7fb4d0004d40 2024-04-24T23:20:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.044+0000 7fb4e891b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e40d0bb0 con 0x7fb4e41522d0 2024-04-24T23:20:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.044+0000 7fb4e891b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e4162d70 con 0x7fb4e41e9230 2024-04-24T23:20:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.044+0000 7fb4e259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4e4151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4e4151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38116/0 (socket says 172.21.15.26:38116) 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e259c640 1 -- 172.21.15.26:0/3562834095 learned_addr learned my addr 172.21.15.26:0/3562834095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e2d9d640 1 --2- 172.21.15.26:0/3562834095 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e41e9230 0x7fb4e41e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4dbfff640 1 --2- 172.21.15.26:0/3562834095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4e41522d0 0x7fb4e41e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e259c640 1 -- 172.21.15.26:0/3562834095 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e41e9230 msgr2=0x7fb4e41e96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e259c640 1 --2- 172.21.15.26:0/3562834095 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e41e9230 0x7fb4e41e96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e259c640 1 -- 172.21.15.26:0/3562834095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4e41522d0 msgr2=0x7fb4e41e8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e259c640 1 --2- 172.21.15.26:0/3562834095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4e41522d0 0x7fb4e41e8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e259c640 1 -- 172.21.15.26:0/3562834095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e4160100 con 0x7fb4d0004d40 2024-04-24T23:20:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e259c640 1 --2- 172.21.15.26:0/3562834095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4e4151d90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4010980 tx=0x7fb4d4010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4daffd640 1 -- 172.21.15.26:0/3562834095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d401a020 con 0x7fb4d0004d40 2024-04-24T23:20:31.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4daffd640 1 -- 172.21.15.26:0/3562834095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d400ea20 con 0x7fb4d0004d40 2024-04-24T23:20:31.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e891b640 1 -- 172.21.15.26:0/3562834095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e41603e0 con 0x7fb4d0004d40 2024-04-24T23:20:31.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4daffd640 1 -- 172.21.15.26:0/3562834095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400edf0 con 0x7fb4d0004d40 2024-04-24T23:20:31.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4e891b640 1 -- 172.21.15.26:0/3562834095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e4154320 con 0x7fb4d0004d40 2024-04-24T23:20:31.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4daffd640 1 -- 172.21.15.26:0/3562834095 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb4d401d020 con 0x7fb4d0004d40 2024-04-24T23:20:31.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.045+0000 7fb4daffd640 1 -- 172.21.15.26:0/3562834095 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb4d40149b0 con 0x7fb4d0004d40 2024-04-24T23:20:31.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.046+0000 7fb4e891b640 1 -- 172.21.15.26:0/3562834095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e41526b0 con 0x7fb4d0004d40 2024-04-24T23:20:31.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.048+0000 7fb4daffd640 1 -- 172.21.15.26:0/3562834095 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb4e41526b0 con 0x7fb4d0004d40 2024-04-24T23:20:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.116+0000 7fb4e891b640 1 -- 172.21.15.26:0/3562834095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb4e4075610 con 0x7fb4d0004d40 2024-04-24T23:20:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.117+0000 7fb4daffd640 1 -- 172.21.15.26:0/3562834095 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fb4d4024020 con 0x7fb4d0004d40 2024-04-24T23:20:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-04-24T23:20:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.118+0000 7fb4e891b640 1 -- 172.21.15.26:0/3562834095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 msgr2=0x7fb4e4151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.118+0000 7fb4e891b640 1 --2- 172.21.15.26:0/3562834095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4e4151d90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4010980 tx=0x7fb4d4010e40 comp rx=0 tx=0).stop 2024-04-24T23:20:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.118+0000 7fb4e891b640 1 -- 172.21.15.26:0/3562834095 shutdown_connections 2024-04-24T23:20:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.118+0000 7fb4e891b640 1 --2- 172.21.15.26:0/3562834095 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4e41e9230 0x7fb4e41e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.118+0000 7fb4e891b640 1 --2- 172.21.15.26:0/3562834095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4e41522d0 0x7fb4e41e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.118+0000 7fb4e891b640 1 --2- 172.21.15.26:0/3562834095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d0004d40 0x7fb4e4151d90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.118+0000 7fb4e891b640 1 -- 172.21.15.26:0/3562834095 >> 172.21.15.26:0/3562834095 conn(0x7fb4e40d1dd0 msgr2=0x7fb4e40aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.118+0000 7fb4e891b640 1 -- 172.21.15.26:0/3562834095 shutdown_connections 2024-04-24T23:20:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:31.118+0000 7fb4e891b640 1 -- 172.21.15.26:0/3562834095 wait complete. 2024-04-24T23:20:31.131 INFO:tasks.ceph:check_enable_crimson: False 2024-04-24T23:20:31.131 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-24T23:20:31.131 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:31.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-24T23:20:31.156 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:31.156 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-24T23:20:31.160 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:31.158+0000 7facbe321040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:20:31.174 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:31.172+0000 7f60b4d28040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:20:31.198 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:31.198 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-24T23:20:31.265 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:20:31.265 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-24T23:20:31.322 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:31.321+0000 7facbe321040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:20:31.330 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:31.330 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-24T23:20:31.331 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:31.330+0000 7f60b4d28040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:20:31.355 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:31.355 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-24T23:20:31.379 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:31.378+0000 7facbe321040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:20:31.388 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:31.386+0000 7f60b4d28040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:20:31.404 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:31.404 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-24T23:20:31.469 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:20:31.469 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-24T23:20:31.533 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd new dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f 0 2024-04-24T23:20:31.639 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:31.637+0000 7facbe321040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:20:31.639 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:31.637+0000 7f60b4d28040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:20:31.673 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.672+0000 7f5e3a929640 1 Processor -- start 2024-04-24T23:20:31.674 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.672+0000 7f5e3a929640 1 -- start start 2024-04-24T23:20:31.675 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e3a929640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e3415e370 0x7f5e340d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e3a929640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e340cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e3a929640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e340d2120 con 0x7f5e3415e750 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e3a929640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5e340d1510 con 0x7f5e3415e370 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e3a929640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5e340d1ed0 con 0x7f5e340d4c10 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e38ec6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e340cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e337fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f5e3415e750 0x7f5e340b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57380/0 (socket says 172.21.15.29:57380) 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e337fe640 1 -- 172.21.15.29:0/1634224992 learned_addr learned my addr 172.21.15.29:0/1634224992 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e38ec6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e340cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60414/0 (socket says 172.21.15.29:60414) 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e33fff640 1 --2- 172.21.15.29:0/1634224992 >> v2:172.21.15.26:3300/0 conn(0x7f5e3415e370 0x7f5e340d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e32ffd640 1 -- 172.21.15.29:0/1634224992 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 714120024 0 0) 0x7f5e340d2120 con 0x7f5e3415e750 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e38ec6640 1 -- 172.21.15.29:0/1634224992 >> v2:172.21.15.26:3300/0 conn(0x7f5e3415e370 msgr2=0x7f5e340d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e38ec6640 1 --2- 172.21.15.29:0/1634224992 >> v2:172.21.15.26:3300/0 conn(0x7f5e3415e370 0x7f5e340d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e38ec6640 1 -- 172.21.15.29:0/1634224992 >> v1:172.21.15.29:6789/0 conn(0x7f5e3415e750 legacy=0x7f5e340b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.676 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.673+0000 7f5e38ec6640 1 -- 172.21.15.29:0/1634224992 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e340b3740 con 0x7f5e340d4c10 2024-04-24T23:20:31.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.674+0000 7f5e38ec6640 1 --2- 172.21.15.29:0/1634224992 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e340cfca0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009d90 tx=0x7f5e2c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caff82b1edc03e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.674+0000 7f5e32ffd640 1 -- 172.21.15.29:0/1634224992 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c007740 con 0x7f5e340d4c10 2024-04-24T23:20:31.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.674+0000 7f5e32ffd640 1 -- 172.21.15.29:0/1634224992 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2c0078e0 con 0x7f5e340d4c10 2024-04-24T23:20:31.677 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.674+0000 7f5e32ffd640 1 -- 172.21.15.29:0/1634224992 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c007cb0 con 0x7f5e340d4c10 2024-04-24T23:20:31.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.675+0000 7f5e3a929640 1 -- 172.21.15.29:0/1634224992 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 msgr2=0x7f5e340cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.675+0000 7f5e3a929640 1 --2- 172.21.15.29:0/1634224992 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e340cfca0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009d90 tx=0x7f5e2c02a660 comp rx=0 tx=0).stop 2024-04-24T23:20:31.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.675+0000 7f5e3a929640 1 -- 172.21.15.29:0/1634224992 shutdown_connections 2024-04-24T23:20:31.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.675+0000 7f5e3a929640 1 --2- 172.21.15.29:0/1634224992 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e340cfca0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.675+0000 7f5e3a929640 1 --2- 172.21.15.29:0/1634224992 >> v2:172.21.15.26:3300/0 conn(0x7f5e3415e370 0x7f5e340d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.675+0000 7f5e3a929640 1 -- 172.21.15.29:0/1634224992 >> 172.21.15.29:0/1634224992 conn(0x7f5e340d40f0 msgr2=0x7f5e340d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.678 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.675+0000 7f5e3a929640 1 -- 172.21.15.29:0/1634224992 shutdown_connections 2024-04-24T23:20:31.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.675+0000 7f5e3a929640 1 -- 172.21.15.29:0/1634224992 wait complete. 2024-04-24T23:20:31.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.676+0000 7f5e3a929640 1 Processor -- start 2024-04-24T23:20:31.679 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.676+0000 7f5e3a929640 1 -- start start 2024-04-24T23:20:31.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e3a929640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e3415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e3a929640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e3415e370 0x7f5e340c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e3a929640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e340ca130 0x7f5e340cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e3a929640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e340d2a60 con 0x7f5e3415e370 2024-04-24T23:20:31.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e3a929640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5e340b3d00 con 0x7f5e340ca130 2024-04-24T23:20:31.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e3a929640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5e340d2be0 con 0x7f5e340d4c10 2024-04-24T23:20:31.681 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e33fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e3415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e38ec6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e340ca130 0x7f5e340cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e38ec6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e340ca130 0x7f5e340cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44368/0 (socket says 172.21.15.29:44368) 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e38ec6640 1 -- 172.21.15.29:0/2235232168 learned_addr learned my addr 172.21.15.29:0/2235232168 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e3415e370 0x7f5e340c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e33fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e3415b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60418/0 (socket says 172.21.15.29:60418) 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e33fff640 1 -- 172.21.15.29:0/2235232168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e340ca130 msgr2=0x7f5e340cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e33fff640 1 --2- 172.21.15.29:0/2235232168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e340ca130 0x7f5e340cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e33fff640 1 -- 172.21.15.29:0/2235232168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e3415e370 msgr2=0x7f5e340c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e33fff640 1 --2- 172.21.15.29:0/2235232168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e3415e370 0x7f5e340c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e33fff640 1 -- 172.21.15.29:0/2235232168 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3415bd60 con 0x7f5e340d4c10 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e38ec6640 1 --2- 172.21.15.29:0/2235232168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e340ca130 0x7f5e340cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.677+0000 7f5e337fe640 1 --2- 172.21.15.29:0/2235232168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e3415e370 0x7f5e340c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:31.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.678+0000 7f5e33fff640 1 --2- 172.21.15.29:0/2235232168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e3415b820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e880 tx=0x7f5e2800ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.678+0000 7f5e30ff9640 1 -- 172.21.15.29:0/2235232168 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28017070 con 0x7f5e340d4c10 2024-04-24T23:20:31.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.678+0000 7f5e30ff9640 1 -- 172.21.15.29:0/2235232168 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2800c940 con 0x7f5e340d4c10 2024-04-24T23:20:31.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.678+0000 7f5e30ff9640 1 -- 172.21.15.29:0/2235232168 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800cd30 con 0x7f5e340d4c10 2024-04-24T23:20:31.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.678+0000 7f5e3a929640 1 -- 172.21.15.29:0/2235232168 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e340ca640 con 0x7f5e340d4c10 2024-04-24T23:20:31.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.678+0000 7f5e3a929640 1 -- 172.21.15.29:0/2235232168 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e340cc760 con 0x7f5e340d4c10 2024-04-24T23:20:31.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.678+0000 7f5e30ff9640 1 -- 172.21.15.29:0/2235232168 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5e2801a070 con 0x7f5e340d4c10 2024-04-24T23:20:31.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.678+0000 7f5e30ff9640 1 -- 172.21.15.29:0/2235232168 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f5e28013470 con 0x7f5e340d4c10 2024-04-24T23:20:31.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.679+0000 7f5e3a929640 1 -- 172.21.15.29:0/2235232168 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e34075610 con 0x7f5e340d4c10 2024-04-24T23:20:31.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.683+0000 7f5e30ff9640 1 -- 172.21.15.29:0/2235232168 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5e280136c0 con 0x7f5e340d4c10 2024-04-24T23:20:31.687 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:31.686+0000 7f60b4d28040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:20:31.690 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:31.689+0000 7facbe321040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:20:31.758 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.756+0000 7f5e3a929640 1 -- 172.21.15.29:0/2235232168 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f", "id": 0} v 0) -- 0x7f5e340ce5b0 con 0x7f5e340d4c10 2024-04-24T23:20:31.761 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.760+0000 7f5e30ff9640 1 -- 172.21.15.29:0/2235232168 <== mon.2 v2:172.21.15.29:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f5e280138a0 con 0x7f5e340d4c10 2024-04-24T23:20:31.761 INFO:teuthology.orchestra.run.smithi029.stdout:0 2024-04-24T23:20:31.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.761+0000 7f5e3a929640 1 -- 172.21.15.29:0/2235232168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 msgr2=0x7f5e3415b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.761+0000 7f5e3a929640 1 --2- 172.21.15.29:0/2235232168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e3415b820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e880 tx=0x7f5e2800ed40 comp rx=0 tx=0).stop 2024-04-24T23:20:31.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.761+0000 7f5e3a929640 1 -- 172.21.15.29:0/2235232168 shutdown_connections 2024-04-24T23:20:31.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.761+0000 7f5e3a929640 1 --2- 172.21.15.29:0/2235232168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e340ca130 0x7f5e340cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.761+0000 7f5e3a929640 1 --2- 172.21.15.29:0/2235232168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e3415e370 0x7f5e340c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.761+0000 7f5e3a929640 1 --2- 172.21.15.29:0/2235232168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e340d4c10 0x7f5e3415b820 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.763 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.761+0000 7f5e3a929640 1 -- 172.21.15.29:0/2235232168 >> 172.21.15.29:0/2235232168 conn(0x7f5e340d40f0 msgr2=0x7f5e3406d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.761+0000 7f5e3a929640 1 -- 172.21.15.29:0/2235232168 shutdown_connections 2024-04-24T23:20:31.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.762+0000 7f5e3a929640 1 -- 172.21.15.29:0/2235232168 wait complete. 2024-04-24T23:20:31.778 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd new 6a144c00-9fcd-45b9-a22b-6964b76e7d44 1 2024-04-24T23:20:31.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.876+0000 7f1e6d7b0640 1 Processor -- start 2024-04-24T23:20:31.878 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.876+0000 7f1e6d7b0640 1 -- start start 2024-04-24T23:20:31.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.877+0000 7f1e6d7b0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e6815b760 0x7f1e6815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.877+0000 7f1e6d7b0640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e6815c080 0x7f1e6806c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.877+0000 7f1e6d7b0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e680b4a90 con 0x7f1e6806c670 2024-04-24T23:20:31.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.877+0000 7f1e6d7b0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6806cae0 con 0x7f1e6815b760 2024-04-24T23:20:31.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.877+0000 7f1e6d7b0640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1e680b5610 con 0x7f1e6815c080 2024-04-24T23:20:31.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.877+0000 7f1e667fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e6815c080 0x7f1e6806c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.877+0000 7f1e677fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f1e6806c670 0x7f1e681516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57388/0 (socket says 172.21.15.29:57388) 2024-04-24T23:20:31.880 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.877+0000 7f1e667fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e6815c080 0x7f1e6806c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44372/0 (socket says 172.21.15.29:44372) 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.877+0000 7f1e677fe640 1 -- 172.21.15.29:0/1897786189 learned_addr learned my addr 172.21.15.29:0/1897786189 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.878+0000 7f1e66ffd640 1 --2- 172.21.15.29:0/1897786189 >> v2:172.21.15.26:3300/0 conn(0x7f1e6815b760 0x7f1e6815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.878+0000 7f1e667fc640 1 -- 172.21.15.29:0/1897786189 >> v2:172.21.15.26:3300/0 conn(0x7f1e6815b760 msgr2=0x7f1e6815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.878+0000 7f1e667fc640 1 --2- 172.21.15.29:0/1897786189 >> v2:172.21.15.26:3300/0 conn(0x7f1e6815b760 0x7f1e6815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.878+0000 7f1e667fc640 1 -- 172.21.15.29:0/1897786189 >> v1:172.21.15.29:6789/0 conn(0x7f1e6806c670 legacy=0x7f1e681516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.878+0000 7f1e65ffb640 1 -- 172.21.15.29:0/1897786189 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2011908588 0 0) 0x7f1e680b4a90 con 0x7f1e6806c670 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.878+0000 7f1e667fc640 1 -- 172.21.15.29:0/1897786189 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e680b4d70 con 0x7f1e6815c080 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.878+0000 7f1e667fc640 1 --2- 172.21.15.29:0/1897786189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6815c080 0x7f1e6806c130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e680678c0 tx=0x7f1e5802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=233468b97ebdd988 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e65ffb640 1 -- 172.21.15.29:0/1897786189 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58037070 con 0x7f1e6815c080 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e65ffb640 1 -- 172.21.15.29:0/1897786189 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e58032440 con 0x7f1e6815c080 2024-04-24T23:20:31.881 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e65ffb640 1 -- 172.21.15.29:0/1897786189 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58032810 con 0x7f1e6815c080 2024-04-24T23:20:31.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/1897786189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6815c080 msgr2=0x7f1e6806c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e6d7b0640 1 --2- 172.21.15.29:0/1897786189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6815c080 0x7f1e6806c130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e680678c0 tx=0x7f1e5802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/1897786189 shutdown_connections 2024-04-24T23:20:31.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e6d7b0640 1 --2- 172.21.15.29:0/1897786189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6815c080 0x7f1e6806c130 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e6d7b0640 1 --2- 172.21.15.29:0/1897786189 >> v2:172.21.15.26:3300/0 conn(0x7f1e6815b760 0x7f1e6815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.884 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/1897786189 >> 172.21.15.29:0/1897786189 conn(0x7f1e680715b0 msgr2=0x7f1e680719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/1897786189 shutdown_connections 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.879+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/1897786189 wait complete. 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.881+0000 7f1e6d7b0640 1 Processor -- start 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.881+0000 7f1e6d7b0640 1 -- start start 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.881+0000 7f1e6d7b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6806c670 0x7f1e68153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.881+0000 7f1e6d7b0640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e6815b760 0x7f1e68145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.881+0000 7f1e6d7b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6815c080 0x7f1e68146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.881+0000 7f1e6d7b0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e680afa40 con 0x7f1e6815c080 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.881+0000 7f1e6d7b0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e680b5330 con 0x7f1e6806c670 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.881+0000 7f1e6d7b0640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1e680afbc0 con 0x7f1e6815b760 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e66ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6806c670 0x7f1e68153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e667fc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e6815b760 0x7f1e68145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e66ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6806c670 0x7f1e68153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44374/0 (socket says 172.21.15.29:44374) 2024-04-24T23:20:31.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e66ffd640 1 -- 172.21.15.29:0/3514119780 learned_addr learned my addr 172.21.15.29:0/3514119780 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e677fe640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6815c080 0x7f1e68146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e66ffd640 1 -- 172.21.15.29:0/3514119780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e6815b760 msgr2=0x7f1e68145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e66ffd640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e6815b760 0x7f1e68145ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e66ffd640 1 -- 172.21.15.29:0/3514119780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6815c080 msgr2=0x7f1e68146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e66ffd640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6815c080 0x7f1e68146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e66ffd640 1 -- 172.21.15.29:0/3514119780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6806f030 con 0x7f1e6806c670 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e667fc640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e6815b760 0x7f1e68145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e677fe640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6815c080 0x7f1e68146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e66ffd640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6806c670 0x7f1e68153ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00e9b0 tx=0x7f1e5c00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e4f7fe640 1 -- 172.21.15.29:0/3514119780 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c017070 con 0x7f1e6806c670 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e4f7fe640 1 -- 172.21.15.29:0/3514119780 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c00c9a0 con 0x7f1e6806c670 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e4f7fe640 1 -- 172.21.15.29:0/3514119780 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00cd90 con 0x7f1e6806c670 2024-04-24T23:20:31.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/3514119780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6806f310 con 0x7f1e6806c670 2024-04-24T23:20:31.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.882+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/3514119780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e680b1780 con 0x7f1e6806c670 2024-04-24T23:20:31.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.883+0000 7f1e4f7fe640 1 -- 172.21.15.29:0/3514119780 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e5c01a070 con 0x7f1e6806c670 2024-04-24T23:20:31.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.883+0000 7f1e4f7fe640 1 -- 172.21.15.29:0/3514119780 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f1e5c01c940 con 0x7f1e6806c670 2024-04-24T23:20:31.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.883+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/3514119780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6806f4e0 con 0x7f1e6806c670 2024-04-24T23:20:31.887 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.886+0000 7f1e4f7fe640 1 -- 172.21.15.29:0/3514119780 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1e6806f4e0 con 0x7f1e6806c670 2024-04-24T23:20:31.960 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.959+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/3514119780 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6a144c00-9fcd-45b9-a22b-6964b76e7d44", "id": 1} v 0) -- 0x7f1e680b1290 con 0x7f1e6806c670 2024-04-24T23:20:31.963 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.962+0000 7f1e4f7fe640 1 -- 172.21.15.29:0/3514119780 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6a144c00-9fcd-45b9-a22b-6964b76e7d44", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f1e5c01cd20 con 0x7f1e6806c670 2024-04-24T23:20:31.963 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-24T23:20:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.963+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/3514119780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6806c670 msgr2=0x7f1e68153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.964 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.963+0000 7f1e6d7b0640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6806c670 0x7f1e68153ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00e9b0 tx=0x7f1e5c00ee70 comp rx=0 tx=0).stop 2024-04-24T23:20:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.963+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/3514119780 shutdown_connections 2024-04-24T23:20:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.963+0000 7f1e6d7b0640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6815c080 0x7f1e68146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.963+0000 7f1e6d7b0640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e6815b760 0x7f1e68145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.963+0000 7f1e6d7b0640 1 --2- 172.21.15.29:0/3514119780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e6806c670 0x7f1e68153ff0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.963+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/3514119780 >> 172.21.15.29:0/3514119780 conn(0x7f1e680715b0 msgr2=0x7f1e6806b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.963+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/3514119780 shutdown_connections 2024-04-24T23:20:31.965 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:31.963+0000 7f1e6d7b0640 1 -- 172.21.15.29:0/3514119780 wait complete. 2024-04-24T23:20:31.975 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd new e43870f1-4c8c-4cde-8dcb-3eb9e80cd853 2 2024-04-24T23:20:32.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.074+0000 7f34775ee640 1 Processor -- start 2024-04-24T23:20:32.076 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.075+0000 7f34775ee640 1 -- start start 2024-04-24T23:20:32.077 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.075+0000 7f34775ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f347015e750 0x7f34700cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.075+0000 7f34775ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34700d2c30 0x7f34700cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.075+0000 7f34775ee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34700d2120 con 0x7f347015e370 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.075+0000 7f34775ee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f34700d1510 con 0x7f34700d2c30 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.075+0000 7f34775ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f34700d1ed0 con 0x7f347015e750 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.076+0000 7f3474b89640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f347015e750 0x7f34700cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.076+0000 7f3474b89640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f347015e750 0x7f34700cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60440/0 (socket says 172.21.15.29:60440) 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.076+0000 7f3474b89640 1 -- 172.21.15.29:0/1478226164 learned_addr learned my addr 172.21.15.29:0/1478226164 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.076+0000 7f3475b8b640 1 --2- 172.21.15.29:0/1478226164 >> v2:172.21.15.26:3300/0 conn(0x7f34700d2c30 0x7f34700cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.076+0000 7f3474b89640 1 -- 172.21.15.29:0/1478226164 >> v2:172.21.15.26:3300/0 conn(0x7f34700d2c30 msgr2=0x7f34700cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.076+0000 7f3474b89640 1 --2- 172.21.15.29:0/1478226164 >> v2:172.21.15.26:3300/0 conn(0x7f34700d2c30 0x7f34700cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.076+0000 7f3474b89640 1 -- 172.21.15.29:0/1478226164 >> v1:172.21.15.29:6789/0 conn(0x7f347015e370 legacy=0x7f34700d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.076+0000 7f3467fff640 1 -- 172.21.15.29:0/1478226164 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 842781629 0 0) 0x7f34700d2120 con 0x7f347015e370 2024-04-24T23:20:32.078 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.076+0000 7f3474b89640 1 -- 172.21.15.29:0/1478226164 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34700b3740 con 0x7f347015e750 2024-04-24T23:20:32.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f3474b89640 1 --2- 172.21.15.29:0/1478226164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f347015e750 0x7f34700cec90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f34700d2050 tx=0x7f346002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b43de54fc6410c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f3467fff640 1 -- 172.21.15.29:0/1478226164 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460037070 con 0x7f347015e750 2024-04-24T23:20:32.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f3467fff640 1 -- 172.21.15.29:0/1478226164 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3460032440 con 0x7f347015e750 2024-04-24T23:20:32.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f3467fff640 1 -- 172.21.15.29:0/1478226164 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460032810 con 0x7f347015e750 2024-04-24T23:20:32.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f34775ee640 1 -- 172.21.15.29:0/1478226164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f347015e750 msgr2=0x7f34700cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f34775ee640 1 --2- 172.21.15.29:0/1478226164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f347015e750 0x7f34700cec90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f34700d2050 tx=0x7f346002a250 comp rx=0 tx=0).stop 2024-04-24T23:20:32.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f34775ee640 1 -- 172.21.15.29:0/1478226164 shutdown_connections 2024-04-24T23:20:32.080 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f34775ee640 1 --2- 172.21.15.29:0/1478226164 >> v2:172.21.15.26:3300/0 conn(0x7f34700d2c30 0x7f34700cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f34775ee640 1 --2- 172.21.15.29:0/1478226164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f347015e750 0x7f34700cec90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.077+0000 7f34775ee640 1 -- 172.21.15.29:0/1478226164 >> 172.21.15.29:0/1478226164 conn(0x7f34700d40f0 msgr2=0x7f34700d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:32.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.078+0000 7f34775ee640 1 -- 172.21.15.29:0/1478226164 shutdown_connections 2024-04-24T23:20:32.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.078+0000 7f34775ee640 1 -- 172.21.15.29:0/1478226164 wait complete. 2024-04-24T23:20:32.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.079+0000 7f34775ee640 1 Processor -- start 2024-04-24T23:20:32.081 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.079+0000 7f34775ee640 1 -- start start 2024-04-24T23:20:32.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.079+0000 7f34775ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34700d2c30 0x7f34701f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.079+0000 7f34775ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f347015e370 0x7f34700cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.079+0000 7f34775ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34700cb700 0x7f34700cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.079+0000 7f34775ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34700b5230 con 0x7f34700cb700 2024-04-24T23:20:32.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.079+0000 7f34775ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34700b3d00 con 0x7f347015e370 2024-04-24T23:20:32.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.079+0000 7f34775ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f34700b53b0 con 0x7f34700d2c30 2024-04-24T23:20:32.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f3474b89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f347015e370 0x7f34700cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.083 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f347538a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34700d2c30 0x7f34701f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f3474b89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f347015e370 0x7f34700cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44386/0 (socket says 172.21.15.29:44386) 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f3474b89640 1 -- 172.21.15.29:0/3467519266 learned_addr learned my addr 172.21.15.29:0/3467519266 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f3475b8b640 1 --2- 172.21.15.29:0/3467519266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34700cb700 0x7f34700cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f347538a640 1 -- 172.21.15.29:0/3467519266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f347015e370 msgr2=0x7f34700cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f347538a640 1 --2- 172.21.15.29:0/3467519266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f347015e370 0x7f34700cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f347538a640 1 -- 172.21.15.29:0/3467519266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34700cb700 msgr2=0x7f34700cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f347538a640 1 --2- 172.21.15.29:0/3467519266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34700cb700 0x7f34700cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f347538a640 1 -- 172.21.15.29:0/3467519266 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34700ce210 con 0x7f34700d2c30 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f347538a640 1 --2- 172.21.15.29:0/3467519266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34700d2c30 0x7f34701f4890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f346c00c450 tx=0x7f346c00c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f3465ffb640 1 -- 172.21.15.29:0/3467519266 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c017070 con 0x7f34700d2c30 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f3465ffb640 1 -- 172.21.15.29:0/3467519266 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c0035f0 con 0x7f34700d2c30 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f34775ee640 1 -- 172.21.15.29:0/3467519266 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34700ce4f0 con 0x7f34700d2c30 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f3465ffb640 1 -- 172.21.15.29:0/3467519266 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c0039e0 con 0x7f34700d2c30 2024-04-24T23:20:32.084 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.080+0000 7f34775ee640 1 -- 172.21.15.29:0/3467519266 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f34700c8f20 con 0x7f34700d2c30 2024-04-24T23:20:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.081+0000 7f3465ffb640 1 -- 172.21.15.29:0/3467519266 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f346c01a070 con 0x7f34700d2c30 2024-04-24T23:20:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.081+0000 7f3465ffb640 1 -- 172.21.15.29:0/3467519266 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f346c012820 con 0x7f34700d2c30 2024-04-24T23:20:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.082+0000 7f34775ee640 1 -- 172.21.15.29:0/3467519266 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3470068450 con 0x7f34700d2c30 2024-04-24T23:20:32.085 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.084+0000 7f3465ffb640 1 -- 172.21.15.29:0/3467519266 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3470068450 con 0x7f34700d2c30 2024-04-24T23:20:32.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.158+0000 7f34775ee640 1 -- 172.21.15.29:0/3467519266 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e43870f1-4c8c-4cde-8dcb-3eb9e80cd853", "id": 2} v 0) -- 0x7f3470075610 con 0x7f34700d2c30 2024-04-24T23:20:32.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.161+0000 7f3465ffb640 1 -- 172.21.15.29:0/3467519266 <== mon.2 v2:172.21.15.29:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e43870f1-4c8c-4cde-8dcb-3eb9e80cd853", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f346c021070 con 0x7f34700d2c30 2024-04-24T23:20:32.163 INFO:teuthology.orchestra.run.smithi029.stdout:2 2024-04-24T23:20:32.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.161+0000 7f34775ee640 1 -- 172.21.15.29:0/3467519266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34700d2c30 msgr2=0x7f34701f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.161+0000 7f34775ee640 1 --2- 172.21.15.29:0/3467519266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34700d2c30 0x7f34701f4890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f346c00c450 tx=0x7f346c00c910 comp rx=0 tx=0).stop 2024-04-24T23:20:32.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.162+0000 7f34775ee640 1 -- 172.21.15.29:0/3467519266 shutdown_connections 2024-04-24T23:20:32.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.162+0000 7f34775ee640 1 --2- 172.21.15.29:0/3467519266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34700cb700 0x7f34700cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.162+0000 7f34775ee640 1 --2- 172.21.15.29:0/3467519266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f347015e370 0x7f34700cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.162+0000 7f34775ee640 1 --2- 172.21.15.29:0/3467519266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34700d2c30 0x7f34701f4890 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.162+0000 7f34775ee640 1 -- 172.21.15.29:0/3467519266 >> 172.21.15.29:0/3467519266 conn(0x7f34700d40f0 msgr2=0x7f347006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:32.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.162+0000 7f34775ee640 1 -- 172.21.15.29:0/3467519266 shutdown_connections 2024-04-24T23:20:32.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.162+0000 7f34775ee640 1 -- 172.21.15.29:0/3467519266 wait complete. 2024-04-24T23:20:32.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd new 64ec8a53-51a8-4500-a714-36c1a8f342ac 3 2024-04-24T23:20:32.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.272+0000 7fac8c44d640 1 Processor -- start 2024-04-24T23:20:32.274 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.272+0000 7fac8c44d640 1 -- start start 2024-04-24T23:20:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8c44d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac841475d0 0x7fac840b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8c44d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8406b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8c44d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac84162e00 con 0x7fac84162970 2024-04-24T23:20:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8c44d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fac8406bad0 con 0x7fac841475d0 2024-04-24T23:20:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8c44d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fac840d0db0 con 0x7fac8406b120 2024-04-24T23:20:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8a1e9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac841475d0 0x7fac840b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.276 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8a9ea640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8406b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8a1e9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac841475d0 0x7fac840b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44398/0 (socket says 172.21.15.29:44398) 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac899e8640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fac84162970 0x7fac840d06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57410/0 (socket says 172.21.15.29:57410) 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8a1e9640 1 -- 172.21.15.29:0/3164752272 learned_addr learned my addr 172.21.15.29:0/3164752272 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.273+0000 7fac8a9ea640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8406b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60468/0 (socket says 172.21.15.29:60468) 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.274+0000 7fac891e7640 1 -- 172.21.15.29:0/3164752272 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1804154516 0 0) 0x7fac84162e00 con 0x7fac84162970 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.274+0000 7fac891e7640 1 -- 172.21.15.29:0/3164752272 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac6c003520 con 0x7fac84162970 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.274+0000 7fac8a1e9640 1 -- 172.21.15.29:0/3164752272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 msgr2=0x7fac8406b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.274+0000 7fac8a1e9640 1 --2- 172.21.15.29:0/3164752272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8406b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.274+0000 7fac8a1e9640 1 -- 172.21.15.29:0/3164752272 >> v1:172.21.15.29:6789/0 conn(0x7fac84162970 legacy=0x7fac840d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.274+0000 7fac8a1e9640 1 -- 172.21.15.29:0/3164752272 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8406ffa0 con 0x7fac841475d0 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.274+0000 7fac8a9ea640 1 --2- 172.21.15.29:0/3164752272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:32.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.274+0000 7fac8a1e9640 1 --2- 172.21.15.29:0/3164752272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 0x7fac840b37f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009870 tx=0x7fac7c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89897bc71c16f525 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac891e7640 1 -- 172.21.15.29:0/3164752272 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c037070 con 0x7fac841475d0 2024-04-24T23:20:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac891e7640 1 -- 172.21.15.29:0/3164752272 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7c032440 con 0x7fac841475d0 2024-04-24T23:20:32.278 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac891e7640 1 -- 172.21.15.29:0/3164752272 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c032810 con 0x7fac841475d0 2024-04-24T23:20:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac8c44d640 1 -- 172.21.15.29:0/3164752272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 msgr2=0x7fac840b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac8c44d640 1 --2- 172.21.15.29:0/3164752272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 0x7fac840b37f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009870 tx=0x7fac7c02a250 comp rx=0 tx=0).stop 2024-04-24T23:20:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac8c44d640 1 -- 172.21.15.29:0/3164752272 shutdown_connections 2024-04-24T23:20:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac8c44d640 1 --2- 172.21.15.29:0/3164752272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac8c44d640 1 --2- 172.21.15.29:0/3164752272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 0x7fac840b37f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac8c44d640 1 -- 172.21.15.29:0/3164752272 >> 172.21.15.29:0/3164752272 conn(0x7fac840b2d00 msgr2=0x7fac840b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac8c44d640 1 -- 172.21.15.29:0/3164752272 shutdown_connections 2024-04-24T23:20:32.281 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.275+0000 7fac8c44d640 1 -- 172.21.15.29:0/3164752272 wait complete. 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.277+0000 7fac8c44d640 1 Processor -- start 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.277+0000 7fac8c44d640 1 -- start start 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.277+0000 7fac8c44d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.277+0000 7fac8c44d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 0x7fac840cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.277+0000 7fac8c44d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac84162970 0x7fac840ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8c44d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac840d0180 con 0x7fac84162970 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8c44d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac84070560 con 0x7fac841475d0 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8c44d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fac841479b0 con 0x7fac8406b120 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a1e9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a1e9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8415dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60472/0 (socket says 172.21.15.29:60472) 2024-04-24T23:20:32.282 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a1e9640 1 -- 172.21.15.29:0/133870442 learned_addr learned my addr 172.21.15.29:0/133870442 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac899e8640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 0x7fac840cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a9ea640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac84162970 0x7fac840ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a1e9640 1 -- 172.21.15.29:0/133870442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 msgr2=0x7fac840cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a1e9640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 0x7fac840cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a1e9640 1 -- 172.21.15.29:0/133870442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac84162970 msgr2=0x7fac840ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a1e9640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac84162970 0x7fac840ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a1e9640 1 -- 172.21.15.29:0/133870442 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac841f4fd0 con 0x7fac8406b120 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac899e8640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 0x7fac840cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8a1e9640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8415dbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac7c000c00 tx=0x7fac7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac82ffd640 1 -- 172.21.15.29:0/133870442 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c041020 con 0x7fac8406b120 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac82ffd640 1 -- 172.21.15.29:0/133870442 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7c03d620 con 0x7fac8406b120 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8c44d640 1 -- 172.21.15.29:0/133870442 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac841f8560 con 0x7fac8406b120 2024-04-24T23:20:32.283 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac82ffd640 1 -- 172.21.15.29:0/133870442 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c037040 con 0x7fac8406b120 2024-04-24T23:20:32.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.278+0000 7fac8c44d640 1 -- 172.21.15.29:0/133870442 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fac841f7a80 con 0x7fac8406b120 2024-04-24T23:20:32.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.279+0000 7fac82ffd640 1 -- 172.21.15.29:0/133870442 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac7c044020 con 0x7fac8406b120 2024-04-24T23:20:32.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.279+0000 7fac82ffd640 1 -- 172.21.15.29:0/133870442 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fac7c046c20 con 0x7fac8406b120 2024-04-24T23:20:32.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.279+0000 7fac8c44d640 1 -- 172.21.15.29:0/133870442 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac84075610 con 0x7fac8406b120 2024-04-24T23:20:32.284 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.281+0000 7fac82ffd640 1 -- 172.21.15.29:0/133870442 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fac7c04b030 con 0x7fac8406b120 2024-04-24T23:20:32.369 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.368+0000 7fac8c44d640 1 -- 172.21.15.29:0/133870442 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "osd new", "uuid": "64ec8a53-51a8-4500-a714-36c1a8f342ac", "id": 3} v 0) -- 0x7fac8406be90 con 0x7fac8406b120 2024-04-24T23:20:32.372 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.371+0000 7fac82ffd640 1 -- 172.21.15.29:0/133870442 <== mon.2 v2:172.21.15.29:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64ec8a53-51a8-4500-a714-36c1a8f342ac", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fac7c046400 con 0x7fac8406b120 2024-04-24T23:20:32.373 INFO:teuthology.orchestra.run.smithi029.stdout:3 2024-04-24T23:20:32.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.372+0000 7fac8c44d640 1 -- 172.21.15.29:0/133870442 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 msgr2=0x7fac8415dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.372+0000 7fac8c44d640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8415dbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac7c000c00 tx=0x7fac7c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.372+0000 7fac8c44d640 1 -- 172.21.15.29:0/133870442 shutdown_connections 2024-04-24T23:20:32.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.372+0000 7fac8c44d640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac84162970 0x7fac840ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.372+0000 7fac8c44d640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac841475d0 0x7fac840cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.372+0000 7fac8c44d640 1 --2- 172.21.15.29:0/133870442 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac8406b120 0x7fac8415dbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.372+0000 7fac8c44d640 1 -- 172.21.15.29:0/133870442 >> 172.21.15.29:0/133870442 conn(0x7fac840b2d00 msgr2=0x7fac840b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:32.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.372+0000 7fac8c44d640 1 -- 172.21.15.29:0/133870442 shutdown_connections 2024-04-24T23:20:32.374 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.372+0000 7fac8c44d640 1 -- 172.21.15.29:0/133870442 wait complete. 2024-04-24T23:20:32.384 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd new fe83d1a2-0532-4aad-aba5-c02850051d13 4 2024-04-24T23:20:32.426 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:32.424+0000 7f60b4d28040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:20:32.484 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.483+0000 7feec8170640 1 Processor -- start 2024-04-24T23:20:32.485 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.483+0000 7feec8170640 1 -- start start 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec8170640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feec015bc90 0x7feec0161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec8170640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec00cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec8170640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec0162340 con 0x7feec00cedc0 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec8170640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7feec00ce650 con 0x7feec015bc90 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec8170640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feec00ce930 con 0x7feec00cc420 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec570b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec00cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec5f0c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feec015bc90 0x7feec0161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec5f0c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feec015bc90 0x7feec0161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44430/0 (socket says 172.21.15.29:44430) 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec570b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec00cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60474/0 (socket says 172.21.15.29:60474) 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec5f0c640 1 -- 172.21.15.29:0/1207099756 learned_addr learned my addr 172.21.15.29:0/1207099756 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.486 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec570b640 1 -- 172.21.15.29:0/1207099756 >> v2:172.21.15.29:3300/0 conn(0x7feec015bc90 msgr2=0x7feec0161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:32.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec570b640 1 --2- 172.21.15.29:0/1207099756 >> v2:172.21.15.29:3300/0 conn(0x7feec015bc90 0x7feec0161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec570b640 1 -- 172.21.15.29:0/1207099756 >> v1:172.21.15.26:6789/0 conn(0x7feec00cedc0 legacy=0x7feec006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.487 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.484+0000 7feec570b640 1 -- 172.21.15.29:0/1207099756 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec015f7d0 con 0x7feec00cc420 2024-04-24T23:20:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.485+0000 7feec570b640 1 --2- 172.21.15.29:0/1207099756 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec00cc8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feebc0097a0 tx=0x7feebc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=106a377e1919d6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.485+0000 7feec4f0a640 1 -- 172.21.15.29:0/1207099756 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc037070 con 0x7feec00cc420 2024-04-24T23:20:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.485+0000 7feec4f0a640 1 -- 172.21.15.29:0/1207099756 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc032440 con 0x7feec00cc420 2024-04-24T23:20:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.485+0000 7feec4f0a640 1 -- 172.21.15.29:0/1207099756 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc032810 con 0x7feec00cc420 2024-04-24T23:20:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.486+0000 7feec8170640 1 -- 172.21.15.29:0/1207099756 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 msgr2=0x7feec00cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.486+0000 7feec8170640 1 --2- 172.21.15.29:0/1207099756 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec00cc8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feebc0097a0 tx=0x7feebc02a180 comp rx=0 tx=0).stop 2024-04-24T23:20:32.488 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.486+0000 7feec8170640 1 -- 172.21.15.29:0/1207099756 shutdown_connections 2024-04-24T23:20:32.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.486+0000 7feec8170640 1 --2- 172.21.15.29:0/1207099756 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec00cc8a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.486+0000 7feec8170640 1 --2- 172.21.15.29:0/1207099756 >> v2:172.21.15.29:3300/0 conn(0x7feec015bc90 0x7feec0161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.486+0000 7feec8170640 1 -- 172.21.15.29:0/1207099756 >> 172.21.15.29:0/1207099756 conn(0x7feec009aa00 msgr2=0x7feec009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:32.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.486+0000 7feec8170640 1 -- 172.21.15.29:0/1207099756 shutdown_connections 2024-04-24T23:20:32.489 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.486+0000 7feec8170640 1 -- 172.21.15.29:0/1207099756 wait complete. 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.487+0000 7feec8170640 1 Processor -- start 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.487+0000 7feec8170640 1 -- start start 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec8170640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec8170640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feec00cedc0 0x7feec015da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec8170640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feec015bc90 0x7feec0159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec8170640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feec0162bd0 con 0x7feec015bc90 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec8170640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feec01628f0 con 0x7feec00cedc0 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec8170640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feec015c070 con 0x7feec00cc420 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec570b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feec00cedc0 0x7feec015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec5f0c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec570b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feec00cedc0 0x7feec015da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44432/0 (socket says 172.21.15.29:44432) 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec570b640 1 -- 172.21.15.29:0/3741490858 learned_addr learned my addr 172.21.15.29:0/3741490858 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.492 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec670d640 1 --2- 172.21.15.29:0/3741490858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feec015bc90 0x7feec0159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec570b640 1 -- 172.21.15.29:0/3741490858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 msgr2=0x7feec014f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec570b640 1 --2- 172.21.15.29:0/3741490858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec014f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec570b640 1 -- 172.21.15.29:0/3741490858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feec015bc90 msgr2=0x7feec0159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec570b640 1 --2- 172.21.15.29:0/3741490858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feec015bc90 0x7feec0159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec570b640 1 -- 172.21.15.29:0/3741490858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec014f780 con 0x7feec00cedc0 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.488+0000 7feec570b640 1 --2- 172.21.15.29:0/3741490858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feec00cedc0 0x7feec015da90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feebc035040 tx=0x7feebc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.489+0000 7feeb67fc640 1 -- 172.21.15.29:0/3741490858 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc037040 con 0x7feec00cedc0 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.489+0000 7feeb67fc640 1 -- 172.21.15.29:0/3741490858 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc030500 con 0x7feec00cedc0 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.489+0000 7feec8170640 1 -- 172.21.15.29:0/3741490858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec00af6a0 con 0x7feec00cedc0 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.489+0000 7feeb67fc640 1 -- 172.21.15.29:0/3741490858 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc0308d0 con 0x7feec00cedc0 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.489+0000 7feec8170640 1 -- 172.21.15.29:0/3741490858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec00aff40 con 0x7feec00cedc0 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.489+0000 7feeb67fc640 1 -- 172.21.15.29:0/3741490858 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feebc044020 con 0x7feec00cedc0 2024-04-24T23:20:32.493 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.489+0000 7feeb67fc640 1 -- 172.21.15.29:0/3741490858 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7feebc03bb60 con 0x7feec00cedc0 2024-04-24T23:20:32.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.490+0000 7feec8170640 1 -- 172.21.15.29:0/3741490858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec0075610 con 0x7feec00cedc0 2024-04-24T23:20:32.494 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.492+0000 7feeb67fc640 1 -- 172.21.15.29:0/3741490858 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7feec014f780 con 0x7feec00cedc0 2024-04-24T23:20:32.518 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:32.516+0000 7f60b4d28040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:20:32.526 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:32.525+0000 7facbe321040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:20:32.562 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:32.561+0000 7f60b4d28040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:20:32.568 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.566+0000 7feec8170640 1 -- 172.21.15.29:0/3741490858 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fe83d1a2-0532-4aad-aba5-c02850051d13", "id": 4} v 0) -- 0x7feec00b19e0 con 0x7feec00cedc0 2024-04-24T23:20:32.571 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.569+0000 7feeb67fc640 1 -- 172.21.15.29:0/3741490858 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fe83d1a2-0532-4aad-aba5-c02850051d13", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7feec015f7d0 con 0x7feec00cedc0 2024-04-24T23:20:32.571 INFO:teuthology.orchestra.run.smithi029.stdout:4 2024-04-24T23:20:32.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.570+0000 7feec8170640 1 -- 172.21.15.29:0/3741490858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feec00cedc0 msgr2=0x7feec015da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.570+0000 7feec8170640 1 --2- 172.21.15.29:0/3741490858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feec00cedc0 0x7feec015da90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feebc035040 tx=0x7feebc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.570+0000 7feec8170640 1 -- 172.21.15.29:0/3741490858 shutdown_connections 2024-04-24T23:20:32.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.570+0000 7feec8170640 1 --2- 172.21.15.29:0/3741490858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feec015bc90 0x7feec0159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.572 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.570+0000 7feec8170640 1 --2- 172.21.15.29:0/3741490858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feec00cedc0 0x7feec015da90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.570+0000 7feec8170640 1 --2- 172.21.15.29:0/3741490858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feec00cc420 0x7feec014f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.570+0000 7feec8170640 1 -- 172.21.15.29:0/3741490858 >> 172.21.15.29:0/3741490858 conn(0x7feec009aa00 msgr2=0x7feec00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:32.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.570+0000 7feec8170640 1 -- 172.21.15.29:0/3741490858 shutdown_connections 2024-04-24T23:20:32.573 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.570+0000 7feec8170640 1 -- 172.21.15.29:0/3741490858 wait complete. 2024-04-24T23:20:32.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd new 6f2cff4a-8ea8-42ac-9f50-f870906ae05d 5 2024-04-24T23:20:32.621 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:32.619+0000 7facbe321040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:20:32.667 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:32.665+0000 7facbe321040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:20:32.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.680+0000 7f51a3483640 1 Processor -- start 2024-04-24T23:20:32.682 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.681+0000 7f51a3483640 1 -- start start 2024-04-24T23:20:32.683 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.681+0000 7f51a3483640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c155470 0x7f519c15f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.681+0000 7f51a3483640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f519c15b3a0 0x7f519c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.681+0000 7f51a3483640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f519c15bd60 con 0x7f519c15b7e0 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.681+0000 7f51a3483640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f519c155850 con 0x7f519c15b3a0 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.681+0000 7f51a3483640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f519c0b4480 con 0x7f519c155470 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a0a1e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f519c15b3a0 0x7f519c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a121f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c155470 0x7f519c15f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a0a1e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f519c15b3a0 0x7f519c06def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44436/0 (socket says 172.21.15.29:44436) 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a121f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c155470 0x7f519c15f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60494/0 (socket says 172.21.15.29:60494) 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a0a1e640 1 -- 172.21.15.29:0/2293786493 learned_addr learned my addr 172.21.15.29:0/2293786493 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a0a1e640 1 -- 172.21.15.29:0/2293786493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c155470 msgr2=0x7f519c15f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a0a1e640 1 --2- 172.21.15.29:0/2293786493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c155470 0x7f519c15f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a0a1e640 1 -- 172.21.15.29:0/2293786493 >> v1:172.21.15.26:6789/0 conn(0x7f519c15b7e0 legacy=0x7f519c0720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a0a1e640 1 -- 172.21.15.29:0/2293786493 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c15c040 con 0x7f519c15b3a0 2024-04-24T23:20:32.685 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.682+0000 7f51a121f640 1 --2- 172.21.15.29:0/2293786493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c155470 0x7f519c15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:32.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f51a0a1e640 1 --2- 172.21.15.29:0/2293786493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c15b3a0 0x7f519c06def0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f518c009870 tx=0x7f518c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5161a9b0804beed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f5193fff640 1 -- 172.21.15.29:0/2293786493 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c037070 con 0x7f519c15b3a0 2024-04-24T23:20:32.686 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f5193fff640 1 -- 172.21.15.29:0/2293786493 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518c032440 con 0x7f519c15b3a0 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f5193fff640 1 -- 172.21.15.29:0/2293786493 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c032810 con 0x7f519c15b3a0 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f51a3483640 1 -- 172.21.15.29:0/2293786493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c15b3a0 msgr2=0x7f519c06def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f51a3483640 1 --2- 172.21.15.29:0/2293786493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c15b3a0 0x7f519c06def0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f518c009870 tx=0x7f518c02a1c0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f51a3483640 1 -- 172.21.15.29:0/2293786493 shutdown_connections 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f51a3483640 1 --2- 172.21.15.29:0/2293786493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c15b3a0 0x7f519c06def0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f51a3483640 1 --2- 172.21.15.29:0/2293786493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c155470 0x7f519c15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.683+0000 7f51a3483640 1 -- 172.21.15.29:0/2293786493 >> 172.21.15.29:0/2293786493 conn(0x7f519c0d0f80 msgr2=0x7f519c0d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.684+0000 7f51a3483640 1 -- 172.21.15.29:0/2293786493 shutdown_connections 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.684+0000 7f51a3483640 1 -- 172.21.15.29:0/2293786493 wait complete. 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.685+0000 7f51a3483640 1 Processor -- start 2024-04-24T23:20:32.687 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.685+0000 7f51a3483640 1 -- start start 2024-04-24T23:20:32.689 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.685+0000 7f51a3483640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c155470 0x7f519c144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.685+0000 7f51a3483640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c15b3a0 0x7f519c143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.685+0000 7f51a3483640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519c15b7e0 0x7f519c145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.685+0000 7f51a3483640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f519c0b5aa0 con 0x7f519c15b7e0 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.685+0000 7f51a3483640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f519c0b5920 con 0x7f519c155470 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.685+0000 7f51a3483640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f519c0b5c20 con 0x7f519c15b3a0 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a0a1e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c15b3a0 0x7f519c143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a121f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c155470 0x7f519c144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a0a1e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c15b3a0 0x7f519c143430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60506/0 (socket says 172.21.15.29:60506) 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a121f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c155470 0x7f519c144f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44452/0 (socket says 172.21.15.29:44452) 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a0a1e640 1 -- 172.21.15.29:0/3390167344 learned_addr learned my addr 172.21.15.29:0/3390167344 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.690 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a1a20640 1 --2- 172.21.15.29:0/3390167344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519c15b7e0 0x7f519c145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a0a1e640 1 -- 172.21.15.29:0/3390167344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c155470 msgr2=0x7f519c144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a0a1e640 1 --2- 172.21.15.29:0/3390167344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c155470 0x7f519c144f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a0a1e640 1 -- 172.21.15.29:0/3390167344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519c15b7e0 msgr2=0x7f519c145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a0a1e640 1 --2- 172.21.15.29:0/3390167344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519c15b7e0 0x7f519c145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a0a1e640 1 -- 172.21.15.29:0/3390167344 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c143ef0 con 0x7f519c15b3a0 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a0a1e640 1 --2- 172.21.15.29:0/3390167344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c15b3a0 0x7f519c143430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f518c009870 tx=0x7f518c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f5191ffb640 1 -- 172.21.15.29:0/3390167344 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c041020 con 0x7f519c15b3a0 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f5191ffb640 1 -- 172.21.15.29:0/3390167344 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518c030640 con 0x7f519c15b3a0 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f5191ffb640 1 -- 172.21.15.29:0/3390167344 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c037040 con 0x7f519c15b3a0 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.686+0000 7f51a3483640 1 -- 172.21.15.29:0/3390167344 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c1441d0 con 0x7f519c15b3a0 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.687+0000 7f51a3483640 1 -- 172.21.15.29:0/3390167344 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c144790 con 0x7f519c15b3a0 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.687+0000 7f5191ffb640 1 -- 172.21.15.29:0/3390167344 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f518c044020 con 0x7f519c15b3a0 2024-04-24T23:20:32.691 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.687+0000 7f5191ffb640 1 -- 172.21.15.29:0/3390167344 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f518c0066e0 con 0x7f519c15b3a0 2024-04-24T23:20:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.687+0000 7f51a3483640 1 -- 172.21.15.29:0/3390167344 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c075610 con 0x7f519c15b3a0 2024-04-24T23:20:32.692 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.690+0000 7f5191ffb640 1 -- 172.21.15.29:0/3390167344 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f518c04b070 con 0x7f519c15b3a0 2024-04-24T23:20:32.737 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:32.735+0000 7f60b4d28040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:20:32.764 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.763+0000 7f51a3483640 1 -- 172.21.15.29:0/3390167344 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6f2cff4a-8ea8-42ac-9f50-f870906ae05d", "id": 5} v 0) -- 0x7f519c145570 con 0x7f519c15b3a0 2024-04-24T23:20:32.767 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.766+0000 7f5191ffb640 1 -- 172.21.15.29:0/3390167344 <== mon.2 v2:172.21.15.29:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6f2cff4a-8ea8-42ac-9f50-f870906ae05d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f518c005990 con 0x7f519c15b3a0 2024-04-24T23:20:32.768 INFO:teuthology.orchestra.run.smithi029.stdout:5 2024-04-24T23:20:32.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.767+0000 7f51a3483640 1 -- 172.21.15.29:0/3390167344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c15b3a0 msgr2=0x7f519c143430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.767+0000 7f51a3483640 1 --2- 172.21.15.29:0/3390167344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c15b3a0 0x7f519c143430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f518c009870 tx=0x7f518c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.767+0000 7f51a3483640 1 -- 172.21.15.29:0/3390167344 shutdown_connections 2024-04-24T23:20:32.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.767+0000 7f51a3483640 1 --2- 172.21.15.29:0/3390167344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519c15b7e0 0x7f519c145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.767+0000 7f51a3483640 1 --2- 172.21.15.29:0/3390167344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f519c15b3a0 0x7f519c143430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.767+0000 7f51a3483640 1 --2- 172.21.15.29:0/3390167344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f519c155470 0x7f519c144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.767+0000 7f51a3483640 1 -- 172.21.15.29:0/3390167344 >> 172.21.15.29:0/3390167344 conn(0x7f519c0d0f80 msgr2=0x7f519c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:32.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.767+0000 7f51a3483640 1 -- 172.21.15.29:0/3390167344 shutdown_connections 2024-04-24T23:20:32.769 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.767+0000 7f51a3483640 1 -- 172.21.15.29:0/3390167344 wait complete. 2024-04-24T23:20:32.779 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd new f684c2b3-ae8a-43da-9091-d7c4dacbdd7a 6 2024-04-24T23:20:32.793 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:32.791+0000 7f60b4d28040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:20:32.844 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:32.843+0000 7facbe321040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:20:32.848 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:32.846+0000 7f60b4d28040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:20:32.896 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:32.894+0000 7facbe321040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:20:32.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.918+0000 7f27b42d4640 1 Processor -- start 2024-04-24T23:20:32.920 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.918+0000 7f27b42d4640 1 -- start start 2024-04-24T23:20:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b42d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 0x7f27ac162830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.921 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b42d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f27ac06c440 0x7f27ac06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b42d4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27ac06d180 0x7f27ac0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b42d4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f27ac0b3740 con 0x7f27ac06c440 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b42d4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f27ac06d5d0 con 0x7f27ac06d180 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b42d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f27ac0d0310 con 0x7f27ac15e310 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b2871640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27ac06d180 0x7f27ac0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b2070640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 0x7f27ac162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b2871640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f27ac06d180 0x7f27ac0ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44466/0 (socket says 172.21.15.29:44466) 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b2070640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 0x7f27ac162830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60518/0 (socket says 172.21.15.29:60518) 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.919+0000 7f27b2871640 1 -- 172.21.15.29:0/3948508564 learned_addr learned my addr 172.21.15.29:0/3948508564 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.920+0000 7f27b186f640 1 --2- 172.21.15.29:0/3948508564 >> v2:172.21.15.26:3300/0 conn(0x7f27ac06c440 0x7f27ac06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.920+0000 7f27b2871640 1 -- 172.21.15.29:0/3948508564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 msgr2=0x7f27ac162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.920+0000 7f27b2871640 1 --2- 172.21.15.29:0/3948508564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 0x7f27ac162830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.922 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.920+0000 7f27b2871640 1 -- 172.21.15.29:0/3948508564 >> v2:172.21.15.26:3300/0 conn(0x7f27ac06c440 msgr2=0x7f27ac06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:32.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.920+0000 7f27b2871640 1 --2- 172.21.15.29:0/3948508564 >> v2:172.21.15.26:3300/0 conn(0x7f27ac06c440 0x7f27ac06cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.923 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.920+0000 7f27b2871640 1 -- 172.21.15.29:0/3948508564 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac0d05f0 con 0x7f27ac06d180 2024-04-24T23:20:32.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.920+0000 7f27b2871640 1 --2- 172.21.15.29:0/3948508564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06d180 0x7f27ac0ce440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0723e0 tx=0x7f27a402a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30bae477b8efe653 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b106e640 1 -- 172.21.15.29:0/3948508564 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4036070 con 0x7f27ac06d180 2024-04-24T23:20:32.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b106e640 1 -- 172.21.15.29:0/3948508564 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a4031600 con 0x7f27ac06d180 2024-04-24T23:20:32.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b106e640 1 -- 172.21.15.29:0/3948508564 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40319d0 con 0x7f27ac06d180 2024-04-24T23:20:32.924 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b42d4640 1 -- 172.21.15.29:0/3948508564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06d180 msgr2=0x7f27ac0ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b42d4640 1 --2- 172.21.15.29:0/3948508564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06d180 0x7f27ac0ce440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0723e0 tx=0x7f27a402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b42d4640 1 -- 172.21.15.29:0/3948508564 shutdown_connections 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b42d4640 1 --2- 172.21.15.29:0/3948508564 >> v2:172.21.15.26:3300/0 conn(0x7f27ac06c440 0x7f27ac06cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b42d4640 1 --2- 172.21.15.29:0/3948508564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06d180 0x7f27ac0ce440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b42d4640 1 --2- 172.21.15.29:0/3948508564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 0x7f27ac162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b42d4640 1 -- 172.21.15.29:0/3948508564 >> 172.21.15.29:0/3948508564 conn(0x7f27ac0d1dd0 msgr2=0x7f27ac0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b42d4640 1 -- 172.21.15.29:0/3948508564 shutdown_connections 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.921+0000 7f27b42d4640 1 -- 172.21.15.29:0/3948508564 wait complete. 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.923+0000 7f27b42d4640 1 Processor -- start 2024-04-24T23:20:32.925 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.923+0000 7f27b42d4640 1 -- start start 2024-04-24T23:20:32.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.923+0000 7f27b42d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06c440 0x7f27ac1480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.923+0000 7f27b42d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27ac06d180 0x7f27ac1f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.923+0000 7f27b42d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 0x7f27ac1f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.923+0000 7f27b42d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac0d0e90 con 0x7f27ac06d180 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.923+0000 7f27b42d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac0d0bb0 con 0x7f27ac06c440 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.923+0000 7f27b42d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f27ac162d70 con 0x7f27ac15e310 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.923+0000 7f27b2070640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06c440 0x7f27ac1480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b2070640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06c440 0x7f27ac1480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:44470/0 (socket says 172.21.15.29:44470) 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b2871640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 0x7f27ac1f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b2070640 1 -- 172.21.15.29:0/338091333 learned_addr learned my addr 172.21.15.29:0/338091333 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b186f640 1 --2- 172.21.15.29:0/338091333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27ac06d180 0x7f27ac1f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b2070640 1 -- 172.21.15.29:0/338091333 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 msgr2=0x7f27ac1f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b2070640 1 --2- 172.21.15.29:0/338091333 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 0x7f27ac1f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b2070640 1 -- 172.21.15.29:0/338091333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27ac06d180 msgr2=0x7f27ac1f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:32.930 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b2070640 1 --2- 172.21.15.29:0/338091333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27ac06d180 0x7f27ac1f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b2070640 1 -- 172.21.15.29:0/338091333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac0c7950 con 0x7f27ac06c440 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b2070640 1 --2- 172.21.15.29:0/338091333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06c440 0x7f27ac1480a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f279c00e870 tx=0x7f279c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27a2ffd640 1 -- 172.21.15.29:0/338091333 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c017070 con 0x7f27ac06c440 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27a2ffd640 1 -- 172.21.15.29:0/338091333 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279c00c940 con 0x7f27ac06c440 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b42d4640 1 -- 172.21.15.29:0/338091333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ac0c7c30 con 0x7f27ac06c440 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27a2ffd640 1 -- 172.21.15.29:0/338091333 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c00cd30 con 0x7f27ac06c440 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27b42d4640 1 -- 172.21.15.29:0/338091333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ac0c81f0 con 0x7f27ac06c440 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27a2ffd640 1 -- 172.21.15.29:0/338091333 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f279c01a070 con 0x7f27ac06c440 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.924+0000 7f27a2ffd640 1 -- 172.21.15.29:0/338091333 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f279c01ca40 con 0x7f27ac06c440 2024-04-24T23:20:32.931 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.925+0000 7f27b42d4640 1 -- 172.21.15.29:0/338091333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ac0d13b0 con 0x7f27ac06c440 2024-04-24T23:20:32.932 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:32.927+0000 7f27a2ffd640 1 -- 172.21.15.29:0/338091333 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f27ac0d13b0 con 0x7f27ac06c440 2024-04-24T23:20:32.952 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:32.951+0000 7facbe321040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:20:32.966 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:32.964+0000 7f60b4d28040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:20:33.003 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.001+0000 7f27b42d4640 1 -- 172.21.15.29:0/338091333 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f684c2b3-ae8a-43da-9091-d7c4dacbdd7a", "id": 6} v 0) -- 0x7f27ac075610 con 0x7f27ac06c440 2024-04-24T23:20:33.006 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.004+0000 7f27a2ffd640 1 -- 172.21.15.29:0/338091333 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f684c2b3-ae8a-43da-9091-d7c4dacbdd7a", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f279c01c450 con 0x7f27ac06c440 2024-04-24T23:20:33.006 INFO:teuthology.orchestra.run.smithi029.stdout:6 2024-04-24T23:20:33.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.005+0000 7f27b42d4640 1 -- 172.21.15.29:0/338091333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06c440 msgr2=0x7f27ac1480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.005+0000 7f27b42d4640 1 --2- 172.21.15.29:0/338091333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06c440 0x7f27ac1480a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f279c00e870 tx=0x7f279c00ed30 comp rx=0 tx=0).stop 2024-04-24T23:20:33.007 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.005+0000 7f27b42d4640 1 -- 172.21.15.29:0/338091333 shutdown_connections 2024-04-24T23:20:33.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.005+0000 7f27b42d4640 1 --2- 172.21.15.29:0/338091333 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27ac15e310 0x7f27ac1f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.005+0000 7f27b42d4640 1 --2- 172.21.15.29:0/338091333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27ac06d180 0x7f27ac1f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.005+0000 7f27b42d4640 1 --2- 172.21.15.29:0/338091333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27ac06c440 0x7f27ac1480a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.006+0000 7f27b42d4640 1 -- 172.21.15.29:0/338091333 >> 172.21.15.29:0/338091333 conn(0x7f27ac0d1dd0 msgr2=0x7f27ac06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:33.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.006+0000 7f27b42d4640 1 -- 172.21.15.29:0/338091333 shutdown_connections 2024-04-24T23:20:33.008 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.006+0000 7f27b42d4640 1 -- 172.21.15.29:0/338091333 wait complete. 2024-04-24T23:20:33.021 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd new 6b4fa289-00aa-481d-a9c1-87634fa1d532 7 2024-04-24T23:20:33.077 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:33.076+0000 7facbe321040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:20:33.161 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.159+0000 7fb02804d640 1 Processor -- start 2024-04-24T23:20:33.162 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.160+0000 7fb02804d640 1 -- start start 2024-04-24T23:20:33.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.160+0000 7fb02804d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb02015c140 0x7fb02015f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.160+0000 7fb02804d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb02015f750 con 0x7fb020158b20 2024-04-24T23:20:33.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.160+0000 7fb02804d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb02015b660 con 0x7fb0200cf9a0 2024-04-24T23:20:33.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.160+0000 7fb02804d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb02015b940 con 0x7fb02015c140 2024-04-24T23:20:33.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.160+0000 7fb0265ea640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb02015c140 0x7fb02015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.163 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.160+0000 7fb0255e8640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb0200cf9a0 0x7fb02015ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.29:57414/0 (socket says 172.21.15.29:57414) 2024-04-24T23:20:33.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.160+0000 7fb0255e8640 1 -- 172.21.15.29:0/163788775 learned_addr learned my addr 172.21.15.29:0/163788775 (peer_addr_for_me v1:172.21.15.29:0/0) 2024-04-24T23:20:33.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.161+0000 7fb0265ea640 1 -- 172.21.15.29:0/163788775 >> v1:172.21.15.29:6789/0 conn(0x7fb0200cf9a0 legacy=0x7fb02015ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.161+0000 7fb0265ea640 1 -- 172.21.15.29:0/163788775 >> v1:172.21.15.26:6789/0 conn(0x7fb020158b20 legacy=0x7fb0200cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.161+0000 7fb024de7640 1 -- 172.21.15.29:0/163788775 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3958381590 0 0) 0x7fb02015b660 con 0x7fb0200cf9a0 2024-04-24T23:20:33.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.161+0000 7fb0265ea640 1 -- 172.21.15.29:0/163788775 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02015bc20 con 0x7fb02015c140 2024-04-24T23:20:33.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.161+0000 7fb0265ea640 1 --2- 172.21.15.29:0/163788775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb02015c140 0x7fb02015f150 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb018009dc0 tx=0x7fb01802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129d4ec2e0c6e21d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:33.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.162+0000 7fb024de7640 1 -- 172.21.15.29:0/163788775 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018007740 con 0x7fb02015c140 2024-04-24T23:20:33.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.162+0000 7fb024de7640 1 -- 172.21.15.29:0/163788775 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0180078e0 con 0x7fb02015c140 2024-04-24T23:20:33.164 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.162+0000 7fb024de7640 1 -- 172.21.15.29:0/163788775 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018007cb0 con 0x7fb02015c140 2024-04-24T23:20:33.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.162+0000 7fb02804d640 1 -- 172.21.15.29:0/163788775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb02015c140 msgr2=0x7fb02015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.162+0000 7fb02804d640 1 --2- 172.21.15.29:0/163788775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb02015c140 0x7fb02015f150 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb018009dc0 tx=0x7fb01802a6d0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.162+0000 7fb02804d640 1 -- 172.21.15.29:0/163788775 shutdown_connections 2024-04-24T23:20:33.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.162+0000 7fb02804d640 1 --2- 172.21.15.29:0/163788775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb02015c140 0x7fb02015f150 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.165 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.162+0000 7fb02804d640 1 -- 172.21.15.29:0/163788775 >> 172.21.15.29:0/163788775 conn(0x7fb0200cd730 msgr2=0x7fb0200cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:33.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.162+0000 7fb02804d640 1 -- 172.21.15.29:0/163788775 shutdown_connections 2024-04-24T23:20:33.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.163+0000 7fb02804d640 1 -- 172.21.15.29:0/163788775 wait complete. 2024-04-24T23:20:33.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.164+0000 7fb02804d640 1 Processor -- start 2024-04-24T23:20:33.166 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.164+0000 7fb02804d640 1 -- start start 2024-04-24T23:20:33.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.164+0000 7fb02804d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0200cf9a0 0x7fb0201540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.164+0000 7fb02804d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb020158b20 0x7fb02015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.164+0000 7fb02804d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02015c140 0x7fb02006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.164+0000 7fb02804d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb020166160 con 0x7fb020158b20 2024-04-24T23:20:33.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.164+0000 7fb02804d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb020165fe0 con 0x7fb02015c140 2024-04-24T23:20:33.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.164+0000 7fb02804d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb0201662e0 con 0x7fb0200cf9a0 2024-04-24T23:20:33.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb025de9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0200cf9a0 0x7fb0201540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.168 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb0265ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02015c140 0x7fb02006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb025de9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0200cf9a0 0x7fb0201540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.29:60538/0 (socket says 172.21.15.29:60538) 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb025de9640 1 -- 172.21.15.29:0/3328066185 learned_addr learned my addr 172.21.15.29:0/3328066185 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb0255e8640 1 --2- 172.21.15.29:0/3328066185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb020158b20 0x7fb02015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb025de9640 1 -- 172.21.15.29:0/3328066185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02015c140 msgr2=0x7fb02006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb025de9640 1 --2- 172.21.15.29:0/3328066185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02015c140 0x7fb02006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb025de9640 1 -- 172.21.15.29:0/3328066185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb020158b20 msgr2=0x7fb02015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb025de9640 1 --2- 172.21.15.29:0/3328066185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb020158b20 0x7fb02015ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb025de9640 1 -- 172.21.15.29:0/3328066185 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb020154a20 con 0x7fb0200cf9a0 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb025de9640 1 --2- 172.21.15.29:0/3328066185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0200cf9a0 0x7fb0201540d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c4d0 tx=0x7fb01000c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb0167fc640 1 -- 172.21.15.29:0/3328066185 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100052d0 con 0x7fb0200cf9a0 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb0167fc640 1 -- 172.21.15.29:0/3328066185 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb010005470 con 0x7fb0200cf9a0 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb02804d640 1 -- 172.21.15.29:0/3328066185 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb020154d00 con 0x7fb0200cf9a0 2024-04-24T23:20:33.169 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb0167fc640 1 -- 172.21.15.29:0/3328066185 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010012850 con 0x7fb0200cf9a0 2024-04-24T23:20:33.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb02804d640 1 -- 172.21.15.29:0/3328066185 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0201552c0 con 0x7fb0200cf9a0 2024-04-24T23:20:33.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb0167fc640 1 -- 172.21.15.29:0/3328066185 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb0100129f0 con 0x7fb0200cf9a0 2024-04-24T23:20:33.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.165+0000 7fb0167fc640 1 -- 172.21.15.29:0/3328066185 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fb01001ba60 con 0x7fb0200cf9a0 2024-04-24T23:20:33.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.166+0000 7fb02804d640 1 -- 172.21.15.29:0/3328066185 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb020075610 con 0x7fb0200cf9a0 2024-04-24T23:20:33.170 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.168+0000 7fb0167fc640 1 -- 172.21.15.29:0/3328066185 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb01001b400 con 0x7fb0200cf9a0 2024-04-24T23:20:33.243 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.241+0000 7fb02804d640 1 -- 172.21.15.29:0/3328066185 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6b4fa289-00aa-481d-a9c1-87634fa1d532", "id": 7} v 0) -- 0x7fb02015af10 con 0x7fb0200cf9a0 2024-04-24T23:20:33.246 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.245+0000 7fb0167fc640 1 -- 172.21.15.29:0/3328066185 <== mon.2 v2:172.21.15.29:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b4fa289-00aa-481d-a9c1-87634fa1d532", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fb01001b5e0 con 0x7fb0200cf9a0 2024-04-24T23:20:33.246 INFO:teuthology.orchestra.run.smithi029.stdout:7 2024-04-24T23:20:33.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.245+0000 7fb02804d640 1 -- 172.21.15.29:0/3328066185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0200cf9a0 msgr2=0x7fb0201540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.245+0000 7fb02804d640 1 --2- 172.21.15.29:0/3328066185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0200cf9a0 0x7fb0201540d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c4d0 tx=0x7fb01000c990 comp rx=0 tx=0).stop 2024-04-24T23:20:33.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.245+0000 7fb02804d640 1 -- 172.21.15.29:0/3328066185 shutdown_connections 2024-04-24T23:20:33.247 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.245+0000 7fb02804d640 1 --2- 172.21.15.29:0/3328066185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb02015c140 0x7fb02006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.245+0000 7fb02804d640 1 --2- 172.21.15.29:0/3328066185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb020158b20 0x7fb02015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.245+0000 7fb02804d640 1 --2- 172.21.15.29:0/3328066185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0200cf9a0 0x7fb0201540d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.246+0000 7fb02804d640 1 -- 172.21.15.29:0/3328066185 >> 172.21.15.29:0/3328066185 conn(0x7fb0200cd730 msgr2=0x7fb0200cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:33.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.246+0000 7fb02804d640 1 -- 172.21.15.29:0/3328066185 shutdown_connections 2024-04-24T23:20:33.248 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:20:33.246+0000 7fb02804d640 1 -- 172.21.15.29:0/3328066185 wait complete. 2024-04-24T23:20:33.257 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-24T23:20:33.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-24T23:20:33.259 INFO:tasks.ceph.osd.0:Started 2024-04-24T23:20:33.260 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-24T23:20:33.260 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-24T23:20:33.262 INFO:tasks.ceph.osd.1:Started 2024-04-24T23:20:33.262 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-24T23:20:33.262 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-24T23:20:33.264 INFO:tasks.ceph.osd.2:Started 2024-04-24T23:20:33.264 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-24T23:20:33.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-24T23:20:33.266 INFO:tasks.ceph.osd.3:Started 2024-04-24T23:20:33.267 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-24T23:20:33.267 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-24T23:20:33.299 INFO:tasks.ceph.osd.4:Started 2024-04-24T23:20:33.300 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-24T23:20:33.300 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-24T23:20:33.302 INFO:tasks.ceph.osd.5:Started 2024-04-24T23:20:33.302 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T23:20:33.302 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T23:20:33.305 INFO:tasks.ceph.osd.6:Started 2024-04-24T23:20:33.305 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-24T23:20:33.305 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-24T23:20:33.310 INFO:tasks.ceph.osd.7:Started 2024-04-24T23:20:33.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:20:33.393 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:33.391+0000 7f60b4d28040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:20:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.426+0000 7f4b8d177640 1 Processor -- start 2024-04-24T23:20:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.426+0000 7f4b8d177640 1 -- start start 2024-04-24T23:20:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b8d177640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b8815e2b0 0x7f4b8806c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b8d177640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b880b2040 con 0x7f4b880cef10 2024-04-24T23:20:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b8d177640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b880b22e0 con 0x7f4b880b1b90 2024-04-24T23:20:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b8d177640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4b880b2ef0 con 0x7f4b8815e2b0 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b86d9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4b8815e2b0 0x7f4b8806c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b7ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4b880b1b90 0x7f4b880ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:39616/0 (socket says 172.21.15.26:39616) 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b8759e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4b880cef10 0x7f4b880d04f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44322/0 (socket says 172.21.15.26:44322) 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b7ffff640 1 -- 172.21.15.26:0/1575164006 learned_addr learned my addr 172.21.15.26:0/1575164006 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b8659c640 1 -- 172.21.15.26:0/1575164006 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 764601457 0 0) 0x7f4b880b2040 con 0x7f4b880cef10 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b8659c640 1 -- 172.21.15.26:0/1575164006 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b74003520 con 0x7f4b880cef10 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b8659c640 1 -- 172.21.15.26:0/1575164006 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1540708509 0 0) 0x7f4b880b22e0 con 0x7f4b880b1b90 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b8659c640 1 -- 172.21.15.26:0/1575164006 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b880b2040 con 0x7f4b880b1b90 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b86d9d640 1 -- 172.21.15.26:0/1575164006 >> v1:172.21.15.29:6789/0 conn(0x7f4b880b1b90 legacy=0x7f4b880ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.427+0000 7f4b86d9d640 1 -- 172.21.15.26:0/1575164006 >> v1:172.21.15.26:6789/0 conn(0x7f4b880cef10 legacy=0x7f4b880d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.428+0000 7f4b86d9d640 1 -- 172.21.15.26:0/1575164006 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b880b25c0 con 0x7f4b8815e2b0 2024-04-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.428+0000 7f4b86d9d640 1 --2- 172.21.15.26:0/1575164006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b8815e2b0 0x7f4b8806c600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4b780097a0 tx=0x7f4b7802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11d1991be7efea06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.428+0000 7f4b8659c640 1 -- 172.21.15.26:0/1575164006 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78037070 con 0x7f4b8815e2b0 2024-04-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.428+0000 7f4b8659c640 1 -- 172.21.15.26:0/1575164006 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b78032440 con 0x7f4b8815e2b0 2024-04-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.428+0000 7f4b8659c640 1 -- 172.21.15.26:0/1575164006 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780327f0 con 0x7f4b8815e2b0 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.429+0000 7f4b8d177640 1 -- 172.21.15.26:0/1575164006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b8815e2b0 msgr2=0x7f4b8806c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.429+0000 7f4b8d177640 1 --2- 172.21.15.26:0/1575164006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b8815e2b0 0x7f4b8806c600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4b780097a0 tx=0x7f4b7802a230 comp rx=0 tx=0).stop 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.429+0000 7f4b8d177640 1 -- 172.21.15.26:0/1575164006 shutdown_connections 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.429+0000 7f4b8d177640 1 --2- 172.21.15.26:0/1575164006 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b8815e2b0 0x7f4b8806c600 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.429+0000 7f4b8d177640 1 -- 172.21.15.26:0/1575164006 >> 172.21.15.26:0/1575164006 conn(0x7f4b880d1f80 msgr2=0x7f4b880d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.429+0000 7f4b8d177640 1 -- 172.21.15.26:0/1575164006 shutdown_connections 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.429+0000 7f4b8d177640 1 -- 172.21.15.26:0/1575164006 wait complete. 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.431+0000 7f4b8d177640 1 Processor -- start 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.431+0000 7f4b8d177640 1 -- start start 2024-04-24T23:20:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.431+0000 7f4b8d177640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b880b1b90 0x7f4b880b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.431+0000 7f4b8d177640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b880cef10 0x7f4b8815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.431+0000 7f4b8d177640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b8815e2b0 0x7f4b881f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b8d177640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8815e690 con 0x7f4b880cef10 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b8d177640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b880b2b80 con 0x7f4b8815e2b0 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b8d177640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8815e810 con 0x7f4b880b1b90 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b880cef10 0x7f4b8815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b880cef10 0x7f4b8815ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38160/0 (socket says 172.21.15.26:38160) 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7ffff640 1 -- 172.21.15.26:0/750339022 learned_addr learned my addr 172.21.15.26:0/750339022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b86d9d640 1 --2- 172.21.15.26:0/750339022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b880b1b90 0x7f4b880b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b8759e640 1 --2- 172.21.15.26:0/750339022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b8815e2b0 0x7f4b881f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7ffff640 1 -- 172.21.15.26:0/750339022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b880b1b90 msgr2=0x7f4b880b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7ffff640 1 --2- 172.21.15.26:0/750339022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b880b1b90 0x7f4b880b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7ffff640 1 -- 172.21.15.26:0/750339022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b8815e2b0 msgr2=0x7f4b881f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7ffff640 1 --2- 172.21.15.26:0/750339022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b8815e2b0 0x7f4b881f3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7ffff640 1 -- 172.21.15.26:0/750339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b881f48f0 con 0x7f4b880cef10 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7ffff640 1 --2- 172.21.15.26:0/750339022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b880cef10 0x7f4b8815ea20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c500 tx=0x7f4b7000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7f7fe640 1 -- 172.21.15.26:0/750339022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70017070 con 0x7f4b880cef10 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7f7fe640 1 -- 172.21.15.26:0/750339022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b70005ce0 con 0x7f4b880cef10 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b8d177640 1 -- 172.21.15.26:0/750339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b881f4bd0 con 0x7f4b880cef10 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b7f7fe640 1 -- 172.21.15.26:0/750339022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700072b0 con 0x7f4b880cef10 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.432+0000 7f4b8d177640 1 -- 172.21.15.26:0/750339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b880b5070 con 0x7f4b880cef10 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.433+0000 7f4b7f7fe640 1 -- 172.21.15.26:0/750339022 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b7001a070 con 0x7f4b880cef10 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.433+0000 7f4b7f7fe640 1 -- 172.21.15.26:0/750339022 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4b70007450 con 0x7f4b880cef10 2024-04-24T23:20:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.434+0000 7f4b8d177640 1 -- 172.21.15.26:0/750339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b880681e0 con 0x7f4b880cef10 2024-04-24T23:20:33.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.436+0000 7f4b7f7fe640 1 -- 172.21.15.26:0/750339022 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4b70021020 con 0x7f4b880cef10 2024-04-24T23:20:33.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.498+0000 7f4b8d177640 1 -- 172.21.15.26:0/750339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b880b3350 con 0x7f4b880cef10 2024-04-24T23:20:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.498+0000 7f4b7f7fe640 1 -- 172.21.15.26:0/750339022 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4b7001c440 con 0x7f4b880cef10 2024-04-24T23:20:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:33.500 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:33.243284+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:20:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.499+0000 7f4b8d177640 1 -- 172.21.15.26:0/750339022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b880cef10 msgr2=0x7f4b8815ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.499+0000 7f4b8d177640 1 --2- 172.21.15.26:0/750339022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b880cef10 0x7f4b8815ea20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c500 tx=0x7f4b7000c900 comp rx=0 tx=0).stop 2024-04-24T23:20:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.499+0000 7f4b8d177640 1 -- 172.21.15.26:0/750339022 shutdown_connections 2024-04-24T23:20:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.499+0000 7f4b8d177640 1 --2- 172.21.15.26:0/750339022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b8815e2b0 0x7f4b881f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.499+0000 7f4b8d177640 1 --2- 172.21.15.26:0/750339022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b880cef10 0x7f4b8815ea20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.499+0000 7f4b8d177640 1 --2- 172.21.15.26:0/750339022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b880b1b90 0x7f4b880b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.499+0000 7f4b8d177640 1 -- 172.21.15.26:0/750339022 >> 172.21.15.26:0/750339022 conn(0x7f4b880d1f80 msgr2=0x7f4b880d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.499+0000 7f4b8d177640 1 -- 172.21.15.26:0/750339022 shutdown_connections 2024-04-24T23:20:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:33.499+0000 7f4b8d177640 1 -- 172.21.15.26:0/750339022 wait complete. 2024-04-24T23:20:33.512 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-24T23:20:33.513 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-24T23:20:33.519 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:33.517+0000 7facbe321040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:20:33.669 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:33.667+0000 7f60b4d28040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:20:33.724 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:33.722+0000 7f60b4d28040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:20:33.790 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:33.789+0000 7facbe321040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:20:33.845 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:33.843+0000 7facbe321040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:20:33.845 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:33.844+0000 7f60b4d28040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:20:33.868 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-04-24T23:20:33.866+0000 7f5c3f015640 -1 Falling back to public interface 2024-04-24T23:20:33.872 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-04-24T23:20:33.870+0000 7f681343a640 -1 Falling back to public interface 2024-04-24T23:20:33.884 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-04-24T23:20:33.882+0000 7fe45c61e640 -1 Falling back to public interface 2024-04-24T23:20:33.885 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-04-24T23:20:33.883+0000 7f9a07a0d640 -1 Falling back to public interface 2024-04-24T23:20:33.907 INFO:tasks.ceph.osd.5.smithi029.stderr:2024-04-24T23:20:33.905+0000 7fdab8810640 -1 Falling back to public interface 2024-04-24T23:20:33.920 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-04-24T23:20:33.918+0000 7f2abdfab640 -1 Falling back to public interface 2024-04-24T23:20:33.920 INFO:tasks.ceph.osd.6.smithi029.stderr:2024-04-24T23:20:33.919+0000 7fe1f0b23640 -1 Falling back to public interface 2024-04-24T23:20:33.928 INFO:tasks.ceph.osd.7.smithi029.stderr:2024-04-24T23:20:33.926+0000 7fedd2818640 -1 Falling back to public interface 2024-04-24T23:20:33.972 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:33.971+0000 7facbe321040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:20:34.142 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:34.140+0000 7f60b4d28040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:20:34.192 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:34.191+0000 7f60b4d28040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:20:34.266 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:34.265+0000 7facbe321040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:20:34.316 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:34.315+0000 7facbe321040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:20:34.318 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T23:20:34.358 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:34.356+0000 7f60b4d28040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:20:34.410 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.408+0000 7ff0d415a640 1 Processor -- start 2024-04-24T23:20:34.410 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.409+0000 7ff0d415a640 1 -- start start 2024-04-24T23:20:34.413 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d415a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc13dcb0 0x7ff0cc072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:34.413 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d415a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff0cc15f350 0x7ff0cc0725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:34.413 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d415a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0cc15b5c0 con 0x7ff0cc13e090 2024-04-24T23:20:34.413 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d415a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff0cc15f990 con 0x7ff0cc15f350 2024-04-24T23:20:34.413 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d415a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc15b8a0 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.413 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d26f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff0cc15f350 0x7ff0cc0725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:34.413 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d16f5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff0cc13e090 0x7ff0cc15ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56300/0 (socket says 172.21.15.26:56300) 2024-04-24T23:20:34.413 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d16f5640 1 -- 172.21.15.26:0/1609579680 learned_addr learned my addr 172.21.15.26:0/1609579680 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d1ef6640 1 --2- 172.21.15.26:0/1609579680 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc13dcb0 0x7ff0cc072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d26f7640 1 -- 172.21.15.26:0/1609579680 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc13dcb0 msgr2=0x7ff0cc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d26f7640 1 --2- 172.21.15.26:0/1609579680 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc13dcb0 0x7ff0cc072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d26f7640 1 -- 172.21.15.26:0/1609579680 >> v1:172.21.15.26:6789/0 conn(0x7ff0cc13e090 legacy=0x7ff0cc15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d0ef4640 1 -- 172.21.15.26:0/1609579680 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1901455600 0 0) 0x7ff0cc15b5c0 con 0x7ff0cc13e090 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.410+0000 7ff0d26f7640 1 -- 172.21.15.26:0/1609579680 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc15bb80 con 0x7ff0cc15f350 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.411+0000 7ff0d26f7640 1 --2- 172.21.15.26:0/1609579680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc15f350 0x7ff0cc0725b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009d90 tx=0x7ff0c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d7817f5590bd06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.411+0000 7ff0d0ef4640 1 -- 172.21.15.26:0/1609579680 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4007740 con 0x7ff0cc15f350 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.411+0000 7ff0d0ef4640 1 -- 172.21.15.26:0/1609579680 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c40078e0 con 0x7ff0cc15f350 2024-04-24T23:20:34.414 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.411+0000 7ff0d0ef4640 1 -- 172.21.15.26:0/1609579680 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4007cb0 con 0x7ff0cc15f350 2024-04-24T23:20:34.415 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.412+0000 7ff0d415a640 1 -- 172.21.15.26:0/1609579680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc15f350 msgr2=0x7ff0cc0725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:34.415 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.412+0000 7ff0d415a640 1 --2- 172.21.15.26:0/1609579680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc15f350 0x7ff0cc0725b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009d90 tx=0x7ff0c402a660 comp rx=0 tx=0).stop 2024-04-24T23:20:34.415 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.412+0000 7ff0d415a640 1 -- 172.21.15.26:0/1609579680 shutdown_connections 2024-04-24T23:20:34.415 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.412+0000 7ff0d415a640 1 --2- 172.21.15.26:0/1609579680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc15f350 0x7ff0cc0725b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:34.415 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.412+0000 7ff0d415a640 1 --2- 172.21.15.26:0/1609579680 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc13dcb0 0x7ff0cc072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:34.415 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.412+0000 7ff0d415a640 1 -- 172.21.15.26:0/1609579680 >> 172.21.15.26:0/1609579680 conn(0x7ff0cc06cb70 msgr2=0x7ff0cc06e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:34.415 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.412+0000 7ff0d415a640 1 -- 172.21.15.26:0/1609579680 shutdown_connections 2024-04-24T23:20:34.415 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.412+0000 7ff0d415a640 1 -- 172.21.15.26:0/1609579680 wait complete. 2024-04-24T23:20:34.416 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.414+0000 7ff0d415a640 1 Processor -- start 2024-04-24T23:20:34.416 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.414+0000 7ff0d415a640 1 -- start start 2024-04-24T23:20:34.418 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d415a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc13dcb0 0x7ff0cc13a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:34.418 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d415a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0cc13e090 0x7ff0cc151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d415a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc15f350 0x7ff0cc0b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d415a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc0cf6a0 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d415a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc15c360 con 0x7ff0cc13e090 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d415a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc0cf820 con 0x7ff0cc15f350 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d1ef6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc13dcb0 0x7ff0cc13a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d1ef6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc13dcb0 0x7ff0cc13a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55634/0 (socket says 172.21.15.26:55634) 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d1ef6640 1 -- 172.21.15.26:0/892797494 learned_addr learned my addr 172.21.15.26:0/892797494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d26f7640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc15f350 0x7ff0cc0b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d16f5640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0cc13e090 0x7ff0cc151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d1ef6640 1 -- 172.21.15.26:0/892797494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc15f350 msgr2=0x7ff0cc0b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d1ef6640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc15f350 0x7ff0cc0b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d1ef6640 1 -- 172.21.15.26:0/892797494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0cc13e090 msgr2=0x7ff0cc151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d1ef6640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0cc13e090 0x7ff0cc151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:34.419 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d1ef6640 1 -- 172.21.15.26:0/892797494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc151b50 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.415+0000 7ff0d26f7640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc15f350 0x7ff0cc0b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.416+0000 7ff0d1ef6640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc13dcb0 0x7ff0cc13a340 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800e9b0 tx=0x7ff0c800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.416+0000 7ff0c27fc640 1 -- 172.21.15.26:0/892797494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8017070 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.416+0000 7ff0c27fc640 1 -- 172.21.15.26:0/892797494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c800c9a0 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.416+0000 7ff0d415a640 1 -- 172.21.15.26:0/892797494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc0b3290 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.416+0000 7ff0c27fc640 1 -- 172.21.15.26:0/892797494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800cd90 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.416+0000 7ff0d415a640 1 -- 172.21.15.26:0/892797494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc0b3850 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.416+0000 7ff0c27fc640 1 -- 172.21.15.26:0/892797494 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff0c801a070 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.416+0000 7ff0c27fc640 1 -- 172.21.15.26:0/892797494 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ff0c80066b0 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.420 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.417+0000 7ff0d415a640 1 -- 172.21.15.26:0/892797494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0cc072c30 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.421 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.419+0000 7ff0c27fc640 1 -- 172.21.15.26:0/892797494 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff0cc072c30 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.481 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:34.480+0000 7facbe321040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:20:34.484 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.482+0000 7ff0d415a640 1 -- 172.21.15.26:0/892797494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0cc15ebc0 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.485 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.483+0000 7ff0c27fc640 1 -- 172.21.15.26:0/892797494 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ff0cc151b50 con 0x7ff0cc13dcb0 2024-04-24T23:20:34.485 INFO:teuthology.misc.health.smithi026.stdout: 2024-04-24T23:20:34.485 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":10,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:33.243284+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:20:34.486 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.484+0000 7ff0d415a640 1 -- 172.21.15.26:0/892797494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc13dcb0 msgr2=0x7ff0cc13a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:34.486 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.484+0000 7ff0d415a640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc13dcb0 0x7ff0cc13a340 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800e9b0 tx=0x7ff0c800ee70 comp rx=0 tx=0).stop 2024-04-24T23:20:34.486 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.484+0000 7ff0d415a640 1 -- 172.21.15.26:0/892797494 shutdown_connections 2024-04-24T23:20:34.486 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.484+0000 7ff0d415a640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff0cc15f350 0x7ff0cc0b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:34.486 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.484+0000 7ff0d415a640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0cc13e090 0x7ff0cc151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:34.486 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.484+0000 7ff0d415a640 1 --2- 172.21.15.26:0/892797494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0cc13dcb0 0x7ff0cc13a340 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:34.487 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.484+0000 7ff0d415a640 1 -- 172.21.15.26:0/892797494 >> 172.21.15.26:0/892797494 conn(0x7ff0cc06cb70 msgr2=0x7ff0cc13c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:34.487 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.484+0000 7ff0d415a640 1 -- 172.21.15.26:0/892797494 shutdown_connections 2024-04-24T23:20:34.487 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:34.484+0000 7ff0d415a640 1 -- 172.21.15.26:0/892797494 wait complete. 2024-04-24T23:20:34.495 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T23:20:34.507 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:34.506+0000 7f60b4d28040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:20:34.560 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:34.558+0000 7f60b4d28040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:20:34.625 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:34.623+0000 7facbe321040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:20:34.633 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:34.632+0000 7f60b4d28040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:20:34.678 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:34.675+0000 7facbe321040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:20:34.681 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:34.679+0000 7f60b4d28040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:20:34.751 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:34.749+0000 7facbe321040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:20:34.798 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:34.797+0000 7facbe321040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:20:35.186 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:35.184+0000 7f60b4d28040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:20:35.299 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:35.298+0000 7facbe321040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:20:35.307 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:35.305+0000 7f60b4d28040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:20:35.423 INFO:tasks.ceph.mgr.x.smithi029.stderr:2024-04-24T23:20:35.421+0000 7facbe321040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:20:36.322 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:36.320+0000 7f6055850640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T23:20:36.322 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-24T23:20:36.320+0000 7f6055850640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T23:20:37.356 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-04-24T23:20:37.354+0000 7f5c3f015640 -1 osd.2 0 log_to_monitors true 2024-04-24T23:20:37.362 INFO:tasks.ceph.osd.5.smithi029.stderr:2024-04-24T23:20:37.360+0000 7fdab8810640 -1 osd.5 0 log_to_monitors true 2024-04-24T23:20:37.397 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-04-24T23:20:37.395+0000 7f681343a640 -1 osd.3 0 log_to_monitors true 2024-04-24T23:20:37.414 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-04-24T23:20:37.412+0000 7fe45c61e640 -1 osd.1 0 log_to_monitors true 2024-04-24T23:20:37.417 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-04-24T23:20:37.416+0000 7f9a07a0d640 -1 osd.0 0 log_to_monitors true 2024-04-24T23:20:37.418 INFO:tasks.ceph.osd.7.smithi029.stderr:2024-04-24T23:20:37.417+0000 7fedd2818640 -1 osd.7 0 log_to_monitors true 2024-04-24T23:20:37.419 INFO:tasks.ceph.osd.6.smithi029.stderr:2024-04-24T23:20:37.417+0000 7fe1f0b23640 -1 osd.6 0 log_to_monitors true 2024-04-24T23:20:37.433 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-04-24T23:20:37.432+0000 7f2abdfab640 -1 osd.4 0 log_to_monitors true 2024-04-24T23:20:40.642 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-04-24T23:20:40.640+0000 7fe457847640 -1 osd.1 0 waiting for initial osdmap 2024-04-24T23:20:40.653 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-04-24T23:20:40.651+0000 7f9a02ad0640 -1 osd.0 0 waiting for initial osdmap 2024-04-24T23:20:40.655 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-04-24T23:20:40.653+0000 7f680ebad640 -1 osd.3 0 waiting for initial osdmap 2024-04-24T23:20:40.658 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-04-24T23:20:40.657+0000 7fe44fe79640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:40.668 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-04-24T23:20:40.667+0000 7f99fb102640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:40.669 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-04-24T23:20:40.668+0000 7f68071df640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:40.714 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-04-24T23:20:40.712+0000 7f2ab971e640 -1 osd.4 0 waiting for initial osdmap 2024-04-24T23:20:40.734 INFO:tasks.ceph.osd.5.smithi029.stderr:2024-04-24T23:20:40.732+0000 7fdab39bc640 -1 osd.5 0 waiting for initial osdmap 2024-04-24T23:20:40.734 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-04-24T23:20:40.733+0000 7f2ab1d50640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:40.749 INFO:tasks.ceph.osd.5.smithi029.stderr:2024-04-24T23:20:40.748+0000 7fdaabfee640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:40.757 INFO:tasks.ceph.osd.6.smithi029.stderr:2024-04-24T23:20:40.755+0000 7fe1ec296640 -1 osd.6 0 waiting for initial osdmap 2024-04-24T23:20:40.768 INFO:tasks.ceph.osd.6.smithi029.stderr:2024-04-24T23:20:40.767+0000 7fe1e48c8640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:40.787 INFO:tasks.ceph.osd.7.smithi029.stderr:2024-04-24T23:20:40.785+0000 7fedcf1fc640 -1 osd.7 0 waiting for initial osdmap 2024-04-24T23:20:40.800 INFO:tasks.ceph.osd.7.smithi029.stderr:2024-04-24T23:20:40.799+0000 7fedc5ff6640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:41.226 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-04-24T23:20:41.224+0000 7f5c3a0d8640 -1 osd.2 0 waiting for initial osdmap 2024-04-24T23:20:41.234 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-04-24T23:20:41.233+0000 7f5c3270a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:41.301 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T23:20:41.396 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.394+0000 7f4044615640 1 Processor -- start 2024-04-24T23:20:41.396 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.394+0000 7f4044615640 1 -- start start 2024-04-24T23:20:41.399 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4044615640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f403c0cd650 0x7f403c0d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.399 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4044615640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c09b5b0 con 0x7f403c06cf40 2024-04-24T23:20:41.399 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4044615640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c0d2960 con 0x7f403c15e530 2024-04-24T23:20:41.399 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4044615640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f403c09b380 con 0x7f403c0cd650 2024-04-24T23:20:41.399 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4042bb2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f403c0cd650 0x7f403c0d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.399 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4041bb0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f403c06cf40 0x7f403c0d0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56314/0 (socket says 172.21.15.26:56314) 2024-04-24T23:20:41.399 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4042bb2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f403c0cd650 0x7f403c0d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55744/0 (socket says 172.21.15.26:55744) 2024-04-24T23:20:41.399 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4041bb0640 1 -- 172.21.15.26:0/512197416 learned_addr learned my addr 172.21.15.26:0/512197416 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:41.399 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4042bb2640 1 -- 172.21.15.26:0/512197416 >> v1:172.21.15.29:6789/0 conn(0x7f403c15e530 legacy=0x7f403c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4042bb2640 1 -- 172.21.15.26:0/512197416 >> v1:172.21.15.26:6789/0 conn(0x7f403c06cf40 legacy=0x7f403c0d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f40413af640 1 -- 172.21.15.26:0/512197416 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180282268 0 0) 0x7f403c09b5b0 con 0x7f403c06cf40 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.395+0000 7f4042bb2640 1 -- 172.21.15.26:0/512197416 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0d2c40 con 0x7f403c0cd650 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.396+0000 7f4042bb2640 1 --2- 172.21.15.26:0/512197416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0cd650 0x7f403c0d0790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d90 tx=0x7f403402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3534cf1767d00b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.396+0000 7f40413af640 1 -- 172.21.15.26:0/512197416 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034031600 con 0x7f403c0cd650 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.396+0000 7f40413af640 1 -- 172.21.15.26:0/512197416 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40340317a0 con 0x7f403c0cd650 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.396+0000 7f40413af640 1 -- 172.21.15.26:0/512197416 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034031b70 con 0x7f403c0cd650 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.396+0000 7f4044615640 1 -- 172.21.15.26:0/512197416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0cd650 msgr2=0x7f403c0d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.396+0000 7f4044615640 1 --2- 172.21.15.26:0/512197416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0cd650 0x7f403c0d0790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d90 tx=0x7f403402a5d0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.397+0000 7f4044615640 1 -- 172.21.15.26:0/512197416 shutdown_connections 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.397+0000 7f4044615640 1 --2- 172.21.15.26:0/512197416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0cd650 0x7f403c0d0790 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.400 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.397+0000 7f4044615640 1 -- 172.21.15.26:0/512197416 >> 172.21.15.26:0/512197416 conn(0x7f403c0d1e40 msgr2=0x7f403c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:41.401 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.397+0000 7f4044615640 1 -- 172.21.15.26:0/512197416 shutdown_connections 2024-04-24T23:20:41.401 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.397+0000 7f4044615640 1 -- 172.21.15.26:0/512197416 wait complete. 2024-04-24T23:20:41.403 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.398+0000 7f4044615640 1 Processor -- start 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.398+0000 7f4044615640 1 -- start start 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.398+0000 7f4044615640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f403c06cf40 0x7f403c0b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.398+0000 7f4044615640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f403c0cd650 0x7f403c0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.398+0000 7f4044615640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0b4a40 0x7f403c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.398+0000 7f4044615640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f403c06d520 con 0x7f403c0b4a40 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.398+0000 7f4044615640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f403c06c450 con 0x7f403c0cd650 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.398+0000 7f4044615640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f403c0b13a0 con 0x7f403c06cf40 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4042bb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0b4a40 0x7f403c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4042bb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0b4a40 0x7f403c0b4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55754/0 (socket says 172.21.15.26:55754) 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4042bb2640 1 -- 172.21.15.26:0/2073758298 learned_addr learned my addr 172.21.15.26:0/2073758298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:41.404 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4041bb0640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f403c0cd650 0x7f403c0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f40423b1640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f403c06cf40 0x7f403c0b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4042bb2640 1 -- 172.21.15.26:0/2073758298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f403c06cf40 msgr2=0x7f403c0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4042bb2640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f403c06cf40 0x7f403c0b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4042bb2640 1 -- 172.21.15.26:0/2073758298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f403c0cd650 msgr2=0x7f403c0b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4042bb2640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f403c0cd650 0x7f403c0b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4042bb2640 1 -- 172.21.15.26:0/2073758298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c1f5270 con 0x7f403c0b4a40 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4042bb2640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0b4a40 0x7f403c0b4ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d90 tx=0x7f4034004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4032ffd640 1 -- 172.21.15.26:0/2073758298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034040070 con 0x7f403c0b4a40 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4032ffd640 1 -- 172.21.15.26:0/2073758298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4034030b60 con 0x7f403c0b4a40 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4032ffd640 1 -- 172.21.15.26:0/2073758298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403403c420 con 0x7f403c0b4a40 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c160bc0 con 0x7f403c0b4a40 2024-04-24T23:20:41.405 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.399+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c160600 con 0x7f403c0b4a40 2024-04-24T23:20:41.406 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.400+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c0cf980 con 0x7f403c0b4a40 2024-04-24T23:20:41.406 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.401+0000 7f4032ffd640 1 -- 172.21.15.26:0/2073758298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f4034039040 con 0x7f403c0b4a40 2024-04-24T23:20:41.406 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.401+0000 7f4032ffd640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f402006bda0 0x7f402006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.406 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.401+0000 7f4032ffd640 1 -- 172.21.15.26:0/2073758298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f40340ab010 con 0x7f403c0b4a40 2024-04-24T23:20:41.406 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.401+0000 7f40423b1640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f402006bda0 0x7f402006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.406 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.402+0000 7f40423b1640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f402006bda0 0x7f402006e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4038000f60 tx=0x7f4038009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:41.407 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.406+0000 7f4032ffd640 1 -- 172.21.15.26:0/2073758298 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f403c0cf980 con 0x7f403c0b4a40 2024-04-24T23:20:41.593 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.591+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f403c0b6f10 con 0x7f403c0b4a40 2024-04-24T23:20:41.593 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.592+0000 7f4032ffd640 1 -- 172.21.15.26:0/2073758298 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f403407a980 con 0x7f403c0b4a40 2024-04-24T23:20:41.594 INFO:teuthology.misc.health.smithi026.stdout: 2024-04-24T23:20:41.594 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":12,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:39.319832+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:20:41.596 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f402006bda0 msgr2=0x7f402006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.596 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f402006bda0 0x7f402006e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4038000f60 tx=0x7f4038009040 comp rx=0 tx=0).stop 2024-04-24T23:20:41.596 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0b4a40 msgr2=0x7f403c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.596 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0b4a40 0x7f403c0b4ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d90 tx=0x7f4034004a40 comp rx=0 tx=0).stop 2024-04-24T23:20:41.596 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 shutdown_connections 2024-04-24T23:20:41.596 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f402006bda0 0x7f402006e260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.596 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c0b4a40 0x7f403c0b4ef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.596 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f403c0cd650 0x7f403c0b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.597 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 --2- 172.21.15.26:0/2073758298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f403c06cf40 0x7f403c0b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.597 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 >> 172.21.15.26:0/2073758298 conn(0x7f403c0d1e40 msgr2=0x7f403c0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:41.597 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 shutdown_connections 2024-04-24T23:20:41.597 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:41.594+0000 7f4044615640 1 -- 172.21.15.26:0/2073758298 wait complete. 2024-04-24T23:20:41.605 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T23:20:48.412 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T23:20:48.517 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.515+0000 7fabeef6a640 1 Processor -- start 2024-04-24T23:20:48.518 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.515+0000 7fabeef6a640 1 -- start start 2024-04-24T23:20:48.528 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabeef6a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe815e350 0x7fabe80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.528 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabeef6a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fabe80b37a0 0x7fabe80ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.528 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabeef6a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabe8073640 con 0x7fabe80731b0 2024-04-24T23:20:48.528 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabeef6a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fabe80ad070 con 0x7fabe80b37a0 2024-04-24T23:20:48.528 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabeef6a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fabe80ad350 con 0x7fabe815e350 2024-04-24T23:20:48.528 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabe7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fabe80731b0 0x7fabe80b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37832/0 (socket says 172.21.15.26:37832) 2024-04-24T23:20:48.529 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabe7fff640 1 -- 172.21.15.26:0/3562345373 learned_addr learned my addr 172.21.15.26:0/3562345373 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:48.529 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabecd06640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe815e350 0x7fabe80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.529 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabed507640 1 --2- 172.21.15.26:0/3562345373 >> v2:172.21.15.29:3300/0 conn(0x7fabe80b37a0 0x7fabe80ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.529 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599825654 0 0) 0x7fabe8073640 con 0x7fabe80731b0 2024-04-24T23:20:48.529 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.516+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabd0003520 con 0x7fabe80731b0 2024-04-24T23:20:48.529 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 173583620 0 0) 0x7fabd0003520 con 0x7fabe80731b0 2024-04-24T23:20:48.530 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe815e350 msgr2=0x7fabe80b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.530 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe815e350 0x7fabe80b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.530 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 >> v2:172.21.15.29:3300/0 conn(0x7fabe80b37a0 msgr2=0x7fabe80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.530 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 --2- 172.21.15.26:0/3562345373 >> v2:172.21.15.29:3300/0 conn(0x7fabe80b37a0 0x7fabe80ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.530 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe80ad630 con 0x7fabe80731b0 2024-04-24T23:20:48.530 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fabdc001670 con 0x7fabe80731b0 2024-04-24T23:20:48.531 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 >> v1:172.21.15.26:6789/0 conn(0x7fabe80731b0 legacy=0x7fabe80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.531 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabd00039b0 0x7fabd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.531 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe815e350 0x7fabd0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.531 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabd0004d40 0x7fabd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.531 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fabe80ad070 con 0x7fabd0004d40 2024-04-24T23:20:48.531 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fabe80ad350 con 0x7fabd00039b0 2024-04-24T23:20:48.532 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fabd000b9c0 con 0x7fabe815e350 2024-04-24T23:20:48.532 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabecd06640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabd00039b0 0x7fabd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.532 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabed507640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabd0004d40 0x7fabd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.532 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabe7fff640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe815e350 0x7fabd0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.532 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabed507640 1 -- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe815e350 msgr2=0x7fabd0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.532 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabed507640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe815e350 0x7fabd0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.533 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabed507640 1 -- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabd00039b0 msgr2=0x7fabd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.533 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabed507640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabd00039b0 0x7fabd0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.534 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.517+0000 7fabed507640 1 -- 172.21.15.26:0/3562345373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd0003520 con 0x7fabd0004d40 2024-04-24T23:20:48.534 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.518+0000 7fabed507640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabd0004d40 0x7fabd00051c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009f90 tx=0x7fabd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecde0f6c1a2d43e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.534 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.518+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0009c10 con 0x7fabd0004d40 2024-04-24T23:20:48.534 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.518+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe0036070 con 0x7fabd0004d40 2024-04-24T23:20:48.534 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.518+0000 7fabe77fe640 1 -- 172.21.15.26:0/3562345373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0031880 con 0x7fabd0004d40 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.519+0000 7fabeef6a640 1 -- 172.21.15.26:0/3562345373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabd0004d40 msgr2=0x7fabd00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.519+0000 7fabeef6a640 1 --2- 172.21.15.26:0/3562345373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabd0004d40 0x7fabd00051c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009f90 tx=0x7fabd0005830 comp rx=0 tx=0).stop 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.519+0000 7fabed507640 1 -- 172.21.15.26:0/3562345373 reap_dead start 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.519+0000 7fabeef6a640 1 -- 172.21.15.26:0/3562345373 shutdown_connections 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.519+0000 7fabeef6a640 1 -- 172.21.15.26:0/3562345373 >> 172.21.15.26:0/3562345373 conn(0x7fabe80b57d0 msgr2=0x7fabe80b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.519+0000 7fabeef6a640 1 -- 172.21.15.26:0/3562345373 shutdown_connections 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.519+0000 7fabeef6a640 1 -- 172.21.15.26:0/3562345373 wait complete. 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.520+0000 7fabeef6a640 1 Processor -- start 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.520+0000 7fabeef6a640 1 -- start start 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabeef6a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe8151ab0 0x7fabe8153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabeef6a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe8152010 0x7fabe80cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabeef6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe81606d0 0x7fabe8160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabeef6a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fabe815e730 con 0x7fabe8152010 2024-04-24T23:20:48.535 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabeef6a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8071bb0 con 0x7fabe81606d0 2024-04-24T23:20:48.536 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabeef6a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fabe80af350 con 0x7fabe8151ab0 2024-04-24T23:20:48.536 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabe7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe8152010 0x7fabe80cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.536 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabe7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe8152010 0x7fabe80cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33174/0 (socket says 172.21.15.26:33174) 2024-04-24T23:20:48.536 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabe7fff640 1 -- 172.21.15.26:0/2247156021 learned_addr learned my addr 172.21.15.26:0/2247156021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:48.536 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabe7fff640 1 -- 172.21.15.26:0/2247156021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe8151ab0 msgr2=0x7fabe8153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.536 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabecd06640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe8151ab0 0x7fabe8153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.536 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabed507640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe81606d0 0x7fabe8160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.536 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabe7fff640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe8151ab0 0x7fabe8153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.536 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabe7fff640 1 -- 172.21.15.26:0/2247156021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe81606d0 msgr2=0x7fabe8160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.537 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabe7fff640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe81606d0 0x7fabe8160b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.537 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabe7fff640 1 -- 172.21.15.26:0/2247156021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe815fdd0 con 0x7fabe8152010 2024-04-24T23:20:48.537 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.521+0000 7fabe7fff640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe8152010 0x7fabe80cd1e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0072c0 tx=0x7fabdc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.537 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.522+0000 7fabe57fa640 1 -- 172.21.15.26:0/2247156021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00f960 con 0x7fabe8152010 2024-04-24T23:20:48.537 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.522+0000 7fabe57fa640 1 -- 172.21.15.26:0/2247156021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabdc018070 con 0x7fabe8152010 2024-04-24T23:20:48.537 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.522+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe8161630 con 0x7fabe8152010 2024-04-24T23:20:48.537 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.522+0000 7fabe57fa640 1 -- 172.21.15.26:0/2247156021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00fd60 con 0x7fabe8152010 2024-04-24T23:20:48.537 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.522+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe80cd720 con 0x7fabe8152010 2024-04-24T23:20:48.537 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.523+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe80b4810 con 0x7fabe8152010 2024-04-24T23:20:48.538 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.523+0000 7fabe57fa640 1 -- 172.21.15.26:0/2247156021 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fabdc01d460 con 0x7fabe8152010 2024-04-24T23:20:48.538 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.524+0000 7fabe57fa640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fabd406be60 0x7fabd406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.538 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.524+0000 7fabe57fa640 1 -- 172.21.15.26:0/2247156021 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fabdc08ac00 con 0x7fabe8152010 2024-04-24T23:20:48.538 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.524+0000 7fabecd06640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fabd406be60 0x7fabd406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.538 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.524+0000 7fabecd06640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fabd406be60 0x7fabd406e320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009980 tx=0x7fabd8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.539 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.529+0000 7fabe57fa640 1 -- 172.21.15.26:0/2247156021 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fabe80b4810 con 0x7fabe8152010 2024-04-24T23:20:48.714 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.712+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabe804f570 con 0x7fabe8152010 2024-04-24T23:20:48.715 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.713+0000 7fabe57fa640 1 -- 172.21.15.26:0/2247156021 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12824 (secure 0 0 0) 0x7fabdc091080 con 0x7fabe8152010 2024-04-24T23:20:48.715 INFO:teuthology.misc.health.smithi026.stdout: 2024-04-24T23:20:48.715 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":16,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:45.439439+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:20:48.717 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fabd406be60 msgr2=0x7fabd406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.717 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fabd406be60 0x7fabd406e320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009980 tx=0x7fabd8008040 comp rx=0 tx=0).stop 2024-04-24T23:20:48.717 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe8152010 msgr2=0x7fabe80cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.717 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe8152010 0x7fabe80cd1e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0072c0 tx=0x7fabdc007780 comp rx=0 tx=0).stop 2024-04-24T23:20:48.717 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 shutdown_connections 2024-04-24T23:20:48.717 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fabd406be60 0x7fabd406e320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.717 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fabe81606d0 0x7fabe8160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.718 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabe8152010 0x7fabe80cd1e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.718 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 --2- 172.21.15.26:0/2247156021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fabe8151ab0 0x7fabe8153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.718 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 >> 172.21.15.26:0/2247156021 conn(0x7fabe80b57d0 msgr2=0x7fabe80add30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.718 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 shutdown_connections 2024-04-24T23:20:48.718 INFO:teuthology.misc.health.smithi026.stderr:2024-04-24T23:20:48.715+0000 7fabeef6a640 1 -- 172.21.15.26:0/2247156021 wait complete. 2024-04-24T23:20:48.726 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-24T23:20:48.727 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-24T23:20:48.727 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:20:48.727 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:20:48.729 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:20:48.730 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:20:48.730 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:20:48.732 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:20:48.732 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:20:48.732 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:20:48.734 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:20:48.735 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:20:48.735 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:20:48.737 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:20:48.737 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-24T23:20:48.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:20:48.807 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:20:48.807 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:20:48.808 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-04-24T23:20:48.806+0000 7f9ba6055640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-24T23:20:48.821 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:20:48.822 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:20:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.841+0000 7fa1bf0ba640 1 Processor -- start 2024-04-24T23:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.841+0000 7fa1bf0ba640 1 -- start start 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bf0ba640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa1b815c2e0 0x7fa1b80cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bf0ba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80cdbc0 0x7fa1b80ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bf0ba640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa1b80ceed0 0x7fa1b80b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bf0ba640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa1b80d0af0 con 0x7fa1b80ceed0 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bf0ba640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa1b80d0dd0 con 0x7fa1b815c2e0 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bf0ba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b80ccba0 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bd657640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa1b80ceed0 0x7fa1b80b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bd657640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa1b80ceed0 0x7fa1b80b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33208/0 (socket says 172.21.15.26:33208) 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bd657640 1 -- 172.21.15.26:0/590216946 learned_addr learned my addr 172.21.15.26:0/590216946 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bce56640 1 --2- 172.21.15.26:0/590216946 >> v2:172.21.15.29:3300/0 conn(0x7fa1b815c2e0 0x7fa1b80cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1b7fff640 1 --2- 172.21.15.26:0/590216946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80cdbc0 0x7fa1b80ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bd657640 1 -- 172.21.15.26:0/590216946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80cdbc0 msgr2=0x7fa1b80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bd657640 1 --2- 172.21.15.26:0/590216946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80cdbc0 0x7fa1b80ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bd657640 1 -- 172.21.15.26:0/590216946 >> v2:172.21.15.29:3300/0 conn(0x7fa1b815c2e0 msgr2=0x7fa1b80cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bd657640 1 --2- 172.21.15.26:0/590216946 >> v2:172.21.15.29:3300/0 conn(0x7fa1b815c2e0 0x7fa1b80cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bd657640 1 -- 172.21.15.26:0/590216946 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b80cce80 con 0x7fa1b80ceed0 2024-04-24T23:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.842+0000 7fa1bce56640 1 --2- 172.21.15.26:0/590216946 >> v2:172.21.15.29:3300/0 conn(0x7fa1b815c2e0 0x7fa1b80cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1bd657640 1 --2- 172.21.15.26:0/590216946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80ceed0 0x7fa1b80b1860 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80cf650 tx=0x7fa1b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286024bb5e973d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1b77fe640 1 -- 172.21.15.26:0/590216946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0036070 con 0x7fa1b80ceed0 2024-04-24T23:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1b77fe640 1 -- 172.21.15.26:0/590216946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b0031600 con 0x7fa1b80ceed0 2024-04-24T23:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1b77fe640 1 -- 172.21.15.26:0/590216946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00319d0 con 0x7fa1b80ceed0 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/590216946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80ceed0 msgr2=0x7fa1b80b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/590216946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80ceed0 0x7fa1b80b1860 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80cf650 tx=0x7fa1b002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/590216946 shutdown_connections 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/590216946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80ceed0 0x7fa1b80b1860 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/590216946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80cdbc0 0x7fa1b80ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/590216946 >> v2:172.21.15.29:3300/0 conn(0x7fa1b815c2e0 0x7fa1b80cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.843+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/590216946 >> 172.21.15.26:0/590216946 conn(0x7fa1b80d1f80 msgr2=0x7fa1b80d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.844+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/590216946 shutdown_connections 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.844+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/590216946 wait complete. 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.845+0000 7fa1bf0ba640 1 Processor -- start 2024-04-24T23:20:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.845+0000 7fa1bf0ba640 1 -- start start 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.845+0000 7fa1bf0ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80cdbc0 0x7fa1b815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.845+0000 7fa1bf0ba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80ceed0 0x7fa1b8151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bf0ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b815c2e0 0x7fa1b814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bf0ba640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b80b1f20 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bf0ba640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b80b1da0 con 0x7fa1b815c2e0 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bf0ba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b80b20a0 con 0x7fa1b80ceed0 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bce56640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80cdbc0 0x7fa1b815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bce56640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80cdbc0 0x7fa1b815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33222/0 (socket says 172.21.15.26:33222) 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bce56640 1 -- 172.21.15.26:0/1818597881 learned_addr learned my addr 172.21.15.26:0/1818597881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1b7fff640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80ceed0 0x7fa1b8151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bd657640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b815c2e0 0x7fa1b814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bce56640 1 -- 172.21.15.26:0/1818597881 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80ceed0 msgr2=0x7fa1b8151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bce56640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80ceed0 0x7fa1b8151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bce56640 1 -- 172.21.15.26:0/1818597881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b815c2e0 msgr2=0x7fa1b814f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bce56640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b815c2e0 0x7fa1b814f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bce56640 1 -- 172.21.15.26:0/1818597881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b8159b20 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bd657640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b815c2e0 0x7fa1b814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bce56640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80cdbc0 0x7fa1b815b6b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00e7e0 tx=0x7fa1ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1b57fa640 1 -- 172.21.15.26:0/1818597881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00f040 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1b57fa640 1 -- 172.21.15.26:0/1818597881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ac004030 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b8159e00 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1b57fa640 1 -- 172.21.15.26:0/1818597881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac004400 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.846+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b814fc50 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.847+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b80681e0 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.848+0000 7fa1b57fa640 1 -- 172.21.15.26:0/1818597881 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1ac00c6d0 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.848+0000 7fa1b57fa640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa19c06be30 0x7fa19c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.848+0000 7fa1b57fa640 1 -- 172.21.15.26:0/1818597881 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa1ac08a8a0 con 0x7fa1b80cdbc0 2024-04-24T23:20:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.848+0000 7fa1b7fff640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa19c06be30 0x7fa19c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.849+0000 7fa1b7fff640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa19c06be30 0x7fa19c06e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80b0d50 tx=0x7fa1a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:48.852+0000 7fa1b57fa640 1 -- 172.21.15.26:0/1818597881 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1ac017020 con 0x7fa1b80cdbc0 2024-04-24T23:20:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.040+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1b80ce770 con 0x7fa1b80cdbc0 2024-04-24T23:20:49.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.041+0000 7fa1b57fa640 1 -- 172.21.15.26:0/1818597881 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12824 (secure 0 0 0) 0x7fa1ac059630 con 0x7fa1b80cdbc0 2024-04-24T23:20:49.043 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:49.043 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:45.439439+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:20:49.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa19c06be30 msgr2=0x7fa19c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa19c06be30 0x7fa19c06e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80b0d50 tx=0x7fa1a4008040 comp rx=0 tx=0).stop 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80cdbc0 msgr2=0x7fa1b815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80cdbc0 0x7fa1b815b6b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00e7e0 tx=0x7fa1ac00eca0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 shutdown_connections 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa19c06be30 0x7fa19c06e2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1b815c2e0 0x7fa1b814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa1b80ceed0 0x7fa1b8151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 --2- 172.21.15.26:0/1818597881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1b80cdbc0 0x7fa1b815b6b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 >> 172.21.15.26:0/1818597881 conn(0x7fa1b80d1f80 msgr2=0x7fa1b80b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 shutdown_connections 2024-04-24T23:20:49.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.043+0000 7fa1bf0ba640 1 -- 172.21.15.26:0/1818597881 wait complete. 2024-04-24T23:20:49.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:20:49.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:20:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.158+0000 7f4c508e1640 1 Processor -- start 2024-04-24T23:20:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.158+0000 7f4c508e1640 1 -- start start 2024-04-24T23:20:49.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c508e1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c4c0d0750 0x7f4c4c0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c508e1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c4c0aeb10 con 0x7f4c4c0b4ce0 2024-04-24T23:20:49.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c508e1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c4c0aedf0 con 0x7f4c4c15be40 2024-04-24T23:20:49.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c508e1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4c4c09df20 con 0x7f4c4c0d0750 2024-04-24T23:20:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c49d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c4c0d0750 0x7f4c4c0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c4ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4c4c0b4ce0 0x7f4c4c15c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37848/0 (socket says 172.21.15.26:37848) 2024-04-24T23:20:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c4ad9d640 1 -- 172.21.15.26:0/233768085 learned_addr learned my addr 172.21.15.26:0/233768085 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c49d9b640 1 -- 172.21.15.26:0/233768085 >> v1:172.21.15.29:6789/0 conn(0x7f4c4c15be40 legacy=0x7f4c4c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c49d9b640 1 -- 172.21.15.26:0/233768085 >> v1:172.21.15.26:6789/0 conn(0x7f4c4c0b4ce0 legacy=0x7f4c4c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c49d9b640 1 -- 172.21.15.26:0/233768085 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c0af0d0 con 0x7f4c4c0d0750 2024-04-24T23:20:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.159+0000 7f4c4959a640 1 -- 172.21.15.26:0/233768085 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307419571 0 0) 0x7f4c4c0aeb10 con 0x7f4c4c0b4ce0 2024-04-24T23:20:49.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.160+0000 7f4c49d9b640 1 --2- 172.21.15.26:0/233768085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 0x7f4c4c0acd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c400097a0 tx=0x7f4c4002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cbd0db2ccd1933 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.160+0000 7f4c4959a640 1 -- 172.21.15.26:0/233768085 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40037070 con 0x7f4c4c0d0750 2024-04-24T23:20:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.160+0000 7f4c4959a640 1 -- 172.21.15.26:0/233768085 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c40032440 con 0x7f4c4c0d0750 2024-04-24T23:20:49.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.160+0000 7f4c4959a640 1 -- 172.21.15.26:0/233768085 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40032810 con 0x7f4c4c0d0750 2024-04-24T23:20:49.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.161+0000 7f4c508e1640 1 -- 172.21.15.26:0/233768085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 msgr2=0x7f4c4c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.161+0000 7f4c508e1640 1 --2- 172.21.15.26:0/233768085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 0x7f4c4c0acd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c400097a0 tx=0x7f4c4002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.161+0000 7f4c508e1640 1 -- 172.21.15.26:0/233768085 shutdown_connections 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.161+0000 7f4c508e1640 1 --2- 172.21.15.26:0/233768085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 0x7f4c4c0acd30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.161+0000 7f4c508e1640 1 -- 172.21.15.26:0/233768085 >> 172.21.15.26:0/233768085 conn(0x7f4c4c06b540 msgr2=0x7f4c4c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.161+0000 7f4c508e1640 1 -- 172.21.15.26:0/233768085 shutdown_connections 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.161+0000 7f4c508e1640 1 -- 172.21.15.26:0/233768085 wait complete. 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.162+0000 7f4c508e1640 1 Processor -- start 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.162+0000 7f4c508e1640 1 -- start start 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c508e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c0b4ce0 0x7f4c4c09d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c508e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 0x7f4c4c09bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c508e1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c4c15be40 0x7f4c4c14d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c508e1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c06cd10 con 0x7f4c4c0d0750 2024-04-24T23:20:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c508e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c06cb90 con 0x7f4c4c0b4ce0 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c508e1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4c4c15c930 con 0x7f4c4c15be40 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c49d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 0x7f4c4c09bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c49d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 0x7f4c4c09bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33238/0 (socket says 172.21.15.26:33238) 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c49d9b640 1 -- 172.21.15.26:0/922850167 learned_addr learned my addr 172.21.15.26:0/922850167 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c4a59c640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c0b4ce0 0x7f4c4c09d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c4ad9d640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c4c15be40 0x7f4c4c14d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c49d9b640 1 -- 172.21.15.26:0/922850167 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c4c15be40 msgr2=0x7f4c4c14d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c49d9b640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c4c15be40 0x7f4c4c14d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c49d9b640 1 -- 172.21.15.26:0/922850167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c0b4ce0 msgr2=0x7f4c4c09d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c49d9b640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c0b4ce0 0x7f4c4c09d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c49d9b640 1 -- 172.21.15.26:0/922850167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c09c440 con 0x7f4c4c0d0750 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.163+0000 7f4c49d9b640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 0x7f4c4c09bcd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4c400097a0 tx=0x7f4c400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.164+0000 7f4c3affd640 1 -- 172.21.15.26:0/922850167 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40041070 con 0x7f4c4c0d0750 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.164+0000 7f4c3affd640 1 -- 172.21.15.26:0/922850167 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4003c870 con 0x7f4c4c0d0750 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.164+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c09c720 con 0x7f4c4c0d0750 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.164+0000 7f4c3affd640 1 -- 172.21.15.26:0/922850167 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40037040 con 0x7f4c4c0d0750 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.164+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c09cce0 con 0x7f4c4c0d0750 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.165+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c075610 con 0x7f4c4c0d0750 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.165+0000 7f4c3affd640 1 -- 172.21.15.26:0/922850167 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c40044070 con 0x7f4c4c0d0750 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.166+0000 7f4c3affd640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c2c06be30 0x7f4c2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.166+0000 7f4c3affd640 1 -- 172.21.15.26:0/922850167 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4c400ab810 con 0x7f4c4c0d0750 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.166+0000 7f4c4a59c640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c2c06be30 0x7f4c2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.166+0000 7f4c4a59c640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c2c06be30 0x7f4c2c06e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006fd0 tx=0x7f4c3c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.170+0000 7f4c3affd640 1 -- 172.21.15.26:0/922850167 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4c4007a630 con 0x7f4c4c0d0750 2024-04-24T23:20:49.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.352+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c4c0b2fa0 con 0x7f4c4c0d0750 2024-04-24T23:20:49.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.353+0000 7f4c3affd640 1 -- 172.21.15.26:0/922850167 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f4c40079f30 con 0x7f4c4c0d0750 2024-04-24T23:20:49.355 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c2c06be30 msgr2=0x7f4c2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c2c06be30 0x7f4c2c06e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006fd0 tx=0x7f4c3c008040 comp rx=0 tx=0).stop 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 msgr2=0x7f4c4c09bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 0x7f4c4c09bcd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4c400097a0 tx=0x7f4c400046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 shutdown_connections 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c2c06be30 0x7f4c2c06e2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c4c15be40 0x7f4c4c14d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c4c0d0750 0x7f4c4c09bcd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 --2- 172.21.15.26:0/922850167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c4c0b4ce0 0x7f4c4c09d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 >> 172.21.15.26:0/922850167 conn(0x7f4c4c06b540 msgr2=0x7f4c4c06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 shutdown_connections 2024-04-24T23:20:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.355+0000 7f4c508e1640 1 -- 172.21.15.26:0/922850167 wait complete. 2024-04-24T23:20:49.368 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:20:49.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.467+0000 7f09bf427640 1 Processor -- start 2024-04-24T23:20:49.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.468+0000 7f09bf427640 1 -- start start 2024-04-24T23:20:49.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.468+0000 7f09bf427640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f09b815e380 0x7f09b80b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.468+0000 7f09bf427640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b4fd0 0x7f09b8072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.468+0000 7f09bf427640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09b80b0030 0x7f09b80b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.468+0000 7f09bf427640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f09b815e760 con 0x7f09b80b0030 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.468+0000 7f09bf427640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f09b80725b0 con 0x7f09b815e380 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.468+0000 7f09bf427640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f09b80b09a0 con 0x7f09b80b4fd0 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bd9c4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09b80b0030 0x7f09b80b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bd9c4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09b80b0030 0x7f09b80b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33254/0 (socket says 172.21.15.26:33254) 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bd9c4640 1 -- 172.21.15.26:0/705552935 learned_addr learned my addr 172.21.15.26:0/705552935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bc9c2640 1 --2- 172.21.15.26:0/705552935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b4fd0 0x7f09b8072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bd1c3640 1 --2- 172.21.15.26:0/705552935 >> v2:172.21.15.29:3300/0 conn(0x7f09b815e380 0x7f09b80b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bd9c4640 1 -- 172.21.15.26:0/705552935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b4fd0 msgr2=0x7f09b8072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:49.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bd9c4640 1 --2- 172.21.15.26:0/705552935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b4fd0 0x7f09b8072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bd9c4640 1 -- 172.21.15.26:0/705552935 >> v2:172.21.15.29:3300/0 conn(0x7f09b815e380 msgr2=0x7f09b80b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bd9c4640 1 --2- 172.21.15.26:0/705552935 >> v2:172.21.15.29:3300/0 conn(0x7f09b815e380 0x7f09b80b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.469+0000 7f09bd9c4640 1 -- 172.21.15.26:0/705552935 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b80b5700 con 0x7f09b80b0030 2024-04-24T23:20:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bd9c4640 1 --2- 172.21.15.26:0/705552935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b0030 0x7f09b80b0460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f09b806f5f0 tx=0x7f09b002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b684fc6a4f9e3eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09affff640 1 -- 172.21.15.26:0/705552935 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0036070 con 0x7f09b80b0030 2024-04-24T23:20:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09affff640 1 -- 172.21.15.26:0/705552935 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b0031600 con 0x7f09b80b0030 2024-04-24T23:20:49.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09affff640 1 -- 172.21.15.26:0/705552935 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b00319d0 con 0x7f09b80b0030 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bf427640 1 -- 172.21.15.26:0/705552935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b0030 msgr2=0x7f09b80b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bf427640 1 --2- 172.21.15.26:0/705552935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b0030 0x7f09b80b0460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f09b806f5f0 tx=0x7f09b002a750 comp rx=0 tx=0).stop 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bf427640 1 -- 172.21.15.26:0/705552935 shutdown_connections 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bf427640 1 --2- 172.21.15.26:0/705552935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b0030 0x7f09b80b0460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bf427640 1 --2- 172.21.15.26:0/705552935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b4fd0 0x7f09b8072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bf427640 1 --2- 172.21.15.26:0/705552935 >> v2:172.21.15.29:3300/0 conn(0x7f09b815e380 0x7f09b80b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bf427640 1 -- 172.21.15.26:0/705552935 >> 172.21.15.26:0/705552935 conn(0x7f09b806e2c0 msgr2=0x7f09b806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bf427640 1 -- 172.21.15.26:0/705552935 shutdown_connections 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.470+0000 7f09bf427640 1 -- 172.21.15.26:0/705552935 wait complete. 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.471+0000 7f09bf427640 1 Processor -- start 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bf427640 1 -- start start 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bf427640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b0030 0x7f09b80c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bf427640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b4fd0 0x7f09b81f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bf427640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09b815e380 0x7f09b814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bf427640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f09b80b5410 con 0x7f09b80b4fd0 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bf427640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8072240 con 0x7f09b815e380 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bf427640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f09b8072f80 con 0x7f09b80b0030 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bc9c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b4fd0 0x7f09b81f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bc9c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b4fd0 0x7f09b81f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33262/0 (socket says 172.21.15.26:33262) 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bc9c2640 1 -- 172.21.15.26:0/521182266 learned_addr learned my addr 172.21.15.26:0/521182266 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bd1c3640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b0030 0x7f09b80c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.472+0000 7f09bd9c4640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09b815e380 0x7f09b814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09bc9c2640 1 -- 172.21.15.26:0/521182266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b0030 msgr2=0x7f09b80c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09bc9c2640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b0030 0x7f09b80c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09bc9c2640 1 -- 172.21.15.26:0/521182266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09b815e380 msgr2=0x7f09b814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09bc9c2640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09b815e380 0x7f09b814f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09bc9c2640 1 -- 172.21.15.26:0/521182266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b80c65a0 con 0x7f09b80b4fd0 2024-04-24T23:20:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09bc9c2640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b4fd0 0x7f09b81f4580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09a800e890 tx=0x7f09a800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09adffb640 1 -- 172.21.15.26:0/521182266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8017070 con 0x7f09b80b4fd0 2024-04-24T23:20:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09adffb640 1 -- 172.21.15.26:0/521182266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a800c940 con 0x7f09b80b4fd0 2024-04-24T23:20:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b81f4db0 con 0x7f09b80b4fd0 2024-04-24T23:20:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09adffb640 1 -- 172.21.15.26:0/521182266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a800cd10 con 0x7f09b80b4fd0 2024-04-24T23:20:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.473+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b81f5370 con 0x7f09b80b4fd0 2024-04-24T23:20:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.474+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b80694c0 con 0x7f09b80b4fd0 2024-04-24T23:20:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.475+0000 7f09adffb640 1 -- 172.21.15.26:0/521182266 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09a801a070 con 0x7f09b80b4fd0 2024-04-24T23:20:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.475+0000 7f09adffb640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099c06be30 0x7f099c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.475+0000 7f09adffb640 1 -- 172.21.15.26:0/521182266 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f09a808aa50 con 0x7f09b80b4fd0 2024-04-24T23:20:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.475+0000 7f09bd1c3640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099c06be30 0x7f099c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.475+0000 7f09bd1c3640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099c06be30 0x7f099c06e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009aa0 tx=0x7f09b4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:49.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.479+0000 7f09adffb640 1 -- 172.21.15.26:0/521182266 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09a8059100 con 0x7f09b80b4fd0 2024-04-24T23:20:49.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.699+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09b815d800 con 0x7f09b80b4fd0 2024-04-24T23:20:49.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.699+0000 7f09adffb640 1 -- 172.21.15.26:0/521182266 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f09a805d400 con 0x7f09b80b4fd0 2024-04-24T23:20:49.701 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:49.701 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:20:49.701 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2024-04-24T23:20:49.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099c06be30 msgr2=0x7f099c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099c06be30 0x7f099c06e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009aa0 tx=0x7f09b4009190 comp rx=0 tx=0).stop 2024-04-24T23:20:49.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b4fd0 msgr2=0x7f09b81f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:49.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b4fd0 0x7f09b81f4580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09a800e890 tx=0x7f09a800ed50 comp rx=0 tx=0).stop 2024-04-24T23:20:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 shutdown_connections 2024-04-24T23:20:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099c06be30 0x7f099c06e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09b815e380 0x7f09b814f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b80b4fd0 0x7f09b81f4580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 --2- 172.21.15.26:0/521182266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09b80b0030 0x7f09b80c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 >> 172.21.15.26:0/521182266 conn(0x7f09b806e2c0 msgr2=0x7f09b806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 shutdown_connections 2024-04-24T23:20:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:49.701+0000 7f09bf427640 1 -- 172.21.15.26:0/521182266 wait complete. 2024-04-24T23:20:51.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:20:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.826+0000 7f1eac3f4640 1 Processor -- start 2024-04-24T23:20:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.826+0000 7f1eac3f4640 1 -- start start 2024-04-24T23:20:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eac3f4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ea415e290 0x7f1ea40d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eac3f4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 0x7f1ea40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eac3f4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea40b5840 con 0x7f1ea40b3ac0 2024-04-24T23:20:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eac3f4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea40b5e50 con 0x7f1ea415e290 2024-04-24T23:20:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eac3f4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea40cfb30 con 0x7f1ea40b72f0 2024-04-24T23:20:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eaa190640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ea415e290 0x7f1ea40d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eaa190640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ea415e290 0x7f1ea40d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33268/0 (socket says 172.21.15.26:33268) 2024-04-24T23:20:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eaa190640 1 -- 172.21.15.26:0/3097391820 learned_addr learned my addr 172.21.15.26:0/3097391820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eaa190640 1 -- 172.21.15.26:0/3097391820 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 msgr2=0x7f1ea40b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eaa991640 1 --2- 172.21.15.26:0/3097391820 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 0x7f1ea40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eaa190640 1 --2- 172.21.15.26:0/3097391820 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 0x7f1ea40b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eaa190640 1 -- 172.21.15.26:0/3097391820 >> v1:172.21.15.29:6789/0 conn(0x7f1ea40b3ac0 legacy=0x7f1ea406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eaa190640 1 -- 172.21.15.26:0/3097391820 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea40cfe10 con 0x7f1ea415e290 2024-04-24T23:20:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.827+0000 7f1eaa991640 1 --2- 172.21.15.26:0/3097391820 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 0x7f1ea40b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.828+0000 7f1eaa190640 1 --2- 172.21.15.26:0/3097391820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 0x7f1ea40d1730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009870 tx=0x7f1ea002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce617c748ae0fb5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.828+0000 7f1ea918e640 1 -- 172.21.15.26:0/3097391820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0037070 con 0x7f1ea415e290 2024-04-24T23:20:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.828+0000 7f1ea918e640 1 -- 172.21.15.26:0/3097391820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea0032440 con 0x7f1ea415e290 2024-04-24T23:20:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.828+0000 7f1ea918e640 1 -- 172.21.15.26:0/3097391820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0032810 con 0x7f1ea415e290 2024-04-24T23:20:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.829+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3097391820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 msgr2=0x7f1ea40d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.829+0000 7f1eac3f4640 1 --2- 172.21.15.26:0/3097391820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 0x7f1ea40d1730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009870 tx=0x7f1ea002a210 comp rx=0 tx=0).stop 2024-04-24T23:20:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.829+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3097391820 shutdown_connections 2024-04-24T23:20:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.829+0000 7f1eac3f4640 1 --2- 172.21.15.26:0/3097391820 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 0x7f1ea40b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.829+0000 7f1eac3f4640 1 --2- 172.21.15.26:0/3097391820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 0x7f1ea40d1730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.829+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3097391820 >> 172.21.15.26:0/3097391820 conn(0x7f1ea406b120 msgr2=0x7f1ea406b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.829+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3097391820 shutdown_connections 2024-04-24T23:20:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.829+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3097391820 wait complete. 2024-04-24T23:20:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.830+0000 7f1eac3f4640 1 Processor -- start 2024-04-24T23:20:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.830+0000 7f1eac3f4640 1 -- start start 2024-04-24T23:20:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eac3f4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ea40b3ac0 0x7f1ea40b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eac3f4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 0x7f1ea415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eac3f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 0x7f1ea40c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eac3f4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea415e670 con 0x7f1ea415e290 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eac3f4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea40b5ae0 con 0x7f1ea40b3ac0 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eac3f4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea40d1c70 con 0x7f1ea40b72f0 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eaa991640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 0x7f1ea40c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eaa991640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 0x7f1ea40c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33282/0 (socket says 172.21.15.26:33282) 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eaa991640 1 -- 172.21.15.26:0/3444885301 learned_addr learned my addr 172.21.15.26:0/3444885301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eaa190640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ea40b3ac0 0x7f1ea40b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1ea998f640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 0x7f1ea415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eaa991640 1 -- 172.21.15.26:0/3444885301 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 msgr2=0x7f1ea415b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eaa991640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 0x7f1ea415b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eaa991640 1 -- 172.21.15.26:0/3444885301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ea40b3ac0 msgr2=0x7f1ea40b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eaa991640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ea40b3ac0 0x7f1ea40b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.831+0000 7f1eaa991640 1 -- 172.21.15.26:0/3444885301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea40b12d0 con 0x7f1ea415e290 2024-04-24T23:20:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.832+0000 7f1eaa991640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 0x7f1ea40c7c60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00cef0 tx=0x7f1e9c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.832+0000 7f1e9affd640 1 -- 172.21.15.26:0/3444885301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c01a070 con 0x7f1ea415e290 2024-04-24T23:20:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.832+0000 7f1e9affd640 1 -- 172.21.15.26:0/3444885301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e9c00d860 con 0x7f1ea415e290 2024-04-24T23:20:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.832+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea40b15b0 con 0x7f1ea415e290 2024-04-24T23:20:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.832+0000 7f1e9affd640 1 -- 172.21.15.26:0/3444885301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c00dc50 con 0x7f1ea415e290 2024-04-24T23:20:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.832+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea415baf0 con 0x7f1ea415e290 2024-04-24T23:20:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.833+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea40694c0 con 0x7f1ea415e290 2024-04-24T23:20:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.833+0000 7f1e9affd640 1 -- 172.21.15.26:0/3444885301 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e9c01d070 con 0x7f1ea415e290 2024-04-24T23:20:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.834+0000 7f1e9affd640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8806be60 0x7f1e8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.834+0000 7f1e9affd640 1 -- 172.21.15.26:0/3444885301 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1e9c08b270 con 0x7f1ea415e290 2024-04-24T23:20:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.834+0000 7f1eaa190640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8806be60 0x7f1e8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:51.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.834+0000 7f1eaa190640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8806be60 0x7f1e8806e320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0034000 tx=0x7f1ea002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:51.838+0000 7f1e9affd640 1 -- 172.21.15.26:0/3444885301 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e9c059a70 con 0x7f1ea415e290 2024-04-24T23:20:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.059+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ea40b4850 con 0x7f1ea415e290 2024-04-24T23:20:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.060+0000 7f1e9affd640 1 -- 172.21.15.26:0/3444885301 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3864 (secure 0 0 0) 0x7f1e9c05dd70 con 0x7f1ea415e290 2024-04-24T23:20:52.062 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:52.062 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2829963318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2829963318},{"type":"v1","addr":"172.21.15.26:6837","nonce":2829963318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2687649505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2687649505},{"type":"v1","addr":"172.21.15.29:6833","nonce":2687649505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1604449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1604449840},{"type":"v1","addr":"172.21.15.29:6835","nonce":1604449840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:20:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2024-04-24T23:20:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8806be60 msgr2=0x7f1e8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8806be60 0x7f1e8806e320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0034000 tx=0x7f1ea002e000 comp rx=0 tx=0).stop 2024-04-24T23:20:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 msgr2=0x7f1ea40c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 0x7f1ea40c7c60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00cef0 tx=0x7f1e9c00a710 comp rx=0 tx=0).stop 2024-04-24T23:20:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 shutdown_connections 2024-04-24T23:20:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8806be60 0x7f1e8806e320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea415e290 0x7f1ea40c7c60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ea40b72f0 0x7f1ea415b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 --2- 172.21.15.26:0/3444885301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ea40b3ac0 0x7f1ea40b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 >> 172.21.15.26:0/3444885301 conn(0x7f1ea406b120 msgr2=0x7f1ea40d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 shutdown_connections 2024-04-24T23:20:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.062+0000 7f1eac3f4640 1 -- 172.21.15.26:0/3444885301 wait complete. 2024-04-24T23:20:52.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:20:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.173+0000 7fdd1d314640 1 Processor -- start 2024-04-24T23:20:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.173+0000 7fdd1d314640 1 -- start start 2024-04-24T23:20:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.174+0000 7fdd1d314640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdd180cef10 0x7fdd1806c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.174+0000 7fdd1d314640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd180b1b90 0x7fdd180d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.174+0000 7fdd1d314640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd180b22e0 con 0x7fdd1815e2b0 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.174+0000 7fdd1d314640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fdd180b2ef0 con 0x7fdd180b1b90 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.174+0000 7fdd1d314640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdd180b25c0 con 0x7fdd180cef10 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.174+0000 7fdd177fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd180b1b90 0x7fdd180d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.174+0000 7fdd177fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd180b1b90 0x7fdd180d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33294/0 (socket says 172.21.15.26:33294) 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.174+0000 7fdd177fe640 1 -- 172.21.15.26:0/500378235 learned_addr learned my addr 172.21.15.26:0/500378235 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.174+0000 7fdd167fc640 1 --2- 172.21.15.26:0/500378235 >> v2:172.21.15.29:3300/0 conn(0x7fdd180cef10 0x7fdd1806c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.175+0000 7fdd177fe640 1 -- 172.21.15.26:0/500378235 >> v2:172.21.15.29:3300/0 conn(0x7fdd180cef10 msgr2=0x7fdd1806c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.175+0000 7fdd177fe640 1 --2- 172.21.15.26:0/500378235 >> v2:172.21.15.29:3300/0 conn(0x7fdd180cef10 0x7fdd1806c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.175+0000 7fdd177fe640 1 -- 172.21.15.26:0/500378235 >> v1:172.21.15.29:6789/0 conn(0x7fdd1815e2b0 legacy=0x7fdd180ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.175+0000 7fdd177fe640 1 -- 172.21.15.26:0/500378235 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd180b28a0 con 0x7fdd180b1b90 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.175+0000 7fdd167fc640 1 --2- 172.21.15.26:0/500378235 >> v2:172.21.15.29:3300/0 conn(0x7fdd180cef10 0x7fdd1806c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.175+0000 7fdd177fe640 1 --2- 172.21.15.26:0/500378235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b1b90 0x7fdd180d04f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd180cfb00 tx=0x7fdd0802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db526a54923f6b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.175+0000 7fdd15ffb640 1 -- 172.21.15.26:0/500378235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08036070 con 0x7fdd180b1b90 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.175+0000 7fdd15ffb640 1 -- 172.21.15.26:0/500378235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08031600 con 0x7fdd180b1b90 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.175+0000 7fdd15ffb640 1 -- 172.21.15.26:0/500378235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd080319d0 con 0x7fdd180b1b90 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.176+0000 7fdd1d314640 1 -- 172.21.15.26:0/500378235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b1b90 msgr2=0x7fdd180d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.176+0000 7fdd1d314640 1 --2- 172.21.15.26:0/500378235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b1b90 0x7fdd180d04f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd180cfb00 tx=0x7fdd0802a730 comp rx=0 tx=0).stop 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.176+0000 7fdd1d314640 1 -- 172.21.15.26:0/500378235 shutdown_connections 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.176+0000 7fdd1d314640 1 --2- 172.21.15.26:0/500378235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b1b90 0x7fdd180d04f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.176+0000 7fdd1d314640 1 --2- 172.21.15.26:0/500378235 >> v2:172.21.15.29:3300/0 conn(0x7fdd180cef10 0x7fdd1806c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.176+0000 7fdd1d314640 1 -- 172.21.15.26:0/500378235 >> 172.21.15.26:0/500378235 conn(0x7fdd180d1f80 msgr2=0x7fdd180d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.176+0000 7fdd1d314640 1 -- 172.21.15.26:0/500378235 shutdown_connections 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.176+0000 7fdd1d314640 1 -- 172.21.15.26:0/500378235 wait complete. 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.177+0000 7fdd1d314640 1 Processor -- start 2024-04-24T23:20:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.177+0000 7fdd1d314640 1 -- start start 2024-04-24T23:20:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd1d314640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd180b1b90 0x7fdd180b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd1d314640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180cef10 0x7fdd180b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd1d314640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1815e2b0 0x7fdd1815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd1d314640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1815e690 con 0x7fdd180cef10 2024-04-24T23:20:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd1d314640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdd180b2000 con 0x7fdd180b1b90 2024-04-24T23:20:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd1d314640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1815e810 con 0x7fdd1815e2b0 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd167fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180cef10 0x7fdd180b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd167fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180cef10 0x7fdd180b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33300/0 (socket says 172.21.15.26:33300) 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd167fc640 1 -- 172.21.15.26:0/836144168 learned_addr learned my addr 172.21.15.26:0/836144168 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd16ffd640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd180b1b90 0x7fdd180b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.178+0000 7fdd177fe640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1815e2b0 0x7fdd1815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd167fc640 1 -- 172.21.15.26:0/836144168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1815e2b0 msgr2=0x7fdd1815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd167fc640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1815e2b0 0x7fdd1815a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd167fc640 1 -- 172.21.15.26:0/836144168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd180b1b90 msgr2=0x7fdd180b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd167fc640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd180b1b90 0x7fdd180b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd167fc640 1 -- 172.21.15.26:0/836144168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd180b6510 con 0x7fdd180cef10 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd167fc640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180cef10 0x7fdd180b72d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000e7e0 tx=0x7fdd0000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd0f7fe640 1 -- 172.21.15.26:0/836144168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000f040 con 0x7fdd180cef10 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd0f7fe640 1 -- 172.21.15.26:0/836144168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd00004030 con 0x7fdd180cef10 2024-04-24T23:20:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd180b67f0 con 0x7fdd180cef10 2024-04-24T23:20:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd0f7fe640 1 -- 172.21.15.26:0/836144168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00004400 con 0x7fdd180cef10 2024-04-24T23:20:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.179+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd180b6db0 con 0x7fdd180cef10 2024-04-24T23:20:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.180+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd180681e0 con 0x7fdd180cef10 2024-04-24T23:20:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.180+0000 7fdd0f7fe640 1 -- 172.21.15.26:0/836144168 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd0000c6d0 con 0x7fdd180cef10 2024-04-24T23:20:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.181+0000 7fdd0f7fe640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdcf806be30 0x7fdcf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.181+0000 7fdd0f7fe640 1 -- 172.21.15.26:0/836144168 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdd0008a950 con 0x7fdd180cef10 2024-04-24T23:20:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.181+0000 7fdd16ffd640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdcf806be30 0x7fdcf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.181+0000 7fdd16ffd640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdcf806be30 0x7fdcf806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdd10007fd0 tx=0x7fdd10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.185+0000 7fdd0f7fe640 1 -- 172.21.15.26:0/836144168 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd00017020 con 0x7fdd180cef10 2024-04-24T23:20:52.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.368+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd180b16f0 con 0x7fdd180cef10 2024-04-24T23:20:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.369+0000 7fdd0f7fe640 1 -- 172.21.15.26:0/836144168 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12824 (secure 0 0 0) 0x7fdd00059640 con 0x7fdd180cef10 2024-04-24T23:20:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:52.371 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:45.439439+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:20:52.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.370+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdcf806be30 msgr2=0x7fdcf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.370+0000 7fdd1d314640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdcf806be30 0x7fdcf806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdd10007fd0 tx=0x7fdd10009040 comp rx=0 tx=0).stop 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.370+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180cef10 msgr2=0x7fdd180b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.370+0000 7fdd1d314640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180cef10 0x7fdd180b72d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000e7e0 tx=0x7fdd0000eca0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.371+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 shutdown_connections 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.371+0000 7fdd1d314640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdcf806be30 0x7fdcf806e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.371+0000 7fdd1d314640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1815e2b0 0x7fdd1815a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.371+0000 7fdd1d314640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180cef10 0x7fdd180b72d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.371+0000 7fdd1d314640 1 --2- 172.21.15.26:0/836144168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd180b1b90 0x7fdd180b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.371+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 >> 172.21.15.26:0/836144168 conn(0x7fdd180d1f80 msgr2=0x7fdd180d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.371+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 shutdown_connections 2024-04-24T23:20:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.371+0000 7fdd1d314640 1 -- 172.21.15.26:0/836144168 wait complete. 2024-04-24T23:20:52.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:20:52.384 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:20:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.485+0000 7fbd97c4d640 1 Processor -- start 2024-04-24T23:20:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.485+0000 7fbd97c4d640 1 -- start start 2024-04-24T23:20:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.486+0000 7fbd97c4d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbd9015e370 0x7fbd900d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.486+0000 7fbd97c4d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbd9015e750 0x7fbd900d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.486+0000 7fbd97c4d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd900d4c10 0x7fbd900b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.486+0000 7fbd97c4d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbd900d1510 con 0x7fbd900d4c10 2024-04-24T23:20:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.486+0000 7fbd97c4d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbd900d1ed0 con 0x7fbd9015e750 2024-04-24T23:20:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.486+0000 7fbd97c4d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbd900d2120 con 0x7fbd9015e370 2024-04-24T23:20:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.486+0000 7fbd961ea640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd900d4c10 0x7fbd900b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.486+0000 7fbd961ea640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd900d4c10 0x7fbd900b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33302/0 (socket says 172.21.15.26:33302) 2024-04-24T23:20:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.486+0000 7fbd961ea640 1 -- 172.21.15.26:0/1167781244 learned_addr learned my addr 172.21.15.26:0/1167781244 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.487+0000 7fbd961ea640 1 -- 172.21.15.26:0/1167781244 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbd9015e370 msgr2=0x7fbd900d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.487+0000 7fbd961ea640 1 --2- 172.21.15.26:0/1167781244 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbd9015e370 0x7fbd900d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.487+0000 7fbd951e8640 1 --2- 172.21.15.26:0/1167781244 >> v2:172.21.15.29:3300/0 conn(0x7fbd9015e750 0x7fbd900d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.487+0000 7fbd961ea640 1 -- 172.21.15.26:0/1167781244 >> v2:172.21.15.29:3300/0 conn(0x7fbd9015e750 msgr2=0x7fbd900d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.487+0000 7fbd961ea640 1 --2- 172.21.15.26:0/1167781244 >> v2:172.21.15.29:3300/0 conn(0x7fbd9015e750 0x7fbd900d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.487+0000 7fbd961ea640 1 -- 172.21.15.26:0/1167781244 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd900b3740 con 0x7fbd900d4c10 2024-04-24T23:20:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.487+0000 7fbd951e8640 1 --2- 172.21.15.26:0/1167781244 >> v2:172.21.15.29:3300/0 conn(0x7fbd9015e750 0x7fbd900d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.487+0000 7fbd961ea640 1 --2- 172.21.15.26:0/1167781244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd900d4c10 0x7fbd900b4220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd9008c810 tx=0x7fbd8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b924f62d724293 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd949e7640 1 -- 172.21.15.26:0/1167781244 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80036070 con 0x7fbd900d4c10 2024-04-24T23:20:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd949e7640 1 -- 172.21.15.26:0/1167781244 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd80031600 con 0x7fbd900d4c10 2024-04-24T23:20:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd949e7640 1 -- 172.21.15.26:0/1167781244 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd800319d0 con 0x7fbd900d4c10 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1167781244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd900d4c10 msgr2=0x7fbd900b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1167781244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd900d4c10 0x7fbd900b4220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd9008c810 tx=0x7fbd8002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1167781244 shutdown_connections 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1167781244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd900d4c10 0x7fbd900b4220 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1167781244 >> v2:172.21.15.29:3300/0 conn(0x7fbd9015e750 0x7fbd900d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1167781244 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbd9015e370 0x7fbd900d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1167781244 >> 172.21.15.26:0/1167781244 conn(0x7fbd900d40f0 msgr2=0x7fbd900d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1167781244 shutdown_connections 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.488+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1167781244 wait complete. 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.489+0000 7fbd97c4d640 1 Processor -- start 2024-04-24T23:20:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.489+0000 7fbd97c4d640 1 -- start start 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd97c4d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbd900d4c10 0x7fbd901f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd97c4d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015e370 0x7fbd901614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd97c4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd900b2450 0x7fbd901f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd97c4d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbd900b4760 con 0x7fbd9015e370 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd97c4d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbd900b3d00 con 0x7fbd900b2450 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd97c4d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbd900b48e0 con 0x7fbd900d4c10 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd951e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015e370 0x7fbd901614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd951e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015e370 0x7fbd901614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33310/0 (socket says 172.21.15.26:33310) 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd951e8640 1 -- 172.21.15.26:0/1700499707 learned_addr learned my addr 172.21.15.26:0/1700499707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd959e9640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbd900d4c10 0x7fbd901f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.490+0000 7fbd961ea640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd900b2450 0x7fbd901f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd951e8640 1 -- 172.21.15.26:0/1700499707 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbd900d4c10 msgr2=0x7fbd901f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd951e8640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbd900d4c10 0x7fbd901f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd951e8640 1 -- 172.21.15.26:0/1700499707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd900b2450 msgr2=0x7fbd901f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd951e8640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd900b2450 0x7fbd901f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd951e8640 1 -- 172.21.15.26:0/1700499707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd901f40f0 con 0x7fbd9015e370 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd951e8640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015e370 0x7fbd901614a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c820 tx=0x7fbd8400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd8e7fc640 1 -- 172.21.15.26:0/1700499707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84015070 con 0x7fbd9015e370 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd8e7fc640 1 -- 172.21.15.26:0/1700499707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd84007820 con 0x7fbd9015e370 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd901f43b0 con 0x7fbd9015e370 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd8e7fc640 1 -- 172.21.15.26:0/1700499707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84007bd0 con 0x7fbd9015e370 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.491+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd901f4970 con 0x7fbd9015e370 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.492+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd90075610 con 0x7fbd9015e370 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.492+0000 7fbd8e7fc640 1 -- 172.21.15.26:0/1700499707 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd84018070 con 0x7fbd9015e370 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.493+0000 7fbd8e7fc640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbd7c06be30 0x7fbd7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.493+0000 7fbd8e7fc640 1 -- 172.21.15.26:0/1700499707 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbd8408a880 con 0x7fbd9015e370 2024-04-24T23:20:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.493+0000 7fbd959e9640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbd7c06be30 0x7fbd7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.493+0000 7fbd959e9640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbd7c06be30 0x7fbd7c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009990 tx=0x7fbd88009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:52.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.497+0000 7fbd8e7fc640 1 -- 172.21.15.26:0/1700499707 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbd84058f30 con 0x7fbd9015e370 2024-04-24T23:20:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.681+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbd900cf970 con 0x7fbd9015e370 2024-04-24T23:20:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.681+0000 7fbd8e7fc640 1 -- 172.21.15.26:0/1700499707 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fbd8405d230 con 0x7fbd9015e370 2024-04-24T23:20:52.683 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.683+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbd7c06be30 msgr2=0x7fbd7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.683+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbd7c06be30 0x7fbd7c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009990 tx=0x7fbd88009110 comp rx=0 tx=0).stop 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.683+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015e370 msgr2=0x7fbd901614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.683+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015e370 0x7fbd901614a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c820 tx=0x7fbd8400cc20 comp rx=0 tx=0).stop 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.684+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 shutdown_connections 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.684+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbd7c06be30 0x7fbd7c06e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.684+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbd900b2450 0x7fbd901f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.684+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015e370 0x7fbd901614a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.684+0000 7fbd97c4d640 1 --2- 172.21.15.26:0/1700499707 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbd900d4c10 0x7fbd901f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.684+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 >> 172.21.15.26:0/1700499707 conn(0x7fbd900d40f0 msgr2=0x7fbd9006cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:52.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.684+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 shutdown_connections 2024-04-24T23:20:52.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.684+0000 7fbd97c4d640 1 -- 172.21.15.26:0/1700499707 wait complete. 2024-04-24T23:20:52.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:20:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.794+0000 7f2319570640 1 Processor -- start 2024-04-24T23:20:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.794+0000 7f2319570640 1 -- start start 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.795+0000 7f2319570640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 0x7f23140b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.795+0000 7f2319570640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f231406ca40 con 0x7f23140cef10 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.795+0000 7f2319570640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23140b22e0 con 0x7f231415e2b0 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.795+0000 7f2319570640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f23140b2ef0 con 0x7f231406c600 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.795+0000 7f230bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f23140cef10 0x7f23140d04f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37864/0 (socket says 172.21.15.26:37864) 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.795+0000 7f230bfff640 1 -- 172.21.15.26:0/3420876126 learned_addr learned my addr 172.21.15.26:0/3420876126 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.796+0000 7f23137fe640 1 --2- 172.21.15.26:0/3420876126 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 0x7f23140b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.796+0000 7f23127fc640 1 -- 172.21.15.26:0/3420876126 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 350761820 0 0) 0x7f22fc001110 con 0x7f23140cef10 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.796+0000 7f23127fc640 1 -- 172.21.15.26:0/3420876126 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2300003520 con 0x7f23140cef10 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.796+0000 7f23127fc640 1 -- 172.21.15.26:0/3420876126 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207939052 0 0) 0x7f23140b22e0 con 0x7f231415e2b0 2024-04-24T23:20:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.796+0000 7f23137fe640 1 -- 172.21.15.26:0/3420876126 >> v1:172.21.15.29:6789/0 conn(0x7f231415e2b0 legacy=0x7f23140ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.796+0000 7f23137fe640 1 -- 172.21.15.26:0/3420876126 >> v1:172.21.15.26:6789/0 conn(0x7f23140cef10 legacy=0x7f23140d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.796+0000 7f23137fe640 1 -- 172.21.15.26:0/3420876126 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23140b25c0 con 0x7f231406c600 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f23137fe640 1 --2- 172.21.15.26:0/3420876126 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 0x7f23140b1b90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f230c009dc0 tx=0x7f230c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6923e633c32fa02 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f23127fc640 1 -- 172.21.15.26:0/3420876126 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c007740 con 0x7f231406c600 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f23127fc640 1 -- 172.21.15.26:0/3420876126 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230c0078e0 con 0x7f231406c600 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f23127fc640 1 -- 172.21.15.26:0/3420876126 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c007c90 con 0x7f231406c600 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f2319570640 1 -- 172.21.15.26:0/3420876126 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 msgr2=0x7f23140b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f2319570640 1 --2- 172.21.15.26:0/3420876126 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 0x7f23140b1b90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f230c009dc0 tx=0x7f230c02a6d0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f2319570640 1 -- 172.21.15.26:0/3420876126 shutdown_connections 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f2319570640 1 --2- 172.21.15.26:0/3420876126 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 0x7f23140b1b90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f2319570640 1 -- 172.21.15.26:0/3420876126 >> 172.21.15.26:0/3420876126 conn(0x7f23140d1f80 msgr2=0x7f23140d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f2319570640 1 -- 172.21.15.26:0/3420876126 shutdown_connections 2024-04-24T23:20:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.797+0000 7f2319570640 1 -- 172.21.15.26:0/3420876126 wait complete. 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f2319570640 1 Processor -- start 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f2319570640 1 -- start start 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f2319570640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 0x7f231415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f2319570640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140cef10 0x7f23141e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f2319570640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f231415e2b0 0x7f231409f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f2319570640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f231415e690 con 0x7f23140cef10 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f2319570640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f23140b2b80 con 0x7f231415e2b0 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f2319570640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f231415e810 con 0x7f231406c600 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f230bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140cef10 0x7f23141e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f230bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140cef10 0x7f23141e6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33322/0 (socket says 172.21.15.26:33322) 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.799+0000 7f230bfff640 1 -- 172.21.15.26:0/2905772576 learned_addr learned my addr 172.21.15.26:0/2905772576 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f2312ffd640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 0x7f231415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f23137fe640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f231415e2b0 0x7f231409f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f230bfff640 1 -- 172.21.15.26:0/2905772576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 msgr2=0x7f231415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f230bfff640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 0x7f231415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f230bfff640 1 -- 172.21.15.26:0/2905772576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f231415e2b0 msgr2=0x7f231409f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f230bfff640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f231415e2b0 0x7f231409f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f230bfff640 1 -- 172.21.15.26:0/2905772576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23141e6f80 con 0x7f23140cef10 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f230bfff640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140cef10 0x7f23141e6a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00c760 tx=0x7f22fc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f230b7fe640 1 -- 172.21.15.26:0/2905772576 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc017070 con 0x7f23140cef10 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f230b7fe640 1 -- 172.21.15.26:0/2905772576 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22fc005ad0 con 0x7f23140cef10 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f230b7fe640 1 -- 172.21.15.26:0/2905772576 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc007e20 con 0x7f23140cef10 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23141e7200 con 0x7f23140cef10 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.800+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23141e77c0 con 0x7f23140cef10 2024-04-24T23:20:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.801+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23140681e0 con 0x7f23140cef10 2024-04-24T23:20:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.802+0000 7f230b7fe640 1 -- 172.21.15.26:0/2905772576 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22fc01a070 con 0x7f23140cef10 2024-04-24T23:20:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.802+0000 7f230b7fe640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f22f406be30 0x7f22f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.802+0000 7f230b7fe640 1 -- 172.21.15.26:0/2905772576 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f22fc089dd0 con 0x7f23140cef10 2024-04-24T23:20:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.802+0000 7f2312ffd640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f22f406be30 0x7f22f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.802+0000 7f2312ffd640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f22f406be30 0x7f22f406e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2304000f60 tx=0x7f2304009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.806+0000 7f230b7fe640 1 -- 172.21.15.26:0/2905772576 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23141e6f80 con 0x7f23140cef10 2024-04-24T23:20:52.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.989+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2314074e00 con 0x7f23140cef10 2024-04-24T23:20:52.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.990+0000 7f230b7fe640 1 -- 172.21.15.26:0/2905772576 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f22fc08b020 con 0x7f23140cef10 2024-04-24T23:20:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:52.993 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:20:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:20:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-24T23:20:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-24T23:20:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:20:52.994 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:20:52.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.991+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f22f406be30 msgr2=0x7f22f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.991+0000 7f2319570640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f22f406be30 0x7f22f406e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2304000f60 tx=0x7f2304009040 comp rx=0 tx=0).stop 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.991+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140cef10 msgr2=0x7f23141e6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.991+0000 7f2319570640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140cef10 0x7f23141e6a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00c760 tx=0x7f22fc00cc20 comp rx=0 tx=0).stop 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.991+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 shutdown_connections 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.991+0000 7f2319570640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f22f406be30 0x7f22f406e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.991+0000 7f2319570640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f231415e2b0 0x7f231409f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.992+0000 7f2319570640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23140cef10 0x7f23141e6a40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.992+0000 7f2319570640 1 --2- 172.21.15.26:0/2905772576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f231406c600 0x7f231415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.992+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 >> 172.21.15.26:0/2905772576 conn(0x7f23140d1f80 msgr2=0x7f23140ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.992+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 shutdown_connections 2024-04-24T23:20:52.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:52.992+0000 7f2319570640 1 -- 172.21.15.26:0/2905772576 wait complete. 2024-04-24T23:20:53.004 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:20:53.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:20:53.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.106+0000 7f4fd71ce640 1 Processor -- start 2024-04-24T23:20:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.107+0000 7f4fd71ce640 1 -- start start 2024-04-24T23:20:53.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.107+0000 7f4fd71ce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4fd015e3f0 0x7f4fd015e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.107+0000 7f4fd71ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4fd00d2c50 0x7f4fd00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.107+0000 7f4fd71ce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd006ce20 con 0x7f4fd00b3160 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.107+0000 7f4fd71ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd00d1d00 con 0x7f4fd00d2c50 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.107+0000 7f4fd71ce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd006d0a0 con 0x7f4fd015e3f0 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.107+0000 7f4fcffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4fd00d2c50 0x7f4fd00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.107+0000 7f4fcffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4fd00d2c50 0x7f4fd00cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33336/0 (socket says 172.21.15.26:33336) 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.107+0000 7f4fcffff640 1 -- 172.21.15.26:0/1508718709 learned_addr learned my addr 172.21.15.26:0/1508718709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.108+0000 7f4fd4f6a640 1 --2- 172.21.15.26:0/1508718709 >> v2:172.21.15.29:3300/0 conn(0x7f4fd015e3f0 0x7f4fd015e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.108+0000 7f4fcffff640 1 -- 172.21.15.26:0/1508718709 >> v2:172.21.15.29:3300/0 conn(0x7f4fd015e3f0 msgr2=0x7f4fd015e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.108+0000 7f4fcffff640 1 --2- 172.21.15.26:0/1508718709 >> v2:172.21.15.29:3300/0 conn(0x7f4fd015e3f0 0x7f4fd015e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.108+0000 7f4fcffff640 1 -- 172.21.15.26:0/1508718709 >> v1:172.21.15.29:6789/0 conn(0x7f4fd00b3160 legacy=0x7f4fd009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.108+0000 7f4fcffff640 1 -- 172.21.15.26:0/1508718709 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd006d380 con 0x7f4fd00d2c50 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.108+0000 7f4fd4f6a640 1 --2- 172.21.15.26:0/1508718709 >> v2:172.21.15.29:3300/0 conn(0x7f4fd015e3f0 0x7f4fd015e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:53.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.108+0000 7f4fcffff640 1 --2- 172.21.15.26:0/1508718709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00d2c50 0x7f4fd00cc570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0068af0 tx=0x7f4fc002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d68176e4416be72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fcf7fe640 1 -- 172.21.15.26:0/1508718709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0036070 con 0x7f4fd00d2c50 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fcf7fe640 1 -- 172.21.15.26:0/1508718709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc0031600 con 0x7f4fd00d2c50 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fcf7fe640 1 -- 172.21.15.26:0/1508718709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00319d0 con 0x7f4fd00d2c50 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fd71ce640 1 -- 172.21.15.26:0/1508718709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00d2c50 msgr2=0x7f4fd00cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fd71ce640 1 --2- 172.21.15.26:0/1508718709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00d2c50 0x7f4fd00cc570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0068af0 tx=0x7f4fc002a6f0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fd71ce640 1 -- 172.21.15.26:0/1508718709 shutdown_connections 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fd71ce640 1 --2- 172.21.15.26:0/1508718709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd00d2c50 0x7f4fd00cc570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fd71ce640 1 --2- 172.21.15.26:0/1508718709 >> v2:172.21.15.29:3300/0 conn(0x7f4fd015e3f0 0x7f4fd015e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fd71ce640 1 -- 172.21.15.26:0/1508718709 >> 172.21.15.26:0/1508718709 conn(0x7f4fd00d1f20 msgr2=0x7f4fd00d2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fd71ce640 1 -- 172.21.15.26:0/1508718709 shutdown_connections 2024-04-24T23:20:53.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.109+0000 7f4fd71ce640 1 -- 172.21.15.26:0/1508718709 wait complete. 2024-04-24T23:20:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.110+0000 7f4fd71ce640 1 Processor -- start 2024-04-24T23:20:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.110+0000 7f4fd71ce640 1 -- start start 2024-04-24T23:20:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd71ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fd00b3160 0x7f4fd00c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd71ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4fd00d2c50 0x7f4fd00c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd71ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd015e3f0 0x7f4fd00d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd71ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd006d660 con 0x7f4fd015e3f0 2024-04-24T23:20:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd71ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd00d06d0 con 0x7f4fd00b3160 2024-04-24T23:20:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd71ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd00ccab0 con 0x7f4fd00d2c50 2024-04-24T23:20:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd576b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd015e3f0 0x7f4fd00d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd576b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd015e3f0 0x7f4fd00d0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33344/0 (socket says 172.21.15.26:33344) 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd576b640 1 -- 172.21.15.26:0/3039058041 learned_addr learned my addr 172.21.15.26:0/3039058041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fcffff640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4fd00d2c50 0x7f4fd00c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.111+0000 7f4fd4f6a640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fd00b3160 0x7f4fd00c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fd576b640 1 -- 172.21.15.26:0/3039058041 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4fd00d2c50 msgr2=0x7f4fd00c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fd576b640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4fd00d2c50 0x7f4fd00c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fd576b640 1 -- 172.21.15.26:0/3039058041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fd00b3160 msgr2=0x7f4fd00c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fd576b640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fd00b3160 0x7f4fd00c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fd576b640 1 -- 172.21.15.26:0/3039058041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd01f3d40 con 0x7f4fd015e3f0 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fd576b640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd015e3f0 0x7f4fd00d0af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c410 tx=0x7f4fb800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fcd7fa640 1 -- 172.21.15.26:0/3039058041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8011790 con 0x7f4fd015e3f0 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fcd7fa640 1 -- 172.21.15.26:0/3039058041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb8017070 con 0x7f4fd015e3f0 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd01f8200 con 0x7f4fd015e3f0 2024-04-24T23:20:53.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fcd7fa640 1 -- 172.21.15.26:0/3039058041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8011bb0 con 0x7f4fd015e3f0 2024-04-24T23:20:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.112+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd01f42c0 con 0x7f4fd015e3f0 2024-04-24T23:20:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.113+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd00694c0 con 0x7f4fd015e3f0 2024-04-24T23:20:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.113+0000 7f4fcd7fa640 1 -- 172.21.15.26:0/3039058041 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4fb801c460 con 0x7f4fd015e3f0 2024-04-24T23:20:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.114+0000 7f4fcd7fa640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4fb006be30 0x7f4fb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.114+0000 7f4fd4f6a640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4fb006be30 0x7f4fb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.114+0000 7f4fcd7fa640 1 -- 172.21.15.26:0/3039058041 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4fb808a770 con 0x7f4fd015e3f0 2024-04-24T23:20:53.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.114+0000 7f4fd4f6a640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4fb006be30 0x7f4fb006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fd015d3c0 tx=0x7f4fc8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.118+0000 7f4fcd7fa640 1 -- 172.21.15.26:0/3039058041 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4fb8058eb0 con 0x7f4fd015e3f0 2024-04-24T23:20:53.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.301+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4fd00b7e50 con 0x7f4fd015e3f0 2024-04-24T23:20:53.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.452+0000 7f4fcd7fa640 1 -- 172.21.15.26:0/3039058041 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f4fb805d1b0 con 0x7f4fd015e3f0 2024-04-24T23:20:53.454 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-24T23:20:53.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4fb006be30 msgr2=0x7f4fb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4fb006be30 0x7f4fb006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fd015d3c0 tx=0x7f4fc8008040 comp rx=0 tx=0).stop 2024-04-24T23:20:53.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd015e3f0 msgr2=0x7f4fd00d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd015e3f0 0x7f4fd00d0af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c410 tx=0x7f4fb800c8d0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 shutdown_connections 2024-04-24T23:20:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4fb006be30 0x7f4fb006e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd015e3f0 0x7f4fd00d0af0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4fd00d2c50 0x7f4fd00c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 --2- 172.21.15.26:0/3039058041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4fd00b3160 0x7f4fd00c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.454+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 >> 172.21.15.26:0/3039058041 conn(0x7f4fd00d1f20 msgr2=0x7f4fd006f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.455+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 shutdown_connections 2024-04-24T23:20:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.455+0000 7f4fd71ce640 1 -- 172.21.15.26:0/3039058041 wait complete. 2024-04-24T23:20:53.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:20:53.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.577+0000 7efdd4fa9640 1 Processor -- start 2024-04-24T23:20:53.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.577+0000 7efdd4fa9640 1 -- start start 2024-04-24T23:20:53.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.577+0000 7efdd4fa9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efdd015e370 0x7efdd00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.577+0000 7efdd4fa9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd00d2120 con 0x7efdd015e750 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.577+0000 7efdd4fa9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd00b3740 con 0x7efdd00d4c10 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.577+0000 7efdd4fa9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efdd00d1510 con 0x7efdd015e370 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdc7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efdd015e750 0x7efdd00b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37874/0 (socket says 172.21.15.26:37874) 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdc7fff640 1 -- 172.21.15.26:0/2598124790 learned_addr learned my addr 172.21.15.26:0/2598124790 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdce59c640 1 --2- 172.21.15.26:0/2598124790 >> v2:172.21.15.29:3300/0 conn(0x7efdd015e370 0x7efdd00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1953270369 0 0) 0x7efdd00d2120 con 0x7efdd015e750 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdbc003520 con 0x7efdd015e750 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2260171027 0 0) 0x7efdbc003520 con 0x7efdd015e750 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 >> v2:172.21.15.29:3300/0 conn(0x7efdd015e370 msgr2=0x7efdd00d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 --2- 172.21.15.26:0/2598124790 >> v2:172.21.15.29:3300/0 conn(0x7efdd015e370 0x7efdd00d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 >> v1:172.21.15.29:6789/0 conn(0x7efdd00d4c10 legacy=0x7efdd00b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd00b3a20 con 0x7efdd015e750 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7efdb8001670 con 0x7efdd015e750 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 >> v1:172.21.15.26:6789/0 conn(0x7efdd015e750 legacy=0x7efdd00b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efdbc003a00 0x7efdbc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdbc004800 0x7efdbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdbc0051c0 0x7efdbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc00c090 con 0x7efdbc0051c0 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc00c240 con 0x7efdbc004800 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc00c440 con 0x7efdbc003a00 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdced9d640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdbc0051c0 0x7efdbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdce59c640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efdbc003a00 0x7efdbc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.578+0000 7efdc7fff640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdbc004800 0x7efdbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdced9d640 1 -- 172.21.15.26:0/2598124790 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efdbc003a00 msgr2=0x7efdbc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdced9d640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efdbc003a00 0x7efdbc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdced9d640 1 -- 172.21.15.26:0/2598124790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdbc004800 msgr2=0x7efdbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdced9d640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdbc004800 0x7efdbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdced9d640 1 -- 172.21.15.26:0/2598124790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc003520 con 0x7efdbc0051c0 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdced9d640 1 -- 172.21.15.26:0/2598124790 reap_dead start 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdced9d640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdbc0051c0 0x7efdbc00b9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efdc8001e50 tx=0x7efdbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30f85e034015942 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8001a70 con 0x7efdbc0051c0 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc8003970 con 0x7efdbc0051c0 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.579+0000 7efdcdd9b640 1 -- 172.21.15.26:0/2598124790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8003d60 con 0x7efdbc0051c0 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.580+0000 7efdd4fa9640 1 -- 172.21.15.26:0/2598124790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdbc0051c0 msgr2=0x7efdbc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.580+0000 7efdd4fa9640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdbc0051c0 0x7efdbc00b9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efdc8001e50 tx=0x7efdbc005830 comp rx=0 tx=0).stop 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.580+0000 7efdd4fa9640 1 -- 172.21.15.26:0/2598124790 shutdown_connections 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.580+0000 7efdd4fa9640 1 --2- 172.21.15.26:0/2598124790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdbc0051c0 0x7efdbc00b9c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.580+0000 7efdd4fa9640 1 -- 172.21.15.26:0/2598124790 >> 172.21.15.26:0/2598124790 conn(0x7efdd00d40f0 msgr2=0x7efdd00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.580+0000 7efdd4fa9640 1 -- 172.21.15.26:0/2598124790 shutdown_connections 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.580+0000 7efdd4fa9640 1 -- 172.21.15.26:0/2598124790 wait complete. 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.581+0000 7efdd4fa9640 1 Processor -- start 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.581+0000 7efdd4fa9640 1 -- start start 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdd4fa9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efdbc0051c0 0x7efdd00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdd4fa9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd01e7b70 0x7efdd01e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdd4fa9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd01e5b10 0x7efdd01e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdd4fa9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efdd00d2520 con 0x7efdd01e5b10 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdd4fa9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efdd00d1ed0 con 0x7efdd01e7b70 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdd4fa9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efdd00d26a0 con 0x7efdbc0051c0 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdced9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd01e5b10 0x7efdd01e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdced9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd01e5b10 0x7efdd01e5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33374/0 (socket says 172.21.15.26:33374) 2024-04-24T23:20:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdced9d640 1 -- 172.21.15.26:0/1635557450 learned_addr learned my addr 172.21.15.26:0/1635557450 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdce59c640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efdbc0051c0 0x7efdd00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.582+0000 7efdc7fff640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd01e7b70 0x7efdd01e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdced9d640 1 -- 172.21.15.26:0/1635557450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efdbc0051c0 msgr2=0x7efdd00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdced9d640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efdbc0051c0 0x7efdd00cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdced9d640 1 -- 172.21.15.26:0/1635557450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd01e7b70 msgr2=0x7efdd01e7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdced9d640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd01e7b70 0x7efdd01e7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdced9d640 1 -- 172.21.15.26:0/1635557450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd01543c0 con 0x7efdd01e5b10 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdced9d640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd01e5b10 0x7efdd01e5fa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdc8001f70 tx=0x7efdc80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdc6ffd640 1 -- 172.21.15.26:0/1635557450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8001a70 con 0x7efdd01e5b10 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdc6ffd640 1 -- 172.21.15.26:0/1635557450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc802ac70 con 0x7efdd01e5b10 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd00cbce0 con 0x7efdd01e5b10 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdc6ffd640 1 -- 172.21.15.26:0/1635557450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8006d70 con 0x7efdd01e5b10 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.583+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd00ca700 con 0x7efdd01e5b10 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.584+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd00cc200 con 0x7efdd01e5b10 2024-04-24T23:20:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.584+0000 7efdc6ffd640 1 -- 172.21.15.26:0/1635557450 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efdc8007030 con 0x7efdd01e5b10 2024-04-24T23:20:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.585+0000 7efdc6ffd640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efdb006be30 0x7efdb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.585+0000 7efdce59c640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efdb006be30 0x7efdb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.585+0000 7efdc6ffd640 1 -- 172.21.15.26:0/1635557450 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7efdc80aa890 con 0x7efdd01e5b10 2024-04-24T23:20:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.585+0000 7efdce59c640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efdb006be30 0x7efdb006e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efdd00d0000 tx=0x7efdc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.590+0000 7efdc6ffd640 1 -- 172.21.15.26:0/1635557450 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efdd00cc200 con 0x7efdd01e5b10 2024-04-24T23:20:53.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:53.776+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efdd00cfd50 con 0x7efdd01e5b10 2024-04-24T23:20:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.458+0000 7efdc6ffd640 1 -- 172.21.15.26:0/1635557450 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7efdc8043060 con 0x7efdd01e5b10 2024-04-24T23:20:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-24T23:20:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.462+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efdb006be30 msgr2=0x7efdb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.462+0000 7efdd4fa9640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efdb006be30 0x7efdb006e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efdd00d0000 tx=0x7efdc0008040 comp rx=0 tx=0).stop 2024-04-24T23:20:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.462+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd01e5b10 msgr2=0x7efdd01e5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.462+0000 7efdd4fa9640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd01e5b10 0x7efdd01e5fa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdc8001f70 tx=0x7efdc80014a0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.463+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 shutdown_connections 2024-04-24T23:20:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.463+0000 7efdd4fa9640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efdb006be30 0x7efdb006e2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.463+0000 7efdd4fa9640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd01e5b10 0x7efdd01e5fa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.463+0000 7efdd4fa9640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efdd01e7b70 0x7efdd01e7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.463+0000 7efdd4fa9640 1 --2- 172.21.15.26:0/1635557450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efdbc0051c0 0x7efdd00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.463+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 >> 172.21.15.26:0/1635557450 conn(0x7efdd00d40f0 msgr2=0x7efdd00cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.463+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 shutdown_connections 2024-04-24T23:20:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.465+0000 7efdd4fa9640 1 -- 172.21.15.26:0/1635557450 wait complete. 2024-04-24T23:20:54.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:20:54.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.598+0000 7f35ce95a640 1 Processor -- start 2024-04-24T23:20:54.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.598+0000 7f35ce95a640 1 -- start start 2024-04-24T23:20:54.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.599+0000 7f35ce95a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35c80d0ce0 0x7f35c806e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:54.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.599+0000 7f35ce95a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8071720 0x7f35c8071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:54.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.599+0000 7f35ce95a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35c806b1c0 con 0x7f35c8158ae0 2024-04-24T23:20:54.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.599+0000 7f35ce95a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f35c80720b0 con 0x7f35c80d0ce0 2024-04-24T23:20:54.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.599+0000 7f35ce95a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f35c806ec90 con 0x7f35c8071720 2024-04-24T23:20:54.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.599+0000 7f35ccef7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8071720 0x7f35c8071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:54.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.599+0000 7f35c77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35c80d0ce0 0x7f35c806e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:54.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.600+0000 7f35c77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f35c80d0ce0 0x7f35c806e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:33108/0 (socket says 172.21.15.26:33108) 2024-04-24T23:20:54.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.600+0000 7f35c77fe640 1 -- 172.21.15.26:0/4020936 learned_addr learned my addr 172.21.15.26:0/4020936 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:54.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.600+0000 7f35ccef7640 1 -- 172.21.15.26:0/4020936 >> v2:172.21.15.29:3300/0 conn(0x7f35c80d0ce0 msgr2=0x7f35c806e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:54.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.600+0000 7f35ccef7640 1 --2- 172.21.15.26:0/4020936 >> v2:172.21.15.29:3300/0 conn(0x7f35c80d0ce0 0x7f35c806e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.600+0000 7f35ccef7640 1 -- 172.21.15.26:0/4020936 >> v1:172.21.15.29:6789/0 conn(0x7f35c8158ae0 legacy=0x7f35c815e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:54.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.600+0000 7f35ccef7640 1 -- 172.21.15.26:0/4020936 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c806ef70 con 0x7f35c8071720 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35ccef7640 1 --2- 172.21.15.26:0/4020936 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8071720 0x7f35c8071b70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009d90 tx=0x7f35c002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a56bc145f5855e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35c6ffd640 1 -- 172.21.15.26:0/4020936 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0031600 con 0x7f35c8071720 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35c6ffd640 1 -- 172.21.15.26:0/4020936 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c00317a0 con 0x7f35c8071720 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35c6ffd640 1 -- 172.21.15.26:0/4020936 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0031b70 con 0x7f35c8071720 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35ce95a640 1 -- 172.21.15.26:0/4020936 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8071720 msgr2=0x7f35c8071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35ce95a640 1 --2- 172.21.15.26:0/4020936 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8071720 0x7f35c8071b70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009d90 tx=0x7f35c002a5d0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35ce95a640 1 -- 172.21.15.26:0/4020936 shutdown_connections 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35ce95a640 1 --2- 172.21.15.26:0/4020936 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8071720 0x7f35c8071b70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35ce95a640 1 --2- 172.21.15.26:0/4020936 >> v2:172.21.15.29:3300/0 conn(0x7f35c80d0ce0 0x7f35c806e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35ce95a640 1 -- 172.21.15.26:0/4020936 >> 172.21.15.26:0/4020936 conn(0x7f35c80b2970 msgr2=0x7f35c80d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35ce95a640 1 -- 172.21.15.26:0/4020936 shutdown_connections 2024-04-24T23:20:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.601+0000 7f35ce95a640 1 -- 172.21.15.26:0/4020936 wait complete. 2024-04-24T23:20:54.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.603+0000 7f35ce95a640 1 Processor -- start 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.603+0000 7f35ce95a640 1 -- start start 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.603+0000 7f35ce95a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c8071720 0x7f35c8146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.603+0000 7f35ce95a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35c80d0ce0 0x7f35c80b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.603+0000 7f35ce95a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8158ae0 0x7f35c806b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.603+0000 7f35ce95a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f35c806f530 con 0x7f35c8071720 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.603+0000 7f35ce95a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f35c815f060 con 0x7f35c80d0ce0 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.603+0000 7f35ce95a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f35c80aff60 con 0x7f35c8158ae0 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c8071720 0x7f35c8146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c8071720 0x7f35c8146fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54466/0 (socket says 172.21.15.26:54466) 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c7fff640 1 -- 172.21.15.26:0/1806801934 learned_addr learned my addr 172.21.15.26:0/1806801934 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35ccef7640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8158ae0 0x7f35c806b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:54.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c77fe640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35c80d0ce0 0x7f35c80b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c7fff640 1 -- 172.21.15.26:0/1806801934 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8158ae0 msgr2=0x7f35c806b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c7fff640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8158ae0 0x7f35c806b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c7fff640 1 -- 172.21.15.26:0/1806801934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35c80d0ce0 msgr2=0x7f35c80b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c7fff640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35c80d0ce0 0x7f35c80b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c7fff640 1 -- 172.21.15.26:0/1806801934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c80b4690 con 0x7f35c8071720 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c7fff640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c8071720 0x7f35c8146fd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35b000c3c0 tx=0x7f35b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c4ff9640 1 -- 172.21.15.26:0/1806801934 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b00052d0 con 0x7f35c8071720 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c4ff9640 1 -- 172.21.15.26:0/1806801934 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b0005490 con 0x7f35c8071720 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c80b4970 con 0x7f35c8071720 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35c4ff9640 1 -- 172.21.15.26:0/1806801934 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0007260 con 0x7f35c8071720 2024-04-24T23:20:54.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.604+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c80b4f30 con 0x7f35c8071720 2024-04-24T23:20:54.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.606+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c8075610 con 0x7f35c8071720 2024-04-24T23:20:54.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.606+0000 7f35c4ff9640 1 -- 172.21.15.26:0/1806801934 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35b0007400 con 0x7f35c8071720 2024-04-24T23:20:54.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.607+0000 7f35c4ff9640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35a806be30 0x7f35a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:54.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.607+0000 7f35c77fe640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35a806be30 0x7f35a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:54.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.607+0000 7f35c4ff9640 1 -- 172.21.15.26:0/1806801934 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f35b008a1a0 con 0x7f35c8071720 2024-04-24T23:20:54.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.607+0000 7f35c77fe640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35a806be30 0x7f35a806e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f35c80b0840 tx=0x7f35b8009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:54.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.611+0000 7f35c4ff9640 1 -- 172.21.15.26:0/1806801934 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f35b00585f0 con 0x7f35c8071720 2024-04-24T23:20:54.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:54.831+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f35c8071cf0 con 0x7f35c8071720 2024-04-24T23:20:55.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.471+0000 7f35c4ff9640 1 -- 172.21.15.26:0/1806801934 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f35b005c8f0 con 0x7f35c8071720 2024-04-24T23:20:55.473 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T23:20:55.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.478+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35a806be30 msgr2=0x7f35a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.478+0000 7f35ce95a640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35a806be30 0x7f35a806e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f35c80b0840 tx=0x7f35b8009240 comp rx=0 tx=0).stop 2024-04-24T23:20:55.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.478+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c8071720 msgr2=0x7f35c8146fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.478+0000 7f35ce95a640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c8071720 0x7f35c8146fd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35b000c3c0 tx=0x7f35b000c880 comp rx=0 tx=0).stop 2024-04-24T23:20:55.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.481+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 shutdown_connections 2024-04-24T23:20:55.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.481+0000 7f35ce95a640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35a806be30 0x7f35a806e2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.481+0000 7f35ce95a640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f35c8158ae0 0x7f35c806b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.481+0000 7f35ce95a640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35c80d0ce0 0x7f35c80b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.481+0000 7f35ce95a640 1 --2- 172.21.15.26:0/1806801934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35c8071720 0x7f35c8146fd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.481+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 >> 172.21.15.26:0/1806801934 conn(0x7f35c80b2970 msgr2=0x7f35c80b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:55.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.482+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 shutdown_connections 2024-04-24T23:20:55.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.483+0000 7f35ce95a640 1 -- 172.21.15.26:0/1806801934 wait complete. 2024-04-24T23:20:55.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:20:55.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.607+0000 7f0d5e5b7640 1 Processor -- start 2024-04-24T23:20:55.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.607+0000 7f0d5e5b7640 1 -- start start 2024-04-24T23:20:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d5e5b7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0d580b4530 0x7f0d580b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d5e5b7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d580b5700 con 0x7f0d580b0030 2024-04-24T23:20:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d5e5b7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d580b59e0 con 0x7f0d5815e380 2024-04-24T23:20:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d5e5b7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0d580b49a0 con 0x7f0d580b4530 2024-04-24T23:20:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d577fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0d580b4530 0x7f0d580b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d5cb54640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0d580b0030 0x7f0d580b0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46250/0 (socket says 172.21.15.26:46250) 2024-04-24T23:20:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d577fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0d580b4530 0x7f0d580b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54468/0 (socket says 172.21.15.26:54468) 2024-04-24T23:20:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d5cb54640 1 -- 172.21.15.26:0/1865694479 learned_addr learned my addr 172.21.15.26:0/1865694479 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d577fe640 1 -- 172.21.15.26:0/1865694479 >> v1:172.21.15.29:6789/0 conn(0x7f0d5815e380 legacy=0x7f0d58072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:55.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d577fe640 1 -- 172.21.15.26:0/1865694479 >> v1:172.21.15.26:6789/0 conn(0x7f0d580b0030 legacy=0x7f0d580b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d577fe640 1 -- 172.21.15.26:0/1865694479 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d58072240 con 0x7f0d580b4530 2024-04-24T23:20:55.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.608+0000 7f0d56ffd640 1 -- 172.21.15.26:0/1865694479 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4191981718 0 0) 0x7f0d580b5700 con 0x7f0d580b0030 2024-04-24T23:20:55.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.609+0000 7f0d577fe640 1 --2- 172.21.15.26:0/1865694479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b4530 0x7f0d580b4fd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d480097a0 tx=0x7f0d4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6abb2b40a2c06b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:55.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.609+0000 7f0d56ffd640 1 -- 172.21.15.26:0/1865694479 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48037070 con 0x7f0d580b4530 2024-04-24T23:20:55.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.609+0000 7f0d56ffd640 1 -- 172.21.15.26:0/1865694479 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d48032440 con 0x7f0d580b4530 2024-04-24T23:20:55.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.609+0000 7f0d56ffd640 1 -- 172.21.15.26:0/1865694479 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48032810 con 0x7f0d580b4530 2024-04-24T23:20:55.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.609+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1865694479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b4530 msgr2=0x7f0d580b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.609+0000 7f0d5e5b7640 1 --2- 172.21.15.26:0/1865694479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b4530 0x7f0d580b4fd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d480097a0 tx=0x7f0d4802a270 comp rx=0 tx=0).stop 2024-04-24T23:20:55.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.609+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1865694479 shutdown_connections 2024-04-24T23:20:55.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.609+0000 7f0d5e5b7640 1 --2- 172.21.15.26:0/1865694479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b4530 0x7f0d580b4fd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.609+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1865694479 >> 172.21.15.26:0/1865694479 conn(0x7f0d5806e2c0 msgr2=0x7f0d5806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:55.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.610+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1865694479 shutdown_connections 2024-04-24T23:20:55.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.610+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1865694479 wait complete. 2024-04-24T23:20:55.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.611+0000 7f0d5e5b7640 1 Processor -- start 2024-04-24T23:20:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.611+0000 7f0d5e5b7640 1 -- start start 2024-04-24T23:20:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d5e5b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b0030 0x7f0d5814b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d5e5b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d580b4530 0x7f0d58155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d5e5b7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0d5815e380 0x7f0d58156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d5e5b7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580ae8c0 con 0x7f0d580b0030 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d5e5b7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580ae740 con 0x7f0d580b4530 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d5e5b7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0d580aea40 con 0x7f0d5815e380 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b0030 0x7f0d5814b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b0030 0x7f0d5814b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54472/0 (socket says 172.21.15.26:54472) 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d57fff640 1 -- 172.21.15.26:0/1407619038 learned_addr learned my addr 172.21.15.26:0/1407619038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d577fe640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d580b4530 0x7f0d58155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d5cb54640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0d5815e380 0x7f0d58156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d57fff640 1 -- 172.21.15.26:0/1407619038 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0d5815e380 msgr2=0x7f0d58156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d57fff640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0d5815e380 0x7f0d58156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d57fff640 1 -- 172.21.15.26:0/1407619038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d580b4530 msgr2=0x7f0d58155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d57fff640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d580b4530 0x7f0d58155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d57fff640 1 -- 172.21.15.26:0/1407619038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d581578c0 con 0x7f0d580b0030 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d577fe640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d580b4530 0x7f0d58155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d57fff640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b0030 0x7f0d5814b690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00b420 tx=0x7f0d4c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d54ff9640 1 -- 172.21.15.26:0/1407619038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c011830 con 0x7f0d580b0030 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d54ff9640 1 -- 172.21.15.26:0/1407619038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c016070 con 0x7f0d580b0030 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d58157ba0 con 0x7f0d580b0030 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d54ff9640 1 -- 172.21.15.26:0/1407619038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c011c10 con 0x7f0d580b0030 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.612+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d58158160 con 0x7f0d580b0030 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.613+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d580694c0 con 0x7f0d580b0030 2024-04-24T23:20:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.614+0000 7f0d54ff9640 1 -- 172.21.15.26:0/1407619038 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d4c01b460 con 0x7f0d580b0030 2024-04-24T23:20:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.614+0000 7f0d54ff9640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0d4406be30 0x7f0d4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.614+0000 7f0d54ff9640 1 -- 172.21.15.26:0/1407619038 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f0d4c08ab70 con 0x7f0d580b0030 2024-04-24T23:20:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.614+0000 7f0d577fe640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0d4406be30 0x7f0d4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.615+0000 7f0d577fe640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0d4406be30 0x7f0d4406e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d48034000 tx=0x7f0d4802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.619+0000 7f0d54ff9640 1 -- 172.21.15.26:0/1407619038 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d4c059520 con 0x7f0d580b0030 2024-04-24T23:20:55.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.802+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d5815d1c0 con 0x7f0d580b0030 2024-04-24T23:20:55.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.803+0000 7f0d54ff9640 1 -- 172.21.15.26:0/1407619038 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16558 (secure 0 0 0) 0x7f0d4c058e20 con 0x7f0d580b0030 2024-04-24T23:20:55.805 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:55.806 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":19,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:54.833220+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:20:55.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0d4406be30 msgr2=0x7f0d4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0d4406be30 0x7f0d4406e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d48034000 tx=0x7f0d4802e000 comp rx=0 tx=0).stop 2024-04-24T23:20:55.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b0030 msgr2=0x7f0d5814b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b0030 0x7f0d5814b690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00b420 tx=0x7f0d4c00b8e0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 shutdown_connections 2024-04-24T23:20:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0d4406be30 0x7f0d4406e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0d5815e380 0x7f0d58156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d580b4530 0x7f0d58155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 --2- 172.21.15.26:0/1407619038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d580b0030 0x7f0d5814b690 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 >> 172.21.15.26:0/1407619038 conn(0x7f0d5806e2c0 msgr2=0x7f0d5815d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 shutdown_connections 2024-04-24T23:20:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.805+0000 7f0d5e5b7640 1 -- 172.21.15.26:0/1407619038 wait complete. 2024-04-24T23:20:55.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:20:55.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.916+0000 7f862ffee640 1 Processor -- start 2024-04-24T23:20:55.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.917+0000 7f862ffee640 1 -- start start 2024-04-24T23:20:55.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.917+0000 7f862ffee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86280b3130 0x7f86280b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.917+0000 7f862ffee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86280d32f0 con 0x7f86280d4870 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.917+0000 7f862ffee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86280d0270 con 0x7f862815e570 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.917+0000 7f862ffee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f86280d1ab0 con 0x7f86280b3130 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.917+0000 7f862e58b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86280b3130 0x7f86280b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.918+0000 7f862d589640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86280d4870 0x7f86280b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46266/0 (socket says 172.21.15.26:46266) 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.918+0000 7f862e58b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86280b3130 0x7f86280b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54474/0 (socket says 172.21.15.26:54474) 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.918+0000 7f862d589640 1 -- 172.21.15.26:0/3979931115 learned_addr learned my addr 172.21.15.26:0/3979931115 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.918+0000 7f862e58b640 1 -- 172.21.15.26:0/3979931115 >> v1:172.21.15.29:6789/0 conn(0x7f862815e570 legacy=0x7f86280d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.918+0000 7f862e58b640 1 -- 172.21.15.26:0/3979931115 >> v1:172.21.15.26:6789/0 conn(0x7f86280d4870 legacy=0x7f86280b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.918+0000 7f862e58b640 1 -- 172.21.15.26:0/3979931115 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86280d2d30 con 0x7f86280b3130 2024-04-24T23:20:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.918+0000 7f862cd88640 1 -- 172.21.15.26:0/3979931115 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4045923405 0 0) 0x7f861c001110 con 0x7f86280d4870 2024-04-24T23:20:55.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862e58b640 1 --2- 172.21.15.26:0/3979931115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b3130 0x7f86280b4810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8618009d90 tx=0x7f861802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cfd0b2a5e0104f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862cd88640 1 -- 172.21.15.26:0/3979931115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618007740 con 0x7f86280b3130 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862cd88640 1 -- 172.21.15.26:0/3979931115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86180078e0 con 0x7f86280b3130 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862cd88640 1 -- 172.21.15.26:0/3979931115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618007cb0 con 0x7f86280b3130 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862ffee640 1 -- 172.21.15.26:0/3979931115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b3130 msgr2=0x7f86280b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862ffee640 1 --2- 172.21.15.26:0/3979931115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b3130 0x7f86280b4810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8618009d90 tx=0x7f861802a660 comp rx=0 tx=0).stop 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862ffee640 1 -- 172.21.15.26:0/3979931115 shutdown_connections 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862ffee640 1 --2- 172.21.15.26:0/3979931115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280b3130 0x7f86280b4810 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862ffee640 1 -- 172.21.15.26:0/3979931115 >> 172.21.15.26:0/3979931115 conn(0x7f862808dbe0 msgr2=0x7f862809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862ffee640 1 -- 172.21.15.26:0/3979931115 shutdown_connections 2024-04-24T23:20:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.919+0000 7f862ffee640 1 -- 172.21.15.26:0/3979931115 wait complete. 2024-04-24T23:20:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.920+0000 7f862ffee640 1 Processor -- start 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.920+0000 7f862ffee640 1 -- start start 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862ffee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86280b3130 0x7f8628155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862ffee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280d4870 0x7f862815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862ffee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f862815e570 0x7f8628159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862ffee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86280cfa70 con 0x7f86280d4870 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862ffee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86280d2ae0 con 0x7f86280b3130 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862ffee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f86280cfd50 con 0x7f862815e570 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862d589640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280d4870 0x7f862815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862d589640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280d4870 0x7f862815b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54480/0 (socket says 172.21.15.26:54480) 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862d589640 1 -- 172.21.15.26:0/326319373 learned_addr learned my addr 172.21.15.26:0/326319373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862dd8a640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86280b3130 0x7f8628155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.921+0000 7f862e58b640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f862815e570 0x7f8628159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f862d589640 1 -- 172.21.15.26:0/326319373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f862815e570 msgr2=0x7f8628159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f862d589640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f862815e570 0x7f8628159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f862d589640 1 -- 172.21.15.26:0/326319373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86280b3130 msgr2=0x7f8628155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f862d589640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86280b3130 0x7f8628155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f862d589640 1 -- 172.21.15.26:0/326319373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862815a650 con 0x7f86280d4870 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f862d589640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280d4870 0x7f862815b6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c780 tx=0x7f861c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f86267fc640 1 -- 172.21.15.26:0/326319373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c018070 con 0x7f86280d4870 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f86267fc640 1 -- 172.21.15.26:0/326319373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861c0072d0 con 0x7f86280d4870 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862815a930 con 0x7f86280d4870 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f86267fc640 1 -- 172.21.15.26:0/326319373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c0047e0 con 0x7f86280d4870 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.922+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862815aef0 con 0x7f86280d4870 2024-04-24T23:20:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.923+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86280694c0 con 0x7f86280d4870 2024-04-24T23:20:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.924+0000 7f86267fc640 1 -- 172.21.15.26:0/326319373 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f861c01b070 con 0x7f86280d4870 2024-04-24T23:20:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.924+0000 7f86267fc640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f861406be30 0x7f861406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.924+0000 7f86267fc640 1 -- 172.21.15.26:0/326319373 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f861c08a0e0 con 0x7f86280d4870 2024-04-24T23:20:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.924+0000 7f862dd8a640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f861406be30 0x7f861406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.924+0000 7f862dd8a640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f861406be30 0x7f861406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8620000f60 tx=0x7f8620009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:55.928+0000 7f86267fc640 1 -- 172.21.15.26:0/326319373 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86280d32f0 con 0x7f86280d4870 2024-04-24T23:20:56.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.111+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86280b2d80 con 0x7f86280d4870 2024-04-24T23:20:56.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.112+0000 7f86267fc640 1 -- 172.21.15.26:0/326319373 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16558 (secure 0 0 0) 0x7f861c007d00 con 0x7f86280d4870 2024-04-24T23:20:56.114 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:56.114 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":19,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:54.833220+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:20:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f861406be30 msgr2=0x7f861406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f861406be30 0x7f861406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8620000f60 tx=0x7f8620009040 comp rx=0 tx=0).stop 2024-04-24T23:20:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280d4870 msgr2=0x7f862815b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280d4870 0x7f862815b6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c780 tx=0x7f861c00cc40 comp rx=0 tx=0).stop 2024-04-24T23:20:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 shutdown_connections 2024-04-24T23:20:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f861406be30 0x7f861406e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f862815e570 0x7f8628159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86280d4870 0x7f862815b6d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 --2- 172.21.15.26:0/326319373 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86280b3130 0x7f8628155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 >> 172.21.15.26:0/326319373 conn(0x7f862808dbe0 msgr2=0x7f862806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:56.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 shutdown_connections 2024-04-24T23:20:56.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.114+0000 7f862ffee640 1 -- 172.21.15.26:0/326319373 wait complete. 2024-04-24T23:20:56.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:20:56.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.229+0000 7f02ca215640 1 Processor -- start 2024-04-24T23:20:56.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.229+0000 7f02ca215640 1 -- start start 2024-04-24T23:20:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.229+0000 7f02ca215640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f02c40d2c30 0x7f02c40cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.229+0000 7f02ca215640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c40d2120 con 0x7f02c415e370 2024-04-24T23:20:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.229+0000 7f02ca215640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c40b3740 con 0x7f02c415e750 2024-04-24T23:20:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.229+0000 7f02ca215640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f02c40d1510 con 0x7f02c40d2c30 2024-04-24T23:20:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f02c415e370 0x7f02c40d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46272/0 (socket says 172.21.15.26:46272) 2024-04-24T23:20:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c37fe640 1 -- 172.21.15.26:0/3420571248 learned_addr learned my addr 172.21.15.26:0/3420571248 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c3fff640 1 --2- 172.21.15.26:0/3420571248 >> v2:172.21.15.29:3300/0 conn(0x7f02c40d2c30 0x7f02c40cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2409311373 0 0) 0x7f02c40d2120 con 0x7f02c415e370 2024-04-24T23:20:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02ac003520 con 0x7f02c415e370 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1483071994 0 0) 0x7f02ac003520 con 0x7f02c415e370 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 >> v2:172.21.15.29:3300/0 conn(0x7f02c40d2c30 msgr2=0x7f02c40cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 --2- 172.21.15.26:0/3420571248 >> v2:172.21.15.29:3300/0 conn(0x7f02c40d2c30 0x7f02c40cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 >> v1:172.21.15.29:6789/0 conn(0x7f02c415e750 legacy=0x7f02c40cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c40b3a20 con 0x7f02c415e370 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f02b4001670 con 0x7f02c415e370 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 >> v1:172.21.15.26:6789/0 conn(0x7f02c415e370 legacy=0x7f02c40d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ac0039b0 0x7f02ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ac0051c0 0x7f02ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac00c030 con 0x7f02ac004800 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac00c1e0 con 0x7f02ac0051c0 2024-04-24T23:20:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f02ac00c3e0 con 0x7f02ac0039b0 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c2ffd640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c37fe640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ac0039b0 0x7f02ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.230+0000 7f02c3fff640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ac0051c0 0x7f02ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c2ffd640 1 -- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ac0039b0 msgr2=0x7f02ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c2ffd640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ac0039b0 0x7f02ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c2ffd640 1 -- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ac0051c0 msgr2=0x7f02ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c2ffd640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ac0051c0 0x7f02ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c2ffd640 1 -- 172.21.15.26:0/3420571248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ac003520 con 0x7f02ac004800 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c3fff640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ac0051c0 0x7f02ac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c37fe640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ac0039b0 0x7f02ac003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c3fff640 1 -- 172.21.15.26:0/3420571248 reap_dead start 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c2ffd640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02b8001750 tx=0x7f02ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed89a67a1a4456 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8001370 con 0x7f02ac004800 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b80052d0 con 0x7f02ac004800 2024-04-24T23:20:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.231+0000 7f02c27fc640 1 -- 172.21.15.26:0/3420571248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b802abc0 con 0x7f02ac004800 2024-04-24T23:20:56.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.232+0000 7f02ca215640 1 -- 172.21.15.26:0/3420571248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ac004800 msgr2=0x7f02ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.232+0000 7f02ca215640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02b8001750 tx=0x7f02ac005830 comp rx=0 tx=0).stop 2024-04-24T23:20:56.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.232+0000 7f02ca215640 1 -- 172.21.15.26:0/3420571248 shutdown_connections 2024-04-24T23:20:56.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.232+0000 7f02ca215640 1 --2- 172.21.15.26:0/3420571248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.232+0000 7f02ca215640 1 -- 172.21.15.26:0/3420571248 >> 172.21.15.26:0/3420571248 conn(0x7f02c40d40f0 msgr2=0x7f02c40d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:56.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.232+0000 7f02ca215640 1 -- 172.21.15.26:0/3420571248 shutdown_connections 2024-04-24T23:20:56.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.232+0000 7f02ca215640 1 -- 172.21.15.26:0/3420571248 wait complete. 2024-04-24T23:20:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.233+0000 7f02ca215640 1 Processor -- start 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.233+0000 7f02ca215640 1 -- start start 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02ca215640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ac004800 0x7f02c41f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02ca215640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b0c50 0x7f02c40b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02ca215640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c40b1610 0x7f02c4145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02ca215640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f02c40d4c10 con 0x7f02c40b0c50 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02ca215640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02c40d1ed0 con 0x7f02c40b1610 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02ca215640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f02c40d4d90 con 0x7f02ac004800 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b0c50 0x7f02c40b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b0c50 0x7f02c40b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54500/0 (socket says 172.21.15.26:54500) 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02c2ffd640 1 -- 172.21.15.26:0/1664834802 learned_addr learned my addr 172.21.15.26:0/1664834802 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02c3fff640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c40b1610 0x7f02c4145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02c2ffd640 1 -- 172.21.15.26:0/1664834802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ac004800 msgr2=0x7f02c41f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02c2ffd640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ac004800 0x7f02c41f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02c2ffd640 1 -- 172.21.15.26:0/1664834802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c40b1610 msgr2=0x7f02c4145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02c2ffd640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c40b1610 0x7f02c4145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.234+0000 7f02c2ffd640 1 -- 172.21.15.26:0/1664834802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c4147fb0 con 0x7f02c40b0c50 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.235+0000 7f02c2ffd640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b0c50 0x7f02c40b10d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000c00 tx=0x7f02b8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.235+0000 7f02abfff640 1 -- 172.21.15.26:0/1664834802 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b803f030 con 0x7f02c40b0c50 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.235+0000 7f02abfff640 1 -- 172.21.15.26:0/1664834802 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b8005480 con 0x7f02c40b0c50 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.235+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c4145230 con 0x7f02c40b0c50 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.235+0000 7f02abfff640 1 -- 172.21.15.26:0/1664834802 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8037040 con 0x7f02c40b0c50 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.235+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c41457f0 con 0x7f02c40b0c50 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.236+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c4075610 con 0x7f02c40b0c50 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.236+0000 7f02abfff640 1 -- 172.21.15.26:0/1664834802 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02b8042070 con 0x7f02c40b0c50 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.237+0000 7f02abfff640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02b006be30 0x7f02b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.237+0000 7f02abfff640 1 -- 172.21.15.26:0/1664834802 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f02b80aa520 con 0x7f02c40b0c50 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.237+0000 7f02c37fe640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02b006be30 0x7f02b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.237+0000 7f02c37fe640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02b006be30 0x7f02b006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02b4001eb0 tx=0x7f02b4001150 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:56.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.241+0000 7f02abfff640 1 -- 172.21.15.26:0/1664834802 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02b8049070 con 0x7f02c40b0c50 2024-04-24T23:20:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.461+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f02c40cac50 con 0x7f02c40b0c50 2024-04-24T23:20:56.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.825+0000 7f02abfff640 1 -- 172.21.15.26:0/1664834802 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f02b8078f60 con 0x7f02c40b0c50 2024-04-24T23:20:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02b006be30 msgr2=0x7f02b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02b006be30 0x7f02b006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02b4001eb0 tx=0x7f02b4001150 comp rx=0 tx=0).stop 2024-04-24T23:20:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b0c50 msgr2=0x7f02c40b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b0c50 0x7f02c40b10d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000c00 tx=0x7f02b8001120 comp rx=0 tx=0).stop 2024-04-24T23:20:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 shutdown_connections 2024-04-24T23:20:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02b006be30 0x7f02b006e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02c40b1610 0x7f02c4145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b0c50 0x7f02c40b10d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 --2- 172.21.15.26:0/1664834802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ac004800 0x7f02c41f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 >> 172.21.15.26:0/1664834802 conn(0x7f02c40d40f0 msgr2=0x7f02c40cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 shutdown_connections 2024-04-24T23:20:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.827+0000 7f02ca215640 1 -- 172.21.15.26:0/1664834802 wait complete. 2024-04-24T23:20:56.841 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:20:56.841 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:20:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.947+0000 7efcc1483640 1 Processor -- start 2024-04-24T23:20:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.947+0000 7efcc1483640 1 -- start start 2024-04-24T23:20:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcc1483640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efcbc158ae0 0x7efcbc06e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcc1483640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcbc0d0ce0 0x7efcbc15e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcc1483640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcbc06b1c0 con 0x7efcbc15ee90 2024-04-24T23:20:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcc1483640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7efcbc06ec90 con 0x7efcbc158ae0 2024-04-24T23:20:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcc1483640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efcbc06ef70 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcbaffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efcbc158ae0 0x7efcbc06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcbb7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efcbc15ee90 0x7efcbc071720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46280/0 (socket says 172.21.15.26:46280) 2024-04-24T23:20:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcbb7fe640 1 -- 172.21.15.26:0/27511751 learned_addr learned my addr 172.21.15.26:0/27511751 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcbaffd640 1 -- 172.21.15.26:0/27511751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcbc0d0ce0 msgr2=0x7efcbc15e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcbaffd640 1 --2- 172.21.15.26:0/27511751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcbc0d0ce0 0x7efcbc15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcbaffd640 1 -- 172.21.15.26:0/27511751 >> v1:172.21.15.26:6789/0 conn(0x7efcbc15ee90 legacy=0x7efcbc071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcbaffd640 1 -- 172.21.15.26:0/27511751 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcbc06f250 con 0x7efcbc158ae0 2024-04-24T23:20:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.948+0000 7efcb9ffb640 1 -- 172.21.15.26:0/27511751 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794184068 0 0) 0x7efcbc06b1c0 con 0x7efcbc15ee90 2024-04-24T23:20:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.949+0000 7efcbaffd640 1 --2- 172.21.15.26:0/27511751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc158ae0 0x7efcbc06e620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efcb40097a0 tx=0x7efcb402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e5c95f6879c7f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.949+0000 7efcb9ffb640 1 -- 172.21.15.26:0/27511751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4037070 con 0x7efcbc158ae0 2024-04-24T23:20:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.949+0000 7efcb9ffb640 1 -- 172.21.15.26:0/27511751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb402ae50 con 0x7efcbc158ae0 2024-04-24T23:20:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.949+0000 7efcb9ffb640 1 -- 172.21.15.26:0/27511751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4032660 con 0x7efcbc158ae0 2024-04-24T23:20:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.949+0000 7efcc1483640 1 -- 172.21.15.26:0/27511751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc158ae0 msgr2=0x7efcbc06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.949+0000 7efcc1483640 1 --2- 172.21.15.26:0/27511751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc158ae0 0x7efcbc06e620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efcb40097a0 tx=0x7efcb402a100 comp rx=0 tx=0).stop 2024-04-24T23:20:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.950+0000 7efcc1483640 1 -- 172.21.15.26:0/27511751 shutdown_connections 2024-04-24T23:20:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.950+0000 7efcc1483640 1 --2- 172.21.15.26:0/27511751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcbc0d0ce0 0x7efcbc15e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.950+0000 7efcc1483640 1 --2- 172.21.15.26:0/27511751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc158ae0 0x7efcbc06e620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.950+0000 7efcc1483640 1 -- 172.21.15.26:0/27511751 >> 172.21.15.26:0/27511751 conn(0x7efcbc0b2970 msgr2=0x7efcbc0d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.950+0000 7efcc1483640 1 -- 172.21.15.26:0/27511751 shutdown_connections 2024-04-24T23:20:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.950+0000 7efcc1483640 1 -- 172.21.15.26:0/27511751 wait complete. 2024-04-24T23:20:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.951+0000 7efcc1483640 1 Processor -- start 2024-04-24T23:20:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.951+0000 7efcc1483640 1 -- start start 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcc1483640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc0d0ce0 0x7efcbc147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcc1483640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcbc158ae0 0x7efcbc0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcc1483640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcbc15ee90 0x7efcbc0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcc1483640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc0b00e0 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcc1483640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc0aff60 con 0x7efcbc15ee90 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcc1483640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efcbc071e30 con 0x7efcbc158ae0 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc0d0ce0 0x7efcbc147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc0d0ce0 0x7efcbc147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54518/0 (socket says 172.21.15.26:54518) 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbaffd640 1 -- 172.21.15.26:0/3651748529 learned_addr learned my addr 172.21.15.26:0/3651748529 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcba7fc640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcbc158ae0 0x7efcbc0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbaffd640 1 -- 172.21.15.26:0/3651748529 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcbc158ae0 msgr2=0x7efcbc0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbaffd640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcbc158ae0 0x7efcbc0aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbaffd640 1 -- 172.21.15.26:0/3651748529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcbc15ee90 msgr2=0x7efcbc0b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbb7fe640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcbc15ee90 0x7efcbc0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbaffd640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcbc15ee90 0x7efcbc0b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbaffd640 1 -- 172.21.15.26:0/3651748529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcbc0ade50 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcbaffd640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc0d0ce0 0x7efcbc147040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efcb40097a0 tx=0x7efcb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcb37fe640 1 -- 172.21.15.26:0/3651748529 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4042070 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcb37fe640 1 -- 172.21.15.26:0/3651748529 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb403e670 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcbc0ae130 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.952+0000 7efcb37fe640 1 -- 172.21.15.26:0/3651748529 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4037040 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.953+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcbc0ae6f0 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.953+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcbc075610 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.954+0000 7efcb37fe640 1 -- 172.21.15.26:0/3651748529 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcb4045070 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.954+0000 7efcb37fe640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc9c06be30 0x7efc9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.954+0000 7efcb37fe640 1 -- 172.21.15.26:0/3651748529 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efcb40ab760 con 0x7efcbc0d0ce0 2024-04-24T23:20:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.954+0000 7efcba7fc640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc9c06be30 0x7efc9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.955+0000 7efcba7fc640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc9c06be30 0x7efc9c06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcbc06d600 tx=0x7efca4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:56.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:56.958+0000 7efcb37fe640 1 -- 172.21.15.26:0/3651748529 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efcb407a220 con 0x7efcbc0d0ce0 2024-04-24T23:20:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.180+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7efcbc0b5b40 con 0x7efcbc0d0ce0 2024-04-24T23:20:57.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.828+0000 7efcb37fe640 1 -- 172.21.15.26:0/3651748529 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7efcb4079b20 con 0x7efcbc0d0ce0 2024-04-24T23:20:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc9c06be30 msgr2=0x7efc9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc9c06be30 0x7efc9c06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcbc06d600 tx=0x7efca4008040 comp rx=0 tx=0).stop 2024-04-24T23:20:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc0d0ce0 msgr2=0x7efcbc147040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc0d0ce0 0x7efcbc147040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efcb40097a0 tx=0x7efcb40046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 shutdown_connections 2024-04-24T23:20:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc9c06be30 0x7efc9c06e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcbc15ee90 0x7efcbc0b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcbc158ae0 0x7efcbc0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 --2- 172.21.15.26:0/3651748529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcbc0d0ce0 0x7efcbc147040 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 >> 172.21.15.26:0/3651748529 conn(0x7efcbc0b2970 msgr2=0x7efcbc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 shutdown_connections 2024-04-24T23:20:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.831+0000 7efcc1483640 1 -- 172.21.15.26:0/3651748529 wait complete. 2024-04-24T23:20:57.845 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:20:57.845 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:20:57.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.948+0000 7f06116ee640 1 Processor -- start 2024-04-24T23:20:57.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.948+0000 7f06116ee640 1 -- start start 2024-04-24T23:20:57.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f06116ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f060c0cdd70 0x7f060c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:57.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f06116ee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060c0cfbc0 con 0x7f060c145910 2024-04-24T23:20:57.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f06116ee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060c06d7f0 con 0x7f060c06d350 2024-04-24T23:20:57.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f06116ee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f060c145cf0 con 0x7f060c0cdd70 2024-04-24T23:20:57.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f060b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f060c0cdd70 0x7f060c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:57.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f060affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f060c145910 0x7f060c15d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46284/0 (socket says 172.21.15.26:46284) 2024-04-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f060affd640 1 -- 172.21.15.26:0/986657948 learned_addr learned my addr 172.21.15.26:0/986657948 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f060b7fe640 1 -- 172.21.15.26:0/986657948 >> v1:172.21.15.29:6789/0 conn(0x7f060c06d350 legacy=0x7f060c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f060b7fe640 1 -- 172.21.15.26:0/986657948 >> v1:172.21.15.26:6789/0 conn(0x7f060c145910 legacy=0x7f060c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f0609ffb640 1 -- 172.21.15.26:0/986657948 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819958513 0 0) 0x7f060c0cfbc0 con 0x7f060c145910 2024-04-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.949+0000 7f060b7fe640 1 -- 172.21.15.26:0/986657948 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c06dad0 con 0x7f060c0cdd70 2024-04-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.950+0000 7f060b7fe640 1 --2- 172.21.15.26:0/986657948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c0cdd70 0x7f060c06e890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0604009dc0 tx=0x7f060402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de834d678293cb26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.950+0000 7f0609ffb640 1 -- 172.21.15.26:0/986657948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604007740 con 0x7f060c0cdd70 2024-04-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.950+0000 7f0609ffb640 1 -- 172.21.15.26:0/986657948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06040078e0 con 0x7f060c0cdd70 2024-04-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.950+0000 7f0609ffb640 1 -- 172.21.15.26:0/986657948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604007cb0 con 0x7f060c0cdd70 2024-04-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.951+0000 7f06116ee640 1 -- 172.21.15.26:0/986657948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c0cdd70 msgr2=0x7f060c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.951+0000 7f06116ee640 1 --2- 172.21.15.26:0/986657948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c0cdd70 0x7f060c06e890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0604009dc0 tx=0x7f060402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.951+0000 7f06116ee640 1 -- 172.21.15.26:0/986657948 shutdown_connections 2024-04-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.951+0000 7f06116ee640 1 --2- 172.21.15.26:0/986657948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c0cdd70 0x7f060c06e890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.951+0000 7f06116ee640 1 -- 172.21.15.26:0/986657948 >> 172.21.15.26:0/986657948 conn(0x7f060c06b300 msgr2=0x7f060c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.951+0000 7f06116ee640 1 -- 172.21.15.26:0/986657948 shutdown_connections 2024-04-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.951+0000 7f06116ee640 1 -- 172.21.15.26:0/986657948 wait complete. 2024-04-24T23:20:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.952+0000 7f06116ee640 1 Processor -- start 2024-04-24T23:20:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.952+0000 7f06116ee640 1 -- start start 2024-04-24T23:20:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f06116ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c06d350 0x7f060c139200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f06116ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f060c0cdd70 0x7f060c0ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f06116ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060c145910 0x7f060c0ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f06116ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f060c06e370 con 0x7f060c06d350 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f06116ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f060c06e090 con 0x7f060c145910 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f06116ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f060c15d770 con 0x7f060c0cdd70 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c06d350 0x7f060c139200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c06d350 0x7f060c139200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54536/0 (socket says 172.21.15.26:54536) 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060affd640 1 -- 172.21.15.26:0/298021596 learned_addr learned my addr 172.21.15.26:0/298021596 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060a7fc640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f060c0cdd70 0x7f060c0ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060b7fe640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060c145910 0x7f060c0ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060affd640 1 -- 172.21.15.26:0/298021596 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f060c0cdd70 msgr2=0x7f060c0ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060affd640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f060c0cdd70 0x7f060c0ac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060affd640 1 -- 172.21.15.26:0/298021596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060c145910 msgr2=0x7f060c0ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060affd640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060c145910 0x7f060c0ac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060affd640 1 -- 172.21.15.26:0/298021596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c139740 con 0x7f060c06d350 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.953+0000 7f060affd640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c06d350 0x7f060c139200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c420 tx=0x7f05fc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.954+0000 7f05f37fe640 1 -- 172.21.15.26:0/298021596 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0074e0 con 0x7f060c06d350 2024-04-24T23:20:57.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.954+0000 7f05f37fe640 1 -- 172.21.15.26:0/298021596 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc0047e0 con 0x7f060c06d350 2024-04-24T23:20:57.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.954+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c13c890 con 0x7f060c06d350 2024-04-24T23:20:57.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.954+0000 7f05f37fe640 1 -- 172.21.15.26:0/298021596 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc004c10 con 0x7f060c06d350 2024-04-24T23:20:57.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.954+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c0a9b70 con 0x7f060c06d350 2024-04-24T23:20:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.955+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060c06b910 con 0x7f060c06d350 2024-04-24T23:20:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.955+0000 7f05f37fe640 1 -- 172.21.15.26:0/298021596 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05fc011070 con 0x7f060c06d350 2024-04-24T23:20:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.956+0000 7f05f37fe640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05f806be30 0x7f05f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.956+0000 7f060a7fc640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05f806be30 0x7f05f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.956+0000 7f05f37fe640 1 -- 172.21.15.26:0/298021596 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f05fc08aa60 con 0x7f060c06d350 2024-04-24T23:20:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.956+0000 7f060a7fc640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05f806be30 0x7f05f806e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0600000f60 tx=0x7f0600009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:57.960+0000 7f05f37fe640 1 -- 172.21.15.26:0/298021596 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f060c06b910 con 0x7f060c06d350 2024-04-24T23:20:58.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.181+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f060c0728a0 con 0x7f060c06d350 2024-04-24T23:20:58.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.182+0000 7f05f37fe640 1 -- 172.21.15.26:0/298021596 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7f060c139740 con 0x7f060c06d350 2024-04-24T23:20:58.184 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:58.184 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2829963318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2829963318},{"type":"v1","addr":"172.21.15.26:6837","nonce":2829963318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2687649505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2687649505},{"type":"v1","addr":"172.21.15.29:6833","nonce":2687649505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:20:54.833525+0000","modified":"2024-04-24T23:20:57.181910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1604449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1604449840},{"type":"v1","addr":"172.21.15.29:6835","nonce":1604449840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-24T23:20:58.184 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05f806be30 msgr2=0x7f05f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05f806be30 0x7f05f806e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0600000f60 tx=0x7f0600009040 comp rx=0 tx=0).stop 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c06d350 msgr2=0x7f060c139200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c06d350 0x7f060c139200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c420 tx=0x7f05fc00c8e0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 shutdown_connections 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05f806be30 0x7f05f806e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f060c145910 0x7f060c0ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f060c0cdd70 0x7f060c0ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 --2- 172.21.15.26:0/298021596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f060c06d350 0x7f060c139200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 >> 172.21.15.26:0/298021596 conn(0x7f060c06b300 msgr2=0x7f060c072be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 shutdown_connections 2024-04-24T23:20:58.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.184+0000 7f06116ee640 1 -- 172.21.15.26:0/298021596 wait complete. 2024-04-24T23:20:58.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:20:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.295+0000 7ff15fc3f640 1 Processor -- start 2024-04-24T23:20:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.295+0000 7ff15fc3f640 1 -- start start 2024-04-24T23:20:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15fc3f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff15815a480 0x7ff1580b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15fc3f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff15815a860 0x7ff15806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15fc3f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1580b2e80 con 0x7ff1580b4e30 2024-04-24T23:20:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15fc3f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff1580b3120 con 0x7ff15815a860 2024-04-24T23:20:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15fc3f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff1580b33c0 con 0x7ff15815a480 2024-04-24T23:20:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15d1da640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff15815a860 0x7ff15806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15d1da640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff15815a860 0x7ff15806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54544/0 (socket says 172.21.15.26:54544) 2024-04-24T23:20:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15e1dc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1580b4e30 0x7ff15815e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46298/0 (socket says 172.21.15.26:46298) 2024-04-24T23:20:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15d1da640 1 -- 172.21.15.26:0/3165672170 learned_addr learned my addr 172.21.15.26:0/3165672170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15d9db640 1 --2- 172.21.15.26:0/3165672170 >> v2:172.21.15.29:3300/0 conn(0x7ff15815a480 0x7ff1580b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15d1da640 1 -- 172.21.15.26:0/3165672170 >> v2:172.21.15.29:3300/0 conn(0x7ff15815a480 msgr2=0x7ff1580b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15d1da640 1 --2- 172.21.15.26:0/3165672170 >> v2:172.21.15.29:3300/0 conn(0x7ff15815a480 0x7ff1580b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15d1da640 1 -- 172.21.15.26:0/3165672170 >> v1:172.21.15.26:6789/0 conn(0x7ff1580b4e30 legacy=0x7ff15815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.296+0000 7ff15d1da640 1 -- 172.21.15.26:0/3165672170 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1580b36a0 con 0x7ff15815a860 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.297+0000 7ff15d1da640 1 --2- 172.21.15.26:0/3165672170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a860 0x7ff15806dbe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff15806eab0 tx=0x7ff14802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcea78029743f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.297+0000 7ff15c9d9640 1 -- 172.21.15.26:0/3165672170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148033640 con 0x7ff15815a860 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.297+0000 7ff15c9d9640 1 -- 172.21.15.26:0/3165672170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff148038070 con 0x7ff15815a860 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.297+0000 7ff15c9d9640 1 -- 172.21.15.26:0/3165672170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148033a00 con 0x7ff15815a860 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.297+0000 7ff15fc3f640 1 -- 172.21.15.26:0/3165672170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a860 msgr2=0x7ff15806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.297+0000 7ff15fc3f640 1 --2- 172.21.15.26:0/3165672170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a860 0x7ff15806dbe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff15806eab0 tx=0x7ff14802c8b0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.298+0000 7ff15fc3f640 1 -- 172.21.15.26:0/3165672170 shutdown_connections 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.298+0000 7ff15fc3f640 1 --2- 172.21.15.26:0/3165672170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a860 0x7ff15806dbe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.298+0000 7ff15fc3f640 1 --2- 172.21.15.26:0/3165672170 >> v2:172.21.15.29:3300/0 conn(0x7ff15815a480 0x7ff1580b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.298+0000 7ff15fc3f640 1 -- 172.21.15.26:0/3165672170 >> 172.21.15.26:0/3165672170 conn(0x7ff15806b300 msgr2=0x7ff15806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.298+0000 7ff15fc3f640 1 -- 172.21.15.26:0/3165672170 shutdown_connections 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.298+0000 7ff15fc3f640 1 -- 172.21.15.26:0/3165672170 wait complete. 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.299+0000 7ff15fc3f640 1 Processor -- start 2024-04-24T23:20:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.299+0000 7ff15fc3f640 1 -- start start 2024-04-24T23:20:58.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.299+0000 7ff15fc3f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580b4e30 0x7ff158148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.299+0000 7ff15fc3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a480 0x7ff1581584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.299+0000 7ff15fc3f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff15815a860 0x7ff158154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.299+0000 7ff15fc3f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff158072080 con 0x7ff15815a480 2024-04-24T23:20:58.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.299+0000 7ff15fc3f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1580b3c60 con 0x7ff15815a860 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.299+0000 7ff15fc3f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff158072200 con 0x7ff1580b4e30 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d1da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a480 0x7ff1581584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d1da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a480 0x7ff1581584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54550/0 (socket says 172.21.15.26:54550) 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d1da640 1 -- 172.21.15.26:0/1476315011 learned_addr learned my addr 172.21.15.26:0/1476315011 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15e1dc640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff15815a860 0x7ff158154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d9db640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580b4e30 0x7ff158148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d1da640 1 -- 172.21.15.26:0/1476315011 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580b4e30 msgr2=0x7ff158148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d1da640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580b4e30 0x7ff158148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d1da640 1 -- 172.21.15.26:0/1476315011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff15815a860 msgr2=0x7ff158154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d1da640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff15815a860 0x7ff158154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d1da640 1 -- 172.21.15.26:0/1476315011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1581e55c0 con 0x7ff15815a480 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15d1da640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a480 0x7ff1581584c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff148000c00 tx=0x7ff1480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff14e7fc640 1 -- 172.21.15.26:0/1476315011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1480465f0 con 0x7ff15815a480 2024-04-24T23:20:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff14e7fc640 1 -- 172.21.15.26:0/1476315011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff148046790 con 0x7ff15815a480 2024-04-24T23:20:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1581e58a0 con 0x7ff15815a480 2024-04-24T23:20:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff14e7fc640 1 -- 172.21.15.26:0/1476315011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148046b80 con 0x7ff15815a480 2024-04-24T23:20:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.300+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15814fd10 con 0x7ff15815a480 2024-04-24T23:20:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.301+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1580694c0 con 0x7ff15815a480 2024-04-24T23:20:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.302+0000 7ff14e7fc640 1 -- 172.21.15.26:0/1476315011 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff14803c3f0 con 0x7ff15815a480 2024-04-24T23:20:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.302+0000 7ff14e7fc640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.302+0000 7ff15d9db640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.302+0000 7ff14e7fc640 1 -- 172.21.15.26:0/1476315011 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff1480add10 con 0x7ff15815a480 2024-04-24T23:20:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.303+0000 7ff15d9db640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff154006fd0 tx=0x7ff154008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.307+0000 7ff14e7fc640 1 -- 172.21.15.26:0/1476315011 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff148041070 con 0x7ff15815a480 2024-04-24T23:20:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.492+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1580b44a0 con 0x7ff15815a480 2024-04-24T23:20:58.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.493+0000 7ff14e7fc640 1 -- 172.21.15.26:0/1476315011 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7ff14807c070 con 0x7ff15815a480 2024-04-24T23:20:58.495 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:58.495 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:20:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 msgr2=0x7ff13c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff154006fd0 tx=0x7ff154008040 comp rx=0 tx=0).stop 2024-04-24T23:20:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a480 msgr2=0x7ff1581584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a480 0x7ff1581584c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff148000c00 tx=0x7ff1480046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 shutdown_connections 2024-04-24T23:20:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff15815a860 0x7ff158154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff15815a480 0x7ff1581584c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 --2- 172.21.15.26:0/1476315011 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580b4e30 0x7ff158148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 >> 172.21.15.26:0/1476315011 conn(0x7ff15806b300 msgr2=0x7ff1581566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 shutdown_connections 2024-04-24T23:20:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.495+0000 7ff15fc3f640 1 -- 172.21.15.26:0/1476315011 wait complete. 2024-04-24T23:20:58.508 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:20:58.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:20:58.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.607+0000 7ff1d5b34640 1 Processor -- start 2024-04-24T23:20:58.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.607+0000 7ff1d5b34640 1 -- start start 2024-04-24T23:20:58.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1d5b34640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1d015e380 0x7ff1d00b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1d5b34640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1d00b4fd0 0x7ff1d0072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1d5b34640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff1d00b0030 0x7ff1d00b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1d5b34640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d015e760 con 0x7ff1d015e380 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1d5b34640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d00725b0 con 0x7ff1d00b0030 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1d5b34640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d00b09a0 con 0x7ff1d00b4fd0 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1cf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1d015e380 0x7ff1d00b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1cf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1d015e380 0x7ff1d00b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54562/0 (socket says 172.21.15.26:54562) 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1cf7fe640 1 -- 172.21.15.26:0/760702151 learned_addr learned my addr 172.21.15.26:0/760702151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1cffff640 1 --2- 172.21.15.26:0/760702151 >> v2:172.21.15.29:3300/0 conn(0x7ff1d00b0030 0x7ff1d00b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1cf7fe640 1 -- 172.21.15.26:0/760702151 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1d00b4fd0 msgr2=0x7ff1d0072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1cf7fe640 1 --2- 172.21.15.26:0/760702151 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1d00b4fd0 0x7ff1d0072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1cf7fe640 1 -- 172.21.15.26:0/760702151 >> v2:172.21.15.29:3300/0 conn(0x7ff1d00b0030 msgr2=0x7ff1d00b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1cf7fe640 1 --2- 172.21.15.26:0/760702151 >> v2:172.21.15.29:3300/0 conn(0x7ff1d00b0030 0x7ff1d00b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.608+0000 7ff1cf7fe640 1 -- 172.21.15.26:0/760702151 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d00b5700 con 0x7ff1d015e380 2024-04-24T23:20:58.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.609+0000 7ff1cf7fe640 1 --2- 172.21.15.26:0/760702151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d015e380 0x7ff1d00b4530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0072030 tx=0x7ff1c002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2d839f63840165 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.609+0000 7ff1ce7fc640 1 -- 172.21.15.26:0/760702151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0037070 con 0x7ff1d015e380 2024-04-24T23:20:58.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.609+0000 7ff1ce7fc640 1 -- 172.21.15.26:0/760702151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c0032440 con 0x7ff1d015e380 2024-04-24T23:20:58.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.609+0000 7ff1ce7fc640 1 -- 172.21.15.26:0/760702151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0032810 con 0x7ff1d015e380 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.610+0000 7ff1d5b34640 1 -- 172.21.15.26:0/760702151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d015e380 msgr2=0x7ff1d00b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.610+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/760702151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d015e380 0x7ff1d00b4530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0072030 tx=0x7ff1c002a2a0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.610+0000 7ff1d5b34640 1 -- 172.21.15.26:0/760702151 shutdown_connections 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.610+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/760702151 >> v2:172.21.15.29:3300/0 conn(0x7ff1d00b0030 0x7ff1d00b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.610+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/760702151 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1d00b4fd0 0x7ff1d0072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.610+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/760702151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d015e380 0x7ff1d00b4530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.610+0000 7ff1d5b34640 1 -- 172.21.15.26:0/760702151 >> 172.21.15.26:0/760702151 conn(0x7ff1d006e2c0 msgr2=0x7ff1d006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.610+0000 7ff1d5b34640 1 -- 172.21.15.26:0/760702151 shutdown_connections 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.610+0000 7ff1d5b34640 1 -- 172.21.15.26:0/760702151 wait complete. 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.611+0000 7ff1d5b34640 1 Processor -- start 2024-04-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.611+0000 7ff1d5b34640 1 -- start start 2024-04-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1d5b34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d00b0030 0x7ff1d00c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1d5b34640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1d00b4fd0 0x7ff1d01519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1d5b34640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1d015e380 0x7ff1d00c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1d5b34640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d00b5410 con 0x7ff1d00b0030 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1d5b34640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0072240 con 0x7ff1d015e380 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1d5b34640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d0072f80 con 0x7ff1d00b4fd0 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d00b0030 0x7ff1d00c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d00b0030 0x7ff1d00c7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54570/0 (socket says 172.21.15.26:54570) 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cf7fe640 1 -- 172.21.15.26:0/167376227 learned_addr learned my addr 172.21.15.26:0/167376227 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cf7fe640 1 -- 172.21.15.26:0/167376227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1d00b4fd0 msgr2=0x7ff1d01519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cf7fe640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1d00b4fd0 0x7ff1d01519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cffff640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1d015e380 0x7ff1d00c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cf7fe640 1 -- 172.21.15.26:0/167376227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1d015e380 msgr2=0x7ff1d00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cf7fe640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1d015e380 0x7ff1d00c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cf7fe640 1 -- 172.21.15.26:0/167376227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d00c7740 con 0x7ff1d00b0030 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1cf7fe640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d00b0030 0x7ff1d00c7160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0000c00 tx=0x7ff1c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1b7fff640 1 -- 172.21.15.26:0/167376227 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0037040 con 0x7ff1d00b0030 2024-04-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1b7fff640 1 -- 172.21.15.26:0/167376227 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c0042070 con 0x7ff1d00b0030 2024-04-24T23:20:58.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d00c7a20 con 0x7ff1d00b0030 2024-04-24T23:20:58.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.612+0000 7ff1b7fff640 1 -- 172.21.15.26:0/167376227 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0030bd0 con 0x7ff1d00b0030 2024-04-24T23:20:58.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.613+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d00b1480 con 0x7ff1d00b0030 2024-04-24T23:20:58.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.613+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d00694c0 con 0x7ff1d00b0030 2024-04-24T23:20:58.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.614+0000 7ff1b7fff640 1 -- 172.21.15.26:0/167376227 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1c0047460 con 0x7ff1d00b0030 2024-04-24T23:20:58.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.614+0000 7ff1b7fff640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1bc06be30 0x7ff1bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.615+0000 7ff1b7fff640 1 -- 172.21.15.26:0/167376227 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff1c00abab0 con 0x7ff1d00b0030 2024-04-24T23:20:58.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.615+0000 7ff1ceffd640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1bc06be30 0x7ff1bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.615+0000 7ff1ceffd640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1bc06be30 0x7ff1bc06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff1d015af10 tx=0x7ff1c4006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.619+0000 7ff1b7fff640 1 -- 172.21.15.26:0/167376227 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1c0045360 con 0x7ff1d00b0030 2024-04-24T23:20:58.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.800+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1d00c6530 con 0x7ff1bc06be30 2024-04-24T23:20:58.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.806+0000 7ff1b7fff640 1 -- 172.21.15.26:0/167376227 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7ff1d00c6530 con 0x7ff1bc06be30 2024-04-24T23:20:58.811 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:58.812 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1bc06be30 msgr2=0x7ff1bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1bc06be30 0x7ff1bc06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff1d015af10 tx=0x7ff1c4006c90 comp rx=0 tx=0).stop 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d00b0030 msgr2=0x7ff1d00c7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d00b0030 0x7ff1d00c7160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0000c00 tx=0x7ff1c00046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 shutdown_connections 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1bc06be30 0x7ff1bc06e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1d015e380 0x7ff1d00c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1d00b4fd0 0x7ff1d01519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 --2- 172.21.15.26:0/167376227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d00b0030 0x7ff1d00c7160 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 >> 172.21.15.26:0/167376227 conn(0x7ff1d006e2c0 msgr2=0x7ff1d006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 shutdown_connections 2024-04-24T23:20:58.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.810+0000 7ff1d5b34640 1 -- 172.21.15.26:0/167376227 wait complete. 2024-04-24T23:20:58.822 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-24T23:20:57.315573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":256132,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718588,"statfs":{"total":773094113280,"available":772831834112,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000737"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T23:20:45.641958+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:20:45.641958+0000","last_peered":"2024-04-24T23:20:45.641958+0000","last_clean":"2024-04-24T23:20:45.641958+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:20:45.641958+0000","last_undegraded":"2024-04-24T23:20:45.641958+0000","last_fullsized":"2024-04-24T23:20:45.641958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35496,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336344,"statfs":{"total":96636764160,"available":96600416256,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:20:58.826 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:20:58.868 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:20:58.909 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:20:58.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.962+0000 7f2ec0528640 1 Processor -- start 2024-04-24T23:20:58.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.962+0000 7f2ec0528640 1 -- start start 2024-04-24T23:20:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.963+0000 7f2ec0528640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2eb815e3a0 0x7f2eb80d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.963+0000 7f2ec0528640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb80b5060 con 0x7f2eb80b4310 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.963+0000 7f2ec0528640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb80d0bf0 con 0x7f2eb80d2770 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.963+0000 7f2ec0528640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f2eb80d0ed0 con 0x7f2eb815e3a0 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.964+0000 7f2ebe2c4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2eb815e3a0 0x7f2eb80d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.964+0000 7f2ebeac5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2eb80b4310 0x7f2eb80b47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46306/0 (socket says 172.21.15.26:46306) 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.964+0000 7f2ebeac5640 1 -- 172.21.15.26:0/3636025405 learned_addr learned my addr 172.21.15.26:0/3636025405 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.964+0000 7f2ebe2c4640 1 -- 172.21.15.26:0/3636025405 >> v1:172.21.15.29:6789/0 conn(0x7f2eb80d2770 legacy=0x7f2eb80d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.964+0000 7f2ebe2c4640 1 -- 172.21.15.26:0/3636025405 >> v1:172.21.15.26:6789/0 conn(0x7f2eb80b4310 legacy=0x7f2eb80b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.964+0000 7f2ebe2c4640 1 -- 172.21.15.26:0/3636025405 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb80d11b0 con 0x7f2eb815e3a0 2024-04-24T23:20:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.964+0000 7f2ebd2c2640 1 -- 172.21.15.26:0/3636025405 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407358878 0 0) 0x7f2eb80b5060 con 0x7f2eb80b4310 2024-04-24T23:20:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ebe2c4640 1 --2- 172.21.15.26:0/3636025405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 0x7f2eb80d16d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40097a0 tx=0x7f2eb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334c4e26e00d4431 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ebd2c2640 1 -- 172.21.15.26:0/3636025405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4037070 con 0x7f2eb815e3a0 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ebd2c2640 1 -- 172.21.15.26:0/3636025405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb4032440 con 0x7f2eb815e3a0 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ebd2c2640 1 -- 172.21.15.26:0/3636025405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4032810 con 0x7f2eb815e3a0 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ec0528640 1 -- 172.21.15.26:0/3636025405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 msgr2=0x7f2eb80d16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ec0528640 1 --2- 172.21.15.26:0/3636025405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 0x7f2eb80d16d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40097a0 tx=0x7f2eb402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ec0528640 1 -- 172.21.15.26:0/3636025405 shutdown_connections 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ec0528640 1 --2- 172.21.15.26:0/3636025405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 0x7f2eb80d16d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ec0528640 1 -- 172.21.15.26:0/3636025405 >> 172.21.15.26:0/3636025405 conn(0x7f2eb808dbd0 msgr2=0x7f2eb809d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ec0528640 1 -- 172.21.15.26:0/3636025405 shutdown_connections 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.965+0000 7f2ec0528640 1 -- 172.21.15.26:0/3636025405 wait complete. 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.967+0000 7f2ec0528640 1 Processor -- start 2024-04-24T23:20:58.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.967+0000 7f2ec0528640 1 -- start start 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.967+0000 7f2ec0528640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2eb80b4310 0x7f2eb80c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.967+0000 7f2ec0528640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2eb80d2770 0x7f2eb80cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.967+0000 7f2ec0528640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 0x7f2eb815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.967+0000 7f2ec0528640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb80b2ff0 con 0x7f2eb815e3a0 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.967+0000 7f2ec0528640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb80b2e70 con 0x7f2eb80d2770 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.967+0000 7f2ec0528640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb80b3170 con 0x7f2eb80b4310 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.967+0000 7f2ebeac5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 0x7f2eb815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebeac5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 0x7f2eb815c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54602/0 (socket says 172.21.15.26:54602) 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebeac5640 1 -- 172.21.15.26:0/2979709185 learned_addr learned my addr 172.21.15.26:0/2979709185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebe2c4640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2eb80b4310 0x7f2eb80c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebdac3640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2eb80d2770 0x7f2eb80cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebeac5640 1 -- 172.21.15.26:0/2979709185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2eb80b4310 msgr2=0x7f2eb80c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebeac5640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2eb80b4310 0x7f2eb80c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebeac5640 1 -- 172.21.15.26:0/2979709185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2eb80d2770 msgr2=0x7f2eb80cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebeac5640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2eb80d2770 0x7f2eb80cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebeac5640 1 -- 172.21.15.26:0/2979709185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb80cb6d0 con 0x7f2eb815e3a0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebdac3640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2eb80d2770 0x7f2eb80cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ebeac5640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 0x7f2eb815c8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c7d0 tx=0x7f2eb000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2eaeffd640 1 -- 172.21.15.26:0/2979709185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0018070 con 0x7f2eb815e3a0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2eaeffd640 1 -- 172.21.15.26:0/2979709185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb000a640 con 0x7f2eb815e3a0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb80cb9b0 con 0x7f2eb815e3a0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2eaeffd640 1 -- 172.21.15.26:0/2979709185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb000aa70 con 0x7f2eb815e3a0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.968+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb80b7ca0 con 0x7f2eb815e3a0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.969+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb80681e0 con 0x7f2eb815e3a0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.970+0000 7f2eaeffd640 1 -- 172.21.15.26:0/2979709185 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2eb001b070 con 0x7f2eb815e3a0 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.970+0000 7f2eaeffd640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2e9c06be30 0x7f2e9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.970+0000 7f2eaeffd640 1 -- 172.21.15.26:0/2979709185 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2eb008a4b0 con 0x7f2eb815e3a0 2024-04-24T23:20:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.970+0000 7f2ebe2c4640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2e9c06be30 0x7f2e9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.971+0000 7f2ebe2c4640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2e9c06be30 0x7f2e9c06e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4034000 tx=0x7f2eb402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:58.974+0000 7f2eaeffd640 1 -- 172.21.15.26:0/2979709185 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2eb0058e60 con 0x7f2eb815e3a0 2024-04-24T23:20:59.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.155+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2eb80b7e70 con 0x7f2e9c06be30 2024-04-24T23:20:59.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.161+0000 7f2eaeffd640 1 -- 172.21.15.26:0/2979709185 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7f2eb80b7e70 con 0x7f2e9c06be30 2024-04-24T23:20:59.165 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:59.166 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:20:59.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2e9c06be30 msgr2=0x7f2e9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2e9c06be30 0x7f2e9c06e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4034000 tx=0x7f2eb402e000 comp rx=0 tx=0).stop 2024-04-24T23:20:59.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 msgr2=0x7f2eb815c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 0x7f2eb815c8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c7d0 tx=0x7f2eb000cc90 comp rx=0 tx=0).stop 2024-04-24T23:20:59.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 shutdown_connections 2024-04-24T23:20:59.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2e9c06be30 0x7f2e9c06e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb815e3a0 0x7f2eb815c8d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2eb80d2770 0x7f2eb80cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 --2- 172.21.15.26:0/2979709185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2eb80b4310 0x7f2eb80c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 >> 172.21.15.26:0/2979709185 conn(0x7f2eb808dbd0 msgr2=0x7f2eb80d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:59.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 shutdown_connections 2024-04-24T23:20:59.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.164+0000 7f2ec0528640 1 -- 172.21.15.26:0/2979709185 wait complete. 2024-04-24T23:20:59.176 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-24T23:20:57.315573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":256132,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718588,"statfs":{"total":773094113280,"available":772831834112,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000737"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T23:20:45.641958+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:20:45.641958+0000","last_peered":"2024-04-24T23:20:45.641958+0000","last_clean":"2024-04-24T23:20:45.641958+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:20:45.641958+0000","last_undegraded":"2024-04-24T23:20:45.641958+0000","last_fullsized":"2024-04-24T23:20:45.641958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:54.453567+0000","last_change":"2024-04-24T23:20:54.453567+0000","last_active":"2024-04-24T23:20:54.453567+0000","last_peered":"2024-04-24T23:20:54.453567+0000","last_clean":"2024-04-24T23:20:54.453567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:54.453567+0000","last_undegraded":"2024-04-24T23:20:54.453567+0000","last_fullsized":"2024-04-24T23:20:54.453567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:20:53.449304+0000","last_change":"2024-04-24T23:20:53.449304+0000","last_active":"2024-04-24T23:20:53.449304+0000","last_peered":"2024-04-24T23:20:53.449304+0000","last_clean":"2024-04-24T23:20:53.449304+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:20:53.449304+0000","last_undegraded":"2024-04-24T23:20:53.449304+0000","last_fullsized":"2024-04-24T23:20:53.449304+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35496,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336344,"statfs":{"total":96636764160,"available":96600416256,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:20:59.180 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:20:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.315+0000 7f495688c640 1 Processor -- start 2024-04-24T23:20:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.315+0000 7f495688c640 1 -- start start 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.316+0000 7f495688c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f495015a480 0x7f49500b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.316+0000 7f495688c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a860 0x7f495006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.316+0000 7f495688c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49500b4e30 0x7f495006e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.316+0000 7f495688c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f49500b2e80 con 0x7f49500b4e30 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.316+0000 7f495688c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f49500b3120 con 0x7f495015a480 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.316+0000 7f495688c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f49500b33c0 con 0x7f495015a860 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.316+0000 7f4954e29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49500b4e30 0x7f495006e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.316+0000 7f4954e29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49500b4e30 0x7f495006e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54610/0 (socket says 172.21.15.26:54610) 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.316+0000 7f4954e29640 1 -- 172.21.15.26:0/2917038151 learned_addr learned my addr 172.21.15.26:0/2917038151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f494f7fe640 1 --2- 172.21.15.26:0/2917038151 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a860 0x7f495006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f494ffff640 1 --2- 172.21.15.26:0/2917038151 >> v2:172.21.15.29:3300/0 conn(0x7f495015a480 0x7f49500b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f4954e29640 1 -- 172.21.15.26:0/2917038151 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a860 msgr2=0x7f495006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f4954e29640 1 --2- 172.21.15.26:0/2917038151 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a860 0x7f495006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f4954e29640 1 -- 172.21.15.26:0/2917038151 >> v2:172.21.15.29:3300/0 conn(0x7f495015a480 msgr2=0x7f49500b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f4954e29640 1 --2- 172.21.15.26:0/2917038151 >> v2:172.21.15.29:3300/0 conn(0x7f495015a480 0x7f49500b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f4954e29640 1 -- 172.21.15.26:0/2917038151 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49500b36a0 con 0x7f49500b4e30 2024-04-24T23:20:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f494f7fe640 1 --2- 172.21.15.26:0/2917038151 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a860 0x7f495006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f494ffff640 1 --2- 172.21.15.26:0/2917038151 >> v2:172.21.15.29:3300/0 conn(0x7f495015a480 0x7f49500b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.317+0000 7f4954e29640 1 --2- 172.21.15.26:0/2917038151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49500b4e30 0x7f495006e120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f495006eab0 tx=0x7f494802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a31a8985bcca9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f494effd640 1 -- 172.21.15.26:0/2917038151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948036070 con 0x7f49500b4e30 2024-04-24T23:20:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f494effd640 1 -- 172.21.15.26:0/2917038151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4948031600 con 0x7f49500b4e30 2024-04-24T23:20:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f494effd640 1 -- 172.21.15.26:0/2917038151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49480319d0 con 0x7f49500b4e30 2024-04-24T23:20:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f495688c640 1 -- 172.21.15.26:0/2917038151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49500b4e30 msgr2=0x7f495006e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f495688c640 1 --2- 172.21.15.26:0/2917038151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49500b4e30 0x7f495006e120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f495006eab0 tx=0x7f494802a610 comp rx=0 tx=0).stop 2024-04-24T23:20:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f495688c640 1 -- 172.21.15.26:0/2917038151 shutdown_connections 2024-04-24T23:20:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f495688c640 1 --2- 172.21.15.26:0/2917038151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49500b4e30 0x7f495006e120 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f495688c640 1 --2- 172.21.15.26:0/2917038151 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a860 0x7f495006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f495688c640 1 --2- 172.21.15.26:0/2917038151 >> v2:172.21.15.29:3300/0 conn(0x7f495015a480 0x7f49500b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f495688c640 1 -- 172.21.15.26:0/2917038151 >> 172.21.15.26:0/2917038151 conn(0x7f495006b300 msgr2=0x7f495006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f495688c640 1 -- 172.21.15.26:0/2917038151 shutdown_connections 2024-04-24T23:20:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.318+0000 7f495688c640 1 -- 172.21.15.26:0/2917038151 wait complete. 2024-04-24T23:20:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.319+0000 7f495688c640 1 Processor -- start 2024-04-24T23:20:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f495688c640 1 -- start start 2024-04-24T23:20:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f495688c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49500b4e30 0x7f49501560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f495688c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a480 0x7f495015d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f495688c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f495015a860 0x7f4950157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f495688c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4950072180 con 0x7f495015a860 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f495688c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f49500b3c60 con 0x7f49500b4e30 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f495688c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4950072300 con 0x7f495015a480 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f4954e29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f495015a860 0x7f4950157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f4954e29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f495015a860 0x7f4950157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54614/0 (socket says 172.21.15.26:54614) 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f4954e29640 1 -- 172.21.15.26:0/779717984 learned_addr learned my addr 172.21.15.26:0/779717984 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f494f7fe640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a480 0x7f495015d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.320+0000 7f494ffff640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49500b4e30 0x7f49501560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f4954e29640 1 -- 172.21.15.26:0/779717984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a480 msgr2=0x7f495015d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f4954e29640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a480 0x7f495015d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f4954e29640 1 -- 172.21.15.26:0/779717984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49500b4e30 msgr2=0x7f49501560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f4954e29640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49500b4e30 0x7f49501560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f4954e29640 1 -- 172.21.15.26:0/779717984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49500ad910 con 0x7f495015a860 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f4954e29640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f495015a860 0x7f4950157250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4948009f40 tx=0x7f4948004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f494cff9640 1 -- 172.21.15.26:0/779717984 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948036050 con 0x7f495015a860 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f494cff9640 1 -- 172.21.15.26:0/779717984 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4948042070 con 0x7f495015a860 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49501556e0 con 0x7f495015a860 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f494cff9640 1 -- 172.21.15.26:0/779717984 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948009510 con 0x7f495015a860 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.321+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49500ade90 con 0x7f495015a860 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.322+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49500694c0 con 0x7f495015a860 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.322+0000 7f494cff9640 1 -- 172.21.15.26:0/779717984 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4948007bd0 con 0x7f495015a860 2024-04-24T23:20:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.323+0000 7f494cff9640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f493006be30 0x7f493006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.323+0000 7f494cff9640 1 -- 172.21.15.26:0/779717984 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f49480abe20 con 0x7f495015a860 2024-04-24T23:20:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.323+0000 7f494ffff640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f493006be30 0x7f493006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.323+0000 7f494ffff640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f493006be30 0x7f493006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4940006b10 tx=0x7f4940008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.327+0000 7f494cff9640 1 -- 172.21.15.26:0/779717984 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4948045360 con 0x7f495015a860 2024-04-24T23:20:59.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.506+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49500b44a0 con 0x7f493006be30 2024-04-24T23:20:59.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.512+0000 7f494cff9640 1 -- 172.21.15.26:0/779717984 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f49500b44a0 con 0x7f493006be30 2024-04-24T23:20:59.517 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f493006be30 msgr2=0x7f493006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f493006be30 0x7f493006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4940006b10 tx=0x7f4940008040 comp rx=0 tx=0).stop 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f495015a860 msgr2=0x7f4950157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f495015a860 0x7f4950157250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4948009f40 tx=0x7f4948004a40 comp rx=0 tx=0).stop 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 shutdown_connections 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f493006be30 0x7f493006e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f495015a860 0x7f4950157250 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f495015a480 0x7f495015d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 --2- 172.21.15.26:0/779717984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49500b4e30 0x7f49501560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.515+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 >> 172.21.15.26:0/779717984 conn(0x7f495006b300 msgr2=0x7f495014a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.516+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 shutdown_connections 2024-04-24T23:20:59.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.516+0000 7f495688c640 1 -- 172.21.15.26:0/779717984 wait complete. 2024-04-24T23:20:59.528 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-24T23:20:59.315992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256368,"kb_used_data":2764,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718352,"statfs":{"total":773094113280,"available":772831592448,"internally_reserved":0,"allocated":2830336,"data_stored":1721528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.862540"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725536+0000","last_change":"2024-04-24T23:20:55.487607+0000","last_active":"2024-04-24T23:20:56.725536+0000","last_peered":"2024-04-24T23:20:56.725536+0000","last_clean":"2024-04-24T23:20:56.725536+0000","last_became_active":"2024-04-24T23:20:55.487320+0000","last_became_peered":"2024-04-24T23:20:55.487320+0000","last_unstale":"2024-04-24T23:20:56.725536+0000","last_undegraded":"2024-04-24T23:20:56.725536+0000","last_fullsized":"2024-04-24T23:20:56.725536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:27.675103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478125+0000","last_change":"2024-04-24T23:20:55.480991+0000","last_active":"2024-04-24T23:20:56.478125+0000","last_peered":"2024-04-24T23:20:56.478125+0000","last_clean":"2024-04-24T23:20:56.478125+0000","last_became_active":"2024-04-24T23:20:55.480654+0000","last_became_peered":"2024-04-24T23:20:55.480654+0000","last_unstale":"2024-04-24T23:20:56.478125+0000","last_undegraded":"2024-04-24T23:20:56.478125+0000","last_fullsized":"2024-04-24T23:20:56.478125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724519+0000","last_change":"2024-04-24T23:20:55.479402+0000","last_active":"2024-04-24T23:20:56.724519+0000","last_peered":"2024-04-24T23:20:56.724519+0000","last_clean":"2024-04-24T23:20:56.724519+0000","last_became_active":"2024-04-24T23:20:55.479013+0000","last_became_peered":"2024-04-24T23:20:55.479013+0000","last_unstale":"2024-04-24T23:20:56.724519+0000","last_undegraded":"2024-04-24T23:20:56.724519+0000","last_fullsized":"2024-04-24T23:20:56.724519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475989+0000","last_change":"2024-04-24T23:20:55.481507+0000","last_active":"2024-04-24T23:20:56.475989+0000","last_peered":"2024-04-24T23:20:56.475989+0000","last_clean":"2024-04-24T23:20:56.475989+0000","last_became_active":"2024-04-24T23:20:55.481144+0000","last_became_peered":"2024-04-24T23:20:55.481144+0000","last_unstale":"2024-04-24T23:20:56.475989+0000","last_undegraded":"2024-04-24T23:20:56.475989+0000","last_fullsized":"2024-04-24T23:20:56.475989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725067+0000","last_change":"2024-04-24T23:20:55.482367+0000","last_active":"2024-04-24T23:20:56.725067+0000","last_peered":"2024-04-24T23:20:56.725067+0000","last_clean":"2024-04-24T23:20:56.725067+0000","last_became_active":"2024-04-24T23:20:55.482047+0000","last_became_peered":"2024-04-24T23:20:55.482047+0000","last_unstale":"2024-04-24T23:20:56.725067+0000","last_undegraded":"2024-04-24T23:20:56.725067+0000","last_fullsized":"2024-04-24T23:20:56.725067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475443+0000","last_change":"2024-04-24T23:20:55.482474+0000","last_active":"2024-04-24T23:20:56.475443+0000","last_peered":"2024-04-24T23:20:56.475443+0000","last_clean":"2024-04-24T23:20:56.475443+0000","last_became_active":"2024-04-24T23:20:55.482157+0000","last_became_peered":"2024-04-24T23:20:55.482157+0000","last_unstale":"2024-04-24T23:20:56.475443+0000","last_undegraded":"2024-04-24T23:20:56.475443+0000","last_fullsized":"2024-04-24T23:20:56.475443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724980+0000","last_change":"2024-04-24T23:20:55.487389+0000","last_active":"2024-04-24T23:20:56.724980+0000","last_peered":"2024-04-24T23:20:56.724980+0000","last_clean":"2024-04-24T23:20:56.724980+0000","last_became_active":"2024-04-24T23:20:55.486724+0000","last_became_peered":"2024-04-24T23:20:55.486724+0000","last_unstale":"2024-04-24T23:20:56.724980+0000","last_undegraded":"2024-04-24T23:20:56.724980+0000","last_fullsized":"2024-04-24T23:20:56.724980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724916+0000","last_change":"2024-04-24T23:20:55.486339+0000","last_active":"2024-04-24T23:20:56.724916+0000","last_peered":"2024-04-24T23:20:56.724916+0000","last_clean":"2024-04-24T23:20:56.724916+0000","last_became_active":"2024-04-24T23:20:55.486072+0000","last_became_peered":"2024-04-24T23:20:55.486072+0000","last_unstale":"2024-04-24T23:20:56.724916+0000","last_undegraded":"2024-04-24T23:20:56.724916+0000","last_fullsized":"2024-04-24T23:20:56.724916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.491641+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.491349+0000","last_became_peered":"2024-04-24T23:20:55.491349+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:58.298429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480758+0000","last_change":"2024-04-24T23:20:55.479221+0000","last_active":"2024-04-24T23:20:56.480758+0000","last_peered":"2024-04-24T23:20:56.480758+0000","last_clean":"2024-04-24T23:20:56.480758+0000","last_became_active":"2024-04-24T23:20:55.478795+0000","last_became_peered":"2024-04-24T23:20:55.478795+0000","last_unstale":"2024-04-24T23:20:56.480758+0000","last_undegraded":"2024-04-24T23:20:56.480758+0000","last_fullsized":"2024-04-24T23:20:56.480758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480763+0000","last_change":"2024-04-24T23:20:55.479615+0000","last_active":"2024-04-24T23:20:56.480763+0000","last_peered":"2024-04-24T23:20:56.480763+0000","last_clean":"2024-04-24T23:20:56.480763+0000","last_became_active":"2024-04-24T23:20:55.479317+0000","last_became_peered":"2024-04-24T23:20:55.479317+0000","last_unstale":"2024-04-24T23:20:56.480763+0000","last_undegraded":"2024-04-24T23:20:56.480763+0000","last_fullsized":"2024-04-24T23:20:56.480763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475028+0000","last_change":"2024-04-24T23:20:55.481915+0000","last_active":"2024-04-24T23:20:56.475028+0000","last_peered":"2024-04-24T23:20:56.475028+0000","last_clean":"2024-04-24T23:20:56.475028+0000","last_became_active":"2024-04-24T23:20:55.481596+0000","last_became_peered":"2024-04-24T23:20:55.481596+0000","last_unstale":"2024-04-24T23:20:56.475028+0000","last_undegraded":"2024-04-24T23:20:56.475028+0000","last_fullsized":"2024-04-24T23:20:56.475028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:20.844340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725493+0000","last_change":"2024-04-24T23:20:55.476416+0000","last_active":"2024-04-24T23:20:56.725493+0000","last_peered":"2024-04-24T23:20:56.725493+0000","last_clean":"2024-04-24T23:20:56.725493+0000","last_became_active":"2024-04-24T23:20:55.476068+0000","last_became_peered":"2024-04-24T23:20:55.476068+0000","last_unstale":"2024-04-24T23:20:56.725493+0000","last_undegraded":"2024-04-24T23:20:56.725493+0000","last_fullsized":"2024-04-24T23:20:56.725493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724195+0000","last_change":"2024-04-24T23:20:55.479125+0000","last_active":"2024-04-24T23:20:56.724195+0000","last_peered":"2024-04-24T23:20:56.724195+0000","last_clean":"2024-04-24T23:20:56.724195+0000","last_became_active":"2024-04-24T23:20:55.478473+0000","last_became_peered":"2024-04-24T23:20:55.478473+0000","last_unstale":"2024-04-24T23:20:56.724195+0000","last_undegraded":"2024-04-24T23:20:56.724195+0000","last_fullsized":"2024-04-24T23:20:56.724195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:19.475113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477868+0000","last_change":"2024-04-24T23:20:55.474883+0000","last_active":"2024-04-24T23:20:56.477868+0000","last_peered":"2024-04-24T23:20:56.477868+0000","last_clean":"2024-04-24T23:20:56.477868+0000","last_became_active":"2024-04-24T23:20:55.474459+0000","last_became_peered":"2024-04-24T23:20:55.474459+0000","last_unstale":"2024-04-24T23:20:56.477868+0000","last_undegraded":"2024-04-24T23:20:56.477868+0000","last_fullsized":"2024-04-24T23:20:56.477868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475040+0000","last_change":"2024-04-24T23:20:55.483511+0000","last_active":"2024-04-24T23:20:56.475040+0000","last_peered":"2024-04-24T23:20:56.475040+0000","last_clean":"2024-04-24T23:20:56.475040+0000","last_became_active":"2024-04-24T23:20:55.483171+0000","last_became_peered":"2024-04-24T23:20:55.483171+0000","last_unstale":"2024-04-24T23:20:56.475040+0000","last_undegraded":"2024-04-24T23:20:56.475040+0000","last_fullsized":"2024-04-24T23:20:56.475040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475537+0000","last_change":"2024-04-24T23:20:55.489069+0000","last_active":"2024-04-24T23:20:56.475537+0000","last_peered":"2024-04-24T23:20:56.475537+0000","last_clean":"2024-04-24T23:20:56.475537+0000","last_became_active":"2024-04-24T23:20:55.488796+0000","last_became_peered":"2024-04-24T23:20:55.488796+0000","last_unstale":"2024-04-24T23:20:56.475537+0000","last_undegraded":"2024-04-24T23:20:56.475537+0000","last_fullsized":"2024-04-24T23:20:56.475537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724755+0000","last_change":"2024-04-24T23:20:55.480920+0000","last_active":"2024-04-24T23:20:56.724755+0000","last_peered":"2024-04-24T23:20:56.724755+0000","last_clean":"2024-04-24T23:20:56.724755+0000","last_became_active":"2024-04-24T23:20:55.480581+0000","last_became_peered":"2024-04-24T23:20:55.480581+0000","last_unstale":"2024-04-24T23:20:56.724755+0000","last_undegraded":"2024-04-24T23:20:56.724755+0000","last_fullsized":"2024-04-24T23:20:56.724755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725520+0000","last_change":"2024-04-24T23:20:55.488901+0000","last_active":"2024-04-24T23:20:56.725520+0000","last_peered":"2024-04-24T23:20:56.725520+0000","last_clean":"2024-04-24T23:20:56.725520+0000","last_became_active":"2024-04-24T23:20:55.488647+0000","last_became_peered":"2024-04-24T23:20:55.488647+0000","last_unstale":"2024-04-24T23:20:56.725520+0000","last_undegraded":"2024-04-24T23:20:56.725520+0000","last_fullsized":"2024-04-24T23:20:56.725520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480674+0000","last_change":"2024-04-24T23:20:55.476826+0000","last_active":"2024-04-24T23:20:56.480674+0000","last_peered":"2024-04-24T23:20:56.480674+0000","last_clean":"2024-04-24T23:20:56.480674+0000","last_became_active":"2024-04-24T23:20:55.476481+0000","last_became_peered":"2024-04-24T23:20:55.476481+0000","last_unstale":"2024-04-24T23:20:56.480674+0000","last_undegraded":"2024-04-24T23:20:56.480674+0000","last_fullsized":"2024-04-24T23:20:56.480674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724349+0000","last_change":"2024-04-24T23:20:55.491072+0000","last_active":"2024-04-24T23:20:56.724349+0000","last_peered":"2024-04-24T23:20:56.724349+0000","last_clean":"2024-04-24T23:20:56.724349+0000","last_became_active":"2024-04-24T23:20:55.490763+0000","last_became_peered":"2024-04-24T23:20:55.490763+0000","last_unstale":"2024-04-24T23:20:56.724349+0000","last_undegraded":"2024-04-24T23:20:56.724349+0000","last_fullsized":"2024-04-24T23:20:56.724349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475836+0000","last_change":"2024-04-24T23:20:55.487623+0000","last_active":"2024-04-24T23:20:56.475836+0000","last_peered":"2024-04-24T23:20:56.475836+0000","last_clean":"2024-04-24T23:20:56.475836+0000","last_became_active":"2024-04-24T23:20:55.487355+0000","last_became_peered":"2024-04-24T23:20:55.487355+0000","last_unstale":"2024-04-24T23:20:56.475836+0000","last_undegraded":"2024-04-24T23:20:56.475836+0000","last_fullsized":"2024-04-24T23:20:56.475836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725470+0000","last_change":"2024-04-24T23:20:55.476152+0000","last_active":"2024-04-24T23:20:56.725470+0000","last_peered":"2024-04-24T23:20:56.725470+0000","last_clean":"2024-04-24T23:20:56.725470+0000","last_became_active":"2024-04-24T23:20:55.475857+0000","last_became_peered":"2024-04-24T23:20:55.475857+0000","last_unstale":"2024-04-24T23:20:56.725470+0000","last_undegraded":"2024-04-24T23:20:56.725470+0000","last_fullsized":"2024-04-24T23:20:56.725470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475501+0000","last_change":"2024-04-24T23:20:55.484268+0000","last_active":"2024-04-24T23:20:56.475501+0000","last_peered":"2024-04-24T23:20:56.475501+0000","last_clean":"2024-04-24T23:20:56.475501+0000","last_became_active":"2024-04-24T23:20:55.483972+0000","last_became_peered":"2024-04-24T23:20:55.483972+0000","last_unstale":"2024-04-24T23:20:56.475501+0000","last_undegraded":"2024-04-24T23:20:56.475501+0000","last_fullsized":"2024-04-24T23:20:56.475501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725722+0000","last_change":"2024-04-24T23:20:55.480092+0000","last_active":"2024-04-24T23:20:56.725722+0000","last_peered":"2024-04-24T23:20:56.725722+0000","last_clean":"2024-04-24T23:20:56.725722+0000","last_became_active":"2024-04-24T23:20:55.479783+0000","last_became_peered":"2024-04-24T23:20:55.479783+0000","last_unstale":"2024-04-24T23:20:56.725722+0000","last_undegraded":"2024-04-24T23:20:56.725722+0000","last_fullsized":"2024-04-24T23:20:56.725722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480830+0000","last_change":"2024-04-24T23:20:55.488050+0000","last_active":"2024-04-24T23:20:56.480830+0000","last_peered":"2024-04-24T23:20:56.480830+0000","last_clean":"2024-04-24T23:20:56.480830+0000","last_became_active":"2024-04-24T23:20:55.487762+0000","last_became_peered":"2024-04-24T23:20:55.487762+0000","last_unstale":"2024-04-24T23:20:56.480830+0000","last_undegraded":"2024-04-24T23:20:56.480830+0000","last_fullsized":"2024-04-24T23:20:56.480830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478091+0000","last_change":"2024-04-24T23:20:55.478530+0000","last_active":"2024-04-24T23:20:56.478091+0000","last_peered":"2024-04-24T23:20:56.478091+0000","last_clean":"2024-04-24T23:20:56.478091+0000","last_became_active":"2024-04-24T23:20:55.478233+0000","last_became_peered":"2024-04-24T23:20:55.478233+0000","last_unstale":"2024-04-24T23:20:56.478091+0000","last_undegraded":"2024-04-24T23:20:56.478091+0000","last_fullsized":"2024-04-24T23:20:56.478091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725374+0000","last_change":"2024-04-24T23:20:55.480077+0000","last_active":"2024-04-24T23:20:56.725374+0000","last_peered":"2024-04-24T23:20:56.725374+0000","last_clean":"2024-04-24T23:20:56.725374+0000","last_became_active":"2024-04-24T23:20:55.479769+0000","last_became_peered":"2024-04-24T23:20:55.479769+0000","last_unstale":"2024-04-24T23:20:56.725374+0000","last_undegraded":"2024-04-24T23:20:56.725374+0000","last_fullsized":"2024-04-24T23:20:56.725374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724513+0000","last_change":"2024-04-24T23:20:55.481475+0000","last_active":"2024-04-24T23:20:56.724513+0000","last_peered":"2024-04-24T23:20:56.724513+0000","last_clean":"2024-04-24T23:20:56.724513+0000","last_became_active":"2024-04-24T23:20:55.480091+0000","last_became_peered":"2024-04-24T23:20:55.480091+0000","last_unstale":"2024-04-24T23:20:56.724513+0000","last_undegraded":"2024-04-24T23:20:56.724513+0000","last_fullsized":"2024-04-24T23:20:56.724513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476229+0000","last_change":"2024-04-24T23:20:55.485092+0000","last_active":"2024-04-24T23:20:56.476229+0000","last_peered":"2024-04-24T23:20:56.476229+0000","last_clean":"2024-04-24T23:20:56.476229+0000","last_became_active":"2024-04-24T23:20:55.484409+0000","last_became_peered":"2024-04-24T23:20:55.484409+0000","last_unstale":"2024-04-24T23:20:56.476229+0000","last_undegraded":"2024-04-24T23:20:56.476229+0000","last_fullsized":"2024-04-24T23:20:56.476229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.469291+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.468961+0000","last_became_peered":"2024-04-24T23:20:55.468961+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480450+0000","last_change":"2024-04-24T23:20:55.484826+0000","last_active":"2024-04-24T23:20:56.480450+0000","last_peered":"2024-04-24T23:20:56.480450+0000","last_clean":"2024-04-24T23:20:56.480450+0000","last_became_active":"2024-04-24T23:20:55.484500+0000","last_became_peered":"2024-04-24T23:20:55.484500+0000","last_unstale":"2024-04-24T23:20:56.480450+0000","last_undegraded":"2024-04-24T23:20:56.480450+0000","last_fullsized":"2024-04-24T23:20:56.480450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475480+0000","last_change":"2024-04-24T23:20:55.476006+0000","last_active":"2024-04-24T23:20:56.475480+0000","last_peered":"2024-04-24T23:20:56.475480+0000","last_clean":"2024-04-24T23:20:56.475480+0000","last_became_active":"2024-04-24T23:20:55.475090+0000","last_became_peered":"2024-04-24T23:20:55.475090+0000","last_unstale":"2024-04-24T23:20:56.475480+0000","last_undegraded":"2024-04-24T23:20:56.475480+0000","last_fullsized":"2024-04-24T23:20:56.475480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725560+0000","last_change":"2024-04-24T23:20:54.481863+0000","last_active":"2024-04-24T23:20:56.725560+0000","last_peered":"2024-04-24T23:20:56.725560+0000","last_clean":"2024-04-24T23:20:56.725560+0000","last_became_active":"2024-04-24T23:20:54.481498+0000","last_became_peered":"2024-04-24T23:20:54.481498+0000","last_unstale":"2024-04-24T23:20:56.725560+0000","last_undegraded":"2024-04-24T23:20:56.725560+0000","last_fullsized":"2024-04-24T23:20:56.725560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724565+0000","last_change":"2024-04-24T23:20:55.478651+0000","last_active":"2024-04-24T23:20:56.724565+0000","last_peered":"2024-04-24T23:20:56.724565+0000","last_clean":"2024-04-24T23:20:56.724565+0000","last_became_active":"2024-04-24T23:20:55.477783+0000","last_became_peered":"2024-04-24T23:20:55.477783+0000","last_unstale":"2024-04-24T23:20:56.724565+0000","last_undegraded":"2024-04-24T23:20:56.724565+0000","last_fullsized":"2024-04-24T23:20:56.724565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480400+0000","last_change":"2024-04-24T23:20:54.477560+0000","last_active":"2024-04-24T23:20:56.480400+0000","last_peered":"2024-04-24T23:20:56.480400+0000","last_clean":"2024-04-24T23:20:56.480400+0000","last_became_active":"2024-04-24T23:20:54.477261+0000","last_became_peered":"2024-04-24T23:20:54.477261+0000","last_unstale":"2024-04-24T23:20:56.480400+0000","last_undegraded":"2024-04-24T23:20:56.480400+0000","last_fullsized":"2024-04-24T23:20:56.480400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725154+0000","last_change":"2024-04-24T23:20:54.479540+0000","last_active":"2024-04-24T23:20:56.725154+0000","last_peered":"2024-04-24T23:20:56.725154+0000","last_clean":"2024-04-24T23:20:56.725154+0000","last_became_active":"2024-04-24T23:20:54.479206+0000","last_became_peered":"2024-04-24T23:20:54.479206+0000","last_unstale":"2024-04-24T23:20:56.725154+0000","last_undegraded":"2024-04-24T23:20:56.725154+0000","last_fullsized":"2024-04-24T23:20:56.725154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725091+0000","last_change":"2024-04-24T23:20:55.484927+0000","last_active":"2024-04-24T23:20:56.725091+0000","last_peered":"2024-04-24T23:20:56.725091+0000","last_clean":"2024-04-24T23:20:56.725091+0000","last_became_active":"2024-04-24T23:20:55.484611+0000","last_became_peered":"2024-04-24T23:20:55.484611+0000","last_unstale":"2024-04-24T23:20:56.725091+0000","last_undegraded":"2024-04-24T23:20:56.725091+0000","last_fullsized":"2024-04-24T23:20:56.725091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475910+0000","last_change":"2024-04-24T23:20:54.471358+0000","last_active":"2024-04-24T23:20:56.475910+0000","last_peered":"2024-04-24T23:20:56.475910+0000","last_clean":"2024-04-24T23:20:56.475910+0000","last_became_active":"2024-04-24T23:20:54.470800+0000","last_became_peered":"2024-04-24T23:20:54.470800+0000","last_unstale":"2024-04-24T23:20:56.475910+0000","last_undegraded":"2024-04-24T23:20:56.475910+0000","last_fullsized":"2024-04-24T23:20:56.475910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.477535+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.477215+0000","last_became_peered":"2024-04-24T23:20:55.477215+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480761+0000","last_change":"2024-04-24T23:20:54.479450+0000","last_active":"2024-04-24T23:20:56.480761+0000","last_peered":"2024-04-24T23:20:56.480761+0000","last_clean":"2024-04-24T23:20:56.480761+0000","last_became_active":"2024-04-24T23:20:54.479174+0000","last_became_peered":"2024-04-24T23:20:54.479174+0000","last_unstale":"2024-04-24T23:20:56.480761+0000","last_undegraded":"2024-04-24T23:20:56.480761+0000","last_fullsized":"2024-04-24T23:20:56.480761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476156+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.476156+0000","last_peered":"2024-04-24T23:20:56.476156+0000","last_clean":"2024-04-24T23:20:56.476156+0000","last_became_active":"2024-04-24T23:20:55.488014+0000","last_became_peered":"2024-04-24T23:20:55.488014+0000","last_unstale":"2024-04-24T23:20:56.476156+0000","last_undegraded":"2024-04-24T23:20:56.476156+0000","last_fullsized":"2024-04-24T23:20:56.476156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476129+0000","last_change":"2024-04-24T23:20:54.480387+0000","last_active":"2024-04-24T23:20:56.476129+0000","last_peered":"2024-04-24T23:20:56.476129+0000","last_clean":"2024-04-24T23:20:56.476129+0000","last_became_active":"2024-04-24T23:20:54.480037+0000","last_became_peered":"2024-04-24T23:20:54.480037+0000","last_unstale":"2024-04-24T23:20:56.476129+0000","last_undegraded":"2024-04-24T23:20:56.476129+0000","last_fullsized":"2024-04-24T23:20:56.476129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724464+0000","last_change":"2024-04-24T23:20:55.481206+0000","last_active":"2024-04-24T23:20:56.724464+0000","last_peered":"2024-04-24T23:20:56.724464+0000","last_clean":"2024-04-24T23:20:56.724464+0000","last_became_active":"2024-04-24T23:20:55.478018+0000","last_became_peered":"2024-04-24T23:20:55.478018+0000","last_unstale":"2024-04-24T23:20:56.724464+0000","last_undegraded":"2024-04-24T23:20:56.724464+0000","last_fullsized":"2024-04-24T23:20:56.724464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478112+0000","last_change":"2024-04-24T23:20:54.469557+0000","last_active":"2024-04-24T23:20:56.478112+0000","last_peered":"2024-04-24T23:20:56.478112+0000","last_clean":"2024-04-24T23:20:56.478112+0000","last_became_active":"2024-04-24T23:20:54.469251+0000","last_became_peered":"2024-04-24T23:20:54.469251+0000","last_unstale":"2024-04-24T23:20:56.478112+0000","last_undegraded":"2024-04-24T23:20:56.478112+0000","last_fullsized":"2024-04-24T23:20:56.478112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475184+0000","last_change":"2024-04-24T23:20:55.491797+0000","last_active":"2024-04-24T23:20:56.475184+0000","last_peered":"2024-04-24T23:20:56.475184+0000","last_clean":"2024-04-24T23:20:56.475184+0000","last_became_active":"2024-04-24T23:20:55.491453+0000","last_became_peered":"2024-04-24T23:20:55.491453+0000","last_unstale":"2024-04-24T23:20:56.475184+0000","last_undegraded":"2024-04-24T23:20:56.475184+0000","last_fullsized":"2024-04-24T23:20:56.475184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:05.444694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480475+0000","last_change":"2024-04-24T23:20:54.482472+0000","last_active":"2024-04-24T23:20:56.480475+0000","last_peered":"2024-04-24T23:20:56.480475+0000","last_clean":"2024-04-24T23:20:56.480475+0000","last_became_active":"2024-04-24T23:20:54.481351+0000","last_became_peered":"2024-04-24T23:20:54.481351+0000","last_unstale":"2024-04-24T23:20:56.480475+0000","last_undegraded":"2024-04-24T23:20:56.480475+0000","last_fullsized":"2024-04-24T23:20:56.480475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725210+0000","last_change":"2024-04-24T23:20:55.483153+0000","last_active":"2024-04-24T23:20:56.725210+0000","last_peered":"2024-04-24T23:20:56.725210+0000","last_clean":"2024-04-24T23:20:56.725210+0000","last_became_active":"2024-04-24T23:20:55.482859+0000","last_became_peered":"2024-04-24T23:20:55.482859+0000","last_unstale":"2024-04-24T23:20:56.725210+0000","last_undegraded":"2024-04-24T23:20:56.725210+0000","last_fullsized":"2024-04-24T23:20:56.725210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724624+0000","last_change":"2024-04-24T23:20:54.478089+0000","last_active":"2024-04-24T23:20:56.724624+0000","last_peered":"2024-04-24T23:20:56.724624+0000","last_clean":"2024-04-24T23:20:56.724624+0000","last_became_active":"2024-04-24T23:20:54.477302+0000","last_became_peered":"2024-04-24T23:20:54.477302+0000","last_unstale":"2024-04-24T23:20:56.724624+0000","last_undegraded":"2024-04-24T23:20:56.724624+0000","last_fullsized":"2024-04-24T23:20:56.724624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724289+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.724289+0000","last_peered":"2024-04-24T23:20:56.724289+0000","last_clean":"2024-04-24T23:20:56.724289+0000","last_became_active":"2024-04-24T23:20:55.488016+0000","last_became_peered":"2024-04-24T23:20:55.488016+0000","last_unstale":"2024-04-24T23:20:56.724289+0000","last_undegraded":"2024-04-24T23:20:56.724289+0000","last_fullsized":"2024-04-24T23:20:56.724289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:48.795181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476045+0000","last_change":"2024-04-24T23:20:54.473922+0000","last_active":"2024-04-24T23:20:56.476045+0000","last_peered":"2024-04-24T23:20:56.476045+0000","last_clean":"2024-04-24T23:20:56.476045+0000","last_became_active":"2024-04-24T23:20:54.473601+0000","last_became_peered":"2024-04-24T23:20:54.473601+0000","last_unstale":"2024-04-24T23:20:56.476045+0000","last_undegraded":"2024-04-24T23:20:56.476045+0000","last_fullsized":"2024-04-24T23:20:56.476045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725128+0000","last_change":"2024-04-24T23:20:55.488225+0000","last_active":"2024-04-24T23:20:56.725128+0000","last_peered":"2024-04-24T23:20:56.725128+0000","last_clean":"2024-04-24T23:20:56.725128+0000","last_became_active":"2024-04-24T23:20:55.485663+0000","last_became_peered":"2024-04-24T23:20:55.485663+0000","last_unstale":"2024-04-24T23:20:56.725128+0000","last_undegraded":"2024-04-24T23:20:56.725128+0000","last_fullsized":"2024-04-24T23:20:56.725128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:54.478237+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:54.477943+0000","last_became_peered":"2024-04-24T23:20:54.477943+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:55.484884+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:55.484578+0000","last_became_peered":"2024-04-24T23:20:55.484578+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724810+0000","last_change":"2024-04-24T23:20:54.473033+0000","last_active":"2024-04-24T23:20:56.724810+0000","last_peered":"2024-04-24T23:20:56.724810+0000","last_clean":"2024-04-24T23:20:56.724810+0000","last_became_active":"2024-04-24T23:20:54.472748+0000","last_became_peered":"2024-04-24T23:20:54.472748+0000","last_unstale":"2024-04-24T23:20:56.724810+0000","last_undegraded":"2024-04-24T23:20:56.724810+0000","last_fullsized":"2024-04-24T23:20:56.724810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724977+0000","last_change":"2024-04-24T23:20:55.485153+0000","last_active":"2024-04-24T23:20:56.724977+0000","last_peered":"2024-04-24T23:20:56.724977+0000","last_clean":"2024-04-24T23:20:56.724977+0000","last_became_active":"2024-04-24T23:20:55.484876+0000","last_became_peered":"2024-04-24T23:20:55.484876+0000","last_unstale":"2024-04-24T23:20:56.724977+0000","last_undegraded":"2024-04-24T23:20:56.724977+0000","last_fullsized":"2024-04-24T23:20:56.724977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474721+0000","last_change":"2024-04-24T23:20:54.476273+0000","last_active":"2024-04-24T23:20:56.474721+0000","last_peered":"2024-04-24T23:20:56.474721+0000","last_clean":"2024-04-24T23:20:56.474721+0000","last_became_active":"2024-04-24T23:20:54.475993+0000","last_became_peered":"2024-04-24T23:20:54.475993+0000","last_unstale":"2024-04-24T23:20:56.474721+0000","last_undegraded":"2024-04-24T23:20:56.474721+0000","last_fullsized":"2024-04-24T23:20:56.474721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725412+0000","last_change":"2024-04-24T23:20:55.487659+0000","last_active":"2024-04-24T23:20:56.725412+0000","last_peered":"2024-04-24T23:20:56.725412+0000","last_clean":"2024-04-24T23:20:56.725412+0000","last_became_active":"2024-04-24T23:20:55.486557+0000","last_became_peered":"2024-04-24T23:20:55.486557+0000","last_unstale":"2024-04-24T23:20:56.725412+0000","last_undegraded":"2024-04-24T23:20:56.725412+0000","last_fullsized":"2024-04-24T23:20:56.725412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724872+0000","last_change":"2024-04-24T23:20:54.477334+0000","last_active":"2024-04-24T23:20:56.724872+0000","last_peered":"2024-04-24T23:20:56.724872+0000","last_clean":"2024-04-24T23:20:56.724872+0000","last_became_active":"2024-04-24T23:20:54.477058+0000","last_became_peered":"2024-04-24T23:20:54.477058+0000","last_unstale":"2024-04-24T23:20:56.724872+0000","last_undegraded":"2024-04-24T23:20:56.724872+0000","last_fullsized":"2024-04-24T23:20:56.724872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725667+0000","last_change":"2024-04-24T23:20:55.492754+0000","last_active":"2024-04-24T23:20:56.725667+0000","last_peered":"2024-04-24T23:20:56.725667+0000","last_clean":"2024-04-24T23:20:56.725667+0000","last_became_active":"2024-04-24T23:20:55.492474+0000","last_became_peered":"2024-04-24T23:20:55.492474+0000","last_unstale":"2024-04-24T23:20:56.725667+0000","last_undegraded":"2024-04-24T23:20:56.725667+0000","last_fullsized":"2024-04-24T23:20:56.725667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:54.766130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478192+0000","last_change":"2024-04-24T23:20:54.471091+0000","last_active":"2024-04-24T23:20:56.478192+0000","last_peered":"2024-04-24T23:20:56.478192+0000","last_clean":"2024-04-24T23:20:56.478192+0000","last_became_active":"2024-04-24T23:20:54.470811+0000","last_became_peered":"2024-04-24T23:20:54.470811+0000","last_unstale":"2024-04-24T23:20:56.478192+0000","last_undegraded":"2024-04-24T23:20:56.478192+0000","last_fullsized":"2024-04-24T23:20:56.478192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476216+0000","last_change":"2024-04-24T23:20:55.480640+0000","last_active":"2024-04-24T23:20:56.476216+0000","last_peered":"2024-04-24T23:20:56.476216+0000","last_clean":"2024-04-24T23:20:56.476216+0000","last_became_active":"2024-04-24T23:20:55.480327+0000","last_became_peered":"2024-04-24T23:20:55.480327+0000","last_unstale":"2024-04-24T23:20:56.476216+0000","last_undegraded":"2024-04-24T23:20:56.476216+0000","last_fullsized":"2024-04-24T23:20:56.476216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725414+0000","last_change":"2024-04-24T23:20:54.476392+0000","last_active":"2024-04-24T23:20:56.725414+0000","last_peered":"2024-04-24T23:20:56.725414+0000","last_clean":"2024-04-24T23:20:56.725414+0000","last_became_active":"2024-04-24T23:20:54.476051+0000","last_became_peered":"2024-04-24T23:20:54.476051+0000","last_unstale":"2024-04-24T23:20:56.725414+0000","last_undegraded":"2024-04-24T23:20:56.725414+0000","last_fullsized":"2024-04-24T23:20:56.725414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725389+0000","last_change":"2024-04-24T23:20:55.483909+0000","last_active":"2024-04-24T23:20:56.725389+0000","last_peered":"2024-04-24T23:20:56.725389+0000","last_clean":"2024-04-24T23:20:56.725389+0000","last_became_active":"2024-04-24T23:20:55.483625+0000","last_became_peered":"2024-04-24T23:20:55.483625+0000","last_unstale":"2024-04-24T23:20:56.725389+0000","last_undegraded":"2024-04-24T23:20:56.725389+0000","last_fullsized":"2024-04-24T23:20:56.725389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724832+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:20:56.724832+0000","last_peered":"2024-04-24T23:20:56.724832+0000","last_clean":"2024-04-24T23:20:56.724832+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:20:56.724832+0000","last_undegraded":"2024-04-24T23:20:56.724832+0000","last_fullsized":"2024-04-24T23:20:56.724832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474912+0000","last_change":"2024-04-24T23:20:54.470392+0000","last_active":"2024-04-24T23:20:56.474912+0000","last_peered":"2024-04-24T23:20:56.474912+0000","last_clean":"2024-04-24T23:20:56.474912+0000","last_became_active":"2024-04-24T23:20:54.470095+0000","last_became_peered":"2024-04-24T23:20:54.470095+0000","last_unstale":"2024-04-24T23:20:56.474912+0000","last_undegraded":"2024-04-24T23:20:56.474912+0000","last_fullsized":"2024-04-24T23:20:56.474912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725394+0000","last_change":"2024-04-24T23:20:55.485347+0000","last_active":"2024-04-24T23:20:56.725394+0000","last_peered":"2024-04-24T23:20:56.725394+0000","last_clean":"2024-04-24T23:20:56.725394+0000","last_became_active":"2024-04-24T23:20:55.485004+0000","last_became_peered":"2024-04-24T23:20:55.485004+0000","last_unstale":"2024-04-24T23:20:56.725394+0000","last_undegraded":"2024-04-24T23:20:56.725394+0000","last_fullsized":"2024-04-24T23:20:56.725394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724782+0000","last_change":"2024-04-24T23:20:54.482240+0000","last_active":"2024-04-24T23:20:56.724782+0000","last_peered":"2024-04-24T23:20:56.724782+0000","last_clean":"2024-04-24T23:20:56.724782+0000","last_became_active":"2024-04-24T23:20:54.481934+0000","last_became_peered":"2024-04-24T23:20:54.481934+0000","last_unstale":"2024-04-24T23:20:56.724782+0000","last_undegraded":"2024-04-24T23:20:56.724782+0000","last_fullsized":"2024-04-24T23:20:56.724782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:13.029046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724293+0000","last_change":"2024-04-24T23:20:55.474530+0000","last_active":"2024-04-24T23:20:56.724293+0000","last_peered":"2024-04-24T23:20:56.724293+0000","last_clean":"2024-04-24T23:20:56.724293+0000","last_became_active":"2024-04-24T23:20:55.474181+0000","last_became_peered":"2024-04-24T23:20:55.474181+0000","last_unstale":"2024-04-24T23:20:56.724293+0000","last_undegraded":"2024-04-24T23:20:56.724293+0000","last_fullsized":"2024-04-24T23:20:56.724293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:43.728916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724886+0000","last_change":"2024-04-24T23:20:54.478684+0000","last_active":"2024-04-24T23:20:56.724886+0000","last_peered":"2024-04-24T23:20:56.724886+0000","last_clean":"2024-04-24T23:20:56.724886+0000","last_became_active":"2024-04-24T23:20:54.478377+0000","last_became_peered":"2024-04-24T23:20:54.478377+0000","last_unstale":"2024-04-24T23:20:56.724886+0000","last_undegraded":"2024-04-24T23:20:56.724886+0000","last_fullsized":"2024-04-24T23:20:56.724886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.478546+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.478253+0000","last_became_peered":"2024-04-24T23:20:55.478253+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476161+0000","last_change":"2024-04-24T23:20:54.478753+0000","last_active":"2024-04-24T23:20:56.476161+0000","last_peered":"2024-04-24T23:20:56.476161+0000","last_clean":"2024-04-24T23:20:56.476161+0000","last_became_active":"2024-04-24T23:20:54.478441+0000","last_became_peered":"2024-04-24T23:20:54.478441+0000","last_unstale":"2024-04-24T23:20:56.476161+0000","last_undegraded":"2024-04-24T23:20:56.476161+0000","last_fullsized":"2024-04-24T23:20:56.476161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475739+0000","last_change":"2024-04-24T23:20:55.483158+0000","last_active":"2024-04-24T23:20:56.475739+0000","last_peered":"2024-04-24T23:20:56.475739+0000","last_clean":"2024-04-24T23:20:56.475739+0000","last_became_active":"2024-04-24T23:20:55.482865+0000","last_became_peered":"2024-04-24T23:20:55.482865+0000","last_unstale":"2024-04-24T23:20:56.475739+0000","last_undegraded":"2024-04-24T23:20:56.475739+0000","last_fullsized":"2024-04-24T23:20:56.475739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477991+0000","last_change":"2024-04-24T23:20:54.476881+0000","last_active":"2024-04-24T23:20:56.477991+0000","last_peered":"2024-04-24T23:20:56.477991+0000","last_clean":"2024-04-24T23:20:56.477991+0000","last_became_active":"2024-04-24T23:20:54.476588+0000","last_became_peered":"2024-04-24T23:20:54.476588+0000","last_unstale":"2024-04-24T23:20:56.477991+0000","last_undegraded":"2024-04-24T23:20:56.477991+0000","last_fullsized":"2024-04-24T23:20:56.477991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475322+0000","last_change":"2024-04-24T23:20:55.482626+0000","last_active":"2024-04-24T23:20:56.475322+0000","last_peered":"2024-04-24T23:20:56.475322+0000","last_clean":"2024-04-24T23:20:56.475322+0000","last_became_active":"2024-04-24T23:20:55.482266+0000","last_became_peered":"2024-04-24T23:20:55.482266+0000","last_unstale":"2024-04-24T23:20:56.475322+0000","last_undegraded":"2024-04-24T23:20:56.475322+0000","last_fullsized":"2024-04-24T23:20:56.475322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725524+0000","last_change":"2024-04-24T23:20:54.484817+0000","last_active":"2024-04-24T23:20:56.725524+0000","last_peered":"2024-04-24T23:20:56.725524+0000","last_clean":"2024-04-24T23:20:56.725524+0000","last_became_active":"2024-04-24T23:20:54.484553+0000","last_became_peered":"2024-04-24T23:20:54.484553+0000","last_unstale":"2024-04-24T23:20:56.725524+0000","last_undegraded":"2024-04-24T23:20:56.725524+0000","last_fullsized":"2024-04-24T23:20:56.725524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:42.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478041+0000","last_change":"2024-04-24T23:20:55.479475+0000","last_active":"2024-04-24T23:20:56.478041+0000","last_peered":"2024-04-24T23:20:56.478041+0000","last_clean":"2024-04-24T23:20:56.478041+0000","last_became_active":"2024-04-24T23:20:55.478984+0000","last_became_peered":"2024-04-24T23:20:55.478984+0000","last_unstale":"2024-04-24T23:20:56.478041+0000","last_undegraded":"2024-04-24T23:20:56.478041+0000","last_fullsized":"2024-04-24T23:20:56.478041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.479792+0000","last_change":"2024-04-24T23:20:54.478307+0000","last_active":"2024-04-24T23:20:56.479792+0000","last_peered":"2024-04-24T23:20:56.479792+0000","last_clean":"2024-04-24T23:20:56.479792+0000","last_became_active":"2024-04-24T23:20:54.477988+0000","last_became_peered":"2024-04-24T23:20:54.477988+0000","last_unstale":"2024-04-24T23:20:56.479792+0000","last_undegraded":"2024-04-24T23:20:56.479792+0000","last_fullsized":"2024-04-24T23:20:56.479792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725717+0000","last_change":"2024-04-24T23:20:55.487879+0000","last_active":"2024-04-24T23:20:56.725717+0000","last_peered":"2024-04-24T23:20:56.725717+0000","last_clean":"2024-04-24T23:20:56.725717+0000","last_became_active":"2024-04-24T23:20:55.487553+0000","last_became_peered":"2024-04-24T23:20:55.487553+0000","last_unstale":"2024-04-24T23:20:56.725717+0000","last_undegraded":"2024-04-24T23:20:56.725717+0000","last_fullsized":"2024-04-24T23:20:56.725717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480845+0000","last_change":"2024-04-24T23:20:54.480826+0000","last_active":"2024-04-24T23:20:56.480845+0000","last_peered":"2024-04-24T23:20:56.480845+0000","last_clean":"2024-04-24T23:20:56.480845+0000","last_became_active":"2024-04-24T23:20:54.480351+0000","last_became_peered":"2024-04-24T23:20:54.480351+0000","last_unstale":"2024-04-24T23:20:56.480845+0000","last_undegraded":"2024-04-24T23:20:56.480845+0000","last_fullsized":"2024-04-24T23:20:56.480845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.480939+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.478888+0000","last_became_peered":"2024-04-24T23:20:55.478888+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724748+0000","last_change":"2024-04-24T23:20:54.482329+0000","last_active":"2024-04-24T23:20:56.724748+0000","last_peered":"2024-04-24T23:20:56.724748+0000","last_clean":"2024-04-24T23:20:56.724748+0000","last_became_active":"2024-04-24T23:20:54.482059+0000","last_became_peered":"2024-04-24T23:20:54.482059+0000","last_unstale":"2024-04-24T23:20:56.724748+0000","last_undegraded":"2024-04-24T23:20:56.724748+0000","last_fullsized":"2024-04-24T23:20:56.724748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475091+0000","last_change":"2024-04-24T23:20:55.478742+0000","last_active":"2024-04-24T23:20:56.475091+0000","last_peered":"2024-04-24T23:20:56.475091+0000","last_clean":"2024-04-24T23:20:56.475091+0000","last_became_active":"2024-04-24T23:20:55.478465+0000","last_became_peered":"2024-04-24T23:20:55.478465+0000","last_unstale":"2024-04-24T23:20:56.475091+0000","last_undegraded":"2024-04-24T23:20:56.475091+0000","last_fullsized":"2024-04-24T23:20:56.475091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:32.641502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477976+0000","last_change":"2024-04-24T23:20:54.480255+0000","last_active":"2024-04-24T23:20:56.477976+0000","last_peered":"2024-04-24T23:20:56.477976+0000","last_clean":"2024-04-24T23:20:56.477976+0000","last_became_active":"2024-04-24T23:20:54.479897+0000","last_became_peered":"2024-04-24T23:20:54.479897+0000","last_unstale":"2024-04-24T23:20:56.477976+0000","last_undegraded":"2024-04-24T23:20:56.477976+0000","last_fullsized":"2024-04-24T23:20:56.477976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725278+0000","last_change":"2024-04-24T23:20:55.480538+0000","last_active":"2024-04-24T23:20:56.725278+0000","last_peered":"2024-04-24T23:20:56.725278+0000","last_clean":"2024-04-24T23:20:56.725278+0000","last_became_active":"2024-04-24T23:20:55.474763+0000","last_became_peered":"2024-04-24T23:20:55.474763+0000","last_unstale":"2024-04-24T23:20:56.725278+0000","last_undegraded":"2024-04-24T23:20:56.725278+0000","last_fullsized":"2024-04-24T23:20:56.725278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477945+0000","last_change":"2024-04-24T23:20:54.475154+0000","last_active":"2024-04-24T23:20:56.477945+0000","last_peered":"2024-04-24T23:20:56.477945+0000","last_clean":"2024-04-24T23:20:56.477945+0000","last_became_active":"2024-04-24T23:20:54.473925+0000","last_became_peered":"2024-04-24T23:20:54.473925+0000","last_unstale":"2024-04-24T23:20:56.477945+0000","last_undegraded":"2024-04-24T23:20:56.477945+0000","last_fullsized":"2024-04-24T23:20:56.477945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725566+0000","last_change":"2024-04-24T23:20:55.479171+0000","last_active":"2024-04-24T23:20:56.725566+0000","last_peered":"2024-04-24T23:20:56.725566+0000","last_clean":"2024-04-24T23:20:56.725566+0000","last_became_active":"2024-04-24T23:20:55.478856+0000","last_became_peered":"2024-04-24T23:20:55.478856+0000","last_unstale":"2024-04-24T23:20:56.725566+0000","last_undegraded":"2024-04-24T23:20:56.725566+0000","last_fullsized":"2024-04-24T23:20:56.725566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725243+0000","last_change":"2024-04-24T23:20:54.484143+0000","last_active":"2024-04-24T23:20:56.725243+0000","last_peered":"2024-04-24T23:20:56.725243+0000","last_clean":"2024-04-24T23:20:56.725243+0000","last_became_active":"2024-04-24T23:20:54.483830+0000","last_became_peered":"2024-04-24T23:20:54.483830+0000","last_unstale":"2024-04-24T23:20:56.725243+0000","last_undegraded":"2024-04-24T23:20:56.725243+0000","last_fullsized":"2024-04-24T23:20:56.725243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480446+0000","last_change":"2024-04-24T23:20:55.479005+0000","last_active":"2024-04-24T23:20:56.480446+0000","last_peered":"2024-04-24T23:20:56.480446+0000","last_clean":"2024-04-24T23:20:56.480446+0000","last_became_active":"2024-04-24T23:20:55.478706+0000","last_became_peered":"2024-04-24T23:20:55.478706+0000","last_unstale":"2024-04-24T23:20:56.480446+0000","last_undegraded":"2024-04-24T23:20:56.480446+0000","last_fullsized":"2024-04-24T23:20:56.480446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474978+0000","last_change":"2024-04-24T23:20:55.469771+0000","last_active":"2024-04-24T23:20:56.474978+0000","last_peered":"2024-04-24T23:20:56.474978+0000","last_clean":"2024-04-24T23:20:56.474978+0000","last_became_active":"2024-04-24T23:20:55.469411+0000","last_became_peered":"2024-04-24T23:20:55.469411+0000","last_unstale":"2024-04-24T23:20:56.474978+0000","last_undegraded":"2024-04-24T23:20:56.474978+0000","last_fullsized":"2024-04-24T23:20:56.474978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725118+0000","last_change":"2024-04-24T23:20:54.485231+0000","last_active":"2024-04-24T23:20:56.725118+0000","last_peered":"2024-04-24T23:20:56.725118+0000","last_clean":"2024-04-24T23:20:56.725118+0000","last_became_active":"2024-04-24T23:20:54.484912+0000","last_became_peered":"2024-04-24T23:20:54.484912+0000","last_unstale":"2024-04-24T23:20:56.725118+0000","last_undegraded":"2024-04-24T23:20:56.725118+0000","last_fullsized":"2024-04-24T23:20:56.725118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:27.147159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476107+0000","last_change":"2024-04-24T23:20:55.473157+0000","last_active":"2024-04-24T23:20:56.476107+0000","last_peered":"2024-04-24T23:20:56.476107+0000","last_clean":"2024-04-24T23:20:56.476107+0000","last_became_active":"2024-04-24T23:20:55.472854+0000","last_became_peered":"2024-04-24T23:20:55.472854+0000","last_unstale":"2024-04-24T23:20:56.476107+0000","last_undegraded":"2024-04-24T23:20:56.476107+0000","last_fullsized":"2024-04-24T23:20:56.476107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725345+0000","last_change":"2024-04-24T23:20:54.485353+0000","last_active":"2024-04-24T23:20:56.725345+0000","last_peered":"2024-04-24T23:20:56.725345+0000","last_clean":"2024-04-24T23:20:56.725345+0000","last_became_active":"2024-04-24T23:20:54.485065+0000","last_became_peered":"2024-04-24T23:20:54.485065+0000","last_unstale":"2024-04-24T23:20:56.725345+0000","last_undegraded":"2024-04-24T23:20:56.725345+0000","last_fullsized":"2024-04-24T23:20:56.725345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724430+0000","last_change":"2024-04-24T23:20:55.487656+0000","last_active":"2024-04-24T23:20:56.724430+0000","last_peered":"2024-04-24T23:20:56.724430+0000","last_clean":"2024-04-24T23:20:56.724430+0000","last_became_active":"2024-04-24T23:20:55.487317+0000","last_became_peered":"2024-04-24T23:20:55.487317+0000","last_unstale":"2024-04-24T23:20:56.724430+0000","last_undegraded":"2024-04-24T23:20:56.724430+0000","last_fullsized":"2024-04-24T23:20:56.724430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725671+0000","last_change":"2024-04-24T23:20:54.475528+0000","last_active":"2024-04-24T23:20:56.725671+0000","last_peered":"2024-04-24T23:20:56.725671+0000","last_clean":"2024-04-24T23:20:56.725671+0000","last_became_active":"2024-04-24T23:20:54.475227+0000","last_became_peered":"2024-04-24T23:20:54.475227+0000","last_unstale":"2024-04-24T23:20:56.725671+0000","last_undegraded":"2024-04-24T23:20:56.725671+0000","last_fullsized":"2024-04-24T23:20:56.725671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.481689+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.481405+0000","last_became_peered":"2024-04-24T23:20:55.481405+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724580+0000","last_change":"2024-04-24T23:20:54.482023+0000","last_active":"2024-04-24T23:20:56.724580+0000","last_peered":"2024-04-24T23:20:56.724580+0000","last_clean":"2024-04-24T23:20:56.724580+0000","last_became_active":"2024-04-24T23:20:54.481494+0000","last_became_peered":"2024-04-24T23:20:54.481494+0000","last_unstale":"2024-04-24T23:20:56.724580+0000","last_undegraded":"2024-04-24T23:20:56.724580+0000","last_fullsized":"2024-04-24T23:20:56.724580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35528,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336312,"statfs":{"total":96636764160,"available":96600383488,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:20:59.532 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:20:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.666+0000 7fdf49c41640 1 Processor -- start 2024-04-24T23:20:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.666+0000 7fdf49c41640 1 -- start start 2024-04-24T23:20:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.667+0000 7fdf49c41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf440b4100 0x7fdf440b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.667+0000 7fdf49c41640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b5e40 0x7fdf440b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.667+0000 7fdf49c41640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf440b62f0 con 0x7fdf44147e30 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.667+0000 7fdf49c41640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fdf44162e20 con 0x7fdf440b4100 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.667+0000 7fdf49c41640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdf44162860 con 0x7fdf440b5e40 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.667+0000 7fdf42ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf440b4100 0x7fdf440b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.667+0000 7fdf42ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf440b4100 0x7fdf440b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54628/0 (socket says 172.21.15.26:54628) 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.667+0000 7fdf42ffd640 1 -- 172.21.15.26:0/1893868515 learned_addr learned my addr 172.21.15.26:0/1893868515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.667+0000 7fdf43fff640 1 --2- 172.21.15.26:0/1893868515 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b5e40 0x7fdf440b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.668+0000 7fdf42ffd640 1 -- 172.21.15.26:0/1893868515 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b5e40 msgr2=0x7fdf440b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.668+0000 7fdf42ffd640 1 --2- 172.21.15.26:0/1893868515 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b5e40 0x7fdf440b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.668+0000 7fdf42ffd640 1 -- 172.21.15.26:0/1893868515 >> v1:172.21.15.29:6789/0 conn(0x7fdf44147e30 legacy=0x7fdf44161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.668+0000 7fdf42ffd640 1 -- 172.21.15.26:0/1893868515 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf44162b40 con 0x7fdf440b4100 2024-04-24T23:20:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.668+0000 7fdf42ffd640 1 --2- 172.21.15.26:0/1893868515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b4100 0x7fdf440b5900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009870 tx=0x7fdf3802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd25bc837c639e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.668+0000 7fdf427fc640 1 -- 172.21.15.26:0/1893868515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38037070 con 0x7fdf440b4100 2024-04-24T23:20:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf427fc640 1 -- 172.21.15.26:0/1893868515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf38032440 con 0x7fdf440b4100 2024-04-24T23:20:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf427fc640 1 -- 172.21.15.26:0/1893868515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38032810 con 0x7fdf440b4100 2024-04-24T23:20:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf49c41640 1 -- 172.21.15.26:0/1893868515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b4100 msgr2=0x7fdf440b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf49c41640 1 --2- 172.21.15.26:0/1893868515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b4100 0x7fdf440b5900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009870 tx=0x7fdf3802a130 comp rx=0 tx=0).stop 2024-04-24T23:20:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf49c41640 1 -- 172.21.15.26:0/1893868515 shutdown_connections 2024-04-24T23:20:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf49c41640 1 --2- 172.21.15.26:0/1893868515 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b5e40 0x7fdf440b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf49c41640 1 --2- 172.21.15.26:0/1893868515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b4100 0x7fdf440b5900 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf49c41640 1 -- 172.21.15.26:0/1893868515 >> 172.21.15.26:0/1893868515 conn(0x7fdf4409aa00 msgr2=0x7fdf440b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf49c41640 1 -- 172.21.15.26:0/1893868515 shutdown_connections 2024-04-24T23:20:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.669+0000 7fdf49c41640 1 -- 172.21.15.26:0/1893868515 wait complete. 2024-04-24T23:20:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.670+0000 7fdf49c41640 1 Processor -- start 2024-04-24T23:20:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.670+0000 7fdf49c41640 1 -- start start 2024-04-24T23:20:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf49c41640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b4100 0x7fdf4413f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf49c41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b5e40 0x7fdf441e7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf49c41640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf44147e30 0x7fdf441e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf49c41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdf440b1a10 con 0x7fdf440b5e40 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf49c41640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdf440b0fb0 con 0x7fdf44147e30 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf49c41640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdf440b1b90 con 0x7fdf440b4100 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf42ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b5e40 0x7fdf441e7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf42ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b5e40 0x7fdf441e7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54632/0 (socket says 172.21.15.26:54632) 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf42ffd640 1 -- 172.21.15.26:0/439779221 learned_addr learned my addr 172.21.15.26:0/439779221 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf437fe640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b4100 0x7fdf4413f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf43fff640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf44147e30 0x7fdf441e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf42ffd640 1 -- 172.21.15.26:0/439779221 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b4100 msgr2=0x7fdf4413f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf42ffd640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b4100 0x7fdf4413f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf42ffd640 1 -- 172.21.15.26:0/439779221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf44147e30 msgr2=0x7fdf441e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf42ffd640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf44147e30 0x7fdf441e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf42ffd640 1 -- 172.21.15.26:0/439779221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf44141690 con 0x7fdf440b5e40 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.671+0000 7fdf43fff640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf44147e30 0x7fdf441e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.672+0000 7fdf42ffd640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b5e40 0x7fdf441e7ce0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf3802a170 tx=0x7fdf380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.672+0000 7fdf2bfff640 1 -- 172.21.15.26:0/439779221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38041020 con 0x7fdf440b5e40 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.672+0000 7fdf2bfff640 1 -- 172.21.15.26:0/439779221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf38030640 con 0x7fdf440b5e40 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.672+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4413fc90 con 0x7fdf440b5e40 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.672+0000 7fdf2bfff640 1 -- 172.21.15.26:0/439779221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38037040 con 0x7fdf440b5e40 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.672+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf441448f0 con 0x7fdf440b5e40 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.673+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf440694c0 con 0x7fdf440b5e40 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.673+0000 7fdf2bfff640 1 -- 172.21.15.26:0/439779221 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf38044020 con 0x7fdf440b5e40 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.674+0000 7fdf2bfff640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf3006be30 0x7fdf3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.674+0000 7fdf437fe640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf3006be30 0x7fdf3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:59.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.674+0000 7fdf2bfff640 1 -- 172.21.15.26:0/439779221 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf380ab5a0 con 0x7fdf440b5e40 2024-04-24T23:20:59.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.674+0000 7fdf437fe640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf3006be30 0x7fdf3006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf340016e0 tx=0x7fdf34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.678+0000 7fdf2bfff640 1 -- 172.21.15.26:0/439779221 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdf3804b030 con 0x7fdf440b5e40 2024-04-24T23:20:59.821 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:20:59.819+0000 7f4257c3b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:20:59.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.930+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fdf4406bf00 con 0x7fdf440b5e40 2024-04-24T23:20:59.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.930+0000 7fdf2bfff640 1 -- 172.21.15.26:0/439779221 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1011 (secure 0 0 0) 0x7fdf44141690 con 0x7fdf440b5e40 2024-04-24T23:20:59.932 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:20:59.932 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":262520832,"bytes_avail":772831592448,"bytes_total":773094113280,"write_bytes_sec":2445,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4287}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-24T23:20:31.034117+0000","services":{}},"progress_events":{}} 2024-04-24T23:20:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf3006be30 msgr2=0x7fdf3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf3006be30 0x7fdf3006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf340016e0 tx=0x7fdf34009040 comp rx=0 tx=0).stop 2024-04-24T23:20:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b5e40 msgr2=0x7fdf441e7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b5e40 0x7fdf441e7ce0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf3802a170 tx=0x7fdf380046c0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 shutdown_connections 2024-04-24T23:20:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf3006be30 0x7fdf3006e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf44147e30 0x7fdf441e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf440b5e40 0x7fdf441e7ce0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 --2- 172.21.15.26:0/439779221 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf440b4100 0x7fdf4413f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 >> 172.21.15.26:0/439779221 conn(0x7fdf4409aa00 msgr2=0x7fdf440b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 shutdown_connections 2024-04-24T23:20:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:20:59.932+0000 7fdf49c41640 1 -- 172.21.15.26:0/439779221 wait complete. 2024-04-24T23:20:59.945 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-24T23:20:59.945 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:21:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.043+0000 7ff29b8f5640 1 Processor -- start 2024-04-24T23:21:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.043+0000 7ff29b8f5640 1 -- start start 2024-04-24T23:21:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff29b8f5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff29406d350 0x7ff29406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff29b8f5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2940cfbc0 con 0x7ff294145910 2024-04-24T23:21:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff29b8f5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff29406d7f0 con 0x7ff2940cd660 2024-04-24T23:21:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff29b8f5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff294145cf0 con 0x7ff29406d350 2024-04-24T23:21:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff298e90640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff29406d350 0x7ff29406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff299691640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff294145910 0x7ff29415d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46322/0 (socket says 172.21.15.26:46322) 2024-04-24T23:21:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff299691640 1 -- 172.21.15.26:0/644197707 learned_addr learned my addr 172.21.15.26:0/644197707 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff298e90640 1 -- 172.21.15.26:0/644197707 >> v1:172.21.15.29:6789/0 conn(0x7ff2940cd660 legacy=0x7ff29415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff298e90640 1 -- 172.21.15.26:0/644197707 >> v1:172.21.15.26:6789/0 conn(0x7ff294145910 legacy=0x7ff29415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.044+0000 7ff298e90640 1 -- 172.21.15.26:0/644197707 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29406dad0 con 0x7ff29406d350 2024-04-24T23:21:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.045+0000 7ff298e90640 1 --2- 172.21.15.26:0/644197707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff29406d350 0x7ff29406e890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff2880097d0 tx=0x7ff28802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bafc7ab742b428 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.045+0000 7ff293fff640 1 -- 172.21.15.26:0/644197707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288037070 con 0x7ff29406d350 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.045+0000 7ff293fff640 1 -- 172.21.15.26:0/644197707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff288032440 con 0x7ff29406d350 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.045+0000 7ff293fff640 1 -- 172.21.15.26:0/644197707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288032810 con 0x7ff29406d350 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.046+0000 7ff29b8f5640 1 -- 172.21.15.26:0/644197707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff29406d350 msgr2=0x7ff29406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.046+0000 7ff29b8f5640 1 --2- 172.21.15.26:0/644197707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff29406d350 0x7ff29406e890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff2880097d0 tx=0x7ff28802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.046+0000 7ff29b8f5640 1 -- 172.21.15.26:0/644197707 shutdown_connections 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.046+0000 7ff29b8f5640 1 --2- 172.21.15.26:0/644197707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff29406d350 0x7ff29406e890 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.046+0000 7ff29b8f5640 1 -- 172.21.15.26:0/644197707 >> 172.21.15.26:0/644197707 conn(0x7ff29406b300 msgr2=0x7ff29406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.046+0000 7ff29b8f5640 1 -- 172.21.15.26:0/644197707 shutdown_connections 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.046+0000 7ff29b8f5640 1 -- 172.21.15.26:0/644197707 wait complete. 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.047+0000 7ff29b8f5640 1 Processor -- start 2024-04-24T23:21:00.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.047+0000 7ff29b8f5640 1 -- start start 2024-04-24T23:21:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff29b8f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff29406d350 0x7ff2941f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff29b8f5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2940cd660 0x7ff29415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff29b8f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff294145910 0x7ff29413d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff29b8f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff29406e370 con 0x7ff294145910 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff29b8f5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff29406e090 con 0x7ff2940cd660 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff29b8f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff2940cdaf0 con 0x7ff29406d350 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299e92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff294145910 0x7ff29413d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299e92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff294145910 0x7ff29413d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54662/0 (socket says 172.21.15.26:54662) 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299e92640 1 -- 172.21.15.26:0/3240640022 learned_addr learned my addr 172.21.15.26:0/3240640022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299691640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff29406d350 0x7ff2941f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff298e90640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2940cd660 0x7ff29415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299e92640 1 -- 172.21.15.26:0/3240640022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff29406d350 msgr2=0x7ff2941f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299e92640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff29406d350 0x7ff2941f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299e92640 1 -- 172.21.15.26:0/3240640022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2940cd660 msgr2=0x7ff29415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299e92640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2940cd660 0x7ff29415e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299e92640 1 -- 172.21.15.26:0/3240640022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2941e4970 con 0x7ff294145910 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.048+0000 7ff299e92640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff294145910 0x7ff29413d8d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c7a0 tx=0x7ff28400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.049+0000 7ff291ffb640 1 -- 172.21.15.26:0/3240640022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284003b70 con 0x7ff294145910 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.049+0000 7ff291ffb640 1 -- 172.21.15.26:0/3240640022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff284003d30 con 0x7ff294145910 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.049+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2941e4c50 con 0x7ff294145910 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.049+0000 7ff291ffb640 1 -- 172.21.15.26:0/3240640022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28400aa90 con 0x7ff294145910 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.049+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2941e5210 con 0x7ff294145910 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.050+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2940694c0 con 0x7ff294145910 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.050+0000 7ff291ffb640 1 -- 172.21.15.26:0/3240640022 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff28400ac30 con 0x7ff294145910 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.050+0000 7ff291ffb640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff28006be30 0x7ff28006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.051+0000 7ff291ffb640 1 -- 172.21.15.26:0/3240640022 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff28408ac70 con 0x7ff294145910 2024-04-24T23:21:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.051+0000 7ff299691640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff28006be30 0x7ff28006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.051+0000 7ff299691640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff28006be30 0x7ff28006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005880 tx=0x7ff28c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.055+0000 7ff291ffb640 1 -- 172.21.15.26:0/3240640022 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff284058fd0 con 0x7ff294145910 2024-04-24T23:21:00.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.236+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff29406c130 con 0x7ff28006be30 2024-04-24T23:21:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.242+0000 7ff291ffb640 1 -- 172.21.15.26:0/3240640022 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7ff29406c130 con 0x7ff28006be30 2024-04-24T23:21:00.246 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff28006be30 msgr2=0x7ff28006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff28006be30 0x7ff28006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005880 tx=0x7ff28c009040 comp rx=0 tx=0).stop 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff294145910 msgr2=0x7ff29413d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff294145910 0x7ff29413d8d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c7a0 tx=0x7ff28400cc60 comp rx=0 tx=0).stop 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 shutdown_connections 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff28006be30 0x7ff28006e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff294145910 0x7ff29413d8d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2940cd660 0x7ff29415e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 --2- 172.21.15.26:0/3240640022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff29406d350 0x7ff2941f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 >> 172.21.15.26:0/3240640022 conn(0x7ff29406b300 msgr2=0x7ff2940b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 shutdown_connections 2024-04-24T23:21:00.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:00.245+0000 7ff29b8f5640 1 -- 172.21.15.26:0/3240640022 wait complete. 2024-04-24T23:21:00.258 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-24T23:20:59.315992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256368,"kb_used_data":2764,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718352,"statfs":{"total":773094113280,"available":772831592448,"internally_reserved":0,"allocated":2830336,"data_stored":1721528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.862540"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725536+0000","last_change":"2024-04-24T23:20:55.487607+0000","last_active":"2024-04-24T23:20:56.725536+0000","last_peered":"2024-04-24T23:20:56.725536+0000","last_clean":"2024-04-24T23:20:56.725536+0000","last_became_active":"2024-04-24T23:20:55.487320+0000","last_became_peered":"2024-04-24T23:20:55.487320+0000","last_unstale":"2024-04-24T23:20:56.725536+0000","last_undegraded":"2024-04-24T23:20:56.725536+0000","last_fullsized":"2024-04-24T23:20:56.725536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:27.675103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478125+0000","last_change":"2024-04-24T23:20:55.480991+0000","last_active":"2024-04-24T23:20:56.478125+0000","last_peered":"2024-04-24T23:20:56.478125+0000","last_clean":"2024-04-24T23:20:56.478125+0000","last_became_active":"2024-04-24T23:20:55.480654+0000","last_became_peered":"2024-04-24T23:20:55.480654+0000","last_unstale":"2024-04-24T23:20:56.478125+0000","last_undegraded":"2024-04-24T23:20:56.478125+0000","last_fullsized":"2024-04-24T23:20:56.478125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724519+0000","last_change":"2024-04-24T23:20:55.479402+0000","last_active":"2024-04-24T23:20:56.724519+0000","last_peered":"2024-04-24T23:20:56.724519+0000","last_clean":"2024-04-24T23:20:56.724519+0000","last_became_active":"2024-04-24T23:20:55.479013+0000","last_became_peered":"2024-04-24T23:20:55.479013+0000","last_unstale":"2024-04-24T23:20:56.724519+0000","last_undegraded":"2024-04-24T23:20:56.724519+0000","last_fullsized":"2024-04-24T23:20:56.724519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475989+0000","last_change":"2024-04-24T23:20:55.481507+0000","last_active":"2024-04-24T23:20:56.475989+0000","last_peered":"2024-04-24T23:20:56.475989+0000","last_clean":"2024-04-24T23:20:56.475989+0000","last_became_active":"2024-04-24T23:20:55.481144+0000","last_became_peered":"2024-04-24T23:20:55.481144+0000","last_unstale":"2024-04-24T23:20:56.475989+0000","last_undegraded":"2024-04-24T23:20:56.475989+0000","last_fullsized":"2024-04-24T23:20:56.475989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725067+0000","last_change":"2024-04-24T23:20:55.482367+0000","last_active":"2024-04-24T23:20:56.725067+0000","last_peered":"2024-04-24T23:20:56.725067+0000","last_clean":"2024-04-24T23:20:56.725067+0000","last_became_active":"2024-04-24T23:20:55.482047+0000","last_became_peered":"2024-04-24T23:20:55.482047+0000","last_unstale":"2024-04-24T23:20:56.725067+0000","last_undegraded":"2024-04-24T23:20:56.725067+0000","last_fullsized":"2024-04-24T23:20:56.725067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475443+0000","last_change":"2024-04-24T23:20:55.482474+0000","last_active":"2024-04-24T23:20:56.475443+0000","last_peered":"2024-04-24T23:20:56.475443+0000","last_clean":"2024-04-24T23:20:56.475443+0000","last_became_active":"2024-04-24T23:20:55.482157+0000","last_became_peered":"2024-04-24T23:20:55.482157+0000","last_unstale":"2024-04-24T23:20:56.475443+0000","last_undegraded":"2024-04-24T23:20:56.475443+0000","last_fullsized":"2024-04-24T23:20:56.475443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724980+0000","last_change":"2024-04-24T23:20:55.487389+0000","last_active":"2024-04-24T23:20:56.724980+0000","last_peered":"2024-04-24T23:20:56.724980+0000","last_clean":"2024-04-24T23:20:56.724980+0000","last_became_active":"2024-04-24T23:20:55.486724+0000","last_became_peered":"2024-04-24T23:20:55.486724+0000","last_unstale":"2024-04-24T23:20:56.724980+0000","last_undegraded":"2024-04-24T23:20:56.724980+0000","last_fullsized":"2024-04-24T23:20:56.724980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724916+0000","last_change":"2024-04-24T23:20:55.486339+0000","last_active":"2024-04-24T23:20:56.724916+0000","last_peered":"2024-04-24T23:20:56.724916+0000","last_clean":"2024-04-24T23:20:56.724916+0000","last_became_active":"2024-04-24T23:20:55.486072+0000","last_became_peered":"2024-04-24T23:20:55.486072+0000","last_unstale":"2024-04-24T23:20:56.724916+0000","last_undegraded":"2024-04-24T23:20:56.724916+0000","last_fullsized":"2024-04-24T23:20:56.724916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.491641+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.491349+0000","last_became_peered":"2024-04-24T23:20:55.491349+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:58.298429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480758+0000","last_change":"2024-04-24T23:20:55.479221+0000","last_active":"2024-04-24T23:20:56.480758+0000","last_peered":"2024-04-24T23:20:56.480758+0000","last_clean":"2024-04-24T23:20:56.480758+0000","last_became_active":"2024-04-24T23:20:55.478795+0000","last_became_peered":"2024-04-24T23:20:55.478795+0000","last_unstale":"2024-04-24T23:20:56.480758+0000","last_undegraded":"2024-04-24T23:20:56.480758+0000","last_fullsized":"2024-04-24T23:20:56.480758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480763+0000","last_change":"2024-04-24T23:20:55.479615+0000","last_active":"2024-04-24T23:20:56.480763+0000","last_peered":"2024-04-24T23:20:56.480763+0000","last_clean":"2024-04-24T23:20:56.480763+0000","last_became_active":"2024-04-24T23:20:55.479317+0000","last_became_peered":"2024-04-24T23:20:55.479317+0000","last_unstale":"2024-04-24T23:20:56.480763+0000","last_undegraded":"2024-04-24T23:20:56.480763+0000","last_fullsized":"2024-04-24T23:20:56.480763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475028+0000","last_change":"2024-04-24T23:20:55.481915+0000","last_active":"2024-04-24T23:20:56.475028+0000","last_peered":"2024-04-24T23:20:56.475028+0000","last_clean":"2024-04-24T23:20:56.475028+0000","last_became_active":"2024-04-24T23:20:55.481596+0000","last_became_peered":"2024-04-24T23:20:55.481596+0000","last_unstale":"2024-04-24T23:20:56.475028+0000","last_undegraded":"2024-04-24T23:20:56.475028+0000","last_fullsized":"2024-04-24T23:20:56.475028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:20.844340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725493+0000","last_change":"2024-04-24T23:20:55.476416+0000","last_active":"2024-04-24T23:20:56.725493+0000","last_peered":"2024-04-24T23:20:56.725493+0000","last_clean":"2024-04-24T23:20:56.725493+0000","last_became_active":"2024-04-24T23:20:55.476068+0000","last_became_peered":"2024-04-24T23:20:55.476068+0000","last_unstale":"2024-04-24T23:20:56.725493+0000","last_undegraded":"2024-04-24T23:20:56.725493+0000","last_fullsized":"2024-04-24T23:20:56.725493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724195+0000","last_change":"2024-04-24T23:20:55.479125+0000","last_active":"2024-04-24T23:20:56.724195+0000","last_peered":"2024-04-24T23:20:56.724195+0000","last_clean":"2024-04-24T23:20:56.724195+0000","last_became_active":"2024-04-24T23:20:55.478473+0000","last_became_peered":"2024-04-24T23:20:55.478473+0000","last_unstale":"2024-04-24T23:20:56.724195+0000","last_undegraded":"2024-04-24T23:20:56.724195+0000","last_fullsized":"2024-04-24T23:20:56.724195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:19.475113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477868+0000","last_change":"2024-04-24T23:20:55.474883+0000","last_active":"2024-04-24T23:20:56.477868+0000","last_peered":"2024-04-24T23:20:56.477868+0000","last_clean":"2024-04-24T23:20:56.477868+0000","last_became_active":"2024-04-24T23:20:55.474459+0000","last_became_peered":"2024-04-24T23:20:55.474459+0000","last_unstale":"2024-04-24T23:20:56.477868+0000","last_undegraded":"2024-04-24T23:20:56.477868+0000","last_fullsized":"2024-04-24T23:20:56.477868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475040+0000","last_change":"2024-04-24T23:20:55.483511+0000","last_active":"2024-04-24T23:20:56.475040+0000","last_peered":"2024-04-24T23:20:56.475040+0000","last_clean":"2024-04-24T23:20:56.475040+0000","last_became_active":"2024-04-24T23:20:55.483171+0000","last_became_peered":"2024-04-24T23:20:55.483171+0000","last_unstale":"2024-04-24T23:20:56.475040+0000","last_undegraded":"2024-04-24T23:20:56.475040+0000","last_fullsized":"2024-04-24T23:20:56.475040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475537+0000","last_change":"2024-04-24T23:20:55.489069+0000","last_active":"2024-04-24T23:20:56.475537+0000","last_peered":"2024-04-24T23:20:56.475537+0000","last_clean":"2024-04-24T23:20:56.475537+0000","last_became_active":"2024-04-24T23:20:55.488796+0000","last_became_peered":"2024-04-24T23:20:55.488796+0000","last_unstale":"2024-04-24T23:20:56.475537+0000","last_undegraded":"2024-04-24T23:20:56.475537+0000","last_fullsized":"2024-04-24T23:20:56.475537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724755+0000","last_change":"2024-04-24T23:20:55.480920+0000","last_active":"2024-04-24T23:20:56.724755+0000","last_peered":"2024-04-24T23:20:56.724755+0000","last_clean":"2024-04-24T23:20:56.724755+0000","last_became_active":"2024-04-24T23:20:55.480581+0000","last_became_peered":"2024-04-24T23:20:55.480581+0000","last_unstale":"2024-04-24T23:20:56.724755+0000","last_undegraded":"2024-04-24T23:20:56.724755+0000","last_fullsized":"2024-04-24T23:20:56.724755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725520+0000","last_change":"2024-04-24T23:20:55.488901+0000","last_active":"2024-04-24T23:20:56.725520+0000","last_peered":"2024-04-24T23:20:56.725520+0000","last_clean":"2024-04-24T23:20:56.725520+0000","last_became_active":"2024-04-24T23:20:55.488647+0000","last_became_peered":"2024-04-24T23:20:55.488647+0000","last_unstale":"2024-04-24T23:20:56.725520+0000","last_undegraded":"2024-04-24T23:20:56.725520+0000","last_fullsized":"2024-04-24T23:20:56.725520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480674+0000","last_change":"2024-04-24T23:20:55.476826+0000","last_active":"2024-04-24T23:20:56.480674+0000","last_peered":"2024-04-24T23:20:56.480674+0000","last_clean":"2024-04-24T23:20:56.480674+0000","last_became_active":"2024-04-24T23:20:55.476481+0000","last_became_peered":"2024-04-24T23:20:55.476481+0000","last_unstale":"2024-04-24T23:20:56.480674+0000","last_undegraded":"2024-04-24T23:20:56.480674+0000","last_fullsized":"2024-04-24T23:20:56.480674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724349+0000","last_change":"2024-04-24T23:20:55.491072+0000","last_active":"2024-04-24T23:20:56.724349+0000","last_peered":"2024-04-24T23:20:56.724349+0000","last_clean":"2024-04-24T23:20:56.724349+0000","last_became_active":"2024-04-24T23:20:55.490763+0000","last_became_peered":"2024-04-24T23:20:55.490763+0000","last_unstale":"2024-04-24T23:20:56.724349+0000","last_undegraded":"2024-04-24T23:20:56.724349+0000","last_fullsized":"2024-04-24T23:20:56.724349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475836+0000","last_change":"2024-04-24T23:20:55.487623+0000","last_active":"2024-04-24T23:20:56.475836+0000","last_peered":"2024-04-24T23:20:56.475836+0000","last_clean":"2024-04-24T23:20:56.475836+0000","last_became_active":"2024-04-24T23:20:55.487355+0000","last_became_peered":"2024-04-24T23:20:55.487355+0000","last_unstale":"2024-04-24T23:20:56.475836+0000","last_undegraded":"2024-04-24T23:20:56.475836+0000","last_fullsized":"2024-04-24T23:20:56.475836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725470+0000","last_change":"2024-04-24T23:20:55.476152+0000","last_active":"2024-04-24T23:20:56.725470+0000","last_peered":"2024-04-24T23:20:56.725470+0000","last_clean":"2024-04-24T23:20:56.725470+0000","last_became_active":"2024-04-24T23:20:55.475857+0000","last_became_peered":"2024-04-24T23:20:55.475857+0000","last_unstale":"2024-04-24T23:20:56.725470+0000","last_undegraded":"2024-04-24T23:20:56.725470+0000","last_fullsized":"2024-04-24T23:20:56.725470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475501+0000","last_change":"2024-04-24T23:20:55.484268+0000","last_active":"2024-04-24T23:20:56.475501+0000","last_peered":"2024-04-24T23:20:56.475501+0000","last_clean":"2024-04-24T23:20:56.475501+0000","last_became_active":"2024-04-24T23:20:55.483972+0000","last_became_peered":"2024-04-24T23:20:55.483972+0000","last_unstale":"2024-04-24T23:20:56.475501+0000","last_undegraded":"2024-04-24T23:20:56.475501+0000","last_fullsized":"2024-04-24T23:20:56.475501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725722+0000","last_change":"2024-04-24T23:20:55.480092+0000","last_active":"2024-04-24T23:20:56.725722+0000","last_peered":"2024-04-24T23:20:56.725722+0000","last_clean":"2024-04-24T23:20:56.725722+0000","last_became_active":"2024-04-24T23:20:55.479783+0000","last_became_peered":"2024-04-24T23:20:55.479783+0000","last_unstale":"2024-04-24T23:20:56.725722+0000","last_undegraded":"2024-04-24T23:20:56.725722+0000","last_fullsized":"2024-04-24T23:20:56.725722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480830+0000","last_change":"2024-04-24T23:20:55.488050+0000","last_active":"2024-04-24T23:20:56.480830+0000","last_peered":"2024-04-24T23:20:56.480830+0000","last_clean":"2024-04-24T23:20:56.480830+0000","last_became_active":"2024-04-24T23:20:55.487762+0000","last_became_peered":"2024-04-24T23:20:55.487762+0000","last_unstale":"2024-04-24T23:20:56.480830+0000","last_undegraded":"2024-04-24T23:20:56.480830+0000","last_fullsized":"2024-04-24T23:20:56.480830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478091+0000","last_change":"2024-04-24T23:20:55.478530+0000","last_active":"2024-04-24T23:20:56.478091+0000","last_peered":"2024-04-24T23:20:56.478091+0000","last_clean":"2024-04-24T23:20:56.478091+0000","last_became_active":"2024-04-24T23:20:55.478233+0000","last_became_peered":"2024-04-24T23:20:55.478233+0000","last_unstale":"2024-04-24T23:20:56.478091+0000","last_undegraded":"2024-04-24T23:20:56.478091+0000","last_fullsized":"2024-04-24T23:20:56.478091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725374+0000","last_change":"2024-04-24T23:20:55.480077+0000","last_active":"2024-04-24T23:20:56.725374+0000","last_peered":"2024-04-24T23:20:56.725374+0000","last_clean":"2024-04-24T23:20:56.725374+0000","last_became_active":"2024-04-24T23:20:55.479769+0000","last_became_peered":"2024-04-24T23:20:55.479769+0000","last_unstale":"2024-04-24T23:20:56.725374+0000","last_undegraded":"2024-04-24T23:20:56.725374+0000","last_fullsized":"2024-04-24T23:20:56.725374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724513+0000","last_change":"2024-04-24T23:20:55.481475+0000","last_active":"2024-04-24T23:20:56.724513+0000","last_peered":"2024-04-24T23:20:56.724513+0000","last_clean":"2024-04-24T23:20:56.724513+0000","last_became_active":"2024-04-24T23:20:55.480091+0000","last_became_peered":"2024-04-24T23:20:55.480091+0000","last_unstale":"2024-04-24T23:20:56.724513+0000","last_undegraded":"2024-04-24T23:20:56.724513+0000","last_fullsized":"2024-04-24T23:20:56.724513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476229+0000","last_change":"2024-04-24T23:20:55.485092+0000","last_active":"2024-04-24T23:20:56.476229+0000","last_peered":"2024-04-24T23:20:56.476229+0000","last_clean":"2024-04-24T23:20:56.476229+0000","last_became_active":"2024-04-24T23:20:55.484409+0000","last_became_peered":"2024-04-24T23:20:55.484409+0000","last_unstale":"2024-04-24T23:20:56.476229+0000","last_undegraded":"2024-04-24T23:20:56.476229+0000","last_fullsized":"2024-04-24T23:20:56.476229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.469291+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.468961+0000","last_became_peered":"2024-04-24T23:20:55.468961+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480450+0000","last_change":"2024-04-24T23:20:55.484826+0000","last_active":"2024-04-24T23:20:56.480450+0000","last_peered":"2024-04-24T23:20:56.480450+0000","last_clean":"2024-04-24T23:20:56.480450+0000","last_became_active":"2024-04-24T23:20:55.484500+0000","last_became_peered":"2024-04-24T23:20:55.484500+0000","last_unstale":"2024-04-24T23:20:56.480450+0000","last_undegraded":"2024-04-24T23:20:56.480450+0000","last_fullsized":"2024-04-24T23:20:56.480450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475480+0000","last_change":"2024-04-24T23:20:55.476006+0000","last_active":"2024-04-24T23:20:56.475480+0000","last_peered":"2024-04-24T23:20:56.475480+0000","last_clean":"2024-04-24T23:20:56.475480+0000","last_became_active":"2024-04-24T23:20:55.475090+0000","last_became_peered":"2024-04-24T23:20:55.475090+0000","last_unstale":"2024-04-24T23:20:56.475480+0000","last_undegraded":"2024-04-24T23:20:56.475480+0000","last_fullsized":"2024-04-24T23:20:56.475480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725560+0000","last_change":"2024-04-24T23:20:54.481863+0000","last_active":"2024-04-24T23:20:56.725560+0000","last_peered":"2024-04-24T23:20:56.725560+0000","last_clean":"2024-04-24T23:20:56.725560+0000","last_became_active":"2024-04-24T23:20:54.481498+0000","last_became_peered":"2024-04-24T23:20:54.481498+0000","last_unstale":"2024-04-24T23:20:56.725560+0000","last_undegraded":"2024-04-24T23:20:56.725560+0000","last_fullsized":"2024-04-24T23:20:56.725560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724565+0000","last_change":"2024-04-24T23:20:55.478651+0000","last_active":"2024-04-24T23:20:56.724565+0000","last_peered":"2024-04-24T23:20:56.724565+0000","last_clean":"2024-04-24T23:20:56.724565+0000","last_became_active":"2024-04-24T23:20:55.477783+0000","last_became_peered":"2024-04-24T23:20:55.477783+0000","last_unstale":"2024-04-24T23:20:56.724565+0000","last_undegraded":"2024-04-24T23:20:56.724565+0000","last_fullsized":"2024-04-24T23:20:56.724565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480400+0000","last_change":"2024-04-24T23:20:54.477560+0000","last_active":"2024-04-24T23:20:56.480400+0000","last_peered":"2024-04-24T23:20:56.480400+0000","last_clean":"2024-04-24T23:20:56.480400+0000","last_became_active":"2024-04-24T23:20:54.477261+0000","last_became_peered":"2024-04-24T23:20:54.477261+0000","last_unstale":"2024-04-24T23:20:56.480400+0000","last_undegraded":"2024-04-24T23:20:56.480400+0000","last_fullsized":"2024-04-24T23:20:56.480400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725154+0000","last_change":"2024-04-24T23:20:54.479540+0000","last_active":"2024-04-24T23:20:56.725154+0000","last_peered":"2024-04-24T23:20:56.725154+0000","last_clean":"2024-04-24T23:20:56.725154+0000","last_became_active":"2024-04-24T23:20:54.479206+0000","last_became_peered":"2024-04-24T23:20:54.479206+0000","last_unstale":"2024-04-24T23:20:56.725154+0000","last_undegraded":"2024-04-24T23:20:56.725154+0000","last_fullsized":"2024-04-24T23:20:56.725154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725091+0000","last_change":"2024-04-24T23:20:55.484927+0000","last_active":"2024-04-24T23:20:56.725091+0000","last_peered":"2024-04-24T23:20:56.725091+0000","last_clean":"2024-04-24T23:20:56.725091+0000","last_became_active":"2024-04-24T23:20:55.484611+0000","last_became_peered":"2024-04-24T23:20:55.484611+0000","last_unstale":"2024-04-24T23:20:56.725091+0000","last_undegraded":"2024-04-24T23:20:56.725091+0000","last_fullsized":"2024-04-24T23:20:56.725091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475910+0000","last_change":"2024-04-24T23:20:54.471358+0000","last_active":"2024-04-24T23:20:56.475910+0000","last_peered":"2024-04-24T23:20:56.475910+0000","last_clean":"2024-04-24T23:20:56.475910+0000","last_became_active":"2024-04-24T23:20:54.470800+0000","last_became_peered":"2024-04-24T23:20:54.470800+0000","last_unstale":"2024-04-24T23:20:56.475910+0000","last_undegraded":"2024-04-24T23:20:56.475910+0000","last_fullsized":"2024-04-24T23:20:56.475910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.477535+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.477215+0000","last_became_peered":"2024-04-24T23:20:55.477215+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480761+0000","last_change":"2024-04-24T23:20:54.479450+0000","last_active":"2024-04-24T23:20:56.480761+0000","last_peered":"2024-04-24T23:20:56.480761+0000","last_clean":"2024-04-24T23:20:56.480761+0000","last_became_active":"2024-04-24T23:20:54.479174+0000","last_became_peered":"2024-04-24T23:20:54.479174+0000","last_unstale":"2024-04-24T23:20:56.480761+0000","last_undegraded":"2024-04-24T23:20:56.480761+0000","last_fullsized":"2024-04-24T23:20:56.480761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476156+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.476156+0000","last_peered":"2024-04-24T23:20:56.476156+0000","last_clean":"2024-04-24T23:20:56.476156+0000","last_became_active":"2024-04-24T23:20:55.488014+0000","last_became_peered":"2024-04-24T23:20:55.488014+0000","last_unstale":"2024-04-24T23:20:56.476156+0000","last_undegraded":"2024-04-24T23:20:56.476156+0000","last_fullsized":"2024-04-24T23:20:56.476156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476129+0000","last_change":"2024-04-24T23:20:54.480387+0000","last_active":"2024-04-24T23:20:56.476129+0000","last_peered":"2024-04-24T23:20:56.476129+0000","last_clean":"2024-04-24T23:20:56.476129+0000","last_became_active":"2024-04-24T23:20:54.480037+0000","last_became_peered":"2024-04-24T23:20:54.480037+0000","last_unstale":"2024-04-24T23:20:56.476129+0000","last_undegraded":"2024-04-24T23:20:56.476129+0000","last_fullsized":"2024-04-24T23:20:56.476129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724464+0000","last_change":"2024-04-24T23:20:55.481206+0000","last_active":"2024-04-24T23:20:56.724464+0000","last_peered":"2024-04-24T23:20:56.724464+0000","last_clean":"2024-04-24T23:20:56.724464+0000","last_became_active":"2024-04-24T23:20:55.478018+0000","last_became_peered":"2024-04-24T23:20:55.478018+0000","last_unstale":"2024-04-24T23:20:56.724464+0000","last_undegraded":"2024-04-24T23:20:56.724464+0000","last_fullsized":"2024-04-24T23:20:56.724464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478112+0000","last_change":"2024-04-24T23:20:54.469557+0000","last_active":"2024-04-24T23:20:56.478112+0000","last_peered":"2024-04-24T23:20:56.478112+0000","last_clean":"2024-04-24T23:20:56.478112+0000","last_became_active":"2024-04-24T23:20:54.469251+0000","last_became_peered":"2024-04-24T23:20:54.469251+0000","last_unstale":"2024-04-24T23:20:56.478112+0000","last_undegraded":"2024-04-24T23:20:56.478112+0000","last_fullsized":"2024-04-24T23:20:56.478112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475184+0000","last_change":"2024-04-24T23:20:55.491797+0000","last_active":"2024-04-24T23:20:56.475184+0000","last_peered":"2024-04-24T23:20:56.475184+0000","last_clean":"2024-04-24T23:20:56.475184+0000","last_became_active":"2024-04-24T23:20:55.491453+0000","last_became_peered":"2024-04-24T23:20:55.491453+0000","last_unstale":"2024-04-24T23:20:56.475184+0000","last_undegraded":"2024-04-24T23:20:56.475184+0000","last_fullsized":"2024-04-24T23:20:56.475184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:05.444694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480475+0000","last_change":"2024-04-24T23:20:54.482472+0000","last_active":"2024-04-24T23:20:56.480475+0000","last_peered":"2024-04-24T23:20:56.480475+0000","last_clean":"2024-04-24T23:20:56.480475+0000","last_became_active":"2024-04-24T23:20:54.481351+0000","last_became_peered":"2024-04-24T23:20:54.481351+0000","last_unstale":"2024-04-24T23:20:56.480475+0000","last_undegraded":"2024-04-24T23:20:56.480475+0000","last_fullsized":"2024-04-24T23:20:56.480475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725210+0000","last_change":"2024-04-24T23:20:55.483153+0000","last_active":"2024-04-24T23:20:56.725210+0000","last_peered":"2024-04-24T23:20:56.725210+0000","last_clean":"2024-04-24T23:20:56.725210+0000","last_became_active":"2024-04-24T23:20:55.482859+0000","last_became_peered":"2024-04-24T23:20:55.482859+0000","last_unstale":"2024-04-24T23:20:56.725210+0000","last_undegraded":"2024-04-24T23:20:56.725210+0000","last_fullsized":"2024-04-24T23:20:56.725210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724624+0000","last_change":"2024-04-24T23:20:54.478089+0000","last_active":"2024-04-24T23:20:56.724624+0000","last_peered":"2024-04-24T23:20:56.724624+0000","last_clean":"2024-04-24T23:20:56.724624+0000","last_became_active":"2024-04-24T23:20:54.477302+0000","last_became_peered":"2024-04-24T23:20:54.477302+0000","last_unstale":"2024-04-24T23:20:56.724624+0000","last_undegraded":"2024-04-24T23:20:56.724624+0000","last_fullsized":"2024-04-24T23:20:56.724624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724289+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.724289+0000","last_peered":"2024-04-24T23:20:56.724289+0000","last_clean":"2024-04-24T23:20:56.724289+0000","last_became_active":"2024-04-24T23:20:55.488016+0000","last_became_peered":"2024-04-24T23:20:55.488016+0000","last_unstale":"2024-04-24T23:20:56.724289+0000","last_undegraded":"2024-04-24T23:20:56.724289+0000","last_fullsized":"2024-04-24T23:20:56.724289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:48.795181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476045+0000","last_change":"2024-04-24T23:20:54.473922+0000","last_active":"2024-04-24T23:20:56.476045+0000","last_peered":"2024-04-24T23:20:56.476045+0000","last_clean":"2024-04-24T23:20:56.476045+0000","last_became_active":"2024-04-24T23:20:54.473601+0000","last_became_peered":"2024-04-24T23:20:54.473601+0000","last_unstale":"2024-04-24T23:20:56.476045+0000","last_undegraded":"2024-04-24T23:20:56.476045+0000","last_fullsized":"2024-04-24T23:20:56.476045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725128+0000","last_change":"2024-04-24T23:20:55.488225+0000","last_active":"2024-04-24T23:20:56.725128+0000","last_peered":"2024-04-24T23:20:56.725128+0000","last_clean":"2024-04-24T23:20:56.725128+0000","last_became_active":"2024-04-24T23:20:55.485663+0000","last_became_peered":"2024-04-24T23:20:55.485663+0000","last_unstale":"2024-04-24T23:20:56.725128+0000","last_undegraded":"2024-04-24T23:20:56.725128+0000","last_fullsized":"2024-04-24T23:20:56.725128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:54.478237+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:54.477943+0000","last_became_peered":"2024-04-24T23:20:54.477943+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:55.484884+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:55.484578+0000","last_became_peered":"2024-04-24T23:20:55.484578+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724810+0000","last_change":"2024-04-24T23:20:54.473033+0000","last_active":"2024-04-24T23:20:56.724810+0000","last_peered":"2024-04-24T23:20:56.724810+0000","last_clean":"2024-04-24T23:20:56.724810+0000","last_became_active":"2024-04-24T23:20:54.472748+0000","last_became_peered":"2024-04-24T23:20:54.472748+0000","last_unstale":"2024-04-24T23:20:56.724810+0000","last_undegraded":"2024-04-24T23:20:56.724810+0000","last_fullsized":"2024-04-24T23:20:56.724810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724977+0000","last_change":"2024-04-24T23:20:55.485153+0000","last_active":"2024-04-24T23:20:56.724977+0000","last_peered":"2024-04-24T23:20:56.724977+0000","last_clean":"2024-04-24T23:20:56.724977+0000","last_became_active":"2024-04-24T23:20:55.484876+0000","last_became_peered":"2024-04-24T23:20:55.484876+0000","last_unstale":"2024-04-24T23:20:56.724977+0000","last_undegraded":"2024-04-24T23:20:56.724977+0000","last_fullsized":"2024-04-24T23:20:56.724977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474721+0000","last_change":"2024-04-24T23:20:54.476273+0000","last_active":"2024-04-24T23:20:56.474721+0000","last_peered":"2024-04-24T23:20:56.474721+0000","last_clean":"2024-04-24T23:20:56.474721+0000","last_became_active":"2024-04-24T23:20:54.475993+0000","last_became_peered":"2024-04-24T23:20:54.475993+0000","last_unstale":"2024-04-24T23:20:56.474721+0000","last_undegraded":"2024-04-24T23:20:56.474721+0000","last_fullsized":"2024-04-24T23:20:56.474721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725412+0000","last_change":"2024-04-24T23:20:55.487659+0000","last_active":"2024-04-24T23:20:56.725412+0000","last_peered":"2024-04-24T23:20:56.725412+0000","last_clean":"2024-04-24T23:20:56.725412+0000","last_became_active":"2024-04-24T23:20:55.486557+0000","last_became_peered":"2024-04-24T23:20:55.486557+0000","last_unstale":"2024-04-24T23:20:56.725412+0000","last_undegraded":"2024-04-24T23:20:56.725412+0000","last_fullsized":"2024-04-24T23:20:56.725412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724872+0000","last_change":"2024-04-24T23:20:54.477334+0000","last_active":"2024-04-24T23:20:56.724872+0000","last_peered":"2024-04-24T23:20:56.724872+0000","last_clean":"2024-04-24T23:20:56.724872+0000","last_became_active":"2024-04-24T23:20:54.477058+0000","last_became_peered":"2024-04-24T23:20:54.477058+0000","last_unstale":"2024-04-24T23:20:56.724872+0000","last_undegraded":"2024-04-24T23:20:56.724872+0000","last_fullsized":"2024-04-24T23:20:56.724872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725667+0000","last_change":"2024-04-24T23:20:55.492754+0000","last_active":"2024-04-24T23:20:56.725667+0000","last_peered":"2024-04-24T23:20:56.725667+0000","last_clean":"2024-04-24T23:20:56.725667+0000","last_became_active":"2024-04-24T23:20:55.492474+0000","last_became_peered":"2024-04-24T23:20:55.492474+0000","last_unstale":"2024-04-24T23:20:56.725667+0000","last_undegraded":"2024-04-24T23:20:56.725667+0000","last_fullsized":"2024-04-24T23:20:56.725667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:54.766130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478192+0000","last_change":"2024-04-24T23:20:54.471091+0000","last_active":"2024-04-24T23:20:56.478192+0000","last_peered":"2024-04-24T23:20:56.478192+0000","last_clean":"2024-04-24T23:20:56.478192+0000","last_became_active":"2024-04-24T23:20:54.470811+0000","last_became_peered":"2024-04-24T23:20:54.470811+0000","last_unstale":"2024-04-24T23:20:56.478192+0000","last_undegraded":"2024-04-24T23:20:56.478192+0000","last_fullsized":"2024-04-24T23:20:56.478192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476216+0000","last_change":"2024-04-24T23:20:55.480640+0000","last_active":"2024-04-24T23:20:56.476216+0000","last_peered":"2024-04-24T23:20:56.476216+0000","last_clean":"2024-04-24T23:20:56.476216+0000","last_became_active":"2024-04-24T23:20:55.480327+0000","last_became_peered":"2024-04-24T23:20:55.480327+0000","last_unstale":"2024-04-24T23:20:56.476216+0000","last_undegraded":"2024-04-24T23:20:56.476216+0000","last_fullsized":"2024-04-24T23:20:56.476216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725414+0000","last_change":"2024-04-24T23:20:54.476392+0000","last_active":"2024-04-24T23:20:56.725414+0000","last_peered":"2024-04-24T23:20:56.725414+0000","last_clean":"2024-04-24T23:20:56.725414+0000","last_became_active":"2024-04-24T23:20:54.476051+0000","last_became_peered":"2024-04-24T23:20:54.476051+0000","last_unstale":"2024-04-24T23:20:56.725414+0000","last_undegraded":"2024-04-24T23:20:56.725414+0000","last_fullsized":"2024-04-24T23:20:56.725414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725389+0000","last_change":"2024-04-24T23:20:55.483909+0000","last_active":"2024-04-24T23:20:56.725389+0000","last_peered":"2024-04-24T23:20:56.725389+0000","last_clean":"2024-04-24T23:20:56.725389+0000","last_became_active":"2024-04-24T23:20:55.483625+0000","last_became_peered":"2024-04-24T23:20:55.483625+0000","last_unstale":"2024-04-24T23:20:56.725389+0000","last_undegraded":"2024-04-24T23:20:56.725389+0000","last_fullsized":"2024-04-24T23:20:56.725389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724832+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:20:56.724832+0000","last_peered":"2024-04-24T23:20:56.724832+0000","last_clean":"2024-04-24T23:20:56.724832+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:20:56.724832+0000","last_undegraded":"2024-04-24T23:20:56.724832+0000","last_fullsized":"2024-04-24T23:20:56.724832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474912+0000","last_change":"2024-04-24T23:20:54.470392+0000","last_active":"2024-04-24T23:20:56.474912+0000","last_peered":"2024-04-24T23:20:56.474912+0000","last_clean":"2024-04-24T23:20:56.474912+0000","last_became_active":"2024-04-24T23:20:54.470095+0000","last_became_peered":"2024-04-24T23:20:54.470095+0000","last_unstale":"2024-04-24T23:20:56.474912+0000","last_undegraded":"2024-04-24T23:20:56.474912+0000","last_fullsized":"2024-04-24T23:20:56.474912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725394+0000","last_change":"2024-04-24T23:20:55.485347+0000","last_active":"2024-04-24T23:20:56.725394+0000","last_peered":"2024-04-24T23:20:56.725394+0000","last_clean":"2024-04-24T23:20:56.725394+0000","last_became_active":"2024-04-24T23:20:55.485004+0000","last_became_peered":"2024-04-24T23:20:55.485004+0000","last_unstale":"2024-04-24T23:20:56.725394+0000","last_undegraded":"2024-04-24T23:20:56.725394+0000","last_fullsized":"2024-04-24T23:20:56.725394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724782+0000","last_change":"2024-04-24T23:20:54.482240+0000","last_active":"2024-04-24T23:20:56.724782+0000","last_peered":"2024-04-24T23:20:56.724782+0000","last_clean":"2024-04-24T23:20:56.724782+0000","last_became_active":"2024-04-24T23:20:54.481934+0000","last_became_peered":"2024-04-24T23:20:54.481934+0000","last_unstale":"2024-04-24T23:20:56.724782+0000","last_undegraded":"2024-04-24T23:20:56.724782+0000","last_fullsized":"2024-04-24T23:20:56.724782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:13.029046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724293+0000","last_change":"2024-04-24T23:20:55.474530+0000","last_active":"2024-04-24T23:20:56.724293+0000","last_peered":"2024-04-24T23:20:56.724293+0000","last_clean":"2024-04-24T23:20:56.724293+0000","last_became_active":"2024-04-24T23:20:55.474181+0000","last_became_peered":"2024-04-24T23:20:55.474181+0000","last_unstale":"2024-04-24T23:20:56.724293+0000","last_undegraded":"2024-04-24T23:20:56.724293+0000","last_fullsized":"2024-04-24T23:20:56.724293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:43.728916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724886+0000","last_change":"2024-04-24T23:20:54.478684+0000","last_active":"2024-04-24T23:20:56.724886+0000","last_peered":"2024-04-24T23:20:56.724886+0000","last_clean":"2024-04-24T23:20:56.724886+0000","last_became_active":"2024-04-24T23:20:54.478377+0000","last_became_peered":"2024-04-24T23:20:54.478377+0000","last_unstale":"2024-04-24T23:20:56.724886+0000","last_undegraded":"2024-04-24T23:20:56.724886+0000","last_fullsized":"2024-04-24T23:20:56.724886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.478546+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.478253+0000","last_became_peered":"2024-04-24T23:20:55.478253+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476161+0000","last_change":"2024-04-24T23:20:54.478753+0000","last_active":"2024-04-24T23:20:56.476161+0000","last_peered":"2024-04-24T23:20:56.476161+0000","last_clean":"2024-04-24T23:20:56.476161+0000","last_became_active":"2024-04-24T23:20:54.478441+0000","last_became_peered":"2024-04-24T23:20:54.478441+0000","last_unstale":"2024-04-24T23:20:56.476161+0000","last_undegraded":"2024-04-24T23:20:56.476161+0000","last_fullsized":"2024-04-24T23:20:56.476161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475739+0000","last_change":"2024-04-24T23:20:55.483158+0000","last_active":"2024-04-24T23:20:56.475739+0000","last_peered":"2024-04-24T23:20:56.475739+0000","last_clean":"2024-04-24T23:20:56.475739+0000","last_became_active":"2024-04-24T23:20:55.482865+0000","last_became_peered":"2024-04-24T23:20:55.482865+0000","last_unstale":"2024-04-24T23:20:56.475739+0000","last_undegraded":"2024-04-24T23:20:56.475739+0000","last_fullsized":"2024-04-24T23:20:56.475739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477991+0000","last_change":"2024-04-24T23:20:54.476881+0000","last_active":"2024-04-24T23:20:56.477991+0000","last_peered":"2024-04-24T23:20:56.477991+0000","last_clean":"2024-04-24T23:20:56.477991+0000","last_became_active":"2024-04-24T23:20:54.476588+0000","last_became_peered":"2024-04-24T23:20:54.476588+0000","last_unstale":"2024-04-24T23:20:56.477991+0000","last_undegraded":"2024-04-24T23:20:56.477991+0000","last_fullsized":"2024-04-24T23:20:56.477991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475322+0000","last_change":"2024-04-24T23:20:55.482626+0000","last_active":"2024-04-24T23:20:56.475322+0000","last_peered":"2024-04-24T23:20:56.475322+0000","last_clean":"2024-04-24T23:20:56.475322+0000","last_became_active":"2024-04-24T23:20:55.482266+0000","last_became_peered":"2024-04-24T23:20:55.482266+0000","last_unstale":"2024-04-24T23:20:56.475322+0000","last_undegraded":"2024-04-24T23:20:56.475322+0000","last_fullsized":"2024-04-24T23:20:56.475322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725524+0000","last_change":"2024-04-24T23:20:54.484817+0000","last_active":"2024-04-24T23:20:56.725524+0000","last_peered":"2024-04-24T23:20:56.725524+0000","last_clean":"2024-04-24T23:20:56.725524+0000","last_became_active":"2024-04-24T23:20:54.484553+0000","last_became_peered":"2024-04-24T23:20:54.484553+0000","last_unstale":"2024-04-24T23:20:56.725524+0000","last_undegraded":"2024-04-24T23:20:56.725524+0000","last_fullsized":"2024-04-24T23:20:56.725524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:42.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478041+0000","last_change":"2024-04-24T23:20:55.479475+0000","last_active":"2024-04-24T23:20:56.478041+0000","last_peered":"2024-04-24T23:20:56.478041+0000","last_clean":"2024-04-24T23:20:56.478041+0000","last_became_active":"2024-04-24T23:20:55.478984+0000","last_became_peered":"2024-04-24T23:20:55.478984+0000","last_unstale":"2024-04-24T23:20:56.478041+0000","last_undegraded":"2024-04-24T23:20:56.478041+0000","last_fullsized":"2024-04-24T23:20:56.478041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.479792+0000","last_change":"2024-04-24T23:20:54.478307+0000","last_active":"2024-04-24T23:20:56.479792+0000","last_peered":"2024-04-24T23:20:56.479792+0000","last_clean":"2024-04-24T23:20:56.479792+0000","last_became_active":"2024-04-24T23:20:54.477988+0000","last_became_peered":"2024-04-24T23:20:54.477988+0000","last_unstale":"2024-04-24T23:20:56.479792+0000","last_undegraded":"2024-04-24T23:20:56.479792+0000","last_fullsized":"2024-04-24T23:20:56.479792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725717+0000","last_change":"2024-04-24T23:20:55.487879+0000","last_active":"2024-04-24T23:20:56.725717+0000","last_peered":"2024-04-24T23:20:56.725717+0000","last_clean":"2024-04-24T23:20:56.725717+0000","last_became_active":"2024-04-24T23:20:55.487553+0000","last_became_peered":"2024-04-24T23:20:55.487553+0000","last_unstale":"2024-04-24T23:20:56.725717+0000","last_undegraded":"2024-04-24T23:20:56.725717+0000","last_fullsized":"2024-04-24T23:20:56.725717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480845+0000","last_change":"2024-04-24T23:20:54.480826+0000","last_active":"2024-04-24T23:20:56.480845+0000","last_peered":"2024-04-24T23:20:56.480845+0000","last_clean":"2024-04-24T23:20:56.480845+0000","last_became_active":"2024-04-24T23:20:54.480351+0000","last_became_peered":"2024-04-24T23:20:54.480351+0000","last_unstale":"2024-04-24T23:20:56.480845+0000","last_undegraded":"2024-04-24T23:20:56.480845+0000","last_fullsized":"2024-04-24T23:20:56.480845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.480939+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.478888+0000","last_became_peered":"2024-04-24T23:20:55.478888+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724748+0000","last_change":"2024-04-24T23:20:54.482329+0000","last_active":"2024-04-24T23:20:56.724748+0000","last_peered":"2024-04-24T23:20:56.724748+0000","last_clean":"2024-04-24T23:20:56.724748+0000","last_became_active":"2024-04-24T23:20:54.482059+0000","last_became_peered":"2024-04-24T23:20:54.482059+0000","last_unstale":"2024-04-24T23:20:56.724748+0000","last_undegraded":"2024-04-24T23:20:56.724748+0000","last_fullsized":"2024-04-24T23:20:56.724748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475091+0000","last_change":"2024-04-24T23:20:55.478742+0000","last_active":"2024-04-24T23:20:56.475091+0000","last_peered":"2024-04-24T23:20:56.475091+0000","last_clean":"2024-04-24T23:20:56.475091+0000","last_became_active":"2024-04-24T23:20:55.478465+0000","last_became_peered":"2024-04-24T23:20:55.478465+0000","last_unstale":"2024-04-24T23:20:56.475091+0000","last_undegraded":"2024-04-24T23:20:56.475091+0000","last_fullsized":"2024-04-24T23:20:56.475091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:32.641502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477976+0000","last_change":"2024-04-24T23:20:54.480255+0000","last_active":"2024-04-24T23:20:56.477976+0000","last_peered":"2024-04-24T23:20:56.477976+0000","last_clean":"2024-04-24T23:20:56.477976+0000","last_became_active":"2024-04-24T23:20:54.479897+0000","last_became_peered":"2024-04-24T23:20:54.479897+0000","last_unstale":"2024-04-24T23:20:56.477976+0000","last_undegraded":"2024-04-24T23:20:56.477976+0000","last_fullsized":"2024-04-24T23:20:56.477976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725278+0000","last_change":"2024-04-24T23:20:55.480538+0000","last_active":"2024-04-24T23:20:56.725278+0000","last_peered":"2024-04-24T23:20:56.725278+0000","last_clean":"2024-04-24T23:20:56.725278+0000","last_became_active":"2024-04-24T23:20:55.474763+0000","last_became_peered":"2024-04-24T23:20:55.474763+0000","last_unstale":"2024-04-24T23:20:56.725278+0000","last_undegraded":"2024-04-24T23:20:56.725278+0000","last_fullsized":"2024-04-24T23:20:56.725278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477945+0000","last_change":"2024-04-24T23:20:54.475154+0000","last_active":"2024-04-24T23:20:56.477945+0000","last_peered":"2024-04-24T23:20:56.477945+0000","last_clean":"2024-04-24T23:20:56.477945+0000","last_became_active":"2024-04-24T23:20:54.473925+0000","last_became_peered":"2024-04-24T23:20:54.473925+0000","last_unstale":"2024-04-24T23:20:56.477945+0000","last_undegraded":"2024-04-24T23:20:56.477945+0000","last_fullsized":"2024-04-24T23:20:56.477945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725566+0000","last_change":"2024-04-24T23:20:55.479171+0000","last_active":"2024-04-24T23:20:56.725566+0000","last_peered":"2024-04-24T23:20:56.725566+0000","last_clean":"2024-04-24T23:20:56.725566+0000","last_became_active":"2024-04-24T23:20:55.478856+0000","last_became_peered":"2024-04-24T23:20:55.478856+0000","last_unstale":"2024-04-24T23:20:56.725566+0000","last_undegraded":"2024-04-24T23:20:56.725566+0000","last_fullsized":"2024-04-24T23:20:56.725566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725243+0000","last_change":"2024-04-24T23:20:54.484143+0000","last_active":"2024-04-24T23:20:56.725243+0000","last_peered":"2024-04-24T23:20:56.725243+0000","last_clean":"2024-04-24T23:20:56.725243+0000","last_became_active":"2024-04-24T23:20:54.483830+0000","last_became_peered":"2024-04-24T23:20:54.483830+0000","last_unstale":"2024-04-24T23:20:56.725243+0000","last_undegraded":"2024-04-24T23:20:56.725243+0000","last_fullsized":"2024-04-24T23:20:56.725243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480446+0000","last_change":"2024-04-24T23:20:55.479005+0000","last_active":"2024-04-24T23:20:56.480446+0000","last_peered":"2024-04-24T23:20:56.480446+0000","last_clean":"2024-04-24T23:20:56.480446+0000","last_became_active":"2024-04-24T23:20:55.478706+0000","last_became_peered":"2024-04-24T23:20:55.478706+0000","last_unstale":"2024-04-24T23:20:56.480446+0000","last_undegraded":"2024-04-24T23:20:56.480446+0000","last_fullsized":"2024-04-24T23:20:56.480446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474978+0000","last_change":"2024-04-24T23:20:55.469771+0000","last_active":"2024-04-24T23:20:56.474978+0000","last_peered":"2024-04-24T23:20:56.474978+0000","last_clean":"2024-04-24T23:20:56.474978+0000","last_became_active":"2024-04-24T23:20:55.469411+0000","last_became_peered":"2024-04-24T23:20:55.469411+0000","last_unstale":"2024-04-24T23:20:56.474978+0000","last_undegraded":"2024-04-24T23:20:56.474978+0000","last_fullsized":"2024-04-24T23:20:56.474978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725118+0000","last_change":"2024-04-24T23:20:54.485231+0000","last_active":"2024-04-24T23:20:56.725118+0000","last_peered":"2024-04-24T23:20:56.725118+0000","last_clean":"2024-04-24T23:20:56.725118+0000","last_became_active":"2024-04-24T23:20:54.484912+0000","last_became_peered":"2024-04-24T23:20:54.484912+0000","last_unstale":"2024-04-24T23:20:56.725118+0000","last_undegraded":"2024-04-24T23:20:56.725118+0000","last_fullsized":"2024-04-24T23:20:56.725118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:27.147159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476107+0000","last_change":"2024-04-24T23:20:55.473157+0000","last_active":"2024-04-24T23:20:56.476107+0000","last_peered":"2024-04-24T23:20:56.476107+0000","last_clean":"2024-04-24T23:20:56.476107+0000","last_became_active":"2024-04-24T23:20:55.472854+0000","last_became_peered":"2024-04-24T23:20:55.472854+0000","last_unstale":"2024-04-24T23:20:56.476107+0000","last_undegraded":"2024-04-24T23:20:56.476107+0000","last_fullsized":"2024-04-24T23:20:56.476107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725345+0000","last_change":"2024-04-24T23:20:54.485353+0000","last_active":"2024-04-24T23:20:56.725345+0000","last_peered":"2024-04-24T23:20:56.725345+0000","last_clean":"2024-04-24T23:20:56.725345+0000","last_became_active":"2024-04-24T23:20:54.485065+0000","last_became_peered":"2024-04-24T23:20:54.485065+0000","last_unstale":"2024-04-24T23:20:56.725345+0000","last_undegraded":"2024-04-24T23:20:56.725345+0000","last_fullsized":"2024-04-24T23:20:56.725345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724430+0000","last_change":"2024-04-24T23:20:55.487656+0000","last_active":"2024-04-24T23:20:56.724430+0000","last_peered":"2024-04-24T23:20:56.724430+0000","last_clean":"2024-04-24T23:20:56.724430+0000","last_became_active":"2024-04-24T23:20:55.487317+0000","last_became_peered":"2024-04-24T23:20:55.487317+0000","last_unstale":"2024-04-24T23:20:56.724430+0000","last_undegraded":"2024-04-24T23:20:56.724430+0000","last_fullsized":"2024-04-24T23:20:56.724430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725671+0000","last_change":"2024-04-24T23:20:54.475528+0000","last_active":"2024-04-24T23:20:56.725671+0000","last_peered":"2024-04-24T23:20:56.725671+0000","last_clean":"2024-04-24T23:20:56.725671+0000","last_became_active":"2024-04-24T23:20:54.475227+0000","last_became_peered":"2024-04-24T23:20:54.475227+0000","last_unstale":"2024-04-24T23:20:56.725671+0000","last_undegraded":"2024-04-24T23:20:56.725671+0000","last_fullsized":"2024-04-24T23:20:56.725671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.481689+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.481405+0000","last_became_peered":"2024-04-24T23:20:55.481405+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724580+0000","last_change":"2024-04-24T23:20:54.482023+0000","last_active":"2024-04-24T23:20:56.724580+0000","last_peered":"2024-04-24T23:20:56.724580+0000","last_clean":"2024-04-24T23:20:56.724580+0000","last_became_active":"2024-04-24T23:20:54.481494+0000","last_became_peered":"2024-04-24T23:20:54.481494+0000","last_unstale":"2024-04-24T23:20:56.724580+0000","last_undegraded":"2024-04-24T23:20:56.724580+0000","last_fullsized":"2024-04-24T23:20:56.724580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35528,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336312,"statfs":{"total":96636764160,"available":96600383488,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:21:03.262 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:21:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.374+0000 7f8ce289e640 1 Processor -- start 2024-04-24T23:21:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.374+0000 7f8ce289e640 1 -- start start 2024-04-24T23:21:03.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8ce289e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc143c40 0x7f8cdc0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8ce289e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8cdc0cc860 0x7f8cdc072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8ce289e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cdc0cd0b0 con 0x7f8cdc15eb00 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8ce289e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8cdc071b70 con 0x7f8cdc0cc860 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8ce289e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc071240 con 0x7f8cdc143c40 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8ce0e3b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8cdc15eb00 0x7f8cdc0b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46326/0 (socket says 172.21.15.26:46326) 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8ce0e3b640 1 -- 172.21.15.26:0/299630580 learned_addr learned my addr 172.21.15.26:0/299630580 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8cdb7fe640 1 --2- 172.21.15.26:0/299630580 >> v2:172.21.15.29:3300/0 conn(0x7f8cdc0cc860 0x7f8cdc072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8cdbfff640 1 --2- 172.21.15.26:0/299630580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc143c40 0x7f8cdc0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8cdaffd640 1 -- 172.21.15.26:0/299630580 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1319076385 0 0) 0x7f8cdc0cd0b0 con 0x7f8cdc15eb00 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.375+0000 7f8cdaffd640 1 -- 172.21.15.26:0/299630580 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cc8005730 con 0x7f8cdc15eb00 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.376+0000 7f8cdaffd640 1 -- 172.21.15.26:0/299630580 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3602244989 0 0) 0x7f8cc8005730 con 0x7f8cdc15eb00 2024-04-24T23:21:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.376+0000 7f8cdb7fe640 1 -- 172.21.15.26:0/299630580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc143c40 msgr2=0x7f8cdc0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.376+0000 7f8cdb7fe640 1 --2- 172.21.15.26:0/299630580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc143c40 0x7f8cdc0cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.376+0000 7f8cdb7fe640 1 -- 172.21.15.26:0/299630580 >> v1:172.21.15.26:6789/0 conn(0x7f8cdc15eb00 legacy=0x7f8cdc0b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.376+0000 7f8cdb7fe640 1 -- 172.21.15.26:0/299630580 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc071520 con 0x7f8cdc0cc860 2024-04-24T23:21:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.376+0000 7f8cdbfff640 1 --2- 172.21.15.26:0/299630580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc143c40 0x7f8cdc0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.376+0000 7f8cdb7fe640 1 --2- 172.21.15.26:0/299630580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cdc0cc860 0x7f8cdc072000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0097a0 tx=0x7f8ccc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d542df7ee119ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8cdaffd640 1 -- 172.21.15.26:0/299630580 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc037070 con 0x7f8cdc0cc860 2024-04-24T23:21:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8cdaffd640 1 -- 172.21.15.26:0/299630580 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ccc02aba0 con 0x7f8cdc0cc860 2024-04-24T23:21:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8cdaffd640 1 -- 172.21.15.26:0/299630580 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40019f0 con 0x7f8cdc0cc860 2024-04-24T23:21:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8ce289e640 1 -- 172.21.15.26:0/299630580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cdc0cc860 msgr2=0x7f8cdc072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8ce289e640 1 --2- 172.21.15.26:0/299630580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cdc0cc860 0x7f8cdc072000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0097a0 tx=0x7f8ccc02a110 comp rx=0 tx=0).stop 2024-04-24T23:21:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8ce289e640 1 -- 172.21.15.26:0/299630580 shutdown_connections 2024-04-24T23:21:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8ce289e640 1 --2- 172.21.15.26:0/299630580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cdc0cc860 0x7f8cdc072000 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8ce289e640 1 --2- 172.21.15.26:0/299630580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc143c40 0x7f8cdc0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8ce289e640 1 -- 172.21.15.26:0/299630580 >> 172.21.15.26:0/299630580 conn(0x7f8cdc06b300 msgr2=0x7f8cdc06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8ce289e640 1 -- 172.21.15.26:0/299630580 shutdown_connections 2024-04-24T23:21:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.377+0000 7f8ce289e640 1 -- 172.21.15.26:0/299630580 wait complete. 2024-04-24T23:21:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.378+0000 7f8ce289e640 1 Processor -- start 2024-04-24T23:21:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.379+0000 7f8ce289e640 1 -- start start 2024-04-24T23:21:03.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.379+0000 7f8ce289e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cdc0cc860 0x7f8cdc13d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.379+0000 7f8ce289e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cdc143c40 0x7f8cdc13e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.379+0000 7f8ce289e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc15eb00 0x7f8cdc140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.379+0000 7f8ce289e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc072540 con 0x7f8cdc143c40 2024-04-24T23:21:03.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.379+0000 7f8ce289e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc144020 con 0x7f8cdc0cc860 2024-04-24T23:21:03.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.379+0000 7f8ce289e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc13dc20 con 0x7f8cdc15eb00 2024-04-24T23:21:03.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.379+0000 7f8cdb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cdc143c40 0x7f8cdc13e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cdb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cdc143c40 0x7f8cdc13e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54670/0 (socket says 172.21.15.26:54670) 2024-04-24T23:21:03.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cdb7fe640 1 -- 172.21.15.26:0/2447995598 learned_addr learned my addr 172.21.15.26:0/2447995598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8ce0e3b640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc15eb00 0x7f8cdc140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cdbfff640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cdc0cc860 0x7f8cdc13d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cdb7fe640 1 -- 172.21.15.26:0/2447995598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc15eb00 msgr2=0x7f8cdc140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cdb7fe640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc15eb00 0x7f8cdc140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cdb7fe640 1 -- 172.21.15.26:0/2447995598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cdc0cc860 msgr2=0x7f8cdc13d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cdb7fe640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cdc0cc860 0x7f8cdc13d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cdb7fe640 1 -- 172.21.15.26:0/2447995598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc13df00 con 0x7f8cdc143c40 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cdb7fe640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cdc143c40 0x7f8cdc13e9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc035040 tx=0x7f8ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cd8ff9640 1 -- 172.21.15.26:0/2447995598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc041020 con 0x7f8cdc143c40 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cd8ff9640 1 -- 172.21.15.26:0/2447995598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ccc03d620 con 0x7f8cdc143c40 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8cd8ff9640 1 -- 172.21.15.26:0/2447995598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc037040 con 0x7f8cdc143c40 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc13e1e0 con 0x7f8cdc143c40 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.380+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc141580 con 0x7f8cdc143c40 2024-04-24T23:21:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.381+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc0694c0 con 0x7f8cdc143c40 2024-04-24T23:21:03.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.382+0000 7f8cd8ff9640 1 -- 172.21.15.26:0/2447995598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ccc044020 con 0x7f8cdc143c40 2024-04-24T23:21:03.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.382+0000 7f8cd8ff9640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8cbc06be30 0x7f8cbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.382+0000 7f8cd8ff9640 1 -- 172.21.15.26:0/2447995598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ccc0ab160 con 0x7f8cdc143c40 2024-04-24T23:21:03.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.382+0000 7f8cdbfff640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8cbc06be30 0x7f8cbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.383+0000 7f8cdbfff640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8cbc06be30 0x7f8cbc06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009560 tx=0x7f8cc4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.387+0000 7f8cd8ff9640 1 -- 172.21.15.26:0/2447995598 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ccc04b030 con 0x7f8cdc143c40 2024-04-24T23:21:03.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.573+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8cdc0b4d20 con 0x7f8cbc06be30 2024-04-24T23:21:03.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.579+0000 7f8cd8ff9640 1 -- 172.21.15.26:0/2447995598 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f8cdc0b4d20 con 0x7f8cbc06be30 2024-04-24T23:21:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:03.585 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8cbc06be30 msgr2=0x7f8cbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8cbc06be30 0x7f8cbc06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009560 tx=0x7f8cc4009190 comp rx=0 tx=0).stop 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cdc143c40 msgr2=0x7f8cdc13e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cdc143c40 0x7f8cdc13e9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc035040 tx=0x7f8ccc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 shutdown_connections 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8cbc06be30 0x7f8cbc06e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8cdc15eb00 0x7f8cdc140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cdc143c40 0x7f8cdc13e9e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 --2- 172.21.15.26:0/2447995598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8cdc0cc860 0x7f8cdc13d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.582+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 >> 172.21.15.26:0/2447995598 conn(0x7f8cdc06b300 msgr2=0x7f8cdc13f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:03.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.583+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 shutdown_connections 2024-04-24T23:21:03.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.583+0000 7f8ce289e640 1 -- 172.21.15.26:0/2447995598 wait complete. 2024-04-24T23:21:03.595 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-24T23:21:03.316900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256448,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718272,"statfs":{"total":773094113280,"available":772831510528,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001727"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725536+0000","last_change":"2024-04-24T23:20:55.487607+0000","last_active":"2024-04-24T23:20:56.725536+0000","last_peered":"2024-04-24T23:20:56.725536+0000","last_clean":"2024-04-24T23:20:56.725536+0000","last_became_active":"2024-04-24T23:20:55.487320+0000","last_became_peered":"2024-04-24T23:20:55.487320+0000","last_unstale":"2024-04-24T23:20:56.725536+0000","last_undegraded":"2024-04-24T23:20:56.725536+0000","last_fullsized":"2024-04-24T23:20:56.725536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:27.675103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478125+0000","last_change":"2024-04-24T23:20:55.480991+0000","last_active":"2024-04-24T23:20:56.478125+0000","last_peered":"2024-04-24T23:20:56.478125+0000","last_clean":"2024-04-24T23:20:56.478125+0000","last_became_active":"2024-04-24T23:20:55.480654+0000","last_became_peered":"2024-04-24T23:20:55.480654+0000","last_unstale":"2024-04-24T23:20:56.478125+0000","last_undegraded":"2024-04-24T23:20:56.478125+0000","last_fullsized":"2024-04-24T23:20:56.478125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724519+0000","last_change":"2024-04-24T23:20:55.479402+0000","last_active":"2024-04-24T23:20:56.724519+0000","last_peered":"2024-04-24T23:20:56.724519+0000","last_clean":"2024-04-24T23:20:56.724519+0000","last_became_active":"2024-04-24T23:20:55.479013+0000","last_became_peered":"2024-04-24T23:20:55.479013+0000","last_unstale":"2024-04-24T23:20:56.724519+0000","last_undegraded":"2024-04-24T23:20:56.724519+0000","last_fullsized":"2024-04-24T23:20:56.724519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475989+0000","last_change":"2024-04-24T23:20:55.481507+0000","last_active":"2024-04-24T23:20:56.475989+0000","last_peered":"2024-04-24T23:20:56.475989+0000","last_clean":"2024-04-24T23:20:56.475989+0000","last_became_active":"2024-04-24T23:20:55.481144+0000","last_became_peered":"2024-04-24T23:20:55.481144+0000","last_unstale":"2024-04-24T23:20:56.475989+0000","last_undegraded":"2024-04-24T23:20:56.475989+0000","last_fullsized":"2024-04-24T23:20:56.475989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725067+0000","last_change":"2024-04-24T23:20:55.482367+0000","last_active":"2024-04-24T23:20:56.725067+0000","last_peered":"2024-04-24T23:20:56.725067+0000","last_clean":"2024-04-24T23:20:56.725067+0000","last_became_active":"2024-04-24T23:20:55.482047+0000","last_became_peered":"2024-04-24T23:20:55.482047+0000","last_unstale":"2024-04-24T23:20:56.725067+0000","last_undegraded":"2024-04-24T23:20:56.725067+0000","last_fullsized":"2024-04-24T23:20:56.725067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475443+0000","last_change":"2024-04-24T23:20:55.482474+0000","last_active":"2024-04-24T23:20:56.475443+0000","last_peered":"2024-04-24T23:20:56.475443+0000","last_clean":"2024-04-24T23:20:56.475443+0000","last_became_active":"2024-04-24T23:20:55.482157+0000","last_became_peered":"2024-04-24T23:20:55.482157+0000","last_unstale":"2024-04-24T23:20:56.475443+0000","last_undegraded":"2024-04-24T23:20:56.475443+0000","last_fullsized":"2024-04-24T23:20:56.475443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724980+0000","last_change":"2024-04-24T23:20:55.487389+0000","last_active":"2024-04-24T23:20:56.724980+0000","last_peered":"2024-04-24T23:20:56.724980+0000","last_clean":"2024-04-24T23:20:56.724980+0000","last_became_active":"2024-04-24T23:20:55.486724+0000","last_became_peered":"2024-04-24T23:20:55.486724+0000","last_unstale":"2024-04-24T23:20:56.724980+0000","last_undegraded":"2024-04-24T23:20:56.724980+0000","last_fullsized":"2024-04-24T23:20:56.724980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724916+0000","last_change":"2024-04-24T23:20:55.486339+0000","last_active":"2024-04-24T23:20:56.724916+0000","last_peered":"2024-04-24T23:20:56.724916+0000","last_clean":"2024-04-24T23:20:56.724916+0000","last_became_active":"2024-04-24T23:20:55.486072+0000","last_became_peered":"2024-04-24T23:20:55.486072+0000","last_unstale":"2024-04-24T23:20:56.724916+0000","last_undegraded":"2024-04-24T23:20:56.724916+0000","last_fullsized":"2024-04-24T23:20:56.724916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.491641+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.491349+0000","last_became_peered":"2024-04-24T23:20:55.491349+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:58.298429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480758+0000","last_change":"2024-04-24T23:20:55.479221+0000","last_active":"2024-04-24T23:20:56.480758+0000","last_peered":"2024-04-24T23:20:56.480758+0000","last_clean":"2024-04-24T23:20:56.480758+0000","last_became_active":"2024-04-24T23:20:55.478795+0000","last_became_peered":"2024-04-24T23:20:55.478795+0000","last_unstale":"2024-04-24T23:20:56.480758+0000","last_undegraded":"2024-04-24T23:20:56.480758+0000","last_fullsized":"2024-04-24T23:20:56.480758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480763+0000","last_change":"2024-04-24T23:20:55.479615+0000","last_active":"2024-04-24T23:20:56.480763+0000","last_peered":"2024-04-24T23:20:56.480763+0000","last_clean":"2024-04-24T23:20:56.480763+0000","last_became_active":"2024-04-24T23:20:55.479317+0000","last_became_peered":"2024-04-24T23:20:55.479317+0000","last_unstale":"2024-04-24T23:20:56.480763+0000","last_undegraded":"2024-04-24T23:20:56.480763+0000","last_fullsized":"2024-04-24T23:20:56.480763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475028+0000","last_change":"2024-04-24T23:20:55.481915+0000","last_active":"2024-04-24T23:20:56.475028+0000","last_peered":"2024-04-24T23:20:56.475028+0000","last_clean":"2024-04-24T23:20:56.475028+0000","last_became_active":"2024-04-24T23:20:55.481596+0000","last_became_peered":"2024-04-24T23:20:55.481596+0000","last_unstale":"2024-04-24T23:20:56.475028+0000","last_undegraded":"2024-04-24T23:20:56.475028+0000","last_fullsized":"2024-04-24T23:20:56.475028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:20.844340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725493+0000","last_change":"2024-04-24T23:20:55.476416+0000","last_active":"2024-04-24T23:20:56.725493+0000","last_peered":"2024-04-24T23:20:56.725493+0000","last_clean":"2024-04-24T23:20:56.725493+0000","last_became_active":"2024-04-24T23:20:55.476068+0000","last_became_peered":"2024-04-24T23:20:55.476068+0000","last_unstale":"2024-04-24T23:20:56.725493+0000","last_undegraded":"2024-04-24T23:20:56.725493+0000","last_fullsized":"2024-04-24T23:20:56.725493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724195+0000","last_change":"2024-04-24T23:20:55.479125+0000","last_active":"2024-04-24T23:20:56.724195+0000","last_peered":"2024-04-24T23:20:56.724195+0000","last_clean":"2024-04-24T23:20:56.724195+0000","last_became_active":"2024-04-24T23:20:55.478473+0000","last_became_peered":"2024-04-24T23:20:55.478473+0000","last_unstale":"2024-04-24T23:20:56.724195+0000","last_undegraded":"2024-04-24T23:20:56.724195+0000","last_fullsized":"2024-04-24T23:20:56.724195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:19.475113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477868+0000","last_change":"2024-04-24T23:20:55.474883+0000","last_active":"2024-04-24T23:20:56.477868+0000","last_peered":"2024-04-24T23:20:56.477868+0000","last_clean":"2024-04-24T23:20:56.477868+0000","last_became_active":"2024-04-24T23:20:55.474459+0000","last_became_peered":"2024-04-24T23:20:55.474459+0000","last_unstale":"2024-04-24T23:20:56.477868+0000","last_undegraded":"2024-04-24T23:20:56.477868+0000","last_fullsized":"2024-04-24T23:20:56.477868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475040+0000","last_change":"2024-04-24T23:20:55.483511+0000","last_active":"2024-04-24T23:20:56.475040+0000","last_peered":"2024-04-24T23:20:56.475040+0000","last_clean":"2024-04-24T23:20:56.475040+0000","last_became_active":"2024-04-24T23:20:55.483171+0000","last_became_peered":"2024-04-24T23:20:55.483171+0000","last_unstale":"2024-04-24T23:20:56.475040+0000","last_undegraded":"2024-04-24T23:20:56.475040+0000","last_fullsized":"2024-04-24T23:20:56.475040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475537+0000","last_change":"2024-04-24T23:20:55.489069+0000","last_active":"2024-04-24T23:20:56.475537+0000","last_peered":"2024-04-24T23:20:56.475537+0000","last_clean":"2024-04-24T23:20:56.475537+0000","last_became_active":"2024-04-24T23:20:55.488796+0000","last_became_peered":"2024-04-24T23:20:55.488796+0000","last_unstale":"2024-04-24T23:20:56.475537+0000","last_undegraded":"2024-04-24T23:20:56.475537+0000","last_fullsized":"2024-04-24T23:20:56.475537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724755+0000","last_change":"2024-04-24T23:20:55.480920+0000","last_active":"2024-04-24T23:20:56.724755+0000","last_peered":"2024-04-24T23:20:56.724755+0000","last_clean":"2024-04-24T23:20:56.724755+0000","last_became_active":"2024-04-24T23:20:55.480581+0000","last_became_peered":"2024-04-24T23:20:55.480581+0000","last_unstale":"2024-04-24T23:20:56.724755+0000","last_undegraded":"2024-04-24T23:20:56.724755+0000","last_fullsized":"2024-04-24T23:20:56.724755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725520+0000","last_change":"2024-04-24T23:20:55.488901+0000","last_active":"2024-04-24T23:20:56.725520+0000","last_peered":"2024-04-24T23:20:56.725520+0000","last_clean":"2024-04-24T23:20:56.725520+0000","last_became_active":"2024-04-24T23:20:55.488647+0000","last_became_peered":"2024-04-24T23:20:55.488647+0000","last_unstale":"2024-04-24T23:20:56.725520+0000","last_undegraded":"2024-04-24T23:20:56.725520+0000","last_fullsized":"2024-04-24T23:20:56.725520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480674+0000","last_change":"2024-04-24T23:20:55.476826+0000","last_active":"2024-04-24T23:20:56.480674+0000","last_peered":"2024-04-24T23:20:56.480674+0000","last_clean":"2024-04-24T23:20:56.480674+0000","last_became_active":"2024-04-24T23:20:55.476481+0000","last_became_peered":"2024-04-24T23:20:55.476481+0000","last_unstale":"2024-04-24T23:20:56.480674+0000","last_undegraded":"2024-04-24T23:20:56.480674+0000","last_fullsized":"2024-04-24T23:20:56.480674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724349+0000","last_change":"2024-04-24T23:20:55.491072+0000","last_active":"2024-04-24T23:20:56.724349+0000","last_peered":"2024-04-24T23:20:56.724349+0000","last_clean":"2024-04-24T23:20:56.724349+0000","last_became_active":"2024-04-24T23:20:55.490763+0000","last_became_peered":"2024-04-24T23:20:55.490763+0000","last_unstale":"2024-04-24T23:20:56.724349+0000","last_undegraded":"2024-04-24T23:20:56.724349+0000","last_fullsized":"2024-04-24T23:20:56.724349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475836+0000","last_change":"2024-04-24T23:20:55.487623+0000","last_active":"2024-04-24T23:20:56.475836+0000","last_peered":"2024-04-24T23:20:56.475836+0000","last_clean":"2024-04-24T23:20:56.475836+0000","last_became_active":"2024-04-24T23:20:55.487355+0000","last_became_peered":"2024-04-24T23:20:55.487355+0000","last_unstale":"2024-04-24T23:20:56.475836+0000","last_undegraded":"2024-04-24T23:20:56.475836+0000","last_fullsized":"2024-04-24T23:20:56.475836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725470+0000","last_change":"2024-04-24T23:20:55.476152+0000","last_active":"2024-04-24T23:20:56.725470+0000","last_peered":"2024-04-24T23:20:56.725470+0000","last_clean":"2024-04-24T23:20:56.725470+0000","last_became_active":"2024-04-24T23:20:55.475857+0000","last_became_peered":"2024-04-24T23:20:55.475857+0000","last_unstale":"2024-04-24T23:20:56.725470+0000","last_undegraded":"2024-04-24T23:20:56.725470+0000","last_fullsized":"2024-04-24T23:20:56.725470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475501+0000","last_change":"2024-04-24T23:20:55.484268+0000","last_active":"2024-04-24T23:20:56.475501+0000","last_peered":"2024-04-24T23:20:56.475501+0000","last_clean":"2024-04-24T23:20:56.475501+0000","last_became_active":"2024-04-24T23:20:55.483972+0000","last_became_peered":"2024-04-24T23:20:55.483972+0000","last_unstale":"2024-04-24T23:20:56.475501+0000","last_undegraded":"2024-04-24T23:20:56.475501+0000","last_fullsized":"2024-04-24T23:20:56.475501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725722+0000","last_change":"2024-04-24T23:20:55.480092+0000","last_active":"2024-04-24T23:20:56.725722+0000","last_peered":"2024-04-24T23:20:56.725722+0000","last_clean":"2024-04-24T23:20:56.725722+0000","last_became_active":"2024-04-24T23:20:55.479783+0000","last_became_peered":"2024-04-24T23:20:55.479783+0000","last_unstale":"2024-04-24T23:20:56.725722+0000","last_undegraded":"2024-04-24T23:20:56.725722+0000","last_fullsized":"2024-04-24T23:20:56.725722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480830+0000","last_change":"2024-04-24T23:20:55.488050+0000","last_active":"2024-04-24T23:20:56.480830+0000","last_peered":"2024-04-24T23:20:56.480830+0000","last_clean":"2024-04-24T23:20:56.480830+0000","last_became_active":"2024-04-24T23:20:55.487762+0000","last_became_peered":"2024-04-24T23:20:55.487762+0000","last_unstale":"2024-04-24T23:20:56.480830+0000","last_undegraded":"2024-04-24T23:20:56.480830+0000","last_fullsized":"2024-04-24T23:20:56.480830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478091+0000","last_change":"2024-04-24T23:20:55.478530+0000","last_active":"2024-04-24T23:20:56.478091+0000","last_peered":"2024-04-24T23:20:56.478091+0000","last_clean":"2024-04-24T23:20:56.478091+0000","last_became_active":"2024-04-24T23:20:55.478233+0000","last_became_peered":"2024-04-24T23:20:55.478233+0000","last_unstale":"2024-04-24T23:20:56.478091+0000","last_undegraded":"2024-04-24T23:20:56.478091+0000","last_fullsized":"2024-04-24T23:20:56.478091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725374+0000","last_change":"2024-04-24T23:20:55.480077+0000","last_active":"2024-04-24T23:20:56.725374+0000","last_peered":"2024-04-24T23:20:56.725374+0000","last_clean":"2024-04-24T23:20:56.725374+0000","last_became_active":"2024-04-24T23:20:55.479769+0000","last_became_peered":"2024-04-24T23:20:55.479769+0000","last_unstale":"2024-04-24T23:20:56.725374+0000","last_undegraded":"2024-04-24T23:20:56.725374+0000","last_fullsized":"2024-04-24T23:20:56.725374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724513+0000","last_change":"2024-04-24T23:20:55.481475+0000","last_active":"2024-04-24T23:20:56.724513+0000","last_peered":"2024-04-24T23:20:56.724513+0000","last_clean":"2024-04-24T23:20:56.724513+0000","last_became_active":"2024-04-24T23:20:55.480091+0000","last_became_peered":"2024-04-24T23:20:55.480091+0000","last_unstale":"2024-04-24T23:20:56.724513+0000","last_undegraded":"2024-04-24T23:20:56.724513+0000","last_fullsized":"2024-04-24T23:20:56.724513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476229+0000","last_change":"2024-04-24T23:20:55.485092+0000","last_active":"2024-04-24T23:20:56.476229+0000","last_peered":"2024-04-24T23:20:56.476229+0000","last_clean":"2024-04-24T23:20:56.476229+0000","last_became_active":"2024-04-24T23:20:55.484409+0000","last_became_peered":"2024-04-24T23:20:55.484409+0000","last_unstale":"2024-04-24T23:20:56.476229+0000","last_undegraded":"2024-04-24T23:20:56.476229+0000","last_fullsized":"2024-04-24T23:20:56.476229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.469291+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.468961+0000","last_became_peered":"2024-04-24T23:20:55.468961+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480450+0000","last_change":"2024-04-24T23:20:55.484826+0000","last_active":"2024-04-24T23:20:56.480450+0000","last_peered":"2024-04-24T23:20:56.480450+0000","last_clean":"2024-04-24T23:20:56.480450+0000","last_became_active":"2024-04-24T23:20:55.484500+0000","last_became_peered":"2024-04-24T23:20:55.484500+0000","last_unstale":"2024-04-24T23:20:56.480450+0000","last_undegraded":"2024-04-24T23:20:56.480450+0000","last_fullsized":"2024-04-24T23:20:56.480450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475480+0000","last_change":"2024-04-24T23:20:55.476006+0000","last_active":"2024-04-24T23:20:56.475480+0000","last_peered":"2024-04-24T23:20:56.475480+0000","last_clean":"2024-04-24T23:20:56.475480+0000","last_became_active":"2024-04-24T23:20:55.475090+0000","last_became_peered":"2024-04-24T23:20:55.475090+0000","last_unstale":"2024-04-24T23:20:56.475480+0000","last_undegraded":"2024-04-24T23:20:56.475480+0000","last_fullsized":"2024-04-24T23:20:56.475480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725560+0000","last_change":"2024-04-24T23:20:54.481863+0000","last_active":"2024-04-24T23:20:56.725560+0000","last_peered":"2024-04-24T23:20:56.725560+0000","last_clean":"2024-04-24T23:20:56.725560+0000","last_became_active":"2024-04-24T23:20:54.481498+0000","last_became_peered":"2024-04-24T23:20:54.481498+0000","last_unstale":"2024-04-24T23:20:56.725560+0000","last_undegraded":"2024-04-24T23:20:56.725560+0000","last_fullsized":"2024-04-24T23:20:56.725560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724565+0000","last_change":"2024-04-24T23:20:55.478651+0000","last_active":"2024-04-24T23:20:56.724565+0000","last_peered":"2024-04-24T23:20:56.724565+0000","last_clean":"2024-04-24T23:20:56.724565+0000","last_became_active":"2024-04-24T23:20:55.477783+0000","last_became_peered":"2024-04-24T23:20:55.477783+0000","last_unstale":"2024-04-24T23:20:56.724565+0000","last_undegraded":"2024-04-24T23:20:56.724565+0000","last_fullsized":"2024-04-24T23:20:56.724565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480400+0000","last_change":"2024-04-24T23:20:54.477560+0000","last_active":"2024-04-24T23:20:56.480400+0000","last_peered":"2024-04-24T23:20:56.480400+0000","last_clean":"2024-04-24T23:20:56.480400+0000","last_became_active":"2024-04-24T23:20:54.477261+0000","last_became_peered":"2024-04-24T23:20:54.477261+0000","last_unstale":"2024-04-24T23:20:56.480400+0000","last_undegraded":"2024-04-24T23:20:56.480400+0000","last_fullsized":"2024-04-24T23:20:56.480400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725154+0000","last_change":"2024-04-24T23:20:54.479540+0000","last_active":"2024-04-24T23:20:56.725154+0000","last_peered":"2024-04-24T23:20:56.725154+0000","last_clean":"2024-04-24T23:20:56.725154+0000","last_became_active":"2024-04-24T23:20:54.479206+0000","last_became_peered":"2024-04-24T23:20:54.479206+0000","last_unstale":"2024-04-24T23:20:56.725154+0000","last_undegraded":"2024-04-24T23:20:56.725154+0000","last_fullsized":"2024-04-24T23:20:56.725154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725091+0000","last_change":"2024-04-24T23:20:55.484927+0000","last_active":"2024-04-24T23:20:56.725091+0000","last_peered":"2024-04-24T23:20:56.725091+0000","last_clean":"2024-04-24T23:20:56.725091+0000","last_became_active":"2024-04-24T23:20:55.484611+0000","last_became_peered":"2024-04-24T23:20:55.484611+0000","last_unstale":"2024-04-24T23:20:56.725091+0000","last_undegraded":"2024-04-24T23:20:56.725091+0000","last_fullsized":"2024-04-24T23:20:56.725091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475910+0000","last_change":"2024-04-24T23:20:54.471358+0000","last_active":"2024-04-24T23:20:56.475910+0000","last_peered":"2024-04-24T23:20:56.475910+0000","last_clean":"2024-04-24T23:20:56.475910+0000","last_became_active":"2024-04-24T23:20:54.470800+0000","last_became_peered":"2024-04-24T23:20:54.470800+0000","last_unstale":"2024-04-24T23:20:56.475910+0000","last_undegraded":"2024-04-24T23:20:56.475910+0000","last_fullsized":"2024-04-24T23:20:56.475910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.477535+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.477215+0000","last_became_peered":"2024-04-24T23:20:55.477215+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480761+0000","last_change":"2024-04-24T23:20:54.479450+0000","last_active":"2024-04-24T23:20:56.480761+0000","last_peered":"2024-04-24T23:20:56.480761+0000","last_clean":"2024-04-24T23:20:56.480761+0000","last_became_active":"2024-04-24T23:20:54.479174+0000","last_became_peered":"2024-04-24T23:20:54.479174+0000","last_unstale":"2024-04-24T23:20:56.480761+0000","last_undegraded":"2024-04-24T23:20:56.480761+0000","last_fullsized":"2024-04-24T23:20:56.480761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476156+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.476156+0000","last_peered":"2024-04-24T23:20:56.476156+0000","last_clean":"2024-04-24T23:20:56.476156+0000","last_became_active":"2024-04-24T23:20:55.488014+0000","last_became_peered":"2024-04-24T23:20:55.488014+0000","last_unstale":"2024-04-24T23:20:56.476156+0000","last_undegraded":"2024-04-24T23:20:56.476156+0000","last_fullsized":"2024-04-24T23:20:56.476156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476129+0000","last_change":"2024-04-24T23:20:54.480387+0000","last_active":"2024-04-24T23:20:56.476129+0000","last_peered":"2024-04-24T23:20:56.476129+0000","last_clean":"2024-04-24T23:20:56.476129+0000","last_became_active":"2024-04-24T23:20:54.480037+0000","last_became_peered":"2024-04-24T23:20:54.480037+0000","last_unstale":"2024-04-24T23:20:56.476129+0000","last_undegraded":"2024-04-24T23:20:56.476129+0000","last_fullsized":"2024-04-24T23:20:56.476129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724464+0000","last_change":"2024-04-24T23:20:55.481206+0000","last_active":"2024-04-24T23:20:56.724464+0000","last_peered":"2024-04-24T23:20:56.724464+0000","last_clean":"2024-04-24T23:20:56.724464+0000","last_became_active":"2024-04-24T23:20:55.478018+0000","last_became_peered":"2024-04-24T23:20:55.478018+0000","last_unstale":"2024-04-24T23:20:56.724464+0000","last_undegraded":"2024-04-24T23:20:56.724464+0000","last_fullsized":"2024-04-24T23:20:56.724464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478112+0000","last_change":"2024-04-24T23:20:54.469557+0000","last_active":"2024-04-24T23:20:56.478112+0000","last_peered":"2024-04-24T23:20:56.478112+0000","last_clean":"2024-04-24T23:20:56.478112+0000","last_became_active":"2024-04-24T23:20:54.469251+0000","last_became_peered":"2024-04-24T23:20:54.469251+0000","last_unstale":"2024-04-24T23:20:56.478112+0000","last_undegraded":"2024-04-24T23:20:56.478112+0000","last_fullsized":"2024-04-24T23:20:56.478112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475184+0000","last_change":"2024-04-24T23:20:55.491797+0000","last_active":"2024-04-24T23:20:56.475184+0000","last_peered":"2024-04-24T23:20:56.475184+0000","last_clean":"2024-04-24T23:20:56.475184+0000","last_became_active":"2024-04-24T23:20:55.491453+0000","last_became_peered":"2024-04-24T23:20:55.491453+0000","last_unstale":"2024-04-24T23:20:56.475184+0000","last_undegraded":"2024-04-24T23:20:56.475184+0000","last_fullsized":"2024-04-24T23:20:56.475184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:05.444694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480475+0000","last_change":"2024-04-24T23:20:54.482472+0000","last_active":"2024-04-24T23:20:56.480475+0000","last_peered":"2024-04-24T23:20:56.480475+0000","last_clean":"2024-04-24T23:20:56.480475+0000","last_became_active":"2024-04-24T23:20:54.481351+0000","last_became_peered":"2024-04-24T23:20:54.481351+0000","last_unstale":"2024-04-24T23:20:56.480475+0000","last_undegraded":"2024-04-24T23:20:56.480475+0000","last_fullsized":"2024-04-24T23:20:56.480475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725210+0000","last_change":"2024-04-24T23:20:55.483153+0000","last_active":"2024-04-24T23:20:56.725210+0000","last_peered":"2024-04-24T23:20:56.725210+0000","last_clean":"2024-04-24T23:20:56.725210+0000","last_became_active":"2024-04-24T23:20:55.482859+0000","last_became_peered":"2024-04-24T23:20:55.482859+0000","last_unstale":"2024-04-24T23:20:56.725210+0000","last_undegraded":"2024-04-24T23:20:56.725210+0000","last_fullsized":"2024-04-24T23:20:56.725210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724624+0000","last_change":"2024-04-24T23:20:54.478089+0000","last_active":"2024-04-24T23:20:56.724624+0000","last_peered":"2024-04-24T23:20:56.724624+0000","last_clean":"2024-04-24T23:20:56.724624+0000","last_became_active":"2024-04-24T23:20:54.477302+0000","last_became_peered":"2024-04-24T23:20:54.477302+0000","last_unstale":"2024-04-24T23:20:56.724624+0000","last_undegraded":"2024-04-24T23:20:56.724624+0000","last_fullsized":"2024-04-24T23:20:56.724624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724289+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.724289+0000","last_peered":"2024-04-24T23:20:56.724289+0000","last_clean":"2024-04-24T23:20:56.724289+0000","last_became_active":"2024-04-24T23:20:55.488016+0000","last_became_peered":"2024-04-24T23:20:55.488016+0000","last_unstale":"2024-04-24T23:20:56.724289+0000","last_undegraded":"2024-04-24T23:20:56.724289+0000","last_fullsized":"2024-04-24T23:20:56.724289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:48.795181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476045+0000","last_change":"2024-04-24T23:20:54.473922+0000","last_active":"2024-04-24T23:20:56.476045+0000","last_peered":"2024-04-24T23:20:56.476045+0000","last_clean":"2024-04-24T23:20:56.476045+0000","last_became_active":"2024-04-24T23:20:54.473601+0000","last_became_peered":"2024-04-24T23:20:54.473601+0000","last_unstale":"2024-04-24T23:20:56.476045+0000","last_undegraded":"2024-04-24T23:20:56.476045+0000","last_fullsized":"2024-04-24T23:20:56.476045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725128+0000","last_change":"2024-04-24T23:20:55.488225+0000","last_active":"2024-04-24T23:20:56.725128+0000","last_peered":"2024-04-24T23:20:56.725128+0000","last_clean":"2024-04-24T23:20:56.725128+0000","last_became_active":"2024-04-24T23:20:55.485663+0000","last_became_peered":"2024-04-24T23:20:55.485663+0000","last_unstale":"2024-04-24T23:20:56.725128+0000","last_undegraded":"2024-04-24T23:20:56.725128+0000","last_fullsized":"2024-04-24T23:20:56.725128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:54.478237+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:54.477943+0000","last_became_peered":"2024-04-24T23:20:54.477943+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:55.484884+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:55.484578+0000","last_became_peered":"2024-04-24T23:20:55.484578+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724810+0000","last_change":"2024-04-24T23:20:54.473033+0000","last_active":"2024-04-24T23:20:56.724810+0000","last_peered":"2024-04-24T23:20:56.724810+0000","last_clean":"2024-04-24T23:20:56.724810+0000","last_became_active":"2024-04-24T23:20:54.472748+0000","last_became_peered":"2024-04-24T23:20:54.472748+0000","last_unstale":"2024-04-24T23:20:56.724810+0000","last_undegraded":"2024-04-24T23:20:56.724810+0000","last_fullsized":"2024-04-24T23:20:56.724810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724977+0000","last_change":"2024-04-24T23:20:55.485153+0000","last_active":"2024-04-24T23:20:56.724977+0000","last_peered":"2024-04-24T23:20:56.724977+0000","last_clean":"2024-04-24T23:20:56.724977+0000","last_became_active":"2024-04-24T23:20:55.484876+0000","last_became_peered":"2024-04-24T23:20:55.484876+0000","last_unstale":"2024-04-24T23:20:56.724977+0000","last_undegraded":"2024-04-24T23:20:56.724977+0000","last_fullsized":"2024-04-24T23:20:56.724977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474721+0000","last_change":"2024-04-24T23:20:54.476273+0000","last_active":"2024-04-24T23:20:56.474721+0000","last_peered":"2024-04-24T23:20:56.474721+0000","last_clean":"2024-04-24T23:20:56.474721+0000","last_became_active":"2024-04-24T23:20:54.475993+0000","last_became_peered":"2024-04-24T23:20:54.475993+0000","last_unstale":"2024-04-24T23:20:56.474721+0000","last_undegraded":"2024-04-24T23:20:56.474721+0000","last_fullsized":"2024-04-24T23:20:56.474721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725412+0000","last_change":"2024-04-24T23:20:55.487659+0000","last_active":"2024-04-24T23:20:56.725412+0000","last_peered":"2024-04-24T23:20:56.725412+0000","last_clean":"2024-04-24T23:20:56.725412+0000","last_became_active":"2024-04-24T23:20:55.486557+0000","last_became_peered":"2024-04-24T23:20:55.486557+0000","last_unstale":"2024-04-24T23:20:56.725412+0000","last_undegraded":"2024-04-24T23:20:56.725412+0000","last_fullsized":"2024-04-24T23:20:56.725412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724872+0000","last_change":"2024-04-24T23:20:54.477334+0000","last_active":"2024-04-24T23:20:56.724872+0000","last_peered":"2024-04-24T23:20:56.724872+0000","last_clean":"2024-04-24T23:20:56.724872+0000","last_became_active":"2024-04-24T23:20:54.477058+0000","last_became_peered":"2024-04-24T23:20:54.477058+0000","last_unstale":"2024-04-24T23:20:56.724872+0000","last_undegraded":"2024-04-24T23:20:56.724872+0000","last_fullsized":"2024-04-24T23:20:56.724872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725667+0000","last_change":"2024-04-24T23:20:55.492754+0000","last_active":"2024-04-24T23:20:56.725667+0000","last_peered":"2024-04-24T23:20:56.725667+0000","last_clean":"2024-04-24T23:20:56.725667+0000","last_became_active":"2024-04-24T23:20:55.492474+0000","last_became_peered":"2024-04-24T23:20:55.492474+0000","last_unstale":"2024-04-24T23:20:56.725667+0000","last_undegraded":"2024-04-24T23:20:56.725667+0000","last_fullsized":"2024-04-24T23:20:56.725667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:54.766130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478192+0000","last_change":"2024-04-24T23:20:54.471091+0000","last_active":"2024-04-24T23:20:56.478192+0000","last_peered":"2024-04-24T23:20:56.478192+0000","last_clean":"2024-04-24T23:20:56.478192+0000","last_became_active":"2024-04-24T23:20:54.470811+0000","last_became_peered":"2024-04-24T23:20:54.470811+0000","last_unstale":"2024-04-24T23:20:56.478192+0000","last_undegraded":"2024-04-24T23:20:56.478192+0000","last_fullsized":"2024-04-24T23:20:56.478192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476216+0000","last_change":"2024-04-24T23:20:55.480640+0000","last_active":"2024-04-24T23:20:56.476216+0000","last_peered":"2024-04-24T23:20:56.476216+0000","last_clean":"2024-04-24T23:20:56.476216+0000","last_became_active":"2024-04-24T23:20:55.480327+0000","last_became_peered":"2024-04-24T23:20:55.480327+0000","last_unstale":"2024-04-24T23:20:56.476216+0000","last_undegraded":"2024-04-24T23:20:56.476216+0000","last_fullsized":"2024-04-24T23:20:56.476216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725414+0000","last_change":"2024-04-24T23:20:54.476392+0000","last_active":"2024-04-24T23:20:56.725414+0000","last_peered":"2024-04-24T23:20:56.725414+0000","last_clean":"2024-04-24T23:20:56.725414+0000","last_became_active":"2024-04-24T23:20:54.476051+0000","last_became_peered":"2024-04-24T23:20:54.476051+0000","last_unstale":"2024-04-24T23:20:56.725414+0000","last_undegraded":"2024-04-24T23:20:56.725414+0000","last_fullsized":"2024-04-24T23:20:56.725414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725389+0000","last_change":"2024-04-24T23:20:55.483909+0000","last_active":"2024-04-24T23:20:56.725389+0000","last_peered":"2024-04-24T23:20:56.725389+0000","last_clean":"2024-04-24T23:20:56.725389+0000","last_became_active":"2024-04-24T23:20:55.483625+0000","last_became_peered":"2024-04-24T23:20:55.483625+0000","last_unstale":"2024-04-24T23:20:56.725389+0000","last_undegraded":"2024-04-24T23:20:56.725389+0000","last_fullsized":"2024-04-24T23:20:56.725389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724832+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:20:56.724832+0000","last_peered":"2024-04-24T23:20:56.724832+0000","last_clean":"2024-04-24T23:20:56.724832+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:20:56.724832+0000","last_undegraded":"2024-04-24T23:20:56.724832+0000","last_fullsized":"2024-04-24T23:20:56.724832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474912+0000","last_change":"2024-04-24T23:20:54.470392+0000","last_active":"2024-04-24T23:20:56.474912+0000","last_peered":"2024-04-24T23:20:56.474912+0000","last_clean":"2024-04-24T23:20:56.474912+0000","last_became_active":"2024-04-24T23:20:54.470095+0000","last_became_peered":"2024-04-24T23:20:54.470095+0000","last_unstale":"2024-04-24T23:20:56.474912+0000","last_undegraded":"2024-04-24T23:20:56.474912+0000","last_fullsized":"2024-04-24T23:20:56.474912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725394+0000","last_change":"2024-04-24T23:20:55.485347+0000","last_active":"2024-04-24T23:20:56.725394+0000","last_peered":"2024-04-24T23:20:56.725394+0000","last_clean":"2024-04-24T23:20:56.725394+0000","last_became_active":"2024-04-24T23:20:55.485004+0000","last_became_peered":"2024-04-24T23:20:55.485004+0000","last_unstale":"2024-04-24T23:20:56.725394+0000","last_undegraded":"2024-04-24T23:20:56.725394+0000","last_fullsized":"2024-04-24T23:20:56.725394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724782+0000","last_change":"2024-04-24T23:20:54.482240+0000","last_active":"2024-04-24T23:20:56.724782+0000","last_peered":"2024-04-24T23:20:56.724782+0000","last_clean":"2024-04-24T23:20:56.724782+0000","last_became_active":"2024-04-24T23:20:54.481934+0000","last_became_peered":"2024-04-24T23:20:54.481934+0000","last_unstale":"2024-04-24T23:20:56.724782+0000","last_undegraded":"2024-04-24T23:20:56.724782+0000","last_fullsized":"2024-04-24T23:20:56.724782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:13.029046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724293+0000","last_change":"2024-04-24T23:20:55.474530+0000","last_active":"2024-04-24T23:20:56.724293+0000","last_peered":"2024-04-24T23:20:56.724293+0000","last_clean":"2024-04-24T23:20:56.724293+0000","last_became_active":"2024-04-24T23:20:55.474181+0000","last_became_peered":"2024-04-24T23:20:55.474181+0000","last_unstale":"2024-04-24T23:20:56.724293+0000","last_undegraded":"2024-04-24T23:20:56.724293+0000","last_fullsized":"2024-04-24T23:20:56.724293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:43.728916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724886+0000","last_change":"2024-04-24T23:20:54.478684+0000","last_active":"2024-04-24T23:20:56.724886+0000","last_peered":"2024-04-24T23:20:56.724886+0000","last_clean":"2024-04-24T23:20:56.724886+0000","last_became_active":"2024-04-24T23:20:54.478377+0000","last_became_peered":"2024-04-24T23:20:54.478377+0000","last_unstale":"2024-04-24T23:20:56.724886+0000","last_undegraded":"2024-04-24T23:20:56.724886+0000","last_fullsized":"2024-04-24T23:20:56.724886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.478546+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.478253+0000","last_became_peered":"2024-04-24T23:20:55.478253+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476161+0000","last_change":"2024-04-24T23:20:54.478753+0000","last_active":"2024-04-24T23:20:56.476161+0000","last_peered":"2024-04-24T23:20:56.476161+0000","last_clean":"2024-04-24T23:20:56.476161+0000","last_became_active":"2024-04-24T23:20:54.478441+0000","last_became_peered":"2024-04-24T23:20:54.478441+0000","last_unstale":"2024-04-24T23:20:56.476161+0000","last_undegraded":"2024-04-24T23:20:56.476161+0000","last_fullsized":"2024-04-24T23:20:56.476161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475739+0000","last_change":"2024-04-24T23:20:55.483158+0000","last_active":"2024-04-24T23:20:56.475739+0000","last_peered":"2024-04-24T23:20:56.475739+0000","last_clean":"2024-04-24T23:20:56.475739+0000","last_became_active":"2024-04-24T23:20:55.482865+0000","last_became_peered":"2024-04-24T23:20:55.482865+0000","last_unstale":"2024-04-24T23:20:56.475739+0000","last_undegraded":"2024-04-24T23:20:56.475739+0000","last_fullsized":"2024-04-24T23:20:56.475739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477991+0000","last_change":"2024-04-24T23:20:54.476881+0000","last_active":"2024-04-24T23:20:56.477991+0000","last_peered":"2024-04-24T23:20:56.477991+0000","last_clean":"2024-04-24T23:20:56.477991+0000","last_became_active":"2024-04-24T23:20:54.476588+0000","last_became_peered":"2024-04-24T23:20:54.476588+0000","last_unstale":"2024-04-24T23:20:56.477991+0000","last_undegraded":"2024-04-24T23:20:56.477991+0000","last_fullsized":"2024-04-24T23:20:56.477991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475322+0000","last_change":"2024-04-24T23:20:55.482626+0000","last_active":"2024-04-24T23:20:56.475322+0000","last_peered":"2024-04-24T23:20:56.475322+0000","last_clean":"2024-04-24T23:20:56.475322+0000","last_became_active":"2024-04-24T23:20:55.482266+0000","last_became_peered":"2024-04-24T23:20:55.482266+0000","last_unstale":"2024-04-24T23:20:56.475322+0000","last_undegraded":"2024-04-24T23:20:56.475322+0000","last_fullsized":"2024-04-24T23:20:56.475322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725524+0000","last_change":"2024-04-24T23:20:54.484817+0000","last_active":"2024-04-24T23:20:56.725524+0000","last_peered":"2024-04-24T23:20:56.725524+0000","last_clean":"2024-04-24T23:20:56.725524+0000","last_became_active":"2024-04-24T23:20:54.484553+0000","last_became_peered":"2024-04-24T23:20:54.484553+0000","last_unstale":"2024-04-24T23:20:56.725524+0000","last_undegraded":"2024-04-24T23:20:56.725524+0000","last_fullsized":"2024-04-24T23:20:56.725524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:42.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478041+0000","last_change":"2024-04-24T23:20:55.479475+0000","last_active":"2024-04-24T23:20:56.478041+0000","last_peered":"2024-04-24T23:20:56.478041+0000","last_clean":"2024-04-24T23:20:56.478041+0000","last_became_active":"2024-04-24T23:20:55.478984+0000","last_became_peered":"2024-04-24T23:20:55.478984+0000","last_unstale":"2024-04-24T23:20:56.478041+0000","last_undegraded":"2024-04-24T23:20:56.478041+0000","last_fullsized":"2024-04-24T23:20:56.478041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.479792+0000","last_change":"2024-04-24T23:20:54.478307+0000","last_active":"2024-04-24T23:20:56.479792+0000","last_peered":"2024-04-24T23:20:56.479792+0000","last_clean":"2024-04-24T23:20:56.479792+0000","last_became_active":"2024-04-24T23:20:54.477988+0000","last_became_peered":"2024-04-24T23:20:54.477988+0000","last_unstale":"2024-04-24T23:20:56.479792+0000","last_undegraded":"2024-04-24T23:20:56.479792+0000","last_fullsized":"2024-04-24T23:20:56.479792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725717+0000","last_change":"2024-04-24T23:20:55.487879+0000","last_active":"2024-04-24T23:20:56.725717+0000","last_peered":"2024-04-24T23:20:56.725717+0000","last_clean":"2024-04-24T23:20:56.725717+0000","last_became_active":"2024-04-24T23:20:55.487553+0000","last_became_peered":"2024-04-24T23:20:55.487553+0000","last_unstale":"2024-04-24T23:20:56.725717+0000","last_undegraded":"2024-04-24T23:20:56.725717+0000","last_fullsized":"2024-04-24T23:20:56.725717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480845+0000","last_change":"2024-04-24T23:20:54.480826+0000","last_active":"2024-04-24T23:20:56.480845+0000","last_peered":"2024-04-24T23:20:56.480845+0000","last_clean":"2024-04-24T23:20:56.480845+0000","last_became_active":"2024-04-24T23:20:54.480351+0000","last_became_peered":"2024-04-24T23:20:54.480351+0000","last_unstale":"2024-04-24T23:20:56.480845+0000","last_undegraded":"2024-04-24T23:20:56.480845+0000","last_fullsized":"2024-04-24T23:20:56.480845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.480939+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.478888+0000","last_became_peered":"2024-04-24T23:20:55.478888+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724748+0000","last_change":"2024-04-24T23:20:54.482329+0000","last_active":"2024-04-24T23:20:56.724748+0000","last_peered":"2024-04-24T23:20:56.724748+0000","last_clean":"2024-04-24T23:20:56.724748+0000","last_became_active":"2024-04-24T23:20:54.482059+0000","last_became_peered":"2024-04-24T23:20:54.482059+0000","last_unstale":"2024-04-24T23:20:56.724748+0000","last_undegraded":"2024-04-24T23:20:56.724748+0000","last_fullsized":"2024-04-24T23:20:56.724748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475091+0000","last_change":"2024-04-24T23:20:55.478742+0000","last_active":"2024-04-24T23:20:56.475091+0000","last_peered":"2024-04-24T23:20:56.475091+0000","last_clean":"2024-04-24T23:20:56.475091+0000","last_became_active":"2024-04-24T23:20:55.478465+0000","last_became_peered":"2024-04-24T23:20:55.478465+0000","last_unstale":"2024-04-24T23:20:56.475091+0000","last_undegraded":"2024-04-24T23:20:56.475091+0000","last_fullsized":"2024-04-24T23:20:56.475091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:32.641502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477976+0000","last_change":"2024-04-24T23:20:54.480255+0000","last_active":"2024-04-24T23:20:56.477976+0000","last_peered":"2024-04-24T23:20:56.477976+0000","last_clean":"2024-04-24T23:20:56.477976+0000","last_became_active":"2024-04-24T23:20:54.479897+0000","last_became_peered":"2024-04-24T23:20:54.479897+0000","last_unstale":"2024-04-24T23:20:56.477976+0000","last_undegraded":"2024-04-24T23:20:56.477976+0000","last_fullsized":"2024-04-24T23:20:56.477976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725278+0000","last_change":"2024-04-24T23:20:55.480538+0000","last_active":"2024-04-24T23:20:56.725278+0000","last_peered":"2024-04-24T23:20:56.725278+0000","last_clean":"2024-04-24T23:20:56.725278+0000","last_became_active":"2024-04-24T23:20:55.474763+0000","last_became_peered":"2024-04-24T23:20:55.474763+0000","last_unstale":"2024-04-24T23:20:56.725278+0000","last_undegraded":"2024-04-24T23:20:56.725278+0000","last_fullsized":"2024-04-24T23:20:56.725278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477945+0000","last_change":"2024-04-24T23:20:54.475154+0000","last_active":"2024-04-24T23:20:56.477945+0000","last_peered":"2024-04-24T23:20:56.477945+0000","last_clean":"2024-04-24T23:20:56.477945+0000","last_became_active":"2024-04-24T23:20:54.473925+0000","last_became_peered":"2024-04-24T23:20:54.473925+0000","last_unstale":"2024-04-24T23:20:56.477945+0000","last_undegraded":"2024-04-24T23:20:56.477945+0000","last_fullsized":"2024-04-24T23:20:56.477945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725566+0000","last_change":"2024-04-24T23:20:55.479171+0000","last_active":"2024-04-24T23:20:56.725566+0000","last_peered":"2024-04-24T23:20:56.725566+0000","last_clean":"2024-04-24T23:20:56.725566+0000","last_became_active":"2024-04-24T23:20:55.478856+0000","last_became_peered":"2024-04-24T23:20:55.478856+0000","last_unstale":"2024-04-24T23:20:56.725566+0000","last_undegraded":"2024-04-24T23:20:56.725566+0000","last_fullsized":"2024-04-24T23:20:56.725566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725243+0000","last_change":"2024-04-24T23:20:54.484143+0000","last_active":"2024-04-24T23:20:56.725243+0000","last_peered":"2024-04-24T23:20:56.725243+0000","last_clean":"2024-04-24T23:20:56.725243+0000","last_became_active":"2024-04-24T23:20:54.483830+0000","last_became_peered":"2024-04-24T23:20:54.483830+0000","last_unstale":"2024-04-24T23:20:56.725243+0000","last_undegraded":"2024-04-24T23:20:56.725243+0000","last_fullsized":"2024-04-24T23:20:56.725243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480446+0000","last_change":"2024-04-24T23:20:55.479005+0000","last_active":"2024-04-24T23:20:56.480446+0000","last_peered":"2024-04-24T23:20:56.480446+0000","last_clean":"2024-04-24T23:20:56.480446+0000","last_became_active":"2024-04-24T23:20:55.478706+0000","last_became_peered":"2024-04-24T23:20:55.478706+0000","last_unstale":"2024-04-24T23:20:56.480446+0000","last_undegraded":"2024-04-24T23:20:56.480446+0000","last_fullsized":"2024-04-24T23:20:56.480446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474978+0000","last_change":"2024-04-24T23:20:55.469771+0000","last_active":"2024-04-24T23:20:56.474978+0000","last_peered":"2024-04-24T23:20:56.474978+0000","last_clean":"2024-04-24T23:20:56.474978+0000","last_became_active":"2024-04-24T23:20:55.469411+0000","last_became_peered":"2024-04-24T23:20:55.469411+0000","last_unstale":"2024-04-24T23:20:56.474978+0000","last_undegraded":"2024-04-24T23:20:56.474978+0000","last_fullsized":"2024-04-24T23:20:56.474978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725118+0000","last_change":"2024-04-24T23:20:54.485231+0000","last_active":"2024-04-24T23:20:56.725118+0000","last_peered":"2024-04-24T23:20:56.725118+0000","last_clean":"2024-04-24T23:20:56.725118+0000","last_became_active":"2024-04-24T23:20:54.484912+0000","last_became_peered":"2024-04-24T23:20:54.484912+0000","last_unstale":"2024-04-24T23:20:56.725118+0000","last_undegraded":"2024-04-24T23:20:56.725118+0000","last_fullsized":"2024-04-24T23:20:56.725118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:27.147159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476107+0000","last_change":"2024-04-24T23:20:55.473157+0000","last_active":"2024-04-24T23:20:56.476107+0000","last_peered":"2024-04-24T23:20:56.476107+0000","last_clean":"2024-04-24T23:20:56.476107+0000","last_became_active":"2024-04-24T23:20:55.472854+0000","last_became_peered":"2024-04-24T23:20:55.472854+0000","last_unstale":"2024-04-24T23:20:56.476107+0000","last_undegraded":"2024-04-24T23:20:56.476107+0000","last_fullsized":"2024-04-24T23:20:56.476107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725345+0000","last_change":"2024-04-24T23:20:54.485353+0000","last_active":"2024-04-24T23:20:56.725345+0000","last_peered":"2024-04-24T23:20:56.725345+0000","last_clean":"2024-04-24T23:20:56.725345+0000","last_became_active":"2024-04-24T23:20:54.485065+0000","last_became_peered":"2024-04-24T23:20:54.485065+0000","last_unstale":"2024-04-24T23:20:56.725345+0000","last_undegraded":"2024-04-24T23:20:56.725345+0000","last_fullsized":"2024-04-24T23:20:56.725345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724430+0000","last_change":"2024-04-24T23:20:55.487656+0000","last_active":"2024-04-24T23:20:56.724430+0000","last_peered":"2024-04-24T23:20:56.724430+0000","last_clean":"2024-04-24T23:20:56.724430+0000","last_became_active":"2024-04-24T23:20:55.487317+0000","last_became_peered":"2024-04-24T23:20:55.487317+0000","last_unstale":"2024-04-24T23:20:56.724430+0000","last_undegraded":"2024-04-24T23:20:56.724430+0000","last_fullsized":"2024-04-24T23:20:56.724430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725671+0000","last_change":"2024-04-24T23:20:54.475528+0000","last_active":"2024-04-24T23:20:56.725671+0000","last_peered":"2024-04-24T23:20:56.725671+0000","last_clean":"2024-04-24T23:20:56.725671+0000","last_became_active":"2024-04-24T23:20:54.475227+0000","last_became_peered":"2024-04-24T23:20:54.475227+0000","last_unstale":"2024-04-24T23:20:56.725671+0000","last_undegraded":"2024-04-24T23:20:56.725671+0000","last_fullsized":"2024-04-24T23:20:56.725671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.481689+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.481405+0000","last_became_peered":"2024-04-24T23:20:55.481405+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724580+0000","last_change":"2024-04-24T23:20:54.482023+0000","last_active":"2024-04-24T23:20:56.724580+0000","last_peered":"2024-04-24T23:20:56.724580+0000","last_clean":"2024-04-24T23:20:56.724580+0000","last_became_active":"2024-04-24T23:20:54.481494+0000","last_became_peered":"2024-04-24T23:20:54.481494+0000","last_unstale":"2024-04-24T23:20:56.724580+0000","last_undegraded":"2024-04-24T23:20:56.724580+0000","last_fullsized":"2024-04-24T23:20:56.724580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:21:03.599 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:21:03.599 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T23:21:03.599 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-24T23:21:03.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-24T23:21:03.600 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-24T23:21:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.734+0000 7f3252aee640 1 Processor -- start 2024-04-24T23:21:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.734+0000 7f3252aee640 1 -- start start 2024-04-24T23:21:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.735+0000 7f3252aee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f324c15e570 0x7f324c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.735+0000 7f3252aee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f324c0b3130 0x7f324c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.735+0000 7f3252aee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c0d22e0 0x7f324c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.735+0000 7f3252aee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f324c0d1ab0 con 0x7f324c0b3130 2024-04-24T23:21:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.735+0000 7f3252aee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f324c0d2ae0 con 0x7f324c15e570 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.735+0000 7f3252aee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f324c0d32f0 con 0x7f324c0d22e0 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.735+0000 7f324bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f324c0b3130 0x7f324c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.735+0000 7f324bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f324c0b3130 0x7f324c0d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39866/0 (socket says 172.21.15.26:39866) 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.735+0000 7f324bfff640 1 -- 172.21.15.26:0/1411988723 learned_addr learned my addr 172.21.15.26:0/1411988723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.736+0000 7f325088a640 1 --2- 172.21.15.26:0/1411988723 >> v2:172.21.15.29:3300/0 conn(0x7f324c15e570 0x7f324c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.736+0000 7f325108b640 1 --2- 172.21.15.26:0/1411988723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c0d22e0 0x7f324c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.736+0000 7f324bfff640 1 -- 172.21.15.26:0/1411988723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c0d22e0 msgr2=0x7f324c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.736+0000 7f324bfff640 1 --2- 172.21.15.26:0/1411988723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c0d22e0 0x7f324c0d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.736+0000 7f324bfff640 1 -- 172.21.15.26:0/1411988723 >> v2:172.21.15.29:3300/0 conn(0x7f324c15e570 msgr2=0x7f324c0d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.736+0000 7f324bfff640 1 --2- 172.21.15.26:0/1411988723 >> v2:172.21.15.29:3300/0 conn(0x7f324c15e570 0x7f324c0d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.736+0000 7f324bfff640 1 -- 172.21.15.26:0/1411988723 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c0d0270 con 0x7f324c0b3130 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.736+0000 7f324bfff640 1 --2- 172.21.15.26:0/1411988723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 0x7f324c0d46a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f324c0d52d0 tx=0x7f323c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc85eee007f67b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.736+0000 7f324b7fe640 1 -- 172.21.15.26:0/1411988723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c037070 con 0x7f324c0b3130 2024-04-24T23:21:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f324b7fe640 1 -- 172.21.15.26:0/1411988723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c032440 con 0x7f324c0b3130 2024-04-24T23:21:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f324b7fe640 1 -- 172.21.15.26:0/1411988723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c032810 con 0x7f324c0b3130 2024-04-24T23:21:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f3252aee640 1 -- 172.21.15.26:0/1411988723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 msgr2=0x7f324c0d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f3252aee640 1 --2- 172.21.15.26:0/1411988723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 0x7f324c0d46a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f324c0d52d0 tx=0x7f323c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f3252aee640 1 -- 172.21.15.26:0/1411988723 shutdown_connections 2024-04-24T23:21:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f3252aee640 1 --2- 172.21.15.26:0/1411988723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 0x7f324c0d46a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f3252aee640 1 --2- 172.21.15.26:0/1411988723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c0d22e0 0x7f324c0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f3252aee640 1 --2- 172.21.15.26:0/1411988723 >> v2:172.21.15.29:3300/0 conn(0x7f324c15e570 0x7f324c0d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f3252aee640 1 -- 172.21.15.26:0/1411988723 >> 172.21.15.26:0/1411988723 conn(0x7f324c08dbe0 msgr2=0x7f324c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f3252aee640 1 -- 172.21.15.26:0/1411988723 shutdown_connections 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.737+0000 7f3252aee640 1 -- 172.21.15.26:0/1411988723 wait complete. 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.738+0000 7f3252aee640 1 Processor -- start 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.738+0000 7f3252aee640 1 -- start start 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f3252aee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 0x7f324c15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f3252aee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f324c0d22e0 0x7f324c0b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f3252aee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c15e570 0x7f324c0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f3252aee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f324c0cfa70 con 0x7f324c0b3130 2024-04-24T23:21:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f3252aee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f324c0d3010 con 0x7f324c0d22e0 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f3252aee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f324c0cfd50 con 0x7f324c15e570 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f325088a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 0x7f324c15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f325088a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 0x7f324c15a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39882/0 (socket says 172.21.15.26:39882) 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f325088a640 1 -- 172.21.15.26:0/2480986435 learned_addr learned my addr 172.21.15.26:0/2480986435 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f324bfff640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f324c0d22e0 0x7f324c0b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.739+0000 7f325108b640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c15e570 0x7f324c0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f325088a640 1 -- 172.21.15.26:0/2480986435 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c15e570 msgr2=0x7f324c0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f325088a640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c15e570 0x7f324c0c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f325088a640 1 -- 172.21.15.26:0/2480986435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f324c0d22e0 msgr2=0x7f324c0b87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f325088a640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f324c0d22e0 0x7f324c0b87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f325088a640 1 -- 172.21.15.26:0/2480986435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c0ca400 con 0x7f324c0b3130 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f325088a640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 0x7f324c15a7d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3244007e40 tx=0x7f324400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f32497fa640 1 -- 172.21.15.26:0/2480986435 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244014620 con 0x7f324c0b3130 2024-04-24T23:21:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f32497fa640 1 -- 172.21.15.26:0/2480986435 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3244019070 con 0x7f324c0b3130 2024-04-24T23:21:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c0b8f60 con 0x7f324c0b3130 2024-04-24T23:21:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f32497fa640 1 -- 172.21.15.26:0/2480986435 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244014a00 con 0x7f324c0b3130 2024-04-24T23:21:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.740+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c0b9520 con 0x7f324c0b3130 2024-04-24T23:21:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.741+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324c0694c0 con 0x7f324c0b3130 2024-04-24T23:21:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.741+0000 7f32497fa640 1 -- 172.21.15.26:0/2480986435 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3244014ba0 con 0x7f324c0b3130 2024-04-24T23:21:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.742+0000 7f32497fa640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f322c06be30 0x7f322c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.742+0000 7f32497fa640 1 -- 172.21.15.26:0/2480986435 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f324408b520 con 0x7f324c0b3130 2024-04-24T23:21:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.742+0000 7f324bfff640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f322c06be30 0x7f322c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.742+0000 7f324bfff640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f322c06be30 0x7f322c06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f323c034000 tx=0x7f323c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.746+0000 7f32497fa640 1 -- 172.21.15.26:0/2480986435 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f324c0ca400 con 0x7f324c0b3130 2024-04-24T23:21:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.977+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f324c06c6a0 con 0x7f324c0b3130 2024-04-24T23:21:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.980+0000 7f32497fa640 1 -- 172.21.15.26:0/2480986435 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172938 (secure 0 0 0) 0x7f3244059910 con 0x7f324c0b3130 2024-04-24T23:21:03.984 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f322c06be30 msgr2=0x7f322c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f322c06be30 0x7f322c06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f323c034000 tx=0x7f323c02e000 comp rx=0 tx=0).stop 2024-04-24T23:21:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 msgr2=0x7f324c15a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 0x7f324c15a7d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3244007e40 tx=0x7f324400e4b0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 shutdown_connections 2024-04-24T23:21:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f322c06be30 0x7f322c06e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f324c15e570 0x7f324c0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f324c0d22e0 0x7f324c0b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 --2- 172.21.15.26:0/2480986435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f324c0b3130 0x7f324c15a7d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 >> 172.21.15.26:0/2480986435 conn(0x7f324c08dbe0 msgr2=0x7f324c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 shutdown_connections 2024-04-24T23:21:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:03.983+0000 7f3252aee640 1 -- 172.21.15.26:0/2480986435 wait complete. 2024-04-24T23:21:03.996 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3792722438},{"type":"v1","addr":"172.21.15.26:6833","nonce":3792722438}]},"active_addr":"172.21.15.26:6833/3792722438","active_change":"2024-04-24T23:20:35.307361+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4099,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3154089153}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":2665236811}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":939755117}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":1034209987}]}]} 2024-04-24T23:21:03.999 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-24T23:21:03.999 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T23:21:03.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.098+0000 7fdaf4c9a640 1 Processor -- start 2024-04-24T23:21:04.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.099+0000 7fdaf4c9a640 1 -- start start 2024-04-24T23:21:04.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.099+0000 7fdaf4c9a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdaf015e570 0x7fdaf00d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.099+0000 7fdaf4c9a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf00d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.099+0000 7fdaf4c9a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf00d32f0 con 0x7fdaf00d22e0 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.099+0000 7fdaf4c9a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf00d1ab0 con 0x7fdaf015e570 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.099+0000 7fdaf4c9a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf00d2ae0 con 0x7fdaf00b3130 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaee59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdaf015e570 0x7fdaf00d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdae7fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf00d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaeed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdaf00d22e0 0x7fdaf00d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:32892/0 (socket says 172.21.15.26:32892) 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaee59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdaf015e570 0x7fdaf00d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:56458/0 (socket says 172.21.15.26:56458) 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaeed9d640 1 -- 172.21.15.26:0/410590107 learned_addr learned my addr 172.21.15.26:0/410590107 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaedd9b640 1 -- 172.21.15.26:0/410590107 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2752151103 0 0) 0x7fdae8001490 con 0x7fdaf00d22e0 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaedd9b640 1 -- 172.21.15.26:0/410590107 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdadc003520 con 0x7fdaf00d22e0 2024-04-24T23:21:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaee59c640 1 -- 172.21.15.26:0/410590107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 msgr2=0x7fdaf00d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaee59c640 1 --2- 172.21.15.26:0/410590107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf00d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaee59c640 1 -- 172.21.15.26:0/410590107 >> v1:172.21.15.29:6789/0 conn(0x7fdaf00d22e0 legacy=0x7fdaf00d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdaee59c640 1 -- 172.21.15.26:0/410590107 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf00d0270 con 0x7fdaf015e570 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.100+0000 7fdae7fff640 1 --2- 172.21.15.26:0/410590107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf00d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.101+0000 7fdaee59c640 1 --2- 172.21.15.26:0/410590107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdaf015e570 0x7fdaf00d4160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdae00097a0 tx=0x7fdae002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffb129fb11b5ad9d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.101+0000 7fdaedd9b640 1 -- 172.21.15.26:0/410590107 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0037070 con 0x7fdaf015e570 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.101+0000 7fdaedd9b640 1 -- 172.21.15.26:0/410590107 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae0032440 con 0x7fdaf015e570 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.101+0000 7fdaedd9b640 1 -- 172.21.15.26:0/410590107 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae00327f0 con 0x7fdaf015e570 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.102+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/410590107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdaf015e570 msgr2=0x7fdaf00d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.102+0000 7fdaf4c9a640 1 --2- 172.21.15.26:0/410590107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdaf015e570 0x7fdaf00d4160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdae00097a0 tx=0x7fdae002a180 comp rx=0 tx=0).stop 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.102+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/410590107 shutdown_connections 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.102+0000 7fdaf4c9a640 1 --2- 172.21.15.26:0/410590107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf00d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.102+0000 7fdaf4c9a640 1 --2- 172.21.15.26:0/410590107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdaf015e570 0x7fdaf00d4160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.102+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/410590107 >> 172.21.15.26:0/410590107 conn(0x7fdaf008dbe0 msgr2=0x7fdaf009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.102+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/410590107 shutdown_connections 2024-04-24T23:21:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.102+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/410590107 wait complete. 2024-04-24T23:21:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.103+0000 7fdaf4c9a640 1 Processor -- start 2024-04-24T23:21:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.103+0000 7fdaf4c9a640 1 -- start start 2024-04-24T23:21:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaf4c9a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf0155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaf4c9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf00d22e0 0x7fdaf01526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaf4c9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdaf015e570 0x7fdaf00a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaf4c9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf00cfa70 con 0x7fdaf00d22e0 2024-04-24T23:21:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaf4c9a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf00d3010 con 0x7fdaf015e570 2024-04-24T23:21:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaf4c9a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf00cfd50 con 0x7fdaf00b3130 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf00d22e0 0x7fdaf01526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf00d22e0 0x7fdaf01526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39898/0 (socket says 172.21.15.26:39898) 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae7fff640 1 -- 172.21.15.26:0/2768923056 learned_addr learned my addr 172.21.15.26:0/2768923056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaeed9d640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdaf015e570 0x7fdaf00a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaee59c640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf0155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae7fff640 1 -- 172.21.15.26:0/2768923056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 msgr2=0x7fdaf0155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae7fff640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf0155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae7fff640 1 -- 172.21.15.26:0/2768923056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdaf015e570 msgr2=0x7fdaf00a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae7fff640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdaf015e570 0x7fdaf00a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae7fff640 1 -- 172.21.15.26:0/2768923056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf00b7e40 con 0x7fdaf00d22e0 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaee59c640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf0155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae7fff640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf00d22e0 0x7fdaf01526e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdad800ec10 tx=0x7fdad800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae6ffd640 1 -- 172.21.15.26:0/2768923056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8017070 con 0x7fdaf00d22e0 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae6ffd640 1 -- 172.21.15.26:0/2768923056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad800cc30 con 0x7fdaf00d22e0 2024-04-24T23:21:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf00b8120 con 0x7fdaf00d22e0 2024-04-24T23:21:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.104+0000 7fdae6ffd640 1 -- 172.21.15.26:0/2768923056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8013470 con 0x7fdaf00d22e0 2024-04-24T23:21:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.105+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf00b86e0 con 0x7fdaf00d22e0 2024-04-24T23:21:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.106+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf00694c0 con 0x7fdaf00d22e0 2024-04-24T23:21:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.106+0000 7fdae6ffd640 1 -- 172.21.15.26:0/2768923056 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdad801a070 con 0x7fdaf00d22e0 2024-04-24T23:21:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.106+0000 7fdae6ffd640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdad006be60 0x7fdad006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.107+0000 7fdae6ffd640 1 -- 172.21.15.26:0/2768923056 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdad808b100 con 0x7fdaf00d22e0 2024-04-24T23:21:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.107+0000 7fdaee59c640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdad006be60 0x7fdad006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.107+0000 7fdaee59c640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdad006be60 0x7fdad006e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdae0034000 tx=0x7fdae002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.111+0000 7fdae6ffd640 1 -- 172.21.15.26:0/2768923056 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdad8059ab0 con 0x7fdaf00d22e0 2024-04-24T23:21:04.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.295+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdaf00d14b0 con 0x7fdaf00d22e0 2024-04-24T23:21:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.296+0000 7fdae6ffd640 1 -- 172.21.15.26:0/2768923056 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fdad80593b0 con 0x7fdaf00d22e0 2024-04-24T23:21:04.298 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:04.298 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdad006be60 msgr2=0x7fdad006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdad006be60 0x7fdad006e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdae0034000 tx=0x7fdae002e000 comp rx=0 tx=0).stop 2024-04-24T23:21:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf00d22e0 msgr2=0x7fdaf01526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf00d22e0 0x7fdaf01526e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdad800ec10 tx=0x7fdad800c520 comp rx=0 tx=0).stop 2024-04-24T23:21:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 shutdown_connections 2024-04-24T23:21:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdad006be60 0x7fdad006e320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdaf015e570 0x7fdaf00a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf00d22e0 0x7fdaf01526e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 --2- 172.21.15.26:0/2768923056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdaf00b3130 0x7fdaf0155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 >> 172.21.15.26:0/2768923056 conn(0x7fdaf008dbe0 msgr2=0x7fdaf00d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 shutdown_connections 2024-04-24T23:21:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.298+0000 7fdaf4c9a640 1 -- 172.21.15.26:0/2768923056 wait complete. 2024-04-24T23:21:04.311 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T23:21:04.312 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.410+0000 7f11507dc640 1 Processor -- start 2024-04-24T23:21:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.410+0000 7f11507dc640 1 -- start start 2024-04-24T23:21:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.411+0000 7f11507dc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f114815e570 0x7f11480d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.411+0000 7f11507dc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f11480b3130 0x7f11480d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.411+0000 7f11507dc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11480d32f0 con 0x7f11480d22e0 2024-04-24T23:21:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.411+0000 7f11507dc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f11480d1ab0 con 0x7f11480b3130 2024-04-24T23:21:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.411+0000 7f11507dc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f11480d2ae0 con 0x7f114815e570 2024-04-24T23:21:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.411+0000 7f114ed79640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f11480d22e0 0x7f11480d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54592/0 (socket says 172.21.15.26:54592) 2024-04-24T23:21:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.411+0000 7f114ed79640 1 -- 172.21.15.26:0/1097689997 learned_addr learned my addr 172.21.15.26:0/1097689997 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114dd77640 1 --2- 172.21.15.26:0/1097689997 >> v2:172.21.15.29:3300/0 conn(0x7f11480b3130 0x7f11480d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114e578640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f114815e570 0x7f11480d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3616630967 0 0) 0x7f1138001490 con 0x7f11480d22e0 2024-04-24T23:21:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1130003520 con 0x7f11480d22e0 2024-04-24T23:21:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2128061886 0 0) 0x7f1130003520 con 0x7f11480d22e0 2024-04-24T23:21:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f114815e570 msgr2=0x7f11480d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f114815e570 0x7f11480d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 >> v2:172.21.15.29:3300/0 conn(0x7f11480b3130 msgr2=0x7f11480d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 --2- 172.21.15.26:0/1097689997 >> v2:172.21.15.29:3300/0 conn(0x7f11480b3130 0x7f11480d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11480d0270 con 0x7f11480d22e0 2024-04-24T23:21:04.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f1138001bc0 con 0x7f11480d22e0 2024-04-24T23:21:04.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 >> v1:172.21.15.26:6789/0 conn(0x7f11480d22e0 legacy=0x7f11480d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f114815e570 0x7f11300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1130003f30 0x7f1130004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1130004d40 0x7f11300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11480d1ab0 con 0x7f1130003f30 2024-04-24T23:21:04.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f11480d2ae0 con 0x7f1130004d40 2024-04-24T23:21:04.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f113000b9c0 con 0x7f114815e570 2024-04-24T23:21:04.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114dd77640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1130003f30 0x7f1130004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114e578640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f114815e570 0x7f11300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114ed79640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1130004d40 0x7f11300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114dd77640 1 -- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f114815e570 msgr2=0x7f11300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114dd77640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f114815e570 0x7f11300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114dd77640 1 -- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1130004d40 msgr2=0x7f11300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114dd77640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1130004d40 0x7f11300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.412+0000 7f114dd77640 1 -- 172.21.15.26:0/1097689997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1130003520 con 0x7f1130003f30 2024-04-24T23:21:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.413+0000 7f114dd77640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1130003f30 0x7f1130004800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f113c009ac0 tx=0x7f1130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1799b6487ae1bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.413+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c036070 con 0x7f1130003f30 2024-04-24T23:21:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.413+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113c031400 con 0x7f1130003f30 2024-04-24T23:21:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.413+0000 7f114d576640 1 -- 172.21.15.26:0/1097689997 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c009690 con 0x7f1130003f30 2024-04-24T23:21:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.414+0000 7f11507dc640 1 -- 172.21.15.26:0/1097689997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1130003f30 msgr2=0x7f1130004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.414+0000 7f11507dc640 1 --2- 172.21.15.26:0/1097689997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1130003f30 0x7f1130004800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f113c009ac0 tx=0x7f1130005830 comp rx=0 tx=0).stop 2024-04-24T23:21:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.414+0000 7f114ed79640 1 -- 172.21.15.26:0/1097689997 reap_dead start 2024-04-24T23:21:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.414+0000 7f11507dc640 1 -- 172.21.15.26:0/1097689997 shutdown_connections 2024-04-24T23:21:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.414+0000 7f11507dc640 1 -- 172.21.15.26:0/1097689997 >> 172.21.15.26:0/1097689997 conn(0x7f114808dbe0 msgr2=0x7f114809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.414+0000 7f11507dc640 1 -- 172.21.15.26:0/1097689997 shutdown_connections 2024-04-24T23:21:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.414+0000 7f11507dc640 1 -- 172.21.15.26:0/1097689997 wait complete. 2024-04-24T23:21:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.415+0000 7f11507dc640 1 Processor -- start 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.415+0000 7f11507dc640 1 -- start start 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f11507dc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1130003f30 0x7f114814f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f11507dc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f114809e5d0 0x7f11480b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f11507dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11480b90a0 0x7f11480a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f11507dc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11480cfa70 con 0x7f11480b90a0 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f11507dc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f11480d3010 con 0x7f114809e5d0 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f11507dc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f11480cfd50 con 0x7f1130003f30 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114ed79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11480b90a0 0x7f11480a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114ed79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11480b90a0 0x7f11480a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39918/0 (socket says 172.21.15.26:39918) 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114ed79640 1 -- 172.21.15.26:0/590505495 learned_addr learned my addr 172.21.15.26:0/590505495 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114e578640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1130003f30 0x7f114814f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114dd77640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f114809e5d0 0x7f11480b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114ed79640 1 -- 172.21.15.26:0/590505495 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1130003f30 msgr2=0x7f114814f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114ed79640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1130003f30 0x7f114814f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114ed79640 1 -- 172.21.15.26:0/590505495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f114809e5d0 msgr2=0x7f11480b8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114ed79640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f114809e5d0 0x7f11480b8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114ed79640 1 -- 172.21.15.26:0/590505495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114814fab0 con 0x7f11480b90a0 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114dd77640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f114809e5d0 0x7f11480b8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f114ed79640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11480b90a0 0x7f11480a0f10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f114808e710 tx=0x7f113800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f1146ffd640 1 -- 172.21.15.26:0/590505495 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113800ec10 con 0x7f11480b90a0 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f1146ffd640 1 -- 172.21.15.26:0/590505495 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1138019070 con 0x7f11480b90a0 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.416+0000 7f1146ffd640 1 -- 172.21.15.26:0/590505495 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138013420 con 0x7f11480b90a0 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.417+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114814f7a0 con 0x7f11480b90a0 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.417+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1148150030 con 0x7f11480b90a0 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.417+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114808e200 con 0x7f11480b90a0 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.418+0000 7f1146ffd640 1 -- 172.21.15.26:0/590505495 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f11380135c0 con 0x7f11480b90a0 2024-04-24T23:21:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.418+0000 7f1146ffd640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f113406be60 0x7f113406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.418+0000 7f1146ffd640 1 -- 172.21.15.26:0/590505495 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f113808b080 con 0x7f11480b90a0 2024-04-24T23:21:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.418+0000 7f114e578640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f113406be60 0x7f113406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.419+0000 7f114e578640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f113406be60 0x7f113406e320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1140009ae0 tx=0x7f1140009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.423+0000 7f1146ffd640 1 -- 172.21.15.26:0/590505495 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f114808e200 con 0x7f11480b90a0 2024-04-24T23:21:04.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.608+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f114815e950 con 0x7f11480b90a0 2024-04-24T23:21:04.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.609+0000 7f1146ffd640 1 -- 172.21.15.26:0/590505495 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f114815e950 con 0x7f11480b90a0 2024-04-24T23:21:04.612 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:04.612 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f113406be60 msgr2=0x7f113406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f113406be60 0x7f113406e320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1140009ae0 tx=0x7f1140009190 comp rx=0 tx=0).stop 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11480b90a0 msgr2=0x7f11480a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11480b90a0 0x7f11480a0f10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f114808e710 tx=0x7f113800e530 comp rx=0 tx=0).stop 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 shutdown_connections 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f113406be60 0x7f113406e320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11480b90a0 0x7f11480a0f10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f114809e5d0 0x7f11480b8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 --2- 172.21.15.26:0/590505495 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1130003f30 0x7f114814f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 >> 172.21.15.26:0/590505495 conn(0x7f114808dbe0 msgr2=0x7f11480caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 shutdown_connections 2024-04-24T23:21:04.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.611+0000 7f11507dc640 1 -- 172.21.15.26:0/590505495 wait complete. 2024-04-24T23:21:04.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T23:21:04.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T23:21:04.628 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T23:21:04.628 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T23:21:04.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T23:21:04.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T23:21:04.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T23:21:04.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T23:21:04.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.826+0000 7faa1f722640 1 Processor -- start 2024-04-24T23:21:04.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.826+0000 7faa1f722640 1 -- start start 2024-04-24T23:21:04.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.828+0000 7faa1f722640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faa100eacf0 0x7faa1004fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.828+0000 7faa1f722640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa1004ffc0 0x7faa10037bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.828+0000 7faa1f722640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa10038100 0x7faa10038590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.828+0000 7faa1f722640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7faa10054320 con 0x7faa10038100 2024-04-24T23:21:04.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.828+0000 7faa1f722640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faa10053710 con 0x7faa100eacf0 2024-04-24T23:21:04.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.828+0000 7faa1f722640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa100539f0 con 0x7faa1004ffc0 2024-04-24T23:21:04.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.828+0000 7faa1dcbf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa10038100 0x7faa10038590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.828+0000 7faa1dcbf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa10038100 0x7faa10038590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39926/0 (socket says 172.21.15.26:39926) 2024-04-24T23:21:04.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.828+0000 7faa1dcbf640 1 -- 172.21.15.26:0/1376577901 learned_addr learned my addr 172.21.15.26:0/1376577901 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.829+0000 7faa1dcbf640 1 -- 172.21.15.26:0/1376577901 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa1004ffc0 msgr2=0x7faa10037bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.829+0000 7faa1dcbf640 1 --2- 172.21.15.26:0/1376577901 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa1004ffc0 0x7faa10037bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.829+0000 7faa1dcbf640 1 -- 172.21.15.26:0/1376577901 >> v2:172.21.15.29:3300/0 conn(0x7faa100eacf0 msgr2=0x7faa1004fa80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.829+0000 7faa1dcbf640 1 --2- 172.21.15.26:0/1376577901 >> v2:172.21.15.29:3300/0 conn(0x7faa100eacf0 0x7faa1004fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.829+0000 7faa1dcbf640 1 -- 172.21.15.26:0/1376577901 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa10053cd0 con 0x7faa10038100 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.830+0000 7faa1dcbf640 1 --2- 172.21.15.26:0/1376577901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa10038100 0x7faa10038590 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faa10007aa0 tx=0x7faa0802a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b98dd05734a7485 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.831+0000 7faa0ffff640 1 -- 172.21.15.26:0/1376577901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08036070 con 0x7faa10038100 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.831+0000 7faa0ffff640 1 -- 172.21.15.26:0/1376577901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa08031600 con 0x7faa10038100 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.832+0000 7faa1f722640 1 -- 172.21.15.26:0/1376577901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa10038100 msgr2=0x7faa10038590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.832+0000 7faa1f722640 1 --2- 172.21.15.26:0/1376577901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa10038100 0x7faa10038590 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faa10007aa0 tx=0x7faa0802a470 comp rx=0 tx=0).stop 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.832+0000 7faa1f722640 1 -- 172.21.15.26:0/1376577901 shutdown_connections 2024-04-24T23:21:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.832+0000 7faa1f722640 1 --2- 172.21.15.26:0/1376577901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa10038100 0x7faa10038590 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.832+0000 7faa1f722640 1 --2- 172.21.15.26:0/1376577901 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa1004ffc0 0x7faa10037bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.832+0000 7faa1f722640 1 --2- 172.21.15.26:0/1376577901 >> v2:172.21.15.29:3300/0 conn(0x7faa100eacf0 0x7faa1004fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.832+0000 7faa1f722640 1 -- 172.21.15.26:0/1376577901 >> 172.21.15.26:0/1376577901 conn(0x7faa10009fa0 msgr2=0x7faa1000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.832+0000 7faa1f722640 1 -- 172.21.15.26:0/1376577901 shutdown_connections 2024-04-24T23:21:04.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.832+0000 7faa1f722640 1 -- 172.21.15.26:0/1376577901 wait complete. 2024-04-24T23:21:04.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.834+0000 7f28dec9a640 1 Processor -- start 2024-04-24T23:21:04.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.834+0000 7faa1f722640 1 Processor -- start 2024-04-24T23:21:04.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.834+0000 7f28dec9a640 1 -- start start 2024-04-24T23:21:04.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.834+0000 7f1a7ffff640 1 Processor -- start 2024-04-24T23:21:04.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7f4556685640 1 Processor -- start 2024-04-24T23:21:04.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.834+0000 7f1a7ffff640 1 -- start start 2024-04-24T23:21:04.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7f28dec9a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28d00deff0 0x7f28d004f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.834+0000 7faa1f722640 1 -- start start 2024-04-24T23:21:04.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1f722640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10038100 0x7faa1017d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1f722640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1004ffc0 0x7faa10172ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1f722640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa100eacf0 0x7faa10173010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1f722640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa10054930 con 0x7faa1004ffc0 2024-04-24T23:21:04.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1f722640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa100547b0 con 0x7faa10038100 2024-04-24T23:21:04.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1f722640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa1004f540 con 0x7faa100eacf0 2024-04-24T23:21:04.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1d4be640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10038100 0x7faa1017d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f6e55789640 1 Processor -- start 2024-04-24T23:21:04.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1ccbd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1004ffc0 0x7faa10172ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1ccbd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1004ffc0 0x7faa10172ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39942/0 (socket says 172.21.15.26:39942) 2024-04-24T23:21:04.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7faa1ccbd640 1 -- 172.21.15.26:0/1837417792 learned_addr learned my addr 172.21.15.26:0/1837417792 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f28dec9a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f28d00df3d0 0x7f28d004fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f28dec9a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28d0054d30 con 0x7f28d0005eb0 2024-04-24T23:21:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f28dec9a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f28d0054630 con 0x7f28d00deff0 2024-04-24T23:21:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f28dec9a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f28d0006dd0 con 0x7f28d00df3d0 2024-04-24T23:21:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f28dd237640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f28d0005eb0 0x7f28d0050150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54608/0 (socket says 172.21.15.26:54608) 2024-04-24T23:21:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f28dd237640 1 -- 172.21.15.26:0/270197879 learned_addr learned my addr 172.21.15.26:0/270197879 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f6e55789640 1 -- start start 2024-04-24T23:21:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7f1a7ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a8006f0e0 0x7f1a800d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1a800d0f30 0x7f1a8006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7f1a7ffff640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8006f8f0 con 0x7f1a800af6d0 2024-04-24T23:21:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7f1a7ffff640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8006f6a0 con 0x7f1a8006f0e0 2024-04-24T23:21:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7f1a7ffff640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8006dc90 con 0x7f1a800d0f30 2024-04-24T23:21:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7f4556685640 1 -- start start 2024-04-24T23:21:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7fa0a7e72640 1 Processor -- start 2024-04-24T23:21:04.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.835+0000 7fa0a7e72640 1 -- start start 2024-04-24T23:21:04.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f28dca36640 1 --2- 172.21.15.26:0/270197879 >> v2:172.21.15.26:3300/0 conn(0x7f28d00deff0 0x7f28d004f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f28d7fff640 1 --2- 172.21.15.26:0/270197879 >> v2:172.21.15.29:3300/0 conn(0x7f28d00df3d0 0x7f28d004fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f4556685640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 0x7f454803c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7faa1d4be640 1 -- 172.21.15.26:0/1837417792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa100eacf0 msgr2=0x7faa10173010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7f4556685640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4548053950 con 0x7f45480583c0 2024-04-24T23:21:04.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7faa1d4be640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa100eacf0 0x7faa10173010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f4556685640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4548006da0 con 0x7f4548007820 2024-04-24T23:21:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f4556685640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f45480e7a20 con 0x7f454803c150 2024-04-24T23:21:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7fa0a7e72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a014c040 0x7fa0a00b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7fa0a7e72640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0a00b4380 0x7fa0a006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7fa0a7e72640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a006cff0 con 0x7fa0a006e900 2024-04-24T23:21:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7fa0a7e72640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a006d2d0 con 0x7fa0a00b4380 2024-04-24T23:21:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7fa0a7e72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a006d5b0 con 0x7fa0a014c040 2024-04-24T23:21:04.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7faa1d4be640 1 -- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1004ffc0 msgr2=0x7faa10172ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f4554e82640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f4548007820 0x7f4548038040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:32894/0 (socket says 172.21.15.26:32894) 2024-04-24T23:21:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f4554e82640 1 -- 172.21.15.26:0/3605607735 learned_addr learned my addr 172.21.15.26:0/3605607735 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7faa1d4be640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1004ffc0 0x7faa10172ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f45578e7640 1 --2- 172.21.15.26:0/3605607735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 0x7f454803c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.836+0000 7faa1d4be640 1 -- 172.21.15.26:0/1837417792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1017d970 con 0x7faa10038100 2024-04-24T23:21:04.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7faa1d4be640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10038100 0x7faa1017d430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa14009fd0 tx=0x7faa1400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7fa0a5c0e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a014c040 0x7fa0a00b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7fa0a5c0e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a014c040 0x7fa0a00b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38112/0 (socket says 172.21.15.26:38112) 2024-04-24T23:21:04.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7fa0a5c0e640 1 -- 172.21.15.26:0/1774014916 learned_addr learned my addr 172.21.15.26:0/1774014916 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f1a7e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a8006f0e0 0x7f1a800d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f1a7e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a8006f0e0 0x7f1a800d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39946/0 (socket says 172.21.15.26:39946) 2024-04-24T23:21:04.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f1a7e7fc640 1 -- 172.21.15.26:0/3648752172 learned_addr learned my addr 172.21.15.26:0/3648752172 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7fa0a540d640 1 --2- 172.21.15.26:0/1774014916 >> v2:172.21.15.29:3300/0 conn(0x7fa0a00b4380 0x7fa0a006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f1a7f7fe640 1 --2- 172.21.15.26:0/3648752172 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1a800d0f30 0x7f1a8006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f28d77fe640 1 -- 172.21.15.26:0/270197879 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1954924706 0 0) 0x7f28d0054d30 con 0x7f28d0005eb0 2024-04-24T23:21:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f28d77fe640 1 -- 172.21.15.26:0/270197879 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28c8003520 con 0x7f28d0005eb0 2024-04-24T23:21:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7faa0dffb640 1 -- 172.21.15.26:0/1837417792 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14015020 con 0x7faa10038100 2024-04-24T23:21:04.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f28d7fff640 1 -- 172.21.15.26:0/270197879 >> v2:172.21.15.26:3300/0 conn(0x7f28d00deff0 msgr2=0x7f28d004f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f28d7fff640 1 --2- 172.21.15.26:0/270197879 >> v2:172.21.15.26:3300/0 conn(0x7f28d00deff0 0x7f28d004f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f28d7fff640 1 -- 172.21.15.26:0/270197879 >> v1:172.21.15.26:6789/0 conn(0x7f28d0005eb0 legacy=0x7f28d0050150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f28d7fff640 1 -- 172.21.15.26:0/270197879 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d00070b0 con 0x7f28d00df3d0 2024-04-24T23:21:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f6e55789640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50154720 0x7f6e500b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f6e55789640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e50154b00 con 0x7f6e500b4380 2024-04-24T23:21:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f6e55789640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e500b47e0 con 0x7f6e50071440 2024-04-24T23:21:04.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f6e55789640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6e50071a20 con 0x7f6e50154720 2024-04-24T23:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f6e4effd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50154720 0x7f6e500b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f6e4effd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50154720 0x7f6e500b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38118/0 (socket says 172.21.15.26:38118) 2024-04-24T23:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f4547fff640 1 -- 172.21.15.26:0/3605607735 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342916959 0 0) 0x7f4540001110 con 0x7f4548007820 2024-04-24T23:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7f4547fff640 1 -- 172.21.15.26:0/3605607735 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f453c003520 con 0x7f4548007820 2024-04-24T23:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f45578e7640 1 -- 172.21.15.26:0/3605607735 >> v1:172.21.15.29:6789/0 conn(0x7f4548007820 legacy=0x7f4548038040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f45578e7640 1 -- 172.21.15.26:0/3605607735 >> v1:172.21.15.26:6789/0 conn(0x7f45480583c0 legacy=0x7f45480587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f45578e7640 1 -- 172.21.15.26:0/3605607735 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45480542c0 con 0x7f454803c150 2024-04-24T23:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7fa0a5c0e640 1 -- 172.21.15.26:0/1774014916 >> v2:172.21.15.29:3300/0 conn(0x7fa0a00b4380 msgr2=0x7fa0a006e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7fa0a5c0e640 1 --2- 172.21.15.26:0/1774014916 >> v2:172.21.15.29:3300/0 conn(0x7fa0a00b4380 0x7fa0a006e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7fa0a5c0e640 1 -- 172.21.15.26:0/1774014916 >> v1:172.21.15.29:6789/0 conn(0x7fa0a006e900 legacy=0x7fa0a0166050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7fa0a5c0e640 1 -- 172.21.15.26:0/1774014916 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a006d890 con 0x7fa0a014c040 2024-04-24T23:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f1a7e7fc640 1 -- 172.21.15.26:0/3648752172 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1a800d0f30 msgr2=0x7f1a8006e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f1a7e7fc640 1 --2- 172.21.15.26:0/3648752172 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1a800d0f30 0x7f1a8006e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f1a7e7fc640 1 -- 172.21.15.26:0/3648752172 >> v1:172.21.15.29:6789/0 conn(0x7f1a800af6d0 legacy=0x7f1a800b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f1a7e7fc640 1 -- 172.21.15.26:0/3648752172 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8006fbd0 con 0x7f1a8006f0e0 2024-04-24T23:21:04.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f6e4effd640 1 -- 172.21.15.26:0/50258965 learned_addr learned my addr 172.21.15.26:0/50258965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f6e4dffb640 1 -- 172.21.15.26:0/50258965 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766693326 0 0) 0x7f6e40001110 con 0x7f6e50071440 2024-04-24T23:21:04.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f6e4dffb640 1 -- 172.21.15.26:0/50258965 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e38003520 con 0x7f6e50071440 2024-04-24T23:21:04.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7f6e4effd640 1 -- 172.21.15.26:0/50258965 >> v1:172.21.15.29:6789/0 conn(0x7f6e50071440 legacy=0x7f6e5015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7ff1fb59e640 1 Processor -- start 2024-04-24T23:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7f6e4effd640 1 -- 172.21.15.26:0/50258965 >> v1:172.21.15.26:6789/0 conn(0x7f6e500b4380 legacy=0x7f6e50070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7f45578e7640 1 --2- 172.21.15.26:0/3605607735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 0x7f454803c5e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4550050770 tx=0x7f45500a5880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91856361526bff7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.837+0000 7faa1f722640 1 -- 172.21.15.26:0/1837417792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1017dc40 con 0x7faa10038100 2024-04-24T23:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7faa1f722640 1 -- 172.21.15.26:0/1837417792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1017e200 con 0x7faa10038100 2024-04-24T23:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.838+0000 7faa0dffb640 1 -- 172.21.15.26:0/1837417792 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa14011850 con 0x7faa10038100 2024-04-24T23:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7f1a7e7fc640 1 --2- 172.21.15.26:0/3648752172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a8006f0e0 0x7f1a800d09f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009e50 tx=0x7f1a7802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf10f6e5895b96f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7f4547fff640 1 -- 172.21.15.26:0/3605607735 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45500ab640 con 0x7f454803c150 2024-04-24T23:21:04.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7faa0dffb640 1 -- 172.21.15.26:0/1837417792 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14011c80 con 0x7faa10038100 2024-04-24T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7f4547fff640 1 -- 172.21.15.26:0/3605607735 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45500ab7e0 con 0x7f454803c150 2024-04-24T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7f4547fff640 1 -- 172.21.15.26:0/3605607735 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45500abc50 con 0x7f454803c150 2024-04-24T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7faa1f722640 1 -- 172.21.15.26:0/1837417792 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9f8001030 con 0x7faa10038100 2024-04-24T23:21:04.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7ff1fb59e640 1 -- start start 2024-04-24T23:21:04.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7fb51fded640 1 Processor -- start 2024-04-24T23:21:04.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7fb51fded640 1 -- start start 2024-04-24T23:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7f28d7fff640 1 --2- 172.21.15.26:0/270197879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d00df3d0 0x7f28d004fc10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f28d0051410 tx=0x7f28c402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=608abced66c98ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.839+0000 7f6e4effd640 1 -- 172.21.15.26:0/50258965 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5015a9c0 con 0x7f6e50154720 2024-04-24T23:21:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7fa0a5c0e640 1 --2- 172.21.15.26:0/1774014916 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a014c040 0x7fa0a00b3e40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa098009e50 tx=0x7fa09802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e256ac75c765b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f1a7dffb640 1 -- 172.21.15.26:0/3648752172 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78036070 con 0x7f1a8006f0e0 2024-04-24T23:21:04.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f1a7dffb640 1 -- 172.21.15.26:0/3648752172 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a78031600 con 0x7f1a8006f0e0 2024-04-24T23:21:04.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f28d77fe640 1 -- 172.21.15.26:0/270197879 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4037070 con 0x7f28d00df3d0 2024-04-24T23:21:04.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f28d77fe640 1 -- 172.21.15.26:0/270197879 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c4032440 con 0x7f28d00df3d0 2024-04-24T23:21:04.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f1a7dffb640 1 -- 172.21.15.26:0/3648752172 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78031a10 con 0x7f1a8006f0e0 2024-04-24T23:21:04.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7faa0dffb640 1 -- 172.21.15.26:0/1837417792 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa14018020 con 0x7faa10038100 2024-04-24T23:21:04.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f28d77fe640 1 -- 172.21.15.26:0/270197879 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c40327f0 con 0x7f28d00df3d0 2024-04-24T23:21:04.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f4556685640 1 -- 172.21.15.26:0/3605607735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 msgr2=0x7f454803c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f4556685640 1 --2- 172.21.15.26:0/3605607735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 0x7f454803c5e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4550050770 tx=0x7f45500a5880 comp rx=0 tx=0).stop 2024-04-24T23:21:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7ff1fb59e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff1f40eacf0 0x7ff1f404fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7fb51fded640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb5180b3530 0x7fb5180d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7fb51fded640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5180b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7ff1fa59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff1f40eacf0 0x7ff1f404fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7ff1fa59c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff1f40eacf0 0x7ff1f404fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:56548/0 (socket says 172.21.15.26:56548) 2024-04-24T23:21:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f4556685640 1 -- 172.21.15.26:0/3605607735 shutdown_connections 2024-04-24T23:21:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f4556685640 1 --2- 172.21.15.26:0/3605607735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 0x7f454803c5e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.840+0000 7f4556685640 1 -- 172.21.15.26:0/3605607735 >> 172.21.15.26:0/3605607735 conn(0x7f45480590c0 msgr2=0x7f45480594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51d388640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5180b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51d388640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5180b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38134/0 (socket says 172.21.15.26:38134) 2024-04-24T23:21:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51d388640 1 -- 172.21.15.26:0/1274306913 learned_addr learned my addr 172.21.15.26:0/1274306913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51db89640 1 --2- 172.21.15.26:0/1274306913 >> v2:172.21.15.26:3300/0 conn(0x7fb5180b3530 0x7fb5180d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51fded640 1 -- 172.21.15.26:0/1274306913 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5180b4770 con 0x7fb5180d27e0 2024-04-24T23:21:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51fded640 1 -- 172.21.15.26:0/1274306913 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb5180cee30 con 0x7fb5180b3530 2024-04-24T23:21:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51fded640 1 -- 172.21.15.26:0/1274306913 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb5180cf0f0 con 0x7fb5180b3910 2024-04-24T23:21:04.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7ff1fa59c640 1 -- 172.21.15.26:0/2714601330 learned_addr learned my addr 172.21.15.26:0/2714601330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f4556685640 1 -- 172.21.15.26:0/3605607735 shutdown_connections 2024-04-24T23:21:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7faa0dffb640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9fc06be30 0x7fa9fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3648752172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a8006f0e0 msgr2=0x7f1a800d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f1a7ffff640 1 --2- 172.21.15.26:0/3648752172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a8006f0e0 0x7f1a800d09f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009e50 tx=0x7f1a7802a730 comp rx=0 tx=0).stop 2024-04-24T23:21:04.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f28dec9a640 1 -- 172.21.15.26:0/270197879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d00df3d0 msgr2=0x7f28d004fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f28dec9a640 1 --2- 172.21.15.26:0/270197879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d00df3d0 0x7f28d004fc10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f28d0051410 tx=0x7f28c402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f28dec9a640 1 -- 172.21.15.26:0/270197879 shutdown_connections 2024-04-24T23:21:04.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2714601330 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f4053710 con 0x7ff1f404ffc0 2024-04-24T23:21:04.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2714601330 --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f40539f0 con 0x7ff1f4006ee0 2024-04-24T23:21:04.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f6e4effd640 1 --2- 172.21.15.26:0/50258965 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50154720 0x7f6e500b3e40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800d3a0 tx=0x7f6e4802cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944462a25a400e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f28dec9a640 1 --2- 172.21.15.26:0/270197879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d00df3d0 0x7f28d004fc10 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f28d0051410 tx=0x7f28c402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f28dec9a640 1 --2- 172.21.15.26:0/270197879 >> v2:172.21.15.26:3300/0 conn(0x7f28d00deff0 0x7f28d004f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2714601330 --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f4054320 con 0x7ff1f40eacf0 2024-04-24T23:21:04.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f28dec9a640 1 -- 172.21.15.26:0/270197879 >> 172.21.15.26:0/270197879 conn(0x7f28d000a3f0 msgr2=0x7f28d000a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fa0a4c0c640 1 -- 172.21.15.26:0/1774014916 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098036070 con 0x7fa0a014c040 2024-04-24T23:21:04.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f4556685640 1 -- 172.21.15.26:0/3605607735 wait complete. 2024-04-24T23:21:04.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7f6e4dffb640 1 -- 172.21.15.26:0/50258965 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800be60 con 0x7f6e50154720 2024-04-24T23:21:04.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fa0a4c0c640 1 -- 172.21.15.26:0/1774014916 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa098031600 con 0x7fa0a014c040 2024-04-24T23:21:04.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fa0a4c0c640 1 -- 172.21.15.26:0/1774014916 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980319d0 con 0x7fa0a014c040 2024-04-24T23:21:04.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7faa0dffb640 1 -- 172.21.15.26:0/1837417792 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faa1408acc0 con 0x7faa10038100 2024-04-24T23:21:04.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7faa0dffb640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7fa9fc074030 0x7fa9fc076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7f6e4dffb640 1 -- 172.21.15.26:0/50258965 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e4802e8c0 con 0x7f6e50154720 2024-04-24T23:21:04.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7f28dec9a640 1 -- 172.21.15.26:0/270197879 shutdown_connections 2024-04-24T23:21:04.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7f6e4dffb640 1 -- 172.21.15.26:0/50258965 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4802ec90 con 0x7f6e50154720 2024-04-24T23:21:04.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7f28dec9a640 1 -- 172.21.15.26:0/270197879 wait complete. 2024-04-24T23:21:04.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7faa0dffb640 1 -- 172.21.15.26:0/1837417792 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa14004510 con 0x7fa9fc074030 2024-04-24T23:21:04.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7faa0dffb640 1 -- 172.21.15.26:0/1837417792 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7faa1408f030 con 0x7faa10038100 2024-04-24T23:21:04.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7faa1ccbd640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9fc06be30 0x7fa9fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7faa1dcbf640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7fa9fc074030 0x7fa9fc076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51d388640 1 -- 172.21.15.26:0/1274306913 >> v2:172.21.15.26:3300/0 conn(0x7fb5180b3530 msgr2=0x7fb5180d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:04.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51d388640 1 --2- 172.21.15.26:0/1274306913 >> v2:172.21.15.26:3300/0 conn(0x7fb5180b3530 0x7fb5180d21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51d388640 1 -- 172.21.15.26:0/1274306913 >> v1:172.21.15.26:6789/0 conn(0x7fb5180d27e0 legacy=0x7fb518166530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.841+0000 7fb51d388640 1 -- 172.21.15.26:0/1274306913 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5180b4a10 con 0x7fb5180b3910 2024-04-24T23:21:04.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3648752172 shutdown_connections 2024-04-24T23:21:04.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7f1a7ffff640 1 --2- 172.21.15.26:0/3648752172 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1a800d0f30 0x7f1a8006e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7f1a7ffff640 1 --2- 172.21.15.26:0/3648752172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a8006f0e0 0x7f1a800d09f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3648752172 >> 172.21.15.26:0/3648752172 conn(0x7f1a8006c230 msgr2=0x7f1a8006c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7faa1dcbf640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7fa9fc074030 0x7fa9fc076450 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7ff1fa59c640 1 -- 172.21.15.26:0/2714601330 >> v1:172.21.15.29:6789/0 conn(0x7ff1f4006ee0 legacy=0x7ff1f40382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7ff1fa59c640 1 -- 172.21.15.26:0/2714601330 >> v1:172.21.15.26:6789/0 conn(0x7ff1f404ffc0 legacy=0x7ff1f4037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7ff1fa59c640 1 -- 172.21.15.26:0/2714601330 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f4053cd0 con 0x7ff1f40eacf0 2024-04-24T23:21:04.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7ff1f959a640 1 -- 172.21.15.26:0/2714601330 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3309000611 0 0) 0x7ff1f40539f0 con 0x7ff1f4006ee0 2024-04-24T23:21:04.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7fa0a7e72640 1 -- 172.21.15.26:0/1774014916 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a014c040 msgr2=0x7fa0a00b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.842+0000 7fa0a7e72640 1 --2- 172.21.15.26:0/1774014916 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a014c040 0x7fa0a00b3e40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa098009e50 tx=0x7fa09802a6d0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7fa0a7e72640 1 -- 172.21.15.26:0/1774014916 shutdown_connections 2024-04-24T23:21:04.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7fa0a7e72640 1 --2- 172.21.15.26:0/1774014916 >> v2:172.21.15.29:3300/0 conn(0x7fa0a00b4380 0x7fa0a006e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7fa0a7e72640 1 --2- 172.21.15.26:0/1774014916 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a014c040 0x7fa0a00b3e40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7fa0a7e72640 1 -- 172.21.15.26:0/1774014916 >> 172.21.15.26:0/1774014916 conn(0x7fa0a006b910 msgr2=0x7fa0a00b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f6e55789640 1 -- 172.21.15.26:0/50258965 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50154720 msgr2=0x7f6e500b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f6e55789640 1 --2- 172.21.15.26:0/50258965 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50154720 0x7f6e500b3e40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800d3a0 tx=0x7f6e4802cb60 comp rx=0 tx=0).stop 2024-04-24T23:21:04.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f4556685640 1 Processor -- start 2024-04-24T23:21:04.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3648752172 shutdown_connections 2024-04-24T23:21:04.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f4556685640 1 -- start start 2024-04-24T23:21:04.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3648752172 wait complete. 2024-04-24T23:21:04.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f6e55789640 1 -- 172.21.15.26:0/50258965 shutdown_connections 2024-04-24T23:21:04.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f6e55789640 1 --2- 172.21.15.26:0/50258965 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50154720 0x7f6e500b3e40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f6e55789640 1 -- 172.21.15.26:0/50258965 >> 172.21.15.26:0/50258965 conn(0x7f6e5006b910 msgr2=0x7f6e500b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7fa0a7e72640 1 -- 172.21.15.26:0/1774014916 shutdown_connections 2024-04-24T23:21:04.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7fa0a7e72640 1 -- 172.21.15.26:0/1774014916 wait complete. 2024-04-24T23:21:04.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f6e55789640 1 -- 172.21.15.26:0/50258965 shutdown_connections 2024-04-24T23:21:04.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7faa0dffb640 1 -- 172.21.15.26:0/1837417792 <== osd.0 v2:172.21.15.26:6819/256768257 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faa14004510 con 0x7fa9fc074030 2024-04-24T23:21:04.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7fb51d388640 1 --2- 172.21.15.26:0/1274306913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5180b4230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb508009870 tx=0x7fb50802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc820f5b0b93053 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7ff1fa59c640 1 --2- 172.21.15.26:0/2714601330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 0x7ff1f404fa80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009870 tx=0x7ff1ec02a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3517ee62981b952 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4556685640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4548007820 0x7f45480d8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f28dec9a640 1 Processor -- start 2024-04-24T23:21:04.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7fb51cb87640 1 -- 172.21.15.26:0/1274306913 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508037070 con 0x7fb5180b3910 2024-04-24T23:21:04.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4556685640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 0x7f4548180bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4556685640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480583c0 0x7f454817ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4556685640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4548007300 con 0x7f45480583c0 2024-04-24T23:21:04.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4556685640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4548007020 con 0x7f4548007820 2024-04-24T23:21:04.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4556685640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4548007ca0 con 0x7f454803c150 2024-04-24T23:21:04.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7fb51cb87640 1 -- 172.21.15.26:0/1274306913 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb508032440 con 0x7fb5180b3910 2024-04-24T23:21:04.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7faa1ccbd640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9fc06be30 0x7fa9fc06e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa180502b0 tx=0x7faa18069d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7fb51cb87640 1 -- 172.21.15.26:0/1274306913 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508032810 con 0x7fb5180b3910 2024-04-24T23:21:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4554e82640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 0x7f4548180bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4555683640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4548007820 0x7f45480d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4555683640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4548007820 0x7f45480d8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:56550/0 (socket says 172.21.15.26:56550) 2024-04-24T23:21:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f45578e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480583c0 0x7f454817ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7f4555683640 1 -- 172.21.15.26:0/2226574997 learned_addr learned my addr 172.21.15.26:0/2226574997 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7ff1f959a640 1 -- 172.21.15.26:0/2714601330 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec037070 con 0x7ff1f40eacf0 2024-04-24T23:21:04.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7ff1f959a640 1 -- 172.21.15.26:0/2714601330 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec032640 con 0x7ff1f40eacf0 2024-04-24T23:21:04.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.844+0000 7ff1f959a640 1 -- 172.21.15.26:0/2714601330 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec032a10 con 0x7ff1f40eacf0 2024-04-24T23:21:04.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f1a7ffff640 1 Processor -- start 2024-04-24T23:21:04.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f1a7ffff640 1 -- start start 2024-04-24T23:21:04.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7fa0a7e72640 1 Processor -- start 2024-04-24T23:21:04.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7fa0a7e72640 1 -- start start 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4554e82640 1 -- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4548007820 msgr2=0x7f45480d8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4554e82640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4548007820 0x7f45480d8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4554e82640 1 -- 172.21.15.26:0/2226574997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480583c0 msgr2=0x7f454817ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4554e82640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45480583c0 0x7f454817ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4554e82640 1 -- 172.21.15.26:0/2226574997 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4548181490 con 0x7f454803c150 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4554e82640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 0x7f4548180bb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f454000c390 tx=0x7f454000c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4545ffb640 1 -- 172.21.15.26:0/2226574997 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540017070 con 0x7f454803c150 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4545ffb640 1 -- 172.21.15.26:0/2226574997 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4540005ce0 con 0x7f454803c150 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4545ffb640 1 -- 172.21.15.26:0/2226574997 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45400072d0 con 0x7f454803c150 2024-04-24T23:21:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4548181770 con 0x7f454803c150 2024-04-24T23:21:04.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7fb51fded640 1 -- 172.21.15.26:0/1274306913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 msgr2=0x7fb5180b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7fb51fded640 1 --2- 172.21.15.26:0/1274306913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5180b4230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb508009870 tx=0x7fb50802a250 comp rx=0 tx=0).stop 2024-04-24T23:21:04.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2714601330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 msgr2=0x7ff1f404fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7ff1fb59e640 1 --2- 172.21.15.26:0/2714601330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 0x7ff1f404fa80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009870 tx=0x7ff1ec02a2f0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f454817d6d0 con 0x7f454803c150 2024-04-24T23:21:04.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.843+0000 7f6e55789640 1 -- 172.21.15.26:0/50258965 wait complete. 2024-04-24T23:21:04.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fb51fded640 1 -- 172.21.15.26:0/1274306913 shutdown_connections 2024-04-24T23:21:04.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2714601330 shutdown_connections 2024-04-24T23:21:04.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7ff1fb59e640 1 --2- 172.21.15.26:0/2714601330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 0x7ff1f404fa80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2714601330 >> 172.21.15.26:0/2714601330 conn(0x7ff1f4009fa0 msgr2=0x7ff1f400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fb51fded640 1 --2- 172.21.15.26:0/1274306913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5180b4230 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fb51fded640 1 --2- 172.21.15.26:0/1274306913 >> v2:172.21.15.26:3300/0 conn(0x7fb5180b3530 0x7fb5180d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a7e72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a006e900 0x7fa0a014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fb51fded640 1 -- 172.21.15.26:0/1274306913 >> 172.21.15.26:0/1274306913 conn(0x7fb51808dbd0 msgr2=0x7fb51809d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f6e55789640 1 Processor -- start 2024-04-24T23:21:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.845+0000 7f6e55789640 1 -- start start 2024-04-24T23:21:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a7e72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a00b4380 0x7fa0a0158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a7e72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0a014c040 0x7fa0a0139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a7e72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a006ef10 con 0x7fa0a00b4380 2024-04-24T23:21:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a7e72640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a006ed90 con 0x7fa0a014c040 2024-04-24T23:21:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a7e72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a0071d40 con 0x7fa0a006e900 2024-04-24T23:21:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a8006f0e0 0x7f1a8009e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a800af6d0 0x7f1a801f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a5c0e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a006e900 0x7fa0a014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1a800d0f30 0x7f1a8015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7f1a7effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a8006f0e0 0x7f1a8009e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a5c0e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a006e900 0x7fa0a014ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38152/0 (socket says 172.21.15.26:38152) 2024-04-24T23:21:04.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a5c0e640 1 -- 172.21.15.26:0/1442602554 learned_addr learned my addr 172.21.15.26:0/1442602554 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7fa0a640f640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0a014c040 0x7fa0a0139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7f1a7effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a8006f0e0 0x7f1a8009e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:56582/0 (socket says 172.21.15.26:56582) 2024-04-24T23:21:04.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7f1a7effd640 1 -- 172.21.15.26:0/3245848534 learned_addr learned my addr 172.21.15.26:0/3245848534 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fa0a540d640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a00b4380 0x7fa0a0158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7f1a7ffff640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8006e940 con 0x7f1a800af6d0 2024-04-24T23:21:04.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f1a7e7fc640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a800af6d0 0x7f1a801f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.846+0000 7f6e55789640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50071440 0x7f6e50148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e55789640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e500b4380 0x7f6e50147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e55789640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e50154720 0x7f6e50147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e55789640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5015a660 con 0x7f6e500b4380 2024-04-24T23:21:04.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e55789640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5015af80 con 0x7f6e50154720 2024-04-24T23:21:04.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e55789640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5015b4a0 con 0x7f6e50071440 2024-04-24T23:21:04.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f1a7f7fe640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1a800d0f30 0x7f1a8015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e4effd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50071440 0x7f6e50148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e4effd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50071440 0x7f6e50148630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38164/0 (socket says 172.21.15.26:38164) 2024-04-24T23:21:04.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e4effd640 1 -- 172.21.15.26:0/1425901843 learned_addr learned my addr 172.21.15.26:0/1425901843 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800afab0 con 0x7f1a8006f0e0 2024-04-24T23:21:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8006eac0 con 0x7f1a800d0f30 2024-04-24T23:21:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fa0a5c0e640 1 -- 172.21.15.26:0/1442602554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0a014c040 msgr2=0x7fa0a0139640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fa0a5c0e640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0a014c040 0x7fa0a0139640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fa0a5c0e640 1 -- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a00b4380 msgr2=0x7fa0a0158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fa0a5c0e640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a00b4380 0x7fa0a0158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fa0a5c0e640 1 -- 172.21.15.26:0/1442602554 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a015a050 con 0x7fa0a006e900 2024-04-24T23:21:04.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fa0a540d640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0a00b4380 0x7fa0a0158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:04.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2714601330 shutdown_connections 2024-04-24T23:21:04.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e4f7fe640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e50154720 0x7f6e50147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fb51fded640 1 -- 172.21.15.26:0/1274306913 shutdown_connections 2024-04-24T23:21:04.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fb51fded640 1 -- 172.21.15.26:0/1274306913 wait complete. 2024-04-24T23:21:04.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f6e4e7fc640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e500b4380 0x7f6e50147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7fa0a5c0e640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a006e900 0x7fa0a014ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa098009f70 tx=0x7fa098004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f4545ffb640 1 -- 172.21.15.26:0/2226574997 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f454001a070 con 0x7f454803c150 2024-04-24T23:21:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f1a7effd640 1 -- 172.21.15.26:0/3245848534 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1a800d0f30 msgr2=0x7f1a8015a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f6e4effd640 1 -- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e50154720 msgr2=0x7f6e50147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f6e4effd640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e50154720 0x7f6e50147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f6e4effd640 1 -- 172.21.15.26:0/1425901843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e500b4380 msgr2=0x7f6e50147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7fa0967fc640 1 -- 172.21.15.26:0/1442602554 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098041020 con 0x7fa0a006e900 2024-04-24T23:21:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7fa0967fc640 1 -- 172.21.15.26:0/1442602554 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0980087b0 con 0x7fa0a006e900 2024-04-24T23:21:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7fa0967fc640 1 -- 172.21.15.26:0/1442602554 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098036040 con 0x7fa0a006e900 2024-04-24T23:21:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f1a7effd640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1a800d0f30 0x7f1a8015a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7fa0a7e72640 1 -- 172.21.15.26:0/1442602554 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a0139e90 con 0x7fa0a006e900 2024-04-24T23:21:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45480551e0 con 0x7f454803c150 2024-04-24T23:21:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f1a7effd640 1 -- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a800af6d0 msgr2=0x7f1a801f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f6e4effd640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e500b4380 0x7f6e50147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f6e4effd640 1 -- 172.21.15.26:0/1425901843 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e500ce350 con 0x7f6e50071440 2024-04-24T23:21:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f1a7effd640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a800af6d0 0x7f1a801f84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f6e4e7fc640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e500b4380 0x7f6e50147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7fa0a7e72640 1 -- 172.21.15.26:0/1442602554 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a013a450 con 0x7fa0a006e900 2024-04-24T23:21:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7f1a7effd640 1 -- 172.21.15.26:0/3245848534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a800b1ac0 con 0x7f1a8006f0e0 2024-04-24T23:21:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f1a7effd640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a8006f0e0 0x7f1a8009e3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c450 tx=0x7f1a7000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f1a67fff640 1 -- 172.21.15.26:0/3245848534 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a700052d0 con 0x7f1a8006f0e0 2024-04-24T23:21:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a801f8a00 con 0x7f1a8006f0e0 2024-04-24T23:21:04.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f6e4effd640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50071440 0x7f6e50148630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007210 tx=0x7f6e48004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.847+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2714601330 wait complete. 2024-04-24T23:21:04.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a800b1d30 con 0x7f1a8006f0e0 2024-04-24T23:21:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f4545ffb640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453406be30 0x7f453406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f4545ffb640 1 -- 172.21.15.26:0/2226574997 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f454008a050 con 0x7f454803c150 2024-04-24T23:21:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.849+0000 7f4545ffb640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f4534073fd0 0x7f45340763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.849+0000 7f28dec9a640 1 -- start start 2024-04-24T23:21:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f1a67fff640 1 -- 172.21.15.26:0/3245848534 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a70005470 con 0x7f1a8006f0e0 2024-04-24T23:21:04.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.848+0000 7f1a67fff640 1 -- 172.21.15.26:0/3245848534 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70012830 con 0x7f1a8006f0e0 2024-04-24T23:21:04.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.849+0000 7fa0a7e72640 1 -- 172.21.15.26:0/1442602554 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0a015a410 con 0x7fa0a006e900 2024-04-24T23:21:04.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.849+0000 7ff1fb59e640 1 Processor -- start 2024-04-24T23:21:04.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.849+0000 7f6e377fe640 1 -- 172.21.15.26:0/1425901843 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48045070 con 0x7f6e50071440 2024-04-24T23:21:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.849+0000 7f6e377fe640 1 -- 172.21.15.26:0/1425901843 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e48008040 con 0x7f6e50071440 2024-04-24T23:21:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e500ce630 con 0x7f6e50071440 2024-04-24T23:21:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e500cebf0 con 0x7f6e50071440 2024-04-24T23:21:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.849+0000 7f4555683640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453406be30 0x7f453406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7f4555683640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453406be30 0x7f453406e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f454c005020 tx=0x7f454c007ef0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7f4545ffb640 1 -- 172.21.15.26:0/2226574997 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f454000ee50 con 0x7f4534073fd0 2024-04-24T23:21:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7f4545ffb640 1 -- 172.21.15.26:0/2226574997 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4540054470 con 0x7f454803c150 2024-04-24T23:21:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7fb51fded640 1 Processor -- start 2024-04-24T23:21:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7fb51fded640 1 -- start start 2024-04-24T23:21:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7f45578e7640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f4534073fd0 0x7f45340763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.849+0000 7ff1fb59e640 1 -- start start 2024-04-24T23:21:04.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7f1a67fff640 1 -- 172.21.15.26:0/3245848534 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a700129d0 con 0x7f1a8006f0e0 2024-04-24T23:21:04.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7ff1fb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f4006ee0 0x7ff1f4045930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7ff1fb59e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1f404ffc0 0x7ff1f4048800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7ff1fb59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 0x7ff1f40d8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fb59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4054930 con 0x7ff1f4006ee0 2024-04-24T23:21:04.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fb59e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40547b0 con 0x7ff1f40eacf0 2024-04-24T23:21:04.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fb59e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f404f540 con 0x7ff1f404ffc0 2024-04-24T23:21:04.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a60001030 con 0x7f1a8006f0e0 2024-04-24T23:21:04.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7f45578e7640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f4534073fd0 0x7f45340763f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f4006ee0 0x7ff1f4045930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 0x7ff1f40d8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 0x7ff1f40d8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:56600/0 (socket says 172.21.15.26:56600) 2024-04-24T23:21:04.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fad9d640 1 -- 172.21.15.26:0/2832315655 learned_addr learned my addr 172.21.15.26:0/2832315655 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1f9d9b640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1f404ffc0 0x7ff1f4048800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7fa0967fc640 1 -- 172.21.15.26:0/1442602554 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa098044020 con 0x7fa0a006e900 2024-04-24T23:21:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7fb51fded640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5180b3530 0x7fb5180caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fa59c640 1 -- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1f404ffc0 msgr2=0x7ff1f4048800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fa59c640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1f404ffc0 0x7ff1f4048800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fa59c640 1 -- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 msgr2=0x7ff1f40d8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fa59c640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 0x7ff1f40d8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fa59c640 1 -- 172.21.15.26:0/2832315655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f40497b0 con 0x7ff1f4006ee0 2024-04-24T23:21:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7ff1fad9d640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1f40eacf0 0x7ff1f40d8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:04.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7f1a67fff640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a6c06bf00 0x7f1a6c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51fded640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5181635a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7f1a67fff640 1 -- 172.21.15.26:0/3245848534 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1a7008a9c0 con 0x7f1a8006f0e0 2024-04-24T23:21:04.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7f4545ffb640 1 -- 172.21.15.26:0/2226574997 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f454000ee50 con 0x7f4534073fd0 2024-04-24T23:21:04.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51db89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5180b3530 0x7fb5180caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51db89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5180b3530 0x7fb5180caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40002/0 (socket says 172.21.15.26:40002) 2024-04-24T23:21:04.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7f1a67fff640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f1a6c074120 0x7f1a6c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7fa0967fc640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa08406bf00 0x7fa08406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fa0967fc640 1 -- 172.21.15.26:0/1442602554 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa0980ab8f0 con 0x7fa0a006e900 2024-04-24T23:21:04.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fa0967fc640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7fa084074120 0x7fa084076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fa0967fc640 1 -- 172.21.15.26:0/1442602554 --> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa084076bf0 con 0x7fa084074120 2024-04-24T23:21:04.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fa0967fc640 1 -- 172.21.15.26:0/1442602554 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa0980b02a0 con 0x7fa0a006e900 2024-04-24T23:21:04.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7f1a67fff640 1 -- 172.21.15.26:0/3245848534 --> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a6c076bf0 con 0x7f1a6c074120 2024-04-24T23:21:04.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7f1a67fff640 1 -- 172.21.15.26:0/3245848534 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1a70091050 con 0x7f1a8006f0e0 2024-04-24T23:21:04.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.850+0000 7f6e377fe640 1 -- 172.21.15.26:0/1425901843 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48008410 con 0x7f6e50071440 2024-04-24T23:21:04.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.851+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e500cefd0 con 0x7f6e50071440 2024-04-24T23:21:04.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7ff1fa59c640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f4006ee0 0x7ff1f4045930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec031000 tx=0x7ff1ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7f6e377fe640 1 -- 172.21.15.26:0/1425901843 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e48048070 con 0x7f6e50071440 2024-04-24T23:21:04.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7f1a7e7fc640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a6c06bf00 0x7f1a6c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51fded640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5180d27e0 0x7fb518165170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51d388640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5181635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51d388640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5181635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38200/0 (socket says 172.21.15.26:38200) 2024-04-24T23:21:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51db89640 1 -- 172.21.15.26:0/1754747243 learned_addr learned my addr 172.21.15.26:0/1754747243 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51e38a640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5180d27e0 0x7fb518165170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51fded640 1 -- 172.21.15.26:0/1754747243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb5180b4fd0 con 0x7fb5180b3530 2024-04-24T23:21:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fa0a640f640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7fa084074120 0x7fa084076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fa0a540d640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa08406bf00 0x7fa08406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51fded640 1 -- 172.21.15.26:0/1754747243 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb5180b52b0 con 0x7fb5180d27e0 2024-04-24T23:21:04.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7fb51fded640 1 -- 172.21.15.26:0/1754747243 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb5180d0590 con 0x7fb5180b3910 2024-04-24T23:21:04.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.852+0000 7f6e377fe640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3c06bf00 0x7f6e3c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7fa0a640f640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7fa084074120 0x7fa084076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7f6e4e7fc640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3c06bf00 0x7f6e3c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7f6e377fe640 1 -- 172.21.15.26:0/1425901843 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e480ab230 con 0x7f6e50071440 2024-04-24T23:21:04.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7f1a7f7fe640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f1a6c074120 0x7f1a6c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7f6e377fe640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f6e3c0740a0 0x7f6e3c0764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7f6e377fe640 1 -- 172.21.15.26:0/1425901843 --> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e48008040 con 0x7f6e3c0740a0 2024-04-24T23:21:04.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7f6e377fe640 1 -- 172.21.15.26:0/1425901843 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6e480b2050 con 0x7f6e50071440 2024-04-24T23:21:04.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7ff1eaffd640 1 -- 172.21.15.26:0/2832315655 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec041070 con 0x7ff1f4006ee0 2024-04-24T23:21:04.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7ff1eaffd640 1 -- 172.21.15.26:0/2832315655 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec030980 con 0x7ff1f4006ee0 2024-04-24T23:21:04.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7ff1eaffd640 1 -- 172.21.15.26:0/2832315655 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec037040 con 0x7ff1f4006ee0 2024-04-24T23:21:04.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7fb51d388640 1 -- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5180d27e0 msgr2=0x7fb518165170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7fb51d388640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5180d27e0 0x7fb518165170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7fb51d388640 1 -- 172.21.15.26:0/1754747243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5180b3530 msgr2=0x7fb5180caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f40491f0 con 0x7ff1f4006ee0 2024-04-24T23:21:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f4043710 con 0x7ff1f4006ee0 2024-04-24T23:21:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f6e4e7fc640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3c06bf00 0x7f6e3c06e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e44007fd0 tx=0x7f6e44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.853+0000 7fa0a540d640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa08406bf00 0x7fa08406e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0a01fb860 tx=0x7fa09c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f1a7e7fc640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a6c06bf00 0x7f1a6c06e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a78033000 tx=0x7f1a78009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7fa0967fc640 1 -- 172.21.15.26:0/1442602554 <== osd.2 v2:172.21.15.26:6800/3780187135 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa084076bf0 con 0x7fa084074120 2024-04-24T23:21:04.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f6e4f7fe640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f6e3c0740a0 0x7f6e3c0764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f1a7f7fe640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f1a6c074120 0x7f1a6c076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7fb51d388640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5180b3530 0x7fb5180caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f28dec9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0005eb0 0x7f28d00d55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f28dec9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d00deff0 0x7f28d00d40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f28dec9a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28d0034920 0x7f28d0036310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f28dec9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0007b90 con 0x7f28d0005eb0 2024-04-24T23:21:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f28dec9a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0007670 con 0x7f28d00deff0 2024-04-24T23:21:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f28dec9a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f28d0039070 con 0x7f28d0034920 2024-04-24T23:21:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f28d7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d00deff0 0x7f28d00d40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f28d7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d00deff0 0x7f28d00d40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:56630/0 (socket says 172.21.15.26:56630) 2024-04-24T23:21:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7f28d7fff640 1 -- 172.21.15.26:0/1278483813 learned_addr learned my addr 172.21.15.26:0/1278483813 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f28dd237640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28d0034920 0x7f28d0036310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f6e4f7fe640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f6e3c0740a0 0x7f6e3c0764c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1f40052b0 con 0x7ff1f4006ee0 2024-04-24T23:21:04.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7fb51d388640 1 -- 172.21.15.26:0/1754747243 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5181604a0 con 0x7fb5180b3910 2024-04-24T23:21:04.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.854+0000 7fb51db89640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5180b3530 0x7fb5180caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:04.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7fb51d388640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5181635a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb508031000 tx=0x7fb5080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f28dca36640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0005eb0 0x7f28d00d55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f1a67fff640 1 -- 172.21.15.26:0/3245848534 <== osd.1 v2:172.21.15.26:6816/1500554534 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a6c076bf0 con 0x7f1a6c074120 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f28dd237640 1 -- 172.21.15.26:0/1278483813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d00deff0 msgr2=0x7f28d00d40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f28dd237640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28d00deff0 0x7f28d00d40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f28dd237640 1 -- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0005eb0 msgr2=0x7f28d00d55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f28dd237640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0005eb0 0x7f28d00d55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f28dd237640 1 -- 172.21.15.26:0/1278483813 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d00d5b30 con 0x7f28d0034920 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f28dca36640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0005eb0 0x7f28d00d55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7f28dd237640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28d0034920 0x7f28d0036310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f28d80502b0 tx=0x7f28d8071610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.856+0000 7f28d57fa640 1 -- 172.21.15.26:0/1278483813 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8071c90 con 0x7f28d0034920 2024-04-24T23:21:04.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.856+0000 7f28d57fa640 1 -- 172.21.15.26:0/1278483813 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d8073320 con 0x7f28d0034920 2024-04-24T23:21:04.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.856+0000 7f28d57fa640 1 -- 172.21.15.26:0/1278483813 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8073550 con 0x7f28d0034920 2024-04-24T23:21:04.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.856+0000 7f6e377fe640 1 -- 172.21.15.26:0/1425901843 <== osd.5 v2:172.21.15.29:6800/1108214380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e48008040 con 0x7f6e3c0740a0 2024-04-24T23:21:04.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7fb50e7fc640 1 -- 172.21.15.26:0/1754747243 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508041020 con 0x7fb5180b3910 2024-04-24T23:21:04.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7fb50e7fc640 1 -- 172.21.15.26:0/1754747243 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb508030990 con 0x7fb5180b3910 2024-04-24T23:21:04.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7fb50e7fc640 1 -- 172.21.15.26:0/1754747243 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508037040 con 0x7fb5180b3910 2024-04-24T23:21:04.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7fb51fded640 1 -- 172.21.15.26:0/1754747243 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51815f400 con 0x7fb5180b3910 2024-04-24T23:21:04.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.855+0000 7fb51fded640 1 -- 172.21.15.26:0/1754747243 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51815f9c0 con 0x7fb5180b3910 2024-04-24T23:21:04.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.857+0000 7fb4fbfff640 1 -- 172.21.15.26:0/1754747243 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5180681e0 con 0x7fb5180b3910 2024-04-24T23:21:04.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.857+0000 7ff1eaffd640 1 -- 172.21.15.26:0/2832315655 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1ec044070 con 0x7ff1f4006ee0 2024-04-24T23:21:04.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.857+0000 7f28dec9a640 1 -- 172.21.15.26:0/1278483813 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d0034e20 con 0x7f28d0034920 2024-04-24T23:21:04.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.857+0000 7f28dec9a640 1 -- 172.21.15.26:0/1278483813 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d00353e0 con 0x7f28d0034920 2024-04-24T23:21:04.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.857+0000 7ff1eaffd640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1dc06bf00 0x7ff1dc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.857+0000 7ff1eaffd640 1 -- 172.21.15.26:0/2832315655 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff1ec0abda0 con 0x7ff1f4006ee0 2024-04-24T23:21:04.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.858+0000 7f28c2ffd640 1 -- 172.21.15.26:0/1278483813 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28d00052b0 con 0x7f28d0034920 2024-04-24T23:21:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.858+0000 7ff1eaffd640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7ff1dc074120 0x7ff1dc076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.858+0000 7ff1eaffd640 1 -- 172.21.15.26:0/2832315655 --> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1ec04ce80 con 0x7ff1dc074120 2024-04-24T23:21:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.858+0000 7ff1eaffd640 1 -- 172.21.15.26:0/2832315655 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff1ec0b0030 con 0x7ff1f4006ee0 2024-04-24T23:21:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.858+0000 7ff1fad9d640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7ff1dc074120 0x7ff1dc076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.858+0000 7ff1f9d9b640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1dc06bf00 0x7ff1dc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.858+0000 7fb50e7fc640 1 -- 172.21.15.26:0/1754747243 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb508044020 con 0x7fb5180b3910 2024-04-24T23:21:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.859+0000 7fb50e7fc640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4fc06bf30 0x7fb4fc06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.859+0000 7ff1fad9d640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7ff1dc074120 0x7ff1dc076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.859+0000 7ff1f9d9b640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1dc06bf00 0x7ff1dc06e3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc064c60 tx=0x7ff1fc069d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.859+0000 7fb50e7fc640 1 -- 172.21.15.26:0/1754747243 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb5080ab630 con 0x7fb5180b3910 2024-04-24T23:21:04.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.859+0000 7fb50e7fc640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7fb4fc074130 0x7fb4fc076550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.860+0000 7fb50e7fc640 1 -- 172.21.15.26:0/1754747243 --> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb508032440 con 0x7fb4fc074130 2024-04-24T23:21:04.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.860+0000 7fb50e7fc640 1 -- 172.21.15.26:0/1754747243 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb5080b2050 con 0x7fb5180b3910 2024-04-24T23:21:04.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.860+0000 7f28d57fa640 1 -- 172.21.15.26:0/1278483813 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f28d8093400 con 0x7f28d0034920 2024-04-24T23:21:04.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.860+0000 7fb51db89640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4fc06bf30 0x7fb4fc06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.860+0000 7ff1eaffd640 1 -- 172.21.15.26:0/2832315655 <== osd.4 v2:172.21.15.29:6808/3208387101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1ec04ce80 con 0x7ff1dc074120 2024-04-24T23:21:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.860+0000 7f28d57fa640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28bc06bf30 0x7f28bc06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.860+0000 7fb51db89640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4fc06bf30 0x7fb4fc06e3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb514006fd0 tx=0x7fb514008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.861+0000 7f28dca36640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28bc06bf30 0x7f28bc06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.861+0000 7fb51e38a640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7fb4fc074130 0x7fb4fc076550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.861+0000 7f28d57fa640 1 -- 172.21.15.26:0/1278483813 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f28d804fa70 con 0x7f28d0034920 2024-04-24T23:21:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.861+0000 7f28dca36640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28bc06bf30 0x7f28bc06e3f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009af0 tx=0x7f28cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.861+0000 7f28d57fa640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f28bc0741b0 0x7f28bc0765d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.862+0000 7fb51e38a640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7fb4fc074130 0x7fb4fc076550 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.861+0000 7f28d57fa640 1 -- 172.21.15.26:0/1278483813 --> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28d8072e50 con 0x7f28bc0741b0 2024-04-24T23:21:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.861+0000 7f28d57fa640 1 -- 172.21.15.26:0/1278483813 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f28d810d050 con 0x7f28d0034920 2024-04-24T23:21:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.862+0000 7f28d7fff640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f28bc0741b0 0x7f28bc0765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.862+0000 7f28d7fff640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f28bc0741b0 0x7f28bc0765d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:04.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.862+0000 7fb50e7fc640 1 -- 172.21.15.26:0/1754747243 <== osd.7 v2:172.21.15.29:6824/2604664408 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb508032440 con 0x7fb4fc074130 2024-04-24T23:21:04.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.863+0000 7f28d57fa640 1 -- 172.21.15.26:0/1278483813 <== osd.3 v2:172.21.15.26:6808/2035350946 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28d8072e50 con 0x7f28bc0741b0 2024-04-24T23:21:04.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.880+0000 7faa1f722640 1 -- 172.21.15.26:0/1837417792 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9f8002dd0 con 0x7fa9fc074030 2024-04-24T23:21:04.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.880+0000 7faa0dffb640 1 -- 172.21.15.26:0/1837417792 <== osd.0 v2:172.21.15.26:6819/256768257 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa9f8002dd0 con 0x7fa9fc074030 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.881+0000 7faa037fe640 1 -- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7fa9fc074030 msgr2=0x7fa9fc076450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.881+0000 7faa037fe640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7fa9fc074030 0x7fa9fc076450 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.882+0000 7faa037fe640 1 -- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9fc06be30 msgr2=0x7fa9fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.882+0000 7faa037fe640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9fc06be30 0x7fa9fc06e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa180502b0 tx=0x7faa18069d40 comp rx=0 tx=0).stop 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.882+0000 7faa037fe640 1 -- 172.21.15.26:0/1837417792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10038100 msgr2=0x7faa1017d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.882+0000 7faa037fe640 1 --2- 172.21.15.26:0/1837417792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10038100 0x7faa1017d430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa14009fd0 tx=0x7faa1400b870 comp rx=0 tx=0).stop 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.882+0000 7faa1dcbf640 1 -- 172.21.15.26:0/1837417792 reap_dead start 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.882+0000 7faa037fe640 1 -- 172.21.15.26:0/1837417792 shutdown_connections 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.882+0000 7faa037fe640 1 -- 172.21.15.26:0/1837417792 >> 172.21.15.26:0/1837417792 conn(0x7faa10009fa0 msgr2=0x7faa10049f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.883+0000 7faa037fe640 1 -- 172.21.15.26:0/1837417792 shutdown_connections 2024-04-24T23:21:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.883+0000 7faa037fe640 1 -- 172.21.15.26:0/1837417792 wait complete. 2024-04-24T23:21:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.886+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45480058e0 con 0x7f4534073fd0 2024-04-24T23:21:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.886+0000 7fa0a7e72640 1 -- 172.21.15.26:0/1442602554 --> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0a0075610 con 0x7fa084074120 2024-04-24T23:21:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.887+0000 7fa0967fc640 1 -- 172.21.15.26:0/1442602554 <== osd.2 v2:172.21.15.26:6800/3780187135 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa0a0075610 con 0x7fa084074120 2024-04-24T23:21:04.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.887+0000 7f4545ffb640 1 -- 172.21.15.26:0/2226574997 <== osd.6 v2:172.21.15.29:6812/744093613 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f45480058e0 con 0x7f4534073fd0 2024-04-24T23:21:04.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.887+0000 7fa083fff640 1 -- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7fa084074120 msgr2=0x7fa084076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.887+0000 7fa083fff640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7fa084074120 0x7fa084076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7fa083fff640 1 -- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa08406bf00 msgr2=0x7fa08406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7fa083fff640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa08406bf00 0x7fa08406e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0a01fb860 tx=0x7fa09c008040 comp rx=0 tx=0).stop 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7fa083fff640 1 -- 172.21.15.26:0/1442602554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a006e900 msgr2=0x7fa0a014ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7fa083fff640 1 --2- 172.21.15.26:0/1442602554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0a006e900 0x7fa0a014ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa098009f70 tx=0x7fa098004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7fa0a640f640 1 -- 172.21.15.26:0/1442602554 reap_dead start 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7fa083fff640 1 -- 172.21.15.26:0/1442602554 shutdown_connections 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7fa083fff640 1 -- 172.21.15.26:0/1442602554 >> 172.21.15.26:0/1442602554 conn(0x7fa0a006b910 msgr2=0x7fa0a0161ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f4534073fd0 msgr2=0x7f45340763f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7f4556685640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f4534073fd0 0x7f45340763f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453406be30 msgr2=0x7f453406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7f4556685640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453406be30 0x7f453406e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f454c005020 tx=0x7f454c007ef0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 msgr2=0x7f4548180bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7f4556685640 1 --2- 172.21.15.26:0/2226574997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f454803c150 0x7f4548180bb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f454000c390 tx=0x7f454000c790 comp rx=0 tx=0).stop 2024-04-24T23:21:04.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7f45578e7640 1 -- 172.21.15.26:0/2226574997 reap_dead start 2024-04-24T23:21:04.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.888+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 --> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a60002d50 con 0x7f1a6c074120 2024-04-24T23:21:04.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.889+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 shutdown_connections 2024-04-24T23:21:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.889+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 >> 172.21.15.26:0/2226574997 conn(0x7f45480590c0 msgr2=0x7f45480e1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.889+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 shutdown_connections 2024-04-24T23:21:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.889+0000 7fa083fff640 1 -- 172.21.15.26:0/1442602554 shutdown_connections 2024-04-24T23:21:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.889+0000 7f1a67fff640 1 -- 172.21.15.26:0/3245848534 <== osd.1 v2:172.21.15.26:6816/1500554534 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1a60002d50 con 0x7f1a6c074120 2024-04-24T23:21:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.890+0000 7fa083fff640 1 -- 172.21.15.26:0/1442602554 wait complete. 2024-04-24T23:21:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.890+0000 7f4556685640 1 -- 172.21.15.26:0/2226574997 wait complete. 2024-04-24T23:21:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.890+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f1a6c074120 msgr2=0x7f1a6c076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.890+0000 7f1a7ffff640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f1a6c074120 0x7f1a6c076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.890+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 --> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e50067af0 con 0x7f6e3c0740a0 2024-04-24T23:21:04.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.891+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a6c06bf00 msgr2=0x7f1a6c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.891+0000 7f1a7ffff640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a6c06bf00 0x7f1a6c06e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a78033000 tx=0x7f1a78009000 comp rx=0 tx=0).stop 2024-04-24T23:21:04.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.891+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a8006f0e0 msgr2=0x7f1a8009e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.891+0000 7f1a7ffff640 1 --2- 172.21.15.26:0/3245848534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a8006f0e0 0x7f1a8009e3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c450 tx=0x7f1a7000c910 comp rx=0 tx=0).stop 2024-04-24T23:21:04.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.891+0000 7f1a7f7fe640 1 -- 172.21.15.26:0/3245848534 reap_dead start 2024-04-24T23:21:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.891+0000 7f6e377fe640 1 -- 172.21.15.26:0/1425901843 <== osd.5 v2:172.21.15.29:6800/1108214380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6e50067af0 con 0x7f6e3c0740a0 2024-04-24T23:21:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.891+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 shutdown_connections 2024-04-24T23:21:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.891+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 >> 172.21.15.26:0/3245848534 conn(0x7f1a8006c230 msgr2=0x7f1a800b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.891+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 shutdown_connections 2024-04-24T23:21:04.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.892+0000 7f1a7ffff640 1 -- 172.21.15.26:0/3245848534 wait complete. 2024-04-24T23:21:04.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 --> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1f404b510 con 0x7ff1dc074120 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f6e3c0740a0 msgr2=0x7f6e3c0764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7f6e55789640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f6e3c0740a0 0x7f6e3c0764c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3c06bf00 msgr2=0x7f6e3c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7f6e55789640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3c06bf00 0x7f6e3c06e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e44007fd0 tx=0x7f6e44009040 comp rx=0 tx=0).stop 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50071440 msgr2=0x7f6e50148630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7f6e55789640 1 --2- 172.21.15.26:0/1425901843 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e50071440 0x7f6e50148630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007210 tx=0x7f6e48004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7f6e4f7fe640 1 -- 172.21.15.26:0/1425901843 reap_dead start 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 shutdown_connections 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.894+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 >> 172.21.15.26:0/1425901843 conn(0x7f6e5006b910 msgr2=0x7f6e500aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.895+0000 7ff1eaffd640 1 -- 172.21.15.26:0/2832315655 <== osd.4 v2:172.21.15.29:6808/3208387101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff1f404b510 con 0x7ff1dc074120 2024-04-24T23:21:04.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.895+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 shutdown_connections 2024-04-24T23:21:04.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.895+0000 7f6e55789640 1 -- 172.21.15.26:0/1425901843 wait complete. 2024-04-24T23:21:04.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.895+0000 7f28c2ffd640 1 -- 172.21.15.26:0/1278483813 --> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28d0054b10 con 0x7f28bc0741b0 2024-04-24T23:21:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.895+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7ff1dc074120 msgr2=0x7ff1dc076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.895+0000 7ff1fb59e640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7ff1dc074120 0x7ff1dc076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.895+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1dc06bf00 msgr2=0x7ff1dc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.895+0000 7ff1fb59e640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff1dc06bf00 0x7ff1dc06e3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc064c60 tx=0x7ff1fc069d40 comp rx=0 tx=0).stop 2024-04-24T23:21:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.895+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f4006ee0 msgr2=0x7ff1f4045930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7ff1fb59e640 1 --2- 172.21.15.26:0/2832315655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f4006ee0 0x7ff1f4045930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec031000 tx=0x7ff1ec0046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7ff1fad9d640 1 -- 172.21.15.26:0/2832315655 reap_dead start 2024-04-24T23:21:04.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 shutdown_connections 2024-04-24T23:21:04.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 >> 172.21.15.26:0/2832315655 conn(0x7ff1f4009fa0 msgr2=0x7ff1f4052c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7f28d57fa640 1 -- 172.21.15.26:0/1278483813 <== osd.3 v2:172.21.15.26:6808/2035350946 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f28d0054b10 con 0x7f28bc0741b0 2024-04-24T23:21:04.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 shutdown_connections 2024-04-24T23:21:04.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7ff1fb59e640 1 -- 172.21.15.26:0/2832315655 wait complete. 2024-04-24T23:21:04.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7fb4fbfff640 1 -- 172.21.15.26:0/1754747243 --> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb518160260 con 0x7fb4fc074130 2024-04-24T23:21:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7f28dec9a640 1 -- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f28bc0741b0 msgr2=0x7f28bc0765d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.896+0000 7f28dec9a640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f28bc0741b0 0x7f28bc0765d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7f28dec9a640 1 -- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28bc06bf30 msgr2=0x7f28bc06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7f28dec9a640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28bc06bf30 0x7f28bc06e3f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009af0 tx=0x7f28cc008040 comp rx=0 tx=0).stop 2024-04-24T23:21:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7f28dec9a640 1 -- 172.21.15.26:0/1278483813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28d0034920 msgr2=0x7f28d0036310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7f28dec9a640 1 --2- 172.21.15.26:0/1278483813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28d0034920 0x7f28d0036310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f28d80502b0 tx=0x7f28d8071610 comp rx=0 tx=0).stop 2024-04-24T23:21:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7fb50e7fc640 1 -- 172.21.15.26:0/1754747243 <== osd.7 v2:172.21.15.29:6824/2604664408 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb518160260 con 0x7fb4fc074130 2024-04-24T23:21:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7f28dd237640 1 -- 172.21.15.26:0/1278483813 reap_dead start 2024-04-24T23:21:04.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7f28dec9a640 1 -- 172.21.15.26:0/1278483813 shutdown_connections 2024-04-24T23:21:04.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7f28dec9a640 1 -- 172.21.15.26:0/1278483813 >> 172.21.15.26:0/1278483813 conn(0x7f28d000a3f0 msgr2=0x7f28d004a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7f28dec9a640 1 -- 172.21.15.26:0/1278483813 shutdown_connections 2024-04-24T23:21:04.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7fb4fbfff640 1 -- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7fb4fc074130 msgr2=0x7fb4fc076550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7f28dec9a640 1 -- 172.21.15.26:0/1278483813 wait complete. 2024-04-24T23:21:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.897+0000 7fb4fbfff640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7fb4fc074130 0x7fb4fc076550 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.898+0000 7fb4fbfff640 1 -- 172.21.15.26:0/1754747243 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4fc06bf30 msgr2=0x7fb4fc06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.898+0000 7fb4fbfff640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4fc06bf30 0x7fb4fc06e3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb514006fd0 tx=0x7fb514008040 comp rx=0 tx=0).stop 2024-04-24T23:21:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.898+0000 7fb4fbfff640 1 -- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 msgr2=0x7fb5181635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.898+0000 7fb4fbfff640 1 --2- 172.21.15.26:0/1754747243 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb5180b3910 0x7fb5181635a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb508031000 tx=0x7fb5080046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.898+0000 7fb51e38a640 1 -- 172.21.15.26:0/1754747243 reap_dead start 2024-04-24T23:21:04.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.898+0000 7fb4fbfff640 1 -- 172.21.15.26:0/1754747243 shutdown_connections 2024-04-24T23:21:04.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.898+0000 7fb4fbfff640 1 -- 172.21.15.26:0/1754747243 >> 172.21.15.26:0/1754747243 conn(0x7fb51808dbd0 msgr2=0x7fb5180d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:04.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.898+0000 7fb4fbfff640 1 -- 172.21.15.26:0/1754747243 shutdown_connections 2024-04-24T23:21:04.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:04.898+0000 7fb4fbfff640 1 -- 172.21.15.26:0/1754747243 wait complete. 2024-04-24T23:21:04.956 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:04.957 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T23:21:04.958 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:04.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T23:21:04.959 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:04.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T23:21:04.960 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:04.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T23:21:04.961 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-04-24T23:21:04.961 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T23:21:04.961 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:04.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T23:21:04.963 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-04-24T23:21:04.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T23:21:04.964 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:04.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T23:21:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.134+0000 7f12548e6640 1 Processor -- start 2024-04-24T23:21:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.134+0000 7f12548e6640 1 -- start start 2024-04-24T23:21:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.135+0000 7f7d1b509640 1 Processor -- start 2024-04-24T23:21:05.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.135+0000 7f12548e6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12480507a0 0x7f12480df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.135+0000 7f12548e6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12480e3630 con 0x7f1248034610 2024-04-24T23:21:05.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.135+0000 7f12548e6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12480e3910 con 0x7f12480e3130 2024-04-24T23:21:05.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.135+0000 7f7d1b509640 1 -- start start 2024-04-24T23:21:05.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.135+0000 7f12548e6640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f12480e3bf0 con 0x7f12480507a0 2024-04-24T23:21:05.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.136+0000 7f124f7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f12480e3130 0x7f1248006290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:32942/0 (socket says 172.21.15.26:32942) 2024-04-24T23:21:05.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.136+0000 7f124f7fe640 1 -- 172.21.15.26:0/979931478 learned_addr learned my addr 172.21.15.26:0/979931478 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:05.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.136+0000 7f124ffff640 1 --2- 172.21.15.26:0/979931478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12480507a0 0x7f12480df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.136+0000 7f124e7fc640 1 -- 172.21.15.26:0/979931478 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644697498 0 0) 0x7f12480e3910 con 0x7f12480e3130 2024-04-24T23:21:05.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.136+0000 7f124e7fc640 1 -- 172.21.15.26:0/979931478 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1240003520 con 0x7f12480e3130 2024-04-24T23:21:05.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.136+0000 7f124ffff640 1 -- 172.21.15.26:0/979931478 >> v1:172.21.15.29:6789/0 conn(0x7f12480e3130 legacy=0x7f1248006290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.136+0000 7f124ffff640 1 -- 172.21.15.26:0/979931478 >> v1:172.21.15.26:6789/0 conn(0x7f1248034610 legacy=0x7f1248050090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d1b509640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d140ce990 0x7f7d140cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d1b509640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7d140b1860 0x7f7d140b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d1b509640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d140b2230 con 0x7f7d1415c2e0 2024-04-24T23:21:05.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d1b509640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7d140cf330 con 0x7f7d140ce990 2024-04-24T23:21:05.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d1b509640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7d140d0af0 con 0x7f7d140b1860 2024-04-24T23:21:05.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d18aa4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d140ce990 0x7f7d140cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d18aa4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7d140ce990 0x7f7d140cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:56632/0 (socket says 172.21.15.26:56632) 2024-04-24T23:21:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d18aa4640 1 -- 172.21.15.26:0/3079076008 learned_addr learned my addr 172.21.15.26:0/3079076008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.136+0000 7f124ffff640 1 -- 172.21.15.26:0/979931478 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12480e3ed0 con 0x7f12480507a0 2024-04-24T23:21:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d18aa4640 1 -- 172.21.15.26:0/3079076008 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7d140b1860 msgr2=0x7f7d140b1cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:21:05.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d18aa4640 1 --2- 172.21.15.26:0/3079076008 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7d140b1860 0x7f7d140b1cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d18aa4640 1 -- 172.21.15.26:0/3079076008 >> v1:172.21.15.26:6789/0 conn(0x7f7d1415c2e0 legacy=0x7f7d140cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.137+0000 7f7d18aa4640 1 -- 172.21.15.26:0/3079076008 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d140d0dd0 con 0x7f7d140ce990 2024-04-24T23:21:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.138+0000 7f124ffff640 1 --2- 172.21.15.26:0/979931478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12480507a0 0x7f12480df660 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f12440097d0 tx=0x7f124402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ab4a53fe8c84404 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.138+0000 7f124e7fc640 1 -- 172.21.15.26:0/979931478 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244037070 con 0x7f12480507a0 2024-04-24T23:21:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.139+0000 7f124e7fc640 1 -- 172.21.15.26:0/979931478 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124402ae40 con 0x7f12480507a0 2024-04-24T23:21:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.139+0000 7f124e7fc640 1 -- 172.21.15.26:0/979931478 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244032660 con 0x7f12480507a0 2024-04-24T23:21:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.139+0000 7f7d18aa4640 1 --2- 172.21.15.26:0/3079076008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d140ce990 0x7f7d140cedf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009870 tx=0x7f7d1002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c202ec5313ca207d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.139+0000 7f7d0bfff640 1 -- 172.21.15.26:0/3079076008 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10037070 con 0x7f7d140ce990 2024-04-24T23:21:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.139+0000 7f7d0bfff640 1 -- 172.21.15.26:0/3079076008 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d10032440 con 0x7f7d140ce990 2024-04-24T23:21:05.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.139+0000 7f12548e6640 1 -- 172.21.15.26:0/979931478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12480507a0 msgr2=0x7f12480df660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.139+0000 7f12548e6640 1 --2- 172.21.15.26:0/979931478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12480507a0 0x7f12480df660 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f12440097d0 tx=0x7f124402a130 comp rx=0 tx=0).stop 2024-04-24T23:21:05.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.140+0000 7efcfde9a640 1 Processor -- start 2024-04-24T23:21:05.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.140+0000 7efcfde9a640 1 -- start start 2024-04-24T23:21:05.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.140+0000 7fa2c5561640 1 Processor -- start 2024-04-24T23:21:05.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.140+0000 7fa2c5561640 1 -- start start 2024-04-24T23:21:05.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.140+0000 7f7d0bfff640 1 -- 172.21.15.26:0/3079076008 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10037070 con 0x7f7d140ce990 2024-04-24T23:21:05.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.140+0000 7f7d1b509640 1 -- 172.21.15.26:0/3079076008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d140ce990 msgr2=0x7f7d140cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.140+0000 7f7d1b509640 1 --2- 172.21.15.26:0/3079076008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d140ce990 0x7f7d140cedf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009870 tx=0x7f7d1002a230 comp rx=0 tx=0).stop 2024-04-24T23:21:05.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.140+0000 7f3a45dd5640 1 Processor -- start 2024-04-24T23:21:05.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f7d1b509640 1 -- 172.21.15.26:0/3079076008 shutdown_connections 2024-04-24T23:21:05.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f7d1b509640 1 --2- 172.21.15.26:0/3079076008 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7d140b1860 0x7f7d140b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f12548e6640 1 -- 172.21.15.26:0/979931478 shutdown_connections 2024-04-24T23:21:05.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f7d1b509640 1 --2- 172.21.15.26:0/3079076008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d140ce990 0x7f7d140cedf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f7d1b509640 1 -- 172.21.15.26:0/3079076008 >> 172.21.15.26:0/3079076008 conn(0x7f7d140d1f80 msgr2=0x7f7d140d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f12548e6640 1 --2- 172.21.15.26:0/979931478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12480507a0 0x7f12480df660 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f12548e6640 1 -- 172.21.15.26:0/979931478 >> 172.21.15.26:0/979931478 conn(0x7f1248055350 msgr2=0x7f1248055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f7d1b509640 1 -- 172.21.15.26:0/3079076008 shutdown_connections 2024-04-24T23:21:05.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7efcfde9a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efcf815a0a0 0x7efcf8070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f7d1b509640 1 -- 172.21.15.26:0/3079076008 wait complete. 2024-04-24T23:21:05.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7efcfde9a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efcf806eb10 0x7efcf80714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7efcfde9a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcf806d620 con 0x7efcf80b44b0 2024-04-24T23:21:05.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7efcfde9a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7efcf80cee80 con 0x7efcf815a0a0 2024-04-24T23:21:05.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7efcfde9a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efcf80ce5e0 con 0x7efcf806eb10 2024-04-24T23:21:05.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f12548e6640 1 -- 172.21.15.26:0/979931478 shutdown_connections 2024-04-24T23:21:05.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7efcf77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efcf815a0a0 0x7efcf8070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7efcf77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efcf815a0a0 0x7efcf8070f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40014/0 (socket says 172.21.15.26:40014) 2024-04-24T23:21:05.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7efcf77fe640 1 -- 172.21.15.26:0/3343871353 learned_addr learned my addr 172.21.15.26:0/3343871353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f12548e6640 1 -- 172.21.15.26:0/979931478 wait complete. 2024-04-24T23:21:05.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7fa2c5561640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c013df50 0x7fa2c0072060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.141+0000 7f3a45dd5640 1 -- start start 2024-04-24T23:21:05.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7efcf77fe640 1 -- 172.21.15.26:0/3343871353 >> v2:172.21.15.29:3300/0 conn(0x7efcf806eb10 msgr2=0x7efcf80714d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:21:05.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7efcf77fe640 1 --2- 172.21.15.26:0/3343871353 >> v2:172.21.15.29:3300/0 conn(0x7efcf806eb10 0x7efcf80714d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7efcf77fe640 1 -- 172.21.15.26:0/3343871353 >> v1:172.21.15.29:6789/0 conn(0x7efcf80b44b0 legacy=0x7efcf80721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7efcf77fe640 1 -- 172.21.15.26:0/3343871353 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf80ce8c0 con 0x7efcf815a0a0 2024-04-24T23:21:05.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7fa2c5561640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2c015f3a0 0x7fa2c00725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7fa2beffd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c013df50 0x7fa2c0072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7f3a45dd5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a400cc300 0x7f3a400b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7f3a45dd5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a400cc6e0 0x7f3a40071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7fa2beffd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c013df50 0x7fa2c0072060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38248/0 (socket says 172.21.15.26:38248) 2024-04-24T23:21:05.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7fa2beffd640 1 -- 172.21.15.26:0/2654893546 learned_addr learned my addr 172.21.15.26:0/2654893546 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7fa2be7fc640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fa2c0141470 0x7fa2c015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:32962/0 (socket says 172.21.15.26:32962) 2024-04-24T23:21:05.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7fa2bf7fe640 1 --2- 172.21.15.26:0/2654893546 >> v2:172.21.15.26:3300/0 conn(0x7fa2c015f3a0 0x7fa2c00725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7f3a45dd5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a401638f0 0x7f3a400724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7f3a45dd5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3a400b46e0 con 0x7f3a400cc6e0 2024-04-24T23:21:05.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7fa2c5561640 1 -- 172.21.15.26:0/2654893546 --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c015f9a0 con 0x7fa2c0141470 2024-04-24T23:21:05.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7f3a45dd5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3a400b49c0 con 0x7f3a401638f0 2024-04-24T23:21:05.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7fa2c5561640 1 -- 172.21.15.26:0/2654893546 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c0140990 con 0x7fa2c015f3a0 2024-04-24T23:21:05.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7f3a45dd5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3a400b4ca0 con 0x7f3a400cc300 2024-04-24T23:21:05.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7fa2bdffb640 1 -- 172.21.15.26:0/2654893546 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 265429529 0 0) 0x7fa2c015f9a0 con 0x7fa2c0141470 2024-04-24T23:21:05.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7fa2c5561640 1 -- 172.21.15.26:0/2654893546 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c0140c70 con 0x7fa2c013df50 2024-04-24T23:21:05.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.142+0000 7f3a3f7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a400cc300 0x7f3a400b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f3a3ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3a401638f0 0x7f3a400724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f3a3effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a400cc6e0 0x7f3a40071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f3a3effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a400cc6e0 0x7f3a40071f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40024/0 (socket says 172.21.15.26:40024) 2024-04-24T23:21:05.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7fa2bdffb640 1 -- 172.21.15.26:0/2654893546 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2a8003520 con 0x7fa2c0141470 2024-04-24T23:21:05.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f3a3effd640 1 -- 172.21.15.26:0/2736830562 learned_addr learned my addr 172.21.15.26:0/2736830562 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f3a3effd640 1 -- 172.21.15.26:0/2736830562 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a400cc300 msgr2=0x7f3a400b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f3a3effd640 1 --2- 172.21.15.26:0/2736830562 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a400cc300 0x7f3a400b3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f3a3effd640 1 -- 172.21.15.26:0/2736830562 >> v2:172.21.15.29:3300/0 conn(0x7f3a401638f0 msgr2=0x7f3a400724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f3a3effd640 1 --2- 172.21.15.26:0/2736830562 >> v2:172.21.15.29:3300/0 conn(0x7f3a401638f0 0x7f3a400724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f3a3effd640 1 -- 172.21.15.26:0/2736830562 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a400b43d0 con 0x7f3a400cc6e0 2024-04-24T23:21:05.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f12548e6640 1 Processor -- start 2024-04-24T23:21:05.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f12548e6640 1 -- start start 2024-04-24T23:21:05.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7fa2beffd640 1 -- 172.21.15.26:0/2654893546 >> v2:172.21.15.26:3300/0 conn(0x7fa2c015f3a0 msgr2=0x7fa2c00725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7fa2beffd640 1 --2- 172.21.15.26:0/2654893546 >> v2:172.21.15.26:3300/0 conn(0x7fa2c015f3a0 0x7fa2c00725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7fa2beffd640 1 -- 172.21.15.26:0/2654893546 >> v1:172.21.15.29:6789/0 conn(0x7fa2c0141470 legacy=0x7fa2c015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7fa2beffd640 1 -- 172.21.15.26:0/2654893546 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c0140f50 con 0x7fa2c013df50 2024-04-24T23:21:05.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7f19d4a37640 1 Processor -- start 2024-04-24T23:21:05.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7f19d4a37640 1 -- start start 2024-04-24T23:21:05.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7f12548e6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1248034610 0x7f12480d5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f7d1b509640 1 Processor -- start 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.143+0000 7f7d1b509640 1 -- start start 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7f7d1b509640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d140b1860 0x7f7d14157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7f7d1b509640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7d140ce990 0x7f7d14151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7f7d1b509640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d1415c2e0 0x7f7d140afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7f7d1b509640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7d140cd160 con 0x7f7d1415c2e0 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7f7d1b509640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7d140cce80 con 0x7f7d140b1860 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7f7d1b509640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7d140cdd90 con 0x7f7d140ce990 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f7d19aa6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d1415c2e0 0x7f7d140afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f7d19aa6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d1415c2e0 0x7f7d140afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40030/0 (socket says 172.21.15.26:40030) 2024-04-24T23:21:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f7d19aa6640 1 -- 172.21.15.26:0/3241330648 learned_addr learned my addr 172.21.15.26:0/3241330648 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f12548e6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12480507a0 0x7f12481806f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f12548e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480e3130 0x7f124817da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f12548e6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1248055130 con 0x7f12480e3130 2024-04-24T23:21:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f12548e6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1248054fb0 con 0x7f12480507a0 2024-04-24T23:21:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f12548e6640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1248007fc0 con 0x7f1248034610 2024-04-24T23:21:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7fa2beffd640 1 --2- 172.21.15.26:0/2654893546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c013df50 0x7fa2c0072060 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009870 tx=0x7fa2b002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef159d092ba5b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f124effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12480507a0 0x7f12481806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f124effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12480507a0 0x7f12481806f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:56648/0 (socket says 172.21.15.26:56648) 2024-04-24T23:21:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f124effd640 1 -- 172.21.15.26:0/1255785323 learned_addr learned my addr 172.21.15.26:0/1255785323 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7efcf77fe640 1 --2- 172.21.15.26:0/3343871353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf815a0a0 0x7efcf8070f90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efcf80cd4e0 tx=0x7efcf002ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1d5fd9128c7e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f7d192a5640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d140b1860 0x7f7d14157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f124ffff640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480e3130 0x7f124817da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7fa2bdffb640 1 -- 172.21.15.26:0/2654893546 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0037070 con 0x7fa2c013df50 2024-04-24T23:21:05.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7feb1de46640 1 Processor -- start 2024-04-24T23:21:05.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.144+0000 7feb1de46640 1 -- start start 2024-04-24T23:21:05.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7fa2bdffb640 1 -- 172.21.15.26:0/2654893546 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b0032440 con 0x7fa2c013df50 2024-04-24T23:21:05.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7fa2bdffb640 1 -- 172.21.15.26:0/2654893546 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0032810 con 0x7fa2c013df50 2024-04-24T23:21:05.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f7d19aa6640 1 -- 172.21.15.26:0/3241330648 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7d140ce990 msgr2=0x7f7d14151830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:21:05.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f7d19aa6640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7d140ce990 0x7f7d14151830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f7d19aa6640 1 -- 172.21.15.26:0/3241330648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d140b1860 msgr2=0x7f7d14157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f7d19aa6640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d140b1860 0x7f7d14157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f7d19aa6640 1 -- 172.21.15.26:0/3241330648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d140ca610 con 0x7f7d1415c2e0 2024-04-24T23:21:05.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f124effd640 1 -- 172.21.15.26:0/1255785323 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1248034610 msgr2=0x7f12480d5760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:21:05.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f124effd640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1248034610 0x7f12480d5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f124effd640 1 -- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480e3130 msgr2=0x7f124817da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f124effd640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480e3130 0x7f124817da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f124effd640 1 -- 172.21.15.26:0/1255785323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1248180ff0 con 0x7f12480507a0 2024-04-24T23:21:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f3a3effd640 1 --2- 172.21.15.26:0/2736830562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc6e0 0x7f3a40071f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a400715e0 tx=0x7f3a3402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12950375a005975 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.145+0000 7f19d4a37640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19c8037fe0 0x7f19c8058420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19d4a37640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19c8058960 0x7f19c803c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19d4a37640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c803c640 0x7f19c80061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19d4a37640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f19c80383c0 con 0x7f19c8058960 2024-04-24T23:21:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19d4a37640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f19c80e7990 con 0x7f19c8037fe0 2024-04-24T23:21:05.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19d4a37640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f19c80570a0 con 0x7f19c803c640 2024-04-24T23:21:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f124effd640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12480507a0 0x7f12481806f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f123c00c460 tx=0x7f123c00c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7feb1de46640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb18166230 0x7feb1806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7feb1de46640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1806ef60 0x7feb180cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7feb1de46640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb1806d960 con 0x7feb1814be70 2024-04-24T23:21:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7feb1de46640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7feb1806e2b0 con 0x7feb18166230 2024-04-24T23:21:05.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7feb1de46640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feb1806dc40 con 0x7feb1806ef60 2024-04-24T23:21:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7efcf67fc640 1 -- 172.21.15.26:0/3343871353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0037020 con 0x7efcf815a0a0 2024-04-24T23:21:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7efcf67fc640 1 -- 172.21.15.26:0/3343871353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf0033870 con 0x7efcf815a0a0 2024-04-24T23:21:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7efcf67fc640 1 -- 172.21.15.26:0/3343871353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf000be60 con 0x7efcf815a0a0 2024-04-24T23:21:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f3a3e7fc640 1 -- 172.21.15.26:0/2736830562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34037070 con 0x7f3a400cc6e0 2024-04-24T23:21:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f3a3e7fc640 1 -- 172.21.15.26:0/2736830562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a34032440 con 0x7f3a400cc6e0 2024-04-24T23:21:05.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f7d19aa6640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d1415c2e0 0x7f7d140afca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b990 tx=0x7f7d0c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19cf59e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c803c640 0x7f19c80061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19cf59e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c803c640 0x7f19c80061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38274/0 (socket says 172.21.15.26:38274) 2024-04-24T23:21:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19cf59e640 1 -- 172.21.15.26:0/136214060 learned_addr learned my addr 172.21.15.26:0/136214060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19ced9d640 1 --2- 172.21.15.26:0/136214060 >> v2:172.21.15.29:3300/0 conn(0x7f19c8037fe0 0x7f19c8058420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19ce59c640 1 --2- 172.21.15.26:0/136214060 >> v2:172.21.15.26:3300/0 conn(0x7f19c8058960 0x7f19c803c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19cf59e640 1 -- 172.21.15.26:0/136214060 >> v2:172.21.15.29:3300/0 conn(0x7f19c8037fe0 msgr2=0x7f19c8058420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19cf59e640 1 --2- 172.21.15.26:0/136214060 >> v2:172.21.15.29:3300/0 conn(0x7f19c8037fe0 0x7f19c8058420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19cf59e640 1 -- 172.21.15.26:0/136214060 >> v2:172.21.15.26:3300/0 conn(0x7f19c8058960 msgr2=0x7f19c803c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19cf59e640 1 --2- 172.21.15.26:0/136214060 >> v2:172.21.15.26:3300/0 conn(0x7f19c8058960 0x7f19c803c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19cf59e640 1 -- 172.21.15.26:0/136214060 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c8057380 con 0x7f19c803c640 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19ce59c640 1 --2- 172.21.15.26:0/136214060 >> v2:172.21.15.26:3300/0 conn(0x7f19c8058960 0x7f19c803c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f19ced9d640 1 --2- 172.21.15.26:0/136214060 >> v2:172.21.15.29:3300/0 conn(0x7f19c8037fe0 0x7f19c8058420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f123bfff640 1 -- 172.21.15.26:0/1255785323 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c016030 con 0x7f12480507a0 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f12548e6640 1 -- 172.21.15.26:0/1255785323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12481812d0 con 0x7f12480507a0 2024-04-24T23:21:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7f12548e6640 1 -- 172.21.15.26:0/1255785323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124817d210 con 0x7f12480507a0 2024-04-24T23:21:05.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7feb16ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb18166230 0x7feb1806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7feb177fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7feb1814be70 0x7feb1806d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:32972/0 (socket says 172.21.15.26:32972) 2024-04-24T23:21:05.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7feb177fe640 1 -- 172.21.15.26:0/1984440235 learned_addr learned my addr 172.21.15.26:0/1984440235 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:05.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7feb17fff640 1 --2- 172.21.15.26:0/1984440235 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1806ef60 0x7feb180cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7fa2c5561640 1 -- 172.21.15.26:0/2654893546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c013df50 msgr2=0x7fa2c0072060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.146+0000 7fa2c5561640 1 --2- 172.21.15.26:0/2654893546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c013df50 0x7fa2c0072060 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009870 tx=0x7fa2b002a270 comp rx=0 tx=0).stop 2024-04-24T23:21:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7fa2c5561640 1 -- 172.21.15.26:0/2654893546 shutdown_connections 2024-04-24T23:21:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7fa2c5561640 1 --2- 172.21.15.26:0/2654893546 >> v2:172.21.15.26:3300/0 conn(0x7fa2c015f3a0 0x7fa2c00725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7fa2c5561640 1 --2- 172.21.15.26:0/2654893546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c013df50 0x7fa2c0072060 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7fa2c5561640 1 -- 172.21.15.26:0/2654893546 >> 172.21.15.26:0/2654893546 conn(0x7fa2c00b2ce0 msgr2=0x7fa2c00b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7fa2c5561640 1 -- 172.21.15.26:0/2654893546 shutdown_connections 2024-04-24T23:21:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7fa2c5561640 1 -- 172.21.15.26:0/2654893546 wait complete. 2024-04-24T23:21:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7f3a3e7fc640 1 -- 172.21.15.26:0/2736830562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34032810 con 0x7f3a400cc6e0 2024-04-24T23:21:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7feb167fc640 1 -- 172.21.15.26:0/1984440235 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2413020713 0 0) 0x7feb1806d960 con 0x7feb1814be70 2024-04-24T23:21:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7feb167fc640 1 -- 172.21.15.26:0/1984440235 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb00003520 con 0x7feb1814be70 2024-04-24T23:21:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7feb16ffd640 1 -- 172.21.15.26:0/1984440235 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1806ef60 msgr2=0x7feb180cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7feb16ffd640 1 --2- 172.21.15.26:0/1984440235 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1806ef60 0x7feb180cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7feb16ffd640 1 -- 172.21.15.26:0/1984440235 >> v1:172.21.15.29:6789/0 conn(0x7feb1814be70 legacy=0x7feb1806d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7efcfde9a640 1 -- 172.21.15.26:0/3343871353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf815a0a0 msgr2=0x7efcf8070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7efcfde9a640 1 --2- 172.21.15.26:0/3343871353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf815a0a0 0x7efcf8070f90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efcf80cd4e0 tx=0x7efcf002ca40 comp rx=0 tx=0).stop 2024-04-24T23:21:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7feb16ffd640 1 -- 172.21.15.26:0/1984440235 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1806df20 con 0x7feb18166230 2024-04-24T23:21:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7f3a45dd5640 1 -- 172.21.15.26:0/2736830562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc6e0 msgr2=0x7f3a40071f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7f123bfff640 1 -- 172.21.15.26:0/1255785323 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123c007020 con 0x7f12480507a0 2024-04-24T23:21:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7f7d09ffb640 1 -- 172.21.15.26:0/3241330648 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c019070 con 0x7f7d1415c2e0 2024-04-24T23:21:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f7d09ffb640 1 -- 172.21.15.26:0/3241330648 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c00cba0 con 0x7f7d1415c2e0 2024-04-24T23:21:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f18b2e4d640 1 Processor -- start 2024-04-24T23:21:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7f123bfff640 1 -- 172.21.15.26:0/1255785323 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c007430 con 0x7f12480507a0 2024-04-24T23:21:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d140ca8b0 con 0x7f7d1415c2e0 2024-04-24T23:21:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f7d09ffb640 1 -- 172.21.15.26:0/3241330648 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c014440 con 0x7f7d1415c2e0 2024-04-24T23:21:05.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f19cf59e640 1 --2- 172.21.15.26:0/136214060 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c803c640 0x7f19c80061f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f19c8008510 tx=0x7f19c402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=946f6da76d54ccb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/2736830562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc6e0 0x7f3a40071f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a400715e0 tx=0x7f3a3402a1d0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d140cae70 con 0x7f7d1415c2e0 2024-04-24T23:21:05.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f19cdd9b640 1 -- 172.21.15.26:0/136214060 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4037070 con 0x7f19c803c640 2024-04-24T23:21:05.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f19cdd9b640 1 -- 172.21.15.26:0/136214060 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c4032440 con 0x7f19c803c640 2024-04-24T23:21:05.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f19cdd9b640 1 -- 172.21.15.26:0/136214060 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4032810 con 0x7f19c803c640 2024-04-24T23:21:05.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f123bfff640 1 -- 172.21.15.26:0/1255785323 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f123c019070 con 0x7f12480507a0 2024-04-24T23:21:05.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7feb16ffd640 1 --2- 172.21.15.26:0/1984440235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb18166230 0x7feb1806ea20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0097a0 tx=0x7feb0c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4872a9d99e45484 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7efcfde9a640 1 -- 172.21.15.26:0/3343871353 shutdown_connections 2024-04-24T23:21:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7efcfde9a640 1 --2- 172.21.15.26:0/3343871353 >> v2:172.21.15.29:3300/0 conn(0x7efcf806eb10 0x7efcf80714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7efcfde9a640 1 --2- 172.21.15.26:0/3343871353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf815a0a0 0x7efcf8070f90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.147+0000 7efcfde9a640 1 -- 172.21.15.26:0/3343871353 >> 172.21.15.26:0/3343871353 conn(0x7efcf806b300 msgr2=0x7efcf806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7efcfde9a640 1 -- 172.21.15.26:0/3343871353 shutdown_connections 2024-04-24T23:21:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7efcfde9a640 1 -- 172.21.15.26:0/3343871353 wait complete. 2024-04-24T23:21:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.148+0000 7f18b2e4d640 1 -- start start 2024-04-24T23:21:05.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f3a45dd5640 1 -- 172.21.15.26:0/2736830562 shutdown_connections 2024-04-24T23:21:05.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/2736830562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc6e0 0x7f3a40071f90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/2736830562 >> v2:172.21.15.29:3300/0 conn(0x7f3a401638f0 0x7f3a400724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/2736830562 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a400cc300 0x7f3a400b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f123bfff640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f123406bf30 0x7f123406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f3a45dd5640 1 -- 172.21.15.26:0/2736830562 >> 172.21.15.26:0/2736830562 conn(0x7f3a4006b300 msgr2=0x7f3a4006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f19d4a37640 1 -- 172.21.15.26:0/136214060 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c803c640 msgr2=0x7f19c80061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7fa2c5561640 1 Processor -- start 2024-04-24T23:21:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f3a45dd5640 1 -- 172.21.15.26:0/2736830562 shutdown_connections 2024-04-24T23:21:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f123bfff640 1 -- 172.21.15.26:0/1255785323 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f123c08a970 con 0x7f12480507a0 2024-04-24T23:21:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f19d4a37640 1 --2- 172.21.15.26:0/136214060 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c803c640 0x7f19c80061f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f19c8008510 tx=0x7f19c402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7fa2c5561640 1 -- start start 2024-04-24T23:21:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7feb167fc640 1 -- 172.21.15.26:0/1984440235 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c037070 con 0x7feb18166230 2024-04-24T23:21:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7efcfde9a640 1 Processor -- start 2024-04-24T23:21:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7feb167fc640 1 -- 172.21.15.26:0/1984440235 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb0c02adb0 con 0x7feb18166230 2024-04-24T23:21:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7feb167fc640 1 -- 172.21.15.26:0/1984440235 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c037070 con 0x7feb18166230 2024-04-24T23:21:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7f124f7fe640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f123406bf30 0x7f123406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7f12548e6640 1 -- 172.21.15.26:0/1255785323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1248008850 con 0x7f12480507a0 2024-04-24T23:21:05.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7f7d09ffb640 1 -- 172.21.15.26:0/3241330648 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d0c01c070 con 0x7f7d1415c2e0 2024-04-24T23:21:05.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f18b2e4d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f18ac15a7d0 0x7f18ac0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f18b2e4d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 0x7f18ac0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f18b2e4d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18ac072790 con 0x7f18ac157df0 2024-04-24T23:21:05.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f18b2e4d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f18ac1581d0 con 0x7f18ac15a7d0 2024-04-24T23:21:05.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f18b2e4d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f18ac0cfc30 con 0x7f18ac0cf2a0 2024-04-24T23:21:05.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f18b13ea640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 0x7f18ac0cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7f18b0be9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f18ac157df0 0x7f18ac0cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54664/0 (socket says 172.21.15.26:54664) 2024-04-24T23:21:05.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7f18b0be9640 1 -- 172.21.15.26:0/2972648817 learned_addr learned my addr 172.21.15.26:0/2972648817 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:05.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7f18abfff640 1 --2- 172.21.15.26:0/2972648817 >> v2:172.21.15.26:3300/0 conn(0x7f18ac15a7d0 0x7f18ac0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.149+0000 7f3a45dd5640 1 -- 172.21.15.26:0/2736830562 wait complete. 2024-04-24T23:21:05.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d140681e0 con 0x7f7d1415c2e0 2024-04-24T23:21:05.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7fa2c5561640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2c013df50 0x7fa2c01f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f18ab7fe640 1 -- 172.21.15.26:0/2972648817 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483469153 0 0) 0x7f18ac072790 con 0x7f18ac157df0 2024-04-24T23:21:05.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f18ab7fe640 1 -- 172.21.15.26:0/2972648817 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1898003520 con 0x7f18ac157df0 2024-04-24T23:21:05.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7feb1de46640 1 -- 172.21.15.26:0/1984440235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb18166230 msgr2=0x7feb1806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7feb1de46640 1 --2- 172.21.15.26:0/1984440235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb18166230 0x7feb1806ea20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0097a0 tx=0x7feb0c02a060 comp rx=0 tx=0).stop 2024-04-24T23:21:05.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7fa2c5561640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2c0141470 0x7fa2c015a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7fa2c5561640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c015f3a0 0x7fa2c006f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7fa2c5561640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00cfac0 con 0x7fa2c0141470 2024-04-24T23:21:05.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7fa2c5561640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00cf940 con 0x7fa2c013df50 2024-04-24T23:21:05.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7fa2c5561640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c00cfc40 con 0x7fa2c015f3a0 2024-04-24T23:21:05.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.150+0000 7efcfde9a640 1 -- start start 2024-04-24T23:21:05.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7feb1de46640 1 -- 172.21.15.26:0/1984440235 shutdown_connections 2024-04-24T23:21:05.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7feb1de46640 1 --2- 172.21.15.26:0/1984440235 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1806ef60 0x7feb180cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7feb1de46640 1 --2- 172.21.15.26:0/1984440235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb18166230 0x7feb1806ea20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7feb1de46640 1 -- 172.21.15.26:0/1984440235 >> 172.21.15.26:0/1984440235 conn(0x7feb180b1b80 msgr2=0x7feb180b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f19d4a37640 1 -- 172.21.15.26:0/136214060 shutdown_connections 2024-04-24T23:21:05.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f19d4a37640 1 --2- 172.21.15.26:0/136214060 >> v2:172.21.15.26:3300/0 conn(0x7f19c8058960 0x7f19c803c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f19d4a37640 1 --2- 172.21.15.26:0/136214060 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c803c640 0x7f19c80061f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f19d4a37640 1 --2- 172.21.15.26:0/136214060 >> v2:172.21.15.29:3300/0 conn(0x7f19c8037fe0 0x7f19c8058420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f3a45dd5640 1 Processor -- start 2024-04-24T23:21:05.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f19d4a37640 1 -- 172.21.15.26:0/136214060 >> 172.21.15.26:0/136214060 conn(0x7f19c8058ef0 msgr2=0x7f19c8059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f7d09ffb640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cf806bf00 0x7f7cf806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f19d4a37640 1 -- 172.21.15.26:0/136214060 shutdown_connections 2024-04-24T23:21:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f19d4a37640 1 -- 172.21.15.26:0/136214060 wait complete. 2024-04-24T23:21:05.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f7d192a5640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cf806bf00 0x7f7cf806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2beffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2c013df50 0x7fa2c01f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2beffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2c013df50 0x7fa2c01f0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:56682/0 (socket says 172.21.15.26:56682) 2024-04-24T23:21:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2beffd640 1 -- 172.21.15.26:0/288426908 learned_addr learned my addr 172.21.15.26:0/288426908 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f7d09ffb640 1 -- 172.21.15.26:0/3241330648 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d0c08ac70 con 0x7f7d1415c2e0 2024-04-24T23:21:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2be7fc640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2c0141470 0x7fa2c015a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2beffd640 1 -- 172.21.15.26:0/288426908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c015f3a0 msgr2=0x7fa2c006f540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:21:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2beffd640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c015f3a0 0x7fa2c006f540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2beffd640 1 -- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2c0141470 msgr2=0x7fa2c015a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2beffd640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2c0141470 0x7fa2c015a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2beffd640 1 -- 172.21.15.26:0/288426908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c013b670 con 0x7fa2c013df50 2024-04-24T23:21:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7f7d192a5640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cf806bf00 0x7f7cf806e3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d04006fd0 tx=0x7f7d04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcfde9a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcf806eb10 0x7efcf8153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcfde9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf80b44b0 0x7efcf815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcfde9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf815a0a0 0x7efcf815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcfde9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efcf815a480 con 0x7efcf80b44b0 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcfde9a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8071cb0 con 0x7efcf815a0a0 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcfde9a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efcf815a600 con 0x7efcf806eb10 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcf77fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcf806eb10 0x7efcf8153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcf77fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcf806eb10 0x7efcf8153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38308/0 (socket says 172.21.15.26:38308) 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcf77fe640 1 -- 172.21.15.26:0/2136838193 learned_addr learned my addr 172.21.15.26:0/2136838193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f18abfff640 1 -- 172.21.15.26:0/2972648817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 msgr2=0x7f18ac0cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f18abfff640 1 --2- 172.21.15.26:0/2972648817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 0x7f18ac0cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f18abfff640 1 -- 172.21.15.26:0/2972648817 >> v1:172.21.15.26:6789/0 conn(0x7f18ac157df0 legacy=0x7f18ac0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.151+0000 7f18abfff640 1 -- 172.21.15.26:0/2972648817 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac15ade0 con 0x7f18ac15a7d0 2024-04-24T23:21:05.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7fa2beffd640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2c013df50 0x7fa2c01f0700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2b002a860 tx=0x7fa2b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7efcf7fff640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf815a0a0 0x7efcf815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.152+0000 7f3a45dd5640 1 -- start start 2024-04-24T23:21:05.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f124f7fe640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f123406bf30 0x7f123406e3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f12500502b0 tx=0x7f125007ab60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcf6ffd640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf80b44b0 0x7efcf815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcf77fe640 1 -- 172.21.15.26:0/2136838193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf815a0a0 msgr2=0x7efcf815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcf77fe640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf815a0a0 0x7efcf815d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcf77fe640 1 -- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf80b44b0 msgr2=0x7efcf815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcf77fe640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf80b44b0 0x7efcf815cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcf77fe640 1 -- 172.21.15.26:0/2136838193 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf81f9050 con 0x7efcf806eb10 2024-04-24T23:21:05.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f18abfff640 1 --2- 172.21.15.26:0/2972648817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 0x7f18ac0cc270 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009d90 tx=0x7f18a402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f110507f0b423f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcf6ffd640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf80b44b0 0x7efcf815cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:05.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7feb1de46640 1 -- 172.21.15.26:0/1984440235 shutdown_connections 2024-04-24T23:21:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7feb1de46640 1 -- 172.21.15.26:0/1984440235 wait complete. 2024-04-24T23:21:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a45dd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc300 0x7f3a40134b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a45dd5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a400cc6e0 0x7f3a40164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a45dd5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a401638f0 0x7f3a401654f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a45dd5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a401665c0 con 0x7f3a400cc300 2024-04-24T23:21:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a45dd5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a40166440 con 0x7f3a400cc6e0 2024-04-24T23:21:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a45dd5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3a40166740 con 0x7f3a401638f0 2024-04-24T23:21:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcf77fe640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcf806eb10 0x7efcf8153300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efcf0036040 tx=0x7efcf0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a3f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc300 0x7f3a40134b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a3f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc300 0x7f3a40134b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40068/0 (socket says 172.21.15.26:40068) 2024-04-24T23:21:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a3f7fe640 1 -- 172.21.15.26:0/3710201353 learned_addr learned my addr 172.21.15.26:0/3710201353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a3ffff640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a401638f0 0x7f3a401654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f3a3effd640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a400cc6e0 0x7f3a40164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcdffff640 1 -- 172.21.15.26:0/2136838193 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0008240 con 0x7efcf806eb10 2024-04-24T23:21:05.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7f19d4a37640 1 Processor -- start 2024-04-24T23:21:05.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.153+0000 7efcdffff640 1 -- 172.21.15.26:0/2136838193 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf00083e0 con 0x7efcf806eb10 2024-04-24T23:21:05.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7f19d4a37640 1 -- start start 2024-04-24T23:21:05.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf81fbab0 con 0x7efcf806eb10 2024-04-24T23:21:05.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7f3a3f7fe640 1 -- 172.21.15.26:0/3710201353 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a401638f0 msgr2=0x7f3a401654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7f3a3f7fe640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a401638f0 0x7f3a401654f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7f3a3f7fe640 1 -- 172.21.15.26:0/3710201353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a400cc6e0 msgr2=0x7f3a40164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7f3a3f7fe640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a400cc6e0 0x7f3a40164fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7f3a3f7fe640 1 -- 172.21.15.26:0/3710201353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4013ce10 con 0x7f3a400cc300 2024-04-24T23:21:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7efcdffff640 1 -- 172.21.15.26:0/2136838193 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00087b0 con 0x7efcf806eb10 2024-04-24T23:21:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf81fc590 con 0x7efcf806eb10 2024-04-24T23:21:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.154+0000 7f3a3f7fe640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc300 0x7f3a40134b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c7f0 tx=0x7f3a3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19d4a37640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c8037fe0 0x7f19c80d9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19d4a37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c803c640 0x7f19c8039f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19d4a37640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c8058960 0x7f19c803a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19ced9d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c8037fe0 0x7f19c80d9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19ce59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c803c640 0x7f19c8039f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7feb1de46640 1 Processor -- start 2024-04-24T23:21:05.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19ced9d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c8037fe0 0x7f19c80d9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:38332/0 (socket says 172.21.15.26:38332) 2024-04-24T23:21:05.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7feb1de46640 1 -- start start 2024-04-24T23:21:05.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19ced9d640 1 -- 172.21.15.26:0/4162227608 learned_addr learned my addr 172.21.15.26:0/4162227608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf81fc760 con 0x7efcf806eb10 2024-04-24T23:21:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19cf59e640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c8058960 0x7f19c803a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8057c20 con 0x7f19c803c640 2024-04-24T23:21:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8057940 con 0x7f19c8058960 2024-04-24T23:21:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f19c8057f00 con 0x7f19c8037fe0 2024-04-24T23:21:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.155+0000 7efcdffff640 1 -- 172.21.15.26:0/2136838193 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcf0044020 con 0x7efcf806eb10 2024-04-24T23:21:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7efcdffff640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efce406be30 0x7efce406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7f19ce59c640 1 -- 172.21.15.26:0/4162227608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c8037fe0 msgr2=0x7f19c80d9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7efcf6ffd640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efce406be30 0x7efce406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7feb1de46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb1806ef60 0x7feb18071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7feb1de46640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1814be70 0x7feb18161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7feb1de46640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb18166230 0x7feb18148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7feb1de46640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feb1814c3d0 con 0x7feb1806ef60 2024-04-24T23:21:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7f19ce59c640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c8037fe0 0x7f19c80d9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7f19ce59c640 1 -- 172.21.15.26:0/4162227608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c8058960 msgr2=0x7f19c803a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7f19ce59c640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c8058960 0x7f19c803a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7f19ce59c640 1 -- 172.21.15.26:0/4162227608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c80511a0 con 0x7f19c803c640 2024-04-24T23:21:05.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7feb1de46640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feb1814c250 con 0x7feb18166230 2024-04-24T23:21:05.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7fa2a77fe640 1 -- 172.21.15.26:0/288426908 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0037040 con 0x7fa2c013df50 2024-04-24T23:21:05.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7f19ce59c640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c803c640 0x7f19c8039f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f19d0000b60 tx=0x7f19d0073690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7feb1de46640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feb180cdb40 con 0x7feb1814be70 2024-04-24T23:21:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb1806ef60 0x7feb18071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb17fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb18166230 0x7feb18148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7fa2a77fe640 1 -- 172.21.15.26:0/288426908 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b0042070 con 0x7fa2c013df50 2024-04-24T23:21:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7fa2a77fe640 1 -- 172.21.15.26:0/288426908 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0030bb0 con 0x7fa2c013df50 2024-04-24T23:21:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb1806ef60 0x7feb18071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40088/0 (socket says 172.21.15.26:40088) 2024-04-24T23:21:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7f123bfff640 1 -- 172.21.15.26:0/1255785323 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1248008850 con 0x7f12480507a0 2024-04-24T23:21:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c013b950 con 0x7fa2c013df50 2024-04-24T23:21:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb177fe640 1 -- 172.21.15.26:0/1883657747 learned_addr learned my addr 172.21.15.26:0/1883657747 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb16ffd640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1814be70 0x7feb18161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.156+0000 7efcdffff640 1 -- 172.21.15.26:0/2136838193 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efcf00abfd0 con 0x7efcf806eb10 2024-04-24T23:21:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7efcf6ffd640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efce406be30 0x7efce406e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efcec007fd0 tx=0x7efcec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c0137500 con 0x7fa2c013df50 2024-04-24T23:21:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb16ffd640 1 -- 172.21.15.26:0/1883657747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb18166230 msgr2=0x7feb18148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb16ffd640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb18166230 0x7feb18148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb16ffd640 1 -- 172.21.15.26:0/1883657747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb1806ef60 msgr2=0x7feb18071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb16ffd640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb1806ef60 0x7feb18071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb16ffd640 1 -- 172.21.15.26:0/1883657747 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb18160d60 con 0x7feb1814be70 2024-04-24T23:21:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.157+0000 7feb16ffd640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1814be70 0x7feb18161b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb0c031000 tx=0x7feb0c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.158+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c00681e0 con 0x7fa2c013df50 2024-04-24T23:21:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.158+0000 7f3a27fff640 1 -- 172.21.15.26:0/3710201353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30015070 con 0x7f3a400cc300 2024-04-24T23:21:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f3a27fff640 1 -- 172.21.15.26:0/3710201353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a30004220 con 0x7f3a400cc300 2024-04-24T23:21:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f3a27fff640 1 -- 172.21.15.26:0/3710201353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30007620 con 0x7f3a400cc300 2024-04-24T23:21:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f7d09ffb640 1 -- 172.21.15.26:0/3241330648 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d0c059600 con 0x7f7d1415c2e0 2024-04-24T23:21:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f19b77fe640 1 -- 172.21.15.26:0/4162227608 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0098070 con 0x7f19c803c640 2024-04-24T23:21:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.158+0000 7f18ab7fe640 1 -- 172.21.15.26:0/2972648817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4007740 con 0x7f18ac15a7d0 2024-04-24T23:21:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f18ab7fe640 1 -- 172.21.15.26:0/2972648817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a40078e0 con 0x7f18ac15a7d0 2024-04-24T23:21:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f18ab7fe640 1 -- 172.21.15.26:0/2972648817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4007cf0 con 0x7f18ac15a7d0 2024-04-24T23:21:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4013d0f0 con 0x7f3a400cc300 2024-04-24T23:21:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4013d6b0 con 0x7f3a400cc300 2024-04-24T23:21:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f18b2e4d640 1 -- 172.21.15.26:0/2972648817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 msgr2=0x7f18ac0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f19b77fe640 1 -- 172.21.15.26:0/4162227608 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d0071790 con 0x7f19c803c640 2024-04-24T23:21:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f19b77fe640 1 -- 172.21.15.26:0/4162227608 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0071960 con 0x7f19c803c640 2024-04-24T23:21:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c800d760 con 0x7f19c803c640 2024-04-24T23:21:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.159+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c8051440 con 0x7f19c803c640 2024-04-24T23:21:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f18b2e4d640 1 --2- 172.21.15.26:0/2972648817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 0x7f18ac0cc270 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009d90 tx=0x7f18a402a680 comp rx=0 tx=0).stop 2024-04-24T23:21:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7fa2a77fe640 1 -- 172.21.15.26:0/288426908 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2b0047460 con 0x7fa2c013df50 2024-04-24T23:21:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f18b2e4d640 1 -- 172.21.15.26:0/2972648817 shutdown_connections 2024-04-24T23:21:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f18b2e4d640 1 --2- 172.21.15.26:0/2972648817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 0x7f18ac0cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f18b2e4d640 1 --2- 172.21.15.26:0/2972648817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 0x7f18ac0cc270 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f18b2e4d640 1 -- 172.21.15.26:0/2972648817 >> 172.21.15.26:0/2972648817 conn(0x7f18ac0b1b80 msgr2=0x7f18ac0b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f18b2e4d640 1 -- 172.21.15.26:0/2972648817 shutdown_connections 2024-04-24T23:21:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f18b2e4d640 1 -- 172.21.15.26:0/2972648817 wait complete. 2024-04-24T23:21:05.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c8005710 con 0x7f19c803c640 2024-04-24T23:21:05.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7fa2a77fe640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2ac06be30 0x7fa2ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4013deb0 con 0x7f3a400cc300 2024-04-24T23:21:05.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.160+0000 7fa2a77fe640 1 -- 172.21.15.26:0/288426908 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa2b00aba00 con 0x7fa2c013df50 2024-04-24T23:21:05.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.161+0000 7fa2be7fc640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2ac06be30 0x7fa2ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.161+0000 7fa2be7fc640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2ac06be30 0x7fa2ac06e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0140670 tx=0x7fa2b4007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.161+0000 7feafffff640 1 -- 172.21.15.26:0/1883657747 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c041030 con 0x7feb1814be70 2024-04-24T23:21:05.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.161+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb18161040 con 0x7feb1814be70 2024-04-24T23:21:05.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.161+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb180aa320 con 0x7feb1814be70 2024-04-24T23:21:05.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.162+0000 7f18b2e4d640 1 Processor -- start 2024-04-24T23:21:05.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.162+0000 7f18b2e4d640 1 -- start start 2024-04-24T23:21:05.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.162+0000 7feafffff640 1 -- 172.21.15.26:0/1883657747 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb0c03b640 con 0x7feb1814be70 2024-04-24T23:21:05.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.162+0000 7feafffff640 1 -- 172.21.15.26:0/1883657747 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c037070 con 0x7feb1814be70 2024-04-24T23:21:05.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7f19b77fe640 1 -- 172.21.15.26:0/4162227608 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19d009b070 con 0x7f19c803c640 2024-04-24T23:21:05.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7efcdffff640 1 -- 172.21.15.26:0/2136838193 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efcf81fc760 con 0x7efcf806eb10 2024-04-24T23:21:05.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7f18b2e4d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 0x7f18ac1472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7f18b2e4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac157df0 0x7f18ac15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7f18b2e4d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 0x7f18ac15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7f18b2e4d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac15b680 con 0x7f18ac15a7d0 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7f18b2e4d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac15b3a0 con 0x7f18ac157df0 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7f18b2e4d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f18ac0ce1d0 con 0x7f18ac0cf2a0 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18b13ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 0x7f18ac15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18b13ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 0x7f18ac15d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40090/0 (socket says 172.21.15.26:40090) 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18b13ea640 1 -- 172.21.15.26:0/644829883 learned_addr learned my addr 172.21.15.26:0/644829883 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18abfff640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac157df0 0x7f18ac15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18b0be9640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 0x7f18ac1472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18b13ea640 1 -- 172.21.15.26:0/644829883 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 msgr2=0x7f18ac1472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7feafffff640 1 -- 172.21.15.26:0/1883657747 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb0c044070 con 0x7feb1814be70 2024-04-24T23:21:05.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18b13ea640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 0x7f18ac1472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18b13ea640 1 -- 172.21.15.26:0/644829883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac157df0 msgr2=0x7f18ac15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18b13ea640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac157df0 0x7f18ac15cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.164+0000 7f18b13ea640 1 -- 172.21.15.26:0/644829883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac1593c0 con 0x7f18ac15a7d0 2024-04-24T23:21:05.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.165+0000 7feafffff640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0406be80 0x7feb0406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.165+0000 7feafffff640 1 -- 172.21.15.26:0/1883657747 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feb0c0aaf30 con 0x7feb1814be70 2024-04-24T23:21:05.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.165+0000 7f3a27fff640 1 -- 172.21.15.26:0/3710201353 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a300079d0 con 0x7f3a400cc300 2024-04-24T23:21:05.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.165+0000 7f18b13ea640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 0x7f18ac15d2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f189400d7b0 tx=0x7f189400dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.165+0000 7feb177fe640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0406be80 0x7feb0406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7f19b77fe640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19bc06be80 0x7f19bc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.163+0000 7f19b77fe640 1 -- 172.21.15.26:0/4162227608 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f19d01061a0 con 0x7f19c803c640 2024-04-24T23:21:05.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.165+0000 7f19ced9d640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19bc06be80 0x7f19bc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.165+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf8005290 con 0x7feb1814be70 2024-04-24T23:21:05.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.165+0000 7fa2a77fe640 1 -- 172.21.15.26:0/288426908 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa2b0045360 con 0x7fa2c013df50 2024-04-24T23:21:05.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.166+0000 7f3a27fff640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.166+0000 7f3a27fff640 1 -- 172.21.15.26:0/3710201353 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3a3008b380 con 0x7f3a400cc300 2024-04-24T23:21:05.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.166+0000 7f3a3effd640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.166+0000 7feb177fe640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0406be80 0x7feb0406e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb080011e0 tx=0x7feb08009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.166+0000 7f19ced9d640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19bc06be80 0x7f19bc06e340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19c0006d40 tx=0x7f19c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.166+0000 7f3a3effd640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a34034000 tx=0x7f3a3402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.170+0000 7feafffff640 1 -- 172.21.15.26:0/1883657747 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feb0c0799c0 con 0x7feb1814be70 2024-04-24T23:21:05.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.170+0000 7f19b77fe640 1 -- 172.21.15.26:0/4162227608 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f19d00a3020 con 0x7f19c803c640 2024-04-24T23:21:05.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.180+0000 7f18a97fa640 1 -- 172.21.15.26:0/644829883 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894015070 con 0x7f18ac15a7d0 2024-04-24T23:21:05.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.181+0000 7f3a27fff640 1 -- 172.21.15.26:0/3710201353 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a30059630 con 0x7f3a400cc300 2024-04-24T23:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.180+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ac1596a0 con 0x7f18ac15a7d0 2024-04-24T23:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.181+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ac159c60 con 0x7f18ac15a7d0 2024-04-24T23:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.181+0000 7f18a97fa640 1 -- 172.21.15.26:0/644829883 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1894007b30 con 0x7f18ac15a7d0 2024-04-24T23:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.182+0000 7f18a97fa640 1 -- 172.21.15.26:0/644829883 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894010470 con 0x7f18ac15a7d0 2024-04-24T23:21:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.185+0000 7f18a97fa640 1 -- 172.21.15.26:0/644829883 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1894018070 con 0x7f18ac15a7d0 2024-04-24T23:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.185+0000 7f18a97fa640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f188c06bf00 0x7f188c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.186+0000 7f18a97fa640 1 -- 172.21.15.26:0/644829883 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f189408ade0 con 0x7f18ac15a7d0 2024-04-24T23:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.186+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1890005290 con 0x7f18ac15a7d0 2024-04-24T23:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.186+0000 7f18b0be9640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f188c06bf00 0x7f188c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.189+0000 7f18b0be9640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f188c06bf00 0x7f188c06e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f189c0048a0 tx=0x7f189c00a1a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.196+0000 7f18a97fa640 1 -- 172.21.15.26:0/644829883 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18940591b0 con 0x7f18ac15a7d0 2024-04-24T23:21:05.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.565+0000 7f12548e6640 1 -- 172.21.15.26:0/1255785323 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f12480530d0 con 0x7f12480507a0 2024-04-24T23:21:05.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.566+0000 7f123bfff640 1 -- 172.21.15.26:0/1255785323 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f123c059390 con 0x7f12480507a0 2024-04-24T23:21:05.567 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:05.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.569+0000 7f1239ffb640 1 -- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f123406bf30 msgr2=0x7f123406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.569+0000 7f1239ffb640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f123406bf30 0x7f123406e3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f12500502b0 tx=0x7f125007ab60 comp rx=0 tx=0).stop 2024-04-24T23:21:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.569+0000 7f1239ffb640 1 -- 172.21.15.26:0/1255785323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12480507a0 msgr2=0x7f12481806f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.569+0000 7f1239ffb640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12480507a0 0x7f12481806f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f123c00c460 tx=0x7f123c00c920 comp rx=0 tx=0).stop 2024-04-24T23:21:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.570+0000 7f1239ffb640 1 -- 172.21.15.26:0/1255785323 shutdown_connections 2024-04-24T23:21:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.570+0000 7f1239ffb640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f123406bf30 0x7f123406e3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.570+0000 7f1239ffb640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480e3130 0x7f124817da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.570+0000 7f1239ffb640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12480507a0 0x7f12481806f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.570+0000 7f1239ffb640 1 --2- 172.21.15.26:0/1255785323 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1248034610 0x7f12480d5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.570+0000 7f1239ffb640 1 -- 172.21.15.26:0/1255785323 >> 172.21.15.26:0/1255785323 conn(0x7f1248055350 msgr2=0x7f124804c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.570+0000 7f1239ffb640 1 -- 172.21.15.26:0/1255785323 shutdown_connections 2024-04-24T23:21:05.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.570+0000 7f1239ffb640 1 -- 172.21.15.26:0/1255785323 wait complete. 2024-04-24T23:21:05.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-24T23:21:05.589 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:05.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.590+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7d140ce770 con 0x7f7d1415c2e0 2024-04-24T23:21:05.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.591+0000 7f7d09ffb640 1 -- 172.21.15.26:0/3241330648 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7d0c058f00 con 0x7f7d1415c2e0 2024-04-24T23:21:05.593 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:05.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.594+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cf806bf00 msgr2=0x7f7cf806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.594+0000 7f7d1b509640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cf806bf00 0x7f7cf806e3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d04006fd0 tx=0x7f7d04008040 comp rx=0 tx=0).stop 2024-04-24T23:21:05.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.594+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d1415c2e0 msgr2=0x7f7d140afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.594+0000 7f7d1b509640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d1415c2e0 0x7f7d140afca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b990 tx=0x7f7d0c00be50 comp rx=0 tx=0).stop 2024-04-24T23:21:05.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.595+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 shutdown_connections 2024-04-24T23:21:05.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.595+0000 7f7d1b509640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cf806bf00 0x7f7cf806e3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.595+0000 7f7d1b509640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d1415c2e0 0x7f7d140afca0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.595+0000 7f7d1b509640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7d140ce990 0x7f7d14151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.595+0000 7f7d1b509640 1 --2- 172.21.15.26:0/3241330648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7d140b1860 0x7f7d14157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.595+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 >> 172.21.15.26:0/3241330648 conn(0x7f7d140d1f80 msgr2=0x7f7d14158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.595+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 shutdown_connections 2024-04-24T23:21:05.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.595+0000 7f7d1b509640 1 -- 172.21.15.26:0/3241330648 wait complete. 2024-04-24T23:21:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.596+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efcf80694c0 con 0x7efcf806eb10 2024-04-24T23:21:05.598 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.596+0000 7efcdffff640 1 -- 172.21.15.26:0/2136838193 <== mon.2 v2:172.21.15.29:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efcf007a3c0 con 0x7efcf806eb10 2024-04-24T23:21:05.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.599+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7feaf8005850 con 0x7feb1814be70 2024-04-24T23:21:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.599+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa2c015e7f0 con 0x7fa2c013df50 2024-04-24T23:21:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.599+0000 7feafffff640 1 -- 172.21.15.26:0/1883657747 <== mon.2 v2:172.21.15.29:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb0c0792c0 con 0x7feb1814be70 2024-04-24T23:21:05.602 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-04-24T23:21:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efce406be30 msgr2=0x7efce406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efce406be30 0x7efce406e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efcec007fd0 tx=0x7efcec009040 comp rx=0 tx=0).stop 2024-04-24T23:21:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcf806eb10 msgr2=0x7efcf8153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcf806eb10 0x7efcf8153300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efcf0036040 tx=0x7efcf0004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 shutdown_connections 2024-04-24T23:21:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efce406be30 0x7efce406e2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efcf815a0a0 0x7efcf815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcf80b44b0 0x7efcf815cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 --2- 172.21.15.26:0/2136838193 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efcf806eb10 0x7efcf8153300 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 >> 172.21.15.26:0/2136838193 conn(0x7efcf806b300 msgr2=0x7efcf80b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.600+0000 7fa2a77fe640 1 -- 172.21.15.26:0/288426908 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa2b007a310 con 0x7fa2c013df50 2024-04-24T23:21:05.603 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.601+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 shutdown_connections 2024-04-24T23:21:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.601+0000 7efcfde9a640 1 -- 172.21.15.26:0/2136838193 wait complete. 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0406be80 msgr2=0x7feb0406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0406be80 0x7feb0406e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb080011e0 tx=0x7feb08009040 comp rx=0 tx=0).stop 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1814be70 msgr2=0x7feb18161b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1814be70 0x7feb18161b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb0c031000 tx=0x7feb0c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 shutdown_connections 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0406be80 0x7feb0406e340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb18166230 0x7feb18148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1814be70 0x7feb18161b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 --2- 172.21.15.26:0/1883657747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb1806ef60 0x7feb18071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 >> 172.21.15.26:0/1883657747 conn(0x7feb180b1b80 msgr2=0x7feb180b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 shutdown_connections 2024-04-24T23:21:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7feb1de46640 1 -- 172.21.15.26:0/1883657747 wait complete. 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2ac06be30 msgr2=0x7fa2ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7fa2c5561640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2ac06be30 0x7fa2ac06e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0140670 tx=0x7fa2b4007b50 comp rx=0 tx=0).stop 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2c013df50 msgr2=0x7fa2c01f0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.603+0000 7fa2c5561640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2c013df50 0x7fa2c01f0700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2b002a860 tx=0x7fa2b00046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.604+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 shutdown_connections 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.604+0000 7fa2c5561640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2ac06be30 0x7fa2ac06e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.604+0000 7fa2c5561640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2c015f3a0 0x7fa2c006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.604+0000 7fa2c5561640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2c0141470 0x7fa2c015a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.604+0000 7fa2c5561640 1 --2- 172.21.15.26:0/288426908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2c013df50 0x7fa2c01f0700 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.604+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 >> 172.21.15.26:0/288426908 conn(0x7fa2c00b2ce0 msgr2=0x7fa2c015bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.604+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 shutdown_connections 2024-04-24T23:21:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.604+0000 7fa2c5561640 1 -- 172.21.15.26:0/288426908 wait complete. 2024-04-24T23:21:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.604+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f19c8053d90 con 0x7f19c803c640 2024-04-24T23:21:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.605+0000 7f19b77fe640 1 -- 172.21.15.26:0/4162227608 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f19d0090070 con 0x7f19c803c640 2024-04-24T23:21:05.608 INFO:teuthology.orchestra.run.smithi026.stdout:55834574856 2024-04-24T23:21:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19bc06be80 msgr2=0x7f19bc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19bc06be80 0x7f19bc06e340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19c0006d40 tx=0x7f19c0008040 comp rx=0 tx=0).stop 2024-04-24T23:21:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c803c640 msgr2=0x7f19c8039f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c803c640 0x7f19c8039f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f19d0000b60 tx=0x7f19d0073690 comp rx=0 tx=0).stop 2024-04-24T23:21:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 shutdown_connections 2024-04-24T23:21:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19bc06be80 0x7f19bc06e340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c8058960 0x7f19c803a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c803c640 0x7f19c8039f00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 --2- 172.21.15.26:0/4162227608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c8037fe0 0x7f19c80d9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.609+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 >> 172.21.15.26:0/4162227608 conn(0x7f19c8058ef0 msgr2=0x7f19c804d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.610+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 shutdown_connections 2024-04-24T23:21:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.610+0000 7f19d4a37640 1 -- 172.21.15.26:0/4162227608 wait complete. 2024-04-24T23:21:05.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.613+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3a400694c0 con 0x7f3a400cc300 2024-04-24T23:21:05.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.614+0000 7f3a27fff640 1 -- 172.21.15.26:0/3710201353 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a3005d930 con 0x7f3a400cc300 2024-04-24T23:21:05.617 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:05.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-24T23:21:05.618 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.617+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3a2c06be30 msgr2=0x7f3a2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a34034000 tx=0x7f3a3402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc300 msgr2=0x7f3a40134b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc300 0x7f3a40134b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c7f0 tx=0x7f3a3000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 shutdown_connections 2024-04-24T23:21:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a401638f0 0x7f3a401654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a400cc6e0 0x7f3a40164fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 --2- 172.21.15.26:0/3710201353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a400cc300 0x7f3a40134b50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 >> 172.21.15.26:0/3710201353 conn(0x7f3a4006b300 msgr2=0x7f3a4006b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 shutdown_connections 2024-04-24T23:21:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.618+0000 7f3a45dd5640 1 -- 172.21.15.26:0/3710201353 wait complete. 2024-04-24T23:21:05.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-04-24T23:21:05.624 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:05.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-24T23:21:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.622+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1890005580 con 0x7f18ac15a7d0 2024-04-24T23:21:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.623+0000 7f18a97fa640 1 -- 172.21.15.26:0/644829883 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f189405d4b0 con 0x7f18ac15a7d0 2024-04-24T23:21:05.625 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:05.625 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-24T23:21:05.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-24T23:21:05.628 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f188c06bf00 msgr2=0x7f188c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f188c06bf00 0x7f188c06e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f189c0048a0 tx=0x7f189c00a1a0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 msgr2=0x7f18ac15d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 0x7f18ac15d2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f189400d7b0 tx=0x7f189400dc70 comp rx=0 tx=0).stop 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 shutdown_connections 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f188c06bf00 0x7f188c06e3c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18ac15a7d0 0x7f18ac15d2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18ac157df0 0x7f18ac15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 --2- 172.21.15.26:0/644829883 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f18ac0cf2a0 0x7f18ac1472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 >> 172.21.15.26:0/644829883 conn(0x7f18ac0b1b80 msgr2=0x7f18ac14ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 shutdown_connections 2024-04-24T23:21:05.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.626+0000 7f18b2e4d640 1 -- 172.21.15.26:0/644829883 wait complete. 2024-04-24T23:21:05.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-04-24T23:21:05.630 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:05.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-24T23:21:05.634 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:05.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-24T23:21:05.639 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:05.639 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T23:21:05.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:21:05.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.779+0000 7fc0aa509640 1 Processor -- start 2024-04-24T23:21:05.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.779+0000 7fc0aa509640 1 -- start start 2024-04-24T23:21:05.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0aa509640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0a415eb00 0x7fc0a415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0aa509640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a415f450 con 0x7fc0a40cf870 2024-04-24T23:21:05.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0aa509640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a40cd0b0 con 0x7fc0a4143c40 2024-04-24T23:21:05.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0aa509640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc0a4071b70 con 0x7fc0a415eb00 2024-04-24T23:21:05.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0a8aa6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0a415eb00 0x7fc0a415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0a37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc0a40cf870 0x7fc0a4072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54672/0 (socket says 172.21.15.26:54672) 2024-04-24T23:21:05.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0a37fe640 1 -- 172.21.15.26:0/3722167119 learned_addr learned my addr 172.21.15.26:0/3722167119 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:05.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0a8aa6640 1 -- 172.21.15.26:0/3722167119 >> v1:172.21.15.29:6789/0 conn(0x7fc0a4143c40 legacy=0x7fc0a40cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:05.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0a8aa6640 1 -- 172.21.15.26:0/3722167119 >> v1:172.21.15.26:6789/0 conn(0x7fc0a40cf870 legacy=0x7fc0a4072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0a2ffd640 1 -- 172.21.15.26:0/3722167119 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2675044586 0 0) 0x7fc098001110 con 0x7fc0a40cf870 2024-04-24T23:21:05.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.780+0000 7fc0a8aa6640 1 -- 172.21.15.26:0/3722167119 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a4071240 con 0x7fc0a415eb00 2024-04-24T23:21:05.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.781+0000 7fc0a8aa6640 1 --2- 172.21.15.26:0/3722167119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 0x7fc0a415ef10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009d90 tx=0x7fc09c02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ea42c45de443b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.781+0000 7fc0a2ffd640 1 -- 172.21.15.26:0/3722167119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c032770 con 0x7fc0a415eb00 2024-04-24T23:21:05.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.781+0000 7fc0a2ffd640 1 -- 172.21.15.26:0/3722167119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09c038070 con 0x7fc0a415eb00 2024-04-24T23:21:05.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.781+0000 7fc0a2ffd640 1 -- 172.21.15.26:0/3722167119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c032b30 con 0x7fc0a415eb00 2024-04-24T23:21:05.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.782+0000 7fc0aa509640 1 -- 172.21.15.26:0/3722167119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 msgr2=0x7fc0a415ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.782+0000 7fc0aa509640 1 --2- 172.21.15.26:0/3722167119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 0x7fc0a415ef10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009d90 tx=0x7fc09c02c8b0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.782+0000 7fc0aa509640 1 -- 172.21.15.26:0/3722167119 shutdown_connections 2024-04-24T23:21:05.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.782+0000 7fc0aa509640 1 --2- 172.21.15.26:0/3722167119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 0x7fc0a415ef10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.782+0000 7fc0aa509640 1 -- 172.21.15.26:0/3722167119 >> 172.21.15.26:0/3722167119 conn(0x7fc0a406b300 msgr2=0x7fc0a406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.782+0000 7fc0aa509640 1 -- 172.21.15.26:0/3722167119 shutdown_connections 2024-04-24T23:21:05.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.782+0000 7fc0aa509640 1 -- 172.21.15.26:0/3722167119 wait complete. 2024-04-24T23:21:05.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.783+0000 7fc0aa509640 1 Processor -- start 2024-04-24T23:21:05.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.783+0000 7fc0aa509640 1 -- start start 2024-04-24T23:21:05.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0aa509640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc0a40cf870 0x7fc0a4159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0aa509640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0a4143c40 0x7fc0a4141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0aa509640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 0x7fc0a415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0aa509640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4144020 con 0x7fc0a415eb00 2024-04-24T23:21:05.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0aa509640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4071800 con 0x7fc0a4143c40 2024-04-24T23:21:05.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0aa509640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc0a40b2e80 con 0x7fc0a40cf870 2024-04-24T23:21:05.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a8aa6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 0x7fc0a415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a8aa6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 0x7fc0a415b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40106/0 (socket says 172.21.15.26:40106) 2024-04-24T23:21:05.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a8aa6640 1 -- 172.21.15.26:0/4089012963 learned_addr learned my addr 172.21.15.26:0/4089012963 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:05.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a37fe640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0a4143c40 0x7fc0a4141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a3fff640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc0a40cf870 0x7fc0a4159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a8aa6640 1 -- 172.21.15.26:0/4089012963 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc0a40cf870 msgr2=0x7fc0a4159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a8aa6640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc0a40cf870 0x7fc0a4159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a8aa6640 1 -- 172.21.15.26:0/4089012963 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0a4143c40 msgr2=0x7fc0a4141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a8aa6640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0a4143c40 0x7fc0a4141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.784+0000 7fc0a8aa6640 1 -- 172.21.15.26:0/4089012963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a415cdb0 con 0x7fc0a415eb00 2024-04-24T23:21:05.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.785+0000 7fc0a8aa6640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 0x7fc0a415b3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009d90 tx=0x7fc09c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.785+0000 7fc0a0ff9640 1 -- 172.21.15.26:0/4089012963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c033810 con 0x7fc0a415eb00 2024-04-24T23:21:05.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.785+0000 7fc0a0ff9640 1 -- 172.21.15.26:0/4089012963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09c032e50 con 0x7fc0a415eb00 2024-04-24T23:21:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.785+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a415d090 con 0x7fc0a415eb00 2024-04-24T23:21:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.785+0000 7fc0a0ff9640 1 -- 172.21.15.26:0/4089012963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c033c10 con 0x7fc0a415eb00 2024-04-24T23:21:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.785+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a4140240 con 0x7fc0a415eb00 2024-04-24T23:21:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.786+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a415d260 con 0x7fc0a415eb00 2024-04-24T23:21:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.786+0000 7fc0a0ff9640 1 -- 172.21.15.26:0/4089012963 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc09c032070 con 0x7fc0a415eb00 2024-04-24T23:21:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.786+0000 7fc0a0ff9640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc09006be30 0x7fc09006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.787+0000 7fc0a3fff640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc09006be30 0x7fc09006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.787+0000 7fc0a0ff9640 1 -- 172.21.15.26:0/4089012963 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc09c0ade30 con 0x7fc0a415eb00 2024-04-24T23:21:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.787+0000 7fc0a3fff640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc09006be30 0x7fc09006e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc094004ea0 tx=0x7fc094009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.791+0000 7fc0a0ff9640 1 -- 172.21.15.26:0/4089012963 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc09c042070 con 0x7fc0a415eb00 2024-04-24T23:21:05.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.970+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0a40694c0 con 0x7fc09006be30 2024-04-24T23:21:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.976+0000 7fc0a0ff9640 1 -- 172.21.15.26:0/4089012963 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fc0a40694c0 con 0x7fc09006be30 2024-04-24T23:21:05.980 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc09006be30 msgr2=0x7fc09006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc09006be30 0x7fc09006e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc094004ea0 tx=0x7fc094009040 comp rx=0 tx=0).stop 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 msgr2=0x7fc0a415b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 0x7fc0a415b3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009d90 tx=0x7fc09c004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 shutdown_connections 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc09006be30 0x7fc09006e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0a415eb00 0x7fc0a415b3c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0a4143c40 0x7fc0a4141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 --2- 172.21.15.26:0/4089012963 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc0a40cf870 0x7fc0a4159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 >> 172.21.15.26:0/4089012963 conn(0x7fc0a406b300 msgr2=0x7fc0a40afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 shutdown_connections 2024-04-24T23:21:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:05.979+0000 7fc0aa509640 1 -- 172.21.15.26:0/4089012963 wait complete. 2024-04-24T23:21:05.991 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-24T23:21:05.317422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256448,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718272,"statfs":{"total":773094113280,"available":772831510528,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.856172"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725536+0000","last_change":"2024-04-24T23:20:55.487607+0000","last_active":"2024-04-24T23:20:56.725536+0000","last_peered":"2024-04-24T23:20:56.725536+0000","last_clean":"2024-04-24T23:20:56.725536+0000","last_became_active":"2024-04-24T23:20:55.487320+0000","last_became_peered":"2024-04-24T23:20:55.487320+0000","last_unstale":"2024-04-24T23:20:56.725536+0000","last_undegraded":"2024-04-24T23:20:56.725536+0000","last_fullsized":"2024-04-24T23:20:56.725536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:27.675103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478125+0000","last_change":"2024-04-24T23:20:55.480991+0000","last_active":"2024-04-24T23:20:56.478125+0000","last_peered":"2024-04-24T23:20:56.478125+0000","last_clean":"2024-04-24T23:20:56.478125+0000","last_became_active":"2024-04-24T23:20:55.480654+0000","last_became_peered":"2024-04-24T23:20:55.480654+0000","last_unstale":"2024-04-24T23:20:56.478125+0000","last_undegraded":"2024-04-24T23:20:56.478125+0000","last_fullsized":"2024-04-24T23:20:56.478125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724519+0000","last_change":"2024-04-24T23:20:55.479402+0000","last_active":"2024-04-24T23:20:56.724519+0000","last_peered":"2024-04-24T23:20:56.724519+0000","last_clean":"2024-04-24T23:20:56.724519+0000","last_became_active":"2024-04-24T23:20:55.479013+0000","last_became_peered":"2024-04-24T23:20:55.479013+0000","last_unstale":"2024-04-24T23:20:56.724519+0000","last_undegraded":"2024-04-24T23:20:56.724519+0000","last_fullsized":"2024-04-24T23:20:56.724519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475989+0000","last_change":"2024-04-24T23:20:55.481507+0000","last_active":"2024-04-24T23:20:56.475989+0000","last_peered":"2024-04-24T23:20:56.475989+0000","last_clean":"2024-04-24T23:20:56.475989+0000","last_became_active":"2024-04-24T23:20:55.481144+0000","last_became_peered":"2024-04-24T23:20:55.481144+0000","last_unstale":"2024-04-24T23:20:56.475989+0000","last_undegraded":"2024-04-24T23:20:56.475989+0000","last_fullsized":"2024-04-24T23:20:56.475989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725067+0000","last_change":"2024-04-24T23:20:55.482367+0000","last_active":"2024-04-24T23:20:56.725067+0000","last_peered":"2024-04-24T23:20:56.725067+0000","last_clean":"2024-04-24T23:20:56.725067+0000","last_became_active":"2024-04-24T23:20:55.482047+0000","last_became_peered":"2024-04-24T23:20:55.482047+0000","last_unstale":"2024-04-24T23:20:56.725067+0000","last_undegraded":"2024-04-24T23:20:56.725067+0000","last_fullsized":"2024-04-24T23:20:56.725067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475443+0000","last_change":"2024-04-24T23:20:55.482474+0000","last_active":"2024-04-24T23:20:56.475443+0000","last_peered":"2024-04-24T23:20:56.475443+0000","last_clean":"2024-04-24T23:20:56.475443+0000","last_became_active":"2024-04-24T23:20:55.482157+0000","last_became_peered":"2024-04-24T23:20:55.482157+0000","last_unstale":"2024-04-24T23:20:56.475443+0000","last_undegraded":"2024-04-24T23:20:56.475443+0000","last_fullsized":"2024-04-24T23:20:56.475443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724980+0000","last_change":"2024-04-24T23:20:55.487389+0000","last_active":"2024-04-24T23:20:56.724980+0000","last_peered":"2024-04-24T23:20:56.724980+0000","last_clean":"2024-04-24T23:20:56.724980+0000","last_became_active":"2024-04-24T23:20:55.486724+0000","last_became_peered":"2024-04-24T23:20:55.486724+0000","last_unstale":"2024-04-24T23:20:56.724980+0000","last_undegraded":"2024-04-24T23:20:56.724980+0000","last_fullsized":"2024-04-24T23:20:56.724980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724916+0000","last_change":"2024-04-24T23:20:55.486339+0000","last_active":"2024-04-24T23:20:56.724916+0000","last_peered":"2024-04-24T23:20:56.724916+0000","last_clean":"2024-04-24T23:20:56.724916+0000","last_became_active":"2024-04-24T23:20:55.486072+0000","last_became_peered":"2024-04-24T23:20:55.486072+0000","last_unstale":"2024-04-24T23:20:56.724916+0000","last_undegraded":"2024-04-24T23:20:56.724916+0000","last_fullsized":"2024-04-24T23:20:56.724916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.491641+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.491349+0000","last_became_peered":"2024-04-24T23:20:55.491349+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:58.298429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480758+0000","last_change":"2024-04-24T23:20:55.479221+0000","last_active":"2024-04-24T23:20:56.480758+0000","last_peered":"2024-04-24T23:20:56.480758+0000","last_clean":"2024-04-24T23:20:56.480758+0000","last_became_active":"2024-04-24T23:20:55.478795+0000","last_became_peered":"2024-04-24T23:20:55.478795+0000","last_unstale":"2024-04-24T23:20:56.480758+0000","last_undegraded":"2024-04-24T23:20:56.480758+0000","last_fullsized":"2024-04-24T23:20:56.480758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480763+0000","last_change":"2024-04-24T23:20:55.479615+0000","last_active":"2024-04-24T23:20:56.480763+0000","last_peered":"2024-04-24T23:20:56.480763+0000","last_clean":"2024-04-24T23:20:56.480763+0000","last_became_active":"2024-04-24T23:20:55.479317+0000","last_became_peered":"2024-04-24T23:20:55.479317+0000","last_unstale":"2024-04-24T23:20:56.480763+0000","last_undegraded":"2024-04-24T23:20:56.480763+0000","last_fullsized":"2024-04-24T23:20:56.480763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475028+0000","last_change":"2024-04-24T23:20:55.481915+0000","last_active":"2024-04-24T23:20:56.475028+0000","last_peered":"2024-04-24T23:20:56.475028+0000","last_clean":"2024-04-24T23:20:56.475028+0000","last_became_active":"2024-04-24T23:20:55.481596+0000","last_became_peered":"2024-04-24T23:20:55.481596+0000","last_unstale":"2024-04-24T23:20:56.475028+0000","last_undegraded":"2024-04-24T23:20:56.475028+0000","last_fullsized":"2024-04-24T23:20:56.475028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:20.844340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725493+0000","last_change":"2024-04-24T23:20:55.476416+0000","last_active":"2024-04-24T23:20:56.725493+0000","last_peered":"2024-04-24T23:20:56.725493+0000","last_clean":"2024-04-24T23:20:56.725493+0000","last_became_active":"2024-04-24T23:20:55.476068+0000","last_became_peered":"2024-04-24T23:20:55.476068+0000","last_unstale":"2024-04-24T23:20:56.725493+0000","last_undegraded":"2024-04-24T23:20:56.725493+0000","last_fullsized":"2024-04-24T23:20:56.725493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724195+0000","last_change":"2024-04-24T23:20:55.479125+0000","last_active":"2024-04-24T23:20:56.724195+0000","last_peered":"2024-04-24T23:20:56.724195+0000","last_clean":"2024-04-24T23:20:56.724195+0000","last_became_active":"2024-04-24T23:20:55.478473+0000","last_became_peered":"2024-04-24T23:20:55.478473+0000","last_unstale":"2024-04-24T23:20:56.724195+0000","last_undegraded":"2024-04-24T23:20:56.724195+0000","last_fullsized":"2024-04-24T23:20:56.724195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:19.475113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477868+0000","last_change":"2024-04-24T23:20:55.474883+0000","last_active":"2024-04-24T23:20:56.477868+0000","last_peered":"2024-04-24T23:20:56.477868+0000","last_clean":"2024-04-24T23:20:56.477868+0000","last_became_active":"2024-04-24T23:20:55.474459+0000","last_became_peered":"2024-04-24T23:20:55.474459+0000","last_unstale":"2024-04-24T23:20:56.477868+0000","last_undegraded":"2024-04-24T23:20:56.477868+0000","last_fullsized":"2024-04-24T23:20:56.477868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475040+0000","last_change":"2024-04-24T23:20:55.483511+0000","last_active":"2024-04-24T23:20:56.475040+0000","last_peered":"2024-04-24T23:20:56.475040+0000","last_clean":"2024-04-24T23:20:56.475040+0000","last_became_active":"2024-04-24T23:20:55.483171+0000","last_became_peered":"2024-04-24T23:20:55.483171+0000","last_unstale":"2024-04-24T23:20:56.475040+0000","last_undegraded":"2024-04-24T23:20:56.475040+0000","last_fullsized":"2024-04-24T23:20:56.475040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475537+0000","last_change":"2024-04-24T23:20:55.489069+0000","last_active":"2024-04-24T23:20:56.475537+0000","last_peered":"2024-04-24T23:20:56.475537+0000","last_clean":"2024-04-24T23:20:56.475537+0000","last_became_active":"2024-04-24T23:20:55.488796+0000","last_became_peered":"2024-04-24T23:20:55.488796+0000","last_unstale":"2024-04-24T23:20:56.475537+0000","last_undegraded":"2024-04-24T23:20:56.475537+0000","last_fullsized":"2024-04-24T23:20:56.475537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724755+0000","last_change":"2024-04-24T23:20:55.480920+0000","last_active":"2024-04-24T23:20:56.724755+0000","last_peered":"2024-04-24T23:20:56.724755+0000","last_clean":"2024-04-24T23:20:56.724755+0000","last_became_active":"2024-04-24T23:20:55.480581+0000","last_became_peered":"2024-04-24T23:20:55.480581+0000","last_unstale":"2024-04-24T23:20:56.724755+0000","last_undegraded":"2024-04-24T23:20:56.724755+0000","last_fullsized":"2024-04-24T23:20:56.724755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725520+0000","last_change":"2024-04-24T23:20:55.488901+0000","last_active":"2024-04-24T23:20:56.725520+0000","last_peered":"2024-04-24T23:20:56.725520+0000","last_clean":"2024-04-24T23:20:56.725520+0000","last_became_active":"2024-04-24T23:20:55.488647+0000","last_became_peered":"2024-04-24T23:20:55.488647+0000","last_unstale":"2024-04-24T23:20:56.725520+0000","last_undegraded":"2024-04-24T23:20:56.725520+0000","last_fullsized":"2024-04-24T23:20:56.725520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480674+0000","last_change":"2024-04-24T23:20:55.476826+0000","last_active":"2024-04-24T23:20:56.480674+0000","last_peered":"2024-04-24T23:20:56.480674+0000","last_clean":"2024-04-24T23:20:56.480674+0000","last_became_active":"2024-04-24T23:20:55.476481+0000","last_became_peered":"2024-04-24T23:20:55.476481+0000","last_unstale":"2024-04-24T23:20:56.480674+0000","last_undegraded":"2024-04-24T23:20:56.480674+0000","last_fullsized":"2024-04-24T23:20:56.480674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724349+0000","last_change":"2024-04-24T23:20:55.491072+0000","last_active":"2024-04-24T23:20:56.724349+0000","last_peered":"2024-04-24T23:20:56.724349+0000","last_clean":"2024-04-24T23:20:56.724349+0000","last_became_active":"2024-04-24T23:20:55.490763+0000","last_became_peered":"2024-04-24T23:20:55.490763+0000","last_unstale":"2024-04-24T23:20:56.724349+0000","last_undegraded":"2024-04-24T23:20:56.724349+0000","last_fullsized":"2024-04-24T23:20:56.724349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475836+0000","last_change":"2024-04-24T23:20:55.487623+0000","last_active":"2024-04-24T23:20:56.475836+0000","last_peered":"2024-04-24T23:20:56.475836+0000","last_clean":"2024-04-24T23:20:56.475836+0000","last_became_active":"2024-04-24T23:20:55.487355+0000","last_became_peered":"2024-04-24T23:20:55.487355+0000","last_unstale":"2024-04-24T23:20:56.475836+0000","last_undegraded":"2024-04-24T23:20:56.475836+0000","last_fullsized":"2024-04-24T23:20:56.475836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725470+0000","last_change":"2024-04-24T23:20:55.476152+0000","last_active":"2024-04-24T23:20:56.725470+0000","last_peered":"2024-04-24T23:20:56.725470+0000","last_clean":"2024-04-24T23:20:56.725470+0000","last_became_active":"2024-04-24T23:20:55.475857+0000","last_became_peered":"2024-04-24T23:20:55.475857+0000","last_unstale":"2024-04-24T23:20:56.725470+0000","last_undegraded":"2024-04-24T23:20:56.725470+0000","last_fullsized":"2024-04-24T23:20:56.725470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475501+0000","last_change":"2024-04-24T23:20:55.484268+0000","last_active":"2024-04-24T23:20:56.475501+0000","last_peered":"2024-04-24T23:20:56.475501+0000","last_clean":"2024-04-24T23:20:56.475501+0000","last_became_active":"2024-04-24T23:20:55.483972+0000","last_became_peered":"2024-04-24T23:20:55.483972+0000","last_unstale":"2024-04-24T23:20:56.475501+0000","last_undegraded":"2024-04-24T23:20:56.475501+0000","last_fullsized":"2024-04-24T23:20:56.475501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725722+0000","last_change":"2024-04-24T23:20:55.480092+0000","last_active":"2024-04-24T23:20:56.725722+0000","last_peered":"2024-04-24T23:20:56.725722+0000","last_clean":"2024-04-24T23:20:56.725722+0000","last_became_active":"2024-04-24T23:20:55.479783+0000","last_became_peered":"2024-04-24T23:20:55.479783+0000","last_unstale":"2024-04-24T23:20:56.725722+0000","last_undegraded":"2024-04-24T23:20:56.725722+0000","last_fullsized":"2024-04-24T23:20:56.725722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480830+0000","last_change":"2024-04-24T23:20:55.488050+0000","last_active":"2024-04-24T23:20:56.480830+0000","last_peered":"2024-04-24T23:20:56.480830+0000","last_clean":"2024-04-24T23:20:56.480830+0000","last_became_active":"2024-04-24T23:20:55.487762+0000","last_became_peered":"2024-04-24T23:20:55.487762+0000","last_unstale":"2024-04-24T23:20:56.480830+0000","last_undegraded":"2024-04-24T23:20:56.480830+0000","last_fullsized":"2024-04-24T23:20:56.480830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478091+0000","last_change":"2024-04-24T23:20:55.478530+0000","last_active":"2024-04-24T23:20:56.478091+0000","last_peered":"2024-04-24T23:20:56.478091+0000","last_clean":"2024-04-24T23:20:56.478091+0000","last_became_active":"2024-04-24T23:20:55.478233+0000","last_became_peered":"2024-04-24T23:20:55.478233+0000","last_unstale":"2024-04-24T23:20:56.478091+0000","last_undegraded":"2024-04-24T23:20:56.478091+0000","last_fullsized":"2024-04-24T23:20:56.478091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725374+0000","last_change":"2024-04-24T23:20:55.480077+0000","last_active":"2024-04-24T23:20:56.725374+0000","last_peered":"2024-04-24T23:20:56.725374+0000","last_clean":"2024-04-24T23:20:56.725374+0000","last_became_active":"2024-04-24T23:20:55.479769+0000","last_became_peered":"2024-04-24T23:20:55.479769+0000","last_unstale":"2024-04-24T23:20:56.725374+0000","last_undegraded":"2024-04-24T23:20:56.725374+0000","last_fullsized":"2024-04-24T23:20:56.725374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724513+0000","last_change":"2024-04-24T23:20:55.481475+0000","last_active":"2024-04-24T23:20:56.724513+0000","last_peered":"2024-04-24T23:20:56.724513+0000","last_clean":"2024-04-24T23:20:56.724513+0000","last_became_active":"2024-04-24T23:20:55.480091+0000","last_became_peered":"2024-04-24T23:20:55.480091+0000","last_unstale":"2024-04-24T23:20:56.724513+0000","last_undegraded":"2024-04-24T23:20:56.724513+0000","last_fullsized":"2024-04-24T23:20:56.724513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476229+0000","last_change":"2024-04-24T23:20:55.485092+0000","last_active":"2024-04-24T23:20:56.476229+0000","last_peered":"2024-04-24T23:20:56.476229+0000","last_clean":"2024-04-24T23:20:56.476229+0000","last_became_active":"2024-04-24T23:20:55.484409+0000","last_became_peered":"2024-04-24T23:20:55.484409+0000","last_unstale":"2024-04-24T23:20:56.476229+0000","last_undegraded":"2024-04-24T23:20:56.476229+0000","last_fullsized":"2024-04-24T23:20:56.476229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.469291+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.468961+0000","last_became_peered":"2024-04-24T23:20:55.468961+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480450+0000","last_change":"2024-04-24T23:20:55.484826+0000","last_active":"2024-04-24T23:20:56.480450+0000","last_peered":"2024-04-24T23:20:56.480450+0000","last_clean":"2024-04-24T23:20:56.480450+0000","last_became_active":"2024-04-24T23:20:55.484500+0000","last_became_peered":"2024-04-24T23:20:55.484500+0000","last_unstale":"2024-04-24T23:20:56.480450+0000","last_undegraded":"2024-04-24T23:20:56.480450+0000","last_fullsized":"2024-04-24T23:20:56.480450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475480+0000","last_change":"2024-04-24T23:20:55.476006+0000","last_active":"2024-04-24T23:20:56.475480+0000","last_peered":"2024-04-24T23:20:56.475480+0000","last_clean":"2024-04-24T23:20:56.475480+0000","last_became_active":"2024-04-24T23:20:55.475090+0000","last_became_peered":"2024-04-24T23:20:55.475090+0000","last_unstale":"2024-04-24T23:20:56.475480+0000","last_undegraded":"2024-04-24T23:20:56.475480+0000","last_fullsized":"2024-04-24T23:20:56.475480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725560+0000","last_change":"2024-04-24T23:20:54.481863+0000","last_active":"2024-04-24T23:20:56.725560+0000","last_peered":"2024-04-24T23:20:56.725560+0000","last_clean":"2024-04-24T23:20:56.725560+0000","last_became_active":"2024-04-24T23:20:54.481498+0000","last_became_peered":"2024-04-24T23:20:54.481498+0000","last_unstale":"2024-04-24T23:20:56.725560+0000","last_undegraded":"2024-04-24T23:20:56.725560+0000","last_fullsized":"2024-04-24T23:20:56.725560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724565+0000","last_change":"2024-04-24T23:20:55.478651+0000","last_active":"2024-04-24T23:20:56.724565+0000","last_peered":"2024-04-24T23:20:56.724565+0000","last_clean":"2024-04-24T23:20:56.724565+0000","last_became_active":"2024-04-24T23:20:55.477783+0000","last_became_peered":"2024-04-24T23:20:55.477783+0000","last_unstale":"2024-04-24T23:20:56.724565+0000","last_undegraded":"2024-04-24T23:20:56.724565+0000","last_fullsized":"2024-04-24T23:20:56.724565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480400+0000","last_change":"2024-04-24T23:20:54.477560+0000","last_active":"2024-04-24T23:20:56.480400+0000","last_peered":"2024-04-24T23:20:56.480400+0000","last_clean":"2024-04-24T23:20:56.480400+0000","last_became_active":"2024-04-24T23:20:54.477261+0000","last_became_peered":"2024-04-24T23:20:54.477261+0000","last_unstale":"2024-04-24T23:20:56.480400+0000","last_undegraded":"2024-04-24T23:20:56.480400+0000","last_fullsized":"2024-04-24T23:20:56.480400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725154+0000","last_change":"2024-04-24T23:20:54.479540+0000","last_active":"2024-04-24T23:20:56.725154+0000","last_peered":"2024-04-24T23:20:56.725154+0000","last_clean":"2024-04-24T23:20:56.725154+0000","last_became_active":"2024-04-24T23:20:54.479206+0000","last_became_peered":"2024-04-24T23:20:54.479206+0000","last_unstale":"2024-04-24T23:20:56.725154+0000","last_undegraded":"2024-04-24T23:20:56.725154+0000","last_fullsized":"2024-04-24T23:20:56.725154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725091+0000","last_change":"2024-04-24T23:20:55.484927+0000","last_active":"2024-04-24T23:20:56.725091+0000","last_peered":"2024-04-24T23:20:56.725091+0000","last_clean":"2024-04-24T23:20:56.725091+0000","last_became_active":"2024-04-24T23:20:55.484611+0000","last_became_peered":"2024-04-24T23:20:55.484611+0000","last_unstale":"2024-04-24T23:20:56.725091+0000","last_undegraded":"2024-04-24T23:20:56.725091+0000","last_fullsized":"2024-04-24T23:20:56.725091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475910+0000","last_change":"2024-04-24T23:20:54.471358+0000","last_active":"2024-04-24T23:20:56.475910+0000","last_peered":"2024-04-24T23:20:56.475910+0000","last_clean":"2024-04-24T23:20:56.475910+0000","last_became_active":"2024-04-24T23:20:54.470800+0000","last_became_peered":"2024-04-24T23:20:54.470800+0000","last_unstale":"2024-04-24T23:20:56.475910+0000","last_undegraded":"2024-04-24T23:20:56.475910+0000","last_fullsized":"2024-04-24T23:20:56.475910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.477535+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.477215+0000","last_became_peered":"2024-04-24T23:20:55.477215+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480761+0000","last_change":"2024-04-24T23:20:54.479450+0000","last_active":"2024-04-24T23:20:56.480761+0000","last_peered":"2024-04-24T23:20:56.480761+0000","last_clean":"2024-04-24T23:20:56.480761+0000","last_became_active":"2024-04-24T23:20:54.479174+0000","last_became_peered":"2024-04-24T23:20:54.479174+0000","last_unstale":"2024-04-24T23:20:56.480761+0000","last_undegraded":"2024-04-24T23:20:56.480761+0000","last_fullsized":"2024-04-24T23:20:56.480761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476156+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.476156+0000","last_peered":"2024-04-24T23:20:56.476156+0000","last_clean":"2024-04-24T23:20:56.476156+0000","last_became_active":"2024-04-24T23:20:55.488014+0000","last_became_peered":"2024-04-24T23:20:55.488014+0000","last_unstale":"2024-04-24T23:20:56.476156+0000","last_undegraded":"2024-04-24T23:20:56.476156+0000","last_fullsized":"2024-04-24T23:20:56.476156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476129+0000","last_change":"2024-04-24T23:20:54.480387+0000","last_active":"2024-04-24T23:20:56.476129+0000","last_peered":"2024-04-24T23:20:56.476129+0000","last_clean":"2024-04-24T23:20:56.476129+0000","last_became_active":"2024-04-24T23:20:54.480037+0000","last_became_peered":"2024-04-24T23:20:54.480037+0000","last_unstale":"2024-04-24T23:20:56.476129+0000","last_undegraded":"2024-04-24T23:20:56.476129+0000","last_fullsized":"2024-04-24T23:20:56.476129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724464+0000","last_change":"2024-04-24T23:20:55.481206+0000","last_active":"2024-04-24T23:20:56.724464+0000","last_peered":"2024-04-24T23:20:56.724464+0000","last_clean":"2024-04-24T23:20:56.724464+0000","last_became_active":"2024-04-24T23:20:55.478018+0000","last_became_peered":"2024-04-24T23:20:55.478018+0000","last_unstale":"2024-04-24T23:20:56.724464+0000","last_undegraded":"2024-04-24T23:20:56.724464+0000","last_fullsized":"2024-04-24T23:20:56.724464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478112+0000","last_change":"2024-04-24T23:20:54.469557+0000","last_active":"2024-04-24T23:20:56.478112+0000","last_peered":"2024-04-24T23:20:56.478112+0000","last_clean":"2024-04-24T23:20:56.478112+0000","last_became_active":"2024-04-24T23:20:54.469251+0000","last_became_peered":"2024-04-24T23:20:54.469251+0000","last_unstale":"2024-04-24T23:20:56.478112+0000","last_undegraded":"2024-04-24T23:20:56.478112+0000","last_fullsized":"2024-04-24T23:20:56.478112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475184+0000","last_change":"2024-04-24T23:20:55.491797+0000","last_active":"2024-04-24T23:20:56.475184+0000","last_peered":"2024-04-24T23:20:56.475184+0000","last_clean":"2024-04-24T23:20:56.475184+0000","last_became_active":"2024-04-24T23:20:55.491453+0000","last_became_peered":"2024-04-24T23:20:55.491453+0000","last_unstale":"2024-04-24T23:20:56.475184+0000","last_undegraded":"2024-04-24T23:20:56.475184+0000","last_fullsized":"2024-04-24T23:20:56.475184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:05.444694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480475+0000","last_change":"2024-04-24T23:20:54.482472+0000","last_active":"2024-04-24T23:20:56.480475+0000","last_peered":"2024-04-24T23:20:56.480475+0000","last_clean":"2024-04-24T23:20:56.480475+0000","last_became_active":"2024-04-24T23:20:54.481351+0000","last_became_peered":"2024-04-24T23:20:54.481351+0000","last_unstale":"2024-04-24T23:20:56.480475+0000","last_undegraded":"2024-04-24T23:20:56.480475+0000","last_fullsized":"2024-04-24T23:20:56.480475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725210+0000","last_change":"2024-04-24T23:20:55.483153+0000","last_active":"2024-04-24T23:20:56.725210+0000","last_peered":"2024-04-24T23:20:56.725210+0000","last_clean":"2024-04-24T23:20:56.725210+0000","last_became_active":"2024-04-24T23:20:55.482859+0000","last_became_peered":"2024-04-24T23:20:55.482859+0000","last_unstale":"2024-04-24T23:20:56.725210+0000","last_undegraded":"2024-04-24T23:20:56.725210+0000","last_fullsized":"2024-04-24T23:20:56.725210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724624+0000","last_change":"2024-04-24T23:20:54.478089+0000","last_active":"2024-04-24T23:20:56.724624+0000","last_peered":"2024-04-24T23:20:56.724624+0000","last_clean":"2024-04-24T23:20:56.724624+0000","last_became_active":"2024-04-24T23:20:54.477302+0000","last_became_peered":"2024-04-24T23:20:54.477302+0000","last_unstale":"2024-04-24T23:20:56.724624+0000","last_undegraded":"2024-04-24T23:20:56.724624+0000","last_fullsized":"2024-04-24T23:20:56.724624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724289+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.724289+0000","last_peered":"2024-04-24T23:20:56.724289+0000","last_clean":"2024-04-24T23:20:56.724289+0000","last_became_active":"2024-04-24T23:20:55.488016+0000","last_became_peered":"2024-04-24T23:20:55.488016+0000","last_unstale":"2024-04-24T23:20:56.724289+0000","last_undegraded":"2024-04-24T23:20:56.724289+0000","last_fullsized":"2024-04-24T23:20:56.724289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:48.795181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476045+0000","last_change":"2024-04-24T23:20:54.473922+0000","last_active":"2024-04-24T23:20:56.476045+0000","last_peered":"2024-04-24T23:20:56.476045+0000","last_clean":"2024-04-24T23:20:56.476045+0000","last_became_active":"2024-04-24T23:20:54.473601+0000","last_became_peered":"2024-04-24T23:20:54.473601+0000","last_unstale":"2024-04-24T23:20:56.476045+0000","last_undegraded":"2024-04-24T23:20:56.476045+0000","last_fullsized":"2024-04-24T23:20:56.476045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725128+0000","last_change":"2024-04-24T23:20:55.488225+0000","last_active":"2024-04-24T23:20:56.725128+0000","last_peered":"2024-04-24T23:20:56.725128+0000","last_clean":"2024-04-24T23:20:56.725128+0000","last_became_active":"2024-04-24T23:20:55.485663+0000","last_became_peered":"2024-04-24T23:20:55.485663+0000","last_unstale":"2024-04-24T23:20:56.725128+0000","last_undegraded":"2024-04-24T23:20:56.725128+0000","last_fullsized":"2024-04-24T23:20:56.725128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:54.478237+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:54.477943+0000","last_became_peered":"2024-04-24T23:20:54.477943+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:55.484884+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:55.484578+0000","last_became_peered":"2024-04-24T23:20:55.484578+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724810+0000","last_change":"2024-04-24T23:20:54.473033+0000","last_active":"2024-04-24T23:20:56.724810+0000","last_peered":"2024-04-24T23:20:56.724810+0000","last_clean":"2024-04-24T23:20:56.724810+0000","last_became_active":"2024-04-24T23:20:54.472748+0000","last_became_peered":"2024-04-24T23:20:54.472748+0000","last_unstale":"2024-04-24T23:20:56.724810+0000","last_undegraded":"2024-04-24T23:20:56.724810+0000","last_fullsized":"2024-04-24T23:20:56.724810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724977+0000","last_change":"2024-04-24T23:20:55.485153+0000","last_active":"2024-04-24T23:20:56.724977+0000","last_peered":"2024-04-24T23:20:56.724977+0000","last_clean":"2024-04-24T23:20:56.724977+0000","last_became_active":"2024-04-24T23:20:55.484876+0000","last_became_peered":"2024-04-24T23:20:55.484876+0000","last_unstale":"2024-04-24T23:20:56.724977+0000","last_undegraded":"2024-04-24T23:20:56.724977+0000","last_fullsized":"2024-04-24T23:20:56.724977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474721+0000","last_change":"2024-04-24T23:20:54.476273+0000","last_active":"2024-04-24T23:20:56.474721+0000","last_peered":"2024-04-24T23:20:56.474721+0000","last_clean":"2024-04-24T23:20:56.474721+0000","last_became_active":"2024-04-24T23:20:54.475993+0000","last_became_peered":"2024-04-24T23:20:54.475993+0000","last_unstale":"2024-04-24T23:20:56.474721+0000","last_undegraded":"2024-04-24T23:20:56.474721+0000","last_fullsized":"2024-04-24T23:20:56.474721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725412+0000","last_change":"2024-04-24T23:20:55.487659+0000","last_active":"2024-04-24T23:20:56.725412+0000","last_peered":"2024-04-24T23:20:56.725412+0000","last_clean":"2024-04-24T23:20:56.725412+0000","last_became_active":"2024-04-24T23:20:55.486557+0000","last_became_peered":"2024-04-24T23:20:55.486557+0000","last_unstale":"2024-04-24T23:20:56.725412+0000","last_undegraded":"2024-04-24T23:20:56.725412+0000","last_fullsized":"2024-04-24T23:20:56.725412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724872+0000","last_change":"2024-04-24T23:20:54.477334+0000","last_active":"2024-04-24T23:20:56.724872+0000","last_peered":"2024-04-24T23:20:56.724872+0000","last_clean":"2024-04-24T23:20:56.724872+0000","last_became_active":"2024-04-24T23:20:54.477058+0000","last_became_peered":"2024-04-24T23:20:54.477058+0000","last_unstale":"2024-04-24T23:20:56.724872+0000","last_undegraded":"2024-04-24T23:20:56.724872+0000","last_fullsized":"2024-04-24T23:20:56.724872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725667+0000","last_change":"2024-04-24T23:20:55.492754+0000","last_active":"2024-04-24T23:20:56.725667+0000","last_peered":"2024-04-24T23:20:56.725667+0000","last_clean":"2024-04-24T23:20:56.725667+0000","last_became_active":"2024-04-24T23:20:55.492474+0000","last_became_peered":"2024-04-24T23:20:55.492474+0000","last_unstale":"2024-04-24T23:20:56.725667+0000","last_undegraded":"2024-04-24T23:20:56.725667+0000","last_fullsized":"2024-04-24T23:20:56.725667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:54.766130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478192+0000","last_change":"2024-04-24T23:20:54.471091+0000","last_active":"2024-04-24T23:20:56.478192+0000","last_peered":"2024-04-24T23:20:56.478192+0000","last_clean":"2024-04-24T23:20:56.478192+0000","last_became_active":"2024-04-24T23:20:54.470811+0000","last_became_peered":"2024-04-24T23:20:54.470811+0000","last_unstale":"2024-04-24T23:20:56.478192+0000","last_undegraded":"2024-04-24T23:20:56.478192+0000","last_fullsized":"2024-04-24T23:20:56.478192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476216+0000","last_change":"2024-04-24T23:20:55.480640+0000","last_active":"2024-04-24T23:20:56.476216+0000","last_peered":"2024-04-24T23:20:56.476216+0000","last_clean":"2024-04-24T23:20:56.476216+0000","last_became_active":"2024-04-24T23:20:55.480327+0000","last_became_peered":"2024-04-24T23:20:55.480327+0000","last_unstale":"2024-04-24T23:20:56.476216+0000","last_undegraded":"2024-04-24T23:20:56.476216+0000","last_fullsized":"2024-04-24T23:20:56.476216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725414+0000","last_change":"2024-04-24T23:20:54.476392+0000","last_active":"2024-04-24T23:20:56.725414+0000","last_peered":"2024-04-24T23:20:56.725414+0000","last_clean":"2024-04-24T23:20:56.725414+0000","last_became_active":"2024-04-24T23:20:54.476051+0000","last_became_peered":"2024-04-24T23:20:54.476051+0000","last_unstale":"2024-04-24T23:20:56.725414+0000","last_undegraded":"2024-04-24T23:20:56.725414+0000","last_fullsized":"2024-04-24T23:20:56.725414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725389+0000","last_change":"2024-04-24T23:20:55.483909+0000","last_active":"2024-04-24T23:20:56.725389+0000","last_peered":"2024-04-24T23:20:56.725389+0000","last_clean":"2024-04-24T23:20:56.725389+0000","last_became_active":"2024-04-24T23:20:55.483625+0000","last_became_peered":"2024-04-24T23:20:55.483625+0000","last_unstale":"2024-04-24T23:20:56.725389+0000","last_undegraded":"2024-04-24T23:20:56.725389+0000","last_fullsized":"2024-04-24T23:20:56.725389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724832+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:20:56.724832+0000","last_peered":"2024-04-24T23:20:56.724832+0000","last_clean":"2024-04-24T23:20:56.724832+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:20:56.724832+0000","last_undegraded":"2024-04-24T23:20:56.724832+0000","last_fullsized":"2024-04-24T23:20:56.724832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474912+0000","last_change":"2024-04-24T23:20:54.470392+0000","last_active":"2024-04-24T23:20:56.474912+0000","last_peered":"2024-04-24T23:20:56.474912+0000","last_clean":"2024-04-24T23:20:56.474912+0000","last_became_active":"2024-04-24T23:20:54.470095+0000","last_became_peered":"2024-04-24T23:20:54.470095+0000","last_unstale":"2024-04-24T23:20:56.474912+0000","last_undegraded":"2024-04-24T23:20:56.474912+0000","last_fullsized":"2024-04-24T23:20:56.474912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725394+0000","last_change":"2024-04-24T23:20:55.485347+0000","last_active":"2024-04-24T23:20:56.725394+0000","last_peered":"2024-04-24T23:20:56.725394+0000","last_clean":"2024-04-24T23:20:56.725394+0000","last_became_active":"2024-04-24T23:20:55.485004+0000","last_became_peered":"2024-04-24T23:20:55.485004+0000","last_unstale":"2024-04-24T23:20:56.725394+0000","last_undegraded":"2024-04-24T23:20:56.725394+0000","last_fullsized":"2024-04-24T23:20:56.725394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724782+0000","last_change":"2024-04-24T23:20:54.482240+0000","last_active":"2024-04-24T23:20:56.724782+0000","last_peered":"2024-04-24T23:20:56.724782+0000","last_clean":"2024-04-24T23:20:56.724782+0000","last_became_active":"2024-04-24T23:20:54.481934+0000","last_became_peered":"2024-04-24T23:20:54.481934+0000","last_unstale":"2024-04-24T23:20:56.724782+0000","last_undegraded":"2024-04-24T23:20:56.724782+0000","last_fullsized":"2024-04-24T23:20:56.724782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:13.029046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724293+0000","last_change":"2024-04-24T23:20:55.474530+0000","last_active":"2024-04-24T23:20:56.724293+0000","last_peered":"2024-04-24T23:20:56.724293+0000","last_clean":"2024-04-24T23:20:56.724293+0000","last_became_active":"2024-04-24T23:20:55.474181+0000","last_became_peered":"2024-04-24T23:20:55.474181+0000","last_unstale":"2024-04-24T23:20:56.724293+0000","last_undegraded":"2024-04-24T23:20:56.724293+0000","last_fullsized":"2024-04-24T23:20:56.724293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:43.728916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724886+0000","last_change":"2024-04-24T23:20:54.478684+0000","last_active":"2024-04-24T23:20:56.724886+0000","last_peered":"2024-04-24T23:20:56.724886+0000","last_clean":"2024-04-24T23:20:56.724886+0000","last_became_active":"2024-04-24T23:20:54.478377+0000","last_became_peered":"2024-04-24T23:20:54.478377+0000","last_unstale":"2024-04-24T23:20:56.724886+0000","last_undegraded":"2024-04-24T23:20:56.724886+0000","last_fullsized":"2024-04-24T23:20:56.724886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.478546+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.478253+0000","last_became_peered":"2024-04-24T23:20:55.478253+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476161+0000","last_change":"2024-04-24T23:20:54.478753+0000","last_active":"2024-04-24T23:20:56.476161+0000","last_peered":"2024-04-24T23:20:56.476161+0000","last_clean":"2024-04-24T23:20:56.476161+0000","last_became_active":"2024-04-24T23:20:54.478441+0000","last_became_peered":"2024-04-24T23:20:54.478441+0000","last_unstale":"2024-04-24T23:20:56.476161+0000","last_undegraded":"2024-04-24T23:20:56.476161+0000","last_fullsized":"2024-04-24T23:20:56.476161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475739+0000","last_change":"2024-04-24T23:20:55.483158+0000","last_active":"2024-04-24T23:20:56.475739+0000","last_peered":"2024-04-24T23:20:56.475739+0000","last_clean":"2024-04-24T23:20:56.475739+0000","last_became_active":"2024-04-24T23:20:55.482865+0000","last_became_peered":"2024-04-24T23:20:55.482865+0000","last_unstale":"2024-04-24T23:20:56.475739+0000","last_undegraded":"2024-04-24T23:20:56.475739+0000","last_fullsized":"2024-04-24T23:20:56.475739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477991+0000","last_change":"2024-04-24T23:20:54.476881+0000","last_active":"2024-04-24T23:20:56.477991+0000","last_peered":"2024-04-24T23:20:56.477991+0000","last_clean":"2024-04-24T23:20:56.477991+0000","last_became_active":"2024-04-24T23:20:54.476588+0000","last_became_peered":"2024-04-24T23:20:54.476588+0000","last_unstale":"2024-04-24T23:20:56.477991+0000","last_undegraded":"2024-04-24T23:20:56.477991+0000","last_fullsized":"2024-04-24T23:20:56.477991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475322+0000","last_change":"2024-04-24T23:20:55.482626+0000","last_active":"2024-04-24T23:20:56.475322+0000","last_peered":"2024-04-24T23:20:56.475322+0000","last_clean":"2024-04-24T23:20:56.475322+0000","last_became_active":"2024-04-24T23:20:55.482266+0000","last_became_peered":"2024-04-24T23:20:55.482266+0000","last_unstale":"2024-04-24T23:20:56.475322+0000","last_undegraded":"2024-04-24T23:20:56.475322+0000","last_fullsized":"2024-04-24T23:20:56.475322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725524+0000","last_change":"2024-04-24T23:20:54.484817+0000","last_active":"2024-04-24T23:20:56.725524+0000","last_peered":"2024-04-24T23:20:56.725524+0000","last_clean":"2024-04-24T23:20:56.725524+0000","last_became_active":"2024-04-24T23:20:54.484553+0000","last_became_peered":"2024-04-24T23:20:54.484553+0000","last_unstale":"2024-04-24T23:20:56.725524+0000","last_undegraded":"2024-04-24T23:20:56.725524+0000","last_fullsized":"2024-04-24T23:20:56.725524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:42.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478041+0000","last_change":"2024-04-24T23:20:55.479475+0000","last_active":"2024-04-24T23:20:56.478041+0000","last_peered":"2024-04-24T23:20:56.478041+0000","last_clean":"2024-04-24T23:20:56.478041+0000","last_became_active":"2024-04-24T23:20:55.478984+0000","last_became_peered":"2024-04-24T23:20:55.478984+0000","last_unstale":"2024-04-24T23:20:56.478041+0000","last_undegraded":"2024-04-24T23:20:56.478041+0000","last_fullsized":"2024-04-24T23:20:56.478041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.479792+0000","last_change":"2024-04-24T23:20:54.478307+0000","last_active":"2024-04-24T23:20:56.479792+0000","last_peered":"2024-04-24T23:20:56.479792+0000","last_clean":"2024-04-24T23:20:56.479792+0000","last_became_active":"2024-04-24T23:20:54.477988+0000","last_became_peered":"2024-04-24T23:20:54.477988+0000","last_unstale":"2024-04-24T23:20:56.479792+0000","last_undegraded":"2024-04-24T23:20:56.479792+0000","last_fullsized":"2024-04-24T23:20:56.479792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725717+0000","last_change":"2024-04-24T23:20:55.487879+0000","last_active":"2024-04-24T23:20:56.725717+0000","last_peered":"2024-04-24T23:20:56.725717+0000","last_clean":"2024-04-24T23:20:56.725717+0000","last_became_active":"2024-04-24T23:20:55.487553+0000","last_became_peered":"2024-04-24T23:20:55.487553+0000","last_unstale":"2024-04-24T23:20:56.725717+0000","last_undegraded":"2024-04-24T23:20:56.725717+0000","last_fullsized":"2024-04-24T23:20:56.725717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480845+0000","last_change":"2024-04-24T23:20:54.480826+0000","last_active":"2024-04-24T23:20:56.480845+0000","last_peered":"2024-04-24T23:20:56.480845+0000","last_clean":"2024-04-24T23:20:56.480845+0000","last_became_active":"2024-04-24T23:20:54.480351+0000","last_became_peered":"2024-04-24T23:20:54.480351+0000","last_unstale":"2024-04-24T23:20:56.480845+0000","last_undegraded":"2024-04-24T23:20:56.480845+0000","last_fullsized":"2024-04-24T23:20:56.480845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.480939+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.478888+0000","last_became_peered":"2024-04-24T23:20:55.478888+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724748+0000","last_change":"2024-04-24T23:20:54.482329+0000","last_active":"2024-04-24T23:20:56.724748+0000","last_peered":"2024-04-24T23:20:56.724748+0000","last_clean":"2024-04-24T23:20:56.724748+0000","last_became_active":"2024-04-24T23:20:54.482059+0000","last_became_peered":"2024-04-24T23:20:54.482059+0000","last_unstale":"2024-04-24T23:20:56.724748+0000","last_undegraded":"2024-04-24T23:20:56.724748+0000","last_fullsized":"2024-04-24T23:20:56.724748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475091+0000","last_change":"2024-04-24T23:20:55.478742+0000","last_active":"2024-04-24T23:20:56.475091+0000","last_peered":"2024-04-24T23:20:56.475091+0000","last_clean":"2024-04-24T23:20:56.475091+0000","last_became_active":"2024-04-24T23:20:55.478465+0000","last_became_peered":"2024-04-24T23:20:55.478465+0000","last_unstale":"2024-04-24T23:20:56.475091+0000","last_undegraded":"2024-04-24T23:20:56.475091+0000","last_fullsized":"2024-04-24T23:20:56.475091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:32.641502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477976+0000","last_change":"2024-04-24T23:20:54.480255+0000","last_active":"2024-04-24T23:20:56.477976+0000","last_peered":"2024-04-24T23:20:56.477976+0000","last_clean":"2024-04-24T23:20:56.477976+0000","last_became_active":"2024-04-24T23:20:54.479897+0000","last_became_peered":"2024-04-24T23:20:54.479897+0000","last_unstale":"2024-04-24T23:20:56.477976+0000","last_undegraded":"2024-04-24T23:20:56.477976+0000","last_fullsized":"2024-04-24T23:20:56.477976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725278+0000","last_change":"2024-04-24T23:20:55.480538+0000","last_active":"2024-04-24T23:20:56.725278+0000","last_peered":"2024-04-24T23:20:56.725278+0000","last_clean":"2024-04-24T23:20:56.725278+0000","last_became_active":"2024-04-24T23:20:55.474763+0000","last_became_peered":"2024-04-24T23:20:55.474763+0000","last_unstale":"2024-04-24T23:20:56.725278+0000","last_undegraded":"2024-04-24T23:20:56.725278+0000","last_fullsized":"2024-04-24T23:20:56.725278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477945+0000","last_change":"2024-04-24T23:20:54.475154+0000","last_active":"2024-04-24T23:20:56.477945+0000","last_peered":"2024-04-24T23:20:56.477945+0000","last_clean":"2024-04-24T23:20:56.477945+0000","last_became_active":"2024-04-24T23:20:54.473925+0000","last_became_peered":"2024-04-24T23:20:54.473925+0000","last_unstale":"2024-04-24T23:20:56.477945+0000","last_undegraded":"2024-04-24T23:20:56.477945+0000","last_fullsized":"2024-04-24T23:20:56.477945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725566+0000","last_change":"2024-04-24T23:20:55.479171+0000","last_active":"2024-04-24T23:20:56.725566+0000","last_peered":"2024-04-24T23:20:56.725566+0000","last_clean":"2024-04-24T23:20:56.725566+0000","last_became_active":"2024-04-24T23:20:55.478856+0000","last_became_peered":"2024-04-24T23:20:55.478856+0000","last_unstale":"2024-04-24T23:20:56.725566+0000","last_undegraded":"2024-04-24T23:20:56.725566+0000","last_fullsized":"2024-04-24T23:20:56.725566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725243+0000","last_change":"2024-04-24T23:20:54.484143+0000","last_active":"2024-04-24T23:20:56.725243+0000","last_peered":"2024-04-24T23:20:56.725243+0000","last_clean":"2024-04-24T23:20:56.725243+0000","last_became_active":"2024-04-24T23:20:54.483830+0000","last_became_peered":"2024-04-24T23:20:54.483830+0000","last_unstale":"2024-04-24T23:20:56.725243+0000","last_undegraded":"2024-04-24T23:20:56.725243+0000","last_fullsized":"2024-04-24T23:20:56.725243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480446+0000","last_change":"2024-04-24T23:20:55.479005+0000","last_active":"2024-04-24T23:20:56.480446+0000","last_peered":"2024-04-24T23:20:56.480446+0000","last_clean":"2024-04-24T23:20:56.480446+0000","last_became_active":"2024-04-24T23:20:55.478706+0000","last_became_peered":"2024-04-24T23:20:55.478706+0000","last_unstale":"2024-04-24T23:20:56.480446+0000","last_undegraded":"2024-04-24T23:20:56.480446+0000","last_fullsized":"2024-04-24T23:20:56.480446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474978+0000","last_change":"2024-04-24T23:20:55.469771+0000","last_active":"2024-04-24T23:20:56.474978+0000","last_peered":"2024-04-24T23:20:56.474978+0000","last_clean":"2024-04-24T23:20:56.474978+0000","last_became_active":"2024-04-24T23:20:55.469411+0000","last_became_peered":"2024-04-24T23:20:55.469411+0000","last_unstale":"2024-04-24T23:20:56.474978+0000","last_undegraded":"2024-04-24T23:20:56.474978+0000","last_fullsized":"2024-04-24T23:20:56.474978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725118+0000","last_change":"2024-04-24T23:20:54.485231+0000","last_active":"2024-04-24T23:20:56.725118+0000","last_peered":"2024-04-24T23:20:56.725118+0000","last_clean":"2024-04-24T23:20:56.725118+0000","last_became_active":"2024-04-24T23:20:54.484912+0000","last_became_peered":"2024-04-24T23:20:54.484912+0000","last_unstale":"2024-04-24T23:20:56.725118+0000","last_undegraded":"2024-04-24T23:20:56.725118+0000","last_fullsized":"2024-04-24T23:20:56.725118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:27.147159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476107+0000","last_change":"2024-04-24T23:20:55.473157+0000","last_active":"2024-04-24T23:20:56.476107+0000","last_peered":"2024-04-24T23:20:56.476107+0000","last_clean":"2024-04-24T23:20:56.476107+0000","last_became_active":"2024-04-24T23:20:55.472854+0000","last_became_peered":"2024-04-24T23:20:55.472854+0000","last_unstale":"2024-04-24T23:20:56.476107+0000","last_undegraded":"2024-04-24T23:20:56.476107+0000","last_fullsized":"2024-04-24T23:20:56.476107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725345+0000","last_change":"2024-04-24T23:20:54.485353+0000","last_active":"2024-04-24T23:20:56.725345+0000","last_peered":"2024-04-24T23:20:56.725345+0000","last_clean":"2024-04-24T23:20:56.725345+0000","last_became_active":"2024-04-24T23:20:54.485065+0000","last_became_peered":"2024-04-24T23:20:54.485065+0000","last_unstale":"2024-04-24T23:20:56.725345+0000","last_undegraded":"2024-04-24T23:20:56.725345+0000","last_fullsized":"2024-04-24T23:20:56.725345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724430+0000","last_change":"2024-04-24T23:20:55.487656+0000","last_active":"2024-04-24T23:20:56.724430+0000","last_peered":"2024-04-24T23:20:56.724430+0000","last_clean":"2024-04-24T23:20:56.724430+0000","last_became_active":"2024-04-24T23:20:55.487317+0000","last_became_peered":"2024-04-24T23:20:55.487317+0000","last_unstale":"2024-04-24T23:20:56.724430+0000","last_undegraded":"2024-04-24T23:20:56.724430+0000","last_fullsized":"2024-04-24T23:20:56.724430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725671+0000","last_change":"2024-04-24T23:20:54.475528+0000","last_active":"2024-04-24T23:20:56.725671+0000","last_peered":"2024-04-24T23:20:56.725671+0000","last_clean":"2024-04-24T23:20:56.725671+0000","last_became_active":"2024-04-24T23:20:54.475227+0000","last_became_peered":"2024-04-24T23:20:54.475227+0000","last_unstale":"2024-04-24T23:20:56.725671+0000","last_undegraded":"2024-04-24T23:20:56.725671+0000","last_fullsized":"2024-04-24T23:20:56.725671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.481689+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.481405+0000","last_became_peered":"2024-04-24T23:20:55.481405+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724580+0000","last_change":"2024-04-24T23:20:54.482023+0000","last_active":"2024-04-24T23:20:56.724580+0000","last_peered":"2024-04-24T23:20:56.724580+0000","last_clean":"2024-04-24T23:20:56.724580+0000","last_became_active":"2024-04-24T23:20:54.481494+0000","last_became_peered":"2024-04-24T23:20:54.481494+0000","last_unstale":"2024-04-24T23:20:56.724580+0000","last_undegraded":"2024-04-24T23:20:56.724580+0000","last_fullsized":"2024-04-24T23:20:56.724580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:21:05.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:21:06.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.131+0000 7f236f9ee640 1 Processor -- start 2024-04-24T23:21:06.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.131+0000 7f236f9ee640 1 -- start start 2024-04-24T23:21:06.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236f9ee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f23680b3ac0 0x7f23680b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236f9ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23680b62e0 0x7f23680b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236f9ee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23680b5840 con 0x7f236815e290 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236f9ee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f23680b5e50 con 0x7f23680b62e0 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236f9ee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f23680cfb30 con 0x7f23680b3ac0 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236df8b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23680b62e0 0x7f23680b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236df8b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23680b62e0 0x7f23680b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40110/0 (socket says 172.21.15.26:40110) 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236df8b640 1 -- 172.21.15.26:0/1822637478 learned_addr learned my addr 172.21.15.26:0/1822637478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236cf89640 1 --2- 172.21.15.26:0/1822637478 >> v2:172.21.15.29:3300/0 conn(0x7f23680b3ac0 0x7f23680b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236df8b640 1 -- 172.21.15.26:0/1822637478 >> v2:172.21.15.29:3300/0 conn(0x7f23680b3ac0 msgr2=0x7f23680b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236df8b640 1 --2- 172.21.15.26:0/1822637478 >> v2:172.21.15.29:3300/0 conn(0x7f23680b3ac0 0x7f23680b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236df8b640 1 -- 172.21.15.26:0/1822637478 >> v1:172.21.15.29:6789/0 conn(0x7f236815e290 legacy=0x7f23680d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:06.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.132+0000 7f236df8b640 1 -- 172.21.15.26:0/1822637478 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23680cfe10 con 0x7f23680b62e0 2024-04-24T23:21:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.133+0000 7f236df8b640 1 --2- 172.21.15.26:0/1822637478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 0x7f23680b6700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f23680c5a30 tx=0x7f236002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915111357d3fba91 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.133+0000 7f235ffff640 1 -- 172.21.15.26:0/1822637478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360036070 con 0x7f23680b62e0 2024-04-24T23:21:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.133+0000 7f235ffff640 1 -- 172.21.15.26:0/1822637478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2360031600 con 0x7f23680b62e0 2024-04-24T23:21:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.133+0000 7f235ffff640 1 -- 172.21.15.26:0/1822637478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23600319d0 con 0x7f23680b62e0 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.134+0000 7f236f9ee640 1 -- 172.21.15.26:0/1822637478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 msgr2=0x7f23680b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.134+0000 7f236f9ee640 1 --2- 172.21.15.26:0/1822637478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 0x7f23680b6700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f23680c5a30 tx=0x7f236002a6c0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.134+0000 7f236f9ee640 1 -- 172.21.15.26:0/1822637478 shutdown_connections 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.134+0000 7f236f9ee640 1 --2- 172.21.15.26:0/1822637478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 0x7f23680b6700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.134+0000 7f236f9ee640 1 --2- 172.21.15.26:0/1822637478 >> v2:172.21.15.29:3300/0 conn(0x7f23680b3ac0 0x7f23680b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.134+0000 7f236f9ee640 1 -- 172.21.15.26:0/1822637478 >> 172.21.15.26:0/1822637478 conn(0x7f236806b120 msgr2=0x7f236806b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.134+0000 7f236f9ee640 1 -- 172.21.15.26:0/1822637478 shutdown_connections 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.134+0000 7f236f9ee640 1 -- 172.21.15.26:0/1822637478 wait complete. 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.135+0000 7f236f9ee640 1 Processor -- start 2024-04-24T23:21:06.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.135+0000 7f236f9ee640 1 -- start start 2024-04-24T23:21:06.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236f9ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f23680b3ac0 0x7f23680c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236f9ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 0x7f2368152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236f9ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f236815e290 0x7f23680c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236f9ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f236815e670 con 0x7f23680b62e0 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236f9ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f23680b5ae0 con 0x7f236815e290 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236f9ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f236806dfd0 con 0x7f23680b3ac0 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236cf89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 0x7f2368152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236cf89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 0x7f2368152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40120/0 (socket says 172.21.15.26:40120) 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236cf89640 1 -- 172.21.15.26:0/494905378 learned_addr learned my addr 172.21.15.26:0/494905378 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236df8b640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f236815e290 0x7f23680c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236d78a640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f23680b3ac0 0x7f23680c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236cf89640 1 -- 172.21.15.26:0/494905378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f23680b3ac0 msgr2=0x7f23680c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236cf89640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f23680b3ac0 0x7f23680c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236cf89640 1 -- 172.21.15.26:0/494905378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f236815e290 msgr2=0x7f23680c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236cf89640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f236815e290 0x7f23680c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236cf89640 1 -- 172.21.15.26:0/494905378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23680c9570 con 0x7f23680b62e0 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236cf89640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 0x7f2368152370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f235800d800 tx=0x7f235800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f235dffb640 1 -- 172.21.15.26:0/494905378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235800b6c0 con 0x7f23680b62e0 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f235dffb640 1 -- 172.21.15.26:0/494905378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2358019070 con 0x7f23680b62e0 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f235dffb640 1 -- 172.21.15.26:0/494905378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235800ba80 con 0x7f23680b62e0 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.136+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23681618c0 con 0x7f23680b62e0 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.137+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23680b6ea0 con 0x7f23680b62e0 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.138+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23680b69b0 con 0x7f23680b62e0 2024-04-24T23:21:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.138+0000 7f235dffb640 1 -- 172.21.15.26:0/494905378 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f235800bc20 con 0x7f23680b62e0 2024-04-24T23:21:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.138+0000 7f235dffb640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f234c06be30 0x7f234c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.138+0000 7f236d78a640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f234c06be30 0x7f234c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.138+0000 7f235dffb640 1 -- 172.21.15.26:0/494905378 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f235808b2d0 con 0x7f23680b62e0 2024-04-24T23:21:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.139+0000 7f236d78a640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f234c06be30 0x7f234c06e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2364006fd0 tx=0x7f2364008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:06.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.143+0000 7f235dffb640 1 -- 172.21.15.26:0/494905378 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2358059c80 con 0x7f23680b62e0 2024-04-24T23:21:06.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.321+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23680694c0 con 0x7f234c06be30 2024-04-24T23:21:06.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.327+0000 7f235dffb640 1 -- 172.21.15.26:0/494905378 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f23680694c0 con 0x7f234c06be30 2024-04-24T23:21:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:06.333 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:21:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f234c06be30 msgr2=0x7f234c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f234c06be30 0x7f234c06e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2364006fd0 tx=0x7f2364008040 comp rx=0 tx=0).stop 2024-04-24T23:21:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 msgr2=0x7f2368152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 0x7f2368152370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f235800d800 tx=0x7f235800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 shutdown_connections 2024-04-24T23:21:06.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f234c06be30 0x7f234c06e2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f236815e290 0x7f23680c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23680b62e0 0x7f2368152370 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 --2- 172.21.15.26:0/494905378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f23680b3ac0 0x7f23680c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 >> 172.21.15.26:0/494905378 conn(0x7f236806b120 msgr2=0x7f236806d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:06.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 shutdown_connections 2024-04-24T23:21:06.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.330+0000 7f236f9ee640 1 -- 172.21.15.26:0/494905378 wait complete. 2024-04-24T23:21:06.343 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-24T23:21:05.317422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256448,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718272,"statfs":{"total":773094113280,"available":772831510528,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.856172"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725536+0000","last_change":"2024-04-24T23:20:55.487607+0000","last_active":"2024-04-24T23:20:56.725536+0000","last_peered":"2024-04-24T23:20:56.725536+0000","last_clean":"2024-04-24T23:20:56.725536+0000","last_became_active":"2024-04-24T23:20:55.487320+0000","last_became_peered":"2024-04-24T23:20:55.487320+0000","last_unstale":"2024-04-24T23:20:56.725536+0000","last_undegraded":"2024-04-24T23:20:56.725536+0000","last_fullsized":"2024-04-24T23:20:56.725536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:25:27.675103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478125+0000","last_change":"2024-04-24T23:20:55.480991+0000","last_active":"2024-04-24T23:20:56.478125+0000","last_peered":"2024-04-24T23:20:56.478125+0000","last_clean":"2024-04-24T23:20:56.478125+0000","last_became_active":"2024-04-24T23:20:55.480654+0000","last_became_peered":"2024-04-24T23:20:55.480654+0000","last_unstale":"2024-04-24T23:20:56.478125+0000","last_undegraded":"2024-04-24T23:20:56.478125+0000","last_fullsized":"2024-04-24T23:20:56.478125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724519+0000","last_change":"2024-04-24T23:20:55.479402+0000","last_active":"2024-04-24T23:20:56.724519+0000","last_peered":"2024-04-24T23:20:56.724519+0000","last_clean":"2024-04-24T23:20:56.724519+0000","last_became_active":"2024-04-24T23:20:55.479013+0000","last_became_peered":"2024-04-24T23:20:55.479013+0000","last_unstale":"2024-04-24T23:20:56.724519+0000","last_undegraded":"2024-04-24T23:20:56.724519+0000","last_fullsized":"2024-04-24T23:20:56.724519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475989+0000","last_change":"2024-04-24T23:20:55.481507+0000","last_active":"2024-04-24T23:20:56.475989+0000","last_peered":"2024-04-24T23:20:56.475989+0000","last_clean":"2024-04-24T23:20:56.475989+0000","last_became_active":"2024-04-24T23:20:55.481144+0000","last_became_peered":"2024-04-24T23:20:55.481144+0000","last_unstale":"2024-04-24T23:20:56.475989+0000","last_undegraded":"2024-04-24T23:20:56.475989+0000","last_fullsized":"2024-04-24T23:20:56.475989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725067+0000","last_change":"2024-04-24T23:20:55.482367+0000","last_active":"2024-04-24T23:20:56.725067+0000","last_peered":"2024-04-24T23:20:56.725067+0000","last_clean":"2024-04-24T23:20:56.725067+0000","last_became_active":"2024-04-24T23:20:55.482047+0000","last_became_peered":"2024-04-24T23:20:55.482047+0000","last_unstale":"2024-04-24T23:20:56.725067+0000","last_undegraded":"2024-04-24T23:20:56.725067+0000","last_fullsized":"2024-04-24T23:20:56.725067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475443+0000","last_change":"2024-04-24T23:20:55.482474+0000","last_active":"2024-04-24T23:20:56.475443+0000","last_peered":"2024-04-24T23:20:56.475443+0000","last_clean":"2024-04-24T23:20:56.475443+0000","last_became_active":"2024-04-24T23:20:55.482157+0000","last_became_peered":"2024-04-24T23:20:55.482157+0000","last_unstale":"2024-04-24T23:20:56.475443+0000","last_undegraded":"2024-04-24T23:20:56.475443+0000","last_fullsized":"2024-04-24T23:20:56.475443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724980+0000","last_change":"2024-04-24T23:20:55.487389+0000","last_active":"2024-04-24T23:20:56.724980+0000","last_peered":"2024-04-24T23:20:56.724980+0000","last_clean":"2024-04-24T23:20:56.724980+0000","last_became_active":"2024-04-24T23:20:55.486724+0000","last_became_peered":"2024-04-24T23:20:55.486724+0000","last_unstale":"2024-04-24T23:20:56.724980+0000","last_undegraded":"2024-04-24T23:20:56.724980+0000","last_fullsized":"2024-04-24T23:20:56.724980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724916+0000","last_change":"2024-04-24T23:20:55.486339+0000","last_active":"2024-04-24T23:20:56.724916+0000","last_peered":"2024-04-24T23:20:56.724916+0000","last_clean":"2024-04-24T23:20:56.724916+0000","last_became_active":"2024-04-24T23:20:55.486072+0000","last_became_peered":"2024-04-24T23:20:55.486072+0000","last_unstale":"2024-04-24T23:20:56.724916+0000","last_undegraded":"2024-04-24T23:20:56.724916+0000","last_fullsized":"2024-04-24T23:20:56.724916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.491641+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.491349+0000","last_became_peered":"2024-04-24T23:20:55.491349+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:58.298429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480758+0000","last_change":"2024-04-24T23:20:55.479221+0000","last_active":"2024-04-24T23:20:56.480758+0000","last_peered":"2024-04-24T23:20:56.480758+0000","last_clean":"2024-04-24T23:20:56.480758+0000","last_became_active":"2024-04-24T23:20:55.478795+0000","last_became_peered":"2024-04-24T23:20:55.478795+0000","last_unstale":"2024-04-24T23:20:56.480758+0000","last_undegraded":"2024-04-24T23:20:56.480758+0000","last_fullsized":"2024-04-24T23:20:56.480758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480763+0000","last_change":"2024-04-24T23:20:55.479615+0000","last_active":"2024-04-24T23:20:56.480763+0000","last_peered":"2024-04-24T23:20:56.480763+0000","last_clean":"2024-04-24T23:20:56.480763+0000","last_became_active":"2024-04-24T23:20:55.479317+0000","last_became_peered":"2024-04-24T23:20:55.479317+0000","last_unstale":"2024-04-24T23:20:56.480763+0000","last_undegraded":"2024-04-24T23:20:56.480763+0000","last_fullsized":"2024-04-24T23:20:56.480763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475028+0000","last_change":"2024-04-24T23:20:55.481915+0000","last_active":"2024-04-24T23:20:56.475028+0000","last_peered":"2024-04-24T23:20:56.475028+0000","last_clean":"2024-04-24T23:20:56.475028+0000","last_became_active":"2024-04-24T23:20:55.481596+0000","last_became_peered":"2024-04-24T23:20:55.481596+0000","last_unstale":"2024-04-24T23:20:56.475028+0000","last_undegraded":"2024-04-24T23:20:56.475028+0000","last_fullsized":"2024-04-24T23:20:56.475028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:20.844340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725493+0000","last_change":"2024-04-24T23:20:55.476416+0000","last_active":"2024-04-24T23:20:56.725493+0000","last_peered":"2024-04-24T23:20:56.725493+0000","last_clean":"2024-04-24T23:20:56.725493+0000","last_became_active":"2024-04-24T23:20:55.476068+0000","last_became_peered":"2024-04-24T23:20:55.476068+0000","last_unstale":"2024-04-24T23:20:56.725493+0000","last_undegraded":"2024-04-24T23:20:56.725493+0000","last_fullsized":"2024-04-24T23:20:56.725493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724195+0000","last_change":"2024-04-24T23:20:55.479125+0000","last_active":"2024-04-24T23:20:56.724195+0000","last_peered":"2024-04-24T23:20:56.724195+0000","last_clean":"2024-04-24T23:20:56.724195+0000","last_became_active":"2024-04-24T23:20:55.478473+0000","last_became_peered":"2024-04-24T23:20:55.478473+0000","last_unstale":"2024-04-24T23:20:56.724195+0000","last_undegraded":"2024-04-24T23:20:56.724195+0000","last_fullsized":"2024-04-24T23:20:56.724195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:19.475113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477868+0000","last_change":"2024-04-24T23:20:55.474883+0000","last_active":"2024-04-24T23:20:56.477868+0000","last_peered":"2024-04-24T23:20:56.477868+0000","last_clean":"2024-04-24T23:20:56.477868+0000","last_became_active":"2024-04-24T23:20:55.474459+0000","last_became_peered":"2024-04-24T23:20:55.474459+0000","last_unstale":"2024-04-24T23:20:56.477868+0000","last_undegraded":"2024-04-24T23:20:56.477868+0000","last_fullsized":"2024-04-24T23:20:56.477868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475040+0000","last_change":"2024-04-24T23:20:55.483511+0000","last_active":"2024-04-24T23:20:56.475040+0000","last_peered":"2024-04-24T23:20:56.475040+0000","last_clean":"2024-04-24T23:20:56.475040+0000","last_became_active":"2024-04-24T23:20:55.483171+0000","last_became_peered":"2024-04-24T23:20:55.483171+0000","last_unstale":"2024-04-24T23:20:56.475040+0000","last_undegraded":"2024-04-24T23:20:56.475040+0000","last_fullsized":"2024-04-24T23:20:56.475040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475537+0000","last_change":"2024-04-24T23:20:55.489069+0000","last_active":"2024-04-24T23:20:56.475537+0000","last_peered":"2024-04-24T23:20:56.475537+0000","last_clean":"2024-04-24T23:20:56.475537+0000","last_became_active":"2024-04-24T23:20:55.488796+0000","last_became_peered":"2024-04-24T23:20:55.488796+0000","last_unstale":"2024-04-24T23:20:56.475537+0000","last_undegraded":"2024-04-24T23:20:56.475537+0000","last_fullsized":"2024-04-24T23:20:56.475537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724755+0000","last_change":"2024-04-24T23:20:55.480920+0000","last_active":"2024-04-24T23:20:56.724755+0000","last_peered":"2024-04-24T23:20:56.724755+0000","last_clean":"2024-04-24T23:20:56.724755+0000","last_became_active":"2024-04-24T23:20:55.480581+0000","last_became_peered":"2024-04-24T23:20:55.480581+0000","last_unstale":"2024-04-24T23:20:56.724755+0000","last_undegraded":"2024-04-24T23:20:56.724755+0000","last_fullsized":"2024-04-24T23:20:56.724755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725520+0000","last_change":"2024-04-24T23:20:55.488901+0000","last_active":"2024-04-24T23:20:56.725520+0000","last_peered":"2024-04-24T23:20:56.725520+0000","last_clean":"2024-04-24T23:20:56.725520+0000","last_became_active":"2024-04-24T23:20:55.488647+0000","last_became_peered":"2024-04-24T23:20:55.488647+0000","last_unstale":"2024-04-24T23:20:56.725520+0000","last_undegraded":"2024-04-24T23:20:56.725520+0000","last_fullsized":"2024-04-24T23:20:56.725520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480674+0000","last_change":"2024-04-24T23:20:55.476826+0000","last_active":"2024-04-24T23:20:56.480674+0000","last_peered":"2024-04-24T23:20:56.480674+0000","last_clean":"2024-04-24T23:20:56.480674+0000","last_became_active":"2024-04-24T23:20:55.476481+0000","last_became_peered":"2024-04-24T23:20:55.476481+0000","last_unstale":"2024-04-24T23:20:56.480674+0000","last_undegraded":"2024-04-24T23:20:56.480674+0000","last_fullsized":"2024-04-24T23:20:56.480674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724349+0000","last_change":"2024-04-24T23:20:55.491072+0000","last_active":"2024-04-24T23:20:56.724349+0000","last_peered":"2024-04-24T23:20:56.724349+0000","last_clean":"2024-04-24T23:20:56.724349+0000","last_became_active":"2024-04-24T23:20:55.490763+0000","last_became_peered":"2024-04-24T23:20:55.490763+0000","last_unstale":"2024-04-24T23:20:56.724349+0000","last_undegraded":"2024-04-24T23:20:56.724349+0000","last_fullsized":"2024-04-24T23:20:56.724349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475836+0000","last_change":"2024-04-24T23:20:55.487623+0000","last_active":"2024-04-24T23:20:56.475836+0000","last_peered":"2024-04-24T23:20:56.475836+0000","last_clean":"2024-04-24T23:20:56.475836+0000","last_became_active":"2024-04-24T23:20:55.487355+0000","last_became_peered":"2024-04-24T23:20:55.487355+0000","last_unstale":"2024-04-24T23:20:56.475836+0000","last_undegraded":"2024-04-24T23:20:56.475836+0000","last_fullsized":"2024-04-24T23:20:56.475836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725470+0000","last_change":"2024-04-24T23:20:55.476152+0000","last_active":"2024-04-24T23:20:56.725470+0000","last_peered":"2024-04-24T23:20:56.725470+0000","last_clean":"2024-04-24T23:20:56.725470+0000","last_became_active":"2024-04-24T23:20:55.475857+0000","last_became_peered":"2024-04-24T23:20:55.475857+0000","last_unstale":"2024-04-24T23:20:56.725470+0000","last_undegraded":"2024-04-24T23:20:56.725470+0000","last_fullsized":"2024-04-24T23:20:56.725470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475501+0000","last_change":"2024-04-24T23:20:55.484268+0000","last_active":"2024-04-24T23:20:56.475501+0000","last_peered":"2024-04-24T23:20:56.475501+0000","last_clean":"2024-04-24T23:20:56.475501+0000","last_became_active":"2024-04-24T23:20:55.483972+0000","last_became_peered":"2024-04-24T23:20:55.483972+0000","last_unstale":"2024-04-24T23:20:56.475501+0000","last_undegraded":"2024-04-24T23:20:56.475501+0000","last_fullsized":"2024-04-24T23:20:56.475501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725722+0000","last_change":"2024-04-24T23:20:55.480092+0000","last_active":"2024-04-24T23:20:56.725722+0000","last_peered":"2024-04-24T23:20:56.725722+0000","last_clean":"2024-04-24T23:20:56.725722+0000","last_became_active":"2024-04-24T23:20:55.479783+0000","last_became_peered":"2024-04-24T23:20:55.479783+0000","last_unstale":"2024-04-24T23:20:56.725722+0000","last_undegraded":"2024-04-24T23:20:56.725722+0000","last_fullsized":"2024-04-24T23:20:56.725722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:33.585112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480830+0000","last_change":"2024-04-24T23:20:55.488050+0000","last_active":"2024-04-24T23:20:56.480830+0000","last_peered":"2024-04-24T23:20:56.480830+0000","last_clean":"2024-04-24T23:20:56.480830+0000","last_became_active":"2024-04-24T23:20:55.487762+0000","last_became_peered":"2024-04-24T23:20:55.487762+0000","last_unstale":"2024-04-24T23:20:56.480830+0000","last_undegraded":"2024-04-24T23:20:56.480830+0000","last_fullsized":"2024-04-24T23:20:56.480830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478091+0000","last_change":"2024-04-24T23:20:55.478530+0000","last_active":"2024-04-24T23:20:56.478091+0000","last_peered":"2024-04-24T23:20:56.478091+0000","last_clean":"2024-04-24T23:20:56.478091+0000","last_became_active":"2024-04-24T23:20:55.478233+0000","last_became_peered":"2024-04-24T23:20:55.478233+0000","last_unstale":"2024-04-24T23:20:56.478091+0000","last_undegraded":"2024-04-24T23:20:56.478091+0000","last_fullsized":"2024-04-24T23:20:56.478091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725374+0000","last_change":"2024-04-24T23:20:55.480077+0000","last_active":"2024-04-24T23:20:56.725374+0000","last_peered":"2024-04-24T23:20:56.725374+0000","last_clean":"2024-04-24T23:20:56.725374+0000","last_became_active":"2024-04-24T23:20:55.479769+0000","last_became_peered":"2024-04-24T23:20:55.479769+0000","last_unstale":"2024-04-24T23:20:56.725374+0000","last_undegraded":"2024-04-24T23:20:56.725374+0000","last_fullsized":"2024-04-24T23:20:56.725374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724513+0000","last_change":"2024-04-24T23:20:55.481475+0000","last_active":"2024-04-24T23:20:56.724513+0000","last_peered":"2024-04-24T23:20:56.724513+0000","last_clean":"2024-04-24T23:20:56.724513+0000","last_became_active":"2024-04-24T23:20:55.480091+0000","last_became_peered":"2024-04-24T23:20:55.480091+0000","last_unstale":"2024-04-24T23:20:56.724513+0000","last_undegraded":"2024-04-24T23:20:56.724513+0000","last_fullsized":"2024-04-24T23:20:56.724513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476229+0000","last_change":"2024-04-24T23:20:55.485092+0000","last_active":"2024-04-24T23:20:56.476229+0000","last_peered":"2024-04-24T23:20:56.476229+0000","last_clean":"2024-04-24T23:20:56.476229+0000","last_became_active":"2024-04-24T23:20:55.484409+0000","last_became_peered":"2024-04-24T23:20:55.484409+0000","last_unstale":"2024-04-24T23:20:56.476229+0000","last_undegraded":"2024-04-24T23:20:56.476229+0000","last_fullsized":"2024-04-24T23:20:56.476229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:08.651059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.469291+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.468961+0000","last_became_peered":"2024-04-24T23:20:55.468961+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480450+0000","last_change":"2024-04-24T23:20:55.484826+0000","last_active":"2024-04-24T23:20:56.480450+0000","last_peered":"2024-04-24T23:20:56.480450+0000","last_clean":"2024-04-24T23:20:56.480450+0000","last_became_active":"2024-04-24T23:20:55.484500+0000","last_became_peered":"2024-04-24T23:20:55.484500+0000","last_unstale":"2024-04-24T23:20:56.480450+0000","last_undegraded":"2024-04-24T23:20:56.480450+0000","last_fullsized":"2024-04-24T23:20:56.480450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475480+0000","last_change":"2024-04-24T23:20:55.476006+0000","last_active":"2024-04-24T23:20:56.475480+0000","last_peered":"2024-04-24T23:20:56.475480+0000","last_clean":"2024-04-24T23:20:56.475480+0000","last_became_active":"2024-04-24T23:20:55.475090+0000","last_became_peered":"2024-04-24T23:20:55.475090+0000","last_unstale":"2024-04-24T23:20:56.475480+0000","last_undegraded":"2024-04-24T23:20:56.475480+0000","last_fullsized":"2024-04-24T23:20:56.475480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:11.022696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725560+0000","last_change":"2024-04-24T23:20:54.481863+0000","last_active":"2024-04-24T23:20:56.725560+0000","last_peered":"2024-04-24T23:20:56.725560+0000","last_clean":"2024-04-24T23:20:56.725560+0000","last_became_active":"2024-04-24T23:20:54.481498+0000","last_became_peered":"2024-04-24T23:20:54.481498+0000","last_unstale":"2024-04-24T23:20:56.725560+0000","last_undegraded":"2024-04-24T23:20:56.725560+0000","last_fullsized":"2024-04-24T23:20:56.725560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724565+0000","last_change":"2024-04-24T23:20:55.478651+0000","last_active":"2024-04-24T23:20:56.724565+0000","last_peered":"2024-04-24T23:20:56.724565+0000","last_clean":"2024-04-24T23:20:56.724565+0000","last_became_active":"2024-04-24T23:20:55.477783+0000","last_became_peered":"2024-04-24T23:20:55.477783+0000","last_unstale":"2024-04-24T23:20:56.724565+0000","last_undegraded":"2024-04-24T23:20:56.724565+0000","last_fullsized":"2024-04-24T23:20:56.724565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480400+0000","last_change":"2024-04-24T23:20:54.477560+0000","last_active":"2024-04-24T23:20:56.480400+0000","last_peered":"2024-04-24T23:20:56.480400+0000","last_clean":"2024-04-24T23:20:56.480400+0000","last_became_active":"2024-04-24T23:20:54.477261+0000","last_became_peered":"2024-04-24T23:20:54.477261+0000","last_unstale":"2024-04-24T23:20:56.480400+0000","last_undegraded":"2024-04-24T23:20:56.480400+0000","last_fullsized":"2024-04-24T23:20:56.480400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725154+0000","last_change":"2024-04-24T23:20:54.479540+0000","last_active":"2024-04-24T23:20:56.725154+0000","last_peered":"2024-04-24T23:20:56.725154+0000","last_clean":"2024-04-24T23:20:56.725154+0000","last_became_active":"2024-04-24T23:20:54.479206+0000","last_became_peered":"2024-04-24T23:20:54.479206+0000","last_unstale":"2024-04-24T23:20:56.725154+0000","last_undegraded":"2024-04-24T23:20:56.725154+0000","last_fullsized":"2024-04-24T23:20:56.725154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725091+0000","last_change":"2024-04-24T23:20:55.484927+0000","last_active":"2024-04-24T23:20:56.725091+0000","last_peered":"2024-04-24T23:20:56.725091+0000","last_clean":"2024-04-24T23:20:56.725091+0000","last_became_active":"2024-04-24T23:20:55.484611+0000","last_became_peered":"2024-04-24T23:20:55.484611+0000","last_unstale":"2024-04-24T23:20:56.725091+0000","last_undegraded":"2024-04-24T23:20:56.725091+0000","last_fullsized":"2024-04-24T23:20:56.725091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475910+0000","last_change":"2024-04-24T23:20:54.471358+0000","last_active":"2024-04-24T23:20:56.475910+0000","last_peered":"2024-04-24T23:20:56.475910+0000","last_clean":"2024-04-24T23:20:56.475910+0000","last_became_active":"2024-04-24T23:20:54.470800+0000","last_became_peered":"2024-04-24T23:20:54.470800+0000","last_unstale":"2024-04-24T23:20:56.475910+0000","last_undegraded":"2024-04-24T23:20:56.475910+0000","last_fullsized":"2024-04-24T23:20:56.475910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.477535+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.477215+0000","last_became_peered":"2024-04-24T23:20:55.477215+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480761+0000","last_change":"2024-04-24T23:20:54.479450+0000","last_active":"2024-04-24T23:20:56.480761+0000","last_peered":"2024-04-24T23:20:56.480761+0000","last_clean":"2024-04-24T23:20:56.480761+0000","last_became_active":"2024-04-24T23:20:54.479174+0000","last_became_peered":"2024-04-24T23:20:54.479174+0000","last_unstale":"2024-04-24T23:20:56.480761+0000","last_undegraded":"2024-04-24T23:20:56.480761+0000","last_fullsized":"2024-04-24T23:20:56.480761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476156+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.476156+0000","last_peered":"2024-04-24T23:20:56.476156+0000","last_clean":"2024-04-24T23:20:56.476156+0000","last_became_active":"2024-04-24T23:20:55.488014+0000","last_became_peered":"2024-04-24T23:20:55.488014+0000","last_unstale":"2024-04-24T23:20:56.476156+0000","last_undegraded":"2024-04-24T23:20:56.476156+0000","last_fullsized":"2024-04-24T23:20:56.476156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476129+0000","last_change":"2024-04-24T23:20:54.480387+0000","last_active":"2024-04-24T23:20:56.476129+0000","last_peered":"2024-04-24T23:20:56.476129+0000","last_clean":"2024-04-24T23:20:56.476129+0000","last_became_active":"2024-04-24T23:20:54.480037+0000","last_became_peered":"2024-04-24T23:20:54.480037+0000","last_unstale":"2024-04-24T23:20:56.476129+0000","last_undegraded":"2024-04-24T23:20:56.476129+0000","last_fullsized":"2024-04-24T23:20:56.476129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724464+0000","last_change":"2024-04-24T23:20:55.481206+0000","last_active":"2024-04-24T23:20:56.724464+0000","last_peered":"2024-04-24T23:20:56.724464+0000","last_clean":"2024-04-24T23:20:56.724464+0000","last_became_active":"2024-04-24T23:20:55.478018+0000","last_became_peered":"2024-04-24T23:20:55.478018+0000","last_unstale":"2024-04-24T23:20:56.724464+0000","last_undegraded":"2024-04-24T23:20:56.724464+0000","last_fullsized":"2024-04-24T23:20:56.724464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478112+0000","last_change":"2024-04-24T23:20:54.469557+0000","last_active":"2024-04-24T23:20:56.478112+0000","last_peered":"2024-04-24T23:20:56.478112+0000","last_clean":"2024-04-24T23:20:56.478112+0000","last_became_active":"2024-04-24T23:20:54.469251+0000","last_became_peered":"2024-04-24T23:20:54.469251+0000","last_unstale":"2024-04-24T23:20:56.478112+0000","last_undegraded":"2024-04-24T23:20:56.478112+0000","last_fullsized":"2024-04-24T23:20:56.478112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475184+0000","last_change":"2024-04-24T23:20:55.491797+0000","last_active":"2024-04-24T23:20:56.475184+0000","last_peered":"2024-04-24T23:20:56.475184+0000","last_clean":"2024-04-24T23:20:56.475184+0000","last_became_active":"2024-04-24T23:20:55.491453+0000","last_became_peered":"2024-04-24T23:20:55.491453+0000","last_unstale":"2024-04-24T23:20:56.475184+0000","last_undegraded":"2024-04-24T23:20:56.475184+0000","last_fullsized":"2024-04-24T23:20:56.475184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:05.444694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480475+0000","last_change":"2024-04-24T23:20:54.482472+0000","last_active":"2024-04-24T23:20:56.480475+0000","last_peered":"2024-04-24T23:20:56.480475+0000","last_clean":"2024-04-24T23:20:56.480475+0000","last_became_active":"2024-04-24T23:20:54.481351+0000","last_became_peered":"2024-04-24T23:20:54.481351+0000","last_unstale":"2024-04-24T23:20:56.480475+0000","last_undegraded":"2024-04-24T23:20:56.480475+0000","last_fullsized":"2024-04-24T23:20:56.480475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725210+0000","last_change":"2024-04-24T23:20:55.483153+0000","last_active":"2024-04-24T23:20:56.725210+0000","last_peered":"2024-04-24T23:20:56.725210+0000","last_clean":"2024-04-24T23:20:56.725210+0000","last_became_active":"2024-04-24T23:20:55.482859+0000","last_became_peered":"2024-04-24T23:20:55.482859+0000","last_unstale":"2024-04-24T23:20:56.725210+0000","last_undegraded":"2024-04-24T23:20:56.725210+0000","last_fullsized":"2024-04-24T23:20:56.725210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:59:54.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724624+0000","last_change":"2024-04-24T23:20:54.478089+0000","last_active":"2024-04-24T23:20:56.724624+0000","last_peered":"2024-04-24T23:20:56.724624+0000","last_clean":"2024-04-24T23:20:56.724624+0000","last_became_active":"2024-04-24T23:20:54.477302+0000","last_became_peered":"2024-04-24T23:20:54.477302+0000","last_unstale":"2024-04-24T23:20:56.724624+0000","last_undegraded":"2024-04-24T23:20:56.724624+0000","last_fullsized":"2024-04-24T23:20:56.724624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:58.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724289+0000","last_change":"2024-04-24T23:20:55.488305+0000","last_active":"2024-04-24T23:20:56.724289+0000","last_peered":"2024-04-24T23:20:56.724289+0000","last_clean":"2024-04-24T23:20:56.724289+0000","last_became_active":"2024-04-24T23:20:55.488016+0000","last_became_peered":"2024-04-24T23:20:55.488016+0000","last_unstale":"2024-04-24T23:20:56.724289+0000","last_undegraded":"2024-04-24T23:20:56.724289+0000","last_fullsized":"2024-04-24T23:20:56.724289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:48.795181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476045+0000","last_change":"2024-04-24T23:20:54.473922+0000","last_active":"2024-04-24T23:20:56.476045+0000","last_peered":"2024-04-24T23:20:56.476045+0000","last_clean":"2024-04-24T23:20:56.476045+0000","last_became_active":"2024-04-24T23:20:54.473601+0000","last_became_peered":"2024-04-24T23:20:54.473601+0000","last_unstale":"2024-04-24T23:20:56.476045+0000","last_undegraded":"2024-04-24T23:20:56.476045+0000","last_fullsized":"2024-04-24T23:20:56.476045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725128+0000","last_change":"2024-04-24T23:20:55.488225+0000","last_active":"2024-04-24T23:20:56.725128+0000","last_peered":"2024-04-24T23:20:56.725128+0000","last_clean":"2024-04-24T23:20:56.725128+0000","last_became_active":"2024-04-24T23:20:55.485663+0000","last_became_peered":"2024-04-24T23:20:55.485663+0000","last_unstale":"2024-04-24T23:20:56.725128+0000","last_undegraded":"2024-04-24T23:20:56.725128+0000","last_fullsized":"2024-04-24T23:20:56.725128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:41.458523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:54.478237+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:54.477943+0000","last_became_peered":"2024-04-24T23:20:54.477943+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480383+0000","last_change":"2024-04-24T23:20:55.484884+0000","last_active":"2024-04-24T23:20:56.480383+0000","last_peered":"2024-04-24T23:20:56.480383+0000","last_clean":"2024-04-24T23:20:56.480383+0000","last_became_active":"2024-04-24T23:20:55.484578+0000","last_became_peered":"2024-04-24T23:20:55.484578+0000","last_unstale":"2024-04-24T23:20:56.480383+0000","last_undegraded":"2024-04-24T23:20:56.480383+0000","last_fullsized":"2024-04-24T23:20:56.480383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:30.937003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724810+0000","last_change":"2024-04-24T23:20:54.473033+0000","last_active":"2024-04-24T23:20:56.724810+0000","last_peered":"2024-04-24T23:20:56.724810+0000","last_clean":"2024-04-24T23:20:56.724810+0000","last_became_active":"2024-04-24T23:20:54.472748+0000","last_became_peered":"2024-04-24T23:20:54.472748+0000","last_unstale":"2024-04-24T23:20:56.724810+0000","last_undegraded":"2024-04-24T23:20:56.724810+0000","last_fullsized":"2024-04-24T23:20:56.724810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724977+0000","last_change":"2024-04-24T23:20:55.485153+0000","last_active":"2024-04-24T23:20:56.724977+0000","last_peered":"2024-04-24T23:20:56.724977+0000","last_clean":"2024-04-24T23:20:56.724977+0000","last_became_active":"2024-04-24T23:20:55.484876+0000","last_became_peered":"2024-04-24T23:20:55.484876+0000","last_unstale":"2024-04-24T23:20:56.724977+0000","last_undegraded":"2024-04-24T23:20:56.724977+0000","last_fullsized":"2024-04-24T23:20:56.724977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474721+0000","last_change":"2024-04-24T23:20:54.476273+0000","last_active":"2024-04-24T23:20:56.474721+0000","last_peered":"2024-04-24T23:20:56.474721+0000","last_clean":"2024-04-24T23:20:56.474721+0000","last_became_active":"2024-04-24T23:20:54.475993+0000","last_became_peered":"2024-04-24T23:20:54.475993+0000","last_unstale":"2024-04-24T23:20:56.474721+0000","last_undegraded":"2024-04-24T23:20:56.474721+0000","last_fullsized":"2024-04-24T23:20:56.474721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725412+0000","last_change":"2024-04-24T23:20:55.487659+0000","last_active":"2024-04-24T23:20:56.725412+0000","last_peered":"2024-04-24T23:20:56.725412+0000","last_clean":"2024-04-24T23:20:56.725412+0000","last_became_active":"2024-04-24T23:20:55.486557+0000","last_became_peered":"2024-04-24T23:20:55.486557+0000","last_unstale":"2024-04-24T23:20:56.725412+0000","last_undegraded":"2024-04-24T23:20:56.725412+0000","last_fullsized":"2024-04-24T23:20:56.725412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724872+0000","last_change":"2024-04-24T23:20:54.477334+0000","last_active":"2024-04-24T23:20:56.724872+0000","last_peered":"2024-04-24T23:20:56.724872+0000","last_clean":"2024-04-24T23:20:56.724872+0000","last_became_active":"2024-04-24T23:20:54.477058+0000","last_became_peered":"2024-04-24T23:20:54.477058+0000","last_unstale":"2024-04-24T23:20:56.724872+0000","last_undegraded":"2024-04-24T23:20:56.724872+0000","last_fullsized":"2024-04-24T23:20:56.724872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:24.246046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725667+0000","last_change":"2024-04-24T23:20:55.492754+0000","last_active":"2024-04-24T23:20:56.725667+0000","last_peered":"2024-04-24T23:20:56.725667+0000","last_clean":"2024-04-24T23:20:56.725667+0000","last_became_active":"2024-04-24T23:20:55.492474+0000","last_became_peered":"2024-04-24T23:20:55.492474+0000","last_unstale":"2024-04-24T23:20:56.725667+0000","last_undegraded":"2024-04-24T23:20:56.725667+0000","last_fullsized":"2024-04-24T23:20:56.725667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:54.766130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478192+0000","last_change":"2024-04-24T23:20:54.471091+0000","last_active":"2024-04-24T23:20:56.478192+0000","last_peered":"2024-04-24T23:20:56.478192+0000","last_clean":"2024-04-24T23:20:56.478192+0000","last_became_active":"2024-04-24T23:20:54.470811+0000","last_became_peered":"2024-04-24T23:20:54.470811+0000","last_unstale":"2024-04-24T23:20:56.478192+0000","last_undegraded":"2024-04-24T23:20:56.478192+0000","last_fullsized":"2024-04-24T23:20:56.478192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476216+0000","last_change":"2024-04-24T23:20:55.480640+0000","last_active":"2024-04-24T23:20:56.476216+0000","last_peered":"2024-04-24T23:20:56.476216+0000","last_clean":"2024-04-24T23:20:56.476216+0000","last_became_active":"2024-04-24T23:20:55.480327+0000","last_became_peered":"2024-04-24T23:20:55.480327+0000","last_unstale":"2024-04-24T23:20:56.476216+0000","last_undegraded":"2024-04-24T23:20:56.476216+0000","last_fullsized":"2024-04-24T23:20:56.476216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:56.878361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725414+0000","last_change":"2024-04-24T23:20:54.476392+0000","last_active":"2024-04-24T23:20:56.725414+0000","last_peered":"2024-04-24T23:20:56.725414+0000","last_clean":"2024-04-24T23:20:56.725414+0000","last_became_active":"2024-04-24T23:20:54.476051+0000","last_became_peered":"2024-04-24T23:20:54.476051+0000","last_unstale":"2024-04-24T23:20:56.725414+0000","last_undegraded":"2024-04-24T23:20:56.725414+0000","last_fullsized":"2024-04-24T23:20:56.725414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725389+0000","last_change":"2024-04-24T23:20:55.483909+0000","last_active":"2024-04-24T23:20:56.725389+0000","last_peered":"2024-04-24T23:20:56.725389+0000","last_clean":"2024-04-24T23:20:56.725389+0000","last_became_active":"2024-04-24T23:20:55.483625+0000","last_became_peered":"2024-04-24T23:20:55.483625+0000","last_unstale":"2024-04-24T23:20:56.725389+0000","last_undegraded":"2024-04-24T23:20:56.725389+0000","last_fullsized":"2024-04-24T23:20:56.725389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724832+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:20:56.724832+0000","last_peered":"2024-04-24T23:20:56.724832+0000","last_clean":"2024-04-24T23:20:56.724832+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:20:56.724832+0000","last_undegraded":"2024-04-24T23:20:56.724832+0000","last_fullsized":"2024-04-24T23:20:56.724832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474912+0000","last_change":"2024-04-24T23:20:54.470392+0000","last_active":"2024-04-24T23:20:56.474912+0000","last_peered":"2024-04-24T23:20:56.474912+0000","last_clean":"2024-04-24T23:20:56.474912+0000","last_became_active":"2024-04-24T23:20:54.470095+0000","last_became_peered":"2024-04-24T23:20:54.470095+0000","last_unstale":"2024-04-24T23:20:56.474912+0000","last_undegraded":"2024-04-24T23:20:56.474912+0000","last_fullsized":"2024-04-24T23:20:56.474912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:24:19.840077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725394+0000","last_change":"2024-04-24T23:20:55.485347+0000","last_active":"2024-04-24T23:20:56.725394+0000","last_peered":"2024-04-24T23:20:56.725394+0000","last_clean":"2024-04-24T23:20:56.725394+0000","last_became_active":"2024-04-24T23:20:55.485004+0000","last_became_peered":"2024-04-24T23:20:55.485004+0000","last_unstale":"2024-04-24T23:20:56.725394+0000","last_undegraded":"2024-04-24T23:20:56.725394+0000","last_fullsized":"2024-04-24T23:20:56.725394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724782+0000","last_change":"2024-04-24T23:20:54.482240+0000","last_active":"2024-04-24T23:20:56.724782+0000","last_peered":"2024-04-24T23:20:56.724782+0000","last_clean":"2024-04-24T23:20:56.724782+0000","last_became_active":"2024-04-24T23:20:54.481934+0000","last_became_peered":"2024-04-24T23:20:54.481934+0000","last_unstale":"2024-04-24T23:20:56.724782+0000","last_undegraded":"2024-04-24T23:20:56.724782+0000","last_fullsized":"2024-04-24T23:20:56.724782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:13.029046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724293+0000","last_change":"2024-04-24T23:20:55.474530+0000","last_active":"2024-04-24T23:20:56.724293+0000","last_peered":"2024-04-24T23:20:56.724293+0000","last_clean":"2024-04-24T23:20:56.724293+0000","last_became_active":"2024-04-24T23:20:55.474181+0000","last_became_peered":"2024-04-24T23:20:55.474181+0000","last_unstale":"2024-04-24T23:20:56.724293+0000","last_undegraded":"2024-04-24T23:20:56.724293+0000","last_fullsized":"2024-04-24T23:20:56.724293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:43.728916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724886+0000","last_change":"2024-04-24T23:20:54.478684+0000","last_active":"2024-04-24T23:20:56.724886+0000","last_peered":"2024-04-24T23:20:56.724886+0000","last_clean":"2024-04-24T23:20:56.724886+0000","last_became_active":"2024-04-24T23:20:54.478377+0000","last_became_peered":"2024-04-24T23:20:54.478377+0000","last_unstale":"2024-04-24T23:20:56.724886+0000","last_undegraded":"2024-04-24T23:20:56.724886+0000","last_fullsized":"2024-04-24T23:20:56.724886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477871+0000","last_change":"2024-04-24T23:20:55.478546+0000","last_active":"2024-04-24T23:20:56.477871+0000","last_peered":"2024-04-24T23:20:56.477871+0000","last_clean":"2024-04-24T23:20:56.477871+0000","last_became_active":"2024-04-24T23:20:55.478253+0000","last_became_peered":"2024-04-24T23:20:55.478253+0000","last_unstale":"2024-04-24T23:20:56.477871+0000","last_undegraded":"2024-04-24T23:20:56.477871+0000","last_fullsized":"2024-04-24T23:20:56.477871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:59.195277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476161+0000","last_change":"2024-04-24T23:20:54.478753+0000","last_active":"2024-04-24T23:20:56.476161+0000","last_peered":"2024-04-24T23:20:56.476161+0000","last_clean":"2024-04-24T23:20:56.476161+0000","last_became_active":"2024-04-24T23:20:54.478441+0000","last_became_peered":"2024-04-24T23:20:54.478441+0000","last_unstale":"2024-04-24T23:20:56.476161+0000","last_undegraded":"2024-04-24T23:20:56.476161+0000","last_fullsized":"2024-04-24T23:20:56.476161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475739+0000","last_change":"2024-04-24T23:20:55.483158+0000","last_active":"2024-04-24T23:20:56.475739+0000","last_peered":"2024-04-24T23:20:56.475739+0000","last_clean":"2024-04-24T23:20:56.475739+0000","last_became_active":"2024-04-24T23:20:55.482865+0000","last_became_peered":"2024-04-24T23:20:55.482865+0000","last_unstale":"2024-04-24T23:20:56.475739+0000","last_undegraded":"2024-04-24T23:20:56.475739+0000","last_fullsized":"2024-04-24T23:20:56.475739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:14.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477991+0000","last_change":"2024-04-24T23:20:54.476881+0000","last_active":"2024-04-24T23:20:56.477991+0000","last_peered":"2024-04-24T23:20:56.477991+0000","last_clean":"2024-04-24T23:20:56.477991+0000","last_became_active":"2024-04-24T23:20:54.476588+0000","last_became_peered":"2024-04-24T23:20:54.476588+0000","last_unstale":"2024-04-24T23:20:56.477991+0000","last_undegraded":"2024-04-24T23:20:56.477991+0000","last_fullsized":"2024-04-24T23:20:56.477991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:42.729526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475322+0000","last_change":"2024-04-24T23:20:55.482626+0000","last_active":"2024-04-24T23:20:56.475322+0000","last_peered":"2024-04-24T23:20:56.475322+0000","last_clean":"2024-04-24T23:20:56.475322+0000","last_became_active":"2024-04-24T23:20:55.482266+0000","last_became_peered":"2024-04-24T23:20:55.482266+0000","last_unstale":"2024-04-24T23:20:56.475322+0000","last_undegraded":"2024-04-24T23:20:56.475322+0000","last_fullsized":"2024-04-24T23:20:56.475322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725524+0000","last_change":"2024-04-24T23:20:54.484817+0000","last_active":"2024-04-24T23:20:56.725524+0000","last_peered":"2024-04-24T23:20:56.725524+0000","last_clean":"2024-04-24T23:20:56.725524+0000","last_became_active":"2024-04-24T23:20:54.484553+0000","last_became_peered":"2024-04-24T23:20:54.484553+0000","last_unstale":"2024-04-24T23:20:56.725524+0000","last_undegraded":"2024-04-24T23:20:56.725524+0000","last_fullsized":"2024-04-24T23:20:56.725524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:42.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.478041+0000","last_change":"2024-04-24T23:20:55.479475+0000","last_active":"2024-04-24T23:20:56.478041+0000","last_peered":"2024-04-24T23:20:56.478041+0000","last_clean":"2024-04-24T23:20:56.478041+0000","last_became_active":"2024-04-24T23:20:55.478984+0000","last_became_peered":"2024-04-24T23:20:55.478984+0000","last_unstale":"2024-04-24T23:20:56.478041+0000","last_undegraded":"2024-04-24T23:20:56.478041+0000","last_fullsized":"2024-04-24T23:20:56.478041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:17.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.479792+0000","last_change":"2024-04-24T23:20:54.478307+0000","last_active":"2024-04-24T23:20:56.479792+0000","last_peered":"2024-04-24T23:20:56.479792+0000","last_clean":"2024-04-24T23:20:56.479792+0000","last_became_active":"2024-04-24T23:20:54.477988+0000","last_became_peered":"2024-04-24T23:20:54.477988+0000","last_unstale":"2024-04-24T23:20:56.479792+0000","last_undegraded":"2024-04-24T23:20:56.479792+0000","last_fullsized":"2024-04-24T23:20:56.479792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725717+0000","last_change":"2024-04-24T23:20:55.487879+0000","last_active":"2024-04-24T23:20:56.725717+0000","last_peered":"2024-04-24T23:20:56.725717+0000","last_clean":"2024-04-24T23:20:56.725717+0000","last_became_active":"2024-04-24T23:20:55.487553+0000","last_became_peered":"2024-04-24T23:20:55.487553+0000","last_unstale":"2024-04-24T23:20:56.725717+0000","last_undegraded":"2024-04-24T23:20:56.725717+0000","last_fullsized":"2024-04-24T23:20:56.725717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:35.838001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480845+0000","last_change":"2024-04-24T23:20:54.480826+0000","last_active":"2024-04-24T23:20:56.480845+0000","last_peered":"2024-04-24T23:20:56.480845+0000","last_clean":"2024-04-24T23:20:56.480845+0000","last_became_active":"2024-04-24T23:20:54.480351+0000","last_became_peered":"2024-04-24T23:20:54.480351+0000","last_unstale":"2024-04-24T23:20:56.480845+0000","last_undegraded":"2024-04-24T23:20:56.480845+0000","last_fullsized":"2024-04-24T23:20:56.480845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:24.361077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724786+0000","last_change":"2024-04-24T23:20:55.480939+0000","last_active":"2024-04-24T23:20:56.724786+0000","last_peered":"2024-04-24T23:20:56.724786+0000","last_clean":"2024-04-24T23:20:56.724786+0000","last_became_active":"2024-04-24T23:20:55.478888+0000","last_became_peered":"2024-04-24T23:20:55.478888+0000","last_unstale":"2024-04-24T23:20:56.724786+0000","last_undegraded":"2024-04-24T23:20:56.724786+0000","last_fullsized":"2024-04-24T23:20:56.724786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:56.037901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724748+0000","last_change":"2024-04-24T23:20:54.482329+0000","last_active":"2024-04-24T23:20:56.724748+0000","last_peered":"2024-04-24T23:20:56.724748+0000","last_clean":"2024-04-24T23:20:56.724748+0000","last_became_active":"2024-04-24T23:20:54.482059+0000","last_became_peered":"2024-04-24T23:20:54.482059+0000","last_unstale":"2024-04-24T23:20:56.724748+0000","last_undegraded":"2024-04-24T23:20:56.724748+0000","last_fullsized":"2024-04-24T23:20:56.724748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.475091+0000","last_change":"2024-04-24T23:20:55.478742+0000","last_active":"2024-04-24T23:20:56.475091+0000","last_peered":"2024-04-24T23:20:56.475091+0000","last_clean":"2024-04-24T23:20:56.475091+0000","last_became_active":"2024-04-24T23:20:55.478465+0000","last_became_peered":"2024-04-24T23:20:55.478465+0000","last_unstale":"2024-04-24T23:20:56.475091+0000","last_undegraded":"2024-04-24T23:20:56.475091+0000","last_fullsized":"2024-04-24T23:20:56.475091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:32.641502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477976+0000","last_change":"2024-04-24T23:20:54.480255+0000","last_active":"2024-04-24T23:20:56.477976+0000","last_peered":"2024-04-24T23:20:56.477976+0000","last_clean":"2024-04-24T23:20:56.477976+0000","last_became_active":"2024-04-24T23:20:54.479897+0000","last_became_peered":"2024-04-24T23:20:54.479897+0000","last_unstale":"2024-04-24T23:20:56.477976+0000","last_undegraded":"2024-04-24T23:20:56.477976+0000","last_fullsized":"2024-04-24T23:20:56.477976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725278+0000","last_change":"2024-04-24T23:20:55.480538+0000","last_active":"2024-04-24T23:20:56.725278+0000","last_peered":"2024-04-24T23:20:56.725278+0000","last_clean":"2024-04-24T23:20:56.725278+0000","last_became_active":"2024-04-24T23:20:55.474763+0000","last_became_peered":"2024-04-24T23:20:55.474763+0000","last_unstale":"2024-04-24T23:20:56.725278+0000","last_undegraded":"2024-04-24T23:20:56.725278+0000","last_fullsized":"2024-04-24T23:20:56.725278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:42.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.477945+0000","last_change":"2024-04-24T23:20:54.475154+0000","last_active":"2024-04-24T23:20:56.477945+0000","last_peered":"2024-04-24T23:20:56.477945+0000","last_clean":"2024-04-24T23:20:56.477945+0000","last_became_active":"2024-04-24T23:20:54.473925+0000","last_became_peered":"2024-04-24T23:20:54.473925+0000","last_unstale":"2024-04-24T23:20:56.477945+0000","last_undegraded":"2024-04-24T23:20:56.477945+0000","last_fullsized":"2024-04-24T23:20:56.477945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:10.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725566+0000","last_change":"2024-04-24T23:20:55.479171+0000","last_active":"2024-04-24T23:20:56.725566+0000","last_peered":"2024-04-24T23:20:56.725566+0000","last_clean":"2024-04-24T23:20:56.725566+0000","last_became_active":"2024-04-24T23:20:55.478856+0000","last_became_peered":"2024-04-24T23:20:55.478856+0000","last_unstale":"2024-04-24T23:20:56.725566+0000","last_undegraded":"2024-04-24T23:20:56.725566+0000","last_fullsized":"2024-04-24T23:20:56.725566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.630308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725243+0000","last_change":"2024-04-24T23:20:54.484143+0000","last_active":"2024-04-24T23:20:56.725243+0000","last_peered":"2024-04-24T23:20:56.725243+0000","last_clean":"2024-04-24T23:20:56.725243+0000","last_became_active":"2024-04-24T23:20:54.483830+0000","last_became_peered":"2024-04-24T23:20:54.483830+0000","last_unstale":"2024-04-24T23:20:56.725243+0000","last_undegraded":"2024-04-24T23:20:56.725243+0000","last_fullsized":"2024-04-24T23:20:56.725243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:31.637239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.480446+0000","last_change":"2024-04-24T23:20:55.479005+0000","last_active":"2024-04-24T23:20:56.480446+0000","last_peered":"2024-04-24T23:20:56.480446+0000","last_clean":"2024-04-24T23:20:56.480446+0000","last_became_active":"2024-04-24T23:20:55.478706+0000","last_became_peered":"2024-04-24T23:20:55.478706+0000","last_unstale":"2024-04-24T23:20:56.480446+0000","last_undegraded":"2024-04-24T23:20:56.480446+0000","last_fullsized":"2024-04-24T23:20:56.480446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.474978+0000","last_change":"2024-04-24T23:20:55.469771+0000","last_active":"2024-04-24T23:20:56.474978+0000","last_peered":"2024-04-24T23:20:56.474978+0000","last_clean":"2024-04-24T23:20:56.474978+0000","last_became_active":"2024-04-24T23:20:55.469411+0000","last_became_peered":"2024-04-24T23:20:55.469411+0000","last_unstale":"2024-04-24T23:20:56.474978+0000","last_undegraded":"2024-04-24T23:20:56.474978+0000","last_fullsized":"2024-04-24T23:20:56.474978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:43.706258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725118+0000","last_change":"2024-04-24T23:20:54.485231+0000","last_active":"2024-04-24T23:20:56.725118+0000","last_peered":"2024-04-24T23:20:56.725118+0000","last_clean":"2024-04-24T23:20:56.725118+0000","last_became_active":"2024-04-24T23:20:54.484912+0000","last_became_peered":"2024-04-24T23:20:54.484912+0000","last_unstale":"2024-04-24T23:20:56.725118+0000","last_undegraded":"2024-04-24T23:20:56.725118+0000","last_fullsized":"2024-04-24T23:20:56.725118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:27.147159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.476107+0000","last_change":"2024-04-24T23:20:55.473157+0000","last_active":"2024-04-24T23:20:56.476107+0000","last_peered":"2024-04-24T23:20:56.476107+0000","last_clean":"2024-04-24T23:20:56.476107+0000","last_became_active":"2024-04-24T23:20:55.472854+0000","last_became_peered":"2024-04-24T23:20:55.472854+0000","last_unstale":"2024-04-24T23:20:56.476107+0000","last_undegraded":"2024-04-24T23:20:56.476107+0000","last_fullsized":"2024-04-24T23:20:56.476107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:53.598933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725345+0000","last_change":"2024-04-24T23:20:54.485353+0000","last_active":"2024-04-24T23:20:56.725345+0000","last_peered":"2024-04-24T23:20:56.725345+0000","last_clean":"2024-04-24T23:20:56.725345+0000","last_became_active":"2024-04-24T23:20:54.485065+0000","last_became_peered":"2024-04-24T23:20:54.485065+0000","last_unstale":"2024-04-24T23:20:56.725345+0000","last_undegraded":"2024-04-24T23:20:56.725345+0000","last_fullsized":"2024-04-24T23:20:56.725345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:04.440431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724430+0000","last_change":"2024-04-24T23:20:55.487656+0000","last_active":"2024-04-24T23:20:56.724430+0000","last_peered":"2024-04-24T23:20:56.724430+0000","last_clean":"2024-04-24T23:20:56.724430+0000","last_became_active":"2024-04-24T23:20:55.487317+0000","last_became_peered":"2024-04-24T23:20:55.487317+0000","last_unstale":"2024-04-24T23:20:56.724430+0000","last_undegraded":"2024-04-24T23:20:56.724430+0000","last_fullsized":"2024-04-24T23:20:56.724430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:19.255769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725671+0000","last_change":"2024-04-24T23:20:54.475528+0000","last_active":"2024-04-24T23:20:56.725671+0000","last_peered":"2024-04-24T23:20:56.725671+0000","last_clean":"2024-04-24T23:20:56.725671+0000","last_became_active":"2024-04-24T23:20:54.475227+0000","last_became_peered":"2024-04-24T23:20:54.475227+0000","last_unstale":"2024-04-24T23:20:56.725671+0000","last_undegraded":"2024-04-24T23:20:56.725671+0000","last_fullsized":"2024-04-24T23:20:56.725671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:39.564395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.725535+0000","last_change":"2024-04-24T23:20:55.481689+0000","last_active":"2024-04-24T23:20:56.725535+0000","last_peered":"2024-04-24T23:20:56.725535+0000","last_clean":"2024-04-24T23:20:56.725535+0000","last_became_active":"2024-04-24T23:20:55.481405+0000","last_became_peered":"2024-04-24T23:20:55.481405+0000","last_unstale":"2024-04-24T23:20:56.725535+0000","last_undegraded":"2024-04-24T23:20:56.725535+0000","last_fullsized":"2024-04-24T23:20:56.725535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:54.453567+0000","last_clean_scrub_stamp":"2024-04-24T23:20:54.453567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:28.460824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:20:56.724580+0000","last_change":"2024-04-24T23:20:54.482023+0000","last_active":"2024-04-24T23:20:56.724580+0000","last_peered":"2024-04-24T23:20:56.724580+0000","last_clean":"2024-04-24T23:20:56.724580+0000","last_became_active":"2024-04-24T23:20:54.481494+0000","last_became_peered":"2024-04-24T23:20:54.481494+0000","last_unstale":"2024-04-24T23:20:56.724580+0000","last_undegraded":"2024-04-24T23:20:56.724580+0000","last_fullsized":"2024-04-24T23:20:56.724580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:53.449304+0000","last_clean_scrub_stamp":"2024-04-24T23:20:53.449304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:07.429756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:21:06.347 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T23:21:06.347 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T23:21:06.347 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-24T23:21:06.347 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T23:21:06.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.482+0000 7f12b8b29640 1 Processor -- start 2024-04-24T23:21:06.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.482+0000 7f12b8b29640 1 -- start start 2024-04-24T23:21:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b8b29640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b4145a80 0x7f12b40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b8b29640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12b415cee0 con 0x7f12b415c5b0 2024-04-24T23:21:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b8b29640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12b415d1c0 con 0x7f12b406dc40 2024-04-24T23:21:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b8b29640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f12b40b3300 con 0x7f12b4145a80 2024-04-24T23:21:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12abfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f12b415c5b0 0x7f12b406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54684/0 (socket says 172.21.15.26:54684) 2024-04-24T23:21:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12abfff640 1 -- 172.21.15.26:0/223046343 learned_addr learned my addr 172.21.15.26:0/223046343 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b259c640 1 --2- 172.21.15.26:0/223046343 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b4145a80 0x7f12b40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b1d9b640 1 -- 172.21.15.26:0/223046343 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 740052999 0 0) 0x7f12b415cee0 con 0x7f12b415c5b0 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b1d9b640 1 -- 172.21.15.26:0/223046343 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12a0003520 con 0x7f12b415c5b0 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b1d9b640 1 -- 172.21.15.26:0/223046343 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3764794169 0 0) 0x7f12b415d1c0 con 0x7f12b406dc40 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b1d9b640 1 -- 172.21.15.26:0/223046343 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12b415cee0 con 0x7f12b406dc40 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b259c640 1 -- 172.21.15.26:0/223046343 >> v1:172.21.15.29:6789/0 conn(0x7f12b406dc40 legacy=0x7f12b40b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b259c640 1 -- 172.21.15.26:0/223046343 >> v1:172.21.15.26:6789/0 conn(0x7f12b415c5b0 legacy=0x7f12b406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b1d9b640 1 -- 172.21.15.26:0/223046343 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 990195391 0 0) 0x7f12a0003520 con 0x7f12b415c5b0 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.483+0000 7f12b259c640 1 -- 172.21.15.26:0/223046343 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b415d4a0 con 0x7f12b4145a80 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.484+0000 7f12b259c640 1 --2- 172.21.15.26:0/223046343 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b4145a80 0x7f12b40ce040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009800 tx=0x7f12a402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff51ad249942cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.484+0000 7f12b1d9b640 1 -- 172.21.15.26:0/223046343 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4037070 con 0x7f12b4145a80 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.484+0000 7f12b1d9b640 1 -- 172.21.15.26:0/223046343 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a402ac50 con 0x7f12b4145a80 2024-04-24T23:21:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.484+0000 7f12b1d9b640 1 -- 172.21.15.26:0/223046343 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c001670 con 0x7f12b4145a80 2024-04-24T23:21:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.485+0000 7f12b8b29640 1 -- 172.21.15.26:0/223046343 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b4145a80 msgr2=0x7f12b40ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.485+0000 7f12b8b29640 1 --2- 172.21.15.26:0/223046343 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b4145a80 0x7f12b40ce040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009800 tx=0x7f12a402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.485+0000 7f12b8b29640 1 -- 172.21.15.26:0/223046343 shutdown_connections 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.485+0000 7f12b8b29640 1 --2- 172.21.15.26:0/223046343 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b4145a80 0x7f12b40ce040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.485+0000 7f12b8b29640 1 -- 172.21.15.26:0/223046343 >> 172.21.15.26:0/223046343 conn(0x7f12b406b540 msgr2=0x7f12b406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.485+0000 7f12b8b29640 1 -- 172.21.15.26:0/223046343 shutdown_connections 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.485+0000 7f12b8b29640 1 -- 172.21.15.26:0/223046343 wait complete. 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.486+0000 7f12b8b29640 1 Processor -- start 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.486+0000 7f12b8b29640 1 -- start start 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b8b29640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b406dc40 0x7f12b40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b8b29640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b4145a80 0x7f12b415b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b8b29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b415c5b0 0x7f12b40c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b8b29640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12b415cb90 con 0x7f12b415c5b0 2024-04-24T23:21:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b8b29640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b415ca10 con 0x7f12b4145a80 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b8b29640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f12b40b3ea0 con 0x7f12b406dc40 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b415c5b0 0x7f12b40c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b415c5b0 0x7f12b40c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40136/0 (socket says 172.21.15.26:40136) 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b2d9d640 1 -- 172.21.15.26:0/667621902 learned_addr learned my addr 172.21.15.26:0/667621902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12abfff640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b4145a80 0x7f12b415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b259c640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b406dc40 0x7f12b40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b2d9d640 1 -- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b406dc40 msgr2=0x7f12b40ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b2d9d640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b406dc40 0x7f12b40ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b2d9d640 1 -- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b4145a80 msgr2=0x7f12b415b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b2d9d640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b4145a80 0x7f12b415b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b2d9d640 1 -- 172.21.15.26:0/667621902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b41313c0 con 0x7f12b415c5b0 2024-04-24T23:21:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12b259c640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b406dc40 0x7f12b40ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.487+0000 7f12abfff640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b4145a80 0x7f12b415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.488+0000 7f12b2d9d640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b415c5b0 0x7f12b40c4e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c7a0 tx=0x7f12ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.488+0000 7f12aaffd640 1 -- 172.21.15.26:0/667621902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac003b70 con 0x7f12b415c5b0 2024-04-24T23:21:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.488+0000 7f12aaffd640 1 -- 172.21.15.26:0/667621902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ac003d10 con 0x7f12b415c5b0 2024-04-24T23:21:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.488+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b4131680 con 0x7f12b415c5b0 2024-04-24T23:21:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.488+0000 7f12aaffd640 1 -- 172.21.15.26:0/667621902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac00aa90 con 0x7f12b415c5b0 2024-04-24T23:21:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.488+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b4131c40 con 0x7f12b415c5b0 2024-04-24T23:21:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.489+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b40694c0 con 0x7f12b415c5b0 2024-04-24T23:21:06.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.489+0000 7f12aaffd640 1 -- 172.21.15.26:0/667621902 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12ac00ac30 con 0x7f12b415c5b0 2024-04-24T23:21:06.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.490+0000 7f12aaffd640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f129406be60 0x7f129406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.490+0000 7f12b259c640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f129406be60 0x7f129406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.490+0000 7f12aaffd640 1 -- 172.21.15.26:0/667621902 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f12ac08aee0 con 0x7f12b415c5b0 2024-04-24T23:21:06.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.490+0000 7f12b259c640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f129406be60 0x7f129406e320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12a4034000 tx=0x7f12a402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.494+0000 7f12aaffd640 1 -- 172.21.15.26:0/667621902 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f12ac059770 con 0x7f12b415c5b0 2024-04-24T23:21:06.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.712+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f12b40b53f0 con 0x7f12b415c5b0 2024-04-24T23:21:06.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.712+0000 7f12aaffd640 1 -- 172.21.15.26:0/667621902 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f12ac059070 con 0x7f12b415c5b0 2024-04-24T23:21:06.714 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:06.715 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T23:21:06.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f129406be60 msgr2=0x7f129406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f129406be60 0x7f129406e320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12a4034000 tx=0x7f12a402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:06.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b415c5b0 msgr2=0x7f12b40c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b415c5b0 0x7f12b40c4e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c7a0 tx=0x7f12ac00cc60 comp rx=0 tx=0).stop 2024-04-24T23:21:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 shutdown_connections 2024-04-24T23:21:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f129406be60 0x7f129406e320 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b415c5b0 0x7f12b40c4e60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b4145a80 0x7f12b415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 --2- 172.21.15.26:0/667621902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12b406dc40 0x7f12b40ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 >> 172.21.15.26:0/667621902 conn(0x7f12b406b540 msgr2=0x7f12b41400e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 shutdown_connections 2024-04-24T23:21:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.714+0000 7f12b8b29640 1 -- 172.21.15.26:0/667621902 wait complete. 2024-04-24T23:21:06.727 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-24T23:21:06.727 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:06.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.826+0000 7ff5cced4640 1 Processor -- start 2024-04-24T23:21:06.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.826+0000 7ff5cced4640 1 -- start start 2024-04-24T23:21:06.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5cced4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c80b3f50 0x7ff5c80b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5cced4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c80b5840 con 0x7ff5c815e290 2024-04-24T23:21:06.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5cced4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c80b43c0 con 0x7ff5c80b3ac0 2024-04-24T23:21:06.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5cced4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c80b5e50 con 0x7ff5c80b3f50 2024-04-24T23:21:06.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5c659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff5c815e290 0x7ff5c80d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54692/0 (socket says 172.21.15.26:54692) 2024-04-24T23:21:06.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5c659c640 1 -- 172.21.15.26:0/879736406 learned_addr learned my addr 172.21.15.26:0/879736406 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:06.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2723687538 0 0) 0x7ff5c80b5840 con 0x7ff5c815e290 2024-04-24T23:21:06.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5b4003520 con 0x7ff5c815e290 2024-04-24T23:21:06.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5c6d9d640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c80b3f50 0x7ff5c80b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2620371687 0 0) 0x7ff5b4003520 con 0x7ff5c815e290 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c80b3f50 msgr2=0x7ff5c80b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.827+0000 7ff5c5d9b640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c80b3f50 0x7ff5c80b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 >> v1:172.21.15.29:6789/0 conn(0x7ff5c80b3ac0 legacy=0x7ff5c80b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80cfb30 con 0x7ff5c815e290 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7ff5b8001690 con 0x7ff5c815e290 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 >> v1:172.21.15.26:6789/0 conn(0x7ff5c815e290 legacy=0x7ff5c80d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5b40039e0 0x7ff5b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b4004800 0x7ff5b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c80b3f50 0x7ff5b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c80b5e50 con 0x7ff5b40039e0 2024-04-24T23:21:06.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b400ba70 con 0x7ff5b4004800 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b400bc70 con 0x7ff5c80b3f50 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c659c640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5b40039e0 0x7ff5b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5bffff640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b4004800 0x7ff5b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c6d9d640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c80b3f50 0x7ff5b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c659c640 1 -- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c80b3f50 msgr2=0x7ff5b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c659c640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c80b3f50 0x7ff5b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c659c640 1 -- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b4004800 msgr2=0x7ff5b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c659c640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b4004800 0x7ff5b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.828+0000 7ff5c659c640 1 -- 172.21.15.26:0/879736406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b4003520 con 0x7ff5b40039e0 2024-04-24T23:21:06.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5c659c640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5b40039e0 0x7ff5b4003e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c5a60 tx=0x7ff5b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2569b20566a6ae1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8007d50 con 0x7ff5b40039e0 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b8005af0 con 0x7ff5b40039e0 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5c5d9b640 1 -- 172.21.15.26:0/879736406 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8007070 con 0x7ff5b40039e0 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5cced4640 1 -- 172.21.15.26:0/879736406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5b40039e0 msgr2=0x7ff5b4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5cced4640 1 --2- 172.21.15.26:0/879736406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5b40039e0 0x7ff5b4003e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c5a60 tx=0x7ff5b4005830 comp rx=0 tx=0).stop 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5c6d9d640 1 -- 172.21.15.26:0/879736406 reap_dead start 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5cced4640 1 -- 172.21.15.26:0/879736406 shutdown_connections 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5cced4640 1 -- 172.21.15.26:0/879736406 >> 172.21.15.26:0/879736406 conn(0x7ff5c806b120 msgr2=0x7ff5c806b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5cced4640 1 -- 172.21.15.26:0/879736406 shutdown_connections 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.829+0000 7ff5cced4640 1 -- 172.21.15.26:0/879736406 wait complete. 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.831+0000 7ff5cced4640 1 Processor -- start 2024-04-24T23:21:06.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.831+0000 7ff5cced4640 1 -- start start 2024-04-24T23:21:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.831+0000 7ff5cced4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40039e0 0x7ff5c80c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.831+0000 7ff5cced4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5c80c71e0 0x7ff5c815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5cced4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c814e330 0x7ff5c8160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5cced4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c80b5ae0 con 0x7ff5c80c71e0 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5cced4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c80d00f0 con 0x7ff5b40039e0 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5cced4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c815e670 con 0x7ff5c814e330 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5c80c71e0 0x7ff5c815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5c80c71e0 0x7ff5c815fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40158/0 (socket says 172.21.15.26:40158) 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5bffff640 1 -- 172.21.15.26:0/2032115120 learned_addr learned my addr 172.21.15.26:0/2032115120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5c659c640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40039e0 0x7ff5c80c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5c6d9d640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c814e330 0x7ff5c8160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5bffff640 1 -- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c814e330 msgr2=0x7ff5c8160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5bffff640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c814e330 0x7ff5c8160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5bffff640 1 -- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40039e0 msgr2=0x7ff5c80c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5bffff640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40039e0 0x7ff5c80c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5bffff640 1 -- 172.21.15.26:0/2032115120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80c6140 con 0x7ff5c80c71e0 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5c6d9d640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c814e330 0x7ff5c8160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5c659c640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40039e0 0x7ff5c80c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5bffff640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5c80c71e0 0x7ff5c815fa20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000f620 tx=0x7ff5b000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5beffd640 1 -- 172.21.15.26:0/2032115120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00052e0 con 0x7ff5c80c71e0 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5beffd640 1 -- 172.21.15.26:0/2032115120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b0018070 con 0x7ff5c80c71e0 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c80c6420 con 0x7ff5c80c71e0 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5beffd640 1 -- 172.21.15.26:0/2032115120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0003810 con 0x7ff5c80c71e0 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.832+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c80c69e0 con 0x7ff5c80c71e0 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.833+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c80694c0 con 0x7ff5c80c71e0 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.834+0000 7ff5beffd640 1 -- 172.21.15.26:0/2032115120 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5b00039b0 con 0x7ff5c80c71e0 2024-04-24T23:21:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.834+0000 7ff5beffd640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5a806be60 0x7ff5a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.834+0000 7ff5beffd640 1 -- 172.21.15.26:0/2032115120 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5b008ab40 con 0x7ff5c80c71e0 2024-04-24T23:21:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.834+0000 7ff5c659c640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5a806be60 0x7ff5a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.835+0000 7ff5c659c640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5a806be60 0x7ff5a806e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8032000 tx=0x7ff5b802d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:06.839+0000 7ff5beffd640 1 -- 172.21.15.26:0/2032115120 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff5b00594e0 con 0x7ff5c80c71e0 2024-04-24T23:21:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.021+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5c8070430 con 0x7ff5c80c71e0 2024-04-24T23:21:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.022+0000 7ff5beffd640 1 -- 172.21.15.26:0/2032115120 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7ff5b0058de0 con 0x7ff5c80c71e0 2024-04-24T23:21:07.024 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:07.025 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5a806be60 msgr2=0x7ff5a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5a806be60 0x7ff5a806e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8032000 tx=0x7ff5b802d000 comp rx=0 tx=0).stop 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5c80c71e0 msgr2=0x7ff5c815fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5c80c71e0 0x7ff5c815fa20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000f620 tx=0x7ff5b000fae0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 shutdown_connections 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5a806be60 0x7ff5a806e320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5c814e330 0x7ff5c8160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5c80c71e0 0x7ff5c815fa20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 --2- 172.21.15.26:0/2032115120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5b40039e0 0x7ff5c80c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 >> 172.21.15.26:0/2032115120 conn(0x7ff5c806b120 msgr2=0x7ff5c80d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:07.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 shutdown_connections 2024-04-24T23:21:07.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.025+0000 7ff5cced4640 1 -- 172.21.15.26:0/2032115120 wait complete. 2024-04-24T23:21:07.038 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:07.038 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.140+0000 7fa0f0b70640 1 Processor -- start 2024-04-24T23:21:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.140+0000 7fa0f0b70640 1 -- start start 2024-04-24T23:21:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.141+0000 7fa0f0b70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec15e610 0x7fa0ec0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.141+0000 7fa0f0b70640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0ec0b3110 0x7fa0ec0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.141+0000 7fa0f0b70640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0ec0d2950 con 0x7fa0ec0cfce0 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.141+0000 7fa0f0b70640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ec0d2c30 con 0x7fa0ec0b3110 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.141+0000 7fa0f0b70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec0d2f10 con 0x7fa0ec15e610 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.141+0000 7fa0e9d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0ec0b3110 0x7fa0ec0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.141+0000 7fa0ead9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa0ec0cfce0 0x7fa0ec0d0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54694/0 (socket says 172.21.15.26:54694) 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.141+0000 7fa0ead9d640 1 -- 172.21.15.26:0/554005652 learned_addr learned my addr 172.21.15.26:0/554005652 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.141+0000 7fa0ea59c640 1 --2- 172.21.15.26:0/554005652 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec15e610 0x7fa0ec0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.142+0000 7fa0e9d9b640 1 -- 172.21.15.26:0/554005652 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec15e610 msgr2=0x7fa0ec0b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.142+0000 7fa0e9d9b640 1 --2- 172.21.15.26:0/554005652 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec15e610 0x7fa0ec0b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.142+0000 7fa0e9d9b640 1 -- 172.21.15.26:0/554005652 >> v1:172.21.15.26:6789/0 conn(0x7fa0ec0cfce0 legacy=0x7fa0ec0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.142+0000 7fa0e959a640 1 -- 172.21.15.26:0/554005652 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280277628 0 0) 0x7fa0ec0d2950 con 0x7fa0ec0cfce0 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.142+0000 7fa0e9d9b640 1 -- 172.21.15.26:0/554005652 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec0cefa0 con 0x7fa0ec0b3110 2024-04-24T23:21:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.142+0000 7fa0ea59c640 1 --2- 172.21.15.26:0/554005652 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec15e610 0x7fa0ec0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.142+0000 7fa0e9d9b640 1 --2- 172.21.15.26:0/554005652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec0b3110 0x7fa0ec0cf7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40097a0 tx=0x7fa0d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3faa9d9ab63215b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.142+0000 7fa0e959a640 1 -- 172.21.15.26:0/554005652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4037070 con 0x7fa0ec0b3110 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0e959a640 1 -- 172.21.15.26:0/554005652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d4032440 con 0x7fa0ec0b3110 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0e959a640 1 -- 172.21.15.26:0/554005652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4032810 con 0x7fa0ec0b3110 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0f0b70640 1 -- 172.21.15.26:0/554005652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec0b3110 msgr2=0x7fa0ec0cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0f0b70640 1 --2- 172.21.15.26:0/554005652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec0b3110 0x7fa0ec0cf7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40097a0 tx=0x7fa0d402a180 comp rx=0 tx=0).stop 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0f0b70640 1 -- 172.21.15.26:0/554005652 shutdown_connections 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0f0b70640 1 --2- 172.21.15.26:0/554005652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec0b3110 0x7fa0ec0cf7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0f0b70640 1 --2- 172.21.15.26:0/554005652 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec15e610 0x7fa0ec0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0f0b70640 1 -- 172.21.15.26:0/554005652 >> 172.21.15.26:0/554005652 conn(0x7fa0ec0d4040 msgr2=0x7fa0ec0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0f0b70640 1 -- 172.21.15.26:0/554005652 shutdown_connections 2024-04-24T23:21:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.143+0000 7fa0f0b70640 1 -- 172.21.15.26:0/554005652 wait complete. 2024-04-24T23:21:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.144+0000 7fa0f0b70640 1 Processor -- start 2024-04-24T23:21:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0f0b70640 1 -- start start 2024-04-24T23:21:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0f0b70640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0ec0b3110 0x7fa0ec15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0f0b70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec0cfce0 0x7fa0ec0b9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0f0b70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec15e610 0x7fa0ec156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0f0b70640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec0d0140 con 0x7fa0ec15e610 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0f0b70640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec0cec50 con 0x7fa0ec0b3110 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0f0b70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec0d02c0 con 0x7fa0ec0cfce0 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0ead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec15e610 0x7fa0ec156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0ead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec15e610 0x7fa0ec156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40174/0 (socket says 172.21.15.26:40174) 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.145+0000 7fa0ead9d640 1 -- 172.21.15.26:0/905645691 learned_addr learned my addr 172.21.15.26:0/905645691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0ea59c640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0ec0b3110 0x7fa0ec15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0e9d9b640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec0cfce0 0x7fa0ec0b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0ead9d640 1 -- 172.21.15.26:0/905645691 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec0cfce0 msgr2=0x7fa0ec0b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0ead9d640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec0cfce0 0x7fa0ec0b9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0ead9d640 1 -- 172.21.15.26:0/905645691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0ec0b3110 msgr2=0x7fa0ec15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0ead9d640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0ec0b3110 0x7fa0ec15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0ead9d640 1 -- 172.21.15.26:0/905645691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec15f080 con 0x7fa0ec15e610 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0ea59c640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0ec0b3110 0x7fa0ec15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0ead9d640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec15e610 0x7fa0ec156f30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00c7a0 tx=0x7fa0dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0e2ffd640 1 -- 172.21.15.26:0/905645691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc018070 con 0x7fa0ec15e610 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0e2ffd640 1 -- 172.21.15.26:0/905645691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0dc00a640 con 0x7fa0ec15e610 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec15f360 con 0x7fa0ec15e610 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0e2ffd640 1 -- 172.21.15.26:0/905645691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc00aab0 con 0x7fa0ec15e610 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.146+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec15f920 con 0x7fa0ec15e610 2024-04-24T23:21:07.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.147+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec075610 con 0x7fa0ec15e610 2024-04-24T23:21:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.147+0000 7fa0e2ffd640 1 -- 172.21.15.26:0/905645691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa0dc01b070 con 0x7fa0ec15e610 2024-04-24T23:21:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.148+0000 7fa0e2ffd640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0cc06be60 0x7fa0cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.148+0000 7fa0e2ffd640 1 -- 172.21.15.26:0/905645691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa0dc08a470 con 0x7fa0ec15e610 2024-04-24T23:21:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.148+0000 7fa0ea59c640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0cc06be60 0x7fa0cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.148+0000 7fa0ea59c640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0cc06be60 0x7fa0cc06e320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec06dad0 tx=0x7fa0e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.152+0000 7fa0e2ffd640 1 -- 172.21.15.26:0/905645691 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa0dc058e20 con 0x7fa0ec15e610 2024-04-24T23:21:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.333+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0ec0ce570 con 0x7fa0ec15e610 2024-04-24T23:21:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.334+0000 7fa0e2ffd640 1 -- 172.21.15.26:0/905645691 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa0dc058720 con 0x7fa0ec15e610 2024-04-24T23:21:07.336 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0cc06be60 msgr2=0x7fa0cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0cc06be60 0x7fa0cc06e320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec06dad0 tx=0x7fa0e4008040 comp rx=0 tx=0).stop 2024-04-24T23:21:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec15e610 msgr2=0x7fa0ec156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec15e610 0x7fa0ec156f30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00c7a0 tx=0x7fa0dc00cc60 comp rx=0 tx=0).stop 2024-04-24T23:21:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 shutdown_connections 2024-04-24T23:21:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0cc06be60 0x7fa0cc06e320 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0ec15e610 0x7fa0ec156f30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0ec0cfce0 0x7fa0ec0b9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 --2- 172.21.15.26:0/905645691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0ec0b3110 0x7fa0ec15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 >> 172.21.15.26:0/905645691 conn(0x7fa0ec0d4040 msgr2=0x7fa0ec162210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 shutdown_connections 2024-04-24T23:21:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.336+0000 7fa0f0b70640 1 -- 172.21.15.26:0/905645691 wait complete. 2024-04-24T23:21:07.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:07.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.450+0000 7fa66b858640 1 Processor -- start 2024-04-24T23:21:07.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.450+0000 7fa66b858640 1 -- start start 2024-04-24T23:21:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.451+0000 7fa66b858640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa66415e370 0x7fa6640d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.451+0000 7fa66b858640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6640d4c10 0x7fa6640cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.451+0000 7fa66b858640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6640d2120 con 0x7fa66415e750 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.451+0000 7fa66b858640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa6640d1510 con 0x7fa6640d4c10 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.451+0000 7fa66b858640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa6640d1ed0 con 0x7fa66415e370 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.451+0000 7fa668df3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa66415e750 0x7fa6640b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54710/0 (socket says 172.21.15.26:54710) 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.451+0000 7fa668df3640 1 -- 172.21.15.26:0/340672111 learned_addr learned my addr 172.21.15.26:0/340672111 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa6695f4640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa66415e370 0x7fa6640d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa669df5640 1 --2- 172.21.15.26:0/340672111 >> v2:172.21.15.29:3300/0 conn(0x7fa6640d4c10 0x7fa6640cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1862945201 0 0) 0x7fa6640d2120 con 0x7fa66415e750 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa650003520 con 0x7fa66415e750 2024-04-24T23:21:07.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3423646374 0 0) 0x7fa650003520 con 0x7fa66415e750 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa66415e370 msgr2=0x7fa6640d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa66415e370 0x7fa6640d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 >> v2:172.21.15.29:3300/0 conn(0x7fa6640d4c10 msgr2=0x7fa6640cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 --2- 172.21.15.26:0/340672111 >> v2:172.21.15.29:3300/0 conn(0x7fa6640d4c10 0x7fa6640cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6640b3740 con 0x7fa66415e750 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fa654001670 con 0x7fa66415e750 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 >> v1:172.21.15.26:6789/0 conn(0x7fa66415e750 legacy=0x7fa6640b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa66415e370 0x7fa6500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa650003f30 0x7fa650004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa650004d40 0x7fa6500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6640d1ed0 con 0x7fa650004d40 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa65000b9c0 con 0x7fa650003f30 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa65000bb40 con 0x7fa66415e370 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa669df5640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa650004d40 0x7fa6500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa6695f4640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa66415e370 0x7fa6500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa668df3640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa650003f30 0x7fa650004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa669df5640 1 -- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa66415e370 msgr2=0x7fa6500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa669df5640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa66415e370 0x7fa6500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa669df5640 1 -- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa650003f30 msgr2=0x7fa650004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa669df5640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa650003f30 0x7fa650004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa669df5640 1 -- 172.21.15.26:0/340672111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa650003520 con 0x7fa650004d40 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.452+0000 7fa668df3640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa650003f30 0x7fa650004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:07.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.453+0000 7fa669df5640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa650004d40 0x7fa6500051c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009fc0 tx=0x7fa650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717d4c03ec2086d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.453+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c009c10 con 0x7fa650004d40 2024-04-24T23:21:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.453+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65c036070 con 0x7fa650004d40 2024-04-24T23:21:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.453+0000 7fa65bfff640 1 -- 172.21.15.26:0/340672111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c02d070 con 0x7fa650004d40 2024-04-24T23:21:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.454+0000 7fa66b858640 1 -- 172.21.15.26:0/340672111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa650004d40 msgr2=0x7fa6500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.454+0000 7fa66b858640 1 --2- 172.21.15.26:0/340672111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa650004d40 0x7fa6500051c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009fc0 tx=0x7fa650005830 comp rx=0 tx=0).stop 2024-04-24T23:21:07.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.454+0000 7fa669df5640 1 -- 172.21.15.26:0/340672111 reap_dead start 2024-04-24T23:21:07.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.454+0000 7fa66b858640 1 -- 172.21.15.26:0/340672111 shutdown_connections 2024-04-24T23:21:07.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.454+0000 7fa66b858640 1 -- 172.21.15.26:0/340672111 >> 172.21.15.26:0/340672111 conn(0x7fa6640d40f0 msgr2=0x7fa6640d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:07.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.454+0000 7fa66b858640 1 -- 172.21.15.26:0/340672111 shutdown_connections 2024-04-24T23:21:07.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.454+0000 7fa66b858640 1 -- 172.21.15.26:0/340672111 wait complete. 2024-04-24T23:21:07.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.455+0000 7fa66b858640 1 Processor -- start 2024-04-24T23:21:07.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.455+0000 7fa66b858640 1 -- start start 2024-04-24T23:21:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa66b858640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa66415c710 0x7fa6641f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa66b858640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6641e8680 0x7fa66406cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa66b858640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa66415d4f0 0x7fa66406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa66b858640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6640d2520 con 0x7fa66415c710 2024-04-24T23:21:07.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa66b858640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa6640b3d00 con 0x7fa66415d4f0 2024-04-24T23:21:07.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa66b858640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa6640d26a0 con 0x7fa6641e8680 2024-04-24T23:21:07.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa6695f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa66415c710 0x7fa6641f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa6695f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa66415c710 0x7fa6641f4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40190/0 (socket says 172.21.15.26:40190) 2024-04-24T23:21:07.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa6695f4640 1 -- 172.21.15.26:0/4140780464 learned_addr learned my addr 172.21.15.26:0/4140780464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:07.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa669df5640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa66415d4f0 0x7fa66406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa668df3640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6641e8680 0x7fa66406cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa6695f4640 1 -- 172.21.15.26:0/4140780464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6641e8680 msgr2=0x7fa66406cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa6695f4640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6641e8680 0x7fa66406cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa6695f4640 1 -- 172.21.15.26:0/4140780464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa66415d4f0 msgr2=0x7fa66406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa6695f4640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa66415d4f0 0x7fa66406d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa6695f4640 1 -- 172.21.15.26:0/4140780464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66415d2a0 con 0x7fa66415c710 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa6695f4640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa66415c710 0x7fa6641f4170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6600119a0 tx=0x7fa660011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa659ffb640 1 -- 172.21.15.26:0/4140780464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66000fa90 con 0x7fa66415c710 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa659ffb640 1 -- 172.21.15.26:0/4140780464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa66000fc30 con 0x7fa66415c710 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.456+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66415cfc0 con 0x7fa66415c710 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.457+0000 7fa659ffb640 1 -- 172.21.15.26:0/4140780464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660014440 con 0x7fa66415c710 2024-04-24T23:21:07.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.457+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6640cbf50 con 0x7fa66415c710 2024-04-24T23:21:07.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.458+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa664075610 con 0x7fa66415c710 2024-04-24T23:21:07.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.458+0000 7fa659ffb640 1 -- 172.21.15.26:0/4140780464 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa660012040 con 0x7fa66415c710 2024-04-24T23:21:07.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.459+0000 7fa659ffb640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa64806be30 0x7fa64806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.459+0000 7fa659ffb640 1 -- 172.21.15.26:0/4140780464 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa66008b3b0 con 0x7fa66415c710 2024-04-24T23:21:07.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.459+0000 7fa668df3640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa64806be30 0x7fa64806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.459+0000 7fa668df3640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa64806be30 0x7fa64806e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa6641e4e50 tx=0x7fa654007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.463+0000 7fa659ffb640 1 -- 172.21.15.26:0/4140780464 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa660019030 con 0x7fa66415c710 2024-04-24T23:21:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.643+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa66406ca40 con 0x7fa66415c710 2024-04-24T23:21:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.644+0000 7fa659ffb640 1 -- 172.21.15.26:0/4140780464 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa660059700 con 0x7fa66415c710 2024-04-24T23:21:07.646 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa64806be30 msgr2=0x7fa64806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa64806be30 0x7fa64806e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa6641e4e50 tx=0x7fa654007040 comp rx=0 tx=0).stop 2024-04-24T23:21:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa66415c710 msgr2=0x7fa6641f4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa66415c710 0x7fa6641f4170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6600119a0 tx=0x7fa660011e60 comp rx=0 tx=0).stop 2024-04-24T23:21:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 shutdown_connections 2024-04-24T23:21:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa64806be30 0x7fa64806e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa66415d4f0 0x7fa66406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6641e8680 0x7fa66406cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 --2- 172.21.15.26:0/4140780464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa66415c710 0x7fa6641f4170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 >> 172.21.15.26:0/4140780464 conn(0x7fa6640d40f0 msgr2=0x7fa6640b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 shutdown_connections 2024-04-24T23:21:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.646+0000 7fa66b858640 1 -- 172.21.15.26:0/4140780464 wait complete. 2024-04-24T23:21:07.658 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.759+0000 7f4ddf412640 1 Processor -- start 2024-04-24T23:21:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.760+0000 7f4ddf412640 1 -- start start 2024-04-24T23:21:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.760+0000 7f4ddf412640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4dd815e750 0x7f4dd80cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.760+0000 7f4ddf412640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd80d2120 con 0x7f4dd80d2c30 2024-04-24T23:21:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.760+0000 7f4ddf412640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd80b3740 con 0x7f4dd815e370 2024-04-24T23:21:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.760+0000 7f4ddf412640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd80d1510 con 0x7f4dd815e750 2024-04-24T23:21:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.760+0000 7f4ddc9ad640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4dd815e750 0x7f4dd80cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.760+0000 7f4ddd9af640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4dd80d2c30 0x7f4dd80cf1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54712/0 (socket says 172.21.15.26:54712) 2024-04-24T23:21:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.760+0000 7f4ddd9af640 1 -- 172.21.15.26:0/3880666160 learned_addr learned my addr 172.21.15.26:0/3880666160 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.761+0000 7f4ddc9ad640 1 -- 172.21.15.26:0/3880666160 >> v1:172.21.15.29:6789/0 conn(0x7f4dd815e370 legacy=0x7f4dd80d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.761+0000 7f4ddc9ad640 1 -- 172.21.15.26:0/3880666160 >> v1:172.21.15.26:6789/0 conn(0x7f4dd80d2c30 legacy=0x7f4dd80cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.761+0000 7f4ddc9ad640 1 -- 172.21.15.26:0/3880666160 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd80b3a20 con 0x7f4dd815e750 2024-04-24T23:21:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.761+0000 7f4dd7fff640 1 -- 172.21.15.26:0/3880666160 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244573107 0 0) 0x7f4dd80d2120 con 0x7f4dd80d2c30 2024-04-24T23:21:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.761+0000 7f4ddc9ad640 1 --2- 172.21.15.26:0/3880666160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e750 0x7f4dd80cec90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0097a0 tx=0x7f4dcc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87953ac88e39ee13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4dd7fff640 1 -- 172.21.15.26:0/3880666160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc037070 con 0x7f4dd815e750 2024-04-24T23:21:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4dd7fff640 1 -- 172.21.15.26:0/3880666160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dcc032440 con 0x7f4dd815e750 2024-04-24T23:21:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4dd7fff640 1 -- 172.21.15.26:0/3880666160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc032810 con 0x7f4dd815e750 2024-04-24T23:21:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4ddf412640 1 -- 172.21.15.26:0/3880666160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e750 msgr2=0x7f4dd80cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4ddf412640 1 --2- 172.21.15.26:0/3880666160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e750 0x7f4dd80cec90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0097a0 tx=0x7f4dcc02a230 comp rx=0 tx=0).stop 2024-04-24T23:21:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4ddf412640 1 -- 172.21.15.26:0/3880666160 shutdown_connections 2024-04-24T23:21:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4ddf412640 1 --2- 172.21.15.26:0/3880666160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e750 0x7f4dd80cec90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4ddf412640 1 -- 172.21.15.26:0/3880666160 >> 172.21.15.26:0/3880666160 conn(0x7f4dd80d40f0 msgr2=0x7f4dd80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4ddf412640 1 -- 172.21.15.26:0/3880666160 shutdown_connections 2024-04-24T23:21:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.762+0000 7f4ddf412640 1 -- 172.21.15.26:0/3880666160 wait complete. 2024-04-24T23:21:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.763+0000 7f4ddf412640 1 Processor -- start 2024-04-24T23:21:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.763+0000 7f4ddf412640 1 -- start start 2024-04-24T23:21:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddf412640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4dd80d2c30 0x7f4dd8153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddf412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e370 0x7f4dd80b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddf412640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4dd80b25f0 0x7f4dd80b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddf412640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd80b5230 con 0x7f4dd815e370 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddf412640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd80d1ed0 con 0x7f4dd80b25f0 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddf412640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd80b53b0 con 0x7f4dd80d2c30 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddc9ad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e370 0x7f4dd80b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddc9ad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e370 0x7f4dd80b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40206/0 (socket says 172.21.15.26:40206) 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddc9ad640 1 -- 172.21.15.26:0/2405081126 learned_addr learned my addr 172.21.15.26:0/2405081126 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddd9af640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4dd80b25f0 0x7f4dd80b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.764+0000 7f4ddd1ae640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4dd80d2c30 0x7f4dd8153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4ddc9ad640 1 -- 172.21.15.26:0/2405081126 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4dd80d2c30 msgr2=0x7f4dd8153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4ddc9ad640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4dd80d2c30 0x7f4dd8153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4ddc9ad640 1 -- 172.21.15.26:0/2405081126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4dd80b25f0 msgr2=0x7f4dd80b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4ddc9ad640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4dd80b25f0 0x7f4dd80b7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4ddc9ad640 1 -- 172.21.15.26:0/2405081126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd80b8cd0 con 0x7f4dd815e370 2024-04-24T23:21:07.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4ddc9ad640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e370 0x7f4dd80b20b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0098c0 tx=0x7f4dcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4dd5ffb640 1 -- 172.21.15.26:0/2405081126 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc037040 con 0x7f4dd815e370 2024-04-24T23:21:07.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4dd5ffb640 1 -- 172.21.15.26:0/2405081126 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dcc042070 con 0x7f4dd815e370 2024-04-24T23:21:07.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd80b8fb0 con 0x7f4dd815e370 2024-04-24T23:21:07.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4dd5ffb640 1 -- 172.21.15.26:0/2405081126 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc030a70 con 0x7f4dd815e370 2024-04-24T23:21:07.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.765+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd80b9570 con 0x7f4dd815e370 2024-04-24T23:21:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.766+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd80b2a00 con 0x7f4dd815e370 2024-04-24T23:21:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.767+0000 7f4dd5ffb640 1 -- 172.21.15.26:0/2405081126 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4dcc030c10 con 0x7f4dd815e370 2024-04-24T23:21:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.767+0000 7f4dd5ffb640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dc406be30 0x7f4dc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.767+0000 7f4ddd1ae640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dc406be30 0x7f4dc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.767+0000 7f4dd5ffb640 1 -- 172.21.15.26:0/2405081126 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4dcc0ab720 con 0x7f4dd815e370 2024-04-24T23:21:07.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.768+0000 7f4ddd1ae640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dc406be30 0x7f4dc406e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4dd81f72d0 tx=0x7f4dc8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.771+0000 7f4dd5ffb640 1 -- 172.21.15.26:0/2405081126 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4dd80b2a00 con 0x7f4dd815e370 2024-04-24T23:21:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.957+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4dd80d4ff0 con 0x7f4dd815e370 2024-04-24T23:21:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.957+0000 7f4dd5ffb640 1 -- 172.21.15.26:0/2405081126 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4dcc079b10 con 0x7f4dd815e370 2024-04-24T23:21:07.959 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dc406be30 msgr2=0x7f4dc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dc406be30 0x7f4dc406e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4dd81f72d0 tx=0x7f4dc8009040 comp rx=0 tx=0).stop 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e370 msgr2=0x7f4dd80b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e370 0x7f4dd80b20b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0098c0 tx=0x7f4dcc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 shutdown_connections 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dc406be30 0x7f4dc406e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4dd80b25f0 0x7f4dd80b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd815e370 0x7f4dd80b20b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 --2- 172.21.15.26:0/2405081126 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4dd80d2c30 0x7f4dd8153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.959+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 >> 172.21.15.26:0/2405081126 conn(0x7f4dd80d40f0 msgr2=0x7f4dd80b5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.960+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 shutdown_connections 2024-04-24T23:21:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:07.960+0000 7f4ddf412640 1 -- 172.21.15.26:0/2405081126 wait complete. 2024-04-24T23:21:07.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:08.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.071+0000 7f3ebe763640 1 Processor -- start 2024-04-24T23:21:08.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.071+0000 7f3ebe763640 1 -- start start 2024-04-24T23:21:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.071+0000 7f3ebe763640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3eb815e380 0x7f3eb80b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.071+0000 7f3ebe763640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb80b5440 con 0x7f3eb80b0030 2024-04-24T23:21:08.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.072+0000 7f3ebe763640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb80b5700 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.072+0000 7f3ebe763640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3eb815e760 con 0x7f3eb815e380 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.072+0000 7f3eb7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3eb815e380 0x7f3eb80b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.072+0000 7f3ebcd00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3eb80b0030 0x7f3eb80b0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54722/0 (socket says 172.21.15.26:54722) 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.072+0000 7f3ebcd00640 1 -- 172.21.15.26:0/745550679 learned_addr learned my addr 172.21.15.26:0/745550679 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.072+0000 7f3eb7fff640 1 -- 172.21.15.26:0/745550679 >> v1:172.21.15.29:6789/0 conn(0x7f3eb80b4fd0 legacy=0x7f3eb8072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.072+0000 7f3eb7fff640 1 -- 172.21.15.26:0/745550679 >> v1:172.21.15.26:6789/0 conn(0x7f3eb80b0030 legacy=0x7f3eb80b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.072+0000 7f3eb7fff640 1 -- 172.21.15.26:0/745550679 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb80b59e0 con 0x7f3eb815e380 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3eb7fff640 1 --2- 172.21.15.26:0/745550679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb815e380 0x7f3eb80b4530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80097a0 tx=0x7f3ea802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65349fdd2f42c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3eb6ffd640 1 -- 172.21.15.26:0/745550679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8037070 con 0x7f3eb815e380 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3eb6ffd640 1 -- 172.21.15.26:0/745550679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea8032640 con 0x7f3eb815e380 2024-04-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3eb6ffd640 1 -- 172.21.15.26:0/745550679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8032a10 con 0x7f3eb815e380 2024-04-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3ebe763640 1 -- 172.21.15.26:0/745550679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb815e380 msgr2=0x7f3eb80b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3ebe763640 1 --2- 172.21.15.26:0/745550679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb815e380 0x7f3eb80b4530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80097a0 tx=0x7f3ea802a2c0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3ebe763640 1 -- 172.21.15.26:0/745550679 shutdown_connections 2024-04-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3ebe763640 1 --2- 172.21.15.26:0/745550679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb815e380 0x7f3eb80b4530 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3ebe763640 1 -- 172.21.15.26:0/745550679 >> 172.21.15.26:0/745550679 conn(0x7f3eb806e2c0 msgr2=0x7f3eb806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:08.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3ebe763640 1 -- 172.21.15.26:0/745550679 shutdown_connections 2024-04-24T23:21:08.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.073+0000 7f3ebe763640 1 -- 172.21.15.26:0/745550679 wait complete. 2024-04-24T23:21:08.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.074+0000 7f3ebe763640 1 Processor -- start 2024-04-24T23:21:08.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3ebe763640 1 -- start start 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3ebe763640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb80b0030 0x7f3eb8159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3ebe763640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb80b4fd0 0x7f3eb80b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3ebe763640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb815e380 0x7f3eb80b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3ebe763640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb80ae530 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3ebe763640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb80725b0 con 0x7f3eb80b0030 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3ebe763640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb80ae6b0 con 0x7f3eb815e380 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3eb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb80b4fd0 0x7f3eb80b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3eb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb80b4fd0 0x7f3eb80b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40224/0 (socket says 172.21.15.26:40224) 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.075+0000 7f3eb77fe640 1 -- 172.21.15.26:0/944992197 learned_addr learned my addr 172.21.15.26:0/944992197 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb7fff640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb80b0030 0x7f3eb8159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3ebcd00640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb815e380 0x7f3eb80b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb77fe640 1 -- 172.21.15.26:0/944992197 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb815e380 msgr2=0x7f3eb80b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb77fe640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb815e380 0x7f3eb80b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb77fe640 1 -- 172.21.15.26:0/944992197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb80b0030 msgr2=0x7f3eb8159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb77fe640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb80b0030 0x7f3eb8159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb77fe640 1 -- 172.21.15.26:0/944992197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb80b2a20 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3ebcd00640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb815e380 0x7f3eb80b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb77fe640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb80b4fd0 0x7f3eb80b1fa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c400 tx=0x7f3ea000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb4ff9640 1 -- 172.21.15.26:0/944992197 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea00052d0 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb4ff9640 1 -- 172.21.15.26:0/944992197 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea0005490 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb80b2c90 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3eb4ff9640 1 -- 172.21.15.26:0/944992197 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0007260 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.076+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb80b3250 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.077+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb80694c0 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.077+0000 7f3eb4ff9640 1 -- 172.21.15.26:0/944992197 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ea0007400 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.078+0000 7f3eb4ff9640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9806be30 0x7f3e9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.078+0000 7f3eb4ff9640 1 -- 172.21.15.26:0/944992197 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3ea008a3a0 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.078+0000 7f3eb7fff640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9806be30 0x7f3e9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.078+0000 7f3eb7fff640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9806be30 0x7f3e9806e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8034000 tx=0x7f3ea802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.082+0000 7f3eb4ff9640 1 -- 172.21.15.26:0/944992197 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ea0058d50 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.306+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3eb80b45a0 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.307+0000 7f3eb4ff9640 1 -- 172.21.15.26:0/944992197 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7f3ea0058650 con 0x7f3eb80b4fd0 2024-04-24T23:21:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:08.309 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2829963318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2829963318},{"type":"v1","addr":"172.21.15.26:6837","nonce":2829963318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2687649505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2687649505},{"type":"v1","addr":"172.21.15.29:6833","nonce":2687649505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:20:54.833525+0000","modified":"2024-04-24T23:20:57.181910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1604449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1604449840},{"type":"v1","addr":"172.21.15.29:6835","nonce":1604449840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-24T23:21:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9806be30 msgr2=0x7f3e9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9806be30 0x7f3e9806e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8034000 tx=0x7f3ea802e000 comp rx=0 tx=0).stop 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb80b4fd0 msgr2=0x7f3eb80b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb80b4fd0 0x7f3eb80b1fa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c400 tx=0x7f3ea000c8c0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 shutdown_connections 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9806be30 0x7f3e9806e2f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb815e380 0x7f3eb80b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb80b4fd0 0x7f3eb80b1fa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 --2- 172.21.15.26:0/944992197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb80b0030 0x7f3eb8159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 >> 172.21.15.26:0/944992197 conn(0x7f3eb806e2c0 msgr2=0x7f3eb806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 shutdown_connections 2024-04-24T23:21:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.309+0000 7f3ebe763640 1 -- 172.21.15.26:0/944992197 wait complete. 2024-04-24T23:21:08.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:08.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.420+0000 7fd82a190640 1 Processor -- start 2024-04-24T23:21:08.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.420+0000 7fd82a190640 1 -- start start 2024-04-24T23:21:08.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.421+0000 7fd82a190640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240732b0 0x7fd8240aed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.421+0000 7fd82a190640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd8240af290 0x7fd8240af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.421+0000 7fd82a190640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8240afc60 con 0x7fd82415c210 2024-04-24T23:21:08.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.421+0000 7fd82a190640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd8240ad170 con 0x7fd8240af290 2024-04-24T23:21:08.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.421+0000 7fd82a190640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd8240ad730 con 0x7fd8240732b0 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.421+0000 7fd823fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd8240af290 0x7fd8240af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.421+0000 7fd8237fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd82415c210 0x7fd8240b1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54726/0 (socket says 172.21.15.26:54726) 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.421+0000 7fd8237fe640 1 -- 172.21.15.26:0/1412387258 learned_addr learned my addr 172.21.15.26:0/1412387258 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.421+0000 7fd822ffd640 1 --2- 172.21.15.26:0/1412387258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240732b0 0x7fd8240aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.422+0000 7fd823fff640 1 -- 172.21.15.26:0/1412387258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240732b0 msgr2=0x7fd8240aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.422+0000 7fd823fff640 1 --2- 172.21.15.26:0/1412387258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240732b0 0x7fd8240aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.422+0000 7fd823fff640 1 -- 172.21.15.26:0/1412387258 >> v1:172.21.15.26:6789/0 conn(0x7fd82415c210 legacy=0x7fd8240b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.422+0000 7fd823fff640 1 -- 172.21.15.26:0/1412387258 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8240b09a0 con 0x7fd8240af290 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.422+0000 7fd8227fc640 1 -- 172.21.15.26:0/1412387258 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1404929730 0 0) 0x7fd8240afc60 con 0x7fd82415c210 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.422+0000 7fd823fff640 1 --2- 172.21.15.26:0/1412387258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240af290 0x7fd8240af720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009d90 tx=0x7fd81c02a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1748e42b0808e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.422+0000 7fd8227fc640 1 -- 172.21.15.26:0/1412387258 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c007740 con 0x7fd8240af290 2024-04-24T23:21:08.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd8227fc640 1 -- 172.21.15.26:0/1412387258 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd81c0078e0 con 0x7fd8240af290 2024-04-24T23:21:08.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd8227fc640 1 -- 172.21.15.26:0/1412387258 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c007cb0 con 0x7fd8240af290 2024-04-24T23:21:08.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd82a190640 1 -- 172.21.15.26:0/1412387258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240af290 msgr2=0x7fd8240af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd82a190640 1 --2- 172.21.15.26:0/1412387258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240af290 0x7fd8240af720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009d90 tx=0x7fd81c02a530 comp rx=0 tx=0).stop 2024-04-24T23:21:08.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd82a190640 1 -- 172.21.15.26:0/1412387258 shutdown_connections 2024-04-24T23:21:08.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd82a190640 1 --2- 172.21.15.26:0/1412387258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240af290 0x7fd8240af720 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd82a190640 1 --2- 172.21.15.26:0/1412387258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240732b0 0x7fd8240aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd82a190640 1 -- 172.21.15.26:0/1412387258 >> 172.21.15.26:0/1412387258 conn(0x7fd824072790 msgr2=0x7fd824072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd82a190640 1 -- 172.21.15.26:0/1412387258 shutdown_connections 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.423+0000 7fd82a190640 1 -- 172.21.15.26:0/1412387258 wait complete. 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.424+0000 7fd82a190640 1 Processor -- start 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.424+0000 7fd82a190640 1 -- start start 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd82a190640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240732b0 0x7fd8241506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd82a190640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240af290 0x7fd8240b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd82a190640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd82415c210 0x7fd82415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd82a190640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd8240cff50 con 0x7fd8240732b0 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd82a190640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd8240ad3c0 con 0x7fd82415c210 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd82a190640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd8240d00d0 con 0x7fd8240af290 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd8237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240732b0 0x7fd8241506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd8237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240732b0 0x7fd8241506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40234/0 (socket says 172.21.15.26:40234) 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd8237fe640 1 -- 172.21.15.26:0/1265289422 learned_addr learned my addr 172.21.15.26:0/1265289422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:08.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd822ffd640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240af290 0x7fd8240b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd823fff640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd82415c210 0x7fd82415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd8237fe640 1 -- 172.21.15.26:0/1265289422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240af290 msgr2=0x7fd8240b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd8237fe640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240af290 0x7fd8240b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd8237fe640 1 -- 172.21.15.26:0/1265289422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd82415c210 msgr2=0x7fd82415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd8237fe640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd82415c210 0x7fd82415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd8237fe640 1 -- 172.21.15.26:0/1265289422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd824158000 con 0x7fd8240732b0 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.425+0000 7fd823fff640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd82415c210 0x7fd82415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.426+0000 7fd8237fe640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240732b0 0x7fd8241506c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd81400c460 tx=0x7fd81400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.426+0000 7fd80bfff640 1 -- 172.21.15.26:0/1265289422 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814018070 con 0x7fd8240732b0 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.426+0000 7fd80bfff640 1 -- 172.21.15.26:0/1265289422 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd814007500 con 0x7fd8240732b0 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.426+0000 7fd80bfff640 1 -- 172.21.15.26:0/1265289422 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814004a20 con 0x7fd8240732b0 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.426+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd824150c00 con 0x7fd8240732b0 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.426+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82409ba90 con 0x7fd8240732b0 2024-04-24T23:21:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.427+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8240694c0 con 0x7fd8240732b0 2024-04-24T23:21:08.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.427+0000 7fd80bfff640 1 -- 172.21.15.26:0/1265289422 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd814004bc0 con 0x7fd8240732b0 2024-04-24T23:21:08.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.428+0000 7fd80bfff640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd81006be30 0x7fd81006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.428+0000 7fd822ffd640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd81006be30 0x7fd81006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.428+0000 7fd80bfff640 1 -- 172.21.15.26:0/1265289422 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd81408a0d0 con 0x7fd8240732b0 2024-04-24T23:21:08.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.428+0000 7fd822ffd640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd81006be30 0x7fd81006e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8241585c0 tx=0x7fd818009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.432+0000 7fd80bfff640 1 -- 172.21.15.26:0/1265289422 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd824150c00 con 0x7fd8240732b0 2024-04-24T23:21:08.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.615+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd82406c3a0 con 0x7fd8240732b0 2024-04-24T23:21:08.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.616+0000 7fd80bfff640 1 -- 172.21.15.26:0/1265289422 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fd824158000 con 0x7fd8240732b0 2024-04-24T23:21:08.618 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:08.619 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:08.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.618+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd81006be30 msgr2=0x7fd81006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.618+0000 7fd82a190640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd81006be30 0x7fd81006e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8241585c0 tx=0x7fd818009190 comp rx=0 tx=0).stop 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.618+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240732b0 msgr2=0x7fd8241506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.618+0000 7fd82a190640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240732b0 0x7fd8241506c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd81400c460 tx=0x7fd81400c920 comp rx=0 tx=0).stop 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.619+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 shutdown_connections 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.619+0000 7fd82a190640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd81006be30 0x7fd81006e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.619+0000 7fd82a190640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd82415c210 0x7fd82415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.619+0000 7fd82a190640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8240af290 0x7fd8240b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.619+0000 7fd82a190640 1 --2- 172.21.15.26:0/1265289422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8240732b0 0x7fd8241506c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.619+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 >> 172.21.15.26:0/1265289422 conn(0x7fd824072790 msgr2=0x7fd8240b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.619+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 shutdown_connections 2024-04-24T23:21:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.619+0000 7fd82a190640 1 -- 172.21.15.26:0/1265289422 wait complete. 2024-04-24T23:21:08.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:08.632 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.733+0000 7feeabe9e640 1 Processor -- start 2024-04-24T23:21:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.733+0000 7feeabe9e640 1 -- start start 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feeabe9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 0x7feea40d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feeabe9e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feea40d18b0 0x7feea406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feeabe9e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea415e920 con 0x7feea406b660 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feeabe9e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7feea415eba0 con 0x7feea40d18b0 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feeabe9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feea415ee80 con 0x7feea40cc9b0 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feea9439640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feea40d18b0 0x7feea406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feeaa43b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feea406b660 0x7feea4071ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54730/0 (socket says 172.21.15.26:54730) 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feeaa43b640 1 -- 172.21.15.26:0/1270763146 learned_addr learned my addr 172.21.15.26:0/1270763146 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feea9c3a640 1 --2- 172.21.15.26:0/1270763146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 0x7feea40d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feea9439640 1 -- 172.21.15.26:0/1270763146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 msgr2=0x7feea40d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feea9439640 1 --2- 172.21.15.26:0/1270763146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 0x7feea40d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feea9439640 1 -- 172.21.15.26:0/1270763146 >> v1:172.21.15.26:6789/0 conn(0x7feea406b660 legacy=0x7feea4071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feea9439640 1 -- 172.21.15.26:0/1270763146 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea415f160 con 0x7feea40d18b0 2024-04-24T23:21:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feea8c38640 1 -- 172.21.15.26:0/1270763146 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2255169032 0 0) 0x7fee9c001490 con 0x7feea406b660 2024-04-24T23:21:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.734+0000 7feea9c3a640 1 --2- 172.21.15.26:0/1270763146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 0x7feea40d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.735+0000 7feea9439640 1 --2- 172.21.15.26:0/1270763146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 0x7feea406b120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fee940097a0 tx=0x7fee9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863a0347fc75200a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.735+0000 7feea8c38640 1 -- 172.21.15.26:0/1270763146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94037070 con 0x7feea40d18b0 2024-04-24T23:21:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.735+0000 7feea8c38640 1 -- 172.21.15.26:0/1270763146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee94032440 con 0x7feea40d18b0 2024-04-24T23:21:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.735+0000 7feea8c38640 1 -- 172.21.15.26:0/1270763146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94032810 con 0x7feea40d18b0 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.736+0000 7feeabe9e640 1 -- 172.21.15.26:0/1270763146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 msgr2=0x7feea406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.736+0000 7feeabe9e640 1 --2- 172.21.15.26:0/1270763146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 0x7feea406b120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fee940097a0 tx=0x7fee9402a180 comp rx=0 tx=0).stop 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.736+0000 7feeabe9e640 1 -- 172.21.15.26:0/1270763146 shutdown_connections 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.736+0000 7feeabe9e640 1 --2- 172.21.15.26:0/1270763146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 0x7feea406b120 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.736+0000 7feeabe9e640 1 --2- 172.21.15.26:0/1270763146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 0x7feea40d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.736+0000 7feeabe9e640 1 -- 172.21.15.26:0/1270763146 >> 172.21.15.26:0/1270763146 conn(0x7feea40d0d20 msgr2=0x7feea40d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.736+0000 7feeabe9e640 1 -- 172.21.15.26:0/1270763146 shutdown_connections 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.736+0000 7feeabe9e640 1 -- 172.21.15.26:0/1270763146 wait complete. 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.737+0000 7feeabe9e640 1 Processor -- start 2024-04-24T23:21:08.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.737+0000 7feeabe9e640 1 -- start start 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeabe9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feea406b660 0x7feea41528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeabe9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 0x7feea415b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeabe9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 0x7feea40c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeabe9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feea40d08f0 con 0x7feea40d18b0 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeabe9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feea40cec20 con 0x7feea406b660 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeabe9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feea4072600 con 0x7feea40cc9b0 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeaa43b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 0x7feea40c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeaa43b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 0x7feea40c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40252/0 (socket says 172.21.15.26:40252) 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeaa43b640 1 -- 172.21.15.26:0/1080213564 learned_addr learned my addr 172.21.15.26:0/1080213564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feea9439640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 0x7feea415b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feea9c3a640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feea406b660 0x7feea41528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeaa43b640 1 -- 172.21.15.26:0/1080213564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 msgr2=0x7feea415b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeaa43b640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 0x7feea415b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeaa43b640 1 -- 172.21.15.26:0/1080213564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feea406b660 msgr2=0x7feea41528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeaa43b640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feea406b660 0x7feea41528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeaa43b640 1 -- 172.21.15.26:0/1080213564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea415a850 con 0x7feea40d18b0 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feea9c3a640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feea406b660 0x7feea41528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeaa43b640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 0x7feea40c6610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c7f0 tx=0x7fee9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7fee9a7fc640 1 -- 172.21.15.26:0/1080213564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c018070 con 0x7feea40d18b0 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7fee9a7fc640 1 -- 172.21.15.26:0/1080213564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9c00a5e0 con 0x7feea40d18b0 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea415ab30 con 0x7feea40d18b0 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.738+0000 7fee9a7fc640 1 -- 172.21.15.26:0/1080213564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00aa10 con 0x7feea40d18b0 2024-04-24T23:21:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.739+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea415b0f0 con 0x7feea40d18b0 2024-04-24T23:21:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.740+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea4075610 con 0x7feea40d18b0 2024-04-24T23:21:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.740+0000 7fee9a7fc640 1 -- 172.21.15.26:0/1080213564 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee9c01b070 con 0x7feea40d18b0 2024-04-24T23:21:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.740+0000 7fee9a7fc640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee8806be30 0x7fee8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.740+0000 7feea9c3a640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee8806be30 0x7fee8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.741+0000 7fee9a7fc640 1 -- 172.21.15.26:0/1080213564 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fee9c08a3d0 con 0x7feea40d18b0 2024-04-24T23:21:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.741+0000 7feea9c3a640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee8806be30 0x7fee8806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feea0006fd0 tx=0x7feea0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:08.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.745+0000 7fee9a7fc640 1 -- 172.21.15.26:0/1080213564 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feea415e920 con 0x7feea40d18b0 2024-04-24T23:21:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.930+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feea4072fd0 con 0x7feea40d18b0 2024-04-24T23:21:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.930+0000 7fee9a7fc640 1 -- 172.21.15.26:0/1080213564 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fee9c058e10 con 0x7feea40d18b0 2024-04-24T23:21:08.933 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:08.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee8806be30 msgr2=0x7fee8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee8806be30 0x7fee8806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feea0006fd0 tx=0x7feea0008040 comp rx=0 tx=0).stop 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 msgr2=0x7feea40c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 0x7feea40c6610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c7f0 tx=0x7fee9c00ccb0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 shutdown_connections 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee8806be30 0x7fee8806e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea40d18b0 0x7feea40c6610 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feea40cc9b0 0x7feea415b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 --2- 172.21.15.26:0/1080213564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feea406b660 0x7feea41528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 >> 172.21.15.26:0/1080213564 conn(0x7feea40d0d20 msgr2=0x7feea406f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 shutdown_connections 2024-04-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:08.932+0000 7feeabe9e640 1 -- 172.21.15.26:0/1080213564 wait complete. 2024-04-24T23:21:08.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.046+0000 7f2065314640 1 Processor -- start 2024-04-24T23:21:09.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.047+0000 7f2065314640 1 -- start start 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.047+0000 7f2065314640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f206015e370 0x7f20600d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.047+0000 7f2065314640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f206015e750 0x7f20600d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.047+0000 7f2065314640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20600d2120 con 0x7f20600d4c10 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.047+0000 7f2065314640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f20600d1510 con 0x7f206015e750 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.047+0000 7f2065314640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f20600d1ed0 con 0x7f206015e370 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.047+0000 7f205e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f206015e750 0x7f20600d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.048+0000 7f205e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f206015e750 0x7f20600d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40266/0 (socket says 172.21.15.26:40266) 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.048+0000 7f205e7fc640 1 -- 172.21.15.26:0/831492215 learned_addr learned my addr 172.21.15.26:0/831492215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.048+0000 7f205effd640 1 --2- 172.21.15.26:0/831492215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f206015e370 0x7f20600d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.048+0000 7f205e7fc640 1 -- 172.21.15.26:0/831492215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f206015e370 msgr2=0x7f20600d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.048+0000 7f205e7fc640 1 --2- 172.21.15.26:0/831492215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f206015e370 0x7f20600d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.048+0000 7f205e7fc640 1 -- 172.21.15.26:0/831492215 >> v1:172.21.15.29:6789/0 conn(0x7f20600d4c10 legacy=0x7f20600b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.048+0000 7f205e7fc640 1 -- 172.21.15.26:0/831492215 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20600b3740 con 0x7f206015e750 2024-04-24T23:21:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.048+0000 7f205effd640 1 --2- 172.21.15.26:0/831492215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f206015e370 0x7f20600d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.048+0000 7f205e7fc640 1 --2- 172.21.15.26:0/831492215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f206015e750 0x7f20600d2a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2054009870 tx=0x7f205402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c320a5301c13ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f205dffb640 1 -- 172.21.15.26:0/831492215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054037070 con 0x7f206015e750 2024-04-24T23:21:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f205dffb640 1 -- 172.21.15.26:0/831492215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2054032440 con 0x7f206015e750 2024-04-24T23:21:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f205dffb640 1 -- 172.21.15.26:0/831492215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054032810 con 0x7f206015e750 2024-04-24T23:21:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f2065314640 1 -- 172.21.15.26:0/831492215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f206015e750 msgr2=0x7f20600d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f2065314640 1 --2- 172.21.15.26:0/831492215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f206015e750 0x7f20600d2a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2054009870 tx=0x7f205402a210 comp rx=0 tx=0).stop 2024-04-24T23:21:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f2065314640 1 -- 172.21.15.26:0/831492215 shutdown_connections 2024-04-24T23:21:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f2065314640 1 --2- 172.21.15.26:0/831492215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f206015e750 0x7f20600d2a60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f2065314640 1 --2- 172.21.15.26:0/831492215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f206015e370 0x7f20600d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f2065314640 1 -- 172.21.15.26:0/831492215 >> 172.21.15.26:0/831492215 conn(0x7f20600d40f0 msgr2=0x7f20600d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f2065314640 1 -- 172.21.15.26:0/831492215 shutdown_connections 2024-04-24T23:21:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.049+0000 7f2065314640 1 -- 172.21.15.26:0/831492215 wait complete. 2024-04-24T23:21:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f2065314640 1 Processor -- start 2024-04-24T23:21:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f2065314640 1 -- start start 2024-04-24T23:21:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f2065314640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20600d4c10 0x7f20600b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f2065314640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206015e370 0x7f20600b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f2065314640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20600b1c10 0x7f20601f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f2065314640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20600cfca0 con 0x7f20600d4c10 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f2065314640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20600b3d00 con 0x7f206015e370 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f2065314640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f20600cfe20 con 0x7f20600b1c10 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f205effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20600d4c10 0x7f20600b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.051+0000 7f205effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20600d4c10 0x7f20600b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40282/0 (socket says 172.21.15.26:40282) 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f205effd640 1 -- 172.21.15.26:0/2662257459 learned_addr learned my addr 172.21.15.26:0/2662257459 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f205e7fc640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206015e370 0x7f20600b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f205f7fe640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20600b1c10 0x7f20601f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f205effd640 1 -- 172.21.15.26:0/2662257459 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20600b1c10 msgr2=0x7f20601f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f205effd640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20600b1c10 0x7f20601f2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f205effd640 1 -- 172.21.15.26:0/2662257459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206015e370 msgr2=0x7f20600b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f205effd640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206015e370 0x7f20600b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f205effd640 1 -- 172.21.15.26:0/2662257459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20600b8f30 con 0x7f20600d4c10 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f205effd640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20600d4c10 0x7f20600b6f80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f205800c7f0 tx=0x7f205800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f20477fe640 1 -- 172.21.15.26:0/2662257459 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058015070 con 0x7f20600d4c10 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f20477fe640 1 -- 172.21.15.26:0/2662257459 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2058007a30 con 0x7f20600d4c10 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f20477fe640 1 -- 172.21.15.26:0/2662257459 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058007e00 con 0x7f20600d4c10 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20600b9210 con 0x7f20600d4c10 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.052+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20600b97d0 con 0x7f20600d4c10 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.053+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2060075610 con 0x7f20600d4c10 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.054+0000 7f20477fe640 1 -- 172.21.15.26:0/2662257459 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2058018070 con 0x7f20600d4c10 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.054+0000 7f20477fe640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f204c06be30 0x7f204c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.054+0000 7f20477fe640 1 -- 172.21.15.26:0/2662257459 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f205808ada0 con 0x7f20600d4c10 2024-04-24T23:21:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.054+0000 7f205e7fc640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f204c06be30 0x7f204c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.054+0000 7f205e7fc640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f204c06be30 0x7f204c06e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2054034000 tx=0x7f205402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:09.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.058+0000 7f20477fe640 1 -- 172.21.15.26:0/2662257459 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2058059100 con 0x7f20600d4c10 2024-04-24T23:21:09.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.238+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f206006bc90 con 0x7f20600d4c10 2024-04-24T23:21:09.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.239+0000 7f20477fe640 1 -- 172.21.15.26:0/2662257459 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f205805d400 con 0x7f20600d4c10 2024-04-24T23:21:09.242 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f204c06be30 msgr2=0x7f204c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f204c06be30 0x7f204c06e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2054034000 tx=0x7f205402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20600d4c10 msgr2=0x7f20600b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20600d4c10 0x7f20600b6f80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f205800c7f0 tx=0x7f205800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 shutdown_connections 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f204c06be30 0x7f204c06e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20600b1c10 0x7f20601f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f206015e370 0x7f20600b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 --2- 172.21.15.26:0/2662257459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20600d4c10 0x7f20600b6f80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 >> 172.21.15.26:0/2662257459 conn(0x7f20600d40f0 msgr2=0x7f20600d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 shutdown_connections 2024-04-24T23:21:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.241+0000 7f2065314640 1 -- 172.21.15.26:0/2662257459 wait complete. 2024-04-24T23:21:09.255 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:09.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.356+0000 7f0493da9640 1 Processor -- start 2024-04-24T23:21:09.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.356+0000 7f0493da9640 1 -- start start 2024-04-24T23:21:09.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0493da9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 0x7f048c0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0493da9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f048c15ef60 con 0x7f048c0d0750 2024-04-24T23:21:09.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0493da9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f048c15f240 con 0x7f048c0ad7b0 2024-04-24T23:21:09.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0493da9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f048c15f520 con 0x7f048c15adc0 2024-04-24T23:21:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0492346640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f048c0d0750 0x7f048c072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54736/0 (socket says 172.21.15.26:54736) 2024-04-24T23:21:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0492346640 1 -- 172.21.15.26:0/4098200484 learned_addr learned my addr 172.21.15.26:0/4098200484 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0491b45640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 0x7f048c0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0490b43640 1 -- 172.21.15.26:0/4098200484 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1856749951 0 0) 0x7f048c15ef60 con 0x7f048c0d0750 2024-04-24T23:21:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0490b43640 1 -- 172.21.15.26:0/4098200484 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0478003520 con 0x7f048c0d0750 2024-04-24T23:21:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0491b45640 1 -- 172.21.15.26:0/4098200484 >> v1:172.21.15.29:6789/0 conn(0x7f048c0ad7b0 legacy=0x7f048c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0491b45640 1 -- 172.21.15.26:0/4098200484 >> v1:172.21.15.26:6789/0 conn(0x7f048c0d0750 legacy=0x7f048c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.357+0000 7f0491b45640 1 -- 172.21.15.26:0/4098200484 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c15f800 con 0x7f048c15adc0 2024-04-24T23:21:09.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.358+0000 7f0491b45640 1 --2- 172.21.15.26:0/4098200484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 0x7f048c0b4370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04880097d0 tx=0x7f048802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6821467d2165ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:09.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.358+0000 7f0490b43640 1 -- 172.21.15.26:0/4098200484 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488037070 con 0x7f048c15adc0 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.358+0000 7f0490b43640 1 -- 172.21.15.26:0/4098200484 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0488032440 con 0x7f048c15adc0 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.358+0000 7f0490b43640 1 -- 172.21.15.26:0/4098200484 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488032810 con 0x7f048c15adc0 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.359+0000 7f0493da9640 1 -- 172.21.15.26:0/4098200484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 msgr2=0x7f048c0b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.359+0000 7f0493da9640 1 --2- 172.21.15.26:0/4098200484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 0x7f048c0b4370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04880097d0 tx=0x7f048802a180 comp rx=0 tx=0).stop 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.359+0000 7f0493da9640 1 -- 172.21.15.26:0/4098200484 shutdown_connections 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.359+0000 7f0493da9640 1 --2- 172.21.15.26:0/4098200484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 0x7f048c0b4370 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.359+0000 7f0493da9640 1 -- 172.21.15.26:0/4098200484 >> 172.21.15.26:0/4098200484 conn(0x7f048c06b540 msgr2=0x7f048c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.359+0000 7f0493da9640 1 -- 172.21.15.26:0/4098200484 shutdown_connections 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.359+0000 7f0493da9640 1 -- 172.21.15.26:0/4098200484 wait complete. 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.360+0000 7f0493da9640 1 Processor -- start 2024-04-24T23:21:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.360+0000 7f0493da9640 1 -- start start 2024-04-24T23:21:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0493da9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f048c0ad7b0 0x7f048c151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0493da9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c0d0750 0x7f048c142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0493da9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 0x7f048c1432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0493da9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f048c15b1a0 con 0x7f048c0d0750 2024-04-24T23:21:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0493da9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f048c0b48b0 con 0x7f048c0ad7b0 2024-04-24T23:21:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0493da9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f048c15e9c0 con 0x7f048c15adc0 2024-04-24T23:21:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491344640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c0d0750 0x7f048c142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491344640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c0d0750 0x7f048c142d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40284/0 (socket says 172.21.15.26:40284) 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491344640 1 -- 172.21.15.26:0/776219132 learned_addr learned my addr 172.21.15.26:0/776219132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0492346640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 0x7f048c1432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491b45640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f048c0ad7b0 0x7f048c151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491344640 1 -- 172.21.15.26:0/776219132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 msgr2=0x7f048c1432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491344640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 0x7f048c1432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491344640 1 -- 172.21.15.26:0/776219132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f048c0ad7b0 msgr2=0x7f048c151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491344640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f048c0ad7b0 0x7f048c151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491344640 1 -- 172.21.15.26:0/776219132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c0b0d70 con 0x7f048c0d0750 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491b45640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f048c0ad7b0 0x7f048c151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.361+0000 7f0491344640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c0d0750 0x7f048c142d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c450 tx=0x7f047c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.362+0000 7f04827fc640 1 -- 172.21.15.26:0/776219132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c0052d0 con 0x7f048c0d0750 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.362+0000 7f04827fc640 1 -- 172.21.15.26:0/776219132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047c005470 con 0x7f048c0d0750 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.362+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c151940 con 0x7f048c0d0750 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.362+0000 7f04827fc640 1 -- 172.21.15.26:0/776219132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c012870 con 0x7f048c0d0750 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.362+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c0b15b0 con 0x7f048c0d0750 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.363+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c075610 con 0x7f048c0d0750 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.363+0000 7f04827fc640 1 -- 172.21.15.26:0/776219132 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f047c012a10 con 0x7f048c0d0750 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.363+0000 7f04827fc640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f047006be30 0x7f047006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.364+0000 7f04827fc640 1 -- 172.21.15.26:0/776219132 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f047c08a9c0 con 0x7f048c0d0750 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.364+0000 7f0491b45640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f047006be30 0x7f047006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.364+0000 7f0491b45640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f047006be30 0x7f047006e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0488034000 tx=0x7f048802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:09.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.368+0000 7f04827fc640 1 -- 172.21.15.26:0/776219132 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f047c08ce20 con 0x7f048c0d0750 2024-04-24T23:21:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.550+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f048c15e730 con 0x7f048c0d0750 2024-04-24T23:21:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.550+0000 7f04827fc640 1 -- 172.21.15.26:0/776219132 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f047c05f030 con 0x7f048c0d0750 2024-04-24T23:21:09.553 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f047006be30 msgr2=0x7f047006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f047006be30 0x7f047006e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0488034000 tx=0x7f048802e000 comp rx=0 tx=0).stop 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c0d0750 msgr2=0x7f048c142d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c0d0750 0x7f048c142d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c450 tx=0x7f047c00c910 comp rx=0 tx=0).stop 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 shutdown_connections 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f047006be30 0x7f047006e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f048c15adc0 0x7f048c1432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c0d0750 0x7f048c142d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 --2- 172.21.15.26:0/776219132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f048c0ad7b0 0x7f048c151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 >> 172.21.15.26:0/776219132 conn(0x7f048c06b540 msgr2=0x7f048c0b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 shutdown_connections 2024-04-24T23:21:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.552+0000 7f0493da9640 1 -- 172.21.15.26:0/776219132 wait complete. 2024-04-24T23:21:09.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.664+0000 7f80512c4640 1 Processor -- start 2024-04-24T23:21:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.664+0000 7f80512c4640 1 -- start start 2024-04-24T23:21:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f80512c4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f804c0b4530 0x7f804c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f80512c4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f804c0b0030 0x7f804c0b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f80512c4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804c0b0980 con 0x7f804c15e380 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f80512c4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f804c0b49a0 con 0x7f804c0b0030 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f80512c4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f804c15e760 con 0x7f804c0b4530 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f804b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f804c0b0030 0x7f804c0b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f804affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f804c15e380 0x7f804c072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54738/0 (socket says 172.21.15.26:54738) 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f804affd640 1 -- 172.21.15.26:0/469344887 learned_addr learned my addr 172.21.15.26:0/469344887 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f804b7fe640 1 -- 172.21.15.26:0/469344887 >> v2:172.21.15.29:3300/0 conn(0x7f804c0b4530 msgr2=0x7f804c0b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f8043fff640 1 --2- 172.21.15.26:0/469344887 >> v2:172.21.15.29:3300/0 conn(0x7f804c0b4530 0x7f804c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f804b7fe640 1 --2- 172.21.15.26:0/469344887 >> v2:172.21.15.29:3300/0 conn(0x7f804c0b4530 0x7f804c0b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f804b7fe640 1 -- 172.21.15.26:0/469344887 >> v1:172.21.15.26:6789/0 conn(0x7f804c15e380 legacy=0x7f804c072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f804a7fc640 1 -- 172.21.15.26:0/469344887 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3976865203 0 0) 0x7f803c001110 con 0x7f804c15e380 2024-04-24T23:21:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.665+0000 7f804b7fe640 1 -- 172.21.15.26:0/469344887 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c0b5700 con 0x7f804c0b0030 2024-04-24T23:21:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.666+0000 7f804b7fe640 1 --2- 172.21.15.26:0/469344887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b0030 0x7f804c0b0440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f804c0b4d30 tx=0x7f804402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6ef2782827db4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.666+0000 7f804a7fc640 1 -- 172.21.15.26:0/469344887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044007740 con 0x7f804c0b0030 2024-04-24T23:21:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.666+0000 7f804a7fc640 1 -- 172.21.15.26:0/469344887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80440078e0 con 0x7f804c0b0030 2024-04-24T23:21:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.666+0000 7f804a7fc640 1 -- 172.21.15.26:0/469344887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044007cb0 con 0x7f804c0b0030 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.667+0000 7f80512c4640 1 -- 172.21.15.26:0/469344887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b0030 msgr2=0x7f804c0b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.667+0000 7f80512c4640 1 --2- 172.21.15.26:0/469344887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b0030 0x7f804c0b0440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f804c0b4d30 tx=0x7f804402a710 comp rx=0 tx=0).stop 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.667+0000 7f80512c4640 1 -- 172.21.15.26:0/469344887 shutdown_connections 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.667+0000 7f80512c4640 1 --2- 172.21.15.26:0/469344887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b0030 0x7f804c0b0440 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.667+0000 7f80512c4640 1 --2- 172.21.15.26:0/469344887 >> v2:172.21.15.29:3300/0 conn(0x7f804c0b4530 0x7f804c0b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.667+0000 7f80512c4640 1 -- 172.21.15.26:0/469344887 >> 172.21.15.26:0/469344887 conn(0x7f804c06e2c0 msgr2=0x7f804c06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.667+0000 7f80512c4640 1 -- 172.21.15.26:0/469344887 shutdown_connections 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.667+0000 7f80512c4640 1 -- 172.21.15.26:0/469344887 wait complete. 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.668+0000 7f80512c4640 1 Processor -- start 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.668+0000 7f80512c4640 1 -- start start 2024-04-24T23:21:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.668+0000 7f80512c4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f804c0b0030 0x7f804c0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.668+0000 7f80512c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b4530 0x7f804c0b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.668+0000 7f80512c4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f804c15e380 0x7f804c0cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.668+0000 7f80512c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f804c072240 con 0x7f804c0b4530 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.668+0000 7f80512c4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f804c0725b0 con 0x7f804c15e380 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.668+0000 7f80512c4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f804c0ae530 con 0x7f804c0b0030 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f8043fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b4530 0x7f804c0b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f8043fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b4530 0x7f804c0b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40300/0 (socket says 172.21.15.26:40300) 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f8043fff640 1 -- 172.21.15.26:0/3735241305 learned_addr learned my addr 172.21.15.26:0/3735241305 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f804b7fe640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f804c15e380 0x7f804c0cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f804affd640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f804c0b0030 0x7f804c0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f8043fff640 1 -- 172.21.15.26:0/3735241305 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f804c0b0030 msgr2=0x7f804c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f8043fff640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f804c0b0030 0x7f804c0b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f8043fff640 1 -- 172.21.15.26:0/3735241305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f804c15e380 msgr2=0x7f804c0cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f8043fff640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f804c15e380 0x7f804c0cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f8043fff640 1 -- 172.21.15.26:0/3735241305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c0b2ec0 con 0x7f804c0b4530 2024-04-24T23:21:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f8043fff640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b4530 0x7f804c0b25b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f803400c7f0 tx=0x7f803400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f80437fe640 1 -- 172.21.15.26:0/3735241305 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034015070 con 0x7f804c0b4530 2024-04-24T23:21:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f80437fe640 1 -- 172.21.15.26:0/3735241305 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8034004240 con 0x7f804c0b4530 2024-04-24T23:21:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c0b31a0 con 0x7f804c0b4530 2024-04-24T23:21:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f80437fe640 1 -- 172.21.15.26:0/3735241305 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034007620 con 0x7f804c0b4530 2024-04-24T23:21:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.669+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c0b3760 con 0x7f804c0b4530 2024-04-24T23:21:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.670+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804c0694c0 con 0x7f804c0b4530 2024-04-24T23:21:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.671+0000 7f80437fe640 1 -- 172.21.15.26:0/3735241305 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80340079d0 con 0x7f804c0b4530 2024-04-24T23:21:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.671+0000 7f80437fe640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f802c06be30 0x7f802c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.671+0000 7f80437fe640 1 -- 172.21.15.26:0/3735241305 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f803408b300 con 0x7f804c0b4530 2024-04-24T23:21:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.671+0000 7f804affd640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f802c06be30 0x7f802c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.672+0000 7f804affd640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f802c06be30 0x7f802c06e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f803c000f60 tx=0x7f803c007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.675+0000 7f80437fe640 1 -- 172.21.15.26:0/3735241305 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8034059cb0 con 0x7f804c0b4530 2024-04-24T23:21:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.895+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f804c15ce40 con 0x7f804c0b4530 2024-04-24T23:21:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.895+0000 7f80437fe640 1 -- 172.21.15.26:0/3735241305 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7f80340595b0 con 0x7f804c0b4530 2024-04-24T23:21:09.898 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:09.898 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2829963318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2829963318},{"type":"v1","addr":"172.21.15.26:6837","nonce":2829963318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2687649505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2687649505},{"type":"v1","addr":"172.21.15.29:6833","nonce":2687649505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:20:54.833525+0000","modified":"2024-04-24T23:20:57.181910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1604449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1604449840},{"type":"v1","addr":"172.21.15.29:6835","nonce":1604449840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-24T23:21:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-04-24T23:21:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.897+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f802c06be30 msgr2=0x7f802c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.897+0000 7f80512c4640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f802c06be30 0x7f802c06e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f803c000f60 tx=0x7f803c007c00 comp rx=0 tx=0).stop 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.897+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b4530 msgr2=0x7f804c0b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.897+0000 7f80512c4640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b4530 0x7f804c0b25b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f803400c7f0 tx=0x7f803400ccb0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.898+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 shutdown_connections 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.898+0000 7f80512c4640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f802c06be30 0x7f802c06e2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.898+0000 7f80512c4640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f804c15e380 0x7f804c0cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.898+0000 7f80512c4640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c0b4530 0x7f804c0b25b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.898+0000 7f80512c4640 1 --2- 172.21.15.26:0/3735241305 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f804c0b0030 0x7f804c0b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.898+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 >> 172.21.15.26:0/3735241305 conn(0x7f804c06e2c0 msgr2=0x7f804c158710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.898+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 shutdown_connections 2024-04-24T23:21:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:09.898+0000 7f80512c4640 1 -- 172.21.15.26:0/3735241305 wait complete. 2024-04-24T23:21:09.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.009+0000 7ff8a06bd640 1 Processor -- start 2024-04-24T23:21:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.009+0000 7ff8a06bd640 1 -- start start 2024-04-24T23:21:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff8a06bd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff89815e380 0x7ff8980b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff8a06bd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b0030 0x7ff8980b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff8a06bd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8980b0980 con 0x7ff8980b4fd0 2024-04-24T23:21:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff8a06bd640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff89815e760 con 0x7ff89815e380 2024-04-24T23:21:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff8a06bd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff8980725b0 con 0x7ff8980b0030 2024-04-24T23:21:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff89e459640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff89815e380 0x7ff8980b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff89e459640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff89815e380 0x7ff8980b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40308/0 (socket says 172.21.15.26:40308) 2024-04-24T23:21:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff89e459640 1 -- 172.21.15.26:0/2676108633 learned_addr learned my addr 172.21.15.26:0/2676108633 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff89ec5a640 1 --2- 172.21.15.26:0/2676108633 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b0030 0x7ff8980b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff89e459640 1 -- 172.21.15.26:0/2676108633 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b0030 msgr2=0x7ff8980b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff89e459640 1 --2- 172.21.15.26:0/2676108633 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b0030 0x7ff8980b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff89e459640 1 -- 172.21.15.26:0/2676108633 >> v1:172.21.15.29:6789/0 conn(0x7ff8980b4fd0 legacy=0x7ff898072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.010+0000 7ff89e459640 1 -- 172.21.15.26:0/2676108633 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8980b5440 con 0x7ff89815e380 2024-04-24T23:21:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.011+0000 7ff89e459640 1 --2- 172.21.15.26:0/2676108633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 0x7ff8980b4530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff894009870 tx=0x7ff89402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2f6b27758be515 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.011+0000 7ff89d457640 1 -- 172.21.15.26:0/2676108633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894037070 con 0x7ff89815e380 2024-04-24T23:21:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.011+0000 7ff89d457640 1 -- 172.21.15.26:0/2676108633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89402ae50 con 0x7ff89815e380 2024-04-24T23:21:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.011+0000 7ff89d457640 1 -- 172.21.15.26:0/2676108633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894032860 con 0x7ff89815e380 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.012+0000 7ff8a06bd640 1 -- 172.21.15.26:0/2676108633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 msgr2=0x7ff8980b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.012+0000 7ff8a06bd640 1 --2- 172.21.15.26:0/2676108633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 0x7ff8980b4530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff894009870 tx=0x7ff89402a300 comp rx=0 tx=0).stop 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.012+0000 7ff8a06bd640 1 -- 172.21.15.26:0/2676108633 shutdown_connections 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.012+0000 7ff8a06bd640 1 --2- 172.21.15.26:0/2676108633 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b0030 0x7ff8980b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.012+0000 7ff8a06bd640 1 --2- 172.21.15.26:0/2676108633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 0x7ff8980b4530 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.012+0000 7ff8a06bd640 1 -- 172.21.15.26:0/2676108633 >> 172.21.15.26:0/2676108633 conn(0x7ff89806e2c0 msgr2=0x7ff89806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.012+0000 7ff8a06bd640 1 -- 172.21.15.26:0/2676108633 shutdown_connections 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.012+0000 7ff8a06bd640 1 -- 172.21.15.26:0/2676108633 wait complete. 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.013+0000 7ff8a06bd640 1 Processor -- start 2024-04-24T23:21:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.013+0000 7ff8a06bd640 1 -- start start 2024-04-24T23:21:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff8a06bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8980b0030 0x7ff8981f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff8a06bd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b4fd0 0x7ff898156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff8a06bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 0x7ff898148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff8a06bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff898072240 con 0x7ff89815e380 2024-04-24T23:21:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff8a06bd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8980b59e0 con 0x7ff8980b0030 2024-04-24T23:21:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff8a06bd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff8980ae530 con 0x7ff8980b4fd0 2024-04-24T23:21:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89ec5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 0x7ff898148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89ec5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 0x7ff898148ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40310/0 (socket says 172.21.15.26:40310) 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89ec5a640 1 -- 172.21.15.26:0/1192318108 learned_addr learned my addr 172.21.15.26:0/1192318108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89dc58640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b4fd0 0x7ff898156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89e459640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8980b0030 0x7ff8981f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89ec5a640 1 -- 172.21.15.26:0/1192318108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b4fd0 msgr2=0x7ff898156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89ec5a640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b4fd0 0x7ff898156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89ec5a640 1 -- 172.21.15.26:0/1192318108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8980b0030 msgr2=0x7ff8981f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89ec5a640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8980b0030 0x7ff8981f7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89ec5a640 1 -- 172.21.15.26:0/1192318108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff898149010 con 0x7ff89815e380 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff89ec5a640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 0x7ff898148ad0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff890007e70 tx=0x7ff89000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff88effd640 1 -- 172.21.15.26:0/1192318108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890014620 con 0x7ff89815e380 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff88effd640 1 -- 172.21.15.26:0/1192318108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff890019070 con 0x7ff89815e380 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8980b23c0 con 0x7ff89815e380 2024-04-24T23:21:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff88effd640 1 -- 172.21.15.26:0/1192318108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890014a20 con 0x7ff89815e380 2024-04-24T23:21:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.014+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8980b2980 con 0x7ff89815e380 2024-04-24T23:21:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.015+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8980694c0 con 0x7ff89815e380 2024-04-24T23:21:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.016+0000 7ff88effd640 1 -- 172.21.15.26:0/1192318108 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff890014bc0 con 0x7ff89815e380 2024-04-24T23:21:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.016+0000 7ff88effd640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff87c06be30 0x7ff87c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.016+0000 7ff89e459640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff87c06be30 0x7ff87c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.016+0000 7ff88effd640 1 -- 172.21.15.26:0/1192318108 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff89008b7b0 con 0x7ff89815e380 2024-04-24T23:21:10.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.017+0000 7ff89e459640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff87c06be30 0x7ff87c06e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff894034000 tx=0x7ff89402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:10.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.021+0000 7ff88effd640 1 -- 172.21.15.26:0/1192318108 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff89005a160 con 0x7ff89815e380 2024-04-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.202+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8981560f0 con 0x7ff89815e380 2024-04-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.203+0000 7ff88effd640 1 -- 172.21.15.26:0/1192318108 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7ff890059a60 con 0x7ff89815e380 2024-04-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff87c06be30 msgr2=0x7ff87c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff87c06be30 0x7ff87c06e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff894034000 tx=0x7ff89402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 msgr2=0x7ff898148ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 0x7ff898148ad0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff890007e70 tx=0x7ff89000e4b0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 shutdown_connections 2024-04-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff87c06be30 0x7ff87c06e2f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89815e380 0x7ff898148ad0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8980b4fd0 0x7ff898156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 --2- 172.21.15.26:0/1192318108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8980b0030 0x7ff8981f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 >> 172.21.15.26:0/1192318108 conn(0x7ff89806e2c0 msgr2=0x7ff89815e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 shutdown_connections 2024-04-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.205+0000 7ff8a06bd640 1 -- 172.21.15.26:0/1192318108 wait complete. 2024-04-24T23:21:10.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.317+0000 7fdcc3996640 1 Processor -- start 2024-04-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.317+0000 7fdcc3996640 1 -- start start 2024-04-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc3996640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdcbc06f4f0 0x7fdcbc0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc3996640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcbc0addb0 0x7fdcbc154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc3996640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcbc0b0220 con 0x7fdcbc06f110 2024-04-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc3996640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fdcbc0af180 con 0x7fdcbc0addb0 2024-04-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc3996640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdcbc0af460 con 0x7fdcbc06f4f0 2024-04-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc1f33640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcbc0addb0 0x7fdcbc154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc1f33640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcbc0addb0 0x7fdcbc154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40316/0 (socket says 172.21.15.26:40316) 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc1f33640 1 -- 172.21.15.26:0/679798128 learned_addr learned my addr 172.21.15.26:0/679798128 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc0f31640 1 --2- 172.21.15.26:0/679798128 >> v2:172.21.15.29:3300/0 conn(0x7fdcbc06f4f0 0x7fdcbc0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc1f33640 1 -- 172.21.15.26:0/679798128 >> v2:172.21.15.29:3300/0 conn(0x7fdcbc06f4f0 msgr2=0x7fdcbc0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc1f33640 1 --2- 172.21.15.26:0/679798128 >> v2:172.21.15.29:3300/0 conn(0x7fdcbc06f4f0 0x7fdcbc0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc1f33640 1 -- 172.21.15.26:0/679798128 >> v1:172.21.15.29:6789/0 conn(0x7fdcbc06f110 legacy=0x7fdcbc15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc1f33640 1 -- 172.21.15.26:0/679798128 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc0af740 con 0x7fdcbc0addb0 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.318+0000 7fdcc0f31640 1 --2- 172.21.15.26:0/679798128 >> v2:172.21.15.29:3300/0 conn(0x7fdcbc06f4f0 0x7fdcbc0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.319+0000 7fdcc1f33640 1 --2- 172.21.15.26:0/679798128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc0addb0 0x7fdcbc154a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc158df0 tx=0x7fdcb402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abde1055d08f7858 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.319+0000 7fdcb3fff640 1 -- 172.21.15.26:0/679798128 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4036070 con 0x7fdcbc0addb0 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.319+0000 7fdcb3fff640 1 -- 172.21.15.26:0/679798128 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb4031600 con 0x7fdcbc0addb0 2024-04-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.319+0000 7fdcb3fff640 1 -- 172.21.15.26:0/679798128 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40319d0 con 0x7fdcbc0addb0 2024-04-24T23:21:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.320+0000 7fdcc3996640 1 -- 172.21.15.26:0/679798128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc0addb0 msgr2=0x7fdcbc154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.320+0000 7fdcc3996640 1 --2- 172.21.15.26:0/679798128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc0addb0 0x7fdcbc154a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc158df0 tx=0x7fdcb402a730 comp rx=0 tx=0).stop 2024-04-24T23:21:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.320+0000 7fdcc3996640 1 -- 172.21.15.26:0/679798128 shutdown_connections 2024-04-24T23:21:10.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.320+0000 7fdcc3996640 1 --2- 172.21.15.26:0/679798128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc0addb0 0x7fdcbc154a50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.320+0000 7fdcc3996640 1 --2- 172.21.15.26:0/679798128 >> v2:172.21.15.29:3300/0 conn(0x7fdcbc06f4f0 0x7fdcbc0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.320+0000 7fdcc3996640 1 -- 172.21.15.26:0/679798128 >> 172.21.15.26:0/679798128 conn(0x7fdcbc0729a0 msgr2=0x7fdcbc06b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.320+0000 7fdcc3996640 1 -- 172.21.15.26:0/679798128 shutdown_connections 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.320+0000 7fdcc3996640 1 -- 172.21.15.26:0/679798128 wait complete. 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.321+0000 7fdcc3996640 1 Processor -- start 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.321+0000 7fdcc3996640 1 -- start start 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc3996640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc06f110 0x7fdcbc145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc3996640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc06f4f0 0x7fdcbc1e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc3996640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdcbc0addb0 0x7fdcbc1486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc3996640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc0aee50 con 0x7fdcbc06f110 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc3996640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc0afd00 con 0x7fdcbc06f4f0 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc3996640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdcbc06fb00 con 0x7fdcbc0addb0 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1732640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc06f110 0x7fdcbc145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1732640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc06f110 0x7fdcbc145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40328/0 (socket says 172.21.15.26:40328) 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1732640 1 -- 172.21.15.26:0/2102316103 learned_addr learned my addr 172.21.15.26:0/2102316103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1f33640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdcbc0addb0 0x7fdcbc1486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc0f31640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc06f4f0 0x7fdcbc1e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1732640 1 -- 172.21.15.26:0/2102316103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdcbc0addb0 msgr2=0x7fdcbc1486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1732640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdcbc0addb0 0x7fdcbc1486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1732640 1 -- 172.21.15.26:0/2102316103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc06f4f0 msgr2=0x7fdcbc1e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1732640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc06f4f0 0x7fdcbc1e73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1732640 1 -- 172.21.15.26:0/2102316103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc1e78e0 con 0x7fdcbc06f110 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1f33640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdcbc0addb0 0x7fdcbc1486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcc1732640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc06f110 0x7fdcbc145340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c400 tx=0x7fdcb800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.322+0000 7fdcb1ffb640 1 -- 172.21.15.26:0/2102316103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80052d0 con 0x7fdcbc06f110 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.323+0000 7fdcb1ffb640 1 -- 172.21.15.26:0/2102316103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb8005490 con 0x7fdcbc06f110 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.323+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc148be0 con 0x7fdcbc06f110 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.323+0000 7fdcb1ffb640 1 -- 172.21.15.26:0/2102316103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8007260 con 0x7fdcbc06f110 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.323+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc147bc0 con 0x7fdcbc06f110 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.324+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcbc0694c0 con 0x7fdcbc06f110 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.324+0000 7fdcb1ffb640 1 -- 172.21.15.26:0/2102316103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcb8007400 con 0x7fdcbc06f110 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.325+0000 7fdcb1ffb640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdca006be30 0x7fdca006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.325+0000 7fdcc0f31640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdca006be30 0x7fdca006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.325+0000 7fdcb1ffb640 1 -- 172.21.15.26:0/2102316103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdcb808a400 con 0x7fdcbc06f110 2024-04-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.325+0000 7fdcc0f31640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdca006be30 0x7fdca006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc148f40 tx=0x7fdcac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.329+0000 7fdcb1ffb640 1 -- 172.21.15.26:0/2102316103 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdcbc1e78e0 con 0x7fdcbc06f110 2024-04-24T23:21:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.554+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcbc0b4540 con 0x7fdcbc06f110 2024-04-24T23:21:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.554+0000 7fdcb1ffb640 1 -- 172.21.15.26:0/2102316103 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7fdcb8058da0 con 0x7fdcbc06f110 2024-04-24T23:21:10.557 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:10.557 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2829963318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2829963318},{"type":"v1","addr":"172.21.15.26:6837","nonce":2829963318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2687649505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2687649505},{"type":"v1","addr":"172.21.15.29:6833","nonce":2687649505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:20:54.833525+0000","modified":"2024-04-24T23:20:57.181910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1604449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1604449840},{"type":"v1","addr":"172.21.15.29:6835","nonce":1604449840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-24T23:21:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-04-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdca006be30 msgr2=0x7fdca006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdca006be30 0x7fdca006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc148f40 tx=0x7fdcac008040 comp rx=0 tx=0).stop 2024-04-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc06f110 msgr2=0x7fdcbc145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc06f110 0x7fdcbc145340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c400 tx=0x7fdcb800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 shutdown_connections 2024-04-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdca006be30 0x7fdca006e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdcbc0addb0 0x7fdcbc1486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdcbc06f4f0 0x7fdcbc1e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 --2- 172.21.15.26:0/2102316103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcbc06f110 0x7fdcbc145340 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 >> 172.21.15.26:0/2102316103 conn(0x7fdcbc0729a0 msgr2=0x7fdcbc0b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 shutdown_connections 2024-04-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:10.557+0000 7fdcc3996640 1 -- 172.21.15.26:0/2102316103 wait complete. 2024-04-24T23:21:10.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:20:54.833525+0000', 'modified': '2024-04-24T23:20:57.181910+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/1604449840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 1604449840}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1604449840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4287, 'qdb_cluster': [4287]} 2024-04-24T23:21:10.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:21:10.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:21:11.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.080+0000 7faf217d5640 1 Processor -- start 2024-04-24T23:21:11.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.080+0000 7faf217d5640 1 -- start start 2024-04-24T23:21:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf217d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf1c06f110 0x7faf1c15f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf217d5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faf1c06f4f0 0x7faf1c0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf217d5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1c15f750 con 0x7faf1c0addb0 2024-04-24T23:21:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf217d5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7faf1c0b0220 con 0x7faf1c06f110 2024-04-24T23:21:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf217d5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faf1c0af180 con 0x7faf1c06f4f0 2024-04-24T23:21:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf1affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf1c06f110 0x7faf1c15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf1b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf1c0addb0 0x7faf1c154a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54754/0 (socket says 172.21.15.26:54754) 2024-04-24T23:21:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf1affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf1c06f110 0x7faf1c15f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40340/0 (socket says 172.21.15.26:40340) 2024-04-24T23:21:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf1b7fe640 1 -- 172.21.15.26:0/1216517095 learned_addr learned my addr 172.21.15.26:0/1216517095 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/1216517095 >> v2:172.21.15.29:3300/0 conn(0x7faf1c06f4f0 0x7faf1c0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf1affd640 1 -- 172.21.15.26:0/1216517095 >> v2:172.21.15.29:3300/0 conn(0x7faf1c06f4f0 msgr2=0x7faf1c0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf1affd640 1 --2- 172.21.15.26:0/1216517095 >> v2:172.21.15.29:3300/0 conn(0x7faf1c06f4f0 0x7faf1c0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf1affd640 1 -- 172.21.15.26:0/1216517095 >> v1:172.21.15.26:6789/0 conn(0x7faf1c0addb0 legacy=0x7faf1c154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.081+0000 7faf1affd640 1 -- 172.21.15.26:0/1216517095 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c0af460 con 0x7faf1c06f110 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.082+0000 7faf1affd640 1 --2- 172.21.15.26:0/1216517095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f110 0x7faf1c15f210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf1c158df0 tx=0x7faf0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704fde5cd5e19955 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.082+0000 7faf19ffb640 1 -- 172.21.15.26:0/1216517095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c037070 con 0x7faf1c06f110 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.082+0000 7faf19ffb640 1 -- 172.21.15.26:0/1216517095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c032440 con 0x7faf1c06f110 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.082+0000 7faf19ffb640 1 -- 172.21.15.26:0/1216517095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c032810 con 0x7faf1c06f110 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.083+0000 7faf217d5640 1 -- 172.21.15.26:0/1216517095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f110 msgr2=0x7faf1c15f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.083+0000 7faf217d5640 1 --2- 172.21.15.26:0/1216517095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f110 0x7faf1c15f210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf1c158df0 tx=0x7faf0c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.083+0000 7faf217d5640 1 -- 172.21.15.26:0/1216517095 shutdown_connections 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.083+0000 7faf217d5640 1 --2- 172.21.15.26:0/1216517095 >> v2:172.21.15.29:3300/0 conn(0x7faf1c06f4f0 0x7faf1c0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.083+0000 7faf217d5640 1 --2- 172.21.15.26:0/1216517095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f110 0x7faf1c15f210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.083+0000 7faf217d5640 1 -- 172.21.15.26:0/1216517095 >> 172.21.15.26:0/1216517095 conn(0x7faf1c0729a0 msgr2=0x7faf1c06b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.083+0000 7faf217d5640 1 -- 172.21.15.26:0/1216517095 shutdown_connections 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.083+0000 7faf217d5640 1 -- 172.21.15.26:0/1216517095 wait complete. 2024-04-24T23:21:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.084+0000 7faf217d5640 1 Processor -- start 2024-04-24T23:21:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.084+0000 7faf217d5640 1 -- start start 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf217d5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faf1c06f110 0x7faf1c155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf217d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f4f0 0x7faf1c160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf217d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf1c0addb0 0x7faf1c1f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf217d5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0afd00 con 0x7faf1c06f4f0 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf217d5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0afa20 con 0x7faf1c0addb0 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf217d5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c0aee50 con 0x7faf1c06f110 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f4f0 0x7faf1c160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f4f0 0x7faf1c160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40354/0 (socket says 172.21.15.26:40354) 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1a7fc640 1 -- 172.21.15.26:0/406220186 learned_addr learned my addr 172.21.15.26:0/406220186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1affd640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faf1c06f110 0x7faf1c155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1b7fe640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf1c0addb0 0x7faf1c1f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1a7fc640 1 -- 172.21.15.26:0/406220186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faf1c06f110 msgr2=0x7faf1c155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faf1c06f110 0x7faf1c155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1a7fc640 1 -- 172.21.15.26:0/406220186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf1c0addb0 msgr2=0x7faf1c1f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf1c0addb0 0x7faf1c1f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1a7fc640 1 -- 172.21.15.26:0/406220186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c156790 con 0x7faf1c06f4f0 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f4f0 0x7faf1c160bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf1000e9b0 tx=0x7faf1000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.085+0000 7faf037fe640 1 -- 172.21.15.26:0/406220186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10017070 con 0x7faf1c06f4f0 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.086+0000 7faf037fe640 1 -- 172.21.15.26:0/406220186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf1000c9a0 con 0x7faf1c06f4f0 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.086+0000 7faf217d5640 1 -- 172.21.15.26:0/406220186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c156a70 con 0x7faf1c06f4f0 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.086+0000 7faf037fe640 1 -- 172.21.15.26:0/406220186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1000cd70 con 0x7faf1c06f4f0 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.086+0000 7faf217d5640 1 -- 172.21.15.26:0/406220186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c1563f0 con 0x7faf1c06f4f0 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.087+0000 7faf037fe640 1 -- 172.21.15.26:0/406220186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf1001a070 con 0x7faf1c06f4f0 2024-04-24T23:21:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.087+0000 7faf037fe640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faf0806be30 0x7faf0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.088+0000 7faf037fe640 1 -- 172.21.15.26:0/406220186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faf1008b190 con 0x7faf1c06f4f0 2024-04-24T23:21:11.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.088+0000 7faf1affd640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faf0806be30 0x7faf0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.088+0000 7faf1affd640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faf0806be30 0x7faf0806e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf0c034000 tx=0x7faf0c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.089+0000 7faf245c1740 1 Processor -- start 2024-04-24T23:21:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.089+0000 7faf245c1740 1 -- start start 2024-04-24T23:21:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf245c1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f461034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf245c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 0x558f46133000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf245c1740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133540 0x558f46108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf245c1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558f4606bf00 con 0x558f46150b60 2024-04-24T23:21:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf245c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558f46014750 con 0x558f46133c80 2024-04-24T23:21:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf245c1740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x558f46067ec0 con 0x558f46133540 2024-04-24T23:21:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f461034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f461034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40370/0 (socket says 172.21.15.26:40370) 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1affd640 1 -- 172.21.15.26:0/3471698532 learned_addr learned my addr 172.21.15.26:0/3471698532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 0x558f46133000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1b7fe640 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133540 0x558f46108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1affd640 1 -- 172.21.15.26:0/3471698532 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133540 msgr2=0x558f46108350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1affd640 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133540 0x558f46108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1affd640 1 -- 172.21.15.26:0/3471698532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 msgr2=0x558f46133000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1affd640 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 0x558f46133000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1affd640 1 -- 172.21.15.26:0/3471698532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f45d802d0 con 0x558f46150b60 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 0x558f46133000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf1affd640 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f461034f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faf0c02a790 tx=0x7faf0c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a278b91abda18a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf00ff9640 1 -- 172.21.15.26:0/3471698532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c04d070 con 0x558f46150b60 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf00ff9640 1 -- 172.21.15.26:0/3471698532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c032d90 con 0x558f46150b60 2024-04-24T23:21:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.090+0000 7faf00ff9640 1 -- 172.21.15.26:0/3471698532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c037040 con 0x558f46150b60 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.091+0000 7faf245c1740 1 -- 172.21.15.26:0/3471698532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 msgr2=0x558f461034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.091+0000 7faf245c1740 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f461034f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faf0c02a790 tx=0x7faf0c02c000 comp rx=0 tx=0).stop 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.091+0000 7faf245c1740 1 -- 172.21.15.26:0/3471698532 shutdown_connections 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.091+0000 7faf245c1740 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133540 0x558f46108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.091+0000 7faf245c1740 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 0x558f46133000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.091+0000 7faf245c1740 1 --2- 172.21.15.26:0/3471698532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f461034f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.091+0000 7faf245c1740 1 -- 172.21.15.26:0/3471698532 >> 172.21.15.26:0/3471698532 conn(0x558f46105760 msgr2=0x558f4614e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.091+0000 7faf245c1740 1 -- 172.21.15.26:0/3471698532 shutdown_connections 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.091+0000 7faf245c1740 1 -- 172.21.15.26:0/3471698532 wait complete. 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.092+0000 7faf245c1740 1 -- start start 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.092+0000 7faf245c1740 1 Processor -- start 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.092+0000 7faf245c1740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133c80 0x558f46145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.092+0000 7faf245c1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f46123dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.092+0000 7faf245c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46136910 0x558f46102250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.092+0000 7faf245c1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558f46107910 con 0x558f46150b60 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.092+0000 7faf245c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558f46027cf0 con 0x558f46136910 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf245c1740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x558f46106d00 con 0x558f46133c80 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf245c1740 10 client.0 _ll_register_callbacks cb 0x558f460276e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f46123dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f46123dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40374/0 (socket says 172.21.15.26:40374) 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1a7fc640 1 -- 172.21.15.26:0/3894381259 learned_addr learned my addr 172.21.15.26:0/3894381259 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1a7fc640 1 -- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133c80 msgr2=0x558f46145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1affd640 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133c80 0x558f46145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1b7fe640 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46136910 0x558f46102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133c80 0x558f46145680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1a7fc640 1 -- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46136910 msgr2=0x558f46102250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46136910 0x558f46102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1a7fc640 1 -- 172.21.15.26:0/3894381259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f45e4a6b0 con 0x558f46150b60 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f46123dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faf10089770 tx=0x7faf10059d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71ac8aac2709d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faef9ffb640 1 -- 172.21.15.26:0/3894381259 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10021040 con 0x558f46150b60 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faef9ffb640 1 -- 172.21.15.26:0/3894381259 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faf10058b20 con 0x558f46150b60 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf245c1740 1 -- 172.21.15.26:0/3894381259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0c032d90 con 0x558f46150b60 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faef9ffb640 1 -- 172.21.15.26:0/3894381259 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7faf100620b0 con 0x558f46150b60 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf245c1740 10 client.4665 fetch_fsmap learned FSMap version 7 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faf245c1740 1 -- 172.21.15.26:0/3894381259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x558f46146a80 con 0x558f46150b60 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faef9ffb640 1 -- 172.21.15.26:0/3894381259 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c032d90 con 0x558f46150b60 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.093+0000 7faef9ffb640 1 -- 172.21.15.26:0/3894381259 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7faf10089ae0 con 0x558f46150b60 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.094+0000 7faf245c1740 10 client.4665 fetch_fsmap finished waiting for FSMap version 7 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.094+0000 7faf245c1740 10 client.4665 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:21:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.094+0000 7faf245c1740 20 client.4665 populate_metadata read hostname 'smithi026' 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.094+0000 7faf245c1740 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] conn(0x558f461213e0 0x558f461347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.094+0000 7faf245c1740 4 client.4665 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.094+0000 7faf245c1740 1 -- 172.21.15.26:0/3894381259 --> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f46079440 con 0x558f461213e0 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.094+0000 7faf1affd640 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] conn(0x558f461213e0 0x558f461347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.094+0000 7faf1affd640 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] conn(0x558f461213e0 0x558f461347b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a55a1abae7d8573 server_cookie=d714dbb35877c97f in_seq=0 out_seq=0 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.094+0000 7faef9ffb640 10 client.4665 ms_handle_connect on v2:172.21.15.29:6834/1604449840 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faef9ffb640 1 -- 172.21.15.26:0/3894381259 <== mds.0 v2:172.21.15.29:6834/1604449840 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f46079440 con 0x558f461213e0 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faef9ffb640 10 client.4665 handle_command_reply: tid=0 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 client.4665 shutdown 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 -- 172.21.15.26:0/3894381259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 msgr2=0x558f46123dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f46123dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faf10089770 tx=0x7faf10059d10 comp rx=0 tx=0).stop 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 -- 172.21.15.26:0/3894381259 shutdown_connections 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] conn(0x558f461213e0 0x558f461347b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46136910 0x558f46102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46150b60 0x558f46123dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faef9ffb640 0 client.4665 ms_handle_reset on v2:172.21.15.29:6834/1604449840 2024-04-24T23:21:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 --2- 172.21.15.26:0/3894381259 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f46133c80 0x558f46145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 -- 172.21.15.26:0/3894381259 >> 172.21.15.26:0/3894381259 conn(0x558f46133540 msgr2=0x558f4614e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 -- 172.21.15.26:0/3894381259 shutdown_connections 2024-04-24T23:21:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.095+0000 7faf245c1740 1 -- 172.21.15.26:0/3894381259 wait complete. 2024-04-24T23:21:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.096+0000 7faf245c1740 20 client.4665 trim_cache size 0 max 16384 2024-04-24T23:21:11.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.114+0000 7faf245c1740 1 Processor -- start 2024-04-24T23:21:11.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.114+0000 7faf245c1740 1 -- start start 2024-04-24T23:21:11.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf245c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133540 0x558f4612e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf245c1740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 0x558f4615d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf245c1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133c80 0x558f46104230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf245c1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c148500 con 0x558f46133c80 2024-04-24T23:21:11.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf245c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c158cd0 con 0x558f46133540 2024-04-24T23:21:11.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf245c1740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c147eb0 con 0x558f461213e0 2024-04-24T23:21:11.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133c80 0x558f46104230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133c80 0x558f46104230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40376/0 (socket says 172.21.15.26:40376) 2024-04-24T23:21:11.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1b7fe640 1 -- 172.21.15.26:0/4010015523 learned_addr learned my addr 172.21.15.26:0/4010015523 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1affd640 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133540 0x558f4612e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 0x558f4615d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1b7fe640 1 -- 172.21.15.26:0/4010015523 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 msgr2=0x558f4615d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1b7fe640 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 0x558f4615d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1b7fe640 1 -- 172.21.15.26:0/4010015523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133540 msgr2=0x558f4612e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1b7fe640 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133540 0x558f4612e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1b7fe640 1 -- 172.21.15.26:0/4010015523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f46134ec0 con 0x558f46133c80 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1affd640 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133540 0x558f4612e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faf1b7fe640 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133c80 0x558f46104230 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faf14001e90 tx=0x7faf14010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5438e2837493c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faefbfff640 1 -- 172.21.15.26:0/4010015523 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14014660 con 0x558f46133c80 2024-04-24T23:21:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faefbfff640 1 -- 172.21.15.26:0/4010015523 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf14019070 con 0x558f46133c80 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.115+0000 7faefbfff640 1 -- 172.21.15.26:0/4010015523 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14014a00 con 0x558f46133c80 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 -- 172.21.15.26:0/4010015523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133c80 msgr2=0x558f46104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133c80 0x558f46104230 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faf14001e90 tx=0x7faf14010610 comp rx=0 tx=0).stop 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 -- 172.21.15.26:0/4010015523 shutdown_connections 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133c80 0x558f46104230 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 0x558f4615d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 --2- 172.21.15.26:0/4010015523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133540 0x558f4612e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 -- 172.21.15.26:0/4010015523 >> 172.21.15.26:0/4010015523 conn(0x558f461308a0 msgr2=0x558f461347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 -- 172.21.15.26:0/4010015523 shutdown_connections 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 -- 172.21.15.26:0/4010015523 wait complete. 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.116+0000 7faf245c1740 1 -- start start 2024-04-24T23:21:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf245c1740 1 Processor -- start 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf245c1740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 0x558f4614ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf245c1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133540 0x558f461c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf245c1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 0x558f461c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf245c1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558f46121260 con 0x558f46133540 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf245c1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558f461220a0 con 0x558f46133c80 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf245c1740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x558f461218b0 con 0x558f461213e0 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf245c1740 10 client.0 _ll_register_callbacks cb 0x7faf100590a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf1a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133540 0x558f461c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf1a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133540 0x558f461c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40380/0 (socket says 172.21.15.26:40380) 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf1a7fc640 1 -- 172.21.15.26:0/1946666072 learned_addr learned my addr 172.21.15.26:0/1946666072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf1affd640 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 0x558f4614ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.117+0000 7faf1b7fe640 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 0x558f461c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf1a7fc640 1 -- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 msgr2=0x558f4614ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 0x558f4614ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf1a7fc640 1 -- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 msgr2=0x558f461c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 0x558f461c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf1a7fc640 1 -- 172.21.15.26:0/1946666072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f46134c30 con 0x558f46133540 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf1a7fc640 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133540 0x558f461c7750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf100897a0 tx=0x7faf10089620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed87c18da14d212f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faef9ffb640 1 -- 172.21.15.26:0/1946666072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10021040 con 0x558f46133540 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faef9ffb640 1 -- 172.21.15.26:0/1946666072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faf10078520 con 0x558f46133540 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf245c1740 1 -- 172.21.15.26:0/1946666072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf14019070 con 0x558f46133540 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faef9ffb640 1 -- 172.21.15.26:0/1946666072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7faf10089200 con 0x558f46133540 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf245c1740 10 client.4671 fetch_fsmap learned FSMap version 7 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf245c1740 1 -- 172.21.15.26:0/1946666072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x558f461c9650 con 0x558f46133540 2024-04-24T23:21:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faef9ffb640 1 -- 172.21.15.26:0/1946666072 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf14019070 con 0x558f46133540 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faef9ffb640 1 -- 172.21.15.26:0/1946666072 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7faf10055220 con 0x558f46133540 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf245c1740 10 client.4671 fetch_fsmap finished waiting for FSMap version 7 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf245c1740 10 client.4671 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf245c1740 20 client.4671 populate_metadata read hostname 'smithi026' 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf245c1740 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] conn(0x558f46136910 0x558f4615d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf245c1740 4 client.4671 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.118+0000 7faf245c1740 1 -- 172.21.15.26:0/1946666072 --> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] -- command(tid 0: {"prefix": "status"}) -- 0x558f461481d0 con 0x558f46136910 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf1affd640 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] conn(0x558f46136910 0x558f4615d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf1affd640 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] conn(0x558f46136910 0x558f4615d170 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b97531c9945d4c2b server_cookie=37b5fc713ac9c9c1 in_seq=0 out_seq=0 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faef9ffb640 10 client.4671 ms_handle_connect on v2:172.21.15.29:6834/1604449840 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faef9ffb640 1 -- 172.21.15.26:0/1946666072 <== mds.0 v2:172.21.15.29:6834/1604449840 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x558f461481d0 con 0x558f46136910 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faef9ffb640 10 client.4671 handle_command_reply: tid=0 2024-04-24T23:21:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 client.4671 shutdown 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 -- 172.21.15.26:0/1946666072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133540 msgr2=0x558f461c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133540 0x558f461c7750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf100897a0 tx=0x7faf10089620 comp rx=0 tx=0).stop 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 -- 172.21.15.26:0/1946666072 shutdown_connections 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:6834/1604449840,v1:172.21.15.29:6835/1604449840] conn(0x558f46136910 0x558f4615d170 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558f46133c80 0x558f461c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faef9ffb640 0 client.4671 ms_handle_reset on v2:172.21.15.29:6834/1604449840 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f46133540 0x558f461c7750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 --2- 172.21.15.26:0/1946666072 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558f461213e0 0x558f4614ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 -- 172.21.15.26:0/1946666072 >> 172.21.15.26:0/1946666072 conn(0x558f461308a0 msgr2=0x558f461342f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 -- 172.21.15.26:0/1946666072 shutdown_connections 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 1 -- 172.21.15.26:0/1946666072 wait complete. 2024-04-24T23:21:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.119+0000 7faf245c1740 20 client.4671 trim_cache size 0 max 16384 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4287, 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 15.649947016, 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 7, 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 20, 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 20, 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 22.297273665999999 2024-04-24T23:21:11.131 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:21:11.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 -- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faf0806be30 msgr2=0x7faf0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faf0806be30 0x7faf0806e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf0c034000 tx=0x7faf0c02e000 comp rx=0 tx=0).stop 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 -- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f4f0 msgr2=0x7faf1c160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f4f0 0x7faf1c160bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf1000e9b0 tx=0x7faf1000ee70 comp rx=0 tx=0).stop 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 -- 172.21.15.26:0/406220186 shutdown_connections 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faf0806be30 0x7faf0806e2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf1c0addb0 0x7faf1c1f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c06f4f0 0x7faf1c160bf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 --2- 172.21.15.26:0/406220186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faf1c06f110 0x7faf1c155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 -- 172.21.15.26:0/406220186 >> 172.21.15.26:0/406220186 conn(0x7faf1c0729a0 msgr2=0x7faf1c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 -- 172.21.15.26:0/406220186 shutdown_connections 2024-04-24T23:21:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.120+0000 7faf217d5640 1 -- 172.21.15.26:0/406220186 wait complete. 2024-04-24T23:21:11.134 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T23:21:11.145 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T23:21:11.145 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T23:21:11.146 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T23:21:11.146 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T23:21:11.182 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T23:21:11.182 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-24T23:21:11.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:11.182 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:21:11.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:11.198 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-24T23:21:11.265 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-04-24T23:21:11.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:11.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:11.267 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:21:11.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:11.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-24T23:21:11.388 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-04-24T23:21:11.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:11.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:11.389 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-24T23:21:11.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:11.409 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-04-24T23:21:11.481 INFO:teuthology.orchestra.run.smithi029.stderr:Cannot find device "ceph-brx" 2024-04-24T23:21:11.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:11.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:11.484 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-24T23:21:11.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:11.541 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-04-24T23:21:11.609 INFO:teuthology.orchestra.run.smithi029.stderr:Cannot find device "ceph-brx" 2024-04-24T23:21:11.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:11.610 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-24T23:21:11.611 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:21:11.611 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:21:11.611 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:21:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.709+0000 7fe015e10640 1 Processor -- start 2024-04-24T23:21:11.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.709+0000 7fe015e10640 1 -- start start 2024-04-24T23:21:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.710+0000 7fe015e10640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe01015e290 0x7fe0100d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.710+0000 7fe015e10640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0100b3ac0 0x7fe0100b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.710+0000 7fe015e10640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0100b5840 con 0x7fe01006c230 2024-04-24T23:21:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.710+0000 7fe015e10640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe0100b4460 con 0x7fe01015e290 2024-04-24T23:21:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.710+0000 7fe015e10640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe0100b5e50 con 0x7fe0100b3ac0 2024-04-24T23:21:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.710+0000 7fe00f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe01015e290 0x7fe0100d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.710+0000 7fe00ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe01006c230 0x7fe01006c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54770/0 (socket says 172.21.15.26:54770) 2024-04-24T23:21:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.710+0000 7fe00ffff640 1 -- 172.21.15.26:0/51777149 learned_addr learned my addr 172.21.15.26:0/51777149 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.710+0000 7fe00effd640 1 --2- 172.21.15.26:0/51777149 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0100b3ac0 0x7fe0100b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.711+0000 7fe00f7fe640 1 -- 172.21.15.26:0/51777149 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0100b3ac0 msgr2=0x7fe0100b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.711+0000 7fe00f7fe640 1 --2- 172.21.15.26:0/51777149 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0100b3ac0 0x7fe0100b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.711+0000 7fe00f7fe640 1 -- 172.21.15.26:0/51777149 >> v1:172.21.15.26:6789/0 conn(0x7fe01006c230 legacy=0x7fe01006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.711+0000 7fe00f7fe640 1 -- 172.21.15.26:0/51777149 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0100cfb30 con 0x7fe01015e290 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.711+0000 7fe00e7fc640 1 -- 172.21.15.26:0/51777149 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016786160 0 0) 0x7fe0100b5840 con 0x7fe01006c230 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.711+0000 7fe00f7fe640 1 --2- 172.21.15.26:0/51777149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01015e290 0x7fe0100d1730 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe0000097a0 tx=0x7fe00002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bc1bb1c88f3433 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.711+0000 7fe00e7fc640 1 -- 172.21.15.26:0/51777149 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000037070 con 0x7fe01015e290 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.711+0000 7fe00e7fc640 1 -- 172.21.15.26:0/51777149 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe000032440 con 0x7fe01015e290 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.711+0000 7fe00e7fc640 1 -- 172.21.15.26:0/51777149 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000032810 con 0x7fe01015e290 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.712+0000 7fe015e10640 1 -- 172.21.15.26:0/51777149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01015e290 msgr2=0x7fe0100d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.712+0000 7fe015e10640 1 --2- 172.21.15.26:0/51777149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01015e290 0x7fe0100d1730 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe0000097a0 tx=0x7fe00002a180 comp rx=0 tx=0).stop 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.712+0000 7fe015e10640 1 -- 172.21.15.26:0/51777149 shutdown_connections 2024-04-24T23:21:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.712+0000 7fe015e10640 1 --2- 172.21.15.26:0/51777149 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0100b3ac0 0x7fe0100b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.712+0000 7fe015e10640 1 --2- 172.21.15.26:0/51777149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01015e290 0x7fe0100d1730 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.712+0000 7fe015e10640 1 -- 172.21.15.26:0/51777149 >> 172.21.15.26:0/51777149 conn(0x7fe01006b120 msgr2=0x7fe01006b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.712+0000 7fe015e10640 1 -- 172.21.15.26:0/51777149 shutdown_connections 2024-04-24T23:21:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.712+0000 7fe015e10640 1 -- 172.21.15.26:0/51777149 wait complete. 2024-04-24T23:21:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.713+0000 7fe015e10640 1 Processor -- start 2024-04-24T23:21:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.713+0000 7fe015e10640 1 -- start start 2024-04-24T23:21:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe015e10640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01006c230 0x7fe0100c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe015e10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0100b3ac0 0x7fe0100ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe015e10640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe01015e290 0x7fe0100c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe015e10640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0100b5ae0 con 0x7fe01006c230 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe015e10640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0100d00f0 con 0x7fe0100b3ac0 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe015e10640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe01015e670 con 0x7fe01015e290 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01006c230 0x7fe0100c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01006c230 0x7fe0100c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40394/0 (socket says 172.21.15.26:40394) 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00f7fe640 1 -- 172.21.15.26:0/3809659040 learned_addr learned my addr 172.21.15.26:0/3809659040 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00ffff640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe01015e290 0x7fe0100c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00effd640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0100b3ac0 0x7fe0100ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00f7fe640 1 -- 172.21.15.26:0/3809659040 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe01015e290 msgr2=0x7fe0100c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00f7fe640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe01015e290 0x7fe0100c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00f7fe640 1 -- 172.21.15.26:0/3809659040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0100b3ac0 msgr2=0x7fe0100ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00f7fe640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0100b3ac0 0x7fe0100ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.714+0000 7fe00f7fe640 1 -- 172.21.15.26:0/3809659040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0100c93a0 con 0x7fe01006c230 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.715+0000 7fe00f7fe640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01006c230 0x7fe0100c64a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe0000097a0 tx=0x7fe0000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.715+0000 7fdff7fff640 1 -- 172.21.15.26:0/3809659040 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000037050 con 0x7fe01006c230 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.715+0000 7fdff7fff640 1 -- 172.21.15.26:0/3809659040 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe000038040 con 0x7fe01006c230 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.715+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0100c9680 con 0x7fe01006c230 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.715+0000 7fdff7fff640 1 -- 172.21.15.26:0/3809659040 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000004470 con 0x7fe01006c230 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.715+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0100c9c40 con 0x7fe01006c230 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.716+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0100694c0 con 0x7fe01006c230 2024-04-24T23:21:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.716+0000 7fdff7fff640 1 -- 172.21.15.26:0/3809659040 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe000030900 con 0x7fe01006c230 2024-04-24T23:21:11.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.717+0000 7fdff7fff640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdffc06be30 0x7fdffc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.717+0000 7fdff7fff640 1 -- 172.21.15.26:0/3809659040 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe0000ab9d0 con 0x7fe01006c230 2024-04-24T23:21:11.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.717+0000 7fe00effd640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdffc06be30 0x7fdffc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.717+0000 7fe00effd640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdffc06be30 0x7fdffc06e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe0101f70a0 tx=0x7fe004008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.722+0000 7fdff7fff640 1 -- 172.21.15.26:0/3809659040 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe00007a370 con 0x7fe01006c230 2024-04-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.944+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe0100c76e0 con 0x7fe01006c230 2024-04-24T23:21:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.944+0000 7fdff7fff640 1 -- 172.21.15.26:0/3809659040 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fe000080020 con 0x7fe01006c230 2024-04-24T23:21:11.947 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:21:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdffc06be30 msgr2=0x7fdffc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdffc06be30 0x7fdffc06e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe0101f70a0 tx=0x7fe004008040 comp rx=0 tx=0).stop 2024-04-24T23:21:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01006c230 msgr2=0x7fe0100c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01006c230 0x7fe0100c64a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe0000097a0 tx=0x7fe0000046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 shutdown_connections 2024-04-24T23:21:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdffc06be30 0x7fdffc06e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe01015e290 0x7fe0100c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0100b3ac0 0x7fe0100ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 --2- 172.21.15.26:0/3809659040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe01006c230 0x7fe0100c64a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 >> 172.21.15.26:0/3809659040 conn(0x7fe01006b120 msgr2=0x7fe01015d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 shutdown_connections 2024-04-24T23:21:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:11.946+0000 7fe015e10640 1 -- 172.21.15.26:0/3809659040 wait complete. 2024-04-24T23:21:11.960 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:21:11.960 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:21:11.960 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-24T23:21:11.960 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:21:11.960 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:21:11.960 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:21:11.960 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:21:11.960 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:21:11.960 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:21:11.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:11.961 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-24T23:21:11.976 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:21:11.976 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:21:11.976 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:21:11.976 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:11.976 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:21:11.976 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:11.976 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:11.976 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42721sec preferred_lft 42721sec 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:11.977 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:11.978 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-24T23:21:11.978 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:21:11.978 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:11.978 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:11.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:21:11.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-04-24T23:21:11.979 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-04-24T23:21:11.979 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:21:11.979 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:12.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:12.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:12.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:12.154 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:21:12.179 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-24T23:21:12.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:12.181 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-24T23:21:12.247 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-24T23:21:12.247 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:21:12.247 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:21:12.247 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:21:12.247 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:21:12.248 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:12.248 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:12.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:21:12.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:21:12.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:21:12.248 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:12.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:12.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:12.510 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:21:12.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:12.527 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-04-24T23:21:12.584 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:12.584 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:12.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:21:12.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T23:21:12.584 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:12.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:12.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:12.704 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T23:21:12.704 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:12.704 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:12.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T23:21:12.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:21:12.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T23:21:12.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T23:21:12.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T23:21:12.705 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:12.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:12.847 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:12.847 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:12.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-24T23:21:12.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:21:12.847 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:12.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:12.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:12.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:12.928 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:21:12.956 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:21:12.956 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:12.956 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:21:13.012 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:21:13.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:13.013 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:21:13.118 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:21:13.118 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:21:13.118 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:21:13.229 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:21:13.229 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:21:13.229 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:21:13.229 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:13.229 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:13.234 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:21:13.234 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:21:13.234 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:21:13.234 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:13.234 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:13.234 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:21:13.234 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:21:13.235 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:21:13.236 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:21:13.236 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:21:13.236 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:21:13.236 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:21:13.236 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:21:13.238 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:21:13.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:13.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:21:13.269 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:13.370 INFO:teuthology.orchestra.run.smithi026.stdout:client4570 2024-04-24T23:21:13.370 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:21:13.370 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:13.387 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:13.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:13.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:13.488 INFO:teuthology.orchestra.run.smithi026.stdout:client4570 2024-04-24T23:21:13.489 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:13.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:13.550 INFO:teuthology.orchestra.run.smithi026.stdout:client4570 2024-04-24T23:21:13.550 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5a872e4b-97fd-4255-95a5-3ea24d1048cf.client4570/status') 2024-04-24T23:21:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:13.613 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.4570 (3)172.21.15.26:0/1413685160 2024-04-24T23:21:13.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-04-24T23:21:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-04-24T23:21:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-04-24T23:21:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:70 bytes copied, 2.461e-05 s, 2.8 MB/s 2024-04-24T23:21:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:13.622 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-24T23:21:13.622 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:21:13.622 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T23:21:13.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:21:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.723+0000 7f1934064640 1 Processor -- start 2024-04-24T23:21:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.724+0000 7f1934064640 1 -- start start 2024-04-24T23:21:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.724+0000 7f1934064640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f192c15eb00 0x7f192c15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.724+0000 7f1934064640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192c15f450 con 0x7f192c0cf870 2024-04-24T23:21:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.724+0000 7f1934064640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192c0cd0b0 con 0x7f192c143c40 2024-04-24T23:21:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.724+0000 7f1934064640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f192c071b70 con 0x7f192c15eb00 2024-04-24T23:21:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.725+0000 7f19315ff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f192c0cf870 0x7f192c072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41154/0 (socket says 172.21.15.26:41154) 2024-04-24T23:21:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.725+0000 7f19315ff640 1 -- 172.21.15.26:0/440409771 learned_addr learned my addr 172.21.15.26:0/440409771 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.725+0000 7f1932601640 1 --2- 172.21.15.26:0/440409771 >> v2:172.21.15.26:3300/0 conn(0x7f192c15eb00 0x7f192c15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.725+0000 7f1930dfe640 1 -- 172.21.15.26:0/440409771 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3087000458 0 0) 0x7f1928001110 con 0x7f192c0cf870 2024-04-24T23:21:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.725+0000 7f1930dfe640 1 -- 172.21.15.26:0/440409771 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1918003520 con 0x7f192c0cf870 2024-04-24T23:21:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.725+0000 7f1932601640 1 -- 172.21.15.26:0/440409771 >> v1:172.21.15.29:6789/0 conn(0x7f192c143c40 legacy=0x7f192c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.725+0000 7f1932601640 1 -- 172.21.15.26:0/440409771 >> v1:172.21.15.26:6789/0 conn(0x7f192c0cf870 legacy=0x7f192c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.725+0000 7f1932601640 1 -- 172.21.15.26:0/440409771 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192c071240 con 0x7f192c15eb00 2024-04-24T23:21:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.726+0000 7f1932601640 1 --2- 172.21.15.26:0/440409771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c15eb00 0x7f192c15ef10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1924009d90 tx=0x7f192402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2915b23dfe1178 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.726+0000 7f1930dfe640 1 -- 172.21.15.26:0/440409771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924007740 con 0x7f192c15eb00 2024-04-24T23:21:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.726+0000 7f1930dfe640 1 -- 172.21.15.26:0/440409771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19240078e0 con 0x7f192c15eb00 2024-04-24T23:21:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.726+0000 7f1930dfe640 1 -- 172.21.15.26:0/440409771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924007cb0 con 0x7f192c15eb00 2024-04-24T23:21:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.727+0000 7f1934064640 1 -- 172.21.15.26:0/440409771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c15eb00 msgr2=0x7f192c15ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.727+0000 7f1934064640 1 --2- 172.21.15.26:0/440409771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c15eb00 0x7f192c15ef10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1924009d90 tx=0x7f192402a6f0 comp rx=0 tx=0).stop 2024-04-24T23:21:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.727+0000 7f1934064640 1 -- 172.21.15.26:0/440409771 shutdown_connections 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.727+0000 7f1934064640 1 --2- 172.21.15.26:0/440409771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c15eb00 0x7f192c15ef10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.727+0000 7f1934064640 1 -- 172.21.15.26:0/440409771 >> 172.21.15.26:0/440409771 conn(0x7f192c06b300 msgr2=0x7f192c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.727+0000 7f1934064640 1 -- 172.21.15.26:0/440409771 shutdown_connections 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.727+0000 7f1934064640 1 -- 172.21.15.26:0/440409771 wait complete. 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.728+0000 7f1934064640 1 Processor -- start 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.728+0000 7f1934064640 1 -- start start 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f1934064640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f192c0cf870 0x7f192c15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f1934064640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c143c40 0x7f192c0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f1934064640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f192c15eb00 0x7f192c0ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f1934064640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f192c144020 con 0x7f192c143c40 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f1934064640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f192c071800 con 0x7f192c15eb00 2024-04-24T23:21:13.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f1934064640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f192c0b2e80 con 0x7f192c0cf870 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19315ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c143c40 0x7f192c0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19315ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c143c40 0x7f192c0ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53364/0 (socket says 172.21.15.26:53364) 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19315ff640 1 -- 172.21.15.26:0/2391970155 learned_addr learned my addr 172.21.15.26:0/2391970155 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f1931e00640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f192c0cf870 0x7f192c15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f1932601640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f192c15eb00 0x7f192c0ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19315ff640 1 -- 172.21.15.26:0/2391970155 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f192c0cf870 msgr2=0x7f192c15a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19315ff640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f192c0cf870 0x7f192c15a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19315ff640 1 -- 172.21.15.26:0/2391970155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f192c15eb00 msgr2=0x7f192c0ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19315ff640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f192c15eb00 0x7f192c0ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19315ff640 1 -- 172.21.15.26:0/2391970155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192c132590 con 0x7f192c143c40 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19315ff640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c143c40 0x7f192c0ac970 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f192800c730 tx=0x7f192800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.729+0000 7f19227fc640 1 -- 172.21.15.26:0/2391970155 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928017070 con 0x7f192c143c40 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.730+0000 7f19227fc640 1 -- 172.21.15.26:0/2391970155 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1928005ab0 con 0x7f192c143c40 2024-04-24T23:21:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.730+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192c132870 con 0x7f192c143c40 2024-04-24T23:21:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.730+0000 7f19227fc640 1 -- 172.21.15.26:0/2391970155 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928007070 con 0x7f192c143c40 2024-04-24T23:21:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.730+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192c132e30 con 0x7f192c143c40 2024-04-24T23:21:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.731+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f192c0694c0 con 0x7f192c143c40 2024-04-24T23:21:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.731+0000 7f19227fc640 1 -- 172.21.15.26:0/2391970155 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f192801a070 con 0x7f192c143c40 2024-04-24T23:21:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.731+0000 7f19227fc640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f191006be30 0x7f191006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.732+0000 7f19227fc640 1 -- 172.21.15.26:0/2391970155 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f192808a270 con 0x7f192c143c40 2024-04-24T23:21:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.732+0000 7f1931e00640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f191006be30 0x7f191006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.732+0000 7f1931e00640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f191006be30 0x7f191006e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f191c001780 tx=0x7f191c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:13.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.736+0000 7f19227fc640 1 -- 172.21.15.26:0/2391970155 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f192c15f450 con 0x7f192c143c40 2024-04-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.959+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f192c15bb20 con 0x7f192c143c40 2024-04-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.959+0000 7f19227fc640 1 -- 172.21.15.26:0/2391970155 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f1928058530 con 0x7f192c143c40 2024-04-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f191006be30 msgr2=0x7f191006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f191006be30 0x7f191006e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f191c001780 tx=0x7f191c009040 comp rx=0 tx=0).stop 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c143c40 msgr2=0x7f192c0ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c143c40 0x7f192c0ac970 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f192800c730 tx=0x7f192800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 shutdown_connections 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f191006be30 0x7f191006e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f192c15eb00 0x7f192c0ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f192c143c40 0x7f192c0ac970 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 --2- 172.21.15.26:0/2391970155 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f192c0cf870 0x7f192c15a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 >> 172.21.15.26:0/2391970155 conn(0x7f192c06b300 msgr2=0x7f192c15e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 shutdown_connections 2024-04-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:13.962+0000 7f1934064640 1 -- 172.21.15.26:0/2391970155 wait complete. 2024-04-24T23:21:13.974 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:21:13.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:21:13.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-24T23:21:13.975 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-24T23:21:13.975 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T23:21:13.975 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:21:13.975 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:21:13.975 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:21:13.975 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-24T23:21:13.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:13.975 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-24T23:21:13.991 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:21:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:21:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-24T23:21:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42719sec preferred_lft 42719sec 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c2:ea:e0:db:00:e0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ccf2:1eff:fe9b:4749/64 scope link 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:21:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c2:ea:e0:db:00:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:21:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c0ea:e0ff:fedb:e0/64 scope link tentative 2024-04-24T23:21:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:13.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:13.994 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:21:14.054 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-24T23:21:14.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:14.057 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-24T23:21:14.113 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-24T23:21:14.113 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:21:14.114 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:21:14.114 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:21:14.114 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:21:14.114 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:14.114 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:14.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:21:14.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:21:14.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:21:14.114 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:14.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:14.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:14.240 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:21:14.256 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:21:14.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:14.256 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-04-24T23:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-24T23:21:14.311 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:14.311 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:14.311 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:21:14.312 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-24T23:21:14.312 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:14.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:14.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:14.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:14.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T23:21:14.453 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:21:14.453 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:21:14.453 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:21:14.454 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:14.454 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:21:14.454 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:14.455 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:21:14.455 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether da:bd:65:9c:aa:d8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T23:21:14.455 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T23:21:14.455 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:14.455 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::d8bd:65ff:fe9c:aad8/64 scope link 2024-04-24T23:21:14.455 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:14.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:14.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether da:bd:65:9c:aa:d8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::d8bd:65ff:fe9c:aad8/64 scope link 2024-04-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:14.523 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-24T23:21:14.523 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:14.523 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:14.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-24T23:21:14.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T23:21:14.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-24T23:21:14.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-24T23:21:14.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-24T23:21:14.524 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:14.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:14.673 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:14.673 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:14.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.1 up 2024-04-24T23:21:14.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T23:21:14.674 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:14.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:14.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:14.750 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-24T23:21:14.765 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-24T23:21:14.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:14.766 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-24T23:21:14.820 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:21:14.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:14.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-24T23:21:14.903 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-24T23:21:14.903 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-24T23:21:14.903 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:21:14.911 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-24T23:21:14.911 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-24T23:21:14.911 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:21:14.911 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:14.911 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:14.929 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-24T23:21:14.930 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-24T23:21:14.930 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:21:14.930 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:14.930 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:14.930 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:21:14.930 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:21:14.930 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:21:14.931 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-24T23:21:14.931 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:21:14.931 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:21:14.931 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:21:14.931 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:21:14.932 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:21:14.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:14.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-24T23:21:14.997 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:15.095 INFO:teuthology.orchestra.run.smithi026.stdout:client4692 2024-04-24T23:21:15.096 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:21:15.096 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:15.112 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:15.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:15.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:15.209 INFO:teuthology.orchestra.run.smithi026.stdout:client4692 2024-04-24T23:21:15.209 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:15.266 INFO:teuthology.orchestra.run.smithi026.stdout:client4692 2024-04-24T23:21:15.266 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5a872e4b-97fd-4255-95a5-3ea24d1048cf.client4692/status') 2024-04-24T23:21:15.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:15.325 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.4692 (3)192.168.0.2:0/3315129998 2024-04-24T23:21:15.325 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-04-24T23:21:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-04-24T23:21:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-04-24T23:21:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:69 bytes copied, 2.3741e-05 s, 2.9 MB/s 2024-04-24T23:21:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:15.332 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-24T23:21:15.332 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:21:15.332 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:21:15.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:21:15.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.432+0000 7fb3388f5640 1 Processor -- start 2024-04-24T23:21:15.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.432+0000 7fb3388f5640 1 -- start start 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.432+0000 7fb3388f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb334143c40 0x7fb3340cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb3388f5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb33415eb00 0x7fb33415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb3388f5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3340cd0b0 con 0x7fb3340cc860 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb3388f5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb33415f490 con 0x7fb33415eb00 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb3388f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb334071b70 con 0x7fb334143c40 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb332d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb33415eb00 0x7fb33415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb332d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb33415eb00 0x7fb33415ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53376/0 (socket says 172.21.15.26:53376) 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb332d9d640 1 -- 172.21.15.26:0/272495041 learned_addr learned my addr 172.21.15.26:0/272495041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb33259c640 1 --2- 172.21.15.26:0/272495041 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb334143c40 0x7fb3340cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb332d9d640 1 -- 172.21.15.26:0/272495041 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb334143c40 msgr2=0x7fb3340cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb332d9d640 1 --2- 172.21.15.26:0/272495041 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb334143c40 0x7fb3340cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb332d9d640 1 -- 172.21.15.26:0/272495041 >> v1:172.21.15.29:6789/0 conn(0x7fb3340cc860 legacy=0x7fb334072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb332d9d640 1 -- 172.21.15.26:0/272495041 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb334071240 con 0x7fb33415eb00 2024-04-24T23:21:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.433+0000 7fb33259c640 1 --2- 172.21.15.26:0/272495041 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb334143c40 0x7fb3340cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.434+0000 7fb332d9d640 1 --2- 172.21.15.26:0/272495041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33415eb00 0x7fb33415ef50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009e50 tx=0x7fb32c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c59db20ad1c9fde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.434+0000 7fb331d9b640 1 -- 172.21.15.26:0/272495041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c036070 con 0x7fb33415eb00 2024-04-24T23:21:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.434+0000 7fb331d9b640 1 -- 172.21.15.26:0/272495041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32c031600 con 0x7fb33415eb00 2024-04-24T23:21:15.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.434+0000 7fb331d9b640 1 -- 172.21.15.26:0/272495041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0319d0 con 0x7fb33415eb00 2024-04-24T23:21:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.435+0000 7fb3388f5640 1 -- 172.21.15.26:0/272495041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33415eb00 msgr2=0x7fb33415ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.435+0000 7fb3388f5640 1 --2- 172.21.15.26:0/272495041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33415eb00 0x7fb33415ef50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009e50 tx=0x7fb32c02a6c0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.435+0000 7fb3388f5640 1 -- 172.21.15.26:0/272495041 shutdown_connections 2024-04-24T23:21:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.435+0000 7fb3388f5640 1 --2- 172.21.15.26:0/272495041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb33415eb00 0x7fb33415ef50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.435+0000 7fb3388f5640 1 --2- 172.21.15.26:0/272495041 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb334143c40 0x7fb3340cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.435+0000 7fb3388f5640 1 -- 172.21.15.26:0/272495041 >> 172.21.15.26:0/272495041 conn(0x7fb33406b300 msgr2=0x7fb33406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.435+0000 7fb3388f5640 1 -- 172.21.15.26:0/272495041 shutdown_connections 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.435+0000 7fb3388f5640 1 -- 172.21.15.26:0/272495041 wait complete. 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.436+0000 7fb3388f5640 1 Processor -- start 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.436+0000 7fb3388f5640 1 -- start start 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb3388f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3340cc860 0x7fb33414baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb3388f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb33413ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb3388f5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb33415eb00 0x7fb334137630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb3388f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb334144020 con 0x7fb334143c40 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb3388f5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb334071800 con 0x7fb33415eb00 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb3388f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb3340ccca0 con 0x7fb3340cc860 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb33413ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb33413ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53388/0 (socket says 172.21.15.26:53388) 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32bfff640 1 -- 172.21.15.26:0/3212095377 learned_addr learned my addr 172.21.15.26:0/3212095377 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb332d9d640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb33415eb00 0x7fb334137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb33259c640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3340cc860 0x7fb33414baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32bfff640 1 -- 172.21.15.26:0/3212095377 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3340cc860 msgr2=0x7fb33414baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32bfff640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3340cc860 0x7fb33414baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32bfff640 1 -- 172.21.15.26:0/3212095377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb33415eb00 msgr2=0x7fb334137630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32bfff640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb33415eb00 0x7fb334137630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32bfff640 1 -- 172.21.15.26:0/3212095377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33413d6a0 con 0x7fb334143c40 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb332d9d640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb33415eb00 0x7fb334137630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32bfff640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb33413ce50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00c430 tx=0x7fb31c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32affd640 1 -- 172.21.15.26:0/3212095377 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c011790 con 0x7fb334143c40 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.437+0000 7fb32affd640 1 -- 172.21.15.26:0/3212095377 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb31c017070 con 0x7fb334143c40 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.438+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33413d980 con 0x7fb334143c40 2024-04-24T23:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.438+0000 7fb32affd640 1 -- 172.21.15.26:0/3212095377 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c011bb0 con 0x7fb334143c40 2024-04-24T23:21:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.438+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3341490b0 con 0x7fb334143c40 2024-04-24T23:21:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.439+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3340694c0 con 0x7fb334143c40 2024-04-24T23:21:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.439+0000 7fb32affd640 1 -- 172.21.15.26:0/3212095377 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb31c01c460 con 0x7fb334143c40 2024-04-24T23:21:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.440+0000 7fb32affd640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb31406be30 0x7fb31406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.440+0000 7fb33259c640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb31406be30 0x7fb31406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.440+0000 7fb32affd640 1 -- 172.21.15.26:0/3212095377 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb31c08ad30 con 0x7fb334143c40 2024-04-24T23:21:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.440+0000 7fb33259c640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb31406be30 0x7fb31406e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb324008850 tx=0x7fb324008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.444+0000 7fb32affd640 1 -- 172.21.15.26:0/3212095377 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb31c0596e0 con 0x7fb334143c40 2024-04-24T23:21:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.667+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb3340b4d20 con 0x7fb334143c40 2024-04-24T23:21:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.667+0000 7fb32affd640 1 -- 172.21.15.26:0/3212095377 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fb31c058fe0 con 0x7fb334143c40 2024-04-24T23:21:15.669 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:21:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb31406be30 msgr2=0x7fb31406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb31406be30 0x7fb31406e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb324008850 tx=0x7fb324008040 comp rx=0 tx=0).stop 2024-04-24T23:21:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 msgr2=0x7fb33413ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb33413ce50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00c430 tx=0x7fb31c00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 shutdown_connections 2024-04-24T23:21:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb31406be30 0x7fb31406e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb33415eb00 0x7fb334137630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb33413ce50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 --2- 172.21.15.26:0/3212095377 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3340cc860 0x7fb33414baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 >> 172.21.15.26:0/3212095377 conn(0x7fb33406b300 msgr2=0x7fb3340b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:15.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 shutdown_connections 2024-04-24T23:21:15.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:15.669+0000 7fb3388f5640 1 -- 172.21.15.26:0/3212095377 wait complete. 2024-04-24T23:21:15.681 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:21:15.682 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:21:15.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-24T23:21:15.682 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:21:15.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:21:15.682 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:21:15.682 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:21:15.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:21:15.682 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:21:15.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:15.682 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-24T23:21:15.702 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:21:15.702 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:21:15.702 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:15.703 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-24T23:21:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:21:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42730sec preferred_lft 42730sec 2024-04-24T23:21:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link noprefixroute 2024-04-24T23:21:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:15.704 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:15.704 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-24T23:21:15.704 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:21:15.704 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:15.704 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-24T23:21:15.704 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:21:15.704 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr flush dev ceph-brx 2024-04-24T23:21:15.705 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set ceph-brx up 2024-04-24T23:21:15.705 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:21:15.705 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-24T23:21:15.801 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:15.903 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:15.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:15.910 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:21:15.936 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-24T23:21:15.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:15.938 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-24T23:21:16.007 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-24T23:21:16.007 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:21:16.007 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:21:16.007 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:21:16.007 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:21:16.008 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:16.008 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-24T23:21:16.008 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:21:16.008 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:21:16.008 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:21:16.008 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-24T23:21:16.097 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:16.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:16.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:16.296 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-24T23:21:16.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:16.311 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list-id 2024-04-24T23:21:16.366 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:16.366 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-24T23:21:16.366 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:21:16.366 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-24T23:21:16.366 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-24T23:21:16.455 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:16.484 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-24T23:21:16.484 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:16.484 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-24T23:21:16.484 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-24T23:21:16.484 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:21:16.484 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-24T23:21:16.484 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-24T23:21:16.484 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-24T23:21:16.485 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-24T23:21:16.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:16.607 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:16.613 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:16.613 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-24T23:21:16.613 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.0 up 2024-04-24T23:21:16.613 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:21:16.613 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-24T23:21:16.662 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:16.684 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:16.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:16.690 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:21:16.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:16.705 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:21:16.760 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:21:16.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:16.760 DEBUG:teuthology.orchestra.run.smithi029:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi029.stdout:invalid new device string format 2024-04-24T23:21:16.980 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-24T23:21:16.980 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-24T23:21:16.980 INFO:teuthology.orchestra.run.smithi029.stdout:invalid new device string format 2024-04-24T23:21:16.980 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:16.980 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:16.984 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-24T23:21:16.985 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-24T23:21:16.985 INFO:teuthology.orchestra.run.smithi029.stdout:invalid new device string format 2024-04-24T23:21:16.985 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:16.985 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:16.985 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:21:16.985 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:21:16.985 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:21:16.986 INFO:teuthology.orchestra.run.smithi029.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:21:16.986 INFO:teuthology.orchestra.run.smithi029.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:21:16.986 INFO:teuthology.orchestra.run.smithi029.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:21:16.986 INFO:teuthology.orchestra.run.smithi029.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:21:16.986 INFO:teuthology.orchestra.run.smithi029.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:21:16.990 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:21:16.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:16.990 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:21:17.014 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:17.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:17.106 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:17.112 INFO:teuthology.orchestra.run.smithi029.stdout:client4579 2024-04-24T23:21:17.112 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:21:17.112 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:17.128 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi029.stdout:client4579 2024-04-24T23:21:17.225 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:17.275 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:17.277 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:17.283 INFO:teuthology.orchestra.run.smithi029.stdout:client4579 2024-04-24T23:21:17.283 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5a872e4b-97fd-4255-95a5-3ea24d1048cf.client4579/status') 2024-04-24T23:21:17.332 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi029.stdout:instance: client.4579 (3)192.168.0.1:0/3902612595 2024-04-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi029.stdout:blocklisted: false 2024-04-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi029.stderr:0+1 records in 2024-04-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi029.stderr:0+1 records out 2024-04-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi029.stderr:69 bytes copied, 2.6521e-05 s, 2.6 MB/s 2024-04-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:17.349 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-24T23:21:17.350 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:21:17.350 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T23:21:17.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:21:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.462+0000 7f3ec3753640 1 Processor -- start 2024-04-24T23:21:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.462+0000 7f3ec3753640 1 -- start start 2024-04-24T23:21:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.463+0000 7f3ec3753640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ebc15e380 0x7f3ebc0b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.463+0000 7f3ec3753640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3ebc0b4fd0 0x7f3ebc072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.463+0000 7f3ec3753640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ebc0b5440 con 0x7f3ebc0b0030 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.463+0000 7f3ec3753640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3ebc15e760 con 0x7f3ebc15e380 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.463+0000 7f3ec3753640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3ebc0725b0 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.463+0000 7f3ec14ef640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ebc15e380 0x7f3ebc0b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.463+0000 7f3ec14ef640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ebc15e380 0x7f3ebc0b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53396/0 (socket says 172.21.15.26:53396) 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.463+0000 7f3ec14ef640 1 -- 172.21.15.26:0/1085474104 learned_addr learned my addr 172.21.15.26:0/1085474104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.464+0000 7f3ec0cee640 1 --2- 172.21.15.26:0/1085474104 >> v2:172.21.15.29:3300/0 conn(0x7f3ebc0b4fd0 0x7f3ebc072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.464+0000 7f3ec14ef640 1 -- 172.21.15.26:0/1085474104 >> v2:172.21.15.29:3300/0 conn(0x7f3ebc0b4fd0 msgr2=0x7f3ebc072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.464+0000 7f3ec14ef640 1 --2- 172.21.15.26:0/1085474104 >> v2:172.21.15.29:3300/0 conn(0x7f3ebc0b4fd0 0x7f3ebc072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.464+0000 7f3ec14ef640 1 -- 172.21.15.26:0/1085474104 >> v1:172.21.15.29:6789/0 conn(0x7f3ebc0b0030 legacy=0x7f3ebc0b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.464+0000 7f3ec14ef640 1 -- 172.21.15.26:0/1085474104 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ebc0b5700 con 0x7f3ebc15e380 2024-04-24T23:21:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.464+0000 7f3ec0cee640 1 --2- 172.21.15.26:0/1085474104 >> v2:172.21.15.29:3300/0 conn(0x7f3ebc0b4fd0 0x7f3ebc072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.464+0000 7f3ec14ef640 1 --2- 172.21.15.26:0/1085474104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc15e380 0x7f3ebc0b4530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0b4d30 tx=0x7f3eb402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f875c30494d97dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.464+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1085474104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4037070 con 0x7f3ebc15e380 2024-04-24T23:21:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1085474104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb4032440 con 0x7f3ebc15e380 2024-04-24T23:21:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ebbfff640 1 -- 172.21.15.26:0/1085474104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4032810 con 0x7f3ebc15e380 2024-04-24T23:21:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ec3753640 1 -- 172.21.15.26:0/1085474104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc15e380 msgr2=0x7f3ebc0b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ec3753640 1 --2- 172.21.15.26:0/1085474104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc15e380 0x7f3ebc0b4530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0b4d30 tx=0x7f3eb402a2c0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ec3753640 1 -- 172.21.15.26:0/1085474104 shutdown_connections 2024-04-24T23:21:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ec3753640 1 --2- 172.21.15.26:0/1085474104 >> v2:172.21.15.29:3300/0 conn(0x7f3ebc0b4fd0 0x7f3ebc072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ec3753640 1 --2- 172.21.15.26:0/1085474104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc15e380 0x7f3ebc0b4530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ec3753640 1 -- 172.21.15.26:0/1085474104 >> 172.21.15.26:0/1085474104 conn(0x7f3ebc06e2c0 msgr2=0x7f3ebc06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ec3753640 1 -- 172.21.15.26:0/1085474104 shutdown_connections 2024-04-24T23:21:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.465+0000 7f3ec3753640 1 -- 172.21.15.26:0/1085474104 wait complete. 2024-04-24T23:21:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.466+0000 7f3ec3753640 1 Processor -- start 2024-04-24T23:21:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec3753640 1 -- start start 2024-04-24T23:21:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec3753640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ebc0b0030 0x7f3ebc14d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec3753640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc0b4fd0 0x7f3ebc0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec3753640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ebc15e380 0x7f3ebc0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec3753640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc072f80 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec3753640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc072240 con 0x7f3ebc0b0030 2024-04-24T23:21:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec3753640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3ebc0ae530 con 0x7f3ebc15e380 2024-04-24T23:21:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec0cee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc0b4fd0 0x7f3ebc0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec0cee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc0b4fd0 0x7f3ebc0ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53400/0 (socket says 172.21.15.26:53400) 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec0cee640 1 -- 172.21.15.26:0/3813128800 learned_addr learned my addr 172.21.15.26:0/3813128800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.467+0000 7f3ec14ef640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ebc0b0030 0x7f3ebc14d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec1cf0640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ebc15e380 0x7f3ebc0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec0cee640 1 -- 172.21.15.26:0/3813128800 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ebc15e380 msgr2=0x7f3ebc0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec0cee640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ebc15e380 0x7f3ebc0b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec0cee640 1 -- 172.21.15.26:0/3813128800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ebc0b0030 msgr2=0x7f3ebc14d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec0cee640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ebc0b0030 0x7f3ebc14d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec0cee640 1 -- 172.21.15.26:0/3813128800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ebc1f7320 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec1cf0640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ebc15e380 0x7f3ebc0b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec0cee640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc0b4fd0 0x7f3ebc0ce090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000c7f0 tx=0x7f3eb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3eb9ffb640 1 -- 172.21.15.26:0/3813128800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0015070 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3eb9ffb640 1 -- 172.21.15.26:0/3813128800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb0007a30 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ebc1f7600 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3eb9ffb640 1 -- 172.21.15.26:0/3813128800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0007e00 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.468+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ebc1f7bc0 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.469+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ebc0694c0 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.470+0000 7f3eb9ffb640 1 -- 172.21.15.26:0/3813128800 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3eb0018070 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.470+0000 7f3eb9ffb640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ea806be30 0x7f3ea806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.471+0000 7f3eb9ffb640 1 -- 172.21.15.26:0/3813128800 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3eb008ae40 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.471+0000 7f3ec14ef640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ea806be30 0x7f3ea806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.471+0000 7f3ec14ef640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ea806be30 0x7f3ea806e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4034000 tx=0x7f3eb402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:17.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.475+0000 7f3eb9ffb640 1 -- 172.21.15.26:0/3813128800 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3eb0059100 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.696+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ebc0af680 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.696+0000 7f3eb9ffb640 1 -- 172.21.15.26:0/3813128800 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f3eb005d400 con 0x7f3ebc0b4fd0 2024-04-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ea806be30 msgr2=0x7f3ea806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ea806be30 0x7f3ea806e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4034000 tx=0x7f3eb402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc0b4fd0 msgr2=0x7f3ebc0ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc0b4fd0 0x7f3ebc0ce090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000c7f0 tx=0x7f3eb000cbf0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 shutdown_connections 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ea806be30 0x7f3ea806e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ebc15e380 0x7f3ebc0b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ebc0b4fd0 0x7f3ebc0ce090 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 --2- 172.21.15.26:0/3813128800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ebc0b0030 0x7f3ebc14d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 >> 172.21.15.26:0/3813128800 conn(0x7f3ebc06e2c0 msgr2=0x7f3ebc06b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 shutdown_connections 2024-04-24T23:21:17.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:17.698+0000 7f3ec3753640 1 -- 172.21.15.26:0/3813128800 wait complete. 2024-04-24T23:21:17.710 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:21:17.710 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:21:17.710 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi029.front.sepia.ceph.com 2024-04-24T23:21:17.711 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-24T23:21:17.711 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T23:21:17.711 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:21:17.711 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:21:17.711 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:21:17.711 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-24T23:21:17.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:17.711 DEBUG:teuthology.orchestra.run.smithi029:> ip addr 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f0 2024-04-24T23:21:17.728 INFO:teuthology.orchestra.run.smithi029.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:6b:fd:a1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp7s0f1 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f0 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: inet 172.21.15.29/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft 42728sec preferred_lft 42728sec 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::ec4:7aff:febd:125a/64 scope link noprefixroute 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0c:c4:7a:bd:12:5b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout: altname enp1s0f1 2024-04-24T23:21:17.729 INFO:teuthology.orchestra.run.smithi029.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0a:51:f7:b0:78:e9 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::4c9f:61ff:fe90:a0e1/64 scope link 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run.smithi029.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether 0a:51:f7:b0:78:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::851:f7ff:feb0:78e9/64 scope link tentative 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:17.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:17.731 DEBUG:teuthology.orchestra.run.smithi029:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:21:17.792 INFO:teuthology.orchestra.run.smithi029.stdout:1 2024-04-24T23:21:17.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:17.794 DEBUG:teuthology.orchestra.run.smithi029:> route 2024-04-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi029.stdout:Kernel IP routing table 2024-04-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi029.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi029.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi029.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:21:17.851 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:17.851 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-24T23:21:17.851 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:21:17.852 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:21:17.852 DEBUG:teuthology.orchestra.run.smithi029:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:21:17.852 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-24T23:21:17.941 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:17.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:17.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:17.979 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:21:17.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:17.994 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list-id 2024-04-24T23:21:18.049 INFO:teuthology.orchestra.run.smithi029.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-24T23:21:18.049 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:18.049 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-24T23:21:18.050 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:21:18.050 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-24T23:21:18.050 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-24T23:21:18.137 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:18.159 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:18.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:18.165 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T23:21:18.189 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:21:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:21:18.189 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:21:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-24T23:21:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:18.190 INFO:teuthology.orchestra.run.smithi029.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:21:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether da:d3:2d:7d:ea:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T23:21:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T23:21:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::d8d3:2dff:fe7d:ea47/64 scope link 2024-04-24T23:21:18.190 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:18.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:18.191 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T23:21:18.255 INFO:teuthology.orchestra.run.smithi029.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:21:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:21:18.255 INFO:teuthology.orchestra.run.smithi029.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:21:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 ::1/128 scope host 2024-04-24T23:21:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:18.256 INFO:teuthology.orchestra.run.smithi029.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:21:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: link/ether da:d3:2d:7d:ea:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T23:21:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T23:21:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: inet6 fe80::d8d3:2dff:fe7d:ea47/64 scope link 2024-04-24T23:21:18.256 INFO:teuthology.orchestra.run.smithi029.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:18.258 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-24T23:21:18.258 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:18.258 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-24T23:21:18.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-24T23:21:18.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T23:21:18.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-24T23:21:18.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-24T23:21:18.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-24T23:21:18.258 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-24T23:21:18.348 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:18.404 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:18.411 DEBUG:teuthology.orchestra.run.smithi029:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:18.411 DEBUG:teuthology.orchestra.run.smithi029:> set -e 2024-04-24T23:21:18.411 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set brx.1 up 2024-04-24T23:21:18.411 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T23:21:18.411 DEBUG:teuthology.orchestra.run.smithi029:> ') 2024-04-24T23:21:18.461 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:18.482 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:18.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:18.491 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-24T23:21:18.507 INFO:teuthology.orchestra.run.smithi029.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-24T23:21:18.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:18.507 DEBUG:teuthology.orchestra.run.smithi029:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-24T23:21:18.562 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:21:18.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:18.563 DEBUG:teuthology.orchestra.run.smithi029:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-24T23:21:18.645 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-24T23:21:18.645 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-24T23:21:18.645 INFO:teuthology.orchestra.run.smithi029.stdout:invalid new device string format 2024-04-24T23:21:18.653 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-24T23:21:18.654 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-24T23:21:18.654 INFO:teuthology.orchestra.run.smithi029.stdout:invalid new device string format 2024-04-24T23:21:18.654 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:18.654 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:18.658 INFO:teuthology.orchestra.run.smithi029.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-24T23:21:18.658 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-24T23:21:18.658 INFO:teuthology.orchestra.run.smithi029.stdout:invalid new device string format 2024-04-24T23:21:18.658 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:18.659 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:18.659 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:21:18.659 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:21:18.659 INFO:teuthology.orchestra.run.smithi029.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:21:18.659 INFO:teuthology.orchestra.run.smithi029.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-24T23:21:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:21:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:21:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:21:18.659 INFO:teuthology.orchestra.run.smithi029.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:21:18.661 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:21:18.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:18.661 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-24T23:21:18.726 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:18.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:18.817 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:18.823 INFO:teuthology.orchestra.run.smithi029.stdout:client4585 2024-04-24T23:21:18.823 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:21:18.823 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:18.838 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:18.927 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:18.929 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:18.935 INFO:teuthology.orchestra.run.smithi029.stdout:client4585 2024-04-24T23:21:18.935 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:18.984 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:18.985 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:18.991 INFO:teuthology.orchestra.run.smithi029.stdout:client4585 2024-04-24T23:21:18.991 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5a872e4b-97fd-4255-95a5-3ea24d1048cf.client4585/status') 2024-04-24T23:21:19.042 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:19.051 INFO:teuthology.orchestra.run.smithi029.stdout:instance: client.4585 (3)192.168.0.2:0/3545251717 2024-04-24T23:21:19.051 INFO:teuthology.orchestra.run.smithi029.stdout:blocklisted: false 2024-04-24T23:21:19.051 INFO:teuthology.orchestra.run.smithi029.stderr:0+1 records in 2024-04-24T23:21:19.051 INFO:teuthology.orchestra.run.smithi029.stderr:0+1 records out 2024-04-24T23:21:19.051 INFO:teuthology.orchestra.run.smithi029.stderr:69 bytes copied, 2.2967e-05 s, 3.0 MB/s 2024-04-24T23:21:19.052 INFO:teuthology.orchestra.run.smithi029.stderr:2024-04-24T23:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:19.058 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-24T23:21:19.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:19.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.194+0000 7f6baf73b640 1 Processor -- start 2024-04-24T23:21:19.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.195+0000 7f6baf73b640 1 -- start start 2024-04-24T23:21:19.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.195+0000 7f6baf73b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 0x7f6ba80d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.195+0000 7f6baf73b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ba80d22e0 0x7f6ba80d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6baf73b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba80d32f0 con 0x7f6ba80b3130 2024-04-24T23:21:19.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6baf73b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80d1ab0 con 0x7f6ba80d22e0 2024-04-24T23:21:19.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6baf73b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba80d2ae0 con 0x7f6ba815e570 2024-04-24T23:21:19.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6badcd8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ba80d22e0 0x7f6ba80d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6baccd6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6ba80b3130 0x7f6ba80d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41160/0 (socket says 172.21.15.26:41160) 2024-04-24T23:21:19.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6baccd6640 1 -- 172.21.15.26:0/520677552 learned_addr learned my addr 172.21.15.26:0/520677552 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:19.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6bad4d7640 1 --2- 172.21.15.26:0/520677552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 0x7f6ba80d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6badcd8640 1 -- 172.21.15.26:0/520677552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 msgr2=0x7f6ba80d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6badcd8640 1 --2- 172.21.15.26:0/520677552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 0x7f6ba80d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6badcd8640 1 -- 172.21.15.26:0/520677552 >> v1:172.21.15.26:6789/0 conn(0x7f6ba80b3130 legacy=0x7f6ba80d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6badcd8640 1 -- 172.21.15.26:0/520677552 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80d0270 con 0x7f6ba80d22e0 2024-04-24T23:21:19.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.196+0000 7f6b9ffff640 1 -- 172.21.15.26:0/520677552 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2854991802 0 0) 0x7f6b98001110 con 0x7f6ba80b3130 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.197+0000 7f6badcd8640 1 --2- 172.21.15.26:0/520677552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 0x7f6ba80d46a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009d90 tx=0x7f6ba002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8479acfc900a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.197+0000 7f6b9ffff640 1 -- 172.21.15.26:0/520677552 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0007740 con 0x7f6ba80d22e0 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.197+0000 7f6b9ffff640 1 -- 172.21.15.26:0/520677552 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba00078e0 con 0x7f6ba80d22e0 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.197+0000 7f6b9ffff640 1 -- 172.21.15.26:0/520677552 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0007cb0 con 0x7f6ba80d22e0 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.198+0000 7f6baf73b640 1 -- 172.21.15.26:0/520677552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 msgr2=0x7f6ba80d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.198+0000 7f6baf73b640 1 --2- 172.21.15.26:0/520677552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 0x7f6ba80d46a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009d90 tx=0x7f6ba002a660 comp rx=0 tx=0).stop 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.198+0000 7f6baf73b640 1 -- 172.21.15.26:0/520677552 shutdown_connections 2024-04-24T23:21:19.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.198+0000 7f6baf73b640 1 --2- 172.21.15.26:0/520677552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 0x7f6ba80d46a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.198+0000 7f6baf73b640 1 --2- 172.21.15.26:0/520677552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 0x7f6ba80d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.198+0000 7f6baf73b640 1 -- 172.21.15.26:0/520677552 >> 172.21.15.26:0/520677552 conn(0x7f6ba808dbe0 msgr2=0x7f6ba809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.198+0000 7f6baf73b640 1 -- 172.21.15.26:0/520677552 shutdown_connections 2024-04-24T23:21:19.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.198+0000 7f6baf73b640 1 -- 172.21.15.26:0/520677552 wait complete. 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.199+0000 7f6baf73b640 1 Processor -- start 2024-04-24T23:21:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.199+0000 7f6baf73b640 1 -- start start 2024-04-24T23:21:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baf73b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80b3130 0x7f6ba80b8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baf73b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 0x7f6ba815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baf73b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 0x7f6ba81528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baf73b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba80cfa70 con 0x7f6ba80d22e0 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baf73b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba80d3010 con 0x7f6ba80b3130 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baf73b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba80cfd50 con 0x7f6ba815e570 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baccd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 0x7f6ba815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baccd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 0x7f6ba815cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53420/0 (socket says 172.21.15.26:53420) 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baccd6640 1 -- 172.21.15.26:0/2516561367 learned_addr learned my addr 172.21.15.26:0/2516561367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6badcd8640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 0x7f6ba81528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6bad4d7640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80b3130 0x7f6ba80b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baccd6640 1 -- 172.21.15.26:0/2516561367 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 msgr2=0x7f6ba81528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baccd6640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 0x7f6ba81528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baccd6640 1 -- 172.21.15.26:0/2516561367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80b3130 msgr2=0x7f6ba80b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baccd6640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80b3130 0x7f6ba80b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baccd6640 1 -- 172.21.15.26:0/2516561367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80b6e90 con 0x7f6ba80d22e0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6bad4d7640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80b3130 0x7f6ba80b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6baccd6640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 0x7f6ba815cb60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c780 tx=0x7f6b9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.200+0000 7f6b9dffb640 1 -- 172.21.15.26:0/2516561367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98018070 con 0x7f6ba80d22e0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.201+0000 7f6b9dffb640 1 -- 172.21.15.26:0/2516561367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b980072d0 con 0x7f6ba80d22e0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.201+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba8159f50 con 0x7f6ba80d22e0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.201+0000 7f6b9dffb640 1 -- 172.21.15.26:0/2516561367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980047e0 con 0x7f6ba80d22e0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.201+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba815a7f0 con 0x7f6ba80d22e0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.202+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba80b7060 con 0x7f6ba80d22e0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.202+0000 7f6b9dffb640 1 -- 172.21.15.26:0/2516561367 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b9801b070 con 0x7f6ba80d22e0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.203+0000 7f6b9dffb640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.203+0000 7f6b9dffb640 1 -- 172.21.15.26:0/2516561367 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b9808a190 con 0x7f6ba80d22e0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.203+0000 7f6bad4d7640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.203+0000 7f6bad4d7640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009a20 tx=0x7f6ba4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.207+0000 7f6b9dffb640 1 -- 172.21.15.26:0/2516561367 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ba80d32f0 con 0x7f6ba80d22e0 2024-04-24T23:21:19.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.392+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ba815e950 con 0x7f6ba80d22e0 2024-04-24T23:21:19.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.393+0000 7f6b9dffb640 1 -- 172.21.15.26:0/2516561367 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f6ba815e950 con 0x7f6ba80d22e0 2024-04-24T23:21:19.395 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:19.395 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b8c06be30 msgr2=0x7f6b8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009a20 tx=0x7f6ba4008040 comp rx=0 tx=0).stop 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 msgr2=0x7f6ba815cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 0x7f6ba815cb60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c780 tx=0x7f6b9800cc40 comp rx=0 tx=0).stop 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 shutdown_connections 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ba815e570 0x7f6ba81528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba80d22e0 0x7f6ba815cb60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 --2- 172.21.15.26:0/2516561367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80b3130 0x7f6ba80b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 >> 172.21.15.26:0/2516561367 conn(0x7f6ba808dbe0 msgr2=0x7f6ba80d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 shutdown_connections 2024-04-24T23:21:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.395+0000 7f6baf73b640 1 -- 172.21.15.26:0/2516561367 wait complete. 2024-04-24T23:21:19.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:19.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.512+0000 7faa0e34c640 1 Processor -- start 2024-04-24T23:21:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.512+0000 7faa0e34c640 1 -- start start 2024-04-24T23:21:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa0e34c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa0815adc0 0x7faa080b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa0e34c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faa080ad7b0 0x7faa080d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa0e34c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815e9c0 0x7faa080d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa0e34c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7faa0815ef60 con 0x7faa0815adc0 2024-04-24T23:21:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa0e34c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faa0815f240 con 0x7faa080ad7b0 2024-04-24T23:21:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa0e34c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa0815f520 con 0x7faa0815e9c0 2024-04-24T23:21:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa07fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa0815adc0 0x7faa080b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa07fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa0815adc0 0x7faa080b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53434/0 (socket says 172.21.15.26:53434) 2024-04-24T23:21:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa07fff640 1 -- 172.21.15.26:0/695386914 learned_addr learned my addr 172.21.15.26:0/695386914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa07fff640 1 -- 172.21.15.26:0/695386914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815e9c0 msgr2=0x7faa080d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa07fff640 1 --2- 172.21.15.26:0/695386914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815e9c0 0x7faa080d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa07fff640 1 -- 172.21.15.26:0/695386914 >> v2:172.21.15.29:3300/0 conn(0x7faa080ad7b0 msgr2=0x7faa080d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa07fff640 1 --2- 172.21.15.26:0/695386914 >> v2:172.21.15.29:3300/0 conn(0x7faa080ad7b0 0x7faa080d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.513+0000 7faa07fff640 1 -- 172.21.15.26:0/695386914 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0815f800 con 0x7faa0815adc0 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.514+0000 7faa07fff640 1 --2- 172.21.15.26:0/695386914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa0815adc0 0x7faa080b4370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009870 tx=0x7fa9f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373cd00d160b86b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.514+0000 7faa06ffd640 1 -- 172.21.15.26:0/695386914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8037070 con 0x7faa0815adc0 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.514+0000 7faa06ffd640 1 -- 172.21.15.26:0/695386914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f8032440 con 0x7faa0815adc0 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.514+0000 7faa06ffd640 1 -- 172.21.15.26:0/695386914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8032810 con 0x7faa0815adc0 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.515+0000 7faa0e34c640 1 -- 172.21.15.26:0/695386914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa0815adc0 msgr2=0x7faa080b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.515+0000 7faa0e34c640 1 --2- 172.21.15.26:0/695386914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa0815adc0 0x7faa080b4370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009870 tx=0x7fa9f802a180 comp rx=0 tx=0).stop 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.515+0000 7faa0e34c640 1 -- 172.21.15.26:0/695386914 shutdown_connections 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.515+0000 7faa0e34c640 1 --2- 172.21.15.26:0/695386914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815e9c0 0x7faa080d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.515+0000 7faa0e34c640 1 --2- 172.21.15.26:0/695386914 >> v2:172.21.15.29:3300/0 conn(0x7faa080ad7b0 0x7faa080d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.515+0000 7faa0e34c640 1 --2- 172.21.15.26:0/695386914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa0815adc0 0x7faa080b4370 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.515+0000 7faa0e34c640 1 -- 172.21.15.26:0/695386914 >> 172.21.15.26:0/695386914 conn(0x7faa0806b540 msgr2=0x7faa0806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.515+0000 7faa0e34c640 1 -- 172.21.15.26:0/695386914 shutdown_connections 2024-04-24T23:21:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.515+0000 7faa0e34c640 1 -- 172.21.15.26:0/695386914 wait complete. 2024-04-24T23:21:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.516+0000 7faa0e34c640 1 Processor -- start 2024-04-24T23:21:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.516+0000 7faa0e34c640 1 -- start start 2024-04-24T23:21:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa0e34c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa080ad7b0 0x7faa0814b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa0e34c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815adc0 0x7faa081e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa0e34c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa0815e9c0 0x7faa0815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa0e34c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa0815b1a0 con 0x7faa080ad7b0 2024-04-24T23:21:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa0e34c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa080b48b0 con 0x7faa0815e9c0 2024-04-24T23:21:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa0e34c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa0806dca0 con 0x7faa0815adc0 2024-04-24T23:21:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa080ad7b0 0x7faa0814b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa080ad7b0 0x7faa0814b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53446/0 (socket says 172.21.15.26:53446) 2024-04-24T23:21:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa07fff640 1 -- 172.21.15.26:0/3585326887 learned_addr learned my addr 172.21.15.26:0/3585326887 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa077fe640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815adc0 0x7faa081e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa0c8e9640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa0815e9c0 0x7faa0815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa07fff640 1 -- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815adc0 msgr2=0x7faa081e4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa07fff640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815adc0 0x7faa081e4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa07fff640 1 -- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa0815e9c0 msgr2=0x7faa0815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa07fff640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa0815e9c0 0x7faa0815d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa07fff640 1 -- 172.21.15.26:0/3585326887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0814bab0 con 0x7faa080ad7b0 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa077fe640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815adc0 0x7faa081e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa0c8e9640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa0815e9c0 0x7faa0815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.517+0000 7faa07fff640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa080ad7b0 0x7faa0814b4d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009870 tx=0x7fa9f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.518+0000 7faa04ff9640 1 -- 172.21.15.26:0/3585326887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8041070 con 0x7faa080ad7b0 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.518+0000 7faa04ff9640 1 -- 172.21.15.26:0/3585326887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f803c870 con 0x7faa080ad7b0 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.518+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0814bd90 con 0x7faa080ad7b0 2024-04-24T23:21:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.518+0000 7faa04ff9640 1 -- 172.21.15.26:0/3585326887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8037040 con 0x7faa080ad7b0 2024-04-24T23:21:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.518+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0814c350 con 0x7faa080ad7b0 2024-04-24T23:21:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.519+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa080cd7b0 con 0x7faa080ad7b0 2024-04-24T23:21:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.519+0000 7faa04ff9640 1 -- 172.21.15.26:0/3585326887 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9f8044070 con 0x7faa080ad7b0 2024-04-24T23:21:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.520+0000 7faa04ff9640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9f406be30 0x7fa9f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.520+0000 7faa04ff9640 1 -- 172.21.15.26:0/3585326887 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9f80aba60 con 0x7faa080ad7b0 2024-04-24T23:21:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.520+0000 7faa077fe640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9f406be30 0x7fa9f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.520+0000 7faa077fe640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9f406be30 0x7fa9f406e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006fd0 tx=0x7fa9fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.524+0000 7faa04ff9640 1 -- 172.21.15.26:0/3585326887 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faa080cd7b0 con 0x7faa080ad7b0 2024-04-24T23:21:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.707+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa080b31b0 con 0x7faa080ad7b0 2024-04-24T23:21:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.708+0000 7faa04ff9640 1 -- 172.21.15.26:0/3585326887 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa9f807a400 con 0x7faa080ad7b0 2024-04-24T23:21:19.710 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:19.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9f406be30 msgr2=0x7fa9f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9f406be30 0x7fa9f406e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006fd0 tx=0x7fa9fc008040 comp rx=0 tx=0).stop 2024-04-24T23:21:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa080ad7b0 msgr2=0x7faa0814b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa080ad7b0 0x7faa0814b4d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009870 tx=0x7fa9f80046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 shutdown_connections 2024-04-24T23:21:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa9f406be30 0x7fa9f406e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa0815e9c0 0x7faa0815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa0815adc0 0x7faa081e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 --2- 172.21.15.26:0/3585326887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa080ad7b0 0x7faa0814b4d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 >> 172.21.15.26:0/3585326887 conn(0x7faa0806b540 msgr2=0x7faa080b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 shutdown_connections 2024-04-24T23:21:19.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.710+0000 7faa0e34c640 1 -- 172.21.15.26:0/3585326887 wait complete. 2024-04-24T23:21:19.723 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:19.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.825+0000 7feedb55a640 1 Processor -- start 2024-04-24T23:21:19.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.825+0000 7feedb55a640 1 -- start start 2024-04-24T23:21:19.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feedb55a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40ce640 0x7feed40d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feedb55a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed40d28e0 0x7feed40cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feedb55a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed406d750 con 0x7feed406c4d0 2024-04-24T23:21:19.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feedb55a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7feed40d1110 con 0x7feed40d28e0 2024-04-24T23:21:19.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feedb55a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feed40cc410 con 0x7feed40ce640 2024-04-24T23:21:19.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feed8af5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed40d28e0 0x7feed40cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feed8af5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed40d28e0 0x7feed40cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53452/0 (socket says 172.21.15.26:53452) 2024-04-24T23:21:19.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feed8af5640 1 -- 172.21.15.26:0/2127129559 learned_addr learned my addr 172.21.15.26:0/2127129559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:19.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feed92f6640 1 --2- 172.21.15.26:0/2127129559 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40ce640 0x7feed40d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feed8af5640 1 -- 172.21.15.26:0/2127129559 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40ce640 msgr2=0x7feed40d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:19.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feed8af5640 1 --2- 172.21.15.26:0/2127129559 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40ce640 0x7feed40d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feed8af5640 1 -- 172.21.15.26:0/2127129559 >> v1:172.21.15.29:6789/0 conn(0x7feed406c4d0 legacy=0x7feed406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.826+0000 7feed8af5640 1 -- 172.21.15.26:0/2127129559 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed40b1720 con 0x7feed40d28e0 2024-04-24T23:21:19.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.827+0000 7feed8af5640 1 --2- 172.21.15.26:0/2127129559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40d28e0 0x7feed40cea20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feec8009870 tx=0x7feec802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7034cb4d75c1f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.827+0000 7feed3fff640 1 -- 172.21.15.26:0/2127129559 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8037070 con 0x7feed40d28e0 2024-04-24T23:21:19.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.827+0000 7feed3fff640 1 -- 172.21.15.26:0/2127129559 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec802ae50 con 0x7feed40d28e0 2024-04-24T23:21:19.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.827+0000 7feed3fff640 1 -- 172.21.15.26:0/2127129559 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8032860 con 0x7feed40d28e0 2024-04-24T23:21:19.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.828+0000 7feedb55a640 1 -- 172.21.15.26:0/2127129559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40d28e0 msgr2=0x7feed40cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.828+0000 7feedb55a640 1 --2- 172.21.15.26:0/2127129559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40d28e0 0x7feed40cea20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feec8009870 tx=0x7feec802a300 comp rx=0 tx=0).stop 2024-04-24T23:21:19.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.828+0000 7feedb55a640 1 -- 172.21.15.26:0/2127129559 shutdown_connections 2024-04-24T23:21:19.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.828+0000 7feedb55a640 1 --2- 172.21.15.26:0/2127129559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40d28e0 0x7feed40cea20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.828+0000 7feedb55a640 1 --2- 172.21.15.26:0/2127129559 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40ce640 0x7feed40d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.828+0000 7feedb55a640 1 -- 172.21.15.26:0/2127129559 >> 172.21.15.26:0/2127129559 conn(0x7feed40d1dc0 msgr2=0x7feed40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:19.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.828+0000 7feedb55a640 1 -- 172.21.15.26:0/2127129559 shutdown_connections 2024-04-24T23:21:19.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.828+0000 7feedb55a640 1 -- 172.21.15.26:0/2127129559 wait complete. 2024-04-24T23:21:19.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.829+0000 7feedb55a640 1 Processor -- start 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.829+0000 7feedb55a640 1 -- start start 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feedb55a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed406c4d0 0x7feed40c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feedb55a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40ce640 0x7feed4150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feedb55a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40d28e0 0x7feed414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feedb55a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feed40cc660 con 0x7feed406c4d0 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feedb55a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feed40d1360 con 0x7feed40ce640 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feedb55a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feed40cc940 con 0x7feed40d28e0 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed92f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed406c4d0 0x7feed40c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed92f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed406c4d0 0x7feed40c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53456/0 (socket says 172.21.15.26:53456) 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed92f6640 1 -- 172.21.15.26:0/3246905034 learned_addr learned my addr 172.21.15.26:0/3246905034 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed8af5640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40ce640 0x7feed4150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed9af7640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40d28e0 0x7feed414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed92f6640 1 -- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40d28e0 msgr2=0x7feed414d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed92f6640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40d28e0 0x7feed414d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed92f6640 1 -- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40ce640 msgr2=0x7feed4150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed92f6640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40ce640 0x7feed4150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed92f6640 1 -- 172.21.15.26:0/3246905034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed4150b40 con 0x7feed406c4d0 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed9af7640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40d28e0 0x7feed414d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed8af5640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40ce640 0x7feed4150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.830+0000 7feed92f6640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed406c4d0 0x7feed40c7610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9b0 tx=0x7feecc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.831+0000 7feed1ffb640 1 -- 172.21.15.26:0/3246905034 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc017070 con 0x7feed406c4d0 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.831+0000 7feed1ffb640 1 -- 172.21.15.26:0/3246905034 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feecc00c9a0 con 0x7feed406c4d0 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.831+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed40c7cf0 con 0x7feed406c4d0 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.831+0000 7feed1ffb640 1 -- 172.21.15.26:0/3246905034 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc00cd90 con 0x7feed406c4d0 2024-04-24T23:21:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.831+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed40c82b0 con 0x7feed406c4d0 2024-04-24T23:21:19.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.832+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed4075610 con 0x7feed406c4d0 2024-04-24T23:21:19.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.832+0000 7feed1ffb640 1 -- 172.21.15.26:0/3246905034 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feecc01a070 con 0x7feed406c4d0 2024-04-24T23:21:19.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.833+0000 7feed1ffb640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feec006be30 0x7feec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:19.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.833+0000 7feed1ffb640 1 -- 172.21.15.26:0/3246905034 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feecc08aea0 con 0x7feed406c4d0 2024-04-24T23:21:19.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.833+0000 7feed8af5640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feec006be30 0x7feec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:19.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.833+0000 7feed8af5640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feec006be30 0x7feec006e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feec8034000 tx=0x7feec802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:19.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:19.837+0000 7feed1ffb640 1 -- 172.21.15.26:0/3246905034 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feed4150b40 con 0x7feed406c4d0 2024-04-24T23:21:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.018+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feed40b1500 con 0x7feed406c4d0 2024-04-24T23:21:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.019+0000 7feed1ffb640 1 -- 172.21.15.26:0/3246905034 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7feecc059840 con 0x7feed406c4d0 2024-04-24T23:21:20.021 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feec006be30 msgr2=0x7feec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feec006be30 0x7feec006e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feec8034000 tx=0x7feec802e000 comp rx=0 tx=0).stop 2024-04-24T23:21:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed406c4d0 msgr2=0x7feed40c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed406c4d0 0x7feed40c7610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9b0 tx=0x7feecc00ee70 comp rx=0 tx=0).stop 2024-04-24T23:21:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 shutdown_connections 2024-04-24T23:21:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feec006be30 0x7feec006e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40d28e0 0x7feed414d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40ce640 0x7feed4150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 --2- 172.21.15.26:0/3246905034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed406c4d0 0x7feed40c7610 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 >> 172.21.15.26:0/3246905034 conn(0x7feed40d1dc0 msgr2=0x7feed40b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 shutdown_connections 2024-04-24T23:21:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.021+0000 7feedb55a640 1 -- 172.21.15.26:0/3246905034 wait complete. 2024-04-24T23:21:20.034 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:20.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.136+0000 7f91d11a2640 1 Processor -- start 2024-04-24T23:21:20.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.137+0000 7f91d11a2640 1 -- start start 2024-04-24T23:21:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.137+0000 7f91d11a2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f91cc06d5a0 0x7f91cc0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.137+0000 7f91d11a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc070f30 con 0x7f91cc15e120 2024-04-24T23:21:20.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.137+0000 7f91d11a2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc06dcf0 con 0x7f91cc0b3d20 2024-04-24T23:21:20.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.137+0000 7f91d11a2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f91cc06dfd0 con 0x7f91cc06d5a0 2024-04-24T23:21:20.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.137+0000 7f91cad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f91cc15e120 0x7f91cc15e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41162/0 (socket says 172.21.15.26:41162) 2024-04-24T23:21:20.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91cad9d640 1 -- 172.21.15.26:0/3146650091 learned_addr learned my addr 172.21.15.26:0/3146650091 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:20.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91ca59c640 1 --2- 172.21.15.26:0/3146650091 >> v2:172.21.15.29:3300/0 conn(0x7f91cc06d5a0 0x7f91cc0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992034517 0 0) 0x7f91cc070f30 con 0x7f91cc15e120 2024-04-24T23:21:20.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91b8003520 con 0x7f91cc15e120 2024-04-24T23:21:20.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3119705954 0 0) 0x7f91b8003520 con 0x7f91cc15e120 2024-04-24T23:21:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 >> v2:172.21.15.29:3300/0 conn(0x7f91cc06d5a0 msgr2=0x7f91cc0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 --2- 172.21.15.26:0/3146650091 >> v2:172.21.15.29:3300/0 conn(0x7f91cc06d5a0 0x7f91cc0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 >> v1:172.21.15.29:6789/0 conn(0x7f91cc0b3d20 legacy=0x7f91cc150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc06e2b0 con 0x7f91cc15e120 2024-04-24T23:21:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f91bc0019f0 con 0x7f91cc15e120 2024-04-24T23:21:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 >> v1:172.21.15.26:6789/0 conn(0x7f91cc15e120 legacy=0x7f91cc15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b80039b0 0x7f91b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91b8004800 0x7f91b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91b80051c0 0x7f91b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc06dfd0 con 0x7f91b80039b0 2024-04-24T23:21:20.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f91b800c060 con 0x7f91b8004800 2024-04-24T23:21:20.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f91b800c260 con 0x7f91b80051c0 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91cad9d640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b80039b0 0x7f91b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91ca59c640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91b8004800 0x7f91b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.138+0000 7f91cb59e640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91b80051c0 0x7f91b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91cad9d640 1 -- 172.21.15.26:0/3146650091 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91b80051c0 msgr2=0x7f91b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91cad9d640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91b80051c0 0x7f91b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91cad9d640 1 -- 172.21.15.26:0/3146650091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91b8004800 msgr2=0x7f91b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91cad9d640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91b8004800 0x7f91b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91cad9d640 1 -- 172.21.15.26:0/3146650091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b8003520 con 0x7f91b80039b0 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91cb59e640 1 -- 172.21.15.26:0/3146650091 reap_dead start 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91cad9d640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b80039b0 0x7f91b8003dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00a830 tx=0x7f91b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e615acca19b30e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00a480 con 0x7f91b80039b0 2024-04-24T23:21:20.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc02f070 con 0x7f91b80039b0 2024-04-24T23:21:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.139+0000 7f91c9d9b640 1 -- 172.21.15.26:0/3146650091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc02bcb0 con 0x7f91b80039b0 2024-04-24T23:21:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.140+0000 7f91d11a2640 1 -- 172.21.15.26:0/3146650091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b80039b0 msgr2=0x7f91b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.140+0000 7f91d11a2640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b80039b0 0x7f91b8003dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00a830 tx=0x7f91b8005830 comp rx=0 tx=0).stop 2024-04-24T23:21:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.140+0000 7f91d11a2640 1 -- 172.21.15.26:0/3146650091 shutdown_connections 2024-04-24T23:21:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.140+0000 7f91d11a2640 1 --2- 172.21.15.26:0/3146650091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b80039b0 0x7f91b8003dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.140+0000 7f91d11a2640 1 -- 172.21.15.26:0/3146650091 >> 172.21.15.26:0/3146650091 conn(0x7f91cc06b540 msgr2=0x7f91cc06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.140+0000 7f91d11a2640 1 -- 172.21.15.26:0/3146650091 shutdown_connections 2024-04-24T23:21:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.140+0000 7f91d11a2640 1 -- 172.21.15.26:0/3146650091 wait complete. 2024-04-24T23:21:20.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.141+0000 7f91d11a2640 1 Processor -- start 2024-04-24T23:21:20.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.141+0000 7f91d11a2640 1 -- start start 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91d11a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91b80039b0 0x7f91cc1efce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91d11a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91cc14c6d0 0x7f91cc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91d11a2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91cc14cb50 0x7f91cc0ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91d11a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0b37e0 con 0x7f91cc14c6d0 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91d11a2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc06da10 con 0x7f91b80039b0 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91d11a2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc0b3960 con 0x7f91cc14cb50 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91ca59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91cc14c6d0 0x7f91cc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91ca59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91cc14c6d0 0x7f91cc0adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53478/0 (socket says 172.21.15.26:53478) 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91ca59c640 1 -- 172.21.15.26:0/3491979271 learned_addr learned my addr 172.21.15.26:0/3491979271 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91cb59e640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91cc14cb50 0x7f91cc0ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.142+0000 7f91cad9d640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91b80039b0 0x7f91cc1efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91ca59c640 1 -- 172.21.15.26:0/3491979271 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91cc14cb50 msgr2=0x7f91cc0ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91ca59c640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91cc14cb50 0x7f91cc0ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91ca59c640 1 -- 172.21.15.26:0/3491979271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91b80039b0 msgr2=0x7f91cc1efce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91ca59c640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91b80039b0 0x7f91cc1efce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91ca59c640 1 -- 172.21.15.26:0/3491979271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc14f380 con 0x7f91cc14c6d0 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91ca59c640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91cc14c6d0 0x7f91cc0adfb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f91b4011a70 tx=0x7f91b4011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91c37fe640 1 -- 172.21.15.26:0/3491979271 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b400fb50 con 0x7f91cc14c6d0 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91c37fe640 1 -- 172.21.15.26:0/3491979271 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b400fcf0 con 0x7f91cc14c6d0 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91cc149ab0 con 0x7f91cc14c6d0 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91c37fe640 1 -- 172.21.15.26:0/3491979271 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b400fea0 con 0x7f91cc14c6d0 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.143+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91cc14f0a0 con 0x7f91cc14c6d0 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.144+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91cc075610 con 0x7f91cc14c6d0 2024-04-24T23:21:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.144+0000 7f91c37fe640 1 -- 172.21.15.26:0/3491979271 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91b4012040 con 0x7f91cc14c6d0 2024-04-24T23:21:20.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.145+0000 7f91c37fe640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.145+0000 7f91cad9d640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.145+0000 7f91c37fe640 1 -- 172.21.15.26:0/3491979271 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f91b408b400 con 0x7f91cc14c6d0 2024-04-24T23:21:20.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.146+0000 7f91cad9d640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f91bc033000 tx=0x7f91bc007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:20.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.149+0000 7f91c37fe640 1 -- 172.21.15.26:0/3491979271 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91b4019030 con 0x7f91cc14c6d0 2024-04-24T23:21:20.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.332+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f91cc0ced80 con 0x7f91cc14c6d0 2024-04-24T23:21:20.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.332+0000 7f91c37fe640 1 -- 172.21.15.26:0/3491979271 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f91b40597f0 con 0x7f91cc14c6d0 2024-04-24T23:21:20.334 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:20.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 msgr2=0x7f91ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f91bc033000 tx=0x7f91bc007000 comp rx=0 tx=0).stop 2024-04-24T23:21:20.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91cc14c6d0 msgr2=0x7f91cc0adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91cc14c6d0 0x7f91cc0adfb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f91b4011a70 tx=0x7f91b4011f30 comp rx=0 tx=0).stop 2024-04-24T23:21:20.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 shutdown_connections 2024-04-24T23:21:20.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91cc14cb50 0x7f91cc0ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91cc14c6d0 0x7f91cc0adfb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 --2- 172.21.15.26:0/3491979271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91b80039b0 0x7f91cc1efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 >> 172.21.15.26:0/3491979271 conn(0x7f91cc06b540 msgr2=0x7f91cc1520c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:20.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 shutdown_connections 2024-04-24T23:21:20.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.334+0000 7f91d11a2640 1 -- 172.21.15.26:0/3491979271 wait complete. 2024-04-24T23:21:20.347 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.445+0000 7f1e99264640 1 Processor -- start 2024-04-24T23:21:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.446+0000 7f1e99264640 1 -- start start 2024-04-24T23:21:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.446+0000 7f1e99264640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1e94143c40 0x7f1e940cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.446+0000 7f1e99264640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 0x7f1e94072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.446+0000 7f1e99264640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e940cd0b0 con 0x7f1e9415eb00 2024-04-24T23:21:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.446+0000 7f1e99264640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1e94071b70 con 0x7f1e94143c40 2024-04-24T23:21:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.446+0000 7f1e99264640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1e94071240 con 0x7f1e940cc860 2024-04-24T23:21:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e937fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1e9415eb00 0x7f1e940b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41166/0 (socket says 172.21.15.26:41166) 2024-04-24T23:21:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e937fe640 1 -- 172.21.15.26:0/2376145464 learned_addr learned my addr 172.21.15.26:0/2376145464 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e92ffd640 1 --2- 172.21.15.26:0/2376145464 >> v2:172.21.15.29:3300/0 conn(0x7f1e94143c40 0x7f1e940cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e927fc640 1 --2- 172.21.15.26:0/2376145464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 0x7f1e94072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e91ffb640 1 -- 172.21.15.26:0/2376145464 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553511660 0 0) 0x7f1e940cd0b0 con 0x7f1e9415eb00 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e91ffb640 1 -- 172.21.15.26:0/2376145464 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e7c003520 con 0x7f1e9415eb00 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e92ffd640 1 -- 172.21.15.26:0/2376145464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 msgr2=0x7f1e94072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e92ffd640 1 --2- 172.21.15.26:0/2376145464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 0x7f1e94072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e92ffd640 1 -- 172.21.15.26:0/2376145464 >> v1:172.21.15.26:6789/0 conn(0x7f1e9415eb00 legacy=0x7f1e940b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.447+0000 7f1e92ffd640 1 -- 172.21.15.26:0/2376145464 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e94071520 con 0x7f1e94143c40 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.448+0000 7f1e92ffd640 1 --2- 172.21.15.26:0/2376145464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 0x7f1e940cf870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0097a0 tx=0x7f1e8c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ce91dcc85a49eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.448+0000 7f1e91ffb640 1 -- 172.21.15.26:0/2376145464 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c037070 con 0x7f1e94143c40 2024-04-24T23:21:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.448+0000 7f1e91ffb640 1 -- 172.21.15.26:0/2376145464 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c032440 con 0x7f1e94143c40 2024-04-24T23:21:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.448+0000 7f1e91ffb640 1 -- 172.21.15.26:0/2376145464 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c0327f0 con 0x7f1e94143c40 2024-04-24T23:21:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.448+0000 7f1e99264640 1 -- 172.21.15.26:0/2376145464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 msgr2=0x7f1e940cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.448+0000 7f1e99264640 1 --2- 172.21.15.26:0/2376145464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 0x7f1e940cf870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0097a0 tx=0x7f1e8c02a180 comp rx=0 tx=0).stop 2024-04-24T23:21:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.448+0000 7f1e99264640 1 -- 172.21.15.26:0/2376145464 shutdown_connections 2024-04-24T23:21:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.448+0000 7f1e99264640 1 --2- 172.21.15.26:0/2376145464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 0x7f1e94072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.448+0000 7f1e99264640 1 --2- 172.21.15.26:0/2376145464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 0x7f1e940cf870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.449+0000 7f1e99264640 1 -- 172.21.15.26:0/2376145464 >> 172.21.15.26:0/2376145464 conn(0x7f1e9406b300 msgr2=0x7f1e9406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.449+0000 7f1e99264640 1 -- 172.21.15.26:0/2376145464 shutdown_connections 2024-04-24T23:21:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.449+0000 7f1e99264640 1 -- 172.21.15.26:0/2376145464 wait complete. 2024-04-24T23:21:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.450+0000 7f1e99264640 1 Processor -- start 2024-04-24T23:21:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.450+0000 7f1e99264640 1 -- start start 2024-04-24T23:21:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e99264640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 0x7f1e941f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e99264640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 0x7f1e9406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e99264640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e9415eb00 0x7f1e9406da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e99264640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e94072540 con 0x7f1e9415eb00 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e99264640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e94144020 con 0x7f1e94143c40 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e99264640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1e941f55e0 con 0x7f1e940cc860 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e9415eb00 0x7f1e9406da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e9415eb00 0x7f1e9406da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53494/0 (socket says 172.21.15.26:53494) 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e937fe640 1 -- 172.21.15.26:0/3524387371 learned_addr learned my addr 172.21.15.26:0/3524387371 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e92ffd640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 0x7f1e941f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e927fc640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 0x7f1e9406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e937fe640 1 -- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 msgr2=0x7f1e941f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e937fe640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 0x7f1e941f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e937fe640 1 -- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 msgr2=0x7f1e9406d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e937fe640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 0x7f1e9406d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e937fe640 1 -- 172.21.15.26:0/3524387371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e941f4ad0 con 0x7f1e9415eb00 2024-04-24T23:21:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e92ffd640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 0x7f1e941f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e927fc640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 0x7f1e9406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e937fe640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e9415eb00 0x7f1e9406da20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400cad0 tx=0x7f1e8400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/3524387371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84017070 con 0x7f1e9415eb00 2024-04-24T23:21:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/3524387371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8400ac30 con 0x7f1e9415eb00 2024-04-24T23:21:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e941f4db0 con 0x7f1e9415eb00 2024-04-24T23:21:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/3524387371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84012400 con 0x7f1e9415eb00 2024-04-24T23:21:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.451+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e940ca500 con 0x7f1e9415eb00 2024-04-24T23:21:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.453+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e941f52d0 con 0x7f1e9415eb00 2024-04-24T23:21:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.453+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/3524387371 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e8401a070 con 0x7f1e9415eb00 2024-04-24T23:21:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.453+0000 7f1e7b7fe640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be30 0x7f1e8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.453+0000 7f1e92ffd640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be30 0x7f1e8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.454+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/3524387371 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e8408ac40 con 0x7f1e9415eb00 2024-04-24T23:21:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.454+0000 7f1e92ffd640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be30 0x7f1e8006e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c034000 tx=0x7f1e8c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.458+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/3524387371 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e941f52d0 con 0x7f1e9415eb00 2024-04-24T23:21:20.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.643+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e940694c0 con 0x7f1e9415eb00 2024-04-24T23:21:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.644+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/3524387371 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f1e840595f0 con 0x7f1e9415eb00 2024-04-24T23:21:20.647 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:20.647 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.646+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be30 msgr2=0x7f1e8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.646+0000 7f1e99264640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be30 0x7f1e8006e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c034000 tx=0x7f1e8c02e000 comp rx=0 tx=0).stop 2024-04-24T23:21:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.646+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e9415eb00 msgr2=0x7f1e9406da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.647+0000 7f1e99264640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e9415eb00 0x7f1e9406da20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400cad0 tx=0x7f1e8400cf90 comp rx=0 tx=0).stop 2024-04-24T23:21:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.647+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 shutdown_connections 2024-04-24T23:21:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.647+0000 7f1e99264640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be30 0x7f1e8006e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.647+0000 7f1e99264640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e9415eb00 0x7f1e9406da20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.647+0000 7f1e99264640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e94143c40 0x7f1e9406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.647+0000 7f1e99264640 1 --2- 172.21.15.26:0/3524387371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940cc860 0x7f1e941f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.647+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 >> 172.21.15.26:0/3524387371 conn(0x7f1e9406b300 msgr2=0x7f1e9415bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.647+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 shutdown_connections 2024-04-24T23:21:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.647+0000 7f1e99264640 1 -- 172.21.15.26:0/3524387371 wait complete. 2024-04-24T23:21:20.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:20.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.762+0000 7f58d466b640 1 Processor -- start 2024-04-24T23:21:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.762+0000 7f58d466b640 1 -- start start 2024-04-24T23:21:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.763+0000 7f58d466b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc15e750 0x7f58cc0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.763+0000 7f58d466b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58cc0d2c30 0x7f58cc0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.763+0000 7f58d466b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58cc0d2120 con 0x7f58cc15e370 2024-04-24T23:21:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.763+0000 7f58d466b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc0d1510 con 0x7f58cc0d2c30 2024-04-24T23:21:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.763+0000 7f58d466b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc0d1ed0 con 0x7f58cc15e750 2024-04-24T23:21:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.763+0000 7f58d2c08640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58cc0d2c30 0x7f58cc0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.764+0000 7f58d2c08640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58cc0d2c30 0x7f58cc0cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53502/0 (socket says 172.21.15.26:53502) 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.764+0000 7f58d2c08640 1 -- 172.21.15.26:0/1911778132 learned_addr learned my addr 172.21.15.26:0/1911778132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.764+0000 7f58d1c06640 1 --2- 172.21.15.26:0/1911778132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc15e750 0x7f58cc0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.764+0000 7f58d2c08640 1 -- 172.21.15.26:0/1911778132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc15e750 msgr2=0x7f58cc0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.764+0000 7f58d2c08640 1 --2- 172.21.15.26:0/1911778132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc15e750 0x7f58cc0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.764+0000 7f58d2c08640 1 -- 172.21.15.26:0/1911778132 >> v1:172.21.15.29:6789/0 conn(0x7f58cc15e370 legacy=0x7f58cc0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.764+0000 7f58d2c08640 1 -- 172.21.15.26:0/1911778132 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc0b3740 con 0x7f58cc0d2c30 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.764+0000 7f58d1c06640 1 --2- 172.21.15.26:0/1911778132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc15e750 0x7f58cc0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.764+0000 7f58d2c08640 1 --2- 172.21.15.26:0/1911778132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc0d2c30 0x7f58cc0cf1d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0d2050 tx=0x7f58bc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fc653ed6c64488 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d1405640 1 -- 172.21.15.26:0/1911778132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc036070 con 0x7f58cc0d2c30 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d1405640 1 -- 172.21.15.26:0/1911778132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc031600 con 0x7f58cc0d2c30 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d1405640 1 -- 172.21.15.26:0/1911778132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc0319d0 con 0x7f58cc0d2c30 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d466b640 1 -- 172.21.15.26:0/1911778132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc0d2c30 msgr2=0x7f58cc0cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d466b640 1 --2- 172.21.15.26:0/1911778132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc0d2c30 0x7f58cc0cf1d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0d2050 tx=0x7f58bc02a6c0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d466b640 1 -- 172.21.15.26:0/1911778132 shutdown_connections 2024-04-24T23:21:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d466b640 1 --2- 172.21.15.26:0/1911778132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc0d2c30 0x7f58cc0cf1d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d466b640 1 --2- 172.21.15.26:0/1911778132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc15e750 0x7f58cc0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d466b640 1 -- 172.21.15.26:0/1911778132 >> 172.21.15.26:0/1911778132 conn(0x7f58cc0d40f0 msgr2=0x7f58cc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d466b640 1 -- 172.21.15.26:0/1911778132 shutdown_connections 2024-04-24T23:21:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.765+0000 7f58d466b640 1 -- 172.21.15.26:0/1911778132 wait complete. 2024-04-24T23:21:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.766+0000 7f58d466b640 1 Processor -- start 2024-04-24T23:21:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d466b640 1 -- start start 2024-04-24T23:21:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d466b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc0d2c30 0x7f58cc1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d466b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc15e370 0x7f58cc0b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d466b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0caab0 0x7f58cc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d466b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0b5230 con 0x7f58cc15e370 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d466b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0b3d00 con 0x7f58cc0caab0 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d466b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc0b53b0 con 0x7f58cc0d2c30 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d1c06640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc15e370 0x7f58cc0b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d1c06640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc15e370 0x7f58cc0b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53516/0 (socket says 172.21.15.26:53516) 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.767+0000 7f58d1c06640 1 -- 172.21.15.26:0/179573061 learned_addr learned my addr 172.21.15.26:0/179573061 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d2c08640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0caab0 0x7f58cc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d2407640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc0d2c30 0x7f58cc1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d1c06640 1 -- 172.21.15.26:0/179573061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc0d2c30 msgr2=0x7f58cc1f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d1c06640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc0d2c30 0x7f58cc1f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d1c06640 1 -- 172.21.15.26:0/179573061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0caab0 msgr2=0x7f58cc0b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d1c06640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0caab0 0x7f58cc0b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d1c06640 1 -- 172.21.15.26:0/179573061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc1f4ac0 con 0x7f58cc15e370 2024-04-24T23:21:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d2c08640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0caab0 0x7f58cc0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d1c06640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc15e370 0x7f58cc0b1ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f58c000c820 tx=0x7f58c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58caffd640 1 -- 172.21.15.26:0/179573061 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0015070 con 0x7f58cc15e370 2024-04-24T23:21:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58caffd640 1 -- 172.21.15.26:0/179573061 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c0007a30 con 0x7f58cc15e370 2024-04-24T23:21:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc0c7ec0 con 0x7f58cc15e370 2024-04-24T23:21:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58caffd640 1 -- 172.21.15.26:0/179573061 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0007de0 con 0x7f58cc15e370 2024-04-24T23:21:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.768+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc0c8480 con 0x7f58cc15e370 2024-04-24T23:21:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.769+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc075610 con 0x7f58cc15e370 2024-04-24T23:21:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.770+0000 7f58caffd640 1 -- 172.21.15.26:0/179573061 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58c0018070 con 0x7f58cc15e370 2024-04-24T23:21:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.770+0000 7f58caffd640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58b806be60 0x7f58b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.770+0000 7f58caffd640 1 -- 172.21.15.26:0/179573061 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58c008ad10 con 0x7f58cc15e370 2024-04-24T23:21:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.770+0000 7f58d2407640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58b806be60 0x7f58b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:20.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.770+0000 7f58d2407640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58b806be60 0x7f58b806e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f58c4004ea0 tx=0x7f58c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:20.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.774+0000 7f58caffd640 1 -- 172.21.15.26:0/179573061 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58c00596c0 con 0x7f58cc15e370 2024-04-24T23:21:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.958+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f58cc0c9240 con 0x7f58cc15e370 2024-04-24T23:21:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.958+0000 7f58caffd640 1 -- 172.21.15.26:0/179573061 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f58c0058fc0 con 0x7f58cc15e370 2024-04-24T23:21:20.961 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:20.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58b806be60 msgr2=0x7f58b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58b806be60 0x7f58b806e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f58c4004ea0 tx=0x7f58c4009040 comp rx=0 tx=0).stop 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc15e370 msgr2=0x7f58cc0b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc15e370 0x7f58cc0b1ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f58c000c820 tx=0x7f58c000cce0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 shutdown_connections 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58b806be60 0x7f58b806e320 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58cc0caab0 0x7f58cc0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58cc15e370 0x7f58cc0b1ca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 --2- 172.21.15.26:0/179573061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58cc0d2c30 0x7f58cc1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.960+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 >> 172.21.15.26:0/179573061 conn(0x7f58cc0d40f0 msgr2=0x7f58cc06ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.961+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 shutdown_connections 2024-04-24T23:21:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:20.961+0000 7f58d466b640 1 -- 172.21.15.26:0/179573061 wait complete. 2024-04-24T23:21:20.973 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.075+0000 7f26a5bc1640 1 Processor -- start 2024-04-24T23:21:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.075+0000 7f26a5bc1640 1 -- start start 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f26a5bc1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00d28e0 0x7f26a00d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f26a5bc1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26a015da30 0x7f26a015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f26a5bc1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a006d750 con 0x7f26a00ce640 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f26a5bc1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f26a00d1110 con 0x7f26a015da30 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f26a5bc1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f26a00cc410 con 0x7f26a00d28e0 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26a015da30 0x7f26a015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f26a00ce640 0x7f26a00cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41176/0 (socket says 172.21.15.26:41176) 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269f7fe640 1 -- 172.21.15.26:0/2664370229 learned_addr learned my addr 172.21.15.26:0/2664370229 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26a015da30 0x7f26a015deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53524/0 (socket says 172.21.15.26:53524) 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269effd640 1 --2- 172.21.15.26:0/2664370229 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00d28e0 0x7f26a00d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269ffff640 1 -- 172.21.15.26:0/2664370229 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00d28e0 msgr2=0x7f26a00d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269ffff640 1 --2- 172.21.15.26:0/2664370229 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00d28e0 0x7f26a00d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269ffff640 1 -- 172.21.15.26:0/2664370229 >> v1:172.21.15.26:6789/0 conn(0x7f26a00ce640 legacy=0x7f26a00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269e7fc640 1 -- 172.21.15.26:0/2664370229 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1926246263 0 0) 0x7f26a006d750 con 0x7f26a00ce640 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.076+0000 7f269ffff640 1 -- 172.21.15.26:0/2664370229 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a00b1720 con 0x7f26a015da30 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.077+0000 7f269ffff640 1 --2- 172.21.15.26:0/2664370229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a015da30 0x7f26a015deb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2698009d90 tx=0x7f269802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94800a60f25506e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.077+0000 7f269e7fc640 1 -- 172.21.15.26:0/2664370229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698007740 con 0x7f26a015da30 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.077+0000 7f269e7fc640 1 -- 172.21.15.26:0/2664370229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26980078e0 con 0x7f26a015da30 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.077+0000 7f269e7fc640 1 -- 172.21.15.26:0/2664370229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698007cb0 con 0x7f26a015da30 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.078+0000 7f26a5bc1640 1 -- 172.21.15.26:0/2664370229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a015da30 msgr2=0x7f26a015deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.078+0000 7f26a5bc1640 1 --2- 172.21.15.26:0/2664370229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a015da30 0x7f26a015deb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2698009d90 tx=0x7f269802a6f0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.078+0000 7f26a5bc1640 1 -- 172.21.15.26:0/2664370229 shutdown_connections 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.078+0000 7f26a5bc1640 1 --2- 172.21.15.26:0/2664370229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a015da30 0x7f26a015deb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.078+0000 7f26a5bc1640 1 --2- 172.21.15.26:0/2664370229 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00d28e0 0x7f26a00d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.078+0000 7f26a5bc1640 1 -- 172.21.15.26:0/2664370229 >> 172.21.15.26:0/2664370229 conn(0x7f26a00d1dc0 msgr2=0x7f26a00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.078+0000 7f26a5bc1640 1 -- 172.21.15.26:0/2664370229 shutdown_connections 2024-04-24T23:21:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.078+0000 7f26a5bc1640 1 -- 172.21.15.26:0/2664370229 wait complete. 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.079+0000 7f26a5bc1640 1 Processor -- start 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.079+0000 7f26a5bc1640 1 -- start start 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f26a5bc1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00ce640 0x7f26a00b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f26a5bc1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a00d28e0 0x7f26a01f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f26a5bc1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a015da30 0x7f26a01f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f26a5bc1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26a00cc940 con 0x7f26a00d28e0 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f26a5bc1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f26a00cc660 con 0x7f26a015da30 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f26a5bc1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f26a00ccc20 con 0x7f26a00ce640 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a00d28e0 0x7f26a01f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a00d28e0 0x7f26a01f6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53536/0 (socket says 172.21.15.26:53536) 2024-04-24T23:21:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269effd640 1 -- 172.21.15.26:0/4287063762 learned_addr learned my addr 172.21.15.26:0/4287063762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269f7fe640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00ce640 0x7f26a00b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269ffff640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a015da30 0x7f26a01f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269effd640 1 -- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00ce640 msgr2=0x7f26a00b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269effd640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00ce640 0x7f26a00b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269effd640 1 -- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a015da30 msgr2=0x7f26a01f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269effd640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a015da30 0x7f26a01f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269effd640 1 -- 172.21.15.26:0/4287063762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a00b5660 con 0x7f26a00d28e0 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269f7fe640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00ce640 0x7f26a00b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269ffff640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a015da30 0x7f26a01f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f269effd640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a00d28e0 0x7f26a01f6ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f269400c7f0 tx=0x7f269400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.080+0000 7f2687fff640 1 -- 172.21.15.26:0/4287063762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694015070 con 0x7f26a00d28e0 2024-04-24T23:21:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.081+0000 7f2687fff640 1 -- 172.21.15.26:0/4287063762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2694004220 con 0x7f26a00d28e0 2024-04-24T23:21:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.081+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a00b58d0 con 0x7f26a00d28e0 2024-04-24T23:21:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.081+0000 7f2687fff640 1 -- 172.21.15.26:0/4287063762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694007620 con 0x7f26a00d28e0 2024-04-24T23:21:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.081+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a00b52e0 con 0x7f26a00d28e0 2024-04-24T23:21:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.081+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a0075610 con 0x7f26a00d28e0 2024-04-24T23:21:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.082+0000 7f2687fff640 1 -- 172.21.15.26:0/4287063762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26940077c0 con 0x7f26a00d28e0 2024-04-24T23:21:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.082+0000 7f2687fff640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f268c06be30 0x7f268c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.083+0000 7f2687fff640 1 -- 172.21.15.26:0/4287063762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f269408ad10 con 0x7f26a00d28e0 2024-04-24T23:21:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.083+0000 7f269f7fe640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f268c06be30 0x7f268c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.083+0000 7f269f7fe640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f268c06be30 0x7f268c06e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2690000f60 tx=0x7f2690009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.087+0000 7f2687fff640 1 -- 172.21.15.26:0/4287063762 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f26940596c0 con 0x7f26a00d28e0 2024-04-24T23:21:21.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.269+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f26a00b1500 con 0x7f26a00d28e0 2024-04-24T23:21:21.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.269+0000 7f2687fff640 1 -- 172.21.15.26:0/4287063762 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f2694058fc0 con 0x7f26a00d28e0 2024-04-24T23:21:21.271 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f268c06be30 msgr2=0x7f268c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f268c06be30 0x7f268c06e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2690000f60 tx=0x7f2690009040 comp rx=0 tx=0).stop 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a00d28e0 msgr2=0x7f26a01f6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a00d28e0 0x7f26a01f6ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f269400c7f0 tx=0x7f269400ccb0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 shutdown_connections 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f268c06be30 0x7f268c06e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26a015da30 0x7f26a01f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a00d28e0 0x7f26a01f6ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 --2- 172.21.15.26:0/4287063762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26a00ce640 0x7f26a00b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 >> 172.21.15.26:0/4287063762 conn(0x7f26a00d1dc0 msgr2=0x7f26a00cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 shutdown_connections 2024-04-24T23:21:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.271+0000 7f26a5bc1640 1 -- 172.21.15.26:0/4287063762 wait complete. 2024-04-24T23:21:21.284 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.386+0000 7f0c5a55a640 1 Processor -- start 2024-04-24T23:21:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.386+0000 7f0c5a55a640 1 -- start start 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.387+0000 7f0c5a55a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0c5415e370 0x7f0c540d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.387+0000 7f0c5a55a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c5415e750 0x7f0c540d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.387+0000 7f0c5a55a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0c540d4c10 0x7f0c540b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.387+0000 7f0c5a55a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0c540d1510 con 0x7f0c540d4c10 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.387+0000 7f0c5a55a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0c540d1ed0 con 0x7f0c5415e370 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.387+0000 7f0c5a55a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0c540d2120 con 0x7f0c5415e750 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.387+0000 7f0c58af7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0c540d4c10 0x7f0c540b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.387+0000 7f0c58af7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0c540d4c10 0x7f0c540b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53546/0 (socket says 172.21.15.26:53546) 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.387+0000 7f0c58af7640 1 -- 172.21.15.26:0/3709980908 learned_addr learned my addr 172.21.15.26:0/3709980908 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.388+0000 7f0c537fe640 1 --2- 172.21.15.26:0/3709980908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c5415e750 0x7f0c540d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.388+0000 7f0c53fff640 1 --2- 172.21.15.26:0/3709980908 >> v2:172.21.15.29:3300/0 conn(0x7f0c5415e370 0x7f0c540d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.388+0000 7f0c58af7640 1 -- 172.21.15.26:0/3709980908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c5415e750 msgr2=0x7f0c540d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.388+0000 7f0c58af7640 1 --2- 172.21.15.26:0/3709980908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c5415e750 0x7f0c540d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.388+0000 7f0c58af7640 1 -- 172.21.15.26:0/3709980908 >> v2:172.21.15.29:3300/0 conn(0x7f0c5415e370 msgr2=0x7f0c540d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.388+0000 7f0c58af7640 1 --2- 172.21.15.26:0/3709980908 >> v2:172.21.15.29:3300/0 conn(0x7f0c5415e370 0x7f0c540d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.388+0000 7f0c58af7640 1 -- 172.21.15.26:0/3709980908 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c540b3740 con 0x7f0c540d4c10 2024-04-24T23:21:21.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.388+0000 7f0c58af7640 1 --2- 172.21.15.26:0/3709980908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c540d4c10 0x7f0c540b4220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0c5408c7e0 tx=0x7f0c4c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee750462cf1572c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c52ffd640 1 -- 172.21.15.26:0/3709980908 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c036070 con 0x7f0c540d4c10 2024-04-24T23:21:21.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c52ffd640 1 -- 172.21.15.26:0/3709980908 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c4c031600 con 0x7f0c540d4c10 2024-04-24T23:21:21.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c52ffd640 1 -- 172.21.15.26:0/3709980908 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c0319d0 con 0x7f0c540d4c10 2024-04-24T23:21:21.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c5a55a640 1 -- 172.21.15.26:0/3709980908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c540d4c10 msgr2=0x7f0c540b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/3709980908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c540d4c10 0x7f0c540b4220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0c5408c7e0 tx=0x7f0c4c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c5a55a640 1 -- 172.21.15.26:0/3709980908 shutdown_connections 2024-04-24T23:21:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/3709980908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c540d4c10 0x7f0c540b4220 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/3709980908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c5415e750 0x7f0c540d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/3709980908 >> v2:172.21.15.29:3300/0 conn(0x7f0c5415e370 0x7f0c540d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c5a55a640 1 -- 172.21.15.26:0/3709980908 >> 172.21.15.26:0/3709980908 conn(0x7f0c540d40f0 msgr2=0x7f0c540d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c5a55a640 1 -- 172.21.15.26:0/3709980908 shutdown_connections 2024-04-24T23:21:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.389+0000 7f0c5a55a640 1 -- 172.21.15.26:0/3709980908 wait complete. 2024-04-24T23:21:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.390+0000 7f0c5a55a640 1 Processor -- start 2024-04-24T23:21:21.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c5a55a640 1 -- start start 2024-04-24T23:21:21.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c5a55a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c540d4c10 0x7f0c541f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c5a55a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5415e370 0x7f0c5415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c5a55a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c54160540 0x7f0c540b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c5a55a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0c540b4760 con 0x7f0c5415e370 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c5a55a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c540b3d00 con 0x7f0c54160540 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c5a55a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0c540b48e0 con 0x7f0c540d4c10 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c537fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5415e370 0x7f0c5415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c537fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5415e370 0x7f0c5415c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53560/0 (socket says 172.21.15.26:53560) 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.391+0000 7f0c537fe640 1 -- 172.21.15.26:0/2131768563 learned_addr learned my addr 172.21.15.26:0/2131768563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c58af7640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c54160540 0x7f0c540b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c53fff640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c540d4c10 0x7f0c541f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c537fe640 1 -- 172.21.15.26:0/2131768563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c540d4c10 msgr2=0x7f0c541f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c537fe640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c540d4c10 0x7f0c541f8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c537fe640 1 -- 172.21.15.26:0/2131768563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c54160540 msgr2=0x7f0c540b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c537fe640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c54160540 0x7f0c540b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c537fe640 1 -- 172.21.15.26:0/2131768563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c541f4250 con 0x7f0c5415e370 2024-04-24T23:21:21.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c58af7640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c54160540 0x7f0c540b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c537fe640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5415e370 0x7f0c5415c1c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800c7f0 tx=0x7f0c4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c50ff9640 1 -- 172.21.15.26:0/2131768563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48015070 con 0x7f0c5415e370 2024-04-24T23:21:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c50ff9640 1 -- 172.21.15.26:0/2131768563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c48004220 con 0x7f0c5415e370 2024-04-24T23:21:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c50ff9640 1 -- 172.21.15.26:0/2131768563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48007620 con 0x7f0c5415e370 2024-04-24T23:21:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c541f4530 con 0x7f0c5415e370 2024-04-24T23:21:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.392+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c541f3f40 con 0x7f0c5415e370 2024-04-24T23:21:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.393+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c540cdf80 con 0x7f0c5415e370 2024-04-24T23:21:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.394+0000 7f0c50ff9640 1 -- 172.21.15.26:0/2131768563 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c480079d0 con 0x7f0c5415e370 2024-04-24T23:21:21.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.394+0000 7f0c50ff9640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0c4006be30 0x7f0c4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.394+0000 7f0c50ff9640 1 -- 172.21.15.26:0/2131768563 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c4808b130 con 0x7f0c5415e370 2024-04-24T23:21:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.394+0000 7f0c53fff640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0c4006be30 0x7f0c4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.395+0000 7f0c53fff640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0c4006be30 0x7f0c4006e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c44006fd0 tx=0x7f0c44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.398+0000 7f0c50ff9640 1 -- 172.21.15.26:0/2131768563 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c540cdf80 con 0x7f0c5415e370 2024-04-24T23:21:21.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.581+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0c54075610 con 0x7f0c5415e370 2024-04-24T23:21:21.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.582+0000 7f0c50ff9640 1 -- 172.21.15.26:0/2131768563 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0c48059b70 con 0x7f0c5415e370 2024-04-24T23:21:21.583 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.583+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0c4006be30 msgr2=0x7f0c4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.583+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0c4006be30 0x7f0c4006e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c44006fd0 tx=0x7f0c44008040 comp rx=0 tx=0).stop 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.583+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5415e370 msgr2=0x7f0c5415c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.583+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5415e370 0x7f0c5415c1c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800c7f0 tx=0x7f0c4800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.583+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 shutdown_connections 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.583+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0c4006be30 0x7f0c4006e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.583+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c54160540 0x7f0c540b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.583+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5415e370 0x7f0c5415c1c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.583+0000 7f0c5a55a640 1 --2- 172.21.15.26:0/2131768563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0c540d4c10 0x7f0c541f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.584+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 >> 172.21.15.26:0/2131768563 conn(0x7f0c540d40f0 msgr2=0x7f0c540d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.584+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 shutdown_connections 2024-04-24T23:21:21.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.584+0000 7f0c5a55a640 1 -- 172.21.15.26:0/2131768563 wait complete. 2024-04-24T23:21:21.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:21.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.695+0000 7f9cc649a640 1 Processor -- start 2024-04-24T23:21:21.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.695+0000 7f9cc649a640 1 -- start start 2024-04-24T23:21:21.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cc649a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9cc00b3f70 0x7f9cc00cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cc649a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc0156b40 con 0x7f9cc0156760 2024-04-24T23:21:21.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cc649a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc00cfed0 con 0x7f9cc00b3ac0 2024-04-24T23:21:21.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cc649a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc00d01b0 con 0x7f9cc00b3f70 2024-04-24T23:21:21.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9cc0156760 0x7f9cc015d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41184/0 (socket says 172.21.15.26:41184) 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbffff640 1 -- 172.21.15.26:0/4027849446 learned_addr learned my addr 172.21.15.26:0/4027849446 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cc4a37640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9cc00b3f70 0x7f9cc00cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3056819468 0 0) 0x7f9cc0156b40 con 0x7f9cc0156760 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ca8003520 con 0x7f9cc0156760 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 150923321 0 0) 0x7f9cc00cfed0 con 0x7f9cc00b3ac0 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cc0156b40 con 0x7f9cc00b3ac0 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3122751003 0 0) 0x7f9ca8003520 con 0x7f9cc0156760 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9cc00b3f70 msgr2=0x7f9cc00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9cc00b3f70 0x7f9cc00cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 >> v1:172.21.15.29:6789/0 conn(0x7f9cc00b3ac0 legacy=0x7f9cc00ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc00d0490 con 0x7f9cc0156760 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f9cb4001690 con 0x7f9cc0156760 2024-04-24T23:21:21.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 >> v1:172.21.15.26:6789/0 conn(0x7f9cc0156760 legacy=0x7f9cc015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ca8003b70 0x7f9ca8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca8004800 0x7f9ca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.696+0000 7f9cbeffd640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9cc00b3f70 0x7f9ca80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc00d01b0 con 0x7f9ca8004800 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca800ba70 con 0x7f9ca8003b70 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca800bc40 con 0x7f9cc00b3f70 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbf7fe640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca8004800 0x7f9ca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbffff640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ca8003b70 0x7f9ca8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cc4a37640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9cc00b3f70 0x7f9ca80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbf7fe640 1 -- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9cc00b3f70 msgr2=0x7f9ca80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbf7fe640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9cc00b3f70 0x7f9ca80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbf7fe640 1 -- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ca8003b70 msgr2=0x7f9ca8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbf7fe640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ca8003b70 0x7f9ca8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cbf7fe640 1 -- 172.21.15.26:0/4027849446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca8003520 con 0x7f9ca8004800 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.697+0000 7f9cc4a37640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9cc00b3f70 0x7f9ca80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cbf7fe640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca8004800 0x7f9ca8004c80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00018d0 tx=0x7f9ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2f0cd1cd2b9881 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00014f0 con 0x7f9ca8004800 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb00052d0 con 0x7f9ca8004800 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cbeffd640 1 -- 172.21.15.26:0/4027849446 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb002ac30 con 0x7f9ca8004800 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cc649a640 1 -- 172.21.15.26:0/4027849446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca8004800 msgr2=0x7f9ca8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cc649a640 1 --2- 172.21.15.26:0/4027849446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca8004800 0x7f9ca8004c80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00018d0 tx=0x7f9ca8005830 comp rx=0 tx=0).stop 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cc4a37640 1 -- 172.21.15.26:0/4027849446 reap_dead start 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cc649a640 1 -- 172.21.15.26:0/4027849446 shutdown_connections 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cc649a640 1 -- 172.21.15.26:0/4027849446 >> 172.21.15.26:0/4027849446 conn(0x7f9cc006b120 msgr2=0x7f9cc006b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cc649a640 1 -- 172.21.15.26:0/4027849446 shutdown_connections 2024-04-24T23:21:21.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.698+0000 7f9cc649a640 1 -- 172.21.15.26:0/4027849446 wait complete. 2024-04-24T23:21:21.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.699+0000 7f9cc649a640 1 Processor -- start 2024-04-24T23:21:21.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cc649a640 1 -- start start 2024-04-24T23:21:21.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cc649a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ca8004800 0x7f9cc015c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cc649a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cc015df90 0x7f9cc00b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cc649a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cc00b7520 0x7f9cc01f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cc649a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc00d0a50 con 0x7f9cc015df90 2024-04-24T23:21:21.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cc649a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc00b43a0 con 0x7f9cc00b7520 2024-04-24T23:21:21.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cc649a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc00cc7e0 con 0x7f9ca8004800 2024-04-24T23:21:21.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cc015df90 0x7f9cc00b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cc015df90 0x7f9cc00b6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53590/0 (socket says 172.21.15.26:53590) 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.700+0000 7f9cbf7fe640 1 -- 172.21.15.26:0/2809404549 learned_addr learned my addr 172.21.15.26:0/2809404549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbffff640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ca8004800 0x7f9cc015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cc4a37640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cc00b7520 0x7f9cc01f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbf7fe640 1 -- 172.21.15.26:0/2809404549 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ca8004800 msgr2=0x7f9cc015c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbf7fe640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ca8004800 0x7f9cc015c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbf7fe640 1 -- 172.21.15.26:0/2809404549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cc00b7520 msgr2=0x7f9cc01f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbf7fe640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cc00b7520 0x7f9cc01f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbf7fe640 1 -- 172.21.15.26:0/2809404549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc015cef0 con 0x7f9cc015df90 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbffff640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ca8004800 0x7f9cc015c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbf7fe640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cc015df90 0x7f9cc00b6fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0005040 tx=0x7f9cb00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbcff9640 1 -- 172.21.15.26:0/2809404549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb003f020 con 0x7f9cc015df90 2024-04-24T23:21:21.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbcff9640 1 -- 172.21.15.26:0/2809404549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb0006e50 con 0x7f9cc015df90 2024-04-24T23:21:21.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc015d1d0 con 0x7f9cc015df90 2024-04-24T23:21:21.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cbcff9640 1 -- 172.21.15.26:0/2809404549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0037040 con 0x7f9cc015df90 2024-04-24T23:21:21.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.701+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc015d790 con 0x7f9cc015df90 2024-04-24T23:21:21.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.702+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc00694c0 con 0x7f9cc015df90 2024-04-24T23:21:21.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.703+0000 7f9cbcff9640 1 -- 172.21.15.26:0/2809404549 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cb0042020 con 0x7f9cc015df90 2024-04-24T23:21:21.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.703+0000 7f9cbcff9640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cac06be60 0x7f9cac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.703+0000 7f9cbcff9640 1 -- 172.21.15.26:0/2809404549 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9cb00aa910 con 0x7f9cc015df90 2024-04-24T23:21:21.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.703+0000 7f9cbffff640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cac06be60 0x7f9cac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.704+0000 7f9cbffff640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cac06be60 0x7f9cac06e320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4001050 tx=0x7f9cb4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.708+0000 7f9cbcff9640 1 -- 172.21.15.26:0/2809404549 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9cb0049030 con 0x7f9cc015df90 2024-04-24T23:21:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.894+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cc015f400 con 0x7f9cc015df90 2024-04-24T23:21:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.895+0000 7f9cbcff9640 1 -- 172.21.15.26:0/2809404549 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f9cb00792c0 con 0x7f9cc015df90 2024-04-24T23:21:21.897 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:21.897 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.897+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cac06be60 msgr2=0x7f9cac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.897+0000 7f9cc649a640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cac06be60 0x7f9cac06e320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4001050 tx=0x7f9cb4009040 comp rx=0 tx=0).stop 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.897+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cc015df90 msgr2=0x7f9cc00b6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.897+0000 7f9cc649a640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cc015df90 0x7f9cc00b6fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0005040 tx=0x7f9cb00012d0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.898+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 shutdown_connections 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.898+0000 7f9cc649a640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cac06be60 0x7f9cac06e320 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.898+0000 7f9cc649a640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9cc00b7520 0x7f9cc01f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.898+0000 7f9cc649a640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cc015df90 0x7f9cc00b6fe0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.898+0000 7f9cc649a640 1 --2- 172.21.15.26:0/2809404549 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ca8004800 0x7f9cc015c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.898+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 >> 172.21.15.26:0/2809404549 conn(0x7f9cc006b120 msgr2=0x7f9cc015aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.898+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 shutdown_connections 2024-04-24T23:21:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:21.898+0000 7f9cc649a640 1 -- 172.21.15.26:0/2809404549 wait complete. 2024-04-24T23:21:21.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:21.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:22.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.014+0000 7f6a3277e640 1 Processor -- start 2024-04-24T23:21:22.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.014+0000 7f6a3277e640 1 -- start start 2024-04-24T23:21:22.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a3277e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a2c15be40 0x7f6a2c0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a3277e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0d0750 0x7f6a2c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a3277e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a2c0aeb10 con 0x7f6a2c0b4ce0 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a3277e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6a2c09df20 con 0x7f6a2c15be40 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a3277e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c0aedf0 con 0x7f6a2c0d0750 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a2bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a2c15be40 0x7f6a2c0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a2bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a2c15be40 0x7f6a2c0acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53606/0 (socket says 172.21.15.26:53606) 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a2bfff640 1 -- 172.21.15.26:0/1042933453 learned_addr learned my addr 172.21.15.26:0/1042933453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a2b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6a2c0b4ce0 0x7f6a2c0d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41198/0 (socket says 172.21.15.26:41198) 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.015+0000 7f6a30d1b640 1 --2- 172.21.15.26:0/1042933453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0d0750 0x7f6a2c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.016+0000 7f6a2bfff640 1 -- 172.21.15.26:0/1042933453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0d0750 msgr2=0x7f6a2c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.016+0000 7f6a2bfff640 1 --2- 172.21.15.26:0/1042933453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0d0750 0x7f6a2c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.016+0000 7f6a2bfff640 1 -- 172.21.15.26:0/1042933453 >> v1:172.21.15.26:6789/0 conn(0x7f6a2c0b4ce0 legacy=0x7f6a2c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.016+0000 7f6a2bfff640 1 -- 172.21.15.26:0/1042933453 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c0af0d0 con 0x7f6a2c15be40 2024-04-24T23:21:22.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.016+0000 7f6a2affd640 1 -- 172.21.15.26:0/1042933453 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165936605 0 0) 0x7f6a2c0aeb10 con 0x7f6a2c0b4ce0 2024-04-24T23:21:22.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.016+0000 7f6a30d1b640 1 --2- 172.21.15.26:0/1042933453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0d0750 0x7f6a2c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.016+0000 7f6a2bfff640 1 --2- 172.21.15.26:0/1042933453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c15be40 0x7f6a2c0acd30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009870 tx=0x7f6a1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444b245b31545ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.016+0000 7f6a2affd640 1 -- 172.21.15.26:0/1042933453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c037070 con 0x7f6a2c15be40 2024-04-24T23:21:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.016+0000 7f6a2affd640 1 -- 172.21.15.26:0/1042933453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1c032440 con 0x7f6a2c15be40 2024-04-24T23:21:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.017+0000 7f6a2affd640 1 -- 172.21.15.26:0/1042933453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c032810 con 0x7f6a2c15be40 2024-04-24T23:21:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.017+0000 7f6a3277e640 1 -- 172.21.15.26:0/1042933453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c15be40 msgr2=0x7f6a2c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.017+0000 7f6a3277e640 1 --2- 172.21.15.26:0/1042933453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c15be40 0x7f6a2c0acd30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009870 tx=0x7f6a1c02a270 comp rx=0 tx=0).stop 2024-04-24T23:21:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.017+0000 7f6a3277e640 1 -- 172.21.15.26:0/1042933453 shutdown_connections 2024-04-24T23:21:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.017+0000 7f6a3277e640 1 --2- 172.21.15.26:0/1042933453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0d0750 0x7f6a2c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.017+0000 7f6a3277e640 1 --2- 172.21.15.26:0/1042933453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c15be40 0x7f6a2c0acd30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.017+0000 7f6a3277e640 1 -- 172.21.15.26:0/1042933453 >> 172.21.15.26:0/1042933453 conn(0x7f6a2c06b540 msgr2=0x7f6a2c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.017+0000 7f6a3277e640 1 -- 172.21.15.26:0/1042933453 shutdown_connections 2024-04-24T23:21:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.017+0000 7f6a3277e640 1 -- 172.21.15.26:0/1042933453 wait complete. 2024-04-24T23:21:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.018+0000 7f6a3277e640 1 Processor -- start 2024-04-24T23:21:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.018+0000 7f6a3277e640 1 -- start start 2024-04-24T23:21:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a3277e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0b4ce0 0x7f6a2c1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a3277e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c0d0750 0x7f6a2c0b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a3277e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a2c15be40 0x7f6a2c0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a3277e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c06cd10 con 0x7f6a2c0d0750 2024-04-24T23:21:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a3277e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c06cb90 con 0x7f6a2c15be40 2024-04-24T23:21:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a3277e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c15c760 con 0x7f6a2c0b4ce0 2024-04-24T23:21:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a2b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c0d0750 0x7f6a2c0b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a2b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c0d0750 0x7f6a2c0b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53614/0 (socket says 172.21.15.26:53614) 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a2b7fe640 1 -- 172.21.15.26:0/2632794169 learned_addr learned my addr 172.21.15.26:0/2632794169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a2bfff640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0b4ce0 0x7f6a2c1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.019+0000 7f6a30d1b640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a2c15be40 0x7f6a2c0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a2b7fe640 1 -- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0b4ce0 msgr2=0x7f6a2c1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a2b7fe640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0b4ce0 0x7f6a2c1598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a2b7fe640 1 -- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a2c15be40 msgr2=0x7f6a2c0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a2b7fe640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a2c15be40 0x7f6a2c0b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a2b7fe640 1 -- 172.21.15.26:0/2632794169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c1f5500 con 0x7f6a2c0d0750 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a30d1b640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a2c15be40 0x7f6a2c0b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a2bfff640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0b4ce0 0x7f6a2c1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a2b7fe640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c0d0750 0x7f6a2c0b2b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c500 tx=0x7f6a1400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a28ff9640 1 -- 172.21.15.26:0/2632794169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14018070 con 0x7f6a2c0d0750 2024-04-24T23:21:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a28ff9640 1 -- 172.21.15.26:0/2632794169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a14007580 con 0x7f6a2c0d0750 2024-04-24T23:21:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a28ff9640 1 -- 172.21.15.26:0/2632794169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14004a20 con 0x7f6a2c0d0750 2024-04-24T23:21:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c0b3730 con 0x7f6a2c0d0750 2024-04-24T23:21:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.020+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c159de0 con 0x7f6a2c0d0750 2024-04-24T23:21:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.021+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2c075610 con 0x7f6a2c0d0750 2024-04-24T23:21:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.021+0000 7f6a28ff9640 1 -- 172.21.15.26:0/2632794169 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a1401b070 con 0x7f6a2c0d0750 2024-04-24T23:21:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.022+0000 7f6a28ff9640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6a0c06be60 0x7f6a0c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.022+0000 7f6a28ff9640 1 -- 172.21.15.26:0/2632794169 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6a1408a070 con 0x7f6a2c0d0750 2024-04-24T23:21:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.022+0000 7f6a2bfff640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6a0c06be60 0x7f6a0c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.022+0000 7f6a2bfff640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6a0c06be60 0x7f6a0c06e320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c034000 tx=0x7f6a1c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.026+0000 7f6a28ff9640 1 -- 172.21.15.26:0/2632794169 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6a14058ab0 con 0x7f6a2c0d0750 2024-04-24T23:21:22.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.209+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6a2c0b08c0 con 0x7f6a2c0d0750 2024-04-24T23:21:22.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.209+0000 7f6a28ff9640 1 -- 172.21.15.26:0/2632794169 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f6a140583b0 con 0x7f6a2c0d0750 2024-04-24T23:21:22.211 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6a0c06be60 msgr2=0x7f6a0c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6a0c06be60 0x7f6a0c06e320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c034000 tx=0x7f6a1c02e000 comp rx=0 tx=0).stop 2024-04-24T23:21:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c0d0750 msgr2=0x7f6a2c0b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c0d0750 0x7f6a2c0b2b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c500 tx=0x7f6a1400c9c0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 shutdown_connections 2024-04-24T23:21:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6a0c06be60 0x7f6a0c06e320 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6a2c15be40 0x7f6a2c0b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a2c0d0750 0x7f6a2c0b2b50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 --2- 172.21.15.26:0/2632794169 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6a2c0b4ce0 0x7f6a2c1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.211+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 >> 172.21.15.26:0/2632794169 conn(0x7f6a2c06b540 msgr2=0x7f6a2c158b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.212+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 shutdown_connections 2024-04-24T23:21:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.212+0000 7f6a3277e640 1 -- 172.21.15.26:0/2632794169 wait complete. 2024-04-24T23:21:22.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:22.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.327+0000 7fb43fc15640 1 Processor -- start 2024-04-24T23:21:22.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.327+0000 7fb43fc15640 1 -- start start 2024-04-24T23:21:22.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43fc15640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb43815e370 0x7fb4380d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43fc15640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb43815e750 0x7fb4380d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43fc15640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4380d2120 con 0x7fb4380d4c10 2024-04-24T23:21:22.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43fc15640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb4380d1510 con 0x7fb43815e370 2024-04-24T23:21:22.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43fc15640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb4380d1ed0 con 0x7fb43815e750 2024-04-24T23:21:22.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43d9b1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb43815e370 0x7fb4380d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43d9b1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb43815e370 0x7fb4380d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53626/0 (socket says 172.21.15.26:53626) 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43d9b1640 1 -- 172.21.15.26:0/72352145 learned_addr learned my addr 172.21.15.26:0/72352145 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43e1b2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb4380d4c10 0x7fb4380b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41208/0 (socket says 172.21.15.26:41208) 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43d9b1640 1 -- 172.21.15.26:0/72352145 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb43815e750 msgr2=0x7fb4380d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43d1b0640 1 --2- 172.21.15.26:0/72352145 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb43815e750 0x7fb4380d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43d9b1640 1 --2- 172.21.15.26:0/72352145 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb43815e750 0x7fb4380d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43c9af640 1 -- 172.21.15.26:0/72352145 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375489971 0 0) 0x7fb4380d2120 con 0x7fb4380d4c10 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43d9b1640 1 -- 172.21.15.26:0/72352145 >> v1:172.21.15.26:6789/0 conn(0x7fb4380d4c10 legacy=0x7fb4380b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43d9b1640 1 -- 172.21.15.26:0/72352145 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4380b3740 con 0x7fb43815e370 2024-04-24T23:21:22.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.328+0000 7fb43d1b0640 1 --2- 172.21.15.26:0/72352145 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb43815e750 0x7fb4380d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.329+0000 7fb43d9b1640 1 --2- 172.21.15.26:0/72352145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43815e370 0x7fb4380d2520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb430009870 tx=0x7fb43002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d318149bd785321d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.329+0000 7fb43c9af640 1 -- 172.21.15.26:0/72352145 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430037070 con 0x7fb43815e370 2024-04-24T23:21:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.329+0000 7fb43c9af640 1 -- 172.21.15.26:0/72352145 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb430032440 con 0x7fb43815e370 2024-04-24T23:21:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.329+0000 7fb43c9af640 1 -- 172.21.15.26:0/72352145 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430032810 con 0x7fb43815e370 2024-04-24T23:21:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.330+0000 7fb43fc15640 1 -- 172.21.15.26:0/72352145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43815e370 msgr2=0x7fb4380d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.330+0000 7fb43fc15640 1 --2- 172.21.15.26:0/72352145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43815e370 0x7fb4380d2520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb430009870 tx=0x7fb43002a270 comp rx=0 tx=0).stop 2024-04-24T23:21:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.330+0000 7fb43fc15640 1 -- 172.21.15.26:0/72352145 shutdown_connections 2024-04-24T23:21:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.330+0000 7fb43fc15640 1 --2- 172.21.15.26:0/72352145 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb43815e750 0x7fb4380d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.330+0000 7fb43fc15640 1 --2- 172.21.15.26:0/72352145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43815e370 0x7fb4380d2520 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.330+0000 7fb43fc15640 1 -- 172.21.15.26:0/72352145 >> 172.21.15.26:0/72352145 conn(0x7fb4380d40f0 msgr2=0x7fb4380d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.330+0000 7fb43fc15640 1 -- 172.21.15.26:0/72352145 shutdown_connections 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.330+0000 7fb43fc15640 1 -- 172.21.15.26:0/72352145 wait complete. 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.331+0000 7fb43fc15640 1 Processor -- start 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.331+0000 7fb43fc15640 1 -- start start 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43fc15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4380d4c10 0x7fb438150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43fc15640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb43815e370 0x7fb4381546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43fc15640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4380c9950 0x7fb4380d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43fc15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4380cfca0 con 0x7fb4380d4c10 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43fc15640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4380b3d00 con 0x7fb43815e370 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43fc15640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb4380cfe20 con 0x7fb4380c9950 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d9b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4380d4c10 0x7fb438150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d9b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4380d4c10 0x7fb438150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53642/0 (socket says 172.21.15.26:53642) 2024-04-24T23:21:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d9b1640 1 -- 172.21.15.26:0/2171873861 learned_addr learned my addr 172.21.15.26:0/2171873861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d9b1640 1 -- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4380c9950 msgr2=0x7fb4380d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d1b0640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb43815e370 0x7fb4381546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43e1b2640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4380c9950 0x7fb4380d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d9b1640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4380c9950 0x7fb4380d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d9b1640 1 -- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb43815e370 msgr2=0x7fb4381546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d9b1640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb43815e370 0x7fb4381546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d9b1640 1 -- 172.21.15.26:0/2171873861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4380cc210 con 0x7fb4380d4c10 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d1b0640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb43815e370 0x7fb4381546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43e1b2640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4380c9950 0x7fb4380d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.332+0000 7fb43d9b1640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4380d4c10 0x7fb438150970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb430000c00 tx=0x7fb4300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.333+0000 7fb42e7fc640 1 -- 172.21.15.26:0/2171873861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430037040 con 0x7fb4380d4c10 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.333+0000 7fb42e7fc640 1 -- 172.21.15.26:0/2171873861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb430042070 con 0x7fb4380d4c10 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.333+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4380cc4f0 con 0x7fb4380d4c10 2024-04-24T23:21:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.333+0000 7fb42e7fc640 1 -- 172.21.15.26:0/2171873861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430030bd0 con 0x7fb4380d4c10 2024-04-24T23:21:22.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.333+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4380ccab0 con 0x7fb4380d4c10 2024-04-24T23:21:22.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.334+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb438075610 con 0x7fb4380d4c10 2024-04-24T23:21:22.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.334+0000 7fb42e7fc640 1 -- 172.21.15.26:0/2171873861 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb430047460 con 0x7fb4380d4c10 2024-04-24T23:21:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.335+0000 7fb42e7fc640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb41c06be30 0x7fb41c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.335+0000 7fb42e7fc640 1 -- 172.21.15.26:0/2171873861 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb4300abb60 con 0x7fb4380d4c10 2024-04-24T23:21:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.335+0000 7fb43d1b0640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb41c06be30 0x7fb41c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.335+0000 7fb43d1b0640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb41c06be30 0x7fb41c06e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb428006fd0 tx=0x7fb428008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.339+0000 7fb42e7fc640 1 -- 172.21.15.26:0/2171873861 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb430045360 con 0x7fb4380d4c10 2024-04-24T23:21:22.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.527+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4380b49f0 con 0x7fb4380d4c10 2024-04-24T23:21:22.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.528+0000 7fb42e7fc640 1 -- 172.21.15.26:0/2171873861 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb43007a470 con 0x7fb4380d4c10 2024-04-24T23:21:22.530 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb41c06be30 msgr2=0x7fb41c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb41c06be30 0x7fb41c06e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb428006fd0 tx=0x7fb428008040 comp rx=0 tx=0).stop 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4380d4c10 msgr2=0x7fb438150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4380d4c10 0x7fb438150970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb430000c00 tx=0x7fb4300046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 shutdown_connections 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb41c06be30 0x7fb41c06e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4380c9950 0x7fb4380d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb43815e370 0x7fb4381546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 --2- 172.21.15.26:0/2171873861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4380d4c10 0x7fb438150970 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 >> 172.21.15.26:0/2171873861 conn(0x7fb4380d40f0 msgr2=0x7fb43806d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 shutdown_connections 2024-04-24T23:21:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.530+0000 7fb43fc15640 1 -- 172.21.15.26:0/2171873861 wait complete. 2024-04-24T23:21:22.543 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.644+0000 7f21cb596640 1 Processor -- start 2024-04-24T23:21:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.644+0000 7f21cb596640 1 -- start start 2024-04-24T23:21:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21cb596640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21c40b4ce0 0x7f21c40acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21cb596640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c40aeb10 con 0x7f21c415be40 2024-04-24T23:21:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21cb596640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c40aedf0 con 0x7f21c40d0750 2024-04-24T23:21:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21cb596640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f21c409df20 con 0x7f21c40b4ce0 2024-04-24T23:21:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21c9b33640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21c40b4ce0 0x7f21c40acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21c9b33640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21c40b4ce0 0x7f21c40acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53656/0 (socket says 172.21.15.26:53656) 2024-04-24T23:21:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21c9332640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21c415be40 0x7f21c40d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41210/0 (socket says 172.21.15.26:41210) 2024-04-24T23:21:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21c9b33640 1 -- 172.21.15.26:0/4110193204 learned_addr learned my addr 172.21.15.26:0/4110193204 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21c9b33640 1 -- 172.21.15.26:0/4110193204 >> v1:172.21.15.29:6789/0 conn(0x7f21c40d0750 legacy=0x7f21c415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21c9b33640 1 -- 172.21.15.26:0/4110193204 >> v1:172.21.15.26:6789/0 conn(0x7f21c415be40 legacy=0x7f21c40d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.645+0000 7f21c9b33640 1 -- 172.21.15.26:0/4110193204 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c40af0d0 con 0x7f21c40b4ce0 2024-04-24T23:21:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.646+0000 7f21c9b33640 1 --2- 172.21.15.26:0/4110193204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40b4ce0 0x7f21c40acd30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009e50 tx=0x7f21bc02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc0d5d6a72df70b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.646+0000 7f21bbfff640 1 -- 172.21.15.26:0/4110193204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc036070 con 0x7f21c40b4ce0 2024-04-24T23:21:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.646+0000 7f21bbfff640 1 -- 172.21.15.26:0/4110193204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc031600 con 0x7f21c40b4ce0 2024-04-24T23:21:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.646+0000 7f21bbfff640 1 -- 172.21.15.26:0/4110193204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0319d0 con 0x7f21c40b4ce0 2024-04-24T23:21:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.647+0000 7f21cb596640 1 -- 172.21.15.26:0/4110193204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40b4ce0 msgr2=0x7f21c40acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.647+0000 7f21cb596640 1 --2- 172.21.15.26:0/4110193204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40b4ce0 0x7f21c40acd30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009e50 tx=0x7f21bc02a760 comp rx=0 tx=0).stop 2024-04-24T23:21:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.647+0000 7f21cb596640 1 -- 172.21.15.26:0/4110193204 shutdown_connections 2024-04-24T23:21:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.647+0000 7f21cb596640 1 --2- 172.21.15.26:0/4110193204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40b4ce0 0x7f21c40acd30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.647+0000 7f21cb596640 1 -- 172.21.15.26:0/4110193204 >> 172.21.15.26:0/4110193204 conn(0x7f21c406b540 msgr2=0x7f21c406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:22.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.647+0000 7f21cb596640 1 -- 172.21.15.26:0/4110193204 shutdown_connections 2024-04-24T23:21:22.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.647+0000 7f21cb596640 1 -- 172.21.15.26:0/4110193204 wait complete. 2024-04-24T23:21:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.648+0000 7f21cb596640 1 Processor -- start 2024-04-24T23:21:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.648+0000 7f21cb596640 1 -- start start 2024-04-24T23:21:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21cb596640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21c40b4ce0 0x7f21c4142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21cb596640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40d0750 0x7f21c415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21cb596640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21c415be40 0x7f21c415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21cb596640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21c406cd10 con 0x7f21c40d0750 2024-04-24T23:21:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21cb596640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f21c406cb90 con 0x7f21c415be40 2024-04-24T23:21:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21cb596640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f21c415c930 con 0x7f21c40b4ce0 2024-04-24T23:21:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c8b31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40d0750 0x7f21c415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c8b31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40d0750 0x7f21c415dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53658/0 (socket says 172.21.15.26:53658) 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c8b31640 1 -- 172.21.15.26:0/2415902069 learned_addr learned my addr 172.21.15.26:0/2415902069 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c9b33640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21c415be40 0x7f21c415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c8b31640 1 -- 172.21.15.26:0/2415902069 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21c40b4ce0 msgr2=0x7f21c4142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c9332640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21c40b4ce0 0x7f21c4142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c8b31640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21c40b4ce0 0x7f21c4142740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c8b31640 1 -- 172.21.15.26:0/2415902069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21c415be40 msgr2=0x7f21c415e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c8b31640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21c415be40 0x7f21c415e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c8b31640 1 -- 172.21.15.26:0/2415902069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c4153160 con 0x7f21c40d0750 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c9b33640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21c415be40 0x7f21c415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21c8b31640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40d0750 0x7f21c415dd70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f21b400c410 tx=0x7f21b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21b9ffb640 1 -- 172.21.15.26:0/2415902069 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4017070 con 0x7f21c40d0750 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.649+0000 7f21b9ffb640 1 -- 172.21.15.26:0/2415902069 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b40035f0 con 0x7f21c40d0750 2024-04-24T23:21:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.650+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c4153440 con 0x7f21c40d0750 2024-04-24T23:21:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.650+0000 7f21b9ffb640 1 -- 172.21.15.26:0/2415902069 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4003a20 con 0x7f21c40d0750 2024-04-24T23:21:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.650+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c4153a00 con 0x7f21c40d0750 2024-04-24T23:21:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.651+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c40ae3e0 con 0x7f21c40d0750 2024-04-24T23:21:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.651+0000 7f21b9ffb640 1 -- 172.21.15.26:0/2415902069 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21b401a070 con 0x7f21c40d0750 2024-04-24T23:21:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.652+0000 7f21b9ffb640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21a806be60 0x7f21a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.652+0000 7f21b9ffb640 1 -- 172.21.15.26:0/2415902069 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f21b408a2b0 con 0x7f21c40d0750 2024-04-24T23:21:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.652+0000 7f21c9332640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21a806be60 0x7f21a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.652+0000 7f21c9332640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21a806be60 0x7f21a806e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f21c0000f60 tx=0x7f21c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.656+0000 7f21b9ffb640 1 -- 172.21.15.26:0/2415902069 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f21c40ae3e0 con 0x7f21c40d0750 2024-04-24T23:21:22.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.837+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f21c4075610 con 0x7f21c40d0750 2024-04-24T23:21:22.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.838+0000 7f21b9ffb640 1 -- 172.21.15.26:0/2415902069 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f21b4058cf0 con 0x7f21c40d0750 2024-04-24T23:21:22.840 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21a806be60 msgr2=0x7f21a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21a806be60 0x7f21a806e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f21c0000f60 tx=0x7f21c0009040 comp rx=0 tx=0).stop 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40d0750 msgr2=0x7f21c415dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40d0750 0x7f21c415dd70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f21b400c410 tx=0x7f21b400c8d0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 shutdown_connections 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21a806be60 0x7f21a806e320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21c415be40 0x7f21c415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c40d0750 0x7f21c415dd70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 --2- 172.21.15.26:0/2415902069 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21c40b4ce0 0x7f21c4142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 >> 172.21.15.26:0/2415902069 conn(0x7f21c406b540 msgr2=0x7f21c406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 shutdown_connections 2024-04-24T23:21:22.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.840+0000 7f21cb596640 1 -- 172.21.15.26:0/2415902069 wait complete. 2024-04-24T23:21:22.852 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:22.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.951+0000 7fa5b35c1640 1 Processor -- start 2024-04-24T23:21:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.951+0000 7fa5b35c1640 1 -- start start 2024-04-24T23:21:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.952+0000 7fa5b35c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac0d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.952+0000 7fa5b35c1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa5ac0b3ac0 0x7fa5ac0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.952+0000 7fa5b35c1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ac0b5840 con 0x7fa5ac06c230 2024-04-24T23:21:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.952+0000 7fa5b35c1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa5ac0b4460 con 0x7fa5ac0b3ac0 2024-04-24T23:21:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.952+0000 7fa5b35c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ac0b5e50 con 0x7fa5ac15e290 2024-04-24T23:21:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.952+0000 7fa5b135d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac0d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.952+0000 7fa5b0b5c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa5ac0b3ac0 0x7fa5ac0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.952+0000 7fa5b135d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac0d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:32816/0 (socket says 172.21.15.26:32816) 2024-04-24T23:21:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.952+0000 7fa5b135d640 1 -- 172.21.15.26:0/3510410215 learned_addr learned my addr 172.21.15.26:0/3510410215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.953+0000 7fa5b0b5c640 1 -- 172.21.15.26:0/3510410215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 msgr2=0x7fa5ac0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.953+0000 7fa5b0b5c640 1 --2- 172.21.15.26:0/3510410215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac0d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.953+0000 7fa5b0b5c640 1 -- 172.21.15.26:0/3510410215 >> v1:172.21.15.29:6789/0 conn(0x7fa5ac06c230 legacy=0x7fa5ac06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.953+0000 7fa5b0b5c640 1 -- 172.21.15.26:0/3510410215 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac0cfb30 con 0x7fa5ac0b3ac0 2024-04-24T23:21:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.953+0000 7fa5b135d640 1 --2- 172.21.15.26:0/3510410215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac0d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.953+0000 7fa5b0b5c640 1 --2- 172.21.15.26:0/3510410215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5ac0b3ac0 0x7fa5ac0b3f20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0097a0 tx=0x7fa59c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0941a1cb432ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5a3fff640 1 -- 172.21.15.26:0/3510410215 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c037070 con 0x7fa5ac0b3ac0 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5a3fff640 1 -- 172.21.15.26:0/3510410215 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c032440 con 0x7fa5ac0b3ac0 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5a3fff640 1 -- 172.21.15.26:0/3510410215 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c032810 con 0x7fa5ac0b3ac0 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5b35c1640 1 -- 172.21.15.26:0/3510410215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5ac0b3ac0 msgr2=0x7fa5ac0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5b35c1640 1 --2- 172.21.15.26:0/3510410215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5ac0b3ac0 0x7fa5ac0b3f20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0097a0 tx=0x7fa59c02a180 comp rx=0 tx=0).stop 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5b35c1640 1 -- 172.21.15.26:0/3510410215 shutdown_connections 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5b35c1640 1 --2- 172.21.15.26:0/3510410215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5ac0b3ac0 0x7fa5ac0b3f20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5b35c1640 1 --2- 172.21.15.26:0/3510410215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac0d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5b35c1640 1 -- 172.21.15.26:0/3510410215 >> 172.21.15.26:0/3510410215 conn(0x7fa5ac06b120 msgr2=0x7fa5ac06b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5b35c1640 1 -- 172.21.15.26:0/3510410215 shutdown_connections 2024-04-24T23:21:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.954+0000 7fa5b35c1640 1 -- 172.21.15.26:0/3510410215 wait complete. 2024-04-24T23:21:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.955+0000 7fa5b35c1640 1 Processor -- start 2024-04-24T23:21:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.955+0000 7fa5b35c1640 1 -- start start 2024-04-24T23:21:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b35c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5ac06c230 0x7fa5ac161290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b35c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5ac0b3ac0 0x7fa5ac1f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b35c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac1f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b35c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac0b5ae0 con 0x7fa5ac06c230 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b35c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac0d00f0 con 0x7fa5ac0b3ac0 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b35c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ac15e670 con 0x7fa5ac15e290 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b135d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5ac06c230 0x7fa5ac161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b135d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5ac06c230 0x7fa5ac161290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53674/0 (socket says 172.21.15.26:53674) 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b135d640 1 -- 172.21.15.26:0/2641731991 learned_addr learned my addr 172.21.15.26:0/2641731991 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b0b5c640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5ac0b3ac0 0x7fa5ac1f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.956+0000 7fa5b1b5e640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac1f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5b135d640 1 -- 172.21.15.26:0/2641731991 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 msgr2=0x7fa5ac1f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5b135d640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac1f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5b135d640 1 -- 172.21.15.26:0/2641731991 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5ac0b3ac0 msgr2=0x7fa5ac1f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5b135d640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5ac0b3ac0 0x7fa5ac1f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5b135d640 1 -- 172.21.15.26:0/2641731991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac15f860 con 0x7fa5ac06c230 2024-04-24T23:21:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5b135d640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5ac06c230 0x7fa5ac161290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9e0 tx=0x7fa5a800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5a1ffb640 1 -- 172.21.15.26:0/2641731991 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8017070 con 0x7fa5ac06c230 2024-04-24T23:21:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5a1ffb640 1 -- 172.21.15.26:0/2641731991 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a800c9d0 con 0x7fa5ac06c230 2024-04-24T23:21:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5a1ffb640 1 -- 172.21.15.26:0/2641731991 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800cd80 con 0x7fa5ac06c230 2024-04-24T23:21:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac1f7280 con 0x7fa5ac06c230 2024-04-24T23:21:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.957+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac1f7840 con 0x7fa5ac06c230 2024-04-24T23:21:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.958+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ac0694c0 con 0x7fa5ac06c230 2024-04-24T23:21:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.959+0000 7fa5a1ffb640 1 -- 172.21.15.26:0/2641731991 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5a801a070 con 0x7fa5ac06c230 2024-04-24T23:21:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.959+0000 7fa5a1ffb640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa59006be30 0x7fa59006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.959+0000 7fa5a1ffb640 1 -- 172.21.15.26:0/2641731991 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa5a808b020 con 0x7fa5ac06c230 2024-04-24T23:21:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.959+0000 7fa5b0b5c640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa59006be30 0x7fa59006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.960+0000 7fa5b0b5c640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa59006be30 0x7fa59006e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa59c034000 tx=0x7fa59c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:22.964+0000 7fa5a1ffb640 1 -- 172.21.15.26:0/2641731991 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa5a8059390 con 0x7fa5ac06c230 2024-04-24T23:21:23.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.148+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5ac06e440 con 0x7fa5ac06c230 2024-04-24T23:21:23.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.149+0000 7fa5a1ffb640 1 -- 172.21.15.26:0/2641731991 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fa5a805d690 con 0x7fa5ac06c230 2024-04-24T23:21:23.151 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:23.151 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa59006be30 msgr2=0x7fa59006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa59006be30 0x7fa59006e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa59c034000 tx=0x7fa59c02e000 comp rx=0 tx=0).stop 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5ac06c230 msgr2=0x7fa5ac161290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5ac06c230 0x7fa5ac161290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9e0 tx=0x7fa5a800eea0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 shutdown_connections 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa59006be30 0x7fa59006e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5ac15e290 0x7fa5ac1f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5ac0b3ac0 0x7fa5ac1f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 --2- 172.21.15.26:0/2641731991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5ac06c230 0x7fa5ac161290 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 >> 172.21.15.26:0/2641731991 conn(0x7fa5ac06b120 msgr2=0x7fa5ac06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 shutdown_connections 2024-04-24T23:21:23.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.152+0000 7fa5b35c1640 1 -- 172.21.15.26:0/2641731991 wait complete. 2024-04-24T23:21:23.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:23.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:23.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.267+0000 7f1e60cd4640 1 Processor -- start 2024-04-24T23:21:23.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.267+0000 7f1e60cd4640 1 -- start start 2024-04-24T23:21:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.268+0000 7f1e60cd4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e5c0d0750 0x7f1e5c0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.268+0000 7f1e60cd4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 0x7f1e5c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.268+0000 7f1e60cd4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e5c0aeb10 con 0x7f1e5c15be40 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.268+0000 7f1e60cd4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5c09df20 con 0x7f1e5c0d0750 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.268+0000 7f1e60cd4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c0aedf0 con 0x7f1e5c0b4ce0 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.268+0000 7f1e59d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e5c0d0750 0x7f1e5c0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.268+0000 7f1e5a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1e5c15be40 0x7f1e5c0d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41224/0 (socket says 172.21.15.26:41224) 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.268+0000 7f1e59d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e5c0d0750 0x7f1e5c0acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53684/0 (socket says 172.21.15.26:53684) 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.268+0000 7f1e5a59c640 1 -- 172.21.15.26:0/48162382 learned_addr learned my addr 172.21.15.26:0/48162382 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.269+0000 7f1e5ad9d640 1 --2- 172.21.15.26:0/48162382 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 0x7f1e5c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.269+0000 7f1e59d9b640 1 -- 172.21.15.26:0/48162382 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 msgr2=0x7f1e5c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.269+0000 7f1e59d9b640 1 --2- 172.21.15.26:0/48162382 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 0x7f1e5c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.269+0000 7f1e59d9b640 1 -- 172.21.15.26:0/48162382 >> v1:172.21.15.26:6789/0 conn(0x7f1e5c15be40 legacy=0x7f1e5c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.269+0000 7f1e59d9b640 1 -- 172.21.15.26:0/48162382 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c0af0d0 con 0x7f1e5c0d0750 2024-04-24T23:21:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.269+0000 7f1e59d9b640 1 --2- 172.21.15.26:0/48162382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c0d0750 0x7f1e5c0acd30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097a0 tx=0x7f1e5002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b73b08862c7a13d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.269+0000 7f1e5959a640 1 -- 172.21.15.26:0/48162382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50037070 con 0x7f1e5c0d0750 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.269+0000 7f1e5959a640 1 -- 172.21.15.26:0/48162382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e50032440 con 0x7f1e5c0d0750 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.269+0000 7f1e5959a640 1 -- 172.21.15.26:0/48162382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50032810 con 0x7f1e5c0d0750 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.270+0000 7f1e60cd4640 1 -- 172.21.15.26:0/48162382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c0d0750 msgr2=0x7f1e5c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.270+0000 7f1e60cd4640 1 --2- 172.21.15.26:0/48162382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c0d0750 0x7f1e5c0acd30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097a0 tx=0x7f1e5002a180 comp rx=0 tx=0).stop 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.270+0000 7f1e60cd4640 1 -- 172.21.15.26:0/48162382 shutdown_connections 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.270+0000 7f1e60cd4640 1 --2- 172.21.15.26:0/48162382 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 0x7f1e5c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.270+0000 7f1e60cd4640 1 --2- 172.21.15.26:0/48162382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c0d0750 0x7f1e5c0acd30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.270+0000 7f1e60cd4640 1 -- 172.21.15.26:0/48162382 >> 172.21.15.26:0/48162382 conn(0x7f1e5c06b540 msgr2=0x7f1e5c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.270+0000 7f1e60cd4640 1 -- 172.21.15.26:0/48162382 shutdown_connections 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.270+0000 7f1e60cd4640 1 -- 172.21.15.26:0/48162382 wait complete. 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.271+0000 7f1e60cd4640 1 Processor -- start 2024-04-24T23:21:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.271+0000 7f1e60cd4640 1 -- start start 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e60cd4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 0x7f1e5c1f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e60cd4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e5c0d0750 0x7f1e5c070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e60cd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c15be40 0x7f1e5c1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e60cd4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c06cd10 con 0x7f1e5c15be40 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e60cd4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c06cb90 con 0x7f1e5c0d0750 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e60cd4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c15c760 con 0x7f1e5c0b4ce0 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e5ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c15be40 0x7f1e5c1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e5ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c15be40 0x7f1e5c1f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53690/0 (socket says 172.21.15.26:53690) 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e5ad9d640 1 -- 172.21.15.26:0/3415177087 learned_addr learned my addr 172.21.15.26:0/3415177087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.272+0000 7f1e5a59c640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 0x7f1e5c1f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e59d9b640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e5c0d0750 0x7f1e5c070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e5ad9d640 1 -- 172.21.15.26:0/3415177087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 msgr2=0x7f1e5c1f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e5ad9d640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 0x7f1e5c1f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e5ad9d640 1 -- 172.21.15.26:0/3415177087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e5c0d0750 msgr2=0x7f1e5c070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e5ad9d640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e5c0d0750 0x7f1e5c070e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e5ad9d640 1 -- 172.21.15.26:0/3415177087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c0714a0 con 0x7f1e5c15be40 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e5a59c640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 0x7f1e5c1f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e5ad9d640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c15be40 0x7f1e5c1f5030 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c007e40 tx=0x7f1e4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e4affd640 1 -- 172.21.15.26:0/3415177087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c014620 con 0x7f1e5c15be40 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e4affd640 1 -- 172.21.15.26:0/3415177087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e4c019070 con 0x7f1e5c15be40 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e4affd640 1 -- 172.21.15.26:0/3415177087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c014a20 con 0x7f1e5c15be40 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c071780 con 0x7f1e5c15be40 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.273+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c071d40 con 0x7f1e5c15be40 2024-04-24T23:21:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.274+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c075610 con 0x7f1e5c15be40 2024-04-24T23:21:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.275+0000 7f1e4affd640 1 -- 172.21.15.26:0/3415177087 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e4c014bc0 con 0x7f1e5c15be40 2024-04-24T23:21:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.275+0000 7f1e4affd640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e3c06be30 0x7f1e3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.275+0000 7f1e4affd640 1 -- 172.21.15.26:0/3415177087 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e4c08b770 con 0x7f1e5c15be40 2024-04-24T23:21:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.275+0000 7f1e5a59c640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e3c06be30 0x7f1e3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.276+0000 7f1e5a59c640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e3c06be30 0x7f1e3c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0ceae0 tx=0x7f1e54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.279+0000 7f1e4affd640 1 -- 172.21.15.26:0/3415177087 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e4c05a120 con 0x7f1e5c15be40 2024-04-24T23:21:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.461+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e5c15a000 con 0x7f1e5c15be40 2024-04-24T23:21:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.462+0000 7f1e4affd640 1 -- 172.21.15.26:0/3415177087 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1e4c059a20 con 0x7f1e5c15be40 2024-04-24T23:21:23.464 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e3c06be30 msgr2=0x7f1e3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e3c06be30 0x7f1e3c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0ceae0 tx=0x7f1e54009040 comp rx=0 tx=0).stop 2024-04-24T23:21:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c15be40 msgr2=0x7f1e5c1f5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c15be40 0x7f1e5c1f5030 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c007e40 tx=0x7f1e4c00e4b0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 shutdown_connections 2024-04-24T23:21:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e3c06be30 0x7f1e3c06e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e5c15be40 0x7f1e5c1f5030 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e5c0d0750 0x7f1e5c070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 --2- 172.21.15.26:0/3415177087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e5c0b4ce0 0x7f1e5c1f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 >> 172.21.15.26:0/3415177087 conn(0x7f1e5c06b540 msgr2=0x7f1e5c0b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 shutdown_connections 2024-04-24T23:21:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.464+0000 7f1e60cd4640 1 -- 172.21.15.26:0/3415177087 wait complete. 2024-04-24T23:21:23.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:23.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.579+0000 7f9d22e76640 1 Processor -- start 2024-04-24T23:21:23.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.579+0000 7f9d22e76640 1 -- start start 2024-04-24T23:21:23.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.580+0000 7f9d22e76640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d1c15cc30 0x7f9d1c15d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.580+0000 7f9d22e76640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9d1c0afce0 0x7f9d1c071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.580+0000 7f9d22e76640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d1c0b4780 con 0x7f9d1c071c60 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.580+0000 7f9d22e76640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1c0b5aa0 con 0x7f9d1c15cc30 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.580+0000 7f9d22e76640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1c0b4a00 con 0x7f9d1c0afce0 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.580+0000 7f9d20c12640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d1c15cc30 0x7f9d1c15d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.580+0000 7f9d21413640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9d1c071c60 0x7f9d1c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41236/0 (socket says 172.21.15.26:41236) 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.580+0000 7f9d20c12640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d1c15cc30 0x7f9d1c15d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53706/0 (socket says 172.21.15.26:53706) 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.580+0000 7f9d21413640 1 -- 172.21.15.26:0/1926134254 learned_addr learned my addr 172.21.15.26:0/1926134254 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.581+0000 7f9d20c12640 1 -- 172.21.15.26:0/1926134254 >> v2:172.21.15.29:3300/0 conn(0x7f9d1c0afce0 msgr2=0x7f9d1c071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.581+0000 7f9d20c12640 1 --2- 172.21.15.26:0/1926134254 >> v2:172.21.15.29:3300/0 conn(0x7f9d1c0afce0 0x7f9d1c071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.581+0000 7f9d20c12640 1 -- 172.21.15.26:0/1926134254 >> v1:172.21.15.26:6789/0 conn(0x7f9d1c071c60 legacy=0x7f9d1c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.581+0000 7f9d20c12640 1 -- 172.21.15.26:0/1926134254 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c0b4ce0 con 0x7f9d1c15cc30 2024-04-24T23:21:23.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.581+0000 7f9d1b7fe640 1 -- 172.21.15.26:0/1926134254 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49878358 0 0) 0x7f9d14001490 con 0x7f9d1c071c60 2024-04-24T23:21:23.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.581+0000 7f9d20c12640 1 --2- 172.21.15.26:0/1926134254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 0x7f9d1c15d010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0b30f0 tx=0x7f9d0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a41bbb11a471b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:23.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.581+0000 7f9d1b7fe640 1 -- 172.21.15.26:0/1926134254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04037070 con 0x7f9d1c15cc30 2024-04-24T23:21:23.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.581+0000 7f9d1b7fe640 1 -- 172.21.15.26:0/1926134254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d04032440 con 0x7f9d1c15cc30 2024-04-24T23:21:23.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.581+0000 7f9d1b7fe640 1 -- 172.21.15.26:0/1926134254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04032810 con 0x7f9d1c15cc30 2024-04-24T23:21:23.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.582+0000 7f9d22e76640 1 -- 172.21.15.26:0/1926134254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 msgr2=0x7f9d1c15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.582+0000 7f9d22e76640 1 --2- 172.21.15.26:0/1926134254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 0x7f9d1c15d010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0b30f0 tx=0x7f9d0402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.582+0000 7f9d22e76640 1 -- 172.21.15.26:0/1926134254 shutdown_connections 2024-04-24T23:21:23.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.582+0000 7f9d22e76640 1 --2- 172.21.15.26:0/1926134254 >> v2:172.21.15.29:3300/0 conn(0x7f9d1c0afce0 0x7f9d1c071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.582+0000 7f9d22e76640 1 --2- 172.21.15.26:0/1926134254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 0x7f9d1c15d010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.582+0000 7f9d22e76640 1 -- 172.21.15.26:0/1926134254 >> 172.21.15.26:0/1926134254 conn(0x7f9d1c0b2970 msgr2=0x7f9d1c0d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.582+0000 7f9d22e76640 1 -- 172.21.15.26:0/1926134254 shutdown_connections 2024-04-24T23:21:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.582+0000 7f9d22e76640 1 -- 172.21.15.26:0/1926134254 wait complete. 2024-04-24T23:21:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.583+0000 7f9d22e76640 1 Processor -- start 2024-04-24T23:21:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.583+0000 7f9d22e76640 1 -- start start 2024-04-24T23:21:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d22e76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c071c60 0x7f9d1c0ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d22e76640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d1c0afce0 0x7f9d1c0ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d22e76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 0x7f9d1c1e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d22e76640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c0b5580 con 0x7f9d1c15cc30 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d22e76640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c0b52a0 con 0x7f9d1c071c60 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d22e76640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c15d550 con 0x7f9d1c0afce0 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d21413640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 0x7f9d1c1e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d21413640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 0x7f9d1c1e6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53716/0 (socket says 172.21.15.26:53716) 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d21413640 1 -- 172.21.15.26:0/682350048 learned_addr learned my addr 172.21.15.26:0/682350048 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d20c12640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c071c60 0x7f9d1c0ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d1bfff640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d1c0afce0 0x7f9d1c0ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.584+0000 7f9d21413640 1 -- 172.21.15.26:0/682350048 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d1c0afce0 msgr2=0x7f9d1c0ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d21413640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d1c0afce0 0x7f9d1c0ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d21413640 1 -- 172.21.15.26:0/682350048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c071c60 msgr2=0x7f9d1c0ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d21413640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c071c60 0x7f9d1c0ab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d21413640 1 -- 172.21.15.26:0/682350048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c1e5c40 con 0x7f9d1c15cc30 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d20c12640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c071c60 0x7f9d1c0ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d21413640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 0x7f9d1c1e6a00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400c700 tx=0x7f9d1400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d197fa640 1 -- 172.21.15.26:0/682350048 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14018070 con 0x7f9d1c15cc30 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d197fa640 1 -- 172.21.15.26:0/682350048 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1400ce10 con 0x7f9d1c15cc30 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c1e5f20 con 0x7f9d1c15cc30 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d197fa640 1 -- 172.21.15.26:0/682350048 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400a860 con 0x7f9d1c15cc30 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.585+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c1e64e0 con 0x7f9d1c15cc30 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.586+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c075610 con 0x7f9d1c15cc30 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.586+0000 7f9d197fa640 1 -- 172.21.15.26:0/682350048 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d1401b070 con 0x7f9d1c15cc30 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.587+0000 7f9d197fa640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cfc06be30 0x7f9cfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.587+0000 7f9d197fa640 1 -- 172.21.15.26:0/682350048 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9d1408a490 con 0x7f9d1c15cc30 2024-04-24T23:21:23.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.587+0000 7f9d20c12640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cfc06be30 0x7f9cfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.587+0000 7f9d20c12640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cfc06be30 0x7f9cfc06e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d04034000 tx=0x7f9d0402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.591+0000 7f9d197fa640 1 -- 172.21.15.26:0/682350048 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9d1c0b4780 con 0x7f9d1c15cc30 2024-04-24T23:21:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.774+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d1c0d0c10 con 0x7f9d1c15cc30 2024-04-24T23:21:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.775+0000 7f9d197fa640 1 -- 172.21.15.26:0/682350048 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f9d14058e30 con 0x7f9d1c15cc30 2024-04-24T23:21:23.777 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cfc06be30 msgr2=0x7f9cfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cfc06be30 0x7f9cfc06e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d04034000 tx=0x7f9d0402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 msgr2=0x7f9d1c1e6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 0x7f9d1c1e6a00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400c700 tx=0x7f9d1400cbc0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 shutdown_connections 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9cfc06be30 0x7f9cfc06e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d1c15cc30 0x7f9d1c1e6a00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d1c0afce0 0x7f9d1c0ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 --2- 172.21.15.26:0/682350048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c071c60 0x7f9d1c0ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 >> 172.21.15.26:0/682350048 conn(0x7f9d1c0b2970 msgr2=0x7f9d1c15b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 shutdown_connections 2024-04-24T23:21:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.777+0000 7f9d22e76640 1 -- 172.21.15.26:0/682350048 wait complete. 2024-04-24T23:21:23.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.891+0000 7f6fbbb70640 1 Processor -- start 2024-04-24T23:21:23.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.892+0000 7f6fbbb70640 1 -- start start 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.892+0000 7f6fbbb70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb415e370 0x7f6fb40d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.892+0000 7f6fbbb70640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6fb415e750 0x7f6fb40d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.892+0000 7f6fbbb70640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb40d2120 con 0x7f6fb40d4c10 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.892+0000 7f6fbbb70640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6fb40d1510 con 0x7f6fb415e750 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.892+0000 7f6fbbb70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb40d1ed0 con 0x7f6fb415e370 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fba10d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6fb40d4c10 0x7f6fb40b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41984/0 (socket says 172.21.15.26:41984) 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fba10d640 1 -- 172.21.15.26:0/17988888 learned_addr learned my addr 172.21.15.26:0/17988888 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fb990c640 1 --2- 172.21.15.26:0/17988888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb415e370 0x7f6fb40d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fb910b640 1 --2- 172.21.15.26:0/17988888 >> v2:172.21.15.29:3300/0 conn(0x7f6fb415e750 0x7f6fb40d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fb890a640 1 -- 172.21.15.26:0/17988888 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603089656 0 0) 0x7f6fb40d2120 con 0x7f6fb40d4c10 2024-04-24T23:21:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fb890a640 1 -- 172.21.15.26:0/17988888 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fa0003520 con 0x7f6fb40d4c10 2024-04-24T23:21:23.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fb990c640 1 -- 172.21.15.26:0/17988888 >> v2:172.21.15.29:3300/0 conn(0x7f6fb415e750 msgr2=0x7f6fb40d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:23.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fb990c640 1 --2- 172.21.15.26:0/17988888 >> v2:172.21.15.29:3300/0 conn(0x7f6fb415e750 0x7f6fb40d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fb990c640 1 -- 172.21.15.26:0/17988888 >> v1:172.21.15.26:6789/0 conn(0x7f6fb40d4c10 legacy=0x7f6fb40b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.893+0000 7f6fb990c640 1 -- 172.21.15.26:0/17988888 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb40b3740 con 0x7f6fb415e370 2024-04-24T23:21:23.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.894+0000 7f6fb990c640 1 --2- 172.21.15.26:0/17988888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb415e370 0x7f6fb40d2520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00097a0 tx=0x7f6fb002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4de1c1138910209f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:23.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.894+0000 7f6fb890a640 1 -- 172.21.15.26:0/17988888 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0037070 con 0x7f6fb415e370 2024-04-24T23:21:23.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.894+0000 7f6fb890a640 1 -- 172.21.15.26:0/17988888 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb0032440 con 0x7f6fb415e370 2024-04-24T23:21:23.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.894+0000 7f6fb890a640 1 -- 172.21.15.26:0/17988888 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00327f0 con 0x7f6fb415e370 2024-04-24T23:21:23.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.895+0000 7f6fbbb70640 1 -- 172.21.15.26:0/17988888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb415e370 msgr2=0x7f6fb40d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.895+0000 7f6fbbb70640 1 --2- 172.21.15.26:0/17988888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb415e370 0x7f6fb40d2520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00097a0 tx=0x7f6fb002a180 comp rx=0 tx=0).stop 2024-04-24T23:21:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.895+0000 7f6fbbb70640 1 -- 172.21.15.26:0/17988888 shutdown_connections 2024-04-24T23:21:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.895+0000 7f6fbbb70640 1 --2- 172.21.15.26:0/17988888 >> v2:172.21.15.29:3300/0 conn(0x7f6fb415e750 0x7f6fb40d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.895+0000 7f6fbbb70640 1 --2- 172.21.15.26:0/17988888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb415e370 0x7f6fb40d2520 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.895+0000 7f6fbbb70640 1 -- 172.21.15.26:0/17988888 >> 172.21.15.26:0/17988888 conn(0x7f6fb40d40f0 msgr2=0x7f6fb40d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.895+0000 7f6fbbb70640 1 -- 172.21.15.26:0/17988888 shutdown_connections 2024-04-24T23:21:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.895+0000 7f6fbbb70640 1 -- 172.21.15.26:0/17988888 wait complete. 2024-04-24T23:21:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.896+0000 7f6fbbb70640 1 Processor -- start 2024-04-24T23:21:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.896+0000 7f6fbbb70640 1 -- start start 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fbbb70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb40d4c10 0x7f6fb40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fbbb70640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fb415e370 0x7f6fb40cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fbbb70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fb4153190 0x7f6fb40cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fbbb70640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb40cfca0 con 0x7f6fb4153190 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fbbb70640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb40b3d00 con 0x7f6fb415e370 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fbbb70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb40cfe20 con 0x7f6fb40d4c10 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fba10d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fb4153190 0x7f6fb40cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fba10d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fb4153190 0x7f6fb40cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57982/0 (socket says 172.21.15.26:57982) 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fba10d640 1 -- 172.21.15.26:0/2055393427 learned_addr learned my addr 172.21.15.26:0/2055393427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:23.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fb990c640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb40d4c10 0x7f6fb40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fb910b640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fb415e370 0x7f6fb40cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fba10d640 1 -- 172.21.15.26:0/2055393427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb40d4c10 msgr2=0x7f6fb40b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fba10d640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb40d4c10 0x7f6fb40b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fba10d640 1 -- 172.21.15.26:0/2055393427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fb415e370 msgr2=0x7f6fb40cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fba10d640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fb415e370 0x7f6fb40cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fba10d640 1 -- 172.21.15.26:0/2055393427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb40b25b0 con 0x7f6fb4153190 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fba10d640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fb4153190 0x7f6fb40cb1c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00cb40 tx=0x7f6fac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6faa7fc640 1 -- 172.21.15.26:0/2055393427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac017070 con 0x7f6fb4153190 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6faa7fc640 1 -- 172.21.15.26:0/2055393427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac00ac40 con 0x7f6fb4153190 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb40cdc00 con 0x7f6fb4153190 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6faa7fc640 1 -- 172.21.15.26:0/2055393427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac012620 con 0x7f6fb4153190 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.897+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb40c7dc0 con 0x7f6fb4153190 2024-04-24T23:21:23.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.899+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb40ce120 con 0x7f6fb4153190 2024-04-24T23:21:23.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.899+0000 7f6faa7fc640 1 -- 172.21.15.26:0/2055393427 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fac01a070 con 0x7f6fb4153190 2024-04-24T23:21:23.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.899+0000 7f6faa7fc640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f9806be60 0x7f6f9806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:23.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.900+0000 7f6fb990c640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f9806be60 0x7f6f9806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:23.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.900+0000 7f6faa7fc640 1 -- 172.21.15.26:0/2055393427 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6fac08aab0 con 0x7f6fb4153190 2024-04-24T23:21:23.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.900+0000 7f6fb990c640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f9806be60 0x7f6f9806e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0034000 tx=0x7f6fb002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:23.904+0000 7f6faa7fc640 1 -- 172.21.15.26:0/2055393427 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6fac058ea0 con 0x7f6fb4153190 2024-04-24T23:21:24.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.088+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6fb4075610 con 0x7f6fb4153190 2024-04-24T23:21:24.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.088+0000 7f6faa7fc640 1 -- 172.21.15.26:0/2055393427 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f6fac05d1a0 con 0x7f6fb4153190 2024-04-24T23:21:24.090 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f9806be60 msgr2=0x7f6f9806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f9806be60 0x7f6f9806e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0034000 tx=0x7f6fb002e000 comp rx=0 tx=0).stop 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fb4153190 msgr2=0x7f6fb40cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fb4153190 0x7f6fb40cb1c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00cb40 tx=0x7f6fac00a060 comp rx=0 tx=0).stop 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 shutdown_connections 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f9806be60 0x7f6f9806e320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fb4153190 0x7f6fb40cb1c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fb415e370 0x7f6fb40cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 --2- 172.21.15.26:0/2055393427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fb40d4c10 0x7f6fb40b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.090+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 >> 172.21.15.26:0/2055393427 conn(0x7f6fb40d40f0 msgr2=0x7f6fb415c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.091+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 shutdown_connections 2024-04-24T23:21:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.091+0000 7f6fbbb70640 1 -- 172.21.15.26:0/2055393427 wait complete. 2024-04-24T23:21:24.134 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-04-24T23:21:24.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.233+0000 7f5391fee640 1 Processor -- start 2024-04-24T23:21:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.233+0000 7f5391fee640 1 -- start start 2024-04-24T23:21:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f5391fee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f538c0d4a30 0x7f538c0d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f5391fee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 0x7f538c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f5391fee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f538c15e750 con 0x7f538c0d0b10 2024-04-24T23:21:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f5391fee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f538c0d2640 con 0x7f538c0d4a30 2024-04-24T23:21:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f5391fee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f538c0d1d30 con 0x7f538c0d0ef0 2024-04-24T23:21:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f538affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f538c0d4a30 0x7f538c0d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f538bfff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 0x7f538c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f538affd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f538c0d4a30 0x7f538c0d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:41000/0 (socket says 172.21.15.26:41000) 2024-04-24T23:21:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f538affd640 1 -- 172.21.15.26:0/200793984 learned_addr learned my addr 172.21.15.26:0/200793984 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.234+0000 7f538b7fe640 1 --1- 172.21.15.26:0/200793984 >> v1:172.21.15.29:6789/0 conn(0x7f538c0d0b10 0x7f538c0cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:56296/0 (socket says 172.21.15.26:56296) 2024-04-24T23:21:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.235+0000 7f538affd640 1 -- 172.21.15.26:0/200793984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 msgr2=0x7f538c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.235+0000 7f538affd640 1 --2- 172.21.15.26:0/200793984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 0x7f538c0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.235+0000 7f538affd640 1 -- 172.21.15.26:0/200793984 >> v1:172.21.15.29:6789/0 conn(0x7f538c0d0b10 legacy=0x7f538c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.235+0000 7f538affd640 1 -- 172.21.15.26:0/200793984 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c0cff40 con 0x7f538c0d4a30 2024-04-24T23:21:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.235+0000 7f538a7fc640 1 -- 172.21.15.26:0/200793984 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412084087 0 0) 0x7f537c001110 con 0x7f538c0d0b10 2024-04-24T23:21:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.235+0000 7f538affd640 1 --2- 172.21.15.26:0/200793984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f538c0d4a30 0x7f538c0d2ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5380009870 tx=0x7f538002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c06a540007ce97e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.235+0000 7f538a7fc640 1 -- 172.21.15.26:0/200793984 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380037070 con 0x7f538c0d4a30 2024-04-24T23:21:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f538a7fc640 1 -- 172.21.15.26:0/200793984 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380032440 con 0x7f538c0d4a30 2024-04-24T23:21:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f538a7fc640 1 -- 172.21.15.26:0/200793984 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380032810 con 0x7f538c0d4a30 2024-04-24T23:21:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f5391fee640 1 -- 172.21.15.26:0/200793984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f538c0d4a30 msgr2=0x7f538c0d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f5391fee640 1 --2- 172.21.15.26:0/200793984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f538c0d4a30 0x7f538c0d2ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5380009870 tx=0x7f538002a270 comp rx=0 tx=0).stop 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f5391fee640 1 -- 172.21.15.26:0/200793984 shutdown_connections 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f5391fee640 1 --2- 172.21.15.26:0/200793984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 0x7f538c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f5391fee640 1 --2- 172.21.15.26:0/200793984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f538c0d4a30 0x7f538c0d2ad0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f5391fee640 1 -- 172.21.15.26:0/200793984 >> 172.21.15.26:0/200793984 conn(0x7f538c09d500 msgr2=0x7f538c0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f5391fee640 1 -- 172.21.15.26:0/200793984 shutdown_connections 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.236+0000 7f5391fee640 1 -- 172.21.15.26:0/200793984 wait complete. 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.237+0000 7f5391fee640 1 Processor -- start 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.237+0000 7f5391fee640 1 -- start start 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.238+0000 7f5391fee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f538c0d0b10 0x7f538c160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.238+0000 7f5391fee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 0x7f538c0b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.238+0000 7f5391fee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f538c0d4a30 0x7f538c15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.238+0000 7f5391fee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f538c0d3010 con 0x7f538c0d4a30 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.238+0000 7f5391fee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f538c0d22d0 con 0x7f538c0d0b10 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.238+0000 7f5391fee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f538c0d3190 con 0x7f538c0d0ef0 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.238+0000 7f538bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f538c0d4a30 0x7f538c15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.238+0000 7f538bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f538c0d4a30 0x7f538c15cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57992/0 (socket says 172.21.15.26:57992) 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.238+0000 7f538bfff640 1 -- 172.21.15.26:0/423307119 learned_addr learned my addr 172.21.15.26:0/423307119 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f538affd640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 0x7f538c0b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f538b7fe640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f538c0d0b10 0x7f538c160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f538bfff640 1 -- 172.21.15.26:0/423307119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 msgr2=0x7f538c0b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f538bfff640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 0x7f538c0b6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f538bfff640 1 -- 172.21.15.26:0/423307119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f538c0d0b10 msgr2=0x7f538c160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f538bfff640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f538c0d0b10 0x7f538c160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f538bfff640 1 -- 172.21.15.26:0/423307119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c0b7280 con 0x7f538c0d4a30 2024-04-24T23:21:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f538bfff640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f538c0d4a30 0x7f538c15cda0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f538400efc0 tx=0x7f538400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f5373fff640 1 -- 172.21.15.26:0/423307119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384018020 con 0x7f538c0d4a30 2024-04-24T23:21:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f5373fff640 1 -- 172.21.15.26:0/423307119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5384014480 con 0x7f538c0d4a30 2024-04-24T23:21:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538c160bc0 con 0x7f538c0d4a30 2024-04-24T23:21:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f5373fff640 1 -- 172.21.15.26:0/423307119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384014870 con 0x7f538c0d4a30 2024-04-24T23:21:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.239+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538c161180 con 0x7f538c0d4a30 2024-04-24T23:21:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.240+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f538c0694c0 con 0x7f538c0d4a30 2024-04-24T23:21:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.241+0000 7f5373fff640 1 -- 172.21.15.26:0/423307119 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5384005d10 con 0x7f538c0d4a30 2024-04-24T23:21:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.241+0000 7f5373fff640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f537806be30 0x7f537806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.241+0000 7f5373fff640 1 -- 172.21.15.26:0/423307119 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f538408bc00 con 0x7f538c0d4a30 2024-04-24T23:21:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.241+0000 7f538b7fe640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f537806be30 0x7f537806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.241+0000 7f538b7fe640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f537806be30 0x7f537806e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f537c000f60 tx=0x7f537c007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.245+0000 7f5373fff640 1 -- 172.21.15.26:0/423307119 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5384092080 con 0x7f538c0d4a30 2024-04-24T23:21:24.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.428+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f538c074eb0 con 0x7f538c0d4a30 2024-04-24T23:21:24.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.429+0000 7f5373fff640 1 -- 172.21.15.26:0/423307119 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f5384022030 con 0x7f538c0d4a30 2024-04-24T23:21:24.431 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:24.431 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:24.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f537806be30 msgr2=0x7f537806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f537806be30 0x7f537806e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f537c000f60 tx=0x7f537c007c00 comp rx=0 tx=0).stop 2024-04-24T23:21:24.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f538c0d4a30 msgr2=0x7f538c15cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f538c0d4a30 0x7f538c15cda0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f538400efc0 tx=0x7f538400c490 comp rx=0 tx=0).stop 2024-04-24T23:21:24.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 shutdown_connections 2024-04-24T23:21:24.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f537806be30 0x7f537806e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f538c0d4a30 0x7f538c15cda0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f538c0d0ef0 0x7f538c0b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 --2- 172.21.15.26:0/423307119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f538c0d0b10 0x7f538c160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 >> 172.21.15.26:0/423307119 conn(0x7f538c09d500 msgr2=0x7f538c0cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 shutdown_connections 2024-04-24T23:21:24.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.431+0000 7f5391fee640 1 -- 172.21.15.26:0/423307119 wait complete. 2024-04-24T23:21:24.444 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:24.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.546+0000 7f02f7d30640 1 Processor -- start 2024-04-24T23:21:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.547+0000 7f02f7d30640 1 -- start start 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.547+0000 7f02f7d30640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00d2c30 0x7f02f00cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.547+0000 7f02f7d30640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02f00d2120 con 0x7f02f015e370 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.547+0000 7f02f7d30640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02f00b3740 con 0x7f02f015e750 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.547+0000 7f02f7d30640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f02f00d1510 con 0x7f02f00d2c30 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.547+0000 7f02f5acc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f02f015e370 0x7f02f00d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41994/0 (socket says 172.21.15.26:41994) 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.547+0000 7f02f5acc640 1 -- 172.21.15.26:0/2135884955 learned_addr learned my addr 172.21.15.26:0/2135884955 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228125525 0 0) 0x7f02f00d2120 con 0x7f02f015e370 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02d8003520 con 0x7f02f015e370 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f62cd640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00d2c30 0x7f02f00cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3029719294 0 0) 0x7f02d8003520 con 0x7f02f015e370 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00d2c30 msgr2=0x7f02f00cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00d2c30 0x7f02f00cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 >> v1:172.21.15.29:6789/0 conn(0x7f02f015e750 legacy=0x7f02f00cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f00b3a20 con 0x7f02f015e370 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f02e80019f0 con 0x7f02f015e370 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 >> v1:172.21.15.26:6789/0 conn(0x7f02f015e370 legacy=0x7f02f00d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02d80039e0 0x7f02d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d8004800 0x7f02d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d80051c0 0x7f02d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f02d800c090 con 0x7f02d80051c0 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02d800c210 con 0x7f02d8004800 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f02d800c410 con 0x7f02d80039e0 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f62cd640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00d2c30 0x7f02f00cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f62cd640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d80051c0 0x7f02d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f5acc640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02d80039e0 0x7f02d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.548+0000 7f02f52cb640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d8004800 0x7f02d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.549+0000 7f02f62cd640 1 -- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02d80039e0 msgr2=0x7f02d8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.549+0000 7f02f62cd640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02d80039e0 0x7f02d8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.549+0000 7f02f62cd640 1 -- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d8004800 msgr2=0x7f02d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.549+0000 7f02f62cd640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d8004800 0x7f02d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.549+0000 7f02f62cd640 1 -- 172.21.15.26:0/2135884955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d8003520 con 0x7f02d80051c0 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.549+0000 7f02f62cd640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d80051c0 0x7f02d800b9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f02e0006fd0 tx=0x7f02d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ba7d047fe2d05c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.549+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0037070 con 0x7f02d80051c0 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.549+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e002aae0 con 0x7f02d80051c0 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.549+0000 7f02f4aca640 1 -- 172.21.15.26:0/2135884955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e000f040 con 0x7f02d80051c0 2024-04-24T23:21:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.550+0000 7f02f7d30640 1 -- 172.21.15.26:0/2135884955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d80051c0 msgr2=0x7f02d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.550+0000 7f02f7d30640 1 --2- 172.21.15.26:0/2135884955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d80051c0 0x7f02d800b9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f02e0006fd0 tx=0x7f02d8005830 comp rx=0 tx=0).stop 2024-04-24T23:21:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.550+0000 7f02f62cd640 1 -- 172.21.15.26:0/2135884955 reap_dead start 2024-04-24T23:21:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.550+0000 7f02f7d30640 1 -- 172.21.15.26:0/2135884955 shutdown_connections 2024-04-24T23:21:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.550+0000 7f02f7d30640 1 -- 172.21.15.26:0/2135884955 >> 172.21.15.26:0/2135884955 conn(0x7f02f00d40f0 msgr2=0x7f02f00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.550+0000 7f02f7d30640 1 -- 172.21.15.26:0/2135884955 shutdown_connections 2024-04-24T23:21:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.550+0000 7f02f7d30640 1 -- 172.21.15.26:0/2135884955 wait complete. 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.551+0000 7f02f7d30640 1 Processor -- start 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.551+0000 7f02f7d30640 1 -- start start 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f7d30640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80051c0 0x7f02f014a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f7d30640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02f0156bc0 0x7f02f00c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f7d30640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00ca450 0x7f02f00ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f7d30640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00b5230 con 0x7f02f0156bc0 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f7d30640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00d1ed0 con 0x7f02d80051c0 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f7d30640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f02f00b53b0 con 0x7f02f00ca450 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f52cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02f0156bc0 0x7f02f00c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f52cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02f0156bc0 0x7f02f00c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58022/0 (socket says 172.21.15.26:58022) 2024-04-24T23:21:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f52cb640 1 -- 172.21.15.26:0/2325914633 learned_addr learned my addr 172.21.15.26:0/2325914633 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f62cd640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00ca450 0x7f02f00ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.552+0000 7f02f5acc640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80051c0 0x7f02f014a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02f52cb640 1 -- 172.21.15.26:0/2325914633 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00ca450 msgr2=0x7f02f00ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02f52cb640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00ca450 0x7f02f00ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02f52cb640 1 -- 172.21.15.26:0/2325914633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80051c0 msgr2=0x7f02f014a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02f52cb640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80051c0 0x7f02f014a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02f52cb640 1 -- 172.21.15.26:0/2325914633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f014ae40 con 0x7f02f0156bc0 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02f52cb640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02f0156bc0 0x7f02f00c84c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f02e400f850 tx=0x7f02e400fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02ee7fc640 1 -- 172.21.15.26:0/2325914633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e40050d0 con 0x7f02f0156bc0 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02ee7fc640 1 -- 172.21.15.26:0/2325914633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e4018070 con 0x7f02f0156bc0 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f014b120 con 0x7f02f0156bc0 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02ee7fc640 1 -- 172.21.15.26:0/2325914633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e400d730 con 0x7f02f0156bc0 2024-04-24T23:21:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.553+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f014aae0 con 0x7f02f0156bc0 2024-04-24T23:21:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.554+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f0075610 con 0x7f02f0156bc0 2024-04-24T23:21:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.554+0000 7f02ee7fc640 1 -- 172.21.15.26:0/2325914633 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02e400d8d0 con 0x7f02f0156bc0 2024-04-24T23:21:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.555+0000 7f02ee7fc640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02dc06be60 0x7f02dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.555+0000 7f02ee7fc640 1 -- 172.21.15.26:0/2325914633 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f02e408abd0 con 0x7f02f0156bc0 2024-04-24T23:21:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.555+0000 7f02f5acc640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02dc06be60 0x7f02dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.555+0000 7f02f5acc640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02dc06be60 0x7f02dc06e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f02e8004d00 tx=0x7f02e8006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.559+0000 7f02ee7fc640 1 -- 172.21.15.26:0/2325914633 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02e4059580 con 0x7f02f0156bc0 2024-04-24T23:21:24.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.743+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f02f006c7e0 con 0x7f02f0156bc0 2024-04-24T23:21:24.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.743+0000 7f02ee7fc640 1 -- 172.21.15.26:0/2325914633 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f02e4058e80 con 0x7f02f0156bc0 2024-04-24T23:21:24.745 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02dc06be60 msgr2=0x7f02dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02dc06be60 0x7f02dc06e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f02e8004d00 tx=0x7f02e8006040 comp rx=0 tx=0).stop 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02f0156bc0 msgr2=0x7f02f00c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02f0156bc0 0x7f02f00c84c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f02e400f850 tx=0x7f02e400fc50 comp rx=0 tx=0).stop 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 shutdown_connections 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02dc06be60 0x7f02dc06e320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02f00ca450 0x7f02f00ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02f0156bc0 0x7f02f00c84c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 --2- 172.21.15.26:0/2325914633 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02d80051c0 0x7f02f014a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.745+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 >> 172.21.15.26:0/2325914633 conn(0x7f02f00d40f0 msgr2=0x7f02f006cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.746+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 shutdown_connections 2024-04-24T23:21:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.746+0000 7f02f7d30640 1 -- 172.21.15.26:0/2325914633 wait complete. 2024-04-24T23:21:24.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:24.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.860+0000 7f884b79b640 1 Processor -- start 2024-04-24T23:21:24.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f884b79b640 1 -- start start 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f884b79b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f884415e370 0x7f88440d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f884b79b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88440d4c10 0x7f88440cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f884b79b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88440d2120 con 0x7f884415e750 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f884b79b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f88440d1510 con 0x7f884415e370 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f884b79b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f88440d1ed0 con 0x7f88440d4c10 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f8849537640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f884415e370 0x7f88440d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f8849537640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f884415e370 0x7f88440d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58034/0 (socket says 172.21.15.26:58034) 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f8849537640 1 -- 172.21.15.26:0/950826397 learned_addr learned my addr 172.21.15.26:0/950826397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.861+0000 7f8848d36640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f884415e750 0x7f88440b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41998/0 (socket says 172.21.15.26:41998) 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.862+0000 7f8849d38640 1 --2- 172.21.15.26:0/950826397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88440d4c10 0x7f88440cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.862+0000 7f8849537640 1 -- 172.21.15.26:0/950826397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88440d4c10 msgr2=0x7f88440cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.862+0000 7f8849537640 1 --2- 172.21.15.26:0/950826397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88440d4c10 0x7f88440cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.862+0000 7f8849537640 1 -- 172.21.15.26:0/950826397 >> v1:172.21.15.26:6789/0 conn(0x7f884415e750 legacy=0x7f88440b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.862+0000 7f8843fff640 1 -- 172.21.15.26:0/950826397 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2941983839 0 0) 0x7f88440d2120 con 0x7f884415e750 2024-04-24T23:21:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.862+0000 7f8849537640 1 -- 172.21.15.26:0/950826397 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88440b3740 con 0x7f884415e370 2024-04-24T23:21:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.862+0000 7f8849537640 1 --2- 172.21.15.26:0/950826397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 0x7f88440d2520 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8834009e50 tx=0x7f883402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0775cb6170f5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f8843fff640 1 -- 172.21.15.26:0/950826397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834036070 con 0x7f884415e370 2024-04-24T23:21:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f8843fff640 1 -- 172.21.15.26:0/950826397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8834031600 con 0x7f884415e370 2024-04-24T23:21:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f8843fff640 1 -- 172.21.15.26:0/950826397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88340319d0 con 0x7f884415e370 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f884b79b640 1 -- 172.21.15.26:0/950826397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 msgr2=0x7f88440d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f884b79b640 1 --2- 172.21.15.26:0/950826397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 0x7f88440d2520 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8834009e50 tx=0x7f883402a710 comp rx=0 tx=0).stop 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f884b79b640 1 -- 172.21.15.26:0/950826397 shutdown_connections 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f884b79b640 1 --2- 172.21.15.26:0/950826397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88440d4c10 0x7f88440cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f884b79b640 1 --2- 172.21.15.26:0/950826397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 0x7f88440d2520 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f884b79b640 1 -- 172.21.15.26:0/950826397 >> 172.21.15.26:0/950826397 conn(0x7f88440d40f0 msgr2=0x7f88440d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f884b79b640 1 -- 172.21.15.26:0/950826397 shutdown_connections 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.863+0000 7f884b79b640 1 -- 172.21.15.26:0/950826397 wait complete. 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.864+0000 7f884b79b640 1 Processor -- start 2024-04-24T23:21:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f884b79b640 1 -- start start 2024-04-24T23:21:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f884b79b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88440d4c10 0x7f88441f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f884b79b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 0x7f88440b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f884b79b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88441f7e10 0x7f88441f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f884b79b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f88440d2a60 con 0x7f884415e370 2024-04-24T23:21:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f884b79b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f88440b3d00 con 0x7f88440d4c10 2024-04-24T23:21:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f884b79b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f88440d2be0 con 0x7f88441f7e10 2024-04-24T23:21:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f8848d36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 0x7f88440b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f8848d36640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 0x7f88440b6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58038/0 (socket says 172.21.15.26:58038) 2024-04-24T23:21:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f8848d36640 1 -- 172.21.15.26:0/2794138770 learned_addr learned my addr 172.21.15.26:0/2794138770 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f8849537640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88440d4c10 0x7f88441f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.865+0000 7f8849d38640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88441f7e10 0x7f88441f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8848d36640 1 -- 172.21.15.26:0/2794138770 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88441f7e10 msgr2=0x7f88441f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8848d36640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88441f7e10 0x7f88441f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8848d36640 1 -- 172.21.15.26:0/2794138770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88440d4c10 msgr2=0x7f88441f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8848d36640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88440d4c10 0x7f88441f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8848d36640 1 -- 172.21.15.26:0/2794138770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88440b1700 con 0x7f884415e370 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8849d38640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88441f7e10 0x7f88441f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8848d36640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 0x7f88440b6ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f883800c410 tx=0x7f883800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8841ffb640 1 -- 172.21.15.26:0/2794138770 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88380074e0 con 0x7f884415e370 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8841ffb640 1 -- 172.21.15.26:0/2794138770 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88380047e0 con 0x7f884415e370 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88440b19e0 con 0x7f884415e370 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f8841ffb640 1 -- 172.21.15.26:0/2794138770 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838004bd0 con 0x7f884415e370 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.866+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88440b1fa0 con 0x7f884415e370 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.867+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8844075610 con 0x7f884415e370 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.868+0000 7f8841ffb640 1 -- 172.21.15.26:0/2794138770 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f883801a020 con 0x7f884415e370 2024-04-24T23:21:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.868+0000 7f8841ffb640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f883006be60 0x7f883006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.868+0000 7f8841ffb640 1 -- 172.21.15.26:0/2794138770 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f883808a550 con 0x7f884415e370 2024-04-24T23:21:24.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.868+0000 7f8849537640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f883006be60 0x7f883006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.868+0000 7f8849537640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f883006be60 0x7f883006e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8834033000 tx=0x7f8834009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:24.872+0000 7f8841ffb640 1 -- 172.21.15.26:0/2794138770 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8838058f00 con 0x7f884415e370 2024-04-24T23:21:25.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.053+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88440b4650 con 0x7f884415e370 2024-04-24T23:21:25.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.053+0000 7f8841ffb640 1 -- 172.21.15.26:0/2794138770 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f8838058800 con 0x7f884415e370 2024-04-24T23:21:25.055 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f883006be60 msgr2=0x7f883006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f883006be60 0x7f883006e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8834033000 tx=0x7f8834009000 comp rx=0 tx=0).stop 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 msgr2=0x7f88440b6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 0x7f88440b6ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f883800c410 tx=0x7f883800c8d0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 shutdown_connections 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f883006be60 0x7f883006e320 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f88441f7e10 0x7f88441f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f884415e370 0x7f88440b6ce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 --2- 172.21.15.26:0/2794138770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f88440d4c10 0x7f88441f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 >> 172.21.15.26:0/2794138770 conn(0x7f88440d40f0 msgr2=0x7f884415c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 shutdown_connections 2024-04-24T23:21:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.056+0000 7f884b79b640 1 -- 172.21.15.26:0/2794138770 wait complete. 2024-04-24T23:21:25.068 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.170+0000 7f9445fdb640 1 Processor -- start 2024-04-24T23:21:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.170+0000 7f9445fdb640 1 -- start start 2024-04-24T23:21:25.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.171+0000 7f9445fdb640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9440158ae0 0x7f944006e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.171+0000 7f9445fdb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94400d0ce0 0x7f944015e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.171+0000 7f9445fdb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f944006b1c0 con 0x7f944015ee90 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.171+0000 7f9445fdb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f944006ec90 con 0x7f94400d0ce0 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.171+0000 7f9445fdb640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f944006ef70 con 0x7f9440158ae0 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.171+0000 7f943effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94400d0ce0 0x7f944015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.171+0000 7f943ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f944015ee90 0x7f9440071720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42002/0 (socket says 172.21.15.26:42002) 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.171+0000 7f943ffff640 1 -- 172.21.15.26:0/1289559300 learned_addr learned my addr 172.21.15.26:0/1289559300 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.172+0000 7f943f7fe640 1 --2- 172.21.15.26:0/1289559300 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9440158ae0 0x7f944006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.172+0000 7f943effd640 1 -- 172.21.15.26:0/1289559300 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9440158ae0 msgr2=0x7f944006e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.172+0000 7f943effd640 1 --2- 172.21.15.26:0/1289559300 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9440158ae0 0x7f944006e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.172+0000 7f943effd640 1 -- 172.21.15.26:0/1289559300 >> v1:172.21.15.26:6789/0 conn(0x7f944015ee90 legacy=0x7f9440071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.172+0000 7f943e7fc640 1 -- 172.21.15.26:0/1289559300 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 355311155 0 0) 0x7f944006b1c0 con 0x7f944015ee90 2024-04-24T23:21:25.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.172+0000 7f943effd640 1 -- 172.21.15.26:0/1289559300 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944006f250 con 0x7f94400d0ce0 2024-04-24T23:21:25.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.172+0000 7f943f7fe640 1 --2- 172.21.15.26:0/1289559300 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9440158ae0 0x7f944006e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:25.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.172+0000 7f943effd640 1 --2- 172.21.15.26:0/1289559300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400d0ce0 0x7f944015e950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f94340097a0 tx=0x7f943402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6668927f1d330700 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:25.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f943e7fc640 1 -- 172.21.15.26:0/1289559300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434037070 con 0x7f94400d0ce0 2024-04-24T23:21:25.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f943e7fc640 1 -- 172.21.15.26:0/1289559300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f943402ae50 con 0x7f94400d0ce0 2024-04-24T23:21:25.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f943e7fc640 1 -- 172.21.15.26:0/1289559300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434032660 con 0x7f94400d0ce0 2024-04-24T23:21:25.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f9445fdb640 1 -- 172.21.15.26:0/1289559300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400d0ce0 msgr2=0x7f944015e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f9445fdb640 1 --2- 172.21.15.26:0/1289559300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400d0ce0 0x7f944015e950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f94340097a0 tx=0x7f943402a100 comp rx=0 tx=0).stop 2024-04-24T23:21:25.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f9445fdb640 1 -- 172.21.15.26:0/1289559300 shutdown_connections 2024-04-24T23:21:25.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f9445fdb640 1 --2- 172.21.15.26:0/1289559300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94400d0ce0 0x7f944015e950 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f9445fdb640 1 --2- 172.21.15.26:0/1289559300 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9440158ae0 0x7f944006e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f9445fdb640 1 -- 172.21.15.26:0/1289559300 >> 172.21.15.26:0/1289559300 conn(0x7f94400b2970 msgr2=0x7f94400d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:25.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f9445fdb640 1 -- 172.21.15.26:0/1289559300 shutdown_connections 2024-04-24T23:21:25.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.173+0000 7f9445fdb640 1 -- 172.21.15.26:0/1289559300 wait complete. 2024-04-24T23:21:25.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.174+0000 7f9445fdb640 1 Processor -- start 2024-04-24T23:21:25.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f9445fdb640 1 -- start start 2024-04-24T23:21:25.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f9445fdb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94400d0ce0 0x7f944015e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f9445fdb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9440158ae0 0x7f944014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f9445fdb640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f944015ee90 0x7f94400b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f9445fdb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94400b00e0 con 0x7f9440158ae0 2024-04-24T23:21:25.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f9445fdb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94400aff60 con 0x7f94400d0ce0 2024-04-24T23:21:25.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f9445fdb640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9440071e30 con 0x7f944015ee90 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f943effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9440158ae0 0x7f944014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f943effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9440158ae0 0x7f944014cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58066/0 (socket says 172.21.15.26:58066) 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f943effd640 1 -- 172.21.15.26:0/2360697743 learned_addr learned my addr 172.21.15.26:0/2360697743 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.175+0000 7f943f7fe640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94400d0ce0 0x7f944015e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f943ffff640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f944015ee90 0x7f94400b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f943effd640 1 -- 172.21.15.26:0/2360697743 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f944015ee90 msgr2=0x7f94400b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f943effd640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f944015ee90 0x7f94400b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f943effd640 1 -- 172.21.15.26:0/2360697743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94400d0ce0 msgr2=0x7f944015e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f943effd640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94400d0ce0 0x7f944015e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f943effd640 1 -- 172.21.15.26:0/2360697743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944015dcc0 con 0x7f9440158ae0 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f943effd640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9440158ae0 0x7f944014cc50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f94340098c0 tx=0x7f94340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f9427fff640 1 -- 172.21.15.26:0/2360697743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434041020 con 0x7f9440158ae0 2024-04-24T23:21:25.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f9427fff640 1 -- 172.21.15.26:0/2360697743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f943403c670 con 0x7f9440158ae0 2024-04-24T23:21:25.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944015d9e0 con 0x7f9440158ae0 2024-04-24T23:21:25.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f9427fff640 1 -- 172.21.15.26:0/2360697743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434037040 con 0x7f9440158ae0 2024-04-24T23:21:25.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.176+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94400adbf0 con 0x7f9440158ae0 2024-04-24T23:21:25.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.177+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9440075610 con 0x7f9440158ae0 2024-04-24T23:21:25.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.178+0000 7f9427fff640 1 -- 172.21.15.26:0/2360697743 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9434044020 con 0x7f9440158ae0 2024-04-24T23:21:25.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.178+0000 7f9427fff640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f942c06be30 0x7f942c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.178+0000 7f943f7fe640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f942c06be30 0x7f942c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.178+0000 7f9427fff640 1 -- 172.21.15.26:0/2360697743 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94340ab410 con 0x7f9440158ae0 2024-04-24T23:21:25.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.179+0000 7f943f7fe640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f942c06be30 0x7f942c06e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f944015b850 tx=0x7f9430008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:25.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.183+0000 7f9427fff640 1 -- 172.21.15.26:0/2360697743 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f943404b030 con 0x7f9440158ae0 2024-04-24T23:21:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.364+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9440067580 con 0x7f9440158ae0 2024-04-24T23:21:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.365+0000 7f9427fff640 1 -- 172.21.15.26:0/2360697743 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f9434079770 con 0x7f9440158ae0 2024-04-24T23:21:25.367 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f942c06be30 msgr2=0x7f942c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f942c06be30 0x7f942c06e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f944015b850 tx=0x7f9430008040 comp rx=0 tx=0).stop 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9440158ae0 msgr2=0x7f944014cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9440158ae0 0x7f944014cc50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f94340098c0 tx=0x7f94340046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 shutdown_connections 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f942c06be30 0x7f942c06e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f944015ee90 0x7f94400b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9440158ae0 0x7f944014cc50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 --2- 172.21.15.26:0/2360697743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94400d0ce0 0x7f944015e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 >> 172.21.15.26:0/2360697743 conn(0x7f94400b2970 msgr2=0x7f9440071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 shutdown_connections 2024-04-24T23:21:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.367+0000 7f9445fdb640 1 -- 172.21.15.26:0/2360697743 wait complete. 2024-04-24T23:21:25.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-04-24T23:21:25.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.479+0000 7fd9162ae640 1 Processor -- start 2024-04-24T23:21:25.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.479+0000 7fd9162ae640 1 -- start start 2024-04-24T23:21:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd9162ae640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd910156760 0x7fd91015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd9162ae640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9100cf330 0x7fd9100b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd9162ae640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd91015d660 con 0x7fd9100b3ac0 2024-04-24T23:21:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd9162ae640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd910156b40 con 0x7fd9100cf330 2024-04-24T23:21:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd9162ae640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9100cfed0 con 0x7fd910156760 2024-04-24T23:21:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd910156760 0x7fd91015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd91484b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9100cf330 0x7fd9100b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd910156760 0x7fd91015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:60572/0 (socket says 172.21.15.26:60572) 2024-04-24T23:21:25.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd90ffff640 1 -- 172.21.15.26:0/3774755913 learned_addr learned my addr 172.21.15.26:0/3774755913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd90f7fe640 1 --1- 172.21.15.26:0/3774755913 >> v1:172.21.15.29:6789/0 conn(0x7fd9100b3ac0 0x7fd9100d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:56314/0 (socket says 172.21.15.26:56314) 2024-04-24T23:21:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd90ffff640 1 -- 172.21.15.26:0/3774755913 >> v2:172.21.15.29:3300/0 conn(0x7fd9100cf330 msgr2=0x7fd9100b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd90ffff640 1 --2- 172.21.15.26:0/3774755913 >> v2:172.21.15.29:3300/0 conn(0x7fd9100cf330 0x7fd9100b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd90ffff640 1 -- 172.21.15.26:0/3774755913 >> v1:172.21.15.29:6789/0 conn(0x7fd9100b3ac0 legacy=0x7fd9100d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.480+0000 7fd90ffff640 1 -- 172.21.15.26:0/3774755913 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9100d01b0 con 0x7fd910156760 2024-04-24T23:21:25.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.481+0000 7fd90ffff640 1 --2- 172.21.15.26:0/3774755913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd910156760 0x7fd91015d120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd900009870 tx=0x7fd90002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf6c476d5ddd1368 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:25.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.481+0000 7fd90effd640 1 -- 172.21.15.26:0/3774755913 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900037070 con 0x7fd910156760 2024-04-24T23:21:25.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.481+0000 7fd90effd640 1 -- 172.21.15.26:0/3774755913 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd900032440 con 0x7fd910156760 2024-04-24T23:21:25.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.481+0000 7fd90effd640 1 -- 172.21.15.26:0/3774755913 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900032810 con 0x7fd910156760 2024-04-24T23:21:25.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.482+0000 7fd9162ae640 1 -- 172.21.15.26:0/3774755913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd910156760 msgr2=0x7fd91015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.482+0000 7fd9162ae640 1 --2- 172.21.15.26:0/3774755913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd910156760 0x7fd91015d120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd900009870 tx=0x7fd90002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.482+0000 7fd9162ae640 1 -- 172.21.15.26:0/3774755913 shutdown_connections 2024-04-24T23:21:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.482+0000 7fd9162ae640 1 --2- 172.21.15.26:0/3774755913 >> v2:172.21.15.29:3300/0 conn(0x7fd9100cf330 0x7fd9100b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.482+0000 7fd9162ae640 1 --2- 172.21.15.26:0/3774755913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd910156760 0x7fd91015d120 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.482+0000 7fd9162ae640 1 -- 172.21.15.26:0/3774755913 >> 172.21.15.26:0/3774755913 conn(0x7fd91006b120 msgr2=0x7fd91006b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.482+0000 7fd9162ae640 1 -- 172.21.15.26:0/3774755913 shutdown_connections 2024-04-24T23:21:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.482+0000 7fd9162ae640 1 -- 172.21.15.26:0/3774755913 wait complete. 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.483+0000 7fd9162ae640 1 Processor -- start 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.483+0000 7fd9162ae640 1 -- start start 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd9162ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b3ac0 0x7fd9101f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd9162ae640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9100cf330 0x7fd910152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd9162ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd910156760 0x7fd9100c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd9162ae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9100d0a50 con 0x7fd9100b3ac0 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd9162ae640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9100d0770 con 0x7fd910156760 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd9162ae640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9100cc7e0 con 0x7fd9100cf330 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b3ac0 0x7fd9101f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b3ac0 0x7fd9101f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58072/0 (socket says 172.21.15.26:58072) 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd90ffff640 1 -- 172.21.15.26:0/3828336167 learned_addr learned my addr 172.21.15.26:0/3828336167 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd90ffff640 1 -- 172.21.15.26:0/3828336167 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9100cf330 msgr2=0x7fd910152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd90ffff640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9100cf330 0x7fd910152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd90ffff640 1 -- 172.21.15.26:0/3828336167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd910156760 msgr2=0x7fd9100c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.484+0000 7fd90ffff640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd910156760 0x7fd9100c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.485+0000 7fd90ffff640 1 -- 172.21.15.26:0/3828336167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9101f28f0 con 0x7fd9100b3ac0 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.485+0000 7fd90ffff640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b3ac0 0x7fd9101f23b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd900000c00 tx=0x7fd9000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.485+0000 7fd90cff9640 1 -- 172.21.15.26:0/3828336167 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900037040 con 0x7fd9100b3ac0 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.485+0000 7fd90cff9640 1 -- 172.21.15.26:0/3828336167 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd900042070 con 0x7fd9100b3ac0 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.485+0000 7fd90cff9640 1 -- 172.21.15.26:0/3828336167 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90003b870 con 0x7fd9100b3ac0 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.485+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd910158130 con 0x7fd9100b3ac0 2024-04-24T23:21:25.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.485+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9101586f0 con 0x7fd9100b3ac0 2024-04-24T23:21:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.486+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9100694c0 con 0x7fd9100b3ac0 2024-04-24T23:21:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.486+0000 7fd90cff9640 1 -- 172.21.15.26:0/3828336167 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd90002ad10 con 0x7fd9100b3ac0 2024-04-24T23:21:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.487+0000 7fd90cff9640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd8fc06be30 0x7fd8fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.487+0000 7fd90cff9640 1 -- 172.21.15.26:0/3828336167 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd9000ab390 con 0x7fd9100b3ac0 2024-04-24T23:21:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.487+0000 7fd90f7fe640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd8fc06be30 0x7fd8fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.487+0000 7fd90f7fe640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd8fc06be30 0x7fd8fc06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd904000f60 tx=0x7fd904007a00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.491+0000 7fd90cff9640 1 -- 172.21.15.26:0/3828336167 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd9101f28f0 con 0x7fd9100b3ac0 2024-04-24T23:21:25.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:25.754+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) -- 0x7fd91006e440 con 0x7fd9100b3ac0 2024-04-24T23:21:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.545+0000 7fd90cff9640 1 -- 172.21.15.26:0/3828336167 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v51) ==== 139+0+0 (secure 0 0 0) 0x7fd900079d40 con 0x7fd9100b3ac0 2024-04-24T23:21:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd8fc06be30 msgr2=0x7fd8fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd8fc06be30 0x7fd8fc06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd904000f60 tx=0x7fd904007a00 comp rx=0 tx=0).stop 2024-04-24T23:21:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b3ac0 msgr2=0x7fd9101f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b3ac0 0x7fd9101f23b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd900000c00 tx=0x7fd9000046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 shutdown_connections 2024-04-24T23:21:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd8fc06be30 0x7fd8fc06e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd910156760 0x7fd9100c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9100cf330 0x7fd910152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 --2- 172.21.15.26:0/3828336167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9100b3ac0 0x7fd9101f23b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 >> 172.21.15.26:0/3828336167 conn(0x7fd91006b120 msgr2=0x7fd9100b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 shutdown_connections 2024-04-24T23:21:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.548+0000 7fd9162ae640 1 -- 172.21.15.26:0/3828336167 wait complete. 2024-04-24T23:21:26.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:21:26.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.677+0000 7f5facbc8640 1 Processor -- start 2024-04-24T23:21:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.677+0000 7f5facbc8640 1 -- start start 2024-04-24T23:21:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5facbc8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5fa80d28e0 0x7f5fa80d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5facbc8640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa815da30 0x7f5fa815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5facbc8640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fa806d750 con 0x7f5fa80ce640 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5facbc8640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa80d1110 con 0x7f5fa80d28e0 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5facbc8640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa80cc410 con 0x7f5fa815da30 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5fa5d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5fa80d28e0 0x7f5fa80d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5fa5d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5fa80d28e0 0x7f5fa80d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58078/0 (socket says 172.21.15.26:58078) 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5fa5d9b640 1 -- 172.21.15.26:0/4163850575 learned_addr learned my addr 172.21.15.26:0/4163850575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5fa6d9d640 1 --2- 172.21.15.26:0/4163850575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa815da30 0x7f5fa815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5fa5d9b640 1 -- 172.21.15.26:0/4163850575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa815da30 msgr2=0x7f5fa815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5fa5d9b640 1 --2- 172.21.15.26:0/4163850575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa815da30 0x7f5fa815deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5fa5d9b640 1 -- 172.21.15.26:0/4163850575 >> v1:172.21.15.29:6789/0 conn(0x7f5fa80ce640 legacy=0x7f5fa80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.678+0000 7f5fa5d9b640 1 -- 172.21.15.26:0/4163850575 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa80b1720 con 0x7f5fa80d28e0 2024-04-24T23:21:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.679+0000 7f5fa5d9b640 1 --2- 172.21.15.26:0/4163850575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80d28e0 0x7f5fa80d00f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009870 tx=0x7f5f9c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6ebc8cf8810253 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.679+0000 7f5fa559a640 1 -- 172.21.15.26:0/4163850575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c037070 con 0x7f5fa80d28e0 2024-04-24T23:21:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.679+0000 7f5fa559a640 1 -- 172.21.15.26:0/4163850575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9c032440 con 0x7f5fa80d28e0 2024-04-24T23:21:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.679+0000 7f5fa559a640 1 -- 172.21.15.26:0/4163850575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c032810 con 0x7f5fa80d28e0 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.679+0000 7f5facbc8640 1 -- 172.21.15.26:0/4163850575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80d28e0 msgr2=0x7f5fa80d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.679+0000 7f5facbc8640 1 --2- 172.21.15.26:0/4163850575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80d28e0 0x7f5fa80d00f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009870 tx=0x7f5f9c02a2a0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.680+0000 7f5facbc8640 1 -- 172.21.15.26:0/4163850575 shutdown_connections 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.680+0000 7f5facbc8640 1 --2- 172.21.15.26:0/4163850575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa815da30 0x7f5fa815deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.680+0000 7f5facbc8640 1 --2- 172.21.15.26:0/4163850575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80d28e0 0x7f5fa80d00f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.680+0000 7f5facbc8640 1 -- 172.21.15.26:0/4163850575 >> 172.21.15.26:0/4163850575 conn(0x7f5fa80d1dc0 msgr2=0x7f5fa80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.680+0000 7f5facbc8640 1 -- 172.21.15.26:0/4163850575 shutdown_connections 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.680+0000 7f5facbc8640 1 -- 172.21.15.26:0/4163850575 wait complete. 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.681+0000 7f5facbc8640 1 Processor -- start 2024-04-24T23:21:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.681+0000 7f5facbc8640 1 -- start start 2024-04-24T23:21:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5facbc8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80ce640 0x7f5fa815c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5facbc8640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa80d28e0 0x7f5fa809dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5facbc8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fa815da30 0x7f5fa8150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5facbc8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa80cc940 con 0x7f5fa80ce640 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5facbc8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa80cc660 con 0x7f5fa815da30 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5facbc8640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa80ccc20 con 0x7f5fa80d28e0 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80ce640 0x7f5fa815c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80ce640 0x7f5fa815c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58082/0 (socket says 172.21.15.26:58082) 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa659c640 1 -- 172.21.15.26:0/2384676794 learned_addr learned my addr 172.21.15.26:0/2384676794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa5d9b640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa80d28e0 0x7f5fa809dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa659c640 1 -- 172.21.15.26:0/2384676794 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa80d28e0 msgr2=0x7f5fa809dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa659c640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa80d28e0 0x7f5fa809dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa6d9d640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fa815da30 0x7f5fa8150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa659c640 1 -- 172.21.15.26:0/2384676794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fa815da30 msgr2=0x7f5fa8150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa659c640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fa815da30 0x7f5fa8150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa659c640 1 -- 172.21.15.26:0/2384676794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa8150d40 con 0x7f5fa80ce640 2024-04-24T23:21:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.682+0000 7f5fa659c640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80ce640 0x7f5fa815c610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800c7f0 tx=0x7f5f9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.683+0000 7f5f96ffd640 1 -- 172.21.15.26:0/2384676794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98016020 con 0x7f5fa80ce640 2024-04-24T23:21:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.683+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa809e360 con 0x7f5fa80ce640 2024-04-24T23:21:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.683+0000 7f5f96ffd640 1 -- 172.21.15.26:0/2384676794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9800aa50 con 0x7f5fa80ce640 2024-04-24T23:21:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.683+0000 7f5f96ffd640 1 -- 172.21.15.26:0/2384676794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9800ae40 con 0x7f5fa80ce640 2024-04-24T23:21:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.683+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa809e920 con 0x7f5fa80ce640 2024-04-24T23:21:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.684+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa8075610 con 0x7f5fa80ce640 2024-04-24T23:21:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.684+0000 7f5f96ffd640 1 -- 172.21.15.26:0/2384676794 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f98019020 con 0x7f5fa80ce640 2024-04-24T23:21:26.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.685+0000 7f5f96ffd640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5f8806be30 0x7f5f8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:26.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.685+0000 7f5fa5d9b640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5f8806be30 0x7f5f8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:26.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.685+0000 7f5f96ffd640 1 -- 172.21.15.26:0/2384676794 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5f9808aba0 con 0x7f5fa80ce640 2024-04-24T23:21:26.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.685+0000 7f5fa5d9b640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5f8806be30 0x7f5f8806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c034000 tx=0x7f5f9c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.689+0000 7f5f96ffd640 1 -- 172.21.15.26:0/2384676794 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5f98020070 con 0x7f5fa80ce640 2024-04-24T23:21:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.871+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5fa80d0ed0 con 0x7f5fa80ce640 2024-04-24T23:21:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.874+0000 7f5f96ffd640 1 -- 172.21.15.26:0/2384676794 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f5f98059560 con 0x7f5fa80ce640 2024-04-24T23:21:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5f8806be30 msgr2=0x7f5f8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5f8806be30 0x7f5f8806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c034000 tx=0x7f5f9c02e000 comp rx=0 tx=0).stop 2024-04-24T23:21:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80ce640 msgr2=0x7f5fa815c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80ce640 0x7f5fa815c610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800c7f0 tx=0x7f5f9800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 shutdown_connections 2024-04-24T23:21:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5f8806be30 0x7f5f8806e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fa815da30 0x7f5fa8150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5fa80d28e0 0x7f5fa809dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 --2- 172.21.15.26:0/2384676794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa80ce640 0x7f5fa815c610 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 >> 172.21.15.26:0/2384676794 conn(0x7f5fa80d1dc0 msgr2=0x7f5fa80cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 shutdown_connections 2024-04-24T23:21:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:26.876+0000 7f5facbc8640 1 -- 172.21.15.26:0/2384676794 wait complete. 2024-04-24T23:21:26.889 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:21:26.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:21:26.925 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:21:26.966 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:21:26.966 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:21:26.966 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:26.981 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:21:26.982 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:27.037 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:21:27.037 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:27.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:27.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.231+0000 7fa329c64640 1 Processor -- start 2024-04-24T23:21:27.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.231+0000 7fa329c64640 1 -- start start 2024-04-24T23:21:27.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa329c64640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa32406b830 0x7fa324073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa329c64640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa32415d7e0 0x7fa3240cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa329c64640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa32415c7c0 con 0x7fa324145980 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa329c64640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa324145d60 con 0x7fa32415d7e0 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa329c64640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa32415caa0 con 0x7fa32406b830 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa323fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa32415d7e0 0x7fa3240cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa323fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa32415d7e0 0x7fa3240cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58098/0 (socket says 172.21.15.26:58098) 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa323fff640 1 -- 172.21.15.26:0/2119834105 learned_addr learned my addr 172.21.15.26:0/2119834105 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa322ffd640 1 --2- 172.21.15.26:0/2119834105 >> v2:172.21.15.29:3300/0 conn(0x7fa32406b830 0x7fa324073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa323fff640 1 -- 172.21.15.26:0/2119834105 >> v2:172.21.15.29:3300/0 conn(0x7fa32406b830 msgr2=0x7fa324073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.232+0000 7fa323fff640 1 --2- 172.21.15.26:0/2119834105 >> v2:172.21.15.29:3300/0 conn(0x7fa32406b830 0x7fa324073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.233+0000 7fa323fff640 1 -- 172.21.15.26:0/2119834105 >> v1:172.21.15.29:6789/0 conn(0x7fa324145980 legacy=0x7fa32406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.233+0000 7fa323fff640 1 -- 172.21.15.26:0/2119834105 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32415cd80 con 0x7fa32415d7e0 2024-04-24T23:21:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.233+0000 7fa323fff640 1 --2- 172.21.15.26:0/2119834105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa32415d7e0 0x7fa3240cd950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa32406d180 tx=0x7fa31c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e923830f1058509d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.233+0000 7fa3227fc640 1 -- 172.21.15.26:0/2119834105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c036070 con 0x7fa32415d7e0 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.233+0000 7fa3227fc640 1 -- 172.21.15.26:0/2119834105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c031600 con 0x7fa32415d7e0 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.233+0000 7fa3227fc640 1 -- 172.21.15.26:0/2119834105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c0319d0 con 0x7fa32415d7e0 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.234+0000 7fa329c64640 1 -- 172.21.15.26:0/2119834105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa32415d7e0 msgr2=0x7fa3240cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.234+0000 7fa329c64640 1 --2- 172.21.15.26:0/2119834105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa32415d7e0 0x7fa3240cd950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa32406d180 tx=0x7fa31c02a6f0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.234+0000 7fa329c64640 1 -- 172.21.15.26:0/2119834105 shutdown_connections 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.234+0000 7fa329c64640 1 --2- 172.21.15.26:0/2119834105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa32415d7e0 0x7fa3240cd950 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.234+0000 7fa329c64640 1 --2- 172.21.15.26:0/2119834105 >> v2:172.21.15.29:3300/0 conn(0x7fa32406b830 0x7fa324073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.234+0000 7fa329c64640 1 -- 172.21.15.26:0/2119834105 >> 172.21.15.26:0/2119834105 conn(0x7fa32406e2c0 msgr2=0x7fa32406e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.234+0000 7fa329c64640 1 -- 172.21.15.26:0/2119834105 shutdown_connections 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.234+0000 7fa329c64640 1 -- 172.21.15.26:0/2119834105 wait complete. 2024-04-24T23:21:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.235+0000 7fa329c64640 1 Processor -- start 2024-04-24T23:21:27.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.235+0000 7fa329c64640 1 -- start start 2024-04-24T23:21:27.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa329c64640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa32406b830 0x7fa3240b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa329c64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa324145980 0x7fa3240c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa329c64640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa32415d7e0 0x7fa324140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa329c64640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa324072320 con 0x7fa324145980 2024-04-24T23:21:27.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa329c64640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa3240721a0 con 0x7fa32415d7e0 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa329c64640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa3240724a0 con 0x7fa32406b830 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa324145980 0x7fa3240c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa324145980 0x7fa3240c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58102/0 (socket says 172.21.15.26:58102) 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa322ffd640 1 -- 172.21.15.26:0/2856304414 learned_addr learned my addr 172.21.15.26:0/2856304414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa323fff640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa32415d7e0 0x7fa324140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa3237fe640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa32406b830 0x7fa3240b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa322ffd640 1 -- 172.21.15.26:0/2856304414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa32406b830 msgr2=0x7fa3240b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa322ffd640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa32406b830 0x7fa3240b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa322ffd640 1 -- 172.21.15.26:0/2856304414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa32415d7e0 msgr2=0x7fa324140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.236+0000 7fa322ffd640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa32415d7e0 0x7fa324140540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.237+0000 7fa322ffd640 1 -- 172.21.15.26:0/2856304414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3241f1e80 con 0x7fa324145980 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.237+0000 7fa323fff640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa32415d7e0 0x7fa324140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.237+0000 7fa322ffd640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa324145980 0x7fa3240c9920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d800 tx=0x7fa31800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.237+0000 7fa30bfff640 1 -- 172.21.15.26:0/2856304414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31800b6c0 con 0x7fa324145980 2024-04-24T23:21:27.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.237+0000 7fa30bfff640 1 -- 172.21.15.26:0/2856304414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa318019070 con 0x7fa324145980 2024-04-24T23:21:27.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.237+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3241f2160 con 0x7fa324145980 2024-04-24T23:21:27.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.237+0000 7fa30bfff640 1 -- 172.21.15.26:0/2856304414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31800ba80 con 0x7fa324145980 2024-04-24T23:21:27.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.237+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241f2720 con 0x7fa324145980 2024-04-24T23:21:27.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.238+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3241f1900 con 0x7fa324145980 2024-04-24T23:21:27.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.238+0000 7fa30bfff640 1 -- 172.21.15.26:0/2856304414 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa31800bc20 con 0x7fa324145980 2024-04-24T23:21:27.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.239+0000 7fa30bfff640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa31006be30 0x7fa31006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.239+0000 7fa3237fe640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa31006be30 0x7fa31006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.239+0000 7fa30bfff640 1 -- 172.21.15.26:0/2856304414 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa31808b330 con 0x7fa324145980 2024-04-24T23:21:27.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.239+0000 7fa3237fe640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa31006be30 0x7fa31006e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa314004ea0 tx=0x7fa314009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:27.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.243+0000 7fa30bfff640 1 -- 172.21.15.26:0/2856304414 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa318059ce0 con 0x7fa324145980 2024-04-24T23:21:27.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.427+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3240694c0 con 0x7fa324145980 2024-04-24T23:21:27.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.428+0000 7fa30bfff640 1 -- 172.21.15.26:0/2856304414 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fa3180595e0 con 0x7fa324145980 2024-04-24T23:21:27.430 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:27.430 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa31006be30 msgr2=0x7fa31006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa31006be30 0x7fa31006e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa314004ea0 tx=0x7fa314009040 comp rx=0 tx=0).stop 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa324145980 msgr2=0x7fa3240c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa324145980 0x7fa3240c9920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d800 tx=0x7fa31800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 shutdown_connections 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa31006be30 0x7fa31006e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa32415d7e0 0x7fa324140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa324145980 0x7fa3240c9920 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 --2- 172.21.15.26:0/2856304414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa32406b830 0x7fa3240b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 >> 172.21.15.26:0/2856304414 conn(0x7fa32406e2c0 msgr2=0x7fa3240cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 shutdown_connections 2024-04-24T23:21:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.430+0000 7fa329c64640 1 -- 172.21.15.26:0/2856304414 wait complete. 2024-04-24T23:21:27.443 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:27.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.548+0000 7f1979d53640 1 Processor -- start 2024-04-24T23:21:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.548+0000 7f1979d53640 1 -- start start 2024-04-24T23:21:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f1979d53640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f197415e310 0x7f1974162830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f1979d53640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f197415f700 con 0x7f197406c440 2024-04-24T23:21:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f1979d53640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19740d0310 con 0x7f197406d350 2024-04-24T23:21:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f1979d53640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f19740b3740 con 0x7f197415e310 2024-04-24T23:21:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f1972ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f197406c440 0x7f197406cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42014/0 (socket says 172.21.15.26:42014) 2024-04-24T23:21:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f1972ffd640 1 -- 172.21.15.26:0/4277404295 learned_addr learned my addr 172.21.15.26:0/4277404295 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f19737fe640 1 --2- 172.21.15.26:0/4277404295 >> v2:172.21.15.29:3300/0 conn(0x7f197415e310 0x7f1974162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f19727fc640 1 -- 172.21.15.26:0/4277404295 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509397035 0 0) 0x7f197415f700 con 0x7f197406c440 2024-04-24T23:21:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f19727fc640 1 -- 172.21.15.26:0/4277404295 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f195c003520 con 0x7f197406c440 2024-04-24T23:21:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f19727fc640 1 -- 172.21.15.26:0/4277404295 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3340419202 0 0) 0x7f195c003520 con 0x7f197406c440 2024-04-24T23:21:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f19737fe640 1 -- 172.21.15.26:0/4277404295 >> v1:172.21.15.29:6789/0 conn(0x7f197406d350 legacy=0x7f19740ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f19737fe640 1 -- 172.21.15.26:0/4277404295 >> v1:172.21.15.26:6789/0 conn(0x7f197406c440 legacy=0x7f197406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.549+0000 7f19737fe640 1 -- 172.21.15.26:0/4277404295 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740d05f0 con 0x7f197415e310 2024-04-24T23:21:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.550+0000 7f19737fe640 1 --2- 172.21.15.26:0/4277404295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 0x7f1974162830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f19640097a0 tx=0x7f196402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2151a946eb3aa75a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.550+0000 7f19727fc640 1 -- 172.21.15.26:0/4277404295 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964037070 con 0x7f197415e310 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.550+0000 7f19727fc640 1 -- 172.21.15.26:0/4277404295 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196402ac50 con 0x7f197415e310 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.550+0000 7f19727fc640 1 -- 172.21.15.26:0/4277404295 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968001670 con 0x7f197415e310 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.551+0000 7f1979d53640 1 -- 172.21.15.26:0/4277404295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 msgr2=0x7f1974162830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.551+0000 7f1979d53640 1 --2- 172.21.15.26:0/4277404295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 0x7f1974162830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f19640097a0 tx=0x7f196402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.551+0000 7f1979d53640 1 -- 172.21.15.26:0/4277404295 shutdown_connections 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.551+0000 7f1979d53640 1 --2- 172.21.15.26:0/4277404295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 0x7f1974162830 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.551+0000 7f1979d53640 1 -- 172.21.15.26:0/4277404295 >> 172.21.15.26:0/4277404295 conn(0x7f19740d1dd0 msgr2=0x7f19740d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.551+0000 7f1979d53640 1 -- 172.21.15.26:0/4277404295 shutdown_connections 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.551+0000 7f1979d53640 1 -- 172.21.15.26:0/4277404295 wait complete. 2024-04-24T23:21:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.552+0000 7f1979d53640 1 Processor -- start 2024-04-24T23:21:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.552+0000 7f1979d53640 1 -- start start 2024-04-24T23:21:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1979d53640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f197406c440 0x7f19741e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1979d53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f197406d350 0x7f1974141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1979d53640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 0x7f19741420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1979d53640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19740d0e90 con 0x7f197406d350 2024-04-24T23:21:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1979d53640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19740d0bb0 con 0x7f197415e310 2024-04-24T23:21:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1979d53640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1974162d70 con 0x7f197406c440 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1972ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f197406d350 0x7f1974141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1972ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f197406d350 0x7f1974141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58112/0 (socket says 172.21.15.26:58112) 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1972ffd640 1 -- 172.21.15.26:0/2989745632 learned_addr learned my addr 172.21.15.26:0/2989745632 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1973fff640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 0x7f19741420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f19737fe640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f197406c440 0x7f19741e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1972ffd640 1 -- 172.21.15.26:0/2989745632 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f197406c440 msgr2=0x7f19741e8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1972ffd640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f197406c440 0x7f19741e8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1972ffd640 1 -- 172.21.15.26:0/2989745632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 msgr2=0x7f19741420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1972ffd640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 0x7f19741420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1972ffd640 1 -- 172.21.15.26:0/2989745632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19741e95e0 con 0x7f197406d350 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.553+0000 7f1973fff640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 0x7f19741420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.554+0000 7f1972ffd640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f197406d350 0x7f1974141ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f19680011e0 tx=0x7f196800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.554+0000 7f195bfff640 1 -- 172.21.15.26:0/2989745632 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968017070 con 0x7f197406d350 2024-04-24T23:21:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.554+0000 7f195bfff640 1 -- 172.21.15.26:0/2989745632 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1968005ce0 con 0x7f197406d350 2024-04-24T23:21:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.554+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19741e98c0 con 0x7f197406d350 2024-04-24T23:21:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.554+0000 7f195bfff640 1 -- 172.21.15.26:0/2989745632 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19680072b0 con 0x7f197406d350 2024-04-24T23:21:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.554+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741e9e80 con 0x7f197406d350 2024-04-24T23:21:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.555+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1974075610 con 0x7f197406d350 2024-04-24T23:21:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.555+0000 7f195bfff640 1 -- 172.21.15.26:0/2989745632 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f196801a070 con 0x7f197406d350 2024-04-24T23:21:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.556+0000 7f195bfff640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f196006be30 0x7f196006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.556+0000 7f195bfff640 1 -- 172.21.15.26:0/2989745632 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f196808a210 con 0x7f197406d350 2024-04-24T23:21:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.556+0000 7f19737fe640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f196006be30 0x7f196006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.556+0000 7f19737fe640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f196006be30 0x7f196006e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1964034000 tx=0x7f196402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:27.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.560+0000 7f195bfff640 1 -- 172.21.15.26:0/2989745632 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f19680584e0 con 0x7f197406d350 2024-04-24T23:21:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.741+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f197406ca20 con 0x7f197406d350 2024-04-24T23:21:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.742+0000 7f195bfff640 1 -- 172.21.15.26:0/2989745632 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f196805c7e0 con 0x7f197406d350 2024-04-24T23:21:27.743 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.743+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f196006be30 msgr2=0x7f196006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.743+0000 7f1979d53640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f196006be30 0x7f196006e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1964034000 tx=0x7f196402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.743+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f197406d350 msgr2=0x7f1974141ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.743+0000 7f1979d53640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f197406d350 0x7f1974141ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f19680011e0 tx=0x7f196800c960 comp rx=0 tx=0).stop 2024-04-24T23:21:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.743+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 shutdown_connections 2024-04-24T23:21:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.744+0000 7f1979d53640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f196006be30 0x7f196006e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.744+0000 7f1979d53640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f197415e310 0x7f19741420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.744+0000 7f1979d53640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f197406d350 0x7f1974141ba0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.744+0000 7f1979d53640 1 --2- 172.21.15.26:0/2989745632 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f197406c440 0x7f19741e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.744+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 >> 172.21.15.26:0/2989745632 conn(0x7f19740d1dd0 msgr2=0x7f19740b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.744+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 shutdown_connections 2024-04-24T23:21:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.744+0000 7f1979d53640 1 -- 172.21.15.26:0/2989745632 wait complete. 2024-04-24T23:21:27.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.858+0000 7f6acec7d640 1 Processor -- start 2024-04-24T23:21:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.858+0000 7f6acec7d640 1 -- start start 2024-04-24T23:21:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acec7d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ac815be40 0x7f6ac80acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acec7d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6ac80b4ce0 0x7f6ac80d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acec7d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ac80aeb10 con 0x7f6ac80d0580 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acec7d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6ac809df20 con 0x7f6ac815be40 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acec7d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6ac80aedf0 con 0x7f6ac80b4ce0 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acca19640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ac815be40 0x7f6ac80acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acca19640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ac815be40 0x7f6ac80acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58124/0 (socket says 172.21.15.26:58124) 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acca19640 1 -- 172.21.15.26:0/894635341 learned_addr learned my addr 172.21.15.26:0/894635341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6ac7fff640 1 --2- 172.21.15.26:0/894635341 >> v2:172.21.15.29:3300/0 conn(0x7f6ac80b4ce0 0x7f6ac80d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acca19640 1 -- 172.21.15.26:0/894635341 >> v2:172.21.15.29:3300/0 conn(0x7f6ac80b4ce0 msgr2=0x7f6ac80d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acca19640 1 --2- 172.21.15.26:0/894635341 >> v2:172.21.15.29:3300/0 conn(0x7f6ac80b4ce0 0x7f6ac80d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acca19640 1 -- 172.21.15.26:0/894635341 >> v1:172.21.15.29:6789/0 conn(0x7f6ac80d0580 legacy=0x7f6ac80b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.859+0000 7f6acca19640 1 -- 172.21.15.26:0/894635341 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac80af0d0 con 0x7f6ac815be40 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.860+0000 7f6acca19640 1 --2- 172.21.15.26:0/894635341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 0x7f6ac80acd30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80b4c90 tx=0x7f6ac002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfb69edf1e42873 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.860+0000 7f6ac77fe640 1 -- 172.21.15.26:0/894635341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0037070 con 0x7f6ac815be40 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.860+0000 7f6ac77fe640 1 -- 172.21.15.26:0/894635341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac0032440 con 0x7f6ac815be40 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.860+0000 7f6ac77fe640 1 -- 172.21.15.26:0/894635341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0032810 con 0x7f6ac815be40 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.861+0000 7f6acec7d640 1 -- 172.21.15.26:0/894635341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 msgr2=0x7f6ac80acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.861+0000 7f6acec7d640 1 --2- 172.21.15.26:0/894635341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 0x7f6ac80acd30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80b4c90 tx=0x7f6ac002a230 comp rx=0 tx=0).stop 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.861+0000 7f6acec7d640 1 -- 172.21.15.26:0/894635341 shutdown_connections 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.861+0000 7f6acec7d640 1 --2- 172.21.15.26:0/894635341 >> v2:172.21.15.29:3300/0 conn(0x7f6ac80b4ce0 0x7f6ac80d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.861+0000 7f6acec7d640 1 --2- 172.21.15.26:0/894635341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 0x7f6ac80acd30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.861+0000 7f6acec7d640 1 -- 172.21.15.26:0/894635341 >> 172.21.15.26:0/894635341 conn(0x7f6ac806b540 msgr2=0x7f6ac806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.861+0000 7f6acec7d640 1 -- 172.21.15.26:0/894635341 shutdown_connections 2024-04-24T23:21:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.861+0000 7f6acec7d640 1 -- 172.21.15.26:0/894635341 wait complete. 2024-04-24T23:21:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.862+0000 7f6acec7d640 1 Processor -- start 2024-04-24T23:21:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.862+0000 7f6acec7d640 1 -- start start 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acec7d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ac80b4ce0 0x7f6ac81f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acec7d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ac80d0580 0x7f6ac809d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acec7d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 0x7f6ac8141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acec7d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac806cd10 con 0x7f6ac815be40 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acec7d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac806cb90 con 0x7f6ac80d0580 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acec7d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac815d230 con 0x7f6ac80b4ce0 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acd21a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 0x7f6ac8141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acd21a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 0x7f6ac8141220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58136/0 (socket says 172.21.15.26:58136) 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acd21a640 1 -- 172.21.15.26:0/3279884423 learned_addr learned my addr 172.21.15.26:0/3279884423 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acca19640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ac80b4ce0 0x7f6ac81f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6ac7fff640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ac80d0580 0x7f6ac809d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acd21a640 1 -- 172.21.15.26:0/3279884423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ac80b4ce0 msgr2=0x7f6ac81f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acd21a640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ac80b4ce0 0x7f6ac81f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acd21a640 1 -- 172.21.15.26:0/3279884423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ac80d0580 msgr2=0x7f6ac809d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acd21a640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ac80d0580 0x7f6ac809d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6acd21a640 1 -- 172.21.15.26:0/3279884423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac809b720 con 0x7f6ac815be40 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.863+0000 7f6ac7fff640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ac80d0580 0x7f6ac809d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.864+0000 7f6acd21a640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 0x7f6ac8141220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000c390 tx=0x7f6ab000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.864+0000 7f6ac57fa640 1 -- 172.21.15.26:0/3279884423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0017070 con 0x7f6ac815be40 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.864+0000 7f6ac57fa640 1 -- 172.21.15.26:0/3279884423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab00035f0 con 0x7f6ac815be40 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.864+0000 7f6ac57fa640 1 -- 172.21.15.26:0/3279884423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00039e0 con 0x7f6ac815be40 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.864+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac809ba00 con 0x7f6ac815be40 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.864+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac80ce900 con 0x7f6ac815be40 2024-04-24T23:21:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.865+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac80cead0 con 0x7f6ac815be40 2024-04-24T23:21:27.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.865+0000 7f6ac57fa640 1 -- 172.21.15.26:0/3279884423 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ab001a070 con 0x7f6ac815be40 2024-04-24T23:21:27.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.866+0000 7f6ac57fa640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6aa806be30 0x7f6aa806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:27.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.866+0000 7f6ac57fa640 1 -- 172.21.15.26:0/3279884423 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6ab008a1b0 con 0x7f6ac815be40 2024-04-24T23:21:27.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.866+0000 7f6acca19640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6aa806be30 0x7f6aa806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:27.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.866+0000 7f6acca19640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6aa806be30 0x7f6aa806e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0034000 tx=0x7f6ac002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:27.870+0000 7f6ac57fa640 1 -- 172.21.15.26:0/3279884423 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ab0058500 con 0x7f6ac815be40 2024-04-24T23:21:28.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.053+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6ac8075610 con 0x7f6ac815be40 2024-04-24T23:21:28.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.054+0000 7f6ac57fa640 1 -- 172.21.15.26:0/3279884423 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f6ab005c800 con 0x7f6ac815be40 2024-04-24T23:21:28.056 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:28.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6aa806be30 msgr2=0x7f6aa806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6aa806be30 0x7f6aa806e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0034000 tx=0x7f6ac002e000 comp rx=0 tx=0).stop 2024-04-24T23:21:28.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 msgr2=0x7f6ac8141220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 0x7f6ac8141220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000c390 tx=0x7f6ab000c850 comp rx=0 tx=0).stop 2024-04-24T23:21:28.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 shutdown_connections 2024-04-24T23:21:28.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6aa806be30 0x7f6aa806e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ac815be40 0x7f6ac8141220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ac80d0580 0x7f6ac809d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 --2- 172.21.15.26:0/3279884423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6ac80b4ce0 0x7f6ac81f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 >> 172.21.15.26:0/3279884423 conn(0x7f6ac806b540 msgr2=0x7f6ac815b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 shutdown_connections 2024-04-24T23:21:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.056+0000 7f6acec7d640 1 -- 172.21.15.26:0/3279884423 wait complete. 2024-04-24T23:21:28.069 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.172+0000 7f0a4ccda640 1 Processor -- start 2024-04-24T23:21:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.172+0000 7f0a4ccda640 1 -- start start 2024-04-24T23:21:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.173+0000 7f0a4ccda640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a4815e370 0x7f0a480d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.173+0000 7f0a4ccda640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a480d2120 con 0x7f0a480d4c10 2024-04-24T23:21:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.173+0000 7f0a4ccda640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a480b3740 con 0x7f0a4815e750 2024-04-24T23:21:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.173+0000 7f0a4ccda640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0a480d1510 con 0x7f0a4815e370 2024-04-24T23:21:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.173+0000 7f0a4659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a4815e370 0x7f0a480d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.173+0000 7f0a46d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0a480d4c10 0x7f0a480b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42016/0 (socket says 172.21.15.26:42016) 2024-04-24T23:21:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.173+0000 7f0a46d9d640 1 -- 172.21.15.26:0/3724922814 learned_addr learned my addr 172.21.15.26:0/3724922814 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.174+0000 7f0a4659c640 1 -- 172.21.15.26:0/3724922814 >> v1:172.21.15.29:6789/0 conn(0x7f0a4815e750 legacy=0x7f0a480b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.174+0000 7f0a4659c640 1 -- 172.21.15.26:0/3724922814 >> v1:172.21.15.26:6789/0 conn(0x7f0a480d4c10 legacy=0x7f0a480b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.174+0000 7f0a4659c640 1 -- 172.21.15.26:0/3724922814 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a480b3a20 con 0x7f0a4815e370 2024-04-24T23:21:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.174+0000 7f0a4659c640 1 --2- 172.21.15.26:0/3724922814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a4815e370 0x7f0a480d2520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0a380097a0 tx=0x7f0a3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4402cff91cb28d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a45d9b640 1 -- 172.21.15.26:0/3724922814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38037070 con 0x7f0a4815e370 2024-04-24T23:21:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a45d9b640 1 -- 172.21.15.26:0/3724922814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a38032440 con 0x7f0a4815e370 2024-04-24T23:21:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a45d9b640 1 -- 172.21.15.26:0/3724922814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38032810 con 0x7f0a4815e370 2024-04-24T23:21:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a4ccda640 1 -- 172.21.15.26:0/3724922814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a4815e370 msgr2=0x7f0a480d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a4ccda640 1 --2- 172.21.15.26:0/3724922814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a4815e370 0x7f0a480d2520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0a380097a0 tx=0x7f0a3802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a4ccda640 1 -- 172.21.15.26:0/3724922814 shutdown_connections 2024-04-24T23:21:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a4ccda640 1 --2- 172.21.15.26:0/3724922814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a4815e370 0x7f0a480d2520 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a4ccda640 1 -- 172.21.15.26:0/3724922814 >> 172.21.15.26:0/3724922814 conn(0x7f0a480d40f0 msgr2=0x7f0a480d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a4ccda640 1 -- 172.21.15.26:0/3724922814 shutdown_connections 2024-04-24T23:21:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.175+0000 7f0a4ccda640 1 -- 172.21.15.26:0/3724922814 wait complete. 2024-04-24T23:21:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a4ccda640 1 Processor -- start 2024-04-24T23:21:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a4ccda640 1 -- start start 2024-04-24T23:21:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a4ccda640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a480d4c10 0x7f0a481567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a4ccda640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a4815e370 0x7f0a4814d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a4ccda640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a480c8c30 0x7f0a481594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a4ccda640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a480cfca0 con 0x7f0a480c8c30 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a4ccda640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0a480d1ed0 con 0x7f0a4815e370 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a4ccda640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0a480cfe20 con 0x7f0a480d4c10 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a46d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a480c8c30 0x7f0a481594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a46d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a480c8c30 0x7f0a481594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58152/0 (socket says 172.21.15.26:58152) 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.177+0000 7f0a46d9d640 1 -- 172.21.15.26:0/2151088280 learned_addr learned my addr 172.21.15.26:0/2151088280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a3ffff640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a4815e370 0x7f0a4814d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a4659c640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a480d4c10 0x7f0a481567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a46d9d640 1 -- 172.21.15.26:0/2151088280 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a480d4c10 msgr2=0x7f0a481567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a46d9d640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a480d4c10 0x7f0a481567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a46d9d640 1 -- 172.21.15.26:0/2151088280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a4815e370 msgr2=0x7f0a4814d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a46d9d640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a4815e370 0x7f0a4814d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a46d9d640 1 -- 172.21.15.26:0/2151088280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a48159a00 con 0x7f0a480c8c30 2024-04-24T23:21:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a3ffff640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a4815e370 0x7f0a4814d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a46d9d640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a480c8c30 0x7f0a481594c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000cbf0 tx=0x7f0a4000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a3effd640 1 -- 172.21.15.26:0/2151088280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40017070 con 0x7f0a480c8c30 2024-04-24T23:21:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a3effd640 1 -- 172.21.15.26:0/2151088280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4000ad40 con 0x7f0a480c8c30 2024-04-24T23:21:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a480c90c0 con 0x7f0a480c8c30 2024-04-24T23:21:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a3effd640 1 -- 172.21.15.26:0/2151088280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40012600 con 0x7f0a480c8c30 2024-04-24T23:21:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.178+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a480c8150 con 0x7f0a480c8c30 2024-04-24T23:21:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.179+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a48075610 con 0x7f0a480c8c30 2024-04-24T23:21:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.180+0000 7f0a3effd640 1 -- 172.21.15.26:0/2151088280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a4001a070 con 0x7f0a480c8c30 2024-04-24T23:21:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.180+0000 7f0a3effd640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a2806be30 0x7f0a2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.180+0000 7f0a3effd640 1 -- 172.21.15.26:0/2151088280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0a4008ab20 con 0x7f0a480c8c30 2024-04-24T23:21:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.180+0000 7f0a4659c640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a2806be30 0x7f0a2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.181+0000 7f0a4659c640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a2806be30 0x7f0a2806e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a38034000 tx=0x7f0a3802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:28.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.189+0000 7f0a3effd640 1 -- 172.21.15.26:0/2151088280 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a40058e80 con 0x7f0a480c8c30 2024-04-24T23:21:28.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.376+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a480b4980 con 0x7f0a480c8c30 2024-04-24T23:21:28.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.377+0000 7f0a3effd640 1 -- 172.21.15.26:0/2151088280 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0a4005d180 con 0x7f0a480c8c30 2024-04-24T23:21:28.378 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a2806be30 msgr2=0x7f0a2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a2806be30 0x7f0a2806e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a38034000 tx=0x7f0a3802e000 comp rx=0 tx=0).stop 2024-04-24T23:21:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a480c8c30 msgr2=0x7f0a481594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a480c8c30 0x7f0a481594c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000cbf0 tx=0x7f0a4000a430 comp rx=0 tx=0).stop 2024-04-24T23:21:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 shutdown_connections 2024-04-24T23:21:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a2806be30 0x7f0a2806e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a480c8c30 0x7f0a481594c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a4815e370 0x7f0a4814d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 --2- 172.21.15.26:0/2151088280 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a480d4c10 0x7f0a481567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 >> 172.21.15.26:0/2151088280 conn(0x7f0a480d40f0 msgr2=0x7f0a480b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 shutdown_connections 2024-04-24T23:21:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.379+0000 7f0a4ccda640 1 -- 172.21.15.26:0/2151088280 wait complete. 2024-04-24T23:21:28.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:21:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.494+0000 7ffb01163640 1 Processor -- start 2024-04-24T23:21:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.494+0000 7ffb01163640 1 -- start start 2024-04-24T23:21:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffb01163640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 0x7ffafc0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffb01163640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffafc0b5840 con 0x7ffafc15e290 2024-04-24T23:21:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffb01163640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffafc0cfb30 con 0x7ffafc0b62e0 2024-04-24T23:21:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffb01163640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ffafc0b4460 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffafad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffafc15e290 0x7ffafc0d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42020/0 (socket says 172.21.15.26:42020) 2024-04-24T23:21:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffafad9d640 1 -- 172.21.15.26:0/3652269874 learned_addr learned my addr 172.21.15.26:0/3652269874 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffaf3fff640 1 --2- 172.21.15.26:0/3652269874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 0x7ffafc0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffafa59c640 1 -- 172.21.15.26:0/3652269874 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4273668749 0 0) 0x7ffafc0b5840 con 0x7ffafc15e290 2024-04-24T23:21:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffafa59c640 1 -- 172.21.15.26:0/3652269874 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffae8003520 con 0x7ffafc15e290 2024-04-24T23:21:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffafa59c640 1 -- 172.21.15.26:0/3652269874 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2292233334 0 0) 0x7ffae8003520 con 0x7ffafc15e290 2024-04-24T23:21:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffaf3fff640 1 -- 172.21.15.26:0/3652269874 >> v1:172.21.15.29:6789/0 conn(0x7ffafc0b62e0 legacy=0x7ffafc0b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffaf3fff640 1 -- 172.21.15.26:0/3652269874 >> v1:172.21.15.26:6789/0 conn(0x7ffafc15e290 legacy=0x7ffafc0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.495+0000 7ffaf3fff640 1 -- 172.21.15.26:0/3652269874 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc0cfe10 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.496+0000 7ffaf3fff640 1 --2- 172.21.15.26:0/3652269874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 0x7ffafc0b3f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffae40097d0 tx=0x7ffae402a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d67cc205575c5297 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.496+0000 7ffafa59c640 1 -- 172.21.15.26:0/3652269874 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4037070 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.496+0000 7ffafa59c640 1 -- 172.21.15.26:0/3652269874 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae402ab40 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.496+0000 7ffafa59c640 1 -- 172.21.15.26:0/3652269874 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec001690 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.497+0000 7ffb01163640 1 -- 172.21.15.26:0/3652269874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 msgr2=0x7ffafc0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.497+0000 7ffb01163640 1 --2- 172.21.15.26:0/3652269874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 0x7ffafc0b3f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffae40097d0 tx=0x7ffae402a0b0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.497+0000 7ffb01163640 1 -- 172.21.15.26:0/3652269874 shutdown_connections 2024-04-24T23:21:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.497+0000 7ffb01163640 1 --2- 172.21.15.26:0/3652269874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 0x7ffafc0b3f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.497+0000 7ffb01163640 1 -- 172.21.15.26:0/3652269874 >> 172.21.15.26:0/3652269874 conn(0x7ffafc06b120 msgr2=0x7ffafc06b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.497+0000 7ffb01163640 1 -- 172.21.15.26:0/3652269874 shutdown_connections 2024-04-24T23:21:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.497+0000 7ffb01163640 1 -- 172.21.15.26:0/3652269874 wait complete. 2024-04-24T23:21:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.498+0000 7ffb01163640 1 Processor -- start 2024-04-24T23:21:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.498+0000 7ffb01163640 1 -- start start 2024-04-24T23:21:28.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffb01163640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0b3ac0 0x7ffafc152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffb01163640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b62e0 0x7ffafc161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffb01163640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 0x7ffafc1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffb01163640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc0b5ae0 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffb01163640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc0b5e50 con 0x7ffafc15e290 2024-04-24T23:21:28.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffb01163640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ffafc15e670 con 0x7ffafc0b62e0 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffafad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0b3ac0 0x7ffafc152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffafad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0b3ac0 0x7ffafc152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58160/0 (socket says 172.21.15.26:58160) 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffafad9d640 1 -- 172.21.15.26:0/3479283102 learned_addr learned my addr 172.21.15.26:0/3479283102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffaf3fff640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b62e0 0x7ffafc161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.499+0000 7ffafb59e640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 0x7ffafc1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffafad9d640 1 -- 172.21.15.26:0/3479283102 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b62e0 msgr2=0x7ffafc161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffafad9d640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b62e0 0x7ffafc161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffafad9d640 1 -- 172.21.15.26:0/3479283102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 msgr2=0x7ffafc1f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffafad9d640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 0x7ffafc1f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffafad9d640 1 -- 172.21.15.26:0/3479283102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc0ca3c0 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffafad9d640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0b3ac0 0x7ffafc152500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00c440 tx=0x7ffaec00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffaf37fe640 1 -- 172.21.15.26:0/3479283102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec018070 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffaf37fe640 1 -- 172.21.15.26:0/3479283102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaec007500 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffafc0ca660 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffaf37fe640 1 -- 172.21.15.26:0/3479283102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec004a20 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.500+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffafc0cac20 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.501+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffafc0694c0 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.502+0000 7ffaf37fe640 1 -- 172.21.15.26:0/3479283102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffaec01b070 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.502+0000 7ffaf37fe640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffadc06be30 0x7ffadc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.502+0000 7ffaf37fe640 1 -- 172.21.15.26:0/3479283102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffaec08a0a0 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.502+0000 7ffaf3fff640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffadc06be30 0x7ffadc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.503+0000 7ffaf3fff640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffadc06be30 0x7ffadc06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffae4034000 tx=0x7ffae402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.509+0000 7ffaf37fe640 1 -- 172.21.15.26:0/3479283102 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffaec058400 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.706+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffafc0b7bf0 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.707+0000 7ffaf37fe640 1 -- 172.21.15.26:0/3479283102 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7ffaec05c700 con 0x7ffafc0b3ac0 2024-04-24T23:21:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffadc06be30 msgr2=0x7ffadc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffadc06be30 0x7ffadc06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffae4034000 tx=0x7ffae402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0b3ac0 msgr2=0x7ffafc152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0b3ac0 0x7ffafc152500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00c440 tx=0x7ffaec00c900 comp rx=0 tx=0).stop 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 shutdown_connections 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffadc06be30 0x7ffadc06e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 0x7ffafc1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b62e0 0x7ffafc161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 --2- 172.21.15.26:0/3479283102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0b3ac0 0x7ffafc152500 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 >> 172.21.15.26:0/3479283102 conn(0x7ffafc06b120 msgr2=0x7ffafc06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 shutdown_connections 2024-04-24T23:21:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:28.709+0000 7ffb01163640 1 -- 172.21.15.26:0/3479283102 wait complete. 2024-04-24T23:21:28.721 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:21:28.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:28.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-24T23:21:28.748 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:21:28.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:28.748 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:21:28.817 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:21:28.817 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:28.837 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:21:28.837 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:28.896 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:21:28.896 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:28.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:28.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:21:29.001 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:21:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.055+0000 7fae79415640 1 Processor -- start 2024-04-24T23:21:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.055+0000 7fae79415640 1 -- start start 2024-04-24T23:21:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae79415640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fae740d4870 0x7fae740b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae79415640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae740b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae79415640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae740d32f0 con 0x7fae7415e570 2024-04-24T23:21:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae79415640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fae740d1ab0 con 0x7fae740d4870 2024-04-24T23:21:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae79415640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fae740d2ae0 con 0x7fae740b3130 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae72ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae7415e570 0x7fae740d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42036/0 (socket says 172.21.15.26:42036) 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae72ffd640 1 -- 172.21.15.26:0/1776700668 learned_addr learned my addr 172.21.15.26:0/1776700668 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae737fe640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae740b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae727fc640 1 --2- 172.21.15.26:0/1776700668 >> v2:172.21.15.29:3300/0 conn(0x7fae740d4870 0x7fae740b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3542791483 0 0) 0x7fae68001110 con 0x7fae7415e570 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.056+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae5c003520 con 0x7fae7415e570 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2267676748 0 0) 0x7fae5c003520 con 0x7fae7415e570 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 msgr2=0x7fae740b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae740b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 >> v2:172.21.15.29:3300/0 conn(0x7fae740d4870 msgr2=0x7fae740b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 --2- 172.21.15.26:0/1776700668 >> v2:172.21.15.29:3300/0 conn(0x7fae740d4870 0x7fae740b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae740d0270 con 0x7fae7415e570 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fae68001860 con 0x7fae7415e570 2024-04-24T23:21:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 >> v1:172.21.15.26:6789/0 conn(0x7fae7415e570 legacy=0x7fae740d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae5c0039b0 0x7fae5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae5c004d40 0x7fae5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae740d1ab0 con 0x7fae5c0039b0 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fae740d2ae0 con 0x7fae5c004d40 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fae5c00b9c0 con 0x7fae740b3130 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae72ffd640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae5c0039b0 0x7fae5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae727fc640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae737fe640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae5c004d40 0x7fae5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae72ffd640 1 -- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 msgr2=0x7fae5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae72ffd640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae72ffd640 1 -- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae5c004d40 msgr2=0x7fae5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae72ffd640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae5c004d40 0x7fae5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.057+0000 7fae72ffd640 1 -- 172.21.15.26:0/1776700668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5c003520 con 0x7fae5c0039b0 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.058+0000 7fae72ffd640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae5c0039b0 0x7fae5c003dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c490 tx=0x7fae5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7edfac3d21f4074 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.058+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6800c0e0 con 0x7fae5c0039b0 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.058+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae68034070 con 0x7fae5c0039b0 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.058+0000 7fae71ffb640 1 -- 172.21.15.26:0/1776700668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68005d10 con 0x7fae5c0039b0 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.058+0000 7fae79415640 1 -- 172.21.15.26:0/1776700668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae5c0039b0 msgr2=0x7fae5c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.058+0000 7fae79415640 1 --2- 172.21.15.26:0/1776700668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae5c0039b0 0x7fae5c003dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c490 tx=0x7fae5c005830 comp rx=0 tx=0).stop 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.058+0000 7fae737fe640 1 -- 172.21.15.26:0/1776700668 reap_dead start 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.058+0000 7fae79415640 1 -- 172.21.15.26:0/1776700668 shutdown_connections 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.058+0000 7fae79415640 1 -- 172.21.15.26:0/1776700668 >> 172.21.15.26:0/1776700668 conn(0x7fae7408dbe0 msgr2=0x7fae7409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.059+0000 7fae79415640 1 -- 172.21.15.26:0/1776700668 shutdown_connections 2024-04-24T23:21:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.059+0000 7fae79415640 1 -- 172.21.15.26:0/1776700668 wait complete. 2024-04-24T23:21:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.060+0000 7fae79415640 1 Processor -- start 2024-04-24T23:21:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.060+0000 7fae79415640 1 -- start start 2024-04-24T23:21:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae79415640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae5c0039b0 0x7fae74158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae79415640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415ae70 0x7fae740b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae79415640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740ca100 0x7fae740b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae79415640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae740cfa70 con 0x7fae740ca100 2024-04-24T23:21:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae79415640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fae740d3010 con 0x7fae7415ae70 2024-04-24T23:21:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae79415640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fae740cfd50 con 0x7fae5c0039b0 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740ca100 0x7fae740b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740ca100 0x7fae740b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58182/0 (socket says 172.21.15.26:58182) 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae737fe640 1 -- 172.21.15.26:0/3052550940 learned_addr learned my addr 172.21.15.26:0/3052550940 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae72ffd640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae5c0039b0 0x7fae74158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae727fc640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415ae70 0x7fae740b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae737fe640 1 -- 172.21.15.26:0/3052550940 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae5c0039b0 msgr2=0x7fae74158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae737fe640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae5c0039b0 0x7fae74158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae737fe640 1 -- 172.21.15.26:0/3052550940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415ae70 msgr2=0x7fae740b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae737fe640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415ae70 0x7fae740b8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae737fe640 1 -- 172.21.15.26:0/3052550940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae740b9730 con 0x7fae740ca100 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae727fc640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415ae70 0x7fae740b8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae737fe640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740ca100 0x7fae740b2270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fae7408c9f0 tx=0x7fae6c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3052550940 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00fde0 con 0x7fae740ca100 2024-04-24T23:21:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3052550940 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6c015430 con 0x7fae740ca100 2024-04-24T23:21:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae74159dd0 con 0x7fae740ca100 2024-04-24T23:21:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.061+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3052550940 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c015820 con 0x7fae740ca100 2024-04-24T23:21:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.062+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7415a390 con 0x7fae740ca100 2024-04-24T23:21:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.062+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae740694c0 con 0x7fae740ca100 2024-04-24T23:21:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.063+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3052550940 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae6c0159c0 con 0x7fae740ca100 2024-04-24T23:21:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.064+0000 7fae5b7fe640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.064+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3052550940 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fae6c08bce0 con 0x7fae740ca100 2024-04-24T23:21:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.064+0000 7fae72ffd640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.064+0000 7fae72ffd640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae740ba280 tx=0x7fae68009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.068+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3052550940 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fae6c019030 con 0x7fae740ca100 2024-04-24T23:21:29.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.252+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae740ca5b0 con 0x7fae740ca100 2024-04-24T23:21:29.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.253+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3052550940 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fae6c010040 con 0x7fae740ca100 2024-04-24T23:21:29.255 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:29.255 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:29.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 msgr2=0x7fae6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae740ba280 tx=0x7fae68009030 comp rx=0 tx=0).stop 2024-04-24T23:21:29.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740ca100 msgr2=0x7fae740b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740ca100 0x7fae740b2270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fae7408c9f0 tx=0x7fae6c00f580 comp rx=0 tx=0).stop 2024-04-24T23:21:29.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 shutdown_connections 2024-04-24T23:21:29.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740ca100 0x7fae740b2270 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415ae70 0x7fae740b8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 --2- 172.21.15.26:0/3052550940 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae5c0039b0 0x7fae74158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 >> 172.21.15.26:0/3052550940 conn(0x7fae7408dbe0 msgr2=0x7fae740d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 shutdown_connections 2024-04-24T23:21:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.255+0000 7fae79415640 1 -- 172.21.15.26:0/3052550940 wait complete. 2024-04-24T23:21:29.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:29.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.410+0000 7fdc63cee640 1 Processor -- start 2024-04-24T23:21:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.410+0000 7fdc63cee640 1 -- start start 2024-04-24T23:21:29.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.411+0000 7fdc63cee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdc5c0d0750 0x7fdc5c165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.411+0000 7fdc63cee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5c15ef60 con 0x7fdc5c0ad7b0 2024-04-24T23:21:29.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.411+0000 7fdc63cee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5c15f240 con 0x7fdc5c15adc0 2024-04-24T23:21:29.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.411+0000 7fdc63cee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5c15f520 con 0x7fdc5c0d0750 2024-04-24T23:21:29.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.411+0000 7fdc61289640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdc5c0ad7b0 0x7fdc5c0d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42048/0 (socket says 172.21.15.26:42048) 2024-04-24T23:21:29.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.411+0000 7fdc61289640 1 -- 172.21.15.26:0/4119874441 learned_addr learned my addr 172.21.15.26:0/4119874441 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:29.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc6228b640 1 --2- 172.21.15.26:0/4119874441 >> v2:172.21.15.29:3300/0 conn(0x7fdc5c0d0750 0x7fdc5c165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3333015597 0 0) 0x7fdc5c15ef60 con 0x7fdc5c0ad7b0 2024-04-24T23:21:29.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc44003520 con 0x7fdc5c0ad7b0 2024-04-24T23:21:29.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1986418153 0 0) 0x7fdc44003520 con 0x7fdc5c0ad7b0 2024-04-24T23:21:29.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 >> v2:172.21.15.29:3300/0 conn(0x7fdc5c0d0750 msgr2=0x7fdc5c165d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:29.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 --2- 172.21.15.26:0/4119874441 >> v2:172.21.15.29:3300/0 conn(0x7fdc5c0d0750 0x7fdc5c165d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 >> v1:172.21.15.29:6789/0 conn(0x7fdc5c15adc0 legacy=0x7fdc5c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c15f800 con 0x7fdc5c0ad7b0 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fdc50001670 con 0x7fdc5c0ad7b0 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 >> v1:172.21.15.26:6789/0 conn(0x7fdc5c0ad7b0 legacy=0x7fdc5c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc44004800 0x7fdc44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc440051c0 0x7fdc4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c15f520 con 0x7fdc440039b0 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4400c160 con 0x7fdc44004800 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4400c360 con 0x7fdc440051c0 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc61a8a640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc6228b640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc440051c0 0x7fdc4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.412+0000 7fdc61289640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc44004800 0x7fdc44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc61a8a640 1 -- 172.21.15.26:0/4119874441 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc440051c0 msgr2=0x7fdc4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc61a8a640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc440051c0 0x7fdc4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc61a8a640 1 -- 172.21.15.26:0/4119874441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc44004800 msgr2=0x7fdc44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc61a8a640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc44004800 0x7fdc44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc61a8a640 1 -- 172.21.15.26:0/4119874441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc44003520 con 0x7fdc440039b0 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc6228b640 1 -- 172.21.15.26:0/4119874441 reap_dead start 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc61a8a640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc44003dd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc54001750 tx=0x7fdc44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c158497a11220214 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54001370 con 0x7fdc440039b0 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc540052d0 con 0x7fdc440039b0 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.413+0000 7fdc60a88640 1 -- 172.21.15.26:0/4119874441 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5402abe0 con 0x7fdc440039b0 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.414+0000 7fdc63cee640 1 -- 172.21.15.26:0/4119874441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 msgr2=0x7fdc44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.414+0000 7fdc63cee640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc44003dd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc54001750 tx=0x7fdc44005830 comp rx=0 tx=0).stop 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.414+0000 7fdc63cee640 1 -- 172.21.15.26:0/4119874441 shutdown_connections 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.414+0000 7fdc63cee640 1 --2- 172.21.15.26:0/4119874441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc44003dd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.414+0000 7fdc63cee640 1 -- 172.21.15.26:0/4119874441 >> 172.21.15.26:0/4119874441 conn(0x7fdc5c06b540 msgr2=0x7fdc5c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:29.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.414+0000 7fdc63cee640 1 -- 172.21.15.26:0/4119874441 shutdown_connections 2024-04-24T23:21:29.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.414+0000 7fdc63cee640 1 -- 172.21.15.26:0/4119874441 wait complete. 2024-04-24T23:21:29.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.415+0000 7fdc63cee640 1 Processor -- start 2024-04-24T23:21:29.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.415+0000 7fdc63cee640 1 -- start start 2024-04-24T23:21:29.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc63cee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc5c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc63cee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc5c15dd90 0x7fdc5c138390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc63cee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5c09c530 0x7fdc5c09c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc63cee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c15e9c0 con 0x7fdc440039b0 2024-04-24T23:21:29.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc63cee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c15b1a0 con 0x7fdc5c09c530 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc63cee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5c15eb40 con 0x7fdc5c15dd90 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc61a8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc5c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc61a8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc5c15bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58202/0 (socket says 172.21.15.26:58202) 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc61a8a640 1 -- 172.21.15.26:0/799864913 learned_addr learned my addr 172.21.15.26:0/799864913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc61289640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc5c15dd90 0x7fdc5c138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc6228b640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5c09c530 0x7fdc5c09c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc61a8a640 1 -- 172.21.15.26:0/799864913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc5c15dd90 msgr2=0x7fdc5c138390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc61a8a640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc5c15dd90 0x7fdc5c138390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc61a8a640 1 -- 172.21.15.26:0/799864913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5c09c530 msgr2=0x7fdc5c09c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc61a8a640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5c09c530 0x7fdc5c09c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.416+0000 7fdc61a8a640 1 -- 172.21.15.26:0/799864913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c15e1f0 con 0x7fdc440039b0 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.417+0000 7fdc61a8a640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc5c15bd30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc54001870 tx=0x7fdc54001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:29.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.417+0000 7fdc5a7fc640 1 -- 172.21.15.26:0/799864913 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54037040 con 0x7fdc440039b0 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.417+0000 7fdc5a7fc640 1 -- 172.21.15.26:0/799864913 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc54000ed0 con 0x7fdc440039b0 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.417+0000 7fdc5a7fc640 1 -- 172.21.15.26:0/799864913 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54007a30 con 0x7fdc440039b0 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.417+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c15c410 con 0x7fdc440039b0 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.417+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c15c9d0 con 0x7fdc440039b0 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.418+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5c075610 con 0x7fdc440039b0 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.418+0000 7fdc5a7fc640 1 -- 172.21.15.26:0/799864913 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc54043020 con 0x7fdc440039b0 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.419+0000 7fdc5a7fc640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc4806be60 0x7fdc4806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.419+0000 7fdc5a7fc640 1 -- 172.21.15.26:0/799864913 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdc540aa250 con 0x7fdc440039b0 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.419+0000 7fdc61289640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc4806be60 0x7fdc4806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.419+0000 7fdc61289640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc4806be60 0x7fdc4806e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0d00d0 tx=0x7fdc50009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:29.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.423+0000 7fdc5a7fc640 1 -- 172.21.15.26:0/799864913 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdc5404a020 con 0x7fdc440039b0 2024-04-24T23:21:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.605+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdc5c15e730 con 0x7fdc440039b0 2024-04-24T23:21:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.605+0000 7fdc5a7fc640 1 -- 172.21.15.26:0/799864913 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fdc54078b70 con 0x7fdc440039b0 2024-04-24T23:21:29.607 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc4806be60 msgr2=0x7fdc4806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc4806be60 0x7fdc4806e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0d00d0 tx=0x7fdc50009040 comp rx=0 tx=0).stop 2024-04-24T23:21:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 msgr2=0x7fdc5c15bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc5c15bd30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc54001870 tx=0x7fdc54001120 comp rx=0 tx=0).stop 2024-04-24T23:21:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 shutdown_connections 2024-04-24T23:21:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc4806be60 0x7fdc4806e320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc5c09c530 0x7fdc5c09c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc5c15dd90 0x7fdc5c138390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 --2- 172.21.15.26:0/799864913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc440039b0 0x7fdc5c15bd30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 >> 172.21.15.26:0/799864913 conn(0x7fdc5c06b540 msgr2=0x7fdc5c06dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 shutdown_connections 2024-04-24T23:21:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.607+0000 7fdc63cee640 1 -- 172.21.15.26:0/799864913 wait complete. 2024-04-24T23:21:29.620 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.722+0000 7f15f359e640 1 Processor -- start 2024-04-24T23:21:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.722+0000 7f15f359e640 1 -- start start 2024-04-24T23:21:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.723+0000 7f15f359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15f40b5260 0x7f15f40d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.723+0000 7f15f359e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f40cfb20 con 0x7f15f40d0930 2024-04-24T23:21:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.723+0000 7f15f359e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f40cfe00 con 0x7f15f4161a80 2024-04-24T23:21:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.723+0000 7f15f359e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f15f40d00e0 con 0x7f15f40b5260 2024-04-24T23:21:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.723+0000 7f15f2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15f40b5260 0x7f15f40d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.723+0000 7f15f2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15f40b5260 0x7f15f40d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58208/0 (socket says 172.21.15.26:58208) 2024-04-24T23:21:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.723+0000 7f15f259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15f40d0930 0x7f15f4161370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42058/0 (socket says 172.21.15.26:42058) 2024-04-24T23:21:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.723+0000 7f15f2d9d640 1 -- 172.21.15.26:0/2904834404 learned_addr learned my addr 172.21.15.26:0/2904834404 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.724+0000 7f15f2d9d640 1 -- 172.21.15.26:0/2904834404 >> v1:172.21.15.29:6789/0 conn(0x7f15f4161a80 legacy=0x7f15f40d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.724+0000 7f15f2d9d640 1 -- 172.21.15.26:0/2904834404 >> v1:172.21.15.26:6789/0 conn(0x7f15f40d0930 legacy=0x7f15f4161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.724+0000 7f15f2d9d640 1 -- 172.21.15.26:0/2904834404 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f40b3a50 con 0x7f15f40b5260 2024-04-24T23:21:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.724+0000 7f15f159a640 1 -- 172.21.15.26:0/2904834404 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 617374984 0 0) 0x7f15f40cfb20 con 0x7f15f40d0930 2024-04-24T23:21:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.724+0000 7f15f2d9d640 1 --2- 172.21.15.26:0/2904834404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 0x7f15f40d1420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009870 tx=0x7f15ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c76ce7e06049675 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.724+0000 7f15f159a640 1 -- 172.21.15.26:0/2904834404 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec037070 con 0x7f15f40b5260 2024-04-24T23:21:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.724+0000 7f15f159a640 1 -- 172.21.15.26:0/2904834404 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ec032440 con 0x7f15f40b5260 2024-04-24T23:21:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.724+0000 7f15f159a640 1 -- 172.21.15.26:0/2904834404 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec032810 con 0x7f15f40b5260 2024-04-24T23:21:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.725+0000 7f15f359e640 1 -- 172.21.15.26:0/2904834404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 msgr2=0x7f15f40d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.725+0000 7f15f359e640 1 --2- 172.21.15.26:0/2904834404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 0x7f15f40d1420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009870 tx=0x7f15ec02a180 comp rx=0 tx=0).stop 2024-04-24T23:21:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.725+0000 7f15f359e640 1 -- 172.21.15.26:0/2904834404 shutdown_connections 2024-04-24T23:21:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.725+0000 7f15f359e640 1 --2- 172.21.15.26:0/2904834404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 0x7f15f40d1420 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.725+0000 7f15f359e640 1 -- 172.21.15.26:0/2904834404 >> 172.21.15.26:0/2904834404 conn(0x7f15f40d3f70 msgr2=0x7f15f40d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.725+0000 7f15f359e640 1 -- 172.21.15.26:0/2904834404 shutdown_connections 2024-04-24T23:21:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.725+0000 7f15f359e640 1 -- 172.21.15.26:0/2904834404 wait complete. 2024-04-24T23:21:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.726+0000 7f15f359e640 1 Processor -- start 2024-04-24T23:21:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.726+0000 7f15f359e640 1 -- start start 2024-04-24T23:21:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 0x7f15f41f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f359e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f40d0930 0x7f15f4150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f359e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f4161a80 0x7f15f414f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f359e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15f40cf740 con 0x7f15f40b5260 2024-04-24T23:21:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f359e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f15f40d3100 con 0x7f15f40d0930 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f359e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f15f40cf8c0 con 0x7f15f4161a80 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 0x7f15f41f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 0x7f15f41f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58222/0 (socket says 172.21.15.26:58222) 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f259c640 1 -- 172.21.15.26:0/3726348058 learned_addr learned my addr 172.21.15.26:0/3726348058 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f2d9d640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f4161a80 0x7f15f414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.727+0000 7f15f1d9b640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f40d0930 0x7f15f4150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15f259c640 1 -- 172.21.15.26:0/3726348058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f4161a80 msgr2=0x7f15f414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15f259c640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f4161a80 0x7f15f414f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15f259c640 1 -- 172.21.15.26:0/3726348058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f40d0930 msgr2=0x7f15f4150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15f259c640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f40d0930 0x7f15f4150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15f259c640 1 -- 172.21.15.26:0/3726348058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f41f3a10 con 0x7f15f40b5260 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15f259c640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 0x7f15f41f34d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c8c0 tx=0x7f15e400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15eaffd640 1 -- 172.21.15.26:0/3726348058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4018070 con 0x7f15f40b5260 2024-04-24T23:21:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15eaffd640 1 -- 172.21.15.26:0/3726348058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e400a640 con 0x7f15f40b5260 2024-04-24T23:21:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15eaffd640 1 -- 172.21.15.26:0/3726348058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e400aa30 con 0x7f15f40b5260 2024-04-24T23:21:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f41542b0 con 0x7f15f40b5260 2024-04-24T23:21:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.728+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f414e3d0 con 0x7f15f40b5260 2024-04-24T23:21:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.729+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f4075610 con 0x7f15f40b5260 2024-04-24T23:21:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.730+0000 7f15eaffd640 1 -- 172.21.15.26:0/3726348058 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15e401b070 con 0x7f15f40b5260 2024-04-24T23:21:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.730+0000 7f15eaffd640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15d406be30 0x7f15d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.730+0000 7f15eaffd640 1 -- 172.21.15.26:0/3726348058 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f15e408a430 con 0x7f15f40b5260 2024-04-24T23:21:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.730+0000 7f15f1d9b640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15d406be30 0x7f15d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.730+0000 7f15f1d9b640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15d406be30 0x7f15d406e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f15f41f7040 tx=0x7f15dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.734+0000 7f15eaffd640 1 -- 172.21.15.26:0/3726348058 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15f41542b0 con 0x7f15f40b5260 2024-04-24T23:21:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.915+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f15f41616f0 con 0x7f15f40b5260 2024-04-24T23:21:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.916+0000 7f15eaffd640 1 -- 172.21.15.26:0/3726348058 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f15e4058820 con 0x7f15f40b5260 2024-04-24T23:21:29.918 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15d406be30 msgr2=0x7f15d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15d406be30 0x7f15d406e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f15f41f7040 tx=0x7f15dc009040 comp rx=0 tx=0).stop 2024-04-24T23:21:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 msgr2=0x7f15f41f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 0x7f15f41f34d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c8c0 tx=0x7f15e400cd80 comp rx=0 tx=0).stop 2024-04-24T23:21:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 shutdown_connections 2024-04-24T23:21:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15d406be30 0x7f15d406e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f4161a80 0x7f15f414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f40d0930 0x7f15f4150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 --2- 172.21.15.26:0/3726348058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f40b5260 0x7f15f41f34d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:29.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 >> 172.21.15.26:0/3726348058 conn(0x7f15f40d3f70 msgr2=0x7f15f40b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:29.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 shutdown_connections 2024-04-24T23:21:29.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:29.918+0000 7f15f359e640 1 -- 172.21.15.26:0/3726348058 wait complete. 2024-04-24T23:21:29.930 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.032+0000 7feaa9ea5640 1 Processor -- start 2024-04-24T23:21:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.032+0000 7feaa9ea5640 1 -- start start 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.033+0000 7feaa9ea5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa415e370 0x7feaa40d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.033+0000 7feaa9ea5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feaa40d4c10 0x7feaa40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.033+0000 7feaa9ea5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa40d2120 con 0x7feaa415e750 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.033+0000 7feaa9ea5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7feaa40d1510 con 0x7feaa40d4c10 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.033+0000 7feaa9ea5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feaa40d1ed0 con 0x7feaa415e370 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.033+0000 7feaa3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feaa40d4c10 0x7feaa40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.033+0000 7feaa3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feaa40d4c10 0x7feaa40cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58238/0 (socket says 172.21.15.26:58238) 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.033+0000 7feaa3fff640 1 -- 172.21.15.26:0/970560576 learned_addr learned my addr 172.21.15.26:0/970560576 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.033+0000 7feaa37fe640 1 --2- 172.21.15.26:0/970560576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa415e370 0x7feaa40d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.034+0000 7feaa3fff640 1 -- 172.21.15.26:0/970560576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa415e370 msgr2=0x7feaa40d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.034+0000 7feaa3fff640 1 --2- 172.21.15.26:0/970560576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa415e370 0x7feaa40d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.034+0000 7feaa3fff640 1 -- 172.21.15.26:0/970560576 >> v1:172.21.15.29:6789/0 conn(0x7feaa415e750 legacy=0x7feaa40b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.034+0000 7feaa3fff640 1 -- 172.21.15.26:0/970560576 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa40b3740 con 0x7feaa40d4c10 2024-04-24T23:21:30.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.034+0000 7feaa3fff640 1 --2- 172.21.15.26:0/970560576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 0x7feaa40cfca0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009e50 tx=0x7fea9c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3688ab8db4fe65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:30.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa27fc640 1 -- 172.21.15.26:0/970560576 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c036070 con 0x7feaa40d4c10 2024-04-24T23:21:30.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa27fc640 1 -- 172.21.15.26:0/970560576 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea9c031600 con 0x7feaa40d4c10 2024-04-24T23:21:30.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa27fc640 1 -- 172.21.15.26:0/970560576 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0319d0 con 0x7feaa40d4c10 2024-04-24T23:21:30.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa9ea5640 1 -- 172.21.15.26:0/970560576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 msgr2=0x7feaa40cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa9ea5640 1 --2- 172.21.15.26:0/970560576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 0x7feaa40cfca0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009e50 tx=0x7fea9c02a710 comp rx=0 tx=0).stop 2024-04-24T23:21:30.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa9ea5640 1 -- 172.21.15.26:0/970560576 shutdown_connections 2024-04-24T23:21:30.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa9ea5640 1 --2- 172.21.15.26:0/970560576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 0x7feaa40cfca0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa9ea5640 1 --2- 172.21.15.26:0/970560576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa415e370 0x7feaa40d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa9ea5640 1 -- 172.21.15.26:0/970560576 >> 172.21.15.26:0/970560576 conn(0x7feaa40d40f0 msgr2=0x7feaa40d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:30.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa9ea5640 1 -- 172.21.15.26:0/970560576 shutdown_connections 2024-04-24T23:21:30.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.035+0000 7feaa9ea5640 1 -- 172.21.15.26:0/970560576 wait complete. 2024-04-24T23:21:30.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.036+0000 7feaa9ea5640 1 Processor -- start 2024-04-24T23:21:30.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.037+0000 7feaa9ea5640 1 -- start start 2024-04-24T23:21:30.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.037+0000 7feaa9ea5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 0x7feaa41576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.037+0000 7feaa9ea5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa415e370 0x7feaa40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.037+0000 7feaa9ea5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa40c8cb0 0x7feaa40c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.037+0000 7feaa9ea5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feaa40d2a60 con 0x7feaa40d4c10 2024-04-24T23:21:30.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.037+0000 7feaa9ea5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feaa40b3d00 con 0x7feaa415e370 2024-04-24T23:21:30.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.037+0000 7feaa9ea5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feaa40d2be0 con 0x7feaa40c8cb0 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.037+0000 7feaa37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 0x7feaa41576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 0x7feaa41576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58244/0 (socket says 172.21.15.26:58244) 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa37fe640 1 -- 172.21.15.26:0/1640217903 learned_addr learned my addr 172.21.15.26:0/1640217903 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa37fe640 1 -- 172.21.15.26:0/1640217903 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa40c8cb0 msgr2=0x7feaa40c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa3fff640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa40c8cb0 0x7feaa40c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa2ffd640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa415e370 0x7feaa40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa37fe640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa40c8cb0 0x7feaa40c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa37fe640 1 -- 172.21.15.26:0/1640217903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa415e370 msgr2=0x7feaa40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa37fe640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa415e370 0x7feaa40cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa37fe640 1 -- 172.21.15.26:0/1640217903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa41f7d50 con 0x7feaa40d4c10 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa37fe640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 0x7feaa41576c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fea9400e9b0 tx=0x7fea9400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7fea8bfff640 1 -- 172.21.15.26:0/1640217903 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94017070 con 0x7feaa40d4c10 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7fea8bfff640 1 -- 172.21.15.26:0/1640217903 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea9400c9a0 con 0x7feaa40d4c10 2024-04-24T23:21:30.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa41f3d30 con 0x7feaa40d4c10 2024-04-24T23:21:30.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7fea8bfff640 1 -- 172.21.15.26:0/1640217903 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9400cd90 con 0x7feaa40d4c10 2024-04-24T23:21:30.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.038+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa41f42f0 con 0x7feaa40d4c10 2024-04-24T23:21:30.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.039+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa4075610 con 0x7feaa40d4c10 2024-04-24T23:21:30.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.040+0000 7fea8bfff640 1 -- 172.21.15.26:0/1640217903 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea9401a070 con 0x7feaa40d4c10 2024-04-24T23:21:30.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.040+0000 7fea8bfff640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea9006be60 0x7fea9006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.040+0000 7fea8bfff640 1 -- 172.21.15.26:0/1640217903 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea9408adc0 con 0x7feaa40d4c10 2024-04-24T23:21:30.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.040+0000 7feaa2ffd640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea9006be60 0x7fea9006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.041+0000 7feaa2ffd640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea9006be60 0x7fea9006e320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feaa4157c00 tx=0x7fea98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:30.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.045+0000 7fea8bfff640 1 -- 172.21.15.26:0/1640217903 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fea94059800 con 0x7feaa40d4c10 2024-04-24T23:21:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.226+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feaa4074bd0 con 0x7feaa40d4c10 2024-04-24T23:21:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.226+0000 7fea8bfff640 1 -- 172.21.15.26:0/1640217903 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fea94059100 con 0x7feaa40d4c10 2024-04-24T23:21:30.228 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:30.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea9006be60 msgr2=0x7fea9006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea9006be60 0x7fea9006e320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feaa4157c00 tx=0x7fea98009040 comp rx=0 tx=0).stop 2024-04-24T23:21:30.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 msgr2=0x7feaa41576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 0x7feaa41576c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fea9400e9b0 tx=0x7fea9400ee70 comp rx=0 tx=0).stop 2024-04-24T23:21:30.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 shutdown_connections 2024-04-24T23:21:30.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea9006be60 0x7fea9006e320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feaa40c8cb0 0x7feaa40c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feaa415e370 0x7feaa40cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 --2- 172.21.15.26:0/1640217903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaa40d4c10 0x7feaa41576c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 >> 172.21.15.26:0/1640217903 conn(0x7feaa40d40f0 msgr2=0x7feaa40d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 shutdown_connections 2024-04-24T23:21:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.228+0000 7feaa9ea5640 1 -- 172.21.15.26:0/1640217903 wait complete. 2024-04-24T23:21:30.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:21:30.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.340+0000 7f7c17b22640 1 Processor -- start 2024-04-24T23:21:30.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.340+0000 7f7c17b22640 1 -- start start 2024-04-24T23:21:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c17b22640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7c100b3ac0 0x7f7c100b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c17b22640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c100b62e0 0x7f7c100b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c17b22640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c100b5840 con 0x7f7c1015e290 2024-04-24T23:21:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c17b22640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7c100b5e50 con 0x7f7c100b62e0 2024-04-24T23:21:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c17b22640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7c100cfb30 con 0x7f7c100b3ac0 2024-04-24T23:21:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c160bf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c100b62e0 0x7f7c100b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c160bf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c100b62e0 0x7f7c100b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58258/0 (socket says 172.21.15.26:58258) 2024-04-24T23:21:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c160bf640 1 -- 172.21.15.26:0/1401724751 learned_addr learned my addr 172.21.15.26:0/1401724751 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c150bd640 1 --2- 172.21.15.26:0/1401724751 >> v2:172.21.15.29:3300/0 conn(0x7f7c100b3ac0 0x7f7c100b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c160bf640 1 -- 172.21.15.26:0/1401724751 >> v2:172.21.15.29:3300/0 conn(0x7f7c100b3ac0 msgr2=0x7f7c100b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c160bf640 1 --2- 172.21.15.26:0/1401724751 >> v2:172.21.15.29:3300/0 conn(0x7f7c100b3ac0 0x7f7c100b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.341+0000 7f7c160bf640 1 -- 172.21.15.26:0/1401724751 >> v1:172.21.15.29:6789/0 conn(0x7f7c1015e290 legacy=0x7f7c100d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.342+0000 7f7c160bf640 1 -- 172.21.15.26:0/1401724751 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c100cfe10 con 0x7f7c100b62e0 2024-04-24T23:21:30.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.342+0000 7f7c160bf640 1 --2- 172.21.15.26:0/1401724751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 0x7f7c100b6700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7c100c5a30 tx=0x7f7c0002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267e5ad3c901266b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:30.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.342+0000 7f7c148bc640 1 -- 172.21.15.26:0/1401724751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00036070 con 0x7f7c100b62e0 2024-04-24T23:21:30.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.342+0000 7f7c148bc640 1 -- 172.21.15.26:0/1401724751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c00031600 con 0x7f7c100b62e0 2024-04-24T23:21:30.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.342+0000 7f7c148bc640 1 -- 172.21.15.26:0/1401724751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c000319d0 con 0x7f7c100b62e0 2024-04-24T23:21:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.343+0000 7f7c17b22640 1 -- 172.21.15.26:0/1401724751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 msgr2=0x7f7c100b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.343+0000 7f7c17b22640 1 --2- 172.21.15.26:0/1401724751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 0x7f7c100b6700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7c100c5a30 tx=0x7f7c0002a6c0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.343+0000 7f7c17b22640 1 -- 172.21.15.26:0/1401724751 shutdown_connections 2024-04-24T23:21:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.343+0000 7f7c17b22640 1 --2- 172.21.15.26:0/1401724751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 0x7f7c100b6700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.343+0000 7f7c17b22640 1 --2- 172.21.15.26:0/1401724751 >> v2:172.21.15.29:3300/0 conn(0x7f7c100b3ac0 0x7f7c100b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.343+0000 7f7c17b22640 1 -- 172.21.15.26:0/1401724751 >> 172.21.15.26:0/1401724751 conn(0x7f7c1006b120 msgr2=0x7f7c1006b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.343+0000 7f7c17b22640 1 -- 172.21.15.26:0/1401724751 shutdown_connections 2024-04-24T23:21:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.343+0000 7f7c17b22640 1 -- 172.21.15.26:0/1401724751 wait complete. 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.344+0000 7f7c17b22640 1 Processor -- start 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.344+0000 7f7c17b22640 1 -- start start 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c17b22640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7c100b3ac0 0x7f7c101f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c17b22640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 0x7f7c101f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c17b22640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1015e290 0x7f7c100b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c17b22640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1015e670 con 0x7f7c100b62e0 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c17b22640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c100b5ae0 con 0x7f7c1015e290 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c17b22640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1006dfd0 con 0x7f7c100b3ac0 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c150bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 0x7f7c101f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c150bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 0x7f7c101f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58272/0 (socket says 172.21.15.26:58272) 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c150bd640 1 -- 172.21.15.26:0/3429310376 learned_addr learned my addr 172.21.15.26:0/3429310376 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c158be640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7c100b3ac0 0x7f7c101f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c160bf640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1015e290 0x7f7c100b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c150bd640 1 -- 172.21.15.26:0/3429310376 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7c100b3ac0 msgr2=0x7f7c101f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c150bd640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7c100b3ac0 0x7f7c101f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c150bd640 1 -- 172.21.15.26:0/3429310376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1015e290 msgr2=0x7f7c100b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c150bd640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1015e290 0x7f7c100b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c150bd640 1 -- 172.21.15.26:0/3429310376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c100c7800 con 0x7f7c100b62e0 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.345+0000 7f7c160bf640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1015e290 0x7f7c100b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.346+0000 7f7c150bd640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 0x7f7c101f6720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d800 tx=0x7f7c0400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.346+0000 7f7c0e7fc640 1 -- 172.21.15.26:0/3429310376 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400b6c0 con 0x7f7c100b62e0 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.346+0000 7f7c0e7fc640 1 -- 172.21.15.26:0/3429310376 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c04019070 con 0x7f7c100b62e0 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.346+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c100ca710 con 0x7f7c100b62e0 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.346+0000 7f7c0e7fc640 1 -- 172.21.15.26:0/3429310376 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400ba80 con 0x7f7c100b62e0 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.346+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c100cacd0 con 0x7f7c100b62e0 2024-04-24T23:21:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.347+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c100694c0 con 0x7f7c100b62e0 2024-04-24T23:21:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.347+0000 7f7c0e7fc640 1 -- 172.21.15.26:0/3429310376 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c0400bc20 con 0x7f7c100b62e0 2024-04-24T23:21:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.348+0000 7f7c0e7fc640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7bfc06be30 0x7f7bfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.348+0000 7f7c0e7fc640 1 -- 172.21.15.26:0/3429310376 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c0408b1f0 con 0x7f7c100b62e0 2024-04-24T23:21:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.348+0000 7f7c158be640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7bfc06be30 0x7f7bfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.348+0000 7f7c158be640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7bfc06be30 0x7f7bfc06e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006fd0 tx=0x7f7c08009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.352+0000 7f7c0e7fc640 1 -- 172.21.15.26:0/3429310376 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7c04059c10 con 0x7f7c100b62e0 2024-04-24T23:21:30.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.549+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7c101f6c60 con 0x7f7c100b62e0 2024-04-24T23:21:30.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.549+0000 7f7c0e7fc640 1 -- 172.21.15.26:0/3429310376 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f7c04059510 con 0x7f7c100b62e0 2024-04-24T23:21:30.551 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:21:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7bfc06be30 msgr2=0x7f7bfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7bfc06be30 0x7f7bfc06e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006fd0 tx=0x7f7c08009060 comp rx=0 tx=0).stop 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 msgr2=0x7f7c101f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 0x7f7c101f6720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d800 tx=0x7f7c0400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 shutdown_connections 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7bfc06be30 0x7f7bfc06e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c1015e290 0x7f7c100b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100b62e0 0x7f7c101f6720 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 --2- 172.21.15.26:0/3429310376 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7c100b3ac0 0x7f7c101f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.551+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 >> 172.21.15.26:0/3429310376 conn(0x7f7c1006b120 msgr2=0x7f7c100cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.552+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 shutdown_connections 2024-04-24T23:21:30.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.552+0000 7f7c17b22640 1 -- 172.21.15.26:0/3429310376 wait complete. 2024-04-24T23:21:30.564 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:21:30.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:30.564 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-24T23:21:30.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:21:30.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:30.596 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:21:30.664 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:21:30.664 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:30.681 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:21:30.681 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:30.737 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:21:30.737 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:30.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:30.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.931+0000 7fb6b0b63640 1 Processor -- start 2024-04-24T23:21:30.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.931+0000 7fb6b0b63640 1 -- start start 2024-04-24T23:21:30.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6b0b63640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb6ac145910 0x7fb6ac06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6b0b63640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb6ac0cd660 0x7fb6ac0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6b0b63640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6ac0cfbc0 con 0x7fb6ac06d350 2024-04-24T23:21:30.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6b0b63640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb6ac145cf0 con 0x7fb6ac145910 2024-04-24T23:21:30.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6b0b63640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ac06d7f0 con 0x7fb6ac0cd660 2024-04-24T23:21:30.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6a9d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb6ac06d350 0x7fb6ac15d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42066/0 (socket says 172.21.15.26:42066) 2024-04-24T23:21:30.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6a9d9b640 1 -- 172.21.15.26:0/1095327272 learned_addr learned my addr 172.21.15.26:0/1095327272 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:30.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6aad9d640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb6ac0cd660 0x7fb6ac0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6aa59c640 1 --2- 172.21.15.26:0/1095327272 >> v2:172.21.15.29:3300/0 conn(0x7fb6ac145910 0x7fb6ac06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 56294602 0 0) 0x7fb6ac0cfbc0 con 0x7fb6ac06d350 2024-04-24T23:21:30.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.932+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb698003520 con 0x7fb6ac06d350 2024-04-24T23:21:30.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 857051367 0 0) 0x7fb698003520 con 0x7fb6ac06d350 2024-04-24T23:21:30.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb6ac0cd660 msgr2=0x7fb6ac0cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:30.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb6ac0cd660 0x7fb6ac0cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 >> v2:172.21.15.29:3300/0 conn(0x7fb6ac145910 msgr2=0x7fb6ac06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 --2- 172.21.15.26:0/1095327272 >> v2:172.21.15.29:3300/0 conn(0x7fb6ac145910 0x7fb6ac06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac06dad0 con 0x7fb6ac06d350 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fb694001670 con 0x7fb6ac06d350 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 >> v1:172.21.15.26:6789/0 conn(0x7fb6ac06d350 legacy=0x7fb6ac15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6980039b0 0x7fb698003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb698004800 0x7fb698004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb6ac0cd660 0x7fb6980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac145cf0 con 0x7fb698004800 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac06d7f0 con 0x7fb6980039b0 2024-04-24T23:21:30.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb69800fdd0 con 0x7fb6ac0cd660 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a9d9b640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb698004800 0x7fb698004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6aa59c640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6980039b0 0x7fb698003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6aad9d640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb6ac0cd660 0x7fb6980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a9d9b640 1 -- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb6ac0cd660 msgr2=0x7fb6980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a9d9b640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb6ac0cd660 0x7fb6980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a9d9b640 1 -- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6980039b0 msgr2=0x7fb698003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a9d9b640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6980039b0 0x7fb698003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.933+0000 7fb6a9d9b640 1 -- 172.21.15.26:0/1095327272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb698003520 con 0x7fb698004800 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.934+0000 7fb6a9d9b640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb698004800 0x7fb698004c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c240 tx=0x7fb698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43400bf7b6a4679f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.934+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694007d10 con 0x7fb698004800 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.934+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb694005ad0 con 0x7fb698004800 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.934+0000 7fb6a959a640 1 -- 172.21.15.26:0/1095327272 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694007070 con 0x7fb698004800 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.935+0000 7fb6b0b63640 1 -- 172.21.15.26:0/1095327272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb698004800 msgr2=0x7fb698004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.935+0000 7fb6b0b63640 1 --2- 172.21.15.26:0/1095327272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb698004800 0x7fb698004c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c240 tx=0x7fb698005830 comp rx=0 tx=0).stop 2024-04-24T23:21:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.935+0000 7fb6aad9d640 1 -- 172.21.15.26:0/1095327272 reap_dead start 2024-04-24T23:21:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.935+0000 7fb6b0b63640 1 -- 172.21.15.26:0/1095327272 shutdown_connections 2024-04-24T23:21:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.935+0000 7fb6b0b63640 1 -- 172.21.15.26:0/1095327272 >> 172.21.15.26:0/1095327272 conn(0x7fb6ac06b300 msgr2=0x7fb6ac06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.935+0000 7fb6b0b63640 1 -- 172.21.15.26:0/1095327272 shutdown_connections 2024-04-24T23:21:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.935+0000 7fb6b0b63640 1 -- 172.21.15.26:0/1095327272 wait complete. 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.936+0000 7fb6b0b63640 1 Processor -- start 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.936+0000 7fb6b0b63640 1 -- start start 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6b0b63640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb698004800 0x7fb6ac136620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6b0b63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6ac14dff0 0x7fb6ac1380f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6b0b63640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac138630 0x7fb6ac143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6b0b63640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac06e370 con 0x7fb6ac138630 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6b0b63640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac06e090 con 0x7fb6ac14dff0 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6b0b63640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ac0cdaf0 con 0x7fb698004800 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac138630 0x7fb6ac143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac138630 0x7fb6ac143830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58290/0 (socket says 172.21.15.26:58290) 2024-04-24T23:21:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6aad9d640 1 -- 172.21.15.26:0/3629445550 learned_addr learned my addr 172.21.15.26:0/3629445550 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6a9d9b640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6ac14dff0 0x7fb6ac1380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6aa59c640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb698004800 0x7fb6ac136620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6aad9d640 1 -- 172.21.15.26:0/3629445550 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb698004800 msgr2=0x7fb6ac136620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6aad9d640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb698004800 0x7fb6ac136620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6aad9d640 1 -- 172.21.15.26:0/3629445550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6ac14dff0 msgr2=0x7fb6ac1380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6aad9d640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6ac14dff0 0x7fb6ac1380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.937+0000 7fb6aad9d640 1 -- 172.21.15.26:0/3629445550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac136b60 con 0x7fb6ac138630 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.938+0000 7fb6aad9d640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac138630 0x7fb6ac143830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0710b0 tx=0x7fb69c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.938+0000 7fb6a2ffd640 1 -- 172.21.15.26:0/3629445550 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00fdb0 con 0x7fb6ac138630 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.938+0000 7fb6a2ffd640 1 -- 172.21.15.26:0/3629445550 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69c01d070 con 0x7fb6ac138630 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.938+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ac138b40 con 0x7fb6ac138630 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.938+0000 7fb6a2ffd640 1 -- 172.21.15.26:0/3629445550 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c015670 con 0x7fb6ac138630 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.938+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ac13fcd0 con 0x7fb6ac138630 2024-04-24T23:21:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.939+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ac13f7e0 con 0x7fb6ac138630 2024-04-24T23:21:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.939+0000 7fb6a2ffd640 1 -- 172.21.15.26:0/3629445550 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb69c015810 con 0x7fb6ac138630 2024-04-24T23:21:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.940+0000 7fb6a2ffd640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb68c06be30 0x7fb68c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.940+0000 7fb6a2ffd640 1 -- 172.21.15.26:0/3629445550 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb69c08b6b0 con 0x7fb6ac138630 2024-04-24T23:21:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.940+0000 7fb6aa59c640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb68c06be30 0x7fb68c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.940+0000 7fb6aa59c640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb68c06be30 0x7fb68c06e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009b00 tx=0x7fb6a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:30.944+0000 7fb6a2ffd640 1 -- 172.21.15.26:0/3629445550 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb6ac136b60 con 0x7fb6ac138630 2024-04-24T23:21:31.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.129+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6ac15cb10 con 0x7fb6ac138630 2024-04-24T23:21:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.130+0000 7fb6a2ffd640 1 -- 172.21.15.26:0/3629445550 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fb69c010040 con 0x7fb6ac138630 2024-04-24T23:21:31.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:31.132 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb68c06be30 msgr2=0x7fb68c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb68c06be30 0x7fb68c06e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009b00 tx=0x7fb6a4008040 comp rx=0 tx=0).stop 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac138630 msgr2=0x7fb6ac143830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac138630 0x7fb6ac143830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0710b0 tx=0x7fb69c00f580 comp rx=0 tx=0).stop 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 shutdown_connections 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb68c06be30 0x7fb68c06e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac138630 0x7fb6ac143830 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6ac14dff0 0x7fb6ac1380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 --2- 172.21.15.26:0/3629445550 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb698004800 0x7fb6ac136620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.132+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 >> 172.21.15.26:0/3629445550 conn(0x7fb6ac06b300 msgr2=0x7fb6ac15ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.133+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 shutdown_connections 2024-04-24T23:21:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.133+0000 7fb6b0b63640 1 -- 172.21.15.26:0/3629445550 wait complete. 2024-04-24T23:21:31.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:31.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.248+0000 7f14aa910640 1 Processor -- start 2024-04-24T23:21:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.248+0000 7f14aa910640 1 -- start start 2024-04-24T23:21:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14aa910640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f14a415c270 0x7f14a40cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14aa910640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f14a415c650 0x7f14a40d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14aa910640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a40cf050 con 0x7f14a40b2950 2024-04-24T23:21:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14aa910640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f14a40cee00 con 0x7f14a415c650 2024-04-24T23:21:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14aa910640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f14a40d28e0 con 0x7f14a415c270 2024-04-24T23:21:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14a37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f14a415c650 0x7f14a40d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14a37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f14a415c650 0x7f14a40d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58306/0 (socket says 172.21.15.26:58306) 2024-04-24T23:21:31.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14a37fe640 1 -- 172.21.15.26:0/411597993 learned_addr learned my addr 172.21.15.26:0/411597993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:31.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14a3fff640 1 --2- 172.21.15.26:0/411597993 >> v2:172.21.15.29:3300/0 conn(0x7f14a415c270 0x7f14a40cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14a37fe640 1 -- 172.21.15.26:0/411597993 >> v2:172.21.15.29:3300/0 conn(0x7f14a415c270 msgr2=0x7f14a40cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:31.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14a37fe640 1 --2- 172.21.15.26:0/411597993 >> v2:172.21.15.29:3300/0 conn(0x7f14a415c270 0x7f14a40cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14a37fe640 1 -- 172.21.15.26:0/411597993 >> v1:172.21.15.29:6789/0 conn(0x7f14a40b2950 legacy=0x7f14a40c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.249+0000 7f14a37fe640 1 -- 172.21.15.26:0/411597993 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a40cdd60 con 0x7f14a415c650 2024-04-24T23:21:31.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.250+0000 7f14a37fe640 1 --2- 172.21.15.26:0/411597993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a415c650 0x7f14a40d03a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14a40d1d20 tx=0x7f149802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e2f9850d4d553 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:31.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.250+0000 7f14a2ffd640 1 -- 172.21.15.26:0/411597993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498037070 con 0x7f14a415c650 2024-04-24T23:21:31.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.250+0000 7f14a2ffd640 1 -- 172.21.15.26:0/411597993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1498032440 con 0x7f14a415c650 2024-04-24T23:21:31.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.250+0000 7f14a2ffd640 1 -- 172.21.15.26:0/411597993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498032810 con 0x7f14a415c650 2024-04-24T23:21:31.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.251+0000 7f14aa910640 1 -- 172.21.15.26:0/411597993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a415c650 msgr2=0x7f14a40d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.251+0000 7f14aa910640 1 --2- 172.21.15.26:0/411597993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a415c650 0x7f14a40d03a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14a40d1d20 tx=0x7f149802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.251+0000 7f14aa910640 1 -- 172.21.15.26:0/411597993 shutdown_connections 2024-04-24T23:21:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.251+0000 7f14aa910640 1 --2- 172.21.15.26:0/411597993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a415c650 0x7f14a40d03a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.251+0000 7f14aa910640 1 --2- 172.21.15.26:0/411597993 >> v2:172.21.15.29:3300/0 conn(0x7f14a415c270 0x7f14a40cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.251+0000 7f14aa910640 1 -- 172.21.15.26:0/411597993 >> 172.21.15.26:0/411597993 conn(0x7f14a40d1dc0 msgr2=0x7f14a40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.251+0000 7f14aa910640 1 -- 172.21.15.26:0/411597993 shutdown_connections 2024-04-24T23:21:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.251+0000 7f14aa910640 1 -- 172.21.15.26:0/411597993 wait complete. 2024-04-24T23:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.252+0000 7f14aa910640 1 Processor -- start 2024-04-24T23:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.252+0000 7f14aa910640 1 -- start start 2024-04-24T23:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14aa910640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a40b2950 0x7f14a41f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14aa910640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14a415c270 0x7f14a4158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14aa910640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f14a415c650 0x7f14a40c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14aa910640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14a40b2e80 con 0x7f14a40b2950 2024-04-24T23:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14aa910640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14a40cd9e0 con 0x7f14a415c270 2024-04-24T23:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14aa910640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f14a40b3000 con 0x7f14a415c650 2024-04-24T23:21:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a40b2950 0x7f14a41f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a40b2950 0x7f14a41f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58318/0 (socket says 172.21.15.26:58318) 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a3fff640 1 -- 172.21.15.26:0/676771178 learned_addr learned my addr 172.21.15.26:0/676771178 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a37fe640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14a415c270 0x7f14a4158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a8ead640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f14a415c650 0x7f14a40c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a3fff640 1 -- 172.21.15.26:0/676771178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f14a415c650 msgr2=0x7f14a40c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a3fff640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f14a415c650 0x7f14a40c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a3fff640 1 -- 172.21.15.26:0/676771178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14a415c270 msgr2=0x7f14a4158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a3fff640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14a415c270 0x7f14a4158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a3fff640 1 -- 172.21.15.26:0/676771178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a40b4e60 con 0x7f14a40b2950 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.253+0000 7f14a37fe640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14a415c270 0x7f14a4158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.254+0000 7f14a3fff640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a40b2950 0x7f14a41f3be0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f149400e870 tx=0x7f149400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.254+0000 7f14a0ff9640 1 -- 172.21.15.26:0/676771178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494017070 con 0x7f14a40b2950 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.254+0000 7f14a0ff9640 1 -- 172.21.15.26:0/676771178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149400c8e0 con 0x7f14a40b2950 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.254+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a40b5140 con 0x7f14a40b2950 2024-04-24T23:21:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.254+0000 7f14a0ff9640 1 -- 172.21.15.26:0/676771178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149400ccd0 con 0x7f14a40b2950 2024-04-24T23:21:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.254+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a4158990 con 0x7f14a40b2950 2024-04-24T23:21:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.255+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a4075610 con 0x7f14a40b2950 2024-04-24T23:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.256+0000 7f14a0ff9640 1 -- 172.21.15.26:0/676771178 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f149401a070 con 0x7f14a40b2950 2024-04-24T23:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.256+0000 7f14a0ff9640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f149006be60 0x7f149006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.256+0000 7f14a0ff9640 1 -- 172.21.15.26:0/676771178 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f149408afe0 con 0x7f14a40b2950 2024-04-24T23:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.256+0000 7f14a37fe640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f149006be60 0x7f149006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.256+0000 7f14a37fe640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f149006be60 0x7f149006e320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1498034000 tx=0x7f149802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.260+0000 7f14a0ff9640 1 -- 172.21.15.26:0/676771178 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f14940599e0 con 0x7f14a40b2950 2024-04-24T23:21:31.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.446+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f14a40b4110 con 0x7f14a40b2950 2024-04-24T23:21:31.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.446+0000 7f14a0ff9640 1 -- 172.21.15.26:0/676771178 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f14940592e0 con 0x7f14a40b2950 2024-04-24T23:21:31.448 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:31.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f149006be60 msgr2=0x7f149006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f149006be60 0x7f149006e320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1498034000 tx=0x7f149802e000 comp rx=0 tx=0).stop 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a40b2950 msgr2=0x7f14a41f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a40b2950 0x7f14a41f3be0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f149400e870 tx=0x7f149400ed30 comp rx=0 tx=0).stop 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 shutdown_connections 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f149006be60 0x7f149006e320 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f14a415c650 0x7f14a40c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14a415c270 0x7f14a4158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 --2- 172.21.15.26:0/676771178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a40b2950 0x7f14a41f3be0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.448+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 >> 172.21.15.26:0/676771178 conn(0x7f14a40d1dc0 msgr2=0x7f14a40d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.449+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 shutdown_connections 2024-04-24T23:21:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.449+0000 7f14aa910640 1 -- 172.21.15.26:0/676771178 wait complete. 2024-04-24T23:21:31.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:31.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.563+0000 7f2955563640 1 Processor -- start 2024-04-24T23:21:31.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.563+0000 7f2955563640 1 -- start start 2024-04-24T23:21:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f2955563640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f295006d5a0 0x7f29500b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f2955563640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f2950150e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f2955563640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2950070f30 con 0x7f295015e120 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f2955563640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f295006dcf0 con 0x7f295006d5a0 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f2955563640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f295006dfd0 con 0x7f29500b3d20 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f294f7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f2950150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f294e7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f295006d5a0 0x7f29500b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f294e7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f295006d5a0 0x7f29500b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:41252/0 (socket says 172.21.15.26:41252) 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f294f7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f2950150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:60788/0 (socket says 172.21.15.26:60788) 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f294e7fc640 1 -- 172.21.15.26:0/2684028307 learned_addr learned my addr 172.21.15.26:0/2684028307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.564+0000 7f294effd640 1 --1- 172.21.15.26:0/2684028307 >> v1:172.21.15.29:6789/0 conn(0x7f295015e120 0x7f295015e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:56434/0 (socket says 172.21.15.26:56434) 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.565+0000 7f294f7fe640 1 -- 172.21.15.26:0/2684028307 >> v2:172.21.15.29:3300/0 conn(0x7f295006d5a0 msgr2=0x7f29500b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.565+0000 7f294f7fe640 1 --2- 172.21.15.26:0/2684028307 >> v2:172.21.15.29:3300/0 conn(0x7f295006d5a0 0x7f29500b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.565+0000 7f294f7fe640 1 -- 172.21.15.26:0/2684028307 >> v1:172.21.15.29:6789/0 conn(0x7f295015e120 legacy=0x7f295015e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.565+0000 7f294f7fe640 1 -- 172.21.15.26:0/2684028307 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295006e2b0 con 0x7f29500b3d20 2024-04-24T23:21:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.565+0000 7f294dffb640 1 -- 172.21.15.26:0/2684028307 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2988221043 0 0) 0x7f2950070f30 con 0x7f295015e120 2024-04-24T23:21:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.565+0000 7f294e7fc640 1 --2- 172.21.15.26:0/2684028307 >> v2:172.21.15.29:3300/0 conn(0x7f295006d5a0 0x7f29500b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f294f7fe640 1 --2- 172.21.15.26:0/2684028307 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f2950150e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2948009d90 tx=0x7f294802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37a87a114d55cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f294dffb640 1 -- 172.21.15.26:0/2684028307 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948007740 con 0x7f29500b3d20 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f294dffb640 1 -- 172.21.15.26:0/2684028307 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29480078e0 con 0x7f29500b3d20 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f294dffb640 1 -- 172.21.15.26:0/2684028307 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948007cb0 con 0x7f29500b3d20 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f2955563640 1 -- 172.21.15.26:0/2684028307 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 msgr2=0x7f2950150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f2955563640 1 --2- 172.21.15.26:0/2684028307 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f2950150e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2948009d90 tx=0x7f294802a660 comp rx=0 tx=0).stop 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f2955563640 1 -- 172.21.15.26:0/2684028307 shutdown_connections 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f2955563640 1 --2- 172.21.15.26:0/2684028307 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f2950150e60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f2955563640 1 --2- 172.21.15.26:0/2684028307 >> v2:172.21.15.29:3300/0 conn(0x7f295006d5a0 0x7f29500b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f2955563640 1 -- 172.21.15.26:0/2684028307 >> 172.21.15.26:0/2684028307 conn(0x7f295006b540 msgr2=0x7f295006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f2955563640 1 -- 172.21.15.26:0/2684028307 shutdown_connections 2024-04-24T23:21:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.566+0000 7f2955563640 1 -- 172.21.15.26:0/2684028307 wait complete. 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.568+0000 7f2955563640 1 Processor -- start 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.568+0000 7f2955563640 1 -- start start 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.568+0000 7f2955563640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f295006d5a0 0x7f2950154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.568+0000 7f2955563640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f29501f1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.568+0000 7f2955563640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015e120 0x7f295009c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.568+0000 7f2955563640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f29501513a0 con 0x7f295006d5a0 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.568+0000 7f2955563640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f295006da10 con 0x7f295015e120 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.568+0000 7f2955563640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2950151520 con 0x7f29500b3d20 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f295006d5a0 0x7f2950154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f295006d5a0 0x7f2950154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58332/0 (socket says 172.21.15.26:58332) 2024-04-24T23:21:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294effd640 1 -- 172.21.15.26:0/242343634 learned_addr learned my addr 172.21.15.26:0/242343634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294f7fe640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015e120 0x7f295009c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294e7fc640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f29501f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294effd640 1 -- 172.21.15.26:0/242343634 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 msgr2=0x7f29501f1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294effd640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f29501f1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294effd640 1 -- 172.21.15.26:0/242343634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015e120 msgr2=0x7f295009c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294effd640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015e120 0x7f295009c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294effd640 1 -- 172.21.15.26:0/242343634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2950151a30 con 0x7f295006d5a0 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f294effd640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f295006d5a0 0x7f2950154360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f294000c460 tx=0x7f294000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f29377fe640 1 -- 172.21.15.26:0/242343634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940018070 con 0x7f295006d5a0 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f29377fe640 1 -- 172.21.15.26:0/242343634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2940007500 con 0x7f295006d5a0 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2950151d10 con 0x7f295006d5a0 2024-04-24T23:21:31.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f29377fe640 1 -- 172.21.15.26:0/242343634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940004a20 con 0x7f295006d5a0 2024-04-24T23:21:31.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.569+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29501522d0 con 0x7f295006d5a0 2024-04-24T23:21:31.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.571+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2950075610 con 0x7f295006d5a0 2024-04-24T23:21:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.571+0000 7f29377fe640 1 -- 172.21.15.26:0/242343634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f294001b070 con 0x7f295006d5a0 2024-04-24T23:21:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.571+0000 7f29377fe640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f293c06be30 0x7f293c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.571+0000 7f29377fe640 1 -- 172.21.15.26:0/242343634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f294008a080 con 0x7f295006d5a0 2024-04-24T23:21:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.571+0000 7f294e7fc640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f293c06be30 0x7f293c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.572+0000 7f294e7fc640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f293c06be30 0x7f293c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2950150b60 tx=0x7f2944008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.576+0000 7f29377fe640 1 -- 172.21.15.26:0/242343634 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2940058ac0 con 0x7f295006d5a0 2024-04-24T23:21:31.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.760+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29500b3980 con 0x7f295006d5a0 2024-04-24T23:21:31.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.761+0000 7f29377fe640 1 -- 172.21.15.26:0/242343634 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f29400583c0 con 0x7f295006d5a0 2024-04-24T23:21:31.763 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f293c06be30 msgr2=0x7f293c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f293c06be30 0x7f293c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2950150b60 tx=0x7f2944008040 comp rx=0 tx=0).stop 2024-04-24T23:21:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f295006d5a0 msgr2=0x7f2950154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f295006d5a0 0x7f2950154360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f294000c460 tx=0x7f294000c920 comp rx=0 tx=0).stop 2024-04-24T23:21:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 shutdown_connections 2024-04-24T23:21:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f293c06be30 0x7f293c06e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f295015e120 0x7f295009c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f29500b3d20 0x7f29501f1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 --2- 172.21.15.26:0/242343634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f295006d5a0 0x7f2950154360 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 >> 172.21.15.26:0/242343634 conn(0x7f295006b540 msgr2=0x7f295015e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:31.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 shutdown_connections 2024-04-24T23:21:31.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.763+0000 7f2955563640 1 -- 172.21.15.26:0/242343634 wait complete. 2024-04-24T23:21:31.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.877+0000 7f835b8bd640 1 Processor -- start 2024-04-24T23:21:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.878+0000 7f835b8bd640 1 -- start start 2024-04-24T23:21:31.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.878+0000 7f835b8bd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f835415e370 0x7f83540d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.878+0000 7f835b8bd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f835415e750 0x7f83540d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.878+0000 7f835b8bd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83540d2120 con 0x7f83540d4c10 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.878+0000 7f835b8bd640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f83540d1510 con 0x7f835415e750 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.878+0000 7f835b8bd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f83540d1ed0 con 0x7f835415e370 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.879+0000 7f8358e58640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f835415e750 0x7f83540d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.879+0000 7f8358e58640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f835415e750 0x7f83540d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58336/0 (socket says 172.21.15.26:58336) 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.879+0000 7f8358e58640 1 -- 172.21.15.26:0/3296656558 learned_addr learned my addr 172.21.15.26:0/3296656558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.879+0000 7f8359659640 1 --2- 172.21.15.26:0/3296656558 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f835415e370 0x7f83540d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.879+0000 7f8358e58640 1 -- 172.21.15.26:0/3296656558 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f835415e370 msgr2=0x7f83540d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.879+0000 7f8358e58640 1 --2- 172.21.15.26:0/3296656558 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f835415e370 0x7f83540d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.879+0000 7f8358e58640 1 -- 172.21.15.26:0/3296656558 >> v1:172.21.15.29:6789/0 conn(0x7f83540d4c10 legacy=0x7f83540b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.879+0000 7f8358e58640 1 -- 172.21.15.26:0/3296656558 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83540b3740 con 0x7f835415e750 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f8358e58640 1 --2- 172.21.15.26:0/3296656558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f835415e750 0x7f83540d2a60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8348009870 tx=0x7f834802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1297c9298bd0b738 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f8353fff640 1 -- 172.21.15.26:0/3296656558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348037070 con 0x7f835415e750 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f8353fff640 1 -- 172.21.15.26:0/3296656558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348032440 con 0x7f835415e750 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f8353fff640 1 -- 172.21.15.26:0/3296656558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348032810 con 0x7f835415e750 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f835b8bd640 1 -- 172.21.15.26:0/3296656558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f835415e750 msgr2=0x7f83540d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f835b8bd640 1 --2- 172.21.15.26:0/3296656558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f835415e750 0x7f83540d2a60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8348009870 tx=0x7f834802a270 comp rx=0 tx=0).stop 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f835b8bd640 1 -- 172.21.15.26:0/3296656558 shutdown_connections 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f835b8bd640 1 --2- 172.21.15.26:0/3296656558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f835415e750 0x7f83540d2a60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f835b8bd640 1 --2- 172.21.15.26:0/3296656558 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f835415e370 0x7f83540d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.880+0000 7f835b8bd640 1 -- 172.21.15.26:0/3296656558 >> 172.21.15.26:0/3296656558 conn(0x7f83540d40f0 msgr2=0x7f83540d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.881+0000 7f835b8bd640 1 -- 172.21.15.26:0/3296656558 shutdown_connections 2024-04-24T23:21:31.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.881+0000 7f835b8bd640 1 -- 172.21.15.26:0/3296656558 wait complete. 2024-04-24T23:21:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.882+0000 7f835b8bd640 1 Processor -- start 2024-04-24T23:21:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.882+0000 7f835b8bd640 1 -- start start 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.882+0000 7f835b8bd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83540d4c10 0x7f8354152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.882+0000 7f835b8bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f835415e370 0x7f8354153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.882+0000 7f835b8bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83540c9d00 0x7f83540ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.882+0000 7f835b8bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83540cfca0 con 0x7f83540c9d00 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.882+0000 7f835b8bd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83540b3d00 con 0x7f835415e370 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.882+0000 7f835b8bd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f83540cfe20 con 0x7f83540d4c10 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359e5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83540c9d00 0x7f83540ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359e5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83540c9d00 0x7f83540ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58348/0 (socket says 172.21.15.26:58348) 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359e5a640 1 -- 172.21.15.26:0/398834736 learned_addr learned my addr 172.21.15.26:0/398834736 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8358e58640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f835415e370 0x7f8354153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359659640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83540d4c10 0x7f8354152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359e5a640 1 -- 172.21.15.26:0/398834736 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83540d4c10 msgr2=0x7f8354152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359e5a640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83540d4c10 0x7f8354152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359e5a640 1 -- 172.21.15.26:0/398834736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f835415e370 msgr2=0x7f8354153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:31.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359e5a640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f835415e370 0x7f8354153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:31.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359e5a640 1 -- 172.21.15.26:0/398834736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83540cab30 con 0x7f83540c9d00 2024-04-24T23:21:31.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8359e5a640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83540c9d00 0x7f83540ca190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f834400c740 tx=0x7f834400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:31.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8351ffb640 1 -- 172.21.15.26:0/398834736 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344016020 con 0x7f83540c9d00 2024-04-24T23:21:31.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8351ffb640 1 -- 172.21.15.26:0/398834736 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834400ce50 con 0x7f83540c9d00 2024-04-24T23:21:31.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83540cd6a0 con 0x7f83540c9d00 2024-04-24T23:21:31.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f8351ffb640 1 -- 172.21.15.26:0/398834736 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344003970 con 0x7f83540c9d00 2024-04-24T23:21:31.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.883+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83540cdc60 con 0x7f83540c9d00 2024-04-24T23:21:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.884+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83540d3530 con 0x7f83540c9d00 2024-04-24T23:21:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.885+0000 7f8351ffb640 1 -- 172.21.15.26:0/398834736 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8344003b10 con 0x7f83540c9d00 2024-04-24T23:21:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.885+0000 7f8351ffb640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f834006be60 0x7f834006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.885+0000 7f8351ffb640 1 -- 172.21.15.26:0/398834736 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f834408a910 con 0x7f83540c9d00 2024-04-24T23:21:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.885+0000 7f8359659640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f834006be60 0x7f834006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.886+0000 7f8359659640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f834006be60 0x7f834006e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f834c006fd0 tx=0x7f834c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:31.890+0000 7f8351ffb640 1 -- 172.21.15.26:0/398834736 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83540d3530 con 0x7f83540c9d00 2024-04-24T23:21:32.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.072+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f835415deb0 con 0x7f83540c9d00 2024-04-24T23:21:32.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.072+0000 7f8351ffb640 1 -- 172.21.15.26:0/398834736 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8344020020 con 0x7f83540c9d00 2024-04-24T23:21:32.074 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:32.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f834006be60 msgr2=0x7f834006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f834006be60 0x7f834006e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f834c006fd0 tx=0x7f834c008040 comp rx=0 tx=0).stop 2024-04-24T23:21:32.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83540c9d00 msgr2=0x7f83540ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83540c9d00 0x7f83540ca190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f834400c740 tx=0x7f834400cc00 comp rx=0 tx=0).stop 2024-04-24T23:21:32.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 shutdown_connections 2024-04-24T23:21:32.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f834006be60 0x7f834006e320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83540c9d00 0x7f83540ca190 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f835415e370 0x7f8354153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 --2- 172.21.15.26:0/398834736 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83540d4c10 0x7f8354152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 >> 172.21.15.26:0/398834736 conn(0x7f83540d40f0 msgr2=0x7f83540b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:32.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 shutdown_connections 2024-04-24T23:21:32.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.074+0000 7f835b8bd640 1 -- 172.21.15.26:0/398834736 wait complete. 2024-04-24T23:21:32.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:21:32.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.187+0000 7f388a4d4640 1 Processor -- start 2024-04-24T23:21:32.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.187+0000 7f388a4d4640 1 -- start start 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f388a4d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f388415e570 0x7f38840d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f388a4d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38840d22e0 0x7f38840d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f388a4d4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38840d32f0 con 0x7f38840b3130 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f388a4d4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f38840d1ab0 con 0x7f38840d22e0 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f388a4d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f38840d2ae0 con 0x7f388415e570 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f3888a71640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38840d22e0 0x7f38840d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f3888a71640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38840d22e0 0x7f38840d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58356/0 (socket says 172.21.15.26:58356) 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f3888a71640 1 -- 172.21.15.26:0/2764251369 learned_addr learned my addr 172.21.15.26:0/2764251369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f38837fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38840b3130 0x7f38840d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42068/0 (socket says 172.21.15.26:42068) 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f3883fff640 1 --2- 172.21.15.26:0/2764251369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f388415e570 0x7f38840d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f3888a71640 1 -- 172.21.15.26:0/2764251369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f388415e570 msgr2=0x7f38840d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f3888a71640 1 --2- 172.21.15.26:0/2764251369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f388415e570 0x7f38840d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f3888a71640 1 -- 172.21.15.26:0/2764251369 >> v1:172.21.15.26:6789/0 conn(0x7f38840b3130 legacy=0x7f38840d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f3888a71640 1 -- 172.21.15.26:0/2764251369 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38840d0270 con 0x7f38840d22e0 2024-04-24T23:21:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.188+0000 7f3882ffd640 1 -- 172.21.15.26:0/2764251369 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396711500 0 0) 0x7f3874001110 con 0x7f38840b3130 2024-04-24T23:21:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.189+0000 7f3888a71640 1 --2- 172.21.15.26:0/2764251369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38840d22e0 0x7f38840d46a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f387c009e50 tx=0x7f387c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9c2f971b83cc83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.189+0000 7f3882ffd640 1 -- 172.21.15.26:0/2764251369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c036070 con 0x7f38840d22e0 2024-04-24T23:21:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.189+0000 7f3882ffd640 1 -- 172.21.15.26:0/2764251369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387c031600 con 0x7f38840d22e0 2024-04-24T23:21:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.189+0000 7f3882ffd640 1 -- 172.21.15.26:0/2764251369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0319d0 con 0x7f38840d22e0 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.190+0000 7f388a4d4640 1 -- 172.21.15.26:0/2764251369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38840d22e0 msgr2=0x7f38840d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.190+0000 7f388a4d4640 1 --2- 172.21.15.26:0/2764251369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38840d22e0 0x7f38840d46a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f387c009e50 tx=0x7f387c02a710 comp rx=0 tx=0).stop 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.190+0000 7f388a4d4640 1 -- 172.21.15.26:0/2764251369 shutdown_connections 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.190+0000 7f388a4d4640 1 --2- 172.21.15.26:0/2764251369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38840d22e0 0x7f38840d46a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.190+0000 7f388a4d4640 1 --2- 172.21.15.26:0/2764251369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f388415e570 0x7f38840d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.190+0000 7f388a4d4640 1 -- 172.21.15.26:0/2764251369 >> 172.21.15.26:0/2764251369 conn(0x7f388408dbe0 msgr2=0x7f388409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.190+0000 7f388a4d4640 1 -- 172.21.15.26:0/2764251369 shutdown_connections 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.190+0000 7f388a4d4640 1 -- 172.21.15.26:0/2764251369 wait complete. 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.191+0000 7f388a4d4640 1 Processor -- start 2024-04-24T23:21:32.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.191+0000 7f388a4d4640 1 -- start start 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f388a4d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38840b3130 0x7f38840b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f388a4d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38840d22e0 0x7f388415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f388a4d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388415e570 0x7f388414f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f388a4d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38840cfa70 con 0x7f388415e570 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f388a4d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38840d3010 con 0x7f38840b3130 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f388a4d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f38840cfd50 con 0x7f38840d22e0 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3888a71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388415e570 0x7f388414f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3888a71640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388415e570 0x7f388414f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58366/0 (socket says 172.21.15.26:58366) 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3888a71640 1 -- 172.21.15.26:0/502603127 learned_addr learned my addr 172.21.15.26:0/502603127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3883fff640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38840b3130 0x7f38840b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f38837fe640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38840d22e0 0x7f388415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3888a71640 1 -- 172.21.15.26:0/502603127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38840d22e0 msgr2=0x7f388415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3888a71640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38840d22e0 0x7f388415bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3888a71640 1 -- 172.21.15.26:0/502603127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38840b3130 msgr2=0x7f38840b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3888a71640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38840b3130 0x7f38840b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3888a71640 1 -- 172.21.15.26:0/502603127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38840b6f90 con 0x7f388415e570 2024-04-24T23:21:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.192+0000 7f3888a71640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388415e570 0x7f388414f1f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f387c034040 tx=0x7f387c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.193+0000 7f3880ff9640 1 -- 172.21.15.26:0/502603127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c036050 con 0x7f388415e570 2024-04-24T23:21:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.193+0000 7f3880ff9640 1 -- 172.21.15.26:0/502603127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387c042070 con 0x7f388415e570 2024-04-24T23:21:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.193+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38840b7270 con 0x7f388415e570 2024-04-24T23:21:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.193+0000 7f3880ff9640 1 -- 172.21.15.26:0/502603127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c009510 con 0x7f388415e570 2024-04-24T23:21:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.193+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38840b7830 con 0x7f388415e570 2024-04-24T23:21:32.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.194+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388406d430 con 0x7f388415e570 2024-04-24T23:21:32.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.194+0000 7f3880ff9640 1 -- 172.21.15.26:0/502603127 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f387c007bd0 con 0x7f388415e570 2024-04-24T23:21:32.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.194+0000 7f3880ff9640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386406be30 0x7f386406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.195+0000 7f3880ff9640 1 -- 172.21.15.26:0/502603127 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f387c0abd40 con 0x7f388415e570 2024-04-24T23:21:32.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.195+0000 7f3883fff640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386406be30 0x7f386406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.195+0000 7f3883fff640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386406be30 0x7f386406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:32.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.199+0000 7f3880ff9640 1 -- 172.21.15.26:0/502603127 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f388406d430 con 0x7f388415e570 2024-04-24T23:21:32.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.400+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f38840694c0 con 0x7f388415e570 2024-04-24T23:21:32.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.400+0000 7f3880ff9640 1 -- 172.21.15.26:0/502603127 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f387c045360 con 0x7f388415e570 2024-04-24T23:21:32.402 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:21:32.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386406be30 msgr2=0x7f386406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386406be30 0x7f386406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c008040 comp rx=0 tx=0).stop 2024-04-24T23:21:32.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388415e570 msgr2=0x7f388414f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388415e570 0x7f388414f1f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f387c034040 tx=0x7f387c004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:32.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 shutdown_connections 2024-04-24T23:21:32.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386406be30 0x7f386406e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388415e570 0x7f388414f1f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38840d22e0 0x7f388415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 --2- 172.21.15.26:0/502603127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38840b3130 0x7f38840b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 >> 172.21.15.26:0/502603127 conn(0x7f388408dbe0 msgr2=0x7f388406cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:32.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 shutdown_connections 2024-04-24T23:21:32.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.402+0000 7f388a4d4640 1 -- 172.21.15.26:0/502603127 wait complete. 2024-04-24T23:21:32.415 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-24T23:21:32.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:32.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-24T23:21:32.441 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:21:32.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:32.442 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T23:21:32.497 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:21:32.497 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:32.516 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:21:32.517 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:32.576 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:21:32.576 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:32.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.735+0000 7f66f3b42640 1 Processor -- start 2024-04-24T23:21:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.735+0000 7f66f3b42640 1 -- start start 2024-04-24T23:21:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f3b42640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f66ec0b3ac0 0x7f66ec0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f3b42640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec0b62e0 0x7f66ec0b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f3b42640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66ec0b5840 con 0x7f66ec15e290 2024-04-24T23:21:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f3b42640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f66ec0b5e50 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f3b42640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f66ec0cfb30 con 0x7f66ec0b62e0 2024-04-24T23:21:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f20df640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec0b62e0 0x7f66ec0b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f10dd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f66ec0b3ac0 0x7f66ec0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f10dd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f66ec0b3ac0 0x7f66ec0b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:41282/0 (socket says 172.21.15.26:41282) 2024-04-24T23:21:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f18de640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f66ec15e290 0x7f66ec0d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:56450/0 (socket says 172.21.15.26:56450) 2024-04-24T23:21:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f20df640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec0b62e0 0x7f66ec0b6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:60828/0 (socket says 172.21.15.26:60828) 2024-04-24T23:21:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f10dd640 1 -- 172.21.15.26:0/2385732280 learned_addr learned my addr 172.21.15.26:0/2385732280 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f10dd640 1 -- 172.21.15.26:0/2385732280 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec0b62e0 msgr2=0x7f66ec0b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f10dd640 1 --2- 172.21.15.26:0/2385732280 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec0b62e0 0x7f66ec0b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f10dd640 1 -- 172.21.15.26:0/2385732280 >> v1:172.21.15.29:6789/0 conn(0x7f66ec15e290 legacy=0x7f66ec0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.736+0000 7f66f10dd640 1 -- 172.21.15.26:0/2385732280 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec0cfe10 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.737+0000 7f66f10dd640 1 --2- 172.21.15.26:0/2385732280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b3ac0 0x7f66ec0b3f50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009870 tx=0x7f66dc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d89d615728d337 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.737+0000 7f66f08dc640 1 -- 172.21.15.26:0/2385732280 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc037070 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.737+0000 7f66f08dc640 1 -- 172.21.15.26:0/2385732280 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66dc032440 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.737+0000 7f66f08dc640 1 -- 172.21.15.26:0/2385732280 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc032810 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.738+0000 7f66f3b42640 1 -- 172.21.15.26:0/2385732280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b3ac0 msgr2=0x7f66ec0b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.738+0000 7f66f3b42640 1 --2- 172.21.15.26:0/2385732280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b3ac0 0x7f66ec0b3f50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009870 tx=0x7f66dc02a270 comp rx=0 tx=0).stop 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.738+0000 7f66f3b42640 1 -- 172.21.15.26:0/2385732280 shutdown_connections 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.738+0000 7f66f3b42640 1 --2- 172.21.15.26:0/2385732280 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec0b62e0 0x7f66ec0b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.738+0000 7f66f3b42640 1 --2- 172.21.15.26:0/2385732280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b3ac0 0x7f66ec0b3f50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.738+0000 7f66f3b42640 1 -- 172.21.15.26:0/2385732280 >> 172.21.15.26:0/2385732280 conn(0x7f66ec06b120 msgr2=0x7f66ec06b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.738+0000 7f66f3b42640 1 -- 172.21.15.26:0/2385732280 shutdown_connections 2024-04-24T23:21:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.738+0000 7f66f3b42640 1 -- 172.21.15.26:0/2385732280 wait complete. 2024-04-24T23:21:32.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.739+0000 7f66f3b42640 1 Processor -- start 2024-04-24T23:21:32.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.739+0000 7f66f3b42640 1 -- start start 2024-04-24T23:21:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f3b42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66ec0b3ac0 0x7f66ec0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f3b42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b62e0 0x7f66ec0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f3b42640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec15e290 0x7f66ec0b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f3b42640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec15e670 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f3b42640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0b5ae0 con 0x7f66ec0b62e0 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f3b42640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f66ec06dfd0 con 0x7f66ec15e290 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f18de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66ec0b3ac0 0x7f66ec0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f18de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66ec0b3ac0 0x7f66ec0cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58382/0 (socket says 172.21.15.26:58382) 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f18de640 1 -- 172.21.15.26:0/1206659146 learned_addr learned my addr 172.21.15.26:0/1206659146 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f20df640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec15e290 0x7f66ec0b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f10dd640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b62e0 0x7f66ec0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f18de640 1 -- 172.21.15.26:0/1206659146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec15e290 msgr2=0x7f66ec0b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f18de640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec15e290 0x7f66ec0b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f18de640 1 -- 172.21.15.26:0/1206659146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b62e0 msgr2=0x7f66ec0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f18de640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b62e0 0x7f66ec0b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f18de640 1 -- 172.21.15.26:0/1206659146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec0cc300 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f10dd640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b62e0 0x7f66ec0b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.740+0000 7f66f18de640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66ec0b3ac0 0x7f66ec0cf190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f66e800c3c0 tx=0x7f66e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.741+0000 7f66e27fc640 1 -- 172.21.15.26:0/1206659146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e80074e0 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.741+0000 7f66e27fc640 1 -- 172.21.15.26:0/1206659146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e8007680 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.741+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ec0c9240 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.741+0000 7f66e27fc640 1 -- 172.21.15.26:0/1206659146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8004a00 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.741+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ec0c9ed0 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.742+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ec0694c0 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.742+0000 7f66e27fc640 1 -- 172.21.15.26:0/1206659146 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66e801a020 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.743+0000 7f66e27fc640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f66d006be60 0x7f66d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.743+0000 7f66f10dd640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f66d006be60 0x7f66d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:32.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.743+0000 7f66e27fc640 1 -- 172.21.15.26:0/1206659146 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f66e808a630 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.743+0000 7f66f10dd640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f66d006be60 0x7f66d006e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f66dc034000 tx=0x7f66dc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:32.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.747+0000 7f66e27fc640 1 -- 172.21.15.26:0/1206659146 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f66e8058fd0 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.930+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66ec0b7520 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.931+0000 7f66e27fc640 1 -- 172.21.15.26:0/1206659146 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f66e80588d0 con 0x7f66ec0b3ac0 2024-04-24T23:21:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:32.934 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:32.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f66d006be60 msgr2=0x7f66d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f66d006be60 0x7f66d006e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f66dc034000 tx=0x7f66dc02e000 comp rx=0 tx=0).stop 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66ec0b3ac0 msgr2=0x7f66ec0cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66ec0b3ac0 0x7f66ec0cf190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f66e800c3c0 tx=0x7f66e800c880 comp rx=0 tx=0).stop 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 shutdown_connections 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f66d006be60 0x7f66d006e320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66ec15e290 0x7f66ec0b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66ec0b62e0 0x7f66ec0b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 --2- 172.21.15.26:0/1206659146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66ec0b3ac0 0x7f66ec0cf190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.933+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 >> 172.21.15.26:0/1206659146 conn(0x7f66ec06b120 msgr2=0x7f66ec06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.934+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 shutdown_connections 2024-04-24T23:21:32.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:32.934+0000 7f66f3b42640 1 -- 172.21.15.26:0/1206659146 wait complete. 2024-04-24T23:21:32.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:32.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:33.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.048+0000 7f4df04d4640 1 Processor -- start 2024-04-24T23:21:33.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.048+0000 7f4df04d4640 1 -- start start 2024-04-24T23:21:33.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4df04d4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4de8158ae0 0x7f4de806e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4df04d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4de80d0ce0 0x7f4de815e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4df04d4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4de806b1c0 con 0x7f4de815ee90 2024-04-24T23:21:33.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4df04d4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4de806ec90 con 0x7f4de80d0ce0 2024-04-24T23:21:33.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4df04d4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4de806ef70 con 0x7f4de8158ae0 2024-04-24T23:21:33.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4deda6f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4de80d0ce0 0x7f4de815e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4deda6f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4de80d0ce0 0x7f4de815e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58388/0 (socket says 172.21.15.26:58388) 2024-04-24T23:21:33.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4deda6f640 1 -- 172.21.15.26:0/3217373852 learned_addr learned my addr 172.21.15.26:0/3217373852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:33.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4dee270640 1 --2- 172.21.15.26:0/3217373852 >> v2:172.21.15.29:3300/0 conn(0x7f4de8158ae0 0x7f4de806e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4deda6f640 1 -- 172.21.15.26:0/3217373852 >> v2:172.21.15.29:3300/0 conn(0x7f4de8158ae0 msgr2=0x7f4de806e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:33.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4deda6f640 1 --2- 172.21.15.26:0/3217373852 >> v2:172.21.15.29:3300/0 conn(0x7f4de8158ae0 0x7f4de806e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4deda6f640 1 -- 172.21.15.26:0/3217373852 >> v1:172.21.15.29:6789/0 conn(0x7f4de815ee90 legacy=0x7f4de8071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:33.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.049+0000 7f4deda6f640 1 -- 172.21.15.26:0/3217373852 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de806f250 con 0x7f4de80d0ce0 2024-04-24T23:21:33.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.050+0000 7f4deda6f640 1 --2- 172.21.15.26:0/3217373852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 0x7f4de815e950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4de8072580 tx=0x7f4dd802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0ee67972413637 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.050+0000 7f4ded26e640 1 -- 172.21.15.26:0/3217373852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8037070 con 0x7f4de80d0ce0 2024-04-24T23:21:33.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.050+0000 7f4ded26e640 1 -- 172.21.15.26:0/3217373852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd8032440 con 0x7f4de80d0ce0 2024-04-24T23:21:33.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.050+0000 7f4ded26e640 1 -- 172.21.15.26:0/3217373852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8032810 con 0x7f4de80d0ce0 2024-04-24T23:21:33.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.051+0000 7f4df04d4640 1 -- 172.21.15.26:0/3217373852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 msgr2=0x7f4de815e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.051+0000 7f4df04d4640 1 --2- 172.21.15.26:0/3217373852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 0x7f4de815e950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4de8072580 tx=0x7f4dd802a160 comp rx=0 tx=0).stop 2024-04-24T23:21:33.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.051+0000 7f4df04d4640 1 -- 172.21.15.26:0/3217373852 shutdown_connections 2024-04-24T23:21:33.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.051+0000 7f4df04d4640 1 --2- 172.21.15.26:0/3217373852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 0x7f4de815e950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.051+0000 7f4df04d4640 1 --2- 172.21.15.26:0/3217373852 >> v2:172.21.15.29:3300/0 conn(0x7f4de8158ae0 0x7f4de806e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.051+0000 7f4df04d4640 1 -- 172.21.15.26:0/3217373852 >> 172.21.15.26:0/3217373852 conn(0x7f4de80b2970 msgr2=0x7f4de80d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:33.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.051+0000 7f4df04d4640 1 -- 172.21.15.26:0/3217373852 shutdown_connections 2024-04-24T23:21:33.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.051+0000 7f4df04d4640 1 -- 172.21.15.26:0/3217373852 wait complete. 2024-04-24T23:21:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.052+0000 7f4df04d4640 1 Processor -- start 2024-04-24T23:21:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.052+0000 7f4df04d4640 1 -- start start 2024-04-24T23:21:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4df04d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 0x7f4de8145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4df04d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8158ae0 0x7f4de81598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4df04d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4de815ee90 0x7f4de815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4df04d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4de80b00e0 con 0x7f4de80d0ce0 2024-04-24T23:21:33.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4df04d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4de80aff60 con 0x7f4de8158ae0 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4df04d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4de8071e30 con 0x7f4de815ee90 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4dee270640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 0x7f4de8145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4dee270640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 0x7f4de8145fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58392/0 (socket says 172.21.15.26:58392) 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4dee270640 1 -- 172.21.15.26:0/3234715821 learned_addr learned my addr 172.21.15.26:0/3234715821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4deda6f640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8158ae0 0x7f4de81598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4deea71640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4de815ee90 0x7f4de815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4dee270640 1 -- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4de815ee90 msgr2=0x7f4de815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4dee270640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4de815ee90 0x7f4de815b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4dee270640 1 -- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8158ae0 msgr2=0x7f4de81598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4dee270640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8158ae0 0x7f4de81598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4dee270640 1 -- 172.21.15.26:0/3234715821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de815fc90 con 0x7f4de80d0ce0 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4deda6f640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8158ae0 0x7f4de81598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4deea71640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4de815ee90 0x7f4de815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:33.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4dee270640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 0x7f4de8145fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4de400d800 tx=0x7f4de400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4ddeffd640 1 -- 172.21.15.26:0/3234715821 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de400b6c0 con 0x7f4de80d0ce0 2024-04-24T23:21:33.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4ddeffd640 1 -- 172.21.15.26:0/3234715821 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4de4019070 con 0x7f4de80d0ce0 2024-04-24T23:21:33.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de815ff70 con 0x7f4de80d0ce0 2024-04-24T23:21:33.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4ddeffd640 1 -- 172.21.15.26:0/3234715821 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de400ba80 con 0x7f4de80d0ce0 2024-04-24T23:21:33.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.053+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de8160530 con 0x7f4de80d0ce0 2024-04-24T23:21:33.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.054+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de8075610 con 0x7f4de80d0ce0 2024-04-24T23:21:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.055+0000 7f4ddeffd640 1 -- 172.21.15.26:0/3234715821 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4de400bc20 con 0x7f4de80d0ce0 2024-04-24T23:21:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.055+0000 7f4ddeffd640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dcc06be30 0x7f4dcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.056+0000 7f4deda6f640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dcc06be30 0x7f4dcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.056+0000 7f4ddeffd640 1 -- 172.21.15.26:0/3234715821 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4de408b320 con 0x7f4de80d0ce0 2024-04-24T23:21:33.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.056+0000 7f4deda6f640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dcc06be30 0x7f4dcc06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8034000 tx=0x7f4dd802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.060+0000 7f4ddeffd640 1 -- 172.21.15.26:0/3234715821 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4de4059cc0 con 0x7f4de80d0ce0 2024-04-24T23:21:33.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.244+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4de80b5b40 con 0x7f4de80d0ce0 2024-04-24T23:21:33.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.244+0000 7f4ddeffd640 1 -- 172.21.15.26:0/3234715821 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f4de40595c0 con 0x7f4de80d0ce0 2024-04-24T23:21:33.246 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dcc06be30 msgr2=0x7f4dcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dcc06be30 0x7f4dcc06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8034000 tx=0x7f4dd802e000 comp rx=0 tx=0).stop 2024-04-24T23:21:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 msgr2=0x7f4de8145fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 0x7f4de8145fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4de400d800 tx=0x7f4de400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 shutdown_connections 2024-04-24T23:21:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4dcc06be30 0x7f4dcc06e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4de815ee90 0x7f4de815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4de8158ae0 0x7f4de81598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 --2- 172.21.15.26:0/3234715821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de80d0ce0 0x7f4de8145fa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 >> 172.21.15.26:0/3234715821 conn(0x7f4de80b2970 msgr2=0x7f4de815d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 shutdown_connections 2024-04-24T23:21:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.246+0000 7f4df04d4640 1 -- 172.21.15.26:0/3234715821 wait complete. 2024-04-24T23:21:33.259 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.361+0000 7f3d54c2c640 1 Processor -- start 2024-04-24T23:21:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.361+0000 7f3d54c2c640 1 -- start start 2024-04-24T23:21:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.361+0000 7f3d54c2c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d500d0750 0x7f3d50165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.361+0000 7f3d54c2c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5015ef60 con 0x7f3d500ad7b0 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.361+0000 7f3d54c2c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5015f240 con 0x7f3d5015adc0 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.361+0000 7f3d54c2c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3d5015f520 con 0x7f3d500d0750 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.362+0000 7f3d4ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d500d0750 0x7f3d50165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.362+0000 7f3d47fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3d500ad7b0 0x7f3d500d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42076/0 (socket says 172.21.15.26:42076) 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.362+0000 7f3d47fff640 1 -- 172.21.15.26:0/2644485480 learned_addr learned my addr 172.21.15.26:0/2644485480 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.362+0000 7f3d4ed9d640 1 -- 172.21.15.26:0/2644485480 >> v1:172.21.15.29:6789/0 conn(0x7f3d5015adc0 legacy=0x7f3d500b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.362+0000 7f3d4ed9d640 1 -- 172.21.15.26:0/2644485480 >> v1:172.21.15.26:6789/0 conn(0x7f3d500ad7b0 legacy=0x7f3d500d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.362+0000 7f3d4dd9b640 1 -- 172.21.15.26:0/2644485480 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1402998735 0 0) 0x7f3d5015ef60 con 0x7f3d500ad7b0 2024-04-24T23:21:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.362+0000 7f3d4ed9d640 1 -- 172.21.15.26:0/2644485480 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5015f800 con 0x7f3d500d0750 2024-04-24T23:21:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d4ed9d640 1 --2- 172.21.15.26:0/2644485480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500d0750 0x7f3d50165d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009dc0 tx=0x7f3d4802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b533cbe762532e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d4dd9b640 1 -- 172.21.15.26:0/2644485480 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48007740 con 0x7f3d500d0750 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d4dd9b640 1 -- 172.21.15.26:0/2644485480 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d480078e0 con 0x7f3d500d0750 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d4dd9b640 1 -- 172.21.15.26:0/2644485480 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48007cb0 con 0x7f3d500d0750 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d54c2c640 1 -- 172.21.15.26:0/2644485480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500d0750 msgr2=0x7f3d50165d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d54c2c640 1 --2- 172.21.15.26:0/2644485480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500d0750 0x7f3d50165d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009dc0 tx=0x7f3d4802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d54c2c640 1 -- 172.21.15.26:0/2644485480 shutdown_connections 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d54c2c640 1 --2- 172.21.15.26:0/2644485480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500d0750 0x7f3d50165d50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d54c2c640 1 -- 172.21.15.26:0/2644485480 >> 172.21.15.26:0/2644485480 conn(0x7f3d5006b540 msgr2=0x7f3d5006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d54c2c640 1 -- 172.21.15.26:0/2644485480 shutdown_connections 2024-04-24T23:21:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.363+0000 7f3d54c2c640 1 -- 172.21.15.26:0/2644485480 wait complete. 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.365+0000 7f3d54c2c640 1 Processor -- start 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.365+0000 7f3d54c2c640 1 -- start start 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.365+0000 7f3d54c2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500ad7b0 0x7f3d5015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.365+0000 7f3d54c2c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d500d0750 0x7f3d5014a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.365+0000 7f3d54c2c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3d5015adc0 0x7f3d500af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.365+0000 7f3d54c2c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5015e9c0 con 0x7f3d500ad7b0 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.365+0000 7f3d54c2c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5015b1a0 con 0x7f3d500d0750 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.365+0000 7f3d54c2c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5015eb40 con 0x7f3d5015adc0 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500ad7b0 0x7f3d5015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500ad7b0 0x7f3d5015ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58416/0 (socket says 172.21.15.26:58416) 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4e59c640 1 -- 172.21.15.26:0/1287274451 learned_addr learned my addr 172.21.15.26:0/1287274451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4ed9d640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3d5015adc0 0x7f3d500af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d47fff640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d500d0750 0x7f3d5014a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4e59c640 1 -- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3d5015adc0 msgr2=0x7f3d500af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4e59c640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3d5015adc0 0x7f3d500af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4e59c640 1 -- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d500d0750 msgr2=0x7f3d5014a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4e59c640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d500d0750 0x7f3d5014a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4e59c640 1 -- 172.21.15.26:0/1287274451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d500afe70 con 0x7f3d500ad7b0 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4ed9d640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3d5015adc0 0x7f3d500af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d47fff640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d500d0750 0x7f3d5014a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d4e59c640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500ad7b0 0x7f3d5015ddf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000c430 tx=0x7f3d4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d46ffd640 1 -- 172.21.15.26:0/1287274451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40017070 con 0x7f3d500ad7b0 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d46ffd640 1 -- 172.21.15.26:0/1287274451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d400052d0 con 0x7f3d500ad7b0 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d500b0150 con 0x7f3d500ad7b0 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d46ffd640 1 -- 172.21.15.26:0/1287274451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40003830 con 0x7f3d500ad7b0 2024-04-24T23:21:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.366+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d500b0710 con 0x7f3d500ad7b0 2024-04-24T23:21:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.367+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d50075610 con 0x7f3d500ad7b0 2024-04-24T23:21:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.368+0000 7f3d46ffd640 1 -- 172.21.15.26:0/1287274451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d4001a070 con 0x7f3d500ad7b0 2024-04-24T23:21:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.368+0000 7f3d46ffd640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d3006be30 0x7f3d3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.368+0000 7f3d47fff640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d3006be30 0x7f3d3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.368+0000 7f3d46ffd640 1 -- 172.21.15.26:0/1287274451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d4008a1e0 con 0x7f3d500ad7b0 2024-04-24T23:21:33.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.369+0000 7f3d47fff640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d3006be30 0x7f3d3006e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d5014d5e0 tx=0x7f3d38007c70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.373+0000 7f3d46ffd640 1 -- 172.21.15.26:0/1287274451 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3d40058b90 con 0x7f3d500ad7b0 2024-04-24T23:21:33.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.561+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3d5015e730 con 0x7f3d500ad7b0 2024-04-24T23:21:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.562+0000 7f3d46ffd640 1 -- 172.21.15.26:0/1287274451 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3d40058490 con 0x7f3d500ad7b0 2024-04-24T23:21:33.564 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d3006be30 msgr2=0x7f3d3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d3006be30 0x7f3d3006e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d5014d5e0 tx=0x7f3d38007c70 comp rx=0 tx=0).stop 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500ad7b0 msgr2=0x7f3d5015ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500ad7b0 0x7f3d5015ddf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000c430 tx=0x7f3d4000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 shutdown_connections 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d3006be30 0x7f3d3006e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3d5015adc0 0x7f3d500af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d500d0750 0x7f3d5014a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 --2- 172.21.15.26:0/1287274451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d500ad7b0 0x7f3d5015ddf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 >> 172.21.15.26:0/1287274451 conn(0x7f3d5006b540 msgr2=0x7f3d5006e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 shutdown_connections 2024-04-24T23:21:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.565+0000 7f3d54c2c640 1 -- 172.21.15.26:0/1287274451 wait complete. 2024-04-24T23:21:33.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.680+0000 7f1472f5a640 1 Processor -- start 2024-04-24T23:21:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.680+0000 7f1472f5a640 1 -- start start 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.680+0000 7f1472f5a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f146c15e750 0x7f146c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.680+0000 7f1472f5a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f146c0d2c30 0x7f146c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.680+0000 7f1472f5a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f146c0d2120 con 0x7f146c15e370 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.680+0000 7f1472f5a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f146c0d1510 con 0x7f146c0d2c30 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.680+0000 7f1472f5a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f146c0d1ed0 con 0x7f146c15e750 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.681+0000 7f14714f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f146c0d2c30 0x7f146c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.681+0000 7f14714f7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f146c0d2c30 0x7f146c0cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58432/0 (socket says 172.21.15.26:58432) 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.681+0000 7f14714f7640 1 -- 172.21.15.26:0/3176153032 learned_addr learned my addr 172.21.15.26:0/3176153032 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.681+0000 7f146bfff640 1 --2- 172.21.15.26:0/3176153032 >> v2:172.21.15.29:3300/0 conn(0x7f146c15e750 0x7f146c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.681+0000 7f14714f7640 1 -- 172.21.15.26:0/3176153032 >> v2:172.21.15.29:3300/0 conn(0x7f146c15e750 msgr2=0x7f146c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.681+0000 7f14714f7640 1 --2- 172.21.15.26:0/3176153032 >> v2:172.21.15.29:3300/0 conn(0x7f146c15e750 0x7f146c0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.681+0000 7f14714f7640 1 -- 172.21.15.26:0/3176153032 >> v1:172.21.15.29:6789/0 conn(0x7f146c15e370 legacy=0x7f146c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.681+0000 7f14714f7640 1 -- 172.21.15.26:0/3176153032 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c0b3740 con 0x7f146c0d2c30 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f14714f7640 1 --2- 172.21.15.26:0/3176153032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c0d2c30 0x7f146c0cf1d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f146c06b4d0 tx=0x7f145c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d29280d9d7bbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f146b7fe640 1 -- 172.21.15.26:0/3176153032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c037070 con 0x7f146c0d2c30 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f146b7fe640 1 -- 172.21.15.26:0/3176153032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f145c032440 con 0x7f146c0d2c30 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f146b7fe640 1 -- 172.21.15.26:0/3176153032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c032810 con 0x7f146c0d2c30 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f1472f5a640 1 -- 172.21.15.26:0/3176153032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c0d2c30 msgr2=0x7f146c0cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f1472f5a640 1 --2- 172.21.15.26:0/3176153032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c0d2c30 0x7f146c0cf1d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f146c06b4d0 tx=0x7f145c02a270 comp rx=0 tx=0).stop 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f1472f5a640 1 -- 172.21.15.26:0/3176153032 shutdown_connections 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f1472f5a640 1 --2- 172.21.15.26:0/3176153032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c0d2c30 0x7f146c0cf1d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f1472f5a640 1 --2- 172.21.15.26:0/3176153032 >> v2:172.21.15.29:3300/0 conn(0x7f146c15e750 0x7f146c0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f1472f5a640 1 -- 172.21.15.26:0/3176153032 >> 172.21.15.26:0/3176153032 conn(0x7f146c0d40f0 msgr2=0x7f146c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f1472f5a640 1 -- 172.21.15.26:0/3176153032 shutdown_connections 2024-04-24T23:21:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.682+0000 7f1472f5a640 1 -- 172.21.15.26:0/3176153032 wait complete. 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.684+0000 7f1472f5a640 1 Processor -- start 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.684+0000 7f1472f5a640 1 -- start start 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.684+0000 7f1472f5a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f146c0d2c30 0x7f146c0b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.684+0000 7f1472f5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c15e370 0x7f146c0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.684+0000 7f1472f5a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f146c0b6380 0x7f146c0b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.684+0000 7f1472f5a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f146c0b5230 con 0x7f146c15e370 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.684+0000 7f1472f5a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f146c0b3d00 con 0x7f146c0b6380 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.684+0000 7f1472f5a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f146c0b53b0 con 0x7f146c0d2c30 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f146bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c15e370 0x7f146c0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f146bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c15e370 0x7f146c0b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58442/0 (socket says 172.21.15.26:58442) 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f146bfff640 1 -- 172.21.15.26:0/3808418673 learned_addr learned my addr 172.21.15.26:0/3808418673 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f14714f7640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f146c0b6380 0x7f146c0b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f1470cf6640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f146c0d2c30 0x7f146c0b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f146bfff640 1 -- 172.21.15.26:0/3808418673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f146c0d2c30 msgr2=0x7f146c0b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f146bfff640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f146c0d2c30 0x7f146c0b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f146bfff640 1 -- 172.21.15.26:0/3808418673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f146c0b6380 msgr2=0x7f146c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f146bfff640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f146c0b6380 0x7f146c0b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f146bfff640 1 -- 172.21.15.26:0/3808418673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c0b6d80 con 0x7f146c15e370 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f1470cf6640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f146c0d2c30 0x7f146c0b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f146bfff640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c15e370 0x7f146c0b5e40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f145400d830 tx=0x7f145400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f14697fa640 1 -- 172.21.15.26:0/3808418673 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145400b6c0 con 0x7f146c15e370 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f14697fa640 1 -- 172.21.15.26:0/3808418673 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1454019070 con 0x7f146c15e370 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c0b7060 con 0x7f146c15e370 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f14697fa640 1 -- 172.21.15.26:0/3808418673 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145400ba80 con 0x7f146c15e370 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.685+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c0b7620 con 0x7f146c15e370 2024-04-24T23:21:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.686+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f146c075610 con 0x7f146c15e370 2024-04-24T23:21:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.687+0000 7f14697fa640 1 -- 172.21.15.26:0/3808418673 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f145400bc20 con 0x7f146c15e370 2024-04-24T23:21:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.687+0000 7f14697fa640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f144c06be30 0x7f144c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.687+0000 7f14697fa640 1 -- 172.21.15.26:0/3808418673 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f145408b280 con 0x7f146c15e370 2024-04-24T23:21:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.687+0000 7f1470cf6640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f144c06be30 0x7f144c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.688+0000 7f1470cf6640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f144c06be30 0x7f144c06e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1464001e00 tx=0x7f1464009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.692+0000 7f14697fa640 1 -- 172.21.15.26:0/3808418673 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1454059c30 con 0x7f146c15e370 2024-04-24T23:21:33.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.873+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f146c0d1060 con 0x7f146c15e370 2024-04-24T23:21:33.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.874+0000 7f14697fa640 1 -- 172.21.15.26:0/3808418673 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1454059530 con 0x7f146c15e370 2024-04-24T23:21:33.875 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f144c06be30 msgr2=0x7f144c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f144c06be30 0x7f144c06e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1464001e00 tx=0x7f1464009040 comp rx=0 tx=0).stop 2024-04-24T23:21:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c15e370 msgr2=0x7f146c0b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c15e370 0x7f146c0b5e40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f145400d830 tx=0x7f145400dcf0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 shutdown_connections 2024-04-24T23:21:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f144c06be30 0x7f144c06e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f146c0b6380 0x7f146c0b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f146c15e370 0x7f146c0b5e40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 --2- 172.21.15.26:0/3808418673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f146c0d2c30 0x7f146c0b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 >> 172.21.15.26:0/3808418673 conn(0x7f146c0d40f0 msgr2=0x7f146c15d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:33.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 shutdown_connections 2024-04-24T23:21:33.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.876+0000 7f1472f5a640 1 -- 172.21.15.26:0/3808418673 wait complete. 2024-04-24T23:21:33.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:21:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.988+0000 7fc128916640 1 Processor -- start 2024-04-24T23:21:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.989+0000 7fc128916640 1 -- start start 2024-04-24T23:21:33.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.989+0000 7fc128916640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc12415e570 0x7fc1240d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.989+0000 7fc128916640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1240d32f0 con 0x7fc1240d22e0 2024-04-24T23:21:33.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.989+0000 7fc128916640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1240d1600 con 0x7fc1240b3130 2024-04-24T23:21:33.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.989+0000 7fc128916640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc1240d1ab0 con 0x7fc12415e570 2024-04-24T23:21:33.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc122d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc1240d22e0 0x7fc1240d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59594/0 (socket says 172.21.15.26:59594) 2024-04-24T23:21:33.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc122d9d640 1 -- 172.21.15.26:0/187988778 learned_addr learned my addr 172.21.15.26:0/187988778 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:33.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc12259c640 1 --2- 172.21.15.26:0/187988778 >> v2:172.21.15.29:3300/0 conn(0x7fc12415e570 0x7fc1240d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4212790970 0 0) 0x7fc11c001490 con 0x7fc1240d22e0 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc110003520 con 0x7fc1240d22e0 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 124589764 0 0) 0x7fc110003520 con 0x7fc1240d22e0 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 >> v2:172.21.15.29:3300/0 conn(0x7fc12415e570 msgr2=0x7fc1240d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 --2- 172.21.15.26:0/187988778 >> v2:172.21.15.29:3300/0 conn(0x7fc12415e570 0x7fc1240d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 >> v1:172.21.15.29:6789/0 conn(0x7fc1240b3130 legacy=0x7fc1240d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1240d0270 con 0x7fc1240d22e0 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fc11c001bc0 con 0x7fc1240d22e0 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 >> v1:172.21.15.26:6789/0 conn(0x7fc1240d22e0 legacy=0x7fc1240d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc1100039b0 0x7fc110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc110004800 0x7fc110004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1100051c0 0x7fc11000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc1240d1ab0 con 0x7fc110004800 2024-04-24T23:21:33.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc11000c060 con 0x7fc1100051c0 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc11000c260 con 0x7fc1100039b0 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc11bfff640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc110004800 0x7fc110004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc12259c640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc1100039b0 0x7fc110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.990+0000 7fc122d9d640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1100051c0 0x7fc11000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc11bfff640 1 -- 172.21.15.26:0/187988778 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc1100039b0 msgr2=0x7fc110003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc11bfff640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc1100039b0 0x7fc110003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc11bfff640 1 -- 172.21.15.26:0/187988778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1100051c0 msgr2=0x7fc11000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc11bfff640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1100051c0 0x7fc11000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc11bfff640 1 -- 172.21.15.26:0/187988778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc110003520 con 0x7fc110004800 2024-04-24T23:21:33.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc122d9d640 1 -- 172.21.15.26:0/187988778 reap_dead start 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc11bfff640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc110004800 0x7fc110004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc10c001750 tx=0x7fc110005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4191609358726d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c001370 con 0x7fc110004800 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10c0052d0 con 0x7fc110004800 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.991+0000 7fc121d9b640 1 -- 172.21.15.26:0/187988778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c02abc0 con 0x7fc110004800 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.992+0000 7fc128916640 1 -- 172.21.15.26:0/187988778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc110004800 msgr2=0x7fc110004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.992+0000 7fc128916640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc110004800 0x7fc110004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc10c001750 tx=0x7fc110005830 comp rx=0 tx=0).stop 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.992+0000 7fc128916640 1 -- 172.21.15.26:0/187988778 shutdown_connections 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.992+0000 7fc128916640 1 --2- 172.21.15.26:0/187988778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc110004800 0x7fc110004c80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.992+0000 7fc128916640 1 -- 172.21.15.26:0/187988778 >> 172.21.15.26:0/187988778 conn(0x7fc12408dbe0 msgr2=0x7fc12409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.992+0000 7fc128916640 1 -- 172.21.15.26:0/187988778 shutdown_connections 2024-04-24T23:21:33.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.992+0000 7fc128916640 1 -- 172.21.15.26:0/187988778 wait complete. 2024-04-24T23:21:34.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.993+0000 7fc128916640 1 Processor -- start 2024-04-24T23:21:34.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.993+0000 7fc128916640 1 -- start start 2024-04-24T23:21:34.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc128916640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc110004800 0x7fc1241f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc128916640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc12415f600 0x7fc1240b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc128916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1241610e0 0x7fc12415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc128916640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc1240d3010 con 0x7fc1241610e0 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc128916640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc1240d2ae0 con 0x7fc110004800 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc128916640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc1240cfa70 con 0x7fc12415f600 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc122d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1241610e0 0x7fc12415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc122d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1241610e0 0x7fc12415a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35212/0 (socket says 172.21.15.26:35212) 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc122d9d640 1 -- 172.21.15.26:0/3417217860 learned_addr learned my addr 172.21.15.26:0/3417217860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc11bfff640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc12415f600 0x7fc1240b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.994+0000 7fc12259c640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc110004800 0x7fc1241f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc122d9d640 1 -- 172.21.15.26:0/3417217860 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc12415f600 msgr2=0x7fc1240b96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc122d9d640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc12415f600 0x7fc1240b96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc122d9d640 1 -- 172.21.15.26:0/3417217860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc110004800 msgr2=0x7fc1241f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc122d9d640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc110004800 0x7fc1241f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc122d9d640 1 -- 172.21.15.26:0/3417217860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1240ba020 con 0x7fc1241610e0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc122d9d640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1241610e0 0x7fc12415a3b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc12408c9f0 tx=0x7fc11c00ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc11affd640 1 -- 172.21.15.26:0/3417217860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c017020 con 0x7fc1241610e0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc11affd640 1 -- 172.21.15.26:0/3417217860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc11c0076c0 con 0x7fc1241610e0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc11affd640 1 -- 172.21.15.26:0/3417217860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c007a90 con 0x7fc1241610e0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1240cc630 con 0x7fc1241610e0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.995+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1240ccbf0 con 0x7fc1241610e0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.996+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1240694c0 con 0x7fc1241610e0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.997+0000 7fc11affd640 1 -- 172.21.15.26:0/3417217860 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc11c01a020 con 0x7fc1241610e0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.997+0000 7fc11affd640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc10406be60 0x7fc10406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.997+0000 7fc11affd640 1 -- 172.21.15.26:0/3417217860 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc11c08b480 con 0x7fc1241610e0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.997+0000 7fc12259c640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc10406be60 0x7fc10406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:33.997+0000 7fc12259c640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc10406be60 0x7fc10406e320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc124068f60 tx=0x7fc114009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:34.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.001+0000 7fc11affd640 1 -- 172.21.15.26:0/3417217860 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc11c021020 con 0x7fc1241610e0 2024-04-24T23:21:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.198+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc1240673c0 con 0x7fc1241610e0 2024-04-24T23:21:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.198+0000 7fc11affd640 1 -- 172.21.15.26:0/3417217860 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fc1240d32f0 con 0x7fc1241610e0 2024-04-24T23:21:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:21:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc10406be60 msgr2=0x7fc10406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc10406be60 0x7fc10406e320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc124068f60 tx=0x7fc114009190 comp rx=0 tx=0).stop 2024-04-24T23:21:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1241610e0 msgr2=0x7fc12415a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1241610e0 0x7fc12415a3b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc12408c9f0 tx=0x7fc11c00ae80 comp rx=0 tx=0).stop 2024-04-24T23:21:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 shutdown_connections 2024-04-24T23:21:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc10406be60 0x7fc10406e320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1241610e0 0x7fc12415a3b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc12415f600 0x7fc1240b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 --2- 172.21.15.26:0/3417217860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc110004800 0x7fc1241f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 >> 172.21.15.26:0/3417217860 conn(0x7fc12408dbe0 msgr2=0x7fc1240b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 shutdown_connections 2024-04-24T23:21:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.200+0000 7fc128916640 1 -- 172.21.15.26:0/3417217860 wait complete. 2024-04-24T23:21:34.213 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-24T23:21:34.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:34.213 DEBUG:teuthology.orchestra.run.smithi029:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-24T23:21:34.245 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:21:34.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:34.246 DEBUG:teuthology.orchestra.run.smithi029:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T23:21:34.304 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:21:34.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.404+0000 7f0a377fb640 1 Processor -- start 2024-04-24T23:21:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.404+0000 7f0a377fb640 1 -- start start 2024-04-24T23:21:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a377fb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a30143c40 0x7f0a300cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a377fb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0a300cc860 0x7f0a30072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a377fb640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a300cd0b0 con 0x7f0a3015eb00 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a377fb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0a30071b70 con 0x7f0a30143c40 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a377fb640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0a30071240 con 0x7f0a300cc860 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a35597640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a30143c40 0x7f0a300cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a35597640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a30143c40 0x7f0a300cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35226/0 (socket says 172.21.15.26:35226) 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a35597640 1 -- 172.21.15.26:0/2352095390 learned_addr learned my addr 172.21.15.26:0/2352095390 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a34d96640 1 --2- 172.21.15.26:0/2352095390 >> v2:172.21.15.29:3300/0 conn(0x7f0a300cc860 0x7f0a30072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a35597640 1 -- 172.21.15.26:0/2352095390 >> v2:172.21.15.29:3300/0 conn(0x7f0a300cc860 msgr2=0x7f0a30072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a35597640 1 --2- 172.21.15.26:0/2352095390 >> v2:172.21.15.29:3300/0 conn(0x7f0a300cc860 0x7f0a30072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a35597640 1 -- 172.21.15.26:0/2352095390 >> v1:172.21.15.29:6789/0 conn(0x7f0a3015eb00 legacy=0x7f0a3015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a35597640 1 -- 172.21.15.26:0/2352095390 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a30071520 con 0x7f0a30143c40 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.405+0000 7f0a34d96640 1 --2- 172.21.15.26:0/2352095390 >> v2:172.21.15.29:3300/0 conn(0x7f0a300cc860 0x7f0a30072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a35597640 1 --2- 172.21.15.26:0/2352095390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a30143c40 0x7f0a300cf870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a3006db50 tx=0x7f0a2c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a7e083186b37c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a27fff640 1 -- 172.21.15.26:0/2352095390 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c037070 con 0x7f0a30143c40 2024-04-24T23:21:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a27fff640 1 -- 172.21.15.26:0/2352095390 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2c032440 con 0x7f0a30143c40 2024-04-24T23:21:34.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a27fff640 1 -- 172.21.15.26:0/2352095390 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c032810 con 0x7f0a30143c40 2024-04-24T23:21:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a377fb640 1 -- 172.21.15.26:0/2352095390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a30143c40 msgr2=0x7f0a300cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a377fb640 1 --2- 172.21.15.26:0/2352095390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a30143c40 0x7f0a300cf870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a3006db50 tx=0x7f0a2c02a230 comp rx=0 tx=0).stop 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a377fb640 1 -- 172.21.15.26:0/2352095390 shutdown_connections 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a377fb640 1 --2- 172.21.15.26:0/2352095390 >> v2:172.21.15.29:3300/0 conn(0x7f0a300cc860 0x7f0a30072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a377fb640 1 --2- 172.21.15.26:0/2352095390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a30143c40 0x7f0a300cf870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.406+0000 7f0a377fb640 1 -- 172.21.15.26:0/2352095390 >> 172.21.15.26:0/2352095390 conn(0x7f0a3006b300 msgr2=0x7f0a3006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.407+0000 7f0a377fb640 1 -- 172.21.15.26:0/2352095390 shutdown_connections 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.407+0000 7f0a377fb640 1 -- 172.21.15.26:0/2352095390 wait complete. 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.408+0000 7f0a377fb640 1 Processor -- start 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.408+0000 7f0a377fb640 1 -- start start 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.408+0000 7f0a377fb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a300cc860 0x7f0a300cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.408+0000 7f0a377fb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a30143c40 0x7f0a3006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.408+0000 7f0a377fb640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a3015eb00 0x7f0a3006e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.408+0000 7f0a377fb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a30072540 con 0x7f0a300cc860 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.408+0000 7f0a377fb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0a30144020 con 0x7f0a30143c40 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.408+0000 7f0a377fb640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0a30132690 con 0x7f0a3015eb00 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.408+0000 7f0a35597640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a300cc860 0x7f0a300cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a35597640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a300cc860 0x7f0a300cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35228/0 (socket says 172.21.15.26:35228) 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a35597640 1 -- 172.21.15.26:0/3136980557 learned_addr learned my addr 172.21.15.26:0/3136980557 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a34d96640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a30143c40 0x7f0a3006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a35597640 1 -- 172.21.15.26:0/3136980557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a3015eb00 msgr2=0x7f0a3006e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a35597640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a3015eb00 0x7f0a3006e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a35597640 1 -- 172.21.15.26:0/3136980557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a30143c40 msgr2=0x7f0a3006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a35597640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a30143c40 0x7f0a3006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a35597640 1 -- 172.21.15.26:0/3136980557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a30140ce0 con 0x7f0a300cc860 2024-04-24T23:21:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a34d96640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a30143c40 0x7f0a3006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a35597640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a300cc860 0x7f0a300cb6e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c000c00 tx=0x7f0a2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a25ffb640 1 -- 172.21.15.26:0/3136980557 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c041020 con 0x7f0a300cc860 2024-04-24T23:21:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a25ffb640 1 -- 172.21.15.26:0/3136980557 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2c030970 con 0x7f0a300cc860 2024-04-24T23:21:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a300ae6f0 con 0x7f0a300cc860 2024-04-24T23:21:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a25ffb640 1 -- 172.21.15.26:0/3136980557 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c037040 con 0x7f0a300cc860 2024-04-24T23:21:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.409+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a30132bc0 con 0x7f0a300cc860 2024-04-24T23:21:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.410+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a300694c0 con 0x7f0a300cc860 2024-04-24T23:21:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.411+0000 7f0a25ffb640 1 -- 172.21.15.26:0/3136980557 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a2c044020 con 0x7f0a300cc860 2024-04-24T23:21:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.411+0000 7f0a25ffb640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a1406be30 0x7f0a1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.411+0000 7f0a25ffb640 1 -- 172.21.15.26:0/3136980557 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0a2c0ab5d0 con 0x7f0a300cc860 2024-04-24T23:21:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.411+0000 7f0a34d96640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a1406be30 0x7f0a1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.412+0000 7f0a34d96640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a1406be30 0x7f0a1406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a3006b990 tx=0x7f0a20008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.416+0000 7f0a25ffb640 1 -- 172.21.15.26:0/3136980557 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a2c04b070 con 0x7f0a300cc860 2024-04-24T23:21:34.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.638+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a3006b6e0 con 0x7f0a300cc860 2024-04-24T23:21:34.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.639+0000 7f0a25ffb640 1 -- 172.21.15.26:0/3136980557 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7f0a30140ce0 con 0x7f0a300cc860 2024-04-24T23:21:34.641 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:34.641 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2829963318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2829963318},{"type":"v1","addr":"172.21.15.26:6837","nonce":2829963318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2687649505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2687649505},{"type":"v1","addr":"172.21.15.29:6833","nonce":2687649505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:20:54.833525+0000","modified":"2024-04-24T23:20:57.181910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1604449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1604449840},{"type":"v1","addr":"172.21.15.29:6835","nonce":1604449840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-24T23:21:34.641 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a1406be30 msgr2=0x7f0a1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a1406be30 0x7f0a1406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a3006b990 tx=0x7f0a20008040 comp rx=0 tx=0).stop 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a300cc860 msgr2=0x7f0a300cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a300cc860 0x7f0a300cb6e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c000c00 tx=0x7f0a2c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 shutdown_connections 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a1406be30 0x7f0a1406e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a3015eb00 0x7f0a3006e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a30143c40 0x7f0a3006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 --2- 172.21.15.26:0/3136980557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a300cc860 0x7f0a300cb6e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 >> 172.21.15.26:0/3136980557 conn(0x7f0a3006b300 msgr2=0x7f0a300b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 shutdown_connections 2024-04-24T23:21:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.641+0000 7f0a377fb640 1 -- 172.21.15.26:0/3136980557 wait complete. 2024-04-24T23:21:34.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.752+0000 7f7cb0676640 1 Processor -- start 2024-04-24T23:21:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.752+0000 7f7cb0676640 1 -- start start 2024-04-24T23:21:34.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.753+0000 7f7cb0676640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ca80b3130 0x7f7ca80b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.753+0000 7f7cb0676640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ca80d32f0 con 0x7f7ca80d4870 2024-04-24T23:21:34.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.753+0000 7f7cb0676640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ca80d0270 con 0x7f7ca815e570 2024-04-24T23:21:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.753+0000 7f7cb0676640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7ca80d1ab0 con 0x7f7ca80b3130 2024-04-24T23:21:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.753+0000 7f7caec13640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ca80b3130 0x7f7ca80b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.753+0000 7f7cadc11640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7ca80d4870 0x7f7ca80b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59610/0 (socket says 172.21.15.26:59610) 2024-04-24T23:21:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.754+0000 7f7cadc11640 1 -- 172.21.15.26:0/1155857679 learned_addr learned my addr 172.21.15.26:0/1155857679 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.754+0000 7f7caec13640 1 -- 172.21.15.26:0/1155857679 >> v1:172.21.15.29:6789/0 conn(0x7f7ca815e570 legacy=0x7f7ca80d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.754+0000 7f7caec13640 1 -- 172.21.15.26:0/1155857679 >> v1:172.21.15.26:6789/0 conn(0x7f7ca80d4870 legacy=0x7f7ca80b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.754+0000 7f7caec13640 1 -- 172.21.15.26:0/1155857679 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca80d2d30 con 0x7f7ca80b3130 2024-04-24T23:21:34.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.754+0000 7f7caec13640 1 --2- 172.21.15.26:0/1155857679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 0x7f7ca80b4810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009d90 tx=0x7f7ca002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fdc85358e1dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:34.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cad410640 1 -- 172.21.15.26:0/1155857679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0031600 con 0x7f7ca80b3130 2024-04-24T23:21:34.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cad410640 1 -- 172.21.15.26:0/1155857679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca00317a0 con 0x7f7ca80b3130 2024-04-24T23:21:34.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cad410640 1 -- 172.21.15.26:0/1155857679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0031b70 con 0x7f7ca80b3130 2024-04-24T23:21:34.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cb0676640 1 -- 172.21.15.26:0/1155857679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 msgr2=0x7f7ca80b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cb0676640 1 --2- 172.21.15.26:0/1155857679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 0x7f7ca80b4810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009d90 tx=0x7f7ca002a610 comp rx=0 tx=0).stop 2024-04-24T23:21:34.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cb0676640 1 -- 172.21.15.26:0/1155857679 shutdown_connections 2024-04-24T23:21:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cb0676640 1 --2- 172.21.15.26:0/1155857679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 0x7f7ca80b4810 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cb0676640 1 -- 172.21.15.26:0/1155857679 >> 172.21.15.26:0/1155857679 conn(0x7f7ca808dbe0 msgr2=0x7f7ca809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cb0676640 1 -- 172.21.15.26:0/1155857679 shutdown_connections 2024-04-24T23:21:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.755+0000 7f7cb0676640 1 -- 172.21.15.26:0/1155857679 wait complete. 2024-04-24T23:21:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.756+0000 7f7cb0676640 1 Processor -- start 2024-04-24T23:21:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cb0676640 1 -- start start 2024-04-24T23:21:34.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cb0676640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 0x7f7ca814f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cb0676640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ca80d4870 0x7f7ca80cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cb0676640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca815e570 0x7f7ca80b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cb0676640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca80cfa70 con 0x7f7ca80b3130 2024-04-24T23:21:34.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cb0676640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca80d2ae0 con 0x7f7ca815e570 2024-04-24T23:21:34.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cb0676640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7ca80cfd50 con 0x7f7ca80d4870 2024-04-24T23:21:34.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cae412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 0x7f7ca814f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cae412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 0x7f7ca814f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35246/0 (socket says 172.21.15.26:35246) 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cae412640 1 -- 172.21.15.26:0/3094926914 learned_addr learned my addr 172.21.15.26:0/3094926914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7caec13640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca815e570 0x7f7ca80b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.757+0000 7f7cadc11640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ca80d4870 0x7f7ca80cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7cae412640 1 -- 172.21.15.26:0/3094926914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ca80d4870 msgr2=0x7f7ca80cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7cae412640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ca80d4870 0x7f7ca80cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7cae412640 1 -- 172.21.15.26:0/3094926914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca815e570 msgr2=0x7f7ca80b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7cae412640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca815e570 0x7f7ca80b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7cae412640 1 -- 172.21.15.26:0/3094926914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca80cd080 con 0x7f7ca80b3130 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7cae412640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 0x7f7ca814f5b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400b3f0 tx=0x7f7ca400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7c9effd640 1 -- 172.21.15.26:0/3094926914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4015020 con 0x7f7ca80b3130 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7c9effd640 1 -- 172.21.15.26:0/3094926914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca4011830 con 0x7f7ca80b3130 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7c9effd640 1 -- 172.21.15.26:0/3094926914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4011c20 con 0x7f7ca80b3130 2024-04-24T23:21:34.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca80cd360 con 0x7f7ca80b3130 2024-04-24T23:21:34.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.758+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca80cd920 con 0x7f7ca80b3130 2024-04-24T23:21:34.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.759+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca80694c0 con 0x7f7ca80b3130 2024-04-24T23:21:34.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.760+0000 7f7c9effd640 1 -- 172.21.15.26:0/3094926914 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ca4018020 con 0x7f7ca80b3130 2024-04-24T23:21:34.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.760+0000 7f7c9effd640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c8c06be30 0x7f7c8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:34.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.760+0000 7f7cadc11640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c8c06be30 0x7f7c8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:34.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.760+0000 7f7c9effd640 1 -- 172.21.15.26:0/3094926914 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ca408a940 con 0x7f7ca80b3130 2024-04-24T23:21:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.761+0000 7f7cadc11640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c8c06be30 0x7f7c8c06e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8161bd0 tx=0x7f7c98007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.765+0000 7f7c9effd640 1 -- 172.21.15.26:0/3094926914 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7ca401f030 con 0x7f7ca80b3130 2024-04-24T23:21:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.947+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ca8161c30 con 0x7f7ca80b3130 2024-04-24T23:21:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.948+0000 7f7c9effd640 1 -- 172.21.15.26:0/3094926914 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f7ca4058d30 con 0x7f7ca80b3130 2024-04-24T23:21:34.950 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:34.950 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:20:56.470441+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c8c06be30 msgr2=0x7f7c8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c8c06be30 0x7f7c8c06e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8161bd0 tx=0x7f7c98007e80 comp rx=0 tx=0).stop 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 msgr2=0x7f7ca814f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 0x7f7ca814f5b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400b3f0 tx=0x7f7ca400b8b0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 shutdown_connections 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c8c06be30 0x7f7c8c06e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ca815e570 0x7f7ca80b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ca80d4870 0x7f7ca80cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 --2- 172.21.15.26:0/3094926914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ca80b3130 0x7f7ca814f5b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:34.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 >> 172.21.15.26:0/3094926914 conn(0x7f7ca808dbe0 msgr2=0x7f7ca806d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:34.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 shutdown_connections 2024-04-24T23:21:34.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:34.950+0000 7f7cb0676640 1 -- 172.21.15.26:0/3094926914 wait complete. 2024-04-24T23:21:34.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:20:53.333231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:20:53.808050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:34.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.065+0000 7f8348e72640 1 Processor -- start 2024-04-24T23:21:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.065+0000 7f8348e72640 1 -- start start 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8348e72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440d18b0 0x7f834406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8348e72640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834415e920 con 0x7f834406b660 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8348e72640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834415eba0 con 0x7f83440cc9b0 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8348e72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f834415ee80 con 0x7f83440d18b0 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8342d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f834406b660 0x7f8344157fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59626/0 (socket says 172.21.15.26:59626) 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8342d9d640 1 -- 172.21.15.26:0/3575526668 learned_addr learned my addr 172.21.15.26:0/3575526668 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f833bfff640 1 --2- 172.21.15.26:0/3575526668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440d18b0 0x7f834406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8341d9b640 1 -- 172.21.15.26:0/3575526668 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1817793499 0 0) 0x7f833c001490 con 0x7f834406b660 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8341d9b640 1 -- 172.21.15.26:0/3575526668 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8330003520 con 0x7f834406b660 2024-04-24T23:21:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8341d9b640 1 -- 172.21.15.26:0/3575526668 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4018192407 0 0) 0x7f834415eba0 con 0x7f83440cc9b0 2024-04-24T23:21:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8341d9b640 1 -- 172.21.15.26:0/3575526668 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f833c001490 con 0x7f83440cc9b0 2024-04-24T23:21:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f8341d9b640 1 -- 172.21.15.26:0/3575526668 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1393073554 0 0) 0x7f8330003520 con 0x7f834406b660 2024-04-24T23:21:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.066+0000 7f833bfff640 1 -- 172.21.15.26:0/3575526668 >> v1:172.21.15.29:6789/0 conn(0x7f83440cc9b0 legacy=0x7f83440d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.067+0000 7f833bfff640 1 -- 172.21.15.26:0/3575526668 >> v1:172.21.15.26:6789/0 conn(0x7f834406b660 legacy=0x7f8344157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.067+0000 7f833bfff640 1 -- 172.21.15.26:0/3575526668 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834415f160 con 0x7f83440d18b0 2024-04-24T23:21:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.067+0000 7f833bfff640 1 --2- 172.21.15.26:0/3575526668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440d18b0 0x7f834406b120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f832c0097d0 tx=0x7f832c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b48c935c2024041 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.067+0000 7f8341d9b640 1 -- 172.21.15.26:0/3575526668 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c037070 con 0x7f83440d18b0 2024-04-24T23:21:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.067+0000 7f8341d9b640 1 -- 172.21.15.26:0/3575526668 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f832c02ac30 con 0x7f83440d18b0 2024-04-24T23:21:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.067+0000 7f8341d9b640 1 -- 172.21.15.26:0/3575526668 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c001bc0 con 0x7f83440d18b0 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.068+0000 7f8348e72640 1 -- 172.21.15.26:0/3575526668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440d18b0 msgr2=0x7f834406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.068+0000 7f8348e72640 1 --2- 172.21.15.26:0/3575526668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440d18b0 0x7f834406b120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f832c0097d0 tx=0x7f832c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.068+0000 7f8348e72640 1 -- 172.21.15.26:0/3575526668 shutdown_connections 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.068+0000 7f8348e72640 1 --2- 172.21.15.26:0/3575526668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440d18b0 0x7f834406b120 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.068+0000 7f8348e72640 1 -- 172.21.15.26:0/3575526668 >> 172.21.15.26:0/3575526668 conn(0x7f83440d0d20 msgr2=0x7f83440d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.068+0000 7f8348e72640 1 -- 172.21.15.26:0/3575526668 shutdown_connections 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.068+0000 7f8348e72640 1 -- 172.21.15.26:0/3575526668 wait complete. 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.069+0000 7f8348e72640 1 Processor -- start 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.069+0000 7f8348e72640 1 -- start start 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f8348e72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834406b660 0x7f83440b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f8348e72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440cc9b0 0x7f83440b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f8348e72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83440d18b0 0x7f834415a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f8348e72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83441586d0 con 0x7f834406b660 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f8348e72640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83440cec20 con 0x7f83440d18b0 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f8348e72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8344158850 con 0x7f83440cc9b0 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f834259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834406b660 0x7f83440b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f834259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834406b660 0x7f83440b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35256/0 (socket says 172.21.15.26:35256) 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f834259c640 1 -- 172.21.15.26:0/3129832155 learned_addr learned my addr 172.21.15.26:0/3129832155 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f834259c640 1 -- 172.21.15.26:0/3129832155 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440cc9b0 msgr2=0x7f83440b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f834259c640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440cc9b0 0x7f83440b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f8342d9d640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83440d18b0 0x7f834415a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f834259c640 1 -- 172.21.15.26:0/3129832155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83440d18b0 msgr2=0x7f834415a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.070+0000 7f834259c640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83440d18b0 0x7f834415a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.071+0000 7f834259c640 1 -- 172.21.15.26:0/3129832155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83440b41b0 con 0x7f834406b660 2024-04-24T23:21:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.071+0000 7f834259c640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834406b660 0x7f83440b6520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f833400c490 tx=0x7f833400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.071+0000 7f833affd640 1 -- 172.21.15.26:0/3129832155 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83340052d0 con 0x7f834406b660 2024-04-24T23:21:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.071+0000 7f833affd640 1 -- 172.21.15.26:0/3129832155 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8334005470 con 0x7f834406b660 2024-04-24T23:21:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.071+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83440b4490 con 0x7f834406b660 2024-04-24T23:21:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.071+0000 7f833affd640 1 -- 172.21.15.26:0/3129832155 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334012850 con 0x7f834406b660 2024-04-24T23:21:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.071+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83440b4d30 con 0x7f834406b660 2024-04-24T23:21:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.072+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83440b4660 con 0x7f834406b660 2024-04-24T23:21:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.072+0000 7f833affd640 1 -- 172.21.15.26:0/3129832155 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83340129f0 con 0x7f834406b660 2024-04-24T23:21:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.073+0000 7f833affd640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f832406be30 0x7f832406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.073+0000 7f833affd640 1 -- 172.21.15.26:0/3129832155 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f833408ab30 con 0x7f834406b660 2024-04-24T23:21:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.073+0000 7f833bfff640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f832406be30 0x7f832406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.073+0000 7f833bfff640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f832406be30 0x7f832406e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f832c034000 tx=0x7f832c02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:35.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.077+0000 7f833affd640 1 -- 172.21.15.26:0/3129832155 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83340594e0 con 0x7f834406b660 2024-04-24T23:21:35.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.258+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8344075610 con 0x7f834406b660 2024-04-24T23:21:35.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.258+0000 7f833affd640 1 -- 172.21.15.26:0/3129832155 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8334058de0 con 0x7f834406b660 2024-04-24T23:21:35.260 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f832406be30 msgr2=0x7f832406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f832406be30 0x7f832406e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f832c034000 tx=0x7f832c02c000 comp rx=0 tx=0).stop 2024-04-24T23:21:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834406b660 msgr2=0x7f83440b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834406b660 0x7f83440b6520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f833400c490 tx=0x7f833400c950 comp rx=0 tx=0).stop 2024-04-24T23:21:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 shutdown_connections 2024-04-24T23:21:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f832406be30 0x7f832406e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83440d18b0 0x7f834415a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83440cc9b0 0x7f83440b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 --2- 172.21.15.26:0/3129832155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834406b660 0x7f83440b6520 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 >> 172.21.15.26:0/3129832155 conn(0x7f83440d0d20 msgr2=0x7f83440742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:35.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 shutdown_connections 2024-04-24T23:21:35.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.260+0000 7f8348e72640 1 -- 172.21.15.26:0/3129832155 wait complete. 2024-04-24T23:21:35.273 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:21:35.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.377+0000 7fbec5bc8640 1 Processor -- start 2024-04-24T23:21:35.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.377+0000 7fbec5bc8640 1 -- start start 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbec5bc8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbec015be40 0x7fbec00acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbec5bc8640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 0x7fbec015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbec5bc8640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec00aeb10 con 0x7fbec00b4ce0 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbec5bc8640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbec009df20 con 0x7fbec015be40 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbec5bc8640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbec00aedf0 con 0x7fbec00d0750 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbebf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbec015be40 0x7fbec00acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbebeffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbec00b4ce0 0x7fbec00d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59636/0 (socket says 172.21.15.26:59636) 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbebeffd640 1 -- 172.21.15.26:0/989345813 learned_addr learned my addr 172.21.15.26:0/989345813 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbebffff640 1 --2- 172.21.15.26:0/989345813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 0x7fbec015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbebf7fe640 1 -- 172.21.15.26:0/989345813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 msgr2=0x7fbec015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbebf7fe640 1 --2- 172.21.15.26:0/989345813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 0x7fbec015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbebf7fe640 1 -- 172.21.15.26:0/989345813 >> v1:172.21.15.26:6789/0 conn(0x7fbec00b4ce0 legacy=0x7fbec00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbebe7fc640 1 -- 172.21.15.26:0/989345813 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2326442331 0 0) 0x7fbec00aeb10 con 0x7fbec00b4ce0 2024-04-24T23:21:35.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.378+0000 7fbebf7fe640 1 -- 172.21.15.26:0/989345813 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec00af0d0 con 0x7fbec015be40 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.379+0000 7fbebf7fe640 1 --2- 172.21.15.26:0/989345813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 0x7fbec00acd30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00097a0 tx=0x7fbeb002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23bf94b75eb1a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.379+0000 7fbebe7fc640 1 -- 172.21.15.26:0/989345813 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0037070 con 0x7fbec015be40 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.379+0000 7fbebe7fc640 1 -- 172.21.15.26:0/989345813 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb0032440 con 0x7fbec015be40 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.379+0000 7fbebe7fc640 1 -- 172.21.15.26:0/989345813 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0032810 con 0x7fbec015be40 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.380+0000 7fbec5bc8640 1 -- 172.21.15.26:0/989345813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 msgr2=0x7fbec00acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.380+0000 7fbec5bc8640 1 --2- 172.21.15.26:0/989345813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 0x7fbec00acd30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00097a0 tx=0x7fbeb002a180 comp rx=0 tx=0).stop 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.380+0000 7fbec5bc8640 1 -- 172.21.15.26:0/989345813 shutdown_connections 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.380+0000 7fbec5bc8640 1 --2- 172.21.15.26:0/989345813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 0x7fbec015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.380+0000 7fbec5bc8640 1 --2- 172.21.15.26:0/989345813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 0x7fbec00acd30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.380+0000 7fbec5bc8640 1 -- 172.21.15.26:0/989345813 >> 172.21.15.26:0/989345813 conn(0x7fbec006b540 msgr2=0x7fbec006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:35.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.380+0000 7fbec5bc8640 1 -- 172.21.15.26:0/989345813 shutdown_connections 2024-04-24T23:21:35.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.380+0000 7fbec5bc8640 1 -- 172.21.15.26:0/989345813 wait complete. 2024-04-24T23:21:35.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.381+0000 7fbec5bc8640 1 Processor -- start 2024-04-24T23:21:35.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.381+0000 7fbec5bc8640 1 -- start start 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbec5bc8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbec00b4ce0 0x7fbec01598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbec5bc8640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 0x7fbec00c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbec5bc8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 0x7fbec014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbec5bc8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbec006cd10 con 0x7fbec015be40 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbec5bc8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbec006cb90 con 0x7fbec00b4ce0 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbec5bc8640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbec015c760 con 0x7fbec00d0750 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 0x7fbec014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 0x7fbec014c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35276/0 (socket says 172.21.15.26:35276) 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebffff640 1 -- 172.21.15.26:0/607655088 learned_addr learned my addr 172.21.15.26:0/607655088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebf7fe640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbec00b4ce0 0x7fbec01598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebeffd640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 0x7fbec00c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebffff640 1 -- 172.21.15.26:0/607655088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 msgr2=0x7fbec00c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebffff640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 0x7fbec00c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebffff640 1 -- 172.21.15.26:0/607655088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbec00b4ce0 msgr2=0x7fbec01598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebffff640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbec00b4ce0 0x7fbec01598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebffff640 1 -- 172.21.15.26:0/607655088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec00c5910 con 0x7fbec015be40 2024-04-24T23:21:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebf7fe640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbec00b4ce0 0x7fbec01598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.382+0000 7fbebffff640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 0x7fbec014c2d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800efc0 tx=0x7fbeb800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.383+0000 7fbea7fff640 1 -- 172.21.15.26:0/607655088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8018020 con 0x7fbec015be40 2024-04-24T23:21:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.383+0000 7fbea7fff640 1 -- 172.21.15.26:0/607655088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb8014480 con 0x7fbec015be40 2024-04-24T23:21:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.383+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec009b640 con 0x7fbec015be40 2024-04-24T23:21:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.383+0000 7fbea7fff640 1 -- 172.21.15.26:0/607655088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8014870 con 0x7fbec015be40 2024-04-24T23:21:35.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.383+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec009bc00 con 0x7fbec015be40 2024-04-24T23:21:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.384+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec01f2210 con 0x7fbec015be40 2024-04-24T23:21:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.384+0000 7fbea7fff640 1 -- 172.21.15.26:0/607655088 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbeb8005d10 con 0x7fbec015be40 2024-04-24T23:21:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.384+0000 7fbea7fff640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbeac06be30 0x7fbeac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.384+0000 7fbea7fff640 1 -- 172.21.15.26:0/607655088 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbeb808b970 con 0x7fbec015be40 2024-04-24T23:21:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.384+0000 7fbebf7fe640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbeac06be30 0x7fbeac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.385+0000 7fbebf7fe640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbeac06be30 0x7fbeac06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0034000 tx=0x7fbeb002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.389+0000 7fbea7fff640 1 -- 172.21.15.26:0/607655088 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbeb8022070 con 0x7fbec015be40 2024-04-24T23:21:35.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.578+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbec0075610 con 0x7fbec015be40 2024-04-24T23:21:35.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.578+0000 7fbea7fff640 1 -- 172.21.15.26:0/607655088 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fbeb8010040 con 0x7fbec015be40 2024-04-24T23:21:35.583 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:21:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbeac06be30 msgr2=0x7fbeac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbeac06be30 0x7fbeac06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0034000 tx=0x7fbeb002e000 comp rx=0 tx=0).stop 2024-04-24T23:21:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 msgr2=0x7fbec014c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 0x7fbec014c2d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800efc0 tx=0x7fbeb800c490 comp rx=0 tx=0).stop 2024-04-24T23:21:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 shutdown_connections 2024-04-24T23:21:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbeac06be30 0x7fbeac06e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec015be40 0x7fbec014c2d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbec00d0750 0x7fbec00c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 --2- 172.21.15.26:0/607655088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbec00b4ce0 0x7fbec01598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 >> 172.21.15.26:0/607655088 conn(0x7fbec006b540 msgr2=0x7fbec0158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:35.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 shutdown_connections 2024-04-24T23:21:35.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.580+0000 7fbec5bc8640 1 -- 172.21.15.26:0/607655088 wait complete. 2024-04-24T23:21:35.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:21:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.695+0000 7f01bcc09640 1 Processor -- start 2024-04-24T23:21:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.696+0000 7f01bcc09640 1 -- start start 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.696+0000 7f01bcc09640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f01b815e750 0x7f01b80cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.696+0000 7f01bcc09640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b80d2120 con 0x7f01b80d2c30 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.696+0000 7f01bcc09640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b80b3740 con 0x7f01b815e370 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.696+0000 7f01bcc09640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f01b80d1510 con 0x7f01b815e750 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.697+0000 7f01b6d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f01b80d2c30 0x7f01b80cf1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59638/0 (socket says 172.21.15.26:59638) 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.697+0000 7f01b6d9d640 1 -- 172.21.15.26:0/1546594080 learned_addr learned my addr 172.21.15.26:0/1546594080 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.697+0000 7f01b5d9b640 1 --2- 172.21.15.26:0/1546594080 >> v2:172.21.15.29:3300/0 conn(0x7f01b815e750 0x7f01b80cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.697+0000 7f01b559a640 1 -- 172.21.15.26:0/1546594080 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849898683 0 0) 0x7f01b80d2120 con 0x7f01b80d2c30 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.697+0000 7f01b559a640 1 -- 172.21.15.26:0/1546594080 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01a4003520 con 0x7f01b80d2c30 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.697+0000 7f01b5d9b640 1 -- 172.21.15.26:0/1546594080 >> v1:172.21.15.29:6789/0 conn(0x7f01b815e370 legacy=0x7f01b80d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.697+0000 7f01b559a640 1 -- 172.21.15.26:0/1546594080 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4112382032 0 0) 0x7f01a4003520 con 0x7f01b80d2c30 2024-04-24T23:21:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.697+0000 7f01b5d9b640 1 -- 172.21.15.26:0/1546594080 >> v1:172.21.15.26:6789/0 conn(0x7f01b80d2c30 legacy=0x7f01b80cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.697+0000 7f01b5d9b640 1 -- 172.21.15.26:0/1546594080 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b80b3a20 con 0x7f01b815e750 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.698+0000 7f01b5d9b640 1 --2- 172.21.15.26:0/1546594080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01b815e750 0x7f01b80cec90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01a00097a0 tx=0x7f01a002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=300ff40b8e7f226c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.698+0000 7f01b559a640 1 -- 172.21.15.26:0/1546594080 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0037070 con 0x7f01b815e750 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.698+0000 7f01b559a640 1 -- 172.21.15.26:0/1546594080 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a002ac50 con 0x7f01b815e750 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.698+0000 7f01b559a640 1 -- 172.21.15.26:0/1546594080 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a80019f0 con 0x7f01b815e750 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.699+0000 7f01bcc09640 1 -- 172.21.15.26:0/1546594080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01b815e750 msgr2=0x7f01b80cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.699+0000 7f01bcc09640 1 --2- 172.21.15.26:0/1546594080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01b815e750 0x7f01b80cec90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01a00097a0 tx=0x7f01a002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.699+0000 7f01bcc09640 1 -- 172.21.15.26:0/1546594080 shutdown_connections 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.699+0000 7f01bcc09640 1 --2- 172.21.15.26:0/1546594080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01b815e750 0x7f01b80cec90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.699+0000 7f01bcc09640 1 -- 172.21.15.26:0/1546594080 >> 172.21.15.26:0/1546594080 conn(0x7f01b80d40f0 msgr2=0x7f01b80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.699+0000 7f01bcc09640 1 -- 172.21.15.26:0/1546594080 shutdown_connections 2024-04-24T23:21:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.699+0000 7f01bcc09640 1 -- 172.21.15.26:0/1546594080 wait complete. 2024-04-24T23:21:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.700+0000 7f01bcc09640 1 Processor -- start 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.700+0000 7f01bcc09640 1 -- start start 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01bcc09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b80d2c30 0x7f01b81525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01bcc09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01b815e370 0x7f01b80cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01bcc09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01b80cc0a0 0x7f01b80b9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01bcc09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f01b80b5230 con 0x7f01b80d2c30 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01bcc09640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f01b80d1ed0 con 0x7f01b80cc0a0 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01bcc09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f01b80b53b0 con 0x7f01b815e370 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b80d2c30 0x7f01b81525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b80d2c30 0x7f01b81525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35278/0 (socket says 172.21.15.26:35278) 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b659c640 1 -- 172.21.15.26:0/1314518457 learned_addr learned my addr 172.21.15.26:0/1314518457 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b5d9b640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01b815e370 0x7f01b80cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b659c640 1 -- 172.21.15.26:0/1314518457 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01b815e370 msgr2=0x7f01b80cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b6d9d640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01b80cc0a0 0x7f01b80b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b659c640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01b815e370 0x7f01b80cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b659c640 1 -- 172.21.15.26:0/1314518457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01b80cc0a0 msgr2=0x7f01b80b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b659c640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01b80cc0a0 0x7f01b80b9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.701+0000 7f01b659c640 1 -- 172.21.15.26:0/1314518457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b80b88a0 con 0x7f01b80d2c30 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.702+0000 7f01b659c640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b80d2c30 0x7f01b81525e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c450 tx=0x7f01b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.702+0000 7f01aeffd640 1 -- 172.21.15.26:0/1314518457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0017070 con 0x7f01b80d2c30 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.702+0000 7f01aeffd640 1 -- 172.21.15.26:0/1314518457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b00052d0 con 0x7f01b80d2c30 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.702+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b80cc540 con 0x7f01b80d2c30 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.702+0000 7f01aeffd640 1 -- 172.21.15.26:0/1314518457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0003830 con 0x7f01b80d2c30 2024-04-24T23:21:35.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.702+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b80b8e20 con 0x7f01b80d2c30 2024-04-24T23:21:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.703+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b8075610 con 0x7f01b80d2c30 2024-04-24T23:21:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.703+0000 7f01aeffd640 1 -- 172.21.15.26:0/1314518457 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01b001a070 con 0x7f01b80d2c30 2024-04-24T23:21:35.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.704+0000 7f01aeffd640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f019806be30 0x7f019806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.704+0000 7f01aeffd640 1 -- 172.21.15.26:0/1314518457 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f01b008a2f0 con 0x7f01b80d2c30 2024-04-24T23:21:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.704+0000 7f01b5d9b640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f019806be30 0x7f019806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:35.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.704+0000 7f01b5d9b640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f019806be30 0x7f019806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f01a0034000 tx=0x7f01a002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:35.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.708+0000 7f01aeffd640 1 -- 172.21.15.26:0/1314518457 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01b0058ba0 con 0x7f01b80d2c30 2024-04-24T23:21:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.894+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f01b806be40 con 0x7f01b80d2c30 2024-04-24T23:21:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.894+0000 7f01aeffd640 1 -- 172.21.15.26:0/1314518457 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f01b00584a0 con 0x7f01b80d2c30 2024-04-24T23:21:35.896 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:21:35.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.896+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f019806be30 msgr2=0x7f019806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.896+0000 7f01bcc09640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f019806be30 0x7f019806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f01a0034000 tx=0x7f01a002e000 comp rx=0 tx=0).stop 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.896+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b80d2c30 msgr2=0x7f01b81525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.896+0000 7f01bcc09640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b80d2c30 0x7f01b81525e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c450 tx=0x7f01b000c910 comp rx=0 tx=0).stop 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.896+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 shutdown_connections 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.896+0000 7f01bcc09640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f019806be30 0x7f019806e2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.896+0000 7f01bcc09640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01b80cc0a0 0x7f01b80b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.896+0000 7f01bcc09640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01b815e370 0x7f01b80cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.896+0000 7f01bcc09640 1 --2- 172.21.15.26:0/1314518457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b80d2c30 0x7f01b81525e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.897+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 >> 172.21.15.26:0/1314518457 conn(0x7f01b80d40f0 msgr2=0x7f01b815d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.897+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 shutdown_connections 2024-04-24T23:21:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:35.897+0000 7f01bcc09640 1 -- 172.21.15.26:0/1314518457 wait complete. 2024-04-24T23:21:35.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:36.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.008+0000 7fd7f4fee640 1 Processor -- start 2024-04-24T23:21:36.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.008+0000 7fd7f4fee640 1 -- start start 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7f4fee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7f006f4f0 0x7fd7f00ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7f4fee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f00addb0 0x7fd7f0154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7f4fee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f00b0220 con 0x7fd7f006f110 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7f4fee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f00af180 con 0x7fd7f006f4f0 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7f4fee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f00af460 con 0x7fd7f00addb0 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7eed9d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f00addb0 0x7fd7f0154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7e7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7f006f4f0 0x7fd7f00ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7eed9d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f00addb0 0x7fd7f0154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:46866/0 (socket says 172.21.15.26:46866) 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7e7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7f006f4f0 0x7fd7f00ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:59702/0 (socket says 172.21.15.26:59702) 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7eed9d640 1 -- 172.21.15.26:0/3274588895 learned_addr learned my addr 172.21.15.26:0/3274588895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:36.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.009+0000 7fd7ee59c640 1 --1- 172.21.15.26:0/3274588895 >> v1:172.21.15.29:6789/0 conn(0x7fd7f006f110 0x7fd7f015f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:35062/0 (socket says 172.21.15.26:35062) 2024-04-24T23:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.010+0000 7fd7eed9d640 1 -- 172.21.15.26:0/3274588895 >> v2:172.21.15.29:3300/0 conn(0x7fd7f006f4f0 msgr2=0x7fd7f00ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.010+0000 7fd7eed9d640 1 --2- 172.21.15.26:0/3274588895 >> v2:172.21.15.29:3300/0 conn(0x7fd7f006f4f0 0x7fd7f00ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.010+0000 7fd7eed9d640 1 -- 172.21.15.26:0/3274588895 >> v1:172.21.15.29:6789/0 conn(0x7fd7f006f110 legacy=0x7fd7f015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.010+0000 7fd7eed9d640 1 -- 172.21.15.26:0/3274588895 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f00af740 con 0x7fd7f00addb0 2024-04-24T23:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.010+0000 7fd7eed9d640 1 --2- 172.21.15.26:0/3274588895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f00addb0 0x7fd7f0154a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009e50 tx=0x7fd7e802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1853b15ddc2ce2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7edd9b640 1 -- 172.21.15.26:0/3274588895 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8036070 con 0x7fd7f00addb0 2024-04-24T23:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7edd9b640 1 -- 172.21.15.26:0/3274588895 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e8031600 con 0x7fd7f00addb0 2024-04-24T23:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7edd9b640 1 -- 172.21.15.26:0/3274588895 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80319d0 con 0x7fd7f00addb0 2024-04-24T23:21:36.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7f4fee640 1 -- 172.21.15.26:0/3274588895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f00addb0 msgr2=0x7fd7f0154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7f4fee640 1 --2- 172.21.15.26:0/3274588895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f00addb0 0x7fd7f0154a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009e50 tx=0x7fd7e802a710 comp rx=0 tx=0).stop 2024-04-24T23:21:36.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7f4fee640 1 -- 172.21.15.26:0/3274588895 shutdown_connections 2024-04-24T23:21:36.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7f4fee640 1 --2- 172.21.15.26:0/3274588895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f00addb0 0x7fd7f0154a50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7f4fee640 1 --2- 172.21.15.26:0/3274588895 >> v2:172.21.15.29:3300/0 conn(0x7fd7f006f4f0 0x7fd7f00ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7f4fee640 1 -- 172.21.15.26:0/3274588895 >> 172.21.15.26:0/3274588895 conn(0x7fd7f00729a0 msgr2=0x7fd7f006b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:36.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7f4fee640 1 -- 172.21.15.26:0/3274588895 shutdown_connections 2024-04-24T23:21:36.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.011+0000 7fd7f4fee640 1 -- 172.21.15.26:0/3274588895 wait complete. 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.013+0000 7fd7f4fee640 1 Processor -- start 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.013+0000 7fd7f4fee640 1 -- start start 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.013+0000 7fd7f4fee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f006f110 0x7fd7f00c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.013+0000 7fd7f4fee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7f006f4f0 0x7fd7f01f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.013+0000 7fd7f4fee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f00addb0 0x7fd7f0158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.013+0000 7fd7f4fee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f00aee50 con 0x7fd7f00addb0 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.013+0000 7fd7f4fee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f00afd00 con 0x7fd7f006f4f0 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.013+0000 7fd7f4fee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f006fb00 con 0x7fd7f006f110 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.013+0000 7fd7eed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f00addb0 0x7fd7f0158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7eed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f00addb0 0x7fd7f0158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35280/0 (socket says 172.21.15.26:35280) 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7eed9d640 1 -- 172.21.15.26:0/86611330 learned_addr learned my addr 172.21.15.26:0/86611330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7ee59c640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f006f110 0x7fd7f00c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7e7fff640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7f006f4f0 0x7fd7f01f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7eed9d640 1 -- 172.21.15.26:0/86611330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f006f110 msgr2=0x7fd7f00c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7eed9d640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f006f110 0x7fd7f00c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7eed9d640 1 -- 172.21.15.26:0/86611330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7f006f4f0 msgr2=0x7fd7f01f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7eed9d640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7f006f4f0 0x7fd7f01f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7eed9d640 1 -- 172.21.15.26:0/86611330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f00c6b90 con 0x7fd7f00addb0 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7eed9d640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f00addb0 0x7fd7f0158dc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8034040 tx=0x7fd7e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7e6ffd640 1 -- 172.21.15.26:0/86611330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8036050 con 0x7fd7f00addb0 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7e6ffd640 1 -- 172.21.15.26:0/86611330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e8042070 con 0x7fd7f00addb0 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f00c6e70 con 0x7fd7f00addb0 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7e6ffd640 1 -- 172.21.15.26:0/86611330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8009510 con 0x7fd7f00addb0 2024-04-24T23:21:36.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.014+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f00c7430 con 0x7fd7f00addb0 2024-04-24T23:21:36.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.015+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f00694c0 con 0x7fd7f00addb0 2024-04-24T23:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.016+0000 7fd7e6ffd640 1 -- 172.21.15.26:0/86611330 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7e8007c70 con 0x7fd7f00addb0 2024-04-24T23:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.016+0000 7fd7e6ffd640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7d006be30 0x7fd7d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.016+0000 7fd7e6ffd640 1 -- 172.21.15.26:0/86611330 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7fd7e80abc00 con 0x7fd7f00addb0 2024-04-24T23:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.016+0000 7fd7ee59c640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7d006be30 0x7fd7d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.017+0000 7fd7ee59c640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7d006be30 0x7fd7d006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0007fd0 tx=0x7fd7e0007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:36.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.021+0000 7fd7e6ffd640 1 -- 172.21.15.26:0/86611330 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7e80b2080 con 0x7fd7f00addb0 2024-04-24T23:21:36.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.241+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7f00b6900 con 0x7fd7f00addb0 2024-04-24T23:21:36.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.241+0000 7fd7e6ffd640 1 -- 172.21.15.26:0/86611330 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7fd7e8079ee0 con 0x7fd7f00addb0 2024-04-24T23:21:36.243 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:36.244 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2829963318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2829963318},{"type":"v1","addr":"172.21.15.26:6837","nonce":2829963318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2687649505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2687649505},{"type":"v1","addr":"172.21.15.29:6833","nonce":2687649505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:20:54.833525+0000","modified":"2024-04-24T23:20:57.181910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1604449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1604449840},{"type":"v1","addr":"172.21.15.29:6835","nonce":1604449840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-24T23:21:36.244 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-04-24T23:21:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7d006be30 msgr2=0x7fd7d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7d006be30 0x7fd7d006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0007fd0 tx=0x7fd7e0007e80 comp rx=0 tx=0).stop 2024-04-24T23:21:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f00addb0 msgr2=0x7fd7f0158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f00addb0 0x7fd7f0158dc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8034040 tx=0x7fd7e8004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 shutdown_connections 2024-04-24T23:21:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7d006be30 0x7fd7d006e2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7f00addb0 0x7fd7f0158dc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7f006f4f0 0x7fd7f01f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 --2- 172.21.15.26:0/86611330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7f006f110 0x7fd7f00c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 >> 172.21.15.26:0/86611330 conn(0x7fd7f00729a0 msgr2=0x7fd7f01611b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 shutdown_connections 2024-04-24T23:21:36.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.244+0000 7fd7f4fee640 1 -- 172.21.15.26:0/86611330 wait complete. 2024-04-24T23:21:36.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.356+0000 7f3885691640 1 Processor -- start 2024-04-24T23:21:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.356+0000 7f3885691640 1 -- start start 2024-04-24T23:21:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f3885691640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f388015e220 0x7f38800cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f3885691640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38800cf340 con 0x7f38800d2240 2024-04-24T23:21:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f3885691640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38800cf620 con 0x7f38800d2de0 2024-04-24T23:21:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f3885691640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f38800cf900 con 0x7f388015e220 2024-04-24T23:21:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f387effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f388015e220 0x7f38800cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f387e7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38800d2240 0x7f38800d26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59644/0 (socket says 172.21.15.26:59644) 2024-04-24T23:21:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f387e7fc640 1 -- 172.21.15.26:0/2132734112 learned_addr learned my addr 172.21.15.26:0/2132734112 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f387effd640 1 -- 172.21.15.26:0/2132734112 >> v1:172.21.15.29:6789/0 conn(0x7f38800d2de0 legacy=0x7f38800cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f387effd640 1 -- 172.21.15.26:0/2132734112 >> v1:172.21.15.26:6789/0 conn(0x7f38800d2240 legacy=0x7f38800d26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f387dffb640 1 -- 172.21.15.26:0/2132734112 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787049774 0 0) 0x7f38800cf340 con 0x7f38800d2240 2024-04-24T23:21:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.357+0000 7f387effd640 1 -- 172.21.15.26:0/2132734112 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38800cfbe0 con 0x7f388015e220 2024-04-24T23:21:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.358+0000 7f387effd640 1 --2- 172.21.15.26:0/2132734112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388015e220 0x7f38800cecd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3870009d90 tx=0x7f387002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e657c588ff77f303 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.358+0000 7f387dffb640 1 -- 172.21.15.26:0/2132734112 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870031600 con 0x7f388015e220 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.358+0000 7f387dffb640 1 -- 172.21.15.26:0/2132734112 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38700317a0 con 0x7f388015e220 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.358+0000 7f387dffb640 1 -- 172.21.15.26:0/2132734112 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870031b70 con 0x7f388015e220 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.359+0000 7f3885691640 1 -- 172.21.15.26:0/2132734112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388015e220 msgr2=0x7f38800cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.359+0000 7f3885691640 1 --2- 172.21.15.26:0/2132734112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388015e220 0x7f38800cecd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3870009d90 tx=0x7f387002a610 comp rx=0 tx=0).stop 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.359+0000 7f3885691640 1 -- 172.21.15.26:0/2132734112 shutdown_connections 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.359+0000 7f3885691640 1 --2- 172.21.15.26:0/2132734112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f388015e220 0x7f38800cecd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.359+0000 7f3885691640 1 -- 172.21.15.26:0/2132734112 >> 172.21.15.26:0/2132734112 conn(0x7f388009d500 msgr2=0x7f38800d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.359+0000 7f3885691640 1 -- 172.21.15.26:0/2132734112 shutdown_connections 2024-04-24T23:21:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.359+0000 7f3885691640 1 -- 172.21.15.26:0/2132734112 wait complete. 2024-04-24T23:21:36.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.360+0000 7f3885691640 1 Processor -- start 2024-04-24T23:21:36.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.360+0000 7f3885691640 1 -- start start 2024-04-24T23:21:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f3885691640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38800d2240 0x7f388015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f3885691640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38800d2de0 0x7f38801edf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f3885691640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f388015e220 0x7f3880157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f3885691640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f388015e600 con 0x7f38800d2de0 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f3885691640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38800d1ee0 con 0x7f388015e220 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f3885691640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f388015e780 con 0x7f38800d2240 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38800d2de0 0x7f38801edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38800d2de0 0x7f38801edf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35298/0 (socket says 172.21.15.26:35298) 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387e7fc640 1 -- 172.21.15.26:0/3312363687 learned_addr learned my addr 172.21.15.26:0/3312363687 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387effd640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38800d2240 0x7f388015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387f7fe640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f388015e220 0x7f3880157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387e7fc640 1 -- 172.21.15.26:0/3312363687 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38800d2240 msgr2=0x7f388015aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387e7fc640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38800d2240 0x7f388015aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387e7fc640 1 -- 172.21.15.26:0/3312363687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f388015e220 msgr2=0x7f3880157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387e7fc640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f388015e220 0x7f3880157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387e7fc640 1 -- 172.21.15.26:0/3312363687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3880158590 con 0x7f38800d2de0 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.361+0000 7f387f7fe640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f388015e220 0x7f3880157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.362+0000 7f387e7fc640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38800d2de0 0x7f38801edf30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f387400c430 tx=0x7f387400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.362+0000 7f38677fe640 1 -- 172.21.15.26:0/3312363687 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38740074e0 con 0x7f38800d2de0 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.362+0000 7f38677fe640 1 -- 172.21.15.26:0/3312363687 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38740047e0 con 0x7f38800d2de0 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.362+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3880159f70 con 0x7f38800d2de0 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.362+0000 7f38677fe640 1 -- 172.21.15.26:0/3312363687 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874004bd0 con 0x7f38800d2de0 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.362+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388015a530 con 0x7f38800d2de0 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.363+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38800694c0 con 0x7f38800d2de0 2024-04-24T23:21:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.363+0000 7f38677fe640 1 -- 172.21.15.26:0/3312363687 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3874011070 con 0x7f38800d2de0 2024-04-24T23:21:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.364+0000 7f38677fe640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386c06be30 0x7f386c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.364+0000 7f387effd640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386c06be30 0x7f386c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.364+0000 7f38677fe640 1 -- 172.21.15.26:0/3312363687 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f387408aa40 con 0x7f38800d2de0 2024-04-24T23:21:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.364+0000 7f387effd640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386c06be30 0x7f386c06e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3870034000 tx=0x7f387002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.368+0000 7f38677fe640 1 -- 172.21.15.26:0/3312363687 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3874059400 con 0x7f38800d2de0 2024-04-24T23:21:36.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.551+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38800d50d0 con 0x7f38800d2de0 2024-04-24T23:21:36.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.552+0000 7f38677fe640 1 -- 172.21.15.26:0/3312363687 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16738 (secure 0 0 0) 0x7f3874058d00 con 0x7f38800d2de0 2024-04-24T23:21:36.554 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:36.554 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:35.565675+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:36.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386c06be30 msgr2=0x7f386c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386c06be30 0x7f386c06e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3870034000 tx=0x7f387002f000 comp rx=0 tx=0).stop 2024-04-24T23:21:36.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38800d2de0 msgr2=0x7f38801edf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38800d2de0 0x7f38801edf30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f387400c430 tx=0x7f387400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 shutdown_connections 2024-04-24T23:21:36.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f386c06be30 0x7f386c06e2f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f388015e220 0x7f3880157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38800d2de0 0x7f38801edf30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 --2- 172.21.15.26:0/3312363687 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38800d2240 0x7f388015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 >> 172.21.15.26:0/3312363687 conn(0x7f388009d500 msgr2=0x7f388015cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:36.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 shutdown_connections 2024-04-24T23:21:36.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.554+0000 7f3885691640 1 -- 172.21.15.26:0/3312363687 wait complete. 2024-04-24T23:21:36.567 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:21:36.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:36.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.667+0000 7faca399e640 1 Processor -- start 2024-04-24T23:21:36.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.668+0000 7faca399e640 1 -- start start 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.668+0000 7faca399e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c15e570 0x7fac9c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.668+0000 7faca399e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac9c0b3130 0x7fac9c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.668+0000 7faca399e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c0d32f0 con 0x7fac9c0d22e0 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.668+0000 7faca399e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c0d1ab0 con 0x7fac9c0b3130 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.668+0000 7faca399e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c0d2ae0 con 0x7fac9c15e570 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca0f39640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac9c0b3130 0x7fac9c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca173a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c15e570 0x7fac9c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca1f3b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fac9c0d22e0 0x7fac9c0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:35088/0 (socket says 172.21.15.26:35088) 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca173a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c15e570 0x7fac9c0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:46902/0 (socket says 172.21.15.26:46902) 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca0f39640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fac9c0b3130 0x7fac9c0d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:59736/0 (socket says 172.21.15.26:59736) 2024-04-24T23:21:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca1f3b640 1 -- 172.21.15.26:0/2963023348 learned_addr learned my addr 172.21.15.26:0/2963023348 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7fac9bfff640 1 -- 172.21.15.26:0/2963023348 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784225821 0 0) 0x7fac8c001490 con 0x7fac9c0d22e0 2024-04-24T23:21:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7fac9bfff640 1 -- 172.21.15.26:0/2963023348 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac84003520 con 0x7fac9c0d22e0 2024-04-24T23:21:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca173a640 1 -- 172.21.15.26:0/2963023348 >> v2:172.21.15.29:3300/0 conn(0x7fac9c0b3130 msgr2=0x7fac9c0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca173a640 1 --2- 172.21.15.26:0/2963023348 >> v2:172.21.15.29:3300/0 conn(0x7fac9c0b3130 0x7fac9c0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca173a640 1 -- 172.21.15.26:0/2963023348 >> v1:172.21.15.29:6789/0 conn(0x7fac9c0d22e0 legacy=0x7fac9c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca173a640 1 -- 172.21.15.26:0/2963023348 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0d0270 con 0x7fac9c15e570 2024-04-24T23:21:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.669+0000 7faca0f39640 1 --2- 172.21.15.26:0/2963023348 >> v2:172.21.15.29:3300/0 conn(0x7fac9c0b3130 0x7fac9c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.670+0000 7faca173a640 1 --2- 172.21.15.26:0/2963023348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c15e570 0x7fac9c0d4160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac940097a0 tx=0x7fac9402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7203b33fd9cd4db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.670+0000 7fac9bfff640 1 -- 172.21.15.26:0/2963023348 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94037070 con 0x7fac9c15e570 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.670+0000 7fac9bfff640 1 -- 172.21.15.26:0/2963023348 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac94032440 con 0x7fac9c15e570 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.670+0000 7fac9bfff640 1 -- 172.21.15.26:0/2963023348 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94032810 con 0x7fac9c15e570 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.671+0000 7faca399e640 1 -- 172.21.15.26:0/2963023348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c15e570 msgr2=0x7fac9c0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.671+0000 7faca399e640 1 --2- 172.21.15.26:0/2963023348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c15e570 0x7fac9c0d4160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac940097a0 tx=0x7fac9402a180 comp rx=0 tx=0).stop 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.671+0000 7faca399e640 1 -- 172.21.15.26:0/2963023348 shutdown_connections 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.671+0000 7faca399e640 1 --2- 172.21.15.26:0/2963023348 >> v2:172.21.15.29:3300/0 conn(0x7fac9c0b3130 0x7fac9c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.671+0000 7faca399e640 1 --2- 172.21.15.26:0/2963023348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c15e570 0x7fac9c0d4160 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.671+0000 7faca399e640 1 -- 172.21.15.26:0/2963023348 >> 172.21.15.26:0/2963023348 conn(0x7fac9c08dbe0 msgr2=0x7fac9c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.671+0000 7faca399e640 1 -- 172.21.15.26:0/2963023348 shutdown_connections 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.671+0000 7faca399e640 1 -- 172.21.15.26:0/2963023348 wait complete. 2024-04-24T23:21:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.672+0000 7faca399e640 1 Processor -- start 2024-04-24T23:21:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.672+0000 7faca399e640 1 -- start start 2024-04-24T23:21:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca399e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c0b3130 0x7fac9c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca399e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c0d22e0 0x7fac9c0a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca399e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c15e570 0x7fac9c0a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca399e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c0cfa70 con 0x7fac9c0b3130 2024-04-24T23:21:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca399e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c0d3010 con 0x7fac9c15e570 2024-04-24T23:21:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca399e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c0cfd50 con 0x7fac9c0d22e0 2024-04-24T23:21:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca173a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c0b3130 0x7fac9c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca173a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c0b3130 0x7fac9c14f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35308/0 (socket says 172.21.15.26:35308) 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca173a640 1 -- 172.21.15.26:0/2583135564 learned_addr learned my addr 172.21.15.26:0/2583135564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca0f39640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c0d22e0 0x7fac9c0a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca1f3b640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c15e570 0x7fac9c0a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca173a640 1 -- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c0d22e0 msgr2=0x7fac9c0a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca173a640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c0d22e0 0x7fac9c0a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca173a640 1 -- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c15e570 msgr2=0x7fac9c0a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca173a640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c15e570 0x7fac9c0a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca173a640 1 -- 172.21.15.26:0/2583135564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0b8270 con 0x7fac9c0b3130 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca0f39640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c0d22e0 0x7fac9c0a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca1f3b640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c15e570 0x7fac9c0a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.673+0000 7faca173a640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c0b3130 0x7fac9c14f4f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fac94035040 tx=0x7fac940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.674+0000 7fac99ffb640 1 -- 172.21.15.26:0/2583135564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94041030 con 0x7fac9c0b3130 2024-04-24T23:21:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.674+0000 7fac99ffb640 1 -- 172.21.15.26:0/2583135564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9403d4e0 con 0x7fac9c0b3130 2024-04-24T23:21:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.674+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c0b8550 con 0x7fac9c0b3130 2024-04-24T23:21:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.674+0000 7fac99ffb640 1 -- 172.21.15.26:0/2583135564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94037040 con 0x7fac9c0b3130 2024-04-24T23:21:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.674+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c0b8b10 con 0x7fac9c0b3130 2024-04-24T23:21:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.675+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c0694c0 con 0x7fac9c0b3130 2024-04-24T23:21:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.675+0000 7fac99ffb640 1 -- 172.21.15.26:0/2583135564 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac94044070 con 0x7fac9c0b3130 2024-04-24T23:21:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.676+0000 7fac99ffb640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fac8806be30 0x7fac8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.676+0000 7fac99ffb640 1 -- 172.21.15.26:0/2583135564 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fac940ab520 con 0x7fac9c0b3130 2024-04-24T23:21:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.676+0000 7faca0f39640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fac8806be30 0x7fac8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.676+0000 7faca0f39640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fac8806be30 0x7fac8806e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0b9f00 tx=0x7fac90008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.680+0000 7fac99ffb640 1 -- 172.21.15.26:0/2583135564 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fac94044e40 con 0x7fac9c0b3130 2024-04-24T23:21:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.903+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac9c0cf830 con 0x7fac9c0b3130 2024-04-24T23:21:36.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.904+0000 7fac99ffb640 1 -- 172.21.15.26:0/2583135564 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7fac94079e60 con 0x7fac9c0b3130 2024-04-24T23:21:36.906 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:36.906 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2829963318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2829963318},{"type":"v1","addr":"172.21.15.26:6837","nonce":2829963318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2687649505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2687649505},{"type":"v1","addr":"172.21.15.29:6833","nonce":2687649505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:20:54.833525+0000","modified":"2024-04-24T23:20:57.181910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1604449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1604449840},{"type":"v1","addr":"172.21.15.29:6835","nonce":1604449840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-24T23:21:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fac8806be30 msgr2=0x7fac8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fac8806be30 0x7fac8806e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0b9f00 tx=0x7fac90008040 comp rx=0 tx=0).stop 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c0b3130 msgr2=0x7fac9c14f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c0b3130 0x7fac9c14f4f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fac94035040 tx=0x7fac940046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 shutdown_connections 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fac8806be30 0x7fac8806e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c15e570 0x7fac9c0a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fac9c0d22e0 0x7fac9c0a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 --2- 172.21.15.26:0/2583135564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c0b3130 0x7fac9c14f4f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 >> 172.21.15.26:0/2583135564 conn(0x7fac9c08dbe0 msgr2=0x7fac9c0d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 shutdown_connections 2024-04-24T23:21:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:36.906+0000 7faca399e640 1 -- 172.21.15.26:0/2583135564 wait complete. 2024-04-24T23:21:36.918 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.017+0000 7fd228ed2640 1 Processor -- start 2024-04-24T23:21:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.017+0000 7fd228ed2640 1 -- start start 2024-04-24T23:21:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd228ed2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 0x7fd2240d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd228ed2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2240b72f0 0x7fd2240b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd228ed2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2240b5840 con 0x7fd2240b3ac0 2024-04-24T23:21:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd228ed2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd2240b4460 con 0x7fd2240b72f0 2024-04-24T23:21:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd228ed2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd2240b5e50 con 0x7fd22415e290 2024-04-24T23:21:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd222d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2240b72f0 0x7fd2240b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd222d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2240b72f0 0x7fd2240b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35318/0 (socket says 172.21.15.26:35318) 2024-04-24T23:21:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd222d9d640 1 -- 172.21.15.26:0/3630841176 learned_addr learned my addr 172.21.15.26:0/3630841176 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd22259c640 1 --2- 172.21.15.26:0/3630841176 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 0x7fd2240d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd222d9d640 1 -- 172.21.15.26:0/3630841176 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 msgr2=0x7fd2240d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd222d9d640 1 --2- 172.21.15.26:0/3630841176 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 0x7fd2240d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd222d9d640 1 -- 172.21.15.26:0/3630841176 >> v1:172.21.15.29:6789/0 conn(0x7fd2240b3ac0 legacy=0x7fd22406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.018+0000 7fd222d9d640 1 -- 172.21.15.26:0/3630841176 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2240cfb30 con 0x7fd2240b72f0 2024-04-24T23:21:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd222d9d640 1 --2- 172.21.15.26:0/3630841176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b72f0 0x7fd2240b3f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009e50 tx=0x7fd21c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0818475ec44ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd221d9b640 1 -- 172.21.15.26:0/3630841176 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c036070 con 0x7fd2240b72f0 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd221d9b640 1 -- 172.21.15.26:0/3630841176 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21c031600 con 0x7fd2240b72f0 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd221d9b640 1 -- 172.21.15.26:0/3630841176 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0319d0 con 0x7fd2240b72f0 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd228ed2640 1 -- 172.21.15.26:0/3630841176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b72f0 msgr2=0x7fd2240b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd228ed2640 1 --2- 172.21.15.26:0/3630841176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b72f0 0x7fd2240b3f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009e50 tx=0x7fd21c02a710 comp rx=0 tx=0).stop 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd228ed2640 1 -- 172.21.15.26:0/3630841176 shutdown_connections 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd228ed2640 1 --2- 172.21.15.26:0/3630841176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b72f0 0x7fd2240b3f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd228ed2640 1 --2- 172.21.15.26:0/3630841176 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 0x7fd2240d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.019+0000 7fd228ed2640 1 -- 172.21.15.26:0/3630841176 >> 172.21.15.26:0/3630841176 conn(0x7fd22406b120 msgr2=0x7fd22406b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.020+0000 7fd228ed2640 1 -- 172.21.15.26:0/3630841176 shutdown_connections 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.020+0000 7fd228ed2640 1 -- 172.21.15.26:0/3630841176 wait complete. 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.021+0000 7fd228ed2640 1 Processor -- start 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.021+0000 7fd228ed2640 1 -- start start 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd228ed2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b3ac0 0x7fd224161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd228ed2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2240b72f0 0x7fd224147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd228ed2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 0x7fd2241e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd228ed2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2240b5ae0 con 0x7fd2240b3ac0 2024-04-24T23:21:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd228ed2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd2240d00f0 con 0x7fd2240b72f0 2024-04-24T23:21:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd228ed2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd22415e670 con 0x7fd22415e290 2024-04-24T23:21:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd22259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b3ac0 0x7fd224161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd22259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b3ac0 0x7fd224161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35326/0 (socket says 172.21.15.26:35326) 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd22259c640 1 -- 172.21.15.26:0/3453265499 learned_addr learned my addr 172.21.15.26:0/3453265499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd222d9d640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 0x7fd2241e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd21bfff640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2240b72f0 0x7fd224147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd22259c640 1 -- 172.21.15.26:0/3453265499 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 msgr2=0x7fd2241e9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd22259c640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 0x7fd2241e9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd22259c640 1 -- 172.21.15.26:0/3453265499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2240b72f0 msgr2=0x7fd224147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd22259c640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2240b72f0 0x7fd224147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd22259c640 1 -- 172.21.15.26:0/3453265499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2241479c0 con 0x7fd2240b3ac0 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd222d9d640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 0x7fd2241e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd22259c640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b3ac0 0x7fd224161090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd21400d620 tx=0x7fd21400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.022+0000 7fd21affd640 1 -- 172.21.15.26:0/3453265499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21400b620 con 0x7fd2240b3ac0 2024-04-24T23:21:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.023+0000 7fd21affd640 1 -- 172.21.15.26:0/3453265499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd214019070 con 0x7fd2240b3ac0 2024-04-24T23:21:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.023+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2240c85c0 con 0x7fd2240b3ac0 2024-04-24T23:21:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.023+0000 7fd21affd640 1 -- 172.21.15.26:0/3453265499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21400ba00 con 0x7fd2240b3ac0 2024-04-24T23:21:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.023+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2241615d0 con 0x7fd2240b3ac0 2024-04-24T23:21:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.024+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2240694c0 con 0x7fd2240b3ac0 2024-04-24T23:21:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.024+0000 7fd21affd640 1 -- 172.21.15.26:0/3453265499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd21400bba0 con 0x7fd2240b3ac0 2024-04-24T23:21:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.024+0000 7fd21affd640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd20406be30 0x7fd20406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.025+0000 7fd21affd640 1 -- 172.21.15.26:0/3453265499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd21408b2e0 con 0x7fd2240b3ac0 2024-04-24T23:21:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.025+0000 7fd21bfff640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd20406be30 0x7fd20406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.025+0000 7fd21bfff640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd20406be30 0x7fd20406e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2241f7890 tx=0x7fd20c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.029+0000 7fd21affd640 1 -- 172.21.15.26:0/3453265499 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd214059c10 con 0x7fd2240b3ac0 2024-04-24T23:21:37.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.258+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd22415a9f0 con 0x7fd2240b3ac0 2024-04-24T23:21:37.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.259+0000 7fd21affd640 1 -- 172.21.15.26:0/3453265499 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5175 (secure 0 0 0) 0x7fd214059510 con 0x7fd2240b3ac0 2024-04-24T23:21:37.261 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:37.261 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1014415490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1014415490},{"type":"v1","addr":"172.21.15.26:6836","nonce":1014415490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2829963318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2829963318},{"type":"v1","addr":"172.21.15.26:6837","nonce":2829963318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2687649505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2687649505},{"type":"v1","addr":"172.21.15.29:6833","nonce":2687649505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:20:54.833525+0000","modified":"2024-04-24T23:20:57.181910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/1604449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1604449840},{"type":"v1","addr":"172.21.15.29:6835","nonce":1604449840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-24T23:21:37.261 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-04-24T23:21:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd20406be30 msgr2=0x7fd20406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd20406be30 0x7fd20406e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2241f7890 tx=0x7fd20c009040 comp rx=0 tx=0).stop 2024-04-24T23:21:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b3ac0 msgr2=0x7fd224161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b3ac0 0x7fd224161090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd21400d620 tx=0x7fd21400dae0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 shutdown_connections 2024-04-24T23:21:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd20406be30 0x7fd20406e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd22415e290 0x7fd2241e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2240b72f0 0x7fd224147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 --2- 172.21.15.26:0/3453265499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2240b3ac0 0x7fd224161090 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 >> 172.21.15.26:0/3453265499 conn(0x7fd22406b120 msgr2=0x7fd2240b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 shutdown_connections 2024-04-24T23:21:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.261+0000 7fd228ed2640 1 -- 172.21.15.26:0/3453265499 wait complete. 2024-04-24T23:21:37.274 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.374+0000 7f8f012e1640 1 Processor -- start 2024-04-24T23:21:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.374+0000 7f8f012e1640 1 -- start start 2024-04-24T23:21:37.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.375+0000 7f8f012e1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8efc15c2e0 0x7f8efc0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.375+0000 7f8f012e1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8efc0d0af0 con 0x7f8efc0cdbc0 2024-04-24T23:21:37.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.375+0000 7f8f012e1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8efc0d0dd0 con 0x7f8efc0cf0a0 2024-04-24T23:21:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.375+0000 7f8f012e1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8efc0ccba0 con 0x7f8efc15c2e0 2024-04-24T23:21:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.375+0000 7f8efaffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8efc15c2e0 0x7f8efc0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.375+0000 7f8efa7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8efc0cdbc0 0x7f8efc0ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59652/0 (socket says 172.21.15.26:59652) 2024-04-24T23:21:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.375+0000 7f8efa7fc640 1 -- 172.21.15.26:0/204653822 learned_addr learned my addr 172.21.15.26:0/204653822 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.376+0000 7f8efaffd640 1 -- 172.21.15.26:0/204653822 >> v1:172.21.15.29:6789/0 conn(0x7f8efc0cf0a0 legacy=0x7f8efc0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.376+0000 7f8efaffd640 1 -- 172.21.15.26:0/204653822 >> v1:172.21.15.26:6789/0 conn(0x7f8efc0cdbc0 legacy=0x7f8efc0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.376+0000 7f8ef9ffb640 1 -- 172.21.15.26:0/204653822 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2116447277 0 0) 0x7f8efc0d0af0 con 0x7f8efc0cdbc0 2024-04-24T23:21:37.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.376+0000 7f8efaffd640 1 -- 172.21.15.26:0/204653822 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc0cce80 con 0x7f8efc15c2e0 2024-04-24T23:21:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.376+0000 7f8efaffd640 1 --2- 172.21.15.26:0/204653822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 0x7f8efc0cd680 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40097a0 tx=0x7f8ef402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172146c936d7ec20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8ef9ffb640 1 -- 172.21.15.26:0/204653822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4037070 con 0x7f8efc15c2e0 2024-04-24T23:21:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8ef9ffb640 1 -- 172.21.15.26:0/204653822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef4032440 con 0x7f8efc15c2e0 2024-04-24T23:21:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8ef9ffb640 1 -- 172.21.15.26:0/204653822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4032810 con 0x7f8efc15c2e0 2024-04-24T23:21:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8f012e1640 1 -- 172.21.15.26:0/204653822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 msgr2=0x7f8efc0cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8f012e1640 1 --2- 172.21.15.26:0/204653822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 0x7f8efc0cd680 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40097a0 tx=0x7f8ef402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8f012e1640 1 -- 172.21.15.26:0/204653822 shutdown_connections 2024-04-24T23:21:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8f012e1640 1 --2- 172.21.15.26:0/204653822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 0x7f8efc0cd680 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8f012e1640 1 -- 172.21.15.26:0/204653822 >> 172.21.15.26:0/204653822 conn(0x7f8efc0d1f80 msgr2=0x7f8efc0d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8f012e1640 1 -- 172.21.15.26:0/204653822 shutdown_connections 2024-04-24T23:21:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.377+0000 7f8f012e1640 1 -- 172.21.15.26:0/204653822 wait complete. 2024-04-24T23:21:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.378+0000 7f8f012e1640 1 Processor -- start 2024-04-24T23:21:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.378+0000 7f8f012e1640 1 -- start start 2024-04-24T23:21:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8f012e1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8efc0cdbc0 0x7f8efc0b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8f012e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8efc0cf0a0 0x7f8efc14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8f012e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 0x7f8efc0af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8f012e1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc0b20f0 con 0x7f8efc15c2e0 2024-04-24T23:21:37.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8f012e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc0b1f70 con 0x7f8efc0cf0a0 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8f012e1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8efc0b2270 con 0x7f8efc0cdbc0 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 0x7f8efc0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8efb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 0x7f8efc0af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35340/0 (socket says 172.21.15.26:35340) 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8efb7fe640 1 -- 172.21.15.26:0/618230030 learned_addr learned my addr 172.21.15.26:0/618230030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.379+0000 7f8efa7fc640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8efc0cf0a0 0x7f8efc14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8efaffd640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8efc0cdbc0 0x7f8efc0b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8efb7fe640 1 -- 172.21.15.26:0/618230030 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8efc0cdbc0 msgr2=0x7f8efc0b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8efc0cdbc0 0x7f8efc0b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8efb7fe640 1 -- 172.21.15.26:0/618230030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8efc0cf0a0 msgr2=0x7f8efc14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8efc0cf0a0 0x7f8efc14dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8efb7fe640 1 -- 172.21.15.26:0/618230030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc0b7a30 con 0x7f8efc15c2e0 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8efa7fc640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8efc0cf0a0 0x7f8efc14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8efb7fe640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 0x7f8efc0af710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c7b0 tx=0x7f8eec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8ef37fe640 1 -- 172.21.15.26:0/618230030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec004b60 con 0x7f8efc15c2e0 2024-04-24T23:21:37.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8ef37fe640 1 -- 172.21.15.26:0/618230030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec004d00 con 0x7f8efc15c2e0 2024-04-24T23:21:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8efc14dff0 con 0x7f8efc15c2e0 2024-04-24T23:21:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8ef37fe640 1 -- 172.21.15.26:0/618230030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec00aa90 con 0x7f8efc15c2e0 2024-04-24T23:21:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.380+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8efc0b7470 con 0x7f8efc15c2e0 2024-04-24T23:21:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.381+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8efc0d19b0 con 0x7f8efc15c2e0 2024-04-24T23:21:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.382+0000 7f8ef37fe640 1 -- 172.21.15.26:0/618230030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8eec00ac30 con 0x7f8efc15c2e0 2024-04-24T23:21:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.382+0000 7f8ef37fe640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8edc06be30 0x7f8edc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.382+0000 7f8ef37fe640 1 -- 172.21.15.26:0/618230030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8eec08b160 con 0x7f8efc15c2e0 2024-04-24T23:21:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.382+0000 7f8efaffd640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8edc06be30 0x7f8edc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.383+0000 7f8efaffd640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8edc06be30 0x7f8edc06e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4034000 tx=0x7f8ef402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:37.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.386+0000 7f8ef37fe640 1 -- 172.21.15.26:0/618230030 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8efc0d19b0 con 0x7f8efc15c2e0 2024-04-24T23:21:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.571+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8efc0681e0 con 0x7f8efc15c2e0 2024-04-24T23:21:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.572+0000 7f8ef37fe640 1 -- 172.21.15.26:0/618230030 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16738 (secure 0 0 0) 0x7f8eec020070 con 0x7f8efc15c2e0 2024-04-24T23:21:37.574 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:37.574 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:36.571165+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:20:53.333231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:20:53.808050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8edc06be30 msgr2=0x7f8edc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8edc06be30 0x7f8edc06e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4034000 tx=0x7f8ef402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 msgr2=0x7f8efc0af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 0x7f8efc0af710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c7b0 tx=0x7f8eec00cc70 comp rx=0 tx=0).stop 2024-04-24T23:21:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 shutdown_connections 2024-04-24T23:21:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8edc06be30 0x7f8edc06e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc15c2e0 0x7f8efc0af710 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8efc0cf0a0 0x7f8efc14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 --2- 172.21.15.26:0/618230030 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8efc0cdbc0 0x7f8efc0b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 >> 172.21.15.26:0/618230030 conn(0x7f8efc0d1f80 msgr2=0x7f8efc06f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.574+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 shutdown_connections 2024-04-24T23:21:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.575+0000 7f8f012e1640 1 -- 172.21.15.26:0/618230030 wait complete. 2024-04-24T23:21:37.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:21:37.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.688+0000 7f841ccf5640 1 Processor -- start 2024-04-24T23:21:37.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.688+0000 7f841ccf5640 1 -- start start 2024-04-24T23:21:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f841ccf5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 0x7f841806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f841ccf5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f841806d350 0x7f841815d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f841ccf5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841815d570 con 0x7f84180cd660 2024-04-24T23:21:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f841ccf5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8418145cf0 con 0x7f841806d350 2024-04-24T23:21:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f841ccf5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f84180cfbc0 con 0x7f8418145910 2024-04-24T23:21:37.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f840ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f841806d350 0x7f841815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f841659c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 0x7f841806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f840ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f841806d350 0x7f841815d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:59774/0 (socket says 172.21.15.26:59774) 2024-04-24T23:21:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f840ffff640 1 -- 172.21.15.26:0/3426416469 learned_addr learned my addr 172.21.15.26:0/3426416469 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.689+0000 7f8416d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f84180cd660 0x7f841815d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:35120/0 (socket says 172.21.15.26:35120) 2024-04-24T23:21:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.690+0000 7f840ffff640 1 -- 172.21.15.26:0/3426416469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 msgr2=0x7f841806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.690+0000 7f840ffff640 1 --2- 172.21.15.26:0/3426416469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 0x7f841806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.690+0000 7f840ffff640 1 -- 172.21.15.26:0/3426416469 >> v1:172.21.15.29:6789/0 conn(0x7f84180cd660 legacy=0x7f841815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.690+0000 7f840ffff640 1 -- 172.21.15.26:0/3426416469 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841806d7f0 con 0x7f841806d350 2024-04-24T23:21:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.690+0000 7f8415d9b640 1 -- 172.21.15.26:0/3426416469 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 276559135 0 0) 0x7f8410001490 con 0x7f84180cd660 2024-04-24T23:21:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.690+0000 7f840ffff640 1 --2- 172.21.15.26:0/3426416469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841806d350 0x7f841815d030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8400009870 tx=0x7f840002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2799ef9d3bc0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f8415d9b640 1 -- 172.21.15.26:0/3426416469 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400037070 con 0x7f841806d350 2024-04-24T23:21:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f8415d9b640 1 -- 172.21.15.26:0/3426416469 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400032440 con 0x7f841806d350 2024-04-24T23:21:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f8415d9b640 1 -- 172.21.15.26:0/3426416469 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400032810 con 0x7f841806d350 2024-04-24T23:21:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f841ccf5640 1 -- 172.21.15.26:0/3426416469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841806d350 msgr2=0x7f841815d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f841ccf5640 1 --2- 172.21.15.26:0/3426416469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841806d350 0x7f841815d030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8400009870 tx=0x7f840002a270 comp rx=0 tx=0).stop 2024-04-24T23:21:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f841ccf5640 1 -- 172.21.15.26:0/3426416469 shutdown_connections 2024-04-24T23:21:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f841ccf5640 1 --2- 172.21.15.26:0/3426416469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841806d350 0x7f841815d030 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f841ccf5640 1 --2- 172.21.15.26:0/3426416469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 0x7f841806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f841ccf5640 1 -- 172.21.15.26:0/3426416469 >> 172.21.15.26:0/3426416469 conn(0x7f841806b300 msgr2=0x7f841806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f841ccf5640 1 -- 172.21.15.26:0/3426416469 shutdown_connections 2024-04-24T23:21:37.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.691+0000 7f841ccf5640 1 -- 172.21.15.26:0/3426416469 wait complete. 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.692+0000 7f841ccf5640 1 Processor -- start 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.692+0000 7f841ccf5640 1 -- start start 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f841ccf5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841806d350 0x7f84181433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f841ccf5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84180cd660 0x7f84180aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f841ccf5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 0x7f84181f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f841ccf5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f841806e090 con 0x7f84180cd660 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f841ccf5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f841806ddb0 con 0x7f841806d350 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f841ccf5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f841806e370 con 0x7f8418145910 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f840ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84180cd660 0x7f84180aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f840ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84180cd660 0x7f84180aa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35352/0 (socket says 172.21.15.26:35352) 2024-04-24T23:21:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f840ffff640 1 -- 172.21.15.26:0/1501162469 learned_addr learned my addr 172.21.15.26:0/1501162469 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f841659c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841806d350 0x7f84181433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.693+0000 7f8416d9d640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 0x7f84181f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f840ffff640 1 -- 172.21.15.26:0/1501162469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 msgr2=0x7f84181f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f840ffff640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 0x7f84181f2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f840ffff640 1 -- 172.21.15.26:0/1501162469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841806d350 msgr2=0x7f84181433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f840ffff640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841806d350 0x7f84181433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f840ffff640 1 -- 172.21.15.26:0/1501162469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84181f5c10 con 0x7f84180cd660 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f840ffff640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84180cd660 0x7f84180aa2c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8400035040 tx=0x7f84000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f840effd640 1 -- 172.21.15.26:0/1501162469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400037040 con 0x7f84180cd660 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f840effd640 1 -- 172.21.15.26:0/1501162469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400004030 con 0x7f84180cd660 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84180a9500 con 0x7f84180cd660 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f840effd640 1 -- 172.21.15.26:0/1501162469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400004420 con 0x7f84180cd660 2024-04-24T23:21:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.694+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84180a9da0 con 0x7f84180cd660 2024-04-24T23:21:37.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.695+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84180694c0 con 0x7f84180cd660 2024-04-24T23:21:37.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.696+0000 7f840effd640 1 -- 172.21.15.26:0/1501162469 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f840003a070 con 0x7f84180cd660 2024-04-24T23:21:37.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.696+0000 7f840effd640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83f806be30 0x7f83f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:37.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.696+0000 7f840effd640 1 -- 172.21.15.26:0/1501162469 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f84000ab2f0 con 0x7f84180cd660 2024-04-24T23:21:37.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.696+0000 7f841659c640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83f806be30 0x7f83f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:37.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.697+0000 7f841659c640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83f806be30 0x7f83f806e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8408009b40 tx=0x7f8408009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:37.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.700+0000 7f840effd640 1 -- 172.21.15.26:0/1501162469 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8400041020 con 0x7f84180cd660 2024-04-24T23:21:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.922+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f84180a96d0 con 0x7f84180cd660 2024-04-24T23:21:37.924 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:21:37.922+0000 7f26a0f44640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:21:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.924+0000 7f840effd640 1 -- 172.21.15.26:0/1501162469 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f8400079660 con 0x7f84180cd660 2024-04-24T23:21:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:21:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83f806be30 msgr2=0x7f83f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83f806be30 0x7f83f806e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8408009b40 tx=0x7f8408009190 comp rx=0 tx=0).stop 2024-04-24T23:21:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84180cd660 msgr2=0x7f84180aa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84180cd660 0x7f84180aa2c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8400035040 tx=0x7f84000046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 shutdown_connections 2024-04-24T23:21:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83f806be30 0x7f83f806e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8418145910 0x7f84181f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84180cd660 0x7f84180aa2c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 --2- 172.21.15.26:0/1501162469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f841806d350 0x7f84181433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.926+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 >> 172.21.15.26:0/1501162469 conn(0x7f841806b300 msgr2=0x7f8418143f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.927+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 shutdown_connections 2024-04-24T23:21:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:37.927+0000 7f841ccf5640 1 -- 172.21.15.26:0/1501162469 wait complete. 2024-04-24T23:21:37.933 INFO:tasks.ceph.mds.d.smithi029.stderr: -7508> 2024-04-24T23:20:59.819+0000 7f4257c3b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:21:37.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:21:37.967 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:21:38.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.040+0000 7f21a87f9640 1 Processor -- start 2024-04-24T23:21:38.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.040+0000 7f21a87f9640 1 -- start start 2024-04-24T23:21:38.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.041+0000 7f21a87f9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f21a015e750 0x7f21a00cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.041+0000 7f21a87f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a00d2120 con 0x7f21a00d2c30 2024-04-24T23:21:38.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.041+0000 7f21a87f9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a00b3740 con 0x7f21a015e370 2024-04-24T23:21:38.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.041+0000 7f21a87f9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f21a00d1510 con 0x7f21a015e750 2024-04-24T23:21:38.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.041+0000 7f21a6d96640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21a00d2c30 0x7f21a00cf1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59668/0 (socket says 172.21.15.26:59668) 2024-04-24T23:21:38.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.041+0000 7f21a6d96640 1 -- 172.21.15.26:0/445771648 learned_addr learned my addr 172.21.15.26:0/445771648 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:38.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.041+0000 7f21a5d94640 1 --2- 172.21.15.26:0/445771648 >> v2:172.21.15.29:3300/0 conn(0x7f21a015e750 0x7f21a00cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.041+0000 7f21a5593640 1 -- 172.21.15.26:0/445771648 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 110177193 0 0) 0x7f21a00d2120 con 0x7f21a00d2c30 2024-04-24T23:21:38.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.041+0000 7f21a5593640 1 -- 172.21.15.26:0/445771648 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f218c003520 con 0x7f21a00d2c30 2024-04-24T23:21:38.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.042+0000 7f21a5d94640 1 -- 172.21.15.26:0/445771648 >> v1:172.21.15.29:6789/0 conn(0x7f21a015e370 legacy=0x7f21a00d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.042+0000 7f21a5593640 1 -- 172.21.15.26:0/445771648 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2938333598 0 0) 0x7f21a00b3740 con 0x7f21a015e370 2024-04-24T23:21:38.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.042+0000 7f21a5d94640 1 -- 172.21.15.26:0/445771648 >> v1:172.21.15.26:6789/0 conn(0x7f21a00d2c30 legacy=0x7f21a00cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.042+0000 7f21a5d94640 1 -- 172.21.15.26:0/445771648 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a00b3a20 con 0x7f21a015e750 2024-04-24T23:21:38.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.042+0000 7f21a5d94640 1 --2- 172.21.15.26:0/445771648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a015e750 0x7f21a00cec90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f21900097a0 tx=0x7f219002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59408b1bd3d8ec3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:38.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.042+0000 7f21a5593640 1 -- 172.21.15.26:0/445771648 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190037070 con 0x7f21a015e750 2024-04-24T23:21:38.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.042+0000 7f21a5593640 1 -- 172.21.15.26:0/445771648 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2190032440 con 0x7f21a015e750 2024-04-24T23:21:38.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.043+0000 7f21a5593640 1 -- 172.21.15.26:0/445771648 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190032810 con 0x7f21a015e750 2024-04-24T23:21:38.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.043+0000 7f21a87f9640 1 -- 172.21.15.26:0/445771648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a015e750 msgr2=0x7f21a00cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.043+0000 7f21a87f9640 1 --2- 172.21.15.26:0/445771648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a015e750 0x7f21a00cec90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f21900097a0 tx=0x7f219002a230 comp rx=0 tx=0).stop 2024-04-24T23:21:38.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.043+0000 7f21a87f9640 1 -- 172.21.15.26:0/445771648 shutdown_connections 2024-04-24T23:21:38.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.043+0000 7f21a87f9640 1 --2- 172.21.15.26:0/445771648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a015e750 0x7f21a00cec90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.043+0000 7f21a87f9640 1 -- 172.21.15.26:0/445771648 >> 172.21.15.26:0/445771648 conn(0x7f21a00d40f0 msgr2=0x7f21a00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:38.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.043+0000 7f21a87f9640 1 -- 172.21.15.26:0/445771648 shutdown_connections 2024-04-24T23:21:38.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.043+0000 7f21a87f9640 1 -- 172.21.15.26:0/445771648 wait complete. 2024-04-24T23:21:38.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.044+0000 7f21a87f9640 1 Processor -- start 2024-04-24T23:21:38.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.044+0000 7f21a87f9640 1 -- start start 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a87f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a00d2c30 0x7f21a015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a87f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21a015e370 0x7f21a0157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a87f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00cbf90 0x7f21a00cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a87f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21a00b5230 con 0x7f21a00cbf90 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a87f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f21a00d1ed0 con 0x7f21a00d2c30 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a87f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f21a00b53b0 con 0x7f21a015e370 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6d96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00cbf90 0x7f21a00cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6d96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00cbf90 0x7f21a00cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35366/0 (socket says 172.21.15.26:35366) 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6d96640 1 -- 172.21.15.26:0/335861554 learned_addr learned my addr 172.21.15.26:0/335861554 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6595640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a00d2c30 0x7f21a015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6d96640 1 -- 172.21.15.26:0/335861554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21a015e370 msgr2=0x7f21a0157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a5d94640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21a015e370 0x7f21a0157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6d96640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21a015e370 0x7f21a0157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6d96640 1 -- 172.21.15.26:0/335861554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a00d2c30 msgr2=0x7f21a015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6d96640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a00d2c30 0x7f21a015a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6d96640 1 -- 172.21.15.26:0/335861554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a0151eb0 con 0x7f21a00cbf90 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.045+0000 7f21a6595640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a00d2c30 0x7f21a015a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.046+0000 7f21a6d96640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00cbf90 0x7f21a00cc440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f219800c3f0 tx=0x7f2198001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.046+0000 7f2196ffd640 1 -- 172.21.15.26:0/335861554 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198017070 con 0x7f21a00cbf90 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.046+0000 7f2196ffd640 1 -- 172.21.15.26:0/335861554 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2198005ce0 con 0x7f21a00cbf90 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.046+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a00cb390 con 0x7f21a00cbf90 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.046+0000 7f2196ffd640 1 -- 172.21.15.26:0/335861554 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21980072f0 con 0x7f21a00cbf90 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.046+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a00cb630 con 0x7f21a00cbf90 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.047+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a00d12e0 con 0x7f21a00cbf90 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.047+0000 7f2196ffd640 1 -- 172.21.15.26:0/335861554 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f219801a070 con 0x7f21a00cbf90 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.048+0000 7f2196ffd640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f218406be60 0x7f218406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.048+0000 7f2196ffd640 1 -- 172.21.15.26:0/335861554 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f219808a370 con 0x7f21a00cbf90 2024-04-24T23:21:38.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.048+0000 7f21a6595640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f218406be60 0x7f218406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.048+0000 7f21a6595640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f218406be60 0x7f218406e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f219c0012e0 tx=0x7f219c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:38.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.052+0000 7f2196ffd640 1 -- 172.21.15.26:0/335861554 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f21a00d12e0 con 0x7f21a00cbf90 2024-04-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.278+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f21a00b1060 con 0x7f21a00cbf90 2024-04-24T23:21:38.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.925+0000 7f2196ffd640 1 -- 172.21.15.26:0/335861554 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f2198058bb0 con 0x7f21a00cbf90 2024-04-24T23:21:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f218406be60 msgr2=0x7f218406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f218406be60 0x7f218406e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f219c0012e0 tx=0x7f219c009040 comp rx=0 tx=0).stop 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00cbf90 msgr2=0x7f21a00cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00cbf90 0x7f21a00cc440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f219800c3f0 tx=0x7f2198001e70 comp rx=0 tx=0).stop 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 shutdown_connections 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f218406be60 0x7f218406e320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00cbf90 0x7f21a00cc440 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21a015e370 0x7f21a0157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 --2- 172.21.15.26:0/335861554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21a00d2c30 0x7f21a015a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.928+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 >> 172.21.15.26:0/335861554 conn(0x7f21a00d40f0 msgr2=0x7f21a00d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.929+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 shutdown_connections 2024-04-24T23:21:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:38.929+0000 7f21a87f9640 1 -- 172.21.15.26:0/335861554 wait complete. 2024-04-24T23:21:38.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:21:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.047+0000 7fa5a6c15640 1 Processor -- start 2024-04-24T23:21:39.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.047+0000 7fa5a6c15640 1 -- start start 2024-04-24T23:21:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa5a6c15640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5a00d0c00 0x7fa5a00cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa5a6c15640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c730 0x7fa5a00d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa5a6c15640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a00cfe20 con 0x7fa5a015c350 2024-04-24T23:21:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa5a6c15640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a00cdf40 con 0x7fa5a00d0c00 2024-04-24T23:21:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa5a6c15640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a00b2fd0 con 0x7fa5a015c730 2024-04-24T23:21:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa597fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5a00d0c00 0x7fa5a00cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa597fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5a00d0c00 0x7fa5a00cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35378/0 (socket says 172.21.15.26:35378) 2024-04-24T23:21:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa597fff640 1 -- 172.21.15.26:0/4217173708 learned_addr learned my addr 172.21.15.26:0/4217173708 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa5a49b1640 1 --1- 172.21.15.26:0/4217173708 >> v1:172.21.15.26:6789/0 conn(0x7fa5a015c350 0x7fa5a00d04f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59672/0 (socket says 172.21.15.26:59672) 2024-04-24T23:21:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa5a51b2640 1 --2- 172.21.15.26:0/4217173708 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c730 0x7fa5a00d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa597fff640 1 -- 172.21.15.26:0/4217173708 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c730 msgr2=0x7fa5a00d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa597fff640 1 --2- 172.21.15.26:0/4217173708 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c730 0x7fa5a00d2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa597fff640 1 -- 172.21.15.26:0/4217173708 >> v1:172.21.15.26:6789/0 conn(0x7fa5a015c350 legacy=0x7fa5a00d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa597fff640 1 -- 172.21.15.26:0/4217173708 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a00d0100 con 0x7fa5a00d0c00 2024-04-24T23:21:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.048+0000 7fa5977fe640 1 -- 172.21.15.26:0/4217173708 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3666091847 0 0) 0x7fa5a00cfe20 con 0x7fa5a015c350 2024-04-24T23:21:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.049+0000 7fa597fff640 1 --2- 172.21.15.26:0/4217173708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 0x7fa5a00cef50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa590009870 tx=0x7fa59002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4afe5c97d7e00e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.049+0000 7fa5977fe640 1 -- 172.21.15.26:0/4217173708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590037070 con 0x7fa5a00d0c00 2024-04-24T23:21:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.049+0000 7fa5977fe640 1 -- 172.21.15.26:0/4217173708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590032440 con 0x7fa5a00d0c00 2024-04-24T23:21:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.049+0000 7fa5977fe640 1 -- 172.21.15.26:0/4217173708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590032810 con 0x7fa5a00d0c00 2024-04-24T23:21:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.049+0000 7fa5a6c15640 1 -- 172.21.15.26:0/4217173708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 msgr2=0x7fa5a00cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.049+0000 7fa5a6c15640 1 --2- 172.21.15.26:0/4217173708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 0x7fa5a00cef50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa590009870 tx=0x7fa59002a130 comp rx=0 tx=0).stop 2024-04-24T23:21:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.050+0000 7fa5a6c15640 1 -- 172.21.15.26:0/4217173708 shutdown_connections 2024-04-24T23:21:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.050+0000 7fa5a6c15640 1 --2- 172.21.15.26:0/4217173708 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c730 0x7fa5a00d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.050+0000 7fa5a6c15640 1 --2- 172.21.15.26:0/4217173708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 0x7fa5a00cef50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.050+0000 7fa5a6c15640 1 -- 172.21.15.26:0/4217173708 >> 172.21.15.26:0/4217173708 conn(0x7fa5a00d1f10 msgr2=0x7fa5a00d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.050+0000 7fa5a6c15640 1 -- 172.21.15.26:0/4217173708 shutdown_connections 2024-04-24T23:21:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.050+0000 7fa5a6c15640 1 -- 172.21.15.26:0/4217173708 wait complete. 2024-04-24T23:21:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.051+0000 7fa5a6c15640 1 Processor -- start 2024-04-24T23:21:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.051+0000 7fa5a6c15640 1 -- start start 2024-04-24T23:21:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a6c15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 0x7fa5a006d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a6c15640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c350 0x7fa5a00cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a6c15640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5a015c730 0x7fa5a01534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a6c15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00c5cc0 con 0x7fa5a00d0c00 2024-04-24T23:21:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a6c15640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00c5b40 con 0x7fa5a015c730 2024-04-24T23:21:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a6c15640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a00c5e40 con 0x7fa5a015c350 2024-04-24T23:21:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a49b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 0x7fa5a006d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a49b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 0x7fa5a006d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35388/0 (socket says 172.21.15.26:35388) 2024-04-24T23:21:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a49b1640 1 -- 172.21.15.26:0/2402920402 learned_addr learned my addr 172.21.15.26:0/2402920402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa597fff640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c350 0x7fa5a00cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a51b2640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5a015c730 0x7fa5a01534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a49b1640 1 -- 172.21.15.26:0/2402920402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c350 msgr2=0x7fa5a00cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a49b1640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c350 0x7fa5a00cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a49b1640 1 -- 172.21.15.26:0/2402920402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5a015c730 msgr2=0x7fa5a01534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a49b1640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5a015c730 0x7fa5a01534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a49b1640 1 -- 172.21.15.26:0/2402920402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a01599a0 con 0x7fa5a00d0c00 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a51b2640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5a015c730 0x7fa5a01534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a49b1640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 0x7fa5a006d2c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00c500 tx=0x7fa59c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5957fa640 1 -- 172.21.15.26:0/2402920402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c018070 con 0x7fa5a00d0c00 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5957fa640 1 -- 172.21.15.26:0/2402920402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c007580 con 0x7fa5a00d0c00 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5957fa640 1 -- 172.21.15.26:0/2402920402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c004a20 con 0x7fa5a00d0c00 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.052+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a0153a00 con 0x7fa5a00d0c00 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.053+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a006dbd0 con 0x7fa5a00d0c00 2024-04-24T23:21:39.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.054+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a00694c0 con 0x7fa5a00d0c00 2024-04-24T23:21:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.054+0000 7fa5957fa640 1 -- 172.21.15.26:0/2402920402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa59c01b070 con 0x7fa5a00d0c00 2024-04-24T23:21:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.054+0000 7fa5957fa640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa58c06be30 0x7fa58c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.054+0000 7fa5957fa640 1 -- 172.21.15.26:0/2402920402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fa59c08a0f0 con 0x7fa5a00d0c00 2024-04-24T23:21:39.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.054+0000 7fa597fff640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa58c06be30 0x7fa58c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:39.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.055+0000 7fa597fff640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa58c06be30 0x7fa58c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa590034000 tx=0x7fa59002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.059+0000 7fa5957fa640 1 -- 172.21.15.26:0/2402920402 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa5a0153a00 con 0x7fa5a00d0c00 2024-04-24T23:21:39.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.241+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5a00704e0 con 0x7fa5a00d0c00 2024-04-24T23:21:39.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.929+0000 7fa5957fa640 1 -- 172.21.15.26:0/2402920402 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7fa59c058410 con 0x7fa5a00d0c00 2024-04-24T23:21:39.931 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.932+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa58c06be30 msgr2=0x7fa58c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa58c06be30 0x7fa58c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa590034000 tx=0x7fa59002e000 comp rx=0 tx=0).stop 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 msgr2=0x7fa5a006d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 0x7fa5a006d2c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00c500 tx=0x7fa59c00c9c0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 shutdown_connections 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa58c06be30 0x7fa58c06e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5a015c730 0x7fa5a01534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa5a015c350 0x7fa5a00cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 --2- 172.21.15.26:0/2402920402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a00d0c00 0x7fa5a006d2c0 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00c500 tx=0x7fa59c00c9c0 comp rx=0 tx=0).stop 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 >> 172.21.15.26:0/2402920402 conn(0x7fa5a00d1f10 msgr2=0x7fa5a0157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 shutdown_connections 2024-04-24T23:21:39.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:39.933+0000 7fa5a6c15640 1 -- 172.21.15.26:0/2402920402 wait complete. 2024-04-24T23:21:39.948 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:21:40.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.058+0000 7f80740b3640 1 Processor -- start 2024-04-24T23:21:40.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.058+0000 7f80740b3640 1 -- start start 2024-04-24T23:21:40.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f80740b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f806c15ea80 0x7f806c141160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:40.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f80740b3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f806c0711a0 0x7f806c0715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:40.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f80740b3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c0cf050 con 0x7f806c15ee60 2024-04-24T23:21:40.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f80740b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f806c0cf330 con 0x7f806c15ea80 2024-04-24T23:21:40.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f80740b3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f806c0cf610 con 0x7f806c0711a0 2024-04-24T23:21:40.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f8071e4f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f806c15ea80 0x7f806c141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:40.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f8071e4f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f806c15ea80 0x7f806c141160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35390/0 (socket says 172.21.15.26:35390) 2024-04-24T23:21:40.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f8071e4f640 1 -- 172.21.15.26:0/1897650983 learned_addr learned my addr 172.21.15.26:0/1897650983 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f8072650640 1 --2- 172.21.15.26:0/1897650983 >> v2:172.21.15.29:3300/0 conn(0x7f806c0711a0 0x7f806c0715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f8071e4f640 1 -- 172.21.15.26:0/1897650983 >> v2:172.21.15.29:3300/0 conn(0x7f806c0711a0 msgr2=0x7f806c0715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f8071e4f640 1 --2- 172.21.15.26:0/1897650983 >> v2:172.21.15.29:3300/0 conn(0x7f806c0711a0 0x7f806c0715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.059+0000 7f8071e4f640 1 -- 172.21.15.26:0/1897650983 >> v1:172.21.15.29:6789/0 conn(0x7f806c15ee60 legacy=0x7f806c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.060+0000 7f8071e4f640 1 -- 172.21.15.26:0/1897650983 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c0cf8f0 con 0x7f806c15ea80 2024-04-24T23:21:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.060+0000 7f8071e4f640 1 --2- 172.21.15.26:0/1897650983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c15ea80 0x7f806c141160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f806c06eff0 tx=0x7f806402c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a937979a5cae82e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.060+0000 7f8070e4d640 1 -- 172.21.15.26:0/1897650983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064009600 con 0x7f806c15ea80 2024-04-24T23:21:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.060+0000 7f8070e4d640 1 -- 172.21.15.26:0/1897650983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80640097a0 con 0x7f806c15ea80 2024-04-24T23:21:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.060+0000 7f8070e4d640 1 -- 172.21.15.26:0/1897650983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064009b70 con 0x7f806c15ea80 2024-04-24T23:21:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.061+0000 7f80740b3640 1 -- 172.21.15.26:0/1897650983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c15ea80 msgr2=0x7f806c141160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.061+0000 7f80740b3640 1 --2- 172.21.15.26:0/1897650983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c15ea80 0x7f806c141160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f806c06eff0 tx=0x7f806402c900 comp rx=0 tx=0).stop 2024-04-24T23:21:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.061+0000 7f80740b3640 1 -- 172.21.15.26:0/1897650983 shutdown_connections 2024-04-24T23:21:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.061+0000 7f80740b3640 1 --2- 172.21.15.26:0/1897650983 >> v2:172.21.15.29:3300/0 conn(0x7f806c0711a0 0x7f806c0715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.061+0000 7f80740b3640 1 --2- 172.21.15.26:0/1897650983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c15ea80 0x7f806c141160 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.061+0000 7f80740b3640 1 -- 172.21.15.26:0/1897650983 >> 172.21.15.26:0/1897650983 conn(0x7f806c06b300 msgr2=0x7f806c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.061+0000 7f80740b3640 1 -- 172.21.15.26:0/1897650983 shutdown_connections 2024-04-24T23:21:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.061+0000 7f80740b3640 1 -- 172.21.15.26:0/1897650983 wait complete. 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.062+0000 7f80740b3640 1 Processor -- start 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.062+0000 7f80740b3640 1 -- start start 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f80740b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c0711a0 0x7f806c15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f80740b3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f806c15ea80 0x7f806c15e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f80740b3640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f806c15ee60 0x7f806c1517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f80740b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f806c13e090 con 0x7f806c0711a0 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f80740b3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f806c15f2d0 con 0x7f806c15ea80 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f80740b3640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f806c13e210 con 0x7f806c15ee60 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f8071e4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c0711a0 0x7f806c15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f8071e4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c0711a0 0x7f806c15a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35394/0 (socket says 172.21.15.26:35394) 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f8071e4f640 1 -- 172.21.15.26:0/4058324452 learned_addr learned my addr 172.21.15.26:0/4058324452 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f8071e4f640 1 -- 172.21.15.26:0/4058324452 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f806c15ee60 msgr2=0x7f806c1517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f8071e4f640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f806c15ee60 0x7f806c1517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f8071e4f640 1 -- 172.21.15.26:0/4058324452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f806c15ea80 msgr2=0x7f806c15e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f8071e4f640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f806c15ea80 0x7f806c15e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.063+0000 7f8071e4f640 1 -- 172.21.15.26:0/4058324452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c154810 con 0x7f806c0711a0 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.064+0000 7f8071e4f640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c0711a0 0x7f806c15a1d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f806402cef0 tx=0x7f8064004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.064+0000 7f806a7fc640 1 -- 172.21.15.26:0/4058324452 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064042070 con 0x7f806c0711a0 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.064+0000 7f806a7fc640 1 -- 172.21.15.26:0/4058324452 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8064034c80 con 0x7f806c0711a0 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.064+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806c154af0 con 0x7f806c0711a0 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.064+0000 7f806a7fc640 1 -- 172.21.15.26:0/4058324452 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064009600 con 0x7f806c0711a0 2024-04-24T23:21:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.064+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806c1550b0 con 0x7f806c0711a0 2024-04-24T23:21:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.065+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f806c0694c0 con 0x7f806c0711a0 2024-04-24T23:21:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.065+0000 7f806a7fc640 1 -- 172.21.15.26:0/4058324452 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8064045070 con 0x7f806c0711a0 2024-04-24T23:21:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.066+0000 7f806a7fc640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f805806be30 0x7f805806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.066+0000 7f806a7fc640 1 -- 172.21.15.26:0/4058324452 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f80640ac040 con 0x7f806c0711a0 2024-04-24T23:21:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.066+0000 7f807164e640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f805806be30 0x7f805806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.066+0000 7f807164e640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f805806be30 0x7f805806e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8060006fd0 tx=0x7f8060006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.070+0000 7f806a7fc640 1 -- 172.21.15.26:0/4058324452 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f806407ab60 con 0x7f806c0711a0 2024-04-24T23:21:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.253+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f806c072f00 con 0x7f806c0711a0 2024-04-24T23:21:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.931+0000 7f806a7fc640 1 -- 172.21.15.26:0/4058324452 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f806407a460 con 0x7f806c0711a0 2024-04-24T23:21:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-24T23:21:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f805806be30 msgr2=0x7f805806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f805806be30 0x7f805806e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8060006fd0 tx=0x7f8060006bc0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c0711a0 msgr2=0x7f806c15a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c0711a0 0x7f806c15a1d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f806402cef0 tx=0x7f8064004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 shutdown_connections 2024-04-24T23:21:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f805806be30 0x7f805806e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f806c15ee60 0x7f806c1517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f806c15ea80 0x7f806c15e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 --2- 172.21.15.26:0/4058324452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c0711a0 0x7f806c15a1d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 >> 172.21.15.26:0/4058324452 conn(0x7f806c06b300 msgr2=0x7f806c06dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 shutdown_connections 2024-04-24T23:21:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:40.934+0000 7f80740b3640 1 -- 172.21.15.26:0/4058324452 wait complete. 2024-04-24T23:21:40.950 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:21:40.950 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:21:40.950 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:21:40.950 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:21:40.951 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:21:40.951 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:21:40.951 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:21:40.951 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:21:40.951 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:21:40.951 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:21:40.951 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:21:40.951 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:21:40.952 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:21:40.952 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:21:40.952 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:21:40.952 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:21:40.964 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:21:40.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:21:40.965 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:21:40.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:21:40.966 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:21:40.967 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:21:40.968 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:21:40.968 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:21:40.970 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:21:40.970 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:40.971 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:21:40.972 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:41.012 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:21:41.012 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:41.013 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:21:41.013 DEBUG:teuthology.parallel:result is None 2024-04-24T23:21:41.013 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:21:41.051 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:21:41.052 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:21:41.096 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:21:41.097 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:21:41.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.120+0000 7f285d134640 1 Processor -- start 2024-04-24T23:21:41.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.120+0000 7f285d134640 1 -- start start 2024-04-24T23:21:41.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285d134640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285815e570 0x7f28580d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:41.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285d134640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28580b3130 0x7f28580d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285d134640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28580d32f0 con 0x7f28580d22e0 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285d134640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f28580d1ab0 con 0x7f28580b3130 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285d134640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f28580d2ae0 con 0x7f285815e570 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28580b3130 0x7f28580d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28580b3130 0x7f28580d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35428/0 (socket says 172.21.15.26:35428) 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285659c640 1 -- 172.21.15.26:0/3908292939 learned_addr learned my addr 172.21.15.26:0/3908292939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285759e640 1 --1- 172.21.15.26:0/3908292939 >> v1:172.21.15.26:6789/0 conn(0x7f28580d22e0 0x7f28580d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59684/0 (socket says 172.21.15.26:59684) 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f2856d9d640 1 --2- 172.21.15.26:0/3908292939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285815e570 0x7f28580d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285659c640 1 -- 172.21.15.26:0/3908292939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285815e570 msgr2=0x7f28580d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285659c640 1 --2- 172.21.15.26:0/3908292939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285815e570 0x7f28580d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285659c640 1 -- 172.21.15.26:0/3908292939 >> v1:172.21.15.26:6789/0 conn(0x7f28580d22e0 legacy=0x7f28580d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f285659c640 1 -- 172.21.15.26:0/3908292939 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28580d0270 con 0x7f28580b3130 2024-04-24T23:21:41.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.121+0000 7f2855d9b640 1 -- 172.21.15.26:0/3908292939 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357060872 0 0) 0x7f2850001490 con 0x7f28580d22e0 2024-04-24T23:21:41.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.122+0000 7f285659c640 1 --2- 172.21.15.26:0/3908292939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580b3130 0x7f28580d46a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f284c009870 tx=0x7f284c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f84965b555e3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:41.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.122+0000 7f2855d9b640 1 -- 172.21.15.26:0/3908292939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c037070 con 0x7f28580b3130 2024-04-24T23:21:41.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.122+0000 7f2855d9b640 1 -- 172.21.15.26:0/3908292939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c032440 con 0x7f28580b3130 2024-04-24T23:21:41.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.122+0000 7f2855d9b640 1 -- 172.21.15.26:0/3908292939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c032810 con 0x7f28580b3130 2024-04-24T23:21:41.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.123+0000 7f285d134640 1 -- 172.21.15.26:0/3908292939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580b3130 msgr2=0x7f28580d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:41.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.123+0000 7f285d134640 1 --2- 172.21.15.26:0/3908292939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580b3130 0x7f28580d46a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f284c009870 tx=0x7f284c02a270 comp rx=0 tx=0).stop 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.123+0000 7f285d134640 1 -- 172.21.15.26:0/3908292939 shutdown_connections 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.123+0000 7f285d134640 1 --2- 172.21.15.26:0/3908292939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580b3130 0x7f28580d46a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.123+0000 7f285d134640 1 --2- 172.21.15.26:0/3908292939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285815e570 0x7f28580d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.123+0000 7f285d134640 1 -- 172.21.15.26:0/3908292939 >> 172.21.15.26:0/3908292939 conn(0x7f285808dbe0 msgr2=0x7f285809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.123+0000 7f285d134640 1 -- 172.21.15.26:0/3908292939 shutdown_connections 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.123+0000 7f285d134640 1 -- 172.21.15.26:0/3908292939 wait complete. 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.124+0000 7f285d134640 1 Processor -- start 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.124+0000 7f285d134640 1 -- start start 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285d134640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28580b3130 0x7f28580b9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285d134640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580d22e0 0x7f28580b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285d134640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285815e570 0x7f28580b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285d134640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28580cfa70 con 0x7f28580d22e0 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285d134640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f28580d3010 con 0x7f285815e570 2024-04-24T23:21:41.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285d134640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f28580cfd50 con 0x7f28580b3130 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580d22e0 0x7f28580b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580d22e0 0x7f28580b6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35432/0 (socket says 172.21.15.26:35432) 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285659c640 1 -- 172.21.15.26:0/2874201741 learned_addr learned my addr 172.21.15.26:0/2874201741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285759e640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285815e570 0x7f28580b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f2856d9d640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28580b3130 0x7f28580b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.125+0000 7f285659c640 1 -- 172.21.15.26:0/2874201741 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28580b3130 msgr2=0x7f28580b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f285659c640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28580b3130 0x7f28580b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f285659c640 1 -- 172.21.15.26:0/2874201741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285815e570 msgr2=0x7f28580b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f285659c640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285815e570 0x7f28580b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f285659c640 1 -- 172.21.15.26:0/2874201741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2858159b70 con 0x7f28580d22e0 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f285759e640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285815e570 0x7f28580b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f285659c640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580d22e0 0x7f28580b6df0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f284c02a2b0 tx=0x7f284c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:41.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f283f7fe640 1 -- 172.21.15.26:0/2874201741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c041020 con 0x7f28580d22e0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f283f7fe640 1 -- 172.21.15.26:0/2874201741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c0306c0 con 0x7f28580d22e0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f283f7fe640 1 -- 172.21.15.26:0/2874201741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c037040 con 0x7f28580d22e0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2858159e50 con 0x7f28580d22e0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.126+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285815a6f0 con 0x7f28580d22e0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.127+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28580694c0 con 0x7f28580d22e0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.128+0000 7f283f7fe640 1 -- 172.21.15.26:0/2874201741 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f284c044020 con 0x7f28580d22e0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.128+0000 7f283f7fe640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284406be30 0x7f284406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.128+0000 7f283f7fe640 1 -- 172.21.15.26:0/2874201741 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f284c0ab3b0 con 0x7f28580d22e0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.128+0000 7f2856d9d640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284406be30 0x7f284406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.128+0000 7f2856d9d640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284406be30 0x7f284406e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2848006fd0 tx=0x7f2848008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.133+0000 7f283f7fe640 1 -- 172.21.15.26:0/2874201741 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f284c04b030 con 0x7f28580d22e0 2024-04-24T23:21:41.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.330+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f28580b2d80 con 0x7f28580d22e0 2024-04-24T23:21:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.947+0000 7f283f7fe640 1 -- 172.21.15.26:0/2874201741 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f284c07a060 con 0x7f28580d22e0 2024-04-24T23:21:41.949 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284406be30 msgr2=0x7f284406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284406be30 0x7f284406e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2848006fd0 tx=0x7f2848008040 comp rx=0 tx=0).stop 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580d22e0 msgr2=0x7f28580b6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580d22e0 0x7f28580b6df0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f284c02a2b0 tx=0x7f284c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 shutdown_connections 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284406be30 0x7f284406e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285815e570 0x7f28580b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28580d22e0 0x7f28580b6df0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 --2- 172.21.15.26:0/2874201741 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28580b3130 0x7f28580b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 >> 172.21.15.26:0/2874201741 conn(0x7f285808dbe0 msgr2=0x7f28580b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:41.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 shutdown_connections 2024-04-24T23:21:41.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:41.951+0000 7f285d134640 1 -- 172.21.15.26:0/2874201741 wait complete. 2024-04-24T23:21:41.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:21:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.113+0000 7f9d0a464640 1 Processor -- start 2024-04-24T23:21:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.113+0000 7f9d0a464640 1 -- start start 2024-04-24T23:21:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d0a464640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d04145910 0x7f9d0406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d0a464640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d0406d350 0x7f9d0415d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d0a464640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9d040cd660 0x7f9d040cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d0a464640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9d040cdff0 con 0x7f9d04145910 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d0a464640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9d04145cf0 con 0x7f9d040cd660 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d0a464640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9d0415d570 con 0x7f9d0406d350 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d03fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d04145910 0x7f9d0406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d03fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d04145910 0x7f9d0406e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35438/0 (socket says 172.21.15.26:35438) 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d037fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d0406d350 0x7f9d0415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d03fff640 1 -- 172.21.15.26:0/3212213424 learned_addr learned my addr 172.21.15.26:0/3212213424 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d08a01640 1 --2- 172.21.15.26:0/3212213424 >> v2:172.21.15.29:3300/0 conn(0x7f9d040cd660 0x7f9d040cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d03fff640 1 -- 172.21.15.26:0/3212213424 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d0406d350 msgr2=0x7f9d0415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d03fff640 1 --2- 172.21.15.26:0/3212213424 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d0406d350 0x7f9d0415d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d03fff640 1 -- 172.21.15.26:0/3212213424 >> v2:172.21.15.29:3300/0 conn(0x7f9d040cd660 msgr2=0x7f9d040cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d03fff640 1 --2- 172.21.15.26:0/3212213424 >> v2:172.21.15.29:3300/0 conn(0x7f9d040cd660 0x7f9d040cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.114+0000 7f9d03fff640 1 -- 172.21.15.26:0/3212213424 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d040cfbc0 con 0x7f9d04145910 2024-04-24T23:21:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.115+0000 7f9d03fff640 1 --2- 172.21.15.26:0/3212213424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 0x7f9d0406e890 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d040710b0 tx=0x7f9cec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3643eb2e70fb81e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.115+0000 7f9d02ffd640 1 -- 172.21.15.26:0/3212213424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec037070 con 0x7f9d04145910 2024-04-24T23:21:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.115+0000 7f9d02ffd640 1 -- 172.21.15.26:0/3212213424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cec032440 con 0x7f9d04145910 2024-04-24T23:21:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.115+0000 7f9d02ffd640 1 -- 172.21.15.26:0/3212213424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec032810 con 0x7f9d04145910 2024-04-24T23:21:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.116+0000 7f9d0a464640 1 -- 172.21.15.26:0/3212213424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 msgr2=0x7f9d0406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.116+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3212213424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 0x7f9d0406e890 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d040710b0 tx=0x7f9cec02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.116+0000 7f9d0a464640 1 -- 172.21.15.26:0/3212213424 shutdown_connections 2024-04-24T23:21:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.116+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3212213424 >> v2:172.21.15.29:3300/0 conn(0x7f9d040cd660 0x7f9d040cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.116+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3212213424 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d0406d350 0x7f9d0415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.116+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3212213424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 0x7f9d0406e890 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.116+0000 7f9d0a464640 1 -- 172.21.15.26:0/3212213424 >> 172.21.15.26:0/3212213424 conn(0x7f9d0406b300 msgr2=0x7f9d0406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.116+0000 7f9d0a464640 1 -- 172.21.15.26:0/3212213424 shutdown_connections 2024-04-24T23:21:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.116+0000 7f9d0a464640 1 -- 172.21.15.26:0/3212213424 wait complete. 2024-04-24T23:21:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.117+0000 7f9d0a464640 1 Processor -- start 2024-04-24T23:21:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.117+0000 7f9d0a464640 1 -- start start 2024-04-24T23:21:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d0a464640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d0406d350 0x7f9d040b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d0a464640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d040cd660 0x7f9d041f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d0a464640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 0x7f9d041e4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d0a464640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0406ddb0 con 0x7f9d04145910 2024-04-24T23:21:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d0a464640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0406dad0 con 0x7f9d0406d350 2024-04-24T23:21:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d0a464640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9d0406e090 con 0x7f9d040cd660 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d08a01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 0x7f9d041e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d08a01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 0x7f9d041e4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35450/0 (socket says 172.21.15.26:35450) 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d08a01640 1 -- 172.21.15.26:0/3902472829 learned_addr learned my addr 172.21.15.26:0/3902472829 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d03fff640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d0406d350 0x7f9d040b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d037fe640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d040cd660 0x7f9d041f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d08a01640 1 -- 172.21.15.26:0/3902472829 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d040cd660 msgr2=0x7f9d041f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d08a01640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d040cd660 0x7f9d041f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d08a01640 1 -- 172.21.15.26:0/3902472829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d0406d350 msgr2=0x7f9d040b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d08a01640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d0406d350 0x7f9d040b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d08a01640 1 -- 172.21.15.26:0/3902472829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d040b33f0 con 0x7f9d04145910 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.118+0000 7f9d03fff640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d0406d350 0x7f9d040b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.119+0000 7f9d08a01640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 0x7f9d041e4440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009dd0 tx=0x7f9cfc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.119+0000 7f9d00ff9640 1 -- 172.21.15.26:0/3902472829 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc018020 con 0x7f9d04145910 2024-04-24T23:21:42.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.119+0000 7f9d00ff9640 1 -- 172.21.15.26:0/3902472829 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cfc014480 con 0x7f9d04145910 2024-04-24T23:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.119+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d040b36d0 con 0x7f9d04145910 2024-04-24T23:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.119+0000 7f9d00ff9640 1 -- 172.21.15.26:0/3902472829 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc014870 con 0x7f9d04145910 2024-04-24T23:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.119+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d040b3c90 con 0x7f9d04145910 2024-04-24T23:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.120+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d040694c0 con 0x7f9d04145910 2024-04-24T23:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.120+0000 7f9d00ff9640 1 -- 172.21.15.26:0/3902472829 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cfc01b020 con 0x7f9d04145910 2024-04-24T23:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.121+0000 7f9d00ff9640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ce406be30 0x7f9ce406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.121+0000 7f9d00ff9640 1 -- 172.21.15.26:0/3902472829 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9cfc08b4f0 con 0x7f9d04145910 2024-04-24T23:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.121+0000 7f9d03fff640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ce406be30 0x7f9ce406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.121+0000 7f9d03fff640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ce406be30 0x7f9ce406e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9cec034000 tx=0x7f9cec02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.125+0000 7f9d00ff9640 1 -- 172.21.15.26:0/3902472829 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9cfc022030 con 0x7f9d04145910 2024-04-24T23:21:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.376+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9d0415ece0 con 0x7f9d04145910 2024-04-24T23:21:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.377+0000 7f9d00ff9640 1 -- 172.21.15.26:0/3902472829 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f9cfc010040 con 0x7f9d04145910 2024-04-24T23:21:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmg5ZUFBAAkapM9La3f+wSyLczFkm/0w==", 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPS3cHxAASacBQN+95Ds6N5668Je+DA==", 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmStpLGxAAz7molt8venundBs+CcFSbg==", 2024-04-24T23:21:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylml0TBJhAAXQkjExbw1EXc90XlcAEBBw==", 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:21:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjkylmQt74DBAAZLk9JTH3qTKRouUawqNURw==", 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-04-24T23:21:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCmkylmi6IJCxAAWBnNl2y0XIUW/YoAIt6VIQ==", 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCpkylmRcgOCRAA79pdcH0OlB2VROLSRE4dcw==", 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCskylm8SopBxAArvUph95+zD5/0LmNM8WPIA==", 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCwkylmbHipDxAAVUI9OpiMFiM/HFo03fEzSA==", 2024-04-24T23:21:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCzkylmDm4CDhAA1T/X+ywXEUCL96OGyi6S9Q==", 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC2kylmwWyVDBAAJsFFLltURqVqGLNjK9OCJQ==", 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5kylm/nLXChAA6kk0UI2hchkN2lhPO1sXHw==", 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ==", 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylm/BVuMxAAAdEb6dd3Q5eMc+I3oszOyg==", 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-04-24T23:21:42.388 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmjnJGMBAAvwAmiHrkGkDFTeL/WZbdWw==", 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPh3qOBAAa4cbQeF96mdpgNVb/l3xPg==", 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:21:42.389 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCgkylm1ZfsMxAAExymnmhW44TcXzRblp95UA==", 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.390 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mds", 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC/kylmKZ7+ARAA9p+nFRE5OCUNdaPeHlyc6Q==", 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mds" 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mgr", 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC/kylm3Kb+ARAA1FAQs7DHQ1TR/R/6KAm5uA==", 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-osd", 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC/kylmhq/+ARAA4efrwfKx4hTdmwf+n+cs2w==", 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-osd" 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd", 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC/kylmBbf+ARAAz9XbSRvJhSMrHjqmlCU5Dg==", 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC/kylmCb/+ARAAVOzmx66CqaE0erNByGoyjw==", 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rgw", 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC/kylmp8b+ARAAEov8lj6aCMNVJrZS9fnzTQ==", 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-24T23:21:42.393 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmq+8yERAAMKjK+5f2Qr35ral8L2qJMw==", 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:42.394 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-04-24T23:21:42.395 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmqYXWDBAAnuAn2h4Gb9qSFBoAZX73qQ==", 2024-04-24T23:21:42.395 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:21:42.395 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:21:42.395 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:21:42.395 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:21:42.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:42.395 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-24T23:21:42.395 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.378+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ce406be30 msgr2=0x7f9ce406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.378+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ce406be30 0x7f9ce406e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9cec034000 tx=0x7f9cec02e000 comp rx=0 tx=0).stop 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.378+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 msgr2=0x7f9d041e4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.378+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 0x7f9d041e4440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009dd0 tx=0x7f9cfc00c490 comp rx=0 tx=0).stop 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.379+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 shutdown_connections 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.379+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ce406be30 0x7f9ce406e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.379+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d04145910 0x7f9d041e4440 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.379+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9d040cd660 0x7f9d041f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.379+0000 7f9d0a464640 1 --2- 172.21.15.26:0/3902472829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d0406d350 0x7f9d040b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.379+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 >> 172.21.15.26:0/3902472829 conn(0x7f9d0406b300 msgr2=0x7f9d0415e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.379+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 shutdown_connections 2024-04-24T23:21:42.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.379+0000 7f9d0a464640 1 -- 172.21.15.26:0/3902472829 wait complete. 2024-04-24T23:21:42.398 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-24T23:21:42.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.499+0000 7ff34f815640 1 Processor -- start 2024-04-24T23:21:42.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.499+0000 7ff34f815640 1 -- start start 2024-04-24T23:21:42.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34f815640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d4a30 0x7ff3480d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34f815640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3480d0ef0 0x7ff3480cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34f815640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34815e750 con 0x7ff3480d0b10 2024-04-24T23:21:42.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34f815640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff3480d2640 con 0x7ff3480d0ef0 2024-04-24T23:21:42.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34f815640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff3480d1d30 con 0x7ff3480d4a30 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34ddb2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3480d0ef0 0x7ff3480cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34d5b1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff3480d0b10 0x7ff3480cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59700/0 (socket says 172.21.15.26:59700) 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34ddb2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3480d0ef0 0x7ff3480cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35464/0 (socket says 172.21.15.26:35464) 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34d5b1640 1 -- 172.21.15.26:0/124946433 learned_addr learned my addr 172.21.15.26:0/124946433 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34cdb0640 1 --2- 172.21.15.26:0/124946433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d4a30 0x7ff3480d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34ddb2640 1 -- 172.21.15.26:0/124946433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d4a30 msgr2=0x7ff3480d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34ddb2640 1 --2- 172.21.15.26:0/124946433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d4a30 0x7ff3480d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34ddb2640 1 -- 172.21.15.26:0/124946433 >> v1:172.21.15.26:6789/0 conn(0x7ff3480d0b10 legacy=0x7ff3480cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff34ddb2640 1 -- 172.21.15.26:0/124946433 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3480cff40 con 0x7ff3480d0ef0 2024-04-24T23:21:42.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.500+0000 7ff33ffff640 1 -- 172.21.15.26:0/124946433 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581762901 0 0) 0x7ff344001110 con 0x7ff3480d0b10 2024-04-24T23:21:42.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.501+0000 7ff34ddb2640 1 --2- 172.21.15.26:0/124946433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0ef0 0x7ff3480cb230 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff340009d90 tx=0x7ff34002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5d004cb60be724 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.501+0000 7ff33ffff640 1 -- 172.21.15.26:0/124946433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340007740 con 0x7ff3480d0ef0 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.501+0000 7ff33ffff640 1 -- 172.21.15.26:0/124946433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3400078e0 con 0x7ff3480d0ef0 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.501+0000 7ff33ffff640 1 -- 172.21.15.26:0/124946433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340007cb0 con 0x7ff3480d0ef0 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.502+0000 7ff34f815640 1 -- 172.21.15.26:0/124946433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0ef0 msgr2=0x7ff3480cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.502+0000 7ff34f815640 1 --2- 172.21.15.26:0/124946433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0ef0 0x7ff3480cb230 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff340009d90 tx=0x7ff34002a660 comp rx=0 tx=0).stop 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.502+0000 7ff34f815640 1 -- 172.21.15.26:0/124946433 shutdown_connections 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.502+0000 7ff34f815640 1 --2- 172.21.15.26:0/124946433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0ef0 0x7ff3480cb230 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.502+0000 7ff34f815640 1 --2- 172.21.15.26:0/124946433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d4a30 0x7ff3480d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.502+0000 7ff34f815640 1 -- 172.21.15.26:0/124946433 >> 172.21.15.26:0/124946433 conn(0x7ff34809d500 msgr2=0x7ff3480d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:42.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.502+0000 7ff34f815640 1 -- 172.21.15.26:0/124946433 shutdown_connections 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.502+0000 7ff34f815640 1 -- 172.21.15.26:0/124946433 wait complete. 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.503+0000 7ff34f815640 1 Processor -- start 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.503+0000 7ff34f815640 1 -- start start 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34f815640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0b10 0x7ff348160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34f815640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d0ef0 0x7ff34815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34f815640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3480d4a30 0x7ff34814e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34f815640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3480d3010 con 0x7ff3480d0b10 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34f815640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff3480d22d0 con 0x7ff3480d4a30 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34f815640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff3480d3190 con 0x7ff3480d0ef0 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34d5b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0b10 0x7ff348160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34d5b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0b10 0x7ff348160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35478/0 (socket says 172.21.15.26:35478) 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34d5b1640 1 -- 172.21.15.26:0/4168729506 learned_addr learned my addr 172.21.15.26:0/4168729506 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34cdb0640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d0ef0 0x7ff34815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34ddb2640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3480d4a30 0x7ff34814e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34d5b1640 1 -- 172.21.15.26:0/4168729506 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d0ef0 msgr2=0x7ff34815e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34d5b1640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d0ef0 0x7ff34815e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34d5b1640 1 -- 172.21.15.26:0/4168729506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3480d4a30 msgr2=0x7ff34814e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34d5b1640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3480d4a30 0x7ff34814e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34d5b1640 1 -- 172.21.15.26:0/4168729506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34815f180 con 0x7ff3480d0b10 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.504+0000 7ff34d5b1640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0b10 0x7ff348160220 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff34400c460 tx=0x7ff34400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.505+0000 7ff33dffb640 1 -- 172.21.15.26:0/4168729506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344018070 con 0x7ff3480d0b10 2024-04-24T23:21:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.505+0000 7ff33dffb640 1 -- 172.21.15.26:0/4168729506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff344007500 con 0x7ff3480d0b10 2024-04-24T23:21:42.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.505+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34815f460 con 0x7ff3480d0b10 2024-04-24T23:21:42.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.505+0000 7ff33dffb640 1 -- 172.21.15.26:0/4168729506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344004a20 con 0x7ff3480d0b10 2024-04-24T23:21:42.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.505+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34815fa20 con 0x7ff3480d0b10 2024-04-24T23:21:42.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.506+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3480694c0 con 0x7ff3480d0b10 2024-04-24T23:21:42.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.506+0000 7ff33dffb640 1 -- 172.21.15.26:0/4168729506 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff34401b070 con 0x7ff3480d0b10 2024-04-24T23:21:42.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.507+0000 7ff33dffb640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff32c06be30 0x7ff32c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.507+0000 7ff34cdb0640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff32c06be30 0x7ff32c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.507+0000 7ff33dffb640 1 -- 172.21.15.26:0/4168729506 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7ff344089d50 con 0x7ff3480d0b10 2024-04-24T23:21:42.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.507+0000 7ff34cdb0640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff32c06be30 0x7ff32c06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff348159a70 tx=0x7ff338009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:42.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.511+0000 7ff33dffb640 1 -- 172.21.15.26:0/4168729506 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff34408d030 con 0x7ff3480d0b10 2024-04-24T23:21:42.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.767+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ff3480caac0 con 0x7ff3480d0b10 2024-04-24T23:21:42.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.769+0000 7ff33dffb640 1 -- 172.21.15.26:0/4168729506 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7ff34815e750 con 0x7ff3480d0b10 2024-04-24T23:21:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff32c06be30 msgr2=0x7ff32c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff32c06be30 0x7ff32c06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff348159a70 tx=0x7ff338009190 comp rx=0 tx=0).stop 2024-04-24T23:21:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0b10 msgr2=0x7ff348160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0b10 0x7ff348160220 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff34400c460 tx=0x7ff34400c920 comp rx=0 tx=0).stop 2024-04-24T23:21:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 shutdown_connections 2024-04-24T23:21:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff32c06be30 0x7ff32c06e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3480d4a30 0x7ff34814e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3480d0ef0 0x7ff34815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 --2- 172.21.15.26:0/4168729506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3480d0b10 0x7ff348160220 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 >> 172.21.15.26:0/4168729506 conn(0x7ff34809d500 msgr2=0x7ff3480cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 shutdown_connections 2024-04-24T23:21:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.771+0000 7ff34f815640 1 -- 172.21.15.26:0/4168729506 wait complete. 2024-04-24T23:21:42.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-24T23:21:42.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.884+0000 7f08eaf12640 1 Processor -- start 2024-04-24T23:21:42.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.884+0000 7f08eaf12640 1 -- start start 2024-04-24T23:21:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08eaf12640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08e415e290 0x7f08e40d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08eaf12640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f08e40b72f0 0x7f08e40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08eaf12640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e40b5840 con 0x7f08e40b3ac0 2024-04-24T23:21:42.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08eaf12640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f08e40b4460 con 0x7f08e40b72f0 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08eaf12640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f08e40b5e50 con 0x7f08e415e290 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08dbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f08e40b3ac0 0x7f08e406c230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59716/0 (socket says 172.21.15.26:59716) 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08dbfff640 1 -- 172.21.15.26:0/79333242 learned_addr learned my addr 172.21.15.26:0/79333242 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08e8cae640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08e415e290 0x7f08e40d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08e94af640 1 --2- 172.21.15.26:0/79333242 >> v2:172.21.15.29:3300/0 conn(0x7f08e40b72f0 0x7f08e40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2972491156 0 0) 0x7f08e40b5840 con 0x7f08e40b3ac0 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08cc003520 con 0x7f08e40b3ac0 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2686634348 0 0) 0x7f08cc003520 con 0x7f08e40b3ac0 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08e415e290 msgr2=0x7f08e40d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08e415e290 0x7f08e40d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 >> v2:172.21.15.29:3300/0 conn(0x7f08e40b72f0 msgr2=0x7f08e40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 --2- 172.21.15.26:0/79333242 >> v2:172.21.15.29:3300/0 conn(0x7f08e40b72f0 0x7f08e40b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e40cfb30 con 0x7f08e40b3ac0 2024-04-24T23:21:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f08d4001670 con 0x7f08e40b3ac0 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 >> v1:172.21.15.26:6789/0 conn(0x7f08e40b3ac0 legacy=0x7f08e406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.885+0000 7f08db7fe640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08e415e290 0x7f08cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08db7fe640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08cc003f30 0x7f08cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08db7fe640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc004d40 0x7f08cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f08e40b5e50 con 0x7f08cc004d40 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc00b9c0 con 0x7f08cc003f30 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc00bb40 con 0x7f08e415e290 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08e94af640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc004d40 0x7f08cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08dbfff640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08cc003f30 0x7f08cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08e8cae640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08e415e290 0x7f08cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08e94af640 1 -- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08e415e290 msgr2=0x7f08cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:42.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08e94af640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08e415e290 0x7f08cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08e94af640 1 -- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08cc003f30 msgr2=0x7f08cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08e94af640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08cc003f30 0x7f08cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.886+0000 7f08e94af640 1 -- 172.21.15.26:0/79333242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc003520 con 0x7f08cc004d40 2024-04-24T23:21:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08e94af640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc004d40 0x7f08cc0051c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009fc0 tx=0x7f08cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201f11a4af02eb4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc009c10 con 0x7f08cc004d40 2024-04-24T23:21:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08dc036070 con 0x7f08cc004d40 2024-04-24T23:21:42.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08db7fe640 1 -- 172.21.15.26:0/79333242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc02d070 con 0x7f08cc004d40 2024-04-24T23:21:42.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08eaf12640 1 -- 172.21.15.26:0/79333242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc004d40 msgr2=0x7f08cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08eaf12640 1 --2- 172.21.15.26:0/79333242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc004d40 0x7f08cc0051c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009fc0 tx=0x7f08cc005830 comp rx=0 tx=0).stop 2024-04-24T23:21:42.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08e94af640 1 -- 172.21.15.26:0/79333242 reap_dead start 2024-04-24T23:21:42.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08eaf12640 1 -- 172.21.15.26:0/79333242 shutdown_connections 2024-04-24T23:21:42.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08eaf12640 1 -- 172.21.15.26:0/79333242 >> 172.21.15.26:0/79333242 conn(0x7f08e406b120 msgr2=0x7f08e406b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:42.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08eaf12640 1 -- 172.21.15.26:0/79333242 shutdown_connections 2024-04-24T23:21:42.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.887+0000 7f08eaf12640 1 -- 172.21.15.26:0/79333242 wait complete. 2024-04-24T23:21:42.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.889+0000 7f08eaf12640 1 Processor -- start 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.889+0000 7f08eaf12640 1 -- start start 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.889+0000 7f08eaf12640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08cc004d40 0x7f08e415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.889+0000 7f08eaf12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e40c9030 0x7f08e41e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.889+0000 7f08eaf12640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08e40c9490 0x7f08e41e7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.889+0000 7f08eaf12640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f08e40b5ae0 con 0x7f08e40c9030 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.889+0000 7f08eaf12640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f08e40d00f0 con 0x7f08e40c9490 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.889+0000 7f08eaf12640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f08e415e670 con 0x7f08cc004d40 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e40c9030 0x7f08e41e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e40c9030 0x7f08e41e6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35492/0 (socket says 172.21.15.26:35492) 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08dbfff640 1 -- 172.21.15.26:0/127383056 learned_addr learned my addr 172.21.15.26:0/127383056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08e8cae640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08cc004d40 0x7f08e415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08e94af640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08e40c9490 0x7f08e41e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08dbfff640 1 -- 172.21.15.26:0/127383056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08cc004d40 msgr2=0x7f08e415af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08dbfff640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08cc004d40 0x7f08e415af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08dbfff640 1 -- 172.21.15.26:0/127383056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08e40c9490 msgr2=0x7f08e41e7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08dbfff640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08e40c9490 0x7f08e41e7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08dbfff640 1 -- 172.21.15.26:0/127383056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e40c9a40 con 0x7f08e40c9030 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08dbfff640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e40c9030 0x7f08e41e6b10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f08d4007340 tx=0x7f08d4007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08d97fa640 1 -- 172.21.15.26:0/127383056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d400f940 con 0x7f08e40c9030 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08d97fa640 1 -- 172.21.15.26:0/127383056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d4018070 con 0x7f08e40c9030 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e40c9d20 con 0x7f08e40c9030 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08d97fa640 1 -- 172.21.15.26:0/127383056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d400fd60 con 0x7f08e40c9030 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.890+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e40ca2e0 con 0x7f08e40c9030 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.891+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e40694c0 con 0x7f08e40c9030 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.892+0000 7f08d97fa640 1 -- 172.21.15.26:0/127383056 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08d401d460 con 0x7f08e40c9030 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.892+0000 7f08d97fa640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f08d006be30 0x7f08d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:42.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.892+0000 7f08d97fa640 1 -- 172.21.15.26:0/127383056 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f08d408ab10 con 0x7f08e40c9030 2024-04-24T23:21:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.892+0000 7f08e8cae640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f08d006be30 0x7f08d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:42.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.893+0000 7f08e8cae640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f08d006be30 0x7f08d006e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009bc0 tx=0x7f08e0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:42.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:42.896+0000 7f08d97fa640 1 -- 172.21.15.26:0/127383056 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f08d4059810 con 0x7f08e40c9030 2024-04-24T23:21:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.163+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f08e40cf470 con 0x7f08e40c9030 2024-04-24T23:21:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.165+0000 7f08d97fa640 1 -- 172.21.15.26:0/127383056 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f08d4059110 con 0x7f08e40c9030 2024-04-24T23:21:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f08d006be30 msgr2=0x7f08d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f08d006be30 0x7f08d006e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009bc0 tx=0x7f08e0009190 comp rx=0 tx=0).stop 2024-04-24T23:21:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e40c9030 msgr2=0x7f08e41e6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e40c9030 0x7f08e41e6b10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f08d4007340 tx=0x7f08d4007740 comp rx=0 tx=0).stop 2024-04-24T23:21:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 shutdown_connections 2024-04-24T23:21:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f08d006be30 0x7f08d006e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f08e40c9490 0x7f08e41e7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e40c9030 0x7f08e41e6b10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 --2- 172.21.15.26:0/127383056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f08cc004d40 0x7f08e415af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 >> 172.21.15.26:0/127383056 conn(0x7f08e406b120 msgr2=0x7f08e415d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.167+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 shutdown_connections 2024-04-24T23:21:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.168+0000 7f08eaf12640 1 -- 172.21.15.26:0/127383056 wait complete. 2024-04-24T23:21:43.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-24T23:21:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.282+0000 7faa2e714640 1 Processor -- start 2024-04-24T23:21:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.282+0000 7faa2e714640 1 -- start start 2024-04-24T23:21:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.283+0000 7faa2e714640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faa280cdd70 0x7faa2806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.283+0000 7faa2e714640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa280cfbc0 con 0x7faa2806d350 2024-04-24T23:21:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.283+0000 7faa2e714640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa2806d7f0 con 0x7faa28145910 2024-04-24T23:21:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.283+0000 7faa2e714640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faa28145cf0 con 0x7faa280cdd70 2024-04-24T23:21:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.283+0000 7faa277fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa2806d350 0x7faa280cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59718/0 (socket says 172.21.15.26:59718) 2024-04-24T23:21:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.283+0000 7faa277fe640 1 -- 172.21.15.26:0/2345073863 learned_addr learned my addr 172.21.15.26:0/2345073863 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.283+0000 7faa2ccb1640 1 --2- 172.21.15.26:0/2345073863 >> v2:172.21.15.29:3300/0 conn(0x7faa280cdd70 0x7faa2806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.283+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3039965353 0 0) 0x7faa280cfbc0 con 0x7faa2806d350 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.283+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa10003520 con 0x7faa2806d350 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2732426729 0 0) 0x7faa10003520 con 0x7faa2806d350 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 >> v2:172.21.15.29:3300/0 conn(0x7faa280cdd70 msgr2=0x7faa2806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 --2- 172.21.15.26:0/2345073863 >> v2:172.21.15.29:3300/0 conn(0x7faa280cdd70 0x7faa2806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 >> v1:172.21.15.29:6789/0 conn(0x7faa28145910 legacy=0x7faa2815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2806dad0 con 0x7faa2806d350 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7faa18001670 con 0x7faa2806d350 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 >> v1:172.21.15.26:6789/0 conn(0x7faa2806d350 legacy=0x7faa280cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10004800 0x7faa10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa100051c0 0x7faa1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa28145cf0 con 0x7faa100039b0 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa1000c160 con 0x7faa10004800 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa1000c360 con 0x7faa100051c0 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa2ccb1640 1 --2- 172.21.15.26:0/2345073863 >> v2:172.21.15.29:3300/0 conn(0x7faa280cdd70 0x7faa2806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa27fff640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa277fe640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10004800 0x7faa10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa2ccb1640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa100051c0 0x7faa1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa27fff640 1 -- 172.21.15.26:0/2345073863 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa100051c0 msgr2=0x7faa1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa27fff640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa100051c0 0x7faa1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa27fff640 1 -- 172.21.15.26:0/2345073863 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10004800 msgr2=0x7faa10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa27fff640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10004800 0x7faa10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa27fff640 1 -- 172.21.15.26:0/2345073863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa10003520 con 0x7faa100039b0 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa2ccb1640 1 -- 172.21.15.26:0/2345073863 reap_dead start 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.284+0000 7faa277fe640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa10004800 0x7faa10004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.285+0000 7faa27fff640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa10003dd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faa1c001750 tx=0x7faa10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf21db2132ee9f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.285+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c001370 con 0x7faa100039b0 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.285+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c0052d0 con 0x7faa100039b0 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.285+0000 7faa26ffd640 1 -- 172.21.15.26:0/2345073863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c02abe0 con 0x7faa100039b0 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.285+0000 7faa2e714640 1 -- 172.21.15.26:0/2345073863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 msgr2=0x7faa10003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.285+0000 7faa2e714640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa10003dd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faa1c001750 tx=0x7faa10005830 comp rx=0 tx=0).stop 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.286+0000 7faa2e714640 1 -- 172.21.15.26:0/2345073863 shutdown_connections 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.286+0000 7faa2e714640 1 --2- 172.21.15.26:0/2345073863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa10003dd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.286+0000 7faa2e714640 1 -- 172.21.15.26:0/2345073863 >> 172.21.15.26:0/2345073863 conn(0x7faa2806b300 msgr2=0x7faa2806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.286+0000 7faa2e714640 1 -- 172.21.15.26:0/2345073863 shutdown_connections 2024-04-24T23:21:43.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.286+0000 7faa2e714640 1 -- 172.21.15.26:0/2345073863 wait complete. 2024-04-24T23:21:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.287+0000 7faa2e714640 1 Processor -- start 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.287+0000 7faa2e714640 1 -- start start 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa2e714640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa28150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa2e714640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa2813e550 0x7faa2813e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa2e714640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa2813a0c0 0x7faa2813a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa2e714640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa2806e370 con 0x7faa100039b0 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa2e714640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa2806e090 con 0x7faa2813a0c0 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa2e714640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa2806e890 con 0x7faa2813e550 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa28150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa28150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35506/0 (socket says 172.21.15.26:35506) 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa27fff640 1 -- 172.21.15.26:0/690180576 learned_addr learned my addr 172.21.15.26:0/690180576 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa27fff640 1 -- 172.21.15.26:0/690180576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa2813e550 msgr2=0x7faa2813e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa27fff640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa2813e550 0x7faa2813e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa27fff640 1 -- 172.21.15.26:0/690180576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa2813a0c0 msgr2=0x7faa2813a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa27fff640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa2813a0c0 0x7faa2813a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa27fff640 1 -- 172.21.15.26:0/690180576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa28139300 con 0x7faa100039b0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa27fff640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa28150370 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa1c001870 tx=0x7faa1c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa24ff9640 1 -- 172.21.15.26:0/690180576 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c03f030 con 0x7faa100039b0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa24ff9640 1 -- 172.21.15.26:0/690180576 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c005480 con 0x7faa100039b0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa281395e0 con 0x7faa100039b0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.288+0000 7faa24ff9640 1 -- 172.21.15.26:0/690180576 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c037040 con 0x7faa100039b0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.289+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2813aa80 con 0x7faa100039b0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.290+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa28143c50 con 0x7faa100039b0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.290+0000 7faa24ff9640 1 -- 172.21.15.26:0/690180576 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa1c042070 con 0x7faa100039b0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.290+0000 7faa24ff9640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa1406be30 0x7faa1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.290+0000 7faa24ff9640 1 -- 172.21.15.26:0/690180576 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7faa1c007640 con 0x7faa100039b0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.290+0000 7faa277fe640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa1406be30 0x7faa1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.291+0000 7faa277fe640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa1406be30 0x7faa1406e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faa18001cc0 tx=0x7faa18001e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.295+0000 7faa24ff9640 1 -- 172.21.15.26:0/690180576 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faa28143c50 con 0x7faa100039b0 2024-04-24T23:21:43.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.550+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7faa280694c0 con 0x7faa100039b0 2024-04-24T23:21:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.552+0000 7faa24ff9640 1 -- 172.21.15.26:0/690180576 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7faa1c0433b0 con 0x7faa100039b0 2024-04-24T23:21:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa1406be30 msgr2=0x7faa1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa1406be30 0x7faa1406e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faa18001cc0 tx=0x7faa18001e30 comp rx=0 tx=0).stop 2024-04-24T23:21:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 msgr2=0x7faa28150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa28150370 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa1c001870 tx=0x7faa1c001120 comp rx=0 tx=0).stop 2024-04-24T23:21:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 shutdown_connections 2024-04-24T23:21:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa1406be30 0x7faa1406e2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa2813a0c0 0x7faa2813a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa2813e550 0x7faa2813e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 --2- 172.21.15.26:0/690180576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa100039b0 0x7faa28150370 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 >> 172.21.15.26:0/690180576 conn(0x7faa2806b300 msgr2=0x7faa2806b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 shutdown_connections 2024-04-24T23:21:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.554+0000 7faa2e714640 1 -- 172.21.15.26:0/690180576 wait complete. 2024-04-24T23:21:43.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-24T23:21:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.666+0000 7f795a761640 1 Processor -- start 2024-04-24T23:21:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.666+0000 7f795a761640 1 -- start start 2024-04-24T23:21:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.667+0000 7f795a761640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f795406d350 0x7f795406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.667+0000 7f795a761640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79540cd660 0x7f79540cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.667+0000 7f795a761640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79540cfbc0 con 0x7f7954145910 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.667+0000 7f795a761640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7954145cf0 con 0x7f795406d350 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.667+0000 7f795a761640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f795406d7f0 con 0x7f79540cd660 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.667+0000 7f7958cfe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79540cd660 0x7f79540cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.667+0000 7f79537fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f795406d350 0x7f795406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.667+0000 7f7953fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7954145910 0x7f795415d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:35172/0 (socket says 172.21.15.26:35172) 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.667+0000 7f7953fff640 1 -- 172.21.15.26:0/2291748658 learned_addr learned my addr 172.21.15.26:0/2291748658 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.668+0000 7f7958cfe640 1 -- 172.21.15.26:0/2291748658 >> v2:172.21.15.29:3300/0 conn(0x7f795406d350 msgr2=0x7f795406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.668+0000 7f7958cfe640 1 --2- 172.21.15.26:0/2291748658 >> v2:172.21.15.29:3300/0 conn(0x7f795406d350 0x7f795406e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.668+0000 7f7958cfe640 1 -- 172.21.15.26:0/2291748658 >> v1:172.21.15.29:6789/0 conn(0x7f7954145910 legacy=0x7f795415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.668+0000 7f7958cfe640 1 -- 172.21.15.26:0/2291748658 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795406dad0 con 0x7f79540cd660 2024-04-24T23:21:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.668+0000 7f79537fe640 1 --2- 172.21.15.26:0/2291748658 >> v2:172.21.15.29:3300/0 conn(0x7f795406d350 0x7f795406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.668+0000 7f7958cfe640 1 --2- 172.21.15.26:0/2291748658 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79540cd660 0x7f79540cdaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f794c009dc0 tx=0x7f794c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c4d6fd5aed58ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f7952ffd640 1 -- 172.21.15.26:0/2291748658 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c02ae60 con 0x7f79540cd660 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f7952ffd640 1 -- 172.21.15.26:0/2291748658 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794c007740 con 0x7f79540cd660 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f7952ffd640 1 -- 172.21.15.26:0/2291748658 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c007b10 con 0x7f79540cd660 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f795a761640 1 -- 172.21.15.26:0/2291748658 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79540cd660 msgr2=0x7f79540cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f795a761640 1 --2- 172.21.15.26:0/2291748658 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79540cd660 0x7f79540cdaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f794c009dc0 tx=0x7f794c02a680 comp rx=0 tx=0).stop 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f795a761640 1 -- 172.21.15.26:0/2291748658 shutdown_connections 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f795a761640 1 --2- 172.21.15.26:0/2291748658 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79540cd660 0x7f79540cdaf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f795a761640 1 --2- 172.21.15.26:0/2291748658 >> v2:172.21.15.29:3300/0 conn(0x7f795406d350 0x7f795406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f795a761640 1 -- 172.21.15.26:0/2291748658 >> 172.21.15.26:0/2291748658 conn(0x7f795406b300 msgr2=0x7f795406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f795a761640 1 -- 172.21.15.26:0/2291748658 shutdown_connections 2024-04-24T23:21:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.669+0000 7f795a761640 1 -- 172.21.15.26:0/2291748658 wait complete. 2024-04-24T23:21:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.670+0000 7f795a761640 1 Processor -- start 2024-04-24T23:21:43.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.671+0000 7f795a761640 1 -- start start 2024-04-24T23:21:43.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.671+0000 7f795a761640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f795406d350 0x7f7954155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.671+0000 7f795a761640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79540cd660 0x7f795413c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.671+0000 7f795a761640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7954145910 0x7f795413c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.671+0000 7f795a761640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f795406e370 con 0x7f79540cd660 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.671+0000 7f795a761640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f795406e090 con 0x7f7954145910 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.671+0000 7f795a761640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f795415d770 con 0x7f795406d350 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.671+0000 7f79537fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79540cd660 0x7f795413c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f79537fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79540cd660 0x7f795413c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35512/0 (socket says 172.21.15.26:35512) 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f79537fe640 1 -- 172.21.15.26:0/1255189341 learned_addr learned my addr 172.21.15.26:0/1255189341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f7953fff640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f795406d350 0x7f7954155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f7958cfe640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7954145910 0x7f795413c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f79537fe640 1 -- 172.21.15.26:0/1255189341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f795406d350 msgr2=0x7f7954155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f79537fe640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f795406d350 0x7f7954155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f79537fe640 1 -- 172.21.15.26:0/1255189341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7954145910 msgr2=0x7f795413c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f79537fe640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7954145910 0x7f795413c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f79537fe640 1 -- 172.21.15.26:0/1255189341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79540b3c70 con 0x7f79540cd660 2024-04-24T23:21:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f79537fe640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79540cd660 0x7f795413c270 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f794400ebe0 tx=0x7f794400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f7950ff9640 1 -- 172.21.15.26:0/1255189341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944017070 con 0x7f79540cd660 2024-04-24T23:21:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f7950ff9640 1 -- 172.21.15.26:0/1255189341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794400cc30 con 0x7f79540cd660 2024-04-24T23:21:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795413d540 con 0x7f79540cd660 2024-04-24T23:21:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f7950ff9640 1 -- 172.21.15.26:0/1255189341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944013470 con 0x7f79540cd660 2024-04-24T23:21:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.672+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7954155dc0 con 0x7f79540cd660 2024-04-24T23:21:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.673+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79540694c0 con 0x7f79540cd660 2024-04-24T23:21:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.674+0000 7f7950ff9640 1 -- 172.21.15.26:0/1255189341 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f794401a070 con 0x7f79540cd660 2024-04-24T23:21:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.674+0000 7f7950ff9640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f793406be30 0x7f793406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.674+0000 7f7950ff9640 1 -- 172.21.15.26:0/1255189341 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f794408acc0 con 0x7f79540cd660 2024-04-24T23:21:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.674+0000 7f7953fff640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f793406be30 0x7f793406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.675+0000 7f7953fff640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f793406be30 0x7f793406e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f793c007fd0 tx=0x7f793c007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:43.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.678+0000 7f7950ff9640 1 -- 172.21.15.26:0/1255189341 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7944059400 con 0x7f79540cd660 2024-04-24T23:21:43.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.933+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f795406eca0 con 0x7f79540cd660 2024-04-24T23:21:43.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.935+0000 7f7950ff9640 1 -- 172.21.15.26:0/1255189341 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f794405d700 con 0x7f79540cd660 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f793406be30 msgr2=0x7f793406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f793406be30 0x7f793406e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f793c007fd0 tx=0x7f793c007e80 comp rx=0 tx=0).stop 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79540cd660 msgr2=0x7f795413c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79540cd660 0x7f795413c270 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f794400ebe0 tx=0x7f794400c520 comp rx=0 tx=0).stop 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 shutdown_connections 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f793406be30 0x7f793406e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7954145910 0x7f795413c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79540cd660 0x7f795413c270 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 --2- 172.21.15.26:0/1255189341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f795406d350 0x7f7954155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:43.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 >> 172.21.15.26:0/1255189341 conn(0x7f795406b300 msgr2=0x7f795415e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:43.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 shutdown_connections 2024-04-24T23:21:43.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:43.937+0000 7f795a761640 1 -- 172.21.15.26:0/1255189341 wait complete. 2024-04-24T23:21:43.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-24T23:21:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.049+0000 7fa2b9da2640 1 Processor -- start 2024-04-24T23:21:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.049+0000 7fa2b9da2640 1 -- start start 2024-04-24T23:21:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.050+0000 7fa2b9da2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b415e570 0x7fa2b40d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.050+0000 7fa2b9da2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa2b40d22e0 0x7fa2b40d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.050+0000 7fa2b9da2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b40d32f0 con 0x7fa2b40b3130 2024-04-24T23:21:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.050+0000 7fa2b9da2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b40d1ab0 con 0x7fa2b40d22e0 2024-04-24T23:21:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.050+0000 7fa2b9da2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b40d2ae0 con 0x7fa2b415e570 2024-04-24T23:21:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.050+0000 7fa2b2ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa2b40b3130 0x7fa2b40d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56676/0 (socket says 172.21.15.26:56676) 2024-04-24T23:21:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.050+0000 7fa2b2ffd640 1 -- 172.21.15.26:0/4052982291 learned_addr learned my addr 172.21.15.26:0/4052982291 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.050+0000 7fa2b37fe640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b415e570 0x7fa2b40d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b3fff640 1 --2- 172.21.15.26:0/4052982291 >> v2:172.21.15.29:3300/0 conn(0x7fa2b40d22e0 0x7fa2b40d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182378893 0 0) 0x7fa2a8001110 con 0x7fa2b40b3130 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa29c003520 con 0x7fa2b40b3130 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2202666075 0 0) 0x7fa29c003520 con 0x7fa2b40b3130 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b415e570 msgr2=0x7fa2b40d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b415e570 0x7fa2b40d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 >> v2:172.21.15.29:3300/0 conn(0x7fa2b40d22e0 msgr2=0x7fa2b40d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 --2- 172.21.15.26:0/4052982291 >> v2:172.21.15.29:3300/0 conn(0x7fa2b40d22e0 0x7fa2b40d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b40d0270 con 0x7fa2b40b3130 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fa2a8001840 con 0x7fa2b40b3130 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b3fff640 1 --2- 172.21.15.26:0/4052982291 >> v2:172.21.15.29:3300/0 conn(0x7fa2b40d22e0 0x7fa2b40d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 >> v1:172.21.15.26:6789/0 conn(0x7fa2b40b3130 legacy=0x7fa2b40d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa29c0039b0 0x7fa29c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b415e570 0x7fa29c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29c004d40 0x7fa29c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b40d1ab0 con 0x7fa29c004d40 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b40d2ae0 con 0x7fa29c0039b0 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa29c00b9c0 con 0x7fa2b415e570 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b37fe640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa29c0039b0 0x7fa29c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b3fff640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29c004d40 0x7fa29c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b2ffd640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b415e570 0x7fa29c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b3fff640 1 -- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b415e570 msgr2=0x7fa29c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b3fff640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b415e570 0x7fa29c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b3fff640 1 -- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa29c0039b0 msgr2=0x7fa29c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b3fff640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa29c0039b0 0x7fa29c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.051+0000 7fa2b3fff640 1 -- 172.21.15.26:0/4052982291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c003520 con 0x7fa29c004d40 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.052+0000 7fa2b3fff640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29c004d40 0x7fa29c0051c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e350 tx=0x7fa29c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268449c453fd4976 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.052+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac009de0 con 0x7fa29c004d40 2024-04-24T23:21:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.052+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ac02e660 con 0x7fa29c004d40 2024-04-24T23:21:44.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.052+0000 7fa2b27fc640 1 -- 172.21.15.26:0/4052982291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac02ea10 con 0x7fa29c004d40 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.053+0000 7fa2b9da2640 1 -- 172.21.15.26:0/4052982291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29c004d40 msgr2=0x7fa29c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.053+0000 7fa2b9da2640 1 --2- 172.21.15.26:0/4052982291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29c004d40 0x7fa29c0051c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e350 tx=0x7fa29c005830 comp rx=0 tx=0).stop 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.053+0000 7fa2b3fff640 1 -- 172.21.15.26:0/4052982291 reap_dead start 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.053+0000 7fa2b9da2640 1 -- 172.21.15.26:0/4052982291 shutdown_connections 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.053+0000 7fa2b9da2640 1 -- 172.21.15.26:0/4052982291 >> 172.21.15.26:0/4052982291 conn(0x7fa2b408dbe0 msgr2=0x7fa2b409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.053+0000 7fa2b9da2640 1 -- 172.21.15.26:0/4052982291 shutdown_connections 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.053+0000 7fa2b9da2640 1 -- 172.21.15.26:0/4052982291 wait complete. 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.054+0000 7fa2b9da2640 1 Processor -- start 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.054+0000 7fa2b9da2640 1 -- start start 2024-04-24T23:21:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b9da2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b414f4f0 0x7fa2b415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b9da2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b414f8d0 0x7fa2b40b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b9da2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b415a6e0 0x7fa2b40b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b9da2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b40cfa70 con 0x7fa2b414f4f0 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b9da2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b40d3010 con 0x7fa2b415a6e0 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b9da2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b40cfd50 con 0x7fa2b414f8d0 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b414f4f0 0x7fa2b415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b414f4f0 0x7fa2b415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46740/0 (socket says 172.21.15.26:46740) 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b37fe640 1 -- 172.21.15.26:0/1186660095 learned_addr learned my addr 172.21.15.26:0/1186660095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b2ffd640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b414f8d0 0x7fa2b40b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b3fff640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b415a6e0 0x7fa2b40b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b37fe640 1 -- 172.21.15.26:0/1186660095 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b414f8d0 msgr2=0x7fa2b40b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b37fe640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b414f8d0 0x7fa2b40b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b37fe640 1 -- 172.21.15.26:0/1186660095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b415a6e0 msgr2=0x7fa2b40b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b37fe640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b415a6e0 0x7fa2b40b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.055+0000 7fa2b37fe640 1 -- 172.21.15.26:0/1186660095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b40b6db0 con 0x7fa2b414f4f0 2024-04-24T23:21:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.056+0000 7fa2b37fe640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b414f4f0 0x7fa2b415a1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4011910 tx=0x7fa2a4011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.056+0000 7fa29bfff640 1 -- 172.21.15.26:0/1186660095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400f9d0 con 0x7fa2b414f4f0 2024-04-24T23:21:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.056+0000 7fa29bfff640 1 -- 172.21.15.26:0/1186660095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a400fb70 con 0x7fa2b414f4f0 2024-04-24T23:21:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.056+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b40b7090 con 0x7fa2b414f4f0 2024-04-24T23:21:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.056+0000 7fa29bfff640 1 -- 172.21.15.26:0/1186660095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4014440 con 0x7fa2b414f4f0 2024-04-24T23:21:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.056+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b40b7650 con 0x7fa2b414f4f0 2024-04-24T23:21:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.057+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b40694c0 con 0x7fa2b414f4f0 2024-04-24T23:21:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.057+0000 7fa29bfff640 1 -- 172.21.15.26:0/1186660095 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2a4012040 con 0x7fa2b414f4f0 2024-04-24T23:21:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.058+0000 7fa29bfff640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2a006be30 0x7fa2a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.058+0000 7fa29bfff640 1 -- 172.21.15.26:0/1186660095 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fa2a408b0c0 con 0x7fa2b414f4f0 2024-04-24T23:21:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.058+0000 7fa2b2ffd640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2a006be30 0x7fa2a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.058+0000 7fa2b2ffd640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2a006be30 0x7fa2a006e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2b415c500 tx=0x7fa2a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:44.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.062+0000 7fa29bfff640 1 -- 172.21.15.26:0/1186660095 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa2a4019030 con 0x7fa2b414f4f0 2024-04-24T23:21:44.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.318+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fa2b40cf2f0 con 0x7fa2b414f4f0 2024-04-24T23:21:44.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.320+0000 7fa29bfff640 1 -- 172.21.15.26:0/1186660095 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fa2a4059dc0 con 0x7fa2b414f4f0 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2a006be30 msgr2=0x7fa2a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2a006be30 0x7fa2a006e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2b415c500 tx=0x7fa2a8009040 comp rx=0 tx=0).stop 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b414f4f0 msgr2=0x7fa2b415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b414f4f0 0x7fa2b415a1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4011910 tx=0x7fa2a4011dd0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 shutdown_connections 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa2a006be30 0x7fa2a006e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2b415a6e0 0x7fa2b40b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2b414f8d0 0x7fa2b40b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 --2- 172.21.15.26:0/1186660095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b414f4f0 0x7fa2b415a1a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 >> 172.21.15.26:0/1186660095 conn(0x7fa2b408dbe0 msgr2=0x7fa2b40d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 shutdown_connections 2024-04-24T23:21:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.322+0000 7fa2b9da2640 1 -- 172.21.15.26:0/1186660095 wait complete. 2024-04-24T23:21:44.334 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-24T23:21:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.434+0000 7f571e553640 1 Processor -- start 2024-04-24T23:21:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.434+0000 7f571e553640 1 -- start start 2024-04-24T23:21:44.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f571e553640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f571815eb00 0x7f571815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f571e553640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f571815f450 con 0x7f5718143c40 2024-04-24T23:21:44.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f571e553640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57180cd0b0 con 0x7f57180cf870 2024-04-24T23:21:44.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f571e553640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5718071b70 con 0x7f571815eb00 2024-04-24T23:21:44.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f5717fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5718143c40 0x7f57180cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56682/0 (socket says 172.21.15.26:56682) 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f5717fff640 1 -- 172.21.15.26:0/1969643162 learned_addr learned my addr 172.21.15.26:0/1969643162 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f571caf0640 1 --2- 172.21.15.26:0/1969643162 >> v2:172.21.15.29:3300/0 conn(0x7f571815eb00 0x7f571815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 918133765 0 0) 0x7f5700001110 con 0x7f5718143c40 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5704003520 con 0x7f5718143c40 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.435+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 638553690 0 0) 0x7f5704003520 con 0x7f5718143c40 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 >> v2:172.21.15.29:3300/0 conn(0x7f571815eb00 msgr2=0x7f571815ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 --2- 172.21.15.26:0/1969643162 >> v2:172.21.15.29:3300/0 conn(0x7f571815eb00 0x7f571815ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 >> v1:172.21.15.29:6789/0 conn(0x7f57180cf870 legacy=0x7f5718072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5718071240 con 0x7f5718143c40 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f5700001840 con 0x7f5718143c40 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 >> v1:172.21.15.26:6789/0 conn(0x7f5718143c40 legacy=0x7f57180cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57040039b0 0x7f5704003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57040043b0 0x7f5704004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f570400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5718071b70 con 0x7f5704005080 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f570400bfe0 con 0x7f57040043b0 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f570400c1e0 con 0x7f57040039b0 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f571caf0640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f570400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f57177fe640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57040043b0 0x7f5704004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5717fff640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57040039b0 0x7f5704003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f571caf0640 1 -- 172.21.15.26:0/1969643162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57040039b0 msgr2=0x7f5704003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f571caf0640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57040039b0 0x7f5704003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f571caf0640 1 -- 172.21.15.26:0/1969643162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57040043b0 msgr2=0x7f5704004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f571caf0640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57040043b0 0x7f5704004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f571caf0640 1 -- 172.21.15.26:0/1969643162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5704003520 con 0x7f5704005080 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.436+0000 7f5717fff640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57040039b0 0x7f5704003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f571caf0640 1 -- 172.21.15.26:0/1969643162 reap_dead start 2024-04-24T23:21:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f571caf0640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f570400b9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5710009fc0 tx=0x7f5704005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e141dc79d0f6b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710035020 con 0x7f5704005080 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5710003b90 con 0x7f5704005080 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f5716ffd640 1 -- 172.21.15.26:0/1969643162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710009c10 con 0x7f5704005080 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f571e553640 1 -- 172.21.15.26:0/1969643162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 msgr2=0x7f570400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f571e553640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f570400b9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5710009fc0 tx=0x7f5704005830 comp rx=0 tx=0).stop 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f571e553640 1 -- 172.21.15.26:0/1969643162 shutdown_connections 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f571e553640 1 --2- 172.21.15.26:0/1969643162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f570400b9c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f571e553640 1 -- 172.21.15.26:0/1969643162 >> 172.21.15.26:0/1969643162 conn(0x7f571806b300 msgr2=0x7f571806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f571e553640 1 -- 172.21.15.26:0/1969643162 shutdown_connections 2024-04-24T23:21:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.437+0000 7f571e553640 1 -- 172.21.15.26:0/1969643162 wait complete. 2024-04-24T23:21:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.439+0000 7f571e553640 1 Processor -- start 2024-04-24T23:21:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.439+0000 7f571e553640 1 -- start start 2024-04-24T23:21:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.439+0000 7f571e553640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f5718159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.439+0000 7f571e553640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f571815a830 0x7f5718136090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.439+0000 7f571e553640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f571815acb0 0x7f57181365d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.439+0000 7f571e553640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5718144020 con 0x7f5704005080 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.439+0000 7f571e553640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5718071800 con 0x7f571815a830 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.439+0000 7f571e553640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f571815ef10 con 0x7f571815acb0 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5717fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f5718159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5717fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f5718159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46756/0 (socket says 172.21.15.26:46756) 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5717fff640 1 -- 172.21.15.26:0/3121967739 learned_addr learned my addr 172.21.15.26:0/3121967739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f57177fe640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f571815a830 0x7f5718136090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f571caf0640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f571815acb0 0x7f57181365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5717fff640 1 -- 172.21.15.26:0/3121967739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f571815acb0 msgr2=0x7f57181365d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5717fff640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f571815acb0 0x7f57181365d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5717fff640 1 -- 172.21.15.26:0/3121967739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f571815a830 msgr2=0x7f5718136090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5717fff640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f571815a830 0x7f5718136090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5717fff640 1 -- 172.21.15.26:0/3121967739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57180adc30 con 0x7f5704005080 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5717fff640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f5718159850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f570000e7b0 tx=0x7f570000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5714ff9640 1 -- 172.21.15.26:0/3121967739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570000ee60 con 0x7f5704005080 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5714ff9640 1 -- 172.21.15.26:0/3121967739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5700018070 con 0x7f5704005080 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57180adf10 con 0x7f5704005080 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f5714ff9640 1 -- 172.21.15.26:0/3121967739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570000ca70 con 0x7f5704005080 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.440+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57180ae4d0 con 0x7f5704005080 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.441+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57180694c0 con 0x7f5704005080 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.442+0000 7f5714ff9640 1 -- 172.21.15.26:0/3121967739 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f570000cc10 con 0x7f5704005080 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.442+0000 7f5714ff9640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f56f806be60 0x7f56f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.442+0000 7f5714ff9640 1 -- 172.21.15.26:0/3121967739 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f570008a520 con 0x7f5704005080 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.442+0000 7f57177fe640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f56f806be60 0x7f56f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.443+0000 7f57177fe640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f56f806be60 0x7f56f806e320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5718072e50 tx=0x7f5708009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:44.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.447+0000 7f5714ff9640 1 -- 172.21.15.26:0/3121967739 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f571815f450 con 0x7f5704005080 2024-04-24T23:21:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.700+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f57180b4d20 con 0x7f5704005080 2024-04-24T23:21:44.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.702+0000 7f5714ff9640 1 -- 172.21.15.26:0/3121967739 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f5700022250 con 0x7f5704005080 2024-04-24T23:21:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f56f806be60 msgr2=0x7f56f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f56f806be60 0x7f56f806e320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5718072e50 tx=0x7f5708009040 comp rx=0 tx=0).stop 2024-04-24T23:21:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 msgr2=0x7f5718159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f5718159850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f570000e7b0 tx=0x7f570000ec70 comp rx=0 tx=0).stop 2024-04-24T23:21:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 shutdown_connections 2024-04-24T23:21:44.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f56f806be60 0x7f56f806e320 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f571815acb0 0x7f57181365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f571815a830 0x7f5718136090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 --2- 172.21.15.26:0/3121967739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5704005080 0x7f5718159850 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 >> 172.21.15.26:0/3121967739 conn(0x7f571806b300 msgr2=0x7f571806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 shutdown_connections 2024-04-24T23:21:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.704+0000 7f571e553640 1 -- 172.21.15.26:0/3121967739 wait complete. 2024-04-24T23:21:44.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:44.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.816+0000 7f01080a5640 1 Processor -- start 2024-04-24T23:21:44.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.817+0000 7f01080a5640 1 -- start start 2024-04-24T23:21:44.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.817+0000 7f01080a5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f01000b3130 0x7f01000b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.817+0000 7f01080a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01000d32f0 con 0x7f01000d4870 2024-04-24T23:21:44.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.817+0000 7f01080a5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01000d0270 con 0x7f010015e570 2024-04-24T23:21:44.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.817+0000 7f01080a5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f01000d1ab0 con 0x7f01000b3130 2024-04-24T23:21:44.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.817+0000 7f0105640640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f01000d4870 0x7f01000b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56686/0 (socket says 172.21.15.26:56686) 2024-04-24T23:21:44.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.817+0000 7f0105640640 1 -- 172.21.15.26:0/697273207 learned_addr learned my addr 172.21.15.26:0/697273207 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:44.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0106642640 1 --2- 172.21.15.26:0/697273207 >> v2:172.21.15.29:3300/0 conn(0x7f01000b3130 0x7f01000b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1462112362 0 0) 0x7f00f4001110 con 0x7f01000d4870 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00e8003520 con 0x7f01000d4870 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2141597508 0 0) 0x7f00e8003520 con 0x7f01000d4870 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 >> v2:172.21.15.29:3300/0 conn(0x7f01000b3130 msgr2=0x7f01000b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 --2- 172.21.15.26:0/697273207 >> v2:172.21.15.29:3300/0 conn(0x7f01000b3130 0x7f01000b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 >> v1:172.21.15.29:6789/0 conn(0x7f010015e570 legacy=0x7f01000d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01000d2d30 con 0x7f01000d4870 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f00f4001840 con 0x7f01000d4870 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 >> v1:172.21.15.26:6789/0 conn(0x7f01000d4870 legacy=0x7f01000b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e80039b0 0x7f00e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00e8004800 0x7f00e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00e80051c0 0x7f00e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f01000d1ab0 con 0x7f00e80039b0 2024-04-24T23:21:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f00e800c160 con 0x7f00e80051c0 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f00e800c360 con 0x7f00e8004800 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0105e41640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e80039b0 0x7f00e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0106642640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00e80051c0 0x7f00e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.818+0000 7f0105640640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00e8004800 0x7f00e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0105e41640 1 -- 172.21.15.26:0/697273207 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00e8004800 msgr2=0x7f00e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0105e41640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00e8004800 0x7f00e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0105e41640 1 -- 172.21.15.26:0/697273207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00e80051c0 msgr2=0x7f00e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0105e41640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00e80051c0 0x7f00e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0105e41640 1 -- 172.21.15.26:0/697273207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e8003520 con 0x7f00e80039b0 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0106642640 1 -- 172.21.15.26:0/697273207 reap_dead start 2024-04-24T23:21:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0105e41640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e80039b0 0x7f00e8003dd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f00f8001750 tx=0x7f00e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a4bdafe4b152b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:44.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8001370 con 0x7f00e80039b0 2024-04-24T23:21:44.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f80052d0 con 0x7f00e80039b0 2024-04-24T23:21:44.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.819+0000 7f0104e3f640 1 -- 172.21.15.26:0/697273207 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f802abc0 con 0x7f00e80039b0 2024-04-24T23:21:44.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.820+0000 7f01080a5640 1 -- 172.21.15.26:0/697273207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e80039b0 msgr2=0x7f00e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.820+0000 7f01080a5640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e80039b0 0x7f00e8003dd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f00f8001750 tx=0x7f00e8005830 comp rx=0 tx=0).stop 2024-04-24T23:21:44.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.820+0000 7f01080a5640 1 -- 172.21.15.26:0/697273207 shutdown_connections 2024-04-24T23:21:44.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.820+0000 7f01080a5640 1 --2- 172.21.15.26:0/697273207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00e80039b0 0x7f00e8003dd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.820+0000 7f01080a5640 1 -- 172.21.15.26:0/697273207 >> 172.21.15.26:0/697273207 conn(0x7f010008dbe0 msgr2=0x7f010009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:44.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.820+0000 7f01080a5640 1 -- 172.21.15.26:0/697273207 shutdown_connections 2024-04-24T23:21:44.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.820+0000 7f01080a5640 1 -- 172.21.15.26:0/697273207 wait complete. 2024-04-24T23:21:44.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.821+0000 7f01080a5640 1 Processor -- start 2024-04-24T23:21:44.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.821+0000 7f01080a5640 1 -- start start 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f01080a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00e80039b0 0x7f010014d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f01080a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01000b7dc0 0x7f01000b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f01080a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f010009f860 0x7f010009fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f01080a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f01000cfa70 con 0x7f010009f860 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f01080a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f01000d2ae0 con 0x7f00e80039b0 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f01080a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f01000cfd50 con 0x7f01000b7dc0 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0106642640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f010009f860 0x7f010009fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0106642640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f010009f860 0x7f010009fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46772/0 (socket says 172.21.15.26:46772) 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0106642640 1 -- 172.21.15.26:0/3553493995 learned_addr learned my addr 172.21.15.26:0/3553493995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0105e41640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00e80039b0 0x7f010014d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0105640640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01000b7dc0 0x7f01000b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0106642640 1 -- 172.21.15.26:0/3553493995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01000b7dc0 msgr2=0x7f01000b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0106642640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01000b7dc0 0x7f01000b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0106642640 1 -- 172.21.15.26:0/3553493995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00e80039b0 msgr2=0x7f010014d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0106642640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00e80039b0 0x7f010014d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0106642640 1 -- 172.21.15.26:0/3553493995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01000b93d0 con 0x7f010009f860 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.822+0000 7f0105640640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01000b7dc0 0x7f01000b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.823+0000 7f0106642640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f010009f860 0x7f010009fce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01000b3570 tx=0x7f00f000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.823+0000 7f00fe7fc640 1 -- 172.21.15.26:0/3553493995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f000fdb0 con 0x7f010009f860 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.823+0000 7f00fe7fc640 1 -- 172.21.15.26:0/3553493995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f001d070 con 0x7f010009f860 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.823+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01000b8330 con 0x7f010009f860 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.823+0000 7f00fe7fc640 1 -- 172.21.15.26:0/3553493995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0015670 con 0x7f010009f860 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.823+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01000b88f0 con 0x7f010009f860 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.824+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01000694c0 con 0x7f010009f860 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.824+0000 7f00fe7fc640 1 -- 172.21.15.26:0/3553493995 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00f0015810 con 0x7f010009f860 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.825+0000 7f00fe7fc640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f00ec06be60 0x7f00ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.825+0000 7f00fe7fc640 1 -- 172.21.15.26:0/3553493995 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f00f008b380 con 0x7f010009f860 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.825+0000 7f0105e41640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f00ec06be60 0x7f00ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:44.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.825+0000 7f0105e41640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f00ec06be60 0x7f00ec06e320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f00f8032000 tx=0x7f00f802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:44.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:44.829+0000 7f00fe7fc640 1 -- 172.21.15.26:0/3553493995 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00f0010040 con 0x7f010009f860 2024-04-24T23:21:45.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.014+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01000cf4a0 con 0x7f010009f860 2024-04-24T23:21:45.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.015+0000 7f00fe7fc640 1 -- 172.21.15.26:0/3553493995 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12831 (secure 0 0 0) 0x7f00f005a080 con 0x7f010009f860 2024-04-24T23:21:45.017 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:45.017 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:41.955918+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.016+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f00ec06be60 msgr2=0x7f00ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f00ec06be60 0x7f00ec06e320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f00f8032000 tx=0x7f00f802e000 comp rx=0 tx=0).stop 2024-04-24T23:21:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f010009f860 msgr2=0x7f010009fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f010009f860 0x7f010009fce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01000b3570 tx=0x7f00f000f580 comp rx=0 tx=0).stop 2024-04-24T23:21:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 shutdown_connections 2024-04-24T23:21:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f00ec06be60 0x7f00ec06e320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f010009f860 0x7f010009fce0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f01000b7dc0 0x7f01000b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 --2- 172.21.15.26:0/3553493995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00e80039b0 0x7f010014d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 >> 172.21.15.26:0/3553493995 conn(0x7f010008dbe0 msgr2=0x7f01000d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 shutdown_connections 2024-04-24T23:21:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.017+0000 7f01080a5640 1 -- 172.21.15.26:0/3553493995 wait complete. 2024-04-24T23:21:45.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:21:45.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:21:45.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.132+0000 7f07ff74f640 1 Processor -- start 2024-04-24T23:21:45.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.132+0000 7f07ff74f640 1 -- start start 2024-04-24T23:21:45.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.133+0000 7f07ff74f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f815e370 0x7f07f80d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.133+0000 7f07ff74f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07f80d4c10 0x7f07f80cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.133+0000 7f07ff74f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f80d2120 con 0x7f07f815e750 2024-04-24T23:21:45.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.133+0000 7f07ff74f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f07f80d1510 con 0x7f07f80d4c10 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.133+0000 7f07ff74f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f07f80d1ed0 con 0x7f07f815e370 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.133+0000 7f07fdcec640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07f80d4c10 0x7f07f80cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.134+0000 7f07fdcec640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07f80d4c10 0x7f07f80cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46786/0 (socket says 172.21.15.26:46786) 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.134+0000 7f07fdcec640 1 -- 172.21.15.26:0/3256570174 learned_addr learned my addr 172.21.15.26:0/3256570174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.134+0000 7f07fd4eb640 1 --2- 172.21.15.26:0/3256570174 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f815e370 0x7f07f80d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.134+0000 7f07fdcec640 1 -- 172.21.15.26:0/3256570174 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f815e370 msgr2=0x7f07f80d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.134+0000 7f07fdcec640 1 --2- 172.21.15.26:0/3256570174 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f815e370 0x7f07f80d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.134+0000 7f07fdcec640 1 -- 172.21.15.26:0/3256570174 >> v1:172.21.15.29:6789/0 conn(0x7f07f815e750 legacy=0x7f07f80b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.134+0000 7f07fdcec640 1 -- 172.21.15.26:0/3256570174 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f80b3740 con 0x7f07f80d4c10 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07fdcec640 1 --2- 172.21.15.26:0/3256570174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80d4c10 0x7f07f80cfca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009e50 tx=0x7f07e802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cf7af92fe99ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07f7fff640 1 -- 172.21.15.26:0/3256570174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8036070 con 0x7f07f80d4c10 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07f7fff640 1 -- 172.21.15.26:0/3256570174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e8031600 con 0x7f07f80d4c10 2024-04-24T23:21:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07f7fff640 1 -- 172.21.15.26:0/3256570174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e80319d0 con 0x7f07f80d4c10 2024-04-24T23:21:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07ff74f640 1 -- 172.21.15.26:0/3256570174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80d4c10 msgr2=0x7f07f80cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07ff74f640 1 --2- 172.21.15.26:0/3256570174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80d4c10 0x7f07f80cfca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009e50 tx=0x7f07e802a710 comp rx=0 tx=0).stop 2024-04-24T23:21:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07ff74f640 1 -- 172.21.15.26:0/3256570174 shutdown_connections 2024-04-24T23:21:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07ff74f640 1 --2- 172.21.15.26:0/3256570174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80d4c10 0x7f07f80cfca0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07ff74f640 1 --2- 172.21.15.26:0/3256570174 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f815e370 0x7f07f80d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07ff74f640 1 -- 172.21.15.26:0/3256570174 >> 172.21.15.26:0/3256570174 conn(0x7f07f80d40f0 msgr2=0x7f07f80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07ff74f640 1 -- 172.21.15.26:0/3256570174 shutdown_connections 2024-04-24T23:21:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.135+0000 7f07ff74f640 1 -- 172.21.15.26:0/3256570174 wait complete. 2024-04-24T23:21:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07ff74f640 1 Processor -- start 2024-04-24T23:21:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07ff74f640 1 -- start start 2024-04-24T23:21:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07ff74f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f80d4c10 0x7f07f80b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07ff74f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f815e370 0x7f07f80b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07ff74f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80b1190 0x7f07f81f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07ff74f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07f80d2a60 con 0x7f07f80b1190 2024-04-24T23:21:45.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07ff74f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f07f80b3d00 con 0x7f07f815e370 2024-04-24T23:21:45.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07ff74f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f07f80d2be0 con 0x7f07f80d4c10 2024-04-24T23:21:45.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07fdcec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80b1190 0x7f07f81f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07fdcec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80b1190 0x7f07f81f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46794/0 (socket says 172.21.15.26:46794) 2024-04-24T23:21:45.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.137+0000 7f07fdcec640 1 -- 172.21.15.26:0/2286629375 learned_addr learned my addr 172.21.15.26:0/2286629375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:45.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fccea640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f815e370 0x7f07f80b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fdcec640 1 -- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f80d4c10 msgr2=0x7f07f80b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fd4eb640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f80d4c10 0x7f07f80b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fdcec640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f80d4c10 0x7f07f80b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fdcec640 1 -- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f815e370 msgr2=0x7f07f80b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fdcec640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f815e370 0x7f07f80b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fdcec640 1 -- 172.21.15.26:0/2286629375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f81f4b70 con 0x7f07f80b1190 2024-04-24T23:21:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fccea640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f815e370 0x7f07f80b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fd4eb640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f80d4c10 0x7f07f80b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07fdcec640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80b1190 0x7f07f81f3dd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009f70 tx=0x7f07e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07f5ffb640 1 -- 172.21.15.26:0/2286629375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e803f020 con 0x7f07f80b1190 2024-04-24T23:21:45.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07f5ffb640 1 -- 172.21.15.26:0/2286629375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e8037040 con 0x7f07f80b1190 2024-04-24T23:21:45.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07f5ffb640 1 -- 172.21.15.26:0/2286629375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8036070 con 0x7f07f80b1190 2024-04-24T23:21:45.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f81f5150 con 0x7f07f80b1190 2024-04-24T23:21:45.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.138+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f81f5710 con 0x7f07f80b1190 2024-04-24T23:21:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.140+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f8075610 con 0x7f07f80b1190 2024-04-24T23:21:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.140+0000 7f07f5ffb640 1 -- 172.21.15.26:0/2286629375 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07e8042020 con 0x7f07f80b1190 2024-04-24T23:21:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.140+0000 7f07f5ffb640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07e406be30 0x7f07e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.140+0000 7f07f5ffb640 1 -- 172.21.15.26:0/2286629375 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f07e80aaff0 con 0x7f07f80b1190 2024-04-24T23:21:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.140+0000 7f07fd4eb640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07e406be30 0x7f07e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.141+0000 7f07fd4eb640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07e406be30 0x7f07e406e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f07f80cfb30 tx=0x7f07f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:45.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.145+0000 7f07f5ffb640 1 -- 172.21.15.26:0/2286629375 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07e8049030 con 0x7f07f80b1190 2024-04-24T23:21:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.327+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07f81f58e0 con 0x7f07f80b1190 2024-04-24T23:21:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.327+0000 7f07f5ffb640 1 -- 172.21.15.26:0/2286629375 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f07e8079c50 con 0x7f07f80b1190 2024-04-24T23:21:45.329 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:21:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07e406be30 msgr2=0x7f07e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07e406be30 0x7f07e406e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f07f80cfb30 tx=0x7f07f0008040 comp rx=0 tx=0).stop 2024-04-24T23:21:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80b1190 msgr2=0x7f07f81f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80b1190 0x7f07f81f3dd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009f70 tx=0x7f07e8004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 shutdown_connections 2024-04-24T23:21:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07e406be30 0x7f07e406e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f80b1190 0x7f07f81f3dd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f815e370 0x7f07f80b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 --2- 172.21.15.26:0/2286629375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f80d4c10 0x7f07f80b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 >> 172.21.15.26:0/2286629375 conn(0x7f07f80d40f0 msgr2=0x7f07f815d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.329+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 shutdown_connections 2024-04-24T23:21:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.330+0000 7f07ff74f640 1 -- 172.21.15.26:0/2286629375 wait complete. 2024-04-24T23:21:45.342 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.441+0000 7f07cfc72640 1 Processor -- start 2024-04-24T23:21:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.441+0000 7f07cfc72640 1 -- start start 2024-04-24T23:21:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.442+0000 7f07cfc72640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f07c806f4f0 0x7f07c80ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.442+0000 7f07cfc72640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07c80b0220 con 0x7f07c806f110 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.442+0000 7f07cfc72640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07c80af180 con 0x7f07c80addb0 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.442+0000 7f07cfc72640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f07c80af460 con 0x7f07c806f4f0 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.442+0000 7f07cda0e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f07c806f110 0x7f07c815f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56688/0 (socket says 172.21.15.26:56688) 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.442+0000 7f07cda0e640 1 -- 172.21.15.26:0/1156212020 learned_addr learned my addr 172.21.15.26:0/1156212020 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.442+0000 7f07cd20d640 1 --2- 172.21.15.26:0/1156212020 >> v2:172.21.15.29:3300/0 conn(0x7f07c806f4f0 0x7f07c80ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.442+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1038019753 0 0) 0x7f07c80b0220 con 0x7f07c806f110 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.442+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07b0003520 con 0x7f07c806f110 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1990024854 0 0) 0x7f07b0003520 con 0x7f07c806f110 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 >> v2:172.21.15.29:3300/0 conn(0x7f07c806f4f0 msgr2=0x7f07c80ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 --2- 172.21.15.26:0/1156212020 >> v2:172.21.15.29:3300/0 conn(0x7f07c806f4f0 0x7f07c80ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 >> v1:172.21.15.29:6789/0 conn(0x7f07c80addb0 legacy=0x7f07c8154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c80af740 con 0x7f07c806f110 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f07c0001670 con 0x7f07c806f110 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 >> v1:172.21.15.26:6789/0 conn(0x7f07c806f110 legacy=0x7f07c815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07b00039b0 0x7f07b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07b00043b0 0x7f07b0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07b0005080 0x7f07b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07c80af460 con 0x7f07b0005080 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f07b000bfe0 con 0x7f07b00043b0 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f07b000c1e0 con 0x7f07b00039b0 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07ce20f640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07b0005080 0x7f07b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cda0e640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07b00039b0 0x7f07b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cd20d640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07b00043b0 0x7f07b0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07ce20f640 1 -- 172.21.15.26:0/1156212020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07b00039b0 msgr2=0x7f07b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07ce20f640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07b00039b0 0x7f07b0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07ce20f640 1 -- 172.21.15.26:0/1156212020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07b00043b0 msgr2=0x7f07b0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07ce20f640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07b00043b0 0x7f07b0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07ce20f640 1 -- 172.21.15.26:0/1156212020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b0003520 con 0x7f07b0005080 2024-04-24T23:21:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.443+0000 7f07cd20d640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07b00043b0 0x7f07b0004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.444+0000 7f07ce20f640 1 -- 172.21.15.26:0/1156212020 reap_dead start 2024-04-24T23:21:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.444+0000 7f07ce20f640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07b0005080 0x7f07b000b9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07b8001f60 tx=0x7f07b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2934165d1080bb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.444+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8001b30 con 0x7f07b0005080 2024-04-24T23:21:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.444+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b8031600 con 0x7f07b0005080 2024-04-24T23:21:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.444+0000 7f07cca0c640 1 -- 172.21.15.26:0/1156212020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80319f0 con 0x7f07b0005080 2024-04-24T23:21:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.445+0000 7f07cfc72640 1 -- 172.21.15.26:0/1156212020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07b0005080 msgr2=0x7f07b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.445+0000 7f07cfc72640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07b0005080 0x7f07b000b9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07b8001f60 tx=0x7f07b0005830 comp rx=0 tx=0).stop 2024-04-24T23:21:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.445+0000 7f07cfc72640 1 -- 172.21.15.26:0/1156212020 shutdown_connections 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.445+0000 7f07cfc72640 1 --2- 172.21.15.26:0/1156212020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07b0005080 0x7f07b000b9c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.445+0000 7f07cfc72640 1 -- 172.21.15.26:0/1156212020 >> 172.21.15.26:0/1156212020 conn(0x7f07c80729a0 msgr2=0x7f07c806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.445+0000 7f07cfc72640 1 -- 172.21.15.26:0/1156212020 shutdown_connections 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.445+0000 7f07cfc72640 1 -- 172.21.15.26:0/1156212020 wait complete. 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.446+0000 7f07cfc72640 1 Processor -- start 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.446+0000 7f07cfc72640 1 -- start start 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07cfc72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07b0005080 0x7f07c8140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07cfc72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c81f7090 0x7f07c81f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07cfc72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c80b4c70 0x7f07c80b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07cfc72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07c80aee50 con 0x7f07c80b4c70 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07cfc72640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f07c80afd00 con 0x7f07c81f7090 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07cfc72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f07c806fb00 con 0x7f07b0005080 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07ce20f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c80b4c70 0x7f07c80b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07ce20f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c80b4c70 0x7f07c80b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46816/0 (socket says 172.21.15.26:46816) 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07ce20f640 1 -- 172.21.15.26:0/685897033 learned_addr learned my addr 172.21.15.26:0/685897033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07ce20f640 1 -- 172.21.15.26:0/685897033 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07b0005080 msgr2=0x7f07c8140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07ce20f640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07b0005080 0x7f07c8140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07ce20f640 1 -- 172.21.15.26:0/685897033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c81f7090 msgr2=0x7f07c81f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07ce20f640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c81f7090 0x7f07c81f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.447+0000 7f07ce20f640 1 -- 172.21.15.26:0/685897033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c81f7590 con 0x7f07c80b4c70 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.448+0000 7f07ce20f640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c80b4c70 0x7f07c80b5120 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f07b802aa00 tx=0x7f07b80015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.448+0000 7f07c67fc640 1 -- 172.21.15.26:0/685897033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8001b30 con 0x7f07c80b4c70 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.448+0000 7f07c67fc640 1 -- 172.21.15.26:0/685897033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b8040070 con 0x7f07c80b4c70 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.448+0000 7f07c67fc640 1 -- 172.21.15.26:0/685897033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8005070 con 0x7f07c80b4c70 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.448+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c81f7870 con 0x7f07c80b4c70 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.448+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c81f7e30 con 0x7f07c80b4c70 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.449+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c80694c0 con 0x7f07c80b4c70 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.449+0000 7f07c67fc640 1 -- 172.21.15.26:0/685897033 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07b8039040 con 0x7f07c80b4c70 2024-04-24T23:21:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.450+0000 7f07c67fc640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07b406be30 0x7f07b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.450+0000 7f07c67fc640 1 -- 172.21.15.26:0/685897033 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f07b80ab070 con 0x7f07c80b4c70 2024-04-24T23:21:45.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.450+0000 7f07cda0e640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07b406be30 0x7f07b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.450+0000 7f07cda0e640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07b406be30 0x7f07b406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f07c0007bf0 tx=0x7f07c0000f10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:45.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.454+0000 7f07c67fc640 1 -- 172.21.15.26:0/685897033 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07b8079d70 con 0x7f07c80b4c70 2024-04-24T23:21:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.638+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f07c80b5840 con 0x7f07c80b4c70 2024-04-24T23:21:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.639+0000 7f07c67fc640 1 -- 172.21.15.26:0/685897033 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f07b8079670 con 0x7f07c80b4c70 2024-04-24T23:21:45.640 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:45.641 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:21:45.641 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:21:45.641 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-24T23:21:45.641 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-24T23:21:45.641 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:21:45.641 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:21:45.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.640+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07b406be30 msgr2=0x7f07b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.640+0000 7f07cfc72640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07b406be30 0x7f07b406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f07c0007bf0 tx=0x7f07c0000f10 comp rx=0 tx=0).stop 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.640+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c80b4c70 msgr2=0x7f07c80b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.640+0000 7f07cfc72640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c80b4c70 0x7f07c80b5120 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f07b802aa00 tx=0x7f07b80015b0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.641+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 shutdown_connections 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.641+0000 7f07cfc72640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07b406be30 0x7f07b406e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.641+0000 7f07cfc72640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c80b4c70 0x7f07c80b5120 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.641+0000 7f07cfc72640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07c81f7090 0x7f07c81f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.641+0000 7f07cfc72640 1 --2- 172.21.15.26:0/685897033 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07b0005080 0x7f07c8140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.641+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 >> 172.21.15.26:0/685897033 conn(0x7f07c80729a0 msgr2=0x7f07c8158620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.641+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 shutdown_connections 2024-04-24T23:21:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.641+0000 7f07cfc72640 1 -- 172.21.15.26:0/685897033 wait complete. 2024-04-24T23:21:45.653 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:21:45.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:21:45.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.756+0000 7f1870f4c640 1 Processor -- start 2024-04-24T23:21:45.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.756+0000 7f1870f4c640 1 -- start start 2024-04-24T23:21:45.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.757+0000 7f1870f4c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f186c15e710 0x7f186c0d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.757+0000 7f1870f4c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 0x7f186c0d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.757+0000 7f1870f4c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f186c0d27b0 con 0x7f186c15e330 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.757+0000 7f1870f4c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f186c0cef00 con 0x7f186c15e710 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.757+0000 7f1870f4c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f186c0cf1e0 con 0x7f186c0d4630 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.757+0000 7f1869d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f186c15e710 0x7f186c0d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.757+0000 7f1869d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f186c15e710 0x7f186c0d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46830/0 (socket says 172.21.15.26:46830) 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.757+0000 7f1869d9b640 1 -- 172.21.15.26:0/3213463565 learned_addr learned my addr 172.21.15.26:0/3213463565 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.758+0000 7f186ad9d640 1 --2- 172.21.15.26:0/3213463565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 0x7f186c0d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.758+0000 7f1869d9b640 1 -- 172.21.15.26:0/3213463565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 msgr2=0x7f186c0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.758+0000 7f1869d9b640 1 --2- 172.21.15.26:0/3213463565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 0x7f186c0d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.758+0000 7f1869d9b640 1 -- 172.21.15.26:0/3213463565 >> v1:172.21.15.29:6789/0 conn(0x7f186c15e330 legacy=0x7f186c0d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.758+0000 7f1869d9b640 1 -- 172.21.15.26:0/3213463565 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c0cf4c0 con 0x7f186c15e710 2024-04-24T23:21:45.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.758+0000 7f1869d9b640 1 --2- 172.21.15.26:0/3213463565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 0x7f186c0d20b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1854009870 tx=0x7f185402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7595480be106b853 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:45.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f186959a640 1 -- 172.21.15.26:0/3213463565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854037070 con 0x7f186c15e710 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f186959a640 1 -- 172.21.15.26:0/3213463565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1854032440 con 0x7f186c15e710 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f186959a640 1 -- 172.21.15.26:0/3213463565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854032810 con 0x7f186c15e710 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f1870f4c640 1 -- 172.21.15.26:0/3213463565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 msgr2=0x7f186c0d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f1870f4c640 1 --2- 172.21.15.26:0/3213463565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 0x7f186c0d20b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1854009870 tx=0x7f185402a270 comp rx=0 tx=0).stop 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f1870f4c640 1 -- 172.21.15.26:0/3213463565 shutdown_connections 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f1870f4c640 1 --2- 172.21.15.26:0/3213463565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 0x7f186c0d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f1870f4c640 1 --2- 172.21.15.26:0/3213463565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 0x7f186c0d20b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f1870f4c640 1 -- 172.21.15.26:0/3213463565 >> 172.21.15.26:0/3213463565 conn(0x7f186c08dbc0 msgr2=0x7f186c09d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f1870f4c640 1 -- 172.21.15.26:0/3213463565 shutdown_connections 2024-04-24T23:21:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.759+0000 7f1870f4c640 1 -- 172.21.15.26:0/3213463565 wait complete. 2024-04-24T23:21:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f1870f4c640 1 Processor -- start 2024-04-24T23:21:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f1870f4c640 1 -- start start 2024-04-24T23:21:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f1870f4c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 0x7f186c152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f1870f4c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f186c15e330 0x7f186c0b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f1870f4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 0x7f186c0b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f1870f4c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0cfe40 con 0x7f186c15e710 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f1870f4c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0cfcc0 con 0x7f186c15e330 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f1870f4c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f186c0cffc0 con 0x7f186c0d4630 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f186ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 0x7f186c0b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f186ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 0x7f186c0b8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46844/0 (socket says 172.21.15.26:46844) 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f186ad9d640 1 -- 172.21.15.26:0/749434273 learned_addr learned my addr 172.21.15.26:0/749434273 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f1869d9b640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f186c15e330 0x7f186c0b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.761+0000 7f186a59c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 0x7f186c152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f186ad9d640 1 -- 172.21.15.26:0/749434273 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 msgr2=0x7f186c152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f186ad9d640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 0x7f186c152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f186ad9d640 1 -- 172.21.15.26:0/749434273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f186c15e330 msgr2=0x7f186c0b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f186ad9d640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f186c15e330 0x7f186c0b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f186ad9d640 1 -- 172.21.15.26:0/749434273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c1613d0 con 0x7f186c15e710 2024-04-24T23:21:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f186a59c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 0x7f186c152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f186ad9d640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 0x7f186c0b8de0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f185c007e60 tx=0x7f185c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f1862ffd640 1 -- 172.21.15.26:0/749434273 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c014620 con 0x7f186c15e710 2024-04-24T23:21:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f1862ffd640 1 -- 172.21.15.26:0/749434273 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f185c019070 con 0x7f186c15e710 2024-04-24T23:21:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c1616b0 con 0x7f186c15e710 2024-04-24T23:21:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f1862ffd640 1 -- 172.21.15.26:0/749434273 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c014a00 con 0x7f186c15e710 2024-04-24T23:21:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.762+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c0c7d80 con 0x7f186c15e710 2024-04-24T23:21:45.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.763+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c0694c0 con 0x7f186c15e710 2024-04-24T23:21:45.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.764+0000 7f1862ffd640 1 -- 172.21.15.26:0/749434273 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f185c014ba0 con 0x7f186c15e710 2024-04-24T23:21:45.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.764+0000 7f1862ffd640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f184c06be30 0x7f184c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:45.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.764+0000 7f1862ffd640 1 -- 172.21.15.26:0/749434273 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f185c08b320 con 0x7f186c15e710 2024-04-24T23:21:45.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.764+0000 7f186a59c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f184c06be30 0x7f184c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:45.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.765+0000 7f186a59c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f184c06be30 0x7f184c06e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1864001700 tx=0x7f1864009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.768+0000 7f1862ffd640 1 -- 172.21.15.26:0/749434273 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f185c05a0b0 con 0x7f186c15e710 2024-04-24T23:21:45.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:45.951+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f186c158e00 con 0x7f186c15e710 2024-04-24T23:21:46.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.952+0000 7f1862ffd640 1 -- 172.21.15.26:0/749434273 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f185c0599b0 con 0x7f186c15e710 2024-04-24T23:21:46.954 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f184c06be30 msgr2=0x7f184c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f184c06be30 0x7f184c06e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1864001700 tx=0x7f1864009040 comp rx=0 tx=0).stop 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 msgr2=0x7f186c0b8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 0x7f186c0b8de0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f185c007e60 tx=0x7f185c00e4b0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 shutdown_connections 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f184c06be30 0x7f184c06e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f186c15e710 0x7f186c0b8de0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f186c15e330 0x7f186c0b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 --2- 172.21.15.26:0/749434273 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f186c0d4630 0x7f186c152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 >> 172.21.15.26:0/749434273 conn(0x7f186c08dbc0 msgr2=0x7f186c0b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 shutdown_connections 2024-04-24T23:21:46.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:46.956+0000 7f1870f4c640 1 -- 172.21.15.26:0/749434273 wait complete. 2024-04-24T23:21:46.977 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:21:47.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.093+0000 7fe57fb4f640 1 Processor -- start 2024-04-24T23:21:47.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.093+0000 7fe57fb4f640 1 -- start start 2024-04-24T23:21:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.093+0000 7fe57fb4f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5780b3150 0x7fe5780d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57fb4f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57806c840 con 0x7fe5780cdf30 2024-04-24T23:21:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57fb4f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57806cb20 con 0x7fe5780ced90 2024-04-24T23:21:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57fb4f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe5780d00d0 con 0x7fe5780b3150 2024-04-24T23:21:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57d8eb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe5780cdf30 0x7fe5780ce680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56692/0 (socket says 172.21.15.26:56692) 2024-04-24T23:21:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57d8eb640 1 -- 172.21.15.26:0/1956969985 learned_addr learned my addr 172.21.15.26:0/1956969985 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57e0ec640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5780b3150 0x7fe5780d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147173222 0 0) 0x7fe57806c840 con 0x7fe5780cdf30 2024-04-24T23:21:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe560003520 con 0x7fe5780cdf30 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856487535 0 0) 0x7fe57806cb20 con 0x7fe5780ced90 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe57806c840 con 0x7fe5780ced90 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4136888075 0 0) 0x7fe560003520 con 0x7fe5780cdf30 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5780b3150 msgr2=0x7fe5780d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5780b3150 0x7fe5780d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 >> v1:172.21.15.29:6789/0 conn(0x7fe5780ced90 legacy=0x7fe5780d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57806ce00 con 0x7fe5780cdf30 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fe570001670 con 0x7fe5780cdf30 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 >> v1:172.21.15.26:6789/0 conn(0x7fe5780cdf30 legacy=0x7fe5780ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe560003b70 0x7fe560003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe560004800 0x7fe560004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5600051c0 0x7fe56000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe56000bfb0 con 0x7fe560003b70 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe56000c130 con 0x7fe560004800 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe56000c330 con 0x7fe5600051c0 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.094+0000 7fe57e0ec640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5780b3150 0x7fe5780d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.095+0000 7fe57d8eb640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe560003b70 0x7fe560003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.095+0000 7fe57d0ea640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe560004800 0x7fe560004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.095+0000 7fe57e0ec640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5600051c0 0x7fe56000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.095+0000 7fe57d8eb640 1 -- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5600051c0 msgr2=0x7fe56000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.095+0000 7fe57d8eb640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5600051c0 0x7fe56000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.095+0000 7fe57d8eb640 1 -- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe560004800 msgr2=0x7fe560004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.095+0000 7fe57d8eb640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe560004800 0x7fe560004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.095+0000 7fe57d8eb640 1 -- 172.21.15.26:0/1956969985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe560003520 con 0x7fe560003b70 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57d8eb640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe560003b70 0x7fe560003f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c190 tx=0x7fe560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b131a59ba15973 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570007cc0 con 0x7fe560003b70 2024-04-24T23:21:47.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe570005af0 con 0x7fe560003b70 2024-04-24T23:21:47.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57c8e9640 1 -- 172.21.15.26:0/1956969985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570007070 con 0x7fe560003b70 2024-04-24T23:21:47.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57fb4f640 1 -- 172.21.15.26:0/1956969985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe560003b70 msgr2=0x7fe560003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57fb4f640 1 --2- 172.21.15.26:0/1956969985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe560003b70 0x7fe560003f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c190 tx=0x7fe560005830 comp rx=0 tx=0).stop 2024-04-24T23:21:47.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57e0ec640 1 -- 172.21.15.26:0/1956969985 reap_dead start 2024-04-24T23:21:47.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57fb4f640 1 -- 172.21.15.26:0/1956969985 shutdown_connections 2024-04-24T23:21:47.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57fb4f640 1 -- 172.21.15.26:0/1956969985 >> 172.21.15.26:0/1956969985 conn(0x7fe5780d1e00 msgr2=0x7fe5780d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:47.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57fb4f640 1 -- 172.21.15.26:0/1956969985 shutdown_connections 2024-04-24T23:21:47.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.096+0000 7fe57fb4f640 1 -- 172.21.15.26:0/1956969985 wait complete. 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.098+0000 7fe57fb4f640 1 Processor -- start 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.098+0000 7fe57fb4f640 1 -- start start 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.098+0000 7fe57fb4f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe560003b70 0x7fe5780b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.098+0000 7fe57fb4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5781f7530 0x7fe5781f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.098+0000 7fe57fb4f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5780ca810 0x7fe5780cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.098+0000 7fe57fb4f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe57806d3c0 con 0x7fe5781f7530 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.098+0000 7fe57fb4f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe57806c5f0 con 0x7fe5780ca810 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.098+0000 7fe57fb4f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe57806be30 con 0x7fe560003b70 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d0ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5781f7530 0x7fe5781f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d0ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5781f7530 0x7fe5781f79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46862/0 (socket says 172.21.15.26:46862) 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d0ea640 1 -- 172.21.15.26:0/995272423 learned_addr learned my addr 172.21.15.26:0/995272423 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57e0ec640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5780ca810 0x7fe5780cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d8eb640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe560003b70 0x7fe5780b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d0ea640 1 -- 172.21.15.26:0/995272423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe560003b70 msgr2=0x7fe5780b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d0ea640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe560003b70 0x7fe5780b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d0ea640 1 -- 172.21.15.26:0/995272423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5780ca810 msgr2=0x7fe5780cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d0ea640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5780ca810 0x7fe5780cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d0ea640 1 -- 172.21.15.26:0/995272423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5780cb570 con 0x7fe5781f7530 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57d0ea640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5781f7530 0x7fe5781f79b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00f850 tx=0x7fe56c00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe5767fc640 1 -- 172.21.15.26:0/995272423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00fe40 con 0x7fe5781f7530 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe5767fc640 1 -- 172.21.15.26:0/995272423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c018070 con 0x7fe5781f7530 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe5767fc640 1 -- 172.21.15.26:0/995272423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00d730 con 0x7fe5781f7530 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5780cb850 con 0x7fe5781f7530 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.099+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5780cbe10 con 0x7fe5781f7530 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.100+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5780cc050 con 0x7fe5781f7530 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.101+0000 7fe5767fc640 1 -- 172.21.15.26:0/995272423 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe56c00d8d0 con 0x7fe5781f7530 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.101+0000 7fe5767fc640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe56406be60 0x7fe56406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.101+0000 7fe5767fc640 1 -- 172.21.15.26:0/995272423 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7fe56c08a5f0 con 0x7fe5781f7530 2024-04-24T23:21:47.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.101+0000 7fe57d8eb640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe56406be60 0x7fe56406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.102+0000 7fe57d8eb640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe56406be60 0x7fe56406e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe570032000 tx=0x7fe570009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.106+0000 7fe5767fc640 1 -- 172.21.15.26:0/995272423 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe5780cc050 con 0x7fe5781f7530 2024-04-24T23:21:47.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.290+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe578075610 con 0x7fe5781f7530 2024-04-24T23:21:47.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.954+0000 7fe5767fc640 1 -- 172.21.15.26:0/995272423 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7fe56c021020 con 0x7fe5781f7530 2024-04-24T23:21:47.956 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-24T23:21:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe56406be60 msgr2=0x7fe56406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe56406be60 0x7fe56406e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe570032000 tx=0x7fe570009030 comp rx=0 tx=0).stop 2024-04-24T23:21:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5781f7530 msgr2=0x7fe5781f79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5781f7530 0x7fe5781f79b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00f850 tx=0x7fe56c00fd10 comp rx=0 tx=0).stop 2024-04-24T23:21:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 shutdown_connections 2024-04-24T23:21:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe56406be60 0x7fe56406e320 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5780ca810 0x7fe5780cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5781f7530 0x7fe5781f79b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 --2- 172.21.15.26:0/995272423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe560003b70 0x7fe5780b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.958+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 >> 172.21.15.26:0/995272423 conn(0x7fe5780d1e00 msgr2=0x7fe5780ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.959+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 shutdown_connections 2024-04-24T23:21:47.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:47.959+0000 7fe57fb4f640 1 -- 172.21.15.26:0/995272423 wait complete. 2024-04-24T23:21:47.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:21:48.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.088+0000 7fd3d0170640 1 Processor -- start 2024-04-24T23:21:48.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.089+0000 7fd3d0170640 1 -- start start 2024-04-24T23:21:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.089+0000 7fd3d0170640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3c815e750 0x7fd3c80cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.089+0000 7fd3d0170640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd3c80d2c30 0x7fd3c80cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.089+0000 7fd3d0170640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c80d2120 con 0x7fd3c815e370 2024-04-24T23:21:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.089+0000 7fd3d0170640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c80d1510 con 0x7fd3c80d2c30 2024-04-24T23:21:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.089+0000 7fd3d0170640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c80d1ed0 con 0x7fd3c815e750 2024-04-24T23:21:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3cdf0c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd3c815e370 0x7fd3c80d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56696/0 (socket says 172.21.15.26:56696) 2024-04-24T23:21:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3cdf0c640 1 -- 172.21.15.26:0/1462040764 learned_addr learned my addr 172.21.15.26:0/1462040764 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3cd70b640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3c815e750 0x7fd3c80cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 323350704 0 0) 0x7fd3c80d2120 con 0x7fd3c815e370 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3b4003520 con 0x7fd3c815e370 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ce70d640 1 --2- 172.21.15.26:0/1462040764 >> v2:172.21.15.29:3300/0 conn(0x7fd3c80d2c30 0x7fd3c80cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 944651653 0 0) 0x7fd3b4003520 con 0x7fd3c815e370 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3c815e750 msgr2=0x7fd3c80cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3c815e750 0x7fd3c80cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 >> v2:172.21.15.29:3300/0 conn(0x7fd3c80d2c30 msgr2=0x7fd3c80cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 --2- 172.21.15.26:0/1462040764 >> v2:172.21.15.29:3300/0 conn(0x7fd3c80d2c30 0x7fd3c80cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c80b3740 con 0x7fd3c815e370 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fd3c4001670 con 0x7fd3c815e370 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 >> v1:172.21.15.26:6789/0 conn(0x7fd3c815e370 legacy=0x7fd3c80d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3b40039b0 0x7fd3b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3b4004800 0x7fd3b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3b4005180 0x7fd3b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80d1ed0 con 0x7fd3b40039b0 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b40045a0 con 0x7fd3b4005180 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b400bf00 con 0x7fd3b4004800 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3cdf0c640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3b40039b0 0x7fd3b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.090+0000 7fd3cd70b640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3b4004800 0x7fd3b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.091+0000 7fd3ce70d640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3b4005180 0x7fd3b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.091+0000 7fd3cdf0c640 1 -- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3b4004800 msgr2=0x7fd3b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.091+0000 7fd3cdf0c640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3b4004800 0x7fd3b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.091+0000 7fd3cdf0c640 1 -- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3b4005180 msgr2=0x7fd3b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.091+0000 7fd3cdf0c640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3b4005180 0x7fd3b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.091+0000 7fd3cdf0c640 1 -- 172.21.15.26:0/1462040764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b4003520 con 0x7fd3b40039b0 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.091+0000 7fd3cdf0c640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3b40039b0 0x7fd3b4003df0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400c270 tx=0x7fd3b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d702b6c38a524c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4007d30 con 0x7fd3b40039b0 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c4005ad0 con 0x7fd3b40039b0 2024-04-24T23:21:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3ccf0a640 1 -- 172.21.15.26:0/1462040764 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4007070 con 0x7fd3b40039b0 2024-04-24T23:21:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3d0170640 1 -- 172.21.15.26:0/1462040764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3b40039b0 msgr2=0x7fd3b4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3d0170640 1 --2- 172.21.15.26:0/1462040764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3b40039b0 0x7fd3b4003df0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400c270 tx=0x7fd3b4005830 comp rx=0 tx=0).stop 2024-04-24T23:21:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3ce70d640 1 -- 172.21.15.26:0/1462040764 reap_dead start 2024-04-24T23:21:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3d0170640 1 -- 172.21.15.26:0/1462040764 shutdown_connections 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3d0170640 1 -- 172.21.15.26:0/1462040764 >> 172.21.15.26:0/1462040764 conn(0x7fd3c80d40f0 msgr2=0x7fd3c80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3d0170640 1 -- 172.21.15.26:0/1462040764 shutdown_connections 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.092+0000 7fd3d0170640 1 -- 172.21.15.26:0/1462040764 wait complete. 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.093+0000 7fd3d0170640 1 Processor -- start 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.093+0000 7fd3d0170640 1 -- start start 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3d0170640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3c815ce10 0x7fd3c80b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3d0170640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3c80b9a60 0x7fd3c81e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3d0170640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c81e4d30 0x7fd3c815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3d0170640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80b5230 con 0x7fd3c81e4d30 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3d0170640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80b3d00 con 0x7fd3c815ce10 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3d0170640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c80b53b0 con 0x7fd3c80b9a60 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3ce70d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c81e4d30 0x7fd3c815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3ce70d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c81e4d30 0x7fd3c815a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46884/0 (socket says 172.21.15.26:46884) 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3ce70d640 1 -- 172.21.15.26:0/1620754876 learned_addr learned my addr 172.21.15.26:0/1620754876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3cd70b640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3c80b9a60 0x7fd3c81e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.094+0000 7fd3cdf0c640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3c815ce10 0x7fd3c80b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3ce70d640 1 -- 172.21.15.26:0/1620754876 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3c80b9a60 msgr2=0x7fd3c81e47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3ce70d640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3c80b9a60 0x7fd3c81e47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3ce70d640 1 -- 172.21.15.26:0/1620754876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3c815ce10 msgr2=0x7fd3c80b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3ce70d640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3c815ce10 0x7fd3c80b9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3ce70d640 1 -- 172.21.15.26:0/1620754876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c80c8b20 con 0x7fd3c81e4d30 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3ce70d640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c81e4d30 0x7fd3c815a770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80b5bb0 tx=0x7fd3c00070e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3be7fc640 1 -- 172.21.15.26:0/1620754876 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000fcd0 con 0x7fd3c81e4d30 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3be7fc640 1 -- 172.21.15.26:0/1620754876 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c001d070 con 0x7fd3c81e4d30 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c80c8e00 con 0x7fd3c81e4d30 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3be7fc640 1 -- 172.21.15.26:0/1620754876 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0015670 con 0x7fd3c81e4d30 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.095+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c80b8d20 con 0x7fd3c81e4d30 2024-04-24T23:21:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.096+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c8075610 con 0x7fd3c81e4d30 2024-04-24T23:21:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.096+0000 7fd3be7fc640 1 -- 172.21.15.26:0/1620754876 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3c0015810 con 0x7fd3c81e4d30 2024-04-24T23:21:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.097+0000 7fd3be7fc640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3ac06be60 0x7fd3ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.097+0000 7fd3be7fc640 1 -- 172.21.15.26:0/1620754876 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7fd3c008b4d0 con 0x7fd3c81e4d30 2024-04-24T23:21:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.097+0000 7fd3cdf0c640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3ac06be60 0x7fd3ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.097+0000 7fd3cdf0c640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3ac06be60 0x7fd3ac06e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4033000 tx=0x7fd3c4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.101+0000 7fd3be7fc640 1 -- 172.21.15.26:0/1620754876 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd3c0010040 con 0x7fd3c81e4d30 2024-04-24T23:21:48.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.323+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd3c806d2b0 con 0x7fd3c81e4d30 2024-04-24T23:21:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:48.999+0000 7fd3be7fc640 1 -- 172.21.15.26:0/1620754876 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fd3c0059ee0 con 0x7fd3c81e4d30 2024-04-24T23:21:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 4 and data pool 5 2024-04-24T23:21:49.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.009+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3ac06be60 msgr2=0x7fd3ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.009+0000 7fd3d0170640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3ac06be60 0x7fd3ac06e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4033000 tx=0x7fd3c4009030 comp rx=0 tx=0).stop 2024-04-24T23:21:49.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.009+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c81e4d30 msgr2=0x7fd3c815a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.009+0000 7fd3d0170640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c81e4d30 0x7fd3c815a770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80b5bb0 tx=0x7fd3c00070e0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.011+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 shutdown_connections 2024-04-24T23:21:49.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.011+0000 7fd3d0170640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3ac06be60 0x7fd3ac06e320 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.011+0000 7fd3d0170640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c81e4d30 0x7fd3c815a770 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.011+0000 7fd3d0170640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd3c80b9a60 0x7fd3c81e47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.011+0000 7fd3d0170640 1 --2- 172.21.15.26:0/1620754876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3c815ce10 0x7fd3c80b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.011+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 >> 172.21.15.26:0/1620754876 conn(0x7fd3c80d40f0 msgr2=0x7fd3c80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:49.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.012+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 shutdown_connections 2024-04-24T23:21:49.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.013+0000 7fd3d0170640 1 -- 172.21.15.26:0/1620754876 wait complete. 2024-04-24T23:21:49.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:49.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.142+0000 7f3a1ffee640 1 Processor -- start 2024-04-24T23:21:49.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.142+0000 7f3a1ffee640 1 -- start start 2024-04-24T23:21:49.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1ffee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1806d350 0x7f3a1806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1ffee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a180cfbc0 con 0x7f3a18145910 2024-04-24T23:21:49.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1ffee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a1806d7f0 con 0x7f3a180cd660 2024-04-24T23:21:49.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1ffee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3a18145cf0 con 0x7f3a1806d350 2024-04-24T23:21:49.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1dd8a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3a18145910 0x7f3a1815d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56698/0 (socket says 172.21.15.26:56698) 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1dd8a640 1 -- 172.21.15.26:0/1947689781 learned_addr learned my addr 172.21.15.26:0/1947689781 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1d589640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1806d350 0x7f3a1806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747542969 0 0) 0x7f3a180cfbc0 con 0x7f3a18145910 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a04003520 con 0x7f3a18145910 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2659996695 0 0) 0x7f3a04003520 con 0x7f3a18145910 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1806d350 msgr2=0x7f3a1806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1806d350 0x7f3a1806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 >> v1:172.21.15.29:6789/0 conn(0x7f3a180cd660 legacy=0x7f3a1815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1806dad0 con 0x7f3a18145910 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f3a14001670 con 0x7f3a18145910 2024-04-24T23:21:49.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 >> v1:172.21.15.26:6789/0 conn(0x7f3a18145910 legacy=0x7f3a1815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1806d350 0x7f3a040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.143+0000 7f3a1cd88640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a04003f20 0x7f3a04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1cd88640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04004d40 0x7f3a040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18145cf0 con 0x7f3a04004d40 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0400ba20 con 0x7f3a04003f20 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3a0400bc20 con 0x7f3a1806d350 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1e58b640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04004d40 0x7f3a040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1dd8a640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1806d350 0x7f3a040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1d589640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a04003f20 0x7f3a04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1e58b640 1 -- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1806d350 msgr2=0x7f3a040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1e58b640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1806d350 0x7f3a040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1e58b640 1 -- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a04003f20 msgr2=0x7f3a04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1e58b640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a04003f20 0x7f3a04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1e58b640 1 -- 172.21.15.26:0/1947689781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a04003520 con 0x7f3a04004d40 2024-04-24T23:21:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.144+0000 7f3a1dd8a640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1806d350 0x7f3a040039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1e58b640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04004d40 0x7f3a040051c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3a10001f60 tx=0x7f3a04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f9bcdcbdd66572 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10001b30 con 0x7f3a04004d40 2024-04-24T23:21:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a10031620 con 0x7f3a04004d40 2024-04-24T23:21:49.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1cd88640 1 -- 172.21.15.26:0/1947689781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10031a10 con 0x7f3a04004d40 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1ffee640 1 -- 172.21.15.26:0/1947689781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04004d40 msgr2=0x7f3a040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1ffee640 1 --2- 172.21.15.26:0/1947689781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04004d40 0x7f3a040051c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3a10001f60 tx=0x7f3a04005830 comp rx=0 tx=0).stop 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1e58b640 1 -- 172.21.15.26:0/1947689781 reap_dead start 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1ffee640 1 -- 172.21.15.26:0/1947689781 shutdown_connections 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1ffee640 1 -- 172.21.15.26:0/1947689781 >> 172.21.15.26:0/1947689781 conn(0x7f3a1806b300 msgr2=0x7f3a1806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1ffee640 1 -- 172.21.15.26:0/1947689781 shutdown_connections 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.145+0000 7f3a1ffee640 1 -- 172.21.15.26:0/1947689781 wait complete. 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1ffee640 1 Processor -- start 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1ffee640 1 -- start start 2024-04-24T23:21:49.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1ffee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1813c5f0 0x7f3a180abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1ffee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181f4fc0 0x7f3a181f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1ffee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1815e630 0x7f3a1815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1ffee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1806e370 con 0x7f3a181f4fc0 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1ffee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1806e090 con 0x7f3a1815e630 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1ffee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3a1806e890 con 0x7f3a1813c5f0 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1d589640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181f4fc0 0x7f3a181f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1d589640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181f4fc0 0x7f3a181f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46898/0 (socket says 172.21.15.26:46898) 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.147+0000 7f3a1d589640 1 -- 172.21.15.26:0/374518737 learned_addr learned my addr 172.21.15.26:0/374518737 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1e58b640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1815e630 0x7f3a1815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1dd8a640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1813c5f0 0x7f3a180abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1d589640 1 -- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1813c5f0 msgr2=0x7f3a180abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1d589640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1813c5f0 0x7f3a180abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1d589640 1 -- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1815e630 msgr2=0x7f3a1815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1d589640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1815e630 0x7f3a1815eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1d589640 1 -- 172.21.15.26:0/374518737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a181f1b00 con 0x7f3a181f4fc0 2024-04-24T23:21:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1e58b640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1815e630 0x7f3a1815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:49.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1dd8a640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1813c5f0 0x7f3a180abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1d589640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181f4fc0 0x7f3a181f2290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3a08011570 tx=0x7f3a08011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a0e7fc640 1 -- 172.21.15.26:0/374518737 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08015930 con 0x7f3a181f4fc0 2024-04-24T23:21:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a0e7fc640 1 -- 172.21.15.26:0/374518737 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a08015ad0 con 0x7f3a181f4fc0 2024-04-24T23:21:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a181f5810 con 0x7f3a181f4fc0 2024-04-24T23:21:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a0e7fc640 1 -- 172.21.15.26:0/374518737 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0800f460 con 0x7f3a181f4fc0 2024-04-24T23:21:49.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.148+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a181f1d70 con 0x7f3a181f4fc0 2024-04-24T23:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.149+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a180694c0 con 0x7f3a181f4fc0 2024-04-24T23:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.150+0000 7f3a0e7fc640 1 -- 172.21.15.26:0/374518737 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a0800f600 con 0x7f3a181f4fc0 2024-04-24T23:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.150+0000 7f3a0e7fc640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f39fc06be60 0x7f39fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.150+0000 7f3a1dd8a640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f39fc06be60 0x7f39fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.150+0000 7f3a0e7fc640 1 -- 172.21.15.26:0/374518737 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3a0808b6b0 con 0x7f3a181f4fc0 2024-04-24T23:21:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.151+0000 7f3a1dd8a640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f39fc06be60 0x7f39fc06e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a14005a50 tx=0x7f3a14001e10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.155+0000 7f3a0e7fc640 1 -- 172.21.15.26:0/374518737 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a08019030 con 0x7f3a181f4fc0 2024-04-24T23:21:49.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.340+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a1806b960 con 0x7f3a181f4fc0 2024-04-24T23:21:49.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.342+0000 7f3a0e7fc640 1 -- 172.21.15.26:0/374518737 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16622 (secure 0 0 0) 0x7f3a0805a060 con 0x7f3a181f4fc0 2024-04-24T23:21:49.343 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:49.343 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":32,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:48.324581+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:21:45.983358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:21:47.321239+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:49.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.343+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f39fc06be60 msgr2=0x7f39fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.343+0000 7f3a1ffee640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f39fc06be60 0x7f39fc06e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a14005a50 tx=0x7f3a14001e10 comp rx=0 tx=0).stop 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.343+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181f4fc0 msgr2=0x7f3a181f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.343+0000 7f3a1ffee640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181f4fc0 0x7f3a181f2290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3a08011570 tx=0x7f3a08011a30 comp rx=0 tx=0).stop 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.344+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 shutdown_connections 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.344+0000 7f3a1ffee640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f39fc06be60 0x7f39fc06e320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.344+0000 7f3a1ffee640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1815e630 0x7f3a1815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.344+0000 7f3a1ffee640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a181f4fc0 0x7f3a181f2290 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.344+0000 7f3a1ffee640 1 --2- 172.21.15.26:0/374518737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3a1813c5f0 0x7f3a180abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.344+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 >> 172.21.15.26:0/374518737 conn(0x7f3a1806b300 msgr2=0x7f3a180b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.344+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 shutdown_connections 2024-04-24T23:21:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.344+0000 7f3a1ffee640 1 -- 172.21.15.26:0/374518737 wait complete. 2024-04-24T23:21:49.356 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:49.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.455+0000 7f743beda640 1 Processor -- start 2024-04-24T23:21:49.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.455+0000 7f743beda640 1 -- start start 2024-04-24T23:21:49.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f743beda640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f743406f110 0x7f743415f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f743beda640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f743406f4f0 0x7f74340ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f743beda640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f74340addb0 0x7f7434154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f743beda640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f74340ae220 con 0x7f743406f110 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f743beda640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f743415f750 con 0x7f74340addb0 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f743beda640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f74340b0220 con 0x7f743406f4f0 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f7439c76640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f743406f110 0x7f743415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f7439c76640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f743406f110 0x7f743415f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46908/0 (socket says 172.21.15.26:46908) 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f7439c76640 1 -- 172.21.15.26:0/2994679863 learned_addr learned my addr 172.21.15.26:0/2994679863 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f743a477640 1 --2- 172.21.15.26:0/2994679863 >> v2:172.21.15.29:3300/0 conn(0x7f74340addb0 0x7f7434154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.456+0000 7f7439475640 1 --2- 172.21.15.26:0/2994679863 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f743406f4f0 0x7f74340ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.457+0000 7f7439c76640 1 -- 172.21.15.26:0/2994679863 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f743406f4f0 msgr2=0x7f74340ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.457+0000 7f7439c76640 1 --2- 172.21.15.26:0/2994679863 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f743406f4f0 0x7f74340ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.457+0000 7f7439c76640 1 -- 172.21.15.26:0/2994679863 >> v2:172.21.15.29:3300/0 conn(0x7f74340addb0 msgr2=0x7f7434154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.457+0000 7f7439c76640 1 --2- 172.21.15.26:0/2994679863 >> v2:172.21.15.29:3300/0 conn(0x7f74340addb0 0x7f7434154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.457+0000 7f7439c76640 1 -- 172.21.15.26:0/2994679863 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74340af180 con 0x7f743406f110 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.457+0000 7f7439c76640 1 --2- 172.21.15.26:0/2994679863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 0x7f743415f210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7434161ce0 tx=0x7f742c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e81550f60104cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.457+0000 7f7438c74640 1 -- 172.21.15.26:0/2994679863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c037070 con 0x7f743406f110 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.457+0000 7f7438c74640 1 -- 172.21.15.26:0/2994679863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c032440 con 0x7f743406f110 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f7438c74640 1 -- 172.21.15.26:0/2994679863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c032810 con 0x7f743406f110 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f743beda640 1 -- 172.21.15.26:0/2994679863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 msgr2=0x7f743415f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f743beda640 1 --2- 172.21.15.26:0/2994679863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 0x7f743415f210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7434161ce0 tx=0x7f742c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f743beda640 1 -- 172.21.15.26:0/2994679863 shutdown_connections 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f743beda640 1 --2- 172.21.15.26:0/2994679863 >> v2:172.21.15.29:3300/0 conn(0x7f74340addb0 0x7f7434154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f743beda640 1 --2- 172.21.15.26:0/2994679863 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f743406f4f0 0x7f74340ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f743beda640 1 --2- 172.21.15.26:0/2994679863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 0x7f743415f210 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f743beda640 1 -- 172.21.15.26:0/2994679863 >> 172.21.15.26:0/2994679863 conn(0x7f74340729a0 msgr2=0x7f743406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:49.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f743beda640 1 -- 172.21.15.26:0/2994679863 shutdown_connections 2024-04-24T23:21:49.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.458+0000 7f743beda640 1 -- 172.21.15.26:0/2994679863 wait complete. 2024-04-24T23:21:49.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.459+0000 7f743beda640 1 Processor -- start 2024-04-24T23:21:49.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.459+0000 7f743beda640 1 -- start start 2024-04-24T23:21:49.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f743beda640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 0x7f7434148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f743beda640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f743406f4f0 0x7f74341e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f743beda640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f74340addb0 0x7f74341e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f743beda640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f74340afa20 con 0x7f743406f110 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f743beda640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f74340af740 con 0x7f743406f4f0 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f743beda640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f74340afd00 con 0x7f74340addb0 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f7439c76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 0x7f7434148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f7439c76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 0x7f7434148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46918/0 (socket says 172.21.15.26:46918) 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f7439c76640 1 -- 172.21.15.26:0/3716589136 learned_addr learned my addr 172.21.15.26:0/3716589136 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f7439475640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f743406f4f0 0x7f74341e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f743a477640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f74340addb0 0x7f74341e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f7439c76640 1 -- 172.21.15.26:0/3716589136 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f74340addb0 msgr2=0x7f74341e63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f7439c76640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f74340addb0 0x7f74341e63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f7439c76640 1 -- 172.21.15.26:0/3716589136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f743406f4f0 msgr2=0x7f74341e5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f7439c76640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f743406f4f0 0x7f74341e5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.460+0000 7f7439c76640 1 -- 172.21.15.26:0/3716589136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74341e5690 con 0x7f743406f110 2024-04-24T23:21:49.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.461+0000 7f7439c76640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 0x7f7434148200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f742c02a220 tx=0x7f742c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.461+0000 7f74327fc640 1 -- 172.21.15.26:0/3716589136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c041020 con 0x7f743406f110 2024-04-24T23:21:49.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.461+0000 7f74327fc640 1 -- 172.21.15.26:0/3716589136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c0306c0 con 0x7f743406f110 2024-04-24T23:21:49.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.461+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7434158320 con 0x7f743406f110 2024-04-24T23:21:49.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.461+0000 7f74327fc640 1 -- 172.21.15.26:0/3716589136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c037040 con 0x7f743406f110 2024-04-24T23:21:49.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.461+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74341588e0 con 0x7f743406f110 2024-04-24T23:21:49.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.462+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74340694c0 con 0x7f743406f110 2024-04-24T23:21:49.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.462+0000 7f74327fc640 1 -- 172.21.15.26:0/3716589136 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f742c044020 con 0x7f743406f110 2024-04-24T23:21:49.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.463+0000 7f74327fc640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f742006be30 0x7f742006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.463+0000 7f74327fc640 1 -- 172.21.15.26:0/3716589136 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f742c0ab6a0 con 0x7f743406f110 2024-04-24T23:21:49.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.463+0000 7f7439475640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f742006be30 0x7f742006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.463+0000 7f7439475640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f742006be30 0x7f742006e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74340b66f0 tx=0x7f7428008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.467+0000 7f74327fc640 1 -- 172.21.15.26:0/3716589136 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f742c04b030 con 0x7f743406f110 2024-04-24T23:21:49.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.650+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7434148740 con 0x7f743406f110 2024-04-24T23:21:49.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.651+0000 7f74327fc640 1 -- 172.21.15.26:0/3716589136 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16622 (secure 0 0 0) 0x7f742c07a050 con 0x7f743406f110 2024-04-24T23:21:49.653 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:49.653 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":32,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:48.324581+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:21:45.983358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:21:47.321239+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:49.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f742006be30 msgr2=0x7f742006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f742006be30 0x7f742006e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74340b66f0 tx=0x7f7428008040 comp rx=0 tx=0).stop 2024-04-24T23:21:49.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 msgr2=0x7f7434148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 0x7f7434148200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f742c02a220 tx=0x7f742c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 shutdown_connections 2024-04-24T23:21:49.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f742006be30 0x7f742006e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f74340addb0 0x7f74341e63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f743406f4f0 0x7f74341e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 --2- 172.21.15.26:0/3716589136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f743406f110 0x7f7434148200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 >> 172.21.15.26:0/3716589136 conn(0x7f74340729a0 msgr2=0x7f743415ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 shutdown_connections 2024-04-24T23:21:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.653+0000 7f743beda640 1 -- 172.21.15.26:0/3716589136 wait complete. 2024-04-24T23:21:49.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:21:49.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.768+0000 7ff2915dc640 1 Processor -- start 2024-04-24T23:21:49.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.768+0000 7ff2915dc640 1 -- start start 2024-04-24T23:21:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff2915dc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff28c0ce640 0x7ff28c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff2915dc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c15da30 0x7ff28c15deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff2915dc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff28c06d750 con 0x7ff28c0d28e0 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff2915dc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff28c0d1110 con 0x7ff28c0ce640 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff2915dc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c0cc410 con 0x7ff28c15da30 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff28affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff28c0ce640 0x7ff28c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff28affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff28c0ce640 0x7ff28c0d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46932/0 (socket says 172.21.15.26:46932) 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff28affd640 1 -- 172.21.15.26:0/3246692757 learned_addr learned my addr 172.21.15.26:0/3246692757 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff28b7fe640 1 --2- 172.21.15.26:0/3246692757 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c15da30 0x7ff28c15deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff28affd640 1 -- 172.21.15.26:0/3246692757 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c15da30 msgr2=0x7ff28c15deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff28affd640 1 --2- 172.21.15.26:0/3246692757 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c15da30 0x7ff28c15deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff28affd640 1 -- 172.21.15.26:0/3246692757 >> v1:172.21.15.29:6789/0 conn(0x7ff28c0d28e0 legacy=0x7ff28c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.769+0000 7ff28affd640 1 -- 172.21.15.26:0/3246692757 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c0b1720 con 0x7ff28c0ce640 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.770+0000 7ff28affd640 1 --2- 172.21.15.26:0/3246692757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 0x7ff28c0d00f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff284009870 tx=0x7ff28402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd71c7847c046a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.770+0000 7ff289ffb640 1 -- 172.21.15.26:0/3246692757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284037070 con 0x7ff28c0ce640 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.770+0000 7ff289ffb640 1 -- 172.21.15.26:0/3246692757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff284032440 con 0x7ff28c0ce640 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.770+0000 7ff289ffb640 1 -- 172.21.15.26:0/3246692757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284032810 con 0x7ff28c0ce640 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.771+0000 7ff2915dc640 1 -- 172.21.15.26:0/3246692757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 msgr2=0x7ff28c0d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.771+0000 7ff2915dc640 1 --2- 172.21.15.26:0/3246692757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 0x7ff28c0d00f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff284009870 tx=0x7ff28402a2a0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.771+0000 7ff2915dc640 1 -- 172.21.15.26:0/3246692757 shutdown_connections 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.771+0000 7ff2915dc640 1 --2- 172.21.15.26:0/3246692757 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c15da30 0x7ff28c15deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.771+0000 7ff2915dc640 1 --2- 172.21.15.26:0/3246692757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 0x7ff28c0d00f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.771+0000 7ff2915dc640 1 -- 172.21.15.26:0/3246692757 >> 172.21.15.26:0/3246692757 conn(0x7ff28c0d1dc0 msgr2=0x7ff28c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.771+0000 7ff2915dc640 1 -- 172.21.15.26:0/3246692757 shutdown_connections 2024-04-24T23:21:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.771+0000 7ff2915dc640 1 -- 172.21.15.26:0/3246692757 wait complete. 2024-04-24T23:21:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.772+0000 7ff2915dc640 1 Processor -- start 2024-04-24T23:21:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.772+0000 7ff2915dc640 1 -- start start 2024-04-24T23:21:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff2915dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 0x7ff28c15bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff2915dc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c0d28e0 0x7ff28c09c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff2915dc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff28c15da30 0x7ff28c09f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff2915dc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c0cc940 con 0x7ff28c0ce640 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff2915dc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c0cc660 con 0x7ff28c15da30 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff2915dc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c0ccc20 con 0x7ff28c0d28e0 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 0x7ff28c15bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 0x7ff28c15bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46948/0 (socket says 172.21.15.26:46948) 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28affd640 1 -- 172.21.15.26:0/879624727 learned_addr learned my addr 172.21.15.26:0/879624727 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28a7fc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c0d28e0 0x7ff28c09c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28b7fe640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff28c15da30 0x7ff28c09f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28affd640 1 -- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c0d28e0 msgr2=0x7ff28c09c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28affd640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c0d28e0 0x7ff28c09c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28affd640 1 -- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff28c15da30 msgr2=0x7ff28c09f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28affd640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff28c15da30 0x7ff28c09f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28affd640 1 -- 172.21.15.26:0/879624727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c09e160 con 0x7ff28c0ce640 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28a7fc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c0d28e0 0x7ff28c09c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.773+0000 7ff28b7fe640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff28c15da30 0x7ff28c09f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.774+0000 7ff28affd640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 0x7ff28c15bd00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff284009870 tx=0x7ff2840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.774+0000 7ff2837fe640 1 -- 172.21.15.26:0/879624727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284041020 con 0x7ff28c0ce640 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.774+0000 7ff2837fe640 1 -- 172.21.15.26:0/879624727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2840306c0 con 0x7ff28c0ce640 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.774+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c09e440 con 0x7ff28c0ce640 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.774+0000 7ff2837fe640 1 -- 172.21.15.26:0/879624727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284037040 con 0x7ff28c0ce640 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.774+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c09ea00 con 0x7ff28c0ce640 2024-04-24T23:21:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.775+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c075610 con 0x7ff28c0ce640 2024-04-24T23:21:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.775+0000 7ff2837fe640 1 -- 172.21.15.26:0/879624727 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff284044020 con 0x7ff28c0ce640 2024-04-24T23:21:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.776+0000 7ff2837fe640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff26c06be30 0x7ff26c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.776+0000 7ff2837fe640 1 -- 172.21.15.26:0/879624727 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff2840ab7a0 con 0x7ff28c0ce640 2024-04-24T23:21:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.776+0000 7ff28a7fc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff26c06be30 0x7ff26c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.776+0000 7ff28a7fc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff26c06be30 0x7ff26c06e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0b7670 tx=0x7ff274009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:49.780+0000 7ff2837fe640 1 -- 172.21.15.26:0/879624727 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff28404b030 con 0x7ff28c0ce640 2024-04-24T23:21:50.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:50.008+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff28c070c10 con 0x7ff28c0ce640 2024-04-24T23:21:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.006+0000 7ff2837fe640 1 -- 172.21.15.26:0/879624727 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7ff28c0b1720 con 0x7ff28c0ce640 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff26c06be30 msgr2=0x7ff26c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff26c06be30 0x7ff26c06e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0b7670 tx=0x7ff274009040 comp rx=0 tx=0).stop 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 msgr2=0x7ff28c15bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 0x7ff28c15bd00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff284009870 tx=0x7ff2840046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 shutdown_connections 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff26c06be30 0x7ff26c06e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff28c15da30 0x7ff28c09f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff28c0d28e0 0x7ff28c09c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 --2- 172.21.15.26:0/879624727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28c0ce640 0x7ff28c15bd00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 >> 172.21.15.26:0/879624727 conn(0x7ff28c0d1dc0 msgr2=0x7ff28c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 shutdown_connections 2024-04-24T23:21:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.009+0000 7ff2915dc640 1 -- 172.21.15.26:0/879624727 wait complete. 2024-04-24T23:21:51.024 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:21:51.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:21:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.133+0000 7fe0277f9640 1 Processor -- start 2024-04-24T23:21:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.133+0000 7fe0277f9640 1 -- start start 2024-04-24T23:21:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe0277f9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0200ce640 0x7fe0200d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe0277f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe0277f9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe02006d750 con 0x7fe02006c4d0 2024-04-24T23:21:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe0277f9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe0200d1110 con 0x7fe0200ce640 2024-04-24T23:21:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe0277f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe0200cc410 con 0x7fe0200d28e0 2024-04-24T23:21:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe024d94640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe025595640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0200ce640 0x7fe0200d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe024d94640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:51152/0 (socket says 172.21.15.26:51152) 2024-04-24T23:21:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe025595640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0200ce640 0x7fe0200d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:33310/0 (socket says 172.21.15.26:33310) 2024-04-24T23:21:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.134+0000 7fe024d94640 1 -- 172.21.15.26:0/2308335723 learned_addr learned my addr 172.21.15.26:0/2308335723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.135+0000 7fe025595640 1 -- 172.21.15.26:0/2308335723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 msgr2=0x7fe0200cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.135+0000 7fe025595640 1 --2- 172.21.15.26:0/2308335723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.135+0000 7fe025595640 1 -- 172.21.15.26:0/2308335723 >> v1:172.21.15.29:6789/0 conn(0x7fe02006c4d0 legacy=0x7fe02006c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.135+0000 7fe025595640 1 -- 172.21.15.26:0/2308335723 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0200b1720 con 0x7fe0200ce640 2024-04-24T23:21:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.135+0000 7fe024d94640 1 --2- 172.21.15.26:0/2308335723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.135+0000 7fe025595640 1 --2- 172.21.15.26:0/2308335723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0200ce640 0x7fe0200d00f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0097a0 tx=0x7fe01c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f80d21b66c22699 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe017fff640 1 -- 172.21.15.26:0/2308335723 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c037070 con 0x7fe0200ce640 2024-04-24T23:21:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe017fff640 1 -- 172.21.15.26:0/2308335723 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe01c032440 con 0x7fe0200ce640 2024-04-24T23:21:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe017fff640 1 -- 172.21.15.26:0/2308335723 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c032810 con 0x7fe0200ce640 2024-04-24T23:21:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe0277f9640 1 -- 172.21.15.26:0/2308335723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0200ce640 msgr2=0x7fe0200d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe0277f9640 1 --2- 172.21.15.26:0/2308335723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0200ce640 0x7fe0200d00f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0097a0 tx=0x7fe01c02a210 comp rx=0 tx=0).stop 2024-04-24T23:21:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe0277f9640 1 -- 172.21.15.26:0/2308335723 shutdown_connections 2024-04-24T23:21:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe0277f9640 1 --2- 172.21.15.26:0/2308335723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe0277f9640 1 --2- 172.21.15.26:0/2308335723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0200ce640 0x7fe0200d00f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe0277f9640 1 -- 172.21.15.26:0/2308335723 >> 172.21.15.26:0/2308335723 conn(0x7fe0200d1dc0 msgr2=0x7fe0200d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe0277f9640 1 -- 172.21.15.26:0/2308335723 shutdown_connections 2024-04-24T23:21:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.136+0000 7fe0277f9640 1 -- 172.21.15.26:0/2308335723 wait complete. 2024-04-24T23:21:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.137+0000 7fe0277f9640 1 Processor -- start 2024-04-24T23:21:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe0277f9640 1 -- start start 2024-04-24T23:21:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe0277f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe02006c4d0 0x7fe0200c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe0277f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0200ce640 0x7fe0200b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe0277f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe0277f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0200cc660 con 0x7fe0200ce640 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe0277f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0200d1360 con 0x7fe02006c4d0 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe0277f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe0200cc940 con 0x7fe0200d28e0 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe024d94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0200ce640 0x7fe0200b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe024d94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0200ce640 0x7fe0200b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46958/0 (socket says 172.21.15.26:46958) 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe024d94640 1 -- 172.21.15.26:0/382441164 learned_addr learned my addr 172.21.15.26:0/382441164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe025d96640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.138+0000 7fe025595640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe02006c4d0 0x7fe0200c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe024d94640 1 -- 172.21.15.26:0/382441164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 msgr2=0x7fe0200cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe024d94640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe024d94640 1 -- 172.21.15.26:0/382441164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe02006c4d0 msgr2=0x7fe0200c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe024d94640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe02006c4d0 0x7fe0200c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe024d94640 1 -- 172.21.15.26:0/382441164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0200c9e60 con 0x7fe0200ce640 2024-04-24T23:21:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe025595640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe02006c4d0 0x7fe0200c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe024d94640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0200ce640 0x7fe0200b5f70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e900 tx=0x7fe01000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe015ffb640 1 -- 172.21.15.26:0/382441164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010017070 con 0x7fe0200ce640 2024-04-24T23:21:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe015ffb640 1 -- 172.21.15.26:0/382441164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe01000ca00 con 0x7fe0200ce640 2024-04-24T23:21:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0200ca140 con 0x7fe0200ce640 2024-04-24T23:21:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe015ffb640 1 -- 172.21.15.26:0/382441164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01000cdf0 con 0x7fe0200ce640 2024-04-24T23:21:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.139+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0200ca9e0 con 0x7fe0200ce640 2024-04-24T23:21:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.141+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0200ca310 con 0x7fe0200ce640 2024-04-24T23:21:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.141+0000 7fe015ffb640 1 -- 172.21.15.26:0/382441164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe01001a070 con 0x7fe0200ce640 2024-04-24T23:21:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.141+0000 7fe015ffb640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe00406be60 0x7fe00406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.141+0000 7fe015ffb640 1 -- 172.21.15.26:0/382441164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe01008afb0 con 0x7fe0200ce640 2024-04-24T23:21:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.142+0000 7fe025595640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe00406be60 0x7fe00406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.142+0000 7fe025595640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe00406be60 0x7fe00406e320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe01c034000 tx=0x7fe01c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.146+0000 7fe015ffb640 1 -- 172.21.15.26:0/382441164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe010059310 con 0x7fe0200ce640 2024-04-24T23:21:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:51.369+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe020075610 con 0x7fe0200ce640 2024-04-24T23:21:51.949 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:21:51.948+0000 7fe9ecd69640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:21:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.010+0000 7fe015ffb640 1 -- 172.21.15.26:0/382441164 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fe01005d610 con 0x7fe0200ce640 2024-04-24T23:21:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe00406be60 msgr2=0x7fe00406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe00406be60 0x7fe00406e320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe01c034000 tx=0x7fe01c02e000 comp rx=0 tx=0).stop 2024-04-24T23:21:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0200ce640 msgr2=0x7fe0200b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0200ce640 0x7fe0200b5f70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e900 tx=0x7fe01000edc0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 shutdown_connections 2024-04-24T23:21:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe00406be60 0x7fe00406e320 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0200d28e0 0x7fe0200cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0200ce640 0x7fe0200b5f70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 --2- 172.21.15.26:0/382441164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe02006c4d0 0x7fe0200c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 >> 172.21.15.26:0/382441164 conn(0x7fe0200d1dc0 msgr2=0x7fe02006efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 shutdown_connections 2024-04-24T23:21:52.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.013+0000 7fe0277f9640 1 -- 172.21.15.26:0/382441164 wait complete. 2024-04-24T23:21:52.028 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:21:52.028 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.135+0000 7f378a1db640 1 Processor -- start 2024-04-24T23:21:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.135+0000 7f378a1db640 1 -- start start 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.135+0000 7f378a1db640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 0x7f37840d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.135+0000 7f378a1db640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37840d32f0 con 0x7f37840b3130 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.135+0000 7f378a1db640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37840d1600 con 0x7f37840d22e0 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.135+0000 7f378a1db640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f37840d1ab0 con 0x7f378415e570 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.136+0000 7f3782ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f37840b3130 0x7f37840d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56702/0 (socket says 172.21.15.26:56702) 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.136+0000 7f3782ffd640 1 -- 172.21.15.26:0/3924064296 learned_addr learned my addr 172.21.15.26:0/3924064296 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.136+0000 7f37837fe640 1 --2- 172.21.15.26:0/3924064296 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 0x7f37840d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.136+0000 7f37827fc640 1 -- 172.21.15.26:0/3924064296 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386753309 0 0) 0x7f3778001110 con 0x7f37840b3130 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.136+0000 7f37827fc640 1 -- 172.21.15.26:0/3924064296 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f376c003520 con 0x7f37840b3130 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.136+0000 7f37827fc640 1 -- 172.21.15.26:0/3924064296 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764215982 0 0) 0x7f37840d1600 con 0x7f37840d22e0 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.136+0000 7f37837fe640 1 -- 172.21.15.26:0/3924064296 >> v1:172.21.15.29:6789/0 conn(0x7f37840d22e0 legacy=0x7f37840d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.136+0000 7f37837fe640 1 -- 172.21.15.26:0/3924064296 >> v1:172.21.15.26:6789/0 conn(0x7f37840b3130 legacy=0x7f37840d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.136+0000 7f37837fe640 1 -- 172.21.15.26:0/3924064296 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37840d0270 con 0x7f378415e570 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.137+0000 7f37837fe640 1 --2- 172.21.15.26:0/3924064296 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 0x7f37840d4160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f37740097d0 tx=0x7f377402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2f3f0c26ef75fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.137+0000 7f37827fc640 1 -- 172.21.15.26:0/3924064296 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774037070 con 0x7f378415e570 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.137+0000 7f37827fc640 1 -- 172.21.15.26:0/3924064296 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3774032440 con 0x7f378415e570 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.137+0000 7f37827fc640 1 -- 172.21.15.26:0/3924064296 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37740327f0 con 0x7f378415e570 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.138+0000 7f378a1db640 1 -- 172.21.15.26:0/3924064296 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 msgr2=0x7f37840d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.138+0000 7f378a1db640 1 --2- 172.21.15.26:0/3924064296 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 0x7f37840d4160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f37740097d0 tx=0x7f377402a210 comp rx=0 tx=0).stop 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.138+0000 7f378a1db640 1 -- 172.21.15.26:0/3924064296 shutdown_connections 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.138+0000 7f378a1db640 1 --2- 172.21.15.26:0/3924064296 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 0x7f37840d4160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.138+0000 7f378a1db640 1 -- 172.21.15.26:0/3924064296 >> 172.21.15.26:0/3924064296 conn(0x7f378408dbe0 msgr2=0x7f378409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.138+0000 7f378a1db640 1 -- 172.21.15.26:0/3924064296 shutdown_connections 2024-04-24T23:21:52.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.138+0000 7f378a1db640 1 -- 172.21.15.26:0/3924064296 wait complete. 2024-04-24T23:21:52.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.139+0000 7f378a1db640 1 Processor -- start 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.139+0000 7f378a1db640 1 -- start start 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f378a1db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37840b3130 0x7f37841517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f378a1db640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37840d22e0 0x7f37841f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f378a1db640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 0x7f37840b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f378a1db640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37840d3010 con 0x7f37840b3130 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f378a1db640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f37840d2ae0 con 0x7f37840d22e0 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f378a1db640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f37840cfa70 con 0x7f378415e570 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f37837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37840b3130 0x7f37841517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f37837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37840b3130 0x7f37841517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46962/0 (socket says 172.21.15.26:46962) 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f37837fe640 1 -- 172.21.15.26:0/3014785689 learned_addr learned my addr 172.21.15.26:0/3014785689 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f3782ffd640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37840d22e0 0x7f37841f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f3783fff640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 0x7f37840b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f37837fe640 1 -- 172.21.15.26:0/3014785689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 msgr2=0x7f37840b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f37837fe640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 0x7f37840b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f37837fe640 1 -- 172.21.15.26:0/3014785689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37840d22e0 msgr2=0x7f37841f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f37837fe640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37840d22e0 0x7f37841f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f37837fe640 1 -- 172.21.15.26:0/3014785689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37841f54d0 con 0x7f37840b3130 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.140+0000 7f37837fe640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37840b3130 0x7f37841517d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3774000c00 tx=0x7f37740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.141+0000 7f376bfff640 1 -- 172.21.15.26:0/3014785689 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774037040 con 0x7f37840b3130 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.141+0000 7f376bfff640 1 -- 172.21.15.26:0/3014785689 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3774042070 con 0x7f37840b3130 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.141+0000 7f376bfff640 1 -- 172.21.15.26:0/3014785689 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774030900 con 0x7f37840b3130 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.141+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37840caab0 con 0x7f37840b3130 2024-04-24T23:21:52.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.141+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37840b6ee0 con 0x7f37840b3130 2024-04-24T23:21:52.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.142+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37840cf370 con 0x7f37840b3130 2024-04-24T23:21:52.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.142+0000 7f376bfff640 1 -- 172.21.15.26:0/3014785689 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3774030aa0 con 0x7f37840b3130 2024-04-24T23:21:52.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.143+0000 7f376bfff640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f377006be60 0x7f377006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.143+0000 7f376bfff640 1 -- 172.21.15.26:0/3014785689 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f37740ab040 con 0x7f37840b3130 2024-04-24T23:21:52.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.143+0000 7f3782ffd640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f377006be60 0x7f377006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.143+0000 7f3782ffd640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f377006be60 0x7f377006e320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f37780012b0 tx=0x7f3778007b70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.147+0000 7f376bfff640 1 -- 172.21.15.26:0/3014785689 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f37840cf370 con 0x7f37840b3130 2024-04-24T23:21:52.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.369+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37840694c0 con 0x7f37840b3130 2024-04-24T23:21:52.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.370+0000 7f376bfff640 1 -- 172.21.15.26:0/3014785689 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f377404b020 con 0x7f37840b3130 2024-04-24T23:21:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:52.371 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2201738459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2201738459},{"type":"v1","addr":"172.21.15.29:6834","nonce":2201738459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3972561683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3972561683},{"type":"v1","addr":"172.21.15.29:6835","nonce":3972561683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:21:48.324889+0000","modified":"2024-04-24T23:21:51.371127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/315291888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":315291888},{"type":"v1","addr":"172.21.15.26:6836","nonce":315291888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-04-24T23:21:52.372 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 16 2024-04-24T23:21:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f377006be60 msgr2=0x7f377006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f377006be60 0x7f377006e320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f37780012b0 tx=0x7f3778007b70 comp rx=0 tx=0).stop 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37840b3130 msgr2=0x7f37841517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37840b3130 0x7f37841517d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3774000c00 tx=0x7f37740046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 shutdown_connections 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f377006be60 0x7f377006e320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f378415e570 0x7f37840b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37840d22e0 0x7f37841f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 --2- 172.21.15.26:0/3014785689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37840b3130 0x7f37841517d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 >> 172.21.15.26:0/3014785689 conn(0x7f378408dbe0 msgr2=0x7f37840b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 shutdown_connections 2024-04-24T23:21:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.372+0000 7f378a1db640 1 -- 172.21.15.26:0/3014785689 wait complete. 2024-04-24T23:21:52.384 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:21:52.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.484+0000 7f213d37e640 1 Processor -- start 2024-04-24T23:21:52.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.484+0000 7f213d37e640 1 -- start start 2024-04-24T23:21:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.484+0000 7f213d37e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f21380b4530 0x7f21380b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.484+0000 7f213d37e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21380b0030 0x7f21380b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.484+0000 7f213d37e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21380b0980 con 0x7f213815e380 2024-04-24T23:21:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.484+0000 7f213d37e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f21380b49a0 con 0x7f21380b0030 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.484+0000 7f213d37e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f213815e760 con 0x7f21380b4530 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.485+0000 7f21377fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21380b0030 0x7f21380b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.485+0000 7f21377fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21380b0030 0x7f21380b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46968/0 (socket says 172.21.15.26:46968) 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.485+0000 7f21377fe640 1 -- 172.21.15.26:0/876792113 learned_addr learned my addr 172.21.15.26:0/876792113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.485+0000 7f212ffff640 1 --2- 172.21.15.26:0/876792113 >> v2:172.21.15.29:3300/0 conn(0x7f21380b4530 0x7f21380b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.485+0000 7f21377fe640 1 -- 172.21.15.26:0/876792113 >> v2:172.21.15.29:3300/0 conn(0x7f21380b4530 msgr2=0x7f21380b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.485+0000 7f21377fe640 1 --2- 172.21.15.26:0/876792113 >> v2:172.21.15.29:3300/0 conn(0x7f21380b4530 0x7f21380b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.485+0000 7f21377fe640 1 -- 172.21.15.26:0/876792113 >> v1:172.21.15.29:6789/0 conn(0x7f213815e380 legacy=0x7f2138072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.485+0000 7f21377fe640 1 -- 172.21.15.26:0/876792113 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21380b5700 con 0x7f21380b0030 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f21377fe640 1 --2- 172.21.15.26:0/876792113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b0030 0x7f21380b0440 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f21380b4d30 tx=0x7f213002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b0edb9a4cc53a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f21367fc640 1 -- 172.21.15.26:0/876792113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130036070 con 0x7f21380b0030 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f21367fc640 1 -- 172.21.15.26:0/876792113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2130031600 con 0x7f21380b0030 2024-04-24T23:21:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f21367fc640 1 -- 172.21.15.26:0/876792113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21300319f0 con 0x7f21380b0030 2024-04-24T23:21:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f213d37e640 1 -- 172.21.15.26:0/876792113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b0030 msgr2=0x7f21380b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f213d37e640 1 --2- 172.21.15.26:0/876792113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b0030 0x7f21380b0440 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f21380b4d30 tx=0x7f213002a7c0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f213d37e640 1 -- 172.21.15.26:0/876792113 shutdown_connections 2024-04-24T23:21:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f213d37e640 1 --2- 172.21.15.26:0/876792113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b0030 0x7f21380b0440 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f213d37e640 1 --2- 172.21.15.26:0/876792113 >> v2:172.21.15.29:3300/0 conn(0x7f21380b4530 0x7f21380b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.486+0000 7f213d37e640 1 -- 172.21.15.26:0/876792113 >> 172.21.15.26:0/876792113 conn(0x7f213806e2c0 msgr2=0x7f213806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.487+0000 7f213d37e640 1 -- 172.21.15.26:0/876792113 shutdown_connections 2024-04-24T23:21:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.487+0000 7f213d37e640 1 -- 172.21.15.26:0/876792113 wait complete. 2024-04-24T23:21:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.488+0000 7f213d37e640 1 Processor -- start 2024-04-24T23:21:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.488+0000 7f213d37e640 1 -- start start 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.488+0000 7f213d37e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21380b0030 0x7f21380b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.488+0000 7f213d37e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b4530 0x7f21380b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.488+0000 7f213d37e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f213815e380 0x7f21380b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.488+0000 7f213d37e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2138072240 con 0x7f21380b4530 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.488+0000 7f213d37e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f21380725b0 con 0x7f21380b0030 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.488+0000 7f213d37e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f21380ae530 con 0x7f213815e380 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b4530 0x7f21380b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b4530 0x7f21380b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46984/0 (socket says 172.21.15.26:46984) 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212ffff640 1 -- 172.21.15.26:0/3286118870 learned_addr learned my addr 172.21.15.26:0/3286118870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f2136ffd640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21380b0030 0x7f21380b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f21377fe640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f213815e380 0x7f21380b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212ffff640 1 -- 172.21.15.26:0/3286118870 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f213815e380 msgr2=0x7f21380b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212ffff640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f213815e380 0x7f21380b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212ffff640 1 -- 172.21.15.26:0/3286118870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21380b0030 msgr2=0x7f21380b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212ffff640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21380b0030 0x7f21380b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212ffff640 1 -- 172.21.15.26:0/3286118870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21380b3930 con 0x7f21380b4530 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212ffff640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b4530 0x7f21380b14f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f212000d830 tx=0x7f212000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212f7fe640 1 -- 172.21.15.26:0/3286118870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212000b6c0 con 0x7f21380b4530 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212f7fe640 1 -- 172.21.15.26:0/3286118870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2120019070 con 0x7f21380b4530 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21380b2270 con 0x7f21380b4530 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f212f7fe640 1 -- 172.21.15.26:0/3286118870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212000bac0 con 0x7f21380b4530 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.489+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21380b2830 con 0x7f21380b4530 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.490+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21380694c0 con 0x7f21380b4530 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.491+0000 7f212f7fe640 1 -- 172.21.15.26:0/3286118870 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f212001e460 con 0x7f21380b4530 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.491+0000 7f212f7fe640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211806be30 0x7f211806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.491+0000 7f212f7fe640 1 -- 172.21.15.26:0/3286118870 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f212008b3b0 con 0x7f21380b4530 2024-04-24T23:21:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.491+0000 7f2136ffd640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211806be30 0x7f211806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.492+0000 7f2136ffd640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211806be30 0x7f211806e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f21280016e0 tx=0x7f2128009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.496+0000 7f212f7fe640 1 -- 172.21.15.26:0/3286118870 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f21200597a0 con 0x7f21380b4530 2024-04-24T23:21:52.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.678+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f213815cd20 con 0x7f21380b4530 2024-04-24T23:21:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.680+0000 7f212f7fe640 1 -- 172.21.15.26:0/3286118870 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7f212005daa0 con 0x7f21380b4530 2024-04-24T23:21:52.682 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:52.682 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:49.992052+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:21:45.983358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:21:47.321239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:21:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.681+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211806be30 msgr2=0x7f211806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.681+0000 7f213d37e640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211806be30 0x7f211806e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f21280016e0 tx=0x7f2128009040 comp rx=0 tx=0).stop 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.681+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b4530 msgr2=0x7f21380b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.681+0000 7f213d37e640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b4530 0x7f21380b14f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f212000d830 tx=0x7f212000dcf0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.681+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 shutdown_connections 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.681+0000 7f213d37e640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211806be30 0x7f211806e2f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.681+0000 7f213d37e640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f213815e380 0x7f21380b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.681+0000 7f213d37e640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21380b4530 0x7f21380b14f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.681+0000 7f213d37e640 1 --2- 172.21.15.26:0/3286118870 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21380b0030 0x7f21380b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.682+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 >> 172.21.15.26:0/3286118870 conn(0x7f213806e2c0 msgr2=0x7f213815e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.682+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 shutdown_connections 2024-04-24T23:21:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.682+0000 7f213d37e640 1 -- 172.21.15.26:0/3286118870 wait complete. 2024-04-24T23:21:52.694 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:21:52.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:21:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.794+0000 7f63e4ef5640 1 Processor -- start 2024-04-24T23:21:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.794+0000 7f63e4ef5640 1 -- start start 2024-04-24T23:21:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63e4ef5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63e00cf870 0x7f63e0072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63e4ef5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 0x7f63e015ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63e4ef5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e00cd0b0 con 0x7f63e0143c40 2024-04-24T23:21:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63e4ef5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f63e0071b70 con 0x7f63e00cf870 2024-04-24T23:21:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63e4ef5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f63e015f470 con 0x7f63e015eb00 2024-04-24T23:21:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63ddd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63e00cf870 0x7f63e0072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63ddd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63e00cf870 0x7f63e0072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46996/0 (socket says 172.21.15.26:46996) 2024-04-24T23:21:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63ddd9b640 1 -- 172.21.15.26:0/3903581752 learned_addr learned my addr 172.21.15.26:0/3903581752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63ded9d640 1 --2- 172.21.15.26:0/3903581752 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 0x7f63e015ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63ddd9b640 1 -- 172.21.15.26:0/3903581752 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 msgr2=0x7f63e015ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63ddd9b640 1 --2- 172.21.15.26:0/3903581752 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 0x7f63e015ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.795+0000 7f63ddd9b640 1 -- 172.21.15.26:0/3903581752 >> v1:172.21.15.29:6789/0 conn(0x7f63e0143c40 legacy=0x7f63e00cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.796+0000 7f63ddd9b640 1 -- 172.21.15.26:0/3903581752 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e0071240 con 0x7f63e00cf870 2024-04-24T23:21:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.796+0000 7f63ddd9b640 1 --2- 172.21.15.26:0/3903581752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e00cf870 0x7f63e0072000 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009870 tx=0x7f63d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce682902f74d04a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.796+0000 7f63dd59a640 1 -- 172.21.15.26:0/3903581752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4037070 con 0x7f63e00cf870 2024-04-24T23:21:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.796+0000 7f63dd59a640 1 -- 172.21.15.26:0/3903581752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d4032440 con 0x7f63e00cf870 2024-04-24T23:21:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.796+0000 7f63dd59a640 1 -- 172.21.15.26:0/3903581752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4032810 con 0x7f63e00cf870 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.797+0000 7f63e4ef5640 1 -- 172.21.15.26:0/3903581752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e00cf870 msgr2=0x7f63e0072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.797+0000 7f63e4ef5640 1 --2- 172.21.15.26:0/3903581752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e00cf870 0x7f63e0072000 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009870 tx=0x7f63d402a270 comp rx=0 tx=0).stop 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.797+0000 7f63e4ef5640 1 -- 172.21.15.26:0/3903581752 shutdown_connections 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.797+0000 7f63e4ef5640 1 --2- 172.21.15.26:0/3903581752 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 0x7f63e015ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.797+0000 7f63e4ef5640 1 --2- 172.21.15.26:0/3903581752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e00cf870 0x7f63e0072000 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.797+0000 7f63e4ef5640 1 -- 172.21.15.26:0/3903581752 >> 172.21.15.26:0/3903581752 conn(0x7f63e006b300 msgr2=0x7f63e006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.797+0000 7f63e4ef5640 1 -- 172.21.15.26:0/3903581752 shutdown_connections 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.797+0000 7f63e4ef5640 1 -- 172.21.15.26:0/3903581752 wait complete. 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.798+0000 7f63e4ef5640 1 Processor -- start 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.798+0000 7f63e4ef5640 1 -- start start 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63e4ef5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63e00cf870 0x7f63e01f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63e4ef5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e0143c40 0x7f63e01415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63e4ef5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 0x7f63e015d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63e4ef5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0144020 con 0x7f63e0143c40 2024-04-24T23:21:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63e4ef5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0071800 con 0x7f63e00cf870 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63e4ef5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f63e0072540 con 0x7f63e015eb00 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63ddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e0143c40 0x7f63e01415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63ddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e0143c40 0x7f63e01415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47008/0 (socket says 172.21.15.26:47008) 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63ddd9b640 1 -- 172.21.15.26:0/636137938 learned_addr learned my addr 172.21.15.26:0/636137938 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63de59c640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63e00cf870 0x7f63e01f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63ded9d640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 0x7f63e015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63ddd9b640 1 -- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 msgr2=0x7f63e015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63ddd9b640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 0x7f63e015d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63ddd9b640 1 -- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63e00cf870 msgr2=0x7f63e01f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.799+0000 7f63ddd9b640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63e00cf870 0x7f63e01f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.800+0000 7f63ddd9b640 1 -- 172.21.15.26:0/636137938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e013b8b0 con 0x7f63e0143c40 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.800+0000 7f63de59c640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63e00cf870 0x7f63e01f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.800+0000 7f63ded9d640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 0x7f63e015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.800+0000 7f63ddd9b640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e0143c40 0x7f63e01415a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f63d4000c00 tx=0x7f63d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.800+0000 7f63ceffd640 1 -- 172.21.15.26:0/636137938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4041020 con 0x7f63e0143c40 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.800+0000 7f63ceffd640 1 -- 172.21.15.26:0/636137938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d40306c0 con 0x7f63e0143c40 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.800+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e0140500 con 0x7f63e0143c40 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.800+0000 7f63ceffd640 1 -- 172.21.15.26:0/636137938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4037040 con 0x7f63e0143c40 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.800+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e0140ac0 con 0x7f63e0143c40 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.801+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e00694c0 con 0x7f63e0143c40 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.801+0000 7f63ceffd640 1 -- 172.21.15.26:0/636137938 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63d4044020 con 0x7f63e0143c40 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.802+0000 7f63ceffd640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f63c006be30 0x7f63c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.802+0000 7f63ceffd640 1 -- 172.21.15.26:0/636137938 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f63d40ab6a0 con 0x7f63e0143c40 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.802+0000 7f63de59c640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f63c006be30 0x7f63c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.803+0000 7f63de59c640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f63c006be30 0x7f63c006e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000f60 tx=0x7f63d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.806+0000 7f63ceffd640 1 -- 172.21.15.26:0/636137938 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f63d404b030 con 0x7f63e0143c40 2024-04-24T23:21:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.985+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63e00b4d20 con 0x7f63c006be30 2024-04-24T23:21:52.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.991+0000 7f63ceffd640 1 -- 172.21.15.26:0/636137938 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239790 (secure 0 0 0) 0x7f63e00b4d20 con 0x7f63c006be30 2024-04-24T23:21:52.997 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:52.998 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:21:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f63c006be30 msgr2=0x7f63c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f63c006be30 0x7f63c006e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000f60 tx=0x7f63d8009040 comp rx=0 tx=0).stop 2024-04-24T23:21:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e0143c40 msgr2=0x7f63e01415a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e0143c40 0x7f63e01415a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f63d4000c00 tx=0x7f63d40046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 shutdown_connections 2024-04-24T23:21:53.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f63c006be30 0x7f63c006e2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f63e015eb00 0x7f63e015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63e0143c40 0x7f63e01415a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 --2- 172.21.15.26:0/636137938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63e00cf870 0x7f63e01f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 >> 172.21.15.26:0/636137938 conn(0x7f63e006b300 msgr2=0x7f63e00b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 shutdown_connections 2024-04-24T23:21:53.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:52.994+0000 7f63e4ef5640 1 -- 172.21.15.26:0/636137938 wait complete. 2024-04-24T23:21:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-04-24T23:21:51.325830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257352,"kb_used_data":3700,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754717368,"statfs":{"total":773094113280,"available":772830584832,"internally_reserved":0,"allocated":3788800,"data_stored":2158145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.371857"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.961795+0000","last_change":"2024-04-24T23:21:46.961795+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.961795+0000","last_undegraded":"2024-04-24T23:21:46.961795+0000","last_fullsized":"2024-04-24T23:21:46.961795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:47.072151+0000","last_change":"2024-04-24T23:21:47.072151+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.072151+0000","last_undegraded":"2024-04-24T23:21:47.072151+0000","last_fullsized":"2024-04-24T23:21:47.072151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.961793+0000","last_change":"2024-04-24T23:21:46.961793+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.961793+0000","last_undegraded":"2024-04-24T23:21:46.961793+0000","last_fullsized":"2024-04-24T23:21:46.961793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.956929+0000","last_change":"2024-04-24T23:21:46.956929+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.956929+0000","last_undegraded":"2024-04-24T23:21:46.956929+0000","last_fullsized":"2024-04-24T23:21:46.956929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.964672+0000","last_change":"2024-04-24T23:21:46.964672+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.964672+0000","last_undegraded":"2024-04-24T23:21:46.964672+0000","last_fullsized":"2024-04-24T23:21:46.964672+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.963923+0000","last_change":"2024-04-24T23:21:46.963923+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.963923+0000","last_undegraded":"2024-04-24T23:21:46.963923+0000","last_fullsized":"2024-04-24T23:21:46.963923+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.961738+0000","last_change":"2024-04-24T23:21:46.961738+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.961738+0000","last_undegraded":"2024-04-24T23:21:46.961738+0000","last_fullsized":"2024-04-24T23:21:46.961738+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.963616+0000","last_change":"2024-04-24T23:21:46.963616+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.963616+0000","last_undegraded":"2024-04-24T23:21:46.963616+0000","last_fullsized":"2024-04-24T23:21:46.963616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.961773+0000","last_change":"2024-04-24T23:21:46.961773+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.961773+0000","last_undegraded":"2024-04-24T23:21:46.961773+0000","last_fullsized":"2024-04-24T23:21:46.961773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:47.071384+0000","last_change":"2024-04-24T23:21:47.071384+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.071384+0000","last_undegraded":"2024-04-24T23:21:47.071384+0000","last_fullsized":"2024-04-24T23:21:47.071384+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.956233+0000","last_change":"2024-04-24T23:21:46.956233+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.956233+0000","last_undegraded":"2024-04-24T23:21:46.956233+0000","last_fullsized":"2024-04-24T23:21:46.956233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.961878+0000","last_change":"2024-04-24T23:21:46.961878+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.961878+0000","last_undegraded":"2024-04-24T23:21:46.961878+0000","last_fullsized":"2024-04-24T23:21:46.961878+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.963956+0000","last_change":"2024-04-24T23:21:46.963956+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.963956+0000","last_undegraded":"2024-04-24T23:21:46.963956+0000","last_fullsized":"2024-04-24T23:21:46.963956+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.959736+0000","last_change":"2024-04-24T23:21:46.959736+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.959736+0000","last_undegraded":"2024-04-24T23:21:46.959736+0000","last_fullsized":"2024-04-24T23:21:46.959736+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.962291+0000","last_change":"2024-04-24T23:21:46.962291+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.962291+0000","last_undegraded":"2024-04-24T23:21:46.962291+0000","last_fullsized":"2024-04-24T23:21:46.962291+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.961720+0000","last_change":"2024-04-24T23:21:46.961720+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.961720+0000","last_undegraded":"2024-04-24T23:21:46.961720+0000","last_fullsized":"2024-04-24T23:21:46.961720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.959702+0000","last_change":"2024-04-24T23:21:46.959702+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.959702+0000","last_undegraded":"2024-04-24T23:21:46.959702+0000","last_fullsized":"2024-04-24T23:21:46.959702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.963797+0000","last_change":"2024-04-24T23:21:46.963797+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.963797+0000","last_undegraded":"2024-04-24T23:21:46.963797+0000","last_fullsized":"2024-04-24T23:21:46.963797+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.963791+0000","last_change":"2024-04-24T23:21:46.963791+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.963791+0000","last_undegraded":"2024-04-24T23:21:46.963791+0000","last_fullsized":"2024-04-24T23:21:46.963791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.964037+0000","last_change":"2024-04-24T23:21:46.964037+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.964037+0000","last_undegraded":"2024-04-24T23:21:46.964037+0000","last_fullsized":"2024-04-24T23:21:46.964037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:47.071922+0000","last_change":"2024-04-24T23:21:47.071922+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.071922+0000","last_undegraded":"2024-04-24T23:21:47.071922+0000","last_fullsized":"2024-04-24T23:21:47.071922+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.961768+0000","last_change":"2024-04-24T23:21:46.961768+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.961768+0000","last_undegraded":"2024-04-24T23:21:46.961768+0000","last_fullsized":"2024-04-24T23:21:46.961768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.964616+0000","last_change":"2024-04-24T23:21:46.964616+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.964616+0000","last_undegraded":"2024-04-24T23:21:46.964616+0000","last_fullsized":"2024-04-24T23:21:46.964616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.956250+0000","last_change":"2024-04-24T23:21:46.956250+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.956250+0000","last_undegraded":"2024-04-24T23:21:46.956250+0000","last_fullsized":"2024-04-24T23:21:46.956250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.959089+0000","last_change":"2024-04-24T23:21:46.959089+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.959089+0000","last_undegraded":"2024-04-24T23:21:46.959089+0000","last_fullsized":"2024-04-24T23:21:46.959089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:47.071839+0000","last_change":"2024-04-24T23:21:47.071839+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.071839+0000","last_undegraded":"2024-04-24T23:21:47.071839+0000","last_fullsized":"2024-04-24T23:21:47.071839+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.959139+0000","last_change":"2024-04-24T23:21:46.959139+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.959139+0000","last_undegraded":"2024-04-24T23:21:46.959139+0000","last_fullsized":"2024-04-24T23:21:46.959139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:47.071466+0000","last_change":"2024-04-24T23:21:47.071466+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.071466+0000","last_undegraded":"2024-04-24T23:21:47.071466+0000","last_fullsized":"2024-04-24T23:21:47.071466+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.959116+0000","last_change":"2024-04-24T23:21:46.959116+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.959116+0000","last_undegraded":"2024-04-24T23:21:46.959116+0000","last_fullsized":"2024-04-24T23:21:46.959116+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.963612+0000","last_change":"2024-04-24T23:21:46.963612+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.963612+0000","last_undegraded":"2024-04-24T23:21:46.963612+0000","last_fullsized":"2024-04-24T23:21:46.963612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.962314+0000","last_change":"2024-04-24T23:21:46.962314+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.962314+0000","last_undegraded":"2024-04-24T23:21:46.962314+0000","last_fullsized":"2024-04-24T23:21:46.962314+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:21:47.953412+0000","last_change":"2024-04-24T23:21:47.953412+0000","last_active":"2024-04-24T23:21:47.953412+0000","last_peered":"2024-04-24T23:21:47.953412+0000","last_clean":"2024-04-24T23:21:47.953412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:47.953412+0000","last_undegraded":"2024-04-24T23:21:47.953412+0000","last_fullsized":"2024-04-24T23:21:47.953412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-24T23:21:46.961656+0000","last_change":"2024-04-24T23:21:46.961656+0000","last_active":"2024-04-24T23:21:46.951440+0000","last_peered":"2024-04-24T23:21:46.951440+0000","last_clean":"2024-04-24T23:21:46.951440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:21:46.961656+0000","last_undegraded":"2024-04-24T23:21:46.961656+0000","last_fullsized":"2024-04-24T23:21:46.961656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-24T23:21:46.958304+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:21:46.958304+0000","last_peered":"2024-04-24T23:21:46.958304+0000","last_clean":"2024-04-24T23:21:46.958304+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:21:46.958304+0000","last_undegraded":"2024-04-24T23:21:46.958304+0000","last_fullsized":"2024-04-24T23:21:46.958304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":711817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31504,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340336,"statfs":{"total":96636764160,"available":96604504064,"internally_reserved":0,"allocated":323584,"data_stored":121449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":323584,"data_stored":121449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31504,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340336,"statfs":{"total":96636764160,"available":96604504064,"internally_reserved":0,"allocated":323584,"data_stored":121449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35660,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336180,"statfs":{"total":96636764160,"available":96600248320,"internally_reserved":0,"allocated":323584,"data_stored":121449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":323584,"data_stored":121449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":711817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:21:53.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:21:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.148+0000 7f56af67d640 1 Processor -- start 2024-04-24T23:21:53.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.148+0000 7f56af67d640 1 -- start start 2024-04-24T23:21:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f56af67d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56a80b0740 0x7f56a80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f56af67d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a80b5700 con 0x7f56a80b4530 2024-04-24T23:21:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f56af67d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a80b59e0 con 0x7f56a815e380 2024-04-24T23:21:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f56af67d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f56a80b49a0 con 0x7f56a80b0740 2024-04-24T23:21:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f56acc18640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f56a80b4530 0x7f56a80b0030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56712/0 (socket says 172.21.15.26:56712) 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f56acc18640 1 -- 172.21.15.26:0/3453636105 learned_addr learned my addr 172.21.15.26:0/3453636105 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1195460798 0 0) 0x7f56a80b5700 con 0x7f56a80b4530 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5694003520 con 0x7f56a80b4530 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f56adc1a640 1 --2- 172.21.15.26:0/3453636105 >> v2:172.21.15.29:3300/0 conn(0x7f56a80b0740 0x7f56a80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1934911884 0 0) 0x7f5694003520 con 0x7f56a80b4530 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 >> v2:172.21.15.29:3300/0 conn(0x7f56a80b0740 msgr2=0x7f56a80b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 --2- 172.21.15.26:0/3453636105 >> v2:172.21.15.29:3300/0 conn(0x7f56a80b0740 0x7f56a80b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 >> v1:172.21.15.29:6789/0 conn(0x7f56a815e380 legacy=0x7f56a8072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a8072240 con 0x7f56a80b4530 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f5698001670 con 0x7f56a80b4530 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 >> v1:172.21.15.26:6789/0 conn(0x7f56a80b4530 legacy=0x7f56a80b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f56940039b0 0x7f5694003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5694004800 0x7f5694004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.149+0000 7f569ffff640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56940051c0 0x7f569400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f569400c0c0 con 0x7f56940051c0 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f569400c270 con 0x7f5694004800 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f569400c470 con 0x7f56940039b0 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f56adc1a640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56940051c0 0x7f569400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f56ad419640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f56940039b0 0x7f5694003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f56acc18640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5694004800 0x7f5694004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f56adc1a640 1 -- 172.21.15.26:0/3453636105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f56940039b0 msgr2=0x7f5694003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f56adc1a640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f56940039b0 0x7f5694003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f56adc1a640 1 -- 172.21.15.26:0/3453636105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5694004800 msgr2=0x7f5694004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f56adc1a640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5694004800 0x7f5694004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.150+0000 7f56adc1a640 1 -- 172.21.15.26:0/3453636105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5694003520 con 0x7f56940051c0 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f56adc1a640 1 -- 172.21.15.26:0/3453636105 reap_dead start 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f56adc1a640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56940051c0 0x7f569400b9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f56a0007360 tx=0x7f5694005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fcfc89185ccf90 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0009be0 con 0x7f56940051c0 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a0003bd0 con 0x7f56940051c0 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f569ffff640 1 -- 172.21.15.26:0/3453636105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0004020 con 0x7f56940051c0 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f56af67d640 1 -- 172.21.15.26:0/3453636105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56940051c0 msgr2=0x7f569400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f56af67d640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56940051c0 0x7f569400b9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f56a0007360 tx=0x7f5694005830 comp rx=0 tx=0).stop 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f56af67d640 1 -- 172.21.15.26:0/3453636105 shutdown_connections 2024-04-24T23:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f56af67d640 1 --2- 172.21.15.26:0/3453636105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56940051c0 0x7f569400b9c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f56af67d640 1 -- 172.21.15.26:0/3453636105 >> 172.21.15.26:0/3453636105 conn(0x7f56a806e2c0 msgr2=0x7f56a806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f56af67d640 1 -- 172.21.15.26:0/3453636105 shutdown_connections 2024-04-24T23:21:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.151+0000 7f56af67d640 1 -- 172.21.15.26:0/3453636105 wait complete. 2024-04-24T23:21:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.152+0000 7f56af67d640 1 Processor -- start 2024-04-24T23:21:53.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56af67d640 1 -- start start 2024-04-24T23:21:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56af67d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56940051c0 0x7f56a80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56af67d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f56a814a6f0 0x7f56a80ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56af67d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a814ab70 0x7f56a80aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56af67d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80b5360 con 0x7f56a814ab70 2024-04-24T23:21:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56af67d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80b51e0 con 0x7f56940051c0 2024-04-24T23:21:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56af67d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f56a80b54e0 con 0x7f56a814a6f0 2024-04-24T23:21:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56adc1a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a814ab70 0x7f56a80aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56adc1a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a814ab70 0x7f56a80aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47030/0 (socket says 172.21.15.26:47030) 2024-04-24T23:21:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56adc1a640 1 -- 172.21.15.26:0/3283169158 learned_addr learned my addr 172.21.15.26:0/3283169158 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56acc18640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f56a814a6f0 0x7f56a80ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.153+0000 7f56ad419640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56940051c0 0x7f56a80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f56adc1a640 1 -- 172.21.15.26:0/3283169158 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f56a814a6f0 msgr2=0x7f56a80ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f56adc1a640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f56a814a6f0 0x7f56a80ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f56adc1a640 1 -- 172.21.15.26:0/3283169158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56940051c0 msgr2=0x7f56a80b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f56adc1a640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56940051c0 0x7f56a80b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f56adc1a640 1 -- 172.21.15.26:0/3283169158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a81f46f0 con 0x7f56a814ab70 2024-04-24T23:21:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f56adc1a640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a814ab70 0x7f56a80aec90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f56a0007450 tx=0x7f56a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f569dffb640 1 -- 172.21.15.26:0/3283169158 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0045070 con 0x7f56a814ab70 2024-04-24T23:21:53.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f569dffb640 1 -- 172.21.15.26:0/3283169158 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a0036650 con 0x7f56a814ab70 2024-04-24T23:21:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f569dffb640 1 -- 172.21.15.26:0/3283169158 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0009be0 con 0x7f56a814ab70 2024-04-24T23:21:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a81f49d0 con 0x7f56a814ab70 2024-04-24T23:21:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.154+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a81f4f90 con 0x7f56a814ab70 2024-04-24T23:21:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.155+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a80694c0 con 0x7f56a814ab70 2024-04-24T23:21:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.156+0000 7f569dffb640 1 -- 172.21.15.26:0/3283169158 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56a0048070 con 0x7f56a814ab70 2024-04-24T23:21:53.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.156+0000 7f569dffb640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f568c06be30 0x7f568c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.156+0000 7f569dffb640 1 -- 172.21.15.26:0/3283169158 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f56a00ab6c0 con 0x7f56a814ab70 2024-04-24T23:21:53.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.156+0000 7f56ad419640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f568c06be30 0x7f568c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.157+0000 7f56ad419640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f568c06be30 0x7f568c06e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f56a4000c00 tx=0x7f56a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.160+0000 7f569dffb640 1 -- 172.21.15.26:0/3283169158 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f56a003d070 con 0x7f56a814ab70 2024-04-24T23:21:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.339+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56a80afdb0 con 0x7f568c06be30 2024-04-24T23:21:53.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.345+0000 7f569dffb640 1 -- 172.21.15.26:0/3283169158 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f56a80afdb0 con 0x7f568c06be30 2024-04-24T23:21:53.349 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:53.350 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:21:53.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f568c06be30 msgr2=0x7f568c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f568c06be30 0x7f568c06e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f56a4000c00 tx=0x7f56a4009040 comp rx=0 tx=0).stop 2024-04-24T23:21:53.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a814ab70 msgr2=0x7f56a80aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a814ab70 0x7f56a80aec90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f56a0007450 tx=0x7f56a0004ab0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 shutdown_connections 2024-04-24T23:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f568c06be30 0x7f568c06e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a814ab70 0x7f56a80aec90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f56a814a6f0 0x7f56a80ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 --2- 172.21.15.26:0/3283169158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f56940051c0 0x7f56a80b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 >> 172.21.15.26:0/3283169158 conn(0x7f56a806e2c0 msgr2=0x7f56a815e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 shutdown_connections 2024-04-24T23:21:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.348+0000 7f56af67d640 1 -- 172.21.15.26:0/3283169158 wait complete. 2024-04-24T23:21:53.360 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-04-24T23:21:53.326101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257736,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716984,"statfs":{"total":773094113280,"available":772830191616,"internally_reserved":0,"allocated":4235264,"data_stored":2358420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000814"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407506+0000","last_change":"2024-04-24T23:21:49.009207+0000","last_active":"2024-04-24T23:21:50.407506+0000","last_peered":"2024-04-24T23:21:50.407506+0000","last_clean":"2024-04-24T23:21:50.407506+0000","last_became_active":"2024-04-24T23:21:49.008837+0000","last_became_peered":"2024-04-24T23:21:49.008837+0000","last_unstale":"2024-04-24T23:21:50.407506+0000","last_undegraded":"2024-04-24T23:21:50.407506+0000","last_fullsized":"2024-04-24T23:21:50.407506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:45.017049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999577+0000","last_change":"2024-04-24T23:21:49.015914+0000","last_active":"2024-04-24T23:21:49.999577+0000","last_peered":"2024-04-24T23:21:49.999577+0000","last_clean":"2024-04-24T23:21:49.999577+0000","last_became_active":"2024-04-24T23:21:49.015574+0000","last_became_peered":"2024-04-24T23:21:49.015574+0000","last_unstale":"2024-04-24T23:21:49.999577+0000","last_undegraded":"2024-04-24T23:21:49.999577+0000","last_fullsized":"2024-04-24T23:21:49.999577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:56.571614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000817+0000","last_change":"2024-04-24T23:21:49.003674+0000","last_active":"2024-04-24T23:21:50.000817+0000","last_peered":"2024-04-24T23:21:50.000817+0000","last_clean":"2024-04-24T23:21:50.000817+0000","last_became_active":"2024-04-24T23:21:49.003064+0000","last_became_peered":"2024-04-24T23:21:49.003064+0000","last_unstale":"2024-04-24T23:21:50.000817+0000","last_undegraded":"2024-04-24T23:21:50.000817+0000","last_fullsized":"2024-04-24T23:21:50.000817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:58:17.173142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012596+0000","last_change":"2024-04-24T23:21:49.011881+0000","last_active":"2024-04-24T23:21:50.012596+0000","last_peered":"2024-04-24T23:21:50.012596+0000","last_clean":"2024-04-24T23:21:50.012596+0000","last_became_active":"2024-04-24T23:21:49.011388+0000","last_became_peered":"2024-04-24T23:21:49.011388+0000","last_unstale":"2024-04-24T23:21:50.012596+0000","last_undegraded":"2024-04-24T23:21:50.012596+0000","last_fullsized":"2024-04-24T23:21:50.012596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:13.783082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.406543+0000","last_change":"2024-04-24T23:21:49.017656+0000","last_active":"2024-04-24T23:21:50.406543+0000","last_peered":"2024-04-24T23:21:50.406543+0000","last_clean":"2024-04-24T23:21:50.406543+0000","last_became_active":"2024-04-24T23:21:49.014515+0000","last_became_peered":"2024-04-24T23:21:49.014515+0000","last_unstale":"2024-04-24T23:21:50.406543+0000","last_undegraded":"2024-04-24T23:21:50.406543+0000","last_fullsized":"2024-04-24T23:21:50.406543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:20.501227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999193+0000","last_change":"2024-04-24T23:21:49.025126+0000","last_active":"2024-04-24T23:21:49.999193+0000","last_peered":"2024-04-24T23:21:49.999193+0000","last_clean":"2024-04-24T23:21:49.999193+0000","last_became_active":"2024-04-24T23:21:49.024662+0000","last_became_peered":"2024-04-24T23:21:49.024662+0000","last_unstale":"2024-04-24T23:21:49.999193+0000","last_undegraded":"2024-04-24T23:21:49.999193+0000","last_fullsized":"2024-04-24T23:21:49.999193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:53.562702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408620+0000","last_change":"2024-04-24T23:21:49.019831+0000","last_active":"2024-04-24T23:21:50.408620+0000","last_peered":"2024-04-24T23:21:50.408620+0000","last_clean":"2024-04-24T23:21:50.408620+0000","last_became_active":"2024-04-24T23:21:49.019443+0000","last_became_peered":"2024-04-24T23:21:49.019443+0000","last_unstale":"2024-04-24T23:21:50.408620+0000","last_undegraded":"2024-04-24T23:21:50.408620+0000","last_fullsized":"2024-04-24T23:21:50.408620+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:08.687119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407231+0000","last_change":"2024-04-24T23:21:49.015690+0000","last_active":"2024-04-24T23:21:50.407231+0000","last_peered":"2024-04-24T23:21:50.407231+0000","last_clean":"2024-04-24T23:21:50.407231+0000","last_became_active":"2024-04-24T23:21:49.015393+0000","last_became_peered":"2024-04-24T23:21:49.015393+0000","last_unstale":"2024-04-24T23:21:50.407231+0000","last_undegraded":"2024-04-24T23:21:50.407231+0000","last_fullsized":"2024-04-24T23:21:50.407231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:16.207856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408575+0000","last_change":"2024-04-24T23:21:49.012197+0000","last_active":"2024-04-24T23:21:50.408575+0000","last_peered":"2024-04-24T23:21:50.408575+0000","last_clean":"2024-04-24T23:21:50.408575+0000","last_became_active":"2024-04-24T23:21:49.011868+0000","last_became_peered":"2024-04-24T23:21:49.011868+0000","last_unstale":"2024-04-24T23:21:50.408575+0000","last_undegraded":"2024-04-24T23:21:50.408575+0000","last_fullsized":"2024-04-24T23:21:50.408575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:38.250502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012729+0000","last_change":"2024-04-24T23:21:49.016236+0000","last_active":"2024-04-24T23:21:50.012729+0000","last_peered":"2024-04-24T23:21:50.012729+0000","last_clean":"2024-04-24T23:21:50.012729+0000","last_became_active":"2024-04-24T23:21:49.015968+0000","last_became_peered":"2024-04-24T23:21:49.015968+0000","last_unstale":"2024-04-24T23:21:50.012729+0000","last_undegraded":"2024-04-24T23:21:50.012729+0000","last_fullsized":"2024-04-24T23:21:50.012729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:37.395888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408755+0000","last_change":"2024-04-24T23:21:49.009285+0000","last_active":"2024-04-24T23:21:50.408755+0000","last_peered":"2024-04-24T23:21:50.408755+0000","last_clean":"2024-04-24T23:21:50.408755+0000","last_became_active":"2024-04-24T23:21:49.008996+0000","last_became_peered":"2024-04-24T23:21:49.008996+0000","last_unstale":"2024-04-24T23:21:50.408755+0000","last_undegraded":"2024-04-24T23:21:50.408755+0000","last_fullsized":"2024-04-24T23:21:50.408755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:00.445608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000418+0000","last_change":"2024-04-24T23:21:49.015388+0000","last_active":"2024-04-24T23:21:50.000418+0000","last_peered":"2024-04-24T23:21:50.000418+0000","last_clean":"2024-04-24T23:21:50.000418+0000","last_became_active":"2024-04-24T23:21:49.015072+0000","last_became_peered":"2024-04-24T23:21:49.015072+0000","last_unstale":"2024-04-24T23:21:50.000418+0000","last_undegraded":"2024-04-24T23:21:50.000418+0000","last_fullsized":"2024-04-24T23:21:50.000418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:50.469825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.406544+0000","last_change":"2024-04-24T23:21:49.007734+0000","last_active":"2024-04-24T23:21:50.406544+0000","last_peered":"2024-04-24T23:21:50.406544+0000","last_clean":"2024-04-24T23:21:50.406544+0000","last_became_active":"2024-04-24T23:21:49.007472+0000","last_became_peered":"2024-04-24T23:21:49.007472+0000","last_unstale":"2024-04-24T23:21:50.406544+0000","last_undegraded":"2024-04-24T23:21:50.406544+0000","last_fullsized":"2024-04-24T23:21:50.406544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:03.867719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999268+0000","last_change":"2024-04-24T23:21:49.025273+0000","last_active":"2024-04-24T23:21:49.999268+0000","last_peered":"2024-04-24T23:21:49.999268+0000","last_clean":"2024-04-24T23:21:49.999268+0000","last_became_active":"2024-04-24T23:21:49.024799+0000","last_became_peered":"2024-04-24T23:21:49.024799+0000","last_unstale":"2024-04-24T23:21:49.999268+0000","last_undegraded":"2024-04-24T23:21:49.999268+0000","last_fullsized":"2024-04-24T23:21:49.999268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:58:17.173142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000145+0000","last_change":"2024-04-24T23:21:49.011843+0000","last_active":"2024-04-24T23:21:50.000145+0000","last_peered":"2024-04-24T23:21:50.000145+0000","last_clean":"2024-04-24T23:21:50.000145+0000","last_became_active":"2024-04-24T23:21:49.011532+0000","last_became_peered":"2024-04-24T23:21:49.011532+0000","last_unstale":"2024-04-24T23:21:50.000145+0000","last_undegraded":"2024-04-24T23:21:50.000145+0000","last_fullsized":"2024-04-24T23:21:50.000145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:41.240750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407802+0000","last_change":"2024-04-24T23:21:49.017512+0000","last_active":"2024-04-24T23:21:50.407802+0000","last_peered":"2024-04-24T23:21:50.407802+0000","last_clean":"2024-04-24T23:21:50.407802+0000","last_became_active":"2024-04-24T23:21:49.017101+0000","last_became_peered":"2024-04-24T23:21:49.017101+0000","last_unstale":"2024-04-24T23:21:50.407802+0000","last_undegraded":"2024-04-24T23:21:50.407802+0000","last_fullsized":"2024-04-24T23:21:50.407802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:50.469825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408308+0000","last_change":"2024-04-24T23:21:49.020552+0000","last_active":"2024-04-24T23:21:50.408308+0000","last_peered":"2024-04-24T23:21:50.408308+0000","last_clean":"2024-04-24T23:21:50.408308+0000","last_became_active":"2024-04-24T23:21:49.020245+0000","last_became_peered":"2024-04-24T23:21:49.020245+0000","last_unstale":"2024-04-24T23:21:50.408308+0000","last_undegraded":"2024-04-24T23:21:50.408308+0000","last_fullsized":"2024-04-24T23:21:50.408308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:53.562702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999636+0000","last_change":"2024-04-24T23:21:49.021644+0000","last_active":"2024-04-24T23:21:49.999636+0000","last_peered":"2024-04-24T23:21:49.999636+0000","last_clean":"2024-04-24T23:21:49.999636+0000","last_became_active":"2024-04-24T23:21:49.020999+0000","last_became_peered":"2024-04-24T23:21:49.020999+0000","last_unstale":"2024-04-24T23:21:49.999636+0000","last_undegraded":"2024-04-24T23:21:49.999636+0000","last_fullsized":"2024-04-24T23:21:49.999636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:32.993424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999276+0000","last_change":"2024-04-24T23:21:49.022047+0000","last_active":"2024-04-24T23:21:49.999276+0000","last_peered":"2024-04-24T23:21:49.999276+0000","last_clean":"2024-04-24T23:21:49.999276+0000","last_became_active":"2024-04-24T23:21:49.021762+0000","last_became_peered":"2024-04-24T23:21:49.021762+0000","last_unstale":"2024-04-24T23:21:49.999276+0000","last_undegraded":"2024-04-24T23:21:49.999276+0000","last_fullsized":"2024-04-24T23:21:49.999276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:05.678853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.406710+0000","last_change":"2024-04-24T23:21:49.019693+0000","last_active":"2024-04-24T23:21:50.406710+0000","last_peered":"2024-04-24T23:21:50.406710+0000","last_clean":"2024-04-24T23:21:50.406710+0000","last_became_active":"2024-04-24T23:21:49.018532+0000","last_became_peered":"2024-04-24T23:21:49.018532+0000","last_unstale":"2024-04-24T23:21:50.406710+0000","last_undegraded":"2024-04-24T23:21:50.406710+0000","last_fullsized":"2024-04-24T23:21:50.406710+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:56.984644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407465+0000","last_change":"2024-04-24T23:21:49.006298+0000","last_active":"2024-04-24T23:21:50.407465+0000","last_peered":"2024-04-24T23:21:50.407465+0000","last_clean":"2024-04-24T23:21:50.407465+0000","last_became_active":"2024-04-24T23:21:49.006010+0000","last_became_peered":"2024-04-24T23:21:49.006010+0000","last_unstale":"2024-04-24T23:21:50.407465+0000","last_undegraded":"2024-04-24T23:21:50.407465+0000","last_fullsized":"2024-04-24T23:21:50.407465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.601383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407687+0000","last_change":"2024-04-24T23:21:49.019087+0000","last_active":"2024-04-24T23:21:50.407687+0000","last_peered":"2024-04-24T23:21:50.407687+0000","last_clean":"2024-04-24T23:21:50.407687+0000","last_became_active":"2024-04-24T23:21:49.018788+0000","last_became_peered":"2024-04-24T23:21:49.018788+0000","last_unstale":"2024-04-24T23:21:50.407687+0000","last_undegraded":"2024-04-24T23:21:50.407687+0000","last_fullsized":"2024-04-24T23:21:50.407687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:26.778105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.010465+0000","last_change":"2024-04-24T23:21:49.012129+0000","last_active":"2024-04-24T23:21:50.010465+0000","last_peered":"2024-04-24T23:21:50.010465+0000","last_clean":"2024-04-24T23:21:50.010465+0000","last_became_active":"2024-04-24T23:21:49.011802+0000","last_became_peered":"2024-04-24T23:21:49.011802+0000","last_unstale":"2024-04-24T23:21:50.010465+0000","last_undegraded":"2024-04-24T23:21:50.010465+0000","last_fullsized":"2024-04-24T23:21:50.010465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:51.434713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000068+0000","last_change":"2024-04-24T23:21:49.005326+0000","last_active":"2024-04-24T23:21:50.000068+0000","last_peered":"2024-04-24T23:21:50.000068+0000","last_clean":"2024-04-24T23:21:50.000068+0000","last_became_active":"2024-04-24T23:21:49.005019+0000","last_became_peered":"2024-04-24T23:21:49.005019+0000","last_unstale":"2024-04-24T23:21:50.000068+0000","last_undegraded":"2024-04-24T23:21:50.000068+0000","last_fullsized":"2024-04-24T23:21:50.000068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:50.157262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012841+0000","last_change":"2024-04-24T23:21:49.011713+0000","last_active":"2024-04-24T23:21:50.012841+0000","last_peered":"2024-04-24T23:21:50.012841+0000","last_clean":"2024-04-24T23:21:50.012841+0000","last_became_active":"2024-04-24T23:21:49.011399+0000","last_became_peered":"2024-04-24T23:21:49.011399+0000","last_unstale":"2024-04-24T23:21:50.012841+0000","last_undegraded":"2024-04-24T23:21:50.012841+0000","last_fullsized":"2024-04-24T23:21:50.012841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:41.240750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999711+0000","last_change":"2024-04-24T23:21:49.012339+0000","last_active":"2024-04-24T23:21:49.999711+0000","last_peered":"2024-04-24T23:21:49.999711+0000","last_clean":"2024-04-24T23:21:49.999711+0000","last_became_active":"2024-04-24T23:21:49.012054+0000","last_became_peered":"2024-04-24T23:21:49.012054+0000","last_unstale":"2024-04-24T23:21:49.999711+0000","last_undegraded":"2024-04-24T23:21:49.999711+0000","last_fullsized":"2024-04-24T23:21:49.999711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:12.485921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012728+0000","last_change":"2024-04-24T23:21:49.006930+0000","last_active":"2024-04-24T23:21:50.012728+0000","last_peered":"2024-04-24T23:21:50.012728+0000","last_clean":"2024-04-24T23:21:50.012728+0000","last_became_active":"2024-04-24T23:21:49.006100+0000","last_became_peered":"2024-04-24T23:21:49.006100+0000","last_unstale":"2024-04-24T23:21:50.012728+0000","last_undegraded":"2024-04-24T23:21:50.012728+0000","last_fullsized":"2024-04-24T23:21:50.012728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:16.207856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000446+0000","last_change":"2024-04-24T23:21:49.003530+0000","last_active":"2024-04-24T23:21:50.000446+0000","last_peered":"2024-04-24T23:21:50.000446+0000","last_clean":"2024-04-24T23:21:50.000446+0000","last_became_active":"2024-04-24T23:21:49.003192+0000","last_became_peered":"2024-04-24T23:21:49.003192+0000","last_unstale":"2024-04-24T23:21:50.000446+0000","last_undegraded":"2024-04-24T23:21:50.000446+0000","last_fullsized":"2024-04-24T23:21:50.000446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:34.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407650+0000","last_change":"2024-04-24T23:21:49.016659+0000","last_active":"2024-04-24T23:21:50.407650+0000","last_peered":"2024-04-24T23:21:50.407650+0000","last_clean":"2024-04-24T23:21:50.407650+0000","last_became_active":"2024-04-24T23:21:49.014218+0000","last_became_peered":"2024-04-24T23:21:49.014218+0000","last_unstale":"2024-04-24T23:21:50.407650+0000","last_undegraded":"2024-04-24T23:21:50.407650+0000","last_fullsized":"2024-04-24T23:21:50.407650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:59.118957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408443+0000","last_change":"2024-04-24T23:21:49.018007+0000","last_active":"2024-04-24T23:21:50.408443+0000","last_peered":"2024-04-24T23:21:50.408443+0000","last_clean":"2024-04-24T23:21:50.408443+0000","last_became_active":"2024-04-24T23:21:49.017112+0000","last_became_peered":"2024-04-24T23:21:49.017112+0000","last_unstale":"2024-04-24T23:21:50.408443+0000","last_undegraded":"2024-04-24T23:21:50.408443+0000","last_fullsized":"2024-04-24T23:21:50.408443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:58:17.173142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407531+0000","last_change":"2024-04-24T23:21:49.004709+0000","last_active":"2024-04-24T23:21:50.407531+0000","last_peered":"2024-04-24T23:21:50.407531+0000","last_clean":"2024-04-24T23:21:50.407531+0000","last_became_active":"2024-04-24T23:21:49.004398+0000","last_became_peered":"2024-04-24T23:21:49.004398+0000","last_unstale":"2024-04-24T23:21:50.407531+0000","last_undegraded":"2024-04-24T23:21:50.407531+0000","last_fullsized":"2024-04-24T23:21:50.407531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:51.434713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407393+0000","last_change":"2024-04-24T23:21:49.013581+0000","last_active":"2024-04-24T23:21:50.407393+0000","last_peered":"2024-04-24T23:21:50.407393+0000","last_clean":"2024-04-24T23:21:50.407393+0000","last_became_active":"2024-04-24T23:21:49.013255+0000","last_became_peered":"2024-04-24T23:21:49.013255+0000","last_unstale":"2024-04-24T23:21:50.407393+0000","last_undegraded":"2024-04-24T23:21:50.407393+0000","last_fullsized":"2024-04-24T23:21:50.407393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:41.240750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.010575+0000","last_change":"2024-04-24T23:21:49.010775+0000","last_active":"2024-04-24T23:21:50.010575+0000","last_peered":"2024-04-24T23:21:50.010575+0000","last_clean":"2024-04-24T23:21:50.010575+0000","last_became_active":"2024-04-24T23:21:49.009760+0000","last_became_peered":"2024-04-24T23:21:49.009760+0000","last_unstale":"2024-04-24T23:21:50.010575+0000","last_undegraded":"2024-04-24T23:21:50.010575+0000","last_fullsized":"2024-04-24T23:21:50.010575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.601383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407751+0000","last_change":"2024-04-24T23:21:47.969189+0000","last_active":"2024-04-24T23:21:50.407751+0000","last_peered":"2024-04-24T23:21:50.407751+0000","last_clean":"2024-04-24T23:21:50.407751+0000","last_became_active":"2024-04-24T23:21:47.968655+0000","last_became_peered":"2024-04-24T23:21:47.968655+0000","last_unstale":"2024-04-24T23:21:50.407751+0000","last_undegraded":"2024-04-24T23:21:50.407751+0000","last_fullsized":"2024-04-24T23:21:50.407751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:35.773731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407690+0000","last_change":"2024-04-24T23:21:49.009447+0000","last_active":"2024-04-24T23:21:50.407690+0000","last_peered":"2024-04-24T23:21:50.407690+0000","last_clean":"2024-04-24T23:21:50.407690+0000","last_became_active":"2024-04-24T23:21:49.008924+0000","last_became_peered":"2024-04-24T23:21:49.008924+0000","last_unstale":"2024-04-24T23:21:50.407690+0000","last_undegraded":"2024-04-24T23:21:50.407690+0000","last_fullsized":"2024-04-24T23:21:50.407690+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:25.111679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.010534+0000","last_change":"2024-04-24T23:21:47.972344+0000","last_active":"2024-04-24T23:21:50.010534+0000","last_peered":"2024-04-24T23:21:50.010534+0000","last_clean":"2024-04-24T23:21:50.010534+0000","last_became_active":"2024-04-24T23:21:47.972047+0000","last_became_peered":"2024-04-24T23:21:47.972047+0000","last_unstale":"2024-04-24T23:21:50.010534+0000","last_undegraded":"2024-04-24T23:21:50.010534+0000","last_fullsized":"2024-04-24T23:21:50.010534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:37.844221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408578+0000","last_change":"2024-04-24T23:21:49.011462+0000","last_active":"2024-04-24T23:21:50.408578+0000","last_peered":"2024-04-24T23:21:50.408578+0000","last_clean":"2024-04-24T23:21:50.408578+0000","last_became_active":"2024-04-24T23:21:49.011156+0000","last_became_peered":"2024-04-24T23:21:49.011156+0000","last_unstale":"2024-04-24T23:21:50.408578+0000","last_undegraded":"2024-04-24T23:21:50.408578+0000","last_fullsized":"2024-04-24T23:21:50.408578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:13.783082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999332+0000","last_change":"2024-04-24T23:21:47.977644+0000","last_active":"2024-04-24T23:21:49.999332+0000","last_peered":"2024-04-24T23:21:49.999332+0000","last_clean":"2024-04-24T23:21:49.999332+0000","last_became_active":"2024-04-24T23:21:47.977398+0000","last_became_peered":"2024-04-24T23:21:47.977398+0000","last_unstale":"2024-04-24T23:21:49.999332+0000","last_undegraded":"2024-04-24T23:21:49.999332+0000","last_fullsized":"2024-04-24T23:21:49.999332+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:45.599411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012791+0000","last_change":"2024-04-24T23:21:49.007860+0000","last_active":"2024-04-24T23:21:50.012791+0000","last_peered":"2024-04-24T23:21:50.012791+0000","last_clean":"2024-04-24T23:21:50.012791+0000","last_became_active":"2024-04-24T23:21:49.007556+0000","last_became_peered":"2024-04-24T23:21:49.007556+0000","last_unstale":"2024-04-24T23:21:50.012791+0000","last_undegraded":"2024-04-24T23:21:50.012791+0000","last_fullsized":"2024-04-24T23:21:50.012791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:50.157262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.406724+0000","last_change":"2024-04-24T23:21:47.974278+0000","last_active":"2024-04-24T23:21:50.406724+0000","last_peered":"2024-04-24T23:21:50.406724+0000","last_clean":"2024-04-24T23:21:50.406724+0000","last_became_active":"2024-04-24T23:21:47.973998+0000","last_became_peered":"2024-04-24T23:21:47.973998+0000","last_unstale":"2024-04-24T23:21:50.406724+0000","last_undegraded":"2024-04-24T23:21:50.406724+0000","last_fullsized":"2024-04-24T23:21:50.406724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:50.796302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408502+0000","last_change":"2024-04-24T23:21:49.013301+0000","last_active":"2024-04-24T23:21:50.408502+0000","last_peered":"2024-04-24T23:21:50.408502+0000","last_clean":"2024-04-24T23:21:50.408502+0000","last_became_active":"2024-04-24T23:21:49.012971+0000","last_became_peered":"2024-04-24T23:21:49.012971+0000","last_unstale":"2024-04-24T23:21:50.408502+0000","last_undegraded":"2024-04-24T23:21:50.408502+0000","last_fullsized":"2024-04-24T23:21:50.408502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:05.678853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407969+0000","last_change":"2024-04-24T23:21:47.972662+0000","last_active":"2024-04-24T23:21:50.407969+0000","last_peered":"2024-04-24T23:21:50.407969+0000","last_clean":"2024-04-24T23:21:50.407969+0000","last_became_active":"2024-04-24T23:21:47.972386+0000","last_became_peered":"2024-04-24T23:21:47.972386+0000","last_unstale":"2024-04-24T23:21:50.407969+0000","last_undegraded":"2024-04-24T23:21:50.407969+0000","last_fullsized":"2024-04-24T23:21:50.407969+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:18.264459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000378+0000","last_change":"2024-04-24T23:21:49.013935+0000","last_active":"2024-04-24T23:21:50.000378+0000","last_peered":"2024-04-24T23:21:50.000378+0000","last_clean":"2024-04-24T23:21:50.000378+0000","last_became_active":"2024-04-24T23:21:49.013649+0000","last_became_peered":"2024-04-24T23:21:49.013649+0000","last_unstale":"2024-04-24T23:21:50.000378+0000","last_undegraded":"2024-04-24T23:21:50.000378+0000","last_fullsized":"2024-04-24T23:21:50.000378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:49.429392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407743+0000","last_change":"2024-04-24T23:21:47.971446+0000","last_active":"2024-04-24T23:21:50.407743+0000","last_peered":"2024-04-24T23:21:50.407743+0000","last_clean":"2024-04-24T23:21:50.407743+0000","last_became_active":"2024-04-24T23:21:47.971139+0000","last_became_peered":"2024-04-24T23:21:47.971139+0000","last_unstale":"2024-04-24T23:21:50.407743+0000","last_undegraded":"2024-04-24T23:21:50.407743+0000","last_fullsized":"2024-04-24T23:21:50.407743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:40.076523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407232+0000","last_change":"2024-04-24T23:21:49.006610+0000","last_active":"2024-04-24T23:21:50.407232+0000","last_peered":"2024-04-24T23:21:50.407232+0000","last_clean":"2024-04-24T23:21:50.407232+0000","last_became_active":"2024-04-24T23:21:49.006311+0000","last_became_peered":"2024-04-24T23:21:49.006311+0000","last_unstale":"2024-04-24T23:21:50.407232+0000","last_undegraded":"2024-04-24T23:21:50.407232+0000","last_fullsized":"2024-04-24T23:21:50.407232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:25.111679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.998986+0000","last_change":"2024-04-24T23:21:47.969555+0000","last_active":"2024-04-24T23:21:49.998986+0000","last_peered":"2024-04-24T23:21:49.998986+0000","last_clean":"2024-04-24T23:21:49.998986+0000","last_became_active":"2024-04-24T23:21:47.969260+0000","last_became_peered":"2024-04-24T23:21:47.969260+0000","last_unstale":"2024-04-24T23:21:49.998986+0000","last_undegraded":"2024-04-24T23:21:49.998986+0000","last_fullsized":"2024-04-24T23:21:49.998986+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:58.116984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012415+0000","last_change":"2024-04-24T23:21:48.998367+0000","last_active":"2024-04-24T23:21:50.012415+0000","last_peered":"2024-04-24T23:21:50.012415+0000","last_clean":"2024-04-24T23:21:50.012415+0000","last_became_active":"2024-04-24T23:21:48.998097+0000","last_became_peered":"2024-04-24T23:21:48.998097+0000","last_unstale":"2024-04-24T23:21:50.012415+0000","last_undegraded":"2024-04-24T23:21:50.012415+0000","last_fullsized":"2024-04-24T23:21:50.012415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:35.742750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407292+0000","last_change":"2024-04-24T23:21:47.972762+0000","last_active":"2024-04-24T23:21:50.407292+0000","last_peered":"2024-04-24T23:21:50.407292+0000","last_clean":"2024-04-24T23:21:50.407292+0000","last_became_active":"2024-04-24T23:21:47.972370+0000","last_became_peered":"2024-04-24T23:21:47.972370+0000","last_unstale":"2024-04-24T23:21:50.407292+0000","last_undegraded":"2024-04-24T23:21:50.407292+0000","last_fullsized":"2024-04-24T23:21:50.407292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:47.264003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000841+0000","last_change":"2024-04-24T23:21:49.017524+0000","last_active":"2024-04-24T23:21:50.000841+0000","last_peered":"2024-04-24T23:21:50.000841+0000","last_clean":"2024-04-24T23:21:50.000841+0000","last_became_active":"2024-04-24T23:21:49.017195+0000","last_became_peered":"2024-04-24T23:21:49.017195+0000","last_unstale":"2024-04-24T23:21:50.000841+0000","last_undegraded":"2024-04-24T23:21:50.000841+0000","last_fullsized":"2024-04-24T23:21:50.000841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:05.678853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999503+0000","last_change":"2024-04-24T23:21:47.976015+0000","last_active":"2024-04-24T23:21:49.999503+0000","last_peered":"2024-04-24T23:21:49.999503+0000","last_clean":"2024-04-24T23:21:49.999503+0000","last_became_active":"2024-04-24T23:21:47.975677+0000","last_became_peered":"2024-04-24T23:21:47.975677+0000","last_unstale":"2024-04-24T23:21:49.999503+0000","last_undegraded":"2024-04-24T23:21:49.999503+0000","last_fullsized":"2024-04-24T23:21:49.999503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:25.776132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999430+0000","last_change":"2024-04-24T23:21:49.024036+0000","last_active":"2024-04-24T23:21:49.999430+0000","last_peered":"2024-04-24T23:21:49.999430+0000","last_clean":"2024-04-24T23:21:49.999430+0000","last_became_active":"2024-04-24T23:21:49.023661+0000","last_became_peered":"2024-04-24T23:21:49.023661+0000","last_unstale":"2024-04-24T23:21:49.999430+0000","last_undegraded":"2024-04-24T23:21:49.999430+0000","last_fullsized":"2024-04-24T23:21:49.999430+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:00.445608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.010624+0000","last_change":"2024-04-24T23:21:47.970103+0000","last_active":"2024-04-24T23:21:50.010624+0000","last_peered":"2024-04-24T23:21:50.010624+0000","last_clean":"2024-04-24T23:21:50.010624+0000","last_became_active":"2024-04-24T23:21:47.969156+0000","last_became_peered":"2024-04-24T23:21:47.969156+0000","last_unstale":"2024-04-24T23:21:50.010624+0000","last_undegraded":"2024-04-24T23:21:50.010624+0000","last_fullsized":"2024-04-24T23:21:50.010624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:30.115325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999626+0000","last_change":"2024-04-24T23:21:49.011605+0000","last_active":"2024-04-24T23:21:49.999626+0000","last_peered":"2024-04-24T23:21:49.999626+0000","last_clean":"2024-04-24T23:21:49.999626+0000","last_became_active":"2024-04-24T23:21:49.011287+0000","last_became_peered":"2024-04-24T23:21:49.011287+0000","last_unstale":"2024-04-24T23:21:49.999626+0000","last_undegraded":"2024-04-24T23:21:49.999626+0000","last_fullsized":"2024-04-24T23:21:49.999626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:57.509390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000801+0000","last_change":"2024-04-24T23:21:47.974000+0000","last_active":"2024-04-24T23:21:50.000801+0000","last_peered":"2024-04-24T23:21:50.000801+0000","last_clean":"2024-04-24T23:21:50.000801+0000","last_became_active":"2024-04-24T23:21:47.973681+0000","last_became_peered":"2024-04-24T23:21:47.973681+0000","last_unstale":"2024-04-24T23:21:50.000801+0000","last_undegraded":"2024-04-24T23:21:50.000801+0000","last_fullsized":"2024-04-24T23:21:50.000801+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:50.432741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000740+0000","last_change":"2024-04-24T23:21:49.014731+0000","last_active":"2024-04-24T23:21:50.000740+0000","last_peered":"2024-04-24T23:21:50.000740+0000","last_clean":"2024-04-24T23:21:50.000740+0000","last_became_active":"2024-04-24T23:21:49.014389+0000","last_became_peered":"2024-04-24T23:21:49.014389+0000","last_unstale":"2024-04-24T23:21:50.000740+0000","last_undegraded":"2024-04-24T23:21:50.000740+0000","last_fullsized":"2024-04-24T23:21:50.000740+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:53.562702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999470+0000","last_change":"2024-04-24T23:21:47.977605+0000","last_active":"2024-04-24T23:21:49.999470+0000","last_peered":"2024-04-24T23:21:49.999470+0000","last_clean":"2024-04-24T23:21:49.999470+0000","last_became_active":"2024-04-24T23:21:47.977329+0000","last_became_peered":"2024-04-24T23:21:47.977329+0000","last_unstale":"2024-04-24T23:21:49.999470+0000","last_undegraded":"2024-04-24T23:21:49.999470+0000","last_fullsized":"2024-04-24T23:21:49.999470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:53.356343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012424+0000","last_change":"2024-04-24T23:21:49.012817+0000","last_active":"2024-04-24T23:21:50.012424+0000","last_peered":"2024-04-24T23:21:50.012424+0000","last_clean":"2024-04-24T23:21:50.012424+0000","last_became_active":"2024-04-24T23:21:49.012511+0000","last_became_peered":"2024-04-24T23:21:49.012511+0000","last_unstale":"2024-04-24T23:21:50.012424+0000","last_undegraded":"2024-04-24T23:21:50.012424+0000","last_fullsized":"2024-04-24T23:21:50.012424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:38.250502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407976+0000","last_change":"2024-04-24T23:21:47.973107+0000","last_active":"2024-04-24T23:21:50.407976+0000","last_peered":"2024-04-24T23:21:50.407976+0000","last_clean":"2024-04-24T23:21:50.407976+0000","last_became_active":"2024-04-24T23:21:47.972751+0000","last_became_peered":"2024-04-24T23:21:47.972751+0000","last_unstale":"2024-04-24T23:21:50.407976+0000","last_undegraded":"2024-04-24T23:21:50.407976+0000","last_fullsized":"2024-04-24T23:21:50.407976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:02.075856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999380+0000","last_change":"2024-04-24T23:21:49.015830+0000","last_active":"2024-04-24T23:21:49.999380+0000","last_peered":"2024-04-24T23:21:49.999380+0000","last_clean":"2024-04-24T23:21:49.999380+0000","last_became_active":"2024-04-24T23:21:49.015395+0000","last_became_peered":"2024-04-24T23:21:49.015395+0000","last_unstale":"2024-04-24T23:21:49.999380+0000","last_undegraded":"2024-04-24T23:21:49.999380+0000","last_fullsized":"2024-04-24T23:21:49.999380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:08.687119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012384+0000","last_change":"2024-04-24T23:21:47.973481+0000","last_active":"2024-04-24T23:21:50.012384+0000","last_peered":"2024-04-24T23:21:50.012384+0000","last_clean":"2024-04-24T23:21:50.012384+0000","last_became_active":"2024-04-24T23:21:47.973085+0000","last_became_peered":"2024-04-24T23:21:47.973085+0000","last_unstale":"2024-04-24T23:21:50.012384+0000","last_undegraded":"2024-04-24T23:21:50.012384+0000","last_fullsized":"2024-04-24T23:21:50.012384+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:35.773731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407550+0000","last_change":"2024-04-24T23:21:49.013471+0000","last_active":"2024-04-24T23:21:50.407550+0000","last_peered":"2024-04-24T23:21:50.407550+0000","last_clean":"2024-04-24T23:21:50.407550+0000","last_became_active":"2024-04-24T23:21:49.013031+0000","last_became_peered":"2024-04-24T23:21:49.013031+0000","last_unstale":"2024-04-24T23:21:50.407550+0000","last_undegraded":"2024-04-24T23:21:50.407550+0000","last_fullsized":"2024-04-24T23:21:50.407550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:49.429392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.998954+0000","last_change":"2024-04-24T23:21:47.971677+0000","last_active":"2024-04-24T23:21:49.998954+0000","last_peered":"2024-04-24T23:21:49.998954+0000","last_clean":"2024-04-24T23:21:49.998954+0000","last_became_active":"2024-04-24T23:21:47.971390+0000","last_became_peered":"2024-04-24T23:21:47.971390+0000","last_unstale":"2024-04-24T23:21:49.998954+0000","last_undegraded":"2024-04-24T23:21:49.998954+0000","last_fullsized":"2024-04-24T23:21:49.998954+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:44.015077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407714+0000","last_change":"2024-04-24T23:21:49.010154+0000","last_active":"2024-04-24T23:21:50.407714+0000","last_peered":"2024-04-24T23:21:50.407714+0000","last_clean":"2024-04-24T23:21:50.407714+0000","last_became_active":"2024-04-24T23:21:49.009825+0000","last_became_peered":"2024-04-24T23:21:49.009825+0000","last_unstale":"2024-04-24T23:21:50.407714+0000","last_undegraded":"2024-04-24T23:21:50.407714+0000","last_fullsized":"2024-04-24T23:21:50.407714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.601383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999033+0000","last_change":"2024-04-24T23:21:47.977318+0000","last_active":"2024-04-24T23:21:49.999033+0000","last_peered":"2024-04-24T23:21:49.999033+0000","last_clean":"2024-04-24T23:21:49.999033+0000","last_became_active":"2024-04-24T23:21:47.977045+0000","last_became_peered":"2024-04-24T23:21:47.977045+0000","last_unstale":"2024-04-24T23:21:49.999033+0000","last_undegraded":"2024-04-24T23:21:49.999033+0000","last_fullsized":"2024-04-24T23:21:49.999033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:24.109707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408159+0000","last_change":"2024-04-24T23:21:49.019919+0000","last_active":"2024-04-24T23:21:50.408159+0000","last_peered":"2024-04-24T23:21:50.408159+0000","last_clean":"2024-04-24T23:21:50.408159+0000","last_became_active":"2024-04-24T23:21:49.019607+0000","last_became_peered":"2024-04-24T23:21:49.019607+0000","last_unstale":"2024-04-24T23:21:50.408159+0000","last_undegraded":"2024-04-24T23:21:50.408159+0000","last_fullsized":"2024-04-24T23:21:50.408159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:35.742750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012870+0000","last_change":"2024-04-24T23:21:47.975057+0000","last_active":"2024-04-24T23:21:50.012870+0000","last_peered":"2024-04-24T23:21:50.012870+0000","last_clean":"2024-04-24T23:21:50.012870+0000","last_became_active":"2024-04-24T23:21:47.974757+0000","last_became_peered":"2024-04-24T23:21:47.974757+0000","last_unstale":"2024-04-24T23:21:50.012870+0000","last_undegraded":"2024-04-24T23:21:50.012870+0000","last_fullsized":"2024-04-24T23:21:50.012870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:30.115325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000865+0000","last_change":"2024-04-24T23:21:49.008171+0000","last_active":"2024-04-24T23:21:50.000865+0000","last_peered":"2024-04-24T23:21:50.000865+0000","last_clean":"2024-04-24T23:21:50.000865+0000","last_became_active":"2024-04-24T23:21:49.007831+0000","last_became_peered":"2024-04-24T23:21:49.007831+0000","last_unstale":"2024-04-24T23:21:50.000865+0000","last_undegraded":"2024-04-24T23:21:50.000865+0000","last_fullsized":"2024-04-24T23:21:50.000865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:08.687119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408110+0000","last_change":"2024-04-24T23:21:47.974053+0000","last_active":"2024-04-24T23:21:50.408110+0000","last_peered":"2024-04-24T23:21:50.408110+0000","last_clean":"2024-04-24T23:21:50.408110+0000","last_became_active":"2024-04-24T23:21:47.972594+0000","last_became_peered":"2024-04-24T23:21:47.972594+0000","last_unstale":"2024-04-24T23:21:50.408110+0000","last_undegraded":"2024-04-24T23:21:50.408110+0000","last_fullsized":"2024-04-24T23:21:50.408110+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:30.223906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012685+0000","last_change":"2024-04-24T23:21:49.011718+0000","last_active":"2024-04-24T23:21:50.012685+0000","last_peered":"2024-04-24T23:21:50.012685+0000","last_clean":"2024-04-24T23:21:50.012685+0000","last_became_active":"2024-04-24T23:21:49.010961+0000","last_became_peered":"2024-04-24T23:21:49.010961+0000","last_unstale":"2024-04-24T23:21:50.012685+0000","last_undegraded":"2024-04-24T23:21:50.012685+0000","last_fullsized":"2024-04-24T23:21:50.012685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:35.959790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408332+0000","last_change":"2024-04-24T23:21:47.970748+0000","last_active":"2024-04-24T23:21:50.408332+0000","last_peered":"2024-04-24T23:21:50.408332+0000","last_clean":"2024-04-24T23:21:50.408332+0000","last_became_active":"2024-04-24T23:21:47.970415+0000","last_became_peered":"2024-04-24T23:21:47.970415+0000","last_unstale":"2024-04-24T23:21:50.408332+0000","last_undegraded":"2024-04-24T23:21:50.408332+0000","last_fullsized":"2024-04-24T23:21:50.408332+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:51.084749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.406791+0000","last_change":"2024-04-24T23:21:49.010459+0000","last_active":"2024-04-24T23:21:50.406791+0000","last_peered":"2024-04-24T23:21:50.406791+0000","last_clean":"2024-04-24T23:21:50.406791+0000","last_became_active":"2024-04-24T23:21:49.010153+0000","last_became_peered":"2024-04-24T23:21:49.010153+0000","last_unstale":"2024-04-24T23:21:50.406791+0000","last_undegraded":"2024-04-24T23:21:50.406791+0000","last_fullsized":"2024-04-24T23:21:50.406791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:35.859008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408488+0000","last_change":"2024-04-24T23:21:47.972273+0000","last_active":"2024-04-24T23:21:50.408488+0000","last_peered":"2024-04-24T23:21:50.408488+0000","last_clean":"2024-04-24T23:21:50.408488+0000","last_became_active":"2024-04-24T23:21:47.971961+0000","last_became_peered":"2024-04-24T23:21:47.971961+0000","last_unstale":"2024-04-24T23:21:50.408488+0000","last_undegraded":"2024-04-24T23:21:50.408488+0000","last_fullsized":"2024-04-24T23:21:50.408488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:13.654777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.406506+0000","last_change":"2024-04-24T23:21:49.014845+0000","last_active":"2024-04-24T23:21:50.406506+0000","last_peered":"2024-04-24T23:21:50.406506+0000","last_clean":"2024-04-24T23:21:50.406506+0000","last_became_active":"2024-04-24T23:21:49.014533+0000","last_became_peered":"2024-04-24T23:21:49.014533+0000","last_unstale":"2024-04-24T23:21:50.406506+0000","last_undegraded":"2024-04-24T23:21:50.406506+0000","last_fullsized":"2024-04-24T23:21:50.406506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:36.775704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.010639+0000","last_change":"2024-04-24T23:21:47.977855+0000","last_active":"2024-04-24T23:21:50.010639+0000","last_peered":"2024-04-24T23:21:50.010639+0000","last_clean":"2024-04-24T23:21:50.010639+0000","last_became_active":"2024-04-24T23:21:47.977595+0000","last_became_peered":"2024-04-24T23:21:47.977595+0000","last_unstale":"2024-04-24T23:21:50.010639+0000","last_undegraded":"2024-04-24T23:21:50.010639+0000","last_fullsized":"2024-04-24T23:21:50.010639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:19.499255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.406694+0000","last_change":"2024-04-24T23:21:49.005811+0000","last_active":"2024-04-24T23:21:50.406694+0000","last_peered":"2024-04-24T23:21:50.406694+0000","last_clean":"2024-04-24T23:21:50.406694+0000","last_became_active":"2024-04-24T23:21:49.005514+0000","last_became_peered":"2024-04-24T23:21:49.005514+0000","last_unstale":"2024-04-24T23:21:50.406694+0000","last_undegraded":"2024-04-24T23:21:50.406694+0000","last_fullsized":"2024-04-24T23:21:50.406694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:31.117298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408060+0000","last_change":"2024-04-24T23:21:47.976892+0000","last_active":"2024-04-24T23:21:50.408060+0000","last_peered":"2024-04-24T23:21:50.408060+0000","last_clean":"2024-04-24T23:21:50.408060+0000","last_became_active":"2024-04-24T23:21:47.976567+0000","last_became_peered":"2024-04-24T23:21:47.976567+0000","last_unstale":"2024-04-24T23:21:50.408060+0000","last_undegraded":"2024-04-24T23:21:50.408060+0000","last_fullsized":"2024-04-24T23:21:50.408060+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:19.499255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.406441+0000","last_change":"2024-04-24T23:21:49.006010+0000","last_active":"2024-04-24T23:21:50.406441+0000","last_peered":"2024-04-24T23:21:50.406441+0000","last_clean":"2024-04-24T23:21:50.406441+0000","last_became_active":"2024-04-24T23:21:49.005717+0000","last_became_peered":"2024-04-24T23:21:49.005717+0000","last_unstale":"2024-04-24T23:21:50.406441+0000","last_undegraded":"2024-04-24T23:21:50.406441+0000","last_fullsized":"2024-04-24T23:21:50.406441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:38.846193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407578+0000","last_change":"2024-04-24T23:21:47.974743+0000","last_active":"2024-04-24T23:21:50.407578+0000","last_peered":"2024-04-24T23:21:50.407578+0000","last_clean":"2024-04-24T23:21:50.407578+0000","last_became_active":"2024-04-24T23:21:47.974473+0000","last_became_peered":"2024-04-24T23:21:47.974473+0000","last_unstale":"2024-04-24T23:21:50.407578+0000","last_undegraded":"2024-04-24T23:21:50.407578+0000","last_fullsized":"2024-04-24T23:21:50.407578+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:37.844221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407305+0000","last_change":"2024-04-24T23:21:49.007321+0000","last_active":"2024-04-24T23:21:50.407305+0000","last_peered":"2024-04-24T23:21:50.407305+0000","last_clean":"2024-04-24T23:21:50.407305+0000","last_became_active":"2024-04-24T23:21:49.007019+0000","last_became_peered":"2024-04-24T23:21:49.007019+0000","last_unstale":"2024-04-24T23:21:50.407305+0000","last_undegraded":"2024-04-24T23:21:50.407305+0000","last_fullsized":"2024-04-24T23:21:50.407305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:50.157262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.999919+0000","last_change":"2024-04-24T23:21:47.971363+0000","last_active":"2024-04-24T23:21:49.999919+0000","last_peered":"2024-04-24T23:21:49.999919+0000","last_clean":"2024-04-24T23:21:49.999919+0000","last_became_active":"2024-04-24T23:21:47.970801+0000","last_became_peered":"2024-04-24T23:21:47.970801+0000","last_unstale":"2024-04-24T23:21:49.999919+0000","last_undegraded":"2024-04-24T23:21:49.999919+0000","last_fullsized":"2024-04-24T23:21:49.999919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:44.015077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.406771+0000","last_change":"2024-04-24T23:21:49.018672+0000","last_active":"2024-04-24T23:21:50.406771+0000","last_peered":"2024-04-24T23:21:50.406771+0000","last_clean":"2024-04-24T23:21:50.406771+0000","last_became_active":"2024-04-24T23:21:49.018374+0000","last_became_peered":"2024-04-24T23:21:49.018374+0000","last_unstale":"2024-04-24T23:21:50.406771+0000","last_undegraded":"2024-04-24T23:21:50.406771+0000","last_fullsized":"2024-04-24T23:21:50.406771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:28.669901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012880+0000","last_change":"2024-04-24T23:21:47.971059+0000","last_active":"2024-04-24T23:21:50.012880+0000","last_peered":"2024-04-24T23:21:50.012880+0000","last_clean":"2024-04-24T23:21:50.012880+0000","last_became_active":"2024-04-24T23:21:47.970752+0000","last_became_peered":"2024-04-24T23:21:47.970752+0000","last_unstale":"2024-04-24T23:21:50.012880+0000","last_undegraded":"2024-04-24T23:21:50.012880+0000","last_fullsized":"2024-04-24T23:21:50.012880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:55.982671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.000873+0000","last_change":"2024-04-24T23:21:49.000711+0000","last_active":"2024-04-24T23:21:50.000873+0000","last_peered":"2024-04-24T23:21:50.000873+0000","last_clean":"2024-04-24T23:21:50.000873+0000","last_became_active":"2024-04-24T23:21:49.000400+0000","last_became_peered":"2024-04-24T23:21:49.000400+0000","last_unstale":"2024-04-24T23:21:50.000873+0000","last_undegraded":"2024-04-24T23:21:50.000873+0000","last_fullsized":"2024-04-24T23:21:50.000873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:00.445608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.010643+0000","last_change":"2024-04-24T23:21:47.977458+0000","last_active":"2024-04-24T23:21:50.010643+0000","last_peered":"2024-04-24T23:21:50.010643+0000","last_clean":"2024-04-24T23:21:50.010643+0000","last_became_active":"2024-04-24T23:21:47.977165+0000","last_became_peered":"2024-04-24T23:21:47.977165+0000","last_unstale":"2024-04-24T23:21:50.010643+0000","last_undegraded":"2024-04-24T23:21:50.010643+0000","last_fullsized":"2024-04-24T23:21:50.010643+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:02.865747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.010539+0000","last_change":"2024-04-24T23:21:49.005263+0000","last_active":"2024-04-24T23:21:50.010539+0000","last_peered":"2024-04-24T23:21:50.010539+0000","last_clean":"2024-04-24T23:21:50.010539+0000","last_became_active":"2024-04-24T23:21:49.004988+0000","last_became_peered":"2024-04-24T23:21:49.004988+0000","last_unstale":"2024-04-24T23:21:50.010539+0000","last_undegraded":"2024-04-24T23:21:50.010539+0000","last_fullsized":"2024-04-24T23:21:50.010539+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:25.111679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012650+0000","last_change":"2024-04-24T23:21:47.968023+0000","last_active":"2024-04-24T23:21:50.012650+0000","last_peered":"2024-04-24T23:21:50.012650+0000","last_clean":"2024-04-24T23:21:50.012650+0000","last_became_active":"2024-04-24T23:21:47.967756+0000","last_became_peered":"2024-04-24T23:21:47.967756+0000","last_unstale":"2024-04-24T23:21:50.012650+0000","last_undegraded":"2024-04-24T23:21:50.012650+0000","last_fullsized":"2024-04-24T23:21:50.012650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:02.865747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.408003+0000","last_change":"2024-04-24T23:21:49.008323+0000","last_active":"2024-04-24T23:21:50.408003+0000","last_peered":"2024-04-24T23:21:50.408003+0000","last_clean":"2024-04-24T23:21:50.408003+0000","last_became_active":"2024-04-24T23:21:49.007982+0000","last_became_peered":"2024-04-24T23:21:49.007982+0000","last_unstale":"2024-04-24T23:21:50.408003+0000","last_undegraded":"2024-04-24T23:21:50.408003+0000","last_fullsized":"2024-04-24T23:21:50.408003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:54.358315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.010740+0000","last_change":"2024-04-24T23:21:47.971157+0000","last_active":"2024-04-24T23:21:50.010740+0000","last_peered":"2024-04-24T23:21:50.010740+0000","last_clean":"2024-04-24T23:21:50.010740+0000","last_became_active":"2024-04-24T23:21:47.970882+0000","last_became_peered":"2024-04-24T23:21:47.970882+0000","last_unstale":"2024-04-24T23:21:50.010740+0000","last_undegraded":"2024-04-24T23:21:50.010740+0000","last_fullsized":"2024-04-24T23:21:50.010740+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:35.773731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.010652+0000","last_change":"2024-04-24T23:21:49.008298+0000","last_active":"2024-04-24T23:21:50.010652+0000","last_peered":"2024-04-24T23:21:50.010652+0000","last_clean":"2024-04-24T23:21:50.010652+0000","last_became_active":"2024-04-24T23:21:49.008022+0000","last_became_peered":"2024-04-24T23:21:49.008022+0000","last_unstale":"2024-04-24T23:21:50.010652+0000","last_undegraded":"2024-04-24T23:21:50.010652+0000","last_fullsized":"2024-04-24T23:21:50.010652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:54.358315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.012486+0000","last_change":"2024-04-24T23:21:47.965046+0000","last_active":"2024-04-24T23:21:50.012486+0000","last_peered":"2024-04-24T23:21:50.012486+0000","last_clean":"2024-04-24T23:21:50.012486+0000","last_became_active":"2024-04-24T23:21:47.964769+0000","last_became_peered":"2024-04-24T23:21:47.964769+0000","last_unstale":"2024-04-24T23:21:50.012486+0000","last_undegraded":"2024-04-24T23:21:50.012486+0000","last_fullsized":"2024-04-24T23:21:50.012486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:19.499255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.998970+0000","last_change":"2024-04-24T23:21:49.024500+0000","last_active":"2024-04-24T23:21:49.998970+0000","last_peered":"2024-04-24T23:21:49.998970+0000","last_clean":"2024-04-24T23:21:49.998970+0000","last_became_active":"2024-04-24T23:21:49.024216+0000","last_became_peered":"2024-04-24T23:21:49.024216+0000","last_unstale":"2024-04-24T23:21:49.998970+0000","last_undegraded":"2024-04-24T23:21:49.998970+0000","last_fullsized":"2024-04-24T23:21:49.998970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:34.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407693+0000","last_change":"2024-04-24T23:21:47.967855+0000","last_active":"2024-04-24T23:21:50.407693+0000","last_peered":"2024-04-24T23:21:50.407693+0000","last_clean":"2024-04-24T23:21:50.407693+0000","last_became_active":"2024-04-24T23:21:47.967346+0000","last_became_peered":"2024-04-24T23:21:47.967346+0000","last_unstale":"2024-04-24T23:21:50.407693+0000","last_undegraded":"2024-04-24T23:21:50.407693+0000","last_fullsized":"2024-04-24T23:21:50.407693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:35.773731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407531+0000","last_change":"2024-04-24T23:21:49.013579+0000","last_active":"2024-04-24T23:21:50.407531+0000","last_peered":"2024-04-24T23:21:50.407531+0000","last_clean":"2024-04-24T23:21:50.407531+0000","last_became_active":"2024-04-24T23:21:49.013343+0000","last_became_peered":"2024-04-24T23:21:49.013343+0000","last_unstale":"2024-04-24T23:21:50.407531+0000","last_undegraded":"2024-04-24T23:21:50.407531+0000","last_fullsized":"2024-04-24T23:21:50.407531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:54.358315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:49.998917+0000","last_change":"2024-04-24T23:21:47.975444+0000","last_active":"2024-04-24T23:21:49.998917+0000","last_peered":"2024-04-24T23:21:49.998917+0000","last_clean":"2024-04-24T23:21:49.998917+0000","last_became_active":"2024-04-24T23:21:47.975142+0000","last_became_peered":"2024-04-24T23:21:47.975142+0000","last_unstale":"2024-04-24T23:21:49.998917+0000","last_undegraded":"2024-04-24T23:21:49.998917+0000","last_fullsized":"2024-04-24T23:21:49.998917+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:50.432741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407990+0000","last_change":"2024-04-24T23:21:49.015009+0000","last_active":"2024-04-24T23:21:50.407990+0000","last_peered":"2024-04-24T23:21:50.407990+0000","last_clean":"2024-04-24T23:21:50.407990+0000","last_became_active":"2024-04-24T23:21:49.014723+0000","last_became_peered":"2024-04-24T23:21:49.014723+0000","last_unstale":"2024-04-24T23:21:50.407990+0000","last_undegraded":"2024-04-24T23:21:50.407990+0000","last_fullsized":"2024-04-24T23:21:50.407990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:47.953412+0000","last_clean_scrub_stamp":"2024-04-24T23:21:47.953412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:34.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407088+0000","last_change":"2024-04-24T23:21:47.972974+0000","last_active":"2024-04-24T23:21:50.407088+0000","last_peered":"2024-04-24T23:21:50.407088+0000","last_clean":"2024-04-24T23:21:50.407088+0000","last_became_active":"2024-04-24T23:21:47.972651+0000","last_became_peered":"2024-04-24T23:21:47.972651+0000","last_unstale":"2024-04-24T23:21:50.407088+0000","last_undegraded":"2024-04-24T23:21:50.407088+0000","last_fullsized":"2024-04-24T23:21:50.407088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:21:46.951440+0000","last_clean_scrub_stamp":"2024-04-24T23:21:46.951440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:37.844221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T23:21:50.407864+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:21:50.407864+0000","last_peered":"2024-04-24T23:21:50.407864+0000","last_clean":"2024-04-24T23:21:50.407864+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:21:50.407864+0000","last_undegraded":"2024-04-24T23:21:50.407864+0000","last_fullsized":"2024-04-24T23:21:50.407864+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":737066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":147224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":147280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35708,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336132,"statfs":{"total":96636764160,"available":96600199168,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:21:53.364 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:21:53.365 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:21:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.507+0000 7f8ecd415640 1 Processor -- start 2024-04-24T23:21:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.507+0000 7f8ecd415640 1 -- start start 2024-04-24T23:21:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.508+0000 7f8ecd415640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ec80d0750 0x7f8ec8165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.508+0000 7f8ecd415640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec815ef60 con 0x7f8ec80ad7b0 2024-04-24T23:21:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.508+0000 7f8ecd415640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec815f240 con 0x7f8ec815adc0 2024-04-24T23:21:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.508+0000 7f8ecd415640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec815f520 con 0x7f8ec80d0750 2024-04-24T23:21:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.508+0000 7f8ec67fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8ec80ad7b0 0x7f8ec80d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56728/0 (socket says 172.21.15.26:56728) 2024-04-24T23:21:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.508+0000 7f8ec67fc640 1 -- 172.21.15.26:0/2826487039 learned_addr learned my addr 172.21.15.26:0/2826487039 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.508+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1132899415 0 0) 0x7f8ec815ef60 con 0x7f8ec80ad7b0 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.508+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8eb0003520 con 0x7f8ec80ad7b0 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.508+0000 7f8ec77fe640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ec80d0750 0x7f8ec8165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1663663308 0 0) 0x7f8eb0003520 con 0x7f8ec80ad7b0 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ec80d0750 msgr2=0x7f8ec8165d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ec80d0750 0x7f8ec8165d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 >> v1:172.21.15.29:6789/0 conn(0x7f8ec815adc0 legacy=0x7f8ec80b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec815f800 con 0x7f8ec80ad7b0 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f8ebc001670 con 0x7f8ec80ad7b0 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 >> v1:172.21.15.26:6789/0 conn(0x7f8ec80ad7b0 legacy=0x7f8ec80d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ec80d0750 0x7f8eb00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb0003f20 0x7f8eb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8eb0004d40 0x7f8eb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec815f520 con 0x7f8eb0003f20 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb000bad0 con 0x7f8eb0004d40 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb000dee0 con 0x7f8ec80d0750 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec67fc640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb0003f20 0x7f8eb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec6ffd640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ec80d0750 0x7f8eb00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec77fe640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8eb0004d40 0x7f8eb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec67fc640 1 -- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ec80d0750 msgr2=0x7f8eb00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec67fc640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ec80d0750 0x7f8eb00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec67fc640 1 -- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8eb0004d40 msgr2=0x7f8eb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec67fc640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8eb0004d40 0x7f8eb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.509+0000 7f8ec67fc640 1 -- 172.21.15.26:0/2826487039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb0003520 con 0x7f8eb0003f20 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.510+0000 7f8ec67fc640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb0003f20 0x7f8eb0004800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c210 tx=0x7f8eb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ec2e3afdefe0eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.510+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc007d10 con 0x7f8eb0003f20 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.510+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc005ad0 con 0x7f8eb0003f20 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.510+0000 7f8ec5ffb640 1 -- 172.21.15.26:0/2826487039 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc007070 con 0x7f8eb0003f20 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.510+0000 7f8ecd415640 1 -- 172.21.15.26:0/2826487039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb0003f20 msgr2=0x7f8eb0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.510+0000 7f8ecd415640 1 --2- 172.21.15.26:0/2826487039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8eb0003f20 0x7f8eb0004800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c210 tx=0x7f8eb0005830 comp rx=0 tx=0).stop 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.511+0000 7f8ec77fe640 1 -- 172.21.15.26:0/2826487039 reap_dead start 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.511+0000 7f8ecd415640 1 -- 172.21.15.26:0/2826487039 shutdown_connections 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.511+0000 7f8ecd415640 1 -- 172.21.15.26:0/2826487039 >> 172.21.15.26:0/2826487039 conn(0x7f8ec806b540 msgr2=0x7f8ec806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.511+0000 7f8ecd415640 1 -- 172.21.15.26:0/2826487039 shutdown_connections 2024-04-24T23:21:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.511+0000 7f8ecd415640 1 -- 172.21.15.26:0/2826487039 wait complete. 2024-04-24T23:21:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.512+0000 7f8ecd415640 1 Processor -- start 2024-04-24T23:21:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.512+0000 7f8ecd415640 1 -- start start 2024-04-24T23:21:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ecd415640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8eb0003f20 0x7f8ec80b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ecd415640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec814b760 0x7f8ec8072490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ecd415640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ec80729d0 0x7f8ec815c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ecd415640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec815e9c0 con 0x7f8ec814b760 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ecd415640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec815b1a0 con 0x7f8ec80729d0 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ecd415640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec815eb40 con 0x7f8eb0003f20 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec814b760 0x7f8ec8072490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec814b760 0x7f8ec8072490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47048/0 (socket says 172.21.15.26:47048) 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec67fc640 1 -- 172.21.15.26:0/3112993616 learned_addr learned my addr 172.21.15.26:0/3112993616 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec6ffd640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8eb0003f20 0x7f8ec80b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec77fe640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ec80729d0 0x7f8ec815c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec67fc640 1 -- 172.21.15.26:0/3112993616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8eb0003f20 msgr2=0x7f8ec80b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec67fc640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8eb0003f20 0x7f8ec80b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec67fc640 1 -- 172.21.15.26:0/3112993616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ec80729d0 msgr2=0x7f8ec815c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec67fc640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ec80729d0 0x7f8ec815c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec67fc640 1 -- 172.21.15.26:0/3112993616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec815d060 con 0x7f8ec814b760 2024-04-24T23:21:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec6ffd640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8eb0003f20 0x7f8ec80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8ec67fc640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec814b760 0x7f8ec8072490 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c210 tx=0x7f8ec815f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8eaf7fe640 1 -- 172.21.15.26:0/3112993616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc041020 con 0x7f8ec814b760 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.513+0000 7f8eaf7fe640 1 -- 172.21.15.26:0/3112993616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc001430 con 0x7f8ec814b760 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.514+0000 7f8eaf7fe640 1 -- 172.21.15.26:0/3112993616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc005930 con 0x7f8ec814b760 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.514+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec815d340 con 0x7f8ec814b760 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.514+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec809ae80 con 0x7f8ec814b760 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.515+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec8075610 con 0x7f8ec814b760 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.515+0000 7f8eaf7fe640 1 -- 172.21.15.26:0/3112993616 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ebc044020 con 0x7f8ec814b760 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.515+0000 7f8eaf7fe640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8eb406be30 0x7f8eb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.516+0000 7f8eaf7fe640 1 -- 172.21.15.26:0/3112993616 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8ebc0ab010 con 0x7f8ec814b760 2024-04-24T23:21:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.516+0000 7f8ec6ffd640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8eb406be30 0x7f8eb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.516+0000 7f8ec6ffd640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8eb406be30 0x7f8eb406e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8003700 tx=0x7f8eb8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.520+0000 7f8eaf7fe640 1 -- 172.21.15.26:0/3112993616 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ebc079a50 con 0x7f8ec814b760 2024-04-24T23:21:53.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.772+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8ec8072db0 con 0x7f8ec814b760 2024-04-24T23:21:53.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.774+0000 7f8eaf7fe640 1 -- 172.21.15.26:0/3112993616 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f8ebc079350 con 0x7f8ec814b760 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ== 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-04-24T23:21:53.776 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:21:53.777 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-04-24T23:21:53.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8eb406be30 msgr2=0x7f8eb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8eb406be30 0x7f8eb406e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8003700 tx=0x7f8eb8009040 comp rx=0 tx=0).stop 2024-04-24T23:21:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec814b760 msgr2=0x7f8ec8072490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec814b760 0x7f8ec8072490 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c210 tx=0x7f8ec815f410 comp rx=0 tx=0).stop 2024-04-24T23:21:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 shutdown_connections 2024-04-24T23:21:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8eb406be30 0x7f8eb406e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ec80729d0 0x7f8ec815c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec814b760 0x7f8ec8072490 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 --2- 172.21.15.26:0/3112993616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8eb0003f20 0x7f8ec80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 >> 172.21.15.26:0/3112993616 conn(0x7f8ec806b540 msgr2=0x7f8ec80ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 shutdown_connections 2024-04-24T23:21:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.776+0000 7f8ecd415640 1 -- 172.21.15.26:0/3112993616 wait complete. 2024-04-24T23:21:53.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.889+0000 7f97c24ee640 1 Processor -- start 2024-04-24T23:21:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.889+0000 7f97c24ee640 1 -- start start 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c24ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 0x7f97bc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c24ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97bc0cd660 0x7f97bc0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c24ee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97bc0cfbc0 con 0x7f97bc145910 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c24ee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f97bc0cdff0 con 0x7f97bc0cd660 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c24ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc145cf0 con 0x7f97bc06d350 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c0a8b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97bc0cd660 0x7f97bc0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c0a8b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97bc0cd660 0x7f97bc0cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36910/0 (socket says 172.21.15.26:36910) 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97bbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f97bc145910 0x7f97bc15d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46804/0 (socket says 172.21.15.26:46804) 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c0a8b640 1 -- 172.21.15.26:0/3667537806 learned_addr learned my addr 172.21.15.26:0/3667537806 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97bb7fe640 1 --2- 172.21.15.26:0/3667537806 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 0x7f97bc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c0a8b640 1 -- 172.21.15.26:0/3667537806 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 msgr2=0x7f97bc06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c0a8b640 1 --2- 172.21.15.26:0/3667537806 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 0x7f97bc06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c0a8b640 1 -- 172.21.15.26:0/3667537806 >> v1:172.21.15.26:6789/0 conn(0x7f97bc145910 legacy=0x7f97bc15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97baffd640 1 -- 172.21.15.26:0/3667537806 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3468951747 0 0) 0x7f97bc0cfbc0 con 0x7f97bc145910 2024-04-24T23:21:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.890+0000 7f97c0a8b640 1 -- 172.21.15.26:0/3667537806 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc06d7f0 con 0x7f97bc0cd660 2024-04-24T23:21:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.891+0000 7f97c0a8b640 1 --2- 172.21.15.26:0/3667537806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc0cd660 0x7f97bc0cdab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009e50 tx=0x7f97b402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b93647f6509afe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.891+0000 7f97baffd640 1 -- 172.21.15.26:0/3667537806 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4036070 con 0x7f97bc0cd660 2024-04-24T23:21:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.891+0000 7f97baffd640 1 -- 172.21.15.26:0/3667537806 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b4031600 con 0x7f97bc0cd660 2024-04-24T23:21:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.891+0000 7f97baffd640 1 -- 172.21.15.26:0/3667537806 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40319d0 con 0x7f97bc0cd660 2024-04-24T23:21:53.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.892+0000 7f97c24ee640 1 -- 172.21.15.26:0/3667537806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc0cd660 msgr2=0x7f97bc0cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.892+0000 7f97c24ee640 1 --2- 172.21.15.26:0/3667537806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc0cd660 0x7f97bc0cdab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009e50 tx=0x7f97b402a710 comp rx=0 tx=0).stop 2024-04-24T23:21:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.892+0000 7f97c24ee640 1 -- 172.21.15.26:0/3667537806 shutdown_connections 2024-04-24T23:21:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.892+0000 7f97c24ee640 1 --2- 172.21.15.26:0/3667537806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc0cd660 0x7f97bc0cdab0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.892+0000 7f97c24ee640 1 --2- 172.21.15.26:0/3667537806 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 0x7f97bc06e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.892+0000 7f97c24ee640 1 -- 172.21.15.26:0/3667537806 >> 172.21.15.26:0/3667537806 conn(0x7f97bc06b300 msgr2=0x7f97bc06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.892+0000 7f97c24ee640 1 -- 172.21.15.26:0/3667537806 shutdown_connections 2024-04-24T23:21:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.892+0000 7f97c24ee640 1 -- 172.21.15.26:0/3667537806 wait complete. 2024-04-24T23:21:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.893+0000 7f97c24ee640 1 Processor -- start 2024-04-24T23:21:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.893+0000 7f97c24ee640 1 -- start start 2024-04-24T23:21:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c24ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 0x7f97bc1396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c24ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97bc0cd660 0x7f97bc13ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c24ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc145910 0x7f97bc13b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c24ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc06e090 con 0x7f97bc145910 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c24ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc06ddb0 con 0x7f97bc0cd660 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c24ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc06e370 con 0x7f97bc06d350 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c0a8b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc145910 0x7f97bc13b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c0a8b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc145910 0x7f97bc13b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36926/0 (socket says 172.21.15.26:36926) 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c0a8b640 1 -- 172.21.15.26:0/67566999 learned_addr learned my addr 172.21.15.26:0/67566999 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97bbfff640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 0x7f97bc1396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97bb7fe640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97bc0cd660 0x7f97bc13ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c0a8b640 1 -- 172.21.15.26:0/67566999 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 msgr2=0x7f97bc1396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c0a8b640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 0x7f97bc1396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c0a8b640 1 -- 172.21.15.26:0/67566999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97bc0cd660 msgr2=0x7f97bc13ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c0a8b640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97bc0cd660 0x7f97bc13ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.894+0000 7f97c0a8b640 1 -- 172.21.15.26:0/67566999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc15a790 con 0x7f97bc145910 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.895+0000 7f97c0a8b640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc145910 0x7f97bc13b1b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f97b4034040 tx=0x7f97b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.895+0000 7f97b8ff9640 1 -- 172.21.15.26:0/67566999 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4036050 con 0x7f97bc145910 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.895+0000 7f97b8ff9640 1 -- 172.21.15.26:0/67566999 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b4042070 con 0x7f97bc145910 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.895+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc15cfc0 con 0x7f97bc145910 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.895+0000 7f97b8ff9640 1 -- 172.21.15.26:0/67566999 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4009510 con 0x7f97bc145910 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.895+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc15d580 con 0x7f97bc145910 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.896+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc0694c0 con 0x7f97bc145910 2024-04-24T23:21:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.896+0000 7f97b8ff9640 1 -- 172.21.15.26:0/67566999 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97b4007bd0 con 0x7f97bc145910 2024-04-24T23:21:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.897+0000 7f97b8ff9640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f979c06be30 0x7f979c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.897+0000 7f97b8ff9640 1 -- 172.21.15.26:0/67566999 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f97b40abb40 con 0x7f97bc145910 2024-04-24T23:21:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.897+0000 7f97bbfff640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f979c06be30 0x7f979c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.897+0000 7f97bbfff640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f979c06be30 0x7f979c06e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007fd0 tx=0x7f97a4007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:53.901+0000 7f97b8ff9640 1 -- 172.21.15.26:0/67566999 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f97b4079f30 con 0x7f97bc145910 2024-04-24T23:21:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.122+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97bc139be0 con 0x7f97bc145910 2024-04-24T23:21:54.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.122+0000 7f97b8ff9640 1 -- 172.21.15.26:0/67566999 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f97b407e230 con 0x7f97bc145910 2024-04-24T23:21:54.124 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:54.125 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2201738459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2201738459},{"type":"v1","addr":"172.21.15.29:6834","nonce":2201738459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3972561683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3972561683},{"type":"v1","addr":"172.21.15.29:6835","nonce":3972561683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:21:48.324889+0000","modified":"2024-04-24T23:21:51.371127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/315291888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":315291888},{"type":"v1","addr":"172.21.15.26:6836","nonce":315291888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-04-24T23:21:54.125 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 16 2024-04-24T23:21:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.124+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f979c06be30 msgr2=0x7f979c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.124+0000 7f97c24ee640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f979c06be30 0x7f979c06e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007fd0 tx=0x7f97a4007de0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.124+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc145910 msgr2=0x7f97bc13b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.124+0000 7f97c24ee640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc145910 0x7f97bc13b1b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f97b4034040 tx=0x7f97b4004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.125+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 shutdown_connections 2024-04-24T23:21:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.125+0000 7f97c24ee640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f979c06be30 0x7f979c06e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.125+0000 7f97c24ee640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97bc145910 0x7f97bc13b1b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.125+0000 7f97c24ee640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97bc0cd660 0x7f97bc13ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.125+0000 7f97c24ee640 1 --2- 172.21.15.26:0/67566999 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97bc06d350 0x7f97bc1396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.125+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 >> 172.21.15.26:0/67566999 conn(0x7f97bc06b300 msgr2=0x7f97bc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:54.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.125+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 shutdown_connections 2024-04-24T23:21:54.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.125+0000 7f97c24ee640 1 -- 172.21.15.26:0/67566999 wait complete. 2024-04-24T23:21:54.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:21:48.324889+0000', 'modified': '2024-04-24T23:21:51.371127+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5088}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5088': {'gid': 5088, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6836/315291888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 315291888}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 315291888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5088, 'qdb_cluster': [5088]} 2024-04-24T23:21:54.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:21:54.540 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:21:54.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.643+0000 7fd9b2d7e640 1 Processor -- start 2024-04-24T23:21:54.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.643+0000 7fd9b2d7e640 1 -- start start 2024-04-24T23:21:54.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.644+0000 7fd9b2d7e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9ac15e570 0x7fd9ac0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.644+0000 7fd9b2d7e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9ac0d22e0 0x7fd9ac0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.644+0000 7fd9b2d7e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9ac0d32f0 con 0x7fd9ac0b3130 2024-04-24T23:21:54.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.644+0000 7fd9b2d7e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ac0d1ab0 con 0x7fd9ac15e570 2024-04-24T23:21:54.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.644+0000 7fd9b2d7e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ac0d2ae0 con 0x7fd9ac0d22e0 2024-04-24T23:21:54.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.644+0000 7fd9b0b1a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9ac15e570 0x7fd9ac0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.644+0000 7fd9b0b1a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9ac15e570 0x7fd9ac0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36932/0 (socket says 172.21.15.26:36932) 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.644+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1967991401 learned_addr learned my addr 172.21.15.26:0/1967991401 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.645+0000 7fd9b131b640 1 --2- 172.21.15.26:0/1967991401 >> v2:172.21.15.29:3300/0 conn(0x7fd9ac0d22e0 0x7fd9ac0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.645+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1967991401 >> v2:172.21.15.29:3300/0 conn(0x7fd9ac0d22e0 msgr2=0x7fd9ac0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.645+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/1967991401 >> v2:172.21.15.29:3300/0 conn(0x7fd9ac0d22e0 0x7fd9ac0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.645+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1967991401 >> v1:172.21.15.29:6789/0 conn(0x7fd9ac0b3130 legacy=0x7fd9ac0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.645+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1967991401 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac0d0270 con 0x7fd9ac15e570 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.645+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/1967991401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 0x7fd9ac0d4160 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0b6d60 tx=0x7fd9a402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94119453cc35e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9ab7fe640 1 -- 172.21.15.26:0/1967991401 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4036070 con 0x7fd9ac15e570 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9ab7fe640 1 -- 172.21.15.26:0/1967991401 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a4031600 con 0x7fd9ac15e570 2024-04-24T23:21:54.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9ab7fe640 1 -- 172.21.15.26:0/1967991401 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40319d0 con 0x7fd9ac15e570 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/1967991401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 msgr2=0x7fd9ac0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9b2d7e640 1 --2- 172.21.15.26:0/1967991401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 0x7fd9ac0d4160 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0b6d60 tx=0x7fd9a402a730 comp rx=0 tx=0).stop 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/1967991401 shutdown_connections 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9b2d7e640 1 --2- 172.21.15.26:0/1967991401 >> v2:172.21.15.29:3300/0 conn(0x7fd9ac0d22e0 0x7fd9ac0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9b2d7e640 1 --2- 172.21.15.26:0/1967991401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 0x7fd9ac0d4160 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/1967991401 >> 172.21.15.26:0/1967991401 conn(0x7fd9ac08dbe0 msgr2=0x7fd9ac09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/1967991401 shutdown_connections 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.646+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/1967991401 wait complete. 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.647+0000 7fd9b2d7e640 1 Processor -- start 2024-04-24T23:21:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.647+0000 7fd9b2d7e640 1 -- start start 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.648+0000 7fd9b2d7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0b3130 0x7fd9ac0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.648+0000 7fd9b2d7e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9ac0d22e0 0x7fd9ac150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.648+0000 7fd9b2d7e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 0x7fd9ac0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.648+0000 7fd9b2d7e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac0cfa70 con 0x7fd9ac15e570 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.648+0000 7fd9b2d7e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac0d3010 con 0x7fd9ac0b3130 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.648+0000 7fd9b2d7e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ac0cfd50 con 0x7fd9ac0d22e0 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.648+0000 7fd9b131b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 0x7fd9ac0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.648+0000 7fd9b131b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 0x7fd9ac0cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36938/0 (socket says 172.21.15.26:36938) 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.648+0000 7fd9b131b640 1 -- 172.21.15.26:0/682902576 learned_addr learned my addr 172.21.15.26:0/682902576 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0b3130 0x7fd9ac0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9abfff640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9ac0d22e0 0x7fd9ac150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9b131b640 1 -- 172.21.15.26:0/682902576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9ac0d22e0 msgr2=0x7fd9ac150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9b131b640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9ac0d22e0 0x7fd9ac150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9b131b640 1 -- 172.21.15.26:0/682902576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0b3130 msgr2=0x7fd9ac0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9b131b640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0b3130 0x7fd9ac0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9b131b640 1 -- 172.21.15.26:0/682902576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac15aa90 con 0x7fd9ac15e570 2024-04-24T23:21:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9abfff640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9ac0d22e0 0x7fd9ac150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9b131b640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 0x7fd9ac0cb2a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00d800 tx=0x7fd99c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9a97fa640 1 -- 172.21.15.26:0/682902576 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00b6c0 con 0x7fd9ac15e570 2024-04-24T23:21:54.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9a97fa640 1 -- 172.21.15.26:0/682902576 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99c019070 con 0x7fd9ac15e570 2024-04-24T23:21:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9a97fa640 1 -- 172.21.15.26:0/682902576 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00bac0 con 0x7fd9ac15e570 2024-04-24T23:21:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/682902576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac15ad70 con 0x7fd9ac15e570 2024-04-24T23:21:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.649+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/682902576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac15b330 con 0x7fd9ac15e570 2024-04-24T23:21:54.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.651+0000 7fd9a97fa640 1 -- 172.21.15.26:0/682902576 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd99c00bc60 con 0x7fd9ac15e570 2024-04-24T23:21:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.651+0000 7fd9a97fa640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98c06be30 0x7fd98c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.651+0000 7fd9a97fa640 1 -- 172.21.15.26:0/682902576 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd99c08b270 con 0x7fd9ac15e570 2024-04-24T23:21:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.651+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98c06be30 0x7fd98c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.652+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98c06be30 0x7fd98c06e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4033000 tx=0x7fd9a4009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.652+0000 7fd9b5b6a740 1 Processor -- start 2024-04-24T23:21:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.652+0000 7fd9b5b6a740 1 -- start start 2024-04-24T23:21:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a99c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9cc540 0x55a50a9a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a50a904f00 con 0x55a50a9e9b60 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a50a8ad750 con 0x55a50a9cc540 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a50a900ec0 con 0x55a50a9ccc80 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b0b1a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a99c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b0b1a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a99c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36950/0 (socket says 172.21.15.26:36950) 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1438183761 learned_addr learned my addr 172.21.15.26:0/1438183761 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9abfff640 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b131b640 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9cc540 0x55a50a9a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 msgr2=0x55a50a9cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9cc540 msgr2=0x55a50a9a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9cc540 0x55a50a9a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1438183761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50a6192d0 con 0x55a50a9e9b60 2024-04-24T23:21:54.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b131b640 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9cc540 0x55a50a9a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:54.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9abfff640 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:54.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.653+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a99c4f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009f10 tx=0x7fd9a402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b36ee7479e8f5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1438183761 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4036040 con 0x55a50a9e9b60 2024-04-24T23:21:54.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1438183761 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a404b070 con 0x55a50a9e9b60 2024-04-24T23:21:54.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1438183761 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a403cb40 con 0x55a50a9e9b60 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1438183761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 msgr2=0x55a50a99c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a99c4f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009f10 tx=0x7fd9a402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1438183761 shutdown_connections 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9cc540 0x55a50a9a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/1438183761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a99c4f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1438183761 >> 172.21.15.26:0/1438183761 conn(0x55a50a99e760 msgr2=0x55a50a9e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1438183761 shutdown_connections 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.654+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1438183761 wait complete. 2024-04-24T23:21:54.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.655+0000 7fd9b5b6a740 1 -- start start 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.655+0000 7fd9b5b6a740 1 Processor -- start 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9b99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9e8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a50a9a0910 con 0x55a50a9e9b60 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a50a8c0cf0 con 0x55a50a9ddcc0 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a50a99fd00 con 0x55a50a9ccc80 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9b5b6a740 10 client.0 _ll_register_callbacks cb 0x55a50a8c06e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9ba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36966/0 (socket says 172.21.15.26:36966) 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9abfff640 1 -- 172.21.15.26:0/752616444 learned_addr learned my addr 172.21.15.26:0/752616444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9b99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9b131b640 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9e8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9abfff640 1 -- 172.21.15.26:0/752616444 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 msgr2=0x55a50a9b99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9abfff640 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9b99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9abfff640 1 -- 172.21.15.26:0/752616444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 msgr2=0x55a50a9e8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9abfff640 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9e8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9abfff640 1 -- 172.21.15.26:0/752616444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50a6e36b0 con 0x55a50a9e9b60 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd9abfff640 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9ba140 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd994001e50 tx=0x7fd99400e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3017315544f3b7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.656+0000 7fd98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd98b7fe640 1 -- 172.21.15.26:0/752616444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9940052d0 con 0x55a50a9e9b60 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd98b7fe640 1 -- 172.21.15.26:0/752616444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd9940035f0 con 0x55a50a9e9b60 2024-04-24T23:21:54.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/752616444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a404b070 con 0x55a50a9e9b60 2024-04-24T23:21:54.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd98b7fe640 1 -- 172.21.15.26:0/752616444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fd99401f050 con 0x55a50a9e9b60 2024-04-24T23:21:54.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b5b6a740 10 client.5259 fetch_fsmap learned FSMap version 16 2024-04-24T23:21:54.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/752616444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a50a9ded40 con 0x55a50a9e9b60 2024-04-24T23:21:54.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd98b7fe640 1 -- 172.21.15.26:0/752616444 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a404b070 con 0x55a50a9e9b60 2024-04-24T23:21:54.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd98b7fe640 1 -- 172.21.15.26:0/752616444 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fd994022020 con 0x55a50a9e9b60 2024-04-24T23:21:54.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b5b6a740 10 client.5259 fetch_fsmap finished waiting for FSMap version 16 2024-04-24T23:21:54.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b5b6a740 10 client.5259 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:21:54.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b5b6a740 20 client.5259 populate_metadata read hostname 'smithi026' 2024-04-24T23:21:54.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55a50a9df540 0x55a50a9cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b5b6a740 4 client.5259 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:54.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/752616444 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a50a912440 con 0x55a50a9df540 2024-04-24T23:21:54.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.657+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55a50a9df540 0x55a50a9cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd98b7fe640 10 client.5259 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:21:54.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55a50a9df540 0x55a50a9cd630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99977a62e9a9bcb server_cookie=45fd4f435ac980c1 in_seq=0 out_seq=0 2024-04-24T23:21:54.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd98b7fe640 1 -- 172.21.15.26:0/752616444 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a50a912440 con 0x55a50a9df540 2024-04-24T23:21:54.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd98b7fe640 10 client.5259 handle_command_reply: tid=0 2024-04-24T23:21:54.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 client.5259 shutdown 2024-04-24T23:21:54.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/752616444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 msgr2=0x55a50a9ba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9ba140 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd994001e50 tx=0x7fd99400e650 comp rx=0 tx=0).stop 2024-04-24T23:21:54.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/752616444 shutdown_connections 2024-04-24T23:21:54.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55a50a9df540 0x55a50a9cd630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9e8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9ba140 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd98b7fe640 0 client.5259 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:21:54.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/752616444 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9b99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/752616444 >> 172.21.15.26:0/752616444 conn(0x55a50a9cc540 msgr2=0x55a50a99d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:54.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/752616444 shutdown_connections 2024-04-24T23:21:54.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.658+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/752616444 wait complete. 2024-04-24T23:21:54.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.659+0000 7fd9b5b6a740 20 client.5259 trim_cache size 0 max 16384 2024-04-24T23:21:54.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b5b6a740 1 Processor -- start 2024-04-24T23:21:54.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b5b6a740 1 -- start start 2024-04-24T23:21:54.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9ccc80 0x55a50aa00eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9e9b60 0x55a50a99d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac159cd0 con 0x55a50a9ccc80 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac159060 con 0x55a50a9ddcc0 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ac159680 con 0x55a50a9e9b60 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b0b1a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9ccc80 0x55a50aa00eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b0b1a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9ccc80 0x55a50aa00eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36982/0 (socket says 172.21.15.26:36982) 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.677+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1889514119 learned_addr learned my addr 172.21.15.26:0/1889514119 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1889514119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9e9b60 msgr2=0x55a50a99d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/1889514119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9e9b60 0x55a50a99d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1889514119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 msgr2=0x55a50a9b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/1889514119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9b0b1a640 1 -- 172.21.15.26:0/1889514119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50a99f8a0 con 0x55a50a9ccc80 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/1889514119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9ccc80 0x55a50aa00eb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009f40 tx=0x7fd9a403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447ce65cb5fd3916 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9a1ffb640 1 -- 172.21.15.26:0/1889514119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4036040 con 0x55a50a9ccc80 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9a1ffb640 1 -- 172.21.15.26:0/1889514119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a404b070 con 0x55a50a9ccc80 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9a1ffb640 1 -- 172.21.15.26:0/1889514119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40571c0 con 0x55a50a9ccc80 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.678+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1889514119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9ccc80 msgr2=0x55a50aa00eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/1889514119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9ccc80 0x55a50aa00eb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009f40 tx=0x7fd9a403b000 comp rx=0 tx=0).stop 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1889514119 shutdown_connections 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/1889514119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9e9b60 0x55a50a99d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/1889514119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/1889514119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9ccc80 0x55a50aa00eb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1889514119 >> 172.21.15.26:0/1889514119 conn(0x55a50a9cc540 msgr2=0x55a50a9e6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1889514119 shutdown_connections 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/1889514119 wait complete. 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 -- start start 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.679+0000 7fd9b5b6a740 1 Processor -- start 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b5b6a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a50a9e8ba0 con 0x55a50a9e9b60 2024-04-24T23:21:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a50a9e8750 con 0x55a50a9ddcc0 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b5b6a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a50a9df990 con 0x55a50a9ccc80 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b5b6a740 10 client.0 _ll_register_callbacks cb 0x7fd9940019d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b131b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b131b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9b97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36994/0 (socket says 172.21.15.26:36994) 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b131b640 1 -- 172.21.15.26:0/2363487961 learned_addr learned my addr 172.21.15.26:0/2363487961 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9abfff640 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.680+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b131b640 1 -- 172.21.15.26:0/2363487961 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 msgr2=0x55a50a9b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b131b640 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b131b640 1 -- 172.21.15.26:0/2363487961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 msgr2=0x55a50a9e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b131b640 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9e5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b131b640 1 -- 172.21.15.26:0/2363487961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50a9ead00 con 0x55a50a9e9b60 2024-04-24T23:21:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b131b640 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9b97b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd99c006fd0 tx=0x7fd99c059d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4a4432ec5f27ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd98b7fe640 1 -- 172.21.15.26:0/2363487961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c062100 con 0x55a50a9e9b60 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd98b7fe640 1 -- 172.21.15.26:0/2363487961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd99c055b90 con 0x55a50a9e9b60 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/2363487961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a404b070 con 0x55a50a9e9b60 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd98b7fe640 1 -- 172.21.15.26:0/2363487961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fd99c0893d0 con 0x55a50a9e9b60 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b5b6a740 10 client.5265 fetch_fsmap learned FSMap version 16 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/2363487961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a50aa00590 con 0x55a50a9e9b60 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd98b7fe640 1 -- 172.21.15.26:0/2363487961 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a404b070 con 0x55a50a9e9b60 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd98b7fe640 1 -- 172.21.15.26:0/2363487961 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fd99c013070 con 0x55a50a9e9b60 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b5b6a740 10 client.5265 fetch_fsmap finished waiting for FSMap version 16 2024-04-24T23:21:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b5b6a740 10 client.5265 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b5b6a740 20 client.5265 populate_metadata read hostname 'smithi026' 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55a50a9c6840 0x55a50a9c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b5b6a740 4 client.5265 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/2363487961 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "status"}) -- 0x55a50a90c790 con 0x55a50a9c6840 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.681+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55a50a9c6840 0x55a50a9c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd98b7fe640 10 client.5265 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b0b1a640 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55a50a9c6840 0x55a50a9c6c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f6701bf29e7d31 server_cookie=842ff797419229c8 in_seq=0 out_seq=0 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd98b7fe640 1 -- 172.21.15.26:0/2363487961 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a50a90c790 con 0x55a50a9c6840 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd98b7fe640 10 client.5265 handle_command_reply: tid=0 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 client.5265 shutdown 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/2363487961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 msgr2=0x55a50a9b97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9b97b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd99c006fd0 tx=0x7fd99c059d60 comp rx=0 tx=0).stop 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/2363487961 shutdown_connections 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55a50a9c6840 0x55a50a9c6c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a50a9e9b60 0x55a50a9b97b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a50a9ddcc0 0x55a50a9e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd98b7fe640 0 client.5265 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 --2- 172.21.15.26:0/2363487961 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a50a9ccc80 0x55a50a9b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/2363487961 >> 172.21.15.26:0/2363487961 conn(0x55a50a9cc540 msgr2=0x55a50a9e7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/2363487961 shutdown_connections 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 1 -- 172.21.15.26:0/2363487961 wait complete. 2024-04-24T23:21:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.682+0000 7fd9b5b6a740 20 client.5265 trim_cache size 0 max 16384 2024-04-24T23:21:54.694 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:21:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:21:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:21:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5088, 2024-04-24T23:21:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:21:54.694 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:21:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:21:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.6804936250000004, 2024-04-24T23:21:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 16, 2024-04-24T23:21:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 33, 2024-04-24T23:21:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 33, 2024-04-24T23:21:54.695 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 13.630959110999999 2024-04-24T23:21:54.696 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:21:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98c06be30 msgr2=0x7fd98c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98c06be30 0x7fd98c06e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4033000 tx=0x7fd9a4009000 comp rx=0 tx=0).stop 2024-04-24T23:21:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 msgr2=0x7fd9ac0cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 0x7fd9ac0cb2a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00d800 tx=0x7fd99c00dcc0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/682902576 shutdown_connections 2024-04-24T23:21:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98c06be30 0x7fd98c06e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac15e570 0x7fd9ac0cb2a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9ac0d22e0 0x7fd9ac150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 --2- 172.21.15.26:0/682902576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9ac0b3130 0x7fd9ac0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/682902576 >> 172.21.15.26:0/682902576 conn(0x7fd9ac08dbe0 msgr2=0x7fd9ac0d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/682902576 shutdown_connections 2024-04-24T23:21:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.683+0000 7fd9b2d7e640 1 -- 172.21.15.26:0/682902576 wait complete. 2024-04-24T23:21:54.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:21:54.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.800+0000 7f580867d640 1 Processor -- start 2024-04-24T23:21:54.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.800+0000 7f580867d640 1 -- start start 2024-04-24T23:21:54.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.801+0000 7f580867d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58000d4870 0x7f58000b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.801+0000 7f580867d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58000b3130 0x7f58000b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.801+0000 7f580867d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58000d32f0 con 0x7f580015e570 2024-04-24T23:21:54.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.801+0000 7f580867d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f58000d1ab0 con 0x7f58000d4870 2024-04-24T23:21:54.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.801+0000 7f580867d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f58000d2ae0 con 0x7f58000b3130 2024-04-24T23:21:54.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5805c18640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58000d4870 0x7f58000b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5805c18640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58000d4870 0x7f58000b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37000/0 (socket says 172.21.15.26:37000) 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5805c18640 1 -- 172.21.15.26:0/3146583030 learned_addr learned my addr 172.21.15.26:0/3146583030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5806419640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f580015e570 0x7f58000d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46812/0 (socket says 172.21.15.26:46812) 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5806c1a640 1 --2- 172.21.15.26:0/3146583030 >> v2:172.21.15.29:3300/0 conn(0x7f58000b3130 0x7f58000b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5805c18640 1 -- 172.21.15.26:0/3146583030 >> v2:172.21.15.29:3300/0 conn(0x7f58000b3130 msgr2=0x7f58000b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5805c18640 1 --2- 172.21.15.26:0/3146583030 >> v2:172.21.15.29:3300/0 conn(0x7f58000b3130 0x7f58000b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5805c18640 1 -- 172.21.15.26:0/3146583030 >> v1:172.21.15.26:6789/0 conn(0x7f580015e570 legacy=0x7f58000d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5805c18640 1 -- 172.21.15.26:0/3146583030 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58000d0270 con 0x7f58000d4870 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.802+0000 7f5805417640 1 -- 172.21.15.26:0/3146583030 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1065289195 0 0) 0x7f57fc001110 con 0x7f580015e570 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f5805c18640 1 --2- 172.21.15.26:0/3146583030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000d4870 0x7f58000b4100 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f58000ba250 tx=0x7f57f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9674a0532bfa595 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f5805417640 1 -- 172.21.15.26:0/3146583030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0037070 con 0x7f58000d4870 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f5805417640 1 -- 172.21.15.26:0/3146583030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0032440 con 0x7f58000d4870 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f5805417640 1 -- 172.21.15.26:0/3146583030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0032810 con 0x7f58000d4870 2024-04-24T23:21:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f580867d640 1 -- 172.21.15.26:0/3146583030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000d4870 msgr2=0x7f58000b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f580867d640 1 --2- 172.21.15.26:0/3146583030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000d4870 0x7f58000b4100 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f58000ba250 tx=0x7f57f002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f580867d640 1 -- 172.21.15.26:0/3146583030 shutdown_connections 2024-04-24T23:21:54.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f580867d640 1 --2- 172.21.15.26:0/3146583030 >> v2:172.21.15.29:3300/0 conn(0x7f58000b3130 0x7f58000b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f580867d640 1 --2- 172.21.15.26:0/3146583030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000d4870 0x7f58000b4100 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f580867d640 1 -- 172.21.15.26:0/3146583030 >> 172.21.15.26:0/3146583030 conn(0x7f580008dbe0 msgr2=0x7f580009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:54.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f580867d640 1 -- 172.21.15.26:0/3146583030 shutdown_connections 2024-04-24T23:21:54.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.803+0000 7f580867d640 1 -- 172.21.15.26:0/3146583030 wait complete. 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f580867d640 1 Processor -- start 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f580867d640 1 -- start start 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f580867d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000b3130 0x7f58000cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f580867d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58000d4870 0x7f58001f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f580867d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f580015e570 0x7f5800152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f580867d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f58000cfa70 con 0x7f58000b3130 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f580867d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f58000d3010 con 0x7f58000d4870 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f580867d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f58000cfd50 con 0x7f580015e570 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f5806419640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000b3130 0x7f58000cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f5806419640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000b3130 0x7f58000cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37016/0 (socket says 172.21.15.26:37016) 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.805+0000 7f5806419640 1 -- 172.21.15.26:0/1251577263 learned_addr learned my addr 172.21.15.26:0/1251577263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f5805c18640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58000d4870 0x7f58001f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f5806c1a640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f580015e570 0x7f5800152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f5806419640 1 -- 172.21.15.26:0/1251577263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f580015e570 msgr2=0x7f5800152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f5806419640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f580015e570 0x7f5800152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f5806419640 1 -- 172.21.15.26:0/1251577263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58000d4870 msgr2=0x7f58001f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f5806419640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58000d4870 0x7f58001f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f5806419640 1 -- 172.21.15.26:0/1251577263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58001f4f40 con 0x7f58000b3130 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f5806419640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000b3130 0x7f58000cb2a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c430 tx=0x7f57fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f57f6ffd640 1 -- 172.21.15.26:0/1251577263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0074e0 con 0x7f58000b3130 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f57f6ffd640 1 -- 172.21.15.26:0/1251577263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57fc0047e0 con 0x7f58000b3130 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58000cb880 con 0x7f58000b3130 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f57f6ffd640 1 -- 172.21.15.26:0/1251577263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc004bd0 con 0x7f58000b3130 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.806+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58000cbe40 con 0x7f58000b3130 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.807+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58000694c0 con 0x7f58000b3130 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.808+0000 7f57f6ffd640 1 -- 172.21.15.26:0/1251577263 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57fc011070 con 0x7f58000b3130 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.808+0000 7f57f6ffd640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57e406be30 0x7f57e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.808+0000 7f57f6ffd640 1 -- 172.21.15.26:0/1251577263 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f57fc08a8e0 con 0x7f58000b3130 2024-04-24T23:21:54.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.808+0000 7f5805c18640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57e406be30 0x7f57e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:54.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.809+0000 7f5805c18640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57e406be30 0x7f57e406e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f57f0034000 tx=0x7f57f002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:54.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:54.812+0000 7f57f6ffd640 1 -- 172.21.15.26:0/1251577263 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58001f4f40 con 0x7f58000b3130 2024-04-24T23:21:55.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.041+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58000d0fd0 con 0x7f58000b3130 2024-04-24T23:21:55.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.041+0000 7f57f6ffd640 1 -- 172.21.15.26:0/1251577263 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f58000d32f0 con 0x7f58000b3130 2024-04-24T23:21:55.043 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:21:55.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57e406be30 msgr2=0x7f57e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:55.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57e406be30 0x7f57e406e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f57f0034000 tx=0x7f57f002e000 comp rx=0 tx=0).stop 2024-04-24T23:21:55.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000b3130 msgr2=0x7f58000cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:55.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000b3130 0x7f58000cb2a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c430 tx=0x7f57fc00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:21:55.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 shutdown_connections 2024-04-24T23:21:55.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57e406be30 0x7f57e406e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:55.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f580015e570 0x7f5800152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:55.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58000d4870 0x7f58001f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:55.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 --2- 172.21.15.26:0/1251577263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58000b3130 0x7f58000cb2a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:55.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 >> 172.21.15.26:0/1251577263 conn(0x7f580008dbe0 msgr2=0x7f58000b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:55.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 shutdown_connections 2024-04-24T23:21:55.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.043+0000 7f580867d640 1 -- 172.21.15.26:0/1251577263 wait complete. 2024-04-24T23:21:55.056 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:21:55.056 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:21:55.056 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-24T23:21:55.056 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:21:55.056 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:21:55.056 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:21:55.056 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:21:55.056 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:21:55.056 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:21:55.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:55.057 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-24T23:21:55.073 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42678sec preferred_lft 42678sec 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:21:55.074 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ccf2:1eff:fe9b:4749/64 scope link 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c2:ea:e0:db:00:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c0ea:e0ff:fedb:e0/64 scope link 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::1c82:d0ff:fefa:5b3f/64 scope link 2024-04-24T23:21:55.075 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:21:55.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:55.076 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:21:55.136 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-24T23:21:55.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:55.138 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-24T23:21:55.194 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-24T23:21:55.194 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:21:55.194 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:21:55.194 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:21:55.194 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:21:55.195 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:21:55.195 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:21:55.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:21:55.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:21:55.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:21:55.195 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:21:55.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:21:55.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:55.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:55.321 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:21:55.337 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:21:55.337 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:21:55.337 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:21:55.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:55.337 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-24T23:21:55.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:55.403 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:21:55.458 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:21:55.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:21:55.459 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:21:55.513 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:21:55.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:55.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:21:55.596 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:21:55.596 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:21:55.596 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:21:55.605 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:21:55.605 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:21:55.605 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:21:55.605 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:55.605 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:55.609 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:21:55.609 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:21:55.610 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:21:55.611 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:21:55.612 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:21:55.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:21:55.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:21:55.677 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:21:55.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:21:55.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:21:55.778 INFO:teuthology.orchestra.run.smithi026.stdout:client5277 2024-04-24T23:21:55.779 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:21:55.779 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:21:55.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T23:21:55.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.937+0000 7f208bb9e640 1 Processor -- start 2024-04-24T23:21:55.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.937+0000 7f208bb9e640 1 -- start start 2024-04-24T23:21:55.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.938+0000 7f208bb9e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f208415adc0 0x7f20840b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:55.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.938+0000 7f208bb9e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f208415ef60 con 0x7f20840d0750 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.938+0000 7f208bb9e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f208415f240 con 0x7f20840ad7b0 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.938+0000 7f208bb9e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f208415f520 con 0x7f208415adc0 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.938+0000 7f208993a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f208415adc0 0x7f20840b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.938+0000 7f208993a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f208415adc0 0x7f20840b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37024/0 (socket says 172.21.15.26:37024) 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.938+0000 7f208a13b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f20840d0750 0x7f2084072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46826/0 (socket says 172.21.15.26:46826) 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.938+0000 7f208993a640 1 -- 172.21.15.26:0/2882630260 learned_addr learned my addr 172.21.15.26:0/2882630260 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.939+0000 7f208993a640 1 -- 172.21.15.26:0/2882630260 >> v1:172.21.15.29:6789/0 conn(0x7f20840ad7b0 legacy=0x7f20840d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.939+0000 7f208993a640 1 -- 172.21.15.26:0/2882630260 >> v1:172.21.15.26:6789/0 conn(0x7f20840d0750 legacy=0x7f2084072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.939+0000 7f208993a640 1 -- 172.21.15.26:0/2882630260 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208415f800 con 0x7f208415adc0 2024-04-24T23:21:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.939+0000 7f2088938640 1 -- 172.21.15.26:0/2882630260 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358789537 0 0) 0x7f208415ef60 con 0x7f20840d0750 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.939+0000 7f208993a640 1 --2- 172.21.15.26:0/2882630260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208415adc0 0x7f20840b4370 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f207c00bb40 tx=0x7f207c02c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c3bc2c11e0acd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f2088938640 1 -- 172.21.15.26:0/2882630260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c0327c0 con 0x7f208415adc0 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f2088938640 1 -- 172.21.15.26:0/2882630260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c038070 con 0x7f208415adc0 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f2088938640 1 -- 172.21.15.26:0/2882630260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c032b80 con 0x7f208415adc0 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f208bb9e640 1 -- 172.21.15.26:0/2882630260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208415adc0 msgr2=0x7f20840b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f208bb9e640 1 --2- 172.21.15.26:0/2882630260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208415adc0 0x7f20840b4370 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f207c00bb40 tx=0x7f207c02c900 comp rx=0 tx=0).stop 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f208bb9e640 1 -- 172.21.15.26:0/2882630260 shutdown_connections 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f208bb9e640 1 --2- 172.21.15.26:0/2882630260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208415adc0 0x7f20840b4370 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f208bb9e640 1 -- 172.21.15.26:0/2882630260 >> 172.21.15.26:0/2882630260 conn(0x7f208406b540 msgr2=0x7f208406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f208bb9e640 1 -- 172.21.15.26:0/2882630260 shutdown_connections 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.940+0000 7f208bb9e640 1 -- 172.21.15.26:0/2882630260 wait complete. 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.941+0000 7f208bb9e640 1 Processor -- start 2024-04-24T23:21:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.941+0000 7f208bb9e640 1 -- start start 2024-04-24T23:21:55.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f208bb9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20840ad7b0 0x7f208409c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:55.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f208bb9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20840d0750 0x7f2084146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:55.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f208bb9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f208415adc0 0x7f208415d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:55.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f208bb9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f208415b1a0 con 0x7f20840d0750 2024-04-24T23:21:55.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f208bb9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20840b48b0 con 0x7f20840ad7b0 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f208bb9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f208415e9c0 con 0x7f208415adc0 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f2089139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20840d0750 0x7f2084146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f2089139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20840d0750 0x7f2084146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37036/0 (socket says 172.21.15.26:37036) 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f2089139640 1 -- 172.21.15.26:0/187293111 learned_addr learned my addr 172.21.15.26:0/187293111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f208993a640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20840ad7b0 0x7f208409c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.942+0000 7f208a13b640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f208415adc0 0x7f208415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f2089139640 1 -- 172.21.15.26:0/187293111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f208415adc0 msgr2=0x7f208415d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f2089139640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f208415adc0 0x7f208415d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f2089139640 1 -- 172.21.15.26:0/187293111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20840ad7b0 msgr2=0x7f208409c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f2089139640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20840ad7b0 0x7f208409c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f2089139640 1 -- 172.21.15.26:0/187293111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20840b1ec0 con 0x7f20840d0750 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f208993a640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20840ad7b0 0x7f208409c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f2089139640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20840d0750 0x7f2084146630 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f207800c430 tx=0x7f207800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f20827fc640 1 -- 172.21.15.26:0/187293111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078011790 con 0x7f20840d0750 2024-04-24T23:21:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f20827fc640 1 -- 172.21.15.26:0/187293111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2078017070 con 0x7f20840d0750 2024-04-24T23:21:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208409b4b0 con 0x7f20840d0750 2024-04-24T23:21:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f20827fc640 1 -- 172.21.15.26:0/187293111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078011b70 con 0x7f20840d0750 2024-04-24T23:21:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.943+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208409ba70 con 0x7f20840d0750 2024-04-24T23:21:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.944+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2084075610 con 0x7f20840d0750 2024-04-24T23:21:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.945+0000 7f20827fc640 1 -- 172.21.15.26:0/187293111 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2078011d10 con 0x7f20840d0750 2024-04-24T23:21:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.945+0000 7f20827fc640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f207006be30 0x7f207006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.945+0000 7f20827fc640 1 -- 172.21.15.26:0/187293111 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f207808ac20 con 0x7f20840d0750 2024-04-24T23:21:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.945+0000 7f208993a640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f207006be30 0x7f207006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.945+0000 7f208993a640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f207006be30 0x7f207006e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f207c037000 tx=0x7f207c032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:55.949+0000 7f20827fc640 1 -- 172.21.15.26:0/187293111 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2078059660 con 0x7f20840d0750 2024-04-24T23:21:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:56.181+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f208415e730 con 0x7f20840d0750 2024-04-24T23:21:57.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.025+0000 7f20827fc640 1 -- 172.21.15.26:0/187293111 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7f2078058f60 con 0x7f20840d0750 2024-04-24T23:21:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f207006be30 msgr2=0x7f207006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f207006be30 0x7f207006e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f207c037000 tx=0x7f207c032000 comp rx=0 tx=0).stop 2024-04-24T23:21:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20840d0750 msgr2=0x7f2084146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20840d0750 0x7f2084146630 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f207800c430 tx=0x7f207800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 shutdown_connections 2024-04-24T23:21:57.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f207006be30 0x7f207006e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f208415adc0 0x7f208415d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20840d0750 0x7f2084146630 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 --2- 172.21.15.26:0/187293111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20840ad7b0 0x7f208409c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 >> 172.21.15.26:0/187293111 conn(0x7f208406b540 msgr2=0x7f208415cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:57.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 shutdown_connections 2024-04-24T23:21:57.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.028+0000 7f208bb9e640 1 -- 172.21.15.26:0/187293111 wait complete. 2024-04-24T23:21:57.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.146+0000 7f779174f640 1 Processor -- start 2024-04-24T23:21:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.146+0000 7f779174f640 1 -- start start 2024-04-24T23:21:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.147+0000 7f779174f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f778c15e570 0x7f778c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.147+0000 7f779174f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f778c0b3130 0x7f778c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.147+0000 7f779174f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f778c0d32f0 con 0x7f778c0d22e0 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.147+0000 7f779174f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f778c0d1ab0 con 0x7f778c15e570 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.147+0000 7f779174f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f778c0d2ae0 con 0x7f778c0b3130 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.147+0000 7f778affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f778c15e570 0x7f778c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.147+0000 7f778affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f778c15e570 0x7f778c0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37040/0 (socket says 172.21.15.26:37040) 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.147+0000 7f778affd640 1 -- 172.21.15.26:0/392652375 learned_addr learned my addr 172.21.15.26:0/392652375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.148+0000 7f778a7fc640 1 --2- 172.21.15.26:0/392652375 >> v2:172.21.15.29:3300/0 conn(0x7f778c0b3130 0x7f778c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.148+0000 7f778affd640 1 -- 172.21.15.26:0/392652375 >> v2:172.21.15.29:3300/0 conn(0x7f778c0b3130 msgr2=0x7f778c0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.148+0000 7f778affd640 1 --2- 172.21.15.26:0/392652375 >> v2:172.21.15.29:3300/0 conn(0x7f778c0b3130 0x7f778c0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.148+0000 7f778affd640 1 -- 172.21.15.26:0/392652375 >> v1:172.21.15.29:6789/0 conn(0x7f778c0d22e0 legacy=0x7f778c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.148+0000 7f778affd640 1 -- 172.21.15.26:0/392652375 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c0d0270 con 0x7f778c15e570 2024-04-24T23:21:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.148+0000 7f778affd640 1 --2- 172.21.15.26:0/392652375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 0x7f778c0d4160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f778c08cae0 tx=0x7f778402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5728f3c12693da70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f7789ffb640 1 -- 172.21.15.26:0/392652375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784037070 con 0x7f778c15e570 2024-04-24T23:21:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f7789ffb640 1 -- 172.21.15.26:0/392652375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7784032440 con 0x7f778c15e570 2024-04-24T23:21:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f7789ffb640 1 -- 172.21.15.26:0/392652375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784032810 con 0x7f778c15e570 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f779174f640 1 -- 172.21.15.26:0/392652375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 msgr2=0x7f778c0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f779174f640 1 --2- 172.21.15.26:0/392652375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 0x7f778c0d4160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f778c08cae0 tx=0x7f778402a270 comp rx=0 tx=0).stop 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f779174f640 1 -- 172.21.15.26:0/392652375 shutdown_connections 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f779174f640 1 --2- 172.21.15.26:0/392652375 >> v2:172.21.15.29:3300/0 conn(0x7f778c0b3130 0x7f778c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f779174f640 1 --2- 172.21.15.26:0/392652375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 0x7f778c0d4160 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f779174f640 1 -- 172.21.15.26:0/392652375 >> 172.21.15.26:0/392652375 conn(0x7f778c08dbe0 msgr2=0x7f778c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f779174f640 1 -- 172.21.15.26:0/392652375 shutdown_connections 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.149+0000 7f779174f640 1 -- 172.21.15.26:0/392652375 wait complete. 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f779174f640 1 Processor -- start 2024-04-24T23:21:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f779174f640 1 -- start start 2024-04-24T23:21:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f779174f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f778c0b3130 0x7f778c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f779174f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f778c0d22e0 0x7f778c0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f779174f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 0x7f778c0a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f779174f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f778c0cfa70 con 0x7f778c15e570 2024-04-24T23:21:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f779174f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f778c0d3010 con 0x7f778c0d22e0 2024-04-24T23:21:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f779174f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f778c0cfd50 con 0x7f778c0b3130 2024-04-24T23:21:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 0x7f778c0a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 0x7f778c0a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37050/0 (socket says 172.21.15.26:37050) 2024-04-24T23:21:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.151+0000 7f778b7fe640 1 -- 172.21.15.26:0/1645974908 learned_addr learned my addr 172.21.15.26:0/1645974908 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778affd640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f778c0b3130 0x7f778c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778a7fc640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f778c0d22e0 0x7f778c0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778b7fe640 1 -- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f778c0b3130 msgr2=0x7f778c15c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778b7fe640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f778c0b3130 0x7f778c15c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778b7fe640 1 -- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f778c0d22e0 msgr2=0x7f778c0cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778b7fe640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f778c0d22e0 0x7f778c0cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778b7fe640 1 -- 172.21.15.26:0/1645974908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c09e250 con 0x7f778c15e570 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778a7fc640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f778c0d22e0 0x7f778c0cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778affd640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f778c0b3130 0x7f778c15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f778b7fe640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 0x7f778c0a0930 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f777c00c7d0 tx=0x7f777c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f77737fe640 1 -- 172.21.15.26:0/1645974908 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c003b70 con 0x7f778c15e570 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f77737fe640 1 -- 172.21.15.26:0/1645974908 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f777c003d30 con 0x7f778c15e570 2024-04-24T23:21:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f77737fe640 1 -- 172.21.15.26:0/1645974908 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c00aa90 con 0x7f778c15e570 2024-04-24T23:21:57.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778c09e530 con 0x7f778c15e570 2024-04-24T23:21:57.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.152+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778c0cd460 con 0x7f778c15e570 2024-04-24T23:21:57.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.153+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778c0cd940 con 0x7f778c15e570 2024-04-24T23:21:57.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.154+0000 7f77737fe640 1 -- 172.21.15.26:0/1645974908 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f777c00ac30 con 0x7f778c15e570 2024-04-24T23:21:57.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.154+0000 7f77737fe640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f777806be30 0x7f777806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:57.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.154+0000 7f77737fe640 1 -- 172.21.15.26:0/1645974908 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f777c08ae10 con 0x7f778c15e570 2024-04-24T23:21:57.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.154+0000 7f778affd640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f777806be30 0x7f777806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:57.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.155+0000 7f778affd640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f777806be30 0x7f777806e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7784034000 tx=0x7f778402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.159+0000 7f77737fe640 1 -- 172.21.15.26:0/1645974908 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f778c0cd940 con 0x7f778c15e570 2024-04-24T23:21:57.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.384+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f778c0694c0 con 0x7f778c15e570 2024-04-24T23:21:57.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.384+0000 7f77737fe640 1 -- 172.21.15.26:0/1645974908 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5194 (secure 0 0 0) 0x7f778c0d32f0 con 0x7f778c15e570 2024-04-24T23:21:57.386 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:57.386 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2201738459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2201738459},{"type":"v1","addr":"172.21.15.29:6834","nonce":2201738459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:21:48.324889+0000","modified":"2024-04-24T23:21:57.028993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5088,"mds_1":4867},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4867":{"gid":4867,"name":"d","rank":1,"incarnation":18,"state":"up:creating","state_seq":1,"addr":"172.21.15.29:6835/3972561683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3972561683},{"type":"v1","addr":"172.21.15.29:6835","nonce":3972561683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/315291888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":315291888},{"type":"v1","addr":"172.21.15.26:6836","nonce":315291888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-04-24T23:21:57.386 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 18 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f777806be30 msgr2=0x7f777806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f777806be30 0x7f777806e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7784034000 tx=0x7f778402e000 comp rx=0 tx=0).stop 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 msgr2=0x7f778c0a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 0x7f778c0a0930 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f777c00c7d0 tx=0x7f777c00cc90 comp rx=0 tx=0).stop 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 shutdown_connections 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f777806be30 0x7f777806e2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f778c15e570 0x7f778c0a0930 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f778c0d22e0 0x7f778c0cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 --2- 172.21.15.26:0/1645974908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f778c0b3130 0x7f778c15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 >> 172.21.15.26:0/1645974908 conn(0x7f778c08dbe0 msgr2=0x7f778c0b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:57.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 shutdown_connections 2024-04-24T23:21:57.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:57.387+0000 7f779174f640 1 -- 172.21.15.26:0/1645974908 wait complete. 2024-04-24T23:21:57.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:21:48.324889+0000', 'modified': '2024-04-24T23:21:57.028993+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5088, 'mds_1': 4867}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4867': {'gid': 4867, 'name': 'd', 'rank': 1, 'incarnation': 18, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.29:6835/3972561683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 3972561683}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 3972561683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5088': {'gid': 5088, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6836/315291888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 315291888}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 315291888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5088, 'qdb_cluster': [5088]} 2024-04-24T23:21:57.803 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4867:up:creating 2024-04-24T23:21:58.804 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:21:58.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.904+0000 7feb45be7640 1 Processor -- start 2024-04-24T23:21:58.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.904+0000 7feb45be7640 1 -- start start 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.904+0000 7feb45be7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb4015e570 0x7feb400d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.904+0000 7feb45be7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb400b3130 0x7feb400d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.904+0000 7feb45be7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb400d32f0 con 0x7feb400d22e0 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb45be7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7feb400d1ab0 con 0x7feb4015e570 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb45be7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feb400d2ae0 con 0x7feb400b3130 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb3f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb4015e570 0x7feb400d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb3f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb4015e570 0x7feb400d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37056/0 (socket says 172.21.15.26:37056) 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb3f7fe640 1 -- 172.21.15.26:0/2393421422 learned_addr learned my addr 172.21.15.26:0/2393421422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb3effd640 1 --2- 172.21.15.26:0/2393421422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb400b3130 0x7feb400d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb3f7fe640 1 -- 172.21.15.26:0/2393421422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb400b3130 msgr2=0x7feb400d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb3f7fe640 1 --2- 172.21.15.26:0/2393421422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb400b3130 0x7feb400d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:58.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb3f7fe640 1 -- 172.21.15.26:0/2393421422 >> v1:172.21.15.29:6789/0 conn(0x7feb400d22e0 legacy=0x7feb400d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:58.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.905+0000 7feb3f7fe640 1 -- 172.21.15.26:0/2393421422 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb400d0270 con 0x7feb4015e570 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.906+0000 7feb3f7fe640 1 --2- 172.21.15.26:0/2393421422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb4015e570 0x7feb400d4160 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feb30009870 tx=0x7feb3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b6b9e9bbf58a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.906+0000 7feb3e7fc640 1 -- 172.21.15.26:0/2393421422 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30037070 con 0x7feb4015e570 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.906+0000 7feb3e7fc640 1 -- 172.21.15.26:0/2393421422 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb30032440 con 0x7feb4015e570 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.906+0000 7feb3e7fc640 1 -- 172.21.15.26:0/2393421422 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30032810 con 0x7feb4015e570 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.907+0000 7feb45be7640 1 -- 172.21.15.26:0/2393421422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb4015e570 msgr2=0x7feb400d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.907+0000 7feb45be7640 1 --2- 172.21.15.26:0/2393421422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb4015e570 0x7feb400d4160 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feb30009870 tx=0x7feb3002a270 comp rx=0 tx=0).stop 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.907+0000 7feb45be7640 1 -- 172.21.15.26:0/2393421422 shutdown_connections 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.907+0000 7feb45be7640 1 --2- 172.21.15.26:0/2393421422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb400b3130 0x7feb400d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.907+0000 7feb45be7640 1 --2- 172.21.15.26:0/2393421422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb4015e570 0x7feb400d4160 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:58.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.907+0000 7feb45be7640 1 -- 172.21.15.26:0/2393421422 >> 172.21.15.26:0/2393421422 conn(0x7feb4008dbe0 msgr2=0x7feb4009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.907+0000 7feb45be7640 1 -- 172.21.15.26:0/2393421422 shutdown_connections 2024-04-24T23:21:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.907+0000 7feb45be7640 1 -- 172.21.15.26:0/2393421422 wait complete. 2024-04-24T23:21:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.908+0000 7feb45be7640 1 Processor -- start 2024-04-24T23:21:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.909+0000 7feb45be7640 1 -- start start 2024-04-24T23:21:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb45be7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3130 0x7feb40157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb45be7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb400d22e0 0x7feb400b9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb45be7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb4015e570 0x7feb400b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb45be7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feb400cfa70 con 0x7feb400d22e0 2024-04-24T23:21:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb45be7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feb400d3010 con 0x7feb400b3130 2024-04-24T23:21:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb45be7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feb400cfd50 con 0x7feb4015e570 2024-04-24T23:21:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb400d22e0 0x7feb400b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb400d22e0 0x7feb400b9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37068/0 (socket says 172.21.15.26:37068) 2024-04-24T23:21:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3effd640 1 -- 172.21.15.26:0/2496989555 learned_addr learned my addr 172.21.15.26:0/2496989555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3f7fe640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3130 0x7feb40157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3effd640 1 -- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb4015e570 msgr2=0x7feb400b98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3ffff640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb4015e570 0x7feb400b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3effd640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb4015e570 0x7feb400b98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3effd640 1 -- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3130 msgr2=0x7feb40157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3effd640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3130 0x7feb40157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3effd640 1 -- 172.21.15.26:0/2496989555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4015c2e0 con 0x7feb400d22e0 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3ffff640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb4015e570 0x7feb400b98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.910+0000 7feb3f7fe640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3130 0x7feb40157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.911+0000 7feb3effd640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb400d22e0 0x7feb400b9390 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e9e0 tx=0x7feb3400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.911+0000 7feb27fff640 1 -- 172.21.15.26:0/2496989555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34017070 con 0x7feb400d22e0 2024-04-24T23:21:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.911+0000 7feb27fff640 1 -- 172.21.15.26:0/2496989555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3400c9d0 con 0x7feb400d22e0 2024-04-24T23:21:58.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.911+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb400b85d0 con 0x7feb400d22e0 2024-04-24T23:21:58.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.911+0000 7feb27fff640 1 -- 172.21.15.26:0/2496989555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3400cdc0 con 0x7feb400d22e0 2024-04-24T23:21:58.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.911+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb400b8b90 con 0x7feb400d22e0 2024-04-24T23:21:58.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.912+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb400672e0 con 0x7feb400d22e0 2024-04-24T23:21:58.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.912+0000 7feb27fff640 1 -- 172.21.15.26:0/2496989555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb3401a070 con 0x7feb400d22e0 2024-04-24T23:21:58.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.913+0000 7feb27fff640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb2c06be30 0x7feb2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:58.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.913+0000 7feb27fff640 1 -- 172.21.15.26:0/2496989555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7feb3408ae00 con 0x7feb400d22e0 2024-04-24T23:21:58.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.913+0000 7feb3f7fe640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb2c06be30 0x7feb2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:58.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.913+0000 7feb3f7fe640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb2c06be30 0x7feb2c06e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb30034000 tx=0x7feb3002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:58.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:58.917+0000 7feb27fff640 1 -- 172.21.15.26:0/2496989555 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feb400672e0 con 0x7feb400d22e0 2024-04-24T23:21:59.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:21:59.078 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:21:59.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.182+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb400694c0 con 0x7feb400d22e0 2024-04-24T23:21:59.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.182+0000 7feb27fff640 1 -- 172.21.15.26:0/2496989555 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5216 (secure 0 0 0) 0x7feb34059840 con 0x7feb400d22e0 2024-04-24T23:21:59.184 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:21:59.184 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:21:48.324889+0000","modified":"2024-04-24T23:21:59.037564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5088,"mds_1":4867,"mds_2":4864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4864":{"gid":4864,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6834/2201738459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2201738459},{"type":"v1","addr":"172.21.15.29:6834","nonce":2201738459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4867":{"gid":4867,"name":"d","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6835/3972561683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3972561683},{"type":"v1","addr":"172.21.15.29:6835","nonce":3972561683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/315291888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":315291888},{"type":"v1","addr":"172.21.15.26:6836","nonce":315291888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[4864,4867,5088]},"id":2}]} 2024-04-24T23:21:59.185 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-04-24T23:21:59.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb2c06be30 msgr2=0x7feb2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb2c06be30 0x7feb2c06e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb30034000 tx=0x7feb3002e000 comp rx=0 tx=0).stop 2024-04-24T23:21:59.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb400d22e0 msgr2=0x7feb400b9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb400d22e0 0x7feb400b9390 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e9e0 tx=0x7feb3400eea0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 shutdown_connections 2024-04-24T23:21:59.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb2c06be30 0x7feb2c06e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb4015e570 0x7feb400b98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb400d22e0 0x7feb400b9390 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 --2- 172.21.15.26:0/2496989555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400b3130 0x7feb40157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.184+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 >> 172.21.15.26:0/2496989555 conn(0x7feb4008dbe0 msgr2=0x7feb400b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.185+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 shutdown_connections 2024-04-24T23:21:59.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.185+0000 7feb45be7640 1 -- 172.21.15.26:0/2496989555 wait complete. 2024-04-24T23:21:59.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:21:48.324889+0000', 'modified': '2024-04-24T23:21:59.037564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5088, 'mds_1': 4867, 'mds_2': 4864}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4864': {'gid': 4864, 'name': 'b', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.29:6834/2201738459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 2201738459}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 2201738459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4867': {'gid': 4867, 'name': 'd', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.29:6835/3972561683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 3972561683}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 3972561683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5088': {'gid': 5088, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6836/315291888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 315291888}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 315291888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5088, 'qdb_cluster': [4864, 4867, 5088]} 2024-04-24T23:21:59.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:21:59.600 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:21:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.700+0000 7f794d9ce640 1 Processor -- start 2024-04-24T23:21:59.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.700+0000 7f794d9ce640 1 -- start start 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.700+0000 7f794d9ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7948145910 0x7f794806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.700+0000 7f794d9ce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f794806d350 0x7f794815d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.700+0000 7f794d9ce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f794815d570 con 0x7f79480cd660 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.700+0000 7f794d9ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7948145cf0 con 0x7f7948145910 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.700+0000 7f794d9ce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f79480cfbc0 con 0x7f794806d350 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.701+0000 7f7946ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7948145910 0x7f794806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.701+0000 7f7946ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7948145910 0x7f794806e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37078/0 (socket says 172.21.15.26:37078) 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.701+0000 7f7946ffd640 1 -- 172.21.15.26:0/4052925721 learned_addr learned my addr 172.21.15.26:0/4052925721 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.701+0000 7f79467fc640 1 --2- 172.21.15.26:0/4052925721 >> v2:172.21.15.29:3300/0 conn(0x7f794806d350 0x7f794815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.701+0000 7f7946ffd640 1 -- 172.21.15.26:0/4052925721 >> v2:172.21.15.29:3300/0 conn(0x7f794806d350 msgr2=0x7f794815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.701+0000 7f7946ffd640 1 --2- 172.21.15.26:0/4052925721 >> v2:172.21.15.29:3300/0 conn(0x7f794806d350 0x7f794815d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.701+0000 7f7946ffd640 1 -- 172.21.15.26:0/4052925721 >> v1:172.21.15.29:6789/0 conn(0x7f79480cd660 legacy=0x7f794815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.701+0000 7f7946ffd640 1 -- 172.21.15.26:0/4052925721 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794806d7f0 con 0x7f7948145910 2024-04-24T23:21:59.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f7946ffd640 1 --2- 172.21.15.26:0/4052925721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7948145910 0x7f794806e890 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7930009870 tx=0x7f793002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fd7a74b0efad71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f7945ffb640 1 -- 172.21.15.26:0/4052925721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930037070 con 0x7f7948145910 2024-04-24T23:21:59.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f7945ffb640 1 -- 172.21.15.26:0/4052925721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7930032440 con 0x7f7948145910 2024-04-24T23:21:59.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f7945ffb640 1 -- 172.21.15.26:0/4052925721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930032810 con 0x7f7948145910 2024-04-24T23:21:59.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f794d9ce640 1 -- 172.21.15.26:0/4052925721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7948145910 msgr2=0x7f794806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f794d9ce640 1 --2- 172.21.15.26:0/4052925721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7948145910 0x7f794806e890 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7930009870 tx=0x7f793002a210 comp rx=0 tx=0).stop 2024-04-24T23:21:59.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f794d9ce640 1 -- 172.21.15.26:0/4052925721 shutdown_connections 2024-04-24T23:21:59.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f794d9ce640 1 --2- 172.21.15.26:0/4052925721 >> v2:172.21.15.29:3300/0 conn(0x7f794806d350 0x7f794815d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f794d9ce640 1 --2- 172.21.15.26:0/4052925721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7948145910 0x7f794806e890 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f794d9ce640 1 -- 172.21.15.26:0/4052925721 >> 172.21.15.26:0/4052925721 conn(0x7f794806b300 msgr2=0x7f794806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.702+0000 7f794d9ce640 1 -- 172.21.15.26:0/4052925721 shutdown_connections 2024-04-24T23:21:59.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.703+0000 7f794d9ce640 1 -- 172.21.15.26:0/4052925721 wait complete. 2024-04-24T23:21:59.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.704+0000 7f794d9ce640 1 Processor -- start 2024-04-24T23:21:59.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.704+0000 7f794d9ce640 1 -- start start 2024-04-24T23:21:59.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.704+0000 7f794d9ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794806d350 0x7f794815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.704+0000 7f794d9ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79480cd660 0x7f79481f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.704+0000 7f794d9ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7948145910 0x7f794813d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.704+0000 7f794d9ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f794806e090 con 0x7f794806d350 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f794d9ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f794806ddb0 con 0x7f7948145910 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f794d9ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f794806e370 con 0x7f79480cd660 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f7946ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794806d350 0x7f794815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f7946ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794806d350 0x7f794815ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37086/0 (socket says 172.21.15.26:37086) 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f7946ffd640 1 -- 172.21.15.26:0/3050902669 learned_addr learned my addr 172.21.15.26:0/3050902669 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f79477fe640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7948145910 0x7f794813d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f79467fc640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79480cd660 0x7f79481f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f7946ffd640 1 -- 172.21.15.26:0/3050902669 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79480cd660 msgr2=0x7f79481f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f7946ffd640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79480cd660 0x7f79481f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f7946ffd640 1 -- 172.21.15.26:0/3050902669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7948145910 msgr2=0x7f794813d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f7946ffd640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7948145910 0x7f794813d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f7946ffd640 1 -- 172.21.15.26:0/3050902669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794813ddf0 con 0x7f794806d350 2024-04-24T23:21:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.705+0000 7f7946ffd640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794806d350 0x7f794815ae80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7930000c00 tx=0x7f79300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.706+0000 7f793ffff640 1 -- 172.21.15.26:0/3050902669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930041020 con 0x7f794806d350 2024-04-24T23:21:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.706+0000 7f793ffff640 1 -- 172.21.15.26:0/3050902669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79300306c0 con 0x7f794806d350 2024-04-24T23:21:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.706+0000 7f794d9ce640 1 -- 172.21.15.26:0/3050902669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794813e0d0 con 0x7f794806d350 2024-04-24T23:21:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.706+0000 7f793ffff640 1 -- 172.21.15.26:0/3050902669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930037040 con 0x7f794806d350 2024-04-24T23:21:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.706+0000 7f794d9ce640 1 -- 172.21.15.26:0/3050902669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794813e690 con 0x7f794806d350 2024-04-24T23:21:59.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.707+0000 7f793ffff640 1 -- 172.21.15.26:0/3050902669 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7930044020 con 0x7f794806d350 2024-04-24T23:21:59.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.708+0000 7f793ffff640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f792806be30 0x7f792806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.708+0000 7f793ffff640 1 -- 172.21.15.26:0/3050902669 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f79300ab560 con 0x7f794806d350 2024-04-24T23:21:59.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.708+0000 7f79467fc640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f792806be30 0x7f792806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.708+0000 7f79467fc640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f792806be30 0x7f792806e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79481e76c0 tx=0x7f7938008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.709+0000 7f79507b9740 1 Processor -- start 2024-04-24T23:21:59.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.709+0000 7f79507b9740 1 -- start start 2024-04-24T23:21:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f79507b9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c0c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f79507b9740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 0x560209c3c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f79507b9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3c540 0x560209c11350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f79507b9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560209b74f00 con 0x560209c59b60 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f79507b9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560209b1d750 con 0x560209c3c540 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f79507b9740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560209b70ec0 con 0x560209c3cc80 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f7946ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c0c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f7946ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c0c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37088/0 (socket says 172.21.15.26:37088) 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f7946ffd640 1 -- 172.21.15.26:0/2951071497 learned_addr learned my addr 172.21.15.26:0/2951071497 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f79477fe640 1 --2- 172.21.15.26:0/2951071497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3c540 0x560209c11350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f79467fc640 1 --2- 172.21.15.26:0/2951071497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 0x560209c3c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f7946ffd640 1 -- 172.21.15.26:0/2951071497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 msgr2=0x560209c3c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f7946ffd640 1 --2- 172.21.15.26:0/2951071497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 0x560209c3c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f7946ffd640 1 -- 172.21.15.26:0/2951071497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3c540 msgr2=0x560209c11350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f7946ffd640 1 --2- 172.21.15.26:0/2951071497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3c540 0x560209c11350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f7946ffd640 1 -- 172.21.15.26:0/2951071497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602098892d0 con 0x560209c59b60 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f7946ffd640 1 --2- 172.21.15.26:0/2951071497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c0c4f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7930081ec0 tx=0x7f7930075b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6161377d6580ad43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f793d7fa640 1 -- 172.21.15.26:0/2951071497 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300823f0 con 0x560209c59b60 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f793d7fa640 1 -- 172.21.15.26:0/2951071497 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7930077bc0 con 0x560209c59b60 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.710+0000 7f793d7fa640 1 -- 172.21.15.26:0/2951071497 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930081f60 con 0x560209c59b60 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.711+0000 7f79507b9740 1 -- 172.21.15.26:0/2951071497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 msgr2=0x560209c0c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.711+0000 7f79507b9740 1 --2- 172.21.15.26:0/2951071497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c0c4f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7930081ec0 tx=0x7f7930075b60 comp rx=0 tx=0).stop 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.711+0000 7f79507b9740 1 -- 172.21.15.26:0/2951071497 shutdown_connections 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.711+0000 7f79507b9740 1 --2- 172.21.15.26:0/2951071497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3c540 0x560209c11350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.711+0000 7f79507b9740 1 --2- 172.21.15.26:0/2951071497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 0x560209c3c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.711+0000 7f79507b9740 1 --2- 172.21.15.26:0/2951071497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c0c4f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.711+0000 7f79507b9740 1 -- 172.21.15.26:0/2951071497 >> 172.21.15.26:0/2951071497 conn(0x560209c0e760 msgr2=0x560209c57000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.711+0000 7f79507b9740 1 -- 172.21.15.26:0/2951071497 shutdown_connections 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.711+0000 7f79507b9740 1 -- 172.21.15.26:0/2951071497 wait complete. 2024-04-24T23:21:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.712+0000 7f79507b9740 1 -- start start 2024-04-24T23:21:59.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.712+0000 7f79507b9740 1 Processor -- start 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.712+0000 7f79507b9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3cc80 0x560209c6a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.712+0000 7f79507b9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c3f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.712+0000 7f79507b9740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c58c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.712+0000 7f79507b9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560209b74f00 con 0x560209c59b60 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.712+0000 7f79507b9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560209b30cf0 con 0x560209c3cc80 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.712+0000 7f79507b9740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560209c10910 con 0x560209c37c90 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79507b9740 10 client.0 _ll_register_callbacks cb 0x560209b306e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79467fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c3f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79467fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c3f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37090/0 (socket says 172.21.15.26:37090) 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79467fc640 1 -- 172.21.15.26:0/1621802875 learned_addr learned my addr 172.21.15.26:0/1621802875 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f7946ffd640 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3cc80 0x560209c6a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79477fe640 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c58c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79467fc640 1 -- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 msgr2=0x560209c58c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79467fc640 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c58c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79467fc640 1 -- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3cc80 msgr2=0x560209c6a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79467fc640 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3cc80 0x560209c6a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79467fc640 1 -- 172.21.15.26:0/1621802875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602099536b0 con 0x560209c59b60 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79477fe640 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c58c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79467fc640 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c3f6e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7938010cb0 tx=0x7f7938004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b45e2e3654ec05c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79267fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79267fc640 1 -- 172.21.15.26:0/1621802875 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938029060 con 0x560209c59b60 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79267fc640 1 -- 172.21.15.26:0/1621802875 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f793800f040 con 0x560209c59b60 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79507b9740 1 -- 172.21.15.26:0/1621802875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7930077bc0 con 0x560209c59b60 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79267fc640 1 -- 172.21.15.26:0/1621802875 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f793802f050 con 0x560209c59b60 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79507b9740 10 client.5310 fetch_fsmap learned FSMap version 21 2024-04-24T23:21:59.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.713+0000 7f79507b9740 1 -- 172.21.15.26:0/1621802875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x560209c4df60 con 0x560209c59b60 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f79267fc640 1 -- 172.21.15.26:0/1621802875 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7930077bc0 con 0x560209c59b60 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f79267fc640 1 -- 172.21.15.26:0/1621802875 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 3534+0+0 (secure 0 0 0) 0x7f7938032020 con 0x560209c59b60 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f79507b9740 10 client.5310 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f79507b9740 10 client.5310 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f79507b9740 20 client.5310 populate_metadata read hostname 'smithi026' 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f79507b9740 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x560209c3d4e0 0x560209c4e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f79507b9740 4 client.5310 mds_command: new command op to 4864 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f79507b9740 1 -- 172.21.15.26:0/1621802875 --> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560209b82440 con 0x560209c3d4e0 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f7946ffd640 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x560209c3d4e0 0x560209c4e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f7946ffd640 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x560209c3d4e0 0x560209c4e980 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b18569bb15ae0456 server_cookie=6883cfe7a97f42de in_seq=0 out_seq=0 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.714+0000 7f79267fc640 10 client.5310 ms_handle_connect on v2:172.21.15.29:6832/2201738459 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79267fc640 1 -- 172.21.15.26:0/1621802875 <== mds.2 v2:172.21.15.29:6832/2201738459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560209b82440 con 0x560209c3d4e0 2024-04-24T23:21:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79267fc640 10 client.5310 handle_command_reply: tid=0 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79507b9740 1 client.5310 shutdown 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79507b9740 1 -- 172.21.15.26:0/1621802875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 msgr2=0x560209c3f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79507b9740 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c3f6e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7938010cb0 tx=0x7f7938004730 comp rx=0 tx=0).stop 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79507b9740 1 -- 172.21.15.26:0/1621802875 shutdown_connections 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79507b9740 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x560209c3d4e0 0x560209c4e980 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79507b9740 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c58c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79507b9740 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c3f6e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79507b9740 1 --2- 172.21.15.26:0/1621802875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c3cc80 0x560209c6a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79507b9740 1 -- 172.21.15.26:0/1621802875 >> 172.21.15.26:0/1621802875 conn(0x560209c3c540 msgr2=0x560209c0b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.715+0000 7f79267fc640 0 client.5310 ms_handle_reset on v2:172.21.15.29:6832/2201738459 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.716+0000 7f79507b9740 1 -- 172.21.15.26:0/1621802875 shutdown_connections 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.716+0000 7f79507b9740 1 -- 172.21.15.26:0/1621802875 wait complete. 2024-04-24T23:21:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.716+0000 7f79507b9740 20 client.5310 trim_cache size 0 max 16384 2024-04-24T23:21:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.734+0000 7f79507b9740 1 Processor -- start 2024-04-24T23:21:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.734+0000 7f79507b9740 1 -- start start 2024-04-24T23:21:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79507b9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c37c90 0x560209c07690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79507b9740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 0x560209c0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79507b9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c50450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79507b9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79480b4600 con 0x560209c59b60 2024-04-24T23:21:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79507b9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f79480b4cd0 con 0x560209c37c90 2024-04-24T23:21:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79507b9740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f79481e6690 con 0x560209c3cc80 2024-04-24T23:21:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c50450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c50450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37104/0 (socket says 172.21.15.26:37104) 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79477fe640 1 -- 172.21.15.26:0/2818431498 learned_addr learned my addr 172.21.15.26:0/2818431498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f7946ffd640 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c37c90 0x560209c07690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79467fc640 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 0x560209c0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79477fe640 1 -- 172.21.15.26:0/2818431498 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 msgr2=0x560209c0c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79477fe640 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 0x560209c0c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79477fe640 1 -- 172.21.15.26:0/2818431498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c37c90 msgr2=0x560209c07690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79477fe640 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c37c90 0x560209c07690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79477fe640 1 -- 172.21.15.26:0/2818431498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560209c0f8a0 con 0x560209c59b60 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f7946ffd640 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c37c90 0x560209c07690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f79477fe640 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c50450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7940003fa0 tx=0x7f7940001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4124bc94a4ca1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f793cff9640 1 -- 172.21.15.26:0/2818431498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940006d60 con 0x560209c59b60 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f793cff9640 1 -- 172.21.15.26:0/2818431498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7940005350 con 0x560209c59b60 2024-04-24T23:21:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.735+0000 7f793cff9640 1 -- 172.21.15.26:0/2818431498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940005700 con 0x560209c59b60 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.736+0000 7f79507b9740 1 -- 172.21.15.26:0/2818431498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 msgr2=0x560209c50450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.736+0000 7f79507b9740 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c50450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7940003fa0 tx=0x7f7940001670 comp rx=0 tx=0).stop 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.736+0000 7f79507b9740 1 -- 172.21.15.26:0/2818431498 shutdown_connections 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.736+0000 7f79507b9740 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c59b60 0x560209c50450 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.736+0000 7f79507b9740 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c3cc80 0x560209c0c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.736+0000 7f79507b9740 1 --2- 172.21.15.26:0/2818431498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c37c90 0x560209c07690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.736+0000 7f79507b9740 1 -- 172.21.15.26:0/2818431498 >> 172.21.15.26:0/2818431498 conn(0x560209c3c540 msgr2=0x560209c2c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.736+0000 7f79507b9740 1 -- 172.21.15.26:0/2818431498 shutdown_connections 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.736+0000 7f79507b9740 1 -- 172.21.15.26:0/2818431498 wait complete. 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79507b9740 1 -- start start 2024-04-24T23:21:59.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79507b9740 1 Processor -- start 2024-04-24T23:21:59.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79507b9740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c4ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79507b9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c3cc80 0x560209ccfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79507b9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c59b60 0x560209cd0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79507b9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560209c3d360 con 0x560209c3cc80 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79507b9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560209c4f5c0 con 0x560209c59b60 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79507b9740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560209c3d9b0 con 0x560209c37c90 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79507b9740 10 client.0 _ll_register_callbacks cb 0x7f7938010960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79467fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c3cc80 0x560209ccfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.737+0000 7f79467fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c3cc80 0x560209ccfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37118/0 (socket says 172.21.15.26:37118) 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79467fc640 1 -- 172.21.15.26:0/3430150326 learned_addr learned my addr 172.21.15.26:0/3430150326 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f7946ffd640 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c4ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79477fe640 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c59b60 0x560209cd0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79467fc640 1 -- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 msgr2=0x560209c4ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79467fc640 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c4ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79467fc640 1 -- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c59b60 msgr2=0x560209cd0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79467fc640 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c59b60 0x560209cd0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79467fc640 1 -- 172.21.15.26:0/3430150326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560209c4f220 con 0x560209c3cc80 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f7946ffd640 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c4ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79477fe640 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c59b60 0x560209cd0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79467fc640 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c3cc80 0x560209ccfe00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7938010cb0 tx=0x7f7938014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e254db8ed8fdbb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79267fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79267fc640 1 -- 172.21.15.26:0/3430150326 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938010670 con 0x560209c3cc80 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79267fc640 1 -- 172.21.15.26:0/3430150326 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7938020070 con 0x560209c3cc80 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79507b9740 1 -- 172.21.15.26:0/3430150326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7940005350 con 0x560209c3cc80 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79267fc640 1 -- 172.21.15.26:0/3430150326 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f79380247e0 con 0x560209c3cc80 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79507b9740 10 client.5316 fetch_fsmap learned FSMap version 21 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79507b9740 1 -- 172.21.15.26:0/3430150326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x560209c40200 con 0x560209c3cc80 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.738+0000 7f79267fc640 1 -- 172.21.15.26:0/3430150326 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7940005350 con 0x560209c3cc80 2024-04-24T23:21:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79267fc640 1 -- 172.21.15.26:0/3430150326 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 3534+0+0 (secure 0 0 0) 0x7f7938032040 con 0x560209c3cc80 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79507b9740 10 client.5316 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79507b9740 10 client.5316 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79507b9740 20 client.5316 populate_metadata read hostname 'smithi026' 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79507b9740 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x560209c5a420 0x560209c5a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79507b9740 4 client.5316 mds_command: new command op to 4864 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79507b9740 1 -- 172.21.15.26:0/3430150326 --> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] -- command(tid 0: {"prefix": "status"}) -- 0x560209b7c790 con 0x560209c5a420 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f7946ffd640 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x560209c5a420 0x560209c5a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79267fc640 10 client.5316 ms_handle_connect on v2:172.21.15.29:6832/2201738459 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f7946ffd640 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x560209c5a420 0x560209c5a800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1acec2ac0d62a6de server_cookie=3808799388aa5be9 in_seq=0 out_seq=0 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79267fc640 1 -- 172.21.15.26:0/3430150326 <== mds.2 v2:172.21.15.29:6832/2201738459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560209b7c790 con 0x560209c5a420 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.739+0000 7f79267fc640 10 client.5316 handle_command_reply: tid=0 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 client.5316 shutdown 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 -- 172.21.15.26:0/3430150326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c3cc80 msgr2=0x560209ccfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c3cc80 0x560209ccfe00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7938010cb0 tx=0x7f7938014000 comp rx=0 tx=0).stop 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 -- 172.21.15.26:0/3430150326 shutdown_connections 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x560209c5a420 0x560209c5a800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560209c59b60 0x560209cd0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560209c3cc80 0x560209ccfe00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79267fc640 0 client.5316 ms_handle_reset on v2:172.21.15.29:6832/2201738459 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 --2- 172.21.15.26:0/3430150326 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560209c37c90 0x560209c4ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 -- 172.21.15.26:0/3430150326 >> 172.21.15.26:0/3430150326 conn(0x560209c3c540 msgr2=0x560209c50a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 -- 172.21.15.26:0/3430150326 shutdown_connections 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 1 -- 172.21.15.26:0/3430150326 wait complete. 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f79507b9740 20 client.5316 trim_cache size 0 max 16384 2024-04-24T23:21:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4864, 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 1.7025630119999999, 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 21, 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 33, 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 33, 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 18.643458291999998 2024-04-24T23:21:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:21:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 -- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f792806be30 msgr2=0x7f792806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f792806be30 0x7f792806e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79481e76c0 tx=0x7f7938008040 comp rx=0 tx=0).stop 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 -- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794806d350 msgr2=0x7f794815ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794806d350 0x7f794815ae80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7930000c00 tx=0x7f79300046c0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 -- 172.21.15.26:0/3050902669 shutdown_connections 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f792806be30 0x7f792806e2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7948145910 0x7f794813d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79480cd660 0x7f79481f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 --2- 172.21.15.26:0/3050902669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794806d350 0x7f794815ae80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 -- 172.21.15.26:0/3050902669 >> 172.21.15.26:0/3050902669 conn(0x7f794806b300 msgr2=0x7f79480b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.740+0000 7f794d9ce640 1 -- 172.21.15.26:0/3050902669 shutdown_connections 2024-04-24T23:21:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.741+0000 7f794d9ce640 1 -- 172.21.15.26:0/3050902669 wait complete. 2024-04-24T23:21:59.753 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:21:59.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.853+0000 7f7156ddc640 1 Processor -- start 2024-04-24T23:21:59.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.853+0000 7f7156ddc640 1 -- start start 2024-04-24T23:21:59.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.854+0000 7f7156ddc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500d4870 0x7f71500b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.854+0000 7f7156ddc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f71500b3130 0x7f71500b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.854+0000 7f7156ddc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71500d32f0 con 0x7f715015e570 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.854+0000 7f7156ddc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f71500d1ab0 con 0x7f71500b3130 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.854+0000 7f7156ddc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f71500d2ae0 con 0x7f71500d4870 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.854+0000 7f7154b78640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f715015e570 0x7f71500d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46828/0 (socket says 172.21.15.26:46828) 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.854+0000 7f7154b78640 1 -- 172.21.15.26:0/2826300253 learned_addr learned my addr 172.21.15.26:0/2826300253 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.855+0000 7f7155379640 1 --2- 172.21.15.26:0/2826300253 >> v2:172.21.15.29:3300/0 conn(0x7f71500b3130 0x7f71500b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.855+0000 7f714ffff640 1 --2- 172.21.15.26:0/2826300253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500d4870 0x7f71500b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.855+0000 7f714f7fe640 1 -- 172.21.15.26:0/2826300253 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310235401 0 0) 0x7f7144001110 con 0x7f715015e570 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.855+0000 7f714f7fe640 1 -- 172.21.15.26:0/2826300253 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7140003520 con 0x7f715015e570 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.855+0000 7f7155379640 1 -- 172.21.15.26:0/2826300253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500d4870 msgr2=0x7f71500b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.855+0000 7f7155379640 1 --2- 172.21.15.26:0/2826300253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500d4870 0x7f71500b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.855+0000 7f714f7fe640 1 -- 172.21.15.26:0/2826300253 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1338500475 0 0) 0x7f7140003520 con 0x7f715015e570 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.855+0000 7f7155379640 1 -- 172.21.15.26:0/2826300253 >> v1:172.21.15.26:6789/0 conn(0x7f715015e570 legacy=0x7f71500d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.855+0000 7f7155379640 1 -- 172.21.15.26:0/2826300253 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71500d0270 con 0x7f71500b3130 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f7155379640 1 --2- 172.21.15.26:0/2826300253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500b3130 0x7f71500b4640 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7148009d90 tx=0x7f714802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35aa9e2bd242f711 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f714f7fe640 1 -- 172.21.15.26:0/2826300253 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144001840 con 0x7f71500b3130 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f714f7fe640 1 -- 172.21.15.26:0/2826300253 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7148007540 con 0x7f71500b3130 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f714f7fe640 1 -- 172.21.15.26:0/2826300253 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148007910 con 0x7f71500b3130 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f7156ddc640 1 -- 172.21.15.26:0/2826300253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500b3130 msgr2=0x7f71500b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f7156ddc640 1 --2- 172.21.15.26:0/2826300253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500b3130 0x7f71500b4640 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7148009d90 tx=0x7f714802a5f0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f7156ddc640 1 -- 172.21.15.26:0/2826300253 shutdown_connections 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f7156ddc640 1 --2- 172.21.15.26:0/2826300253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500b3130 0x7f71500b4640 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f7156ddc640 1 --2- 172.21.15.26:0/2826300253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500d4870 0x7f71500b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.856+0000 7f7156ddc640 1 -- 172.21.15.26:0/2826300253 >> 172.21.15.26:0/2826300253 conn(0x7f715008dbe0 msgr2=0x7f715009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.857+0000 7f7156ddc640 1 -- 172.21.15.26:0/2826300253 shutdown_connections 2024-04-24T23:21:59.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.857+0000 7f7156ddc640 1 -- 172.21.15.26:0/2826300253 wait complete. 2024-04-24T23:21:59.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.858+0000 7f7156ddc640 1 Processor -- start 2024-04-24T23:21:59.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.858+0000 7f7156ddc640 1 -- start start 2024-04-24T23:21:59.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f7156ddc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500b3130 0x7f715015e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f7156ddc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500d4870 0x7f71501513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f7156ddc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 0x7f7150152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f7156ddc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71500cfa70 con 0x7f71500d4870 2024-04-24T23:21:59.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f7156ddc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f71500d3010 con 0x7f71500b3130 2024-04-24T23:21:59.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f7156ddc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f71500cfd50 con 0x7f715015e570 2024-04-24T23:21:59.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500d4870 0x7f71501513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500d4870 0x7f71501513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37134/0 (socket says 172.21.15.26:37134) 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714ffff640 1 -- 172.21.15.26:0/2096610593 learned_addr learned my addr 172.21.15.26:0/2096610593 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f7154b78640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500b3130 0x7f715015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f7155379640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 0x7f7150152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714ffff640 1 -- 172.21.15.26:0/2096610593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 msgr2=0x7f7150152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714ffff640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 0x7f7150152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714ffff640 1 -- 172.21.15.26:0/2096610593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500b3130 msgr2=0x7f715015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714ffff640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500b3130 0x7f715015e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714ffff640 1 -- 172.21.15.26:0/2096610593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7150151c20 con 0x7f71500d4870 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714ffff640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500d4870 0x7f71501513d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f713c00e9b0 tx=0x7f713c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.859+0000 7f714d7fa640 1 -- 172.21.15.26:0/2096610593 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c017070 con 0x7f71500d4870 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.860+0000 7f714d7fa640 1 -- 172.21.15.26:0/2096610593 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713c00c9a0 con 0x7f71500d4870 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.860+0000 7f714d7fa640 1 -- 172.21.15.26:0/2096610593 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c00cd90 con 0x7f71500d4870 2024-04-24T23:21:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.860+0000 7f7156ddc640 1 -- 172.21.15.26:0/2096610593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7150151f00 con 0x7f71500d4870 2024-04-24T23:21:59.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.860+0000 7f7156ddc640 1 -- 172.21.15.26:0/2096610593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71501524c0 con 0x7f71500d4870 2024-04-24T23:21:59.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.861+0000 7f714d7fa640 1 -- 172.21.15.26:0/2096610593 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f713c01a070 con 0x7f71500d4870 2024-04-24T23:21:59.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.862+0000 7f714d7fa640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be30 0x7f713406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.862+0000 7f714d7fa640 1 -- 172.21.15.26:0/2096610593 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f713c08b1d0 con 0x7f71500d4870 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.862+0000 7f7154b78640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be30 0x7f713406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.862+0000 7f7154b78640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be30 0x7f713406e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71440013b0 tx=0x7f7144009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.862+0000 7f7159bc6740 1 Processor -- start 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.862+0000 7f7159bc6740 1 -- start start 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c2c80 0x55d9ad2c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2540 0x55d9ad297350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7159bc6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d9ad1faf00 con 0x55d9ad2dfb60 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7159bc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d9ad1a3750 con 0x55d9ad2c2540 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7159bc6740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d9ad1f6ec0 con 0x55d9ad2c2c80 2024-04-24T23:21:59.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7154b78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7154b78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2924f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37138/0 (socket says 172.21.15.26:37138) 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7154b78640 1 -- 172.21.15.26:0/3099216590 learned_addr learned my addr 172.21.15.26:0/3099216590 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f7155379640 1 --2- 172.21.15.26:0/3099216590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2540 0x55d9ad297350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.863+0000 7f714ffff640 1 --2- 172.21.15.26:0/3099216590 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c2c80 0x55d9ad2c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7154b78640 1 -- 172.21.15.26:0/3099216590 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c2c80 msgr2=0x55d9ad2c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7154b78640 1 --2- 172.21.15.26:0/3099216590 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c2c80 0x55d9ad2c2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7154b78640 1 -- 172.21.15.26:0/3099216590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2540 msgr2=0x55d9ad297350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7154b78640 1 --2- 172.21.15.26:0/3099216590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2540 0x55d9ad297350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7154b78640 1 -- 172.21.15.26:0/3099216590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9acf0f2d0 con 0x55d9ad2dfb60 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7154b78640 1 --2- 172.21.15.26:0/3099216590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2924f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7144005bc0 tx=0x7f7144010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a135143ec2ba60e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f713a7fc640 1 -- 172.21.15.26:0/3099216590 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714400dbb0 con 0x55d9ad2dfb60 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f713a7fc640 1 -- 172.21.15.26:0/3099216590 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714400dd50 con 0x55d9ad2dfb60 2024-04-24T23:21:59.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f713a7fc640 1 -- 172.21.15.26:0/3099216590 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144020670 con 0x55d9ad2dfb60 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7159bc6740 1 -- 172.21.15.26:0/3099216590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 msgr2=0x55d9ad2924f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7159bc6740 1 --2- 172.21.15.26:0/3099216590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2924f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7144005bc0 tx=0x7f7144010000 comp rx=0 tx=0).stop 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7159bc6740 1 -- 172.21.15.26:0/3099216590 shutdown_connections 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7159bc6740 1 --2- 172.21.15.26:0/3099216590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2540 0x55d9ad297350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7159bc6740 1 --2- 172.21.15.26:0/3099216590 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c2c80 0x55d9ad2c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7159bc6740 1 --2- 172.21.15.26:0/3099216590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2924f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.864+0000 7f7159bc6740 1 -- 172.21.15.26:0/3099216590 >> 172.21.15.26:0/3099216590 conn(0x55d9ad294760 msgr2=0x55d9ad2dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.865+0000 7f7159bc6740 1 -- 172.21.15.26:0/3099216590 shutdown_connections 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.865+0000 7f7159bc6740 1 -- 172.21.15.26:0/3099216590 wait complete. 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.865+0000 7f7159bc6740 1 -- start start 2024-04-24T23:21:59.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.865+0000 7f7159bc6740 1 Processor -- start 2024-04-24T23:21:59.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 0x55d9ad2c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f7159bc6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d9ad296910 con 0x55d9ad2dfb60 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f7159bc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d9ad1b6cf0 con 0x55d9ad2c2c80 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f7159bc6740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d9ad295d00 con 0x55d9ad2c5430 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f7159bc6740 10 client.0 _ll_register_callbacks cb 0x55d9ad1b66e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f714ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f714ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37142/0 (socket says 172.21.15.26:37142) 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f714ffff640 1 -- 172.21.15.26:0/1085575019 learned_addr learned my addr 172.21.15.26:0/1085575019 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f7155379640 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 0x55d9ad2c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.866+0000 7f7154b78640 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f714ffff640 1 -- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 msgr2=0x55d9ad2c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f714ffff640 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 0x55d9ad2c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f714ffff640 1 -- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2c80 msgr2=0x55d9ad2bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f714ffff640 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2bdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f714ffff640 1 -- 172.21.15.26:0/1085575019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9acfd96b0 con 0x55d9ad2dfb60 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f714ffff640 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f0b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f713c061bc0 tx=0x7f713c059140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67523c8cf2f4d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f712f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f712f7fe640 1 -- 172.21.15.26:0/1085575019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c071070 con 0x55d9ad2dfb60 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f712f7fe640 1 -- 172.21.15.26:0/1085575019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f713c058b60 con 0x55d9ad2dfb60 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f7159bc6740 1 -- 172.21.15.26:0/1085575019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f714400dd50 con 0x55d9ad2dfb60 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f712f7fe640 1 -- 172.21.15.26:0/1085575019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f713c021040 con 0x55d9ad2dfb60 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f7159bc6740 10 client.5328 fetch_fsmap learned FSMap version 21 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f7159bc6740 1 -- 172.21.15.26:0/1085575019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55d9ad2d3f60 con 0x55d9ad2dfb60 2024-04-24T23:21:59.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f712f7fe640 1 -- 172.21.15.26:0/1085575019 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714400dd50 con 0x55d9ad2dfb60 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f712f7fe640 1 -- 172.21.15.26:0/1085575019 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 3534+0+0 (secure 0 0 0) 0x7f713c089b20 con 0x55d9ad2dfb60 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f7159bc6740 10 client.5328 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f7159bc6740 10 client.5328 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f7159bc6740 20 client.5328 populate_metadata read hostname 'smithi026' 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f7159bc6740 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55d9ad2edb80 0x55d9ad2edf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f7159bc6740 4 client.5328 mds_command: new command op to 4867 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.867+0000 7f7159bc6740 1 -- 172.21.15.26:0/1085575019 --> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9ad208440 con 0x55d9ad2edb80 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.868+0000 7f7154b78640 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55d9ad2edb80 0x55d9ad2edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.868+0000 7f712f7fe640 10 client.5328 ms_handle_connect on v2:172.21.15.29:6833/3972561683 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.868+0000 7f7154b78640 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55d9ad2edb80 0x55d9ad2edf60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd9c71bedd706de5 server_cookie=a0db3703ac8763b3 in_seq=0 out_seq=0 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f712f7fe640 1 -- 172.21.15.26:0/1085575019 <== mds.1 v2:172.21.15.29:6833/3972561683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9ad208440 con 0x55d9ad2edb80 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f712f7fe640 10 client.5328 handle_command_reply: tid=0 2024-04-24T23:21:59.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 client.5328 shutdown 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 -- 172.21.15.26:0/1085575019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 msgr2=0x55d9ad2f0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f0b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f713c061bc0 tx=0x7f713c059140 comp rx=0 tx=0).stop 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 -- 172.21.15.26:0/1085575019 shutdown_connections 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55d9ad2edb80 0x55d9ad2edf60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 0x55d9ad2c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f0b50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f712f7fe640 0 client.5328 ms_handle_reset on v2:172.21.15.29:6833/3972561683 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 --2- 172.21.15.26:0/1085575019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 -- 172.21.15.26:0/1085575019 >> 172.21.15.26:0/1085575019 conn(0x55d9ad2c2540 msgr2=0x55d9ad291250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 -- 172.21.15.26:0/1085575019 shutdown_connections 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 1 -- 172.21.15.26:0/1085575019 wait complete. 2024-04-24T23:21:59.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.869+0000 7f7159bc6740 20 client.5328 trim_cache size 0 max 16384 2024-04-24T23:21:59.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.887+0000 7f7159bc6740 1 Processor -- start 2024-04-24T23:21:59.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.887+0000 7f7159bc6740 1 -- start start 2024-04-24T23:21:59.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad292140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c5430 0x55d9ad2f0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2dfb60 0x55d9ad291250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7159bc6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71501ffa60 con 0x55d9ad2c2c80 2024-04-24T23:21:59.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7159bc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f71502000f0 con 0x55d9ad2c5430 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7159bc6740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f71500cbeb0 con 0x55d9ad2dfb60 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7154b78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad292140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7154b78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad292140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37156/0 (socket says 172.21.15.26:37156) 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7154b78640 1 -- 172.21.15.26:0/3193659674 learned_addr learned my addr 172.21.15.26:0/3193659674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f714ffff640 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c5430 0x55d9ad2f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7155379640 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2dfb60 0x55d9ad291250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7154b78640 1 -- 172.21.15.26:0/3193659674 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2dfb60 msgr2=0x55d9ad291250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7154b78640 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2dfb60 0x55d9ad291250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7154b78640 1 -- 172.21.15.26:0/3193659674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c5430 msgr2=0x55d9ad2f0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.888+0000 7f7154b78640 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c5430 0x55d9ad2f0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7154b78640 1 -- 172.21.15.26:0/3193659674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9ad295890 con 0x55d9ad2c2c80 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f714ffff640 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c5430 0x55d9ad2f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:59.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7154b78640 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad292140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7144005880 tx=0x7f7144007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432f4ea4a0ec3490 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7139ffb640 1 -- 172.21.15.26:0/3193659674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714400dbb0 con 0x55d9ad2c2c80 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7139ffb640 1 -- 172.21.15.26:0/3193659674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7144034070 con 0x55d9ad2c2c80 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7139ffb640 1 -- 172.21.15.26:0/3193659674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714402f230 con 0x55d9ad2c2c80 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7159bc6740 1 -- 172.21.15.26:0/3193659674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 msgr2=0x55d9ad292140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7159bc6740 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad292140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7144005880 tx=0x7f7144007000 comp rx=0 tx=0).stop 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7159bc6740 1 -- 172.21.15.26:0/3193659674 shutdown_connections 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7159bc6740 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2dfb60 0x55d9ad291250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7159bc6740 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2c5430 0x55d9ad2f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7159bc6740 1 --2- 172.21.15.26:0/3193659674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad292140 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7159bc6740 1 -- 172.21.15.26:0/3193659674 >> 172.21.15.26:0/3193659674 conn(0x55d9ad2c2540 msgr2=0x55d9ad2dce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7159bc6740 1 -- 172.21.15.26:0/3193659674 shutdown_connections 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.889+0000 7f7159bc6740 1 -- 172.21.15.26:0/3193659674 wait complete. 2024-04-24T23:21:59.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.890+0000 7f7159bc6740 1 -- start start 2024-04-24T23:21:59.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.890+0000 7f7159bc6740 1 Processor -- start 2024-04-24T23:21:59.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2ec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 0x55d9ad2eff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7159bc6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7159bc6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d9ad2f1090 con 0x55d9ad2c2c80 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7159bc6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d9ad2c33e0 con 0x55d9ad2dfb60 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7159bc6740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d9ad2d4e90 con 0x55d9ad2c5430 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7159bc6740 10 client.0 _ll_register_callbacks cb 0x7f713c059370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7154b78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2ec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7154b78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2ec8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37166/0 (socket says 172.21.15.26:37166) 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7154b78640 1 -- 172.21.15.26:0/902851201 learned_addr learned my addr 172.21.15.26:0/902851201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7155379640 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f714ffff640 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 0x55d9ad2eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7154b78640 1 -- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 msgr2=0x55d9ad2eff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7154b78640 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 0x55d9ad2eff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7154b78640 1 -- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2dfb60 msgr2=0x55d9ad2f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7154b78640 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7154b78640 1 -- 172.21.15.26:0/902851201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9ad2d2ec0 con 0x55d9ad2c2c80 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f714ffff640 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 0x55d9ad2eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.891+0000 7f7155379640 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7154b78640 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2ec8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7144001550 tx=0x7f7144012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28c2d50b7b7feb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f712f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f712f7fe640 1 -- 172.21.15.26:0/902851201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714403d060 con 0x55d9ad2c2c80 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f712f7fe640 1 -- 172.21.15.26:0/902851201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f714402f030 con 0x55d9ad2c2c80 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7159bc6740 1 -- 172.21.15.26:0/902851201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7144034070 con 0x55d9ad2c2c80 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f712f7fe640 1 -- 172.21.15.26:0/902851201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f714402f230 con 0x55d9ad2c2c80 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7159bc6740 10 client.5334 fetch_fsmap learned FSMap version 21 2024-04-24T23:21:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7159bc6740 1 -- 172.21.15.26:0/902851201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55d9ad2bc2f0 con 0x55d9ad2c2c80 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f712f7fe640 1 -- 172.21.15.26:0/902851201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7144034070 con 0x55d9ad2c2c80 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f712f7fe640 1 -- 172.21.15.26:0/902851201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 3534+0+0 (secure 0 0 0) 0x7f714402f3d0 con 0x55d9ad2c2c80 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7159bc6740 10 client.5334 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7159bc6740 10 client.5334 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7159bc6740 20 client.5334 populate_metadata read hostname 'smithi026' 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7159bc6740 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55d9ad2ec000 0x55d9ad2fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7159bc6740 4 client.5334 mds_command: new command op to 4867 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.892+0000 7f7159bc6740 1 -- 172.21.15.26:0/902851201 --> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] -- command(tid 0: {"prefix": "status"}) -- 0x55d9ad202790 con 0x55d9ad2ec000 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f714ffff640 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55d9ad2ec000 0x55d9ad2fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f712f7fe640 10 client.5334 ms_handle_connect on v2:172.21.15.29:6833/3972561683 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f714ffff640 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55d9ad2ec000 0x55d9ad2fffe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e24e53c97f8db35 server_cookie=82b26b3d9475c9e2 in_seq=0 out_seq=0 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f712f7fe640 1 -- 172.21.15.26:0/902851201 <== mds.1 v2:172.21.15.29:6833/3972561683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d9ad202790 con 0x55d9ad2ec000 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f712f7fe640 10 client.5334 handle_command_reply: tid=0 2024-04-24T23:21:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 client.5334 shutdown 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 -- 172.21.15.26:0/902851201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 msgr2=0x55d9ad2ec8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2ec8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7144001550 tx=0x7f7144012000 comp rx=0 tx=0).stop 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 -- 172.21.15.26:0/902851201 shutdown_connections 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55d9ad2ec000 0x55d9ad2fffe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d9ad2dfb60 0x55d9ad2f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d9ad2c5430 0x55d9ad2eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f712f7fe640 0 client.5334 ms_handle_reset on v2:172.21.15.29:6833/3972561683 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 --2- 172.21.15.26:0/902851201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9ad2c2c80 0x55d9ad2ec8f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 -- 172.21.15.26:0/902851201 >> 172.21.15.26:0/902851201 conn(0x55d9ad2c2540 msgr2=0x55d9ad297fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 -- 172.21.15.26:0/902851201 shutdown_connections 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.893+0000 7f7159bc6740 1 -- 172.21.15.26:0/902851201 wait complete. 2024-04-24T23:21:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7159bc6740 20 client.5334 trim_cache size 0 max 16384 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4867, 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 2.8636222400000002, 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 21, 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 33, 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 33, 2024-04-24T23:21:59.902 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 18.796720573999998 2024-04-24T23:21:59.903 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:21:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 -- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be30 msgr2=0x7f713406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be30 0x7f713406e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71440013b0 tx=0x7f7144009040 comp rx=0 tx=0).stop 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 -- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500d4870 msgr2=0x7f71501513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500d4870 0x7f71501513d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f713c00e9b0 tx=0x7f713c00ee70 comp rx=0 tx=0).stop 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 -- 172.21.15.26:0/2096610593 shutdown_connections 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be30 0x7f713406e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 0x7f7150152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500d4870 0x7f71501513d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 --2- 172.21.15.26:0/2096610593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500b3130 0x7f715015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 -- 172.21.15.26:0/2096610593 >> 172.21.15.26:0/2096610593 conn(0x7f715008dbe0 msgr2=0x7f715006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 -- 172.21.15.26:0/2096610593 shutdown_connections 2024-04-24T23:21:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:21:59.894+0000 7f7156ddc640 1 -- 172.21.15.26:0/2096610593 wait complete. 2024-04-24T23:21:59.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:22:00.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.006+0000 7f0668cd4640 1 Processor -- start 2024-04-24T23:22:00.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.006+0000 7f0668cd4640 1 -- start start 2024-04-24T23:22:00.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.007+0000 7f0668cd4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640b3130 0x7f06640b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.007+0000 7f0668cd4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06640d32f0 con 0x7f06640d4870 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.007+0000 7f0668cd4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06640d0270 con 0x7f066415e570 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.007+0000 7f0668cd4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f06640d1ab0 con 0x7f06640b3130 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.007+0000 7f065bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f06640d4870 0x7f06640b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46834/0 (socket says 172.21.15.26:46834) 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.007+0000 7f065bfff640 1 -- 172.21.15.26:0/3600291997 learned_addr learned my addr 172.21.15.26:0/3600291997 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.008+0000 7f0662d9d640 1 --2- 172.21.15.26:0/3600291997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640b3130 0x7f06640b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.008+0000 7f0661d9b640 1 -- 172.21.15.26:0/3600291997 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4157683861 0 0) 0x7f064c001110 con 0x7f06640d4870 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.008+0000 7f0661d9b640 1 -- 172.21.15.26:0/3600291997 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0650003520 con 0x7f06640d4870 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.008+0000 7f0661d9b640 1 -- 172.21.15.26:0/3600291997 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3051441687 0 0) 0x7f06640d0270 con 0x7f066415e570 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.008+0000 7f0661d9b640 1 -- 172.21.15.26:0/3600291997 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f064c001110 con 0x7f066415e570 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.008+0000 7f0662d9d640 1 -- 172.21.15.26:0/3600291997 >> v1:172.21.15.29:6789/0 conn(0x7f066415e570 legacy=0x7f06640d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.008+0000 7f0662d9d640 1 -- 172.21.15.26:0/3600291997 >> v1:172.21.15.26:6789/0 conn(0x7f06640d4870 legacy=0x7f06640b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.008+0000 7f0662d9d640 1 -- 172.21.15.26:0/3600291997 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06640d2d30 con 0x7f06640b3130 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0662d9d640 1 --2- 172.21.15.26:0/3600291997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640b3130 0x7f06640b4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f065c009dc0 tx=0x7f065c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f328925c0ec41e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0661d9b640 1 -- 172.21.15.26:0/3600291997 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c007740 con 0x7f06640b3130 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0661d9b640 1 -- 172.21.15.26:0/3600291997 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c0078e0 con 0x7f06640b3130 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0661d9b640 1 -- 172.21.15.26:0/3600291997 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c007cb0 con 0x7f06640b3130 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0668cd4640 1 -- 172.21.15.26:0/3600291997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640b3130 msgr2=0x7f06640b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0668cd4640 1 --2- 172.21.15.26:0/3600291997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640b3130 0x7f06640b4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f065c009dc0 tx=0x7f065c02a6d0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0668cd4640 1 -- 172.21.15.26:0/3600291997 shutdown_connections 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0668cd4640 1 --2- 172.21.15.26:0/3600291997 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640b3130 0x7f06640b4810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0668cd4640 1 -- 172.21.15.26:0/3600291997 >> 172.21.15.26:0/3600291997 conn(0x7f066408dbe0 msgr2=0x7f066409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.009+0000 7f0668cd4640 1 -- 172.21.15.26:0/3600291997 shutdown_connections 2024-04-24T23:22:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.010+0000 7f0668cd4640 1 -- 172.21.15.26:0/3600291997 wait complete. 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.011+0000 7f0668cd4640 1 Processor -- start 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.011+0000 7f0668cd4640 1 -- start start 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.011+0000 7f0668cd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640b3130 0x7f06640b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.011+0000 7f0668cd4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640d4870 0x7f06640b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.011+0000 7f0668cd4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f066415e570 0x7f06640b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.011+0000 7f0668cd4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06640cfa70 con 0x7f06640b3130 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f0668cd4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f06640d2ae0 con 0x7f066415e570 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f0668cd4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f06640cfd50 con 0x7f06640d4870 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f066259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640b3130 0x7f06640b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f066259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640b3130 0x7f06640b98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37172/0 (socket says 172.21.15.26:37172) 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f066259c640 1 -- 172.21.15.26:0/1837714862 learned_addr learned my addr 172.21.15.26:0/1837714862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f0662d9d640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f066415e570 0x7f06640b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f065bfff640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640d4870 0x7f06640b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f066259c640 1 -- 172.21.15.26:0/1837714862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640d4870 msgr2=0x7f06640b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f066259c640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640d4870 0x7f06640b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f066259c640 1 -- 172.21.15.26:0/1837714862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f066415e570 msgr2=0x7f06640b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f066259c640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f066415e570 0x7f06640b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f066259c640 1 -- 172.21.15.26:0/1837714862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066415fb40 con 0x7f06640b3130 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f066259c640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640b3130 0x7f06640b98c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f065400c430 tx=0x7f065400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f065affd640 1 -- 172.21.15.26:0/1837714862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654017070 con 0x7f06640b3130 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f065affd640 1 -- 172.21.15.26:0/1837714862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06540052f0 con 0x7f06640b3130 2024-04-24T23:22:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f065affd640 1 -- 172.21.15.26:0/1837714862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654003830 con 0x7f06640b3130 2024-04-24T23:22:00.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.012+0000 7f0668cd4640 1 -- 172.21.15.26:0/1837714862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066415fde0 con 0x7f06640b3130 2024-04-24T23:22:00.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.013+0000 7f0668cd4640 1 -- 172.21.15.26:0/1837714862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06641603a0 con 0x7f06640b3130 2024-04-24T23:22:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.015+0000 7f065affd640 1 -- 172.21.15.26:0/1837714862 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f065401a070 con 0x7f06640b3130 2024-04-24T23:22:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.015+0000 7f065affd640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f064406be30 0x7f064406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.015+0000 7f065affd640 1 -- 172.21.15.26:0/1837714862 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f065408a2b0 con 0x7f06640b3130 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.015+0000 7f065bfff640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f064406be30 0x7f064406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.016+0000 7f065bfff640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f064406be30 0x7f064406e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f066415b690 tx=0x7f064c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.016+0000 7f066b9e7740 1 Processor -- start 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.016+0000 7f066b9e7740 1 -- start start 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 0x557b36f7d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fadc80 0x557b36fad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fad540 0x557b36f82350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f066b9e7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557b36ee5f00 con 0x557b36fad540 2024-04-24T23:22:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f066b9e7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557b36e8e750 con 0x557b36fcab60 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f066b9e7740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x557b36ee1ec0 con 0x557b36fadc80 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0662d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fad540 0x557b36f82350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0662d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fad540 0x557b36f82350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37184/0 (socket says 172.21.15.26:37184) 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0662d9d640 1 -- 172.21.15.26:0/3384112027 learned_addr learned my addr 172.21.15.26:0/3384112027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f066259c640 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 0x557b36f7d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f065bfff640 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fadc80 0x557b36fad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0662d9d640 1 -- 172.21.15.26:0/3384112027 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fadc80 msgr2=0x557b36fad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0662d9d640 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fadc80 0x557b36fad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0662d9d640 1 -- 172.21.15.26:0/3384112027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 msgr2=0x557b36f7d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0662d9d640 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 0x557b36f7d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0662d9d640 1 -- 172.21.15.26:0/3384112027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b36bfa2d0 con 0x557b36fad540 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f066259c640 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 0x557b36f7d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0662d9d640 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fad540 0x557b36f82350 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f065c009eb0 tx=0x7f065c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494f8798e361cf06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0643fff640 1 -- 172.21.15.26:0/3384112027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c007800 con 0x557b36fad540 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0643fff640 1 -- 172.21.15.26:0/3384112027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c0079a0 con 0x557b36fad540 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.017+0000 7f0643fff640 1 -- 172.21.15.26:0/3384112027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c007d50 con 0x557b36fad540 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.018+0000 7f066b9e7740 1 -- 172.21.15.26:0/3384112027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fad540 msgr2=0x557b36f82350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.018+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fad540 0x557b36f82350 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f065c009eb0 tx=0x7f065c02e000 comp rx=0 tx=0).stop 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.018+0000 7f066b9e7740 1 -- 172.21.15.26:0/3384112027 shutdown_connections 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.018+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fad540 0x557b36f82350 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.018+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fadc80 0x557b36fad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.018+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3384112027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 0x557b36f7d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.018+0000 7f066b9e7740 1 -- 172.21.15.26:0/3384112027 >> 172.21.15.26:0/3384112027 conn(0x557b36f7f760 msgr2=0x557b36fc8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.018+0000 7f066b9e7740 1 -- 172.21.15.26:0/3384112027 shutdown_connections 2024-04-24T23:22:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.018+0000 7f066b9e7740 1 -- 172.21.15.26:0/3384112027 wait complete. 2024-04-24T23:22:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.019+0000 7f066b9e7740 1 -- start start 2024-04-24T23:22:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.019+0000 7f066b9e7740 1 Processor -- start 2024-04-24T23:22:00.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 0x557b36fb0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fcab60 0x557b36f81d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fae170 0x557b36f82250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f066b9e7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557b36f81910 con 0x557b36fae170 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f066b9e7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557b36ea1cf0 con 0x557b36fadc80 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f066b9e7740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x557b36f80d00 con 0x557b36fcab60 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f066b9e7740 10 client.0 _ll_register_callbacks cb 0x557b36ea16e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f0662d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fae170 0x557b36f82250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f0662d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fae170 0x557b36f82250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37192/0 (socket says 172.21.15.26:37192) 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f0662d9d640 1 -- 172.21.15.26:0/512402522 learned_addr learned my addr 172.21.15.26:0/512402522 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f066259c640 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 0x557b36fb0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f065bfff640 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fcab60 0x557b36f81d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f0662d9d640 1 -- 172.21.15.26:0/512402522 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fcab60 msgr2=0x557b36f81d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f0662d9d640 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fcab60 0x557b36f81d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f0662d9d640 1 -- 172.21.15.26:0/512402522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 msgr2=0x557b36fb0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f0662d9d640 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 0x557b36fb0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f0662d9d640 1 -- 172.21.15.26:0/512402522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b36cc46b0 con 0x557b36fae170 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f065bfff640 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fcab60 0x557b36f81d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f0662d9d640 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fae170 0x557b36f82250 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f065c02a710 tx=0x7f065c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14b41c4c26b94ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f06417fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.020+0000 7f06417fa640 1 -- 172.21.15.26:0/512402522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c003ba0 con 0x557b36fae170 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f06417fa640 1 -- 172.21.15.26:0/512402522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f065c003d40 con 0x557b36fae170 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066b9e7740 1 -- 172.21.15.26:0/512402522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f065c0079a0 con 0x557b36fae170 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f06417fa640 1 -- 172.21.15.26:0/512402522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f065c004040 con 0x557b36fae170 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066b9e7740 10 client.5346 fetch_fsmap learned FSMap version 21 2024-04-24T23:22:00.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066b9e7740 1 -- 172.21.15.26:0/512402522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x557b36fd93a0 con 0x557b36fae170 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f06417fa640 1 -- 172.21.15.26:0/512402522 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c0079a0 con 0x557b36fae170 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f06417fa640 1 -- 172.21.15.26:0/512402522 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 3534+0+0 (secure 0 0 0) 0x7f065c044070 con 0x557b36fae170 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066b9e7740 10 client.5346 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066b9e7740 10 client.5346 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066b9e7740 20 client.5346 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066b9e7740 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x557b36fd8750 0x557b36fd8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066b9e7740 4 client.5346 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066b9e7740 1 -- 172.21.15.26:0/512402522 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b36ef3440 con 0x557b36fd8750 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f066259c640 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x557b36fd8750 0x557b36fd8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.021+0000 7f06417fa640 10 client.5346 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066259c640 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x557b36fd8750 0x557b36fd8b30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f341df67067977a server_cookie=a3c61056319f700a in_seq=0 out_seq=0 2024-04-24T23:22:00.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f06417fa640 1 -- 172.21.15.26:0/512402522 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b36ef3440 con 0x557b36fd8750 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f06417fa640 10 client.5346 handle_command_reply: tid=0 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 client.5346 shutdown 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 -- 172.21.15.26:0/512402522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fae170 msgr2=0x557b36f82250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fae170 0x557b36f82250 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f065c02a710 tx=0x7f065c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 -- 172.21.15.26:0/512402522 shutdown_connections 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x557b36fd8750 0x557b36fd8b30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fae170 0x557b36f82250 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fcab60 0x557b36f81d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f06417fa640 0 client.5346 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 --2- 172.21.15.26:0/512402522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 0x557b36fb0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 -- 172.21.15.26:0/512402522 >> 172.21.15.26:0/512402522 conn(0x557b36fad540 msgr2=0x557b36fc8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 -- 172.21.15.26:0/512402522 shutdown_connections 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.022+0000 7f066b9e7740 1 -- 172.21.15.26:0/512402522 wait complete. 2024-04-24T23:22:00.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.023+0000 7f066b9e7740 20 client.5346 trim_cache size 0 max 16384 2024-04-24T23:22:00.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.041+0000 7f066b9e7740 1 Processor -- start 2024-04-24T23:22:00.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.041+0000 7f066b9e7740 1 -- start start 2024-04-24T23:22:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fadc80 0x557b36fc55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 0x557b36fd8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 0x557b36fafb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f066b9e7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06641ff800 con 0x557b36fadc80 2024-04-24T23:22:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f066b9e7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f06641ffed0 con 0x557b36fcab60 2024-04-24T23:22:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f066b9e7740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f06641fe8a0 con 0x557b36fae170 2024-04-24T23:22:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f066259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fadc80 0x557b36fc55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f066259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fadc80 0x557b36fc55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37204/0 (socket says 172.21.15.26:37204) 2024-04-24T23:22:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f066259c640 1 -- 172.21.15.26:0/468467029 learned_addr learned my addr 172.21.15.26:0/468467029 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f0662d9d640 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 0x557b36fafb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.042+0000 7f065bfff640 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 0x557b36fd8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f066259c640 1 -- 172.21.15.26:0/468467029 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 msgr2=0x557b36fd8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f066259c640 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 0x557b36fd8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f066259c640 1 -- 172.21.15.26:0/468467029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 msgr2=0x557b36fafb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f066259c640 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 0x557b36fafb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f066259c640 1 -- 172.21.15.26:0/468467029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b36faf4d0 con 0x557b36fadc80 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f065bfff640 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 0x557b36fd8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f066259c640 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fadc80 0x557b36fc55f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f065400a040 tx=0x7f06540886b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6db1f56bc640107 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f06437fe640 1 -- 172.21.15.26:0/468467029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654056800 con 0x557b36fadc80 2024-04-24T23:22:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f06437fe640 1 -- 172.21.15.26:0/468467029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0654077040 con 0x557b36fadc80 2024-04-24T23:22:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.043+0000 7f06437fe640 1 -- 172.21.15.26:0/468467029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654054b70 con 0x557b36fadc80 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.044+0000 7f066b9e7740 1 -- 172.21.15.26:0/468467029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fadc80 msgr2=0x557b36fc55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.044+0000 7f066b9e7740 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fadc80 0x557b36fc55f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f065400a040 tx=0x7f06540886b0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.044+0000 7f066b9e7740 1 -- 172.21.15.26:0/468467029 shutdown_connections 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.044+0000 7f066b9e7740 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fcab60 0x557b36fafb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.044+0000 7f066b9e7740 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 0x557b36fd8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.044+0000 7f066b9e7740 1 --2- 172.21.15.26:0/468467029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fadc80 0x557b36fc55f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.044+0000 7f066b9e7740 1 -- 172.21.15.26:0/468467029 >> 172.21.15.26:0/468467029 conn(0x557b36fad540 msgr2=0x557b36fb0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.044+0000 7f066b9e7740 1 -- 172.21.15.26:0/468467029 shutdown_connections 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.044+0000 7f066b9e7740 1 -- 172.21.15.26:0/468467029 wait complete. 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.045+0000 7f066b9e7740 1 -- start start 2024-04-24T23:22:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.045+0000 7f066b9e7740 1 Processor -- start 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 0x557b370420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 0x557b36fc5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f066b9e7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fcab60 0x557b36f81a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f066b9e7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557b36fc7830 con 0x557b36fcab60 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f066b9e7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557b36f81910 con 0x557b36fadc80 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f066b9e7740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x557b36fc62f0 con 0x557b36fae170 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f066b9e7740 10 client.0 _ll_register_callbacks cb 0x7f065c009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f0662d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fcab60 0x557b36f81a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f0662d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fcab60 0x557b36f81a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37208/0 (socket says 172.21.15.26:37208) 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f0662d9d640 1 -- 172.21.15.26:0/3570005045 learned_addr learned my addr 172.21.15.26:0/3570005045 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f065bfff640 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 0x557b36fc5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f066259c640 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 0x557b370420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f0662d9d640 1 -- 172.21.15.26:0/3570005045 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 msgr2=0x557b36fc5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f0662d9d640 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 0x557b36fc5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f0662d9d640 1 -- 172.21.15.26:0/3570005045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 msgr2=0x557b370420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f0662d9d640 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 0x557b370420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f0662d9d640 1 -- 172.21.15.26:0/3570005045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b36fd6700 con 0x557b36fcab60 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f0662d9d640 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fcab60 0x557b36f81a90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x557b36fa9000 tx=0x7f065c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a800851f5448117a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f06417fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f06417fa640 1 -- 172.21.15.26:0/3570005045 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c061060 con 0x557b36fcab60 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f06417fa640 1 -- 172.21.15.26:0/3570005045 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f065c007b90 con 0x557b36fcab60 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.046+0000 7f066b9e7740 1 -- 172.21.15.26:0/3570005045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0654077040 con 0x557b36fcab60 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f06417fa640 1 -- 172.21.15.26:0/3570005045 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f065c067050 con 0x557b36fcab60 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066b9e7740 10 client.5352 fetch_fsmap learned FSMap version 21 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066b9e7740 1 -- 172.21.15.26:0/3570005045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x557b36fda2b0 con 0x557b36fcab60 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f06417fa640 1 -- 172.21.15.26:0/3570005045 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0654077040 con 0x557b36fcab60 2024-04-24T23:22:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f06417fa640 1 -- 172.21.15.26:0/3570005045 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 3534+0+0 (secure 0 0 0) 0x7f065c06a020 con 0x557b36fcab60 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066b9e7740 10 client.5352 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066b9e7740 10 client.5352 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066b9e7740 20 client.5352 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x557b36fa9c00 0x557b36fcb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066b9e7740 4 client.5352 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066b9e7740 1 -- 172.21.15.26:0/3570005045 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "status"}) -- 0x557b36f7ebb0 con 0x557b36fa9c00 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066259c640 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x557b36fa9c00 0x557b36fcb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f06417fa640 10 client.5352 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f066259c640 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x557b36fa9c00 0x557b36fcb420 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ab0721c618ee62 server_cookie=d1f5e2db042d0c29 in_seq=0 out_seq=0 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f06417fa640 1 -- 172.21.15.26:0/3570005045 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x557b36f7ebb0 con 0x557b36fa9c00 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.047+0000 7f06417fa640 10 client.5352 handle_command_reply: tid=0 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 client.5352 shutdown 2024-04-24T23:22:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 -- 172.21.15.26:0/3570005045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fcab60 msgr2=0x557b36f81a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fcab60 0x557b36f81a90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x557b36fa9000 tx=0x7f065c02f000 comp rx=0 tx=0).stop 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 -- 172.21.15.26:0/3570005045 shutdown_connections 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x557b36fa9c00 0x557b36fcb420 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b36fcab60 0x557b36f81a90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557b36fae170 0x557b36fc5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 --2- 172.21.15.26:0/3570005045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557b36fadc80 0x557b370420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f06417fa640 0 client.5352 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 -- 172.21.15.26:0/3570005045 >> 172.21.15.26:0/3570005045 conn(0x557b36fad540 msgr2=0x557b36f7c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 -- 172.21.15.26:0/3570005045 shutdown_connections 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 1 -- 172.21.15.26:0/3570005045 wait complete. 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f066b9e7740 20 client.5352 trim_cache size 0 max 16384 2024-04-24T23:22:00.056 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5088, 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 11.046051705, 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 21, 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 33, 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 33, 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 18.996517505 2024-04-24T23:22:00.057 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:22:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f0668cd4640 1 -- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f064406be30 msgr2=0x7f064406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f0668cd4640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f064406be30 0x7f064406e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f066415b690 tx=0x7f064c009040 comp rx=0 tx=0).stop 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f0668cd4640 1 -- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640b3130 msgr2=0x7f06640b98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f0668cd4640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640b3130 0x7f06640b98c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f065400c430 tx=0x7f065400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f0668cd4640 1 -- 172.21.15.26:0/1837714862 shutdown_connections 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f0668cd4640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f064406be30 0x7f064406e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f0668cd4640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f066415e570 0x7f06640b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f0668cd4640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f06640d4870 0x7f06640b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.048+0000 7f0668cd4640 1 --2- 172.21.15.26:0/1837714862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640b3130 0x7f06640b98c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.049+0000 7f0668cd4640 1 -- 172.21.15.26:0/1837714862 >> 172.21.15.26:0/1837714862 conn(0x7f066408dbe0 msgr2=0x7f06640d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.049+0000 7f0668cd4640 1 -- 172.21.15.26:0/1837714862 shutdown_connections 2024-04-24T23:22:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.049+0000 7f0668cd4640 1 -- 172.21.15.26:0/1837714862 wait complete. 2024-04-24T23:22:00.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:22:00.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.160+0000 7f55f50ee640 1 Processor -- start 2024-04-24T23:22:00.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.160+0000 7f55f50ee640 1 -- start start 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55f50ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55f015eb00 0x7f55f015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55f50ee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f015f450 con 0x7f55f0143c40 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55f50ee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f00cd0b0 con 0x7f55f00cf870 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55f50ee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f55f0071b70 con 0x7f55f015eb00 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55ef59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55f015eb00 0x7f55f015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55eed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f55f0143c40 0x7f55f00cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46848/0 (socket says 172.21.15.26:46848) 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55eed9d640 1 -- 172.21.15.26:0/3510442583 learned_addr learned my addr 172.21.15.26:0/3510442583 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55ef59e640 1 --2- 172.21.15.26:0/3510442583 >> v2:172.21.15.26:3300/0 conn(0x7f55f015eb00 0x7f55f015ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37222/0 (socket says 172.21.15.26:37222) 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55ef59e640 1 -- 172.21.15.26:0/3510442583 >> v1:172.21.15.29:6789/0 conn(0x7f55f00cf870 legacy=0x7f55f0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55ef59e640 1 -- 172.21.15.26:0/3510442583 >> v1:172.21.15.26:6789/0 conn(0x7f55f0143c40 legacy=0x7f55f00cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55ef59e640 1 -- 172.21.15.26:0/3510442583 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f0071240 con 0x7f55f015eb00 2024-04-24T23:22:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.161+0000 7f55ee59c640 1 -- 172.21.15.26:0/3510442583 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657476888 0 0) 0x7f55e0001110 con 0x7f55f0143c40 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.162+0000 7f55ef59e640 1 --2- 172.21.15.26:0/3510442583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f015eb00 0x7f55f015ef10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009d90 tx=0x7f55e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c0528a5fd7811b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.162+0000 7f55ee59c640 1 -- 172.21.15.26:0/3510442583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8007740 con 0x7f55f015eb00 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.162+0000 7f55ee59c640 1 -- 172.21.15.26:0/3510442583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e80078e0 con 0x7f55f015eb00 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.162+0000 7f55ee59c640 1 -- 172.21.15.26:0/3510442583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8007cb0 con 0x7f55f015eb00 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.162+0000 7f55f50ee640 1 -- 172.21.15.26:0/3510442583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f015eb00 msgr2=0x7f55f015ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.162+0000 7f55f50ee640 1 --2- 172.21.15.26:0/3510442583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f015eb00 0x7f55f015ef10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009d90 tx=0x7f55e802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.163+0000 7f55f50ee640 1 -- 172.21.15.26:0/3510442583 shutdown_connections 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.163+0000 7f55f50ee640 1 --2- 172.21.15.26:0/3510442583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f015eb00 0x7f55f015ef10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.163+0000 7f55f50ee640 1 -- 172.21.15.26:0/3510442583 >> 172.21.15.26:0/3510442583 conn(0x7f55f006b300 msgr2=0x7f55f006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.163+0000 7f55f50ee640 1 -- 172.21.15.26:0/3510442583 shutdown_connections 2024-04-24T23:22:00.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.163+0000 7f55f50ee640 1 -- 172.21.15.26:0/3510442583 wait complete. 2024-04-24T23:22:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.164+0000 7f55f50ee640 1 Processor -- start 2024-04-24T23:22:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.164+0000 7f55f50ee640 1 -- start start 2024-04-24T23:22:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55f50ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f00cf870 0x7f55f01f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55f50ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f0143c40 0x7f55f01f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55f50ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55f015eb00 0x7f55f0139d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55f50ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0144020 con 0x7f55f00cf870 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55f50ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0071800 con 0x7f55f0143c40 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55f50ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f55f00b2e80 con 0x7f55f015eb00 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55eed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f00cf870 0x7f55f01f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55eed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f00cf870 0x7f55f01f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37226/0 (socket says 172.21.15.26:37226) 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55eed9d640 1 -- 172.21.15.26:0/1520555233 learned_addr learned my addr 172.21.15.26:0/1520555233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55ef59e640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55f015eb00 0x7f55f0139d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55e7fff640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f0143c40 0x7f55f01f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55eed9d640 1 -- 172.21.15.26:0/1520555233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55f015eb00 msgr2=0x7f55f0139d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55eed9d640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55f015eb00 0x7f55f0139d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55eed9d640 1 -- 172.21.15.26:0/1520555233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f0143c40 msgr2=0x7f55f01f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55eed9d640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f0143c40 0x7f55f01f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55eed9d640 1 -- 172.21.15.26:0/1520555233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f015cc40 con 0x7f55f00cf870 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55e7fff640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f0143c40 0x7f55f01f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55eed9d640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f00cf870 0x7f55f01f84f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f55e000c780 tx=0x7f55e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.165+0000 7f55e77fe640 1 -- 172.21.15.26:0/1520555233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0018070 con 0x7f55f00cf870 2024-04-24T23:22:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.166+0000 7f55e77fe640 1 -- 172.21.15.26:0/1520555233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e00072d0 con 0x7f55f00cf870 2024-04-24T23:22:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.166+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f015cf20 con 0x7f55f00cf870 2024-04-24T23:22:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.166+0000 7f55e77fe640 1 -- 172.21.15.26:0/1520555233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e00047e0 con 0x7f55f00cf870 2024-04-24T23:22:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.166+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f013a530 con 0x7f55f00cf870 2024-04-24T23:22:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.166+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f00694c0 con 0x7f55f00cf870 2024-04-24T23:22:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.167+0000 7f55e77fe640 1 -- 172.21.15.26:0/1520555233 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55e001b070 con 0x7f55f00cf870 2024-04-24T23:22:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.167+0000 7f55e77fe640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55d006be30 0x7f55d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.167+0000 7f55e77fe640 1 -- 172.21.15.26:0/1520555233 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f55e008a0f0 con 0x7f55f00cf870 2024-04-24T23:22:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.167+0000 7f55e7fff640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55d006be30 0x7f55d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.168+0000 7f55e7fff640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55d006be30 0x7f55d006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55d8003eb0 tx=0x7f55d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.171+0000 7f55e77fe640 1 -- 172.21.15.26:0/1520555233 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f55f015f450 con 0x7f55f00cf870 2024-04-24T23:22:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.392+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55f0072870 con 0x7f55f00cf870 2024-04-24T23:22:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.393+0000 7f55e77fe640 1 -- 172.21.15.26:0/1520555233 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5216 (secure 0 0 0) 0x7f55e0058b30 con 0x7f55f00cf870 2024-04-24T23:22:00.394 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:00.395 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:21:48.324889+0000","modified":"2024-04-24T23:21:59.037564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5088,"mds_1":4867,"mds_2":4864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4864":{"gid":4864,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6834/2201738459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2201738459},{"type":"v1","addr":"172.21.15.29:6834","nonce":2201738459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4867":{"gid":4867,"name":"d","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6835/3972561683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3972561683},{"type":"v1","addr":"172.21.15.29:6835","nonce":3972561683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/315291888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":315291888},{"type":"v1","addr":"172.21.15.26:6836","nonce":315291888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[4864,4867,5088]},"id":2}]} 2024-04-24T23:22:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55d006be30 msgr2=0x7f55d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55d006be30 0x7f55d006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55d8003eb0 tx=0x7f55d8009040 comp rx=0 tx=0).stop 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f00cf870 msgr2=0x7f55f01f84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f00cf870 0x7f55f01f84f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f55e000c780 tx=0x7f55e000cc40 comp rx=0 tx=0).stop 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 shutdown_connections 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55d006be30 0x7f55d006e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55f015eb00 0x7f55f0139d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f0143c40 0x7f55f01f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 --2- 172.21.15.26:0/1520555233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f00cf870 0x7f55f01f84f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 >> 172.21.15.26:0/1520555233 conn(0x7f55f006b300 msgr2=0x7f55f00b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 shutdown_connections 2024-04-24T23:22:00.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:00.395+0000 7f55f50ee640 1 -- 172.21.15.26:0/1520555233 wait complete. 2024-04-24T23:22:00.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:00.408 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:22:00.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:00.432 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:22:00.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:00.492 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-04-24T23:22:00.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:00.556 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-04-24T23:22:00.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:00.613 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-04-24T23:22:00.669 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:22:00.825 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:22:00.826 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:22:00.826 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:22:02.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:22:02.958 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:22:02.959 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:22:02.959 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:22:04.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:22:05.093 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:22:05.094 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:22:05.094 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:22:07.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:22:07.224 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-04-24T23:22:07.224 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:22:07.224 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:22:07.225 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:22:07.355 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:22:07.356 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:22:07.356 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:22:09.357 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:22:09.486 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:22:09.487 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:22:09.487 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:22:11.488 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:22:11.618 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:22:11.618 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:22:11.619 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:22:13.620 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:22:13.750 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-04-24T23:22:13.751 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:22:13.751 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:22:13.751 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-04-24T23:22:13.870 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:22:13.871 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:22:13.871 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:22:13.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 flush journal 2024-04-24T23:22:13.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.985+0000 7efc5f8c1640 1 Processor -- start 2024-04-24T23:22:13.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.985+0000 7efc5f8c1640 1 -- start start 2024-04-24T23:22:13.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5f8c1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc5815be40 0x7efc580acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:13.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5f8c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc580d0750 0x7efc5815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:13.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5f8c1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc580aeb10 con 0x7efc580b4ce0 2024-04-24T23:22:13.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5f8c1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7efc5809df20 con 0x7efc5815be40 2024-04-24T23:22:13.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5f8c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc580aedf0 con 0x7efc580d0750 2024-04-24T23:22:13.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5d65d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc5815be40 0x7efc580acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:13.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5ce5c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efc580b4ce0 0x7efc580d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38126/0 (socket says 172.21.15.26:38126) 2024-04-24T23:22:13.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5ce5c640 1 -- 172.21.15.26:0/2628770290 learned_addr learned my addr 172.21.15.26:0/2628770290 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:13.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5de5e640 1 --2- 172.21.15.26:0/2628770290 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc580d0750 0x7efc5815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:13.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5d65d640 1 -- 172.21.15.26:0/2628770290 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc580d0750 msgr2=0x7efc5815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:13.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5d65d640 1 --2- 172.21.15.26:0/2628770290 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc580d0750 0x7efc5815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:13.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5d65d640 1 -- 172.21.15.26:0/2628770290 >> v1:172.21.15.26:6789/0 conn(0x7efc580b4ce0 legacy=0x7efc580d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:13.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc5d65d640 1 -- 172.21.15.26:0/2628770290 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc580af0d0 con 0x7efc5815be40 2024-04-24T23:22:13.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.986+0000 7efc4ffff640 1 -- 172.21.15.26:0/2628770290 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775114789 0 0) 0x7efc580aeb10 con 0x7efc580b4ce0 2024-04-24T23:22:13.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.987+0000 7efc5d65d640 1 --2- 172.21.15.26:0/2628770290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5815be40 0x7efc580acd30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc540097a0 tx=0x7efc5402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a566e494f6b908ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.987+0000 7efc4ffff640 1 -- 172.21.15.26:0/2628770290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54037070 con 0x7efc5815be40 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.987+0000 7efc4ffff640 1 -- 172.21.15.26:0/2628770290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc54032440 con 0x7efc5815be40 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.987+0000 7efc4ffff640 1 -- 172.21.15.26:0/2628770290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54032810 con 0x7efc5815be40 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.988+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2628770290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5815be40 msgr2=0x7efc580acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.988+0000 7efc5f8c1640 1 --2- 172.21.15.26:0/2628770290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5815be40 0x7efc580acd30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc540097a0 tx=0x7efc5402a180 comp rx=0 tx=0).stop 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.988+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2628770290 shutdown_connections 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.988+0000 7efc5f8c1640 1 --2- 172.21.15.26:0/2628770290 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc580d0750 0x7efc5815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.988+0000 7efc5f8c1640 1 --2- 172.21.15.26:0/2628770290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc5815be40 0x7efc580acd30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.988+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2628770290 >> 172.21.15.26:0/2628770290 conn(0x7efc5806b540 msgr2=0x7efc5806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.988+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2628770290 shutdown_connections 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.988+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2628770290 wait complete. 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.989+0000 7efc5f8c1640 1 Processor -- start 2024-04-24T23:22:13.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.989+0000 7efc5f8c1640 1 -- start start 2024-04-24T23:22:13.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5f8c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc580b4ce0 0x7efc58157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:13.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5f8c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc580d0750 0x7efc58154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:13.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5f8c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc5815be40 0x7efc5815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:13.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5f8c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc5806cd10 con 0x7efc580d0750 2024-04-24T23:22:13.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5f8c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc5806cb90 con 0x7efc580b4ce0 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5f8c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc5815c760 con 0x7efc5815be40 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5ce5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc580d0750 0x7efc58154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5ce5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc580d0750 0x7efc58154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33724/0 (socket says 172.21.15.26:33724) 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5ce5c640 1 -- 172.21.15.26:0/2583376381 learned_addr learned my addr 172.21.15.26:0/2583376381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5d65d640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc580b4ce0 0x7efc58157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5de5e640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc5815be40 0x7efc5815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5ce5c640 1 -- 172.21.15.26:0/2583376381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc5815be40 msgr2=0x7efc5815ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5ce5c640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc5815be40 0x7efc5815ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5ce5c640 1 -- 172.21.15.26:0/2583376381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc580b4ce0 msgr2=0x7efc58157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5ce5c640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc580b4ce0 0x7efc58157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.990+0000 7efc5ce5c640 1 -- 172.21.15.26:0/2583376381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5814d0d0 con 0x7efc580d0750 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.991+0000 7efc5ce5c640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc580d0750 0x7efc58154f10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efc4800c4a0 tx=0x7efc4800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.991+0000 7efc4dffb640 1 -- 172.21.15.26:0/2583376381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48018070 con 0x7efc580d0750 2024-04-24T23:22:13.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.991+0000 7efc4dffb640 1 -- 172.21.15.26:0/2583376381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc480074e0 con 0x7efc580d0750 2024-04-24T23:22:13.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.991+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2583376381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5815f590 con 0x7efc580d0750 2024-04-24T23:22:13.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.991+0000 7efc4dffb640 1 -- 172.21.15.26:0/2583376381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48004a20 con 0x7efc580d0750 2024-04-24T23:22:13.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.991+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2583376381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5809b060 con 0x7efc580d0750 2024-04-24T23:22:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.993+0000 7efc4dffb640 1 -- 172.21.15.26:0/2583376381 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc4801b070 con 0x7efc580d0750 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.993+0000 7efc4dffb640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc3c06be60 0x7efc3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.993+0000 7efc4dffb640 1 -- 172.21.15.26:0/2583376381 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efc4808a060 con 0x7efc580d0750 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.993+0000 7efc5d65d640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc3c06be60 0x7efc3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.994+0000 7efc5d65d640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc3c06be60 0x7efc3c06e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efc54034000 tx=0x7efc5402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.994+0000 7efc625cd740 1 Processor -- start 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.994+0000 7efc625cd740 1 -- start start 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc625cd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 0x55d46349e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc625cd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d4634bb600 0x55d4634bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc625cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d463473a40 0x55d4634b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc625cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4633d76b0 con 0x55d463473a40 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc625cd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d46337f880 con 0x55d46349dd60 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc625cd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d4633d3670 con 0x55d4634bb600 2024-04-24T23:22:13.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5de5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d463473a40 0x55d4634b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5de5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d463473a40 0x55d4634b9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33734/0 (socket says 172.21.15.26:33734) 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5de5e640 1 -- 172.21.15.26:0/1888215619 learned_addr learned my addr 172.21.15.26:0/1888215619 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5ce5c640 1 --2- 172.21.15.26:0/1888215619 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d4634bb600 0x55d4634bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5d65d640 1 --2- 172.21.15.26:0/1888215619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 0x55d46349e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5de5e640 1 -- 172.21.15.26:0/1888215619 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d4634bb600 msgr2=0x55d4634bbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5de5e640 1 --2- 172.21.15.26:0/1888215619 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d4634bb600 0x55d4634bbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5de5e640 1 -- 172.21.15.26:0/1888215619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 msgr2=0x55d46349e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5de5e640 1 --2- 172.21.15.26:0/1888215619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 0x55d46349e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5de5e640 1 -- 172.21.15.26:0/1888215619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4631b57d0 con 0x55d463473a40 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc5de5e640 1 --2- 172.21.15.26:0/1888215619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d463473a40 0x55d4634b9cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efc500039e0 tx=0x7efc50003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48d5a89c634b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc42ffd640 1 -- 172.21.15.26:0/1888215619 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50015420 con 0x55d463473a40 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc42ffd640 1 -- 172.21.15.26:0/1888215619 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5001a070 con 0x55d463473a40 2024-04-24T23:22:13.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.995+0000 7efc42ffd640 1 -- 172.21.15.26:0/1888215619 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc500157c0 con 0x55d463473a40 2024-04-24T23:22:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.996+0000 7efc625cd740 1 -- 172.21.15.26:0/1888215619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d463473a40 msgr2=0x55d4634b9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.996+0000 7efc625cd740 1 --2- 172.21.15.26:0/1888215619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d463473a40 0x55d4634b9cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efc500039e0 tx=0x7efc50003ea0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.996+0000 7efc625cd740 1 -- 172.21.15.26:0/1888215619 shutdown_connections 2024-04-24T23:22:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.996+0000 7efc625cd740 1 --2- 172.21.15.26:0/1888215619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d463473a40 0x55d4634b9cc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.996+0000 7efc625cd740 1 --2- 172.21.15.26:0/1888215619 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d4634bb600 0x55d4634bbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.996+0000 7efc625cd740 1 --2- 172.21.15.26:0/1888215619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 0x55d46349e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.996+0000 7efc625cd740 1 -- 172.21.15.26:0/1888215619 >> 172.21.15.26:0/1888215619 conn(0x55d463470600 msgr2=0x55d4634b8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.996+0000 7efc625cd740 1 -- 172.21.15.26:0/1888215619 shutdown_connections 2024-04-24T23:22:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.996+0000 7efc625cd740 1 -- 172.21.15.26:0/1888215619 wait complete. 2024-04-24T23:22:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.997+0000 7efc625cd740 1 -- start start 2024-04-24T23:22:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.997+0000 7efc625cd740 1 Processor -- start 2024-04-24T23:22:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc625cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d46349dd60 0x55d46349fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc625cd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4634bb600 0x55d4634c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc625cd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 0x55d4634b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc625cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4634747e0 con 0x55d46349dd60 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc625cd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d463392f50 con 0x55d4634bb600 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc625cd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d46346e530 con 0x55d46349a920 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc625cd740 10 client.0 _ll_register_callbacks cb 0x55d4633ce000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5d65d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d46349dd60 0x55d46349fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5d65d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d46349dd60 0x55d46349fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33738/0 (socket says 172.21.15.26:33738) 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5d65d640 1 -- 172.21.15.26:0/3050251665 learned_addr learned my addr 172.21.15.26:0/3050251665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5ce5c640 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4634bb600 0x55d4634c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5de5e640 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 0x55d4634b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5d65d640 1 -- 172.21.15.26:0/3050251665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 msgr2=0x55d4634b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5d65d640 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 0x55d4634b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5d65d640 1 -- 172.21.15.26:0/3050251665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4634bb600 msgr2=0x55d4634c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5d65d640 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4634bb600 0x55d4634c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5d65d640 1 -- 172.21.15.26:0/3050251665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4630eb3f0 con 0x55d46349dd60 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc5d65d640 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d46349dd60 0x55d46349fcf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efc54000c00 tx=0x7efc5402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417c4b08885453ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc37fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.998+0000 7efc37fff640 1 -- 172.21.15.26:0/3050251665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54037040 con 0x55d46349dd60 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc37fff640 1 -- 172.21.15.26:0/3050251665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efc5403c650 con 0x55d46349dd60 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc625cd740 1 -- 172.21.15.26:0/3050251665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc5001a070 con 0x55d46349dd60 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc37fff640 1 -- 172.21.15.26:0/3050251665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efc54056050 con 0x55d46349dd60 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc625cd740 10 client.5376 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc625cd740 1 -- 172.21.15.26:0/3050251665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55d4630ab0f0 con 0x55d46349dd60 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc37fff640 1 -- 172.21.15.26:0/3050251665 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5001a070 con 0x55d46349dd60 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc37fff640 1 -- 172.21.15.26:0/3050251665 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7efc54059020 con 0x55d46349dd60 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc625cd740 10 client.5376 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc625cd740 10 client.5376 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc625cd740 20 client.5376 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc625cd740 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d4634b0400 0x55d4634cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc625cd740 4 client.5376 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc625cd740 1 -- 172.21.15.26:0/3050251665 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4633e4bf0 con 0x55d4634b0400 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:13.999+0000 7efc5ce5c640 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d4634b0400 0x55d4634cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc37fff640 10 client.5376 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc5ce5c640 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d4634b0400 0x55d4634cc880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b03a739edbbecc server_cookie=baa1831418c62e0e in_seq=0 out_seq=0 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc37fff640 1 -- 172.21.15.26:0/3050251665 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4633e4bf0 con 0x55d4634b0400 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc37fff640 10 client.5376 handle_command_reply: tid=0 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc625cd740 1 client.5376 shutdown 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc625cd740 1 -- 172.21.15.26:0/3050251665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d46349dd60 msgr2=0x55d46349fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc625cd740 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d46349dd60 0x55d46349fcf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efc54000c00 tx=0x7efc5402c070 comp rx=0 tx=0).stop 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc625cd740 1 -- 172.21.15.26:0/3050251665 shutdown_connections 2024-04-24T23:22:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc625cd740 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d4634b0400 0x55d4634cc880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc625cd740 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 0x55d4634b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc625cd740 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4634bb600 0x55d4634c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc37fff640 0 client.5376 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc625cd740 1 --2- 172.21.15.26:0/3050251665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d46349dd60 0x55d46349fcf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.000+0000 7efc625cd740 1 -- 172.21.15.26:0/3050251665 >> 172.21.15.26:0/3050251665 conn(0x55d463473a40 msgr2=0x55d4634bc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.001+0000 7efc625cd740 1 -- 172.21.15.26:0/3050251665 shutdown_connections 2024-04-24T23:22:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.001+0000 7efc625cd740 1 -- 172.21.15.26:0/3050251665 wait complete. 2024-04-24T23:22:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.001+0000 7efc625cd740 20 client.5376 trim_cache size 0 max 16384 2024-04-24T23:22:14.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.016+0000 7efc625cd740 1 Processor -- start 2024-04-24T23:22:14.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.016+0000 7efc625cd740 1 -- start start 2024-04-24T23:22:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.016+0000 7efc625cd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349a920 0x55d4634c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.016+0000 7efc625cd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349dd60 0x55d4634dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.016+0000 7efc625cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634dd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.016+0000 7efc625cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4634c8fa0 con 0x55d4634bb600 2024-04-24T23:22:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.016+0000 7efc625cd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc58200600 con 0x55d46349a920 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.016+0000 7efc625cd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc581ff850 con 0x55d46349dd60 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5de5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5de5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634dd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33748/0 (socket says 172.21.15.26:33748) 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5de5e640 1 -- 172.21.15.26:0/136191233 learned_addr learned my addr 172.21.15.26:0/136191233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5d65d640 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349a920 0x55d4634c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5ce5c640 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349dd60 0x55d4634dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5de5e640 1 -- 172.21.15.26:0/136191233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349dd60 msgr2=0x55d4634dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5de5e640 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349dd60 0x55d4634dcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5de5e640 1 -- 172.21.15.26:0/136191233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349a920 msgr2=0x55d4634c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5de5e640 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349a920 0x55d4634c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5de5e640 1 -- 172.21.15.26:0/136191233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4634b2c50 con 0x55d4634bb600 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5d65d640 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349a920 0x55d4634c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc5de5e640 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634dd260 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efc500039b0 tx=0x7efc5000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bb1eee5ceeccb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc427fc640 1 -- 172.21.15.26:0/136191233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50015760 con 0x55d4634bb600 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc427fc640 1 -- 172.21.15.26:0/136191233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5001a030 con 0x55d4634bb600 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.017+0000 7efc427fc640 1 -- 172.21.15.26:0/136191233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50015b40 con 0x55d4634bb600 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 -- 172.21.15.26:0/136191233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 msgr2=0x55d4634dd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634dd260 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efc500039b0 tx=0x7efc5000e000 comp rx=0 tx=0).stop 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 -- 172.21.15.26:0/136191233 shutdown_connections 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634dd260 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349dd60 0x55d4634dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 --2- 172.21.15.26:0/136191233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349a920 0x55d4634c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 -- 172.21.15.26:0/136191233 >> 172.21.15.26:0/136191233 conn(0x55d463473a40 msgr2=0x55d463499d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 -- 172.21.15.26:0/136191233 shutdown_connections 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 -- 172.21.15.26:0/136191233 wait complete. 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.018+0000 7efc625cd740 1 -- start start 2024-04-24T23:22:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.019+0000 7efc625cd740 1 Processor -- start 2024-04-24T23:22:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.019+0000 7efc625cd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 0x55d463531f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.019+0000 7efc625cd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 0x55d463472ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.019+0000 7efc625cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.019+0000 7efc625cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc5815cbf0 con 0x55d4634bb600 2024-04-24T23:22:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.019+0000 7efc625cd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc581ffed0 con 0x55d46349dd60 2024-04-24T23:22:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.019+0000 7efc625cd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d4634b2980 con 0x55d46349a920 2024-04-24T23:22:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc625cd740 10 client.0 _ll_register_callbacks cb 0x7efc54009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5de5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5de5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634733e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33752/0 (socket says 172.21.15.26:33752) 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5de5e640 1 -- 172.21.15.26:0/4107340218 learned_addr learned my addr 172.21.15.26:0/4107340218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5ce5c640 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 0x55d463472ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5d65d640 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 0x55d463531f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5de5e640 1 -- 172.21.15.26:0/4107340218 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 msgr2=0x55d463531f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5de5e640 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 0x55d463531f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5de5e640 1 -- 172.21.15.26:0/4107340218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 msgr2=0x55d463472ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5de5e640 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 0x55d463472ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5de5e640 1 -- 172.21.15.26:0/4107340218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4634b45c0 con 0x55d4634bb600 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc5de5e640 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634733e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efc50016000 tx=0x7efc50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42822996d4e953d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc37fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc37fff640 1 -- 172.21.15.26:0/4107340218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50035060 con 0x55d4634bb600 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc37fff640 1 -- 172.21.15.26:0/4107340218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efc50011050 con 0x55d4634bb600 2024-04-24T23:22:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc625cd740 1 -- 172.21.15.26:0/4107340218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc5001a030 con 0x55d4634bb600 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc37fff640 1 -- 172.21.15.26:0/4107340218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efc5003b050 con 0x55d4634bb600 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc625cd740 10 client.5382 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.020+0000 7efc625cd740 1 -- 172.21.15.26:0/4107340218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55d4634740d0 con 0x55d4634bb600 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc37fff640 1 -- 172.21.15.26:0/4107340218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5001a030 con 0x55d4634bb600 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc37fff640 1 -- 172.21.15.26:0/4107340218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7efc5003e020 con 0x55d4634bb600 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc625cd740 10 client.5382 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc625cd740 10 client.5382 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc625cd740 20 client.5382 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc625cd740 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d4634bb040 0x55d4634b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc625cd740 4 client.5382 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc625cd740 1 -- 172.21.15.26:0/4107340218 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4633def40 con 0x55d4634bb040 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc5d65d640 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d4634bb040 0x55d4634b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc5d65d640 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d4634bb040 0x55d4634b1050 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebcf02ca1847f068 server_cookie=2a8507055d0c8701 in_seq=0 out_seq=0 2024-04-24T23:22:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.021+0000 7efc37fff640 10 client.5382 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc37fff640 1 -- 172.21.15.26:0/4107340218 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d4633def40 con 0x55d4634bb040 2024-04-24T23:22:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc37fff640 10 client.5382 handle_command_reply: tid=0 2024-04-24T23:22:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 client.5382 shutdown 2024-04-24T23:22:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 -- 172.21.15.26:0/4107340218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 msgr2=0x55d4634733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634733e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efc50016000 tx=0x7efc50004ab0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 -- 172.21.15.26:0/4107340218 shutdown_connections 2024-04-24T23:22:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d4634bb040 0x55d4634b1050 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4634bb600 0x55d4634733e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d46349dd60 0x55d463472ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc37fff640 0 client.5382 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 --2- 172.21.15.26:0/4107340218 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d46349a920 0x55d463531f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 -- 172.21.15.26:0/4107340218 >> 172.21.15.26:0/4107340218 conn(0x55d463473a40 msgr2=0x55d463474e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 -- 172.21.15.26:0/4107340218 shutdown_connections 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 1 -- 172.21.15.26:0/4107340218 wait complete. 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.033+0000 7efc625cd740 20 client.5382 trim_cache size 0 max 16384 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:22:14.036 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:22:14.037 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:22:14.037 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:22:14.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc3c06be60 msgr2=0x7efc3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc3c06be60 0x7efc3c06e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efc54034000 tx=0x7efc5402e000 comp rx=0 tx=0).stop 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc580d0750 msgr2=0x7efc58154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc580d0750 0x7efc58154f10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efc4800c4a0 tx=0x7efc4800c960 comp rx=0 tx=0).stop 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2583376381 shutdown_connections 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc3c06be60 0x7efc3c06e320 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc5815be40 0x7efc5815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc580d0750 0x7efc58154f10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 --2- 172.21.15.26:0/2583376381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc580b4ce0 0x7efc58157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2583376381 >> 172.21.15.26:0/2583376381 conn(0x7efc5806b540 msgr2=0x7efc580b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2583376381 shutdown_connections 2024-04-24T23:22:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.034+0000 7efc5f8c1640 1 -- 172.21.15.26:0/2583376381 wait complete. 2024-04-24T23:22:14.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:1 flush journal 2024-04-24T23:22:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.147+0000 7f1b4055a640 1 Processor -- start 2024-04-24T23:22:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.147+0000 7f1b4055a640 1 -- start start 2024-04-24T23:22:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.148+0000 7f1b4055a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b380d0750 0x7f1b380acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.148+0000 7f1b4055a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b3815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.148+0000 7f1b4055a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b380aeb10 con 0x7f1b3815be40 2024-04-24T23:22:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.148+0000 7f1b4055a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1b3809df20 con 0x7f1b380d0750 2024-04-24T23:22:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.148+0000 7f1b4055a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1b380aedf0 con 0x7f1b380b4ce0 2024-04-24T23:22:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.148+0000 7f1b3e2f6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1b3815be40 0x7f1b380d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38134/0 (socket says 172.21.15.26:38134) 2024-04-24T23:22:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.148+0000 7f1b3daf5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b380d0750 0x7f1b380acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.148+0000 7f1b3e2f6640 1 -- 172.21.15.26:0/3831309025 learned_addr learned my addr 172.21.15.26:0/3831309025 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.149+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/3831309025 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b3815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.149+0000 7f1b3daf5640 1 -- 172.21.15.26:0/3831309025 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 msgr2=0x7f1b3815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.149+0000 7f1b3daf5640 1 --2- 172.21.15.26:0/3831309025 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b3815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.149+0000 7f1b3daf5640 1 -- 172.21.15.26:0/3831309025 >> v1:172.21.15.26:6789/0 conn(0x7f1b3815be40 legacy=0x7f1b380d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.149+0000 7f1b3d2f4640 1 -- 172.21.15.26:0/3831309025 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 699613699 0 0) 0x7f1b380aeb10 con 0x7f1b3815be40 2024-04-24T23:22:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.149+0000 7f1b3daf5640 1 -- 172.21.15.26:0/3831309025 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b380af0d0 con 0x7f1b380d0750 2024-04-24T23:22:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.149+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/3831309025 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b3815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.149+0000 7f1b3daf5640 1 --2- 172.21.15.26:0/3831309025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b380d0750 0x7f1b380acd30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1b280097a0 tx=0x7f1b2802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad26feb0819caf52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.149+0000 7f1b3d2f4640 1 -- 172.21.15.26:0/3831309025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28037070 con 0x7f1b380d0750 2024-04-24T23:22:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b3d2f4640 1 -- 172.21.15.26:0/3831309025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b28032440 con 0x7f1b380d0750 2024-04-24T23:22:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b3d2f4640 1 -- 172.21.15.26:0/3831309025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28032810 con 0x7f1b380d0750 2024-04-24T23:22:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b4055a640 1 -- 172.21.15.26:0/3831309025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b380d0750 msgr2=0x7f1b380acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b4055a640 1 --2- 172.21.15.26:0/3831309025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b380d0750 0x7f1b380acd30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1b280097a0 tx=0x7f1b2802a180 comp rx=0 tx=0).stop 2024-04-24T23:22:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b4055a640 1 -- 172.21.15.26:0/3831309025 shutdown_connections 2024-04-24T23:22:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b4055a640 1 --2- 172.21.15.26:0/3831309025 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b3815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b4055a640 1 --2- 172.21.15.26:0/3831309025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b380d0750 0x7f1b380acd30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b4055a640 1 -- 172.21.15.26:0/3831309025 >> 172.21.15.26:0/3831309025 conn(0x7f1b3806b540 msgr2=0x7f1b3806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b4055a640 1 -- 172.21.15.26:0/3831309025 shutdown_connections 2024-04-24T23:22:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.150+0000 7f1b4055a640 1 -- 172.21.15.26:0/3831309025 wait complete. 2024-04-24T23:22:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.151+0000 7f1b4055a640 1 Processor -- start 2024-04-24T23:22:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.151+0000 7f1b4055a640 1 -- start start 2024-04-24T23:22:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b4055a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b38144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b4055a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b380d0750 0x7f1b3809bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b4055a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b3815be40 0x7f1b3815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b4055a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3806cd10 con 0x7f1b3815be40 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b4055a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3806cb90 con 0x7f1b380d0750 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b4055a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3815c760 con 0x7f1b380b4ce0 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b3eaf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b3815be40 0x7f1b3815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b3eaf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b3815be40 0x7f1b3815cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33778/0 (socket says 172.21.15.26:33778) 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/470344098 learned_addr learned my addr 172.21.15.26:0/470344098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b38144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.152+0000 7f1b3daf5640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b380d0750 0x7f1b3809bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 msgr2=0x7f1b38144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b38144810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b380d0750 msgr2=0x7f1b3809bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b380d0750 0x7f1b3809bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/470344098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3815d2d0 con 0x7f1b3815be40 2024-04-24T23:22:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b38144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b3daf5640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b380d0750 0x7f1b3809bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b3815be40 0x7f1b3815cbf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000cef0 tx=0x7f1b3000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b2effd640 1 -- 172.21.15.26:0/470344098 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3001a070 con 0x7f1b3815be40 2024-04-24T23:22:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b2effd640 1 -- 172.21.15.26:0/470344098 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3000d860 con 0x7f1b3815be40 2024-04-24T23:22:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b4055a640 1 -- 172.21.15.26:0/470344098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3815d5b0 con 0x7f1b3815be40 2024-04-24T23:22:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b2effd640 1 -- 172.21.15.26:0/470344098 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000dc50 con 0x7f1b3815be40 2024-04-24T23:22:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.153+0000 7f1b4055a640 1 -- 172.21.15.26:0/470344098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3815db70 con 0x7f1b3815be40 2024-04-24T23:22:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.155+0000 7f1b2effd640 1 -- 172.21.15.26:0/470344098 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b3001d070 con 0x7f1b3815be40 2024-04-24T23:22:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.155+0000 7f1b2effd640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.155+0000 7f1b2effd640 1 -- 172.21.15.26:0/470344098 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1b3008b740 con 0x7f1b3815be40 2024-04-24T23:22:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.155+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.156+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b34000f60 tx=0x7f1b34007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.156+0000 7f1b43269740 1 Processor -- start 2024-04-24T23:22:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.156+0000 7f1b43269740 1 -- start start 2024-04-24T23:22:14.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b43269740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafead60 0x55f5cafeb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b43269740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb008b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b43269740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafc0a40 0x55f5cb006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b43269740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f5caf246b0 con 0x55f5cafc0a40 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b43269740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f5caecc880 con 0x55f5cb008600 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b43269740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f5caf20670 con 0x55f5cafead60 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b3eaf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafc0a40 0x55f5cb006cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b3eaf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafc0a40 0x55f5cb006cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33780/0 (socket says 172.21.15.26:33780) 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/1518330406 learned_addr learned my addr 172.21.15.26:0/1518330406 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/1518330406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafead60 msgr2=0x55f5cafeb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/1518330406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafead60 0x55f5cafeb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/1518330406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 msgr2=0x55f5cb008b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/1518330406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb008b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.157+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/1518330406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5cad027d0 con 0x55f5cafc0a40 2024-04-24T23:22:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/1518330406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafc0a40 0x55f5cb006cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000ae50 tx=0x7f1b3005a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f65afa69c479d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b1bfff640 1 -- 172.21.15.26:0/1518330406 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30088070 con 0x55f5cafc0a40 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b1bfff640 1 -- 172.21.15.26:0/1518330406 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b30062690 con 0x55f5cafc0a40 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b1bfff640 1 -- 172.21.15.26:0/1518330406 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30056120 con 0x55f5cafc0a40 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b43269740 1 -- 172.21.15.26:0/1518330406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafc0a40 msgr2=0x55f5cb006cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b43269740 1 --2- 172.21.15.26:0/1518330406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafc0a40 0x55f5cb006cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000ae50 tx=0x7f1b3005a280 comp rx=0 tx=0).stop 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b43269740 1 -- 172.21.15.26:0/1518330406 shutdown_connections 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b43269740 1 --2- 172.21.15.26:0/1518330406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafc0a40 0x55f5cb006cc0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b43269740 1 --2- 172.21.15.26:0/1518330406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb008b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b43269740 1 --2- 172.21.15.26:0/1518330406 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafead60 0x55f5cafeb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.158+0000 7f1b43269740 1 -- 172.21.15.26:0/1518330406 >> 172.21.15.26:0/1518330406 conn(0x55f5cafbd600 msgr2=0x55f5cb005b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.159+0000 7f1b43269740 1 -- 172.21.15.26:0/1518330406 shutdown_connections 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.159+0000 7f1b43269740 1 -- 172.21.15.26:0/1518330406 wait complete. 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.159+0000 7f1b43269740 1 -- start start 2024-04-24T23:22:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.159+0000 7f1b43269740 1 Processor -- start 2024-04-24T23:22:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b43269740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cafead60 0x55f5cafe6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b43269740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cb008600 0x55f5cafe8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b43269740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b43269740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f5cafc17e0 con 0x55f5cafe6fa0 2024-04-24T23:22:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b43269740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f5caedff50 con 0x55f5cafead60 2024-04-24T23:22:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b43269740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f5cafbb530 con 0x55f5cb008600 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b43269740 10 client.0 _ll_register_callbacks cb 0x55f5caf1b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b3eaf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b3eaf7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33784/0 (socket says 172.21.15.26:33784) 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/622911517 learned_addr learned my addr 172.21.15.26:0/622911517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cafead60 0x55f5cafe6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.160+0000 7f1b3daf5640 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cb008600 0x55f5cafe8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cb008600 msgr2=0x55f5cafe8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cb008600 0x55f5cafe8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cafead60 msgr2=0x55f5cafe6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cafead60 0x55f5cafe6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b3eaf7640 1 -- 172.21.15.26:0/622911517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5cac383f0 con 0x55f5cafe6fa0 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe9360 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000ae50 tx=0x7f1b3000adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d2cab42caa544f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b197fa640 1 -- 172.21.15.26:0/622911517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30088040 con 0x55f5cafe6fa0 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b43269740 1 -- 172.21.15.26:0/622911517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b30062690 con 0x55f5cafe6fa0 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b197fa640 1 -- 172.21.15.26:0/622911517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1b30057db0 con 0x55f5cafe6fa0 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b197fa640 1 -- 172.21.15.26:0/622911517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1b30081050 con 0x55f5cafe6fa0 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b43269740 10 client.5397 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b43269740 1 -- 172.21.15.26:0/622911517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55f5cabf80f0 con 0x55f5cafe6fa0 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b197fa640 1 -- 172.21.15.26:0/622911517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b30062690 con 0x55f5cafe6fa0 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b197fa640 1 -- 172.21.15.26:0/622911517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7f1b30062200 con 0x55f5cafe6fa0 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b43269740 10 client.5397 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b43269740 10 client.5397 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.d 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b43269740 20 client.5397 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b43269740 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55f5cafecdf0 0x55f5cafed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b43269740 4 client.5397 mds_command: new command op to 4867 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.161+0000 7f1b43269740 1 -- 172.21.15.26:0/622911517 --> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5caf31bf0 con 0x55f5cafecdf0 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.162+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55f5cafecdf0 0x55f5cafed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.162+0000 7f1b197fa640 10 client.5397 ms_handle_connect on v2:172.21.15.29:6833/3972561683 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.162+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55f5cafecdf0 0x55f5cafed1d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1f74fd25f59429f server_cookie=7a587aa624586d5f in_seq=0 out_seq=0 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b197fa640 1 -- 172.21.15.26:0/622911517 <== mds.1 v2:172.21.15.29:6833/3972561683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5caf31bf0 con 0x55f5cafecdf0 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b197fa640 10 client.5397 handle_command_reply: tid=0 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 client.5397 shutdown 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 -- 172.21.15.26:0/622911517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 msgr2=0x55f5cafe9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe9360 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000ae50 tx=0x7f1b3000adc0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 -- 172.21.15.26:0/622911517 shutdown_connections 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55f5cafecdf0 0x55f5cafed1d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe9360 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cb008600 0x55f5cafe8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 --2- 172.21.15.26:0/622911517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cafead60 0x55f5cafe6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b197fa640 0 client.5397 ms_handle_reset on v2:172.21.15.29:6833/3972561683 2024-04-24T23:22:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 -- 172.21.15.26:0/622911517 >> 172.21.15.26:0/622911517 conn(0x55f5cafc0a40 msgr2=0x55f5cb0090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 -- 172.21.15.26:0/622911517 shutdown_connections 2024-04-24T23:22:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 1 -- 172.21.15.26:0/622911517 wait complete. 2024-04-24T23:22:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.163+0000 7f1b43269740 20 client.5397 trim_cache size 0 max 16384 2024-04-24T23:22:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.178+0000 7f1b43269740 1 Processor -- start 2024-04-24T23:22:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.178+0000 7f1b43269740 1 -- start start 2024-04-24T23:22:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b43269740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b43269740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafead60 0x55f5cb020020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b43269740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb006af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b43269740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381fe3b0 con 0x55f5cafe6fa0 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b43269740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381fd620 con 0x55f5cb008600 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b43269740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1b381fdc80 con 0x55f5cafead60 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3e2f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3e2f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33798/0 (socket says 172.21.15.26:33798) 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3e2f6640 1 -- 172.21.15.26:0/2127235612 learned_addr learned my addr 172.21.15.26:0/2127235612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb006af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3daf5640 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafead60 0x55f5cb020020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3e2f6640 1 -- 172.21.15.26:0/2127235612 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafead60 msgr2=0x55f5cb020020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafead60 0x55f5cb020020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3e2f6640 1 -- 172.21.15.26:0/2127235612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 msgr2=0x55f5cb006af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb006af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3e2f6640 1 -- 172.21.15.26:0/2127235612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5cafc1880 con 0x55f5cafe6fa0 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.179+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb006af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe5c50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55f5cafec2f0 tx=0x7f1b34011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb16a564ab4239 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b1b7fe640 1 -- 172.21.15.26:0/2127235612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3402a070 con 0x55f5cafe6fa0 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b1b7fe640 1 -- 172.21.15.26:0/2127235612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34013870 con 0x55f5cafe6fa0 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b1b7fe640 1 -- 172.21.15.26:0/2127235612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34013a20 con 0x55f5cafe6fa0 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b43269740 1 -- 172.21.15.26:0/2127235612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 msgr2=0x55f5cafe5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b43269740 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe5c50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55f5cafec2f0 tx=0x7f1b34011000 comp rx=0 tx=0).stop 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b43269740 1 -- 172.21.15.26:0/2127235612 shutdown_connections 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b43269740 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb006af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b43269740 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafead60 0x55f5cb020020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b43269740 1 --2- 172.21.15.26:0/2127235612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafe6fa0 0x55f5cafe5c50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b43269740 1 -- 172.21.15.26:0/2127235612 >> 172.21.15.26:0/2127235612 conn(0x55f5cafc0a40 msgr2=0x55f5cafc0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b43269740 1 -- 172.21.15.26:0/2127235612 shutdown_connections 2024-04-24T23:22:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.180+0000 7f1b43269740 1 -- 172.21.15.26:0/2127235612 wait complete. 2024-04-24T23:22:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.181+0000 7f1b43269740 1 -- start start 2024-04-24T23:22:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.181+0000 7f1b43269740 1 Processor -- start 2024-04-24T23:22:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b43269740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafe6fa0 0x55f5cb005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b43269740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafead60 0x55f5cb0183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b43269740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b43269740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f5cb006470 con 0x55f5cafead60 2024-04-24T23:22:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b43269740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38151ee0 con 0x55f5cb008600 2024-04-24T23:22:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b43269740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f5cafbcac0 con 0x55f5cafe6fa0 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b43269740 10 client.0 _ll_register_callbacks cb 0x7f1b30086070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3daf5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafead60 0x55f5cb0183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3e2f6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafe6fa0 0x55f5cb005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3daf5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafead60 0x55f5cb0183b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33808/0 (socket says 172.21.15.26:33808) 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3daf5640 1 -- 172.21.15.26:0/3175694628 learned_addr learned my addr 172.21.15.26:0/3175694628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3eaf7640 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3daf5640 1 -- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafe6fa0 msgr2=0x55f5cb005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3daf5640 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafe6fa0 0x55f5cb005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3daf5640 1 -- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 msgr2=0x55f5cb003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3daf5640 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3daf5640 1 -- 172.21.15.26:0/3175694628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5cb06cdb0 con 0x55f5cafead60 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b3daf5640 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafead60 0x55f5cb0183b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1b28035040 tx=0x7f1b28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ec77d16bad3e70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.182+0000 7f1b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b197fa640 1 -- 172.21.15.26:0/3175694628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28044070 con 0x55f5cafead60 2024-04-24T23:22:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b197fa640 1 -- 172.21.15.26:0/3175694628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1b2803d650 con 0x55f5cafead60 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b43269740 1 -- 172.21.15.26:0/3175694628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b34013870 con 0x55f5cafead60 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b197fa640 1 -- 172.21.15.26:0/3175694628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1b2804e050 con 0x55f5cafead60 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b43269740 10 client.5403 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b43269740 1 -- 172.21.15.26:0/3175694628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55f5cb0189e0 con 0x55f5cafead60 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b197fa640 1 -- 172.21.15.26:0/3175694628 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34013870 con 0x55f5cafead60 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b197fa640 1 -- 172.21.15.26:0/3175694628 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7f1b28051020 con 0x55f5cafead60 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b43269740 10 client.5403 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b43269740 10 client.5403 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.d 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b43269740 20 client.5403 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b43269740 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55f5cb06a0d0 0x55f5cb004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b43269740 4 client.5403 mds_command: new command op to 4867 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b43269740 1 -- 172.21.15.26:0/3175694628 --> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5cafba5d0 con 0x55f5cb06a0d0 2024-04-24T23:22:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.183+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55f5cb06a0d0 0x55f5cb004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.184+0000 7f1b3e2f6640 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55f5cb06a0d0 0x55f5cb004010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e25bbe0e60c9d45e server_cookie=92aa70f19d129c4b in_seq=0 out_seq=0 2024-04-24T23:22:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.184+0000 7f1b197fa640 10 client.5403 ms_handle_connect on v2:172.21.15.29:6833/3972561683 2024-04-24T23:22:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b197fa640 1 -- 172.21.15.26:0/3175694628 <== mds.1 v2:172.21.15.29:6833/3972561683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5cafba5d0 con 0x55f5cb06a0d0 2024-04-24T23:22:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b197fa640 10 client.5403 handle_command_reply: tid=0 2024-04-24T23:22:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b43269740 1 client.5403 shutdown 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b43269740 1 -- 172.21.15.26:0/3175694628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafead60 msgr2=0x55f5cb0183b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b43269740 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafead60 0x55f5cb0183b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1b28035040 tx=0x7f1b28004730 comp rx=0 tx=0).stop 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b43269740 1 -- 172.21.15.26:0/3175694628 shutdown_connections 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b43269740 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:6833/3972561683,v1:172.21.15.29:6835/3972561683] conn(0x55f5cb06a0d0 0x55f5cb004010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b43269740 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5cb008600 0x55f5cb003ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b43269740 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cafead60 0x55f5cb0183b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b197fa640 0 client.5403 ms_handle_reset on v2:172.21.15.29:6833/3972561683 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b43269740 1 --2- 172.21.15.26:0/3175694628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f5cafe6fa0 0x55f5cb005a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.191+0000 7f1b43269740 1 -- 172.21.15.26:0/3175694628 >> 172.21.15.26:0/3175694628 conn(0x55f5cafc0a40 msgr2=0x55f5cb02a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b43269740 1 -- 172.21.15.26:0/3175694628 shutdown_connections 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b43269740 1 -- 172.21.15.26:0/3175694628 wait complete. 2024-04-24T23:22:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b43269740 20 client.5403 trim_cache size 0 max 16384 2024-04-24T23:22:14.196 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:22:14.196 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:22:14.196 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:22:14.196 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 -- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b1c06be30 msgr2=0x7f1b1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b34000f60 tx=0x7f1b34007ca0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 -- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b3815be40 msgr2=0x7f1b3815cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b3815be40 0x7f1b3815cbf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000cef0 tx=0x7f1b3000a710 comp rx=0 tx=0).stop 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 -- 172.21.15.26:0/470344098 shutdown_connections 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b3815be40 0x7f1b3815cbf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b380d0750 0x7f1b3809bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 --2- 172.21.15.26:0/470344098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b380b4ce0 0x7f1b38144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 -- 172.21.15.26:0/470344098 >> 172.21.15.26:0/470344098 conn(0x7f1b3806b540 msgr2=0x7f1b380d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 -- 172.21.15.26:0/470344098 shutdown_connections 2024-04-24T23:22:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.192+0000 7f1b4055a640 1 -- 172.21.15.26:0/470344098 wait complete. 2024-04-24T23:22:14.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:2 flush journal 2024-04-24T23:22:14.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.305+0000 7fc3127f9640 1 Processor -- start 2024-04-24T23:22:14.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.306+0000 7fc3127f9640 1 -- start start 2024-04-24T23:22:14.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.306+0000 7fc3127f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc30c0d2c30 0x7fc30c0cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.306+0000 7fc3127f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc30c0d2120 con 0x7fc30c15e370 2024-04-24T23:22:14.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.306+0000 7fc3127f9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc30c0b3740 con 0x7fc30c15e750 2024-04-24T23:22:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.306+0000 7fc3127f9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc30c0d1510 con 0x7fc30c0d2c30 2024-04-24T23:22:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.306+0000 7fc310d96640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc30c0d2c30 0x7fc30c0cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.307+0000 7fc30bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc30c15e370 0x7fc30c0d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38140/0 (socket says 172.21.15.26:38140) 2024-04-24T23:22:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.307+0000 7fc310d96640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc30c0d2c30 0x7fc30c0cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33812/0 (socket says 172.21.15.26:33812) 2024-04-24T23:22:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.307+0000 7fc30bfff640 1 -- 172.21.15.26:0/360240109 learned_addr learned my addr 172.21.15.26:0/360240109 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.307+0000 7fc310d96640 1 -- 172.21.15.26:0/360240109 >> v1:172.21.15.29:6789/0 conn(0x7fc30c15e750 legacy=0x7fc30c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.307+0000 7fc310d96640 1 -- 172.21.15.26:0/360240109 >> v1:172.21.15.26:6789/0 conn(0x7fc30c15e370 legacy=0x7fc30c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.307+0000 7fc310d96640 1 -- 172.21.15.26:0/360240109 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c0b3a20 con 0x7fc30c0d2c30 2024-04-24T23:22:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.307+0000 7fc30affd640 1 -- 172.21.15.26:0/360240109 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3127343087 0 0) 0x7fc30c0d2120 con 0x7fc30c15e370 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc310d96640 1 --2- 172.21.15.26:0/360240109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c0d2c30 0x7fc30c0cf3a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc304009d90 tx=0x7fc30402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544c0da748fa05fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc30affd640 1 -- 172.21.15.26:0/360240109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304007740 con 0x7fc30c0d2c30 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc30affd640 1 -- 172.21.15.26:0/360240109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3040078e0 con 0x7fc30c0d2c30 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc30affd640 1 -- 172.21.15.26:0/360240109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304007cb0 con 0x7fc30c0d2c30 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc3127f9640 1 -- 172.21.15.26:0/360240109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c0d2c30 msgr2=0x7fc30c0cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc3127f9640 1 --2- 172.21.15.26:0/360240109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c0d2c30 0x7fc30c0cf3a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc304009d90 tx=0x7fc30402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc3127f9640 1 -- 172.21.15.26:0/360240109 shutdown_connections 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc3127f9640 1 --2- 172.21.15.26:0/360240109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c0d2c30 0x7fc30c0cf3a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc3127f9640 1 -- 172.21.15.26:0/360240109 >> 172.21.15.26:0/360240109 conn(0x7fc30c0d40f0 msgr2=0x7fc30c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc3127f9640 1 -- 172.21.15.26:0/360240109 shutdown_connections 2024-04-24T23:22:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.308+0000 7fc3127f9640 1 -- 172.21.15.26:0/360240109 wait complete. 2024-04-24T23:22:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.309+0000 7fc3127f9640 1 Processor -- start 2024-04-24T23:22:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.310+0000 7fc3127f9640 1 -- start start 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.310+0000 7fc3127f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c0d2c30 0x7fc30c0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.310+0000 7fc3127f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c15e370 0x7fc30c1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.310+0000 7fc3127f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc30c15bad0 0x7fc30c1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.310+0000 7fc3127f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c0b5230 con 0x7fc30c15e370 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.310+0000 7fc3127f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c0d1ed0 con 0x7fc30c0d2c30 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.310+0000 7fc3127f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc30c0b53b0 con 0x7fc30c15bad0 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.310+0000 7fc30b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c15e370 0x7fc30c1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc30b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c15e370 0x7fc30c1563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33816/0 (socket says 172.21.15.26:33816) 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc30b7fe640 1 -- 172.21.15.26:0/3973901920 learned_addr learned my addr 172.21.15.26:0/3973901920 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc30bfff640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c0d2c30 0x7fc30c0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc310d96640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc30c15bad0 0x7fc30c1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc30b7fe640 1 -- 172.21.15.26:0/3973901920 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc30c15bad0 msgr2=0x7fc30c1568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc30c15bad0 0x7fc30c1568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc30b7fe640 1 -- 172.21.15.26:0/3973901920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c0d2c30 msgr2=0x7fc30c0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c0d2c30 0x7fc30c0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc30b7fe640 1 -- 172.21.15.26:0/3973901920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c0cbb80 con 0x7fc30c15e370 2024-04-24T23:22:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c15e370 0x7fc30c1563b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00c430 tx=0x7fc2fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc308ff9640 1 -- 172.21.15.26:0/3973901920 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc011790 con 0x7fc30c15e370 2024-04-24T23:22:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc308ff9640 1 -- 172.21.15.26:0/3973901920 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2fc017070 con 0x7fc30c15e370 2024-04-24T23:22:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc3127f9640 1 -- 172.21.15.26:0/3973901920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30c0cbe20 con 0x7fc30c15e370 2024-04-24T23:22:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc308ff9640 1 -- 172.21.15.26:0/3973901920 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc011bb0 con 0x7fc30c15e370 2024-04-24T23:22:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.311+0000 7fc3127f9640 1 -- 172.21.15.26:0/3973901920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30c0cc3e0 con 0x7fc30c15e370 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.313+0000 7fc308ff9640 1 -- 172.21.15.26:0/3973901920 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2fc01c460 con 0x7fc30c15e370 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.313+0000 7fc308ff9640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2f806be30 0x7fc2f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.313+0000 7fc308ff9640 1 -- 172.21.15.26:0/3973901920 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc2fc08ab50 con 0x7fc30c15e370 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.313+0000 7fc30bfff640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2f806be30 0x7fc2f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.314+0000 7fc30bfff640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2f806be30 0x7fc2f806e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc300000f60 tx=0x7fc300007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.314+0000 7fc3155e1740 1 Processor -- start 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.314+0000 7fc3155e1740 1 -- start start 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089baf5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 0x56089bb12b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bacaa40 0x56089bb10cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc3155e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56089ba2e6b0 con 0x56089baf4d60 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc3155e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56089b9d6880 con 0x56089bacaa40 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc3155e1740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56089ba2a670 con 0x56089bb12600 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc30bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089baf5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc30bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089baf5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33828/0 (socket says 172.21.15.26:33828) 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc30bfff640 1 -- 172.21.15.26:0/4037726481 learned_addr learned my addr 172.21.15.26:0/4037726481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc310d96640 1 --2- 172.21.15.26:0/4037726481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bacaa40 0x56089bb10cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/4037726481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 0x56089bb12b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc30bfff640 1 -- 172.21.15.26:0/4037726481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 msgr2=0x56089bb12b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc30bfff640 1 --2- 172.21.15.26:0/4037726481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 0x56089bb12b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc30bfff640 1 -- 172.21.15.26:0/4037726481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bacaa40 msgr2=0x56089bb10cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc30bfff640 1 --2- 172.21.15.26:0/4037726481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bacaa40 0x56089bb10cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.315+0000 7fc30bfff640 1 -- 172.21.15.26:0/4037726481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089b80c7d0 con 0x56089baf4d60 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc30bfff640 1 --2- 172.21.15.26:0/4037726481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089baf5140 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc300004f90 tx=0x7fc300013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb95e1886bf7434 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc2f1ffb640 1 -- 172.21.15.26:0/4037726481 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300026070 con 0x56089baf4d60 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc2f1ffb640 1 -- 172.21.15.26:0/4037726481 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc300001cf0 con 0x56089baf4d60 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc2f1ffb640 1 -- 172.21.15.26:0/4037726481 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300013d80 con 0x56089baf4d60 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc3155e1740 1 -- 172.21.15.26:0/4037726481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 msgr2=0x56089baf5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc3155e1740 1 --2- 172.21.15.26:0/4037726481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089baf5140 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc300004f90 tx=0x7fc300013000 comp rx=0 tx=0).stop 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc3155e1740 1 -- 172.21.15.26:0/4037726481 shutdown_connections 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc3155e1740 1 --2- 172.21.15.26:0/4037726481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bacaa40 0x56089bb10cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc3155e1740 1 --2- 172.21.15.26:0/4037726481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 0x56089bb12b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc3155e1740 1 --2- 172.21.15.26:0/4037726481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089baf5140 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc3155e1740 1 -- 172.21.15.26:0/4037726481 >> 172.21.15.26:0/4037726481 conn(0x56089bac7600 msgr2=0x56089bb0fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc3155e1740 1 -- 172.21.15.26:0/4037726481 shutdown_connections 2024-04-24T23:22:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.316+0000 7fc3155e1740 1 -- 172.21.15.26:0/4037726481 wait complete. 2024-04-24T23:22:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.317+0000 7fc3155e1740 1 -- start start 2024-04-24T23:22:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.317+0000 7fc3155e1740 1 Processor -- start 2024-04-24T23:22:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 0x56089bb10b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089bb12600 0x56089bb11040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 0x56089bb07fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc3155e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56089bacb7e0 con 0x56089bb12600 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc3155e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56089b9e9f50 con 0x56089baf4d60 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc3155e1740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56089bac5530 con 0x56089baf5d10 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc3155e1740 10 client.0 _ll_register_callbacks cb 0x56089ba25000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc30b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089bb12600 0x56089bb11040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc30b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089bb12600 0x56089bb11040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33838/0 (socket says 172.21.15.26:33838) 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc30b7fe640 1 -- 172.21.15.26:0/740037348 learned_addr learned my addr 172.21.15.26:0/740037348 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc310d96640 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 0x56089bb07fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc30bfff640 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 0x56089bb10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc30b7fe640 1 -- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 msgr2=0x56089bb07fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 0x56089bb07fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc30b7fe640 1 -- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 msgr2=0x56089bb10b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 0x56089bb10b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.318+0000 7fc30b7fe640 1 -- 172.21.15.26:0/740037348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089b7423f0 con 0x56089bb12600 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089bb12600 0x56089bb11040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0553a0 tx=0x7fc2fc05d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da211b527e908d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc2eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc2eaffd640 1 -- 172.21.15.26:0/740037348 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc058390 con 0x56089bb12600 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc2eaffd640 1 -- 172.21.15.26:0/740037348 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc2fc058530 con 0x56089bb12600 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc3155e1740 1 -- 172.21.15.26:0/740037348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc300001cf0 con 0x56089bb12600 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc2eaffd640 1 -- 172.21.15.26:0/740037348 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc077050 con 0x56089bb12600 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc3155e1740 10 client.5418 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc3155e1740 1 -- 172.21.15.26:0/740037348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56089b7020f0 con 0x56089bb12600 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc2eaffd640 1 -- 172.21.15.26:0/740037348 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc300001cf0 con 0x56089bb12600 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc2eaffd640 1 -- 172.21.15.26:0/740037348 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7fc2fc05f440 con 0x56089bb12600 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc3155e1740 10 client.5418 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc3155e1740 10 client.5418 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.b 2024-04-24T23:22:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc3155e1740 20 client.5418 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc3155e1740 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x56089bb20120 0x56089bb23740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc3155e1740 4 client.5418 mds_command: new command op to 4864 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc3155e1740 1 -- 172.21.15.26:0/740037348 --> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56089ba3bbf0 con 0x56089bb20120 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.319+0000 7fc30bfff640 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x56089bb20120 0x56089bb23740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.320+0000 7fc2eaffd640 10 client.5418 ms_handle_connect on v2:172.21.15.29:6832/2201738459 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.320+0000 7fc30bfff640 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x56089bb20120 0x56089bb23740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50a79b1a14d27cd8 server_cookie=61e74bc528a3629d in_seq=0 out_seq=0 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.320+0000 7fc2eaffd640 1 -- 172.21.15.26:0/740037348 <== mds.2 v2:172.21.15.29:6832/2201738459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56089ba3bbf0 con 0x56089bb20120 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.320+0000 7fc2eaffd640 10 client.5418 handle_command_reply: tid=0 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.320+0000 7fc3155e1740 1 client.5418 shutdown 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 -- 172.21.15.26:0/740037348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089bb12600 msgr2=0x56089bb11040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089bb12600 0x56089bb11040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0553a0 tx=0x7fc2fc05d380 comp rx=0 tx=0).stop 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 -- 172.21.15.26:0/740037348 shutdown_connections 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x56089bb20120 0x56089bb23740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 0x56089bb07fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc2eaffd640 0 client.5418 ms_handle_reset on v2:172.21.15.29:6832/2201738459 2024-04-24T23:22:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089bb12600 0x56089bb11040 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 --2- 172.21.15.26:0/740037348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 0x56089bb10b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 -- 172.21.15.26:0/740037348 >> 172.21.15.26:0/740037348 conn(0x56089bacaa40 msgr2=0x56089bb13980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 -- 172.21.15.26:0/740037348 shutdown_connections 2024-04-24T23:22:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 1 -- 172.21.15.26:0/740037348 wait complete. 2024-04-24T23:22:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.321+0000 7fc3155e1740 20 client.5418 trim_cache size 0 max 16384 2024-04-24T23:22:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.336+0000 7fc3155e1740 1 Processor -- start 2024-04-24T23:22:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.336+0000 7fc3155e1740 1 -- start start 2024-04-24T23:22:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089bacb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 0x56089bac87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bb12600 0x56089bac8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc3155e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c2004f0 con 0x56089baf4d60 2024-04-24T23:22:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc3155e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1ff6c0 con 0x56089bb12600 2024-04-24T23:22:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc3155e1740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc30c1ffdc0 con 0x56089baf5d10 2024-04-24T23:22:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089bacb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089bacb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33852/0 (socket says 172.21.15.26:33852) 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30bfff640 1 -- 172.21.15.26:0/511672329 learned_addr learned my addr 172.21.15.26:0/511672329 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 0x56089bac87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc310d96640 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bb12600 0x56089bac8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30bfff640 1 -- 172.21.15.26:0/511672329 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 msgr2=0x56089bac87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30bfff640 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 0x56089bac87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30bfff640 1 -- 172.21.15.26:0/511672329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bb12600 msgr2=0x56089bac8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30bfff640 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bb12600 0x56089bac8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30bfff640 1 -- 172.21.15.26:0/511672329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089bb81a60 con 0x56089baf4d60 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc310d96640 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bb12600 0x56089bac8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc30bfff640 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089bacb010 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc30001f000 tx=0x7fc300014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41580182ec3780d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc2f17fa640 1 -- 172.21.15.26:0/511672329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300026040 con 0x56089baf4d60 2024-04-24T23:22:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc2f17fa640 1 -- 172.21.15.26:0/511672329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc300001cf0 con 0x56089baf4d60 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.337+0000 7fc2f17fa640 1 -- 172.21.15.26:0/511672329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300032160 con 0x56089baf4d60 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.338+0000 7fc3155e1740 1 -- 172.21.15.26:0/511672329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 msgr2=0x56089bacb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.338+0000 7fc3155e1740 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089bacb010 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc30001f000 tx=0x7fc300014000 comp rx=0 tx=0).stop 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.338+0000 7fc3155e1740 1 -- 172.21.15.26:0/511672329 shutdown_connections 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.338+0000 7fc3155e1740 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089bb12600 0x56089bac8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.338+0000 7fc3155e1740 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089baf5d10 0x56089bac87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.338+0000 7fc3155e1740 1 --2- 172.21.15.26:0/511672329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf4d60 0x56089bacb010 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.338+0000 7fc3155e1740 1 -- 172.21.15.26:0/511672329 >> 172.21.15.26:0/511672329 conn(0x56089bacaa40 msgr2=0x56089bb10e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.338+0000 7fc3155e1740 1 -- 172.21.15.26:0/511672329 shutdown_connections 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.338+0000 7fc3155e1740 1 -- 172.21.15.26:0/511672329 wait complete. 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.339+0000 7fc3155e1740 1 -- start start 2024-04-24T23:22:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.339+0000 7fc3155e1740 1 Processor -- start 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 0x56089baf3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf5d10 0x56089baf3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 0x56089bb32bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56089baf4b70 con 0x56089baf5d10 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c15ae60 con 0x56089baf4d60 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56089bab7090 con 0x56089bb12600 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 10 client.0 _ll_register_callbacks cb 0x7fc2fc00c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf5d10 0x56089baf3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf5d10 0x56089baf3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33866/0 (socket says 172.21.15.26:33866) 2024-04-24T23:22:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30b7fe640 1 -- 172.21.15.26:0/2698791508 learned_addr learned my addr 172.21.15.26:0/2698791508 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30bfff640 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 0x56089baf3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc310d96640 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 0x56089bb32bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30b7fe640 1 -- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 msgr2=0x56089bb32bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 0x56089bb32bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30b7fe640 1 -- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 msgr2=0x56089baf3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 0x56089baf3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30b7fe640 1 -- 172.21.15.26:0/2698791508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089baedea0 con 0x56089baf5d10 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc30b7fe640 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf5d10 0x56089baf3540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc058a10 tx=0x7fc2fc06e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ea6229d9fc3853 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc2eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc2eaffd640 1 -- 172.21.15.26:0/2698791508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc058590 con 0x56089baf5d10 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc2eaffd640 1 -- 172.21.15.26:0/2698791508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc2fc054c90 con 0x56089baf5d10 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 1 -- 172.21.15.26:0/2698791508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc300001cf0 con 0x56089baf5d10 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc2eaffd640 1 -- 172.21.15.26:0/2698791508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc06a050 con 0x56089baf5d10 2024-04-24T23:22:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 10 client.5424 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.340+0000 7fc3155e1740 1 -- 172.21.15.26:0/2698791508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56089bb33f90 con 0x56089baf5d10 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc2eaffd640 1 -- 172.21.15.26:0/2698791508 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc300001cf0 con 0x56089baf5d10 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc2eaffd640 1 -- 172.21.15.26:0/2698791508 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7fc2fc054e90 con 0x56089baf5d10 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc3155e1740 10 client.5424 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc3155e1740 10 client.5424 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.b 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc3155e1740 20 client.5424 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc3155e1740 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x56089bac96c0 0x56089bb33130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc3155e1740 4 client.5424 mds_command: new command op to 4864 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc3155e1740 1 -- 172.21.15.26:0/2698791508 --> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56089ba35f40 con 0x56089bac96c0 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc30bfff640 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x56089bac96c0 0x56089bb33130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc30bfff640 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x56089bac96c0 0x56089bb33130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7841b78563ea20b0 server_cookie=e98fbbcd90aa339d in_seq=0 out_seq=0 2024-04-24T23:22:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.341+0000 7fc2eaffd640 10 client.5424 ms_handle_connect on v2:172.21.15.29:6832/2201738459 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.347+0000 7fc2eaffd640 1 -- 172.21.15.26:0/2698791508 <== mds.2 v2:172.21.15.29:6832/2201738459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56089ba35f40 con 0x56089bac96c0 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.347+0000 7fc2eaffd640 10 client.5424 handle_command_reply: tid=0 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.347+0000 7fc3155e1740 1 client.5424 shutdown 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 -- 172.21.15.26:0/2698791508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf5d10 msgr2=0x56089baf3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf5d10 0x56089baf3540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc058a10 tx=0x7fc2fc06e000 comp rx=0 tx=0).stop 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 -- 172.21.15.26:0/2698791508 shutdown_connections 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:6832/2201738459,v1:172.21.15.29:6834/2201738459] conn(0x56089bac96c0 0x56089bb33130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56089bb12600 0x56089bb32bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc2eaffd640 0 client.5424 ms_handle_reset on v2:172.21.15.29:6832/2201738459 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56089baf5d10 0x56089baf3540 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 --2- 172.21.15.26:0/2698791508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56089baf4d60 0x56089baf3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 -- 172.21.15.26:0/2698791508 >> 172.21.15.26:0/2698791508 conn(0x56089bacaa40 msgr2=0x56089bb0e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 -- 172.21.15.26:0/2698791508 shutdown_connections 2024-04-24T23:22:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 1 -- 172.21.15.26:0/2698791508 wait complete. 2024-04-24T23:22:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3155e1740 20 client.5424 trim_cache size 0 max 16384 2024-04-24T23:22:14.352 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:22:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:22:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:22:14.352 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:22:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3127f9640 1 -- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2f806be30 msgr2=0x7fc2f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.348+0000 7fc3127f9640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2f806be30 0x7fc2f806e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc300000f60 tx=0x7fc300007d50 comp rx=0 tx=0).stop 2024-04-24T23:22:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 -- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c15e370 msgr2=0x7fc30c1563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c15e370 0x7fc30c1563b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00c430 tx=0x7fc2fc00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 -- 172.21.15.26:0/3973901920 shutdown_connections 2024-04-24T23:22:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2f806be30 0x7fc2f806e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc30c15bad0 0x7fc30c1568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc30c15e370 0x7fc30c1563b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 --2- 172.21.15.26:0/3973901920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc30c0d2c30 0x7fc30c0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:14.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 -- 172.21.15.26:0/3973901920 >> 172.21.15.26:0/3973901920 conn(0x7fc30c0d40f0 msgr2=0x7fc30c15d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:14.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 -- 172.21.15.26:0/3973901920 shutdown_connections 2024-04-24T23:22:14.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14.349+0000 7fc3127f9640 1 -- 172.21.15.26:0/3973901920 wait complete. 2024-04-24T23:22:14.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:14.362 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-04-24T23:22:14.383 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/ 2024-04-24T23:22:14.383 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4 2024-04-24T23:22:14.383 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5 2024-04-24T23:22:14.383 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6 2024-04-24T23:22:14.383 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7 2024-04-24T23:22:14.383 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-04-24T23:22:14.384 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:22:14.384 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:22:14.384 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:22:14.384 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:22:14.384 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-04-24T23:22:14.384 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:22:14.490 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627778 2024-04-24T23:22:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:22:14.499 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:22:14.499 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:22:14.499 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:22:14.499 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:22:14.499 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-04-24T23:22:14.499 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:22:14.606 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627779 2024-04-24T23:22:14.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:22:14.615 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:22:14.615 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:22:14.615 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:22:14.615 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:22:14.615 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-04-24T23:22:14.615 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:22:14.721 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627780 2024-04-24T23:22:14.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:22:14.729 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:22:14.730 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:22:14.730 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:22:14.730 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:22:14.730 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-04-24T23:22:14.730 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:22:14.836 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627781 2024-04-24T23:22:14.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:22:14.845 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:22:14.845 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:22:14.845 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:22:14.845 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:22:14.845 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-04-24T23:22:14.845 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:22:14.952 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627782 2024-04-24T23:22:14.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:22:14.961 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:22:14.961 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:22:14.961 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:22:14.961 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:22:14.961 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-04-24T23:22:14.961 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:22:15.068 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627783 2024-04-24T23:22:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:22:15.078 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 tag path /d1/d2/d3 tag123 2024-04-24T23:22:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.220+0000 7f831361c640 1 Processor -- start 2024-04-24T23:22:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.220+0000 7f831361c640 1 -- start start 2024-04-24T23:22:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f831361c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1663e0 0x7f830c0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f831361c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f830c14ba40 0x7f830c14bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f831361c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f830c0b3a60 con 0x7f830c1667c0 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f831361c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f830c166fa0 con 0x7f830c14ba40 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f831361c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f830c0b3d40 con 0x7f830c1663e0 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f8311bb9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f830c14ba40 0x7f830c14bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f8310bb7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f830c1667c0 0x7f830c0b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38150/0 (socket says 172.21.15.26:38150) 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f8310bb7640 1 -- 172.21.15.26:0/1733183784 learned_addr learned my addr 172.21.15.26:0/1733183784 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f8303fff640 1 -- 172.21.15.26:0/1733183784 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093819352 0 0) 0x7f830c0b3a60 con 0x7f830c1667c0 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f8303fff640 1 -- 172.21.15.26:0/1733183784 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82f8003520 con 0x7f830c1667c0 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f83113b8640 1 --2- 172.21.15.26:0/1733183784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1663e0 0x7f830c0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f8311bb9640 1 -- 172.21.15.26:0/1733183784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1663e0 msgr2=0x7f830c0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f8311bb9640 1 --2- 172.21.15.26:0/1733183784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1663e0 0x7f830c0b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f8311bb9640 1 -- 172.21.15.26:0/1733183784 >> v1:172.21.15.26:6789/0 conn(0x7f830c1667c0 legacy=0x7f830c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.221+0000 7f8311bb9640 1 -- 172.21.15.26:0/1733183784 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c0b4020 con 0x7f830c14ba40 2024-04-24T23:22:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.222+0000 7f8311bb9640 1 --2- 172.21.15.26:0/1733183784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c14ba40 0x7f830c14bed0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8304009d90 tx=0x7f830402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9f981e46fa02b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.222+0000 7f8303fff640 1 -- 172.21.15.26:0/1733183784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304007740 con 0x7f830c14ba40 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.222+0000 7f8303fff640 1 -- 172.21.15.26:0/1733183784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83040078e0 con 0x7f830c14ba40 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.222+0000 7f8303fff640 1 -- 172.21.15.26:0/1733183784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304007cb0 con 0x7f830c14ba40 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.223+0000 7f831361c640 1 -- 172.21.15.26:0/1733183784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c14ba40 msgr2=0x7f830c14bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.223+0000 7f831361c640 1 --2- 172.21.15.26:0/1733183784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c14ba40 0x7f830c14bed0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8304009d90 tx=0x7f830402a660 comp rx=0 tx=0).stop 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.223+0000 7f831361c640 1 -- 172.21.15.26:0/1733183784 shutdown_connections 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.223+0000 7f831361c640 1 --2- 172.21.15.26:0/1733183784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c14ba40 0x7f830c14bed0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.223+0000 7f831361c640 1 --2- 172.21.15.26:0/1733183784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1663e0 0x7f830c0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.223+0000 7f831361c640 1 -- 172.21.15.26:0/1733183784 >> 172.21.15.26:0/1733183784 conn(0x7f830c06b300 msgr2=0x7f830c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.223+0000 7f831361c640 1 -- 172.21.15.26:0/1733183784 shutdown_connections 2024-04-24T23:22:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.223+0000 7f831361c640 1 -- 172.21.15.26:0/1733183784 wait complete. 2024-04-24T23:22:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.224+0000 7f831361c640 1 Processor -- start 2024-04-24T23:22:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.224+0000 7f831361c640 1 -- start start 2024-04-24T23:22:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f831361c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f830c14ba40 0x7f830c0aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f831361c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c1663e0 0x7f830c13bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f831361c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1667c0 0x7f830c0aa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f831361c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f830c14c410 con 0x7f830c1663e0 2024-04-24T23:22:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f831361c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f830c0b45e0 con 0x7f830c14ba40 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f831361c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f830c0b3600 con 0x7f830c1667c0 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8310bb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c1663e0 0x7f830c13bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8310bb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c1663e0 0x7f830c13bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33876/0 (socket says 172.21.15.26:33876) 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8310bb7640 1 -- 172.21.15.26:0/474311129 learned_addr learned my addr 172.21.15.26:0/474311129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8311bb9640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1667c0 0x7f830c0aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f83113b8640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f830c14ba40 0x7f830c0aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8310bb7640 1 -- 172.21.15.26:0/474311129 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1667c0 msgr2=0x7f830c0aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8310bb7640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1667c0 0x7f830c0aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8310bb7640 1 -- 172.21.15.26:0/474311129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f830c14ba40 msgr2=0x7f830c0aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8310bb7640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f830c14ba40 0x7f830c0aa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8310bb7640 1 -- 172.21.15.26:0/474311129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c0ab380 con 0x7f830c1663e0 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8310bb7640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c1663e0 0x7f830c13bc40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00c390 tx=0x7f82fc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8301ffb640 1 -- 172.21.15.26:0/474311129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc017070 con 0x7f830c1663e0 2024-04-24T23:22:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8301ffb640 1 -- 172.21.15.26:0/474311129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82fc005ce0 con 0x7f830c1663e0 2024-04-24T23:22:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f831361c640 1 -- 172.21.15.26:0/474311129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c0a9500 con 0x7f830c1663e0 2024-04-24T23:22:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.225+0000 7f8301ffb640 1 -- 172.21.15.26:0/474311129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0072f0 con 0x7f830c1663e0 2024-04-24T23:22:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.226+0000 7f831361c640 1 -- 172.21.15.26:0/474311129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c0a9ac0 con 0x7f830c1663e0 2024-04-24T23:22:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.227+0000 7f8301ffb640 1 -- 172.21.15.26:0/474311129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82fc01a070 con 0x7f830c1663e0 2024-04-24T23:22:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.227+0000 7f8301ffb640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82f006be30 0x7f82f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.227+0000 7f8301ffb640 1 -- 172.21.15.26:0/474311129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f82fc08a2a0 con 0x7f830c1663e0 2024-04-24T23:22:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.227+0000 7f83113b8640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82f006be30 0x7f82f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.228+0000 7f83113b8640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82f006be30 0x7f82f006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8308006fd0 tx=0x7f8308008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.228+0000 7f8316408740 1 Processor -- start 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.228+0000 7f8316408740 1 -- start start 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f8316408740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b2c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f8316408740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728ae3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f8316408740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728ae3640 0x55d728b0edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f8316408740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d728a47940 con 0x55d728b0e9f0 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f8316408740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d7289ef920 con 0x55d728ae3640 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f8316408740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d728a43900 con 0x55d728b0aee0 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f83113b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b2c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f83113b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b2c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33880/0 (socket says 172.21.15.26:33880) 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f83113b8640 1 -- 172.21.15.26:0/567179014 learned_addr learned my addr 172.21.15.26:0/567179014 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f83113b8640 1 -- 172.21.15.26:0/567179014 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 msgr2=0x55d728ae3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f83113b8640 1 --2- 172.21.15.26:0/567179014 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728ae3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f83113b8640 1 -- 172.21.15.26:0/567179014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728ae3640 msgr2=0x55d728b0edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.229+0000 7f83113b8640 1 --2- 172.21.15.26:0/567179014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728ae3640 0x55d728b0edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f83113b8640 1 -- 172.21.15.26:0/567179014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d728827a20 con 0x55d728b0e9f0 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f83113b8640 1 --2- 172.21.15.26:0/567179014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b2c7b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f830800eca0 tx=0x7f8308011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79fb7d34f402dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f82f6ffd640 1 -- 172.21.15.26:0/567179014 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830801a050 con 0x55d728b0e9f0 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f82f6ffd640 1 -- 172.21.15.26:0/567179014 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8308024070 con 0x55d728b0e9f0 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f82f6ffd640 1 -- 172.21.15.26:0/567179014 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830801f660 con 0x55d728b0e9f0 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f8316408740 1 -- 172.21.15.26:0/567179014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 msgr2=0x55d728b2c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f8316408740 1 --2- 172.21.15.26:0/567179014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b2c7b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f830800eca0 tx=0x7f8308011000 comp rx=0 tx=0).stop 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f8316408740 1 -- 172.21.15.26:0/567179014 shutdown_connections 2024-04-24T23:22:15.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f8316408740 1 --2- 172.21.15.26:0/567179014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728ae3640 0x55d728b0edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f8316408740 1 --2- 172.21.15.26:0/567179014 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728ae3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f8316408740 1 --2- 172.21.15.26:0/567179014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b2c7b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f8316408740 1 -- 172.21.15.26:0/567179014 >> 172.21.15.26:0/567179014 conn(0x55d728ae1150 msgr2=0x55d728b2cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f8316408740 1 -- 172.21.15.26:0/567179014 shutdown_connections 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.230+0000 7f8316408740 1 -- 172.21.15.26:0/567179014 wait complete. 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.231+0000 7f8316408740 1 -- start start 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.231+0000 7f8316408740 1 Processor -- start 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8316408740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728b0d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8316408740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b10e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8316408740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b11340 0x55d728afbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8316408740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d728b2de40 con 0x55d728b0e9f0 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8316408740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d728a03030 con 0x55d728b11340 2024-04-24T23:22:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8316408740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d728b2d680 con 0x55d728b0aee0 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8316408740 10 client.0 _ll_register_callbacks cb 0x55d7289fde00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8310bb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b10e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8310bb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b10e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33882/0 (socket says 172.21.15.26:33882) 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8310bb7640 1 -- 172.21.15.26:0/733158600 learned_addr learned my addr 172.21.15.26:0/733158600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8311bb9640 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b11340 0x55d728afbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f83113b8640 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728b0d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8310bb7640 1 -- 172.21.15.26:0/733158600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 msgr2=0x55d728b0d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8310bb7640 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728b0d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8310bb7640 1 -- 172.21.15.26:0/733158600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b11340 msgr2=0x55d728afbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8310bb7640 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b11340 0x55d728afbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.232+0000 7f8310bb7640 1 -- 172.21.15.26:0/733158600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72871b310 con 0x55d728b0e9f0 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8310bb7640 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b10e00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f82fc060c00 tx=0x7f82fc058d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b83dcebee2d0b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f82ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f82ebfff640 1 -- 172.21.15.26:0/733158600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc007690 con 0x55d728b0e9f0 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f82ebfff640 1 -- 172.21.15.26:0/733158600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f82fc0579a0 con 0x55d728b0e9f0 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8316408740 1 -- 172.21.15.26:0/733158600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8308024070 con 0x55d728b0e9f0 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f82ebfff640 1 -- 172.21.15.26:0/733158600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f82fc08a0c0 con 0x55d728b0e9f0 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8316408740 10 client.5439 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8316408740 1 -- 172.21.15.26:0/733158600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55d7288881b0 con 0x55d728b0e9f0 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f82ebfff640 1 -- 172.21.15.26:0/733158600 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8308024070 con 0x55d728b0e9f0 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f82ebfff640 1 -- 172.21.15.26:0/733158600 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7f82fc0542a0 con 0x55d728b0e9f0 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8316408740 10 client.5439 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8316408740 10 client.5439 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8316408740 20 client.5439 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8316408740 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d728b27170 0x55d728b27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8316408740 4 client.5439 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f8316408740 1 -- 172.21.15.26:0/733158600 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d728a54e80 con 0x55d728b27170 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.233+0000 7f83113b8640 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d728b27170 0x55d728b27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.234+0000 7f83113b8640 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d728b27170 0x55d728b27580 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38a49e7a1caefaaa server_cookie=f6441d6e8ac3dc2f in_seq=0 out_seq=0 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.234+0000 7f82ebfff640 10 client.5439 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.234+0000 7f82ebfff640 1 -- 172.21.15.26:0/733158600 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d728a54e80 con 0x55d728b27170 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.234+0000 7f82ebfff640 10 client.5439 handle_command_reply: tid=0 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.234+0000 7f8316408740 1 client.5439 shutdown 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.234+0000 7f8316408740 1 -- 172.21.15.26:0/733158600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 msgr2=0x55d728b10e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.234+0000 7f8316408740 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b10e00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f82fc060c00 tx=0x7f82fc058d90 comp rx=0 tx=0).stop 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.234+0000 7f8316408740 1 -- 172.21.15.26:0/733158600 shutdown_connections 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.235+0000 7f8316408740 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d728b27170 0x55d728b27580 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.235+0000 7f8316408740 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b11340 0x55d728afbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.235+0000 7f8316408740 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b0e9f0 0x55d728b10e00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.235+0000 7f8316408740 1 --2- 172.21.15.26:0/733158600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728b0d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.235+0000 7f82ebfff640 0 client.5439 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:15.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.235+0000 7f8316408740 1 -- 172.21.15.26:0/733158600 >> 172.21.15.26:0/733158600 conn(0x55d728ae3640 msgr2=0x55d728b23540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.235+0000 7f8316408740 1 -- 172.21.15.26:0/733158600 shutdown_connections 2024-04-24T23:22:15.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.235+0000 7f8316408740 1 -- 172.21.15.26:0/733158600 wait complete. 2024-04-24T23:22:15.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.235+0000 7f8316408740 20 client.5439 trim_cache size 0 max 16384 2024-04-24T23:22:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8316408740 1 Processor -- start 2024-04-24T23:22:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8316408740 1 -- start start 2024-04-24T23:22:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8316408740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728b0efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8316408740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0e9f0 0x55d728ae4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8316408740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b43720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8316408740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f830c204890 con 0x55d728b11340 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8316408740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f830c203cb0 con 0x55d728b0e9f0 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8316408740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f830c204160 con 0x55d728b0aee0 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8311bb9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b43720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8311bb9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b43720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33888/0 (socket says 172.21.15.26:33888) 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.254+0000 7f8311bb9640 1 -- 172.21.15.26:0/848326654 learned_addr learned my addr 172.21.15.26:0/848326654 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f8310bb7640 1 --2- 172.21.15.26:0/848326654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0e9f0 0x55d728ae4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f83113b8640 1 --2- 172.21.15.26:0/848326654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728b0efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f8311bb9640 1 -- 172.21.15.26:0/848326654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 msgr2=0x55d728b0efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f8311bb9640 1 --2- 172.21.15.26:0/848326654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728b0efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f8311bb9640 1 -- 172.21.15.26:0/848326654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0e9f0 msgr2=0x55d728ae4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f8311bb9640 1 --2- 172.21.15.26:0/848326654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0e9f0 0x55d728ae4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f8311bb9640 1 -- 172.21.15.26:0/848326654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7286f4490 con 0x55d728b11340 2024-04-24T23:22:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f8311bb9640 1 --2- 172.21.15.26:0/848326654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b43720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8304009eb0 tx=0x7f830402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c19c67330093c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f82f67fc640 1 -- 172.21.15.26:0/848326654 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304007600 con 0x55d728b11340 2024-04-24T23:22:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f82f67fc640 1 -- 172.21.15.26:0/848326654 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83040077a0 con 0x55d728b11340 2024-04-24T23:22:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f82f67fc640 1 -- 172.21.15.26:0/848326654 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304007b90 con 0x55d728b11340 2024-04-24T23:22:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f8316408740 1 -- 172.21.15.26:0/848326654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 msgr2=0x55d728b43720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.255+0000 7f8316408740 1 --2- 172.21.15.26:0/848326654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b43720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8304009eb0 tx=0x7f830402c000 comp rx=0 tx=0).stop 2024-04-24T23:22:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.256+0000 7f8316408740 1 -- 172.21.15.26:0/848326654 shutdown_connections 2024-04-24T23:22:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.256+0000 7f8316408740 1 --2- 172.21.15.26:0/848326654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b43720 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.256+0000 7f8316408740 1 --2- 172.21.15.26:0/848326654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0e9f0 0x55d728ae4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.256+0000 7f8316408740 1 --2- 172.21.15.26:0/848326654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0aee0 0x55d728b0efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.256+0000 7f8316408740 1 -- 172.21.15.26:0/848326654 >> 172.21.15.26:0/848326654 conn(0x55d728ae3640 msgr2=0x55d728b0d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.256+0000 7f8316408740 1 -- 172.21.15.26:0/848326654 shutdown_connections 2024-04-24T23:22:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.256+0000 7f8316408740 1 -- 172.21.15.26:0/848326654 wait complete. 2024-04-24T23:22:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.256+0000 7f8316408740 1 -- start start 2024-04-24T23:22:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.256+0000 7f8316408740 1 Processor -- start 2024-04-24T23:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8316408740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0aee0 0x55d728b8c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8316408740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0e9f0 0x55d728b0b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8316408740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b0b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8316408740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f830c162bb0 con 0x55d728b11340 2024-04-24T23:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8316408740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d728ae0da0 con 0x55d728b0aee0 2024-04-24T23:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8316408740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d728aded60 con 0x55d728b0e9f0 2024-04-24T23:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8316408740 10 client.0 _ll_register_callbacks cb 0x7f82fc005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8311bb9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b0b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8311bb9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b0b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33898/0 (socket says 172.21.15.26:33898) 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8311bb9640 1 -- 172.21.15.26:0/2178084274 learned_addr learned my addr 172.21.15.26:0/2178084274 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f83113b8640 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0aee0 0x55d728b8c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.257+0000 7f8310bb7640 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0e9f0 0x55d728b0b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8311bb9640 1 -- 172.21.15.26:0/2178084274 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0e9f0 msgr2=0x55d728b0b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8311bb9640 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0e9f0 0x55d728b0b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8311bb9640 1 -- 172.21.15.26:0/2178084274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0aee0 msgr2=0x55d728b8c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8311bb9640 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0aee0 0x55d728b8c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8311bb9640 1 -- 172.21.15.26:0/2178084274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d728b2c760 con 0x55d728b11340 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8311bb9640 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b0b860 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f830402ac20 tx=0x7f8304004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f894b17ef1d31cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f82ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f82ebfff640 1 -- 172.21.15.26:0/2178084274 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830404f680 con 0x55d728b11340 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f82ebfff640 1 -- 172.21.15.26:0/2178084274 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f830404f820 con 0x55d728b11340 2024-04-24T23:22:15.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8316408740 1 -- 172.21.15.26:0/2178084274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83040077a0 con 0x55d728b11340 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f82ebfff640 1 -- 172.21.15.26:0/2178084274 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f830405c050 con 0x55d728b11340 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8316408740 10 client.5445 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8316408740 1 -- 172.21.15.26:0/2178084274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55d728b10e40 con 0x55d728b11340 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f82ebfff640 1 -- 172.21.15.26:0/2178084274 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83040077a0 con 0x55d728b11340 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f82ebfff640 1 -- 172.21.15.26:0/2178084274 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7f830404fa60 con 0x55d728b11340 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8316408740 10 client.5445 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8316408740 10 client.5445 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8316408740 20 client.5445 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8316408740 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d728ae3a20 0x55d728ae3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8316408740 4 client.5445 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}] 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f8316408740 1 -- 172.21.15.26:0/2178084274 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}) -- 0x55d728b26250 con 0x55d728ae3a20 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.258+0000 7f83113b8640 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d728ae3a20 0x55d728ae3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.259+0000 7f82ebfff640 10 client.5445 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.259+0000 7f83113b8640 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d728ae3a20 0x55d728ae3e30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d999481457d88e server_cookie=b875f5b1aed6ec8 in_seq=0 out_seq=0 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.259+0000 7f82ebfff640 1 -- 172.21.15.26:0/2178084274 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55d728b26250 con 0x55d728ae3a20 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.259+0000 7f82ebfff640 10 client.5445 handle_command_reply: tid=0 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.259+0000 7f8316408740 1 client.5445 shutdown 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 -- 172.21.15.26:0/2178084274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 msgr2=0x55d728b0b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b0b860 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f830402ac20 tx=0x7f8304004ab0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 -- 172.21.15.26:0/2178084274 shutdown_connections 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x55d728ae3a20 0x55d728ae3e30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d728b11340 0x55d728b0b860 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d728b0e9f0 0x55d728b0b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f82ebfff640 0 client.5445 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 --2- 172.21.15.26:0/2178084274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d728b0aee0 0x55d728b8c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 -- 172.21.15.26:0/2178084274 >> 172.21.15.26:0/2178084274 conn(0x55d728ae3640 msgr2=0x55d728b0cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 -- 172.21.15.26:0/2178084274 shutdown_connections 2024-04-24T23:22:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 1 -- 172.21.15.26:0/2178084274 wait complete. 2024-04-24T23:22:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.260+0000 7f8316408740 20 client.5445 trim_cache size 0 max 16384 2024-04-24T23:22:15.269 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:22:15.269 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-04-24T23:22:15.269 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "tag123", 2024-04-24T23:22:15.269 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-04-24T23:22:15.269 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 -- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82f006be30 msgr2=0x7f82f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82f006be30 0x7f82f006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8308006fd0 tx=0x7f8308008040 comp rx=0 tx=0).stop 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 -- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c1663e0 msgr2=0x7f830c13bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c1663e0 0x7f830c13bc40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00c390 tx=0x7f82fc00c850 comp rx=0 tx=0).stop 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 -- 172.21.15.26:0/474311129 shutdown_connections 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82f006be30 0x7f82f006e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f830c1667c0 0x7f830c0aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f830c1663e0 0x7f830c13bc40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 --2- 172.21.15.26:0/474311129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f830c14ba40 0x7f830c0aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 -- 172.21.15.26:0/474311129 >> 172.21.15.26:0/474311129 conn(0x7f830c06b300 msgr2=0x7f830c0af970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 -- 172.21.15.26:0/474311129 shutdown_connections 2024-04-24T23:22:15.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.261+0000 7f831361c640 1 -- 172.21.15.26:0/474311129 wait complete. 2024-04-24T23:22:15.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:15.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-24T23:22:15.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.375+0000 7f2549fa2640 1 Processor -- start 2024-04-24T23:22:15.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.375+0000 7f2549fa2640 1 -- start start 2024-04-24T23:22:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.376+0000 7f2549fa2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f25440d28e0 0x7f25440d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.376+0000 7f2549fa2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254406d750 con 0x7f254415da30 2024-04-24T23:22:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.376+0000 7f2549fa2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25440b1720 con 0x7f25440ce640 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.376+0000 7f2549fa2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f25440d1110 con 0x7f25440d28e0 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.376+0000 7f2543fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f254415da30 0x7f254415dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38156/0 (socket says 172.21.15.26:38156) 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.376+0000 7f2543fff640 1 -- 172.21.15.26:0/2672160642 learned_addr learned my addr 172.21.15.26:0/2672160642 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.376+0000 7f2542ffd640 1 --2- 172.21.15.26:0/2672160642 >> v2:172.21.15.29:3300/0 conn(0x7f25440d28e0 0x7f25440d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.376+0000 7f25427fc640 1 -- 172.21.15.26:0/2672160642 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 98123200 0 0) 0x7f254406d750 con 0x7f254415da30 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.377+0000 7f25427fc640 1 -- 172.21.15.26:0/2672160642 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f252c003520 con 0x7f254415da30 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.377+0000 7f25427fc640 1 -- 172.21.15.26:0/2672160642 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330228054 0 0) 0x7f2534001110 con 0x7f25440ce640 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.377+0000 7f25427fc640 1 -- 172.21.15.26:0/2672160642 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f254406d750 con 0x7f25440ce640 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.377+0000 7f2542ffd640 1 -- 172.21.15.26:0/2672160642 >> v1:172.21.15.29:6789/0 conn(0x7f25440ce640 legacy=0x7f25440cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.377+0000 7f2542ffd640 1 -- 172.21.15.26:0/2672160642 >> v1:172.21.15.26:6789/0 conn(0x7f254415da30 legacy=0x7f254415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.377+0000 7f25427fc640 1 -- 172.21.15.26:0/2672160642 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 712236811 0 0) 0x7f252c003520 con 0x7f254415da30 2024-04-24T23:22:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.377+0000 7f2542ffd640 1 -- 172.21.15.26:0/2672160642 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25440d1360 con 0x7f25440d28e0 2024-04-24T23:22:15.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f2542ffd640 1 --2- 172.21.15.26:0/2672160642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25440d28e0 0x7f25440d00f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25380097a0 tx=0x7f253802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25e2db8c0a1a6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f25427fc640 1 -- 172.21.15.26:0/2672160642 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538037070 con 0x7f25440d28e0 2024-04-24T23:22:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f25427fc640 1 -- 172.21.15.26:0/2672160642 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253802ace0 con 0x7f25440d28e0 2024-04-24T23:22:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f25427fc640 1 -- 172.21.15.26:0/2672160642 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0019f0 con 0x7f25440d28e0 2024-04-24T23:22:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f2549fa2640 1 -- 172.21.15.26:0/2672160642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25440d28e0 msgr2=0x7f25440d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f2549fa2640 1 --2- 172.21.15.26:0/2672160642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25440d28e0 0x7f25440d00f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25380097a0 tx=0x7f253802a250 comp rx=0 tx=0).stop 2024-04-24T23:22:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f2549fa2640 1 -- 172.21.15.26:0/2672160642 shutdown_connections 2024-04-24T23:22:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f2549fa2640 1 --2- 172.21.15.26:0/2672160642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25440d28e0 0x7f25440d00f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f2549fa2640 1 -- 172.21.15.26:0/2672160642 >> 172.21.15.26:0/2672160642 conn(0x7f25440d1dc0 msgr2=0x7f25440d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f2549fa2640 1 -- 172.21.15.26:0/2672160642 shutdown_connections 2024-04-24T23:22:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.378+0000 7f2549fa2640 1 -- 172.21.15.26:0/2672160642 wait complete. 2024-04-24T23:22:15.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.380+0000 7f2549fa2640 1 Processor -- start 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.380+0000 7f2549fa2640 1 -- start start 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.380+0000 7f2549fa2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25440ce640 0x7f2544157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.380+0000 7f2549fa2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440d28e0 0x7f254409e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.380+0000 7f2549fa2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f254415da30 0x7f25440ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.380+0000 7f2549fa2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25440cc940 con 0x7f25440d28e0 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.380+0000 7f2549fa2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f25440cc410 con 0x7f25440ce640 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.380+0000 7f2549fa2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f25440ccc20 con 0x7f254415da30 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440d28e0 0x7f254409e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440d28e0 0x7f254409e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33912/0 (socket says 172.21.15.26:33912) 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2542ffd640 1 -- 172.21.15.26:0/3936070283 learned_addr learned my addr 172.21.15.26:0/3936070283 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f25437fe640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25440ce640 0x7f2544157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2543fff640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f254415da30 0x7f25440ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2542ffd640 1 -- 172.21.15.26:0/3936070283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f254415da30 msgr2=0x7f25440ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2542ffd640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f254415da30 0x7f25440ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2542ffd640 1 -- 172.21.15.26:0/3936070283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25440ce640 msgr2=0x7f2544157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2542ffd640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25440ce640 0x7f2544157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2542ffd640 1 -- 172.21.15.26:0/3936070283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25440b5ec0 con 0x7f25440d28e0 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2543fff640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f254415da30 0x7f25440ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2542ffd640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440d28e0 0x7f254409e5a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2538035040 tx=0x7f25380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f252bfff640 1 -- 172.21.15.26:0/3936070283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538040020 con 0x7f25440d28e0 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f252bfff640 1 -- 172.21.15.26:0/3936070283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2538038040 con 0x7f25440d28e0 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2549fa2640 1 -- 172.21.15.26:0/3936070283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254409f060 con 0x7f25440d28e0 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f252bfff640 1 -- 172.21.15.26:0/3936070283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538037070 con 0x7f25440d28e0 2024-04-24T23:22:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.381+0000 7f2549fa2640 1 -- 172.21.15.26:0/3936070283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254409f620 con 0x7f25440d28e0 2024-04-24T23:22:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.383+0000 7f252bfff640 1 -- 172.21.15.26:0/3936070283 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2538043020 con 0x7f25440d28e0 2024-04-24T23:22:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.383+0000 7f252bfff640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f253006be60 0x7f253006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.383+0000 7f252bfff640 1 -- 172.21.15.26:0/3936070283 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f25380aaec0 con 0x7f25440d28e0 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.383+0000 7f25437fe640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f253006be60 0x7f253006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.384+0000 7f25437fe640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f253006be60 0x7f253006e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2534007fd0 tx=0x7f2534009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.384+0000 7f254cd92740 1 Processor -- start 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.384+0000 7f254cd92740 1 -- start start 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f254cd92740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d4d60 0x555e103d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f254cd92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e103f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f254cd92740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103aaa40 0x555e103f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f254cd92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555e1030e6b0 con 0x555e103f2600 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f254cd92740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555e102b6880 con 0x555e103aaa40 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f254cd92740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555e1030a670 con 0x555e103d4d60 2024-04-24T23:22:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e103f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e103f2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33928/0 (socket says 172.21.15.26:33928) 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2542ffd640 1 -- 172.21.15.26:0/3211856759 learned_addr learned my addr 172.21.15.26:0/3211856759 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2543fff640 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103aaa40 0x555e103f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f25437fe640 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d4d60 0x555e103d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2542ffd640 1 -- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d4d60 msgr2=0x555e103d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2542ffd640 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d4d60 0x555e103d5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2542ffd640 1 -- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103aaa40 msgr2=0x555e103f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2542ffd640 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103aaa40 0x555e103f0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2542ffd640 1 -- 172.21.15.26:0/3211856759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e100ec7d0 con 0x555e103f2600 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f2543fff640 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103aaa40 0x555e103f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.385+0000 7f25437fe640 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d4d60 0x555e103d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f2542ffd640 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e103f2b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f25380a9410 tx=0x7f2538009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b8ea86c97aab22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f25297fa640 1 -- 172.21.15.26:0/3211856759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538079170 con 0x555e103f2600 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f25297fa640 1 -- 172.21.15.26:0/3211856759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253807d520 con 0x555e103f2600 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f25297fa640 1 -- 172.21.15.26:0/3211856759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538081d50 con 0x555e103f2600 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f254cd92740 1 -- 172.21.15.26:0/3211856759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 msgr2=0x555e103f2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f254cd92740 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e103f2b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f25380a9410 tx=0x7f2538009190 comp rx=0 tx=0).stop 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f254cd92740 1 -- 172.21.15.26:0/3211856759 shutdown_connections 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f254cd92740 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103aaa40 0x555e103f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f254cd92740 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e103f2b50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f254cd92740 1 --2- 172.21.15.26:0/3211856759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d4d60 0x555e103d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f254cd92740 1 -- 172.21.15.26:0/3211856759 >> 172.21.15.26:0/3211856759 conn(0x555e103a7600 msgr2=0x555e103efb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f254cd92740 1 -- 172.21.15.26:0/3211856759 shutdown_connections 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.386+0000 7f254cd92740 1 -- 172.21.15.26:0/3211856759 wait complete. 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.387+0000 7f254cd92740 1 -- start start 2024-04-24T23:22:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.387+0000 7f254cd92740 1 Processor -- start 2024-04-24T23:22:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f254cd92740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 0x555e103e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f254cd92740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103f2600 0x555e103e8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f254cd92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d1700 0x555e103d8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f254cd92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555e103ab7e0 con 0x555e103d1700 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f254cd92740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555e102c9f50 con 0x555e103d4d60 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f254cd92740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555e103a5530 con 0x555e103f2600 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f254cd92740 10 client.0 _ll_register_callbacks cb 0x555e10305000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2543fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d1700 0x555e103d8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2543fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d1700 0x555e103d8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33944/0 (socket says 172.21.15.26:33944) 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2543fff640 1 -- 172.21.15.26:0/3053559417 learned_addr learned my addr 172.21.15.26:0/3053559417 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f25437fe640 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 0x555e103e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2542ffd640 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103f2600 0x555e103e8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2543fff640 1 -- 172.21.15.26:0/3053559417 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103f2600 msgr2=0x555e103e8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2543fff640 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103f2600 0x555e103e8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2543fff640 1 -- 172.21.15.26:0/3053559417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 msgr2=0x555e103e86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2543fff640 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 0x555e103e86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2543fff640 1 -- 172.21.15.26:0/3053559417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e100223f0 con 0x555e103d1700 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.388+0000 7f2543fff640 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d1700 0x555e103d8300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f253c00fa00 tx=0x7f253c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cda7f038650d82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f25227fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f25227fc640 1 -- 172.21.15.26:0/3053559417 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c017020 con 0x555e103d1700 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f25227fc640 1 -- 172.21.15.26:0/3053559417 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f253c00d9a0 con 0x555e103d1700 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f254cd92740 1 -- 172.21.15.26:0/3053559417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f253807d520 con 0x555e103d1700 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f25227fc640 1 -- 172.21.15.26:0/3053559417 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f253c021050 con 0x555e103d1700 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f254cd92740 10 client.5457 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f254cd92740 1 -- 172.21.15.26:0/3053559417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x555e0ffe20f0 con 0x555e103d1700 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f25227fc640 1 -- 172.21.15.26:0/3053559417 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253807d520 con 0x555e103d1700 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f25227fc640 1 -- 172.21.15.26:0/3053559417 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7f253c024020 con 0x555e103d1700 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f254cd92740 10 client.5457 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f254cd92740 10 client.5457 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f254cd92740 20 client.5457 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f254cd92740 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x555e103e70f0 0x555e103e7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f254cd92740 4 client.5457 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f254cd92740 1 -- 172.21.15.26:0/3053559417 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e1031bbf0 con 0x555e103e70f0 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.389+0000 7f25437fe640 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x555e103e70f0 0x555e103e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f25227fc640 10 client.5457 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f25437fe640 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x555e103e70f0 0x555e103e7500 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51f2d8daf56508f server_cookie=f490b9f1c10062f4 in_seq=0 out_seq=0 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f25227fc640 1 -- 172.21.15.26:0/3053559417 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e1031bbf0 con 0x555e103e70f0 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f25227fc640 10 client.5457 handle_command_reply: tid=0 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 client.5457 shutdown 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 -- 172.21.15.26:0/3053559417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d1700 msgr2=0x555e103d8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d1700 0x555e103d8300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f253c00fa00 tx=0x7f253c00fec0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 -- 172.21.15.26:0/3053559417 shutdown_connections 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x555e103e70f0 0x555e103e7500 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d1700 0x555e103d8300 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f25227fc640 0 client.5457 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103f2600 0x555e103e8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 --2- 172.21.15.26:0/3053559417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 0x555e103e86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 -- 172.21.15.26:0/3053559417 >> 172.21.15.26:0/3053559417 conn(0x555e103aaa40 msgr2=0x555e103f3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 -- 172.21.15.26:0/3053559417 shutdown_connections 2024-04-24T23:22:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.390+0000 7f254cd92740 1 -- 172.21.15.26:0/3053559417 wait complete. 2024-04-24T23:22:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.391+0000 7f254cd92740 20 client.5457 trim_cache size 0 max 16384 2024-04-24T23:22:15.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.408+0000 7f254cd92740 1 Processor -- start 2024-04-24T23:22:15.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.408+0000 7f254cd92740 1 -- start start 2024-04-24T23:22:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f254cd92740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 0x555e103ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f254cd92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d4d60 0x555e103e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f254cd92740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103f2600 0x555e103e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f254cd92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2544200930 con 0x555e103d4d60 2024-04-24T23:22:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f254cd92740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f25441ffbf0 con 0x555e103f2600 2024-04-24T23:22:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f254cd92740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2544200200 con 0x555e103d1700 2024-04-24T23:22:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d4d60 0x555e103e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d4d60 0x555e103e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33956/0 (socket says 172.21.15.26:33956) 2024-04-24T23:22:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f2542ffd640 1 -- 172.21.15.26:0/1315875285 learned_addr learned my addr 172.21.15.26:0/1315875285 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f2543fff640 1 --2- 172.21.15.26:0/1315875285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103f2600 0x555e103e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f25437fe640 1 --2- 172.21.15.26:0/1315875285 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 0x555e103ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f2542ffd640 1 -- 172.21.15.26:0/1315875285 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 msgr2=0x555e103ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f2542ffd640 1 --2- 172.21.15.26:0/1315875285 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 0x555e103ab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f2542ffd640 1 -- 172.21.15.26:0/1315875285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103f2600 msgr2=0x555e103e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f2542ffd640 1 --2- 172.21.15.26:0/1315875285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103f2600 0x555e103e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.409+0000 7f2542ffd640 1 -- 172.21.15.26:0/1315875285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e103ffc20 con 0x555e103d4d60 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.410+0000 7f2542ffd640 1 --2- 172.21.15.26:0/1315875285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d4d60 0x555e103e7b60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2538079100 tx=0x7f253803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b75255b82230e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.410+0000 7f2528ff9640 1 -- 172.21.15.26:0/1315875285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538081d50 con 0x555e103d4d60 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.410+0000 7f2528ff9640 1 -- 172.21.15.26:0/1315875285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2538079170 con 0x555e103d4d60 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.410+0000 7f2528ff9640 1 -- 172.21.15.26:0/1315875285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253807d520 con 0x555e103d4d60 2024-04-24T23:22:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.410+0000 7f254cd92740 1 -- 172.21.15.26:0/1315875285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d4d60 msgr2=0x555e103e7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.410+0000 7f254cd92740 1 --2- 172.21.15.26:0/1315875285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d4d60 0x555e103e7b60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2538079100 tx=0x7f253803d000 comp rx=0 tx=0).stop 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.410+0000 7f254cd92740 1 -- 172.21.15.26:0/1315875285 shutdown_connections 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.410+0000 7f254cd92740 1 --2- 172.21.15.26:0/1315875285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103f2600 0x555e103e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.411+0000 7f254cd92740 1 --2- 172.21.15.26:0/1315875285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103d4d60 0x555e103e7b60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.411+0000 7f254cd92740 1 --2- 172.21.15.26:0/1315875285 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 0x555e103ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.411+0000 7f254cd92740 1 -- 172.21.15.26:0/1315875285 >> 172.21.15.26:0/1315875285 conn(0x555e103aaa40 msgr2=0x555e104095a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.411+0000 7f254cd92740 1 -- 172.21.15.26:0/1315875285 shutdown_connections 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.411+0000 7f254cd92740 1 -- 172.21.15.26:0/1315875285 wait complete. 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.411+0000 7f254cd92740 1 -- start start 2024-04-24T23:22:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.411+0000 7f254cd92740 1 Processor -- start 2024-04-24T23:22:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f254cd92740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 0x555e103f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f254cd92740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 0x555e1046ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f254cd92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e1046b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f254cd92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f25440b5730 con 0x555e103f2600 2024-04-24T23:22:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f254cd92740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555e103abe10 con 0x555e103d4d60 2024-04-24T23:22:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f254cd92740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555e1046ba70 con 0x555e103d1700 2024-04-24T23:22:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f254cd92740 10 client.0 _ll_register_callbacks cb 0x7f253c0019d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f2543fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e1046b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f2543fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e1046b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33958/0 (socket says 172.21.15.26:33958) 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f2543fff640 1 -- 172.21.15.26:0/4107877594 learned_addr learned my addr 172.21.15.26:0/4107877594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f2542ffd640 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 0x555e1046ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f25437fe640 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 0x555e103f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f2543fff640 1 -- 172.21.15.26:0/4107877594 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 msgr2=0x555e103f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f2543fff640 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 0x555e103f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f2543fff640 1 -- 172.21.15.26:0/4107877594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 msgr2=0x555e1046ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f2543fff640 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 0x555e1046ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.412+0000 7f2543fff640 1 -- 172.21.15.26:0/4107877594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e103aa230 con 0x555e103f2600 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f2543fff640 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e1046b2a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f253c00f9d0 tx=0x7f253c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2962216bad785952 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f25227fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f25227fc640 1 -- 172.21.15.26:0/4107877594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c029070 con 0x555e103f2600 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f25227fc640 1 -- 172.21.15.26:0/4107877594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f253c0079f0 con 0x555e103f2600 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f254cd92740 1 -- 172.21.15.26:0/4107877594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2538079170 con 0x555e103f2600 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f25227fc640 1 -- 172.21.15.26:0/4107877594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f253c034050 con 0x555e103f2600 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f254cd92740 10 client.5463 fetch_fsmap learned FSMap version 23 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f254cd92740 1 -- 172.21.15.26:0/4107877594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x555e103e8fb0 con 0x555e103f2600 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f25227fc640 1 -- 172.21.15.26:0/4107877594 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2538079170 con 0x555e103f2600 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f25227fc640 1 -- 172.21.15.26:0/4107877594 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) ==== 3546+0+0 (secure 0 0 0) 0x7f253c037020 con 0x555e103f2600 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f254cd92740 10 client.5463 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f254cd92740 10 client.5463 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-24T23:22:15.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f254cd92740 20 client.5463 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f254cd92740 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x555e103fc280 0x555e103cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f254cd92740 4 client.5463 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f254cd92740 1 -- 172.21.15.26:0/4107877594 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e10315f40 con 0x555e103fc280 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.413+0000 7f25437fe640 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x555e103fc280 0x555e103cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f25227fc640 10 client.5463 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f25437fe640 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x555e103fc280 0x555e103cec40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c7dfe0e9594e9b server_cookie=4457e96f3dde93c9 in_seq=0 out_seq=0 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f25227fc640 1 -- 172.21.15.26:0/4107877594 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+207 (crc 0 0 0) 0x555e10315f40 con 0x555e103fc280 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f25227fc640 10 client.5463 handle_command_reply: tid=0 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 client.5463 shutdown 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 -- 172.21.15.26:0/4107877594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 msgr2=0x555e1046b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e1046b2a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f253c00f9d0 tx=0x7f253c012000 comp rx=0 tx=0).stop 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 -- 172.21.15.26:0/4107877594 shutdown_connections 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x555e103fc280 0x555e103cec40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e103f2600 0x555e1046b2a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555e103d4d60 0x555e1046ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 --2- 172.21.15.26:0/4107877594 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555e103d1700 0x555e103f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f25227fc640 0 client.5463 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 -- 172.21.15.26:0/4107877594 >> 172.21.15.26:0/4107877594 conn(0x555e103aaa40 msgr2=0x555e104095a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 -- 172.21.15.26:0/4107877594 shutdown_connections 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 1 -- 172.21.15.26:0/4107877594 wait complete. 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.414+0000 7f254cd92740 20 client.5463 trim_cache size 0 max 16384 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:22:15.424 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:22:15.425 INFO:teuthology.orchestra.run.smithi026.stdout: "tag123": { 2024-04-24T23:22:15.425 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:22:15.425 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "tag123", 2024-04-24T23:22:15.425 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:22:15.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:22:15.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:22:15.425 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:22:15.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 -- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f253006be60 msgr2=0x7f253006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f253006be60 0x7f253006e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2534007fd0 tx=0x7f2534009040 comp rx=0 tx=0).stop 2024-04-24T23:22:15.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 -- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440d28e0 msgr2=0x7f254409e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:15.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440d28e0 0x7f254409e5a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2538035040 tx=0x7f25380046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 -- 172.21.15.26:0/3936070283 shutdown_connections 2024-04-24T23:22:15.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f253006be60 0x7f253006e320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f254415da30 0x7f25440ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440d28e0 0x7f254409e5a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 --2- 172.21.15.26:0/3936070283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25440ce640 0x7f2544157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 -- 172.21.15.26:0/3936070283 >> 172.21.15.26:0/3936070283 conn(0x7f25440d1dc0 msgr2=0x7f25440cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 -- 172.21.15.26:0/3936070283 shutdown_connections 2024-04-24T23:22:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:15.415+0000 7f2549fa2640 1 -- 172.21.15.26:0/3936070283 wait complete. 2024-04-24T23:22:15.428 INFO:tasks.cephfs.filesystem:scrub status for tag:tag123 - {'path': '/d1/d2/d3', 'tag': 'tag123', 'options': 'recursive,force'} 2024-04-24T23:22:29.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:22:29.154 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:22:45.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:45.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-24T23:22:45.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.544+0000 7fc25fde9640 1 Processor -- start 2024-04-24T23:22:45.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.544+0000 7fc25fde9640 1 -- start start 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25fde9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc25806d5a0 0x7fc2580b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25fde9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc258070f30 con 0x7fc25815e120 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25fde9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc25806dcf0 con 0x7fc2580b3d20 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25fde9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc25806dfd0 con 0x7fc25806d5a0 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25d384640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc25806d5a0 0x7fc2580b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25db85640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc25815e120 0x7fc25815e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38210/0 (socket says 172.21.15.26:38210) 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25db85640 1 -- 172.21.15.26:0/3954248515 learned_addr learned my addr 172.21.15.26:0/3954248515 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25d384640 1 -- 172.21.15.26:0/3954248515 >> v1:172.21.15.29:6789/0 conn(0x7fc2580b3d20 legacy=0x7fc258150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25d384640 1 -- 172.21.15.26:0/3954248515 >> v1:172.21.15.26:6789/0 conn(0x7fc25815e120 legacy=0x7fc25815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25d384640 1 -- 172.21.15.26:0/3954248515 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25806e2b0 con 0x7fc25806d5a0 2024-04-24T23:22:45.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.545+0000 7fc25cb83640 1 -- 172.21.15.26:0/3954248515 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022846769 0 0) 0x7fc258070f30 con 0x7fc25815e120 2024-04-24T23:22:45.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.546+0000 7fc25d384640 1 --2- 172.21.15.26:0/3954248515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 0x7fc2580b37e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc24800a940 tx=0x7fc24802b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f7455037f3a8ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.546+0000 7fc25cb83640 1 -- 172.21.15.26:0/3954248515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24800b040 con 0x7fc25806d5a0 2024-04-24T23:22:45.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.546+0000 7fc25cb83640 1 -- 172.21.15.26:0/3954248515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc248030710 con 0x7fc25806d5a0 2024-04-24T23:22:45.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.546+0000 7fc25cb83640 1 -- 172.21.15.26:0/3954248515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248030ae0 con 0x7fc25806d5a0 2024-04-24T23:22:45.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.547+0000 7fc25fde9640 1 -- 172.21.15.26:0/3954248515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 msgr2=0x7fc2580b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.547+0000 7fc25fde9640 1 --2- 172.21.15.26:0/3954248515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 0x7fc2580b37e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc24800a940 tx=0x7fc24802b5a0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.547+0000 7fc25fde9640 1 -- 172.21.15.26:0/3954248515 shutdown_connections 2024-04-24T23:22:45.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.547+0000 7fc25fde9640 1 --2- 172.21.15.26:0/3954248515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 0x7fc2580b37e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.547+0000 7fc25fde9640 1 -- 172.21.15.26:0/3954248515 >> 172.21.15.26:0/3954248515 conn(0x7fc25806b540 msgr2=0x7fc25806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.547+0000 7fc25fde9640 1 -- 172.21.15.26:0/3954248515 shutdown_connections 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.547+0000 7fc25fde9640 1 -- 172.21.15.26:0/3954248515 wait complete. 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.548+0000 7fc25fde9640 1 Processor -- start 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.548+0000 7fc25fde9640 1 -- start start 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25fde9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 0x7fc25814f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25fde9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2580b3d20 0x7fc25809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25fde9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc25815e120 0x7fc2580b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25fde9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc258072440 con 0x7fc25806d5a0 2024-04-24T23:22:45.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25fde9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc25806da10 con 0x7fc25815e120 2024-04-24T23:22:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25fde9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc2580725c0 con 0x7fc2580b3d20 2024-04-24T23:22:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 0x7fc25814f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 0x7fc25814f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60072/0 (socket says 172.21.15.26:60072) 2024-04-24T23:22:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25db85640 1 -- 172.21.15.26:0/4089126993 learned_addr learned my addr 172.21.15.26:0/4089126993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25e386640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc25815e120 0x7fc2580b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25d384640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2580b3d20 0x7fc25809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25db85640 1 -- 172.21.15.26:0/4089126993 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2580b3d20 msgr2=0x7fc25809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25db85640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2580b3d20 0x7fc25809aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25db85640 1 -- 172.21.15.26:0/4089126993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc25815e120 msgr2=0x7fc2580b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25db85640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc25815e120 0x7fc2580b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25db85640 1 -- 172.21.15.26:0/4089126993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2581f5c30 con 0x7fc25806d5a0 2024-04-24T23:22:45.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.549+0000 7fc25db85640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 0x7fc25814f930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc25400c430 tx=0x7fc25400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.550+0000 7fc24e7fc640 1 -- 172.21.15.26:0/4089126993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2540074e0 con 0x7fc25806d5a0 2024-04-24T23:22:45.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.550+0000 7fc24e7fc640 1 -- 172.21.15.26:0/4089126993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2540047e0 con 0x7fc25806d5a0 2024-04-24T23:22:45.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.550+0000 7fc25fde9640 1 -- 172.21.15.26:0/4089126993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2580afb10 con 0x7fc25806d5a0 2024-04-24T23:22:45.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.550+0000 7fc24e7fc640 1 -- 172.21.15.26:0/4089126993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254004bd0 con 0x7fc25806d5a0 2024-04-24T23:22:45.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.550+0000 7fc25fde9640 1 -- 172.21.15.26:0/4089126993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25814fe70 con 0x7fc25806d5a0 2024-04-24T23:22:45.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.551+0000 7fc24e7fc640 1 -- 172.21.15.26:0/4089126993 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc254011070 con 0x7fc25806d5a0 2024-04-24T23:22:45.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.552+0000 7fc24e7fc640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc23c06be30 0x7fc23c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.552+0000 7fc24e7fc640 1 -- 172.21.15.26:0/4089126993 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc25408aa40 con 0x7fc25806d5a0 2024-04-24T23:22:45.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.552+0000 7fc25d384640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc23c06be30 0x7fc23c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.553+0000 7fc25d384640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc23c06be30 0x7fc23c06e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc248035000 tx=0x7fc24800c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.553+0000 7fc262bcc740 1 Processor -- start 2024-04-24T23:22:45.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.553+0000 7fc262bcc740 1 -- start start 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f645b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f64585a40 0x560f645cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc262bcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560f644e96b0 con 0x560f645afd60 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc262bcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560f64491880 con 0x560f64585a40 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc262bcc740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560f644e5670 con 0x560f645cd600 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f645b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f645b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60086/0 (socket says 172.21.15.26:60086) 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25db85640 1 -- 172.21.15.26:0/2231127641 learned_addr learned my addr 172.21.15.26:0/2231127641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25e386640 1 --2- 172.21.15.26:0/2231127641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f64585a40 0x560f645cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25d384640 1 --2- 172.21.15.26:0/2231127641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25db85640 1 -- 172.21.15.26:0/2231127641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 msgr2=0x560f645cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25db85640 1 --2- 172.21.15.26:0/2231127641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25db85640 1 -- 172.21.15.26:0/2231127641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f64585a40 msgr2=0x560f645cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25db85640 1 --2- 172.21.15.26:0/2231127641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f64585a40 0x560f645cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25db85640 1 -- 172.21.15.26:0/2231127641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f642c77d0 con 0x560f645afd60 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc25db85640 1 --2- 172.21.15.26:0/2231127641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f645b0140 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc254088fa0 tx=0x7fc254059480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a610700c6a1e600 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.554+0000 7fc23b7fe640 1 -- 172.21.15.26:0/2231127641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254087070 con 0x560f645afd60 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc23b7fe640 1 -- 172.21.15.26:0/2231127641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc25405d130 con 0x560f645afd60 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc23b7fe640 1 -- 172.21.15.26:0/2231127641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254061960 con 0x560f645afd60 2024-04-24T23:22:45.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc262bcc740 1 -- 172.21.15.26:0/2231127641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 msgr2=0x560f645b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc262bcc740 1 --2- 172.21.15.26:0/2231127641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f645b0140 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc254088fa0 tx=0x7fc254059480 comp rx=0 tx=0).stop 2024-04-24T23:22:45.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc262bcc740 1 -- 172.21.15.26:0/2231127641 shutdown_connections 2024-04-24T23:22:45.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc262bcc740 1 --2- 172.21.15.26:0/2231127641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f64585a40 0x560f645cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc262bcc740 1 --2- 172.21.15.26:0/2231127641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc262bcc740 1 --2- 172.21.15.26:0/2231127641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f645b0140 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc262bcc740 1 -- 172.21.15.26:0/2231127641 >> 172.21.15.26:0/2231127641 conn(0x560f64582600 msgr2=0x560f645cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc262bcc740 1 -- 172.21.15.26:0/2231127641 shutdown_connections 2024-04-24T23:22:45.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.555+0000 7fc262bcc740 1 -- 172.21.15.26:0/2231127641 wait complete. 2024-04-24T23:22:45.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.556+0000 7fc262bcc740 1 -- start start 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.556+0000 7fc262bcc740 1 Processor -- start 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645afd60 0x560f645de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645cd600 0x560f645b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645c1c20 0x560f645ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc262bcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560f645867e0 con 0x560f645cd600 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc262bcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560f644a4f50 con 0x560f645afd60 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc262bcc740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560f64580530 con 0x560f645c1c20 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc262bcc740 10 client.0 _ll_register_callbacks cb 0x560f644e0000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25d384640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645cd600 0x560f645b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25d384640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645cd600 0x560f645b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60090/0 (socket says 172.21.15.26:60090) 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25d384640 1 -- 172.21.15.26:0/3799399929 learned_addr learned my addr 172.21.15.26:0/3799399929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25db85640 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645afd60 0x560f645de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25e386640 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645c1c20 0x560f645ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25d384640 1 -- 172.21.15.26:0/3799399929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645c1c20 msgr2=0x560f645ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25d384640 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645c1c20 0x560f645ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25d384640 1 -- 172.21.15.26:0/3799399929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645afd60 msgr2=0x560f645de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25d384640 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645afd60 0x560f645de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25d384640 1 -- 172.21.15.26:0/3799399929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f641fd3f0 con 0x560f645cd600 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc25d384640 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645cd600 0x560f645b3090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc24803e040 tx=0x7fc248004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1dfb2bc2359205 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.557+0000 7fc238ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc238ff9640 1 -- 172.21.15.26:0/3799399929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24804c030 con 0x560f645cd600 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc238ff9640 1 -- 172.21.15.26:0/3799399929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc24803c6e0 con 0x560f645cd600 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc262bcc740 1 -- 172.21.15.26:0/3799399929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc25405d130 con 0x560f645cd600 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc238ff9640 1 -- 172.21.15.26:0/3799399929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc248058050 con 0x560f645cd600 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc262bcc740 10 client.5478 fetch_fsmap learned FSMap version 24 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc262bcc740 1 -- 172.21.15.26:0/3799399929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x560f641bd0f0 con 0x560f645cd600 2024-04-24T23:22:45.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc238ff9640 1 -- 172.21.15.26:0/3799399929 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc25405d130 con 0x560f645cd600 2024-04-24T23:22:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc238ff9640 1 -- 172.21.15.26:0/3799399929 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 24) ==== 3550+0+0 (secure 0 0 0) 0x7fc24805b020 con 0x560f645cd600 2024-04-24T23:22:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc262bcc740 10 client.5478 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:22:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc262bcc740 10 client.5478 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-24T23:22:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc262bcc740 20 client.5478 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x560f645b0ef0 0x560f645b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc262bcc740 4 client.5478 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc262bcc740 1 -- 172.21.15.26:0/3799399929 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f644f6bf0 con 0x560f645b0ef0 2024-04-24T23:22:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.558+0000 7fc25db85640 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x560f645b0ef0 0x560f645b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc25db85640 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x560f645b0ef0 0x560f645b1380 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=159463562bba524b server_cookie=a38e9203bbf0b478 in_seq=0 out_seq=0 2024-04-24T23:22:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc238ff9640 10 client.5478 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc238ff9640 1 -- 172.21.15.26:0/3799399929 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f644f6bf0 con 0x560f645b0ef0 2024-04-24T23:22:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc238ff9640 10 client.5478 handle_command_reply: tid=0 2024-04-24T23:22:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc262bcc740 1 client.5478 shutdown 2024-04-24T23:22:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc262bcc740 1 -- 172.21.15.26:0/3799399929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645cd600 msgr2=0x560f645b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645cd600 0x560f645b3090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc24803e040 tx=0x7fc248004730 comp rx=0 tx=0).stop 2024-04-24T23:22:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc262bcc740 1 -- 172.21.15.26:0/3799399929 shutdown_connections 2024-04-24T23:22:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x560f645b0ef0 0x560f645b1380 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645c1c20 0x560f645ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645cd600 0x560f645b3090 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc238ff9640 0 client.5478 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:45.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3799399929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645afd60 0x560f645de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.559+0000 7fc262bcc740 1 -- 172.21.15.26:0/3799399929 >> 172.21.15.26:0/3799399929 conn(0x560f64585a40 msgr2=0x560f645ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.560+0000 7fc262bcc740 1 -- 172.21.15.26:0/3799399929 shutdown_connections 2024-04-24T23:22:45.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.560+0000 7fc262bcc740 1 -- 172.21.15.26:0/3799399929 wait complete. 2024-04-24T23:22:45.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.560+0000 7fc262bcc740 20 client.5478 trim_cache size 0 max 16384 2024-04-24T23:22:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc262bcc740 1 Processor -- start 2024-04-24T23:22:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc262bcc740 1 -- start start 2024-04-24T23:22:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645afd60 0x560f64586e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645c1c20 0x560f645af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc262bcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581fe3e0 con 0x560f645c1c20 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc262bcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581fd580 con 0x560f645afd60 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc262bcc740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc2581fdcb0 con 0x560f645cd600 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc25d384640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645c1c20 0x560f645af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc25d384640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645c1c20 0x560f645af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60100/0 (socket says 172.21.15.26:60100) 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.578+0000 7fc25d384640 1 -- 172.21.15.26:0/452247418 learned_addr learned my addr 172.21.15.26:0/452247418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.579+0000 7fc25d384640 1 -- 172.21.15.26:0/452247418 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 msgr2=0x560f645ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.579+0000 7fc25d384640 1 --2- 172.21.15.26:0/452247418 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.579+0000 7fc25d384640 1 -- 172.21.15.26:0/452247418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645afd60 msgr2=0x560f64586e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.579+0000 7fc25d384640 1 --2- 172.21.15.26:0/452247418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645afd60 0x560f64586e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.579+0000 7fc25d384640 1 -- 172.21.15.26:0/452247418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f64572090 con 0x560f645c1c20 2024-04-24T23:22:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.579+0000 7fc25d384640 1 --2- 172.21.15.26:0/452247418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645c1c20 0x560f645af2a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c010 tx=0x7fc248048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e45e91ba68c7cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.579+0000 7fc23affd640 1 -- 172.21.15.26:0/452247418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248061070 con 0x560f645c1c20 2024-04-24T23:22:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.579+0000 7fc23affd640 1 -- 172.21.15.26:0/452247418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc24803c6e0 con 0x560f645c1c20 2024-04-24T23:22:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.579+0000 7fc23affd640 1 -- 172.21.15.26:0/452247418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24804c030 con 0x560f645c1c20 2024-04-24T23:22:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 -- 172.21.15.26:0/452247418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645c1c20 msgr2=0x560f645af2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 --2- 172.21.15.26:0/452247418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645c1c20 0x560f645af2a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c010 tx=0x7fc248048000 comp rx=0 tx=0).stop 2024-04-24T23:22:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 -- 172.21.15.26:0/452247418 shutdown_connections 2024-04-24T23:22:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 --2- 172.21.15.26:0/452247418 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 --2- 172.21.15.26:0/452247418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645c1c20 0x560f645af2a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 --2- 172.21.15.26:0/452247418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645afd60 0x560f64586e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 -- 172.21.15.26:0/452247418 >> 172.21.15.26:0/452247418 conn(0x560f64585a40 msgr2=0x560f64585e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 -- 172.21.15.26:0/452247418 shutdown_connections 2024-04-24T23:22:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 -- 172.21.15.26:0/452247418 wait complete. 2024-04-24T23:22:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 -- start start 2024-04-24T23:22:45.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.580+0000 7fc262bcc740 1 Processor -- start 2024-04-24T23:22:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f6462b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645c1c20 0x560f645adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc262bcc740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc262bcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc25809d450 con 0x560f645afd60 2024-04-24T23:22:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc262bcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560f6457f260 con 0x560f645c1c20 2024-04-24T23:22:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc262bcc740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560f645c9490 con 0x560f645cd600 2024-04-24T23:22:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc262bcc740 10 client.0 _ll_register_callbacks cb 0x7fc24800a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc25db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f6462b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc25db85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f6462b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60110/0 (socket says 172.21.15.26:60110) 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc25db85640 1 -- 172.21.15.26:0/3127015241 learned_addr learned my addr 172.21.15.26:0/3127015241 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc25d384640 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645c1c20 0x560f645adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.581+0000 7fc25e386640 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc25db85640 1 -- 172.21.15.26:0/3127015241 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 msgr2=0x560f645ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc25db85640 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc25db85640 1 -- 172.21.15.26:0/3127015241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645c1c20 msgr2=0x560f645adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc25db85640 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645c1c20 0x560f645adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc25db85640 1 -- 172.21.15.26:0/3127015241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f645b2fc0 con 0x560f645afd60 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc25d384640 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645c1c20 0x560f645adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc25db85640 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f6462b620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc254059630 tx=0x7fc254088ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631ef3957d9997b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc238ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc238ff9640 1 -- 172.21.15.26:0/3127015241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254087070 con 0x560f645afd60 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc238ff9640 1 -- 172.21.15.26:0/3127015241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc25401ad50 con 0x560f645afd60 2024-04-24T23:22:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc262bcc740 1 -- 172.21.15.26:0/3127015241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc24803c6e0 con 0x560f645afd60 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc238ff9640 1 -- 172.21.15.26:0/3127015241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc25405d130 con 0x560f645afd60 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc262bcc740 10 client.5484 fetch_fsmap learned FSMap version 24 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc262bcc740 1 -- 172.21.15.26:0/3127015241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x560f64649a80 con 0x560f645afd60 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc238ff9640 1 -- 172.21.15.26:0/3127015241 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc24803c6e0 con 0x560f645afd60 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc238ff9640 1 -- 172.21.15.26:0/3127015241 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 24) ==== 3550+0+0 (secure 0 0 0) 0x7fc2540553f0 con 0x560f645afd60 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc262bcc740 10 client.5484 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc262bcc740 10 client.5484 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc262bcc740 20 client.5484 populate_metadata read hostname 'smithi026' 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x560f64586e10 0x560f64648c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc262bcc740 4 client.5484 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc262bcc740 1 -- 172.21.15.26:0/3127015241 --> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f644f0f40 con 0x560f64586e10 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.582+0000 7fc25d384640 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x560f64586e10 0x560f64648c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc238ff9640 10 client.5484 ms_handle_connect on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc25d384640 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x560f64586e10 0x560f64648c50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbadfa22074d25a7 server_cookie=9ce8f28e8f77b6d5 in_seq=0 out_seq=0 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc238ff9640 1 -- 172.21.15.26:0/3127015241 <== mds.0 v2:172.21.15.26:6834/315291888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f644f0f40 con 0x560f64586e10 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc238ff9640 10 client.5484 handle_command_reply: tid=0 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 client.5484 shutdown 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 -- 172.21.15.26:0/3127015241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 msgr2=0x560f6462b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f6462b620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc254059630 tx=0x7fc254088ed0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 -- 172.21.15.26:0/3127015241 shutdown_connections 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.26:6834/315291888,v1:172.21.15.26:6836/315291888] conn(0x560f64586e10 0x560f64648c50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f645cd600 0x560f645ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f645c1c20 0x560f645adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc238ff9640 0 client.5484 ms_handle_reset on v2:172.21.15.26:6834/315291888 2024-04-24T23:22:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 --2- 172.21.15.26:0/3127015241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f645afd60 0x560f6462b620 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 -- 172.21.15.26:0/3127015241 >> 172.21.15.26:0/3127015241 conn(0x560f64585a40 msgr2=0x560f645af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 -- 172.21.15.26:0/3127015241 shutdown_connections 2024-04-24T23:22:45.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 1 -- 172.21.15.26:0/3127015241 wait complete. 2024-04-24T23:22:45.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.583+0000 7fc262bcc740 20 client.5484 trim_cache size 0 max 16384 2024-04-24T23:22:45.594 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:22:45.594 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-04-24T23:22:45.595 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-04-24T23:22:45.595 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:22:45.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 -- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc23c06be30 msgr2=0x7fc23c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc23c06be30 0x7fc23c06e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc248035000 tx=0x7fc24800c040 comp rx=0 tx=0).stop 2024-04-24T23:22:45.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 -- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 msgr2=0x7fc25814f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 0x7fc25814f930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc25400c430 tx=0x7fc25400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 -- 172.21.15.26:0/4089126993 shutdown_connections 2024-04-24T23:22:45.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc23c06be30 0x7fc23c06e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc25815e120 0x7fc2580b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2580b3d20 0x7fc25809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 --2- 172.21.15.26:0/4089126993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc25806d5a0 0x7fc25814f930 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 -- 172.21.15.26:0/4089126993 >> 172.21.15.26:0/4089126993 conn(0x7fc25806b540 msgr2=0x7fc2580cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 -- 172.21.15.26:0/4089126993 shutdown_connections 2024-04-24T23:22:45.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.584+0000 7fc25fde9640 1 -- 172.21.15.26:0/4089126993 wait complete. 2024-04-24T23:22:45.598 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:22:45.599 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000002.00000000 scrub_tag 2024-04-24T23:22:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.640+0000 7fcbf207ef40 1 Processor -- start 2024-04-24T23:22:45.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.640+0000 7fcbf207ef40 1 -- start start 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf207ef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 0x556fbf9bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf207ef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 0x556fbf9223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf207ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9228f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf207ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556fbf93bc90 con 0x556fbf93b250 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf207ef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556fbf93b9b0 con 0x556fbf8f6e70 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf207ef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556fbf93bf70 con 0x556fbf8f6a90 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf0380640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf0380640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9228f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60124/0 (socket says 172.21.15.26:60124) 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf0380640 1 -- 172.21.15.26:0/726207670 learned_addr learned my addr 172.21.15.26:0/726207670 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbefb7f640 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 0x556fbf9bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbef37e640 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 0x556fbf9223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf0380640 1 -- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 msgr2=0x556fbf9bfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf0380640 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 0x556fbf9bfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf0380640 1 -- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 msgr2=0x556fbf9223b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf0380640 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 0x556fbf9223b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf0380640 1 -- 172.21.15.26:0/726207670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fbf820cb0 con 0x556fbf93b250 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbef37e640 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 0x556fbf9223b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbefb7f640 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 0x556fbf9bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.641+0000 7fcbf0380640 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9228f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800c460 tx=0x7fcbe800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32fdd97aa3e4f77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbee37c640 1 -- 172.21.15.26:0/726207670 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80115f0 con 0x556fbf93b250 2024-04-24T23:22:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbee37c640 1 -- 172.21.15.26:0/726207670 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe8017070 con 0x556fbf93b250 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbee37c640 1 -- 172.21.15.26:0/726207670 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80119f0 con 0x556fbf93b250 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbf207ef40 1 -- 172.21.15.26:0/726207670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 msgr2=0x556fbf9228f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbf207ef40 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9228f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800c460 tx=0x7fcbe800c920 comp rx=0 tx=0).stop 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbf207ef40 1 -- 172.21.15.26:0/726207670 shutdown_connections 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbf207ef40 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9228f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbf207ef40 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 0x556fbf9223b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbf207ef40 1 --2- 172.21.15.26:0/726207670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 0x556fbf9bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbf207ef40 1 -- 172.21.15.26:0/726207670 >> 172.21.15.26:0/726207670 conn(0x556fbf91eb90 msgr2=0x556fbf9ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbf207ef40 1 -- 172.21.15.26:0/726207670 shutdown_connections 2024-04-24T23:22:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.642+0000 7fcbf207ef40 1 -- 172.21.15.26:0/726207670 wait complete. 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.643+0000 7fcbf207ef40 1 Processor -- start 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.643+0000 7fcbf207ef40 1 -- start start 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.643+0000 7fcbf207ef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 0x556fbf939e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.643+0000 7fcbf207ef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 0x556fbf9ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.643+0000 7fcbf207ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.643+0000 7fcbf207ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556fbf9ca270 con 0x556fbf93b250 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.643+0000 7fcbf207ef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556fbf9c7780 con 0x556fbf8f6e70 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.643+0000 7fcbf207ef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556fbf9cfea0 con 0x556fbf8f6a90 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf0380640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf0380640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9b7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60134/0 (socket says 172.21.15.26:60134) 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf0380640 1 -- 172.21.15.26:0/1448879125 learned_addr learned my addr 172.21.15.26:0/1448879125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbefb7f640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 0x556fbf939e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbef37e640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 0x556fbf9ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf0380640 1 -- 172.21.15.26:0/1448879125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 msgr2=0x556fbf939e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf0380640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556fbf8f6a90 0x556fbf939e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf0380640 1 -- 172.21.15.26:0/1448879125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 msgr2=0x556fbf9ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf0380640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 0x556fbf9ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf0380640 1 -- 172.21.15.26:0/1448879125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fbf9bd060 con 0x556fbf93b250 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbef37e640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556fbf8f6e70 0x556fbf9ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf0380640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9b7e10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x556fbf9cdfa0 tx=0x7fcbe80033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbed37a640 1 -- 172.21.15.26:0/1448879125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8011e40 con 0x556fbf93b250 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbed37a640 1 -- 172.21.15.26:0/1448879125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe8012830 con 0x556fbf93b250 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556fbf935fe0 con 0x556fbf93b250 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbed37a640 1 -- 172.21.15.26:0/1448879125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8012c00 con 0x556fbf93b250 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.644+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fbf9bd300 con 0x556fbf93b250 2024-04-24T23:22:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.646+0000 7fcbed37a640 1 -- 172.21.15.26:0/1448879125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbe8024020 con 0x556fbf93b250 2024-04-24T23:22:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.646+0000 7fcbed37a640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcbdc06a810 0x7fcbdc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.646+0000 7fcbed37a640 1 -- 172.21.15.26:0/1448879125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fcbe808bc10 con 0x556fbf93b250 2024-04-24T23:22:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.646+0000 7fcbefb7f640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcbdc06a810 0x7fcbdc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.646+0000 7fcbf207ef40 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x556fbf9bdde0 0x556fbf9be1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.646+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 --> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] -- osd_op(unknown.0.0:1 4.12 4:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556fbf9be730 con 0x556fbf9bdde0 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbef37e640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x556fbf9bdde0 0x556fbf9be1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbefb7f640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcbdc06a810 0x7fcbdc06ccd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0057000 tx=0x7fcbe0052000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbef37e640 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x556fbf9bdde0 0x556fbf9be1f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbef37e640 1 -- 172.21.15.26:0/1448879125 <== osd.1 v2:172.21.15.26:6816/1500554534 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (10) out=10b] v0'0 uv8 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7fcbe4009870 con 0x556fbf9bdde0 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x556fbf9bdde0 msgr2=0x556fbf9be1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x556fbf9bdde0 0x556fbf9be1f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcbdc06a810 msgr2=0x7fcbdc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcbdc06a810 0x7fcbdc06ccd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0057000 tx=0x7fcbe0052000 comp rx=0 tx=0).stop 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 msgr2=0x556fbf9b7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 --2- 172.21.15.26:0/1448879125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556fbf93b250 0x556fbf9b7e10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x556fbf9cdfa0 tx=0x7fcbe80033a0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf0380640 1 -- 172.21.15.26:0/1448879125 reap_dead start 2024-04-24T23:22:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 shutdown_connections 2024-04-24T23:22:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 >> 172.21.15.26:0/1448879125 conn(0x556fbf91eb90 msgr2=0x556fbf9c0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 shutdown_connections 2024-04-24T23:22:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.647+0000 7fcbf207ef40 1 -- 172.21.15.26:0/1448879125 wait complete. 2024-04-24T23:22:45.655 INFO:teuthology.orchestra.run.smithi026.stdout:tag123 2024-04-24T23:22:45.656 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000003.00000000 scrub_tag 2024-04-24T23:22:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec5545ef40 1 Processor -- start 2024-04-24T23:22:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec5545ef40 1 -- start start 2024-04-24T23:22:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec5545ef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dc3e612190 0x55dc3e68a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec5545ef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dc3e6a04f0 0x55dc3e5fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec5545ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e5fb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec5545ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e6a1830 con 0x55dc3e6a08d0 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec5545ef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e6a2400 con 0x55dc3e612190 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec5545ef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55dc3e6a1db0 con 0x55dc3e6a04f0 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec53760640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e5fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec53760640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e5fb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60146/0 (socket says 172.21.15.26:60146) 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.738+0000 7fec53760640 1 -- 172.21.15.26:0/605838714 learned_addr learned my addr 172.21.15.26:0/605838714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec52f5f640 1 --2- 172.21.15.26:0/605838714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dc3e612190 0x55dc3e68a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec5275e640 1 --2- 172.21.15.26:0/605838714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dc3e6a04f0 0x55dc3e5fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec53760640 1 -- 172.21.15.26:0/605838714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dc3e6a04f0 msgr2=0x55dc3e5fab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec53760640 1 --2- 172.21.15.26:0/605838714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dc3e6a04f0 0x55dc3e5fab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec53760640 1 -- 172.21.15.26:0/605838714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dc3e612190 msgr2=0x55dc3e68a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec53760640 1 --2- 172.21.15.26:0/605838714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dc3e612190 0x55dc3e68a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec53760640 1 -- 172.21.15.26:0/605838714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc3e4f7cb0 con 0x55dc3e6a08d0 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec53760640 1 --2- 172.21.15.26:0/605838714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e5fb070 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00b610 tx=0x7fec4c00bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d9a4bf0638054f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec5175c640 1 -- 172.21.15.26:0/605838714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c014020 con 0x55dc3e6a08d0 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec5175c640 1 -- 172.21.15.26:0/605838714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4c010820 con 0x55dc3e6a08d0 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec5175c640 1 -- 172.21.15.26:0/605838714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c010c10 con 0x55dc3e6a08d0 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec5545ef40 1 -- 172.21.15.26:0/605838714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 msgr2=0x55dc3e5fb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.739+0000 7fec5545ef40 1 --2- 172.21.15.26:0/605838714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e5fb070 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00b610 tx=0x7fec4c00bad0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.740+0000 7fec5545ef40 1 -- 172.21.15.26:0/605838714 shutdown_connections 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.740+0000 7fec5545ef40 1 --2- 172.21.15.26:0/605838714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e5fb070 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.740+0000 7fec5545ef40 1 --2- 172.21.15.26:0/605838714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dc3e6a04f0 0x55dc3e5fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.740+0000 7fec5545ef40 1 --2- 172.21.15.26:0/605838714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dc3e612190 0x55dc3e68a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.740+0000 7fec5545ef40 1 -- 172.21.15.26:0/605838714 >> 172.21.15.26:0/605838714 conn(0x55dc3e5f5b90 msgr2=0x55dc3e5cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.740+0000 7fec5545ef40 1 -- 172.21.15.26:0/605838714 shutdown_connections 2024-04-24T23:22:45.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.740+0000 7fec5545ef40 1 -- 172.21.15.26:0/605838714 wait complete. 2024-04-24T23:22:45.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.740+0000 7fec5545ef40 1 Processor -- start 2024-04-24T23:22:45.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.740+0000 7fec5545ef40 1 -- start start 2024-04-24T23:22:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec5545ef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dc3e612190 0x55dc3e6874d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec5545ef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dc3e6a04f0 0x55dc3e60ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec5545ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e6104c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec5545ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e6129b0 con 0x55dc3e6a08d0 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec5545ef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e5cb570 con 0x55dc3e612190 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec5545ef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55dc3e5cb7c0 con 0x55dc3e6a04f0 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec53760640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e6104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec53760640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e6104c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60148/0 (socket says 172.21.15.26:60148) 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec53760640 1 -- 172.21.15.26:0/384365093 learned_addr learned my addr 172.21.15.26:0/384365093 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec5275e640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dc3e6a04f0 0x55dc3e60ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec52f5f640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dc3e612190 0x55dc3e6874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec53760640 1 -- 172.21.15.26:0/384365093 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dc3e6a04f0 msgr2=0x55dc3e60ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec53760640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dc3e6a04f0 0x55dc3e60ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec53760640 1 -- 172.21.15.26:0/384365093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dc3e612190 msgr2=0x55dc3e6874d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec53760640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dc3e612190 0x55dc3e6874d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec53760640 1 -- 172.21.15.26:0/384365093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc3e699b10 con 0x55dc3e6a08d0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.741+0000 7fec53760640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e6104c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00bb10 tx=0x7fec4c0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.742+0000 7fec3bfff640 1 -- 172.21.15.26:0/384365093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c028070 con 0x55dc3e6a08d0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.742+0000 7fec3bfff640 1 -- 172.21.15.26:0/384365093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4c01baf0 con 0x55dc3e6a08d0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.742+0000 7fec3bfff640 1 -- 172.21.15.26:0/384365093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c019450 con 0x55dc3e6a08d0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.742+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc3e699df0 con 0x55dc3e6a08d0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.742+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc3e69a3b0 con 0x55dc3e6a08d0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.743+0000 7fec3bfff640 1 -- 172.21.15.26:0/384365093 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec4c017020 con 0x55dc3e6a08d0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.743+0000 7fec3bfff640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4006a810 0x7fec4006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.744+0000 7fec3bfff640 1 -- 172.21.15.26:0/384365093 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fec4c08b3e0 con 0x55dc3e6a08d0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.744+0000 7fec52f5f640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4006a810 0x7fec4006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.744+0000 7fec5545ef40 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55dc3e69d1e0 0x55dc3e69d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.744+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 --> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] -- osd_op(unknown.0.0:1 4.12 4:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dc3e69db50 con 0x55dc3e69d1e0 2024-04-24T23:22:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.744+0000 7fec5275e640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55dc3e69d1e0 0x55dc3e69d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.744+0000 7fec52f5f640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4006a810 0x7fec4006ccd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec44057000 tx=0x7fec44052000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.744+0000 7fec5275e640 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55dc3e69d1e0 0x55dc3e69d610 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5275e640 1 -- 172.21.15.26:0/384365093 <== osd.1 v2:172.21.15.26:6816/1500554534 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr (10) out=10b] v0'0 uv9 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7fec48007790 con 0x55dc3e69d1e0 2024-04-24T23:22:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55dc3e69d1e0 msgr2=0x55dc3e69d610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55dc3e69d1e0 0x55dc3e69d610 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4006a810 msgr2=0x7fec4006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4006a810 0x7fec4006ccd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec44057000 tx=0x7fec44052000 comp rx=0 tx=0).stop 2024-04-24T23:22:45.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 msgr2=0x55dc3e6104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 --2- 172.21.15.26:0/384365093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc3e6a08d0 0x55dc3e6104c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00bb10 tx=0x7fec4c0033a0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec53760640 1 -- 172.21.15.26:0/384365093 reap_dead start 2024-04-24T23:22:45.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 shutdown_connections 2024-04-24T23:22:45.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 >> 172.21.15.26:0/384365093 conn(0x55dc3e5f5b90 msgr2=0x55dc3e5cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 shutdown_connections 2024-04-24T23:22:45.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.745+0000 7fec5545ef40 1 -- 172.21.15.26:0/384365093 wait complete. 2024-04-24T23:22:45.756 INFO:teuthology.orchestra.run.smithi026.stdout:tag123 2024-04-24T23:22:45.756 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000004.00000000 scrub_tag 2024-04-24T23:22:45.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.837+0000 7feb5d525f40 1 Processor -- start 2024-04-24T23:22:45.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.837+0000 7feb5d525f40 1 -- start start 2024-04-24T23:22:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb5d525f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e7503a90 0x55b1e75ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb5d525f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7503e70 0x55b1e752f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb5d525f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b1e7548250 0x55b1e752f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb5d525f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b1e7548c90 con 0x55b1e7503e70 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb5d525f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b1e75489b0 con 0x55b1e7503a90 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb5d525f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b1e7548f70 con 0x55b1e7548250 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb59f0c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7503e70 0x55b1e752f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb59f0c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7503e70 0x55b1e752f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60166/0 (socket says 172.21.15.26:60166) 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb59f0c640 1 -- 172.21.15.26:0/920149862 learned_addr learned my addr 172.21.15.26:0/920149862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb5a70d640 1 --2- 172.21.15.26:0/920149862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e7503a90 0x55b1e75ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.838+0000 7feb5af0e640 1 --2- 172.21.15.26:0/920149862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b1e7548250 0x55b1e752f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb59f0c640 1 -- 172.21.15.26:0/920149862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b1e7548250 msgr2=0x55b1e752f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb59f0c640 1 --2- 172.21.15.26:0/920149862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b1e7548250 0x55b1e752f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb59f0c640 1 -- 172.21.15.26:0/920149862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e7503a90 msgr2=0x55b1e75ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb59f0c640 1 --2- 172.21.15.26:0/920149862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e7503a90 0x55b1e75ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb59f0c640 1 -- 172.21.15.26:0/920149862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e742dcb0 con 0x55b1e7503e70 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb59f0c640 1 --2- 172.21.15.26:0/920149862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7503e70 0x55b1e752f3b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feb5000a770 tx=0x7feb5000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ed37ece233e189 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb58f0a640 1 -- 172.21.15.26:0/920149862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50019070 con 0x55b1e7503e70 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb58f0a640 1 -- 172.21.15.26:0/920149862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb50007600 con 0x55b1e7503e70 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb58f0a640 1 -- 172.21.15.26:0/920149862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb500079d0 con 0x55b1e7503e70 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb5d525f40 1 -- 172.21.15.26:0/920149862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7503e70 msgr2=0x55b1e752f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb5d525f40 1 --2- 172.21.15.26:0/920149862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7503e70 0x55b1e752f3b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feb5000a770 tx=0x7feb5000a860 comp rx=0 tx=0).stop 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb5d525f40 1 -- 172.21.15.26:0/920149862 shutdown_connections 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb5d525f40 1 --2- 172.21.15.26:0/920149862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b1e7548250 0x55b1e752f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb5d525f40 1 --2- 172.21.15.26:0/920149862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7503e70 0x55b1e752f3b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb5d525f40 1 --2- 172.21.15.26:0/920149862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e7503a90 0x55b1e75ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb5d525f40 1 -- 172.21.15.26:0/920149862 >> 172.21.15.26:0/920149862 conn(0x55b1e752bb90 msgr2=0x55b1e75d7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb5d525f40 1 -- 172.21.15.26:0/920149862 shutdown_connections 2024-04-24T23:22:45.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.839+0000 7feb5d525f40 1 -- 172.21.15.26:0/920149862 wait complete. 2024-04-24T23:22:45.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.840+0000 7feb5d525f40 1 Processor -- start 2024-04-24T23:22:45.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.840+0000 7feb5d525f40 1 -- start start 2024-04-24T23:22:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5d525f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e7503a90 0x55b1e7543110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5d525f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b1e7503e70 0x55b1e7544920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5d525f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7548250 0x55b1e75d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5d525f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b1e75d7270 con 0x55b1e7548250 2024-04-24T23:22:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5d525f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b1e75d4780 con 0x55b1e7503a90 2024-04-24T23:22:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5d525f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b1e75dcea0 con 0x55b1e7503e70 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5af0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7548250 0x55b1e75d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5af0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7548250 0x55b1e75d75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60180/0 (socket says 172.21.15.26:60180) 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5af0e640 1 -- 172.21.15.26:0/1288729847 learned_addr learned my addr 172.21.15.26:0/1288729847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5af0e640 1 -- 172.21.15.26:0/1288729847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b1e7503e70 msgr2=0x55b1e7544920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5af0e640 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b1e7503e70 0x55b1e7544920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5af0e640 1 -- 172.21.15.26:0/1288729847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e7503a90 msgr2=0x55b1e7543110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5af0e640 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b1e7503a90 0x55b1e7543110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5af0e640 1 -- 172.21.15.26:0/1288729847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e75ce000 con 0x55b1e7548250 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb5af0e640 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7548250 0x55b1e75d75e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7feb5400c210 tx=0x7feb5400c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb4b7fe640 1 -- 172.21.15.26:0/1288729847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54017020 con 0x55b1e7548250 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.841+0000 7feb4b7fe640 1 -- 172.21.15.26:0/1288729847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb54013670 con 0x55b1e7548250 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.842+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b1e75cdd20 con 0x55b1e7548250 2024-04-24T23:22:45.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.842+0000 7feb4b7fe640 1 -- 172.21.15.26:0/1288729847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54013a40 con 0x55b1e7548250 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.842+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e75ce570 con 0x55b1e7548250 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.843+0000 7feb4b7fe640 1 -- 172.21.15.26:0/1288729847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb5401a020 con 0x55b1e7548250 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.843+0000 7feb4b7fe640 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3c06a840 0x7feb3c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.843+0000 7feb4b7fe640 1 -- 172.21.15.26:0/1288729847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7feb54089ff0 con 0x55b1e7548250 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.843+0000 7feb5a70d640 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3c06a840 0x7feb3c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.844+0000 7feb5d525f40 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55b1e75d7b20 0x55b1e75dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.844+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- osd_op(unknown.0.0:1 4.5 4:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b1e752fb70 con 0x55b1e75d7b20 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.844+0000 7feb5a70d640 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3c06a840 0x7feb3c06cd00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb4c057000 tx=0x7feb4c051000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.844+0000 7feb59f0c640 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55b1e75d7b20 0x55b1e75dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.844+0000 7feb59f0c640 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55b1e75d7b20 0x55b1e75dbee0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb59f0c640 1 -- 172.21.15.26:0/1288729847 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7feb5000a250 con 0x55b1e75d7b20 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55b1e75d7b20 msgr2=0x55b1e75dbee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55b1e75d7b20 0x55b1e75dbee0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3c06a840 msgr2=0x7feb3c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3c06a840 0x7feb3c06cd00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb4c057000 tx=0x7feb4c051000 comp rx=0 tx=0).stop 2024-04-24T23:22:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7548250 msgr2=0x55b1e75d75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 --2- 172.21.15.26:0/1288729847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1e7548250 0x55b1e75d75e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7feb5400c210 tx=0x7feb5400c6d0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5af0e640 1 -- 172.21.15.26:0/1288729847 reap_dead start 2024-04-24T23:22:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 shutdown_connections 2024-04-24T23:22:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 >> 172.21.15.26:0/1288729847 conn(0x55b1e752bb90 msgr2=0x55b1e75cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 shutdown_connections 2024-04-24T23:22:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.845+0000 7feb5d525f40 1 -- 172.21.15.26:0/1288729847 wait complete. 2024-04-24T23:22:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:tag123 2024-04-24T23:22:45.851 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000005.00000000 scrub_tag 2024-04-24T23:22:45.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.932+0000 7f8fa1d21f40 1 Processor -- start 2024-04-24T23:22:45.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.932+0000 7f8fa1d21f40 1 -- start start 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8fa1d21f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369a90 0x564632432720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8fa1d21f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564632369e70 0x564632432c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8fa1d21f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5646323ae250 0x56463243e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8fa1d21f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5646323aec90 con 0x564632369a90 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8fa1d21f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5646323ae9b0 con 0x564632369e70 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8fa1d21f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5646323aef70 con 0x5646323ae250 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8f9f824640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369a90 0x564632432720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8f9f824640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369a90 0x564632432720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60190/0 (socket says 172.21.15.26:60190) 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8f9f824640 1 -- 172.21.15.26:0/3423626784 learned_addr learned my addr 172.21.15.26:0/3423626784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8f9f824640 1 -- 172.21.15.26:0/3423626784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5646323ae250 msgr2=0x56463243e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8f9f824640 1 --2- 172.21.15.26:0/3423626784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5646323ae250 0x56463243e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8f9f824640 1 -- 172.21.15.26:0/3423626784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564632369e70 msgr2=0x564632432c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8f9f824640 1 --2- 172.21.15.26:0/3423626784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564632369e70 0x564632432c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8f9f824640 1 -- 172.21.15.26:0/3423626784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564632293cb0 con 0x564632369a90 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.933+0000 7f8f9f824640 1 --2- 172.21.15.26:0/3423626784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369a90 0x564632432720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000a4f0 tx=0x7f8f90051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ada62718381ed19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8f9e822640 1 -- 172.21.15.26:0/3423626784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90063070 con 0x564632369a90 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8f9e822640 1 -- 172.21.15.26:0/3423626784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f90054970 con 0x564632369a90 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8f9e822640 1 -- 172.21.15.26:0/3423626784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90054d20 con 0x564632369a90 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/3423626784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369a90 msgr2=0x564632432720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8fa1d21f40 1 --2- 172.21.15.26:0/3423626784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369a90 0x564632432720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000a4f0 tx=0x7f8f90051000 comp rx=0 tx=0).stop 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/3423626784 shutdown_connections 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8fa1d21f40 1 --2- 172.21.15.26:0/3423626784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5646323ae250 0x56463243e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8fa1d21f40 1 --2- 172.21.15.26:0/3423626784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564632369e70 0x564632432c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8fa1d21f40 1 --2- 172.21.15.26:0/3423626784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369a90 0x564632432720 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/3423626784 >> 172.21.15.26:0/3423626784 conn(0x564632391b90 msgr2=0x56463243d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/3423626784 shutdown_connections 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.934+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/3423626784 wait complete. 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.935+0000 7f8fa1d21f40 1 Processor -- start 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.935+0000 7f8fa1d21f40 1 -- start start 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.935+0000 7f8fa1d21f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564632369a90 0x564632432bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.935+0000 7f8fa1d21f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369e70 0x5646323a8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.935+0000 7f8fa1d21f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5646323ae250 0x5646323a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.935+0000 7f8fa1d21f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56463243c790 con 0x564632369e70 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.935+0000 7f8fa1d21f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56463243adf0 con 0x564632369a90 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.935+0000 7f8fa1d21f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56463243c1d0 con 0x5646323ae250 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.935+0000 7f8f97fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369e70 0x5646323a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f97fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369e70 0x5646323a8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60200/0 (socket says 172.21.15.26:60200) 2024-04-24T23:22:45.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f97fff640 1 -- 172.21.15.26:0/381058397 learned_addr learned my addr 172.21.15.26:0/381058397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8fa0025640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5646323ae250 0x5646323a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f9f824640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564632369a90 0x564632432bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f97fff640 1 -- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5646323ae250 msgr2=0x5646323a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f97fff640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5646323ae250 0x5646323a8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f97fff640 1 -- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564632369a90 msgr2=0x564632432bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f97fff640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564632369a90 0x564632432bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f97fff640 1 -- 172.21.15.26:0/381058397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646323ad030 con 0x564632369e70 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f97fff640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369e70 0x5646323a8700 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800ade0 tx=0x7f8f8800aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f9d820640 1 -- 172.21.15.26:0/381058397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8801a070 con 0x564632369e70 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f9d820640 1 -- 172.21.15.26:0/381058397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f880089e0 con 0x564632369e70 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646323ac4b0 con 0x564632369e70 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8f9d820640 1 -- 172.21.15.26:0/381058397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88008db0 con 0x564632369e70 2024-04-24T23:22:45.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.936+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646323aca70 con 0x564632369e70 2024-04-24T23:22:45.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.938+0000 7f8f9d820640 1 -- 172.21.15.26:0/381058397 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f8801d070 con 0x564632369e70 2024-04-24T23:22:45.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.938+0000 7f8f9d820640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f8006a840 0x7f8f8006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.938+0000 7f8f9d820640 1 -- 172.21.15.26:0/381058397 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8f88088bb0 con 0x564632369e70 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.938+0000 7f8f9f824640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f8006a840 0x7f8f8006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.938+0000 7f8fa1d21f40 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x5646324334f0 0x5646324c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.938+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 --> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] -- osd_op(unknown.0.0:1 4.9 4:9258b67d:::10000000005.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564632433110 con 0x5646324334f0 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.939+0000 7f8fa0025640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x5646324334f0 0x5646324c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.939+0000 7f8f9f824640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f8006a840 0x7f8f8006cd00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f90067000 tx=0x7f8f90053000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.939+0000 7f8fa0025640 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x5646324334f0 0x5646324c9bd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.939+0000 7f8fa0025640 1 -- 172.21.15.26:0/381058397 <== osd.4 v2:172.21.15.29:6808/3208387101 1 ==== osd_op_reply(1 10000000005.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f8f9800c930 con 0x5646324334f0 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x5646324334f0 msgr2=0x5646324c9bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x5646324334f0 0x5646324c9bd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f8006a840 msgr2=0x7f8f8006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f8006a840 0x7f8f8006cd00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f90067000 tx=0x7f8f90053000 comp rx=0 tx=0).stop 2024-04-24T23:22:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369e70 msgr2=0x5646323a8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 --2- 172.21.15.26:0/381058397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564632369e70 0x5646323a8700 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800ade0 tx=0x7f8f8800aed0 comp rx=0 tx=0).stop 2024-04-24T23:22:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa0025640 1 -- 172.21.15.26:0/381058397 reap_dead start 2024-04-24T23:22:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 shutdown_connections 2024-04-24T23:22:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 >> 172.21.15.26:0/381058397 conn(0x564632391b90 msgr2=0x56463243d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 shutdown_connections 2024-04-24T23:22:45.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:45.940+0000 7f8fa1d21f40 1 -- 172.21.15.26:0/381058397 wait complete. 2024-04-24T23:22:45.946 INFO:teuthology.orchestra.run.smithi026.stdout:tag123 2024-04-24T23:22:45.946 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000006.00000000 scrub_tag 2024-04-24T23:22:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.027+0000 7f1c8ad5df40 1 Processor -- start 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.027+0000 7f1c8ad5df40 1 -- start start 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c8ad5df40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee0be86a90 0x55ee0bf4f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c8ad5df40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0be86e70 0x55ee0bf58c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c8ad5df40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee0becce80 0x55ee0beb2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c8ad5df40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee0be2b640 con 0x55ee0be86e70 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c8ad5df40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ee0be2f680 con 0x55ee0be86a90 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c8ad5df40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ee0becb9b0 con 0x55ee0becce80 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c83fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0be86e70 0x55ee0bf58c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c83fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0be86e70 0x55ee0bf58c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60226/0 (socket says 172.21.15.26:60226) 2024-04-24T23:22:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c83fff640 1 -- 172.21.15.26:0/1130695131 learned_addr learned my addr 172.21.15.26:0/1130695131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c83fff640 1 -- 172.21.15.26:0/1130695131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee0becce80 msgr2=0x55ee0beb2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c83fff640 1 --2- 172.21.15.26:0/1130695131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee0becce80 0x55ee0beb2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c83fff640 1 -- 172.21.15.26:0/1130695131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee0be86a90 msgr2=0x55ee0bf4f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c8885e640 1 --2- 172.21.15.26:0/1130695131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee0be86a90 0x55ee0bf4f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c83fff640 1 --2- 172.21.15.26:0/1130695131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee0be86a90 0x55ee0bf4f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c83fff640 1 -- 172.21.15.26:0/1130695131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0bf57820 con 0x55ee0be86e70 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.028+0000 7f1c83fff640 1 --2- 172.21.15.26:0/1130695131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0be86e70 0x55ee0bf58c40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1c74002e10 tx=0x7f1c7400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78921e5b9ff47f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c82ffd640 1 -- 172.21.15.26:0/1130695131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74013020 con 0x55ee0be86e70 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c82ffd640 1 -- 172.21.15.26:0/1130695131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c7400f400 con 0x55ee0be86e70 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c82ffd640 1 -- 172.21.15.26:0/1130695131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400f7f0 con 0x55ee0be86e70 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/1130695131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0be86e70 msgr2=0x55ee0bf58c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c8ad5df40 1 --2- 172.21.15.26:0/1130695131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0be86e70 0x55ee0bf58c40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1c74002e10 tx=0x7f1c7400b5b0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/1130695131 shutdown_connections 2024-04-24T23:22:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c8ad5df40 1 --2- 172.21.15.26:0/1130695131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee0becce80 0x55ee0beb2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c8ad5df40 1 --2- 172.21.15.26:0/1130695131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0be86e70 0x55ee0bf58c40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c8ad5df40 1 --2- 172.21.15.26:0/1130695131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee0be86a90 0x55ee0bf4f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/1130695131 >> 172.21.15.26:0/1130695131 conn(0x55ee0beaeb90 msgr2=0x55ee0beb1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/1130695131 shutdown_connections 2024-04-24T23:22:46.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.029+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/1130695131 wait complete. 2024-04-24T23:22:46.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.030+0000 7f1c8ad5df40 1 Processor -- start 2024-04-24T23:22:46.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.030+0000 7f1c8ad5df40 1 -- start start 2024-04-24T23:22:46.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.030+0000 7f1c8ad5df40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee0be86a90 0x55ee0bf47d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.030+0000 7f1c8ad5df40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee0be86e70 0x55ee0bf4e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.030+0000 7f1c8ad5df40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0becce80 0x55ee0bf4c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.030+0000 7f1c8ad5df40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee0beadfe0 con 0x55ee0becce80 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.030+0000 7f1c8ad5df40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ee0bf51f40 con 0x55ee0be86e70 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.030+0000 7f1c8ad5df40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ee0beb4650 con 0x55ee0be86a90 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8905f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0becce80 0x55ee0bf4c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8905f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0becce80 0x55ee0bf4c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60238/0 (socket says 172.21.15.26:60238) 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8905f640 1 -- 172.21.15.26:0/397576091 learned_addr learned my addr 172.21.15.26:0/397576091 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c83fff640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee0be86e70 0x55ee0bf4e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8885e640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee0be86a90 0x55ee0bf47d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8905f640 1 -- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee0be86a90 msgr2=0x55ee0bf47d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8905f640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee0be86a90 0x55ee0bf47d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8905f640 1 -- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee0be86e70 msgr2=0x55ee0bf4e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8905f640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee0be86e70 0x55ee0bf4e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8905f640 1 -- 172.21.15.26:0/397576091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0bf485f0 con 0x55ee0becce80 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8905f640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0becce80 0x55ee0bf4c360 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1c84025b00 tx=0x7f1c84050f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c81ffb640 1 -- 172.21.15.26:0/397576091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84067070 con 0x55ee0becce80 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c81ffb640 1 -- 172.21.15.26:0/397576091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c84058400 con 0x55ee0becce80 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ee0beb41a0 con 0x55ee0becce80 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c81ffb640 1 -- 172.21.15.26:0/397576091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84050610 con 0x55ee0becce80 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.031+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0beb23a0 con 0x55ee0becce80 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.033+0000 7f1c81ffb640 1 -- 172.21.15.26:0/397576091 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c8406a070 con 0x55ee0becce80 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.033+0000 7f1c81ffb640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1c6c06a810 0x7f1c6c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.033+0000 7f1c81ffb640 1 -- 172.21.15.26:0/397576091 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1c840cd3f0 con 0x55ee0becce80 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.033+0000 7f1c8885e640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1c6c06a810 0x7f1c6c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.033+0000 7f1c8ad5df40 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55ee0bf4ea40 0x55ee0bff5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.033+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- osd_op(unknown.0.0:1 4.5 4:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ee0bf4c8a0 con 0x55ee0bf4ea40 2024-04-24T23:22:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c83fff640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55ee0bf4ea40 0x55ee0bff5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8885e640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1c6c06a810 0x7f1c6c06ccd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c000c00 tx=0x7f1c7c0017f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c83fff640 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55ee0bf4ea40 0x55ee0bff5980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c83fff640 1 -- 172.21.15.26:0/397576091 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f1c7400f630 con 0x55ee0bf4ea40 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55ee0bf4ea40 msgr2=0x55ee0bff5980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8ad5df40 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55ee0bf4ea40 0x55ee0bff5980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1c6c06a810 msgr2=0x7f1c6c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8ad5df40 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1c6c06a810 0x7f1c6c06ccd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c000c00 tx=0x7f1c7c0017f0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0becce80 msgr2=0x55ee0bf4c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8ad5df40 1 --2- 172.21.15.26:0/397576091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee0becce80 0x55ee0bf4c360 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1c84025b00 tx=0x7f1c84050f80 comp rx=0 tx=0).stop 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8905f640 1 -- 172.21.15.26:0/397576091 reap_dead start 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 shutdown_connections 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.034+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 >> 172.21.15.26:0/397576091 conn(0x55ee0beaeb90 msgr2=0x55ee0beb3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.035+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 shutdown_connections 2024-04-24T23:22:46.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.035+0000 7f1c8ad5df40 1 -- 172.21.15.26:0/397576091 wait complete. 2024-04-24T23:22:46.041 INFO:teuthology.orchestra.run.smithi026.stdout:tag123 2024-04-24T23:22:46.042 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000007.00000000 scrub_tag 2024-04-24T23:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.124+0000 7f8b6f867f40 1 Processor -- start 2024-04-24T23:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.124+0000 7f8b6f867f40 1 -- start start 2024-04-24T23:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b6f867f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b5c003630 0x5581df2bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b6f867f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df20e720 0x5581df2b9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b6f867f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581df2aca30 0x5581df2b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b6f867f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5581df2269b0 con 0x5581df20e720 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b6f867f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5581df186640 con 0x5581df2aca30 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b6f867f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5581df226c90 con 0x7f8b5c003630 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df20e720 0x5581df2b9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df20e720 0x5581df2b9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60250/0 (socket says 172.21.15.26:60250) 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b67fff640 1 -- 172.21.15.26:0/1521159511 learned_addr learned my addr 172.21.15.26:0/1521159511 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b6d368640 1 --2- 172.21.15.26:0/1521159511 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b5c003630 0x5581df2bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b6db69640 1 --2- 172.21.15.26:0/1521159511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581df2aca30 0x5581df2b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b67fff640 1 -- 172.21.15.26:0/1521159511 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b5c003630 msgr2=0x5581df2bad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b67fff640 1 --2- 172.21.15.26:0/1521159511 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b5c003630 0x5581df2bad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b67fff640 1 -- 172.21.15.26:0/1521159511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581df2aca30 msgr2=0x5581df2b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b67fff640 1 --2- 172.21.15.26:0/1521159511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581df2aca30 0x5581df2b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b67fff640 1 -- 172.21.15.26:0/1521159511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581df2b1de0 con 0x5581df20e720 2024-04-24T23:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b67fff640 1 --2- 172.21.15.26:0/1521159511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df20e720 0x5581df2b9480 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009780 tx=0x7f8b58009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4c559073dd1eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.125+0000 7f8b677fe640 1 -- 172.21.15.26:0/1521159511 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800ebc0 con 0x5581df20e720 2024-04-24T23:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b677fe640 1 -- 172.21.15.26:0/1521159511 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b58013030 con 0x5581df20e720 2024-04-24T23:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b677fe640 1 -- 172.21.15.26:0/1521159511 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800f450 con 0x5581df20e720 2024-04-24T23:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b6f867f40 1 -- 172.21.15.26:0/1521159511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df20e720 msgr2=0x5581df2b9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b6f867f40 1 --2- 172.21.15.26:0/1521159511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df20e720 0x5581df2b9480 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009780 tx=0x7f8b58009c40 comp rx=0 tx=0).stop 2024-04-24T23:22:46.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b6f867f40 1 -- 172.21.15.26:0/1521159511 shutdown_connections 2024-04-24T23:22:46.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b6f867f40 1 --2- 172.21.15.26:0/1521159511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581df2aca30 0x5581df2b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b6f867f40 1 --2- 172.21.15.26:0/1521159511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df20e720 0x5581df2b9480 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b6f867f40 1 --2- 172.21.15.26:0/1521159511 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b5c003630 0x5581df2bad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b6f867f40 1 -- 172.21.15.26:0/1521159511 >> 172.21.15.26:0/1521159511 conn(0x5581df209b90 msgr2=0x5581df2b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:46.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b6f867f40 1 -- 172.21.15.26:0/1521159511 shutdown_connections 2024-04-24T23:22:46.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.126+0000 7f8b6f867f40 1 -- 172.21.15.26:0/1521159511 wait complete. 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b6f867f40 1 Processor -- start 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b6f867f40 1 -- start start 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b6f867f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5581df20e720 0x5581df2b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b6f867f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df2aca30 0x5581df2a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b6f867f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b5c003630 0x5581df350b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b6f867f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5581df2b6af0 con 0x5581df2aca30 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b6f867f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5581df2b20c0 con 0x7f8b5c003630 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b6f867f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5581df225d20 con 0x5581df20e720 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df2aca30 0x5581df2a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df2aca30 0x5581df2a9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60266/0 (socket says 172.21.15.26:60266) 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.127+0000 7f8b67fff640 1 -- 172.21.15.26:0/2330265344 learned_addr learned my addr 172.21.15.26:0/2330265344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b6d368640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5581df20e720 0x5581df2b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b6db69640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b5c003630 0x5581df350b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b67fff640 1 -- 172.21.15.26:0/2330265344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5581df20e720 msgr2=0x5581df2b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b67fff640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5581df20e720 0x5581df2b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b67fff640 1 -- 172.21.15.26:0/2330265344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b5c003630 msgr2=0x5581df350b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b67fff640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b5c003630 0x5581df350b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b67fff640 1 -- 172.21.15.26:0/2330265344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581df2ba220 con 0x5581df2aca30 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b6db69640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b5c003630 0x5581df350b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b67fff640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df2aca30 0x5581df2a9590 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8b58013000 tx=0x7f8b58001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b667fc640 1 -- 172.21.15.26:0/2330265344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800f860 con 0x5581df2aca30 2024-04-24T23:22:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b667fc640 1 -- 172.21.15.26:0/2330265344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b58009e90 con 0x5581df2aca30 2024-04-24T23:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b667fc640 1 -- 172.21.15.26:0/2330265344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800fc20 con 0x5581df2aca30 2024-04-24T23:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5581df2ad060 con 0x5581df2aca30 2024-04-24T23:22:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.128+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581df2ad620 con 0x5581df2aca30 2024-04-24T23:22:46.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.130+0000 7f8b667fc640 1 -- 172.21.15.26:0/2330265344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b58019430 con 0x5581df2aca30 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.130+0000 7f8b667fc640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b5006a810 0x7f8b5006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.130+0000 7f8b667fc640 1 -- 172.21.15.26:0/2330265344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8b5808bc90 con 0x5581df2aca30 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.130+0000 7f8b6d368640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b5006a810 0x7f8b5006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.130+0000 7f8b6f867f40 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x5581df2adb40 0x5581df2adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.130+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 --> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] -- osd_op(unknown.0.0:1 4.1d 4:bbf5850a:::10000000007.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5581df2a9ad0 con 0x5581df2adb40 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.130+0000 7f8b6db69640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x5581df2adb40 0x5581df2adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.130+0000 7f8b6d368640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b5006a810 0x7f8b5006ccd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5581df2b6aa0 tx=0x7f8b6000c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6db69640 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x5581df2adb40 0x5581df2adf50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6db69640 1 -- 172.21.15.26:0/2330265344 <== osd.3 v2:172.21.15.26:6808/2035350946 1 ==== osd_op_reply(1 10000000007.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f8b6804dab0 con 0x5581df2adb40 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x5581df2adb40 msgr2=0x5581df2adf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x5581df2adb40 0x5581df2adf50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b5006a810 msgr2=0x7f8b5006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b5006a810 0x7f8b5006ccd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5581df2b6aa0 tx=0x7f8b6000c040 comp rx=0 tx=0).stop 2024-04-24T23:22:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df2aca30 msgr2=0x5581df2a9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 --2- 172.21.15.26:0/2330265344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581df2aca30 0x5581df2a9590 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8b58013000 tx=0x7f8b58001ab0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6db69640 1 -- 172.21.15.26:0/2330265344 reap_dead start 2024-04-24T23:22:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 shutdown_connections 2024-04-24T23:22:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 >> 172.21.15.26:0/2330265344 conn(0x5581df209b90 msgr2=0x5581df2ba7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 shutdown_connections 2024-04-24T23:22:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.131+0000 7f8b6f867f40 1 -- 172.21.15.26:0/2330265344 wait complete. 2024-04-24T23:22:46.140 INFO:teuthology.orchestra.run.smithi026.stdout:tag123 2024-04-24T23:22:46.140 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:22:46.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:22:46.216 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:22:46.275 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:22:46.275 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:22:46.275 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:22:46.275 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:22:46.291 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:22:46.291 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:22:46.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:22:46.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.486+0000 7fba7ec2c640 1 Processor -- start 2024-04-24T23:22:46.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.486+0000 7fba7ec2c640 1 -- start start 2024-04-24T23:22:46.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba7ec2c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba780b63e0 0x7fba78142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba7ec2c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7806deb0 con 0x7fba78145890 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba7ec2c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7806e190 con 0x7fba78142310 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba7ec2c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fba7806dc60 con 0x7fba780b63e0 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba6ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba780b63e0 0x7fba78142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba7d1c9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fba78145890 0x7fba7806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38252/0 (socket says 172.21.15.26:38252) 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba7d1c9640 1 -- 172.21.15.26:0/1776005886 learned_addr learned my addr 172.21.15.26:0/1776005886 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba6ffff640 1 -- 172.21.15.26:0/1776005886 >> v1:172.21.15.29:6789/0 conn(0x7fba78142310 legacy=0x7fba780b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba6ffff640 1 -- 172.21.15.26:0/1776005886 >> v1:172.21.15.26:6789/0 conn(0x7fba78145890 legacy=0x7fba7806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba6ffff640 1 -- 172.21.15.26:0/1776005886 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7806e470 con 0x7fba780b63e0 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.487+0000 7fba6f7fe640 1 -- 172.21.15.26:0/1776005886 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68384024 0 0) 0x7fba7806deb0 con 0x7fba78145890 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.488+0000 7fba6ffff640 1 --2- 172.21.15.26:0/1776005886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 0x7fba78142d20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fba7806dde0 tx=0x7fba6802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2442b41c9f08bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.488+0000 7fba6f7fe640 1 -- 172.21.15.26:0/1776005886 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68037070 con 0x7fba780b63e0 2024-04-24T23:22:46.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.488+0000 7fba6f7fe640 1 -- 172.21.15.26:0/1776005886 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6802ae40 con 0x7fba780b63e0 2024-04-24T23:22:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.488+0000 7fba6f7fe640 1 -- 172.21.15.26:0/1776005886 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68032660 con 0x7fba780b63e0 2024-04-24T23:22:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.489+0000 7fba7ec2c640 1 -- 172.21.15.26:0/1776005886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 msgr2=0x7fba78142d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.489+0000 7fba7ec2c640 1 --2- 172.21.15.26:0/1776005886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 0x7fba78142d20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fba7806dde0 tx=0x7fba6802a130 comp rx=0 tx=0).stop 2024-04-24T23:22:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.489+0000 7fba7ec2c640 1 -- 172.21.15.26:0/1776005886 shutdown_connections 2024-04-24T23:22:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.489+0000 7fba7ec2c640 1 --2- 172.21.15.26:0/1776005886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 0x7fba78142d20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.489+0000 7fba7ec2c640 1 -- 172.21.15.26:0/1776005886 >> 172.21.15.26:0/1776005886 conn(0x7fba780729a0 msgr2=0x7fba7806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.489+0000 7fba7ec2c640 1 -- 172.21.15.26:0/1776005886 shutdown_connections 2024-04-24T23:22:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.489+0000 7fba7ec2c640 1 -- 172.21.15.26:0/1776005886 wait complete. 2024-04-24T23:22:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.490+0000 7fba7ec2c640 1 Processor -- start 2024-04-24T23:22:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.490+0000 7fba7ec2c640 1 -- start start 2024-04-24T23:22:46.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7ec2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 0x7fba7815b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7ec2c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba78142310 0x7fba780b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7ec2c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fba78145890 0x7fba780ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7ec2c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba781426f0 con 0x7fba780b63e0 2024-04-24T23:22:46.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7ec2c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fba7806eac0 con 0x7fba78142310 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7ec2c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fba78142870 con 0x7fba78145890 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7c9c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 0x7fba7815b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7c9c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 0x7fba7815b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60278/0 (socket says 172.21.15.26:60278) 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7c9c8640 1 -- 172.21.15.26:0/2480885285 learned_addr learned my addr 172.21.15.26:0/2480885285 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7c9c8640 1 -- 172.21.15.26:0/2480885285 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fba78145890 msgr2=0x7fba780ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7c9c8640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fba78145890 0x7fba780ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7c9c8640 1 -- 172.21.15.26:0/2480885285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba78142310 msgr2=0x7fba780b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba6ffff640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba78142310 0x7fba780b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7c9c8640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba78142310 0x7fba780b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.491+0000 7fba7c9c8640 1 -- 172.21.15.26:0/2480885285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba780b5400 con 0x7fba780b63e0 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.492+0000 7fba7c9c8640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 0x7fba7815b5c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c430 tx=0x7fba7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.492+0000 7fba6d7fa640 1 -- 172.21.15.26:0/2480885285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74011790 con 0x7fba780b63e0 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.492+0000 7fba6d7fa640 1 -- 172.21.15.26:0/2480885285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba74017070 con 0x7fba780b63e0 2024-04-24T23:22:46.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.492+0000 7fba6d7fa640 1 -- 172.21.15.26:0/2480885285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74011bb0 con 0x7fba780b63e0 2024-04-24T23:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.492+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba780b4e40 con 0x7fba780b63e0 2024-04-24T23:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.492+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7814d890 con 0x7fba780b63e0 2024-04-24T23:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.493+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba78067720 con 0x7fba780b63e0 2024-04-24T23:22:46.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.493+0000 7fba6d7fa640 1 -- 172.21.15.26:0/2480885285 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba7401c460 con 0x7fba780b63e0 2024-04-24T23:22:46.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.494+0000 7fba6d7fa640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba6406be30 0x7fba6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.494+0000 7fba6d7fa640 1 -- 172.21.15.26:0/2480885285 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fba7408aca0 con 0x7fba780b63e0 2024-04-24T23:22:46.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.494+0000 7fba6ffff640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba6406be30 0x7fba6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.494+0000 7fba6ffff640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba6406be30 0x7fba6406e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba68034000 tx=0x7fba6802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.498+0000 7fba6d7fa640 1 -- 172.21.15.26:0/2480885285 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba740596e0 con 0x7fba780b63e0 2024-04-24T23:22:46.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.682+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba780694c0 con 0x7fba780b63e0 2024-04-24T23:22:46.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.683+0000 7fba6d7fa640 1 -- 172.21.15.26:0/2480885285 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7fba74058fe0 con 0x7fba780b63e0 2024-04-24T23:22:46.685 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:46.686 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:49.992052+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:21:45.983358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:21:47.321239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba6406be30 msgr2=0x7fba6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba6406be30 0x7fba6406e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba68034000 tx=0x7fba6802e000 comp rx=0 tx=0).stop 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 msgr2=0x7fba7815b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 0x7fba7815b5c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c430 tx=0x7fba7400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 shutdown_connections 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba6406be30 0x7fba6406e2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fba78145890 0x7fba780ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba78142310 0x7fba780b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 --2- 172.21.15.26:0/2480885285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba780b63e0 0x7fba7815b5c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.685+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 >> 172.21.15.26:0/2480885285 conn(0x7fba780729a0 msgr2=0x7fba781432a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:46.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.686+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 shutdown_connections 2024-04-24T23:22:46.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.686+0000 7fba7ec2c640 1 -- 172.21.15.26:0/2480885285 wait complete. 2024-04-24T23:22:46.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:21:45.983358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:21:47.321239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:22:46.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:22:46.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.802+0000 7f5436a7b640 1 Processor -- start 2024-04-24T23:22:46.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.802+0000 7f5436a7b640 1 -- start start 2024-04-24T23:22:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f5436a7b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54300d0750 0x7f54300acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f5436a7b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f54300b4ce0 0x7f543015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f5436a7b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54300aeb10 con 0x7f543015be40 2024-04-24T23:22:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f5436a7b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f543009df20 con 0x7f54300b4ce0 2024-04-24T23:22:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f5436a7b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f54300aedf0 con 0x7f54300d0750 2024-04-24T23:22:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f5434817640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f543015be40 0x7f54300d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38268/0 (socket says 172.21.15.26:38268) 2024-04-24T23:22:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f5434817640 1 -- 172.21.15.26:0/4137581939 learned_addr learned my addr 172.21.15.26:0/4137581939 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 815158963 0 0) 0x7f54300aeb10 con 0x7f543015be40 2024-04-24T23:22:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f5435018640 1 --2- 172.21.15.26:0/4137581939 >> v2:172.21.15.29:3300/0 conn(0x7f54300b4ce0 0x7f543015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5418003520 con 0x7f543015be40 2024-04-24T23:22:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f5427fff640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54300d0750 0x7f54300acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.803+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3029186436 0 0) 0x7f5418003520 con 0x7f543015be40 2024-04-24T23:22:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54300d0750 msgr2=0x7f54300acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54300d0750 0x7f54300acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 >> v2:172.21.15.29:3300/0 conn(0x7f54300b4ce0 msgr2=0x7f543015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 --2- 172.21.15.26:0/4137581939 >> v2:172.21.15.29:3300/0 conn(0x7f54300b4ce0 0x7f543015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54300af0d0 con 0x7f543015be40 2024-04-24T23:22:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f542c001670 con 0x7f543015be40 2024-04-24T23:22:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 >> v1:172.21.15.26:6789/0 conn(0x7f543015be40 legacy=0x7f54300d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f5418003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5418004800 0x7f5418004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54300d0750 0x7f54180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f54300aedf0 con 0x7f54180039b0 2024-04-24T23:22:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f541800b9c0 con 0x7f5418004800 2024-04-24T23:22:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f541800bb40 con 0x7f54300d0750 2024-04-24T23:22:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5434817640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f5418003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5427fff640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5418004800 0x7f5418004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5435018640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54300d0750 0x7f54180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5434817640 1 -- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54300d0750 msgr2=0x7f54180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5434817640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54300d0750 0x7f54180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5434817640 1 -- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5418004800 msgr2=0x7f5418004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5434817640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5418004800 0x7f5418004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5434817640 1 -- 172.21.15.26:0/4137581939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5418003520 con 0x7f54180039b0 2024-04-24T23:22:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5435018640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54300d0750 0x7f54180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.804+0000 7f5427fff640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5418004800 0x7f5418004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.805+0000 7f5434817640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f5418003dd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f542c00c240 tx=0x7f5418005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2063828248c7df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.805+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c02e070 con 0x7f54180039b0 2024-04-24T23:22:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.805+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542c005af0 con 0x7f54180039b0 2024-04-24T23:22:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.805+0000 7f54277fe640 1 -- 172.21.15.26:0/4137581939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c007d30 con 0x7f54180039b0 2024-04-24T23:22:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.805+0000 7f5436a7b640 1 -- 172.21.15.26:0/4137581939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 msgr2=0x7f5418003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.805+0000 7f5436a7b640 1 --2- 172.21.15.26:0/4137581939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f5418003dd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f542c00c240 tx=0x7f5418005830 comp rx=0 tx=0).stop 2024-04-24T23:22:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.805+0000 7f5435018640 1 -- 172.21.15.26:0/4137581939 reap_dead start 2024-04-24T23:22:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.805+0000 7f5436a7b640 1 -- 172.21.15.26:0/4137581939 shutdown_connections 2024-04-24T23:22:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.805+0000 7f5436a7b640 1 -- 172.21.15.26:0/4137581939 >> 172.21.15.26:0/4137581939 conn(0x7f543006b540 msgr2=0x7f543006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.806+0000 7f5436a7b640 1 -- 172.21.15.26:0/4137581939 shutdown_connections 2024-04-24T23:22:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.806+0000 7f5436a7b640 1 -- 172.21.15.26:0/4137581939 wait complete. 2024-04-24T23:22:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.807+0000 7f5436a7b640 1 Processor -- start 2024-04-24T23:22:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.807+0000 7f5436a7b640 1 -- start start 2024-04-24T23:22:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5436a7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f543009c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5436a7b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f543014bd40 0x7f543009c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5436a7b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54300722b0 0x7f5430072760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5436a7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f543006cd10 con 0x7f54180039b0 2024-04-24T23:22:46.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5436a7b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f543006cb90 con 0x7f54300722b0 2024-04-24T23:22:46.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5436a7b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f54300acd30 con 0x7f543014bd40 2024-04-24T23:22:46.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5434817640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f543009c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5434817640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f543009c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60292/0 (socket says 172.21.15.26:60292) 2024-04-24T23:22:46.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5434817640 1 -- 172.21.15.26:0/3849075569 learned_addr learned my addr 172.21.15.26:0/3849075569 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:46.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5427fff640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f543014bd40 0x7f543009c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5435018640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54300722b0 0x7f5430072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5434817640 1 -- 172.21.15.26:0/3849075569 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f543014bd40 msgr2=0x7f543009c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5434817640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f543014bd40 0x7f543009c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5434817640 1 -- 172.21.15.26:0/3849075569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54300722b0 msgr2=0x7f5430072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:46.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5434817640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54300722b0 0x7f5430072760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:46.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5434817640 1 -- 172.21.15.26:0/3849075569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5430072ca0 con 0x7f54180039b0 2024-04-24T23:22:46.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5435018640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54300722b0 0x7f5430072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:46.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5434817640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f543009c300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f542c034040 tx=0x7f54300af2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f54257fa640 1 -- 172.21.15.26:0/3849075569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c042070 con 0x7f54180039b0 2024-04-24T23:22:46.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f54257fa640 1 -- 172.21.15.26:0/3849075569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542c007480 con 0x7f54180039b0 2024-04-24T23:22:46.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5430070fb0 con 0x7f54180039b0 2024-04-24T23:22:46.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f54257fa640 1 -- 172.21.15.26:0/3849075569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c001c70 con 0x7f54180039b0 2024-04-24T23:22:46.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.808+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5430071570 con 0x7f54180039b0 2024-04-24T23:22:46.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.809+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5430075610 con 0x7f54180039b0 2024-04-24T23:22:46.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.810+0000 7f54257fa640 1 -- 172.21.15.26:0/3849075569 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f542c045070 con 0x7f54180039b0 2024-04-24T23:22:46.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.810+0000 7f54257fa640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f541c06be30 0x7f541c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:46.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.810+0000 7f5427fff640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f541c06be30 0x7f541c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:46.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.811+0000 7f54257fa640 1 -- 172.21.15.26:0/3849075569 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f542c0ab840 con 0x7f54180039b0 2024-04-24T23:22:46.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.811+0000 7f5427fff640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f541c06be30 0x7f541c06e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f543006be50 tx=0x7f5420008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:46.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:46.815+0000 7f54257fa640 1 -- 172.21.15.26:0/3849075569 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f542c005ac0 con 0x7f54180039b0 2024-04-24T23:22:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.000+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f543006d120 con 0x7f54180039b0 2024-04-24T23:22:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.001+0000 7f54257fa640 1 -- 172.21.15.26:0/3849075569 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f542c079bb0 con 0x7f54180039b0 2024-04-24T23:22:47.002 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:22:47.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f541c06be30 msgr2=0x7f541c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f541c06be30 0x7f541c06e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f543006be50 tx=0x7f5420008040 comp rx=0 tx=0).stop 2024-04-24T23:22:47.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 msgr2=0x7f543009c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f543009c300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f542c034040 tx=0x7f54300af2a0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 shutdown_connections 2024-04-24T23:22:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f541c06be30 0x7f541c06e2f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54300722b0 0x7f5430072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f543014bd40 0x7f543009c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 --2- 172.21.15.26:0/3849075569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54180039b0 0x7f543009c300 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 >> 172.21.15.26:0/3849075569 conn(0x7f543006b540 msgr2=0x7f54301596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 shutdown_connections 2024-04-24T23:22:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.003+0000 7f5436a7b640 1 -- 172.21.15.26:0/3849075569 wait complete. 2024-04-24T23:22:47.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:22:47.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.117+0000 7fd413f09640 1 Processor -- start 2024-04-24T23:22:47.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.118+0000 7fd413f09640 1 -- start start 2024-04-24T23:22:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.118+0000 7fd413f09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd40c15e370 0x7fd40c0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.118+0000 7fd413f09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c0d4c10 0x7fd40c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.118+0000 7fd413f09640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd40c0d2120 con 0x7fd40c15e750 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.118+0000 7fd413f09640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd40c0d1510 con 0x7fd40c15e370 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.118+0000 7fd413f09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c0d1ed0 con 0x7fd40c0d4c10 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.119+0000 7fd411ca5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd40c15e370 0x7fd40c0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.119+0000 7fd411ca5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd40c15e370 0x7fd40c0d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60298/0 (socket says 172.21.15.26:60298) 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.119+0000 7fd411ca5640 1 -- 172.21.15.26:0/1927806390 learned_addr learned my addr 172.21.15.26:0/1927806390 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.119+0000 7fd4124a6640 1 --2- 172.21.15.26:0/1927806390 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c0d4c10 0x7fd40c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.119+0000 7fd411ca5640 1 -- 172.21.15.26:0/1927806390 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c0d4c10 msgr2=0x7fd40c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.119+0000 7fd411ca5640 1 --2- 172.21.15.26:0/1927806390 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c0d4c10 0x7fd40c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.119+0000 7fd411ca5640 1 -- 172.21.15.26:0/1927806390 >> v1:172.21.15.29:6789/0 conn(0x7fd40c15e750 legacy=0x7fd40c0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.119+0000 7fd411ca5640 1 -- 172.21.15.26:0/1927806390 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c0b3740 con 0x7fd40c15e370 2024-04-24T23:22:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd411ca5640 1 --2- 172.21.15.26:0/1927806390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c15e370 0x7fd40c0d2520 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd404009870 tx=0x7fd40402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc6a8d0f7d38780 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd410ca3640 1 -- 172.21.15.26:0/1927806390 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404037070 con 0x7fd40c15e370 2024-04-24T23:22:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd410ca3640 1 -- 172.21.15.26:0/1927806390 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd404032440 con 0x7fd40c15e370 2024-04-24T23:22:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd410ca3640 1 -- 172.21.15.26:0/1927806390 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404032810 con 0x7fd40c15e370 2024-04-24T23:22:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd413f09640 1 -- 172.21.15.26:0/1927806390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c15e370 msgr2=0x7fd40c0d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd413f09640 1 --2- 172.21.15.26:0/1927806390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c15e370 0x7fd40c0d2520 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd404009870 tx=0x7fd40402a270 comp rx=0 tx=0).stop 2024-04-24T23:22:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd413f09640 1 -- 172.21.15.26:0/1927806390 shutdown_connections 2024-04-24T23:22:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd413f09640 1 --2- 172.21.15.26:0/1927806390 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c0d4c10 0x7fd40c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd413f09640 1 --2- 172.21.15.26:0/1927806390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c15e370 0x7fd40c0d2520 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd413f09640 1 -- 172.21.15.26:0/1927806390 >> 172.21.15.26:0/1927806390 conn(0x7fd40c0d40f0 msgr2=0x7fd40c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.120+0000 7fd413f09640 1 -- 172.21.15.26:0/1927806390 shutdown_connections 2024-04-24T23:22:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.121+0000 7fd413f09640 1 -- 172.21.15.26:0/1927806390 wait complete. 2024-04-24T23:22:47.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd413f09640 1 Processor -- start 2024-04-24T23:22:47.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd413f09640 1 -- start start 2024-04-24T23:22:47.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd413f09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c0d4c10 0x7fd40c0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd413f09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c15e370 0x7fd40c0b8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd413f09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd40c0ca680 0x7fd40c1f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd413f09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c0d2a60 con 0x7fd40c0d4c10 2024-04-24T23:22:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd413f09640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c0b3d00 con 0x7fd40c0ca680 2024-04-24T23:22:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd413f09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c0d2be0 con 0x7fd40c15e370 2024-04-24T23:22:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd411ca5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c0d4c10 0x7fd40c0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd411ca5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c0d4c10 0x7fd40c0cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60308/0 (socket says 172.21.15.26:60308) 2024-04-24T23:22:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.122+0000 7fd411ca5640 1 -- 172.21.15.26:0/383922663 learned_addr learned my addr 172.21.15.26:0/383922663 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd4124a6640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd40c0ca680 0x7fd40c1f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd4114a4640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c15e370 0x7fd40c0b8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd411ca5640 1 -- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c15e370 msgr2=0x7fd40c0b8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd411ca5640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c15e370 0x7fd40c0b8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd411ca5640 1 -- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd40c0ca680 msgr2=0x7fd40c1f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd411ca5640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd40c0ca680 0x7fd40c1f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd411ca5640 1 -- 172.21.15.26:0/383922663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c0b91f0 con 0x7fd40c0d4c10 2024-04-24T23:22:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd4124a6640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd40c0ca680 0x7fd40c1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd4114a4640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c15e370 0x7fd40c0b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd411ca5640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c0d4c10 0x7fd40c0cdc10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd404000c00 tx=0x7fd4040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd40a7fc640 1 -- 172.21.15.26:0/383922663 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404037040 con 0x7fd40c0d4c10 2024-04-24T23:22:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd40a7fc640 1 -- 172.21.15.26:0/383922663 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd404042070 con 0x7fd40c0d4c10 2024-04-24T23:22:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd40a7fc640 1 -- 172.21.15.26:0/383922663 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404030880 con 0x7fd40c0d4c10 2024-04-24T23:22:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c0b9490 con 0x7fd40c0d4c10 2024-04-24T23:22:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.123+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c0b9a50 con 0x7fd40c0d4c10 2024-04-24T23:22:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.124+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c0b9c90 con 0x7fd40c0d4c10 2024-04-24T23:22:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.125+0000 7fd40a7fc640 1 -- 172.21.15.26:0/383922663 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd404030a20 con 0x7fd40c0d4c10 2024-04-24T23:22:47.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.125+0000 7fd40a7fc640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3f806be30 0x7fd3f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.125+0000 7fd40a7fc640 1 -- 172.21.15.26:0/383922663 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd4040ab6f0 con 0x7fd40c0d4c10 2024-04-24T23:22:47.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.125+0000 7fd4114a4640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3f806be30 0x7fd3f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.126+0000 7fd4114a4640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3f806be30 0x7fd3f806e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd40c161f50 tx=0x7fd400009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:47.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.129+0000 7fd40a7fc640 1 -- 172.21.15.26:0/383922663 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd40c0b9c90 con 0x7fd40c0d4c10 2024-04-24T23:22:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.313+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd40c075610 con 0x7fd40c0d4c10 2024-04-24T23:22:47.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.314+0000 7fd40a7fc640 1 -- 172.21.15.26:0/383922663 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7fd40407a0a0 con 0x7fd40c0d4c10 2024-04-24T23:22:47.315 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:22:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3f806be30 msgr2=0x7fd3f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3f806be30 0x7fd3f806e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd40c161f50 tx=0x7fd400009040 comp rx=0 tx=0).stop 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c0d4c10 msgr2=0x7fd40c0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c0d4c10 0x7fd40c0cdc10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd404000c00 tx=0x7fd4040046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 shutdown_connections 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd3f806be30 0x7fd3f806e2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd40c0ca680 0x7fd40c1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd40c15e370 0x7fd40c0b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 --2- 172.21.15.26:0/383922663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd40c0d4c10 0x7fd40c0cdc10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 >> 172.21.15.26:0/383922663 conn(0x7fd40c0d40f0 msgr2=0x7fd40c15c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 shutdown_connections 2024-04-24T23:22:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.316+0000 7fd413f09640 1 -- 172.21.15.26:0/383922663 wait complete. 2024-04-24T23:22:47.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:22:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.431+0000 7f68e1db7640 1 Processor -- start 2024-04-24T23:22:47.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.432+0000 7f68e1db7640 1 -- start start 2024-04-24T23:22:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.432+0000 7f68e1db7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.432+0000 7f68e1db7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f68dc06c540 0x7f68dc06c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.432+0000 7f68e1db7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68dc0cc580 con 0x7f68dc06cee0 2024-04-24T23:22:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.432+0000 7f68e1db7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f68dc0cc8b0 con 0x7f68dc06c540 2024-04-24T23:22:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.432+0000 7f68e1db7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc06d370 con 0x7f68dc1619e0 2024-04-24T23:22:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68daffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f68dc06c540 0x7f68dc06c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68db7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68db7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:60922/0 (socket says 172.21.15.26:60922) 2024-04-24T23:22:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68daffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f68dc06c540 0x7f68dc06c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:43578/0 (socket says 172.21.15.26:43578) 2024-04-24T23:22:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68db7fe640 1 -- 172.21.15.26:0/2824588243 learned_addr learned my addr 172.21.15.26:0/2824588243 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:47.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68dbfff640 1 --1- 172.21.15.26:0/2824588243 >> v1:172.21.15.29:6789/0 conn(0x7f68dc06cee0 0x7f68dc0ce710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:40198/0 (socket says 172.21.15.26:40198) 2024-04-24T23:22:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68da7fc640 1 -- 172.21.15.26:0/2824588243 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479477248 0 0) 0x7f68dc0cc580 con 0x7f68dc06cee0 2024-04-24T23:22:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68db7fe640 1 -- 172.21.15.26:0/2824588243 >> v2:172.21.15.29:3300/0 conn(0x7f68dc06c540 msgr2=0x7f68dc06c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68db7fe640 1 --2- 172.21.15.26:0/2824588243 >> v2:172.21.15.29:3300/0 conn(0x7f68dc06c540 0x7f68dc06c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68db7fe640 1 -- 172.21.15.26:0/2824588243 >> v1:172.21.15.29:6789/0 conn(0x7f68dc06cee0 legacy=0x7f68dc0ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.433+0000 7f68db7fe640 1 -- 172.21.15.26:0/2824588243 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc0d1170 con 0x7f68dc1619e0 2024-04-24T23:22:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.434+0000 7f68db7fe640 1 --2- 172.21.15.26:0/2824588243 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc0d00b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009870 tx=0x7f68cc02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61560849a91dadd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.434+0000 7f68da7fc640 1 -- 172.21.15.26:0/2824588243 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc037070 con 0x7f68dc1619e0 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.434+0000 7f68da7fc640 1 -- 172.21.15.26:0/2824588243 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68cc02ae50 con 0x7f68dc1619e0 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.434+0000 7f68da7fc640 1 -- 172.21.15.26:0/2824588243 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc032860 con 0x7f68dc1619e0 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.434+0000 7f68e1db7640 1 -- 172.21.15.26:0/2824588243 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 msgr2=0x7f68dc0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.434+0000 7f68e1db7640 1 --2- 172.21.15.26:0/2824588243 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc0d00b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009870 tx=0x7f68cc02a300 comp rx=0 tx=0).stop 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.435+0000 7f68e1db7640 1 -- 172.21.15.26:0/2824588243 shutdown_connections 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.435+0000 7f68e1db7640 1 --2- 172.21.15.26:0/2824588243 >> v2:172.21.15.29:3300/0 conn(0x7f68dc06c540 0x7f68dc06c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.435+0000 7f68e1db7640 1 --2- 172.21.15.26:0/2824588243 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc0d00b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.435+0000 7f68e1db7640 1 -- 172.21.15.26:0/2824588243 >> 172.21.15.26:0/2824588243 conn(0x7f68dc0d1e90 msgr2=0x7f68dc0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.435+0000 7f68e1db7640 1 -- 172.21.15.26:0/2824588243 shutdown_connections 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.435+0000 7f68e1db7640 1 -- 172.21.15.26:0/2824588243 wait complete. 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.436+0000 7f68e1db7640 1 Processor -- start 2024-04-24T23:22:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.436+0000 7f68e1db7640 1 -- start start 2024-04-24T23:22:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68e1db7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68dc06c540 0x7f68dc159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68e1db7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68dc06cee0 0x7f68dc0c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68e1db7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc1f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68e1db7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc0d2b20 con 0x7f68dc06c540 2024-04-24T23:22:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68e1db7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc0d1730 con 0x7f68dc06cee0 2024-04-24T23:22:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68e1db7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc0d2ca0 con 0x7f68dc1619e0 2024-04-24T23:22:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68dc06c540 0x7f68dc159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68dc06c540 0x7f68dc159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60314/0 (socket says 172.21.15.26:60314) 2024-04-24T23:22:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68db7fe640 1 -- 172.21.15.26:0/233343994 learned_addr learned my addr 172.21.15.26:0/233343994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68dbfff640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc1f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68daffd640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68dc06cee0 0x7f68dc0c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68db7fe640 1 -- 172.21.15.26:0/233343994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 msgr2=0x7f68dc1f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68db7fe640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc1f38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68db7fe640 1 -- 172.21.15.26:0/233343994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68dc06cee0 msgr2=0x7f68dc0c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68db7fe640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68dc06cee0 0x7f68dc0c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68db7fe640 1 -- 172.21.15.26:0/233343994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc0b5470 con 0x7f68dc06c540 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.437+0000 7f68db7fe640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68dc06c540 0x7f68dc159100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02a340 tx=0x7f68cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.438+0000 7f68c3fff640 1 -- 172.21.15.26:0/233343994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc041020 con 0x7f68dc06c540 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.438+0000 7f68c3fff640 1 -- 172.21.15.26:0/233343994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68cc03c870 con 0x7f68dc06c540 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.438+0000 7f68c3fff640 1 -- 172.21.15.26:0/233343994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc037040 con 0x7f68dc06c540 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.438+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc1f4770 con 0x7f68dc06c540 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.438+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc0b5710 con 0x7f68dc06c540 2024-04-24T23:22:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.439+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68dc075610 con 0x7f68dc06c540 2024-04-24T23:22:47.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.439+0000 7f68c3fff640 1 -- 172.21.15.26:0/233343994 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68cc044020 con 0x7f68dc06c540 2024-04-24T23:22:47.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.440+0000 7f68c3fff640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68c806be60 0x7f68c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.440+0000 7f68c3fff640 1 -- 172.21.15.26:0/233343994 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f68cc0ab770 con 0x7f68dc06c540 2024-04-24T23:22:47.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.440+0000 7f68daffd640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68c806be60 0x7f68c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.440+0000 7f68daffd640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68c806be60 0x7f68c806e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0d03f0 tx=0x7f68d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.444+0000 7f68c3fff640 1 -- 172.21.15.26:0/233343994 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f68cc04b030 con 0x7f68dc06c540 2024-04-24T23:22:47.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.627+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f68dc0c85e0 con 0x7f68dc06c540 2024-04-24T23:22:47.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.627+0000 7f68c3fff640 1 -- 172.21.15.26:0/233343994 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f68cc07a090 con 0x7f68dc06c540 2024-04-24T23:22:47.629 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68c806be60 msgr2=0x7f68c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68c806be60 0x7f68c806e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0d03f0 tx=0x7f68d0008040 comp rx=0 tx=0).stop 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68dc06c540 msgr2=0x7f68dc159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68dc06c540 0x7f68dc159100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02a340 tx=0x7f68cc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 shutdown_connections 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68c806be60 0x7f68c806e320 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f68dc1619e0 0x7f68dc1f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f68dc06cee0 0x7f68dc0c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 --2- 172.21.15.26:0/233343994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68dc06c540 0x7f68dc159100 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 >> 172.21.15.26:0/233343994 conn(0x7f68dc0d1e90 msgr2=0x7f68dc070760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 shutdown_connections 2024-04-24T23:22:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.629+0000 7f68e1db7640 1 -- 172.21.15.26:0/233343994 wait complete. 2024-04-24T23:22:47.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:22:47.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.742+0000 7f7ad7461640 1 Processor -- start 2024-04-24T23:22:47.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.743+0000 7f7ad7461640 1 -- start start 2024-04-24T23:22:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.743+0000 7f7ad7461640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad015a480 0x7f7ad00b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.743+0000 7f7ad7461640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ad015a860 0x7f7ad006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad7461640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7ad00b4e30 0x7f7ad006e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad7461640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad00b2e80 con 0x7f7ad015a860 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad7461640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad00b3120 con 0x7f7ad00b4e30 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad7461640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad00b33c0 con 0x7f7ad015a480 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad49fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ad015a860 0x7f7ad006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad49fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ad015a860 0x7f7ad006dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60328/0 (socket says 172.21.15.26:60328) 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad49fc640 1 -- 172.21.15.26:0/1293914057 learned_addr learned my addr 172.21.15.26:0/1293914057 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad59fe640 1 --2- 172.21.15.26:0/1293914057 >> v2:172.21.15.29:3300/0 conn(0x7f7ad00b4e30 0x7f7ad006e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad51fd640 1 --2- 172.21.15.26:0/1293914057 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad015a480 0x7f7ad00b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad49fc640 1 -- 172.21.15.26:0/1293914057 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad015a480 msgr2=0x7f7ad00b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad49fc640 1 --2- 172.21.15.26:0/1293914057 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad015a480 0x7f7ad00b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad49fc640 1 -- 172.21.15.26:0/1293914057 >> v2:172.21.15.29:3300/0 conn(0x7f7ad00b4e30 msgr2=0x7f7ad006e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad49fc640 1 --2- 172.21.15.26:0/1293914057 >> v2:172.21.15.29:3300/0 conn(0x7f7ad00b4e30 0x7f7ad006e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.744+0000 7f7ad49fc640 1 -- 172.21.15.26:0/1293914057 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad00b36a0 con 0x7f7ad015a860 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.745+0000 7f7ad49fc640 1 --2- 172.21.15.26:0/1293914057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 0x7f7ad006dbe0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00733a0 tx=0x7f7ac002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c75dce88a1e3a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.745+0000 7f7ac7fff640 1 -- 172.21.15.26:0/1293914057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0037070 con 0x7f7ad015a860 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.745+0000 7f7ac7fff640 1 -- 172.21.15.26:0/1293914057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac0032440 con 0x7f7ad015a860 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.745+0000 7f7ac7fff640 1 -- 172.21.15.26:0/1293914057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0032810 con 0x7f7ad015a860 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.745+0000 7f7ad7461640 1 -- 172.21.15.26:0/1293914057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 msgr2=0x7f7ad006dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.745+0000 7f7ad7461640 1 --2- 172.21.15.26:0/1293914057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 0x7f7ad006dbe0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00733a0 tx=0x7f7ac002a140 comp rx=0 tx=0).stop 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.746+0000 7f7ad7461640 1 -- 172.21.15.26:0/1293914057 shutdown_connections 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.746+0000 7f7ad7461640 1 --2- 172.21.15.26:0/1293914057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 0x7f7ad006dbe0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.746+0000 7f7ad7461640 1 --2- 172.21.15.26:0/1293914057 >> v2:172.21.15.29:3300/0 conn(0x7f7ad00b4e30 0x7f7ad006e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.746+0000 7f7ad7461640 1 --2- 172.21.15.26:0/1293914057 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad015a480 0x7f7ad00b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.746+0000 7f7ad7461640 1 -- 172.21.15.26:0/1293914057 >> 172.21.15.26:0/1293914057 conn(0x7f7ad006b300 msgr2=0x7f7ad006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.746+0000 7f7ad7461640 1 -- 172.21.15.26:0/1293914057 shutdown_connections 2024-04-24T23:22:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.746+0000 7f7ad7461640 1 -- 172.21.15.26:0/1293914057 wait complete. 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.747+0000 7f7ad7461640 1 Processor -- start 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.747+0000 7f7ad7461640 1 -- start start 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad7461640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad00b4e30 0x7f7ad013dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad7461640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad015a480 0x7f7ad015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad7461640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 0x7f7ad015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad7461640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad006d350 con 0x7f7ad015a860 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad7461640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad00b3c60 con 0x7f7ad015a480 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad7461640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad006d4d0 con 0x7f7ad00b4e30 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad59fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 0x7f7ad015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad59fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 0x7f7ad015ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60344/0 (socket says 172.21.15.26:60344) 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad59fe640 1 -- 172.21.15.26:0/3576315381 learned_addr learned my addr 172.21.15.26:0/3576315381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad49fc640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad015a480 0x7f7ad015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad51fd640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad00b4e30 0x7f7ad013dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad59fe640 1 -- 172.21.15.26:0/3576315381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad00b4e30 msgr2=0x7f7ad013dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad59fe640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad00b4e30 0x7f7ad013dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad59fe640 1 -- 172.21.15.26:0/3576315381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad015a480 msgr2=0x7f7ad015d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad59fe640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad015a480 0x7f7ad015d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad59fe640 1 -- 172.21.15.26:0/3576315381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad0145090 con 0x7f7ad015a860 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad49fc640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad015a480 0x7f7ad015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad59fe640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 0x7f7ad015ac40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8007e60 tx=0x7f7ac800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ac5ffb640 1 -- 172.21.15.26:0/3576315381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8014620 con 0x7f7ad015a860 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ac5ffb640 1 -- 172.21.15.26:0/3576315381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac8019070 con 0x7f7ad015a860 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad015b8a0 con 0x7f7ad015a860 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.748+0000 7f7ac5ffb640 1 -- 172.21.15.26:0/3576315381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8014a20 con 0x7f7ad015a860 2024-04-24T23:22:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.749+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad0145330 con 0x7f7ad015a860 2024-04-24T23:22:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.750+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad00694c0 con 0x7f7ad015a860 2024-04-24T23:22:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.750+0000 7f7ac5ffb640 1 -- 172.21.15.26:0/3576315381 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ac8014bc0 con 0x7f7ad015a860 2024-04-24T23:22:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.750+0000 7f7ac5ffb640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7ab406be30 0x7f7ab406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.751+0000 7f7ac5ffb640 1 -- 172.21.15.26:0/3576315381 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7ac808b7d0 con 0x7f7ad015a860 2024-04-24T23:22:47.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.751+0000 7f7ad51fd640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7ab406be30 0x7f7ab406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.751+0000 7f7ad51fd640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7ab406be30 0x7f7ab406e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7acc006fd0 tx=0x7f7acc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:47.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.755+0000 7f7ac5ffb640 1 -- 172.21.15.26:0/3576315381 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7ac805a0e0 con 0x7f7ad015a860 2024-04-24T23:22:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.949+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7ad00b44a0 con 0x7f7ad015a860 2024-04-24T23:22:47.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.949+0000 7f7ac5ffb640 1 -- 172.21.15.26:0/3576315381 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f7ac80599e0 con 0x7f7ad015a860 2024-04-24T23:22:47.951 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:22:47.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7ab406be30 msgr2=0x7f7ab406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7ab406be30 0x7f7ab406e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7acc006fd0 tx=0x7f7acc008040 comp rx=0 tx=0).stop 2024-04-24T23:22:47.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 msgr2=0x7f7ad015ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:47.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 0x7f7ad015ac40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8007e60 tx=0x7f7ac800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 shutdown_connections 2024-04-24T23:22:47.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7ab406be30 0x7f7ab406e2f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ad015a860 0x7f7ad015ac40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ad015a480 0x7f7ad015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 --2- 172.21.15.26:0/3576315381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ad00b4e30 0x7f7ad013dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:47.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 >> 172.21.15.26:0/3576315381 conn(0x7f7ad006b300 msgr2=0x7f7ad006d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:47.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 shutdown_connections 2024-04-24T23:22:47.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:47.951+0000 7f7ad7461640 1 -- 172.21.15.26:0/3576315381 wait complete. 2024-04-24T23:22:47.963 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:22:47.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:47.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:22:47.992 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:22:47.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:22:47.992 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:22:48.047 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:22:48.048 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:22:48.048 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:22:48.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:22:48.067 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:22:48.067 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:22:48.067 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:22:48.103 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:22:48.104 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:22:48.104 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:22:48.104 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:22:48.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:22:48.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:22:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.242+0000 7f207d170640 1 Processor -- start 2024-04-24T23:22:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.243+0000 7f207d170640 1 -- start start 2024-04-24T23:22:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.243+0000 7f207d170640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20780d4870 0x7f20780b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.243+0000 7f207d170640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20780b3130 0x7f20780b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.243+0000 7f207d170640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20780d32f0 con 0x7f207815e570 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.243+0000 7f207d170640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f20780d1ab0 con 0x7f20780d4870 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.243+0000 7f207d170640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f20780d2ae0 con 0x7f20780b3130 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.243+0000 7f207659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20780d4870 0x7f20780b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.244+0000 7f207659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20780d4870 0x7f20780b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60360/0 (socket says 172.21.15.26:60360) 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.244+0000 7f207659c640 1 -- 172.21.15.26:0/1128725586 learned_addr learned my addr 172.21.15.26:0/1128725586 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.244+0000 7f207759e640 1 --2- 172.21.15.26:0/1128725586 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20780b3130 0x7f20780b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.244+0000 7f207659c640 1 -- 172.21.15.26:0/1128725586 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20780b3130 msgr2=0x7f20780b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.244+0000 7f207659c640 1 --2- 172.21.15.26:0/1128725586 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20780b3130 0x7f20780b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.244+0000 7f207659c640 1 -- 172.21.15.26:0/1128725586 >> v1:172.21.15.29:6789/0 conn(0x7f207815e570 legacy=0x7f20780d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.244+0000 7f207659c640 1 -- 172.21.15.26:0/1128725586 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20780d0270 con 0x7f20780d4870 2024-04-24T23:22:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.244+0000 7f207759e640 1 --2- 172.21.15.26:0/1128725586 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20780b3130 0x7f20780b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.244+0000 7f207659c640 1 --2- 172.21.15.26:0/1128725586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 0x7f20780b4100 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f207808cae0 tx=0x7f206002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d0c3a70478c14a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f2075d9b640 1 -- 172.21.15.26:0/1128725586 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060037070 con 0x7f20780d4870 2024-04-24T23:22:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f2075d9b640 1 -- 172.21.15.26:0/1128725586 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2060032440 con 0x7f20780d4870 2024-04-24T23:22:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f2075d9b640 1 -- 172.21.15.26:0/1128725586 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060032810 con 0x7f20780d4870 2024-04-24T23:22:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f207d170640 1 -- 172.21.15.26:0/1128725586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 msgr2=0x7f20780b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f207d170640 1 --2- 172.21.15.26:0/1128725586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 0x7f20780b4100 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f207808cae0 tx=0x7f206002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f207d170640 1 -- 172.21.15.26:0/1128725586 shutdown_connections 2024-04-24T23:22:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f207d170640 1 --2- 172.21.15.26:0/1128725586 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20780b3130 0x7f20780b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f207d170640 1 --2- 172.21.15.26:0/1128725586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 0x7f20780b4100 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f207d170640 1 -- 172.21.15.26:0/1128725586 >> 172.21.15.26:0/1128725586 conn(0x7f207808dbe0 msgr2=0x7f207809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f207d170640 1 -- 172.21.15.26:0/1128725586 shutdown_connections 2024-04-24T23:22:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.245+0000 7f207d170640 1 -- 172.21.15.26:0/1128725586 wait complete. 2024-04-24T23:22:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.247+0000 7f207d170640 1 Processor -- start 2024-04-24T23:22:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.247+0000 7f207d170640 1 -- start start 2024-04-24T23:22:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.247+0000 7f207d170640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20780b3130 0x7f20780c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.247+0000 7f207d170640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 0x7f20780b8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.247+0000 7f207d170640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f207815e570 0x7f207815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.247+0000 7f207d170640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20780cfa70 con 0x7f20780d4870 2024-04-24T23:22:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.247+0000 7f207d170640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20780d3010 con 0x7f20780b3130 2024-04-24T23:22:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.247+0000 7f207d170640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f20780cfd50 con 0x7f207815e570 2024-04-24T23:22:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 0x7f20780b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 0x7f20780b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60372/0 (socket says 172.21.15.26:60372) 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207659c640 1 -- 172.21.15.26:0/976561082 learned_addr learned my addr 172.21.15.26:0/976561082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f2076d9d640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20780b3130 0x7f20780c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207759e640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f207815e570 0x7f207815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207659c640 1 -- 172.21.15.26:0/976561082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f207815e570 msgr2=0x7f207815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207659c640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f207815e570 0x7f207815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207659c640 1 -- 172.21.15.26:0/976561082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20780b3130 msgr2=0x7f20780c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207659c640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20780b3130 0x7f20780c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207659c640 1 -- 172.21.15.26:0/976561082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20780b8ef0 con 0x7f20780d4870 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207659c640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 0x7f20780b8470 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2060000c00 tx=0x7f20600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f206f7fe640 1 -- 172.21.15.26:0/976561082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060041020 con 0x7f20780d4870 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f206f7fe640 1 -- 172.21.15.26:0/976561082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2060030970 con 0x7f20780d4870 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f206f7fe640 1 -- 172.21.15.26:0/976561082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060037040 con 0x7f20780d4870 2024-04-24T23:22:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20780b91d0 con 0x7f20780d4870 2024-04-24T23:22:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.248+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20780b9a70 con 0x7f20780d4870 2024-04-24T23:22:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.250+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20780694c0 con 0x7f20780d4870 2024-04-24T23:22:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.250+0000 7f206f7fe640 1 -- 172.21.15.26:0/976561082 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2060044020 con 0x7f20780d4870 2024-04-24T23:22:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.250+0000 7f206f7fe640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f205806be30 0x7f205806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.250+0000 7f206f7fe640 1 -- 172.21.15.26:0/976561082 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f20600ab800 con 0x7f20780d4870 2024-04-24T23:22:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.250+0000 7f2076d9d640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f205806be30 0x7f205806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.251+0000 7f2076d9d640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f205806be30 0x7f205806e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2070006fd0 tx=0x7f2070009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:48.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.255+0000 7f206f7fe640 1 -- 172.21.15.26:0/976561082 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f206004b030 con 0x7f20780d4870 2024-04-24T23:22:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.481+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20780b9560 con 0x7f20780d4870 2024-04-24T23:22:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.481+0000 7f206f7fe640 1 -- 172.21.15.26:0/976561082 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5222 (secure 0 0 0) 0x7f20780d32f0 con 0x7f20780d4870 2024-04-24T23:22:48.483 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:48.483 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:21:48.324889+0000","modified":"2024-04-24T23:22:16.101089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5088,"mds_1":4867,"mds_2":4864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4864":{"gid":4864,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6834/2201738459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2201738459},{"type":"v1","addr":"172.21.15.29:6834","nonce":2201738459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4867":{"gid":4867,"name":"d","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6835/3972561683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3972561683},{"type":"v1","addr":"172.21.15.29:6835","nonce":3972561683}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/315291888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":315291888},{"type":"v1","addr":"172.21.15.26:6836","nonce":315291888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088,4867,4864]},"id":2}]} 2024-04-24T23:22:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 24 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f205806be30 msgr2=0x7f205806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f205806be30 0x7f205806e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2070006fd0 tx=0x7f2070009060 comp rx=0 tx=0).stop 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 msgr2=0x7f20780b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 0x7f20780b8470 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2060000c00 tx=0x7f20600046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 shutdown_connections 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f205806be30 0x7f205806e2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f207815e570 0x7f207815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20780d4870 0x7f20780b8470 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 --2- 172.21.15.26:0/976561082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20780b3130 0x7f20780c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 >> 172.21.15.26:0/976561082 conn(0x7f207808dbe0 msgr2=0x7f207806c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 shutdown_connections 2024-04-24T23:22:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.483+0000 7f207d170640 1 -- 172.21.15.26:0/976561082 wait complete. 2024-04-24T23:22:48.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:22:48.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.595+0000 7f3e7b170640 1 Processor -- start 2024-04-24T23:22:48.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.595+0000 7f3e7b170640 1 -- start start 2024-04-24T23:22:48.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.596+0000 7f3e7b170640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e74143c40 0x7f3e740cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.596+0000 7f3e7b170640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e740cd0b0 con 0x7f3e7415eb00 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.596+0000 7f3e7b170640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e74071240 con 0x7f3e740cc860 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.596+0000 7f3e7b170640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3e74071b70 con 0x7f3e74143c40 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.596+0000 7f3e7970d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3e7415eb00 0x7f3e740b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38274/0 (socket says 172.21.15.26:38274) 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.596+0000 7f3e7970d640 1 -- 172.21.15.26:0/2691825601 learned_addr learned my addr 172.21.15.26:0/2691825601 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.596+0000 7f3e78f0c640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e74143c40 0x7f3e740cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.596+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379849575 0 0) 0x7f3e740cd0b0 con 0x7f3e7415eb00 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.596+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e5c005730 con 0x7f3e7415eb00 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 59697195 0 0) 0x7f3e5c005730 con 0x7f3e7415eb00 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e74143c40 msgr2=0x7f3e740cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e74143c40 0x7f3e740cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 >> v1:172.21.15.29:6789/0 conn(0x7f3e740cc860 legacy=0x7f3e74072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e74071520 con 0x7f3e7415eb00 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f3e6c0019f0 con 0x7f3e7415eb00 2024-04-24T23:22:48.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 >> v1:172.21.15.26:6789/0 conn(0x7f3e7415eb00 legacy=0x7f3e740b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e74143c40 0x7f3e5c005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c006130 0x7f3e5c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5c006f50 0x7f3e5c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74071b70 con 0x7f3e5c006f50 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c00dc30 con 0x7f3e5c006130 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5c00de30 con 0x7f3e74143c40 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e7970d640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5c006f50 0x7f3e5c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e78f0c640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e74143c40 0x7f3e5c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e73fff640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c006130 0x7f3e5c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e7970d640 1 -- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e74143c40 msgr2=0x7f3e5c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e7970d640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e74143c40 0x7f3e5c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e7970d640 1 -- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c006130 msgr2=0x7f3e5c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:48.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e7970d640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e5c006130 0x7f3e5c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.597+0000 7f3e7970d640 1 -- 172.21.15.26:0/2691825601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c005730 con 0x7f3e5c006f50 2024-04-24T23:22:48.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.598+0000 7f3e7970d640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5c006f50 0x7f3e5c0073d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00a860 tx=0x7f3e5c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c059c3a58916665 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:48.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.598+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c00a480 con 0x7f3e5c006f50 2024-04-24T23:22:48.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.598+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6c02f070 con 0x7f3e5c006f50 2024-04-24T23:22:48.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.598+0000 7f3e737fe640 1 -- 172.21.15.26:0/2691825601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c02bcf0 con 0x7f3e5c006f50 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.599+0000 7f3e7b170640 1 -- 172.21.15.26:0/2691825601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5c006f50 msgr2=0x7f3e5c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.599+0000 7f3e7b170640 1 --2- 172.21.15.26:0/2691825601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5c006f50 0x7f3e5c0073d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00a860 tx=0x7f3e5c007a40 comp rx=0 tx=0).stop 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.599+0000 7f3e7970d640 1 -- 172.21.15.26:0/2691825601 reap_dead start 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.599+0000 7f3e7b170640 1 -- 172.21.15.26:0/2691825601 shutdown_connections 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.599+0000 7f3e7b170640 1 -- 172.21.15.26:0/2691825601 >> 172.21.15.26:0/2691825601 conn(0x7f3e7406b300 msgr2=0x7f3e7406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.599+0000 7f3e7b170640 1 -- 172.21.15.26:0/2691825601 shutdown_connections 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.599+0000 7f3e7b170640 1 -- 172.21.15.26:0/2691825601 wait complete. 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.600+0000 7f3e7b170640 1 Processor -- start 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.600+0000 7f3e7b170640 1 -- start start 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7b170640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e5c006f50 0x7f3e74159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7b170640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e7415c620 0x7f3e7415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7b170640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740b3420 0x7f3e740b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7b170640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3e740ccca0 con 0x7f3e740b3420 2024-04-24T23:22:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7b170640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74144020 con 0x7f3e7415c620 2024-04-24T23:22:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7b170640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3e740cce20 con 0x7f3e5c006f50 2024-04-24T23:22:48.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7970d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740b3420 0x7f3e740b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7970d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740b3420 0x7f3e740b38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60396/0 (socket says 172.21.15.26:60396) 2024-04-24T23:22:48.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7970d640 1 -- 172.21.15.26:0/948802061 learned_addr learned my addr 172.21.15.26:0/948802061 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e78f0c640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e5c006f50 0x7f3e74159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e73fff640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e7415c620 0x7f3e7415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7970d640 1 -- 172.21.15.26:0/948802061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e5c006f50 msgr2=0x7f3e74159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7970d640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e5c006f50 0x7f3e74159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7970d640 1 -- 172.21.15.26:0/948802061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e7415c620 msgr2=0x7f3e7415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7970d640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e7415c620 0x7f3e7415ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7970d640 1 -- 172.21.15.26:0/948802061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740b4190 con 0x7f3e740b3420 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7970d640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740b3420 0x7f3e740b38a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00a860 tx=0x7f3e74071ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e717fa640 1 -- 172.21.15.26:0/948802061 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c045070 con 0x7f3e740b3420 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e717fa640 1 -- 172.21.15.26:0/948802061 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6c034470 con 0x7f3e740b3420 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e740b4470 con 0x7f3e740b3420 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.601+0000 7f3e717fa640 1 -- 172.21.15.26:0/948802061 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c00a480 con 0x7f3e740b3420 2024-04-24T23:22:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.602+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e740b4a30 con 0x7f3e740b3420 2024-04-24T23:22:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.603+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7415cfc0 con 0x7f3e740b3420 2024-04-24T23:22:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.603+0000 7f3e717fa640 1 -- 172.21.15.26:0/948802061 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e6c048070 con 0x7f3e740b3420 2024-04-24T23:22:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.603+0000 7f3e717fa640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e6006be60 0x7f3e6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.603+0000 7f3e717fa640 1 -- 172.21.15.26:0/948802061 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3e6c0aaa30 con 0x7f3e740b3420 2024-04-24T23:22:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.603+0000 7f3e78f0c640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e6006be60 0x7f3e6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.604+0000 7f3e78f0c640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e6006be60 0x7f3e6006e320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3e640099e0 tx=0x7f3e64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.608+0000 7f3e717fa640 1 -- 172.21.15.26:0/948802061 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3e7415cfc0 con 0x7f3e740b3420 2024-04-24T23:22:48.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.791+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e740694c0 con 0x7f3e740b3420 2024-04-24T23:22:48.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.792+0000 7f3e717fa640 1 -- 172.21.15.26:0/948802061 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7f3e6c078d90 con 0x7f3e740b3420 2024-04-24T23:22:48.794 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:48.794 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:49.992052+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:21:45.983358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:21:47.321239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:22:48.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e6006be60 msgr2=0x7f3e6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e6006be60 0x7f3e6006e320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3e640099e0 tx=0x7f3e64008040 comp rx=0 tx=0).stop 2024-04-24T23:22:48.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740b3420 msgr2=0x7f3e740b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740b3420 0x7f3e740b38a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00a860 tx=0x7f3e74071ae0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 shutdown_connections 2024-04-24T23:22:48.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e6006be60 0x7f3e6006e320 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e740b3420 0x7f3e740b38a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e7415c620 0x7f3e7415ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 --2- 172.21.15.26:0/948802061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3e5c006f50 0x7f3e74159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 >> 172.21.15.26:0/948802061 conn(0x7f3e7406b300 msgr2=0x7f3e7415aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:48.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 shutdown_connections 2024-04-24T23:22:48.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.795+0000 7f3e7b170640 1 -- 172.21.15.26:0/948802061 wait complete. 2024-04-24T23:22:48.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:21:45.983358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:21:47.321239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:22:48.808 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:22:48.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.909+0000 7f6d7a38b640 1 Processor -- start 2024-04-24T23:22:48.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.909+0000 7f6d7a38b640 1 -- start start 2024-04-24T23:22:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d7a38b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d740ce640 0x7f6d740d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d7a38b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d740d28e0 0x7f6d740cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d7a38b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6d7406c4d0 0x7f6d7406c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d7a38b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6d740d1110 con 0x7f6d740d28e0 2024-04-24T23:22:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d7a38b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6d740cc410 con 0x7f6d7406c4d0 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d7a38b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6d7406d750 con 0x7f6d740ce640 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d737fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d740d28e0 0x7f6d740cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d737fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d740d28e0 0x7f6d740cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60402/0 (socket says 172.21.15.26:60402) 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d737fe640 1 -- 172.21.15.26:0/141073528 learned_addr learned my addr 172.21.15.26:0/141073528 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d73fff640 1 --2- 172.21.15.26:0/141073528 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d740ce640 0x7f6d740d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.910+0000 7f6d78928640 1 --2- 172.21.15.26:0/141073528 >> v2:172.21.15.29:3300/0 conn(0x7f6d7406c4d0 0x7f6d7406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.911+0000 7f6d737fe640 1 -- 172.21.15.26:0/141073528 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d740ce640 msgr2=0x7f6d740d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.911+0000 7f6d737fe640 1 --2- 172.21.15.26:0/141073528 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d740ce640 0x7f6d740d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.911+0000 7f6d737fe640 1 -- 172.21.15.26:0/141073528 >> v2:172.21.15.29:3300/0 conn(0x7f6d7406c4d0 msgr2=0x7f6d7406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.911+0000 7f6d737fe640 1 --2- 172.21.15.26:0/141073528 >> v2:172.21.15.29:3300/0 conn(0x7f6d7406c4d0 0x7f6d7406c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.911+0000 7f6d737fe640 1 -- 172.21.15.26:0/141073528 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d740b1720 con 0x7f6d740d28e0 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.911+0000 7f6d737fe640 1 --2- 172.21.15.26:0/141073528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d740d28e0 0x7f6d740cea20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6d740c58c0 tx=0x7f6d5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146576a2d10a0ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d72ffd640 1 -- 172.21.15.26:0/141073528 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c037070 con 0x7f6d740d28e0 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d72ffd640 1 -- 172.21.15.26:0/141073528 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5c032440 con 0x7f6d740d28e0 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d72ffd640 1 -- 172.21.15.26:0/141073528 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c032810 con 0x7f6d740d28e0 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d7a38b640 1 -- 172.21.15.26:0/141073528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d740d28e0 msgr2=0x7f6d740cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/141073528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d740d28e0 0x7f6d740cea20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6d740c58c0 tx=0x7f6d5c02a270 comp rx=0 tx=0).stop 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d7a38b640 1 -- 172.21.15.26:0/141073528 shutdown_connections 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/141073528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d740d28e0 0x7f6d740cea20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/141073528 >> v2:172.21.15.29:3300/0 conn(0x7f6d7406c4d0 0x7f6d7406c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/141073528 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d740ce640 0x7f6d740d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d7a38b640 1 -- 172.21.15.26:0/141073528 >> 172.21.15.26:0/141073528 conn(0x7f6d740d1dc0 msgr2=0x7f6d740d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:48.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d7a38b640 1 -- 172.21.15.26:0/141073528 shutdown_connections 2024-04-24T23:22:48.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.912+0000 7f6d7a38b640 1 -- 172.21.15.26:0/141073528 wait complete. 2024-04-24T23:22:48.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.914+0000 7f6d7a38b640 1 Processor -- start 2024-04-24T23:22:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.914+0000 7f6d7a38b640 1 -- start start 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d7a38b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d7406c4d0 0x7f6d741e2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d7a38b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d740ce640 0x7f6d740b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d7a38b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d740d28e0 0x7f6d741e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d7a38b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d740cc660 con 0x7f6d7406c4d0 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d7a38b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6d740d1360 con 0x7f6d740ce640 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d7a38b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6d740cc940 con 0x7f6d740d28e0 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d73fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d7406c4d0 0x7f6d741e2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d73fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d7406c4d0 0x7f6d741e2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60418/0 (socket says 172.21.15.26:60418) 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d73fff640 1 -- 172.21.15.26:0/1620836603 learned_addr learned my addr 172.21.15.26:0/1620836603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d73fff640 1 -- 172.21.15.26:0/1620836603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d740d28e0 msgr2=0x7f6d741e41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d73fff640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d740d28e0 0x7f6d741e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d73fff640 1 -- 172.21.15.26:0/1620836603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d740ce640 msgr2=0x7f6d740b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d737fe640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d740ce640 0x7f6d740b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d73fff640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d740ce640 0x7f6d740b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d73fff640 1 -- 172.21.15.26:0/1620836603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d740b6ae0 con 0x7f6d7406c4d0 2024-04-24T23:22:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d73fff640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d7406c4d0 0x7f6d741e2970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e930 tx=0x7f6d6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d70ff9640 1 -- 172.21.15.26:0/1620836603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64017070 con 0x7f6d7406c4d0 2024-04-24T23:22:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d70ff9640 1 -- 172.21.15.26:0/1620836603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d6400c940 con 0x7f6d7406c4d0 2024-04-24T23:22:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.915+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d740b6dc0 con 0x7f6d7406c4d0 2024-04-24T23:22:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.916+0000 7f6d70ff9640 1 -- 172.21.15.26:0/1620836603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400cd30 con 0x7f6d7406c4d0 2024-04-24T23:22:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.916+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d740b7380 con 0x7f6d7406c4d0 2024-04-24T23:22:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.917+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d74075610 con 0x7f6d7406c4d0 2024-04-24T23:22:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.917+0000 7f6d70ff9640 1 -- 172.21.15.26:0/1620836603 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d6401a070 con 0x7f6d7406c4d0 2024-04-24T23:22:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.917+0000 7f6d70ff9640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5406be30 0x7f6d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.918+0000 7f6d70ff9640 1 -- 172.21.15.26:0/1620836603 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6d6408b1b0 con 0x7f6d7406c4d0 2024-04-24T23:22:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.918+0000 7f6d737fe640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5406be30 0x7f6d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.918+0000 7f6d737fe640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5406be30 0x7f6d5406e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c034000 tx=0x7f6d5c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:48.922+0000 7f6d70ff9640 1 -- 172.21.15.26:0/1620836603 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d64059b60 con 0x7f6d7406c4d0 2024-04-24T23:22:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.104+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d740b1500 con 0x7f6d7406c4d0 2024-04-24T23:22:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.105+0000 7f6d70ff9640 1 -- 172.21.15.26:0/1620836603 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f6d64059460 con 0x7f6d7406c4d0 2024-04-24T23:22:49.107 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:22:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5406be30 msgr2=0x7f6d5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5406be30 0x7f6d5406e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c034000 tx=0x7f6d5c02e000 comp rx=0 tx=0).stop 2024-04-24T23:22:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d7406c4d0 msgr2=0x7f6d741e2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d7406c4d0 0x7f6d741e2970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e930 tx=0x7f6d6400edf0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 shutdown_connections 2024-04-24T23:22:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5406be30 0x7f6d5406e2f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d740d28e0 0x7f6d741e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d740ce640 0x7f6d740b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 --2- 172.21.15.26:0/1620836603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d7406c4d0 0x7f6d741e2970 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 >> 172.21.15.26:0/1620836603 conn(0x7f6d740d1dc0 msgr2=0x7f6d741e57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 shutdown_connections 2024-04-24T23:22:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.107+0000 7f6d7a38b640 1 -- 172.21.15.26:0/1620836603 wait complete. 2024-04-24T23:22:49.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:22:49.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.221+0000 7fa814276640 1 Processor -- start 2024-04-24T23:22:49.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.221+0000 7fa814276640 1 -- start start 2024-04-24T23:22:49.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.222+0000 7fa814276640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa80c15e370 0x7fa80c0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.222+0000 7fa814276640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa80c0d2120 con 0x7fa80c0d4c10 2024-04-24T23:22:49.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.222+0000 7fa814276640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa80c0b3740 con 0x7fa80c15e750 2024-04-24T23:22:49.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.222+0000 7fa814276640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa80c0d1510 con 0x7fa80c15e370 2024-04-24T23:22:49.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.222+0000 7fa812012640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa80c15e370 0x7fa80c0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.222+0000 7fa812813640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa80c0d4c10 0x7fa80c0b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38284/0 (socket says 172.21.15.26:38284) 2024-04-24T23:22:49.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.222+0000 7fa812012640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa80c15e370 0x7fa80c0d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60424/0 (socket says 172.21.15.26:60424) 2024-04-24T23:22:49.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.222+0000 7fa812813640 1 -- 172.21.15.26:0/1387193505 learned_addr learned my addr 172.21.15.26:0/1387193505 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:49.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.223+0000 7fa812012640 1 -- 172.21.15.26:0/1387193505 >> v1:172.21.15.29:6789/0 conn(0x7fa80c15e750 legacy=0x7fa80c0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:49.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.223+0000 7fa812012640 1 -- 172.21.15.26:0/1387193505 >> v1:172.21.15.26:6789/0 conn(0x7fa80c0d4c10 legacy=0x7fa80c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.223+0000 7fa811010640 1 -- 172.21.15.26:0/1387193505 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3872024646 0 0) 0x7fa80c0d2120 con 0x7fa80c0d4c10 2024-04-24T23:22:49.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.223+0000 7fa812012640 1 -- 172.21.15.26:0/1387193505 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c0b3a20 con 0x7fa80c15e370 2024-04-24T23:22:49.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.223+0000 7fa812012640 1 --2- 172.21.15.26:0/1387193505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15e370 0x7fa80c0d2520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa8040097a0 tx=0x7fa80402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8420e58e86fc99c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:49.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa811010640 1 -- 172.21.15.26:0/1387193505 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804037070 con 0x7fa80c15e370 2024-04-24T23:22:49.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa811010640 1 -- 172.21.15.26:0/1387193505 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa804032440 con 0x7fa80c15e370 2024-04-24T23:22:49.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa811010640 1 -- 172.21.15.26:0/1387193505 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804032810 con 0x7fa80c15e370 2024-04-24T23:22:49.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa814276640 1 -- 172.21.15.26:0/1387193505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15e370 msgr2=0x7fa80c0d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa814276640 1 --2- 172.21.15.26:0/1387193505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15e370 0x7fa80c0d2520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa8040097a0 tx=0x7fa80402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa814276640 1 -- 172.21.15.26:0/1387193505 shutdown_connections 2024-04-24T23:22:49.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa814276640 1 --2- 172.21.15.26:0/1387193505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15e370 0x7fa80c0d2520 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa814276640 1 -- 172.21.15.26:0/1387193505 >> 172.21.15.26:0/1387193505 conn(0x7fa80c0d40f0 msgr2=0x7fa80c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa814276640 1 -- 172.21.15.26:0/1387193505 shutdown_connections 2024-04-24T23:22:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.224+0000 7fa814276640 1 -- 172.21.15.26:0/1387193505 wait complete. 2024-04-24T23:22:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.225+0000 7fa814276640 1 Processor -- start 2024-04-24T23:22:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.225+0000 7fa814276640 1 -- start start 2024-04-24T23:22:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa814276640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa80c0d4c10 0x7fa80c0c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa814276640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa80c15e370 0x7fa80c0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa814276640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c151120 0x7fa80c1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa814276640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c0cfca0 con 0x7fa80c151120 2024-04-24T23:22:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa814276640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c0d1ed0 con 0x7fa80c0d4c10 2024-04-24T23:22:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa814276640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa80c0cfe20 con 0x7fa80c15e370 2024-04-24T23:22:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa812813640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c151120 0x7fa80c1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa812813640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c151120 0x7fa80c1515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60426/0 (socket says 172.21.15.26:60426) 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa812813640 1 -- 172.21.15.26:0/6754226 learned_addr learned my addr 172.21.15.26:0/6754226 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa812012640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa80c0d4c10 0x7fa80c0c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.226+0000 7fa811811640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa80c15e370 0x7fa80c0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa812813640 1 -- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa80c15e370 msgr2=0x7fa80c0b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa812813640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa80c15e370 0x7fa80c0b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa812813640 1 -- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa80c0d4c10 msgr2=0x7fa80c0c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa812813640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa80c0d4c10 0x7fa80c0c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa812813640 1 -- 172.21.15.26:0/6754226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c0cd140 con 0x7fa80c151120 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa812012640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa80c0d4c10 0x7fa80c0c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa811811640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa80c15e370 0x7fa80c0b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:49.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa812813640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c151120 0x7fa80c1515b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00c810 tx=0x7fa7fc00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:49.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa80affd640 1 -- 172.21.15.26:0/6754226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc018070 con 0x7fa80c151120 2024-04-24T23:22:49.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa80affd640 1 -- 172.21.15.26:0/6754226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7fc00a640 con 0x7fa80c151120 2024-04-24T23:22:49.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c0cce60 con 0x7fa80c151120 2024-04-24T23:22:49.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa80affd640 1 -- 172.21.15.26:0/6754226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00aa30 con 0x7fa80c151120 2024-04-24T23:22:49.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.227+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c15eec0 con 0x7fa80c151120 2024-04-24T23:22:49.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.228+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80c075610 con 0x7fa80c151120 2024-04-24T23:22:49.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.228+0000 7fa80affd640 1 -- 172.21.15.26:0/6754226 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7fc01b070 con 0x7fa80c151120 2024-04-24T23:22:49.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.229+0000 7fa80affd640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7f806be30 0x7fa7f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.229+0000 7fa812012640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7f806be30 0x7fa7f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.229+0000 7fa80affd640 1 -- 172.21.15.26:0/6754226 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa7fc08a540 con 0x7fa80c151120 2024-04-24T23:22:49.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.229+0000 7fa812012640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7f806be30 0x7fa7f806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa804034000 tx=0x7fa80402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:49.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.233+0000 7fa80affd640 1 -- 172.21.15.26:0/6754226 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa7fc058e50 con 0x7fa80c151120 2024-04-24T23:22:49.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.417+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa80c0b4dd0 con 0x7fa80c151120 2024-04-24T23:22:49.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.417+0000 7fa80affd640 1 -- 172.21.15.26:0/6754226 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7fa7fc058750 con 0x7fa80c151120 2024-04-24T23:22:49.419 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:22:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7f806be30 msgr2=0x7fa7f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7f806be30 0x7fa7f806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa804034000 tx=0x7fa80402e000 comp rx=0 tx=0).stop 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c151120 msgr2=0x7fa80c1515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c151120 0x7fa80c1515b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00c810 tx=0x7fa7fc00ccd0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 shutdown_connections 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7f806be30 0x7fa7f806e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c151120 0x7fa80c1515b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa80c15e370 0x7fa80c0b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 --2- 172.21.15.26:0/6754226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa80c0d4c10 0x7fa80c0c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 >> 172.21.15.26:0/6754226 conn(0x7fa80c0d40f0 msgr2=0x7fa80c0b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 shutdown_connections 2024-04-24T23:22:49.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.419+0000 7fa814276640 1 -- 172.21.15.26:0/6754226 wait complete. 2024-04-24T23:22:49.431 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:22:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.534+0000 7f4b12cfc640 1 Processor -- start 2024-04-24T23:22:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.534+0000 7f4b12cfc640 1 -- start start 2024-04-24T23:22:49.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b12cfc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b0c06d5a0 0x7f4b0c0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b12cfc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c0b3d20 0x7f4b0c150e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b12cfc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0c070f30 con 0x7f4b0c15e120 2024-04-24T23:22:49.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b12cfc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0c06dcf0 con 0x7f4b0c06d5a0 2024-04-24T23:22:49.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b12cfc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0c06dfd0 con 0x7f4b0c0b3d20 2024-04-24T23:22:49.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b0bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b0c06d5a0 0x7f4b0c0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b0bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b0c06d5a0 0x7f4b0c0b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60428/0 (socket says 172.21.15.26:60428) 2024-04-24T23:22:49.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b0bfff640 1 -- 172.21.15.26:0/1917732158 learned_addr learned my addr 172.21.15.26:0/1917732158 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:49.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b11299640 1 --2- 172.21.15.26:0/1917732158 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c0b3d20 0x7f4b0c150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b0bfff640 1 -- 172.21.15.26:0/1917732158 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c0b3d20 msgr2=0x7f4b0c150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:49.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.535+0000 7f4b0bfff640 1 --2- 172.21.15.26:0/1917732158 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c0b3d20 0x7f4b0c150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.536+0000 7f4b0bfff640 1 -- 172.21.15.26:0/1917732158 >> v1:172.21.15.29:6789/0 conn(0x7f4b0c15e120 legacy=0x7f4b0c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.536+0000 7f4b0bfff640 1 -- 172.21.15.26:0/1917732158 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c06e2b0 con 0x7f4b0c06d5a0 2024-04-24T23:22:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.536+0000 7f4b0bfff640 1 --2- 172.21.15.26:0/1917732158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06d5a0 0x7f4b0c0b37e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009870 tx=0x7f4b0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798a3c9c5efb5fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.536+0000 7f4b0b7fe640 1 -- 172.21.15.26:0/1917732158 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00037070 con 0x7f4b0c06d5a0 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.536+0000 7f4b0b7fe640 1 -- 172.21.15.26:0/1917732158 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b00032440 con 0x7f4b0c06d5a0 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.536+0000 7f4b0b7fe640 1 -- 172.21.15.26:0/1917732158 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00032810 con 0x7f4b0c06d5a0 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.537+0000 7f4b12cfc640 1 -- 172.21.15.26:0/1917732158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06d5a0 msgr2=0x7f4b0c0b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.537+0000 7f4b12cfc640 1 --2- 172.21.15.26:0/1917732158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06d5a0 0x7f4b0c0b37e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009870 tx=0x7f4b0002a270 comp rx=0 tx=0).stop 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.537+0000 7f4b12cfc640 1 -- 172.21.15.26:0/1917732158 shutdown_connections 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.537+0000 7f4b12cfc640 1 --2- 172.21.15.26:0/1917732158 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c0b3d20 0x7f4b0c150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.537+0000 7f4b12cfc640 1 --2- 172.21.15.26:0/1917732158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06d5a0 0x7f4b0c0b37e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.537+0000 7f4b12cfc640 1 -- 172.21.15.26:0/1917732158 >> 172.21.15.26:0/1917732158 conn(0x7f4b0c06b540 msgr2=0x7f4b0c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.537+0000 7f4b12cfc640 1 -- 172.21.15.26:0/1917732158 shutdown_connections 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.537+0000 7f4b12cfc640 1 -- 172.21.15.26:0/1917732158 wait complete. 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.538+0000 7f4b12cfc640 1 Processor -- start 2024-04-24T23:22:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.538+0000 7f4b12cfc640 1 -- start start 2024-04-24T23:22:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b12cfc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c06d5a0 0x7f4b0c154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b12cfc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b0c0b3d20 0x7f4b0c14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b12cfc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c15e120 0x7f4b0c0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b12cfc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c0d0040 con 0x7f4b0c15e120 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b12cfc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c06da10 con 0x7f4b0c0b3d20 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b12cfc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0c0d01c0 con 0x7f4b0c06d5a0 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b11299640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c15e120 0x7f4b0c0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b11299640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c15e120 0x7f4b0c0af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60442/0 (socket says 172.21.15.26:60442) 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b11299640 1 -- 172.21.15.26:0/3378664259 learned_addr learned my addr 172.21.15.26:0/3378664259 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b11299640 1 -- 172.21.15.26:0/3378664259 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c06d5a0 msgr2=0x7f4b0c154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b0bfff640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b0c0b3d20 0x7f4b0c14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.539+0000 7f4b10a98640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c06d5a0 0x7f4b0c154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b11299640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c06d5a0 0x7f4b0c154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b11299640 1 -- 172.21.15.26:0/3378664259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b0c0b3d20 msgr2=0x7f4b0c14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b11299640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b0c0b3d20 0x7f4b0c14a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b11299640 1 -- 172.21.15.26:0/3378664259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c0cd230 con 0x7f4b0c15e120 2024-04-24T23:22:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b11299640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c15e120 0x7f4b0c0af8c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00d800 tx=0x7f4afc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b097fa640 1 -- 172.21.15.26:0/3378664259 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00b6c0 con 0x7f4b0c15e120 2024-04-24T23:22:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b097fa640 1 -- 172.21.15.26:0/3378664259 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4afc019070 con 0x7f4b0c15e120 2024-04-24T23:22:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0c0cd510 con 0x7f4b0c15e120 2024-04-24T23:22:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b097fa640 1 -- 172.21.15.26:0/3378664259 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00bac0 con 0x7f4b0c15e120 2024-04-24T23:22:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.540+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0c0cdad0 con 0x7f4b0c15e120 2024-04-24T23:22:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.541+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0c075610 con 0x7f4b0c15e120 2024-04-24T23:22:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.541+0000 7f4b097fa640 1 -- 172.21.15.26:0/3378664259 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4afc00bc60 con 0x7f4b0c15e120 2024-04-24T23:22:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.542+0000 7f4b097fa640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4af806be30 0x7f4af806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.542+0000 7f4b097fa640 1 -- 172.21.15.26:0/3378664259 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4afc08b2e0 con 0x7f4b0c15e120 2024-04-24T23:22:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.542+0000 7f4b10a98640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4af806be30 0x7f4af806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.542+0000 7f4b10a98640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4af806be30 0x7f4af806e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4b04007fd0 tx=0x7f4b04009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:49.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.546+0000 7f4b097fa640 1 -- 172.21.15.26:0/3378664259 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4afc059c80 con 0x7f4b0c15e120 2024-04-24T23:22:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.727+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4b0c0aec10 con 0x7f4b0c15e120 2024-04-24T23:22:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.727+0000 7f4b097fa640 1 -- 172.21.15.26:0/3378664259 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f4afc059580 con 0x7f4b0c15e120 2024-04-24T23:22:49.729 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:22:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4af806be30 msgr2=0x7f4af806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4af806be30 0x7f4af806e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4b04007fd0 tx=0x7f4b04009040 comp rx=0 tx=0).stop 2024-04-24T23:22:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c15e120 msgr2=0x7f4b0c0af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c15e120 0x7f4b0c0af8c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00d800 tx=0x7f4afc00dcc0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 shutdown_connections 2024-04-24T23:22:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4af806be30 0x7f4af806e2f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c15e120 0x7f4b0c0af8c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b0c0b3d20 0x7f4b0c14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 --2- 172.21.15.26:0/3378664259 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b0c06d5a0 0x7f4b0c154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 >> 172.21.15.26:0/3378664259 conn(0x7f4b0c06b540 msgr2=0x7f4b0c15bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:49.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 shutdown_connections 2024-04-24T23:22:49.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.729+0000 7f4b12cfc640 1 -- 172.21.15.26:0/3378664259 wait complete. 2024-04-24T23:22:49.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:22:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.840+0000 7f091e929640 1 Processor -- start 2024-04-24T23:22:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.841+0000 7f091e929640 1 -- start start 2024-04-24T23:22:49.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.841+0000 7f091e929640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0918145910 0x7f091806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.841+0000 7f091e929640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f091806d350 0x7f091815d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.841+0000 7f091e929640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f091815d570 con 0x7f09180cd660 2024-04-24T23:22:49.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.841+0000 7f091e929640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0918145cf0 con 0x7f091806d350 2024-04-24T23:22:49.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.841+0000 7f091e929640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f09180cfbc0 con 0x7f0918145910 2024-04-24T23:22:49.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.842+0000 7f09177fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f091806d350 0x7f091815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.842+0000 7f091cec6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f09180cd660 0x7f091815d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38286/0 (socket says 172.21.15.26:38286) 2024-04-24T23:22:49.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.842+0000 7f091cec6640 1 -- 172.21.15.26:0/3173922725 learned_addr learned my addr 172.21.15.26:0/3173922725 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:49.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.842+0000 7f0917fff640 1 --2- 172.21.15.26:0/3173922725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0918145910 0x7f091806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.842+0000 7f09177fe640 1 -- 172.21.15.26:0/3173922725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0918145910 msgr2=0x7f091806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:49.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.842+0000 7f09177fe640 1 --2- 172.21.15.26:0/3173922725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0918145910 0x7f091806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.842+0000 7f09177fe640 1 -- 172.21.15.26:0/3173922725 >> v1:172.21.15.26:6789/0 conn(0x7f09180cd660 legacy=0x7f091815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.842+0000 7f0916ffd640 1 -- 172.21.15.26:0/3173922725 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 541526348 0 0) 0x7f0910001490 con 0x7f09180cd660 2024-04-24T23:22:49.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.842+0000 7f09177fe640 1 -- 172.21.15.26:0/3173922725 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091806d7f0 con 0x7f091806d350 2024-04-24T23:22:49.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f09177fe640 1 --2- 172.21.15.26:0/3173922725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f091806d350 0x7f091815d030 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09080097a0 tx=0x7f090802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69231cf95bfbc0b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f0916ffd640 1 -- 172.21.15.26:0/3173922725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908037070 con 0x7f091806d350 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f0916ffd640 1 -- 172.21.15.26:0/3173922725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0908032440 con 0x7f091806d350 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f0916ffd640 1 -- 172.21.15.26:0/3173922725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908032810 con 0x7f091806d350 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f091e929640 1 -- 172.21.15.26:0/3173922725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f091806d350 msgr2=0x7f091815d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f091e929640 1 --2- 172.21.15.26:0/3173922725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f091806d350 0x7f091815d030 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09080097a0 tx=0x7f090802a180 comp rx=0 tx=0).stop 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f091e929640 1 -- 172.21.15.26:0/3173922725 shutdown_connections 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f091e929640 1 --2- 172.21.15.26:0/3173922725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f091806d350 0x7f091815d030 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f091e929640 1 --2- 172.21.15.26:0/3173922725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0918145910 0x7f091806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f091e929640 1 -- 172.21.15.26:0/3173922725 >> 172.21.15.26:0/3173922725 conn(0x7f091806b300 msgr2=0x7f091806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f091e929640 1 -- 172.21.15.26:0/3173922725 shutdown_connections 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.843+0000 7f091e929640 1 -- 172.21.15.26:0/3173922725 wait complete. 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.845+0000 7f091e929640 1 Processor -- start 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.845+0000 7f091e929640 1 -- start start 2024-04-24T23:22:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.845+0000 7f091e929640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f091806d350 0x7f09181f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.845+0000 7f091e929640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09180cd660 0x7f09181e4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.845+0000 7f091e929640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0918145910 0x7f0918157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.845+0000 7f091e929640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f091806e090 con 0x7f09180cd660 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f091e929640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f091806ddb0 con 0x7f0918145910 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f091e929640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f091806e370 con 0x7f091806d350 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f09177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09180cd660 0x7f09181e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f09177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09180cd660 0x7f09181e4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60468/0 (socket says 172.21.15.26:60468) 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f09177fe640 1 -- 172.21.15.26:0/2005973714 learned_addr learned my addr 172.21.15.26:0/2005973714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f0917fff640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f091806d350 0x7f09181f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f091cec6640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0918145910 0x7f0918157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f09177fe640 1 -- 172.21.15.26:0/2005973714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f091806d350 msgr2=0x7f09181f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f09177fe640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f091806d350 0x7f09181f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f09177fe640 1 -- 172.21.15.26:0/2005973714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0918145910 msgr2=0x7f0918157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f09177fe640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0918145910 0x7f0918157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f09177fe640 1 -- 172.21.15.26:0/2005973714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0918156a70 con 0x7f09180cd660 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f091cec6640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0918145910 0x7f0918157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f09177fe640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09180cd660 0x7f09181e4440 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0908035040 tx=0x7f09080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f0914ff9640 1 -- 172.21.15.26:0/2005973714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908037040 con 0x7f09180cd660 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f0914ff9640 1 -- 172.21.15.26:0/2005973714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0908030500 con 0x7f09180cd660 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f0914ff9640 1 -- 172.21.15.26:0/2005973714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908030910 con 0x7f09180cd660 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0918156d50 con 0x7f09180cd660 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.846+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0918157310 con 0x7f09180cd660 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.847+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09180694c0 con 0x7f09180cd660 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.848+0000 7f0914ff9640 1 -- 172.21.15.26:0/2005973714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0908044020 con 0x7f09180cd660 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.848+0000 7f0914ff9640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f090406be30 0x7f090406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.848+0000 7f0914ff9640 1 -- 172.21.15.26:0/2005973714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f09080ab030 con 0x7f09180cd660 2024-04-24T23:22:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.848+0000 7f0917fff640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f090406be30 0x7f090406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.849+0000 7f0917fff640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f090406be30 0x7f090406e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09181f4f00 tx=0x7f090c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:49.853+0000 7f0914ff9640 1 -- 172.21.15.26:0/2005973714 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09080799e0 con 0x7f09180cd660 2024-04-24T23:22:50.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.074+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0918139a30 con 0x7f09180cd660 2024-04-24T23:22:50.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.074+0000 7f0914ff9640 1 -- 172.21.15.26:0/2005973714 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5222 (secure 0 0 0) 0x7f09080792e0 con 0x7f09180cd660 2024-04-24T23:22:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:50.076 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:21:48.324889+0000","modified":"2024-04-24T23:22:16.101089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5088,"mds_1":4867,"mds_2":4864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4864":{"gid":4864,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6834/2201738459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2201738459},{"type":"v1","addr":"172.21.15.29:6834","nonce":2201738459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4867":{"gid":4867,"name":"d","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6835/3972561683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3972561683},{"type":"v1","addr":"172.21.15.29:6835","nonce":3972561683}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/315291888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":315291888},{"type":"v1","addr":"172.21.15.26:6836","nonce":315291888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088,4867,4864]},"id":2}]} 2024-04-24T23:22:50.076 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 24 2024-04-24T23:22:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f090406be30 msgr2=0x7f090406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f090406be30 0x7f090406e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09181f4f00 tx=0x7f090c009190 comp rx=0 tx=0).stop 2024-04-24T23:22:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09180cd660 msgr2=0x7f09181e4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09180cd660 0x7f09181e4440 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0908035040 tx=0x7f09080046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 shutdown_connections 2024-04-24T23:22:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f090406be30 0x7f090406e2f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0918145910 0x7f0918157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09180cd660 0x7f09181e4440 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 --2- 172.21.15.26:0/2005973714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f091806d350 0x7f09181f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 >> 172.21.15.26:0/2005973714 conn(0x7f091806b300 msgr2=0x7f09180b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:50.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 shutdown_connections 2024-04-24T23:22:50.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.076+0000 7f091e929640 1 -- 172.21.15.26:0/2005973714 wait complete. 2024-04-24T23:22:50.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:22:50.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.188+0000 7f6399953640 1 Processor -- start 2024-04-24T23:22:50.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.188+0000 7f6399953640 1 -- start start 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f6399953640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63940b63e0 0x7f6394142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f6399953640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f639406deb0 con 0x7f6394145890 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f6399953640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f639406e190 con 0x7f6394142310 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f6399953640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f639406dc60 con 0x7f63940b63e0 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f63927fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63940b63e0 0x7f6394142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f63937fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6394145890 0x7f639406ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38296/0 (socket says 172.21.15.26:38296) 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f63927fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63940b63e0 0x7f6394142d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60480/0 (socket says 172.21.15.26:60480) 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f63937fe640 1 -- 172.21.15.26:0/1755151254 learned_addr learned my addr 172.21.15.26:0/1755151254 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f63927fc640 1 -- 172.21.15.26:0/1755151254 >> v1:172.21.15.29:6789/0 conn(0x7f6394142310 legacy=0x7f63940b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f63927fc640 1 -- 172.21.15.26:0/1755151254 >> v1:172.21.15.26:6789/0 conn(0x7f6394145890 legacy=0x7f639406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f6391ffb640 1 -- 172.21.15.26:0/1755151254 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2918756118 0 0) 0x7f639406deb0 con 0x7f6394145890 2024-04-24T23:22:50.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.189+0000 7f63927fc640 1 -- 172.21.15.26:0/1755151254 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639406e470 con 0x7f63940b63e0 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.190+0000 7f63927fc640 1 --2- 172.21.15.26:0/1755151254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63940b63e0 0x7f6394142d20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f639406dde0 tx=0x7f637c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2883e61ab4e0c004 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.190+0000 7f6391ffb640 1 -- 172.21.15.26:0/1755151254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c037070 con 0x7f63940b63e0 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.190+0000 7f6391ffb640 1 -- 172.21.15.26:0/1755151254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c032440 con 0x7f63940b63e0 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.190+0000 7f6391ffb640 1 -- 172.21.15.26:0/1755151254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c032810 con 0x7f63940b63e0 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.191+0000 7f6399953640 1 -- 172.21.15.26:0/1755151254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63940b63e0 msgr2=0x7f6394142d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.191+0000 7f6399953640 1 --2- 172.21.15.26:0/1755151254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63940b63e0 0x7f6394142d20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f639406dde0 tx=0x7f637c02a180 comp rx=0 tx=0).stop 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.191+0000 7f6399953640 1 -- 172.21.15.26:0/1755151254 shutdown_connections 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.191+0000 7f6399953640 1 --2- 172.21.15.26:0/1755151254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63940b63e0 0x7f6394142d20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.191+0000 7f6399953640 1 -- 172.21.15.26:0/1755151254 >> 172.21.15.26:0/1755151254 conn(0x7f63940729a0 msgr2=0x7f639406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.191+0000 7f6399953640 1 -- 172.21.15.26:0/1755151254 shutdown_connections 2024-04-24T23:22:50.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.191+0000 7f6399953640 1 -- 172.21.15.26:0/1755151254 wait complete. 2024-04-24T23:22:50.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.192+0000 7f6399953640 1 Processor -- start 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.192+0000 7f6399953640 1 -- start start 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f6399953640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63940b63e0 0x7f639413f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f6399953640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6394142310 0x7f639415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f6399953640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6394145890 0x7f6394140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f6399953640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63941426f0 con 0x7f6394145890 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f6399953640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f639406eac0 con 0x7f63940b63e0 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f6399953640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6394142870 con 0x7f6394142310 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f63937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6394145890 0x7f6394140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f63937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6394145890 0x7f6394140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60490/0 (socket says 172.21.15.26:60490) 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f63937fe640 1 -- 172.21.15.26:0/432099683 learned_addr learned my addr 172.21.15.26:0/432099683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f63927fc640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6394142310 0x7f639415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f6392ffd640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63940b63e0 0x7f639413f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f63937fe640 1 -- 172.21.15.26:0/432099683 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6394142310 msgr2=0x7f639415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f63937fe640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6394142310 0x7f639415e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f63937fe640 1 -- 172.21.15.26:0/432099683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63940b63e0 msgr2=0x7f639413f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f63937fe640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63940b63e0 0x7f639413f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.193+0000 7f63937fe640 1 -- 172.21.15.26:0/432099683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639415d930 con 0x7f6394145890 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.194+0000 7f63937fe640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6394145890 0x7f6394140380 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f638400c700 tx=0x7f638400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.194+0000 7f638bfff640 1 -- 172.21.15.26:0/432099683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384018070 con 0x7f6394145890 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.194+0000 7f638bfff640 1 -- 172.21.15.26:0/432099683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638400ce10 con 0x7f6394145890 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.194+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639415dc10 con 0x7f6394145890 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.194+0000 7f638bfff640 1 -- 172.21.15.26:0/432099683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638400a860 con 0x7f6394145890 2024-04-24T23:22:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.194+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639415e1d0 con 0x7f6394145890 2024-04-24T23:22:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.195+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6394067720 con 0x7f6394145890 2024-04-24T23:22:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.195+0000 7f638bfff640 1 -- 172.21.15.26:0/432099683 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f638400aa00 con 0x7f6394145890 2024-04-24T23:22:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.196+0000 7f638bfff640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637406be60 0x7f637406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.196+0000 7f638bfff640 1 -- 172.21.15.26:0/432099683 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f638408a350 con 0x7f6394145890 2024-04-24T23:22:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.196+0000 7f6392ffd640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637406be60 0x7f637406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.196+0000 7f6392ffd640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637406be60 0x7f637406e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f638c000f60 tx=0x7f638c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.200+0000 7f638bfff640 1 -- 172.21.15.26:0/432099683 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6384058d10 con 0x7f6394145890 2024-04-24T23:22:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.386+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63940694c0 con 0x7f6394145890 2024-04-24T23:22:50.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.387+0000 7f638bfff640 1 -- 172.21.15.26:0/432099683 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7f6384058610 con 0x7f6394145890 2024-04-24T23:22:50.389 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:50.389 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:49.992052+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:21:45.983358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:21:47.321239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:22:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637406be60 msgr2=0x7f637406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637406be60 0x7f637406e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f638c000f60 tx=0x7f638c009040 comp rx=0 tx=0).stop 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6394145890 msgr2=0x7f6394140380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6394145890 0x7f6394140380 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f638400c700 tx=0x7f638400cbc0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 shutdown_connections 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637406be60 0x7f637406e320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6394145890 0x7f6394140380 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6394142310 0x7f639415e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 --2- 172.21.15.26:0/432099683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f63940b63e0 0x7f639413f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 >> 172.21.15.26:0/432099683 conn(0x7f63940729a0 msgr2=0x7f639406ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 shutdown_connections 2024-04-24T23:22:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.390+0000 7f6399953640 1 -- 172.21.15.26:0/432099683 wait complete. 2024-04-24T23:22:50.402 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:22:50.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:22:50.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.502+0000 7fc695309640 1 Processor -- start 2024-04-24T23:22:50.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.502+0000 7fc695309640 1 -- start start 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc695309640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc69006f4f0 0x7fc6900ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc695309640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6900b0220 con 0x7fc6900addb0 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc695309640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6900af180 con 0x7fc69006f110 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc695309640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc6900af460 con 0x7fc69006f4f0 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc68e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc69006f4f0 0x7fc6900ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc68f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc6900addb0 0x7fc690154a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38304/0 (socket says 172.21.15.26:38304) 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc68e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc69006f4f0 0x7fc6900ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60492/0 (socket says 172.21.15.26:60492) 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc68f7fe640 1 -- 172.21.15.26:0/1821908726 learned_addr learned my addr 172.21.15.26:0/1821908726 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc68e7fc640 1 -- 172.21.15.26:0/1821908726 >> v1:172.21.15.29:6789/0 conn(0x7fc69006f110 legacy=0x7fc69015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc68e7fc640 1 -- 172.21.15.26:0/1821908726 >> v1:172.21.15.26:6789/0 conn(0x7fc6900addb0 legacy=0x7fc690154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.503+0000 7fc68e7fc640 1 -- 172.21.15.26:0/1821908726 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6900af740 con 0x7fc69006f4f0 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.504+0000 7fc68e7fc640 1 --2- 172.21.15.26:0/1821908726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 0x7fc6900ad870 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097a0 tx=0x7fc67802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ae8a29b329322d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.504+0000 7fc68dffb640 1 -- 172.21.15.26:0/1821908726 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678037070 con 0x7fc69006f4f0 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.504+0000 7fc68dffb640 1 -- 172.21.15.26:0/1821908726 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc678032440 con 0x7fc69006f4f0 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.504+0000 7fc68dffb640 1 -- 172.21.15.26:0/1821908726 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678032810 con 0x7fc69006f4f0 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.505+0000 7fc695309640 1 -- 172.21.15.26:0/1821908726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 msgr2=0x7fc6900ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.505+0000 7fc695309640 1 --2- 172.21.15.26:0/1821908726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 0x7fc6900ad870 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097a0 tx=0x7fc67802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.505+0000 7fc695309640 1 -- 172.21.15.26:0/1821908726 shutdown_connections 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.505+0000 7fc695309640 1 --2- 172.21.15.26:0/1821908726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 0x7fc6900ad870 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.505+0000 7fc695309640 1 -- 172.21.15.26:0/1821908726 >> 172.21.15.26:0/1821908726 conn(0x7fc6900729a0 msgr2=0x7fc69006b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.505+0000 7fc695309640 1 -- 172.21.15.26:0/1821908726 shutdown_connections 2024-04-24T23:22:50.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.505+0000 7fc695309640 1 -- 172.21.15.26:0/1821908726 wait complete. 2024-04-24T23:22:50.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.506+0000 7fc695309640 1 Processor -- start 2024-04-24T23:22:50.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.506+0000 7fc695309640 1 -- start start 2024-04-24T23:22:50.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc695309640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc69006f110 0x7fc6901f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc695309640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 0x7fc690160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc695309640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6900addb0 0x7fc690141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc695309640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc6900aee50 con 0x7fc69006f4f0 2024-04-24T23:22:50.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc695309640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc6900afd00 con 0x7fc69006f110 2024-04-24T23:22:50.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc695309640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc69006fb00 con 0x7fc6900addb0 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 0x7fc690160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 0x7fc690160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60494/0 (socket says 172.21.15.26:60494) 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68e7fc640 1 -- 172.21.15.26:0/1256476967 learned_addr learned my addr 172.21.15.26:0/1256476967 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68f7fe640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6900addb0 0x7fc690141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68e7fc640 1 -- 172.21.15.26:0/1256476967 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6900addb0 msgr2=0x7fc690141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68effd640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc69006f110 0x7fc6901f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68e7fc640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6900addb0 0x7fc690141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68e7fc640 1 -- 172.21.15.26:0/1256476967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc69006f110 msgr2=0x7fc6901f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68e7fc640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc69006f110 0x7fc6901f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68e7fc640 1 -- 172.21.15.26:0/1256476967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6901f7cb0 con 0x7fc69006f4f0 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc68e7fc640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 0x7fc690160460 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc678000c00 tx=0x7fc6780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc6877fe640 1 -- 172.21.15.26:0/1256476967 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678037040 con 0x7fc69006f4f0 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc6877fe640 1 -- 172.21.15.26:0/1256476967 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc678042070 con 0x7fc69006f4f0 2024-04-24T23:22:50.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6901f7f90 con 0x7fc69006f4f0 2024-04-24T23:22:50.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.507+0000 7fc6877fe640 1 -- 172.21.15.26:0/1256476967 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678030a70 con 0x7fc69006f4f0 2024-04-24T23:22:50.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.508+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6901609a0 con 0x7fc69006f4f0 2024-04-24T23:22:50.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.508+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6900694c0 con 0x7fc69006f4f0 2024-04-24T23:22:50.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.509+0000 7fc6877fe640 1 -- 172.21.15.26:0/1256476967 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc678030c10 con 0x7fc69006f4f0 2024-04-24T23:22:50.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.509+0000 7fc6877fe640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc67006be30 0x7fc67006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.509+0000 7fc6877fe640 1 -- 172.21.15.26:0/1256476967 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc6780ab8a0 con 0x7fc69006f4f0 2024-04-24T23:22:50.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.509+0000 7fc68effd640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc67006be30 0x7fc67006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.510+0000 7fc68effd640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc67006be30 0x7fc67006e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc688003d90 tx=0x7fc688009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:50.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.514+0000 7fc6877fe640 1 -- 172.21.15.26:0/1256476967 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc678045360 con 0x7fc69006f4f0 2024-04-24T23:22:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.734+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc69006c2b0 con 0x7fc69006f4f0 2024-04-24T23:22:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.735+0000 7fc6877fe640 1 -- 172.21.15.26:0/1256476967 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5222 (secure 0 0 0) 0x7fc67807a2e0 con 0x7fc69006f4f0 2024-04-24T23:22:50.736 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:50.736 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:21:48.324889+0000","modified":"2024-04-24T23:22:16.101089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5088,"mds_1":4867,"mds_2":4864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4864":{"gid":4864,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6834/2201738459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2201738459},{"type":"v1","addr":"172.21.15.29:6834","nonce":2201738459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4867":{"gid":4867,"name":"d","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6835/3972561683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3972561683},{"type":"v1","addr":"172.21.15.29:6835","nonce":3972561683}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/315291888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":315291888},{"type":"v1","addr":"172.21.15.26:6836","nonce":315291888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088,4867,4864]},"id":2}]} 2024-04-24T23:22:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 24 2024-04-24T23:22:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc67006be30 msgr2=0x7fc67006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc67006be30 0x7fc67006e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc688003d90 tx=0x7fc688009040 comp rx=0 tx=0).stop 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 msgr2=0x7fc690160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 0x7fc690160460 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc678000c00 tx=0x7fc6780046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 shutdown_connections 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc67006be30 0x7fc67006e2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6900addb0 0x7fc690141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69006f4f0 0x7fc690160460 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 --2- 172.21.15.26:0/1256476967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc69006f110 0x7fc6901f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 >> 172.21.15.26:0/1256476967 conn(0x7fc6900729a0 msgr2=0x7fc69015c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 shutdown_connections 2024-04-24T23:22:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.737+0000 7fc695309640 1 -- 172.21.15.26:0/1256476967 wait complete. 2024-04-24T23:22:50.750 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:22:50.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.849+0000 7f89411e7640 1 Processor -- start 2024-04-24T23:22:50.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.849+0000 7f89411e7640 1 -- start start 2024-04-24T23:22:50.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f89411e7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c15eb00 0x7f893c15ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f89411e7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893c0cd0b0 con 0x7f893c143c40 2024-04-24T23:22:50.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f89411e7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893c071240 con 0x7f893c0cf870 2024-04-24T23:22:50.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f89411e7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f893c071b70 con 0x7f893c15eb00 2024-04-24T23:22:50.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f893c143c40 0x7f893c0cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38320/0 (socket says 172.21.15.26:38320) 2024-04-24T23:22:50.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893ad9d640 1 -- 172.21.15.26:0/828718347 learned_addr learned my addr 172.21.15.26:0/828718347 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893b59e640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c15eb00 0x7f893c15ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1096028082 0 0) 0x7f893c0cd0b0 con 0x7f893c143c40 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8928005730 con 0x7f893c143c40 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2574216359 0 0) 0x7f8928005730 con 0x7f893c143c40 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c15eb00 msgr2=0x7f893c15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c15eb00 0x7f893c15ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 >> v1:172.21.15.29:6789/0 conn(0x7f893c0cf870 legacy=0x7f893c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c071520 con 0x7f893c143c40 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f892c001670 con 0x7f893c143c40 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 >> v1:172.21.15.26:6789/0 conn(0x7f893c143c40 legacy=0x7f893c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c15eb00 0x7f8928005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.850+0000 7f893a59c640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8928006130 0x7f8928006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893a59c640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f89280073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f893c071b70 con 0x7f8928006f50 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f892800dc30 con 0x7f8928006130 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f892800de30 con 0x7f893c15eb00 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893b59e640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f89280073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f8933fff640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8928006130 0x7f8928006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893ad9d640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c15eb00 0x7f8928005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893b59e640 1 -- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c15eb00 msgr2=0x7f8928005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893b59e640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c15eb00 0x7f8928005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893b59e640 1 -- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8928006130 msgr2=0x7f8928006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893b59e640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8928006130 0x7f8928006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.851+0000 7f893b59e640 1 -- 172.21.15.26:0/828718347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8928005730 con 0x7f8928006f50 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f893b59e640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f89280073d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8934009fc0 tx=0x7f8928007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99f38b4820e8373 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934009c10 con 0x7f8928006f50 2024-04-24T23:22:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8934036070 con 0x7f8928006f50 2024-04-24T23:22:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f893a59c640 1 -- 172.21.15.26:0/828718347 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934031880 con 0x7f8928006f50 2024-04-24T23:22:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f89411e7640 1 -- 172.21.15.26:0/828718347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 msgr2=0x7f89280073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f89411e7640 1 --2- 172.21.15.26:0/828718347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f89280073d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8934009fc0 tx=0x7f8928007a40 comp rx=0 tx=0).stop 2024-04-24T23:22:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f893b59e640 1 -- 172.21.15.26:0/828718347 reap_dead start 2024-04-24T23:22:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f89411e7640 1 -- 172.21.15.26:0/828718347 shutdown_connections 2024-04-24T23:22:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f89411e7640 1 -- 172.21.15.26:0/828718347 >> 172.21.15.26:0/828718347 conn(0x7f893c06b300 msgr2=0x7f893c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f89411e7640 1 -- 172.21.15.26:0/828718347 shutdown_connections 2024-04-24T23:22:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.852+0000 7f89411e7640 1 -- 172.21.15.26:0/828718347 wait complete. 2024-04-24T23:22:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.854+0000 7f89411e7640 1 Processor -- start 2024-04-24T23:22:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.854+0000 7f89411e7640 1 -- start start 2024-04-24T23:22:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.854+0000 7f89411e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f893c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.854+0000 7f89411e7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c0b2350 0x7f893c13e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.854+0000 7f89411e7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f893c0b27d0 0x7f893c13ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.854+0000 7f89411e7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f893c15ef30 con 0x7f8928006f50 2024-04-24T23:22:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.854+0000 7f89411e7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f893c144020 con 0x7f893c0b27d0 2024-04-24T23:22:50.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.854+0000 7f89411e7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f893c15f0b0 con 0x7f893c0b2350 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f893c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f893c15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60514/0 (socket says 172.21.15.26:60514) 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893ad9d640 1 -- 172.21.15.26:0/1752011096 learned_addr learned my addr 172.21.15.26:0/1752011096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893b59e640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f893c0b27d0 0x7f893c13ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f8933fff640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c0b2350 0x7f893c13e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893ad9d640 1 -- 172.21.15.26:0/1752011096 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c0b2350 msgr2=0x7f893c13e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893ad9d640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c0b2350 0x7f893c13e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893ad9d640 1 -- 172.21.15.26:0/1752011096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f893c0b27d0 msgr2=0x7f893c13ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893ad9d640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f893c0b27d0 0x7f893c13ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893ad9d640 1 -- 172.21.15.26:0/1752011096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c138b60 con 0x7f8928006f50 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f893ad9d640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f893c15a930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f892c007370 tx=0x7f892c007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f89337fe640 1 -- 172.21.15.26:0/1752011096 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c01a070 con 0x7f8928006f50 2024-04-24T23:22:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f89337fe640 1 -- 172.21.15.26:0/1752011096 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c00f960 con 0x7f8928006f50 2024-04-24T23:22:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c138e40 con 0x7f8928006f50 2024-04-24T23:22:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f89337fe640 1 -- 172.21.15.26:0/1752011096 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c00fd10 con 0x7f8928006f50 2024-04-24T23:22:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.855+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c139dd0 con 0x7f8928006f50 2024-04-24T23:22:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.856+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893c0694c0 con 0x7f8928006f50 2024-04-24T23:22:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.857+0000 7f89337fe640 1 -- 172.21.15.26:0/1752011096 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f892c01d070 con 0x7f8928006f50 2024-04-24T23:22:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.857+0000 7f89337fe640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f891c06be30 0x7f891c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.857+0000 7f89337fe640 1 -- 172.21.15.26:0/1752011096 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f892c08a9a0 con 0x7f8928006f50 2024-04-24T23:22:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.857+0000 7f8933fff640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f891c06be30 0x7f891c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.858+0000 7f8933fff640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f891c06be30 0x7f891c06e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f893c0af750 tx=0x7f8924009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:50.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:50.862+0000 7f89337fe640 1 -- 172.21.15.26:0/1752011096 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f892c059350 con 0x7f8928006f50 2024-04-24T23:22:51.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.083+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f893c15d400 con 0x7f8928006f50 2024-04-24T23:22:51.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.084+0000 7f89337fe640 1 -- 172.21.15.26:0/1752011096 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5222 (secure 0 0 0) 0x7f892c058c50 con 0x7f8928006f50 2024-04-24T23:22:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:51.086 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:21:48.324889+0000","modified":"2024-04-24T23:22:16.101089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5088,"mds_1":4867,"mds_2":4864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4864":{"gid":4864,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6834/2201738459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2201738459},{"type":"v1","addr":"172.21.15.29:6834","nonce":2201738459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4867":{"gid":4867,"name":"d","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6835/3972561683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3972561683},{"type":"v1","addr":"172.21.15.29:6835","nonce":3972561683}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/315291888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":315291888},{"type":"v1","addr":"172.21.15.26:6836","nonce":315291888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088,4867,4864]},"id":2}]} 2024-04-24T23:22:51.086 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 24 2024-04-24T23:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f891c06be30 msgr2=0x7f891c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f891c06be30 0x7f891c06e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f893c0af750 tx=0x7f8924009040 comp rx=0 tx=0).stop 2024-04-24T23:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 msgr2=0x7f893c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f893c15a930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f892c007370 tx=0x7f892c007830 comp rx=0 tx=0).stop 2024-04-24T23:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 shutdown_connections 2024-04-24T23:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f891c06be30 0x7f891c06e2f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f893c0b27d0 0x7f893c13ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f893c0b2350 0x7f893c13e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 --2- 172.21.15.26:0/1752011096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928006f50 0x7f893c15a930 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 >> 172.21.15.26:0/1752011096 conn(0x7f893c06b300 msgr2=0x7f893c13e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 shutdown_connections 2024-04-24T23:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.086+0000 7f89411e7640 1 -- 172.21.15.26:0/1752011096 wait complete. 2024-04-24T23:22:51.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:22:51.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.198+0000 7fab6a47b640 1 Processor -- start 2024-04-24T23:22:51.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.198+0000 7fab6a47b640 1 -- start start 2024-04-24T23:22:51.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab6a47b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab6a47b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab640cc860 0x7fab64072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab6a47b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab640cd0b0 con 0x7fab6415eb00 2024-04-24T23:22:51.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab6a47b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fab64071b70 con 0x7fab640cc860 2024-04-24T23:22:51.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab6a47b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fab64071240 con 0x7fab64143c40 2024-04-24T23:22:51.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab63fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab637fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab640cc860 0x7fab64072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab63fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:32888/0 (socket says 172.21.15.26:32888) 2024-04-24T23:22:51.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab63fff640 1 -- 172.21.15.26:0/2961034447 learned_addr learned my addr 172.21.15.26:0/2961034447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:51.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.199+0000 7fab637fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab640cc860 0x7fab64072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:43730/0 (socket says 172.21.15.26:43730) 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.200+0000 7fab63fff640 1 -- 172.21.15.26:0/2961034447 >> v2:172.21.15.29:3300/0 conn(0x7fab640cc860 msgr2=0x7fab64072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.200+0000 7fab63fff640 1 --2- 172.21.15.26:0/2961034447 >> v2:172.21.15.29:3300/0 conn(0x7fab640cc860 0x7fab64072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.200+0000 7fab63fff640 1 -- 172.21.15.26:0/2961034447 >> v1:172.21.15.29:6789/0 conn(0x7fab6415eb00 legacy=0x7fab640b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.200+0000 7fab63fff640 1 -- 172.21.15.26:0/2961034447 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab64071520 con 0x7fab64143c40 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.200+0000 7fab637fe640 1 --2- 172.21.15.26:0/2961034447 >> v2:172.21.15.29:3300/0 conn(0x7fab640cc860 0x7fab64072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.200+0000 7fab63fff640 1 --2- 172.21.15.26:0/2961034447 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640cf870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab54009870 tx=0x7fab5402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac55f597cdc2fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab62ffd640 1 -- 172.21.15.26:0/2961034447 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54037070 con 0x7fab64143c40 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab62ffd640 1 -- 172.21.15.26:0/2961034447 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54032440 con 0x7fab64143c40 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab62ffd640 1 -- 172.21.15.26:0/2961034447 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54032810 con 0x7fab64143c40 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab6a47b640 1 -- 172.21.15.26:0/2961034447 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 msgr2=0x7fab640cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab6a47b640 1 --2- 172.21.15.26:0/2961034447 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640cf870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab54009870 tx=0x7fab5402a270 comp rx=0 tx=0).stop 2024-04-24T23:22:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab6a47b640 1 -- 172.21.15.26:0/2961034447 shutdown_connections 2024-04-24T23:22:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab6a47b640 1 --2- 172.21.15.26:0/2961034447 >> v2:172.21.15.29:3300/0 conn(0x7fab640cc860 0x7fab64072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab6a47b640 1 --2- 172.21.15.26:0/2961034447 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640cf870 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab6a47b640 1 -- 172.21.15.26:0/2961034447 >> 172.21.15.26:0/2961034447 conn(0x7fab6406b300 msgr2=0x7fab6406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab6a47b640 1 -- 172.21.15.26:0/2961034447 shutdown_connections 2024-04-24T23:22:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.201+0000 7fab6a47b640 1 -- 172.21.15.26:0/2961034447 wait complete. 2024-04-24T23:22:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.202+0000 7fab6a47b640 1 Processor -- start 2024-04-24T23:22:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.203+0000 7fab6a47b640 1 -- start start 2024-04-24T23:22:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.203+0000 7fab6a47b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab640cc860 0x7fab641f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.203+0000 7fab6a47b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.203+0000 7fab6a47b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab6415eb00 0x7fab641363b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.203+0000 7fab6a47b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab64072540 con 0x7fab640cc860 2024-04-24T23:22:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.203+0000 7fab6a47b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fab64144020 con 0x7fab6415eb00 2024-04-24T23:22:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.203+0000 7fab6a47b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fab641397b0 con 0x7fab64143c40 2024-04-24T23:22:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.203+0000 7fab63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab640cc860 0x7fab641f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab640cc860 0x7fab641f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60518/0 (socket says 172.21.15.26:60518) 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab63fff640 1 -- 172.21.15.26:0/1669781348 learned_addr learned my addr 172.21.15.26:0/1669781348 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab637fe640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab68a18640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab6415eb00 0x7fab641363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab63fff640 1 -- 172.21.15.26:0/1669781348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 msgr2=0x7fab640ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab63fff640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab63fff640 1 -- 172.21.15.26:0/1669781348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab6415eb00 msgr2=0x7fab641363b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab63fff640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab6415eb00 0x7fab641363b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab63fff640 1 -- 172.21.15.26:0/1669781348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab64139a90 con 0x7fab640cc860 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab63fff640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab640cc860 0x7fab641f3a60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fab54000c00 tx=0x7fab540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab60ff9640 1 -- 172.21.15.26:0/1669781348 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54037040 con 0x7fab640cc860 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab60ff9640 1 -- 172.21.15.26:0/1669781348 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54042070 con 0x7fab640cc860 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab64139d70 con 0x7fab640cc860 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab60ff9640 1 -- 172.21.15.26:0/1669781348 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54030880 con 0x7fab640cc860 2024-04-24T23:22:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.204+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab641f3fa0 con 0x7fab640cc860 2024-04-24T23:22:51.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.205+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab640694c0 con 0x7fab640cc860 2024-04-24T23:22:51.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.206+0000 7fab60ff9640 1 -- 172.21.15.26:0/1669781348 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab54030a20 con 0x7fab640cc860 2024-04-24T23:22:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.206+0000 7fab60ff9640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab5006be30 0x7fab5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.206+0000 7fab60ff9640 1 -- 172.21.15.26:0/1669781348 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fab540ab690 con 0x7fab640cc860 2024-04-24T23:22:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.206+0000 7fab637fe640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab5006be30 0x7fab5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.206+0000 7fab637fe640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab5006be30 0x7fab5006e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fab58009560 tx=0x7fab58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.210+0000 7fab60ff9640 1 -- 172.21.15.26:0/1669781348 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fab5407a0d0 con 0x7fab640cc860 2024-04-24T23:22:51.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.393+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab64072c90 con 0x7fab640cc860 2024-04-24T23:22:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.394+0000 7fab60ff9640 1 -- 172.21.15.26:0/1669781348 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7fab540799d0 con 0x7fab640cc860 2024-04-24T23:22:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:51.396 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:21:49.992052+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:21:45.983358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:21:47.321239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:22:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab5006be30 msgr2=0x7fab5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab5006be30 0x7fab5006e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fab58009560 tx=0x7fab58008040 comp rx=0 tx=0).stop 2024-04-24T23:22:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab640cc860 msgr2=0x7fab641f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab640cc860 0x7fab641f3a60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fab54000c00 tx=0x7fab540046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 shutdown_connections 2024-04-24T23:22:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab5006be30 0x7fab5006e2f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab6415eb00 0x7fab641363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab64143c40 0x7fab640ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 --2- 172.21.15.26:0/1669781348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab640cc860 0x7fab641f3a60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 >> 172.21.15.26:0/1669781348 conn(0x7fab6406b300 msgr2=0x7fab6415b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 shutdown_connections 2024-04-24T23:22:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.396+0000 7fab6a47b640 1 -- 172.21.15.26:0/1669781348 wait complete. 2024-04-24T23:22:51.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:22:51.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.510+0000 7f2749c15640 1 Processor -- start 2024-04-24T23:22:51.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.510+0000 7f2749c15640 1 -- start start 2024-04-24T23:22:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.511+0000 7f2749c15640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2744145910 0x7f274406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.511+0000 7f2749c15640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 0x7f27440cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.511+0000 7f2749c15640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27440cfbc0 con 0x7f274406d350 2024-04-24T23:22:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.511+0000 7f2749c15640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2744145cf0 con 0x7f2744145910 2024-04-24T23:22:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.511+0000 7f2749c15640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f274406d7f0 con 0x7f27440cd660 2024-04-24T23:22:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.512+0000 7f2743fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 0x7f27440cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.512+0000 7f27437fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2744145910 0x7f274406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.512+0000 7f2742ffd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f274406d350 0x7f274415d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:40292/0 (socket says 172.21.15.26:40292) 2024-04-24T23:22:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.512+0000 7f2742ffd640 1 -- 172.21.15.26:0/2169522857 learned_addr learned my addr 172.21.15.26:0/2169522857 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.512+0000 7f27437fe640 1 -- 172.21.15.26:0/2169522857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 msgr2=0x7f27440cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.512+0000 7f27437fe640 1 --2- 172.21.15.26:0/2169522857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 0x7f27440cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.512+0000 7f27437fe640 1 -- 172.21.15.26:0/2169522857 >> v1:172.21.15.29:6789/0 conn(0x7f274406d350 legacy=0x7f274415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.512+0000 7f27437fe640 1 -- 172.21.15.26:0/2169522857 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274406dad0 con 0x7f2744145910 2024-04-24T23:22:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.512+0000 7f2743fff640 1 --2- 172.21.15.26:0/2169522857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 0x7f27440cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f27437fe640 1 --2- 172.21.15.26:0/2169522857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2744145910 0x7f274406e890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27340097d0 tx=0x7f273402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94b642c440dc3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f27427fc640 1 -- 172.21.15.26:0/2169522857 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734037070 con 0x7f2744145910 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f27427fc640 1 -- 172.21.15.26:0/2169522857 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2734032440 con 0x7f2744145910 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f27427fc640 1 -- 172.21.15.26:0/2169522857 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734032810 con 0x7f2744145910 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f2749c15640 1 -- 172.21.15.26:0/2169522857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2744145910 msgr2=0x7f274406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f2749c15640 1 --2- 172.21.15.26:0/2169522857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2744145910 0x7f274406e890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27340097d0 tx=0x7f273402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f2749c15640 1 -- 172.21.15.26:0/2169522857 shutdown_connections 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f2749c15640 1 --2- 172.21.15.26:0/2169522857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 0x7f27440cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f2749c15640 1 --2- 172.21.15.26:0/2169522857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2744145910 0x7f274406e890 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.513+0000 7f2749c15640 1 -- 172.21.15.26:0/2169522857 >> 172.21.15.26:0/2169522857 conn(0x7f274406b300 msgr2=0x7f274406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.514+0000 7f2749c15640 1 -- 172.21.15.26:0/2169522857 shutdown_connections 2024-04-24T23:22:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.514+0000 7f2749c15640 1 -- 172.21.15.26:0/2169522857 wait complete. 2024-04-24T23:22:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.515+0000 7f2749c15640 1 Processor -- start 2024-04-24T23:22:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.515+0000 7f2749c15640 1 -- start start 2024-04-24T23:22:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2749c15640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f274406d350 0x7f2744141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2749c15640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 0x7f27441f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2749c15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2744145910 0x7f27441f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2749c15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f274406e370 con 0x7f2744145910 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2749c15640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f274406e090 con 0x7f274406d350 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2749c15640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f274415d770 con 0x7f27440cd660 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2743fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2744145910 0x7f27441f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2743fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2744145910 0x7f27441f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60526/0 (socket says 172.21.15.26:60526) 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2743fff640 1 -- 172.21.15.26:0/3949957017 learned_addr learned my addr 172.21.15.26:0/3949957017 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f27437fe640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f274406d350 0x7f2744141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2742ffd640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 0x7f27441f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2743fff640 1 -- 172.21.15.26:0/3949957017 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 msgr2=0x7f27441f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2743fff640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 0x7f27441f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2743fff640 1 -- 172.21.15.26:0/3949957017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f274406d350 msgr2=0x7f2744141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2743fff640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f274406d350 0x7f2744141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2743fff640 1 -- 172.21.15.26:0/3949957017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274413f7d0 con 0x7f2744145910 2024-04-24T23:22:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2743fff640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2744145910 0x7f27441f5140 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f273c00efc0 tx=0x7f273c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f272bfff640 1 -- 172.21.15.26:0/3949957017 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c018030 con 0x7f2744145910 2024-04-24T23:22:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f272bfff640 1 -- 172.21.15.26:0/3949957017 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c00cde0 con 0x7f2744145910 2024-04-24T23:22:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274413fab0 con 0x7f2744145910 2024-04-24T23:22:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.516+0000 7f272bfff640 1 -- 172.21.15.26:0/3949957017 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c003bb0 con 0x7f2744145910 2024-04-24T23:22:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.517+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274413e9f0 con 0x7f2744145910 2024-04-24T23:22:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.517+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27440694c0 con 0x7f2744145910 2024-04-24T23:22:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.518+0000 7f272bfff640 1 -- 172.21.15.26:0/3949957017 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f273c01b070 con 0x7f2744145910 2024-04-24T23:22:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.518+0000 7f272bfff640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f273006be30 0x7f273006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.518+0000 7f272bfff640 1 -- 172.21.15.26:0/3949957017 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f273c08b740 con 0x7f2744145910 2024-04-24T23:22:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.518+0000 7f27437fe640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f273006be30 0x7f273006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.519+0000 7f27437fe640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f273006be30 0x7f273006e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2734034000 tx=0x7f273402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.523+0000 7f272bfff640 1 -- 172.21.15.26:0/3949957017 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f273c010040 con 0x7f2744145910 2024-04-24T23:22:51.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.745+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f27440008d0 con 0x7f2744145910 2024-04-24T23:22:51.746 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:22:51.745+0000 7f26a0f44640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:22:51.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.747+0000 7f272bfff640 1 -- 172.21.15.26:0/3949957017 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f273c059ab0 con 0x7f2744145910 2024-04-24T23:22:51.748 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f273006be30 msgr2=0x7f273006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f273006be30 0x7f273006e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2734034000 tx=0x7f273402e000 comp rx=0 tx=0).stop 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2744145910 msgr2=0x7f27441f5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2744145910 0x7f27441f5140 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f273c00efc0 tx=0x7f273c009f90 comp rx=0 tx=0).stop 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 shutdown_connections 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f273006be30 0x7f273006e2f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2744145910 0x7f27441f5140 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27440cd660 0x7f27441f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 --2- 172.21.15.26:0/3949957017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f274406d350 0x7f2744141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.749+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 >> 172.21.15.26:0/3949957017 conn(0x7f274406b300 msgr2=0x7f274415db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.750+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 shutdown_connections 2024-04-24T23:22:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.750+0000 7f2749c15640 1 -- 172.21.15.26:0/3949957017 wait complete. 2024-04-24T23:22:51.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:22:51.787 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:22:51.787 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:22:51.788 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:22:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.865+0000 7fc2f4e38640 1 Processor -- start 2024-04-24T23:22:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.865+0000 7fc2f4e38640 1 -- start start 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.866+0000 7fc2f4e38640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2f00ad7b0 0x7fc2f00d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.866+0000 7fc2f4e38640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f015ef60 con 0x7fc2f015e9c0 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.866+0000 7fc2f4e38640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f015f240 con 0x7fc2f015adc0 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.866+0000 7fc2f4e38640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f015f520 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.866+0000 7fc2edd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2f00ad7b0 0x7fc2f00d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.866+0000 7fc2edd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2f00ad7b0 0x7fc2f00d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60550/0 (socket says 172.21.15.26:60550) 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.866+0000 7fc2eed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc2f015e9c0 0x7fc2f0072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38322/0 (socket says 172.21.15.26:38322) 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.866+0000 7fc2edd9b640 1 -- 172.21.15.26:0/2917563704 learned_addr learned my addr 172.21.15.26:0/2917563704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.867+0000 7fc2edd9b640 1 -- 172.21.15.26:0/2917563704 >> v1:172.21.15.29:6789/0 conn(0x7fc2f015adc0 legacy=0x7fc2f00b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.867+0000 7fc2edd9b640 1 -- 172.21.15.26:0/2917563704 >> v1:172.21.15.26:6789/0 conn(0x7fc2f015e9c0 legacy=0x7fc2f0072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.867+0000 7fc2edd9b640 1 -- 172.21.15.26:0/2917563704 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f015f800 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.867+0000 7fc2edd9b640 1 --2- 172.21.15.26:0/2917563704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 0x7fc2f00d0040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80098a0 tx=0x7fc2d802c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36695e08a5944311 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.867+0000 7fc2ed59a640 1 -- 172.21.15.26:0/2917563704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8039070 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.867+0000 7fc2ed59a640 1 -- 172.21.15.26:0/2917563704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d8034640 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.867+0000 7fc2ed59a640 1 -- 172.21.15.26:0/2917563704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8034a10 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.868+0000 7fc2f4e38640 1 -- 172.21.15.26:0/2917563704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 msgr2=0x7fc2f00d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.868+0000 7fc2f4e38640 1 --2- 172.21.15.26:0/2917563704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 0x7fc2f00d0040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80098a0 tx=0x7fc2d802c3d0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.868+0000 7fc2f4e38640 1 -- 172.21.15.26:0/2917563704 shutdown_connections 2024-04-24T23:22:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.868+0000 7fc2f4e38640 1 --2- 172.21.15.26:0/2917563704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 0x7fc2f00d0040 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.868+0000 7fc2f4e38640 1 -- 172.21.15.26:0/2917563704 >> 172.21.15.26:0/2917563704 conn(0x7fc2f006b540 msgr2=0x7fc2f006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.868+0000 7fc2f4e38640 1 -- 172.21.15.26:0/2917563704 shutdown_connections 2024-04-24T23:22:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.868+0000 7fc2f4e38640 1 -- 172.21.15.26:0/2917563704 wait complete. 2024-04-24T23:22:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.869+0000 7fc2f4e38640 1 Processor -- start 2024-04-24T23:22:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2f4e38640 1 -- start start 2024-04-24T23:22:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2f4e38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 0x7fc2f00cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2f4e38640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2f015adc0 0x7fc2f009cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2f4e38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f015e9c0 0x7fc2f0142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2f4e38640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f00d0580 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2f4e38640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f015b1a0 con 0x7fc2f015e9c0 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2f4e38640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f00d0700 con 0x7fc2f015adc0 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2ee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 0x7fc2f00cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2ee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 0x7fc2f00cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60554/0 (socket says 172.21.15.26:60554) 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2ee59c640 1 -- 172.21.15.26:0/539823114 learned_addr learned my addr 172.21.15.26:0/539823114 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2eed9d640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f015e9c0 0x7fc2f0142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.870+0000 7fc2edd9b640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2f015adc0 0x7fc2f009cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2ee59c640 1 -- 172.21.15.26:0/539823114 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2f015adc0 msgr2=0x7fc2f009cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2ee59c640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2f015adc0 0x7fc2f009cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2ee59c640 1 -- 172.21.15.26:0/539823114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f015e9c0 msgr2=0x7fc2f0142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2ee59c640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f015e9c0 0x7fc2f0142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2ee59c640 1 -- 172.21.15.26:0/539823114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f009baf0 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2ee59c640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 0x7fc2f00cba90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c430 tx=0x7fc2e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2e6ffd640 1 -- 172.21.15.26:0/539823114 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8011790 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2e6ffd640 1 -- 172.21.15.26:0/539823114 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e8017070 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2e6ffd640 1 -- 172.21.15.26:0/539823114 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8011bb0 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f009bdd0 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.871+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f009c390 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.872+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f00b31f0 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.872+0000 7fc2e6ffd640 1 -- 172.21.15.26:0/539823114 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2e801c460 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.873+0000 7fc2e6ffd640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2d006be30 0x7fc2d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.873+0000 7fc2e6ffd640 1 -- 172.21.15.26:0/539823114 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5967+0+0 (secure 0 0 0) 0x7fc2e808ad00 con 0x7fc2f00ad7b0 2024-04-24T23:22:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.873+0000 7fc2edd9b640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2d006be30 0x7fc2d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.873+0000 7fc2edd9b640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2d006be30 0x7fc2d006e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8036000 tx=0x7fc2d8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:51.877+0000 7fc2e6ffd640 1 -- 172.21.15.26:0/539823114 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc2f00b31f0 con 0x7fc2f00ad7b0 2024-04-24T23:22:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.103+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc2f015e730 con 0x7fc2f00ad7b0 2024-04-24T23:22:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.748+0000 7fc2e6ffd640 1 -- 172.21.15.26:0/539823114 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7fc2e8058f60 con 0x7fc2f00ad7b0 2024-04-24T23:22:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2d006be30 msgr2=0x7fc2d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2d006be30 0x7fc2d006e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8036000 tx=0x7fc2d8030000 comp rx=0 tx=0).stop 2024-04-24T23:22:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 msgr2=0x7fc2f00cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 0x7fc2f00cba90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c430 tx=0x7fc2e800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 shutdown_connections 2024-04-24T23:22:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2d006be30 0x7fc2d006e2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2f015e9c0 0x7fc2f0142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2f015adc0 0x7fc2f009cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 --2- 172.21.15.26:0/539823114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f00ad7b0 0x7fc2f00cba90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 >> 172.21.15.26:0/539823114 conn(0x7fc2f006b540 msgr2=0x7fc2f006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.751+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 shutdown_connections 2024-04-24T23:22:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.752+0000 7fc2f4e38640 1 -- 172.21.15.26:0/539823114 wait complete. 2024-04-24T23:22:52.767 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:22:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.916+0000 7fcb4e6a5640 1 Processor -- start 2024-04-24T23:22:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.916+0000 7fcb4e6a5640 1 -- start start 2024-04-24T23:22:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.917+0000 7fcb4e6a5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcb4809aa00 0x7fcb480d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.917+0000 7fcb4e6a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb4809ade0 0x7fcb4815db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.917+0000 7fcb4e6a5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb4806f650 con 0x7fcb4815e0a0 2024-04-24T23:22:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.917+0000 7fcb4e6a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fcb480706f0 con 0x7fcb4809ade0 2024-04-24T23:22:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.917+0000 7fcb4e6a5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcb48070260 con 0x7fcb4809aa00 2024-04-24T23:22:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.917+0000 7fcb477fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb4809ade0 0x7fcb4815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.917+0000 7fcb477fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb4809ade0 0x7fcb4815db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60556/0 (socket says 172.21.15.26:60556) 2024-04-24T23:22:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.917+0000 7fcb477fe640 1 -- 172.21.15.26:0/1998758590 learned_addr learned my addr 172.21.15.26:0/1998758590 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.918+0000 7fcb477fe640 1 -- 172.21.15.26:0/1998758590 >> v2:172.21.15.29:3300/0 conn(0x7fcb4809aa00 msgr2=0x7fcb480d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.918+0000 7fcb477fe640 1 --2- 172.21.15.26:0/1998758590 >> v2:172.21.15.29:3300/0 conn(0x7fcb4809aa00 0x7fcb480d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.918+0000 7fcb477fe640 1 -- 172.21.15.26:0/1998758590 >> v1:172.21.15.29:6789/0 conn(0x7fcb4815e0a0 legacy=0x7fcb48161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.918+0000 7fcb477fe640 1 -- 172.21.15.26:0/1998758590 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4806f930 con 0x7fcb4809ade0 2024-04-24T23:22:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.918+0000 7fcb477fe640 1 --2- 172.21.15.26:0/1998758590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809ade0 0x7fcb4815db60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcb48072830 tx=0x7fcb3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb03413c9981a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.918+0000 7fcb46ffd640 1 -- 172.21.15.26:0/1998758590 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38037070 con 0x7fcb4809ade0 2024-04-24T23:22:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.918+0000 7fcb46ffd640 1 -- 172.21.15.26:0/1998758590 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb38032440 con 0x7fcb4809ade0 2024-04-24T23:22:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.918+0000 7fcb46ffd640 1 -- 172.21.15.26:0/1998758590 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38032810 con 0x7fcb4809ade0 2024-04-24T23:22:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.919+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/1998758590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809ade0 msgr2=0x7fcb4815db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.919+0000 7fcb4e6a5640 1 --2- 172.21.15.26:0/1998758590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809ade0 0x7fcb4815db60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcb48072830 tx=0x7fcb3802a230 comp rx=0 tx=0).stop 2024-04-24T23:22:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.919+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/1998758590 shutdown_connections 2024-04-24T23:22:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.919+0000 7fcb4e6a5640 1 --2- 172.21.15.26:0/1998758590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809ade0 0x7fcb4815db60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.919+0000 7fcb4e6a5640 1 --2- 172.21.15.26:0/1998758590 >> v2:172.21.15.29:3300/0 conn(0x7fcb4809aa00 0x7fcb480d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.919+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/1998758590 >> 172.21.15.26:0/1998758590 conn(0x7fcb480b39e0 msgr2=0x7fcb480b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.919+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/1998758590 shutdown_connections 2024-04-24T23:22:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.919+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/1998758590 wait complete. 2024-04-24T23:22:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.920+0000 7fcb4e6a5640 1 Processor -- start 2024-04-24T23:22:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.920+0000 7fcb4e6a5640 1 -- start start 2024-04-24T23:22:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb4e6a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809aa00 0x7fcb480cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb4e6a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb4809ade0 0x7fcb481f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb4e6a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcb4815e0a0 0x7fcb480afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb4e6a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4809b220 con 0x7fcb4809aa00 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb4e6a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4806fef0 con 0x7fcb4809ade0 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb4e6a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fcb480ca700 con 0x7fcb4815e0a0 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809aa00 0x7fcb480cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809aa00 0x7fcb480cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60566/0 (socket says 172.21.15.26:60566) 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb47fff640 1 -- 172.21.15.26:0/4028313565 learned_addr learned my addr 172.21.15.26:0/4028313565 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb477fe640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb4809ade0 0x7fcb481f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.921+0000 7fcb4cc42640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcb4815e0a0 0x7fcb480afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb47fff640 1 -- 172.21.15.26:0/4028313565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcb4815e0a0 msgr2=0x7fcb480afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb47fff640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcb4815e0a0 0x7fcb480afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb47fff640 1 -- 172.21.15.26:0/4028313565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb4809ade0 msgr2=0x7fcb481f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb47fff640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb4809ade0 0x7fcb481f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb47fff640 1 -- 172.21.15.26:0/4028313565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb480c9f90 con 0x7fcb4809aa00 2024-04-24T23:22:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb4cc42640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcb4815e0a0 0x7fcb480afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb47fff640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809aa00 0x7fcb480cc430 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00b410 tx=0x7fcb3c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb44ff9640 1 -- 172.21.15.26:0/4028313565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c015020 con 0x7fcb4809aa00 2024-04-24T23:22:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb44ff9640 1 -- 172.21.15.26:0/4028313565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c011850 con 0x7fcb4809aa00 2024-04-24T23:22:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb480ca970 con 0x7fcb4809aa00 2024-04-24T23:22:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb44ff9640 1 -- 172.21.15.26:0/4028313565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c011c40 con 0x7fcb4809aa00 2024-04-24T23:22:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.922+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb480caf30 con 0x7fcb4809aa00 2024-04-24T23:22:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.923+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb480694c0 con 0x7fcb4809aa00 2024-04-24T23:22:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.923+0000 7fcb44ff9640 1 -- 172.21.15.26:0/4028313565 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb3c018020 con 0x7fcb4809aa00 2024-04-24T23:22:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.924+0000 7fcb44ff9640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcb3406be30 0x7fcb3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.924+0000 7fcb44ff9640 1 -- 172.21.15.26:0/4028313565 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5967+0+0 (secure 0 0 0) 0x7fcb3c08af00 con 0x7fcb4809aa00 2024-04-24T23:22:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.924+0000 7fcb477fe640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcb3406be30 0x7fcb3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.924+0000 7fcb477fe640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcb3406be30 0x7fcb3406e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb38034000 tx=0x7fcb3802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:52.928+0000 7fcb44ff9640 1 -- 172.21.15.26:0/4028313565 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcb3c01f020 con 0x7fcb4809aa00 2024-04-24T23:22:53.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.115+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcb480b1680 con 0x7fcb4809aa00 2024-04-24T23:22:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.752+0000 7fcb44ff9640 1 -- 172.21.15.26:0/4028313565 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7fcb3c0172c0 con 0x7fcb4809aa00 2024-04-24T23:22:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcb3406be30 msgr2=0x7fcb3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcb3406be30 0x7fcb3406e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb38034000 tx=0x7fcb3802e000 comp rx=0 tx=0).stop 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809aa00 msgr2=0x7fcb480cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809aa00 0x7fcb480cc430 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00b410 tx=0x7fcb3c00b8d0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 shutdown_connections 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcb3406be30 0x7fcb3406e2f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcb4815e0a0 0x7fcb480afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb4809ade0 0x7fcb481f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 --2- 172.21.15.26:0/4028313565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb4809aa00 0x7fcb480cc430 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 >> 172.21.15.26:0/4028313565 conn(0x7fcb480b39e0 msgr2=0x7fcb480b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 shutdown_connections 2024-04-24T23:22:53.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.755+0000 7fcb4e6a5640 1 -- 172.21.15.26:0/4028313565 wait complete. 2024-04-24T23:22:53.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.879+0000 7f19d3615640 1 Processor -- start 2024-04-24T23:22:53.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.880+0000 7f19d3615640 1 -- start start 2024-04-24T23:22:53.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.880+0000 7f19d3615640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19cc15e330 0x7f19cc0d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.880+0000 7f19d3615640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19cc15e710 0x7f19cc0d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.880+0000 7f19d3615640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19cc0cfcc0 0x7f19cc0d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.880+0000 7f19d3615640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f19cc0d0170 con 0x7f19cc15e330 2024-04-24T23:22:53.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.880+0000 7f19d3615640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f19cc0d27b0 con 0x7f19cc15e710 2024-04-24T23:22:53.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.880+0000 7f19d3615640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f19cc0cef00 con 0x7f19cc0cfcc0 2024-04-24T23:22:53.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.880+0000 7f19d13b1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19cc15e330 0x7f19cc0d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.880+0000 7f19d13b1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19cc15e330 0x7f19cc0d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33308/0 (socket says 172.21.15.26:33308) 2024-04-24T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.881+0000 7f19d13b1640 1 -- 172.21.15.26:0/3513564778 learned_addr learned my addr 172.21.15.26:0/3513564778 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.881+0000 7f19d13b1640 1 -- 172.21.15.26:0/3513564778 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19cc0cfcc0 msgr2=0x7f19cc0d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.881+0000 7f19d13b1640 1 --2- 172.21.15.26:0/3513564778 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19cc0cfcc0 0x7f19cc0d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.881+0000 7f19d13b1640 1 -- 172.21.15.26:0/3513564778 >> v2:172.21.15.29:3300/0 conn(0x7f19cc15e710 msgr2=0x7f19cc0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.881+0000 7f19d13b1640 1 --2- 172.21.15.26:0/3513564778 >> v2:172.21.15.29:3300/0 conn(0x7f19cc15e710 0x7f19cc0d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.881+0000 7f19d13b1640 1 -- 172.21.15.26:0/3513564778 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc0cf1e0 con 0x7f19cc15e330 2024-04-24T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.881+0000 7f19d13b1640 1 --2- 172.21.15.26:0/3513564778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e330 0x7f19cc0d20b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0d1ce0 tx=0x7f19c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671566fd3fe4ee1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19cbfff640 1 -- 172.21.15.26:0/3513564778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4037070 con 0x7f19cc15e330 2024-04-24T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19cbfff640 1 -- 172.21.15.26:0/3513564778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c4032440 con 0x7f19cc15e330 2024-04-24T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19cbfff640 1 -- 172.21.15.26:0/3513564778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4032810 con 0x7f19cc15e330 2024-04-24T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19d3615640 1 -- 172.21.15.26:0/3513564778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e330 msgr2=0x7f19cc0d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19d3615640 1 --2- 172.21.15.26:0/3513564778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e330 0x7f19cc0d20b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0d1ce0 tx=0x7f19c402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19d3615640 1 -- 172.21.15.26:0/3513564778 shutdown_connections 2024-04-24T23:22:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19d3615640 1 --2- 172.21.15.26:0/3513564778 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19cc0cfcc0 0x7f19cc0d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19d3615640 1 --2- 172.21.15.26:0/3513564778 >> v2:172.21.15.29:3300/0 conn(0x7f19cc15e710 0x7f19cc0d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19d3615640 1 --2- 172.21.15.26:0/3513564778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e330 0x7f19cc0d20b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19d3615640 1 -- 172.21.15.26:0/3513564778 >> 172.21.15.26:0/3513564778 conn(0x7f19cc08dbc0 msgr2=0x7f19cc09d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19d3615640 1 -- 172.21.15.26:0/3513564778 shutdown_connections 2024-04-24T23:22:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.882+0000 7f19d3615640 1 -- 172.21.15.26:0/3513564778 wait complete. 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.884+0000 7f19d3615640 1 Processor -- start 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.884+0000 7f19d3615640 1 -- start start 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.884+0000 7f19d3615640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19cc0cfcc0 0x7f19cc14bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.884+0000 7f19d3615640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19cc15e330 0x7f19cc15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.884+0000 7f19d3615640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e710 0x7f19cc14af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.884+0000 7f19d3615640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc0d4460 con 0x7f19cc15e710 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.884+0000 7f19d3615640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc0cf7a0 con 0x7f19cc15e330 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.884+0000 7f19d3615640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f19cc0d45e0 con 0x7f19cc0cfcc0 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d1bb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e710 0x7f19cc14af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d1bb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e710 0x7f19cc14af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33320/0 (socket says 172.21.15.26:33320) 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d1bb2640 1 -- 172.21.15.26:0/94758242 learned_addr learned my addr 172.21.15.26:0/94758242 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d0bb0640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19cc15e330 0x7f19cc15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d13b1640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19cc0cfcc0 0x7f19cc14bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d1bb2640 1 -- 172.21.15.26:0/94758242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19cc0cfcc0 msgr2=0x7f19cc14bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d1bb2640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19cc0cfcc0 0x7f19cc14bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d1bb2640 1 -- 172.21.15.26:0/94758242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19cc15e330 msgr2=0x7f19cc15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d1bb2640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19cc15e330 0x7f19cc15edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d1bb2640 1 -- 172.21.15.26:0/94758242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc0c8e60 con 0x7f19cc15e710 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d1bb2640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e710 0x7f19cc14af90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00b9a0 tx=0x7f19bc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19c9ffb640 1 -- 172.21.15.26:0/94758242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc019070 con 0x7f19cc15e710 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19c9ffb640 1 -- 172.21.15.26:0/94758242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19bc00cbc0 con 0x7f19cc15e710 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19cc0c9140 con 0x7f19cc15e710 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.885+0000 7f19c9ffb640 1 -- 172.21.15.26:0/94758242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc014440 con 0x7f19cc15e710 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.886+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19cc0c9700 con 0x7f19cc15e710 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.887+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19cc0694c0 con 0x7f19cc15e710 2024-04-24T23:22:53.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.887+0000 7f19c9ffb640 1 -- 172.21.15.26:0/94758242 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19bc01c070 con 0x7f19cc15e710 2024-04-24T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.887+0000 7f19c9ffb640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19b806be30 0x7f19b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.887+0000 7f19c9ffb640 1 -- 172.21.15.26:0/94758242 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5529+0+0 (secure 0 0 0) 0x7f19bc08aa50 con 0x7f19cc15e710 2024-04-24T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.887+0000 7f19d13b1640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19b806be30 0x7f19b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.888+0000 7f19d13b1640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19b806be30 0x7f19b806e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19c4034000 tx=0x7f19c402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:53.891+0000 7f19c9ffb640 1 -- 172.21.15.26:0/94758242 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f19bc058e60 con 0x7f19cc15e710 2024-04-24T23:22:54.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.074+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19cc0cb040 con 0x7f19cc15e710 2024-04-24T23:22:54.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.754+0000 7f19c9ffb640 1 -- 172.21.15.26:0/94758242 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7f19bc05d160 con 0x7f19cc15e710 2024-04-24T23:22:54.756 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-24T23:22:54.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19b806be30 msgr2=0x7f19b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19b806be30 0x7f19b806e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19c4034000 tx=0x7f19c402e000 comp rx=0 tx=0).stop 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e710 msgr2=0x7f19cc14af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e710 0x7f19cc14af90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00b9a0 tx=0x7f19bc00be60 comp rx=0 tx=0).stop 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 shutdown_connections 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19b806be30 0x7f19b806e2f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19cc15e710 0x7f19cc14af90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19cc15e330 0x7f19cc15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 --2- 172.21.15.26:0/94758242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19cc0cfcc0 0x7f19cc14bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 >> 172.21.15.26:0/94758242 conn(0x7f19cc08dbc0 msgr2=0x7f19cc0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 shutdown_connections 2024-04-24T23:22:54.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.758+0000 7f19d3615640 1 -- 172.21.15.26:0/94758242 wait complete. 2024-04-24T23:22:54.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:22:54.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.881+0000 7f72c6170640 1 Processor -- start 2024-04-24T23:22:54.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.881+0000 7f72c6170640 1 -- start start 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72c6170640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f72c015be40 0x7f72c00acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72c6170640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72c00aeb10 con 0x7f72c00b4ce0 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72c6170640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72c00aedf0 con 0x7f72c00d0750 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72c6170640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f72c009df20 con 0x7f72c015be40 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72bf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f72c015be40 0x7f72c00acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72beffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f72c00b4ce0 0x7f72c00d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51286/0 (socket says 172.21.15.26:51286) 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72bf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f72c015be40 0x7f72c00acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33328/0 (socket says 172.21.15.26:33328) 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72beffd640 1 -- 172.21.15.26:0/3414913252 learned_addr learned my addr 172.21.15.26:0/3414913252 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72bf7fe640 1 -- 172.21.15.26:0/3414913252 >> v1:172.21.15.29:6789/0 conn(0x7f72c00d0750 legacy=0x7f72c015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72bf7fe640 1 -- 172.21.15.26:0/3414913252 >> v1:172.21.15.26:6789/0 conn(0x7f72c00b4ce0 legacy=0x7f72c00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72bf7fe640 1 -- 172.21.15.26:0/3414913252 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c00af0d0 con 0x7f72c015be40 2024-04-24T23:22:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.882+0000 7f72be7fc640 1 -- 172.21.15.26:0/3414913252 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085751359 0 0) 0x7f72c00aeb10 con 0x7f72c00b4ce0 2024-04-24T23:22:54.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.883+0000 7f72bf7fe640 1 --2- 172.21.15.26:0/3414913252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c015be40 0x7f72c00acd30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f72b00097a0 tx=0x7f72b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d2ccef25d143dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.883+0000 7f72be7fc640 1 -- 172.21.15.26:0/3414913252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0037070 con 0x7f72c015be40 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.883+0000 7f72be7fc640 1 -- 172.21.15.26:0/3414913252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b0032440 con 0x7f72c015be40 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.883+0000 7f72be7fc640 1 -- 172.21.15.26:0/3414913252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0032810 con 0x7f72c015be40 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.884+0000 7f72c6170640 1 -- 172.21.15.26:0/3414913252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c015be40 msgr2=0x7f72c00acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.884+0000 7f72c6170640 1 --2- 172.21.15.26:0/3414913252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c015be40 0x7f72c00acd30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f72b00097a0 tx=0x7f72b002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.884+0000 7f72c6170640 1 -- 172.21.15.26:0/3414913252 shutdown_connections 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.884+0000 7f72c6170640 1 --2- 172.21.15.26:0/3414913252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c015be40 0x7f72c00acd30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.884+0000 7f72c6170640 1 -- 172.21.15.26:0/3414913252 >> 172.21.15.26:0/3414913252 conn(0x7f72c006b540 msgr2=0x7f72c006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.884+0000 7f72c6170640 1 -- 172.21.15.26:0/3414913252 shutdown_connections 2024-04-24T23:22:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.884+0000 7f72c6170640 1 -- 172.21.15.26:0/3414913252 wait complete. 2024-04-24T23:22:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.885+0000 7f72c6170640 1 Processor -- start 2024-04-24T23:22:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.885+0000 7f72c6170640 1 -- start start 2024-04-24T23:22:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72c6170640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c00b4ce0 0x7f72c0156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72c6170640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c00d0750 0x7f72c0146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72c6170640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72c015be40 0x7f72c01f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72c6170640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f72c006cd10 con 0x7f72c00d0750 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72c6170640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f72c006cb90 con 0x7f72c00b4ce0 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72c6170640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f72c015c930 con 0x7f72c015be40 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72beffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c00d0750 0x7f72c0146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72beffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c00d0750 0x7f72c0146730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33340/0 (socket says 172.21.15.26:33340) 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72beffd640 1 -- 172.21.15.26:0/1624166460 learned_addr learned my addr 172.21.15.26:0/1624166460 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72beffd640 1 -- 172.21.15.26:0/1624166460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72c015be40 msgr2=0x7f72c01f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72beffd640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72c015be40 0x7f72c01f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72beffd640 1 -- 172.21.15.26:0/1624166460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c00b4ce0 msgr2=0x7f72c0156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72beffd640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c00b4ce0 0x7f72c0156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72beffd640 1 -- 172.21.15.26:0/1624166460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c0146c70 con 0x7f72c00d0750 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.886+0000 7f72beffd640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c00d0750 0x7f72c0146730 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f72b400c4d0 tx=0x7f72b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.887+0000 7f72a7fff640 1 -- 172.21.15.26:0/1624166460 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4018070 con 0x7f72c00d0750 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.887+0000 7f72a7fff640 1 -- 172.21.15.26:0/1624166460 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b4007500 con 0x7f72c00d0750 2024-04-24T23:22:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.887+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c01f2120 con 0x7f72c00d0750 2024-04-24T23:22:54.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.887+0000 7f72a7fff640 1 -- 172.21.15.26:0/1624166460 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4004a20 con 0x7f72c00d0750 2024-04-24T23:22:54.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.887+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c009c150 con 0x7f72c00d0750 2024-04-24T23:22:54.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.888+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c0075610 con 0x7f72c00d0750 2024-04-24T23:22:54.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.888+0000 7f72a7fff640 1 -- 172.21.15.26:0/1624166460 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72b401b070 con 0x7f72c00d0750 2024-04-24T23:22:54.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.889+0000 7f72a7fff640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72ac06be30 0x7f72ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:54.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.889+0000 7f72a7fff640 1 -- 172.21.15.26:0/1624166460 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f72b4089ed0 con 0x7f72c00d0750 2024-04-24T23:22:54.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.889+0000 7f72bf7fe640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72ac06be30 0x7f72ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:54.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.889+0000 7f72bf7fe640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72ac06be30 0x7f72ac06e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f72b0034000 tx=0x7f72b002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:54.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:54.893+0000 7f72a7fff640 1 -- 172.21.15.26:0/1624166460 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f72b4058b60 con 0x7f72c00d0750 2024-04-24T23:22:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.086+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f72c015f260 con 0x7f72c00d0750 2024-04-24T23:22:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.089+0000 7f72a7fff640 1 -- 172.21.15.26:0/1624166460 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f72b4058460 con 0x7f72c00d0750 2024-04-24T23:22:55.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72ac06be30 msgr2=0x7f72ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72ac06be30 0x7f72ac06e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f72b0034000 tx=0x7f72b002c000 comp rx=0 tx=0).stop 2024-04-24T23:22:55.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c00d0750 msgr2=0x7f72c0146730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c00d0750 0x7f72c0146730 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f72b400c4d0 tx=0x7f72b400c990 comp rx=0 tx=0).stop 2024-04-24T23:22:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 shutdown_connections 2024-04-24T23:22:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72ac06be30 0x7f72ac06e2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72c015be40 0x7f72c01f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72c00d0750 0x7f72c0146730 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 --2- 172.21.15.26:0/1624166460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72c00b4ce0 0x7f72c0156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 >> 172.21.15.26:0/1624166460 conn(0x7f72c006b540 msgr2=0x7f72c006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 shutdown_connections 2024-04-24T23:22:55.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.091+0000 7f72c6170640 1 -- 172.21.15.26:0/1624166460 wait complete. 2024-04-24T23:22:55.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-04-24T23:22:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.203+0000 7fe1edb14640 1 Processor -- start 2024-04-24T23:22:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.203+0000 7fe1edb14640 1 -- start start 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1edb14640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe1e8145910 0x7fe1e806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1edb14640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e806d350 0x7fe1e815d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1edb14640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1e80cd660 0x7fe1e80cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1edb14640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e80cdff0 con 0x7fe1e80cd660 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1edb14640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e8145cf0 con 0x7fe1e8145910 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1edb14640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e815d570 con 0x7fe1e806d350 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1e80cd660 0x7fe1e80cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1e80cd660 0x7fe1e80cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33356/0 (socket says 172.21.15.26:33356) 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e7fff640 1 -- 172.21.15.26:0/596953211 learned_addr learned my addr 172.21.15.26:0/596953211 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e6ffd640 1 --2- 172.21.15.26:0/596953211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e806d350 0x7fe1e815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e77fe640 1 --2- 172.21.15.26:0/596953211 >> v2:172.21.15.29:3300/0 conn(0x7fe1e8145910 0x7fe1e806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e7fff640 1 -- 172.21.15.26:0/596953211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e806d350 msgr2=0x7fe1e815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e7fff640 1 --2- 172.21.15.26:0/596953211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e806d350 0x7fe1e815d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e7fff640 1 -- 172.21.15.26:0/596953211 >> v2:172.21.15.29:3300/0 conn(0x7fe1e8145910 msgr2=0x7fe1e806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e7fff640 1 --2- 172.21.15.26:0/596953211 >> v2:172.21.15.29:3300/0 conn(0x7fe1e8145910 0x7fe1e806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e7fff640 1 -- 172.21.15.26:0/596953211 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e80cfbc0 con 0x7fe1e80cd660 2024-04-24T23:22:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e6ffd640 1 --2- 172.21.15.26:0/596953211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e806d350 0x7fe1e815d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.204+0000 7fe1e77fe640 1 --2- 172.21.15.26:0/596953211 >> v2:172.21.15.29:3300/0 conn(0x7fe1e8145910 0x7fe1e806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.205+0000 7fe1e7fff640 1 --2- 172.21.15.26:0/596953211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e80cd660 0x7fe1e80cdab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe1e806f040 tx=0x7fe1e002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102bbf248f75404a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.205+0000 7fe1e67fc640 1 -- 172.21.15.26:0/596953211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0036070 con 0x7fe1e80cd660 2024-04-24T23:22:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.205+0000 7fe1e67fc640 1 -- 172.21.15.26:0/596953211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e0031600 con 0x7fe1e80cd660 2024-04-24T23:22:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.205+0000 7fe1e67fc640 1 -- 172.21.15.26:0/596953211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00319d0 con 0x7fe1e80cd660 2024-04-24T23:22:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.206+0000 7fe1edb14640 1 -- 172.21.15.26:0/596953211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e80cd660 msgr2=0x7fe1e80cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.206+0000 7fe1edb14640 1 --2- 172.21.15.26:0/596953211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e80cd660 0x7fe1e80cdab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe1e806f040 tx=0x7fe1e002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.206+0000 7fe1edb14640 1 -- 172.21.15.26:0/596953211 shutdown_connections 2024-04-24T23:22:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.206+0000 7fe1edb14640 1 --2- 172.21.15.26:0/596953211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e80cd660 0x7fe1e80cdab0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.206+0000 7fe1edb14640 1 --2- 172.21.15.26:0/596953211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e806d350 0x7fe1e815d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.206+0000 7fe1edb14640 1 --2- 172.21.15.26:0/596953211 >> v2:172.21.15.29:3300/0 conn(0x7fe1e8145910 0x7fe1e806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.206+0000 7fe1edb14640 1 -- 172.21.15.26:0/596953211 >> 172.21.15.26:0/596953211 conn(0x7fe1e806b300 msgr2=0x7fe1e806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.206+0000 7fe1edb14640 1 -- 172.21.15.26:0/596953211 shutdown_connections 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.206+0000 7fe1edb14640 1 -- 172.21.15.26:0/596953211 wait complete. 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.207+0000 7fe1edb14640 1 Processor -- start 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.207+0000 7fe1edb14640 1 -- start start 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1edb14640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1e806d350 0x7fe1e815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1edb14640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e80cd660 0x7fe1e81330c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1edb14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e8145910 0x7fe1e80b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1edb14640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e806ddb0 con 0x7fe1e8145910 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1edb14640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e806dad0 con 0x7fe1e806d350 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1edb14640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e806e090 con 0x7fe1e80cd660 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e8145910 0x7fe1e80b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e8145910 0x7fe1e80b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33364/0 (socket says 172.21.15.26:33364) 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1e7fff640 1 -- 172.21.15.26:0/3243012706 learned_addr learned my addr 172.21.15.26:0/3243012706 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1e77fe640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1e806d350 0x7fe1e815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1e6ffd640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e80cd660 0x7fe1e81330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1e7fff640 1 -- 172.21.15.26:0/3243012706 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e80cd660 msgr2=0x7fe1e81330c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1e7fff640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e80cd660 0x7fe1e81330c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1e7fff640 1 -- 172.21.15.26:0/3243012706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1e806d350 msgr2=0x7fe1e815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.208+0000 7fe1e7fff640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1e806d350 0x7fe1e815aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.209+0000 7fe1e7fff640 1 -- 172.21.15.26:0/3243012706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e80b33c0 con 0x7fe1e8145910 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.209+0000 7fe1e7fff640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e8145910 0x7fe1e80b4460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009f40 tx=0x7fe1e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.209+0000 7fe1cffff640 1 -- 172.21.15.26:0/3243012706 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0036050 con 0x7fe1e8145910 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.209+0000 7fe1cffff640 1 -- 172.21.15.26:0/3243012706 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e0042070 con 0x7fe1e8145910 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.209+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e80b36a0 con 0x7fe1e8145910 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.209+0000 7fe1cffff640 1 -- 172.21.15.26:0/3243012706 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0009510 con 0x7fe1e8145910 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.209+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e80b3c60 con 0x7fe1e8145910 2024-04-24T23:22:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.210+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e80694c0 con 0x7fe1e8145910 2024-04-24T23:22:55.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.210+0000 7fe1cffff640 1 -- 172.21.15.26:0/3243012706 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1e0007bd0 con 0x7fe1e8145910 2024-04-24T23:22:55.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.211+0000 7fe1cffff640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe1d406be30 0x7fe1d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.211+0000 7fe1cffff640 1 -- 172.21.15.26:0/3243012706 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7fe1e00abb60 con 0x7fe1e8145910 2024-04-24T23:22:55.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.211+0000 7fe1e77fe640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe1d406be30 0x7fe1d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.211+0000 7fe1e77fe640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe1d406be30 0x7fe1d406e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006fd0 tx=0x7fe1d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:55.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.215+0000 7fe1cffff640 1 -- 172.21.15.26:0/3243012706 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1e007a6d0 con 0x7fe1e8145910 2024-04-24T23:22:55.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.472+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) -- 0x7fe1e806baf0 con 0x7fe1e8145910 2024-04-24T23:22:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.761+0000 7fe1cffff640 1 -- 172.21.15.26:0/3243012706 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v119) ==== 136+0+0 (secure 0 0 0) 0x7fe1e0079fd0 con 0x7fe1e8145910 2024-04-24T23:22:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe1d406be30 msgr2=0x7fe1d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe1d406be30 0x7fe1d406e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006fd0 tx=0x7fe1d8008040 comp rx=0 tx=0).stop 2024-04-24T23:22:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e8145910 msgr2=0x7fe1e80b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e8145910 0x7fe1e80b4460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009f40 tx=0x7fe1e0004a40 comp rx=0 tx=0).stop 2024-04-24T23:22:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 shutdown_connections 2024-04-24T23:22:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe1d406be30 0x7fe1d406e2f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1e8145910 0x7fe1e80b4460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1e80cd660 0x7fe1e81330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 --2- 172.21.15.26:0/3243012706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1e806d350 0x7fe1e815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 >> 172.21.15.26:0/3243012706 conn(0x7fe1e806b300 msgr2=0x7fe1e8142c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:55.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 shutdown_connections 2024-04-24T23:22:55.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.764+0000 7fe1edb14640 1 -- 172.21.15.26:0/3243012706 wait complete. 2024-04-24T23:22:55.777 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-04-24T23:22:55.778 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-04-24T23:22:55.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:22:55.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.879+0000 7f6583015640 1 Processor -- start 2024-04-24T23:22:55.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.879+0000 7f6583015640 1 -- start start 2024-04-24T23:22:55.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.880+0000 7f6583015640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f657c15e570 0x7f657c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.880+0000 7f6583015640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0d22e0 0x7f657c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.880+0000 7f6583015640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f657c0d32f0 con 0x7f657c0b3130 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.880+0000 7f6583015640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f657c0d1ab0 con 0x7f657c15e570 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.880+0000 7f6583015640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f657c0d2ae0 con 0x7f657c0d22e0 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f65815b2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0d22e0 0x7f657c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f6580db1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f657c15e570 0x7f657c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f657bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f657c0b3130 0x7f657c0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:41436/0 (socket says 172.21.15.26:41436) 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f6580db1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f657c15e570 0x7f657c0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:52032/0 (socket says 172.21.15.26:52032) 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f657bfff640 1 -- 172.21.15.26:0/2384438853 learned_addr learned my addr 172.21.15.26:0/2384438853 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f65815b2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0d22e0 0x7f657c0d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:59024/0 (socket says 172.21.15.26:59024) 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f6580db1640 1 -- 172.21.15.26:0/2384438853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0d22e0 msgr2=0x7f657c0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f6580db1640 1 --2- 172.21.15.26:0/2384438853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0d22e0 0x7f657c0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f6580db1640 1 -- 172.21.15.26:0/2384438853 >> v1:172.21.15.29:6789/0 conn(0x7f657c0b3130 legacy=0x7f657c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f6580db1640 1 -- 172.21.15.26:0/2384438853 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657c0d0270 con 0x7f657c15e570 2024-04-24T23:22:55.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.881+0000 7f657b7fe640 1 -- 172.21.15.26:0/2384438853 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19622649 0 0) 0x7f6570001110 con 0x7f657c0b3130 2024-04-24T23:22:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.882+0000 7f6580db1640 1 --2- 172.21.15.26:0/2384438853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f657c15e570 0x7f657c0d4160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6574009d90 tx=0x7f657402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5cd9e99459b2c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.882+0000 7f657b7fe640 1 -- 172.21.15.26:0/2384438853 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574007740 con 0x7f657c15e570 2024-04-24T23:22:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.882+0000 7f657b7fe640 1 -- 172.21.15.26:0/2384438853 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65740078e0 con 0x7f657c15e570 2024-04-24T23:22:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.882+0000 7f657b7fe640 1 -- 172.21.15.26:0/2384438853 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574007cb0 con 0x7f657c15e570 2024-04-24T23:22:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.883+0000 7f6583015640 1 -- 172.21.15.26:0/2384438853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f657c15e570 msgr2=0x7f657c0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.883+0000 7f6583015640 1 --2- 172.21.15.26:0/2384438853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f657c15e570 0x7f657c0d4160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6574009d90 tx=0x7f657402a660 comp rx=0 tx=0).stop 2024-04-24T23:22:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.883+0000 7f6583015640 1 -- 172.21.15.26:0/2384438853 shutdown_connections 2024-04-24T23:22:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.883+0000 7f6583015640 1 --2- 172.21.15.26:0/2384438853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0d22e0 0x7f657c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.883+0000 7f6583015640 1 --2- 172.21.15.26:0/2384438853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f657c15e570 0x7f657c0d4160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.883+0000 7f6583015640 1 -- 172.21.15.26:0/2384438853 >> 172.21.15.26:0/2384438853 conn(0x7f657c08dbe0 msgr2=0x7f657c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.883+0000 7f6583015640 1 -- 172.21.15.26:0/2384438853 shutdown_connections 2024-04-24T23:22:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.883+0000 7f6583015640 1 -- 172.21.15.26:0/2384438853 wait complete. 2024-04-24T23:22:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.884+0000 7f6583015640 1 Processor -- start 2024-04-24T23:22:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.884+0000 7f6583015640 1 -- start start 2024-04-24T23:22:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f6583015640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0b3130 0x7f657c0a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f6583015640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657c0d22e0 0x7f657c09df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f6583015640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f657c15e570 0x7f657c15cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f6583015640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f657c0cfa70 con 0x7f657c0d22e0 2024-04-24T23:22:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f6583015640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f657c0d3010 con 0x7f657c15e570 2024-04-24T23:22:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f6583015640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f657c0cfd50 con 0x7f657c0b3130 2024-04-24T23:22:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f657bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657c0d22e0 0x7f657c09df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f657bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657c0d22e0 0x7f657c09df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33366/0 (socket says 172.21.15.26:33366) 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f657bfff640 1 -- 172.21.15.26:0/350853935 learned_addr learned my addr 172.21.15.26:0/350853935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f6580db1640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0b3130 0x7f657c0a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f65815b2640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f657c15e570 0x7f657c15cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f657bfff640 1 -- 172.21.15.26:0/350853935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0b3130 msgr2=0x7f657c0a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f657bfff640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0b3130 0x7f657c0a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f657bfff640 1 -- 172.21.15.26:0/350853935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f657c15e570 msgr2=0x7f657c15cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f657bfff640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f657c15e570 0x7f657c15cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f657bfff640 1 -- 172.21.15.26:0/350853935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657c09e6e0 con 0x7f657c0d22e0 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f657bfff640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657c0d22e0 0x7f657c09df70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f657000c780 tx=0x7f657000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.885+0000 7f65797fa640 1 -- 172.21.15.26:0/350853935 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570018070 con 0x7f657c0d22e0 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.886+0000 7f65797fa640 1 -- 172.21.15.26:0/350853935 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65700072d0 con 0x7f657c0d22e0 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.886+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657c09e9c0 con 0x7f657c0d22e0 2024-04-24T23:22:55.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.886+0000 7f65797fa640 1 -- 172.21.15.26:0/350853935 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65700047e0 con 0x7f657c0d22e0 2024-04-24T23:22:55.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.886+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f657c09ef80 con 0x7f657c0d22e0 2024-04-24T23:22:55.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.887+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657c0694c0 con 0x7f657c0d22e0 2024-04-24T23:22:55.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.887+0000 7f65797fa640 1 -- 172.21.15.26:0/350853935 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f657001b070 con 0x7f657c0d22e0 2024-04-24T23:22:55.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.887+0000 7f65797fa640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f656806be30 0x7f656806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:55.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.888+0000 7f65797fa640 1 -- 172.21.15.26:0/350853935 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f6570089fb0 con 0x7f657c0d22e0 2024-04-24T23:22:55.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.888+0000 7f6580db1640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f656806be30 0x7f656806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:55.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.888+0000 7f6580db1640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f656806be30 0x7f656806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6574034000 tx=0x7f6574033040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:55.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:55.892+0000 7f65797fa640 1 -- 172.21.15.26:0/350853935 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f657c0d32f0 con 0x7f657c0d22e0 2024-04-24T23:22:56.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.078+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f657c0d1380 con 0x7f657c0d22e0 2024-04-24T23:22:56.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.079+0000 7f65797fa640 1 -- 172.21.15.26:0/350853935 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13218 (secure 0 0 0) 0x7f6570058bb0 con 0x7f657c0d22e0 2024-04-24T23:22:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:56.081 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:22:54.753758+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6836/315291888":"2024-04-25T23:22:51.746312+0000","172.21.15.26:6834/315291888":"2024-04-25T23:22:51.746312+0000","172.21.15.29:6834/2201738459":"2024-04-25T23:22:51.746294+0000","172.21.15.29:6832/2201738459":"2024-04-25T23:22:51.746294+0000","172.21.15.29:6835/3972561683":"2024-04-25T23:22:51.746307+0000","172.21.15.29:6833/3972561683":"2024-04-25T23:22:51.746307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:22:56.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.081+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f656806be30 msgr2=0x7f656806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.081+0000 7f6583015640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f656806be30 0x7f656806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6574034000 tx=0x7f6574033040 comp rx=0 tx=0).stop 2024-04-24T23:22:56.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657c0d22e0 msgr2=0x7f657c09df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657c0d22e0 0x7f657c09df70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f657000c780 tx=0x7f657000cc40 comp rx=0 tx=0).stop 2024-04-24T23:22:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 shutdown_connections 2024-04-24T23:22:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f656806be30 0x7f656806e2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f657c15e570 0x7f657c15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657c0d22e0 0x7f657c09df70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 --2- 172.21.15.26:0/350853935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f657c0b3130 0x7f657c0a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 >> 172.21.15.26:0/350853935 conn(0x7f657c08dbe0 msgr2=0x7f657c0d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 shutdown_connections 2024-04-24T23:22:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.082+0000 7f6583015640 1 -- 172.21.15.26:0/350853935 wait complete. 2024-04-24T23:22:56.095 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:22:56.095 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:22:56.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.197+0000 7f0377e8c640 1 Processor -- start 2024-04-24T23:22:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.197+0000 7f0377e8c640 1 -- start start 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0377e8c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f037015adc0 0x7f03700b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0377e8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03700ad7b0 0x7f03700d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0377e8c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f037015ef60 con 0x7f037015e9c0 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0377e8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f037015f240 con 0x7f03700ad7b0 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0377e8c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f037015f520 con 0x7f037015adc0 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0375427640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03700ad7b0 0x7f03700d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0375427640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03700ad7b0 0x7f03700d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33374/0 (socket says 172.21.15.26:33374) 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0375427640 1 -- 172.21.15.26:0/3583810608 learned_addr learned my addr 172.21.15.26:0/3583810608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0375427640 1 -- 172.21.15.26:0/3583810608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f037015adc0 msgr2=0x7f03700b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0375427640 1 --2- 172.21.15.26:0/3583810608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f037015adc0 0x7f03700b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0375427640 1 -- 172.21.15.26:0/3583810608 >> v1:172.21.15.29:6789/0 conn(0x7f037015e9c0 legacy=0x7f037006dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.198+0000 7f0375427640 1 -- 172.21.15.26:0/3583810608 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037015f800 con 0x7f03700ad7b0 2024-04-24T23:22:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.199+0000 7f0375427640 1 --2- 172.21.15.26:0/3583810608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03700ad7b0 0x7f03700d0040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f036000bb10 tx=0x7f036002c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2b7f3d84b04605 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.199+0000 7f0374c26640 1 -- 172.21.15.26:0/3583810608 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036003a070 con 0x7f03700ad7b0 2024-04-24T23:22:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.199+0000 7f0374c26640 1 -- 172.21.15.26:0/3583810608 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03600326f0 con 0x7f03700ad7b0 2024-04-24T23:22:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.199+0000 7f0374c26640 1 -- 172.21.15.26:0/3583810608 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360032ac0 con 0x7f03700ad7b0 2024-04-24T23:22:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.200+0000 7f0377e8c640 1 -- 172.21.15.26:0/3583810608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03700ad7b0 msgr2=0x7f03700d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.200+0000 7f0377e8c640 1 --2- 172.21.15.26:0/3583810608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03700ad7b0 0x7f03700d0040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f036000bb10 tx=0x7f036002c830 comp rx=0 tx=0).stop 2024-04-24T23:22:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.200+0000 7f0377e8c640 1 -- 172.21.15.26:0/3583810608 shutdown_connections 2024-04-24T23:22:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.200+0000 7f0377e8c640 1 --2- 172.21.15.26:0/3583810608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03700ad7b0 0x7f03700d0040 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.200+0000 7f0377e8c640 1 --2- 172.21.15.26:0/3583810608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f037015adc0 0x7f03700b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.200+0000 7f0377e8c640 1 -- 172.21.15.26:0/3583810608 >> 172.21.15.26:0/3583810608 conn(0x7f037006b540 msgr2=0x7f037006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.200+0000 7f0377e8c640 1 -- 172.21.15.26:0/3583810608 shutdown_connections 2024-04-24T23:22:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.200+0000 7f0377e8c640 1 -- 172.21.15.26:0/3583810608 wait complete. 2024-04-24T23:22:56.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.201+0000 7f0377e8c640 1 Processor -- start 2024-04-24T23:22:56.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.201+0000 7f0377e8c640 1 -- start start 2024-04-24T23:22:56.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0377e8c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03700ad7b0 0x7f0370146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0377e8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f037015adc0 0x7f037009d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0377e8c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f037015e9c0 0x7f037015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0377e8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f037015b1a0 con 0x7f037015adc0 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0377e8c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f03700b48b0 con 0x7f037015e9c0 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0377e8c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f03700d0580 con 0x7f03700ad7b0 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375427640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f037015adc0 0x7f037009d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375427640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f037015adc0 0x7f037009d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33386/0 (socket says 172.21.15.26:33386) 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375427640 1 -- 172.21.15.26:0/1262215101 learned_addr learned my addr 172.21.15.26:0/1262215101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0376429640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f037015e9c0 0x7f037015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375c28640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03700ad7b0 0x7f0370146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375427640 1 -- 172.21.15.26:0/1262215101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03700ad7b0 msgr2=0x7f0370146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375427640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03700ad7b0 0x7f0370146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375427640 1 -- 172.21.15.26:0/1262215101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f037015e9c0 msgr2=0x7f037015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375427640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f037015e9c0 0x7f037015dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375427640 1 -- 172.21.15.26:0/1262215101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037015cb00 con 0x7f037015adc0 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0376429640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f037015e9c0 0x7f037015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.202+0000 7f0375427640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f037015adc0 0x7f037009d4e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f036002c870 tx=0x7f03600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.203+0000 7f03667fc640 1 -- 172.21.15.26:0/1262215101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360045070 con 0x7f037015adc0 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.203+0000 7f03667fc640 1 -- 172.21.15.26:0/1262215101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0360035680 con 0x7f037015adc0 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.203+0000 7f03667fc640 1 -- 172.21.15.26:0/1262215101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036003a040 con 0x7f037015adc0 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.203+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037015d0e0 con 0x7f037015adc0 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.203+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f037015d6a0 con 0x7f037015adc0 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.204+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0370075610 con 0x7f037015adc0 2024-04-24T23:22:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.204+0000 7f03667fc640 1 -- 172.21.15.26:0/1262215101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0360048070 con 0x7f037015adc0 2024-04-24T23:22:56.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.205+0000 7f03667fc640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f035406be30 0x7f035406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.205+0000 7f0375c28640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f035406be30 0x7f035406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.205+0000 7f03667fc640 1 -- 172.21.15.26:0/1262215101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f03600adba0 con 0x7f037015adc0 2024-04-24T23:22:56.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.205+0000 7f0375c28640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f035406be30 0x7f035406e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f036c00a810 tx=0x7f036c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.209+0000 7f03667fc640 1 -- 172.21.15.26:0/1262215101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f036007c950 con 0x7f037015adc0 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.390+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0370075610 con 0x7f037015adc0 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.391+0000 7f03667fc640 1 -- 172.21.15.26:0/1262215101 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f036007c250 con 0x7f037015adc0 2024-04-24T23:22:56.392 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f035406be30 msgr2=0x7f035406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f035406be30 0x7f035406e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f036c00a810 tx=0x7f036c008040 comp rx=0 tx=0).stop 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f037015adc0 msgr2=0x7f037009d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f037015adc0 0x7f037009d4e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f036002c870 tx=0x7f03600046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 shutdown_connections 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f035406be30 0x7f035406e2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f037015e9c0 0x7f037015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f037015adc0 0x7f037009d4e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 --2- 172.21.15.26:0/1262215101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03700ad7b0 0x7f0370146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 >> 172.21.15.26:0/1262215101 conn(0x7f037006b540 msgr2=0x7f0370072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 shutdown_connections 2024-04-24T23:22:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.393+0000 7f0377e8c640 1 -- 172.21.15.26:0/1262215101 wait complete. 2024-04-24T23:22:56.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-04-24T23:22:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.505+0000 7f5171e3f640 1 Processor -- start 2024-04-24T23:22:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.505+0000 7f5171e3f640 1 -- start start 2024-04-24T23:22:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f5171e3f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c06d350 0x7f516c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f5171e3f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f516c0cd660 0x7f516c0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f5171e3f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f516c0cfbc0 con 0x7f516c145910 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f5171e3f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f516c0cdff0 con 0x7f516c0cd660 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f5171e3f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f516c145cf0 con 0x7f516c06d350 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f516bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f516c0cd660 0x7f516c0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f516bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f516c0cd660 0x7f516c0cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33390/0 (socket says 172.21.15.26:33390) 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f516bfff640 1 -- 172.21.15.26:0/1408266847 learned_addr learned my addr 172.21.15.26:0/1408266847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f516affd640 1 --2- 172.21.15.26:0/1408266847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c06d350 0x7f516c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f516bfff640 1 -- 172.21.15.26:0/1408266847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c06d350 msgr2=0x7f516c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f516bfff640 1 --2- 172.21.15.26:0/1408266847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c06d350 0x7f516c06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f516bfff640 1 -- 172.21.15.26:0/1408266847 >> v1:172.21.15.29:6789/0 conn(0x7f516c145910 legacy=0x7f516c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.506+0000 7f516bfff640 1 -- 172.21.15.26:0/1408266847 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c06d7f0 con 0x7f516c0cd660 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.507+0000 7f516bfff640 1 --2- 172.21.15.26:0/1408266847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c0cd660 0x7f516c0cdab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5164009870 tx=0x7f516402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170af521449e6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.507+0000 7f516a7fc640 1 -- 172.21.15.26:0/1408266847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164037070 con 0x7f516c0cd660 2024-04-24T23:22:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.507+0000 7f516a7fc640 1 -- 172.21.15.26:0/1408266847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5164032440 con 0x7f516c0cd660 2024-04-24T23:22:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.507+0000 7f516a7fc640 1 -- 172.21.15.26:0/1408266847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164032810 con 0x7f516c0cd660 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.508+0000 7f5171e3f640 1 -- 172.21.15.26:0/1408266847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c0cd660 msgr2=0x7f516c0cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.508+0000 7f5171e3f640 1 --2- 172.21.15.26:0/1408266847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c0cd660 0x7f516c0cdab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5164009870 tx=0x7f516402a270 comp rx=0 tx=0).stop 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.508+0000 7f5171e3f640 1 -- 172.21.15.26:0/1408266847 shutdown_connections 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.508+0000 7f5171e3f640 1 --2- 172.21.15.26:0/1408266847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c0cd660 0x7f516c0cdab0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.508+0000 7f5171e3f640 1 --2- 172.21.15.26:0/1408266847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c06d350 0x7f516c06e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.508+0000 7f5171e3f640 1 -- 172.21.15.26:0/1408266847 >> 172.21.15.26:0/1408266847 conn(0x7f516c06b300 msgr2=0x7f516c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.508+0000 7f5171e3f640 1 -- 172.21.15.26:0/1408266847 shutdown_connections 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.508+0000 7f5171e3f640 1 -- 172.21.15.26:0/1408266847 wait complete. 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.509+0000 7f5171e3f640 1 Processor -- start 2024-04-24T23:22:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.509+0000 7f5171e3f640 1 -- start start 2024-04-24T23:22:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f5171e3f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f516c06d350 0x7f516c0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f5171e3f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c0cd660 0x7f516c1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f5171e3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c145910 0x7f516c13fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f5171e3f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f516c06e090 con 0x7f516c145910 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f5171e3f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f516c06ddb0 con 0x7f516c06d350 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f5171e3f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f516c06e370 con 0x7f516c0cd660 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c145910 0x7f516c13fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c145910 0x7f516c13fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33402/0 (socket says 172.21.15.26:33402) 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516bfff640 1 -- 172.21.15.26:0/4050136879 learned_addr learned my addr 172.21.15.26:0/4050136879 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516b7fe640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f516c06d350 0x7f516c0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516bfff640 1 -- 172.21.15.26:0/4050136879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c0cd660 msgr2=0x7f516c1f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516affd640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c0cd660 0x7f516c1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516bfff640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c0cd660 0x7f516c1f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516bfff640 1 -- 172.21.15.26:0/4050136879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f516c06d350 msgr2=0x7f516c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516bfff640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f516c06d350 0x7f516c0aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516bfff640 1 -- 172.21.15.26:0/4050136879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c1f4c10 con 0x7f516c145910 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.510+0000 7f516b7fe640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f516c06d350 0x7f516c0aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.511+0000 7f516bfff640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c145910 0x7f516c13fd20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5164009870 tx=0x7f51640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.511+0000 7f5153fff640 1 -- 172.21.15.26:0/4050136879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164037040 con 0x7f516c145910 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.511+0000 7f5153fff640 1 -- 172.21.15.26:0/4050136879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5164042070 con 0x7f516c145910 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.511+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516c13e700 con 0x7f516c145910 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.511+0000 7f5153fff640 1 -- 172.21.15.26:0/4050136879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164030920 con 0x7f516c145910 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.511+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516c13e320 con 0x7f516c145910 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.512+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516c0694c0 con 0x7f516c145910 2024-04-24T23:22:56.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.512+0000 7f5153fff640 1 -- 172.21.15.26:0/4050136879 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5164030ac0 con 0x7f516c145910 2024-04-24T23:22:56.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.513+0000 7f5153fff640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f515806be60 0x7f515806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:56.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.513+0000 7f516b7fe640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f515806be60 0x7f515806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:56.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.513+0000 7f5153fff640 1 -- 172.21.15.26:0/4050136879 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f51640ab580 con 0x7f516c145910 2024-04-24T23:22:56.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.513+0000 7f516b7fe640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f515806be60 0x7f515806e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f515c001d70 tx=0x7f515c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:56.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.517+0000 7f5153fff640 1 -- 172.21.15.26:0/4050136879 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f516407a0e0 con 0x7f516c145910 2024-04-24T23:22:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:56.772+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) -- 0x7f516c06b780 con 0x7f516c145910 2024-04-24T23:22:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.766+0000 7f5153fff640 1 -- 172.21.15.26:0/4050136879 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v121) ==== 146+0+0 (secure 0 0 0) 0x7f51640799e0 con 0x7f516c145910 2024-04-24T23:22:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f515806be60 msgr2=0x7f515806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f515806be60 0x7f515806e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f515c001d70 tx=0x7f515c009040 comp rx=0 tx=0).stop 2024-04-24T23:22:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c145910 msgr2=0x7f516c13fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c145910 0x7f516c13fd20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5164009870 tx=0x7f51640046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 shutdown_connections 2024-04-24T23:22:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f515806be60 0x7f515806e320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f516c145910 0x7f516c13fd20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f516c0cd660 0x7f516c1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 --2- 172.21.15.26:0/4050136879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f516c06d350 0x7f516c0aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.769+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 >> 172.21.15.26:0/4050136879 conn(0x7f516c06b300 msgr2=0x7f516c1448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.770+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 shutdown_connections 2024-04-24T23:22:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.770+0000 7f5171e3f640 1 -- 172.21.15.26:0/4050136879 wait complete. 2024-04-24T23:22:57.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:22:57.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.893+0000 7f2861589640 1 Processor -- start 2024-04-24T23:22:57.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.893+0000 7f2861589640 1 -- start start 2024-04-24T23:22:57.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.894+0000 7f2861589640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f285c15e370 0x7f285c0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.894+0000 7f2861589640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f285c15e750 0x7f285c0d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.894+0000 7f2861589640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f285c0d2120 con 0x7f285c0d4c10 2024-04-24T23:22:57.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.894+0000 7f2861589640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f285c0d1510 con 0x7f285c15e750 2024-04-24T23:22:57.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.894+0000 7f2861589640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f285c0d1ed0 con 0x7f285c15e370 2024-04-24T23:22:57.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.894+0000 7f285a7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f285c15e750 0x7f285c0d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.894+0000 7f285a7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f285c15e750 0x7f285c0d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33416/0 (socket says 172.21.15.26:33416) 2024-04-24T23:22:57.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.894+0000 7f285a7fc640 1 -- 172.21.15.26:0/3539810101 learned_addr learned my addr 172.21.15.26:0/3539810101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.894+0000 7f285b7fe640 1 --1- 172.21.15.26:0/3539810101 >> v1:172.21.15.26:6789/0 conn(0x7f285c0d4c10 0x7f285c0b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51292/0 (socket says 172.21.15.26:51292) 2024-04-24T23:22:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.895+0000 7f285affd640 1 --2- 172.21.15.26:0/3539810101 >> v2:172.21.15.29:3300/0 conn(0x7f285c15e370 0x7f285c0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.895+0000 7f285a7fc640 1 -- 172.21.15.26:0/3539810101 >> v2:172.21.15.29:3300/0 conn(0x7f285c15e370 msgr2=0x7f285c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.895+0000 7f285a7fc640 1 --2- 172.21.15.26:0/3539810101 >> v2:172.21.15.29:3300/0 conn(0x7f285c15e370 0x7f285c0d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.895+0000 7f285a7fc640 1 -- 172.21.15.26:0/3539810101 >> v1:172.21.15.26:6789/0 conn(0x7f285c0d4c10 legacy=0x7f285c0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.895+0000 7f285a7fc640 1 -- 172.21.15.26:0/3539810101 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c0b3740 con 0x7f285c15e750 2024-04-24T23:22:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.895+0000 7f2859ffb640 1 -- 172.21.15.26:0/3539810101 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546810338 0 0) 0x7f285c0d2120 con 0x7f285c0d4c10 2024-04-24T23:22:57.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.895+0000 7f285a7fc640 1 --2- 172.21.15.26:0/3539810101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e750 0x7f285c0d2a60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f285c08c7e0 tx=0x7f285002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98cbf69c8a5b039 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2859ffb640 1 -- 172.21.15.26:0/3539810101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850037070 con 0x7f285c15e750 2024-04-24T23:22:57.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2859ffb640 1 -- 172.21.15.26:0/3539810101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2850032440 con 0x7f285c15e750 2024-04-24T23:22:57.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2859ffb640 1 -- 172.21.15.26:0/3539810101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850032810 con 0x7f285c15e750 2024-04-24T23:22:57.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2861589640 1 -- 172.21.15.26:0/3539810101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e750 msgr2=0x7f285c0d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2861589640 1 --2- 172.21.15.26:0/3539810101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e750 0x7f285c0d2a60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f285c08c7e0 tx=0x7f285002a270 comp rx=0 tx=0).stop 2024-04-24T23:22:57.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2861589640 1 -- 172.21.15.26:0/3539810101 shutdown_connections 2024-04-24T23:22:57.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2861589640 1 --2- 172.21.15.26:0/3539810101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e750 0x7f285c0d2a60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2861589640 1 --2- 172.21.15.26:0/3539810101 >> v2:172.21.15.29:3300/0 conn(0x7f285c15e370 0x7f285c0d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2861589640 1 -- 172.21.15.26:0/3539810101 >> 172.21.15.26:0/3539810101 conn(0x7f285c0d40f0 msgr2=0x7f285c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2861589640 1 -- 172.21.15.26:0/3539810101 shutdown_connections 2024-04-24T23:22:57.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.896+0000 7f2861589640 1 -- 172.21.15.26:0/3539810101 wait complete. 2024-04-24T23:22:57.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.898+0000 7f2861589640 1 Processor -- start 2024-04-24T23:22:57.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.898+0000 7f2861589640 1 -- start start 2024-04-24T23:22:57.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.898+0000 7f2861589640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0d4c10 0x7f285c1615b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.898+0000 7f2861589640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c15e370 0x7f285c0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.898+0000 7f2861589640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c0c94a0 0x7f285c15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.898+0000 7f2861589640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f285c0cfca0 con 0x7f285c0d4c10 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.898+0000 7f2861589640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f285c0b3d00 con 0x7f285c0c94a0 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.898+0000 7f2861589640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f285c0cfe20 con 0x7f285c15e370 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.898+0000 7f285affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0d4c10 0x7f285c1615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0d4c10 0x7f285c1615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33432/0 (socket says 172.21.15.26:33432) 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285affd640 1 -- 172.21.15.26:0/1783475581 learned_addr learned my addr 172.21.15.26:0/1783475581 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285b7fe640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c0c94a0 0x7f285c15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285a7fc640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c15e370 0x7f285c0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285affd640 1 -- 172.21.15.26:0/1783475581 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c15e370 msgr2=0x7f285c0b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285affd640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c15e370 0x7f285c0b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285affd640 1 -- 172.21.15.26:0/1783475581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c0c94a0 msgr2=0x7f285c15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285affd640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c0c94a0 0x7f285c15b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285affd640 1 -- 172.21.15.26:0/1783475581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c0caa20 con 0x7f285c0d4c10 2024-04-24T23:22:57.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285b7fe640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c0c94a0 0x7f285c15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:57.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f285affd640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0d4c10 0x7f285c1615b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f285400d830 tx=0x7f285400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f28437fe640 1 -- 172.21.15.26:0/1783475581 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285400b6c0 con 0x7f285c0d4c10 2024-04-24T23:22:57.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f28437fe640 1 -- 172.21.15.26:0/1783475581 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2854019070 con 0x7f285c0d4c10 2024-04-24T23:22:57.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c0c8fa0 con 0x7f285c0d4c10 2024-04-24T23:22:57.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f28437fe640 1 -- 172.21.15.26:0/1783475581 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285400ba80 con 0x7f285c0d4c10 2024-04-24T23:22:57.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.899+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c0c9c60 con 0x7f285c0d4c10 2024-04-24T23:22:57.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.900+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285c075610 con 0x7f285c0d4c10 2024-04-24T23:22:57.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.901+0000 7f28437fe640 1 -- 172.21.15.26:0/1783475581 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f285400bc20 con 0x7f285c0d4c10 2024-04-24T23:22:57.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.901+0000 7f28437fe640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.901+0000 7f28437fe640 1 -- 172.21.15.26:0/1783475581 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f285408afa0 con 0x7f285c0d4c10 2024-04-24T23:22:57.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.901+0000 7f285a7fc640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.902+0000 7f285a7fc640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2850034000 tx=0x7f285002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:57.906+0000 7f28437fe640 1 -- 172.21.15.26:0/1783475581 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2854059c30 con 0x7f285c0d4c10 2024-04-24T23:22:58.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.087+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f285c0b5230 con 0x7f285c0d4c10 2024-04-24T23:22:58.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.090+0000 7f28437fe640 1 -- 172.21.15.26:0/1783475581 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f2854059530 con 0x7f285c0d4c10 2024-04-24T23:22:58.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 msgr2=0x7f284806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2850034000 tx=0x7f285002e000 comp rx=0 tx=0).stop 2024-04-24T23:22:58.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0d4c10 msgr2=0x7f285c1615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0d4c10 0x7f285c1615b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f285400d830 tx=0x7f285400dcf0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 shutdown_connections 2024-04-24T23:22:58.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c0c94a0 0x7f285c15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c15e370 0x7f285c0b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 --2- 172.21.15.26:0/1783475581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0d4c10 0x7f285c1615b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 >> 172.21.15.26:0/1783475581 conn(0x7f285c0d40f0 msgr2=0x7f285c0d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 shutdown_connections 2024-04-24T23:22:58.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.092+0000 7f2861589640 1 -- 172.21.15.26:0/1783475581 wait complete. 2024-04-24T23:22:58.105 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:22:58.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:22:58.152 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:22:58.198 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:22:58.198 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:22:58.198 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:22:58.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:22:58.215 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:22:58.215 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:22:58.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:22:58.256 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:22:58.256 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:22:58.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:22:58.272 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:22:58.272 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:22:58.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:22:58.314 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:22:58.314 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:22:58.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.413+0000 7f28f00d6640 1 Processor -- start 2024-04-24T23:22:58.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.413+0000 7f28f00d6640 1 -- start start 2024-04-24T23:22:58.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.414+0000 7f28f00d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28e80d4a30 0x7f28e80d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.414+0000 7f28f00d6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e815e750 con 0x7f28e80d0b10 2024-04-24T23:22:58.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.414+0000 7f28f00d6640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e80d1d30 con 0x7f28e80d0ef0 2024-04-24T23:22:58.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.414+0000 7f28f00d6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f28e80d2640 con 0x7f28e80d4a30 2024-04-24T23:22:58.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.414+0000 7f28ede72640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f28e80d0b10 0x7f28e80cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51302/0 (socket says 172.21.15.26:51302) 2024-04-24T23:22:58.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.414+0000 7f28ede72640 1 -- 172.21.15.26:0/4019121320 learned_addr learned my addr 172.21.15.26:0/4019121320 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:58.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.414+0000 7f28ed671640 1 --2- 172.21.15.26:0/4019121320 >> v2:172.21.15.26:3300/0 conn(0x7f28e80d4a30 0x7f28e80d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.415+0000 7f28ed671640 1 -- 172.21.15.26:0/4019121320 >> v1:172.21.15.29:6789/0 conn(0x7f28e80d0ef0 legacy=0x7f28e80cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.415+0000 7f28ece70640 1 -- 172.21.15.26:0/4019121320 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2986376097 0 0) 0x7f28e4001110 con 0x7f28e80d0b10 2024-04-24T23:22:58.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.415+0000 7f28ed671640 1 -- 172.21.15.26:0/4019121320 >> v1:172.21.15.26:6789/0 conn(0x7f28e80d0b10 legacy=0x7f28e80cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.415+0000 7f28ed671640 1 -- 172.21.15.26:0/4019121320 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e80cff40 con 0x7f28e80d4a30 2024-04-24T23:22:58.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.415+0000 7f28ed671640 1 --2- 172.21.15.26:0/4019121320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d4a30 0x7f28e80d2ad0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097a0 tx=0x7f28d802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca723baef9bc358 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.415+0000 7f28ece70640 1 -- 172.21.15.26:0/4019121320 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8037070 con 0x7f28e80d4a30 2024-04-24T23:22:58.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.416+0000 7f28ece70640 1 -- 172.21.15.26:0/4019121320 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d8032440 con 0x7f28e80d4a30 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.416+0000 7f28ece70640 1 -- 172.21.15.26:0/4019121320 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8032810 con 0x7f28e80d4a30 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.416+0000 7f28f00d6640 1 -- 172.21.15.26:0/4019121320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d4a30 msgr2=0x7f28e80d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.416+0000 7f28f00d6640 1 --2- 172.21.15.26:0/4019121320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d4a30 0x7f28e80d2ad0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097a0 tx=0x7f28d802a230 comp rx=0 tx=0).stop 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.416+0000 7f28f00d6640 1 -- 172.21.15.26:0/4019121320 shutdown_connections 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.416+0000 7f28f00d6640 1 --2- 172.21.15.26:0/4019121320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d4a30 0x7f28e80d2ad0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.416+0000 7f28f00d6640 1 -- 172.21.15.26:0/4019121320 >> 172.21.15.26:0/4019121320 conn(0x7f28e809d500 msgr2=0x7f28e80d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.416+0000 7f28f00d6640 1 -- 172.21.15.26:0/4019121320 shutdown_connections 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.416+0000 7f28f00d6640 1 -- 172.21.15.26:0/4019121320 wait complete. 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.417+0000 7f28f00d6640 1 Processor -- start 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.417+0000 7f28f00d6640 1 -- start start 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28f00d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d0b10 0x7f28e815d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28f00d6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e80d0ef0 0x7f28e809df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28f00d6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28e80d4a30 0x7f28e8151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28f00d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28e80d3010 con 0x7f28e80d0b10 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28f00d6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f28e80d22d0 con 0x7f28e80d0ef0 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28f00d6640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f28e80d3190 con 0x7f28e80d4a30 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28ede72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d0b10 0x7f28e815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28ede72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d0b10 0x7f28e815d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33450/0 (socket says 172.21.15.26:33450) 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28ede72640 1 -- 172.21.15.26:0/610202161 learned_addr learned my addr 172.21.15.26:0/610202161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28ed671640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e80d0ef0 0x7f28e809df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28ee673640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28e80d4a30 0x7f28e8151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28ede72640 1 -- 172.21.15.26:0/610202161 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28e80d4a30 msgr2=0x7f28e8151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.418+0000 7f28ede72640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28e80d4a30 0x7f28e8151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.419+0000 7f28ede72640 1 -- 172.21.15.26:0/610202161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e80d0ef0 msgr2=0x7f28e809df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.419+0000 7f28ede72640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e80d0ef0 0x7f28e809df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.419+0000 7f28ede72640 1 -- 172.21.15.26:0/610202161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e815e9d0 con 0x7f28e80d0b10 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.419+0000 7f28ede72640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d0b10 0x7f28e815d230 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c3f0 tx=0x7f28e400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.419+0000 7f28de7fc640 1 -- 172.21.15.26:0/610202161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40074e0 con 0x7f28e80d0b10 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.419+0000 7f28de7fc640 1 -- 172.21.15.26:0/610202161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e40047e0 con 0x7f28e80d0b10 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.419+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e815ecb0 con 0x7f28e80d0b10 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.419+0000 7f28de7fc640 1 -- 172.21.15.26:0/610202161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4004c10 con 0x7f28e80d0b10 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.419+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e815f550 con 0x7f28e80d0b10 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.420+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e80694c0 con 0x7f28e80d0b10 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.420+0000 7f28de7fc640 1 -- 172.21.15.26:0/610202161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f28e4011070 con 0x7f28e80d0b10 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.421+0000 7f28de7fc640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28cc06be30 0x7f28cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.421+0000 7f28de7fc640 1 -- 172.21.15.26:0/610202161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f28e408a7a0 con 0x7f28e80d0b10 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.421+0000 7f28ed671640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28cc06be30 0x7f28cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.421+0000 7f28ed671640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28cc06be30 0x7f28cc06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28d8034000 tx=0x7f28d802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.425+0000 7f28de7fc640 1 -- 172.21.15.26:0/610202161 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f28e815e750 con 0x7f28e80d0b10 2024-04-24T23:22:58.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.646+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28e815ee80 con 0x7f28e80d0b10 2024-04-24T23:22:58.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.647+0000 7f28de7fc640 1 -- 172.21.15.26:0/610202161 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3866 (secure 0 0 0) 0x7f28e40593a0 con 0x7f28e80d0b10 2024-04-24T23:22:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:22:58.648 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1794706141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1794706141},{"type":"v1","addr":"172.21.15.26:6837","nonce":1794706141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1845234422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1845234422},{"type":"v1","addr":"172.21.15.29:6834","nonce":1845234422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1768308961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1768308961},{"type":"v1","addr":"172.21.15.29:6835","nonce":1768308961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/518488582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":518488582},{"type":"v1","addr":"172.21.15.26:6836","nonce":518488582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-04-24T23:22:58.649 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 26 2024-04-24T23:22:58.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28cc06be30 msgr2=0x7f28cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28cc06be30 0x7f28cc06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28d8034000 tx=0x7f28d802e000 comp rx=0 tx=0).stop 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d0b10 msgr2=0x7f28e815d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d0b10 0x7f28e815d230 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c3f0 tx=0x7f28e400c8b0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 shutdown_connections 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f28cc06be30 0x7f28cc06e2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28e80d4a30 0x7f28e8151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28e80d0ef0 0x7f28e809df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 --2- 172.21.15.26:0/610202161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e80d0b10 0x7f28e815d230 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 >> 172.21.15.26:0/610202161 conn(0x7f28e809d500 msgr2=0x7f28e80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 shutdown_connections 2024-04-24T23:22:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.649+0000 7f28f00d6640 1 -- 172.21.15.26:0/610202161 wait complete. 2024-04-24T23:22:58.661 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:22:58.662 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:22:58.662 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:22:58.662 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:22:58.662 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:22:58.662 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:22:58.662 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:22:58.662 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:22:58.663 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:22:58.663 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:22:58.663 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:22:58.663 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:22:58.663 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:22:58.663 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:22:58.663 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:22:58.663 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:22:58.672 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:22:58.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:22:58.673 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:22:58.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:22:58.679 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:22:58.679 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:22:58.680 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:22:58.680 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:22:58.715 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:22:58.715 DEBUG:teuthology.parallel:result is None 2024-04-24T23:22:58.716 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:22:58.716 DEBUG:teuthology.parallel:result is None 2024-04-24T23:22:58.722 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:22:58.723 DEBUG:teuthology.parallel:result is None 2024-04-24T23:22:58.723 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:22:58.723 DEBUG:teuthology.parallel:result is None 2024-04-24T23:22:58.724 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:22:58.790 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:22:58.790 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:22:58.810 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:22:58.810 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:22:58.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.829+0000 7f38a445d640 1 Processor -- start 2024-04-24T23:22:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.829+0000 7f38a445d640 1 -- start start 2024-04-24T23:22:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.830+0000 7f38a445d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c15e570 0x7f389c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.830+0000 7f38a445d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f389c0d22e0 0x7f389c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.830+0000 7f38a445d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389c0d32f0 con 0x7f389c0b3130 2024-04-24T23:22:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.830+0000 7f38a445d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f389c0d1ab0 con 0x7f389c0d22e0 2024-04-24T23:22:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.830+0000 7f38a445d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f389c0d2ae0 con 0x7f389c15e570 2024-04-24T23:22:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a19f8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f389c0b3130 0x7f389c0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51334/0 (socket says 172.21.15.26:51334) 2024-04-24T23:22:58.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a19f8640 1 -- 172.21.15.26:0/880350791 learned_addr learned my addr 172.21.15.26:0/880350791 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a21f9640 1 --2- 172.21.15.26:0/880350791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c15e570 0x7f389c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a29fa640 1 --2- 172.21.15.26:0/880350791 >> v2:172.21.15.29:3300/0 conn(0x7f389c0d22e0 0x7f389c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a11f7640 1 -- 172.21.15.26:0/880350791 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 610763746 0 0) 0x7f388c001110 con 0x7f389c0b3130 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a11f7640 1 -- 172.21.15.26:0/880350791 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3888003520 con 0x7f389c0b3130 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a21f9640 1 -- 172.21.15.26:0/880350791 >> v2:172.21.15.29:3300/0 conn(0x7f389c0d22e0 msgr2=0x7f389c0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a21f9640 1 --2- 172.21.15.26:0/880350791 >> v2:172.21.15.29:3300/0 conn(0x7f389c0d22e0 0x7f389c0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a21f9640 1 -- 172.21.15.26:0/880350791 >> v1:172.21.15.26:6789/0 conn(0x7f389c0b3130 legacy=0x7f389c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.831+0000 7f38a21f9640 1 -- 172.21.15.26:0/880350791 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c0d0270 con 0x7f389c15e570 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.832+0000 7f38a21f9640 1 --2- 172.21.15.26:0/880350791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c15e570 0x7f389c0d4160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3898009d90 tx=0x7f389802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146910a229e20b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.832+0000 7f38a11f7640 1 -- 172.21.15.26:0/880350791 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898007740 con 0x7f389c15e570 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.832+0000 7f38a11f7640 1 -- 172.21.15.26:0/880350791 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38980078e0 con 0x7f389c15e570 2024-04-24T23:22:58.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.832+0000 7f38a11f7640 1 -- 172.21.15.26:0/880350791 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898007cb0 con 0x7f389c15e570 2024-04-24T23:22:58.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.832+0000 7f38a445d640 1 -- 172.21.15.26:0/880350791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c15e570 msgr2=0x7f389c0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.832+0000 7f38a445d640 1 --2- 172.21.15.26:0/880350791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c15e570 0x7f389c0d4160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3898009d90 tx=0x7f389802a660 comp rx=0 tx=0).stop 2024-04-24T23:22:58.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.832+0000 7f38a445d640 1 -- 172.21.15.26:0/880350791 shutdown_connections 2024-04-24T23:22:58.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.832+0000 7f38a445d640 1 --2- 172.21.15.26:0/880350791 >> v2:172.21.15.29:3300/0 conn(0x7f389c0d22e0 0x7f389c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.832+0000 7f38a445d640 1 --2- 172.21.15.26:0/880350791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c15e570 0x7f389c0d4160 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.833+0000 7f38a445d640 1 -- 172.21.15.26:0/880350791 >> 172.21.15.26:0/880350791 conn(0x7f389c08dbe0 msgr2=0x7f389c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.833+0000 7f38a445d640 1 -- 172.21.15.26:0/880350791 shutdown_connections 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.833+0000 7f38a445d640 1 -- 172.21.15.26:0/880350791 wait complete. 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.834+0000 7f38a445d640 1 Processor -- start 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.834+0000 7f38a445d640 1 -- start start 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.834+0000 7f38a445d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389c0b3130 0x7f389c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.834+0000 7f38a445d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c0d22e0 0x7f389c0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.834+0000 7f38a445d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389c15e570 0x7f389c0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.834+0000 7f38a445d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f389c0cfa70 con 0x7f389c0b3130 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.834+0000 7f38a445d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f389c0d3010 con 0x7f389c15e570 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.834+0000 7f38a445d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f389c0cfd50 con 0x7f389c0d22e0 2024-04-24T23:22:58.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a21f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389c0b3130 0x7f389c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a21f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389c0b3130 0x7f389c158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33492/0 (socket says 172.21.15.26:33492) 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a21f9640 1 -- 172.21.15.26:0/2600284397 learned_addr learned my addr 172.21.15.26:0/2600284397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a19f8640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c0d22e0 0x7f389c0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a29fa640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389c15e570 0x7f389c0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a21f9640 1 -- 172.21.15.26:0/2600284397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c0d22e0 msgr2=0x7f389c0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a21f9640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c0d22e0 0x7f389c0b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a21f9640 1 -- 172.21.15.26:0/2600284397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389c15e570 msgr2=0x7f389c0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a21f9640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389c15e570 0x7f389c0b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a21f9640 1 -- 172.21.15.26:0/2600284397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c1590b0 con 0x7f389c0b3130 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a29fa640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389c15e570 0x7f389c0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f38a21f9640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389c0b3130 0x7f389c158b70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3898009d90 tx=0x7f3898004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f3892ffd640 1 -- 172.21.15.26:0/2600284397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389803f070 con 0x7f389c0b3130 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f3892ffd640 1 -- 172.21.15.26:0/2600284397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3898031c40 con 0x7f389c0b3130 2024-04-24T23:22:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.835+0000 7f3892ffd640 1 -- 172.21.15.26:0/2600284397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898009450 con 0x7f389c0b3130 2024-04-24T23:22:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.836+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c0b9a20 con 0x7f389c0b3130 2024-04-24T23:22:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.836+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c159430 con 0x7f389c0b3130 2024-04-24T23:22:58.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.837+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c0694c0 con 0x7f389c0b3130 2024-04-24T23:22:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.837+0000 7f3892ffd640 1 -- 172.21.15.26:0/2600284397 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3898044460 con 0x7f389c0b3130 2024-04-24T23:22:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.838+0000 7f3892ffd640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f388006be30 0x7f388006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.838+0000 7f3892ffd640 1 -- 172.21.15.26:0/2600284397 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f38980ac050 con 0x7f389c0b3130 2024-04-24T23:22:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.838+0000 7f38a19f8640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f388006be30 0x7f388006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:58.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.838+0000 7f38a19f8640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f388006be30 0x7f388006e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f388c0012b0 tx=0x7f388c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:58.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:58.842+0000 7f3892ffd640 1 -- 172.21.15.26:0/2600284397 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f389804b030 con 0x7f389c0b3130 2024-04-24T23:22:59.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.042+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f389c0b2d80 con 0x7f389c0b3130 2024-04-24T23:22:59.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:22:59.224 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:22:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.775+0000 7f3892ffd640 1 -- 172.21.15.26:0/2600284397 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) ==== 97+0+0 (secure 0 0 0) 0x7f389c1590b0 con 0x7f389c0b3130 2024-04-24T23:22:59.777 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f388006be30 msgr2=0x7f388006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f388006be30 0x7f388006e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f388c0012b0 tx=0x7f388c009040 comp rx=0 tx=0).stop 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389c0b3130 msgr2=0x7f389c158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389c0b3130 0x7f389c158b70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3898009d90 tx=0x7f3898004a40 comp rx=0 tx=0).stop 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 shutdown_connections 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f388006be30 0x7f388006e2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f389c15e570 0x7f389c0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f389c0d22e0 0x7f389c0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 --2- 172.21.15.26:0/2600284397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389c0b3130 0x7f389c158b70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 >> 172.21.15.26:0/2600284397 conn(0x7f389c08dbe0 msgr2=0x7f389c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 shutdown_connections 2024-04-24T23:22:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.779+0000 7f38a445d640 1 -- 172.21.15.26:0/2600284397 wait complete. 2024-04-24T23:22:59.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:22:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.899+0000 7fe0f1672640 1 Processor -- start 2024-04-24T23:22:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.899+0000 7fe0f1672640 1 -- start start 2024-04-24T23:22:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0f1672640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0ec15a480 0x7fe0ec0b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0f1672640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 0x7fe0ec06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0f1672640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0ec0b2e80 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0f1672640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ec0b3120 con 0x7fe0ec15a480 2024-04-24T23:22:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0f1672640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ec0b33c0 con 0x7fe0ec15a860 2024-04-24T23:22:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0eaffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0ec15a480 0x7fe0ec0b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0eaffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0ec15a480 0x7fe0ec0b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33494/0 (socket says 172.21.15.26:33494) 2024-04-24T23:22:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0eaffd640 1 -- 172.21.15.26:0/310395524 learned_addr learned my addr 172.21.15.26:0/310395524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0ea7fc640 1 --2- 172.21.15.26:0/310395524 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 0x7fe0ec06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0eaffd640 1 -- 172.21.15.26:0/310395524 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 msgr2=0x7fe0ec06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0eaffd640 1 --2- 172.21.15.26:0/310395524 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 0x7fe0ec06dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0eaffd640 1 -- 172.21.15.26:0/310395524 >> v1:172.21.15.29:6789/0 conn(0x7fe0ec0b4e30 legacy=0x7fe0ec15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.900+0000 7fe0eaffd640 1 -- 172.21.15.26:0/310395524 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec0b36a0 con 0x7fe0ec15a480 2024-04-24T23:22:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.901+0000 7fe0eaffd640 1 --2- 172.21.15.26:0/310395524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec15a480 0x7fe0ec0b48f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009870 tx=0x7fe0e402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76c808beb288bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.901+0000 7fe0e9ffb640 1 -- 172.21.15.26:0/310395524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4037070 con 0x7fe0ec15a480 2024-04-24T23:22:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.901+0000 7fe0e9ffb640 1 -- 172.21.15.26:0/310395524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e4032440 con 0x7fe0ec15a480 2024-04-24T23:22:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.901+0000 7fe0e9ffb640 1 -- 172.21.15.26:0/310395524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4032810 con 0x7fe0ec15a480 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.902+0000 7fe0f1672640 1 -- 172.21.15.26:0/310395524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec15a480 msgr2=0x7fe0ec0b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.902+0000 7fe0f1672640 1 --2- 172.21.15.26:0/310395524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec15a480 0x7fe0ec0b48f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009870 tx=0x7fe0e402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.902+0000 7fe0f1672640 1 -- 172.21.15.26:0/310395524 shutdown_connections 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.902+0000 7fe0f1672640 1 --2- 172.21.15.26:0/310395524 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 0x7fe0ec06dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.902+0000 7fe0f1672640 1 --2- 172.21.15.26:0/310395524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec15a480 0x7fe0ec0b48f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.902+0000 7fe0f1672640 1 -- 172.21.15.26:0/310395524 >> 172.21.15.26:0/310395524 conn(0x7fe0ec06b300 msgr2=0x7fe0ec06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.902+0000 7fe0f1672640 1 -- 172.21.15.26:0/310395524 shutdown_connections 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.902+0000 7fe0f1672640 1 -- 172.21.15.26:0/310395524 wait complete. 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.903+0000 7fe0f1672640 1 Processor -- start 2024-04-24T23:22:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.903+0000 7fe0f1672640 1 -- start start 2024-04-24T23:22:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0f1672640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec0b4e30 0x7fe0ec169390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0f1672640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0ec15a480 0x7fe0ec139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0f1672640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 0x7fe0ec141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0f1672640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec072080 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0f1672640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec0b3c60 con 0x7fe0ec15a480 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0f1672640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ec072200 con 0x7fe0ec15a860 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0eaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec0b4e30 0x7fe0ec169390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0eaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec0b4e30 0x7fe0ec169390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33502/0 (socket says 172.21.15.26:33502) 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0eaffd640 1 -- 172.21.15.26:0/4081525859 learned_addr learned my addr 172.21.15.26:0/4081525859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0ea7fc640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0ec15a480 0x7fe0ec139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0eb7fe640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 0x7fe0ec141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0eaffd640 1 -- 172.21.15.26:0/4081525859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 msgr2=0x7fe0ec141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0eaffd640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 0x7fe0ec141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0eaffd640 1 -- 172.21.15.26:0/4081525859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0ec15a480 msgr2=0x7fe0ec139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0eaffd640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0ec15a480 0x7fe0ec139e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.904+0000 7fe0eaffd640 1 -- 172.21.15.26:0/4081525859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec1461a0 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.905+0000 7fe0eaffd640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec0b4e30 0x7fe0ec169390 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4000c00 tx=0x7fe0e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.905+0000 7fe0e37fe640 1 -- 172.21.15.26:0/4081525859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4037040 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.905+0000 7fe0e37fe640 1 -- 172.21.15.26:0/4081525859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e4042070 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.905+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec15ac80 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.905+0000 7fe0e37fe640 1 -- 172.21.15.26:0/4081525859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4030860 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.905+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec15b240 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.906+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ec0694c0 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.906+0000 7fe0e37fe640 1 -- 172.21.15.26:0/4081525859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0e402ad10 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.907+0000 7fe0e37fe640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:59.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.907+0000 7fe0e37fe640 1 -- 172.21.15.26:0/4081525859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7fe0e40ab0e0 con 0x7fe0ec0b4e30 2024-04-24T23:22:59.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.907+0000 7fe0ea7fc640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:59.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.907+0000 7fe0ea7fc640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec13aae0 tx=0x7fe0d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:22:59.911+0000 7fe0e37fe640 1 -- 172.21.15.26:0/4081525859 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe0e4079de0 con 0x7fe0ec0b4e30 2024-04-24T23:23:00.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.165+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe0ec146b10 con 0x7fe0ec0b4e30 2024-04-24T23:23:00.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.166+0000 7fe0e37fe640 1 -- 172.21.15.26:0/4081525859 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7fe0e40796e0 con 0x7fe0ec0b4e30 2024-04-24T23:23:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:00.167 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmg5ZUFBAAkapM9La3f+wSyLczFkm/0w==", 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPS3cHxAASacBQN+95Ds6N5668Je+DA==", 2024-04-24T23:23:00.168 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmStpLGxAAz7molt8venundBs+CcFSbg==", 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:23:00.169 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylml0TBJhAAXQkjExbw1EXc90XlcAEBBw==", 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjkylmQt74DBAAZLk9JTH3qTKRouUawqNURw==", 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCmkylmi6IJCxAAWBnNl2y0XIUW/YoAIt6VIQ==", 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.171 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCpkylmRcgOCRAA79pdcH0OlB2VROLSRE4dcw==", 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.172 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCskylm8SopBxAArvUph95+zD5/0LmNM8WPIA==", 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCwkylmbHipDxAAVUI9OpiMFiM/HFo03fEzSA==", 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.173 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCzkylmDm4CDhAA1T/X+ywXEUCL96OGyi6S9Q==", 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.174 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC2kylmwWyVDBAAJsFFLltURqVqGLNjK9OCJQ==", 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5kylm/nLXChAA6kk0UI2hchkN2lhPO1sXHw==", 2024-04-24T23:23:00.175 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ==", 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.176 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylm/BVuMxAAAdEb6dd3Q5eMc+I3oszOyg==", 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmjnJGMBAAvwAmiHrkGkDFTeL/WZbdWw==", 2024-04-24T23:23:00.177 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPh3qOBAAa4cbQeF96mdpgNVb/l3xPg==", 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:23:00.178 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCgkylm1ZfsMxAAExymnmhW44TcXzRblp95UA==", 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.179 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmq+8yERAAMKjK+5f2Qr35ral8L2qJMw==", 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmqYXWDBAAnuAn2h4Gb9qSFBoAZX73qQ==", 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:23:00.180 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:23:00.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:00.181 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-24T23:23:00.181 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:00.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0cc06be30 msgr2=0x7fe0cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec13aae0 tx=0x7fe0d4008040 comp rx=0 tx=0).stop 2024-04-24T23:23:00.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec0b4e30 msgr2=0x7fe0ec169390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec0b4e30 0x7fe0ec169390 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4000c00 tx=0x7fe0e40046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 shutdown_connections 2024-04-24T23:23:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0ec15a860 0x7fe0ec141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0ec15a480 0x7fe0ec139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 --2- 172.21.15.26:0/4081525859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ec0b4e30 0x7fe0ec169390 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 >> 172.21.15.26:0/4081525859 conn(0x7fe0ec06b300 msgr2=0x7fe0ec0ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 shutdown_connections 2024-04-24T23:23:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.168+0000 7fe0f1672640 1 -- 172.21.15.26:0/4081525859 wait complete. 2024-04-24T23:23:00.184 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:23:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.283+0000 7f34c3d5a640 1 Processor -- start 2024-04-24T23:23:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.283+0000 7f34c3d5a640 1 -- start start 2024-04-24T23:23:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.284+0000 7f34c3d5a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34bc143c40 0x7f34bc0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.284+0000 7f34c3d5a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc15eb00 0x7f34bc15ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.284+0000 7f34c3d5a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34bc0cd0b0 con 0x7f34bc0cc860 2024-04-24T23:23:00.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.284+0000 7f34c3d5a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f34bc071b70 con 0x7f34bc143c40 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.284+0000 7f34c3d5a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc15f470 con 0x7f34bc15eb00 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.284+0000 7f34c1af6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34bc143c40 0x7f34bc0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.284+0000 7f34c1af6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34bc143c40 0x7f34bc0cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33510/0 (socket says 172.21.15.26:33510) 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.284+0000 7f34c1af6640 1 -- 172.21.15.26:0/1729214107 learned_addr learned my addr 172.21.15.26:0/1729214107 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.284+0000 7f34c22f7640 1 --2- 172.21.15.26:0/1729214107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc15eb00 0x7f34bc15ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.285+0000 7f34c1af6640 1 -- 172.21.15.26:0/1729214107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc15eb00 msgr2=0x7f34bc15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.285+0000 7f34c1af6640 1 --2- 172.21.15.26:0/1729214107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc15eb00 0x7f34bc15ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.285+0000 7f34c1af6640 1 -- 172.21.15.26:0/1729214107 >> v1:172.21.15.29:6789/0 conn(0x7f34bc0cc860 legacy=0x7f34bc072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.285+0000 7f34c1af6640 1 -- 172.21.15.26:0/1729214107 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc071240 con 0x7f34bc143c40 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.285+0000 7f34c1af6640 1 --2- 172.21.15.26:0/1729214107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc143c40 0x7f34bc0cf870 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009870 tx=0x7f34b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e927cc60c31a36b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.285+0000 7f34c0af4640 1 -- 172.21.15.26:0/1729214107 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4037070 con 0x7f34bc143c40 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.285+0000 7f34c0af4640 1 -- 172.21.15.26:0/1729214107 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b4032440 con 0x7f34bc143c40 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.285+0000 7f34c0af4640 1 -- 172.21.15.26:0/1729214107 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4032810 con 0x7f34bc143c40 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.286+0000 7f34c3d5a640 1 -- 172.21.15.26:0/1729214107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc143c40 msgr2=0x7f34bc0cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.286+0000 7f34c3d5a640 1 --2- 172.21.15.26:0/1729214107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc143c40 0x7f34bc0cf870 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009870 tx=0x7f34b402a270 comp rx=0 tx=0).stop 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.286+0000 7f34c3d5a640 1 -- 172.21.15.26:0/1729214107 shutdown_connections 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.286+0000 7f34c3d5a640 1 --2- 172.21.15.26:0/1729214107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc15eb00 0x7f34bc15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.286+0000 7f34c3d5a640 1 --2- 172.21.15.26:0/1729214107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc143c40 0x7f34bc0cf870 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.286+0000 7f34c3d5a640 1 -- 172.21.15.26:0/1729214107 >> 172.21.15.26:0/1729214107 conn(0x7f34bc06b300 msgr2=0x7f34bc06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.286+0000 7f34c3d5a640 1 -- 172.21.15.26:0/1729214107 shutdown_connections 2024-04-24T23:23:00.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.286+0000 7f34c3d5a640 1 -- 172.21.15.26:0/1729214107 wait complete. 2024-04-24T23:23:00.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.287+0000 7f34c3d5a640 1 Processor -- start 2024-04-24T23:23:00.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.287+0000 7f34c3d5a640 1 -- start start 2024-04-24T23:23:00.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c3d5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc0cc860 0x7f34bc13ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c3d5a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc143c40 0x7f34bc0b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c3d5a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34bc15eb00 0x7f34bc141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c3d5a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc144020 con 0x7f34bc0cc860 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c3d5a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc071800 con 0x7f34bc15eb00 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c3d5a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc0cccd0 con 0x7f34bc143c40 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c1af6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc0cc860 0x7f34bc13ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c1af6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc0cc860 0x7f34bc13ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33516/0 (socket says 172.21.15.26:33516) 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c1af6640 1 -- 172.21.15.26:0/4048988414 learned_addr learned my addr 172.21.15.26:0/4048988414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c12f5640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc143c40 0x7f34bc0b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c22f7640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34bc15eb00 0x7f34bc141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c1af6640 1 -- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc143c40 msgr2=0x7f34bc0b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c1af6640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc143c40 0x7f34bc0b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c1af6640 1 -- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34bc15eb00 msgr2=0x7f34bc141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c1af6640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34bc15eb00 0x7f34bc141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c1af6640 1 -- 172.21.15.26:0/4048988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc140bb0 con 0x7f34bc0cc860 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c12f5640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc143c40 0x7f34bc0b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.288+0000 7f34c22f7640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34bc15eb00 0x7f34bc141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.289+0000 7f34c1af6640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc0cc860 0x7f34bc13ffd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f34b4000c00 tx=0x7f34b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.289+0000 7f34ba7fc640 1 -- 172.21.15.26:0/4048988414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4037040 con 0x7f34bc0cc860 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.289+0000 7f34ba7fc640 1 -- 172.21.15.26:0/4048988414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b4042070 con 0x7f34bc0cc860 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.289+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc140ef0 con 0x7f34bc0cc860 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.289+0000 7f34ba7fc640 1 -- 172.21.15.26:0/4048988414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4030860 con 0x7f34bc0cc860 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.289+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc140510 con 0x7f34bc0cc860 2024-04-24T23:23:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.290+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc0694c0 con 0x7f34bc0cc860 2024-04-24T23:23:00.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.290+0000 7f34ba7fc640 1 -- 172.21.15.26:0/4048988414 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34b4030a00 con 0x7f34bc0cc860 2024-04-24T23:23:00.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.291+0000 7f34ba7fc640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34a806be30 0x7f34a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.291+0000 7f34ba7fc640 1 -- 172.21.15.26:0/4048988414 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f34b40ab3d0 con 0x7f34bc0cc860 2024-04-24T23:23:00.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.291+0000 7f34c12f5640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34a806be30 0x7f34a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.291+0000 7f34c12f5640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34a806be30 0x7f34a806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f34bc15b8c0 tx=0x7f34b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.295+0000 7f34ba7fc640 1 -- 172.21.15.26:0/4048988414 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f34bc140ef0 con 0x7f34bc0cc860 2024-04-24T23:23:00.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.479+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34bc1410d0 con 0x7f34bc0cc860 2024-04-24T23:23:00.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.479+0000 7f34ba7fc640 1 -- 172.21.15.26:0/4048988414 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+12831 (secure 0 0 0) 0x7f34b407a0d0 con 0x7f34bc0cc860 2024-04-24T23:23:00.481 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":40,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:22:59.782935+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:23:00.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.481+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34a806be30 msgr2=0x7f34a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.481+0000 7f34c3d5a640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34a806be30 0x7f34a806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f34bc15b8c0 tx=0x7f34b0009040 comp rx=0 tx=0).stop 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.481+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc0cc860 msgr2=0x7f34bc13ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.481+0000 7f34c3d5a640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc0cc860 0x7f34bc13ffd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f34b4000c00 tx=0x7f34b40046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.482+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 shutdown_connections 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.482+0000 7f34c3d5a640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34a806be30 0x7f34a806e2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.482+0000 7f34c3d5a640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34bc15eb00 0x7f34bc141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.482+0000 7f34c3d5a640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34bc143c40 0x7f34bc0b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.482+0000 7f34c3d5a640 1 --2- 172.21.15.26:0/4048988414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc0cc860 0x7f34bc13ffd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.482+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 >> 172.21.15.26:0/4048988414 conn(0x7f34bc06b300 msgr2=0x7f34bc0cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.482+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 shutdown_connections 2024-04-24T23:23:00.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.482+0000 7f34c3d5a640 1 -- 172.21.15.26:0/4048988414 wait complete. 2024-04-24T23:23:00.494 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:23:00.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:23:00.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.597+0000 7f27c4be7640 1 Processor -- start 2024-04-24T23:23:00.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.597+0000 7f27c4be7640 1 -- start start 2024-04-24T23:23:00.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.598+0000 7f27c4be7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c0158ae0 0x7f27c006e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.598+0000 7f27c4be7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f27c00d0ce0 0x7f27c015e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.598+0000 7f27c4be7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c006b1c0 con 0x7f27c015ee90 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.598+0000 7f27c4be7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f27c006ec90 con 0x7f27c00d0ce0 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.598+0000 7f27c4be7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f27c006ef70 con 0x7f27c0158ae0 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.599+0000 7f27bdd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f27c00d0ce0 0x7f27c015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.599+0000 7f27bed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f27c015ee90 0x7f27c0071720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51340/0 (socket says 172.21.15.26:51340) 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.599+0000 7f27bed9d640 1 -- 172.21.15.26:0/3902081213 learned_addr learned my addr 172.21.15.26:0/3902081213 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.599+0000 7f27be59c640 1 --2- 172.21.15.26:0/3902081213 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c0158ae0 0x7f27c006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.599+0000 7f27bdd9b640 1 -- 172.21.15.26:0/3902081213 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c0158ae0 msgr2=0x7f27c006e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.599+0000 7f27bdd9b640 1 --2- 172.21.15.26:0/3902081213 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c0158ae0 0x7f27c006e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.599+0000 7f27bdd9b640 1 -- 172.21.15.26:0/3902081213 >> v1:172.21.15.26:6789/0 conn(0x7f27c015ee90 legacy=0x7f27c0071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.599+0000 7f27bdd9b640 1 -- 172.21.15.26:0/3902081213 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c006f250 con 0x7f27c00d0ce0 2024-04-24T23:23:00.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.599+0000 7f27bd59a640 1 -- 172.21.15.26:0/3902081213 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900798527 0 0) 0x7f27c006b1c0 con 0x7f27c015ee90 2024-04-24T23:23:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27bdd9b640 1 --2- 172.21.15.26:0/3902081213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c00d0ce0 0x7f27c015e950 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f27a80097a0 tx=0x7f27a802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa968d3b79fed22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27bd59a640 1 -- 172.21.15.26:0/3902081213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8037070 con 0x7f27c00d0ce0 2024-04-24T23:23:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27bd59a640 1 -- 172.21.15.26:0/3902081213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a802ae50 con 0x7f27c00d0ce0 2024-04-24T23:23:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27bd59a640 1 -- 172.21.15.26:0/3902081213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8032660 con 0x7f27c00d0ce0 2024-04-24T23:23:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27c4be7640 1 -- 172.21.15.26:0/3902081213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c00d0ce0 msgr2=0x7f27c015e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27c4be7640 1 --2- 172.21.15.26:0/3902081213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c00d0ce0 0x7f27c015e950 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f27a80097a0 tx=0x7f27a802a100 comp rx=0 tx=0).stop 2024-04-24T23:23:00.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27c4be7640 1 -- 172.21.15.26:0/3902081213 shutdown_connections 2024-04-24T23:23:00.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27c4be7640 1 --2- 172.21.15.26:0/3902081213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c00d0ce0 0x7f27c015e950 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27c4be7640 1 --2- 172.21.15.26:0/3902081213 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c0158ae0 0x7f27c006e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27c4be7640 1 -- 172.21.15.26:0/3902081213 >> 172.21.15.26:0/3902081213 conn(0x7f27c00b2970 msgr2=0x7f27c00d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:00.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27c4be7640 1 -- 172.21.15.26:0/3902081213 shutdown_connections 2024-04-24T23:23:00.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.600+0000 7f27c4be7640 1 -- 172.21.15.26:0/3902081213 wait complete. 2024-04-24T23:23:00.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.602+0000 7f27c4be7640 1 Processor -- start 2024-04-24T23:23:00.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.602+0000 7f27c4be7640 1 -- start start 2024-04-24T23:23:00.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.602+0000 7f27c4be7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c00d0ce0 0x7f27c01f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.602+0000 7f27c4be7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c0158ae0 0x7f27c01546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.602+0000 7f27c4be7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c015ee90 0x7f27c0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.602+0000 7f27c4be7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27c00b00e0 con 0x7f27c015ee90 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.602+0000 7f27c4be7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f27c00aff60 con 0x7f27c0158ae0 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.602+0000 7f27c4be7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f27c0071e30 con 0x7f27c00d0ce0 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c015ee90 0x7f27c0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c015ee90 0x7f27c0159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33532/0 (socket says 172.21.15.26:33532) 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bed9d640 1 -- 172.21.15.26:0/338507403 learned_addr learned my addr 172.21.15.26:0/338507403 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27be59c640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c00d0ce0 0x7f27c01f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bdd9b640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c0158ae0 0x7f27c01546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bed9d640 1 -- 172.21.15.26:0/338507403 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c00d0ce0 msgr2=0x7f27c01f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bed9d640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c00d0ce0 0x7f27c01f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bed9d640 1 -- 172.21.15.26:0/338507403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c0158ae0 msgr2=0x7f27c01546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bed9d640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c0158ae0 0x7f27c01546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bed9d640 1 -- 172.21.15.26:0/338507403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c0154c10 con 0x7f27c015ee90 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bdd9b640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c0158ae0 0x7f27c01546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27bed9d640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c015ee90 0x7f27c0159f20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f27b000c780 tx=0x7f27b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27b6ffd640 1 -- 172.21.15.26:0/338507403 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0018070 con 0x7f27c015ee90 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27b6ffd640 1 -- 172.21.15.26:0/338507403 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b000a640 con 0x7f27c015ee90 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27b6ffd640 1 -- 172.21.15.26:0/338507403 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b000aab0 con 0x7f27c015ee90 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c0160bf0 con 0x7f27c015ee90 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.603+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c01602f0 con 0x7f27c015ee90 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.604+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c0160dc0 con 0x7f27c015ee90 2024-04-24T23:23:00.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.605+0000 7f27b6ffd640 1 -- 172.21.15.26:0/338507403 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f27b001b070 con 0x7f27c015ee90 2024-04-24T23:23:00.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.605+0000 7f27b6ffd640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f27a006be30 0x7f27a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.605+0000 7f27b6ffd640 1 -- 172.21.15.26:0/338507403 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f27b008a000 con 0x7f27c015ee90 2024-04-24T23:23:00.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.605+0000 7f27be59c640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f27a006be30 0x7f27a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.606+0000 7f27be59c640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f27a006be30 0x7f27a006e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009aa0 tx=0x7f27b8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.609+0000 7f27b6ffd640 1 -- 172.21.15.26:0/338507403 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f27b0058740 con 0x7f27c015ee90 2024-04-24T23:23:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.793+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27c0075610 con 0x7f27c015ee90 2024-04-24T23:23:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.794+0000 7f27b6ffd640 1 -- 172.21.15.26:0/338507403 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f27b005ca40 con 0x7f27c015ee90 2024-04-24T23:23:00.796 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:23:00.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f27a006be30 msgr2=0x7f27a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f27a006be30 0x7f27a006e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009aa0 tx=0x7f27b8009190 comp rx=0 tx=0).stop 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c015ee90 msgr2=0x7f27c0159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c015ee90 0x7f27c0159f20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f27b000c780 tx=0x7f27b000cc40 comp rx=0 tx=0).stop 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 shutdown_connections 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f27a006be30 0x7f27a006e2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c015ee90 0x7f27c0159f20 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f27c0158ae0 0x7f27c01546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 --2- 172.21.15.26:0/338507403 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f27c00d0ce0 0x7f27c01f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 >> 172.21.15.26:0/338507403 conn(0x7f27c00b2970 msgr2=0x7f27c0071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 shutdown_connections 2024-04-24T23:23:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.796+0000 7f27c4be7640 1 -- 172.21.15.26:0/338507403 wait complete. 2024-04-24T23:23:00.808 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:23:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.908+0000 7febbecca640 1 Processor -- start 2024-04-24T23:23:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.908+0000 7febbecca640 1 -- start start 2024-04-24T23:23:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbecca640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7febb8143c40 0x7febb80cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbecca640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 0x7febb8072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbecca640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7febb815eb00 0x7febb815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbecca640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7febb815f490 con 0x7febb8143c40 2024-04-24T23:23:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbecca640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7febb8071b70 con 0x7febb815eb00 2024-04-24T23:23:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbecca640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7febb80cd0b0 con 0x7febb80cc860 2024-04-24T23:23:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbca66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7febb8143c40 0x7febb80cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbca66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7febb8143c40 0x7febb80cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33542/0 (socket says 172.21.15.26:33542) 2024-04-24T23:23:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbca66640 1 -- 172.21.15.26:0/3758825501 learned_addr learned my addr 172.21.15.26:0/3758825501 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febbca66640 1 -- 172.21.15.26:0/3758825501 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 msgr2=0x7febb8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.909+0000 7febb7fff640 1 --2- 172.21.15.26:0/3758825501 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 0x7febb8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.910+0000 7febbd267640 1 --2- 172.21.15.26:0/3758825501 >> v2:172.21.15.29:3300/0 conn(0x7febb815eb00 0x7febb815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.910+0000 7febbca66640 1 --2- 172.21.15.26:0/3758825501 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 0x7febb8072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.910+0000 7febbca66640 1 -- 172.21.15.26:0/3758825501 >> v2:172.21.15.29:3300/0 conn(0x7febb815eb00 msgr2=0x7febb815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.910+0000 7febbca66640 1 --2- 172.21.15.26:0/3758825501 >> v2:172.21.15.29:3300/0 conn(0x7febb815eb00 0x7febb815ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.910+0000 7febbca66640 1 -- 172.21.15.26:0/3758825501 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb8071240 con 0x7febb8143c40 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.910+0000 7febbca66640 1 --2- 172.21.15.26:0/3758825501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb8143c40 0x7febb80cf870 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7febb806d450 tx=0x7febb002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98444a7975dc1b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.910+0000 7febb77fe640 1 -- 172.21.15.26:0/3758825501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0036070 con 0x7febb8143c40 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.910+0000 7febb77fe640 1 -- 172.21.15.26:0/3758825501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb0031600 con 0x7febb8143c40 2024-04-24T23:23:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.910+0000 7febb77fe640 1 -- 172.21.15.26:0/3758825501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb00319d0 con 0x7febb8143c40 2024-04-24T23:23:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.911+0000 7febbecca640 1 -- 172.21.15.26:0/3758825501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb8143c40 msgr2=0x7febb80cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.911+0000 7febbecca640 1 --2- 172.21.15.26:0/3758825501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb8143c40 0x7febb80cf870 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7febb806d450 tx=0x7febb002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.911+0000 7febbecca640 1 -- 172.21.15.26:0/3758825501 shutdown_connections 2024-04-24T23:23:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.911+0000 7febbecca640 1 --2- 172.21.15.26:0/3758825501 >> v2:172.21.15.29:3300/0 conn(0x7febb815eb00 0x7febb815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.911+0000 7febbecca640 1 --2- 172.21.15.26:0/3758825501 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 0x7febb8072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.911+0000 7febbecca640 1 --2- 172.21.15.26:0/3758825501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb8143c40 0x7febb80cf870 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.911+0000 7febbecca640 1 -- 172.21.15.26:0/3758825501 >> 172.21.15.26:0/3758825501 conn(0x7febb806b300 msgr2=0x7febb806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.911+0000 7febbecca640 1 -- 172.21.15.26:0/3758825501 shutdown_connections 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.911+0000 7febbecca640 1 -- 172.21.15.26:0/3758825501 wait complete. 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.912+0000 7febbecca640 1 Processor -- start 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.912+0000 7febbecca640 1 -- start start 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbecca640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 0x7febb81378e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbecca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb8143c40 0x7febb813ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbecca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb815eb00 0x7febb81f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbecca640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7febb8144020 con 0x7febb815eb00 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbecca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febb8071800 con 0x7febb8143c40 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbecca640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7febb8072540 con 0x7febb80cc860 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbd267640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb815eb00 0x7febb81f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbd267640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb815eb00 0x7febb81f3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33550/0 (socket says 172.21.15.26:33550) 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbd267640 1 -- 172.21.15.26:0/3057871459 learned_addr learned my addr 172.21.15.26:0/3057871459 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febb7fff640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb8143c40 0x7febb813ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.913+0000 7febbca66640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 0x7febb81378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febbd267640 1 -- 172.21.15.26:0/3057871459 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 msgr2=0x7febb81378e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febbd267640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 0x7febb81378e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febbd267640 1 -- 172.21.15.26:0/3057871459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb8143c40 msgr2=0x7febb813ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febbd267640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb8143c40 0x7febb813ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febbd267640 1 -- 172.21.15.26:0/3057871459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb8137e20 con 0x7febb815eb00 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febbd267640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb815eb00 0x7febb81f3f80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feba800c550 tx=0x7feba800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febb57fa640 1 -- 172.21.15.26:0/3057871459 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8015070 con 0x7febb815eb00 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febb57fa640 1 -- 172.21.15.26:0/3057871459 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba8004220 con 0x7febb815eb00 2024-04-24T23:23:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb813cfa0 con 0x7febb815eb00 2024-04-24T23:23:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febb57fa640 1 -- 172.21.15.26:0/3057871459 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8003610 con 0x7febb815eb00 2024-04-24T23:23:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.914+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb813d560 con 0x7febb815eb00 2024-04-24T23:23:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.915+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb80694c0 con 0x7febb815eb00 2024-04-24T23:23:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.915+0000 7febb57fa640 1 -- 172.21.15.26:0/3057871459 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feba80037b0 con 0x7febb815eb00 2024-04-24T23:23:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.916+0000 7febb57fa640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feba406be30 0x7feba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.916+0000 7febb57fa640 1 -- 172.21.15.26:0/3057871459 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7feba808a790 con 0x7febb815eb00 2024-04-24T23:23:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.916+0000 7febbca66640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feba406be30 0x7feba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.916+0000 7febbca66640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feba406be30 0x7feba406e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7febb0033000 tx=0x7febb0009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:00.920+0000 7febb57fa640 1 -- 172.21.15.26:0/3057871459 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feba8058ed0 con 0x7febb815eb00 2024-04-24T23:23:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.109+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7febb80b4d20 con 0x7febb815eb00 2024-04-24T23:23:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.109+0000 7febb57fa640 1 -- 172.21.15.26:0/3057871459 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) ==== 84+0+65 (secure 0 0 0) 0x7feba805d1d0 con 0x7febb815eb00 2024-04-24T23:23:01.111 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:01.111 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:23:01.111 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:23:01.111 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-24T23:23:01.111 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-24T23:23:01.111 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:01.111 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:23:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feba406be30 msgr2=0x7feba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feba406be30 0x7feba406e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7febb0033000 tx=0x7febb0009000 comp rx=0 tx=0).stop 2024-04-24T23:23:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb815eb00 msgr2=0x7febb81f3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb815eb00 0x7febb81f3f80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feba800c550 tx=0x7feba800ca10 comp rx=0 tx=0).stop 2024-04-24T23:23:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 shutdown_connections 2024-04-24T23:23:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feba406be30 0x7feba406e2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febb815eb00 0x7febb81f3f80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febb8143c40 0x7febb813ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 --2- 172.21.15.26:0/3057871459 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febb80cc860 0x7febb81378e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 >> 172.21.15.26:0/3057871459 conn(0x7febb806b300 msgr2=0x7febb80b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 shutdown_connections 2024-04-24T23:23:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.111+0000 7febbecca640 1 -- 172.21.15.26:0/3057871459 wait complete. 2024-04-24T23:23:01.123 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:23:01.124 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:23:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.227+0000 7f4d050d4640 1 Processor -- start 2024-04-24T23:23:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.227+0000 7f4d050d4640 1 -- start start 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4d050d4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4d0006e230 0x7f4d00154670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4d050d4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d000cde40 con 0x7f4d0006de50 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4d050d4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d000b50d0 con 0x7f4d00154bb0 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4d050d4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4d000b5380 con 0x7f4d0006e230 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4d0006de50 0x7f4d000b49c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51352/0 (socket says 172.21.15.26:51352) 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfed9d640 1 -- 172.21.15.26:0/3025663810 learned_addr learned my addr 172.21.15.26:0/3025663810 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cf7fff640 1 --2- 172.21.15.26:0/3025663810 >> v2:172.21.15.29:3300/0 conn(0x7f4d0006e230 0x7f4d00154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349182173 0 0) 0x7f4cf0001110 con 0x7f4d0006de50 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cec003440 con 0x7f4d0006de50 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 796262250 0 0) 0x7f4cec003440 con 0x7f4d0006de50 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 >> v2:172.21.15.29:3300/0 conn(0x7f4d0006e230 msgr2=0x7f4d00154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 --2- 172.21.15.26:0/3025663810 >> v2:172.21.15.29:3300/0 conn(0x7f4d0006e230 0x7f4d00154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 >> v1:172.21.15.29:6789/0 conn(0x7f4d00154bb0 legacy=0x7f4d0015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0015ace0 con 0x7f4d0006de50 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f4cf0001840 con 0x7f4d0006de50 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 >> v1:172.21.15.26:6789/0 conn(0x7f4d0006de50 legacy=0x7f4d000b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cec003aa0 0x7f4cec003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec008850 0x7f4cec00acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cec00b4c0 0x7f4cec00d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d000b5380 con 0x7f4cec008850 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec00e260 con 0x7f4cec003aa0 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4cec00e460 con 0x7f4cec00b4c0 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.228+0000 7f4cf7fff640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec008850 0x7f4cec00acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cfed9d640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cec003aa0 0x7f4cec003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cff59e640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cec00b4c0 0x7f4cec00d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cf7fff640 1 -- 172.21.15.26:0/3025663810 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cec00b4c0 msgr2=0x7f4cec00d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cf7fff640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cec00b4c0 0x7f4cec00d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cf7fff640 1 -- 172.21.15.26:0/3025663810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cec003aa0 msgr2=0x7f4cec003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cf7fff640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cec003aa0 0x7f4cec003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cf7fff640 1 -- 172.21.15.26:0/3025663810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec003440 con 0x7f4cec008850 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cff59e640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cec00b4c0 0x7f4cec00d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cff59e640 1 -- 172.21.15.26:0/3025663810 reap_dead start 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.229+0000 7f4cf7fff640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec008850 0x7f4cec00acc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009ac0 tx=0x7f4cec003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713a14a9f35eb387 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8036070 con 0x7f4cec008850 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce8031400 con 0x7f4cec008850 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4cfe59c640 1 -- 172.21.15.26:0/3025663810 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8009690 con 0x7f4cec008850 2024-04-24T23:23:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4d050d4640 1 -- 172.21.15.26:0/3025663810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec008850 msgr2=0x7f4cec00acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4d050d4640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec008850 0x7f4cec00acc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009ac0 tx=0x7f4cec003730 comp rx=0 tx=0).stop 2024-04-24T23:23:01.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4d050d4640 1 -- 172.21.15.26:0/3025663810 shutdown_connections 2024-04-24T23:23:01.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4d050d4640 1 --2- 172.21.15.26:0/3025663810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cec008850 0x7f4cec00acc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4d050d4640 1 -- 172.21.15.26:0/3025663810 >> 172.21.15.26:0/3025663810 conn(0x7f4d0006b300 msgr2=0x7f4d0006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:01.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4d050d4640 1 -- 172.21.15.26:0/3025663810 shutdown_connections 2024-04-24T23:23:01.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.230+0000 7f4d050d4640 1 -- 172.21.15.26:0/3025663810 wait complete. 2024-04-24T23:23:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4d050d4640 1 Processor -- start 2024-04-24T23:23:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4d050d4640 1 -- start start 2024-04-24T23:23:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4d050d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cec008850 0x7f4d000c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4d050d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d001f4fe0 0x7f4d001518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4d050d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d000c4e60 0x7f4d000c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4d050d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0015b580 con 0x7f4d001f4fe0 2024-04-24T23:23:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4d050d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0015b2a0 con 0x7f4cec008850 2024-04-24T23:23:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4d050d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4d00155020 con 0x7f4d000c4e60 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4cf7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d001f4fe0 0x7f4d001518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.232+0000 7f4cf7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d001f4fe0 0x7f4d001518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33570/0 (socket says 172.21.15.26:33570) 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf7fff640 1 -- 172.21.15.26:0/1449169148 learned_addr learned my addr 172.21.15.26:0/1449169148 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cff59e640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d000c4e60 0x7f4d000c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cfed9d640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cec008850 0x7f4d000c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf7fff640 1 -- 172.21.15.26:0/1449169148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d000c4e60 msgr2=0x7f4d000c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf7fff640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d000c4e60 0x7f4d000c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf7fff640 1 -- 172.21.15.26:0/1449169148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cec008850 msgr2=0x7f4d000c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf7fff640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cec008850 0x7f4d000c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf7fff640 1 -- 172.21.15.26:0/1449169148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d001f5460 con 0x7f4d001f4fe0 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf7fff640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d001f4fe0 0x7f4d001518d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034040 tx=0x7f4ce8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf77fe640 1 -- 172.21.15.26:0/1449169148 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce803f020 con 0x7f4d001f4fe0 2024-04-24T23:23:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf77fe640 1 -- 172.21.15.26:0/1449169148 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce8037040 con 0x7f4d001f4fe0 2024-04-24T23:23:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d000c6150 con 0x7f4d001f4fe0 2024-04-24T23:23:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4cf77fe640 1 -- 172.21.15.26:0/1449169148 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8036070 con 0x7f4d001f4fe0 2024-04-24T23:23:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.233+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d000ada80 con 0x7f4d001f4fe0 2024-04-24T23:23:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.234+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d000c6670 con 0x7f4d001f4fe0 2024-04-24T23:23:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.235+0000 7f4cf77fe640 1 -- 172.21.15.26:0/1449169148 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ce8042020 con 0x7f4d001f4fe0 2024-04-24T23:23:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.235+0000 7f4cf77fe640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ce006be30 0x7f4ce006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.235+0000 7f4cf77fe640 1 -- 172.21.15.26:0/1449169148 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f4ce80aaa40 con 0x7f4d001f4fe0 2024-04-24T23:23:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.235+0000 7f4cfed9d640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ce006be30 0x7f4ce006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.236+0000 7f4cfed9d640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ce006be30 0x7f4ce006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00013d0 tx=0x7f4cf0000ef0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.240+0000 7f4cf77fe640 1 -- 172.21.15.26:0/1449169148 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ce8049030 con 0x7f4d001f4fe0 2024-04-24T23:23:01.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.422+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4d000ad870 con 0x7f4d001f4fe0 2024-04-24T23:23:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.781+0000 7f4cf77fe640 1 -- 172.21.15.26:0/1449169148 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) ==== 138+0+0 (secure 0 0 0) 0x7f4ce80aae20 con 0x7f4d001f4fe0 2024-04-24T23:23:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ce006be30 msgr2=0x7f4ce006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ce006be30 0x7f4ce006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00013d0 tx=0x7f4cf0000ef0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d001f4fe0 msgr2=0x7f4d001518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d001f4fe0 0x7f4d001518d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034040 tx=0x7f4ce8004730 comp rx=0 tx=0).stop 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 shutdown_connections 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ce006be30 0x7f4ce006e2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d000c4e60 0x7f4d000c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d001f4fe0 0x7f4d001518d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 --2- 172.21.15.26:0/1449169148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cec008850 0x7f4d000c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 >> 172.21.15.26:0/1449169148 conn(0x7f4d0006b300 msgr2=0x7f4d00153aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 shutdown_connections 2024-04-24T23:23:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.783+0000 7f4d050d4640 1 -- 172.21.15.26:0/1449169148 wait complete. 2024-04-24T23:23:01.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:23:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.907+0000 7f28627c1640 1 Processor -- start 2024-04-24T23:23:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.907+0000 7f28627c1640 1 -- start start 2024-04-24T23:23:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.907+0000 7f28627c1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f285c15adc0 0x7f285c0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.907+0000 7f28627c1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f285c0ad7b0 0x7f285c0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.907+0000 7f28627c1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f285c15ef60 con 0x7f285c15e9c0 2024-04-24T23:23:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.907+0000 7f28627c1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f285c15f240 con 0x7f285c0ad7b0 2024-04-24T23:23:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.907+0000 7f28627c1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f285c15f520 con 0x7f285c15adc0 2024-04-24T23:23:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.908+0000 7f285b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f285c0ad7b0 0x7f285c0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.908+0000 7f285b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f285c0ad7b0 0x7f285c0d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33574/0 (socket says 172.21.15.26:33574) 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.908+0000 7f285b7fe640 1 -- 172.21.15.26:0/1214156295 learned_addr learned my addr 172.21.15.26:0/1214156295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.908+0000 7f285bfff640 1 --2- 172.21.15.26:0/1214156295 >> v2:172.21.15.29:3300/0 conn(0x7f285c15adc0 0x7f285c0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.908+0000 7f285b7fe640 1 -- 172.21.15.26:0/1214156295 >> v2:172.21.15.29:3300/0 conn(0x7f285c15adc0 msgr2=0x7f285c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.908+0000 7f285b7fe640 1 --2- 172.21.15.26:0/1214156295 >> v2:172.21.15.29:3300/0 conn(0x7f285c15adc0 0x7f285c0b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.908+0000 7f285b7fe640 1 -- 172.21.15.26:0/1214156295 >> v1:172.21.15.29:6789/0 conn(0x7f285c15e9c0 legacy=0x7f285c06dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.908+0000 7f285b7fe640 1 -- 172.21.15.26:0/1214156295 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c15f800 con 0x7f285c0ad7b0 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f285b7fe640 1 --2- 172.21.15.26:0/1214156295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0ad7b0 0x7f285c0d0040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2850009870 tx=0x7f285002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fb4e6855c6e87d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f285affd640 1 -- 172.21.15.26:0/1214156295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850037070 con 0x7f285c0ad7b0 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f285affd640 1 -- 172.21.15.26:0/1214156295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285002ae40 con 0x7f285c0ad7b0 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f285affd640 1 -- 172.21.15.26:0/1214156295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850032660 con 0x7f285c0ad7b0 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f28627c1640 1 -- 172.21.15.26:0/1214156295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0ad7b0 msgr2=0x7f285c0d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f28627c1640 1 --2- 172.21.15.26:0/1214156295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0ad7b0 0x7f285c0d0040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2850009870 tx=0x7f285002a130 comp rx=0 tx=0).stop 2024-04-24T23:23:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f28627c1640 1 -- 172.21.15.26:0/1214156295 shutdown_connections 2024-04-24T23:23:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f28627c1640 1 --2- 172.21.15.26:0/1214156295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c0ad7b0 0x7f285c0d0040 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f28627c1640 1 --2- 172.21.15.26:0/1214156295 >> v2:172.21.15.29:3300/0 conn(0x7f285c15adc0 0x7f285c0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f28627c1640 1 -- 172.21.15.26:0/1214156295 >> 172.21.15.26:0/1214156295 conn(0x7f285c06b540 msgr2=0x7f285c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f28627c1640 1 -- 172.21.15.26:0/1214156295 shutdown_connections 2024-04-24T23:23:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.909+0000 7f28627c1640 1 -- 172.21.15.26:0/1214156295 wait complete. 2024-04-24T23:23:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.911+0000 7f28627c1640 1 Processor -- start 2024-04-24T23:23:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.911+0000 7f28627c1640 1 -- start start 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.911+0000 7f28627c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c0ad7b0 0x7f285c0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.911+0000 7f28627c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c15adc0 0x7f285c14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.911+0000 7f28627c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e9c0 0x7f285c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.911+0000 7f28627c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f285c15b1a0 con 0x7f285c15e9c0 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.911+0000 7f28627c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f285c0b48b0 con 0x7f285c15adc0 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.911+0000 7f28627c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f285c0d0580 con 0x7f285c0ad7b0 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2860d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e9c0 0x7f285c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2860d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e9c0 0x7f285c152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33590/0 (socket says 172.21.15.26:33590) 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2860d5e640 1 -- 172.21.15.26:0/1408177233 learned_addr learned my addr 172.21.15.26:0/1408177233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f285bfff640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c0ad7b0 0x7f285c0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f285b7fe640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c15adc0 0x7f285c14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2860d5e640 1 -- 172.21.15.26:0/1408177233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c0ad7b0 msgr2=0x7f285c0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2860d5e640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c0ad7b0 0x7f285c0cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2860d5e640 1 -- 172.21.15.26:0/1408177233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c15adc0 msgr2=0x7f285c14e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2860d5e640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c15adc0 0x7f285c14e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2860d5e640 1 -- 172.21.15.26:0/1408177233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c152610 con 0x7f285c15e9c0 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2860d5e640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e9c0 0x7f285c152dd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f285400b990 tx=0x7f285400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2858ff9640 1 -- 172.21.15.26:0/1408177233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854019070 con 0x7f285c15e9c0 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2858ff9640 1 -- 172.21.15.26:0/1408177233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285400cba0 con 0x7f285c15e9c0 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f2858ff9640 1 -- 172.21.15.26:0/1408177233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854014440 con 0x7f285c15e9c0 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c14fd90 con 0x7f285c15e9c0 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.912+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c14ebd0 con 0x7f285c15e9c0 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.914+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285c075610 con 0x7f285c15e9c0 2024-04-24T23:23:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.914+0000 7f2858ff9640 1 -- 172.21.15.26:0/1408177233 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f285401c070 con 0x7f285c15e9c0 2024-04-24T23:23:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.914+0000 7f2858ff9640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.915+0000 7f2858ff9640 1 -- 172.21.15.26:0/1408177233 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5227+0+0 (secure 0 0 0) 0x7f285408aa20 con 0x7f285c15e9c0 2024-04-24T23:23:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.915+0000 7f285bfff640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.915+0000 7f285bfff640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f284c00aac0 tx=0x7f284c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:01.919+0000 7f2858ff9640 1 -- 172.21.15.26:0/1408177233 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2854058ed0 con 0x7f285c15e9c0 2024-04-24T23:23:02.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.105+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f285c15e730 con 0x7f285c15e9c0 2024-04-24T23:23:02.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.786+0000 7f2858ff9640 1 -- 172.21.15.26:0/1408177233 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) ==== 170+0+0 (secure 0 0 0) 0x7f285405d1d0 con 0x7f285c15e9c0 2024-04-24T23:23:02.788 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-24T23:23:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 msgr2=0x7f284806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f284c00aac0 tx=0x7f284c008040 comp rx=0 tx=0).stop 2024-04-24T23:23:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e9c0 msgr2=0x7f285c152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e9c0 0x7f285c152dd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f285400b990 tx=0x7f285400be50 comp rx=0 tx=0).stop 2024-04-24T23:23:02.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 shutdown_connections 2024-04-24T23:23:02.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f284806be30 0x7f284806e2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:02.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f285c15e9c0 0x7f285c152dd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:02.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f285c15adc0 0x7f285c14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:02.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 --2- 172.21.15.26:0/1408177233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f285c0ad7b0 0x7f285c0cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:02.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.790+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 >> 172.21.15.26:0/1408177233 conn(0x7f285c06b540 msgr2=0x7f285c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.794+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 shutdown_connections 2024-04-24T23:23:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.795+0000 7f28627c1640 1 -- 172.21.15.26:0/1408177233 wait complete. 2024-04-24T23:23:02.817 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:23:02.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.924+0000 7f87ec43f640 1 Processor -- start 2024-04-24T23:23:02.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.924+0000 7f87ec43f640 1 -- start start 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.925+0000 7f87ec43f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e40d0750 0x7f87e4165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.925+0000 7f87ec43f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87e415ef60 con 0x7f87e40ad7b0 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.925+0000 7f87ec43f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87e415f240 con 0x7f87e415adc0 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.925+0000 7f87ec43f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f87e415f520 con 0x7f87e40d0750 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.925+0000 7f87e99da640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87e40ad7b0 0x7f87e40d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51360/0 (socket says 172.21.15.26:51360) 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.925+0000 7f87e99da640 1 -- 172.21.15.26:0/3603881483 learned_addr learned my addr 172.21.15.26:0/3603881483 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.925+0000 7f87ea9dc640 1 --2- 172.21.15.26:0/3603881483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e40d0750 0x7f87e4165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.925+0000 7f87e91d9640 1 -- 172.21.15.26:0/3603881483 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 802304846 0 0) 0x7f87e415ef60 con 0x7f87e40ad7b0 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.925+0000 7f87e91d9640 1 -- 172.21.15.26:0/3603881483 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87d0003520 con 0x7f87e40ad7b0 2024-04-24T23:23:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.926+0000 7f87e91d9640 1 -- 172.21.15.26:0/3603881483 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 788339356 0 0) 0x7f87e415f240 con 0x7f87e415adc0 2024-04-24T23:23:02.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.926+0000 7f87e91d9640 1 -- 172.21.15.26:0/3603881483 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87e415ef60 con 0x7f87e415adc0 2024-04-24T23:23:02.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.926+0000 7f87ea9dc640 1 -- 172.21.15.26:0/3603881483 >> v1:172.21.15.29:6789/0 conn(0x7f87e415adc0 legacy=0x7f87e40b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:02.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.926+0000 7f87ea9dc640 1 -- 172.21.15.26:0/3603881483 >> v1:172.21.15.26:6789/0 conn(0x7f87e40ad7b0 legacy=0x7f87e40d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:02.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.926+0000 7f87ea9dc640 1 -- 172.21.15.26:0/3603881483 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e415f800 con 0x7f87e40d0750 2024-04-24T23:23:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.926+0000 7f87ea9dc640 1 --2- 172.21.15.26:0/3603881483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e40d0750 0x7f87e4165d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009df0 tx=0x7f87dc02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd2a1b843a44979f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87e91d9640 1 -- 172.21.15.26:0/3603881483 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc02ae60 con 0x7f87e40d0750 2024-04-24T23:23:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87e91d9640 1 -- 172.21.15.26:0/3603881483 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87dc007740 con 0x7f87e40d0750 2024-04-24T23:23:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87e91d9640 1 -- 172.21.15.26:0/3603881483 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc007b10 con 0x7f87e40d0750 2024-04-24T23:23:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87ec43f640 1 -- 172.21.15.26:0/3603881483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e40d0750 msgr2=0x7f87e4165d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87ec43f640 1 --2- 172.21.15.26:0/3603881483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e40d0750 0x7f87e4165d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009df0 tx=0x7f87dc02a680 comp rx=0 tx=0).stop 2024-04-24T23:23:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87ec43f640 1 -- 172.21.15.26:0/3603881483 shutdown_connections 2024-04-24T23:23:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87ec43f640 1 --2- 172.21.15.26:0/3603881483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e40d0750 0x7f87e4165d50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:02.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87ec43f640 1 -- 172.21.15.26:0/3603881483 >> 172.21.15.26:0/3603881483 conn(0x7f87e406b540 msgr2=0x7f87e406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:02.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87ec43f640 1 -- 172.21.15.26:0/3603881483 shutdown_connections 2024-04-24T23:23:02.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.927+0000 7f87ec43f640 1 -- 172.21.15.26:0/3603881483 wait complete. 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.928+0000 7f87ec43f640 1 Processor -- start 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.928+0000 7f87ec43f640 1 -- start start 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ec43f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40ad7b0 0x7f87e4149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ec43f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87e40d0750 0x7f87e4159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ec43f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e415adc0 0x7f87e409d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ec43f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f87e415e9c0 con 0x7f87e40ad7b0 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ec43f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f87e415b1a0 con 0x7f87e40d0750 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ec43f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f87e415eb40 con 0x7f87e415adc0 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ea1db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40ad7b0 0x7f87e4149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ea1db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40ad7b0 0x7f87e4149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33602/0 (socket says 172.21.15.26:33602) 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ea1db640 1 -- 172.21.15.26:0/3299511240 learned_addr learned my addr 172.21.15.26:0/3299511240 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87ea9dc640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e415adc0 0x7f87e409d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.929+0000 7f87e99da640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87e40d0750 0x7f87e4159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87ea1db640 1 -- 172.21.15.26:0/3299511240 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e415adc0 msgr2=0x7f87e409d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87ea1db640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e415adc0 0x7f87e409d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87ea1db640 1 -- 172.21.15.26:0/3299511240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87e40d0750 msgr2=0x7f87e4159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87ea1db640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87e40d0750 0x7f87e4159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87ea1db640 1 -- 172.21.15.26:0/3299511240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e4159d50 con 0x7f87e40ad7b0 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87ea1db640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40ad7b0 0x7f87e4149050 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f87e000c430 tx=0x7f87e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87daffd640 1 -- 172.21.15.26:0/3299511240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0017070 con 0x7f87e40ad7b0 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87daffd640 1 -- 172.21.15.26:0/3299511240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e00052f0 con 0x7f87e40ad7b0 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e4158770 con 0x7f87e40ad7b0 2024-04-24T23:23:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87daffd640 1 -- 172.21.15.26:0/3299511240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0003830 con 0x7f87e40ad7b0 2024-04-24T23:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.930+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e4158d30 con 0x7f87e40ad7b0 2024-04-24T23:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.931+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e4075610 con 0x7f87e40ad7b0 2024-04-24T23:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.932+0000 7f87daffd640 1 -- 172.21.15.26:0/3299511240 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87e001a070 con 0x7f87e40ad7b0 2024-04-24T23:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.932+0000 7f87daffd640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c806be30 0x7f87c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.932+0000 7f87daffd640 1 -- 172.21.15.26:0/3299511240 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5609+0+0 (secure 0 0 0) 0x7f87e008a140 con 0x7f87e40ad7b0 2024-04-24T23:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.933+0000 7f87e99da640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c806be30 0x7f87c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.933+0000 7f87e99da640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c806be30 0x7f87c806e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f87e41f7510 tx=0x7f87d4007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:02.937+0000 7f87daffd640 1 -- 172.21.15.26:0/3299511240 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f87e0058500 con 0x7f87e40ad7b0 2024-04-24T23:23:03.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.162+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f87e415e730 con 0x7f87e40ad7b0 2024-04-24T23:23:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.801+0000 7f87daffd640 1 -- 172.21.15.26:0/3299511240 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v29) ==== 172+0+0 (secure 0 0 0) 0x7f87e005c800 con 0x7f87e40ad7b0 2024-04-24T23:23:03.810 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 6 and data pool 7 2024-04-24T23:23:03.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.813+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c806be30 msgr2=0x7f87c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:03.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.813+0000 7f87ec43f640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c806be30 0x7f87c806e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f87e41f7510 tx=0x7f87d4007d30 comp rx=0 tx=0).stop 2024-04-24T23:23:03.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.813+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40ad7b0 msgr2=0x7f87e4149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:03.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.813+0000 7f87ec43f640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40ad7b0 0x7f87e4149050 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f87e000c430 tx=0x7f87e000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.814+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 shutdown_connections 2024-04-24T23:23:03.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.814+0000 7f87ec43f640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c806be30 0x7f87c806e2f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.814+0000 7f87ec43f640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87e415adc0 0x7f87e409d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.814+0000 7f87ec43f640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87e40d0750 0x7f87e4159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.814+0000 7f87ec43f640 1 --2- 172.21.15.26:0/3299511240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87e40ad7b0 0x7f87e4149050 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.814+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 >> 172.21.15.26:0/3299511240 conn(0x7f87e406b540 msgr2=0x7f87e415b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:03.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.814+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 shutdown_connections 2024-04-24T23:23:03.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.815+0000 7f87ec43f640 1 -- 172.21.15.26:0/3299511240 wait complete. 2024-04-24T23:23:03.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:23:03.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.939+0000 7f58f7e38640 1 Processor -- start 2024-04-24T23:23:03.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.940+0000 7f58f7e38640 1 -- start start 2024-04-24T23:23:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.940+0000 7f58f7e38640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f0143c40 0x7f58f00cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.940+0000 7f58f7e38640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58f015eb00 0x7f58f015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.940+0000 7f58f7e38640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f00cd0b0 con 0x7f58f00cc860 2024-04-24T23:23:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.940+0000 7f58f7e38640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f58f015f490 con 0x7f58f015eb00 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.940+0000 7f58f7e38640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f58f0071b70 con 0x7f58f0143c40 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f5bd4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f0143c40 0x7f58f00cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f63d5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58f015eb00 0x7f58f015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f63d5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f58f015eb00 0x7f58f015ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:53886/0 (socket says 172.21.15.26:53886) 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f5bd4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f0143c40 0x7f58f00cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:48726/0 (socket says 172.21.15.26:48726) 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f63d5640 1 -- 172.21.15.26:0/2487543911 learned_addr learned my addr 172.21.15.26:0/2487543911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f63d5640 1 -- 172.21.15.26:0/2487543911 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f0143c40 msgr2=0x7f58f00cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f63d5640 1 --2- 172.21.15.26:0/2487543911 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f0143c40 0x7f58f00cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f63d5640 1 -- 172.21.15.26:0/2487543911 >> v1:172.21.15.29:6789/0 conn(0x7f58f00cc860 legacy=0x7f58f0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f63d5640 1 -- 172.21.15.26:0/2487543911 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f0071240 con 0x7f58f015eb00 2024-04-24T23:23:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.941+0000 7f58f5bd4640 1 --2- 172.21.15.26:0/2487543911 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f0143c40 0x7f58f00cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:03.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.942+0000 7f58f63d5640 1 --2- 172.21.15.26:0/2487543911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58f015eb00 0x7f58f015ef50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009e50 tx=0x7f58e802c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f40041a0ad0155 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:03.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.942+0000 7f58f4bd2640 1 -- 172.21.15.26:0/2487543911 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8032610 con 0x7f58f015eb00 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.942+0000 7f58f4bd2640 1 -- 172.21.15.26:0/2487543911 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e8037070 con 0x7f58f015eb00 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.942+0000 7f58f4bd2640 1 -- 172.21.15.26:0/2487543911 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80329d0 con 0x7f58f015eb00 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.943+0000 7f58f7e38640 1 -- 172.21.15.26:0/2487543911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58f015eb00 msgr2=0x7f58f015ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.943+0000 7f58f7e38640 1 --2- 172.21.15.26:0/2487543911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58f015eb00 0x7f58f015ef50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009e50 tx=0x7f58e802c920 comp rx=0 tx=0).stop 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.943+0000 7f58f7e38640 1 -- 172.21.15.26:0/2487543911 shutdown_connections 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.943+0000 7f58f7e38640 1 --2- 172.21.15.26:0/2487543911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58f015eb00 0x7f58f015ef50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.943+0000 7f58f7e38640 1 --2- 172.21.15.26:0/2487543911 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f0143c40 0x7f58f00cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.943+0000 7f58f7e38640 1 -- 172.21.15.26:0/2487543911 >> 172.21.15.26:0/2487543911 conn(0x7f58f006b300 msgr2=0x7f58f006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.943+0000 7f58f7e38640 1 -- 172.21.15.26:0/2487543911 shutdown_connections 2024-04-24T23:23:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.943+0000 7f58f7e38640 1 -- 172.21.15.26:0/2487543911 wait complete. 2024-04-24T23:23:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.944+0000 7f58f7e38640 1 Processor -- start 2024-04-24T23:23:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.944+0000 7f58f7e38640 1 -- start start 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f7e38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f00cc860 0x7f58f013cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f7e38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58f0143c40 0x7f58f01e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f7e38640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f015eb00 0x7f58f01e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f7e38640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0144020 con 0x7f58f00cc860 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f7e38640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0071800 con 0x7f58f0143c40 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f7e38640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f58f00ccca0 con 0x7f58f015eb00 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f5bd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f00cc860 0x7f58f013cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f5bd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f00cc860 0x7f58f013cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45528/0 (socket says 172.21.15.26:45528) 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f5bd4640 1 -- 172.21.15.26:0/120711782 learned_addr learned my addr 172.21.15.26:0/120711782 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f63d5640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f015eb00 0x7f58f01e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f53d3640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58f0143c40 0x7f58f01e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f5bd4640 1 -- 172.21.15.26:0/120711782 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f015eb00 msgr2=0x7f58f01e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f5bd4640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f015eb00 0x7f58f01e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f5bd4640 1 -- 172.21.15.26:0/120711782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58f0143c40 msgr2=0x7f58f01e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f5bd4640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58f0143c40 0x7f58f01e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.945+0000 7f58f5bd4640 1 -- 172.21.15.26:0/120711782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f00ca130 con 0x7f58f00cc860 2024-04-24T23:23:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.946+0000 7f58f5bd4640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f00cc860 0x7f58f013cef0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00ebe0 tx=0x7f58ec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.946+0000 7f58e67fc640 1 -- 172.21.15.26:0/120711782 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec017070 con 0x7f58f00cc860 2024-04-24T23:23:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.946+0000 7f58e67fc640 1 -- 172.21.15.26:0/120711782 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ec00cc30 con 0x7f58f00cc860 2024-04-24T23:23:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.946+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f00ca410 con 0x7f58f00cc860 2024-04-24T23:23:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.946+0000 7f58e67fc640 1 -- 172.21.15.26:0/120711782 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec013470 con 0x7f58f00cc860 2024-04-24T23:23:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.946+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f00ca9d0 con 0x7f58f00cc860 2024-04-24T23:23:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.947+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f00694c0 con 0x7f58f00cc860 2024-04-24T23:23:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.947+0000 7f58e67fc640 1 -- 172.21.15.26:0/120711782 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58ec01a070 con 0x7f58f00cc860 2024-04-24T23:23:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.948+0000 7f58e67fc640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58d406be30 0x7f58d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.948+0000 7f58e67fc640 1 -- 172.21.15.26:0/120711782 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f58ec08b040 con 0x7f58f00cc860 2024-04-24T23:23:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.948+0000 7f58f53d3640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58d406be30 0x7f58d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.948+0000 7f58f53d3640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58d406be30 0x7f58d406e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58f01f7380 tx=0x7f58e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:03.952+0000 7f58e67fc640 1 -- 172.21.15.26:0/120711782 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58ec059430 con 0x7f58f00cc860 2024-04-24T23:23:04.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.136+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58f00b4d20 con 0x7f58f00cc860 2024-04-24T23:23:04.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.137+0000 7f58e67fc640 1 -- 172.21.15.26:0/120711782 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16566 (secure 0 0 0) 0x7f58ec05d730 con 0x7f58f00cc860 2024-04-24T23:23:04.140 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:04.140 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":43,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:23:03.163813+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:23:01.453865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:23:02.136971+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:23:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.139+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58d406be30 msgr2=0x7f58d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.139+0000 7f58f7e38640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58d406be30 0x7f58d406e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58f01f7380 tx=0x7f58e0009040 comp rx=0 tx=0).stop 2024-04-24T23:23:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.139+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f00cc860 msgr2=0x7f58f013cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.139+0000 7f58f7e38640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f00cc860 0x7f58f013cef0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00ebe0 tx=0x7f58ec00c520 comp rx=0 tx=0).stop 2024-04-24T23:23:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.139+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 shutdown_connections 2024-04-24T23:23:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.139+0000 7f58f7e38640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f58d406be30 0x7f58d406e2f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.139+0000 7f58f7e38640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f58f015eb00 0x7f58f01e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.139+0000 7f58f7e38640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58f0143c40 0x7f58f01e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.140+0000 7f58f7e38640 1 --2- 172.21.15.26:0/120711782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f00cc860 0x7f58f013cef0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.140+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 >> 172.21.15.26:0/120711782 conn(0x7f58f006b300 msgr2=0x7f58f00b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.140+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 shutdown_connections 2024-04-24T23:23:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.140+0000 7f58f7e38640 1 -- 172.21.15.26:0/120711782 wait complete. 2024-04-24T23:23:04.153 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:23:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.252+0000 7fe62a790640 1 Processor -- start 2024-04-24T23:23:04.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.252+0000 7fe62a790640 1 -- start start 2024-04-24T23:23:04.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe62a790640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe62415eb00 0x7fe62415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe62a790640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe62415f450 con 0x7fe624143c40 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe62a790640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6240cd0b0 con 0x7fe6240cf870 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe62a790640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe624071b70 con 0x7fe62415eb00 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe623fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe624143c40 0x7fe6240cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49902/0 (socket says 172.21.15.26:49902) 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe623fff640 1 -- 172.21.15.26:0/2776921794 learned_addr learned my addr 172.21.15.26:0/2776921794 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe628d2d640 1 --2- 172.21.15.26:0/2776921794 >> v2:172.21.15.29:3300/0 conn(0x7fe62415eb00 0x7fe62415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe622ffd640 1 -- 172.21.15.26:0/2776921794 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2011555798 0 0) 0x7fe618001110 con 0x7fe624143c40 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe622ffd640 1 -- 172.21.15.26:0/2776921794 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe60c003520 con 0x7fe624143c40 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe628d2d640 1 -- 172.21.15.26:0/2776921794 >> v1:172.21.15.29:6789/0 conn(0x7fe6240cf870 legacy=0x7fe624072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe622ffd640 1 -- 172.21.15.26:0/2776921794 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3462672466 0 0) 0x7fe60c003520 con 0x7fe624143c40 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe628d2d640 1 -- 172.21.15.26:0/2776921794 >> v1:172.21.15.26:6789/0 conn(0x7fe624143c40 legacy=0x7fe6240cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.253+0000 7fe628d2d640 1 -- 172.21.15.26:0/2776921794 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe624071240 con 0x7fe62415eb00 2024-04-24T23:23:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.254+0000 7fe628d2d640 1 --2- 172.21.15.26:0/2776921794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62415eb00 0x7fe62415ef10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009d90 tx=0x7fe61c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5800f109ebd2758 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.254+0000 7fe622ffd640 1 -- 172.21.15.26:0/2776921794 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618001840 con 0x7fe62415eb00 2024-04-24T23:23:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.254+0000 7fe622ffd640 1 -- 172.21.15.26:0/2776921794 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe61c007540 con 0x7fe62415eb00 2024-04-24T23:23:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.254+0000 7fe622ffd640 1 -- 172.21.15.26:0/2776921794 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c007910 con 0x7fe62415eb00 2024-04-24T23:23:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.255+0000 7fe62a790640 1 -- 172.21.15.26:0/2776921794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62415eb00 msgr2=0x7fe62415ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.255+0000 7fe62a790640 1 --2- 172.21.15.26:0/2776921794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62415eb00 0x7fe62415ef10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009d90 tx=0x7fe61c02a680 comp rx=0 tx=0).stop 2024-04-24T23:23:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.255+0000 7fe62a790640 1 -- 172.21.15.26:0/2776921794 shutdown_connections 2024-04-24T23:23:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.255+0000 7fe62a790640 1 --2- 172.21.15.26:0/2776921794 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62415eb00 0x7fe62415ef10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.255+0000 7fe62a790640 1 -- 172.21.15.26:0/2776921794 >> 172.21.15.26:0/2776921794 conn(0x7fe62406b300 msgr2=0x7fe62406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.255+0000 7fe62a790640 1 -- 172.21.15.26:0/2776921794 shutdown_connections 2024-04-24T23:23:04.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.255+0000 7fe62a790640 1 -- 172.21.15.26:0/2776921794 wait complete. 2024-04-24T23:23:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.256+0000 7fe62a790640 1 Processor -- start 2024-04-24T23:23:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.256+0000 7fe62a790640 1 -- start start 2024-04-24T23:23:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe62a790640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe6240cf870 0x7fe6241f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe62a790640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624143c40 0x7fe62412fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe62a790640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62415eb00 0x7fe62415c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe62a790640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe624144020 con 0x7fe624143c40 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe62a790640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe624071800 con 0x7fe62415eb00 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe62a790640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe6240b2e80 con 0x7fe6240cf870 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe6237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624143c40 0x7fe62412fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe6237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624143c40 0x7fe62412fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45542/0 (socket says 172.21.15.26:45542) 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe6237fe640 1 -- 172.21.15.26:0/3948331786 learned_addr learned my addr 172.21.15.26:0/3948331786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe623fff640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe6240cf870 0x7fe6241f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe628d2d640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62415eb00 0x7fe62415c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe6237fe640 1 -- 172.21.15.26:0/3948331786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe6240cf870 msgr2=0x7fe6241f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe6237fe640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe6240cf870 0x7fe6241f1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe6237fe640 1 -- 172.21.15.26:0/3948331786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62415eb00 msgr2=0x7fe62415c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe6237fe640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62415eb00 0x7fe62415c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.257+0000 7fe6237fe640 1 -- 172.21.15.26:0/3948331786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe624130370 con 0x7fe624143c40 2024-04-24T23:23:04.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.258+0000 7fe6237fe640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624143c40 0x7fe62412fa40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c450 tx=0x7fe61400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.258+0000 7fe620ff9640 1 -- 172.21.15.26:0/3948331786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140052d0 con 0x7fe624143c40 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.258+0000 7fe620ff9640 1 -- 172.21.15.26:0/3948331786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe614005470 con 0x7fe624143c40 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.258+0000 7fe620ff9640 1 -- 172.21.15.26:0/3948331786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614012870 con 0x7fe624143c40 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.258+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe624130650 con 0x7fe624143c40 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.258+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe624130c10 con 0x7fe624143c40 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.259+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62415b3c0 con 0x7fe624143c40 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.259+0000 7fe620ff9640 1 -- 172.21.15.26:0/3948331786 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe614012a10 con 0x7fe624143c40 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.260+0000 7fe620ff9640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe61006be30 0x7fe61006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.260+0000 7fe623fff640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe61006be30 0x7fe61006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.260+0000 7fe620ff9640 1 -- 172.21.15.26:0/3948331786 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fe61408a9c0 con 0x7fe624143c40 2024-04-24T23:23:04.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.260+0000 7fe623fff640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe61006be30 0x7fe61006e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe618005880 tx=0x7fe618009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:04.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.264+0000 7fe620ff9640 1 -- 172.21.15.26:0/3948331786 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe62415b3c0 con 0x7fe624143c40 2024-04-24T23:23:04.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.449+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6240694c0 con 0x7fe624143c40 2024-04-24T23:23:04.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.450+0000 7fe620ff9640 1 -- 172.21.15.26:0/3948331786 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16566 (secure 0 0 0) 0x7fe61408ce20 con 0x7fe624143c40 2024-04-24T23:23:04.452 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:04.452 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":43,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:23:03.163813+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:23:01.453865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:23:02.136971+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:23:04.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.452+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe61006be30 msgr2=0x7fe61006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.452+0000 7fe62a790640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe61006be30 0x7fe61006e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe618005880 tx=0x7fe618009040 comp rx=0 tx=0).stop 2024-04-24T23:23:04.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624143c40 msgr2=0x7fe62412fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624143c40 0x7fe62412fa40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c450 tx=0x7fe61400c910 comp rx=0 tx=0).stop 2024-04-24T23:23:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 shutdown_connections 2024-04-24T23:23:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe61006be30 0x7fe61006e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe62415eb00 0x7fe62415c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624143c40 0x7fe62412fa40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 --2- 172.21.15.26:0/3948331786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe6240cf870 0x7fe6241f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 >> 172.21.15.26:0/3948331786 conn(0x7fe62406b300 msgr2=0x7fe6240b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 shutdown_connections 2024-04-24T23:23:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.453+0000 7fe62a790640 1 -- 172.21.15.26:0/3948331786 wait complete. 2024-04-24T23:23:04.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:23:04.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.567+0000 7f6b78563640 1 Processor -- start 2024-04-24T23:23:04.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.567+0000 7f6b78563640 1 -- start start 2024-04-24T23:23:04.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b78563640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b700ad7b0 0x7f6b700d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b78563640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6b7015e9c0 0x7f6b700d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b78563640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7015ef60 con 0x7f6b7015adc0 2024-04-24T23:23:04.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b78563640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6b7015f240 con 0x7f6b700ad7b0 2024-04-24T23:23:04.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b78563640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6b7015f520 con 0x7f6b7015e9c0 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b75afe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b700ad7b0 0x7f6b700d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b75afe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b700ad7b0 0x7f6b700d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45550/0 (socket says 172.21.15.26:45550) 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b75afe640 1 -- 172.21.15.26:0/1672699008 learned_addr learned my addr 172.21.15.26:0/1672699008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b762ff640 1 --1- 172.21.15.26:0/1672699008 >> v1:172.21.15.26:6789/0 conn(0x7f6b7015adc0 0x7f6b700b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49918/0 (socket says 172.21.15.26:49918) 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.568+0000 7f6b76b00640 1 --2- 172.21.15.26:0/1672699008 >> v2:172.21.15.29:3300/0 conn(0x7f6b7015e9c0 0x7f6b700d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.569+0000 7f6b75afe640 1 -- 172.21.15.26:0/1672699008 >> v2:172.21.15.29:3300/0 conn(0x7f6b7015e9c0 msgr2=0x7f6b700d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.569+0000 7f6b75afe640 1 --2- 172.21.15.26:0/1672699008 >> v2:172.21.15.29:3300/0 conn(0x7f6b7015e9c0 0x7f6b700d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.569+0000 7f6b75afe640 1 -- 172.21.15.26:0/1672699008 >> v1:172.21.15.26:6789/0 conn(0x7f6b7015adc0 legacy=0x7f6b700b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.569+0000 7f6b75afe640 1 -- 172.21.15.26:0/1672699008 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7015f800 con 0x7f6b700ad7b0 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.569+0000 7f6b752fd640 1 -- 172.21.15.26:0/1672699008 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573805249 0 0) 0x7f6b7015ef60 con 0x7f6b7015adc0 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.569+0000 7f6b75afe640 1 --2- 172.21.15.26:0/1672699008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b700ad7b0 0x7f6b700d0040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009870 tx=0x7f6b6402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6de0a435b72ff50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.569+0000 7f6b752fd640 1 -- 172.21.15.26:0/1672699008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64037070 con 0x7f6b700ad7b0 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.569+0000 7f6b752fd640 1 -- 172.21.15.26:0/1672699008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b64032440 con 0x7f6b700ad7b0 2024-04-24T23:23:04.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.569+0000 7f6b752fd640 1 -- 172.21.15.26:0/1672699008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64032810 con 0x7f6b700ad7b0 2024-04-24T23:23:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.570+0000 7f6b78563640 1 -- 172.21.15.26:0/1672699008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b700ad7b0 msgr2=0x7f6b700d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.570+0000 7f6b78563640 1 --2- 172.21.15.26:0/1672699008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b700ad7b0 0x7f6b700d0040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009870 tx=0x7f6b6402a180 comp rx=0 tx=0).stop 2024-04-24T23:23:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.570+0000 7f6b78563640 1 -- 172.21.15.26:0/1672699008 shutdown_connections 2024-04-24T23:23:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.570+0000 7f6b78563640 1 --2- 172.21.15.26:0/1672699008 >> v2:172.21.15.29:3300/0 conn(0x7f6b7015e9c0 0x7f6b700d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.570+0000 7f6b78563640 1 --2- 172.21.15.26:0/1672699008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b700ad7b0 0x7f6b700d0040 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.570+0000 7f6b78563640 1 -- 172.21.15.26:0/1672699008 >> 172.21.15.26:0/1672699008 conn(0x7f6b7006b540 msgr2=0x7f6b7006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.570+0000 7f6b78563640 1 -- 172.21.15.26:0/1672699008 shutdown_connections 2024-04-24T23:23:04.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.570+0000 7f6b78563640 1 -- 172.21.15.26:0/1672699008 wait complete. 2024-04-24T23:23:04.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.571+0000 7f6b78563640 1 Processor -- start 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.571+0000 7f6b78563640 1 -- start start 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b78563640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6b700ad7b0 0x7f6b7014f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b78563640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7015adc0 0x7f6b7009c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b78563640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b7015e9c0 0x7f6b70148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b78563640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7006cb90 con 0x7f6b7015adc0 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b78563640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7015b1a0 con 0x7f6b7015e9c0 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b78563640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7006cd10 con 0x7f6b700ad7b0 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b75afe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7015adc0 0x7f6b7009c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b75afe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7015adc0 0x7f6b7009c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45552/0 (socket says 172.21.15.26:45552) 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b75afe640 1 -- 172.21.15.26:0/1263282731 learned_addr learned my addr 172.21.15.26:0/1263282731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b762ff640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6b700ad7b0 0x7f6b7014f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b76b00640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b7015e9c0 0x7f6b70148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b75afe640 1 -- 172.21.15.26:0/1263282731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6b700ad7b0 msgr2=0x7f6b7014f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b75afe640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6b700ad7b0 0x7f6b7014f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b75afe640 1 -- 172.21.15.26:0/1263282731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b7015e9c0 msgr2=0x7f6b70148de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b75afe640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b7015e9c0 0x7f6b70148de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.572+0000 7f6b75afe640 1 -- 172.21.15.26:0/1263282731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7009d110 con 0x7f6b7015adc0 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.573+0000 7f6b76b00640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b7015e9c0 0x7f6b70148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.573+0000 7f6b75afe640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7015adc0 0x7f6b7009c800 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000c00 tx=0x7f6b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.573+0000 7f6b6effd640 1 -- 172.21.15.26:0/1263282731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64037040 con 0x7f6b7015adc0 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.573+0000 7f6b6effd640 1 -- 172.21.15.26:0/1263282731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b64042070 con 0x7f6b7015adc0 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.573+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7009d3f0 con 0x7f6b7015adc0 2024-04-24T23:23:04.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.573+0000 7f6b6effd640 1 -- 172.21.15.26:0/1263282731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64030980 con 0x7f6b7015adc0 2024-04-24T23:23:04.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.573+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7009cd40 con 0x7f6b7015adc0 2024-04-24T23:23:04.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.574+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b700aee20 con 0x7f6b7015adc0 2024-04-24T23:23:04.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.574+0000 7f6b6effd640 1 -- 172.21.15.26:0/1263282731 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b64030b20 con 0x7f6b7015adc0 2024-04-24T23:23:04.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.575+0000 7f6b6effd640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b5c06be30 0x7f6b5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:04.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.575+0000 7f6b6effd640 1 -- 172.21.15.26:0/1263282731 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f6b640ab830 con 0x7f6b7015adc0 2024-04-24T23:23:04.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.575+0000 7f6b762ff640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b5c06be30 0x7f6b5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:04.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.575+0000 7f6b762ff640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b5c06be30 0x7f6b5c06e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b600012e0 tx=0x7f6b60007ed0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:04.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.579+0000 7f6b6effd640 1 -- 172.21.15.26:0/1263282731 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b6407a150 con 0x7f6b7015adc0 2024-04-24T23:23:04.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:04.804+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6b70075610 con 0x7f6b7015adc0 2024-04-24T23:23:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.806+0000 7f6b6effd640 1 -- 172.21.15.26:0/1263282731 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7f6b64079a50 con 0x7f6b7015adc0 2024-04-24T23:23:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b5c06be30 msgr2=0x7f6b5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b5c06be30 0x7f6b5c06e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b600012e0 tx=0x7f6b60007ed0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7015adc0 msgr2=0x7f6b7009c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7015adc0 0x7f6b7009c800 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000c00 tx=0x7f6b640046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 shutdown_connections 2024-04-24T23:23:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6b5c06be30 0x7f6b5c06e2f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b7015e9c0 0x7f6b70148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7015adc0 0x7f6b7009c800 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 --2- 172.21.15.26:0/1263282731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6b700ad7b0 0x7f6b7014f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 >> 172.21.15.26:0/1263282731 conn(0x7f6b7006b540 msgr2=0x7f6b70158690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 shutdown_connections 2024-04-24T23:23:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.809+0000 7f6b78563640 1 -- 172.21.15.26:0/1263282731 wait complete. 2024-04-24T23:23:05.824 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:23:05.824 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:23:05.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.933+0000 7f3f3384d640 1 Processor -- start 2024-04-24T23:23:05.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.934+0000 7f3f3384d640 1 -- start start 2024-04-24T23:23:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.934+0000 7f3f3384d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3f2c0cdf30 0x7f3f2c0ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.934+0000 7f3f3384d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f2c0d1550 0x7f3f2c0cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f3384d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f2c06c840 con 0x7f3f2c0b3150 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f3384d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3f2c0d00d0 con 0x7f3f2c0cdf30 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f3384d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c06c5f0 con 0x7f3f2c0d1550 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f30de8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3f2c0b3150 0x7f3f2c0b1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49920/0 (socket says 172.21.15.26:49920) 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f30de8640 1 -- 172.21.15.26:0/4020481258 learned_addr learned my addr 172.21.15.26:0/4020481258 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375920245 0 0) 0x7f3f2c06c840 con 0x7f3f2c0b3150 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f31dea640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f2c0d1550 0x7f3f2c0cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f18003520 con 0x7f3f2c0b3150 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f315e9640 1 --2- 172.21.15.26:0/4020481258 >> v2:172.21.15.29:3300/0 conn(0x7f3f2c0cdf30 0x7f3f2c0ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2700257521 0 0) 0x7f3f18003520 con 0x7f3f2c0b3150 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f2c0d1550 msgr2=0x7f3f2c0cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f2c0d1550 0x7f3f2c0cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 >> v2:172.21.15.29:3300/0 conn(0x7f3f2c0cdf30 msgr2=0x7f3f2c0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 --2- 172.21.15.26:0/4020481258 >> v2:172.21.15.29:3300/0 conn(0x7f3f2c0cdf30 0x7f3f2c0ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c06cb20 con 0x7f3f2c0b3150 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f3f1c001670 con 0x7f3f2c0b3150 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 >> v1:172.21.15.26:6789/0 conn(0x7f3f2c0b3150 legacy=0x7f3f2c0b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f2c0d1550 0x7f3f180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f18003f30 0x7f3f18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f18004d40 0x7f3f180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c06c5f0 con 0x7f3f18003f30 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1800b9c0 con 0x7f3f18004d40 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.935+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1800bb40 con 0x7f3f2c0d1550 2024-04-24T23:23:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.936+0000 7f3f30de8640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f18003f30 0x7f3f18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.936+0000 7f3f315e9640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f2c0d1550 0x7f3f180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.936+0000 7f3f31dea640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f18004d40 0x7f3f180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.936+0000 7f3f30de8640 1 -- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f2c0d1550 msgr2=0x7f3f180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.936+0000 7f3f30de8640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f2c0d1550 0x7f3f180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.936+0000 7f3f30de8640 1 -- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f18004d40 msgr2=0x7f3f180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.936+0000 7f3f30de8640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f18004d40 0x7f3f180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.936+0000 7f3f30de8640 1 -- 172.21.15.26:0/4020481258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f18003520 con 0x7f3f18003f30 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.937+0000 7f3f30de8640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f18003f30 0x7f3f18004800 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c290 tx=0x7f3f18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f231ae4f6ea5d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.937+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c007d50 con 0x7f3f18003f30 2024-04-24T23:23:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.937+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f1c02e070 con 0x7f3f18003f30 2024-04-24T23:23:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.937+0000 7f3f23fff640 1 -- 172.21.15.26:0/4020481258 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c005d70 con 0x7f3f18003f30 2024-04-24T23:23:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.937+0000 7f3f3384d640 1 -- 172.21.15.26:0/4020481258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f18003f30 msgr2=0x7f3f18004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.937+0000 7f3f3384d640 1 --2- 172.21.15.26:0/4020481258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f18003f30 0x7f3f18004800 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c290 tx=0x7f3f18005830 comp rx=0 tx=0).stop 2024-04-24T23:23:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.937+0000 7f3f31dea640 1 -- 172.21.15.26:0/4020481258 reap_dead start 2024-04-24T23:23:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.937+0000 7f3f3384d640 1 -- 172.21.15.26:0/4020481258 shutdown_connections 2024-04-24T23:23:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.937+0000 7f3f3384d640 1 -- 172.21.15.26:0/4020481258 >> 172.21.15.26:0/4020481258 conn(0x7f3f2c0d1e00 msgr2=0x7f3f2c0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.938+0000 7f3f3384d640 1 -- 172.21.15.26:0/4020481258 shutdown_connections 2024-04-24T23:23:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.938+0000 7f3f3384d640 1 -- 172.21.15.26:0/4020481258 wait complete. 2024-04-24T23:23:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.939+0000 7f3f3384d640 1 Processor -- start 2024-04-24T23:23:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.939+0000 7f3f3384d640 1 -- start start 2024-04-24T23:23:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.939+0000 7f3f3384d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f18003f30 0x7f3f2c0af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.939+0000 7f3f3384d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c8ed0 0x7f3f2c0ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.939+0000 7f3f3384d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f2c0cab40 0x7f3f2c0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.939+0000 7f3f3384d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c06d3c0 con 0x7f3f2c0cab40 2024-04-24T23:23:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.939+0000 7f3f3384d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c06d0e0 con 0x7f3f2c0c8ed0 2024-04-24T23:23:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.939+0000 7f3f3384d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c06be30 con 0x7f3f18003f30 2024-04-24T23:23:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f31dea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f2c0cab40 0x7f3f2c0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f31dea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f2c0cab40 0x7f3f2c0cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45570/0 (socket says 172.21.15.26:45570) 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f31dea640 1 -- 172.21.15.26:0/2663947787 learned_addr learned my addr 172.21.15.26:0/2663947787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f315e9640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f18003f30 0x7f3f2c0af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f30de8640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c8ed0 0x7f3f2c0ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f31dea640 1 -- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f18003f30 msgr2=0x7f3f2c0af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f31dea640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f18003f30 0x7f3f2c0af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f31dea640 1 -- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c8ed0 msgr2=0x7f3f2c0ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f31dea640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c8ed0 0x7f3f2c0ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f31dea640 1 -- 172.21.15.26:0/2663947787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c0c9560 con 0x7f3f2c0cab40 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f315e9640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f18003f30 0x7f3f2c0af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f30de8640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c8ed0 0x7f3f2c0ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f31dea640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f2c0cab40 0x7f3f2c0cc450 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c09fde0 tx=0x7f3f2400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f21ffb640 1 -- 172.21.15.26:0/2663947787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400fdb0 con 0x7f3f2c0cab40 2024-04-24T23:23:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f21ffb640 1 -- 172.21.15.26:0/2663947787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2401d070 con 0x7f3f2c0cab40 2024-04-24T23:23:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c0c9840 con 0x7f3f2c0cab40 2024-04-24T23:23:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f21ffb640 1 -- 172.21.15.26:0/2663947787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24015670 con 0x7f3f2c0cab40 2024-04-24T23:23:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.940+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c0c9e00 con 0x7f3f2c0cab40 2024-04-24T23:23:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.941+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2c075610 con 0x7f3f2c0cab40 2024-04-24T23:23:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.942+0000 7f3f21ffb640 1 -- 172.21.15.26:0/2663947787 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f24015810 con 0x7f3f2c0cab40 2024-04-24T23:23:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.942+0000 7f3f21ffb640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f1006be30 0x7f3f1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.942+0000 7f3f21ffb640 1 -- 172.21.15.26:0/2663947787 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f3f2408b7a0 con 0x7f3f2c0cab40 2024-04-24T23:23:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.942+0000 7f3f315e9640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f1006be30 0x7f3f1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.943+0000 7f3f315e9640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f1006be30 0x7f3f1006e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009bc0 tx=0x7f3f28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:05.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:05.947+0000 7f3f21ffb640 1 -- 172.21.15.26:0/2663947787 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f24010040 con 0x7f3f2c0cab40 2024-04-24T23:23:06.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.168+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3f2c0c92b0 con 0x7f3f2c0cab40 2024-04-24T23:23:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.807+0000 7f3f21ffb640 1 -- 172.21.15.26:0/2663947787 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f3f2405a0b0 con 0x7f3f2c0cab40 2024-04-24T23:23:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f1006be30 msgr2=0x7f3f1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f1006be30 0x7f3f1006e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009bc0 tx=0x7f3f28008040 comp rx=0 tx=0).stop 2024-04-24T23:23:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f2c0cab40 msgr2=0x7f3f2c0cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f2c0cab40 0x7f3f2c0cc450 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c09fde0 tx=0x7f3f2400f580 comp rx=0 tx=0).stop 2024-04-24T23:23:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 shutdown_connections 2024-04-24T23:23:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f1006be30 0x7f3f1006e2f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f2c0cab40 0x7f3f2c0cc450 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f2c0c8ed0 0x7f3f2c0ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 --2- 172.21.15.26:0/2663947787 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f18003f30 0x7f3f2c0af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 >> 172.21.15.26:0/2663947787 conn(0x7f3f2c0d1e00 msgr2=0x7f3f2c0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 shutdown_connections 2024-04-24T23:23:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.809+0000 7f3f3384d640 1 -- 172.21.15.26:0/2663947787 wait complete. 2024-04-24T23:23:06.824 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:23:06.824 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:06.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.930+0000 7fdb0e676640 1 Processor -- start 2024-04-24T23:23:06.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.930+0000 7fdb0e676640 1 -- start start 2024-04-24T23:23:06.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb0e676640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb08145910 0x7fdb0806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb0e676640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdb080cd660 0x7fdb080cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb0e676640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb080cfbc0 con 0x7fdb0806d350 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb0e676640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fdb08145cf0 con 0x7fdb08145910 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb0e676640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0806d7f0 con 0x7fdb080cd660 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb07fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb08145910 0x7fdb0806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb077fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdb0806d350 0x7fdb0815d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49936/0 (socket says 172.21.15.26:49936) 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb07fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb08145910 0x7fdb0806e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45572/0 (socket says 172.21.15.26:45572) 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb077fe640 1 -- 172.21.15.26:0/704437282 learned_addr learned my addr 172.21.15.26:0/704437282 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb0cc13640 1 --2- 172.21.15.26:0/704437282 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdb080cd660 0x7fdb080cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb07fff640 1 -- 172.21.15.26:0/704437282 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdb080cd660 msgr2=0x7fdb080cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:06.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb07fff640 1 --2- 172.21.15.26:0/704437282 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdb080cd660 0x7fdb080cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb07fff640 1 -- 172.21.15.26:0/704437282 >> v1:172.21.15.26:6789/0 conn(0x7fdb0806d350 legacy=0x7fdb0815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb07fff640 1 -- 172.21.15.26:0/704437282 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0806dad0 con 0x7fdb08145910 2024-04-24T23:23:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.931+0000 7fdb06ffd640 1 -- 172.21.15.26:0/704437282 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 837607154 0 0) 0x7fdb080cfbc0 con 0x7fdb0806d350 2024-04-24T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.932+0000 7fdb07fff640 1 --2- 172.21.15.26:0/704437282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08145910 0x7fdb0806e890 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097d0 tx=0x7fdaf802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9c5c35b265cdd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.932+0000 7fdb06ffd640 1 -- 172.21.15.26:0/704437282 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8037070 con 0x7fdb08145910 2024-04-24T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.932+0000 7fdb06ffd640 1 -- 172.21.15.26:0/704437282 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf8032440 con 0x7fdb08145910 2024-04-24T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.932+0000 7fdb06ffd640 1 -- 172.21.15.26:0/704437282 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8032810 con 0x7fdb08145910 2024-04-24T23:23:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.932+0000 7fdb0e676640 1 -- 172.21.15.26:0/704437282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08145910 msgr2=0x7fdb0806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.932+0000 7fdb0e676640 1 --2- 172.21.15.26:0/704437282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08145910 0x7fdb0806e890 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097d0 tx=0x7fdaf802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.933+0000 7fdb0e676640 1 -- 172.21.15.26:0/704437282 shutdown_connections 2024-04-24T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.933+0000 7fdb0e676640 1 --2- 172.21.15.26:0/704437282 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdb080cd660 0x7fdb080cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.933+0000 7fdb0e676640 1 --2- 172.21.15.26:0/704437282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb08145910 0x7fdb0806e890 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.933+0000 7fdb0e676640 1 -- 172.21.15.26:0/704437282 >> 172.21.15.26:0/704437282 conn(0x7fdb0806b300 msgr2=0x7fdb0806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.933+0000 7fdb0e676640 1 -- 172.21.15.26:0/704437282 shutdown_connections 2024-04-24T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.933+0000 7fdb0e676640 1 -- 172.21.15.26:0/704437282 wait complete. 2024-04-24T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.934+0000 7fdb0e676640 1 Processor -- start 2024-04-24T23:23:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.934+0000 7fdb0e676640 1 -- start start 2024-04-24T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb0e676640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806d350 0x7fdb080b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb0e676640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb080cd660 0x7fdb080a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb0e676640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdb08145910 0x7fdb08134bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb0e676640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0806e370 con 0x7fdb0806d350 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb0e676640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0806e090 con 0x7fdb080cd660 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb0e676640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0815d770 con 0x7fdb08145910 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806d350 0x7fdb080b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb07fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806d350 0x7fdb080b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45588/0 (socket says 172.21.15.26:45588) 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb07fff640 1 -- 172.21.15.26:0/434942494 learned_addr learned my addr 172.21.15.26:0/434942494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb07fff640 1 -- 172.21.15.26:0/434942494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdb08145910 msgr2=0x7fdb08134bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb07fff640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdb08145910 0x7fdb08134bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb07fff640 1 -- 172.21.15.26:0/434942494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb080cd660 msgr2=0x7fdb080a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb077fe640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb080cd660 0x7fdb080a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb07fff640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb080cd660 0x7fdb080a98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb07fff640 1 -- 172.21.15.26:0/434942494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb08130160 con 0x7fdb0806d350 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb07fff640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806d350 0x7fdb080b3ed0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8035040 tx=0x7fdaf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb04ff9640 1 -- 172.21.15.26:0/434942494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8037040 con 0x7fdb0806d350 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb04ff9640 1 -- 172.21.15.26:0/434942494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf8030770 con 0x7fdb0806d350 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb04ff9640 1 -- 172.21.15.26:0/434942494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8030b80 con 0x7fdb0806d350 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.935+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb08130440 con 0x7fdb0806d350 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.936+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb08130a00 con 0x7fdb0806d350 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.936+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb080694c0 con 0x7fdb0806d350 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.937+0000 7fdb04ff9640 1 -- 172.21.15.26:0/434942494 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdaf8044020 con 0x7fdb0806d350 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.937+0000 7fdb04ff9640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdaf406be60 0x7fdaf406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.937+0000 7fdb04ff9640 1 -- 172.21.15.26:0/434942494 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fdaf80ab3a0 con 0x7fdb0806d350 2024-04-24T23:23:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.937+0000 7fdb077fe640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdaf406be60 0x7fdaf406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.938+0000 7fdb077fe640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdaf406be60 0x7fdaf406e320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdafc007fd0 tx=0x7fdafc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:06.942+0000 7fdb04ff9640 1 -- 172.21.15.26:0/434942494 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdaf8032e20 con 0x7fdb0806d350 2024-04-24T23:23:07.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.164+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb0806b6e0 con 0x7fdb0806d350 2024-04-24T23:23:07.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.164+0000 7fdb04ff9640 1 -- 172.21.15.26:0/434942494 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5175 (secure 0 0 0) 0x7fdaf8079d50 con 0x7fdb0806d350 2024-04-24T23:23:07.166 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:07.166 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5329,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:06.169758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-24T23:23:07.166 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2024-04-24T23:23:07.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdaf406be60 msgr2=0x7fdaf406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdaf406be60 0x7fdaf406e320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdafc007fd0 tx=0x7fdafc009040 comp rx=0 tx=0).stop 2024-04-24T23:23:07.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806d350 msgr2=0x7fdb080b3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806d350 0x7fdb080b3ed0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8035040 tx=0x7fdaf80046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 shutdown_connections 2024-04-24T23:23:07.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdaf406be60 0x7fdaf406e320 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdb08145910 0x7fdb08134bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdb080cd660 0x7fdb080a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 --2- 172.21.15.26:0/434942494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb0806d350 0x7fdb080b3ed0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.166+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 >> 172.21.15.26:0/434942494 conn(0x7fdb0806b300 msgr2=0x7fdb0815f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:07.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.167+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 shutdown_connections 2024-04-24T23:23:07.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.167+0000 7fdb0e676640 1 -- 172.21.15.26:0/434942494 wait complete. 2024-04-24T23:23:07.179 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:23:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.278+0000 7fb07a961640 1 Processor -- start 2024-04-24T23:23:07.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.278+0000 7fb07a961640 1 -- start start 2024-04-24T23:23:07.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.279+0000 7fb07a961640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb07415e570 0x7fb0740d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.279+0000 7fb07a961640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb0740d22e0 0x7fb0740d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.279+0000 7fb07a961640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0740d32f0 con 0x7fb0740b3130 2024-04-24T23:23:07.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.279+0000 7fb07a961640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb0740d1ab0 con 0x7fb0740d22e0 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.279+0000 7fb07a961640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb0740d2ae0 con 0x7fb07415e570 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.279+0000 7fb073fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb07415e570 0x7fb0740d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.279+0000 7fb078efe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb0740d22e0 0x7fb0740d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.280+0000 7fb078efe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb0740d22e0 0x7fb0740d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:53994/0 (socket says 172.21.15.26:53994) 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.280+0000 7fb078efe640 1 -- 172.21.15.26:0/590115994 learned_addr learned my addr 172.21.15.26:0/590115994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.280+0000 7fb073fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb07415e570 0x7fb0740d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:48804/0 (socket says 172.21.15.26:48804) 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.280+0000 7fb0737fe640 1 --1- 172.21.15.26:0/590115994 >> v1:172.21.15.29:6789/0 conn(0x7fb0740b3130 0x7fb0740d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:37034/0 (socket says 172.21.15.26:37034) 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.280+0000 7fb078efe640 1 -- 172.21.15.26:0/590115994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb07415e570 msgr2=0x7fb0740d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.280+0000 7fb078efe640 1 --2- 172.21.15.26:0/590115994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb07415e570 0x7fb0740d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.280+0000 7fb078efe640 1 -- 172.21.15.26:0/590115994 >> v1:172.21.15.29:6789/0 conn(0x7fb0740b3130 legacy=0x7fb0740d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.280+0000 7fb078efe640 1 -- 172.21.15.26:0/590115994 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0740d0270 con 0x7fb0740d22e0 2024-04-24T23:23:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.280+0000 7fb072ffd640 1 -- 172.21.15.26:0/590115994 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2762433600 0 0) 0x7fb05c001110 con 0x7fb0740b3130 2024-04-24T23:23:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb078efe640 1 --2- 172.21.15.26:0/590115994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0740d22e0 0x7fb0740d46a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009e50 tx=0x7fb06c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a6ea70b8378f1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb072ffd640 1 -- 172.21.15.26:0/590115994 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c036070 con 0x7fb0740d22e0 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb072ffd640 1 -- 172.21.15.26:0/590115994 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06c031600 con 0x7fb0740d22e0 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb072ffd640 1 -- 172.21.15.26:0/590115994 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0319d0 con 0x7fb0740d22e0 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb07a961640 1 -- 172.21.15.26:0/590115994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0740d22e0 msgr2=0x7fb0740d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb07a961640 1 --2- 172.21.15.26:0/590115994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0740d22e0 0x7fb0740d46a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009e50 tx=0x7fb06c02a710 comp rx=0 tx=0).stop 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb07a961640 1 -- 172.21.15.26:0/590115994 shutdown_connections 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb07a961640 1 --2- 172.21.15.26:0/590115994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0740d22e0 0x7fb0740d46a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb07a961640 1 --2- 172.21.15.26:0/590115994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb07415e570 0x7fb0740d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb07a961640 1 -- 172.21.15.26:0/590115994 >> 172.21.15.26:0/590115994 conn(0x7fb07408dbe0 msgr2=0x7fb07409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb07a961640 1 -- 172.21.15.26:0/590115994 shutdown_connections 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.281+0000 7fb07a961640 1 -- 172.21.15.26:0/590115994 wait complete. 2024-04-24T23:23:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.283+0000 7fb07a961640 1 Processor -- start 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.283+0000 7fb07a961640 1 -- start start 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.283+0000 7fb07a961640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0740b3130 0x7fb074153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.283+0000 7fb07a961640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0740d22e0 0x7fb0740b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.283+0000 7fb07a961640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb07415e570 0x7fb0740b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.283+0000 7fb07a961640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0740cfa70 con 0x7fb0740d22e0 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.283+0000 7fb07a961640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb0740d3010 con 0x7fb07415e570 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.283+0000 7fb07a961640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb0740cfd50 con 0x7fb0740b3130 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb0737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0740d22e0 0x7fb0740b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb0737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0740d22e0 0x7fb0740b8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45594/0 (socket says 172.21.15.26:45594) 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb0737fe640 1 -- 172.21.15.26:0/2722153826 learned_addr learned my addr 172.21.15.26:0/2722153826 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb073fff640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0740b3130 0x7fb074153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb078efe640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb07415e570 0x7fb0740b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb0737fe640 1 -- 172.21.15.26:0/2722153826 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0740b3130 msgr2=0x7fb074153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb0737fe640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0740b3130 0x7fb074153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb0737fe640 1 -- 172.21.15.26:0/2722153826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb07415e570 msgr2=0x7fb0740b89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb0737fe640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb07415e570 0x7fb0740b89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb0737fe640 1 -- 172.21.15.26:0/2722153826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb074150090 con 0x7fb0740d22e0 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb0737fe640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0740d22e0 0x7fb0740b8460 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00c780 tx=0x7fb05c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb070ff9640 1 -- 172.21.15.26:0/2722153826 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c018070 con 0x7fb0740d22e0 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb070ff9640 1 -- 172.21.15.26:0/2722153826 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb05c0072d0 con 0x7fb0740d22e0 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb074150370 con 0x7fb0740d22e0 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb070ff9640 1 -- 172.21.15.26:0/2722153826 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0047e0 con 0x7fb0740d22e0 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.284+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb074150930 con 0x7fb0740d22e0 2024-04-24T23:23:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.285+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0740694c0 con 0x7fb0740d22e0 2024-04-24T23:23:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.286+0000 7fb070ff9640 1 -- 172.21.15.26:0/2722153826 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb05c004980 con 0x7fb0740d22e0 2024-04-24T23:23:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.286+0000 7fb070ff9640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb05406be30 0x7fb05406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.286+0000 7fb070ff9640 1 -- 172.21.15.26:0/2722153826 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fb05c08a270 con 0x7fb0740d22e0 2024-04-24T23:23:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.286+0000 7fb073fff640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb05406be30 0x7fb05406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.287+0000 7fb073fff640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb05406be30 0x7fb05406e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb064009aa0 tx=0x7fb064009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.291+0000 7fb070ff9640 1 -- 172.21.15.26:0/2722153826 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb0740d32f0 con 0x7fb0740d22e0 2024-04-24T23:23:07.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.475+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb07415dd70 con 0x7fb0740d22e0 2024-04-24T23:23:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.476+0000 7fb070ff9640 1 -- 172.21.15.26:0/2722153826 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16553 (secure 0 0 0) 0x7fb05c004c70 con 0x7fb0740d22e0 2024-04-24T23:23:07.478 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:07.478 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":44,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:23:04.795016+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:23:01.453865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:23:02.136971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:23:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb05406be30 msgr2=0x7fb05406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb05406be30 0x7fb05406e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb064009aa0 tx=0x7fb064009190 comp rx=0 tx=0).stop 2024-04-24T23:23:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0740d22e0 msgr2=0x7fb0740b8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0740d22e0 0x7fb0740b8460 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00c780 tx=0x7fb05c00cc40 comp rx=0 tx=0).stop 2024-04-24T23:23:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 shutdown_connections 2024-04-24T23:23:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb05406be30 0x7fb05406e2f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb07415e570 0x7fb0740b89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0740d22e0 0x7fb0740b8460 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 --2- 172.21.15.26:0/2722153826 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0740b3130 0x7fb074153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 >> 172.21.15.26:0/2722153826 conn(0x7fb07408dbe0 msgr2=0x7fb0740d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 shutdown_connections 2024-04-24T23:23:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.479+0000 7fb07a961640 1 -- 172.21.15.26:0/2722153826 wait complete. 2024-04-24T23:23:07.491 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:23:07.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:23:07.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.591+0000 7f100e238640 1 Processor -- start 2024-04-24T23:23:07.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.591+0000 7f100e238640 1 -- start start 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.592+0000 7f100e238640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f100815e570 0x7f10080d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.592+0000 7f100e238640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080b3130 0x7f10080d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.592+0000 7f100e238640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10080d32f0 con 0x7f10080d22e0 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.592+0000 7f100e238640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f10080d1ab0 con 0x7f100815e570 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.592+0000 7f100e238640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f10080d2ae0 con 0x7f10080b3130 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.592+0000 7f10077fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f100815e570 0x7f10080d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.592+0000 7f1007fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f10080d22e0 0x7f10080d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49946/0 (socket says 172.21.15.26:49946) 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.592+0000 7f1007fff640 1 -- 172.21.15.26:0/2816629828 learned_addr learned my addr 172.21.15.26:0/2816629828 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.593+0000 7f1006ffd640 1 --2- 172.21.15.26:0/2816629828 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080b3130 0x7f10080d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.593+0000 7f10077fe640 1 -- 172.21.15.26:0/2816629828 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080b3130 msgr2=0x7f10080d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.593+0000 7f10077fe640 1 --2- 172.21.15.26:0/2816629828 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080b3130 0x7f10080d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.593+0000 7f10077fe640 1 -- 172.21.15.26:0/2816629828 >> v1:172.21.15.26:6789/0 conn(0x7f10080d22e0 legacy=0x7f10080d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.593+0000 7f10067fc640 1 -- 172.21.15.26:0/2816629828 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 230765576 0 0) 0x7f1000001490 con 0x7f10080d22e0 2024-04-24T23:23:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.593+0000 7f10077fe640 1 -- 172.21.15.26:0/2816629828 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10080d0270 con 0x7f100815e570 2024-04-24T23:23:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.593+0000 7f1006ffd640 1 --2- 172.21.15.26:0/2816629828 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080b3130 0x7f10080d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.593+0000 7f10077fe640 1 --2- 172.21.15.26:0/2816629828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 0x7f10080d4160 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80097a0 tx=0x7f0ff802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad068e2e5ed685 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f10067fc640 1 -- 172.21.15.26:0/2816629828 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8037070 con 0x7f100815e570 2024-04-24T23:23:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f10067fc640 1 -- 172.21.15.26:0/2816629828 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff8032440 con 0x7f100815e570 2024-04-24T23:23:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f10067fc640 1 -- 172.21.15.26:0/2816629828 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8032810 con 0x7f100815e570 2024-04-24T23:23:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f100e238640 1 -- 172.21.15.26:0/2816629828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 msgr2=0x7f10080d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f100e238640 1 --2- 172.21.15.26:0/2816629828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 0x7f10080d4160 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80097a0 tx=0x7f0ff802a180 comp rx=0 tx=0).stop 2024-04-24T23:23:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f100e238640 1 -- 172.21.15.26:0/2816629828 shutdown_connections 2024-04-24T23:23:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f100e238640 1 --2- 172.21.15.26:0/2816629828 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080b3130 0x7f10080d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f100e238640 1 --2- 172.21.15.26:0/2816629828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 0x7f10080d4160 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f100e238640 1 -- 172.21.15.26:0/2816629828 >> 172.21.15.26:0/2816629828 conn(0x7f100808dbe0 msgr2=0x7f100809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f100e238640 1 -- 172.21.15.26:0/2816629828 shutdown_connections 2024-04-24T23:23:07.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.594+0000 7f100e238640 1 -- 172.21.15.26:0/2816629828 wait complete. 2024-04-24T23:23:07.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.595+0000 7f100e238640 1 Processor -- start 2024-04-24T23:23:07.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f100e238640 1 -- start start 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f100e238640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10080b3130 0x7f10080cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f100e238640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080d22e0 0x7f100814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f100e238640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 0x7f100815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f100e238640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10080cfa70 con 0x7f100815e570 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f100e238640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f10080d3010 con 0x7f10080b3130 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f100e238640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f10080cfd50 con 0x7f10080d22e0 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f1007fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 0x7f100815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f1007fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 0x7f100815cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45616/0 (socket says 172.21.15.26:45616) 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.596+0000 7f1007fff640 1 -- 172.21.15.26:0/1174464595 learned_addr learned my addr 172.21.15.26:0/1174464595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f10077fe640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10080b3130 0x7f10080cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f1006ffd640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080d22e0 0x7f100814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f1007fff640 1 -- 172.21.15.26:0/1174464595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080d22e0 msgr2=0x7f100814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f1007fff640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080d22e0 0x7f100814f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f1007fff640 1 -- 172.21.15.26:0/1174464595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10080b3130 msgr2=0x7f10080cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f1007fff640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10080b3130 0x7f10080cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f1007fff640 1 -- 172.21.15.26:0/1174464595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10080ccb20 con 0x7f100815e570 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f1007fff640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 0x7f100815cba0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f100000c7f0 tx=0x7f100000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f0feffff640 1 -- 172.21.15.26:0/1174464595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000018070 con 0x7f100815e570 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f0feffff640 1 -- 172.21.15.26:0/1174464595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100000a5e0 con 0x7f100815e570 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f0feffff640 1 -- 172.21.15.26:0/1174464595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100000aa10 con 0x7f100815e570 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10080cce00 con 0x7f100815e570 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.597+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10080cd3c0 con 0x7f100815e570 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.598+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10080694c0 con 0x7f100815e570 2024-04-24T23:23:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.599+0000 7f0feffff640 1 -- 172.21.15.26:0/1174464595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f100001b070 con 0x7f100815e570 2024-04-24T23:23:07.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.599+0000 7f0feffff640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0ff406be30 0x7f0ff406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.599+0000 7f0feffff640 1 -- 172.21.15.26:0/1174464595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f100008a450 con 0x7f100815e570 2024-04-24T23:23:07.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.599+0000 7f10077fe640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0ff406be30 0x7f0ff406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.600+0000 7f10077fe640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0ff406be30 0x7f0ff406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8034000 tx=0x7f0ff802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:07.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.603+0000 7f0feffff640 1 -- 172.21.15.26:0/1174464595 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f10080d32f0 con 0x7f100815e570 2024-04-24T23:23:07.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.783+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f100806be00 con 0x7f0ff406be30 2024-04-24T23:23:07.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.789+0000 7f0feffff640 1 -- 172.21.15.26:0/1174464595 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239819 (secure 0 0 0) 0x7f100806be00 con 0x7f0ff406be30 2024-04-24T23:23:07.793 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0ff406be30 msgr2=0x7f0ff406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0ff406be30 0x7f0ff406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8034000 tx=0x7f0ff802e000 comp rx=0 tx=0).stop 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 msgr2=0x7f100815cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 0x7f100815cba0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f100000c7f0 tx=0x7f100000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 shutdown_connections 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0ff406be30 0x7f0ff406e2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100815e570 0x7f100815cba0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f10080d22e0 0x7f100814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 --2- 172.21.15.26:0/1174464595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f10080b3130 0x7f10080cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 >> 172.21.15.26:0/1174464595 conn(0x7f100808dbe0 msgr2=0x7f100806c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 shutdown_connections 2024-04-24T23:23:07.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.792+0000 7f100e238640 1 -- 172.21.15.26:0/1174464595 wait complete. 2024-04-24T23:23:07.805 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-24T23:23:07.343602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":258456,"kb_used_data":4820,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716264,"statfs":{"total":773094113280,"available":772829454336,"internally_reserved":0,"allocated":4935680,"data_stored":2671681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.561080"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788097+0000","last_change":"2024-04-24T23:23:01.788097+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788097+0000","last_undegraded":"2024-04-24T23:23:01.788097+0000","last_fullsized":"2024-04-24T23:23:01.788097+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790320+0000","last_change":"2024-04-24T23:23:01.790320+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790320+0000","last_undegraded":"2024-04-24T23:23:01.790320+0000","last_fullsized":"2024-04-24T23:23:01.790320+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789734+0000","last_change":"2024-04-24T23:23:01.789734+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789734+0000","last_undegraded":"2024-04-24T23:23:01.789734+0000","last_fullsized":"2024-04-24T23:23:01.789734+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784968+0000","last_change":"2024-04-24T23:23:01.784968+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784968+0000","last_undegraded":"2024-04-24T23:23:01.784968+0000","last_fullsized":"2024-04-24T23:23:01.784968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789127+0000","last_change":"2024-04-24T23:23:01.789127+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789127+0000","last_undegraded":"2024-04-24T23:23:01.789127+0000","last_fullsized":"2024-04-24T23:23:01.789127+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790968+0000","last_change":"2024-04-24T23:23:01.790968+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790968+0000","last_undegraded":"2024-04-24T23:23:01.790968+0000","last_fullsized":"2024-04-24T23:23:01.790968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787563+0000","last_change":"2024-04-24T23:23:01.787563+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787563+0000","last_undegraded":"2024-04-24T23:23:01.787563+0000","last_fullsized":"2024-04-24T23:23:01.787563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786997+0000","last_change":"2024-04-24T23:23:01.786997+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786997+0000","last_undegraded":"2024-04-24T23:23:01.786997+0000","last_fullsized":"2024-04-24T23:23:01.786997+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790641+0000","last_change":"2024-04-24T23:23:01.790641+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790641+0000","last_undegraded":"2024-04-24T23:23:01.790641+0000","last_fullsized":"2024-04-24T23:23:01.790641+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786721+0000","last_change":"2024-04-24T23:23:01.786721+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786721+0000","last_undegraded":"2024-04-24T23:23:01.786721+0000","last_fullsized":"2024-04-24T23:23:01.786721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785531+0000","last_change":"2024-04-24T23:23:01.785531+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785531+0000","last_undegraded":"2024-04-24T23:23:01.785531+0000","last_fullsized":"2024-04-24T23:23:01.785531+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787535+0000","last_change":"2024-04-24T23:23:01.787535+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787535+0000","last_undegraded":"2024-04-24T23:23:01.787535+0000","last_fullsized":"2024-04-24T23:23:01.787535+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788675+0000","last_change":"2024-04-24T23:23:01.788675+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788675+0000","last_undegraded":"2024-04-24T23:23:01.788675+0000","last_fullsized":"2024-04-24T23:23:01.788675+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785023+0000","last_change":"2024-04-24T23:23:01.785023+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785023+0000","last_undegraded":"2024-04-24T23:23:01.785023+0000","last_fullsized":"2024-04-24T23:23:01.785023+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788116+0000","last_change":"2024-04-24T23:23:01.788116+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788116+0000","last_undegraded":"2024-04-24T23:23:01.788116+0000","last_fullsized":"2024-04-24T23:23:01.788116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786472+0000","last_change":"2024-04-24T23:23:01.786472+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786472+0000","last_undegraded":"2024-04-24T23:23:01.786472+0000","last_fullsized":"2024-04-24T23:23:01.786472+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787457+0000","last_change":"2024-04-24T23:23:01.787457+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787457+0000","last_undegraded":"2024-04-24T23:23:01.787457+0000","last_fullsized":"2024-04-24T23:23:01.787457+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.791089+0000","last_change":"2024-04-24T23:23:01.791089+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.791089+0000","last_undegraded":"2024-04-24T23:23:01.791089+0000","last_fullsized":"2024-04-24T23:23:01.791089+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787576+0000","last_change":"2024-04-24T23:23:01.787576+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787576+0000","last_undegraded":"2024-04-24T23:23:01.787576+0000","last_fullsized":"2024-04-24T23:23:01.787576+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788189+0000","last_change":"2024-04-24T23:23:01.788189+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788189+0000","last_undegraded":"2024-04-24T23:23:01.788189+0000","last_fullsized":"2024-04-24T23:23:01.788189+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788082+0000","last_change":"2024-04-24T23:23:01.788082+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788082+0000","last_undegraded":"2024-04-24T23:23:01.788082+0000","last_fullsized":"2024-04-24T23:23:01.788082+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788139+0000","last_change":"2024-04-24T23:23:01.788139+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788139+0000","last_undegraded":"2024-04-24T23:23:01.788139+0000","last_fullsized":"2024-04-24T23:23:01.788139+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785956+0000","last_change":"2024-04-24T23:23:01.785956+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785956+0000","last_undegraded":"2024-04-24T23:23:01.785956+0000","last_fullsized":"2024-04-24T23:23:01.785956+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784973+0000","last_change":"2024-04-24T23:23:01.784973+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784973+0000","last_undegraded":"2024-04-24T23:23:01.784973+0000","last_fullsized":"2024-04-24T23:23:01.784973+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788174+0000","last_change":"2024-04-24T23:23:01.788174+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788174+0000","last_undegraded":"2024-04-24T23:23:01.788174+0000","last_fullsized":"2024-04-24T23:23:01.788174+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784975+0000","last_change":"2024-04-24T23:23:01.784975+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784975+0000","last_undegraded":"2024-04-24T23:23:01.784975+0000","last_fullsized":"2024-04-24T23:23:01.784975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789784+0000","last_change":"2024-04-24T23:23:01.789784+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789784+0000","last_undegraded":"2024-04-24T23:23:01.789784+0000","last_fullsized":"2024-04-24T23:23:01.789784+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784948+0000","last_change":"2024-04-24T23:23:01.784948+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784948+0000","last_undegraded":"2024-04-24T23:23:01.784948+0000","last_fullsized":"2024-04-24T23:23:01.784948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790118+0000","last_change":"2024-04-24T23:23:01.790118+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790118+0000","last_undegraded":"2024-04-24T23:23:01.790118+0000","last_fullsized":"2024-04-24T23:23:01.790118+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789656+0000","last_change":"2024-04-24T23:23:01.789656+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789656+0000","last_undegraded":"2024-04-24T23:23:01.789656+0000","last_fullsized":"2024-04-24T23:23:01.789656+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785959+0000","last_change":"2024-04-24T23:23:01.785959+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785959+0000","last_undegraded":"2024-04-24T23:23:01.785959+0000","last_fullsized":"2024-04-24T23:23:01.785959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787557+0000","last_change":"2024-04-24T23:23:01.787557+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787557+0000","last_undegraded":"2024-04-24T23:23:01.787557+0000","last_fullsized":"2024-04-24T23:23:01.787557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T23:23:01.786332+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:23:01.786332+0000","last_peered":"2024-04-24T23:23:01.786332+0000","last_clean":"2024-04-24T23:23:01.786332+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:23:01.786332+0000","last_undegraded":"2024-04-24T23:23:01.786332+0000","last_fullsized":"2024-04-24T23:23:01.786332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":776009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35800,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336040,"statfs":{"total":96636764160,"available":96600104960,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":781826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:23:07.809 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:23:07.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.945+0000 7f85147d5640 1 Processor -- start 2024-04-24T23:23:07.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.945+0000 7f85147d5640 1 -- start start 2024-04-24T23:23:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f85147d5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f850c143c40 0x7f850c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f85147d5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c15eb00 0x7f850c15ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f85147d5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850c0cd0b0 con 0x7f850c0cc860 2024-04-24T23:23:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f85147d5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f850c071b70 con 0x7f850c143c40 2024-04-24T23:23:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f85147d5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f850c15f470 con 0x7f850c15eb00 2024-04-24T23:23:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f8512571640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f850c143c40 0x7f850c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f8512d72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c15eb00 0x7f850c15ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f8512d72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c15eb00 0x7f850c15ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:48852/0 (socket says 172.21.15.26:48852) 2024-04-24T23:23:07.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f8512d72640 1 -- 172.21.15.26:0/54564107 learned_addr learned my addr 172.21.15.26:0/54564107 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f8512571640 1 -- 172.21.15.26:0/54564107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c15eb00 msgr2=0x7f850c15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f8512571640 1 --2- 172.21.15.26:0/54564107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c15eb00 0x7f850c15ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f8512571640 1 -- 172.21.15.26:0/54564107 >> v1:172.21.15.29:6789/0 conn(0x7f850c0cc860 legacy=0x7f850c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f851156f640 1 -- 172.21.15.26:0/54564107 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670580292 0 0) 0x7f850c0cd0b0 con 0x7f850c0cc860 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f8512571640 1 -- 172.21.15.26:0/54564107 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c071240 con 0x7f850c143c40 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.946+0000 7f8512d72640 1 --2- 172.21.15.26:0/54564107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c15eb00 0x7f850c15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.947+0000 7f8512571640 1 --2- 172.21.15.26:0/54564107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c143c40 0x7f850c0cf870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85080097a0 tx=0x7f850802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b421e9996c6b8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.947+0000 7f851156f640 1 -- 172.21.15.26:0/54564107 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508037070 con 0x7f850c143c40 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.947+0000 7f851156f640 1 -- 172.21.15.26:0/54564107 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8508032440 con 0x7f850c143c40 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.947+0000 7f851156f640 1 -- 172.21.15.26:0/54564107 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508032810 con 0x7f850c143c40 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.948+0000 7f85147d5640 1 -- 172.21.15.26:0/54564107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c143c40 msgr2=0x7f850c0cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.948+0000 7f85147d5640 1 --2- 172.21.15.26:0/54564107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c143c40 0x7f850c0cf870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85080097a0 tx=0x7f850802a180 comp rx=0 tx=0).stop 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.948+0000 7f85147d5640 1 -- 172.21.15.26:0/54564107 shutdown_connections 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.948+0000 7f85147d5640 1 --2- 172.21.15.26:0/54564107 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c15eb00 0x7f850c15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.948+0000 7f85147d5640 1 --2- 172.21.15.26:0/54564107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c143c40 0x7f850c0cf870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.948+0000 7f85147d5640 1 -- 172.21.15.26:0/54564107 >> 172.21.15.26:0/54564107 conn(0x7f850c06b300 msgr2=0x7f850c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:07.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.948+0000 7f85147d5640 1 -- 172.21.15.26:0/54564107 shutdown_connections 2024-04-24T23:23:07.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.948+0000 7f85147d5640 1 -- 172.21.15.26:0/54564107 wait complete. 2024-04-24T23:23:07.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.949+0000 7f85147d5640 1 Processor -- start 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.949+0000 7f85147d5640 1 -- start start 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f85147d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f850c0cc860 0x7f850c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f85147d5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c143c40 0x7f850c0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f85147d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c15eb00 0x7f850c1e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f85147d5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f850c144020 con 0x7f850c0cc860 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f85147d5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f850c071800 con 0x7f850c15eb00 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f85147d5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f850c0cccd0 con 0x7f850c143c40 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512571640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f850c0cc860 0x7f850c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512571640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f850c0cc860 0x7f850c15a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45618/0 (socket says 172.21.15.26:45618) 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512571640 1 -- 172.21.15.26:0/4068222981 learned_addr learned my addr 172.21.15.26:0/4068222981 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8511d70640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c143c40 0x7f850c0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512d72640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c15eb00 0x7f850c1e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512571640 1 -- 172.21.15.26:0/4068222981 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c143c40 msgr2=0x7f850c0afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512571640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c143c40 0x7f850c0afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512571640 1 -- 172.21.15.26:0/4068222981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c15eb00 msgr2=0x7f850c1e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512571640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c15eb00 0x7f850c1e4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512571640 1 -- 172.21.15.26:0/4068222981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c0b0570 con 0x7f850c0cc860 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.950+0000 7f8512d72640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c15eb00 0x7f850c1e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.951+0000 7f8512571640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f850c0cc860 0x7f850c15a8b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8508035040 tx=0x7f85080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.951+0000 7f8502ffd640 1 -- 172.21.15.26:0/4068222981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508037040 con 0x7f850c0cc860 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.951+0000 7f8502ffd640 1 -- 172.21.15.26:0/4068222981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8508030500 con 0x7f850c0cc860 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.951+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c0b0850 con 0x7f850c0cc860 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.951+0000 7f8502ffd640 1 -- 172.21.15.26:0/4068222981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508030910 con 0x7f850c0cc860 2024-04-24T23:23:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.951+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c0b0e10 con 0x7f850c0cc860 2024-04-24T23:23:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.952+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c0694c0 con 0x7f850c0cc860 2024-04-24T23:23:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.952+0000 7f8502ffd640 1 -- 172.21.15.26:0/4068222981 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8508044020 con 0x7f850c0cc860 2024-04-24T23:23:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.953+0000 7f8502ffd640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f84f006be30 0x7f84f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.953+0000 7f8502ffd640 1 -- 172.21.15.26:0/4068222981 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f85080ab0c0 con 0x7f850c0cc860 2024-04-24T23:23:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.953+0000 7f8511d70640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f84f006be30 0x7f84f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.953+0000 7f8511d70640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f84f006be30 0x7f84f006e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f850c0b16b0 tx=0x7f84fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:07.957+0000 7f8502ffd640 1 -- 172.21.15.26:0/4068222981 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8508079b00 con 0x7f850c0cc860 2024-04-24T23:23:08.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.137+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f850c0b4d20 con 0x7f84f006be30 2024-04-24T23:23:08.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.143+0000 7f8502ffd640 1 -- 172.21.15.26:0/4068222981 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239819 (secure 0 0 0) 0x7f850c0b4d20 con 0x7f84f006be30 2024-04-24T23:23:08.147 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:23:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f84f006be30 msgr2=0x7f84f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f84f006be30 0x7f84f006e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f850c0b16b0 tx=0x7f84fc009040 comp rx=0 tx=0).stop 2024-04-24T23:23:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f850c0cc860 msgr2=0x7f850c15a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f850c0cc860 0x7f850c15a8b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8508035040 tx=0x7f85080046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 shutdown_connections 2024-04-24T23:23:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f84f006be30 0x7f84f006e2f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f850c15eb00 0x7f850c1e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f850c143c40 0x7f850c0afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 --2- 172.21.15.26:0/4068222981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f850c0cc860 0x7f850c15a8b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 >> 172.21.15.26:0/4068222981 conn(0x7f850c06b300 msgr2=0x7f850c0cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 shutdown_connections 2024-04-24T23:23:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.146+0000 7f85147d5640 1 -- 172.21.15.26:0/4068222981 wait complete. 2024-04-24T23:23:08.158 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-24T23:23:07.343602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":258456,"kb_used_data":4820,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716264,"statfs":{"total":773094113280,"available":772829454336,"internally_reserved":0,"allocated":4935680,"data_stored":2671681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.561080"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788097+0000","last_change":"2024-04-24T23:23:01.788097+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788097+0000","last_undegraded":"2024-04-24T23:23:01.788097+0000","last_fullsized":"2024-04-24T23:23:01.788097+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790320+0000","last_change":"2024-04-24T23:23:01.790320+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790320+0000","last_undegraded":"2024-04-24T23:23:01.790320+0000","last_fullsized":"2024-04-24T23:23:01.790320+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789734+0000","last_change":"2024-04-24T23:23:01.789734+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789734+0000","last_undegraded":"2024-04-24T23:23:01.789734+0000","last_fullsized":"2024-04-24T23:23:01.789734+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784968+0000","last_change":"2024-04-24T23:23:01.784968+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784968+0000","last_undegraded":"2024-04-24T23:23:01.784968+0000","last_fullsized":"2024-04-24T23:23:01.784968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789127+0000","last_change":"2024-04-24T23:23:01.789127+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789127+0000","last_undegraded":"2024-04-24T23:23:01.789127+0000","last_fullsized":"2024-04-24T23:23:01.789127+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790968+0000","last_change":"2024-04-24T23:23:01.790968+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790968+0000","last_undegraded":"2024-04-24T23:23:01.790968+0000","last_fullsized":"2024-04-24T23:23:01.790968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787563+0000","last_change":"2024-04-24T23:23:01.787563+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787563+0000","last_undegraded":"2024-04-24T23:23:01.787563+0000","last_fullsized":"2024-04-24T23:23:01.787563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786997+0000","last_change":"2024-04-24T23:23:01.786997+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786997+0000","last_undegraded":"2024-04-24T23:23:01.786997+0000","last_fullsized":"2024-04-24T23:23:01.786997+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790641+0000","last_change":"2024-04-24T23:23:01.790641+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790641+0000","last_undegraded":"2024-04-24T23:23:01.790641+0000","last_fullsized":"2024-04-24T23:23:01.790641+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786721+0000","last_change":"2024-04-24T23:23:01.786721+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786721+0000","last_undegraded":"2024-04-24T23:23:01.786721+0000","last_fullsized":"2024-04-24T23:23:01.786721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785531+0000","last_change":"2024-04-24T23:23:01.785531+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785531+0000","last_undegraded":"2024-04-24T23:23:01.785531+0000","last_fullsized":"2024-04-24T23:23:01.785531+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787535+0000","last_change":"2024-04-24T23:23:01.787535+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787535+0000","last_undegraded":"2024-04-24T23:23:01.787535+0000","last_fullsized":"2024-04-24T23:23:01.787535+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788675+0000","last_change":"2024-04-24T23:23:01.788675+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788675+0000","last_undegraded":"2024-04-24T23:23:01.788675+0000","last_fullsized":"2024-04-24T23:23:01.788675+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785023+0000","last_change":"2024-04-24T23:23:01.785023+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785023+0000","last_undegraded":"2024-04-24T23:23:01.785023+0000","last_fullsized":"2024-04-24T23:23:01.785023+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788116+0000","last_change":"2024-04-24T23:23:01.788116+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788116+0000","last_undegraded":"2024-04-24T23:23:01.788116+0000","last_fullsized":"2024-04-24T23:23:01.788116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786472+0000","last_change":"2024-04-24T23:23:01.786472+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786472+0000","last_undegraded":"2024-04-24T23:23:01.786472+0000","last_fullsized":"2024-04-24T23:23:01.786472+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787457+0000","last_change":"2024-04-24T23:23:01.787457+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787457+0000","last_undegraded":"2024-04-24T23:23:01.787457+0000","last_fullsized":"2024-04-24T23:23:01.787457+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.791089+0000","last_change":"2024-04-24T23:23:01.791089+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.791089+0000","last_undegraded":"2024-04-24T23:23:01.791089+0000","last_fullsized":"2024-04-24T23:23:01.791089+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787576+0000","last_change":"2024-04-24T23:23:01.787576+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787576+0000","last_undegraded":"2024-04-24T23:23:01.787576+0000","last_fullsized":"2024-04-24T23:23:01.787576+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788189+0000","last_change":"2024-04-24T23:23:01.788189+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788189+0000","last_undegraded":"2024-04-24T23:23:01.788189+0000","last_fullsized":"2024-04-24T23:23:01.788189+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788082+0000","last_change":"2024-04-24T23:23:01.788082+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788082+0000","last_undegraded":"2024-04-24T23:23:01.788082+0000","last_fullsized":"2024-04-24T23:23:01.788082+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788139+0000","last_change":"2024-04-24T23:23:01.788139+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788139+0000","last_undegraded":"2024-04-24T23:23:01.788139+0000","last_fullsized":"2024-04-24T23:23:01.788139+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785956+0000","last_change":"2024-04-24T23:23:01.785956+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785956+0000","last_undegraded":"2024-04-24T23:23:01.785956+0000","last_fullsized":"2024-04-24T23:23:01.785956+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784973+0000","last_change":"2024-04-24T23:23:01.784973+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784973+0000","last_undegraded":"2024-04-24T23:23:01.784973+0000","last_fullsized":"2024-04-24T23:23:01.784973+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788174+0000","last_change":"2024-04-24T23:23:01.788174+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788174+0000","last_undegraded":"2024-04-24T23:23:01.788174+0000","last_fullsized":"2024-04-24T23:23:01.788174+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784975+0000","last_change":"2024-04-24T23:23:01.784975+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784975+0000","last_undegraded":"2024-04-24T23:23:01.784975+0000","last_fullsized":"2024-04-24T23:23:01.784975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789784+0000","last_change":"2024-04-24T23:23:01.789784+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789784+0000","last_undegraded":"2024-04-24T23:23:01.789784+0000","last_fullsized":"2024-04-24T23:23:01.789784+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784948+0000","last_change":"2024-04-24T23:23:01.784948+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784948+0000","last_undegraded":"2024-04-24T23:23:01.784948+0000","last_fullsized":"2024-04-24T23:23:01.784948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790118+0000","last_change":"2024-04-24T23:23:01.790118+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790118+0000","last_undegraded":"2024-04-24T23:23:01.790118+0000","last_fullsized":"2024-04-24T23:23:01.790118+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789656+0000","last_change":"2024-04-24T23:23:01.789656+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789656+0000","last_undegraded":"2024-04-24T23:23:01.789656+0000","last_fullsized":"2024-04-24T23:23:01.789656+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785959+0000","last_change":"2024-04-24T23:23:01.785959+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785959+0000","last_undegraded":"2024-04-24T23:23:01.785959+0000","last_fullsized":"2024-04-24T23:23:01.785959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787557+0000","last_change":"2024-04-24T23:23:01.787557+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787557+0000","last_undegraded":"2024-04-24T23:23:01.787557+0000","last_fullsized":"2024-04-24T23:23:01.787557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T23:23:01.786332+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:23:01.786332+0000","last_peered":"2024-04-24T23:23:01.786332+0000","last_clean":"2024-04-24T23:23:01.786332+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:23:01.786332+0000","last_undegraded":"2024-04-24T23:23:01.786332+0000","last_fullsized":"2024-04-24T23:23:01.786332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":776009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35800,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336040,"statfs":{"total":96636764160,"available":96600104960,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":781826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:23:08.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:23:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.298+0000 7f1fe7041640 1 Processor -- start 2024-04-24T23:23:08.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.298+0000 7f1fe7041640 1 -- start start 2024-04-24T23:23:08.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.299+0000 7f1fe7041640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1fe015e290 0x7f1fe00d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.299+0000 7f1fe7041640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe006d050 con 0x7f1fe00b3ac0 2024-04-24T23:23:08.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.299+0000 7f1fe7041640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe00b5840 con 0x7f1fe00b72f0 2024-04-24T23:23:08.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.299+0000 7f1fe7041640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe00b5e50 con 0x7f1fe015e290 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.299+0000 7f1fdffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1fe00b3ac0 0x7f1fe006c230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49956/0 (socket says 172.21.15.26:49956) 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.299+0000 7f1fdffff640 1 -- 172.21.15.26:0/1431095236 learned_addr learned my addr 172.21.15.26:0/1431095236 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.299+0000 7f1fe4ddd640 1 --2- 172.21.15.26:0/1431095236 >> v2:172.21.15.29:3300/0 conn(0x7f1fe015e290 0x7f1fe00d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.300+0000 7f1fdf7fe640 1 -- 172.21.15.26:0/1431095236 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313057023 0 0) 0x7f1fe006d050 con 0x7f1fe00b3ac0 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.300+0000 7f1fdf7fe640 1 -- 172.21.15.26:0/1431095236 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fcc003520 con 0x7f1fe00b3ac0 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.300+0000 7f1fdf7fe640 1 -- 172.21.15.26:0/1431095236 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1649854430 0 0) 0x7f1fe00b5840 con 0x7f1fe00b72f0 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.300+0000 7f1fdf7fe640 1 -- 172.21.15.26:0/1431095236 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fe006d050 con 0x7f1fe00b72f0 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.300+0000 7f1fe4ddd640 1 -- 172.21.15.26:0/1431095236 >> v1:172.21.15.29:6789/0 conn(0x7f1fe00b72f0 legacy=0x7f1fe006c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.300+0000 7f1fe4ddd640 1 -- 172.21.15.26:0/1431095236 >> v1:172.21.15.26:6789/0 conn(0x7f1fe00b3ac0 legacy=0x7f1fe006c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.300+0000 7f1fdf7fe640 1 -- 172.21.15.26:0/1431095236 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 598638381 0 0) 0x7f1fcc003520 con 0x7f1fe00b3ac0 2024-04-24T23:23:08.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.300+0000 7f1fe4ddd640 1 -- 172.21.15.26:0/1431095236 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe00cfb30 con 0x7f1fe015e290 2024-04-24T23:23:08.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fe4ddd640 1 --2- 172.21.15.26:0/1431095236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe015e290 0x7f1fe00d1730 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009d90 tx=0x7f1fd802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d4e0c8bc93962 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:08.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fdf7fe640 1 -- 172.21.15.26:0/1431095236 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8001670 con 0x7f1fe015e290 2024-04-24T23:23:08.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fdf7fe640 1 -- 172.21.15.26:0/1431095236 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd8007540 con 0x7f1fe015e290 2024-04-24T23:23:08.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fdf7fe640 1 -- 172.21.15.26:0/1431095236 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8007910 con 0x7f1fe015e290 2024-04-24T23:23:08.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fe7041640 1 -- 172.21.15.26:0/1431095236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe015e290 msgr2=0x7f1fe00d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fe7041640 1 --2- 172.21.15.26:0/1431095236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe015e290 0x7f1fe00d1730 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009d90 tx=0x7f1fd802a680 comp rx=0 tx=0).stop 2024-04-24T23:23:08.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fe7041640 1 -- 172.21.15.26:0/1431095236 shutdown_connections 2024-04-24T23:23:08.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fe7041640 1 --2- 172.21.15.26:0/1431095236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe015e290 0x7f1fe00d1730 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fe7041640 1 -- 172.21.15.26:0/1431095236 >> 172.21.15.26:0/1431095236 conn(0x7f1fe006b120 msgr2=0x7f1fe006b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:08.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.301+0000 7f1fe7041640 1 -- 172.21.15.26:0/1431095236 shutdown_connections 2024-04-24T23:23:08.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.302+0000 7f1fe7041640 1 -- 172.21.15.26:0/1431095236 wait complete. 2024-04-24T23:23:08.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.303+0000 7f1fe7041640 1 Processor -- start 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.303+0000 7f1fe7041640 1 -- start start 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.303+0000 7f1fe7041640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe00b3ac0 0x7f1fe0160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe7041640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe00b72f0 0x7f1fe01f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe7041640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe015e290 0x7f1fe01495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe7041640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe00b5ae0 con 0x7f1fe015e290 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe7041640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe00d00f0 con 0x7f1fe00b3ac0 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe7041640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe015e670 con 0x7f1fe00b72f0 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe55de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe015e290 0x7f1fe01495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe55de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe015e290 0x7f1fe01495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45620/0 (socket says 172.21.15.26:45620) 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe55de640 1 -- 172.21.15.26:0/716194160 learned_addr learned my addr 172.21.15.26:0/716194160 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fdffff640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe00b72f0 0x7f1fe01f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe4ddd640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe00b3ac0 0x7f1fe0160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe55de640 1 -- 172.21.15.26:0/716194160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe00b72f0 msgr2=0x7f1fe01f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe55de640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe00b72f0 0x7f1fe01f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe55de640 1 -- 172.21.15.26:0/716194160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe00b3ac0 msgr2=0x7f1fe0160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe55de640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe00b3ac0 0x7f1fe0160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe55de640 1 -- 172.21.15.26:0/716194160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe015fd80 con 0x7f1fe015e290 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe55de640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe015e290 0x7f1fe01495d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c400 tx=0x7f1fd000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fdd7fa640 1 -- 172.21.15.26:0/716194160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0017070 con 0x7f1fe015e290 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fdd7fa640 1 -- 172.21.15.26:0/716194160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd00052d0 con 0x7f1fe015e290 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fdd7fa640 1 -- 172.21.15.26:0/716194160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00037f0 con 0x7f1fe015e290 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe0160060 con 0x7f1fe015e290 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.304+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe0160620 con 0x7f1fe015e290 2024-04-24T23:23:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.305+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe00694c0 con 0x7f1fe015e290 2024-04-24T23:23:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.306+0000 7f1fdd7fa640 1 -- 172.21.15.26:0/716194160 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fd001a070 con 0x7f1fe015e290 2024-04-24T23:23:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.306+0000 7f1fdd7fa640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc006be30 0x7f1fc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.306+0000 7f1fdd7fa640 1 -- 172.21.15.26:0/716194160 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f1fd008a140 con 0x7f1fe015e290 2024-04-24T23:23:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.306+0000 7f1fe4ddd640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc006be30 0x7f1fc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.307+0000 7f1fe4ddd640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc006be30 0x7f1fc006e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8034000 tx=0x7f1fd802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.311+0000 7f1fdd7fa640 1 -- 172.21.15.26:0/716194160 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1fd0058490 con 0x7f1fe015e290 2024-04-24T23:23:08.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.491+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1fe00b1d60 con 0x7f1fc006be30 2024-04-24T23:23:08.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.496+0000 7f1fdd7fa640 1 -- 172.21.15.26:0/716194160 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239819 (secure 0 0 0) 0x7f1fe00b1d60 con 0x7f1fc006be30 2024-04-24T23:23:08.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:23:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc006be30 msgr2=0x7f1fc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc006be30 0x7f1fc006e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8034000 tx=0x7f1fd802f000 comp rx=0 tx=0).stop 2024-04-24T23:23:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe015e290 msgr2=0x7f1fe01495d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe015e290 0x7f1fe01495d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c400 tx=0x7f1fd000c8c0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 shutdown_connections 2024-04-24T23:23:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc006be30 0x7f1fc006e2f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe015e290 0x7f1fe01495d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe00b72f0 0x7f1fe01f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 --2- 172.21.15.26:0/716194160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe00b3ac0 0x7f1fe0160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 >> 172.21.15.26:0/716194160 conn(0x7f1fe006b120 msgr2=0x7f1fe006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 shutdown_connections 2024-04-24T23:23:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.499+0000 7f1fe7041640 1 -- 172.21.15.26:0/716194160 wait complete. 2024-04-24T23:23:08.512 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-24T23:23:07.343602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":258456,"kb_used_data":4820,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716264,"statfs":{"total":773094113280,"available":772829454336,"internally_reserved":0,"allocated":4935680,"data_stored":2671681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.561080"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788097+0000","last_change":"2024-04-24T23:23:01.788097+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788097+0000","last_undegraded":"2024-04-24T23:23:01.788097+0000","last_fullsized":"2024-04-24T23:23:01.788097+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790320+0000","last_change":"2024-04-24T23:23:01.790320+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790320+0000","last_undegraded":"2024-04-24T23:23:01.790320+0000","last_fullsized":"2024-04-24T23:23:01.790320+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789734+0000","last_change":"2024-04-24T23:23:01.789734+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789734+0000","last_undegraded":"2024-04-24T23:23:01.789734+0000","last_fullsized":"2024-04-24T23:23:01.789734+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784968+0000","last_change":"2024-04-24T23:23:01.784968+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784968+0000","last_undegraded":"2024-04-24T23:23:01.784968+0000","last_fullsized":"2024-04-24T23:23:01.784968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789127+0000","last_change":"2024-04-24T23:23:01.789127+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789127+0000","last_undegraded":"2024-04-24T23:23:01.789127+0000","last_fullsized":"2024-04-24T23:23:01.789127+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790968+0000","last_change":"2024-04-24T23:23:01.790968+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790968+0000","last_undegraded":"2024-04-24T23:23:01.790968+0000","last_fullsized":"2024-04-24T23:23:01.790968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787563+0000","last_change":"2024-04-24T23:23:01.787563+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787563+0000","last_undegraded":"2024-04-24T23:23:01.787563+0000","last_fullsized":"2024-04-24T23:23:01.787563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786997+0000","last_change":"2024-04-24T23:23:01.786997+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786997+0000","last_undegraded":"2024-04-24T23:23:01.786997+0000","last_fullsized":"2024-04-24T23:23:01.786997+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790641+0000","last_change":"2024-04-24T23:23:01.790641+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790641+0000","last_undegraded":"2024-04-24T23:23:01.790641+0000","last_fullsized":"2024-04-24T23:23:01.790641+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786721+0000","last_change":"2024-04-24T23:23:01.786721+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786721+0000","last_undegraded":"2024-04-24T23:23:01.786721+0000","last_fullsized":"2024-04-24T23:23:01.786721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785531+0000","last_change":"2024-04-24T23:23:01.785531+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785531+0000","last_undegraded":"2024-04-24T23:23:01.785531+0000","last_fullsized":"2024-04-24T23:23:01.785531+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787535+0000","last_change":"2024-04-24T23:23:01.787535+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787535+0000","last_undegraded":"2024-04-24T23:23:01.787535+0000","last_fullsized":"2024-04-24T23:23:01.787535+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788675+0000","last_change":"2024-04-24T23:23:01.788675+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788675+0000","last_undegraded":"2024-04-24T23:23:01.788675+0000","last_fullsized":"2024-04-24T23:23:01.788675+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785023+0000","last_change":"2024-04-24T23:23:01.785023+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785023+0000","last_undegraded":"2024-04-24T23:23:01.785023+0000","last_fullsized":"2024-04-24T23:23:01.785023+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788116+0000","last_change":"2024-04-24T23:23:01.788116+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788116+0000","last_undegraded":"2024-04-24T23:23:01.788116+0000","last_fullsized":"2024-04-24T23:23:01.788116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786472+0000","last_change":"2024-04-24T23:23:01.786472+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786472+0000","last_undegraded":"2024-04-24T23:23:01.786472+0000","last_fullsized":"2024-04-24T23:23:01.786472+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787457+0000","last_change":"2024-04-24T23:23:01.787457+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787457+0000","last_undegraded":"2024-04-24T23:23:01.787457+0000","last_fullsized":"2024-04-24T23:23:01.787457+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.791089+0000","last_change":"2024-04-24T23:23:01.791089+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.791089+0000","last_undegraded":"2024-04-24T23:23:01.791089+0000","last_fullsized":"2024-04-24T23:23:01.791089+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787576+0000","last_change":"2024-04-24T23:23:01.787576+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787576+0000","last_undegraded":"2024-04-24T23:23:01.787576+0000","last_fullsized":"2024-04-24T23:23:01.787576+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788189+0000","last_change":"2024-04-24T23:23:01.788189+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788189+0000","last_undegraded":"2024-04-24T23:23:01.788189+0000","last_fullsized":"2024-04-24T23:23:01.788189+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788082+0000","last_change":"2024-04-24T23:23:01.788082+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788082+0000","last_undegraded":"2024-04-24T23:23:01.788082+0000","last_fullsized":"2024-04-24T23:23:01.788082+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788139+0000","last_change":"2024-04-24T23:23:01.788139+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788139+0000","last_undegraded":"2024-04-24T23:23:01.788139+0000","last_fullsized":"2024-04-24T23:23:01.788139+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785956+0000","last_change":"2024-04-24T23:23:01.785956+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785956+0000","last_undegraded":"2024-04-24T23:23:01.785956+0000","last_fullsized":"2024-04-24T23:23:01.785956+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784973+0000","last_change":"2024-04-24T23:23:01.784973+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784973+0000","last_undegraded":"2024-04-24T23:23:01.784973+0000","last_fullsized":"2024-04-24T23:23:01.784973+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788174+0000","last_change":"2024-04-24T23:23:01.788174+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788174+0000","last_undegraded":"2024-04-24T23:23:01.788174+0000","last_fullsized":"2024-04-24T23:23:01.788174+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784975+0000","last_change":"2024-04-24T23:23:01.784975+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784975+0000","last_undegraded":"2024-04-24T23:23:01.784975+0000","last_fullsized":"2024-04-24T23:23:01.784975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789784+0000","last_change":"2024-04-24T23:23:01.789784+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789784+0000","last_undegraded":"2024-04-24T23:23:01.789784+0000","last_fullsized":"2024-04-24T23:23:01.789784+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784948+0000","last_change":"2024-04-24T23:23:01.784948+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784948+0000","last_undegraded":"2024-04-24T23:23:01.784948+0000","last_fullsized":"2024-04-24T23:23:01.784948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790118+0000","last_change":"2024-04-24T23:23:01.790118+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790118+0000","last_undegraded":"2024-04-24T23:23:01.790118+0000","last_fullsized":"2024-04-24T23:23:01.790118+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789656+0000","last_change":"2024-04-24T23:23:01.789656+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789656+0000","last_undegraded":"2024-04-24T23:23:01.789656+0000","last_fullsized":"2024-04-24T23:23:01.789656+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785959+0000","last_change":"2024-04-24T23:23:01.785959+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785959+0000","last_undegraded":"2024-04-24T23:23:01.785959+0000","last_fullsized":"2024-04-24T23:23:01.785959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787557+0000","last_change":"2024-04-24T23:23:01.787557+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787557+0000","last_undegraded":"2024-04-24T23:23:01.787557+0000","last_fullsized":"2024-04-24T23:23:01.787557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T23:23:01.786332+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:23:01.786332+0000","last_peered":"2024-04-24T23:23:01.786332+0000","last_clean":"2024-04-24T23:23:01.786332+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:23:01.786332+0000","last_undegraded":"2024-04-24T23:23:01.786332+0000","last_fullsized":"2024-04-24T23:23:01.786332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":776009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35800,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336040,"statfs":{"total":96636764160,"available":96600104960,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":781826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:23:08.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-04-24T23:23:08.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-04-24T23:23:08.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-04-24T23:23:08.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-04-24T23:23:08.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-24T23:23:08.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-24T23:23:08.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-04-24T23:23:08.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-04-24T23:23:08.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-04-24T23:23:08.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-04-24T23:23:08.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-24T23:23:08.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.788097+0000', 'last_change': '2024-04-24T23:23:01.788097+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.788097+0000', 'last_undegraded': '2024-04-24T23:23:01.788097+0000', 'last_fullsized': '2024-04-24T23:23:01.788097+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.790320+0000', 'last_change': '2024-04-24T23:23:01.790320+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.790320+0000', 'last_undegraded': '2024-04-24T23:23:01.790320+0000', 'last_fullsized': '2024-04-24T23:23:01.790320+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.789734+0000', 'last_change': '2024-04-24T23:23:01.789734+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.789734+0000', 'last_undegraded': '2024-04-24T23:23:01.789734+0000', 'last_fullsized': '2024-04-24T23:23:01.789734+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-24T23:23:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.784968+0000', 'last_change': '2024-04-24T23:23:01.784968+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.784968+0000', 'last_undegraded': '2024-04-24T23:23:01.784968+0000', 'last_fullsized': '2024-04-24T23:23:01.784968+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.789127+0000', 'last_change': '2024-04-24T23:23:01.789127+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.789127+0000', 'last_undegraded': '2024-04-24T23:23:01.789127+0000', 'last_fullsized': '2024-04-24T23:23:01.789127+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-24T23:23:08.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.790968+0000', 'last_change': '2024-04-24T23:23:01.790968+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.790968+0000', 'last_undegraded': '2024-04-24T23:23:01.790968+0000', 'last_fullsized': '2024-04-24T23:23:01.790968+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.787563+0000', 'last_change': '2024-04-24T23:23:01.787563+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.787563+0000', 'last_undegraded': '2024-04-24T23:23:01.787563+0000', 'last_fullsized': '2024-04-24T23:23:01.787563+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.786997+0000', 'last_change': '2024-04-24T23:23:01.786997+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.786997+0000', 'last_undegraded': '2024-04-24T23:23:01.786997+0000', 'last_fullsized': '2024-04-24T23:23:01.786997+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-24T23:23:08.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.790641+0000', 'last_change': '2024-04-24T23:23:01.790641+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.790641+0000', 'last_undegraded': '2024-04-24T23:23:01.790641+0000', 'last_fullsized': '2024-04-24T23:23:01.790641+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.786721+0000', 'last_change': '2024-04-24T23:23:01.786721+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.786721+0000', 'last_undegraded': '2024-04-24T23:23:01.786721+0000', 'last_fullsized': '2024-04-24T23:23:01.786721+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.785531+0000', 'last_change': '2024-04-24T23:23:01.785531+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.785531+0000', 'last_undegraded': '2024-04-24T23:23:01.785531+0000', 'last_fullsized': '2024-04-24T23:23:01.785531+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-24T23:23:08.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.787535+0000', 'last_change': '2024-04-24T23:23:01.787535+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.787535+0000', 'last_undegraded': '2024-04-24T23:23:01.787535+0000', 'last_fullsized': '2024-04-24T23:23:01.787535+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.788675+0000', 'last_change': '2024-04-24T23:23:01.788675+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.788675+0000', 'last_undegraded': '2024-04-24T23:23:01.788675+0000', 'last_fullsized': '2024-04-24T23:23:01.788675+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.785023+0000', 'last_change': '2024-04-24T23:23:01.785023+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.785023+0000', 'last_undegraded': '2024-04-24T23:23:01.785023+0000', 'last_fullsized': '2024-04-24T23:23:01.785023+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-24T23:23:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.788116+0000', 'last_change': '2024-04-24T23:23:01.788116+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.788116+0000', 'last_undegraded': '2024-04-24T23:23:01.788116+0000', 'last_fullsized': '2024-04-24T23:23:01.788116+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.786472+0000', 'last_change': '2024-04-24T23:23:01.786472+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.786472+0000', 'last_undegraded': '2024-04-24T23:23:01.786472+0000', 'last_fullsized': '2024-04-24T23:23:01.786472+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.787457+0000', 'last_change': '2024-04-24T23:23:01.787457+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.787457+0000', 'last_undegraded': '2024-04-24T23:23:01.787457+0000', 'last_fullsized': '2024-04-24T23:23:01.787457+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:23:08.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.791089+0000', 'last_change': '2024-04-24T23:23:01.791089+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.791089+0000', 'last_undegraded': '2024-04-24T23:23:01.791089+0000', 'last_fullsized': '2024-04-24T23:23:01.791089+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.787576+0000', 'last_change': '2024-04-24T23:23:01.787576+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.787576+0000', 'last_undegraded': '2024-04-24T23:23:01.787576+0000', 'last_fullsized': '2024-04-24T23:23:01.787576+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-24T23:23:08.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.788189+0000', 'last_change': '2024-04-24T23:23:01.788189+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.788189+0000', 'last_undegraded': '2024-04-24T23:23:01.788189+0000', 'last_fullsized': '2024-04-24T23:23:01.788189+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.788082+0000', 'last_change': '2024-04-24T23:23:01.788082+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.788082+0000', 'last_undegraded': '2024-04-24T23:23:01.788082+0000', 'last_fullsized': '2024-04-24T23:23:01.788082+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.788139+0000', 'last_change': '2024-04-24T23:23:01.788139+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.788139+0000', 'last_undegraded': '2024-04-24T23:23:01.788139+0000', 'last_fullsized': '2024-04-24T23:23:01.788139+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-24T23:23:08.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.785956+0000', 'last_change': '2024-04-24T23:23:01.785956+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.785956+0000', 'last_undegraded': '2024-04-24T23:23:01.785956+0000', 'last_fullsized': '2024-04-24T23:23:01.785956+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.784973+0000', 'last_change': '2024-04-24T23:23:01.784973+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.784973+0000', 'last_undegraded': '2024-04-24T23:23:01.784973+0000', 'last_fullsized': '2024-04-24T23:23:01.784973+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.788174+0000', 'last_change': '2024-04-24T23:23:01.788174+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.788174+0000', 'last_undegraded': '2024-04-24T23:23:01.788174+0000', 'last_fullsized': '2024-04-24T23:23:01.788174+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-24T23:23:08.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.784975+0000', 'last_change': '2024-04-24T23:23:01.784975+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.784975+0000', 'last_undegraded': '2024-04-24T23:23:01.784975+0000', 'last_fullsized': '2024-04-24T23:23:01.784975+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.789784+0000', 'last_change': '2024-04-24T23:23:01.789784+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.789784+0000', 'last_undegraded': '2024-04-24T23:23:01.789784+0000', 'last_fullsized': '2024-04-24T23:23:01.789784+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.784948+0000', 'last_change': '2024-04-24T23:23:01.784948+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.784948+0000', 'last_undegraded': '2024-04-24T23:23:01.784948+0000', 'last_fullsized': '2024-04-24T23:23:01.784948+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-24T23:23:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.790118+0000', 'last_change': '2024-04-24T23:23:01.790118+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.790118+0000', 'last_undegraded': '2024-04-24T23:23:01.790118+0000', 'last_fullsized': '2024-04-24T23:23:01.790118+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.789656+0000', 'last_change': '2024-04-24T23:23:01.789656+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.789656+0000', 'last_undegraded': '2024-04-24T23:23:01.789656+0000', 'last_fullsized': '2024-04-24T23:23:01.789656+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.785959+0000', 'last_change': '2024-04-24T23:23:01.785959+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.785959+0000', 'last_undegraded': '2024-04-24T23:23:01.785959+0000', 'last_fullsized': '2024-04-24T23:23:01.785959+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:23:08.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-24T23:23:08.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-24T23:23:02.782468+0000', 'last_change': '2024-04-24T23:23:02.782468+0000', 'last_active': '2024-04-24T23:23:02.782468+0000', 'last_peered': '2024-04-24T23:23:02.782468+0000', 'last_clean': '2024-04-24T23:23:02.782468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:02.782468+0000', 'last_undegraded': '2024-04-24T23:23:02.782468+0000', 'last_fullsized': '2024-04-24T23:23:02.782468+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:02.782468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-24T23:23:08.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-24T23:23:08.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:23:01.787557+0000', 'last_change': '2024-04-24T23:23:01.787557+0000', 'last_active': '2024-04-24T23:23:01.778077+0000', 'last_peered': '2024-04-24T23:23:01.778077+0000', 'last_clean': '2024-04-24T23:23:01.778077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:23:01.787557+0000', 'last_undegraded': '2024-04-24T23:23:01.787557+0000', 'last_fullsized': '2024-04-24T23:23:01.787557+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'last_clean_scrub_stamp': '2024-04-24T23:23:01.778077+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:23:08.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:23:08.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.651+0000 7f0425615640 1 Processor -- start 2024-04-24T23:23:08.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.651+0000 7f0425615640 1 -- start start 2024-04-24T23:23:08.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.651+0000 7f0425615640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f042013e090 0x7f04200ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.651+0000 7f0425615640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0420140e10 0x7f04200711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.651+0000 7f0425615640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04200cef80 con 0x7f042015bfe0 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.651+0000 7f0425615640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0420155aa0 con 0x7f042013e090 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.651+0000 7f0425615640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f04200cf260 con 0x7f0420140e10 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.652+0000 7f041effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f042013e090 0x7f04200ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.652+0000 7f041e7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f042015bfe0 0x7f0420140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49968/0 (socket says 172.21.15.26:49968) 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.652+0000 7f041e7fc640 1 -- 172.21.15.26:0/2887266322 learned_addr learned my addr 172.21.15.26:0/2887266322 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.652+0000 7f041f7fe640 1 --2- 172.21.15.26:0/2887266322 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0420140e10 0x7f04200711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.652+0000 7f041effd640 1 -- 172.21.15.26:0/2887266322 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0420140e10 msgr2=0x7f04200711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.652+0000 7f041effd640 1 --2- 172.21.15.26:0/2887266322 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0420140e10 0x7f04200711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.652+0000 7f041effd640 1 -- 172.21.15.26:0/2887266322 >> v1:172.21.15.26:6789/0 conn(0x7f042015bfe0 legacy=0x7f0420140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.652+0000 7f041effd640 1 -- 172.21.15.26:0/2887266322 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04200cf540 con 0x7f042013e090 2024-04-24T23:23:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.652+0000 7f041dffb640 1 -- 172.21.15.26:0/2887266322 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1297259252 0 0) 0x7f04200cef80 con 0x7f042015bfe0 2024-04-24T23:23:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f041effd640 1 --2- 172.21.15.26:0/2887266322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042013e090 0x7f04200ce850 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f041000ba40 tx=0x7f041002c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36abd309bae5dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f041dffb640 1 -- 172.21.15.26:0/2887266322 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410031710 con 0x7f042013e090 2024-04-24T23:23:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f041dffb640 1 -- 172.21.15.26:0/2887266322 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04100318b0 con 0x7f042013e090 2024-04-24T23:23:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f041dffb640 1 -- 172.21.15.26:0/2887266322 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410031c80 con 0x7f042013e090 2024-04-24T23:23:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f0425615640 1 -- 172.21.15.26:0/2887266322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042013e090 msgr2=0x7f04200ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f0425615640 1 --2- 172.21.15.26:0/2887266322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042013e090 0x7f04200ce850 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f041000ba40 tx=0x7f041002c7b0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f0425615640 1 -- 172.21.15.26:0/2887266322 shutdown_connections 2024-04-24T23:23:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f0425615640 1 --2- 172.21.15.26:0/2887266322 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0420140e10 0x7f04200711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f0425615640 1 --2- 172.21.15.26:0/2887266322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042013e090 0x7f04200ce850 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f0425615640 1 -- 172.21.15.26:0/2887266322 >> 172.21.15.26:0/2887266322 conn(0x7f042006b540 msgr2=0x7f042006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.653+0000 7f0425615640 1 -- 172.21.15.26:0/2887266322 shutdown_connections 2024-04-24T23:23:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.654+0000 7f0425615640 1 -- 172.21.15.26:0/2887266322 wait complete. 2024-04-24T23:23:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.655+0000 7f0425615640 1 Processor -- start 2024-04-24T23:23:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.655+0000 7f0425615640 1 -- start start 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f0425615640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f042013e090 0x7f04201547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f0425615640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0420140e10 0x7f04200b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f0425615640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042015bfe0 0x7f042015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f0425615640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f042015c460 con 0x7f042015bfe0 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f0425615640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04200cfb00 con 0x7f0420140e10 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f0425615640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f04200716f0 con 0x7f042013e090 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042015bfe0 0x7f042015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042015bfe0 0x7f042015a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45646/0 (socket says 172.21.15.26:45646) 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041f7fe640 1 -- 172.21.15.26:0/4217758007 learned_addr learned my addr 172.21.15.26:0/4217758007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041effd640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f042013e090 0x7f04201547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041e7fc640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0420140e10 0x7f04200b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041f7fe640 1 -- 172.21.15.26:0/4217758007 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f042013e090 msgr2=0x7f04201547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041f7fe640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f042013e090 0x7f04201547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041f7fe640 1 -- 172.21.15.26:0/4217758007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0420140e10 msgr2=0x7f04200b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041f7fe640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0420140e10 0x7f04200b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041f7fe640 1 -- 172.21.15.26:0/4217758007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042013b480 con 0x7f042015bfe0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f041f7fe640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042015bfe0 0x7f042015a030 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f041800efc0 tx=0x7f041800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f04077fe640 1 -- 172.21.15.26:0/4217758007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418018020 con 0x7f042015bfe0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f04077fe640 1 -- 172.21.15.26:0/4217758007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0418014480 con 0x7f042015bfe0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042013b760 con 0x7f042015bfe0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.656+0000 7f04077fe640 1 -- 172.21.15.26:0/4217758007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418014870 con 0x7f042015bfe0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.657+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042013bd20 con 0x7f042015bfe0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.657+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04200694c0 con 0x7f042015bfe0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.658+0000 7f04077fe640 1 -- 172.21.15.26:0/4217758007 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0418005d10 con 0x7f042015bfe0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.658+0000 7f04077fe640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f040c06be30 0x7f040c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.658+0000 7f04077fe640 1 -- 172.21.15.26:0/4217758007 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f041808bb70 con 0x7f042015bfe0 2024-04-24T23:23:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.659+0000 7f041effd640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f040c06be30 0x7f040c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.659+0000 7f041effd640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f040c06be30 0x7f040c06e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0420067930 tx=0x7f0410031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.663+0000 7f04077fe640 1 -- 172.21.15.26:0/4217758007 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0418022030 con 0x7f042015bfe0 2024-04-24T23:23:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.918+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f042006baa0 con 0x7f042015bfe0 2024-04-24T23:23:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.919+0000 7f04077fe640 1 -- 172.21.15.26:0/4217758007 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1528 (secure 0 0 0) 0x7f0418010040 con 0x7f042015bfe0 2024-04-24T23:23:08.921 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:08.921 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":157,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":264658944,"bytes_avail":772829454336,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":33,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5742}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-24T23:22:43.338650+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:23:08.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f040c06be30 msgr2=0x7f040c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f040c06be30 0x7f040c06e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0420067930 tx=0x7f0410031000 comp rx=0 tx=0).stop 2024-04-24T23:23:08.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042015bfe0 msgr2=0x7f042015a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:08.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042015bfe0 0x7f042015a030 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f041800efc0 tx=0x7f041800c490 comp rx=0 tx=0).stop 2024-04-24T23:23:08.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 shutdown_connections 2024-04-24T23:23:08.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f040c06be30 0x7f040c06e2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f042015bfe0 0x7f042015a030 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0420140e10 0x7f04200b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 --2- 172.21.15.26:0/4217758007 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f042013e090 0x7f04201547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:08.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 >> 172.21.15.26:0/4217758007 conn(0x7f042006b540 msgr2=0x7f0420071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:08.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 shutdown_connections 2024-04-24T23:23:08.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:08.921+0000 7f0425615640 1 -- 172.21.15.26:0/4217758007 wait complete. 2024-04-24T23:23:08.934 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-24T23:23:08.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:23:09.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.034+0000 7f923c253640 1 Processor -- start 2024-04-24T23:23:09.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.034+0000 7f923c253640 1 -- start start 2024-04-24T23:23:09.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f923c253640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f92340cdd70 0x7f923406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:09.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f923c253640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92340cfbc0 con 0x7f923406d350 2024-04-24T23:23:09.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f923c253640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f923406d7f0 con 0x7f9234145910 2024-04-24T23:23:09.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f923c253640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9234145cf0 con 0x7f92340cdd70 2024-04-24T23:23:09.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f923a7f0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f92340cdd70 0x7f923406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:09.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f92397ee640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f923406d350 0x7f92340cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49980/0 (socket says 172.21.15.26:49980) 2024-04-24T23:23:09.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f923a7f0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f92340cdd70 0x7f923406e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45656/0 (socket says 172.21.15.26:45656) 2024-04-24T23:23:09.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f92397ee640 1 -- 172.21.15.26:0/1008652033 learned_addr learned my addr 172.21.15.26:0/1008652033 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:09.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f923a7f0640 1 -- 172.21.15.26:0/1008652033 >> v1:172.21.15.29:6789/0 conn(0x7f9234145910 legacy=0x7f923415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:09.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f923a7f0640 1 -- 172.21.15.26:0/1008652033 >> v1:172.21.15.26:6789/0 conn(0x7f923406d350 legacy=0x7f92340cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:09.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.035+0000 7f923a7f0640 1 -- 172.21.15.26:0/1008652033 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923406dad0 con 0x7f92340cdd70 2024-04-24T23:23:09.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.036+0000 7f923a7f0640 1 --2- 172.21.15.26:0/1008652033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 0x7f923406e890 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f922c009dc0 tx=0x7f922c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5188dd929b276cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:09.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.036+0000 7f9238fed640 1 -- 172.21.15.26:0/1008652033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c007740 con 0x7f92340cdd70 2024-04-24T23:23:09.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.036+0000 7f9238fed640 1 -- 172.21.15.26:0/1008652033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922c0078e0 con 0x7f92340cdd70 2024-04-24T23:23:09.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.036+0000 7f9238fed640 1 -- 172.21.15.26:0/1008652033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c007cb0 con 0x7f92340cdd70 2024-04-24T23:23:09.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.037+0000 7f923c253640 1 -- 172.21.15.26:0/1008652033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 msgr2=0x7f923406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:09.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.037+0000 7f923c253640 1 --2- 172.21.15.26:0/1008652033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 0x7f923406e890 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f922c009dc0 tx=0x7f922c02a640 comp rx=0 tx=0).stop 2024-04-24T23:23:09.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.037+0000 7f923c253640 1 -- 172.21.15.26:0/1008652033 shutdown_connections 2024-04-24T23:23:09.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.037+0000 7f923c253640 1 --2- 172.21.15.26:0/1008652033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 0x7f923406e890 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:09.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.037+0000 7f923c253640 1 -- 172.21.15.26:0/1008652033 >> 172.21.15.26:0/1008652033 conn(0x7f923406b300 msgr2=0x7f923406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:09.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.037+0000 7f923c253640 1 -- 172.21.15.26:0/1008652033 shutdown_connections 2024-04-24T23:23:09.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.037+0000 7f923c253640 1 -- 172.21.15.26:0/1008652033 wait complete. 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.038+0000 7f923c253640 1 Processor -- start 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.038+0000 7f923c253640 1 -- start start 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f923c253640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f923406d350 0x7f9234141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f923c253640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 0x7f9234154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f923c253640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9234145910 0x7f92341554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f923c253640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f923406e370 con 0x7f92340cdd70 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f923c253640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f923406e090 con 0x7f9234145910 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f923c253640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f923415d770 con 0x7f923406d350 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f92397ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 0x7f9234154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f92397ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 0x7f9234154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45672/0 (socket says 172.21.15.26:45672) 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f92397ee640 1 -- 172.21.15.26:0/1287660565 learned_addr learned my addr 172.21.15.26:0/1287660565 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f923a7f0640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9234145910 0x7f92341554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:09.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f92397ee640 1 -- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f923406d350 msgr2=0x7f9234141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f9239fef640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f923406d350 0x7f9234141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f92397ee640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f923406d350 0x7f9234141c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f92397ee640 1 -- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9234145910 msgr2=0x7f92341554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f92397ee640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9234145910 0x7f92341554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f92397ee640 1 -- 172.21.15.26:0/1287660565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923412e9d0 con 0x7f92340cdd70 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f923a7f0640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9234145910 0x7f92341554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f9239fef640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f923406d350 0x7f9234141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.039+0000 7f92397ee640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 0x7f9234154f90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f922400c430 tx=0x7f922400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.040+0000 7f922a7fc640 1 -- 172.21.15.26:0/1287660565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224017070 con 0x7f92340cdd70 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.040+0000 7f922a7fc640 1 -- 172.21.15.26:0/1287660565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9224005ce0 con 0x7f92340cdd70 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.040+0000 7f922a7fc640 1 -- 172.21.15.26:0/1287660565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240072f0 con 0x7f92340cdd70 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.040+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923412ecb0 con 0x7f92340cdd70 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.040+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923412f270 con 0x7f92340cdd70 2024-04-24T23:23:09.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.041+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92340694c0 con 0x7f92340cdd70 2024-04-24T23:23:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.041+0000 7f922a7fc640 1 -- 172.21.15.26:0/1287660565 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f922401a070 con 0x7f92340cdd70 2024-04-24T23:23:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.041+0000 7f922a7fc640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f921806be30 0x7f921806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.042+0000 7f922a7fc640 1 -- 172.21.15.26:0/1287660565 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f922408a390 con 0x7f92340cdd70 2024-04-24T23:23:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.042+0000 7f9239fef640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f921806be30 0x7f921806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:09.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.042+0000 7f9239fef640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f921806be30 0x7f921806e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f923000a810 tx=0x7f9230008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:09.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.046+0000 7f922a7fc640 1 -- 172.21.15.26:0/1287660565 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9224058c10 con 0x7f92340cdd70 2024-04-24T23:23:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.226+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f923415f300 con 0x7f921806be30 2024-04-24T23:23:09.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.231+0000 7f922a7fc640 1 -- 172.21.15.26:0/1287660565 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239819 (secure 0 0 0) 0x7f923415f300 con 0x7f921806be30 2024-04-24T23:23:09.235 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:23:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f921806be30 msgr2=0x7f921806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f921806be30 0x7f921806e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f923000a810 tx=0x7f9230008040 comp rx=0 tx=0).stop 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 msgr2=0x7f9234154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 0x7f9234154f90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f922400c430 tx=0x7f922400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 shutdown_connections 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f921806be30 0x7f921806e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9234145910 0x7f92341554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92340cdd70 0x7f9234154f90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 --2- 172.21.15.26:0/1287660565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f923406d350 0x7f9234141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 >> 172.21.15.26:0/1287660565 conn(0x7f923406b300 msgr2=0x7f92340b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 shutdown_connections 2024-04-24T23:23:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:09.234+0000 7f923c253640 1 -- 172.21.15.26:0/1287660565 wait complete. 2024-04-24T23:23:09.247 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-24T23:23:07.343602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":258456,"kb_used_data":4820,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716264,"statfs":{"total":773094113280,"available":772829454336,"internally_reserved":0,"allocated":4935680,"data_stored":2671681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.561080"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788097+0000","last_change":"2024-04-24T23:23:01.788097+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788097+0000","last_undegraded":"2024-04-24T23:23:01.788097+0000","last_fullsized":"2024-04-24T23:23:01.788097+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790320+0000","last_change":"2024-04-24T23:23:01.790320+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790320+0000","last_undegraded":"2024-04-24T23:23:01.790320+0000","last_fullsized":"2024-04-24T23:23:01.790320+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789734+0000","last_change":"2024-04-24T23:23:01.789734+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789734+0000","last_undegraded":"2024-04-24T23:23:01.789734+0000","last_fullsized":"2024-04-24T23:23:01.789734+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784968+0000","last_change":"2024-04-24T23:23:01.784968+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784968+0000","last_undegraded":"2024-04-24T23:23:01.784968+0000","last_fullsized":"2024-04-24T23:23:01.784968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789127+0000","last_change":"2024-04-24T23:23:01.789127+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789127+0000","last_undegraded":"2024-04-24T23:23:01.789127+0000","last_fullsized":"2024-04-24T23:23:01.789127+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790968+0000","last_change":"2024-04-24T23:23:01.790968+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790968+0000","last_undegraded":"2024-04-24T23:23:01.790968+0000","last_fullsized":"2024-04-24T23:23:01.790968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787563+0000","last_change":"2024-04-24T23:23:01.787563+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787563+0000","last_undegraded":"2024-04-24T23:23:01.787563+0000","last_fullsized":"2024-04-24T23:23:01.787563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786997+0000","last_change":"2024-04-24T23:23:01.786997+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786997+0000","last_undegraded":"2024-04-24T23:23:01.786997+0000","last_fullsized":"2024-04-24T23:23:01.786997+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790641+0000","last_change":"2024-04-24T23:23:01.790641+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790641+0000","last_undegraded":"2024-04-24T23:23:01.790641+0000","last_fullsized":"2024-04-24T23:23:01.790641+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786721+0000","last_change":"2024-04-24T23:23:01.786721+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786721+0000","last_undegraded":"2024-04-24T23:23:01.786721+0000","last_fullsized":"2024-04-24T23:23:01.786721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785531+0000","last_change":"2024-04-24T23:23:01.785531+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785531+0000","last_undegraded":"2024-04-24T23:23:01.785531+0000","last_fullsized":"2024-04-24T23:23:01.785531+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787535+0000","last_change":"2024-04-24T23:23:01.787535+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787535+0000","last_undegraded":"2024-04-24T23:23:01.787535+0000","last_fullsized":"2024-04-24T23:23:01.787535+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788675+0000","last_change":"2024-04-24T23:23:01.788675+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788675+0000","last_undegraded":"2024-04-24T23:23:01.788675+0000","last_fullsized":"2024-04-24T23:23:01.788675+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785023+0000","last_change":"2024-04-24T23:23:01.785023+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785023+0000","last_undegraded":"2024-04-24T23:23:01.785023+0000","last_fullsized":"2024-04-24T23:23:01.785023+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788116+0000","last_change":"2024-04-24T23:23:01.788116+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788116+0000","last_undegraded":"2024-04-24T23:23:01.788116+0000","last_fullsized":"2024-04-24T23:23:01.788116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.786472+0000","last_change":"2024-04-24T23:23:01.786472+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.786472+0000","last_undegraded":"2024-04-24T23:23:01.786472+0000","last_fullsized":"2024-04-24T23:23:01.786472+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787457+0000","last_change":"2024-04-24T23:23:01.787457+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787457+0000","last_undegraded":"2024-04-24T23:23:01.787457+0000","last_fullsized":"2024-04-24T23:23:01.787457+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.791089+0000","last_change":"2024-04-24T23:23:01.791089+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.791089+0000","last_undegraded":"2024-04-24T23:23:01.791089+0000","last_fullsized":"2024-04-24T23:23:01.791089+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787576+0000","last_change":"2024-04-24T23:23:01.787576+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787576+0000","last_undegraded":"2024-04-24T23:23:01.787576+0000","last_fullsized":"2024-04-24T23:23:01.787576+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788189+0000","last_change":"2024-04-24T23:23:01.788189+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788189+0000","last_undegraded":"2024-04-24T23:23:01.788189+0000","last_fullsized":"2024-04-24T23:23:01.788189+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788082+0000","last_change":"2024-04-24T23:23:01.788082+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788082+0000","last_undegraded":"2024-04-24T23:23:01.788082+0000","last_fullsized":"2024-04-24T23:23:01.788082+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788139+0000","last_change":"2024-04-24T23:23:01.788139+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788139+0000","last_undegraded":"2024-04-24T23:23:01.788139+0000","last_fullsized":"2024-04-24T23:23:01.788139+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785956+0000","last_change":"2024-04-24T23:23:01.785956+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785956+0000","last_undegraded":"2024-04-24T23:23:01.785956+0000","last_fullsized":"2024-04-24T23:23:01.785956+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784973+0000","last_change":"2024-04-24T23:23:01.784973+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784973+0000","last_undegraded":"2024-04-24T23:23:01.784973+0000","last_fullsized":"2024-04-24T23:23:01.784973+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.788174+0000","last_change":"2024-04-24T23:23:01.788174+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.788174+0000","last_undegraded":"2024-04-24T23:23:01.788174+0000","last_fullsized":"2024-04-24T23:23:01.788174+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784975+0000","last_change":"2024-04-24T23:23:01.784975+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784975+0000","last_undegraded":"2024-04-24T23:23:01.784975+0000","last_fullsized":"2024-04-24T23:23:01.784975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789784+0000","last_change":"2024-04-24T23:23:01.789784+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789784+0000","last_undegraded":"2024-04-24T23:23:01.789784+0000","last_fullsized":"2024-04-24T23:23:01.789784+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.784948+0000","last_change":"2024-04-24T23:23:01.784948+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.784948+0000","last_undegraded":"2024-04-24T23:23:01.784948+0000","last_fullsized":"2024-04-24T23:23:01.784948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.790118+0000","last_change":"2024-04-24T23:23:01.790118+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.790118+0000","last_undegraded":"2024-04-24T23:23:01.790118+0000","last_fullsized":"2024-04-24T23:23:01.790118+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.789656+0000","last_change":"2024-04-24T23:23:01.789656+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.789656+0000","last_undegraded":"2024-04-24T23:23:01.789656+0000","last_fullsized":"2024-04-24T23:23:01.789656+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.785959+0000","last_change":"2024-04-24T23:23:01.785959+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.785959+0000","last_undegraded":"2024-04-24T23:23:01.785959+0000","last_fullsized":"2024-04-24T23:23:01.785959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:23:02.782468+0000","last_change":"2024-04-24T23:23:02.782468+0000","last_active":"2024-04-24T23:23:02.782468+0000","last_peered":"2024-04-24T23:23:02.782468+0000","last_clean":"2024-04-24T23:23:02.782468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:02.782468+0000","last_undegraded":"2024-04-24T23:23:02.782468+0000","last_fullsized":"2024-04-24T23:23:02.782468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-24T23:23:01.787557+0000","last_change":"2024-04-24T23:23:01.787557+0000","last_active":"2024-04-24T23:23:01.778077+0000","last_peered":"2024-04-24T23:23:01.778077+0000","last_clean":"2024-04-24T23:23:01.778077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:23:01.787557+0000","last_undegraded":"2024-04-24T23:23:01.787557+0000","last_fullsized":"2024-04-24T23:23:01.787557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T23:23:01.786332+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:23:01.786332+0000","last_peered":"2024-04-24T23:23:01.786332+0000","last_clean":"2024-04-24T23:23:01.786332+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:23:01.786332+0000","last_undegraded":"2024-04-24T23:23:01.786332+0000","last_fullsized":"2024-04-24T23:23:01.786332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":776009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35800,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336040,"statfs":{"total":96636764160,"available":96600104960,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":781826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:23:09.780 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:23:09.779+0000 7fc1e7675640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:23:12.251 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:23:12.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.364+0000 7f8df9eb3640 1 Processor -- start 2024-04-24T23:23:12.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.364+0000 7f8df9eb3640 1 -- start start 2024-04-24T23:23:12.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.365+0000 7f8df9eb3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8df40cf870 0x7f8df4072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.365+0000 7f8df9eb3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8df415eb00 0x7f8df415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.365+0000 7f8df9eb3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df415f450 con 0x7f8df4143c40 2024-04-24T23:23:12.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.365+0000 7f8df9eb3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8df4071b70 con 0x7f8df40cf870 2024-04-24T23:23:12.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.365+0000 7f8df9eb3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8df40cd0b0 con 0x7f8df415eb00 2024-04-24T23:23:12.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.365+0000 7f8df2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8df40cf870 0x7f8df4072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.365+0000 7f8df2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8df40cf870 0x7f8df4072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45684/0 (socket says 172.21.15.26:45684) 2024-04-24T23:23:12.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.365+0000 7f8df2ffd640 1 -- 172.21.15.26:0/779717498 learned_addr learned my addr 172.21.15.26:0/779717498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:12.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.365+0000 7f8df3fff640 1 --2- 172.21.15.26:0/779717498 >> v2:172.21.15.29:3300/0 conn(0x7f8df415eb00 0x7f8df415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.366+0000 7f8df2ffd640 1 -- 172.21.15.26:0/779717498 >> v2:172.21.15.29:3300/0 conn(0x7f8df415eb00 msgr2=0x7f8df415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:12.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.366+0000 7f8df2ffd640 1 --2- 172.21.15.26:0/779717498 >> v2:172.21.15.29:3300/0 conn(0x7f8df415eb00 0x7f8df415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.366+0000 7f8df2ffd640 1 -- 172.21.15.26:0/779717498 >> v1:172.21.15.29:6789/0 conn(0x7f8df4143c40 legacy=0x7f8df40cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:12.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.366+0000 7f8df27fc640 1 -- 172.21.15.26:0/779717498 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405111801 0 0) 0x7f8de4001110 con 0x7f8df4143c40 2024-04-24T23:23:12.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.366+0000 7f8df2ffd640 1 -- 172.21.15.26:0/779717498 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df4071240 con 0x7f8df40cf870 2024-04-24T23:23:12.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.366+0000 7f8df2ffd640 1 --2- 172.21.15.26:0/779717498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df40cf870 0x7f8df4072000 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8df4144d10 tx=0x7f8de802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2da8436c4d5c5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:12.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df27fc640 1 -- 172.21.15.26:0/779717498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8037070 con 0x7f8df40cf870 2024-04-24T23:23:12.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df27fc640 1 -- 172.21.15.26:0/779717498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de8032440 con 0x7f8df40cf870 2024-04-24T23:23:12.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df27fc640 1 -- 172.21.15.26:0/779717498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8032810 con 0x7f8df40cf870 2024-04-24T23:23:12.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df9eb3640 1 -- 172.21.15.26:0/779717498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df40cf870 msgr2=0x7f8df4072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:12.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df9eb3640 1 --2- 172.21.15.26:0/779717498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df40cf870 0x7f8df4072000 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8df4144d10 tx=0x7f8de802a270 comp rx=0 tx=0).stop 2024-04-24T23:23:12.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df9eb3640 1 -- 172.21.15.26:0/779717498 shutdown_connections 2024-04-24T23:23:12.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df9eb3640 1 --2- 172.21.15.26:0/779717498 >> v2:172.21.15.29:3300/0 conn(0x7f8df415eb00 0x7f8df415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df9eb3640 1 --2- 172.21.15.26:0/779717498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df40cf870 0x7f8df4072000 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df9eb3640 1 -- 172.21.15.26:0/779717498 >> 172.21.15.26:0/779717498 conn(0x7f8df406b300 msgr2=0x7f8df406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:12.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df9eb3640 1 -- 172.21.15.26:0/779717498 shutdown_connections 2024-04-24T23:23:12.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.367+0000 7f8df9eb3640 1 -- 172.21.15.26:0/779717498 wait complete. 2024-04-24T23:23:12.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.368+0000 7f8df9eb3640 1 Processor -- start 2024-04-24T23:23:12.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.369+0000 7f8df9eb3640 1 -- start start 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.369+0000 7f8df9eb3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8df40cf870 0x7f8df4132570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.369+0000 7f8df9eb3640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8df4143c40 0x7f8df415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.369+0000 7f8df9eb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df415eb00 0x7f8df4141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.369+0000 7f8df9eb3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4144020 con 0x7f8df415eb00 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.369+0000 7f8df9eb3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4071800 con 0x7f8df40cf870 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.369+0000 7f8df9eb3640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8df4072540 con 0x7f8df4143c40 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.369+0000 7f8df3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df415eb00 0x7f8df4141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df415eb00 0x7f8df4141df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45692/0 (socket says 172.21.15.26:45692) 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df3fff640 1 -- 172.21.15.26:0/1163484857 learned_addr learned my addr 172.21.15.26:0/1163484857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df2ffd640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8df4143c40 0x7f8df415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df37fe640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8df40cf870 0x7f8df4132570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df3fff640 1 -- 172.21.15.26:0/1163484857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8df4143c40 msgr2=0x7f8df415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df3fff640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8df4143c40 0x7f8df415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df3fff640 1 -- 172.21.15.26:0/1163484857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8df40cf870 msgr2=0x7f8df4132570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df3fff640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8df40cf870 0x7f8df4132570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df3fff640 1 -- 172.21.15.26:0/1163484857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df412f200 con 0x7f8df415eb00 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df3fff640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df415eb00 0x7f8df4141df0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009dd0 tx=0x7f8dec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8ddbfff640 1 -- 172.21.15.26:0/1163484857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec018020 con 0x7f8df415eb00 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8ddbfff640 1 -- 172.21.15.26:0/1163484857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dec014480 con 0x7f8df415eb00 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8ddbfff640 1 -- 172.21.15.26:0/1163484857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec014870 con 0x7f8df415eb00 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df412f4e0 con 0x7f8df415eb00 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.370+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df412faa0 con 0x7f8df415eb00 2024-04-24T23:23:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.371+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df40694c0 con 0x7f8df415eb00 2024-04-24T23:23:12.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.372+0000 7f8ddbfff640 1 -- 172.21.15.26:0/1163484857 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8dec005d10 con 0x7f8df415eb00 2024-04-24T23:23:12.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.372+0000 7f8ddbfff640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8de006be30 0x7f8de006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.372+0000 7f8ddbfff640 1 -- 172.21.15.26:0/1163484857 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f8dec08b8a0 con 0x7f8df415eb00 2024-04-24T23:23:12.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.372+0000 7f8df37fe640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8de006be30 0x7f8de006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.373+0000 7f8df37fe640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8de006be30 0x7f8de006e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000f60 tx=0x7f8de4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:12.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.377+0000 7f8ddbfff640 1 -- 172.21.15.26:0/1163484857 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8dec022070 con 0x7f8df415eb00 2024-04-24T23:23:12.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.555+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8df40b4d20 con 0x7f8de006be30 2024-04-24T23:23:12.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.561+0000 7f8ddbfff640 1 -- 172.21.15.26:0/1163484857 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251960 (secure 0 0 0) 0x7f8df40b4d20 con 0x7f8de006be30 2024-04-24T23:23:12.565 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:12.566 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.564+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8de006be30 msgr2=0x7f8de006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.564+0000 7f8df9eb3640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8de006be30 0x7f8de006e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000f60 tx=0x7f8de4009040 comp rx=0 tx=0).stop 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.564+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df415eb00 msgr2=0x7f8df4141df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.564+0000 7f8df9eb3640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df415eb00 0x7f8df4141df0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009dd0 tx=0x7f8dec00c490 comp rx=0 tx=0).stop 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.564+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 shutdown_connections 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.564+0000 7f8df9eb3640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8de006be30 0x7f8de006e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.565+0000 7f8df9eb3640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df415eb00 0x7f8df4141df0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.565+0000 7f8df9eb3640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8df4143c40 0x7f8df415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.565+0000 7f8df9eb3640 1 --2- 172.21.15.26:0/1163484857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8df40cf870 0x7f8df4132570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.565+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 >> 172.21.15.26:0/1163484857 conn(0x7f8df406b300 msgr2=0x7f8df40b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.565+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 shutdown_connections 2024-04-24T23:23:12.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.565+0000 7f8df9eb3640 1 -- 172.21.15.26:0/1163484857 wait complete. 2024-04-24T23:23:12.577 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-04-24T23:23:11.344529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258880,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715840,"statfs":{"total":773094113280,"available":772829020160,"internally_reserved":0,"allocated":5382144,"data_stored":2871956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001782"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876931+0000","last_change":"2024-04-24T23:23:03.813800+0000","last_active":"2024-04-24T23:23:04.876931+0000","last_peered":"2024-04-24T23:23:04.876931+0000","last_clean":"2024-04-24T23:23:04.876931+0000","last_became_active":"2024-04-24T23:23:03.813489+0000","last_became_peered":"2024-04-24T23:23:03.813489+0000","last_unstale":"2024-04-24T23:23:04.876931+0000","last_undegraded":"2024-04-24T23:23:04.876931+0000","last_fullsized":"2024-04-24T23:23:04.876931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:42.173666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876990+0000","last_change":"2024-04-24T23:23:03.812029+0000","last_active":"2024-04-24T23:23:04.876990+0000","last_peered":"2024-04-24T23:23:04.876990+0000","last_clean":"2024-04-24T23:23:04.876990+0000","last_became_active":"2024-04-24T23:23:03.811720+0000","last_became_peered":"2024-04-24T23:23:03.811720+0000","last_unstale":"2024-04-24T23:23:04.876990+0000","last_undegraded":"2024-04-24T23:23:04.876990+0000","last_fullsized":"2024-04-24T23:23:04.876990+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:00.174563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807104+0000","last_change":"2024-04-24T23:23:03.805799+0000","last_active":"2024-04-24T23:23:04.807104+0000","last_peered":"2024-04-24T23:23:04.807104+0000","last_clean":"2024-04-24T23:23:04.807104+0000","last_became_active":"2024-04-24T23:23:03.805385+0000","last_became_peered":"2024-04-24T23:23:03.805385+0000","last_unstale":"2024-04-24T23:23:04.807104+0000","last_undegraded":"2024-04-24T23:23:04.807104+0000","last_fullsized":"2024-04-24T23:23:04.807104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:40.145575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875691+0000","last_change":"2024-04-24T23:23:03.817520+0000","last_active":"2024-04-24T23:23:04.875691+0000","last_peered":"2024-04-24T23:23:04.875691+0000","last_clean":"2024-04-24T23:23:04.875691+0000","last_became_active":"2024-04-24T23:23:03.816939+0000","last_became_peered":"2024-04-24T23:23:03.816939+0000","last_unstale":"2024-04-24T23:23:04.875691+0000","last_undegraded":"2024-04-24T23:23:04.875691+0000","last_fullsized":"2024-04-24T23:23:04.875691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:02.849082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876432+0000","last_change":"2024-04-24T23:23:03.814784+0000","last_active":"2024-04-24T23:23:04.876432+0000","last_peered":"2024-04-24T23:23:04.876432+0000","last_clean":"2024-04-24T23:23:04.876432+0000","last_became_active":"2024-04-24T23:23:03.814430+0000","last_became_peered":"2024-04-24T23:23:03.814430+0000","last_unstale":"2024-04-24T23:23:04.876432+0000","last_undegraded":"2024-04-24T23:23:04.876432+0000","last_fullsized":"2024-04-24T23:23:04.876432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:36.194366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804193+0000","last_change":"2024-04-24T23:23:03.820491+0000","last_active":"2024-04-24T23:23:04.804193+0000","last_peered":"2024-04-24T23:23:04.804193+0000","last_clean":"2024-04-24T23:23:04.804193+0000","last_became_active":"2024-04-24T23:23:03.820224+0000","last_became_peered":"2024-04-24T23:23:03.820224+0000","last_unstale":"2024-04-24T23:23:04.804193+0000","last_undegraded":"2024-04-24T23:23:04.804193+0000","last_fullsized":"2024-04-24T23:23:04.804193+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:10.078598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876374+0000","last_change":"2024-04-24T23:23:03.805849+0000","last_active":"2024-04-24T23:23:04.876374+0000","last_peered":"2024-04-24T23:23:04.876374+0000","last_clean":"2024-04-24T23:23:04.876374+0000","last_became_active":"2024-04-24T23:23:03.805508+0000","last_became_peered":"2024-04-24T23:23:03.805508+0000","last_unstale":"2024-04-24T23:23:04.876374+0000","last_undegraded":"2024-04-24T23:23:04.876374+0000","last_fullsized":"2024-04-24T23:23:04.876374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:11.161617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875451+0000","last_change":"2024-04-24T23:23:03.816393+0000","last_active":"2024-04-24T23:23:04.875451+0000","last_peered":"2024-04-24T23:23:04.875451+0000","last_clean":"2024-04-24T23:23:04.875451+0000","last_became_active":"2024-04-24T23:23:03.816087+0000","last_became_peered":"2024-04-24T23:23:03.816087+0000","last_unstale":"2024-04-24T23:23:04.875451+0000","last_undegraded":"2024-04-24T23:23:04.875451+0000","last_fullsized":"2024-04-24T23:23:04.875451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:27.653379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876130+0000","last_change":"2024-04-24T23:23:03.814079+0000","last_active":"2024-04-24T23:23:04.876130+0000","last_peered":"2024-04-24T23:23:04.876130+0000","last_clean":"2024-04-24T23:23:04.876130+0000","last_became_active":"2024-04-24T23:23:03.812660+0000","last_became_peered":"2024-04-24T23:23:03.812660+0000","last_unstale":"2024-04-24T23:23:04.876130+0000","last_undegraded":"2024-04-24T23:23:04.876130+0000","last_fullsized":"2024-04-24T23:23:04.876130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:36.194366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875994+0000","last_change":"2024-04-24T23:23:03.816534+0000","last_active":"2024-04-24T23:23:04.875994+0000","last_peered":"2024-04-24T23:23:04.875994+0000","last_clean":"2024-04-24T23:23:04.875994+0000","last_became_active":"2024-04-24T23:23:03.816264+0000","last_became_peered":"2024-04-24T23:23:03.816264+0000","last_unstale":"2024-04-24T23:23:04.875994+0000","last_undegraded":"2024-04-24T23:23:04.875994+0000","last_fullsized":"2024-04-24T23:23:04.875994+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:19.256811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875924+0000","last_change":"2024-04-24T23:23:03.811718+0000","last_active":"2024-04-24T23:23:04.875924+0000","last_peered":"2024-04-24T23:23:04.875924+0000","last_clean":"2024-04-24T23:23:04.875924+0000","last_became_active":"2024-04-24T23:23:03.811433+0000","last_became_peered":"2024-04-24T23:23:03.811433+0000","last_unstale":"2024-04-24T23:23:04.875924+0000","last_undegraded":"2024-04-24T23:23:04.875924+0000","last_fullsized":"2024-04-24T23:23:04.875924+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:55.568926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876672+0000","last_change":"2024-04-24T23:23:03.798408+0000","last_active":"2024-04-24T23:23:04.876672+0000","last_peered":"2024-04-24T23:23:04.876672+0000","last_clean":"2024-04-24T23:23:04.876672+0000","last_became_active":"2024-04-24T23:23:03.798137+0000","last_became_peered":"2024-04-24T23:23:03.798137+0000","last_unstale":"2024-04-24T23:23:04.876672+0000","last_undegraded":"2024-04-24T23:23:04.876672+0000","last_fullsized":"2024-04-24T23:23:04.876672+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:10.391691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.801311+0000","last_change":"2024-04-24T23:23:03.818507+0000","last_active":"2024-04-24T23:23:04.801311+0000","last_peered":"2024-04-24T23:23:04.801311+0000","last_clean":"2024-04-24T23:23:04.801311+0000","last_became_active":"2024-04-24T23:23:03.818191+0000","last_became_peered":"2024-04-24T23:23:03.818191+0000","last_unstale":"2024-04-24T23:23:04.801311+0000","last_undegraded":"2024-04-24T23:23:04.801311+0000","last_fullsized":"2024-04-24T23:23:04.801311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:20.566195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875638+0000","last_change":"2024-04-24T23:23:03.812340+0000","last_active":"2024-04-24T23:23:04.875638+0000","last_peered":"2024-04-24T23:23:04.875638+0000","last_clean":"2024-04-24T23:23:04.875638+0000","last_became_active":"2024-04-24T23:23:03.812055+0000","last_became_peered":"2024-04-24T23:23:03.812055+0000","last_unstale":"2024-04-24T23:23:04.875638+0000","last_undegraded":"2024-04-24T23:23:04.875638+0000","last_fullsized":"2024-04-24T23:23:04.875638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:40.145575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804191+0000","last_change":"2024-04-24T23:23:03.802546+0000","last_active":"2024-04-24T23:23:04.804191+0000","last_peered":"2024-04-24T23:23:04.804191+0000","last_clean":"2024-04-24T23:23:04.804191+0000","last_became_active":"2024-04-24T23:23:03.802325+0000","last_became_peered":"2024-04-24T23:23:03.802325+0000","last_unstale":"2024-04-24T23:23:04.804191+0000","last_undegraded":"2024-04-24T23:23:04.804191+0000","last_fullsized":"2024-04-24T23:23:04.804191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:52.838619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.802655+0000","last_change":"2024-04-24T23:23:03.810252+0000","last_active":"2024-04-24T23:23:04.802655+0000","last_peered":"2024-04-24T23:23:04.802655+0000","last_clean":"2024-04-24T23:23:04.802655+0000","last_became_active":"2024-04-24T23:23:03.809935+0000","last_became_peered":"2024-04-24T23:23:03.809935+0000","last_unstale":"2024-04-24T23:23:04.802655+0000","last_undegraded":"2024-04-24T23:23:04.802655+0000","last_fullsized":"2024-04-24T23:23:04.802655+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:53.713386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807212+0000","last_change":"2024-04-24T23:23:03.801943+0000","last_active":"2024-04-24T23:23:04.807212+0000","last_peered":"2024-04-24T23:23:04.807212+0000","last_clean":"2024-04-24T23:23:04.807212+0000","last_became_active":"2024-04-24T23:23:03.801621+0000","last_became_peered":"2024-04-24T23:23:03.801621+0000","last_unstale":"2024-04-24T23:23:04.807212+0000","last_undegraded":"2024-04-24T23:23:04.807212+0000","last_fullsized":"2024-04-24T23:23:04.807212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:25.185587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807180+0000","last_change":"2024-04-24T23:23:03.809961+0000","last_active":"2024-04-24T23:23:04.807180+0000","last_peered":"2024-04-24T23:23:04.807180+0000","last_clean":"2024-04-24T23:23:04.807180+0000","last_became_active":"2024-04-24T23:23:03.809679+0000","last_became_peered":"2024-04-24T23:23:03.809679+0000","last_unstale":"2024-04-24T23:23:04.807180+0000","last_undegraded":"2024-04-24T23:23:04.807180+0000","last_fullsized":"2024-04-24T23:23:04.807180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:17.514647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.803135+0000","last_change":"2024-04-24T23:23:03.810289+0000","last_active":"2024-04-24T23:23:04.803135+0000","last_peered":"2024-04-24T23:23:04.803135+0000","last_clean":"2024-04-24T23:23:04.803135+0000","last_became_active":"2024-04-24T23:23:03.809526+0000","last_became_peered":"2024-04-24T23:23:03.809526+0000","last_unstale":"2024-04-24T23:23:04.803135+0000","last_undegraded":"2024-04-24T23:23:04.803135+0000","last_fullsized":"2024-04-24T23:23:04.803135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:26.950107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.801477+0000","last_change":"2024-04-24T23:23:03.813453+0000","last_active":"2024-04-24T23:23:04.801477+0000","last_peered":"2024-04-24T23:23:04.801477+0000","last_clean":"2024-04-24T23:23:04.801477+0000","last_became_active":"2024-04-24T23:23:03.813181+0000","last_became_peered":"2024-04-24T23:23:03.813181+0000","last_unstale":"2024-04-24T23:23:04.801477+0000","last_undegraded":"2024-04-24T23:23:04.801477+0000","last_fullsized":"2024-04-24T23:23:04.801477+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:52.565320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804817+0000","last_change":"2024-04-24T23:23:03.808333+0000","last_active":"2024-04-24T23:23:04.804817+0000","last_peered":"2024-04-24T23:23:04.804817+0000","last_clean":"2024-04-24T23:23:04.804817+0000","last_became_active":"2024-04-24T23:23:03.805471+0000","last_became_peered":"2024-04-24T23:23:03.805471+0000","last_unstale":"2024-04-24T23:23:04.804817+0000","last_undegraded":"2024-04-24T23:23:04.804817+0000","last_fullsized":"2024-04-24T23:23:04.804817+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:47.341154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876186+0000","last_change":"2024-04-24T23:23:03.817239+0000","last_active":"2024-04-24T23:23:04.876186+0000","last_peered":"2024-04-24T23:23:04.876186+0000","last_clean":"2024-04-24T23:23:04.876186+0000","last_became_active":"2024-04-24T23:23:03.816971+0000","last_became_peered":"2024-04-24T23:23:03.816971+0000","last_unstale":"2024-04-24T23:23:04.876186+0000","last_undegraded":"2024-04-24T23:23:04.876186+0000","last_fullsized":"2024-04-24T23:23:04.876186+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:10.391691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.802492+0000","last_change":"2024-04-24T23:23:03.803457+0000","last_active":"2024-04-24T23:23:04.802492+0000","last_peered":"2024-04-24T23:23:04.802492+0000","last_clean":"2024-04-24T23:23:04.802492+0000","last_became_active":"2024-04-24T23:23:03.803156+0000","last_became_peered":"2024-04-24T23:23:03.803156+0000","last_unstale":"2024-04-24T23:23:04.802492+0000","last_undegraded":"2024-04-24T23:23:04.802492+0000","last_fullsized":"2024-04-24T23:23:04.802492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:22.028810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.802469+0000","last_change":"2024-04-24T23:23:03.806456+0000","last_active":"2024-04-24T23:23:04.802469+0000","last_peered":"2024-04-24T23:23:04.802469+0000","last_clean":"2024-04-24T23:23:04.802469+0000","last_became_active":"2024-04-24T23:23:03.806148+0000","last_became_peered":"2024-04-24T23:23:03.806148+0000","last_unstale":"2024-04-24T23:23:04.802469+0000","last_undegraded":"2024-04-24T23:23:04.802469+0000","last_fullsized":"2024-04-24T23:23:04.802469+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:05.707009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.803121+0000","last_change":"2024-04-24T23:23:03.806547+0000","last_active":"2024-04-24T23:23:04.803121+0000","last_peered":"2024-04-24T23:23:04.803121+0000","last_clean":"2024-04-24T23:23:04.803121+0000","last_became_active":"2024-04-24T23:23:03.806223+0000","last_became_peered":"2024-04-24T23:23:03.806223+0000","last_unstale":"2024-04-24T23:23:04.803121+0000","last_undegraded":"2024-04-24T23:23:04.803121+0000","last_fullsized":"2024-04-24T23:23:04.803121+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:58.433649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.801774+0000","last_change":"2024-04-24T23:23:03.818753+0000","last_active":"2024-04-24T23:23:04.801774+0000","last_peered":"2024-04-24T23:23:04.801774+0000","last_clean":"2024-04-24T23:23:04.801774+0000","last_became_active":"2024-04-24T23:23:03.818079+0000","last_became_peered":"2024-04-24T23:23:03.818079+0000","last_unstale":"2024-04-24T23:23:04.801774+0000","last_undegraded":"2024-04-24T23:23:04.801774+0000","last_fullsized":"2024-04-24T23:23:04.801774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:31.731740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807219+0000","last_change":"2024-04-24T23:23:03.802720+0000","last_active":"2024-04-24T23:23:04.807219+0000","last_peered":"2024-04-24T23:23:04.807219+0000","last_clean":"2024-04-24T23:23:04.807219+0000","last_became_active":"2024-04-24T23:23:03.802458+0000","last_became_peered":"2024-04-24T23:23:03.802458+0000","last_unstale":"2024-04-24T23:23:04.807219+0000","last_undegraded":"2024-04-24T23:23:04.807219+0000","last_fullsized":"2024-04-24T23:23:04.807219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:42.047136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876862+0000","last_change":"2024-04-24T23:23:03.797271+0000","last_active":"2024-04-24T23:23:04.876862+0000","last_peered":"2024-04-24T23:23:04.876862+0000","last_clean":"2024-04-24T23:23:04.876862+0000","last_became_active":"2024-04-24T23:23:03.796995+0000","last_became_peered":"2024-04-24T23:23:03.796995+0000","last_unstale":"2024-04-24T23:23:04.876862+0000","last_undegraded":"2024-04-24T23:23:04.876862+0000","last_fullsized":"2024-04-24T23:23:04.876862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:56.026414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.803038+0000","last_change":"2024-04-24T23:23:03.806529+0000","last_active":"2024-04-24T23:23:04.803038+0000","last_peered":"2024-04-24T23:23:04.803038+0000","last_clean":"2024-04-24T23:23:04.803038+0000","last_became_active":"2024-04-24T23:23:03.806185+0000","last_became_peered":"2024-04-24T23:23:03.806185+0000","last_unstale":"2024-04-24T23:23:04.803038+0000","last_undegraded":"2024-04-24T23:23:04.803038+0000","last_fullsized":"2024-04-24T23:23:04.803038+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:52:04.758623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.806873+0000","last_change":"2024-04-24T23:23:03.809606+0000","last_active":"2024-04-24T23:23:04.806873+0000","last_peered":"2024-04-24T23:23:04.806873+0000","last_clean":"2024-04-24T23:23:04.806873+0000","last_became_active":"2024-04-24T23:23:03.809271+0000","last_became_peered":"2024-04-24T23:23:03.809271+0000","last_unstale":"2024-04-24T23:23:04.806873+0000","last_undegraded":"2024-04-24T23:23:04.806873+0000","last_fullsized":"2024-04-24T23:23:04.806873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:27.653379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804295+0000","last_change":"2024-04-24T23:23:03.813107+0000","last_active":"2024-04-24T23:23:04.804295+0000","last_peered":"2024-04-24T23:23:04.804295+0000","last_clean":"2024-04-24T23:23:04.804295+0000","last_became_active":"2024-04-24T23:23:03.812817+0000","last_became_peered":"2024-04-24T23:23:03.812817+0000","last_unstale":"2024-04-24T23:23:04.804295+0000","last_undegraded":"2024-04-24T23:23:04.804295+0000","last_fullsized":"2024-04-24T23:23:04.804295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:53.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.801266+0000","last_change":"2024-04-24T23:23:03.809445+0000","last_active":"2024-04-24T23:23:04.801266+0000","last_peered":"2024-04-24T23:23:04.801266+0000","last_clean":"2024-04-24T23:23:04.801266+0000","last_became_active":"2024-04-24T23:23:03.809137+0000","last_became_peered":"2024-04-24T23:23:03.809137+0000","last_unstale":"2024-04-24T23:23:04.801266+0000","last_undegraded":"2024-04-24T23:23:04.801266+0000","last_fullsized":"2024-04-24T23:23:04.801266+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:11.512747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876304+0000","last_change":"2024-04-24T23:23:03.819884+0000","last_active":"2024-04-24T23:23:04.876304+0000","last_peered":"2024-04-24T23:23:04.876304+0000","last_clean":"2024-04-24T23:23:04.876304+0000","last_became_active":"2024-04-24T23:23:03.819569+0000","last_became_peered":"2024-04-24T23:23:03.819569+0000","last_unstale":"2024-04-24T23:23:04.876304+0000","last_undegraded":"2024-04-24T23:23:04.876304+0000","last_fullsized":"2024-04-24T23:23:04.876304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:58.433649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804480+0000","last_change":"2024-04-24T23:23:02.806330+0000","last_active":"2024-04-24T23:23:04.804480+0000","last_peered":"2024-04-24T23:23:04.804480+0000","last_clean":"2024-04-24T23:23:04.804480+0000","last_became_active":"2024-04-24T23:23:02.805806+0000","last_became_peered":"2024-04-24T23:23:02.805806+0000","last_unstale":"2024-04-24T23:23:04.804480+0000","last_undegraded":"2024-04-24T23:23:04.804480+0000","last_fullsized":"2024-04-24T23:23:04.804480+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:55.022024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.803126+0000","last_change":"2024-04-24T23:23:03.810142+0000","last_active":"2024-04-24T23:23:04.803126+0000","last_peered":"2024-04-24T23:23:04.803126+0000","last_clean":"2024-04-24T23:23:04.803126+0000","last_became_active":"2024-04-24T23:23:03.809680+0000","last_became_peered":"2024-04-24T23:23:03.809680+0000","last_unstale":"2024-04-24T23:23:04.803126+0000","last_undegraded":"2024-04-24T23:23:04.803126+0000","last_fullsized":"2024-04-24T23:23:04.803126+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:51.171025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807269+0000","last_change":"2024-04-24T23:23:02.803537+0000","last_active":"2024-04-24T23:23:04.807269+0000","last_peered":"2024-04-24T23:23:04.807269+0000","last_clean":"2024-04-24T23:23:04.807269+0000","last_became_active":"2024-04-24T23:23:02.803250+0000","last_became_peered":"2024-04-24T23:23:02.803250+0000","last_unstale":"2024-04-24T23:23:04.807269+0000","last_undegraded":"2024-04-24T23:23:04.807269+0000","last_fullsized":"2024-04-24T23:23:04.807269+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:56.215292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876960+0000","last_change":"2024-04-24T23:23:03.815177+0000","last_active":"2024-04-24T23:23:04.876960+0000","last_peered":"2024-04-24T23:23:04.876960+0000","last_clean":"2024-04-24T23:23:04.876960+0000","last_became_active":"2024-04-24T23:23:03.814915+0000","last_became_peered":"2024-04-24T23:23:03.814915+0000","last_unstale":"2024-04-24T23:23:04.876960+0000","last_undegraded":"2024-04-24T23:23:04.876960+0000","last_fullsized":"2024-04-24T23:23:04.876960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:53.978073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876453+0000","last_change":"2024-04-24T23:23:02.804284+0000","last_active":"2024-04-24T23:23:04.876453+0000","last_peered":"2024-04-24T23:23:04.876453+0000","last_clean":"2024-04-24T23:23:04.876453+0000","last_became_active":"2024-04-24T23:23:02.804017+0000","last_became_peered":"2024-04-24T23:23:02.804017+0000","last_unstale":"2024-04-24T23:23:04.876453+0000","last_undegraded":"2024-04-24T23:23:04.876453+0000","last_fullsized":"2024-04-24T23:23:04.876453+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:10.466895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876482+0000","last_change":"2024-04-24T23:23:03.805266+0000","last_active":"2024-04-24T23:23:04.876482+0000","last_peered":"2024-04-24T23:23:04.876482+0000","last_clean":"2024-04-24T23:23:04.876482+0000","last_became_active":"2024-04-24T23:23:03.804968+0000","last_became_peered":"2024-04-24T23:23:03.804968+0000","last_unstale":"2024-04-24T23:23:04.876482+0000","last_undegraded":"2024-04-24T23:23:04.876482+0000","last_fullsized":"2024-04-24T23:23:04.876482+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:51.171025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876259+0000","last_change":"2024-04-24T23:23:02.806875+0000","last_active":"2024-04-24T23:23:04.876259+0000","last_peered":"2024-04-24T23:23:04.876259+0000","last_clean":"2024-04-24T23:23:04.876259+0000","last_became_active":"2024-04-24T23:23:02.806472+0000","last_became_peered":"2024-04-24T23:23:02.806472+0000","last_unstale":"2024-04-24T23:23:04.876259+0000","last_undegraded":"2024-04-24T23:23:04.876259+0000","last_fullsized":"2024-04-24T23:23:04.876259+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:41.042746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807035+0000","last_change":"2024-04-24T23:23:03.804451+0000","last_active":"2024-04-24T23:23:04.807035+0000","last_peered":"2024-04-24T23:23:04.807035+0000","last_clean":"2024-04-24T23:23:04.807035+0000","last_became_active":"2024-04-24T23:23:03.804177+0000","last_became_peered":"2024-04-24T23:23:03.804177+0000","last_unstale":"2024-04-24T23:23:04.807035+0000","last_undegraded":"2024-04-24T23:23:04.807035+0000","last_fullsized":"2024-04-24T23:23:04.807035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:53.293083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804589+0000","last_change":"2024-04-24T23:23:02.803217+0000","last_active":"2024-04-24T23:23:04.804589+0000","last_peered":"2024-04-24T23:23:04.804589+0000","last_clean":"2024-04-24T23:23:04.804589+0000","last_became_active":"2024-04-24T23:23:02.802687+0000","last_became_peered":"2024-04-24T23:23:02.802687+0000","last_unstale":"2024-04-24T23:23:04.804589+0000","last_undegraded":"2024-04-24T23:23:04.804589+0000","last_fullsized":"2024-04-24T23:23:04.804589+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:35.189975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875533+0000","last_change":"2024-04-24T23:23:03.813465+0000","last_active":"2024-04-24T23:23:04.875533+0000","last_peered":"2024-04-24T23:23:04.875533+0000","last_clean":"2024-04-24T23:23:04.875533+0000","last_became_active":"2024-04-24T23:23:03.811765+0000","last_became_peered":"2024-04-24T23:23:03.811765+0000","last_unstale":"2024-04-24T23:23:04.875533+0000","last_undegraded":"2024-04-24T23:23:04.875533+0000","last_fullsized":"2024-04-24T23:23:04.875533+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:53.293083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.806963+0000","last_change":"2024-04-24T23:23:02.795543+0000","last_active":"2024-04-24T23:23:04.806963+0000","last_peered":"2024-04-24T23:23:04.806963+0000","last_clean":"2024-04-24T23:23:04.806963+0000","last_became_active":"2024-04-24T23:23:02.795019+0000","last_became_peered":"2024-04-24T23:23:02.795019+0000","last_unstale":"2024-04-24T23:23:04.806963+0000","last_undegraded":"2024-04-24T23:23:04.806963+0000","last_fullsized":"2024-04-24T23:23:04.806963+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:30.727349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875498+0000","last_change":"2024-04-24T23:23:03.815625+0000","last_active":"2024-04-24T23:23:04.875498+0000","last_peered":"2024-04-24T23:23:04.875498+0000","last_clean":"2024-04-24T23:23:04.875498+0000","last_became_active":"2024-04-24T23:23:03.815341+0000","last_became_peered":"2024-04-24T23:23:03.815341+0000","last_unstale":"2024-04-24T23:23:04.875498+0000","last_undegraded":"2024-04-24T23:23:04.875498+0000","last_fullsized":"2024-04-24T23:23:04.875498+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:08.896446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876010+0000","last_change":"2024-04-24T23:23:02.798953+0000","last_active":"2024-04-24T23:23:04.876010+0000","last_peered":"2024-04-24T23:23:04.876010+0000","last_clean":"2024-04-24T23:23:04.876010+0000","last_became_active":"2024-04-24T23:23:02.798593+0000","last_became_peered":"2024-04-24T23:23:02.798593+0000","last_unstale":"2024-04-24T23:23:04.876010+0000","last_undegraded":"2024-04-24T23:23:04.876010+0000","last_fullsized":"2024-04-24T23:23:04.876010+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:52.288693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.802490+0000","last_change":"2024-04-24T23:23:03.802919+0000","last_active":"2024-04-24T23:23:04.802490+0000","last_peered":"2024-04-24T23:23:04.802490+0000","last_clean":"2024-04-24T23:23:04.802490+0000","last_became_active":"2024-04-24T23:23:03.802603+0000","last_became_peered":"2024-04-24T23:23:03.802603+0000","last_unstale":"2024-04-24T23:23:04.802490+0000","last_undegraded":"2024-04-24T23:23:04.802490+0000","last_fullsized":"2024-04-24T23:23:04.802490+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:57.219682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875509+0000","last_change":"2024-04-24T23:23:02.808922+0000","last_active":"2024-04-24T23:23:04.875509+0000","last_peered":"2024-04-24T23:23:04.875509+0000","last_clean":"2024-04-24T23:23:04.875509+0000","last_became_active":"2024-04-24T23:23:02.808623+0000","last_became_peered":"2024-04-24T23:23:02.808623+0000","last_unstale":"2024-04-24T23:23:04.875509+0000","last_undegraded":"2024-04-24T23:23:04.875509+0000","last_fullsized":"2024-04-24T23:23:04.875509+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:19.561805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807197+0000","last_change":"2024-04-24T23:23:03.811167+0000","last_active":"2024-04-24T23:23:04.807197+0000","last_peered":"2024-04-24T23:23:04.807197+0000","last_clean":"2024-04-24T23:23:04.807197+0000","last_became_active":"2024-04-24T23:23:03.810881+0000","last_became_peered":"2024-04-24T23:23:03.810881+0000","last_unstale":"2024-04-24T23:23:04.807197+0000","last_undegraded":"2024-04-24T23:23:04.807197+0000","last_fullsized":"2024-04-24T23:23:04.807197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:02.849082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807257+0000","last_change":"2024-04-24T23:23:02.798123+0000","last_active":"2024-04-24T23:23:04.807257+0000","last_peered":"2024-04-24T23:23:04.807257+0000","last_clean":"2024-04-24T23:23:04.807257+0000","last_became_active":"2024-04-24T23:23:02.797833+0000","last_became_peered":"2024-04-24T23:23:02.797833+0000","last_unstale":"2024-04-24T23:23:04.807257+0000","last_undegraded":"2024-04-24T23:23:04.807257+0000","last_fullsized":"2024-04-24T23:23:04.807257+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:30.385642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876974+0000","last_change":"2024-04-24T23:23:03.812750+0000","last_active":"2024-04-24T23:23:04.876974+0000","last_peered":"2024-04-24T23:23:04.876974+0000","last_clean":"2024-04-24T23:23:04.876974+0000","last_became_active":"2024-04-24T23:23:03.812433+0000","last_became_peered":"2024-04-24T23:23:03.812433+0000","last_unstale":"2024-04-24T23:23:04.876974+0000","last_undegraded":"2024-04-24T23:23:04.876974+0000","last_fullsized":"2024-04-24T23:23:04.876974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:23.219321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876052+0000","last_change":"2024-04-24T23:23:02.805326+0000","last_active":"2024-04-24T23:23:04.876052+0000","last_peered":"2024-04-24T23:23:04.876052+0000","last_clean":"2024-04-24T23:23:04.876052+0000","last_became_active":"2024-04-24T23:23:02.805030+0000","last_became_peered":"2024-04-24T23:23:02.805030+0000","last_unstale":"2024-04-24T23:23:04.876052+0000","last_undegraded":"2024-04-24T23:23:04.876052+0000","last_fullsized":"2024-04-24T23:23:04.876052+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:10.508357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804471+0000","last_change":"2024-04-24T23:23:03.817218+0000","last_active":"2024-04-24T23:23:04.804471+0000","last_peered":"2024-04-24T23:23:04.804471+0000","last_clean":"2024-04-24T23:23:04.804471+0000","last_became_active":"2024-04-24T23:23:03.816939+0000","last_became_peered":"2024-04-24T23:23:03.816939+0000","last_unstale":"2024-04-24T23:23:04.804471+0000","last_undegraded":"2024-04-24T23:23:04.804471+0000","last_fullsized":"2024-04-24T23:23:04.804471+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:41.227176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876703+0000","last_change":"2024-04-24T23:23:02.805927+0000","last_active":"2024-04-24T23:23:04.876703+0000","last_peered":"2024-04-24T23:23:04.876703+0000","last_clean":"2024-04-24T23:23:04.876703+0000","last_became_active":"2024-04-24T23:23:02.805617+0000","last_became_peered":"2024-04-24T23:23:02.805617+0000","last_unstale":"2024-04-24T23:23:04.876703+0000","last_undegraded":"2024-04-24T23:23:04.876703+0000","last_fullsized":"2024-04-24T23:23:04.876703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:54.564535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876911+0000","last_change":"2024-04-24T23:23:03.806730+0000","last_active":"2024-04-24T23:23:04.876911+0000","last_peered":"2024-04-24T23:23:04.876911+0000","last_clean":"2024-04-24T23:23:04.876911+0000","last_became_active":"2024-04-24T23:23:03.806232+0000","last_became_peered":"2024-04-24T23:23:03.806232+0000","last_unstale":"2024-04-24T23:23:04.876911+0000","last_undegraded":"2024-04-24T23:23:04.876911+0000","last_fullsized":"2024-04-24T23:23:04.876911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:14.086138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.801399+0000","last_change":"2024-04-24T23:23:02.802159+0000","last_active":"2024-04-24T23:23:04.801399+0000","last_peered":"2024-04-24T23:23:04.801399+0000","last_clean":"2024-04-24T23:23:04.801399+0000","last_became_active":"2024-04-24T23:23:02.801879+0000","last_became_peered":"2024-04-24T23:23:02.801879+0000","last_unstale":"2024-04-24T23:23:04.801399+0000","last_undegraded":"2024-04-24T23:23:04.801399+0000","last_fullsized":"2024-04-24T23:23:04.801399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:11.334056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.802850+0000","last_change":"2024-04-24T23:23:03.813459+0000","last_active":"2024-04-24T23:23:04.802850+0000","last_peered":"2024-04-24T23:23:04.802850+0000","last_clean":"2024-04-24T23:23:04.802850+0000","last_became_active":"2024-04-24T23:23:03.813130+0000","last_became_peered":"2024-04-24T23:23:03.813130+0000","last_unstale":"2024-04-24T23:23:04.802850+0000","last_undegraded":"2024-04-24T23:23:04.802850+0000","last_fullsized":"2024-04-24T23:23:04.802850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:11.471285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804690+0000","last_change":"2024-04-24T23:23:02.807162+0000","last_active":"2024-04-24T23:23:04.804690+0000","last_peered":"2024-04-24T23:23:04.804690+0000","last_clean":"2024-04-24T23:23:04.804690+0000","last_became_active":"2024-04-24T23:23:02.806863+0000","last_became_peered":"2024-04-24T23:23:02.806863+0000","last_unstale":"2024-04-24T23:23:04.804690+0000","last_undegraded":"2024-04-24T23:23:04.804690+0000","last_fullsized":"2024-04-24T23:23:04.804690+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:09.387301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875681+0000","last_change":"2024-04-24T23:23:03.811905+0000","last_active":"2024-04-24T23:23:04.875681+0000","last_peered":"2024-04-24T23:23:04.875681+0000","last_clean":"2024-04-24T23:23:04.875681+0000","last_became_active":"2024-04-24T23:23:03.811624+0000","last_became_peered":"2024-04-24T23:23:03.811624+0000","last_unstale":"2024-04-24T23:23:04.875681+0000","last_undegraded":"2024-04-24T23:23:04.875681+0000","last_fullsized":"2024-04-24T23:23:04.875681+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:17.514647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876409+0000","last_change":"2024-04-24T23:23:02.806661+0000","last_active":"2024-04-24T23:23:04.876409+0000","last_peered":"2024-04-24T23:23:04.876409+0000","last_clean":"2024-04-24T23:23:04.876409+0000","last_became_active":"2024-04-24T23:23:02.806295+0000","last_became_peered":"2024-04-24T23:23:02.806295+0000","last_unstale":"2024-04-24T23:23:04.876409+0000","last_undegraded":"2024-04-24T23:23:04.876409+0000","last_fullsized":"2024-04-24T23:23:04.876409+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:01.844691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804139+0000","last_change":"2024-04-24T23:23:03.812925+0000","last_active":"2024-04-24T23:23:04.804139+0000","last_peered":"2024-04-24T23:23:04.804139+0000","last_clean":"2024-04-24T23:23:04.804139+0000","last_became_active":"2024-04-24T23:23:03.812597+0000","last_became_peered":"2024-04-24T23:23:03.812597+0000","last_unstale":"2024-04-24T23:23:04.804139+0000","last_undegraded":"2024-04-24T23:23:04.804139+0000","last_fullsized":"2024-04-24T23:23:04.804139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:58.151804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804353+0000","last_change":"2024-04-24T23:23:02.808298+0000","last_active":"2024-04-24T23:23:04.804353+0000","last_peered":"2024-04-24T23:23:04.804353+0000","last_clean":"2024-04-24T23:23:04.804353+0000","last_became_active":"2024-04-24T23:23:02.808022+0000","last_became_peered":"2024-04-24T23:23:02.808022+0000","last_unstale":"2024-04-24T23:23:04.804353+0000","last_undegraded":"2024-04-24T23:23:04.804353+0000","last_fullsized":"2024-04-24T23:23:04.804353+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:18.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876556+0000","last_change":"2024-04-24T23:23:03.808857+0000","last_active":"2024-04-24T23:23:04.876556+0000","last_peered":"2024-04-24T23:23:04.876556+0000","last_clean":"2024-04-24T23:23:04.876556+0000","last_became_active":"2024-04-24T23:23:03.808537+0000","last_became_peered":"2024-04-24T23:23:03.808537+0000","last_unstale":"2024-04-24T23:23:04.876556+0000","last_undegraded":"2024-04-24T23:23:04.876556+0000","last_fullsized":"2024-04-24T23:23:04.876556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:22.927325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":22,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875559+0000","last_change":"2024-04-24T23:23:02.807596+0000","last_active":"2024-04-24T23:23:04.875559+0000","last_peered":"2024-04-24T23:23:04.875559+0000","last_clean":"2024-04-24T23:23:04.875559+0000","last_became_active":"2024-04-24T23:23:02.806613+0000","last_became_peered":"2024-04-24T23:23:02.806613+0000","last_unstale":"2024-04-24T23:23:04.875559+0000","last_undegraded":"2024-04-24T23:23:04.875559+0000","last_fullsized":"2024-04-24T23:23:04.875559+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:16.913280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.801740+0000","last_change":"2024-04-24T23:23:03.795888+0000","last_active":"2024-04-24T23:23:04.801740+0000","last_peered":"2024-04-24T23:23:04.801740+0000","last_clean":"2024-04-24T23:23:04.801740+0000","last_became_active":"2024-04-24T23:23:03.795614+0000","last_became_peered":"2024-04-24T23:23:03.795614+0000","last_unstale":"2024-04-24T23:23:04.801740+0000","last_undegraded":"2024-04-24T23:23:04.801740+0000","last_fullsized":"2024-04-24T23:23:04.801740+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:17.917671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.801581+0000","last_change":"2024-04-24T23:23:02.813703+0000","last_active":"2024-04-24T23:23:04.801581+0000","last_peered":"2024-04-24T23:23:04.801581+0000","last_clean":"2024-04-24T23:23:04.801581+0000","last_became_active":"2024-04-24T23:23:02.813400+0000","last_became_peered":"2024-04-24T23:23:02.813400+0000","last_unstale":"2024-04-24T23:23:04.801581+0000","last_undegraded":"2024-04-24T23:23:04.801581+0000","last_fullsized":"2024-04-24T23:23:04.801581+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:10.396279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876269+0000","last_change":"2024-04-24T23:23:03.813611+0000","last_active":"2024-04-24T23:23:04.876269+0000","last_peered":"2024-04-24T23:23:04.876269+0000","last_clean":"2024-04-24T23:23:04.876269+0000","last_became_active":"2024-04-24T23:23:03.813296+0000","last_became_peered":"2024-04-24T23:23:03.813296+0000","last_unstale":"2024-04-24T23:23:04.876269+0000","last_undegraded":"2024-04-24T23:23:04.876269+0000","last_fullsized":"2024-04-24T23:23:04.876269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:56.026414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807264+0000","last_change":"2024-04-24T23:23:02.797846+0000","last_active":"2024-04-24T23:23:04.807264+0000","last_peered":"2024-04-24T23:23:04.807264+0000","last_clean":"2024-04-24T23:23:04.807264+0000","last_became_active":"2024-04-24T23:23:02.796234+0000","last_became_peered":"2024-04-24T23:23:02.796234+0000","last_unstale":"2024-04-24T23:23:04.807264+0000","last_undegraded":"2024-04-24T23:23:04.807264+0000","last_fullsized":"2024-04-24T23:23:04.807264+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:21.024419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.803020+0000","last_change":"2024-04-24T23:23:03.820614+0000","last_active":"2024-04-24T23:23:04.803020+0000","last_peered":"2024-04-24T23:23:04.803020+0000","last_clean":"2024-04-24T23:23:04.803020+0000","last_became_active":"2024-04-24T23:23:03.820252+0000","last_became_peered":"2024-04-24T23:23:03.820252+0000","last_unstale":"2024-04-24T23:23:04.803020+0000","last_undegraded":"2024-04-24T23:23:04.803020+0000","last_fullsized":"2024-04-24T23:23:04.803020+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:55.917216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876713+0000","last_change":"2024-04-24T23:23:02.801493+0000","last_active":"2024-04-24T23:23:04.876713+0000","last_peered":"2024-04-24T23:23:04.876713+0000","last_clean":"2024-04-24T23:23:04.876713+0000","last_became_active":"2024-04-24T23:23:02.801208+0000","last_became_peered":"2024-04-24T23:23:02.801208+0000","last_unstale":"2024-04-24T23:23:04.876713+0000","last_undegraded":"2024-04-24T23:23:04.876713+0000","last_fullsized":"2024-04-24T23:23:04.876713+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:39.141185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.802958+0000","last_change":"2024-04-24T23:23:03.807410+0000","last_active":"2024-04-24T23:23:04.802958+0000","last_peered":"2024-04-24T23:23:04.802958+0000","last_clean":"2024-04-24T23:23:04.802958+0000","last_became_active":"2024-04-24T23:23:03.806841+0000","last_became_peered":"2024-04-24T23:23:03.806841+0000","last_unstale":"2024-04-24T23:23:04.802958+0000","last_undegraded":"2024-04-24T23:23:04.802958+0000","last_fullsized":"2024-04-24T23:23:04.802958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:25.474126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804520+0000","last_change":"2024-04-24T23:23:02.805096+0000","last_active":"2024-04-24T23:23:04.804520+0000","last_peered":"2024-04-24T23:23:04.804520+0000","last_clean":"2024-04-24T23:23:04.804520+0000","last_became_active":"2024-04-24T23:23:02.804805+0000","last_became_peered":"2024-04-24T23:23:02.804805+0000","last_unstale":"2024-04-24T23:23:04.804520+0000","last_undegraded":"2024-04-24T23:23:04.804520+0000","last_fullsized":"2024-04-24T23:23:04.804520+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:46.291724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804783+0000","last_change":"2024-04-24T23:23:03.812914+0000","last_active":"2024-04-24T23:23:04.804783+0000","last_peered":"2024-04-24T23:23:04.804783+0000","last_clean":"2024-04-24T23:23:04.804783+0000","last_became_active":"2024-04-24T23:23:03.812637+0000","last_became_peered":"2024-04-24T23:23:03.812637+0000","last_unstale":"2024-04-24T23:23:04.804783+0000","last_undegraded":"2024-04-24T23:23:04.804783+0000","last_fullsized":"2024-04-24T23:23:04.804783+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:18.673856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804870+0000","last_change":"2024-04-24T23:23:02.802985+0000","last_active":"2024-04-24T23:23:04.804870+0000","last_peered":"2024-04-24T23:23:04.804870+0000","last_clean":"2024-04-24T23:23:04.804870+0000","last_became_active":"2024-04-24T23:23:02.802308+0000","last_became_peered":"2024-04-24T23:23:02.802308+0000","last_unstale":"2024-04-24T23:23:04.804870+0000","last_undegraded":"2024-04-24T23:23:04.804870+0000","last_fullsized":"2024-04-24T23:23:04.804870+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:52.973682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876018+0000","last_change":"2024-04-24T23:23:03.805080+0000","last_active":"2024-04-24T23:23:04.876018+0000","last_peered":"2024-04-24T23:23:04.876018+0000","last_clean":"2024-04-24T23:23:04.876018+0000","last_became_active":"2024-04-24T23:23:03.804789+0000","last_became_peered":"2024-04-24T23:23:03.804789+0000","last_unstale":"2024-04-24T23:23:04.876018+0000","last_undegraded":"2024-04-24T23:23:04.876018+0000","last_fullsized":"2024-04-24T23:23:04.876018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:15:26.950107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876309+0000","last_change":"2024-04-24T23:23:02.803536+0000","last_active":"2024-04-24T23:23:04.876309+0000","last_peered":"2024-04-24T23:23:04.876309+0000","last_clean":"2024-04-24T23:23:04.876309+0000","last_became_active":"2024-04-24T23:23:02.803306+0000","last_became_peered":"2024-04-24T23:23:02.803306+0000","last_unstale":"2024-04-24T23:23:04.876309+0000","last_undegraded":"2024-04-24T23:23:04.876309+0000","last_fullsized":"2024-04-24T23:23:04.876309+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:07.892055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.801263+0000","last_change":"2024-04-24T23:23:03.800706+0000","last_active":"2024-04-24T23:23:04.801263+0000","last_peered":"2024-04-24T23:23:04.801263+0000","last_clean":"2024-04-24T23:23:04.801263+0000","last_became_active":"2024-04-24T23:23:03.800430+0000","last_became_peered":"2024-04-24T23:23:03.800430+0000","last_unstale":"2024-04-24T23:23:04.801263+0000","last_undegraded":"2024-04-24T23:23:04.801263+0000","last_fullsized":"2024-04-24T23:23:04.801263+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:53.917369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875967+0000","last_change":"2024-04-24T23:23:02.807190+0000","last_active":"2024-04-24T23:23:04.875967+0000","last_peered":"2024-04-24T23:23:04.875967+0000","last_clean":"2024-04-24T23:23:04.875967+0000","last_became_active":"2024-04-24T23:23:02.806851+0000","last_became_peered":"2024-04-24T23:23:02.806851+0000","last_unstale":"2024-04-24T23:23:04.875967+0000","last_undegraded":"2024-04-24T23:23:04.875967+0000","last_fullsized":"2024-04-24T23:23:04.875967+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:52.912978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875900+0000","last_change":"2024-04-24T23:23:03.812988+0000","last_active":"2024-04-24T23:23:04.875900+0000","last_peered":"2024-04-24T23:23:04.875900+0000","last_clean":"2024-04-24T23:23:04.875900+0000","last_became_active":"2024-04-24T23:23:03.810914+0000","last_became_peered":"2024-04-24T23:23:03.810914+0000","last_unstale":"2024-04-24T23:23:04.875900+0000","last_undegraded":"2024-04-24T23:23:04.875900+0000","last_fullsized":"2024-04-24T23:23:04.875900+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:42.047136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876013+0000","last_change":"2024-04-24T23:23:02.805831+0000","last_active":"2024-04-24T23:23:04.876013+0000","last_peered":"2024-04-24T23:23:04.876013+0000","last_clean":"2024-04-24T23:23:04.876013+0000","last_became_active":"2024-04-24T23:23:02.805514+0000","last_became_peered":"2024-04-24T23:23:02.805514+0000","last_unstale":"2024-04-24T23:23:04.876013+0000","last_undegraded":"2024-04-24T23:23:04.876013+0000","last_fullsized":"2024-04-24T23:23:04.876013+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:24.181197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.803289+0000","last_change":"2024-04-24T23:23:03.813892+0000","last_active":"2024-04-24T23:23:04.803289+0000","last_peered":"2024-04-24T23:23:04.803289+0000","last_clean":"2024-04-24T23:23:04.803289+0000","last_became_active":"2024-04-24T23:23:03.813571+0000","last_became_peered":"2024-04-24T23:23:03.813571+0000","last_unstale":"2024-04-24T23:23:04.803289+0000","last_undegraded":"2024-04-24T23:23:04.803289+0000","last_fullsized":"2024-04-24T23:23:04.803289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:32.559397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804714+0000","last_change":"2024-04-24T23:23:02.804378+0000","last_active":"2024-04-24T23:23:04.804714+0000","last_peered":"2024-04-24T23:23:04.804714+0000","last_clean":"2024-04-24T23:23:04.804714+0000","last_became_active":"2024-04-24T23:23:02.804026+0000","last_became_peered":"2024-04-24T23:23:02.804026+0000","last_unstale":"2024-04-24T23:23:04.804714+0000","last_undegraded":"2024-04-24T23:23:04.804714+0000","last_fullsized":"2024-04-24T23:23:04.804714+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:10.157227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875866+0000","last_change":"2024-04-24T23:23:03.820155+0000","last_active":"2024-04-24T23:23:04.875866+0000","last_peered":"2024-04-24T23:23:04.875866+0000","last_clean":"2024-04-24T23:23:04.875866+0000","last_became_active":"2024-04-24T23:23:03.819685+0000","last_became_peered":"2024-04-24T23:23:03.819685+0000","last_unstale":"2024-04-24T23:23:04.875866+0000","last_undegraded":"2024-04-24T23:23:04.875866+0000","last_fullsized":"2024-04-24T23:23:04.875866+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:25.185587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876831+0000","last_change":"2024-04-24T23:23:02.809392+0000","last_active":"2024-04-24T23:23:04.876831+0000","last_peered":"2024-04-24T23:23:04.876831+0000","last_clean":"2024-04-24T23:23:04.876831+0000","last_became_active":"2024-04-24T23:23:02.808974+0000","last_became_peered":"2024-04-24T23:23:02.808974+0000","last_unstale":"2024-04-24T23:23:04.876831+0000","last_undegraded":"2024-04-24T23:23:04.876831+0000","last_fullsized":"2024-04-24T23:23:04.876831+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:18.252420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876622+0000","last_change":"2024-04-24T23:23:03.808135+0000","last_active":"2024-04-24T23:23:04.876622+0000","last_peered":"2024-04-24T23:23:04.876622+0000","last_clean":"2024-04-24T23:23:04.876622+0000","last_became_active":"2024-04-24T23:23:03.807801+0000","last_became_peered":"2024-04-24T23:23:03.807801+0000","last_unstale":"2024-04-24T23:23:04.876622+0000","last_undegraded":"2024-04-24T23:23:04.876622+0000","last_fullsized":"2024-04-24T23:23:04.876622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:28.782107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876089+0000","last_change":"2024-04-24T23:23:02.803614+0000","last_active":"2024-04-24T23:23:04.876089+0000","last_peered":"2024-04-24T23:23:04.876089+0000","last_clean":"2024-04-24T23:23:04.876089+0000","last_became_active":"2024-04-24T23:23:02.803340+0000","last_became_peered":"2024-04-24T23:23:02.803340+0000","last_unstale":"2024-04-24T23:23:04.876089+0000","last_undegraded":"2024-04-24T23:23:04.876089+0000","last_fullsized":"2024-04-24T23:23:04.876089+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:56.215292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804883+0000","last_change":"2024-04-24T23:23:03.811028+0000","last_active":"2024-04-24T23:23:04.804883+0000","last_peered":"2024-04-24T23:23:04.804883+0000","last_clean":"2024-04-24T23:23:04.804883+0000","last_became_active":"2024-04-24T23:23:03.809394+0000","last_became_peered":"2024-04-24T23:23:03.809394+0000","last_unstale":"2024-04-24T23:23:04.804883+0000","last_undegraded":"2024-04-24T23:23:04.804883+0000","last_fullsized":"2024-04-24T23:23:04.804883+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:30.366162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876589+0000","last_change":"2024-04-24T23:23:02.809445+0000","last_active":"2024-04-24T23:23:04.876589+0000","last_peered":"2024-04-24T23:23:04.876589+0000","last_clean":"2024-04-24T23:23:04.876589+0000","last_became_active":"2024-04-24T23:23:02.809106+0000","last_became_peered":"2024-04-24T23:23:02.809106+0000","last_unstale":"2024-04-24T23:23:04.876589+0000","last_undegraded":"2024-04-24T23:23:04.876589+0000","last_fullsized":"2024-04-24T23:23:04.876589+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:16.510257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.804543+0000","last_change":"2024-04-24T23:23:03.812157+0000","last_active":"2024-04-24T23:23:04.804543+0000","last_peered":"2024-04-24T23:23:04.804543+0000","last_clean":"2024-04-24T23:23:04.804543+0000","last_became_active":"2024-04-24T23:23:03.811886+0000","last_became_peered":"2024-04-24T23:23:03.811886+0000","last_unstale":"2024-04-24T23:23:04.804543+0000","last_undegraded":"2024-04-24T23:23:04.804543+0000","last_fullsized":"2024-04-24T23:23:04.804543+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:39.855527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.807118+0000","last_change":"2024-04-24T23:23:02.799553+0000","last_active":"2024-04-24T23:23:04.807118+0000","last_peered":"2024-04-24T23:23:04.807118+0000","last_clean":"2024-04-24T23:23:04.807118+0000","last_became_active":"2024-04-24T23:23:02.799266+0000","last_became_peered":"2024-04-24T23:23:02.799266+0000","last_unstale":"2024-04-24T23:23:04.807118+0000","last_undegraded":"2024-04-24T23:23:04.807118+0000","last_fullsized":"2024-04-24T23:23:04.807118+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:04.702618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876069+0000","last_change":"2024-04-24T23:23:03.805123+0000","last_active":"2024-04-24T23:23:04.876069+0000","last_peered":"2024-04-24T23:23:04.876069+0000","last_clean":"2024-04-24T23:23:04.876069+0000","last_became_active":"2024-04-24T23:23:03.804811+0000","last_became_peered":"2024-04-24T23:23:03.804811+0000","last_unstale":"2024-04-24T23:23:04.876069+0000","last_undegraded":"2024-04-24T23:23:04.876069+0000","last_fullsized":"2024-04-24T23:23:04.876069+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:47.296115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876179+0000","last_change":"2024-04-24T23:23:02.805495+0000","last_active":"2024-04-24T23:23:04.876179+0000","last_peered":"2024-04-24T23:23:04.876179+0000","last_clean":"2024-04-24T23:23:04.876179+0000","last_became_active":"2024-04-24T23:23:02.805145+0000","last_became_peered":"2024-04-24T23:23:02.805145+0000","last_unstale":"2024-04-24T23:23:04.876179+0000","last_undegraded":"2024-04-24T23:23:04.876179+0000","last_fullsized":"2024-04-24T23:23:04.876179+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:52.708996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.806916+0000","last_change":"2024-04-24T23:23:03.810239+0000","last_active":"2024-04-24T23:23:04.806916+0000","last_peered":"2024-04-24T23:23:04.806916+0000","last_clean":"2024-04-24T23:23:04.806916+0000","last_became_active":"2024-04-24T23:23:03.809967+0000","last_became_peered":"2024-04-24T23:23:03.809967+0000","last_unstale":"2024-04-24T23:23:04.806916+0000","last_undegraded":"2024-04-24T23:23:04.806916+0000","last_fullsized":"2024-04-24T23:23:04.806916+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:08.896446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.875604+0000","last_change":"2024-04-24T23:23:02.813813+0000","last_active":"2024-04-24T23:23:04.875604+0000","last_peered":"2024-04-24T23:23:04.875604+0000","last_clean":"2024-04-24T23:23:04.875604+0000","last_became_active":"2024-04-24T23:23:02.813494+0000","last_became_peered":"2024-04-24T23:23:02.813494+0000","last_unstale":"2024-04-24T23:23:04.875604+0000","last_undegraded":"2024-04-24T23:23:04.875604+0000","last_fullsized":"2024-04-24T23:23:04.875604+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:51.560929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876050+0000","last_change":"2024-04-24T23:23:03.809208+0000","last_active":"2024-04-24T23:23:04.876050+0000","last_peered":"2024-04-24T23:23:04.876050+0000","last_clean":"2024-04-24T23:23:04.876050+0000","last_became_active":"2024-04-24T23:23:03.808865+0000","last_became_peered":"2024-04-24T23:23:03.808865+0000","last_unstale":"2024-04-24T23:23:04.876050+0000","last_undegraded":"2024-04-24T23:23:04.876050+0000","last_fullsized":"2024-04-24T23:23:04.876050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:02.782468+0000","last_clean_scrub_stamp":"2024-04-24T23:23:02.782468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:47.296115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876723+0000","last_change":"2024-04-24T23:23:02.813467+0000","last_active":"2024-04-24T23:23:04.876723+0000","last_peered":"2024-04-24T23:23:04.876723+0000","last_clean":"2024-04-24T23:23:04.876723+0000","last_became_active":"2024-04-24T23:23:02.813260+0000","last_became_peered":"2024-04-24T23:23:02.813260+0000","last_unstale":"2024-04-24T23:23:04.876723+0000","last_undegraded":"2024-04-24T23:23:04.876723+0000","last_fullsized":"2024-04-24T23:23:04.876723+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:23:01.778077+0000","last_clean_scrub_stamp":"2024-04-24T23:23:01.778077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:26.648989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":213,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-24T23:23:04.876143+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:23:04.876143+0000","last_peered":"2024-04-24T23:23:04.876143+0000","last_clean":"2024-04-24T23:23:04.876143+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:23:04.876143+0000","last_undegraded":"2024-04-24T23:23:04.876143+0000","last_fullsized":"2024-04-24T23:23:04.876143+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":801258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":524288,"data_stored":211472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":210870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":212412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":211438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35860,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":532480,"data_stored":212434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":210870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":801202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:23:12.581 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:23:12.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:23:12.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.724+0000 7f7cf3161640 1 Processor -- start 2024-04-24T23:23:12.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.724+0000 7f7cf3161640 1 -- start start 2024-04-24T23:23:12.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cf3161640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 0x7f7cec0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cf3161640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cec0b4ce0 0x7f7cec0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cf3161640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7cec0d0580 0x7f7cec0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cf3161640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7cec09df20 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cf3161640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7cec0d09e0 con 0x7f7cec0d0580 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cf3161640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7cec0aeb10 con 0x7f7cec15be40 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cebfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cec0b4ce0 0x7f7cec0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cebfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cec0b4ce0 0x7f7cec0d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45704/0 (socket says 172.21.15.26:45704) 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cebfff640 1 -- 172.21.15.26:0/3525680731 learned_addr learned my addr 172.21.15.26:0/3525680731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cf0efd640 1 --2- 172.21.15.26:0/3525680731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 0x7f7cec0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cf16fe640 1 --2- 172.21.15.26:0/3525680731 >> v2:172.21.15.29:3300/0 conn(0x7f7cec0d0580 0x7f7cec0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cebfff640 1 -- 172.21.15.26:0/3525680731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 msgr2=0x7f7cec0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cebfff640 1 --2- 172.21.15.26:0/3525680731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 0x7f7cec0acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cebfff640 1 -- 172.21.15.26:0/3525680731 >> v2:172.21.15.29:3300/0 conn(0x7f7cec0d0580 msgr2=0x7f7cec0b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cebfff640 1 --2- 172.21.15.26:0/3525680731 >> v2:172.21.15.29:3300/0 conn(0x7f7cec0d0580 0x7f7cec0b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.725+0000 7f7cebfff640 1 -- 172.21.15.26:0/3525680731 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec0aedf0 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.726+0000 7f7cebfff640 1 --2- 172.21.15.26:0/3525680731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 0x7f7cec0d0040 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7cec06e8b0 tx=0x7f7cd802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f42f618ad6eeb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:12.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.726+0000 7f7ceb7fe640 1 -- 172.21.15.26:0/3525680731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8037070 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.726+0000 7f7ceb7fe640 1 -- 172.21.15.26:0/3525680731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd8032440 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.726+0000 7f7ceb7fe640 1 -- 172.21.15.26:0/3525680731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8032810 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.727+0000 7f7cf3161640 1 -- 172.21.15.26:0/3525680731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 msgr2=0x7f7cec0d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.727+0000 7f7cf3161640 1 --2- 172.21.15.26:0/3525680731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 0x7f7cec0d0040 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7cec06e8b0 tx=0x7f7cd802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.727+0000 7f7cf3161640 1 -- 172.21.15.26:0/3525680731 shutdown_connections 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.727+0000 7f7cf3161640 1 --2- 172.21.15.26:0/3525680731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 0x7f7cec0d0040 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.727+0000 7f7cf3161640 1 --2- 172.21.15.26:0/3525680731 >> v2:172.21.15.29:3300/0 conn(0x7f7cec0d0580 0x7f7cec0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.727+0000 7f7cf3161640 1 --2- 172.21.15.26:0/3525680731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 0x7f7cec0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.727+0000 7f7cf3161640 1 -- 172.21.15.26:0/3525680731 >> 172.21.15.26:0/3525680731 conn(0x7f7cec06b540 msgr2=0x7f7cec06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.727+0000 7f7cf3161640 1 -- 172.21.15.26:0/3525680731 shutdown_connections 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.727+0000 7f7cf3161640 1 -- 172.21.15.26:0/3525680731 wait complete. 2024-04-24T23:23:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.728+0000 7f7cf3161640 1 Processor -- start 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.728+0000 7f7cf3161640 1 -- start start 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf3161640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 0x7f7cec1f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf3161640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cec0d0580 0x7f7cec0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf3161640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 0x7f7cec0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf3161640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec06cb90 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf3161640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec0ae800 con 0x7f7cec0d0580 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf3161640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7cec06cd10 con 0x7f7cec15be40 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf0efd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 0x7f7cec1f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf0efd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 0x7f7cec1f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45710/0 (socket says 172.21.15.26:45710) 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf0efd640 1 -- 172.21.15.26:0/842708518 learned_addr learned my addr 172.21.15.26:0/842708518 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cebfff640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cec0d0580 0x7f7cec0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf16fe640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 0x7f7cec0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf0efd640 1 -- 172.21.15.26:0/842708518 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 msgr2=0x7f7cec0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf0efd640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 0x7f7cec0c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf0efd640 1 -- 172.21.15.26:0/842708518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cec0d0580 msgr2=0x7f7cec0c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf0efd640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cec0d0580 0x7f7cec0c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf0efd640 1 -- 172.21.15.26:0/842708518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec0c57a0 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.729+0000 7f7cf0efd640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 0x7f7cec1f4b30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000e9b0 tx=0x7f7ce000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.730+0000 7f7ce97fa640 1 -- 172.21.15.26:0/842708518 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0017070 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.730+0000 7f7ce97fa640 1 -- 172.21.15.26:0/842708518 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce000c9a0 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.730+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec0c5a80 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.730+0000 7f7ce97fa640 1 -- 172.21.15.26:0/842708518 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000cd90 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.730+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec0c6040 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.731+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec06db70 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.731+0000 7f7ce97fa640 1 -- 172.21.15.26:0/842708518 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ce001a070 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.731+0000 7f7ce97fa640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cd006be30 0x7f7cd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.732+0000 7f7ce97fa640 1 -- 172.21.15.26:0/842708518 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f7ce008b2d0 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.732+0000 7f7cebfff640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cd006be30 0x7f7cd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:12.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.732+0000 7f7cebfff640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cd006be30 0x7f7cd006e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8034000 tx=0x7f7cd802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:12.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.736+0000 7f7ce97fa640 1 -- 172.21.15.26:0/842708518 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7cec06db70 con 0x7f7cec0b4ce0 2024-04-24T23:23:12.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.997+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7cec075610 con 0x7f7cec0b4ce0 2024-04-24T23:23:13.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:12.999+0000 7f7ce97fa640 1 -- 172.21.15.26:0/842708518 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f7ce0059c00 con 0x7f7cec0b4ce0 2024-04-24T23:23:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-24T23:23:13.001 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ== 2024-04-24T23:23:13.001 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-04-24T23:23:13.001 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-04-24T23:23:13.001 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:23:13.001 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-04-24T23:23:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cd006be30 msgr2=0x7f7cd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cd006be30 0x7f7cd006e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8034000 tx=0x7f7cd802e000 comp rx=0 tx=0).stop 2024-04-24T23:23:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 msgr2=0x7f7cec1f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 0x7f7cec1f4b30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000e9b0 tx=0x7f7ce000ee70 comp rx=0 tx=0).stop 2024-04-24T23:23:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 shutdown_connections 2024-04-24T23:23:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7cd006be30 0x7f7cd006e2f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7cec15be40 0x7f7cec0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7cec0d0580 0x7f7cec0c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 --2- 172.21.15.26:0/842708518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cec0b4ce0 0x7f7cec1f4b30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.001+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 >> 172.21.15.26:0/842708518 conn(0x7f7cec06b540 msgr2=0x7f7cec06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.002+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 shutdown_connections 2024-04-24T23:23:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.002+0000 7f7cf3161640 1 -- 172.21.15.26:0/842708518 wait complete. 2024-04-24T23:23:13.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.114+0000 7fab85864640 1 Processor -- start 2024-04-24T23:23:13.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.114+0000 7fab85864640 1 -- start start 2024-04-24T23:23:13.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab85864640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab8015e570 0x7fab800d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab85864640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800b3130 0x7fab800d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab85864640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab800d22e0 0x7fab800d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab85864640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fab800d1ab0 con 0x7fab8015e570 2024-04-24T23:23:13.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab85864640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fab800d2ae0 con 0x7fab800d22e0 2024-04-24T23:23:13.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab85864640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fab800d32f0 con 0x7fab800b3130 2024-04-24T23:23:13.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab7effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab8015e570 0x7fab800d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab7effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab8015e570 0x7fab800d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45720/0 (socket says 172.21.15.26:45720) 2024-04-24T23:23:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab7effd640 1 -- 172.21.15.26:0/696957457 learned_addr learned my addr 172.21.15.26:0/696957457 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab7e7fc640 1 --2- 172.21.15.26:0/696957457 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800b3130 0x7fab800d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.115+0000 7fab7f7fe640 1 --2- 172.21.15.26:0/696957457 >> v2:172.21.15.29:3300/0 conn(0x7fab800d22e0 0x7fab800d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.116+0000 7fab7effd640 1 -- 172.21.15.26:0/696957457 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800b3130 msgr2=0x7fab800d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.116+0000 7fab7effd640 1 --2- 172.21.15.26:0/696957457 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800b3130 0x7fab800d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.116+0000 7fab7effd640 1 -- 172.21.15.26:0/696957457 >> v2:172.21.15.29:3300/0 conn(0x7fab800d22e0 msgr2=0x7fab800d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.116+0000 7fab7effd640 1 --2- 172.21.15.26:0/696957457 >> v2:172.21.15.29:3300/0 conn(0x7fab800d22e0 0x7fab800d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.116+0000 7fab7effd640 1 -- 172.21.15.26:0/696957457 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab800d0270 con 0x7fab8015e570 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.116+0000 7fab7effd640 1 --2- 172.21.15.26:0/696957457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 0x7fab800d4160 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fab800ba280 tx=0x7fab7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25d5f9b36edae22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab7dffb640 1 -- 172.21.15.26:0/696957457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78036070 con 0x7fab8015e570 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab7dffb640 1 -- 172.21.15.26:0/696957457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab78031600 con 0x7fab8015e570 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab7dffb640 1 -- 172.21.15.26:0/696957457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab780319d0 con 0x7fab8015e570 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab85864640 1 -- 172.21.15.26:0/696957457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 msgr2=0x7fab800d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab85864640 1 --2- 172.21.15.26:0/696957457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 0x7fab800d4160 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fab800ba280 tx=0x7fab7802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab85864640 1 -- 172.21.15.26:0/696957457 shutdown_connections 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab85864640 1 --2- 172.21.15.26:0/696957457 >> v2:172.21.15.29:3300/0 conn(0x7fab800d22e0 0x7fab800d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab85864640 1 --2- 172.21.15.26:0/696957457 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800b3130 0x7fab800d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab85864640 1 --2- 172.21.15.26:0/696957457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 0x7fab800d4160 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab85864640 1 -- 172.21.15.26:0/696957457 >> 172.21.15.26:0/696957457 conn(0x7fab8008dbe0 msgr2=0x7fab8009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab85864640 1 -- 172.21.15.26:0/696957457 shutdown_connections 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.117+0000 7fab85864640 1 -- 172.21.15.26:0/696957457 wait complete. 2024-04-24T23:23:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.118+0000 7fab85864640 1 Processor -- start 2024-04-24T23:23:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab85864640 1 -- start start 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab85864640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab800b3130 0x7fab8015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab85864640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800d22e0 0x7fab80158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab85864640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 0x7fab800cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab85864640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab800cfa70 con 0x7fab8015e570 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab85864640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fab800d3010 con 0x7fab800b3130 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab85864640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fab800cfd50 con 0x7fab800d22e0 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 0x7fab800cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 0x7fab800cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45724/0 (socket says 172.21.15.26:45724) 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab7effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab800b3130 0x7fab8015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab7f7fe640 1 -- 172.21.15.26:0/2307877929 learned_addr learned my addr 172.21.15.26:0/2307877929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.119+0000 7fab7e7fc640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800d22e0 0x7fab80158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab7f7fe640 1 -- 172.21.15.26:0/2307877929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800d22e0 msgr2=0x7fab80158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab7f7fe640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800d22e0 0x7fab80158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab7f7fe640 1 -- 172.21.15.26:0/2307877929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab800b3130 msgr2=0x7fab8015aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab7f7fe640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab800b3130 0x7fab8015aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab7f7fe640 1 -- 172.21.15.26:0/2307877929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab801599b0 con 0x7fab8015e570 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab7f7fe640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 0x7fab800cd040 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fab7000e960 tx=0x7fab7000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab67fff640 1 -- 172.21.15.26:0/2307877929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70017070 con 0x7fab8015e570 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab67fff640 1 -- 172.21.15.26:0/2307877929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7000c980 con 0x7fab8015e570 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab80159c90 con 0x7fab8015e570 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab67fff640 1 -- 172.21.15.26:0/2307877929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7000cd70 con 0x7fab8015e570 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.120+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8015a250 con 0x7fab8015e570 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.121+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab800694c0 con 0x7fab8015e570 2024-04-24T23:23:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.122+0000 7fab67fff640 1 -- 172.21.15.26:0/2307877929 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab7001a070 con 0x7fab8015e570 2024-04-24T23:23:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.122+0000 7fab67fff640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab6c06be30 0x7fab6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.122+0000 7fab67fff640 1 -- 172.21.15.26:0/2307877929 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fab7008b140 con 0x7fab8015e570 2024-04-24T23:23:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.122+0000 7fab7effd640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab6c06be30 0x7fab6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.123+0000 7fab7effd640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab6c06be30 0x7fab6c06e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fab78033000 tx=0x7fab78009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.127+0000 7fab67fff640 1 -- 172.21.15.26:0/2307877929 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fab700594a0 con 0x7fab8015e570 2024-04-24T23:23:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.353+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab800b4e30 con 0x7fab8015e570 2024-04-24T23:23:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.353+0000 7fab67fff640 1 -- 172.21.15.26:0/2307877929 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5175 (secure 0 0 0) 0x7fab7005d7a0 con 0x7fab8015e570 2024-04-24T23:23:13.355 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:13.355 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5329,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:06.169758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-24T23:23:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2024-04-24T23:23:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab6c06be30 msgr2=0x7fab6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab6c06be30 0x7fab6c06e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fab78033000 tx=0x7fab78009000 comp rx=0 tx=0).stop 2024-04-24T23:23:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 msgr2=0x7fab800cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 0x7fab800cd040 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fab7000e960 tx=0x7fab7000ee20 comp rx=0 tx=0).stop 2024-04-24T23:23:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 shutdown_connections 2024-04-24T23:23:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab6c06be30 0x7fab6c06e2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab8015e570 0x7fab800cd040 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab800d22e0 0x7fab80158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 --2- 172.21.15.26:0/2307877929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab800b3130 0x7fab8015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 >> 172.21.15.26:0/2307877929 conn(0x7fab8008dbe0 msgr2=0x7fab8006c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 shutdown_connections 2024-04-24T23:23:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.356+0000 7fab85864640 1 -- 172.21.15.26:0/2307877929 wait complete. 2024-04-24T23:23:13.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:23:03.164134+0000', 'modified': '2024-04-24T23:23:06.169758+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5742}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5742': {'gid': 5742, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6836/730964888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 730964888}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 730964888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5742, 'qdb_cluster': [5742]} 2024-04-24T23:23:13.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:23:13.771 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:23:13.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.875+0000 7f41d0c31640 1 Processor -- start 2024-04-24T23:23:13.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.875+0000 7f41d0c31640 1 -- start start 2024-04-24T23:23:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.876+0000 7f41d0c31640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f41cc0b4530 0x7f41cc0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.876+0000 7f41d0c31640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41cc0b5700 con 0x7f41cc0b0030 2024-04-24T23:23:13.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.876+0000 7f41d0c31640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41cc0b59e0 con 0x7f41cc15e380 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.876+0000 7f41d0c31640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f41cc0b49a0 con 0x7f41cc0b4530 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.876+0000 7f41cad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41cc0b0030 0x7f41cc0b0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50840/0 (socket says 172.21.15.26:50840) 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.876+0000 7f41cad9d640 1 -- 172.21.15.26:0/1833428508 learned_addr learned my addr 172.21.15.26:0/1833428508 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.876+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/1833428508 >> v2:172.21.15.29:3300/0 conn(0x7f41cc0b4530 0x7f41cc0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.876+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2153203868 0 0) 0x7f41cc0b5700 con 0x7f41cc0b0030 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.876+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41b4003520 con 0x7f41cc0b0030 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2834056010 0 0) 0x7f41b4003520 con 0x7f41cc0b0030 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 >> v2:172.21.15.29:3300/0 conn(0x7f41cc0b4530 msgr2=0x7f41cc0b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 --2- 172.21.15.26:0/1833428508 >> v2:172.21.15.29:3300/0 conn(0x7f41cc0b4530 0x7f41cc0b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 >> v1:172.21.15.29:6789/0 conn(0x7f41cc15e380 legacy=0x7f41cc072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc072240 con 0x7f41cc0b0030 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f41bc0019f0 con 0x7f41cc0b0030 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 >> v1:172.21.15.26:6789/0 conn(0x7f41cc0b0030 legacy=0x7f41cc0b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41b40039b0 0x7f41b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41b4004800 0x7f41b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41b40051c0 0x7f41b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41b400c040 con 0x7f41b4004800 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f41b400c1f0 con 0x7f41b40051c0 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f41b400c3f0 con 0x7f41b40039b0 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41b4004800 0x7f41b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41ca59c640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41b40039b0 0x7f41b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41cad9d640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41b40051c0 0x7f41b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c9d9b640 1 -- 172.21.15.26:0/1833428508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41b40039b0 msgr2=0x7f41b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41b40039b0 0x7f41b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c9d9b640 1 -- 172.21.15.26:0/1833428508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41b40051c0 msgr2=0x7f41b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41b40051c0 0x7f41b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41c9d9b640 1 -- 172.21.15.26:0/1833428508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b4003520 con 0x7f41b4004800 2024-04-24T23:23:13.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41cad9d640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41b40051c0 0x7f41b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.877+0000 7f41cad9d640 1 -- 172.21.15.26:0/1833428508 reap_dead start 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.878+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41b4004800 0x7f41b4004c80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009ac0 tx=0x7f41b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e20be1776be6071 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.878+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0036070 con 0x7f41b4004800 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.878+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0031400 con 0x7f41b4004800 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.878+0000 7f41c959a640 1 -- 172.21.15.26:0/1833428508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0009690 con 0x7f41b4004800 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.879+0000 7f41d0c31640 1 -- 172.21.15.26:0/1833428508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41b4004800 msgr2=0x7f41b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.879+0000 7f41d0c31640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41b4004800 0x7f41b4004c80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009ac0 tx=0x7f41b4005830 comp rx=0 tx=0).stop 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.879+0000 7f41d0c31640 1 -- 172.21.15.26:0/1833428508 shutdown_connections 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.879+0000 7f41d0c31640 1 --2- 172.21.15.26:0/1833428508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41b4004800 0x7f41b4004c80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.879+0000 7f41d0c31640 1 -- 172.21.15.26:0/1833428508 >> 172.21.15.26:0/1833428508 conn(0x7f41cc06e2c0 msgr2=0x7f41cc06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.879+0000 7f41d0c31640 1 -- 172.21.15.26:0/1833428508 shutdown_connections 2024-04-24T23:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.879+0000 7f41d0c31640 1 -- 172.21.15.26:0/1833428508 wait complete. 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.880+0000 7f41d0c31640 1 Processor -- start 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.880+0000 7f41d0c31640 1 -- start start 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41d0c31640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41b4004800 0x7f41cc14a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41d0c31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc14dc10 0x7f41cc1f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41d0c31640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41cc1480c0 0x7f41cc148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41d0c31640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc0b5360 con 0x7f41cc14dc10 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41d0c31640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc0b51e0 con 0x7f41b4004800 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41d0c31640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f41cc0b54e0 con 0x7f41cc1480c0 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc14dc10 0x7f41cc1f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc14dc10 0x7f41cc1f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60334/0 (socket says 172.21.15.26:60334) 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41c9d9b640 1 -- 172.21.15.26:0/519869483 learned_addr learned my addr 172.21.15.26:0/519869483 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41c9d9b640 1 -- 172.21.15.26:0/519869483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41cc1480c0 msgr2=0x7f41cc148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41cc1480c0 0x7f41cc148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41c9d9b640 1 -- 172.21.15.26:0/519869483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41b4004800 msgr2=0x7f41cc14a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41b4004800 0x7f41cc14a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41c9d9b640 1 -- 172.21.15.26:0/519869483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc14a830 con 0x7f41cc14dc10 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.881+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc14dc10 0x7f41cc1f3220 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f41c002abb0 tx=0x7f41c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.882+0000 7f41baffd640 1 -- 172.21.15.26:0/519869483 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c003f020 con 0x7f41cc14dc10 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.882+0000 7f41baffd640 1 -- 172.21.15.26:0/519869483 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0037040 con 0x7f41cc14dc10 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.882+0000 7f41baffd640 1 -- 172.21.15.26:0/519869483 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0036070 con 0x7f41cc14dc10 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.882+0000 7f41d0c31640 1 -- 172.21.15.26:0/519869483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc14ab10 con 0x7f41cc14dc10 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.882+0000 7f41d0c31640 1 -- 172.21.15.26:0/519869483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc0cea90 con 0x7f41cc14dc10 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.883+0000 7f41baffd640 1 -- 172.21.15.26:0/519869483 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41c0042020 con 0x7f41cc14dc10 2024-04-24T23:23:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.884+0000 7f41baffd640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41ac06be60 0x7f41ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.884+0000 7f41ca59c640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41ac06be60 0x7f41ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.884+0000 7f41baffd640 1 -- 172.21.15.26:0/519869483 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f41c00aae10 con 0x7f41cc14dc10 2024-04-24T23:23:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.884+0000 7f41ca59c640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41ac06be60 0x7f41ac06e320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000c00 tx=0x7f41c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.885+0000 7f41d3a1a740 1 Processor -- start 2024-04-24T23:23:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.885+0000 7f41d3a1a740 1 -- start start 2024-04-24T23:23:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0e3b60 0x556daf0964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0c6540 0x556daf09b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556daeffef00 con 0x556daf0c6c80 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556daefa7750 con 0x556daf0e3b60 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556daeffaec0 con 0x556daf0c6540 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60350/0 (socket says 172.21.15.26:60350) 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41c9d9b640 1 -- 172.21.15.26:0/1692160891 learned_addr learned my addr 172.21.15.26:0/1692160891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41cad9d640 1 --2- 172.21.15.26:0/1692160891 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0c6540 0x556daf09b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41ca59c640 1 --2- 172.21.15.26:0/1692160891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0e3b60 0x556daf0964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41c9d9b640 1 -- 172.21.15.26:0/1692160891 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0c6540 msgr2=0x556daf09b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/1692160891 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0c6540 0x556daf09b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41c9d9b640 1 -- 172.21.15.26:0/1692160891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0e3b60 msgr2=0x556daf0964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/1692160891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0e3b60 0x556daf0964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41c9d9b640 1 -- 172.21.15.26:0/1692160891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556daed132d0 con 0x556daf0c6c80 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/1692160891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0c6000 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f41c00a9310 tx=0x7f41c0079350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89dc3c8f8ce33d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41abfff640 1 -- 172.21.15.26:0/1692160891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0077510 con 0x556daf0c6c80 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41abfff640 1 -- 172.21.15.26:0/1692160891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c00a8070 con 0x556daf0c6c80 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.886+0000 7f41abfff640 1 -- 172.21.15.26:0/1692160891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c007d170 con 0x556daf0c6c80 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 -- 172.21.15.26:0/1692160891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 msgr2=0x556daf0c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/1692160891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0c6000 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f41c00a9310 tx=0x7f41c0079350 comp rx=0 tx=0).stop 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 -- 172.21.15.26:0/1692160891 shutdown_connections 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/1692160891 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0c6540 0x556daf09b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/1692160891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0c6000 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/1692160891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0e3b60 0x556daf0964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 -- 172.21.15.26:0/1692160891 >> 172.21.15.26:0/1692160891 conn(0x556daf098760 msgr2=0x556daf0e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 -- 172.21.15.26:0/1692160891 shutdown_connections 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 -- 172.21.15.26:0/1692160891 wait complete. 2024-04-24T23:23:13.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.887+0000 7f41d3a1a740 1 -- start start 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.888+0000 7f41d3a1a740 1 Processor -- start 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.888+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0c6c80 0x556daf0e2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.888+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0e3b60 0x556daf0e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.888+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0afe40 0x556daf0c5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.888+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556daeffaec0 con 0x556daf0e3b60 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.888+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556daefbacf0 con 0x556daf0c6c80 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.888+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556daeffef00 con 0x556daf0afe40 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41d3a1a740 10 client.0 _ll_register_callbacks cb 0x556daefba6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0e3b60 0x556daf0e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0e3b60 0x556daf0e2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60352/0 (socket says 172.21.15.26:60352) 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41c9d9b640 1 -- 172.21.15.26:0/3637485725 learned_addr learned my addr 172.21.15.26:0/3637485725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41cad9d640 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0afe40 0x556daf0c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41ca59c640 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0c6c80 0x556daf0e2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41c9d9b640 1 -- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0afe40 msgr2=0x556daf0c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0afe40 0x556daf0c5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41c9d9b640 1 -- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0c6c80 msgr2=0x556daf0e2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0c6c80 0x556daf0e2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41c9d9b640 1 -- 172.21.15.26:0/3637485725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556daeddd6b0 con 0x556daf0e3b60 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41cad9d640 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0afe40 0x556daf0c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41ca59c640 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0c6c80 0x556daf0e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0e3b60 0x556daf0e2670 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f41c007d780 tx=0x7f41c0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b9dd14a3df339d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41a97fa640 1 -- 172.21.15.26:0/3637485725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c009d060 con 0x556daf0e3b60 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41a97fa640 1 -- 172.21.15.26:0/3637485725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f41c0081960 con 0x556daf0e3b60 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41d3a1a740 1 -- 172.21.15.26:0/3637485725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c00a8070 con 0x556daf0e3b60 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41a97fa640 1 -- 172.21.15.26:0/3637485725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f41c00981e0 con 0x556daf0e3b60 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41d3a1a740 10 client.5913 fetch_fsmap learned FSMap version 33 2024-04-24T23:23:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.889+0000 7f41d3a1a740 1 -- 172.21.15.26:0/3637485725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x556daf0e3400 con 0x556daf0e3b60 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41a97fa640 1 -- 172.21.15.26:0/3637485725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c00a8070 con 0x556daf0e3b60 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41a97fa640 1 -- 172.21.15.26:0/3637485725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f41c0078660 con 0x556daf0e3b60 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41d3a1a740 10 client.5913 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41d3a1a740 10 client.5913 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41d3a1a740 20 client.5913 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556daf0b0bb0 0x556daf0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41d3a1a740 4 client.5913 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41d3a1a740 1 -- 172.21.15.26:0/3637485725 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556daf00c440 con 0x556daf0b0bb0 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41ca59c640 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556daf0b0bb0 0x556daf0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41a97fa640 10 client.5913 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.890+0000 7f41ca59c640 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556daf0b0bb0 0x556daf0b0f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=632adb1c1884580f server_cookie=c564a64a90338c00 in_seq=0 out_seq=0 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41a97fa640 1 -- 172.21.15.26:0/3637485725 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556daf00c440 con 0x556daf0b0bb0 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41a97fa640 10 client.5913 handle_command_reply: tid=0 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 client.5913 shutdown 2024-04-24T23:23:13.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 -- 172.21.15.26:0/3637485725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0e3b60 msgr2=0x556daf0e2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0e3b60 0x556daf0e2670 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f41c007d780 tx=0x7f41c0009190 comp rx=0 tx=0).stop 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 -- 172.21.15.26:0/3637485725 shutdown_connections 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556daf0b0bb0 0x556daf0b0f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0afe40 0x556daf0c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0e3b60 0x556daf0e2670 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/3637485725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0c6c80 0x556daf0e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41a97fa640 0 client.5913 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 -- 172.21.15.26:0/3637485725 >> 172.21.15.26:0/3637485725 conn(0x556daf0c6540 msgr2=0x556daf095ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 -- 172.21.15.26:0/3637485725 shutdown_connections 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 1 -- 172.21.15.26:0/3637485725 wait complete. 2024-04-24T23:23:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.891+0000 7f41d3a1a740 20 client.5913 trim_cache size 0 max 16384 2024-04-24T23:23:13.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.909+0000 7f41d3a1a740 1 Processor -- start 2024-04-24T23:23:13.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.909+0000 7f41d3a1a740 1 -- start start 2024-04-24T23:23:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf0e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf096440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 0x556daf0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1ffd70 con 0x556daf0c6c80 2024-04-24T23:23:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1ff220 con 0x556daf0afe40 2024-04-24T23:23:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f41cc1ff720 con 0x556daf0e3b60 2024-04-24T23:23:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf096440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf096440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60366/0 (socket says 172.21.15.26:60366) 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41c9d9b640 1 -- 172.21.15.26:0/2705125454 learned_addr learned my addr 172.21.15.26:0/2705125454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41cad9d640 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 0x556daf0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41c9d9b640 1 -- 172.21.15.26:0/2705125454 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 msgr2=0x556daf0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 0x556daf0b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41c9d9b640 1 -- 172.21.15.26:0/2705125454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 msgr2=0x556daf0e4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41ca59c640 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf0e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.910+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf0e4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41c9d9b640 1 -- 172.21.15.26:0/2705125454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556daf0afc50 con 0x556daf0c6c80 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41ca59c640 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf0e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf096440 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f41c00818c0 tx=0x7f41c0094040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe6b15f5d75f209 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41ab7fe640 1 -- 172.21.15.26:0/2705125454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c00754b0 con 0x556daf0c6c80 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41ab7fe640 1 -- 172.21.15.26:0/2705125454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c00a8040 con 0x556daf0c6c80 2024-04-24T23:23:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41ab7fe640 1 -- 172.21.15.26:0/2705125454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c007d3b0 con 0x556daf0c6c80 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2705125454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 msgr2=0x556daf096440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf096440 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f41c00818c0 tx=0x7f41c0094040 comp rx=0 tx=0).stop 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2705125454 shutdown_connections 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 0x556daf0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf096440 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2705125454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf0e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2705125454 >> 172.21.15.26:0/2705125454 conn(0x556daf0c6540 msgr2=0x556daf104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2705125454 shutdown_connections 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.911+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2705125454 wait complete. 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.912+0000 7f41d3a1a740 1 -- start start 2024-04-24T23:23:13.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.912+0000 7f41d3a1a740 1 Processor -- start 2024-04-24T23:23:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41d3a1a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 0x556daf0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556daf0b1c30 con 0x556daf0c6c80 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556daefa7750 con 0x556daf0afe40 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41d3a1a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556daf0e1260 con 0x556daf0e3b60 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41d3a1a740 10 client.0 _ll_register_callbacks cb 0x7f41c0079350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41c9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60374/0 (socket says 172.21.15.26:60374) 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41c9d9b640 1 -- 172.21.15.26:0/2486722469 learned_addr learned my addr 172.21.15.26:0/2486722469 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41c9d9b640 1 -- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 msgr2=0x556daf0b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41ca59c640 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41cad9d640 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 0x556daf0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 0x556daf0b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41c9d9b640 1 -- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 msgr2=0x556daf15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf15ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41c9d9b640 1 -- 172.21.15.26:0/2486722469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556daf0faa70 con 0x556daf0c6c80 2024-04-24T23:23:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41cad9d640 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 0x556daf0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41ca59c640 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41c9d9b640 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0b0f90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f41c00983a0 tx=0x7f41c0079090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c2d77f609b4de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.913+0000 7f41a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41a97fa640 1 -- 172.21.15.26:0/2486722469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0086060 con 0x556daf0c6c80 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41a97fa640 1 -- 172.21.15.26:0/2486722469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f41c0081b60 con 0x556daf0c6c80 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2486722469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c00a8040 con 0x556daf0c6c80 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41a97fa640 1 -- 172.21.15.26:0/2486722469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f41c008c050 con 0x556daf0c6c80 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41d3a1a740 10 client.5919 fetch_fsmap learned FSMap version 33 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2486722469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x556daf0faa70 con 0x556daf0c6c80 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41a97fa640 1 -- 172.21.15.26:0/2486722469 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c00a8040 con 0x556daf0c6c80 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41a97fa640 1 -- 172.21.15.26:0/2486722469 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f41c008f020 con 0x556daf0c6c80 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41d3a1a740 10 client.5919 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41d3a1a740 10 client.5919 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:23:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41d3a1a740 20 client.5919 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556daf0d6a00 0x556daf0d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41d3a1a740 4 client.5919 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2486722469 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "status"}) -- 0x556daf006790 con 0x556daf0d6a00 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.914+0000 7f41ca59c640 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556daf0d6a00 0x556daf0d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41ca59c640 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556daf0d6a00 0x556daf0d6de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8482c5933926f2f server_cookie=4c60ecbdad2abc38 in_seq=0 out_seq=0 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41a97fa640 10 client.5919 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41a97fa640 1 -- 172.21.15.26:0/2486722469 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x556daf006790 con 0x556daf0d6a00 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41a97fa640 10 client.5919 handle_command_reply: tid=0 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 client.5919 shutdown 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2486722469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 msgr2=0x556daf0b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0b0f90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f41c00983a0 tx=0x7f41c0079090 comp rx=0 tx=0).stop 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2486722469 shutdown_connections 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556daf0d6a00 0x556daf0d6de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556daf0e3b60 0x556daf0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556daf0c6c80 0x556daf0b0f90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41a97fa640 0 client.5919 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 --2- 172.21.15.26:0/2486722469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556daf0afe40 0x556daf15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2486722469 >> 172.21.15.26:0/2486722469 conn(0x556daf0c6540 msgr2=0x556daf096350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2486722469 shutdown_connections 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 1 -- 172.21.15.26:0/2486722469 wait complete. 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.915+0000 7f41d3a1a740 20 client.5919 trim_cache size 0 max 16384 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:23:13.923 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5742, 2024-04-24T23:23:13.924 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:23:13.924 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:23:13.924 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:23:13.924 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 10.118092122, 2024-04-24T23:23:13.924 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 33, 2024-04-24T23:23:13.924 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 44, 2024-04-24T23:23:13.924 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 44, 2024-04-24T23:23:13.924 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.124595475 2024-04-24T23:23:13.924 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 -- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41ac06be60 msgr2=0x7f41ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41ac06be60 0x7f41ac06e320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000c00 tx=0x7f41c4009040 comp rx=0 tx=0).stop 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 -- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc14dc10 msgr2=0x7f41cc1f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc14dc10 0x7f41cc1f3220 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f41c002abb0 tx=0x7f41c0004730 comp rx=0 tx=0).stop 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 -- 172.21.15.26:0/519869483 shutdown_connections 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41ac06be60 0x7f41ac06e320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41cc1480c0 0x7f41cc148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc14dc10 0x7f41cc1f3220 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 --2- 172.21.15.26:0/519869483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41b4004800 0x7f41cc14a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:13.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 -- 172.21.15.26:0/519869483 >> 172.21.15.26:0/519869483 conn(0x7f41cc06e2c0 msgr2=0x7f41cc161110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 -- 172.21.15.26:0/519869483 shutdown_connections 2024-04-24T23:23:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:13.916+0000 7f41d0c31640 1 -- 172.21.15.26:0/519869483 wait complete. 2024-04-24T23:23:13.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:23:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.027+0000 7fedaab89640 1 Processor -- start 2024-04-24T23:23:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.027+0000 7fedaab89640 1 -- start start 2024-04-24T23:23:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7fedaab89640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feda4145910 0x7feda406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7fedaab89640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feda40cd660 0x7feda40cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7fedaab89640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda40cfbc0 con 0x7feda406d350 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7fedaab89640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7feda4145cf0 con 0x7feda40cd660 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7fedaab89640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7feda406d7f0 con 0x7feda4145910 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7feda9126640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feda40cd660 0x7feda40cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7feda9126640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feda40cd660 0x7feda40cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60384/0 (socket says 172.21.15.26:60384) 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7feda3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feda406d350 0x7feda415d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50850/0 (socket says 172.21.15.26:50850) 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7feda9126640 1 -- 172.21.15.26:0/1919863967 learned_addr learned my addr 172.21.15.26:0/1919863967 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7feda9126640 1 -- 172.21.15.26:0/1919863967 >> v2:172.21.15.29:3300/0 conn(0x7feda4145910 msgr2=0x7feda406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7feda9126640 1 --2- 172.21.15.26:0/1919863967 >> v2:172.21.15.29:3300/0 conn(0x7feda4145910 0x7feda406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7feda9126640 1 -- 172.21.15.26:0/1919863967 >> v1:172.21.15.26:6789/0 conn(0x7feda406d350 legacy=0x7feda415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.028+0000 7feda9126640 1 -- 172.21.15.26:0/1919863967 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda406dad0 con 0x7feda40cd660 2024-04-24T23:23:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.029+0000 7feda9126640 1 --2- 172.21.15.26:0/1919863967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 0x7feda40cdaf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009e50 tx=0x7fed9c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a8cf1b7acaad22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.029+0000 7feda37fe640 1 -- 172.21.15.26:0/1919863967 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c036070 con 0x7feda40cd660 2024-04-24T23:23:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.029+0000 7feda37fe640 1 -- 172.21.15.26:0/1919863967 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c031600 con 0x7feda40cd660 2024-04-24T23:23:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.029+0000 7feda37fe640 1 -- 172.21.15.26:0/1919863967 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0319d0 con 0x7feda40cd660 2024-04-24T23:23:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.030+0000 7fedaab89640 1 -- 172.21.15.26:0/1919863967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 msgr2=0x7feda40cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.030+0000 7fedaab89640 1 --2- 172.21.15.26:0/1919863967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 0x7feda40cdaf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009e50 tx=0x7fed9c02a6d0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.030+0000 7fedaab89640 1 -- 172.21.15.26:0/1919863967 shutdown_connections 2024-04-24T23:23:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.030+0000 7fedaab89640 1 --2- 172.21.15.26:0/1919863967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 0x7feda40cdaf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.030+0000 7fedaab89640 1 --2- 172.21.15.26:0/1919863967 >> v2:172.21.15.29:3300/0 conn(0x7feda4145910 0x7feda406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.030+0000 7fedaab89640 1 -- 172.21.15.26:0/1919863967 >> 172.21.15.26:0/1919863967 conn(0x7feda406b300 msgr2=0x7feda406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.030+0000 7fedaab89640 1 -- 172.21.15.26:0/1919863967 shutdown_connections 2024-04-24T23:23:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.030+0000 7fedaab89640 1 -- 172.21.15.26:0/1919863967 wait complete. 2024-04-24T23:23:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.031+0000 7fedaab89640 1 Processor -- start 2024-04-24T23:23:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.031+0000 7fedaab89640 1 -- start start 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7fedaab89640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feda406d350 0x7feda4156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7fedaab89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 0x7feda41f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7fedaab89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda4145910 0x7feda4141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7fedaab89640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feda406e370 con 0x7feda40cd660 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7fedaab89640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feda406e090 con 0x7feda4145910 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7fedaab89640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feda415d770 con 0x7feda406d350 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 0x7feda41f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 0x7feda41f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60390/0 (socket says 172.21.15.26:60390) 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda3fff640 1 -- 172.21.15.26:0/3816497483 learned_addr learned my addr 172.21.15.26:0/3816497483 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda8925640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feda406d350 0x7feda4156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda9126640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda4145910 0x7feda4141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda3fff640 1 -- 172.21.15.26:0/3816497483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feda406d350 msgr2=0x7feda4156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda3fff640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feda406d350 0x7feda4156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda3fff640 1 -- 172.21.15.26:0/3816497483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda4145910 msgr2=0x7feda4141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda3fff640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda4145910 0x7feda4141c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda3fff640 1 -- 172.21.15.26:0/3816497483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda413f6b0 con 0x7feda40cd660 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.032+0000 7feda3fff640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 0x7feda41f5b80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c500 tx=0x7fed9000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.033+0000 7feda17fa640 1 -- 172.21.15.26:0/3816497483 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90018070 con 0x7feda40cd660 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.033+0000 7feda17fa640 1 -- 172.21.15.26:0/3816497483 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed90007500 con 0x7feda40cd660 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.033+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda413f990 con 0x7feda40cd660 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.033+0000 7feda17fa640 1 -- 172.21.15.26:0/3816497483 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed900049e0 con 0x7feda40cd660 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.033+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda413ff50 con 0x7feda40cd660 2024-04-24T23:23:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.034+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda40694c0 con 0x7feda40cd660 2024-04-24T23:23:14.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.034+0000 7feda17fa640 1 -- 172.21.15.26:0/3816497483 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed90004b80 con 0x7feda40cd660 2024-04-24T23:23:14.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.035+0000 7feda17fa640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:14.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.035+0000 7feda17fa640 1 -- 172.21.15.26:0/3816497483 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fed9008a080 con 0x7feda40cd660 2024-04-24T23:23:14.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.035+0000 7feda8925640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:14.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.035+0000 7feda8925640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed9800b810 tx=0x7fed98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:14.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.039+0000 7feda17fa640 1 -- 172.21.15.26:0/3816497483 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fed900583f0 con 0x7feda40cd660 2024-04-24T23:23:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.261+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feda40cdf30 con 0x7feda40cd660 2024-04-24T23:23:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.262+0000 7feda17fa640 1 -- 172.21.15.26:0/3816497483 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7fed9005c6f0 con 0x7feda40cd660 2024-04-24T23:23:14.263 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:23:14.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 msgr2=0x7fed8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed9800b810 tx=0x7fed98008040 comp rx=0 tx=0).stop 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 msgr2=0x7feda41f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 0x7feda41f5b80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c500 tx=0x7fed9000c9c0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 shutdown_connections 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda4145910 0x7feda4141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda40cd660 0x7feda41f5b80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 --2- 172.21.15.26:0/3816497483 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feda406d350 0x7feda4156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 >> 172.21.15.26:0/3816497483 conn(0x7feda406b300 msgr2=0x7feda415e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 shutdown_connections 2024-04-24T23:23:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14.264+0000 7fedaab89640 1 -- 172.21.15.26:0/3816497483 wait complete. 2024-04-24T23:23:14.276 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:23:14.276 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:23:14.276 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-24T23:23:14.276 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:23:14.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:23:14.276 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:23:14.276 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:23:14.276 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:23:14.276 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:23:14.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:14.277 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-24T23:23:14.292 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:23:14.293 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42598sec preferred_lft 42598sec 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:23:14.294 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ccf2:1eff:fe9b:4749/64 scope link 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c2:ea:e0:db:00:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c0ea:e0ff:fedb:e0/64 scope link 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::1c82:d0ff:fefa:5b3f/64 scope link 2024-04-24T23:23:14.295 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:23:14.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:14.296 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:23:14.356 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-24T23:23:14.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:14.358 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-24T23:23:14.414 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-24T23:23:14.414 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:23:14.414 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:23:14.414 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:23:14.414 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:23:14.414 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:23:14.414 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:23:14.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:23:14.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:23:14.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:23:14.415 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:23:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:23:14.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:23:14.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:14.541 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:23:14.556 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:23:14.556 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:23:14.556 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:23:14.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:14.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-24T23:23:14.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:23:14.621 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:23:14.676 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:23:14.676 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:23:14.677 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:23:14.732 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:23:14.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:14.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:23:14.814 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:23:14.815 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:23:14.815 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:23:14.823 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:23:14.823 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:23:14.823 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:23:14.823 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:23:14.823 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:23:14.832 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:23:14.832 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:23:14.832 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:23:14.832 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:23:14.832 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:23:14.832 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:23:14.832 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:23:14.832 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:23:14.839 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:23:14.839 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:23:14.839 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:23:14.839 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:23:14.839 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:23:14.840 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:23:14.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:14.840 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:23:14.904 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:23:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:23:14.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:23:15.006 INFO:teuthology.orchestra.run.smithi026.stdout:client5458 2024-04-24T23:23:15.006 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:23:15.006 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:23:15.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T23:23:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.165+0000 7fc7b8330640 1 Processor -- start 2024-04-24T23:23:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.165+0000 7fc7b8330640 1 -- start start 2024-04-24T23:23:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.166+0000 7fc7b8330640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015adc0 0x7fc7b00b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.166+0000 7fc7b8330640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7b00ad7b0 0x7fc7b00d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.166+0000 7fc7b8330640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b015ef60 con 0x7fc7b015e9c0 2024-04-24T23:23:15.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.166+0000 7fc7b8330640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b015f240 con 0x7fc7b00ad7b0 2024-04-24T23:23:15.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.166+0000 7fc7b8330640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b015f520 con 0x7fc7b015adc0 2024-04-24T23:23:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.166+0000 7fc7b60cc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015adc0 0x7fc7b00b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.167+0000 7fc7b60cc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015adc0 0x7fc7b00b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:42060/0 (socket says 172.21.15.26:42060) 2024-04-24T23:23:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.167+0000 7fc7b58cb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7b00ad7b0 0x7fc7b00d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.167+0000 7fc7b60cc640 1 -- 172.21.15.26:0/2444758445 learned_addr learned my addr 172.21.15.26:0/2444758445 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.167+0000 7fc7b60cc640 1 -- 172.21.15.26:0/2444758445 >> v2:172.21.15.29:3300/0 conn(0x7fc7b00ad7b0 msgr2=0x7fc7b00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.167+0000 7fc7b60cc640 1 --2- 172.21.15.26:0/2444758445 >> v2:172.21.15.29:3300/0 conn(0x7fc7b00ad7b0 0x7fc7b00d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.167+0000 7fc7b60cc640 1 -- 172.21.15.26:0/2444758445 >> v1:172.21.15.29:6789/0 conn(0x7fc7b015e9c0 legacy=0x7fc7b006dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.167+0000 7fc7b60cc640 1 -- 172.21.15.26:0/2444758445 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b015f800 con 0x7fc7b015adc0 2024-04-24T23:23:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.167+0000 7fc7b58cb640 1 --2- 172.21.15.26:0/2444758445 >> v2:172.21.15.29:3300/0 conn(0x7fc7b00ad7b0 0x7fc7b00d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.168+0000 7fc7b60cc640 1 --2- 172.21.15.26:0/2444758445 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015adc0 0x7fc7b00b4370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800bb10 tx=0x7fc7a802c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f2a54b5abcd22b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.168+0000 7fc7b50ca640 1 -- 172.21.15.26:0/2444758445 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8032790 con 0x7fc7b015adc0 2024-04-24T23:23:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.168+0000 7fc7b50ca640 1 -- 172.21.15.26:0/2444758445 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a8038070 con 0x7fc7b015adc0 2024-04-24T23:23:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.168+0000 7fc7b50ca640 1 -- 172.21.15.26:0/2444758445 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8032b50 con 0x7fc7b015adc0 2024-04-24T23:23:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.168+0000 7fc7b8330640 1 -- 172.21.15.26:0/2444758445 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015adc0 msgr2=0x7fc7b00b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.168+0000 7fc7b8330640 1 --2- 172.21.15.26:0/2444758445 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015adc0 0x7fc7b00b4370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800bb10 tx=0x7fc7a802c890 comp rx=0 tx=0).stop 2024-04-24T23:23:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.169+0000 7fc7b8330640 1 -- 172.21.15.26:0/2444758445 shutdown_connections 2024-04-24T23:23:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.169+0000 7fc7b8330640 1 --2- 172.21.15.26:0/2444758445 >> v2:172.21.15.29:3300/0 conn(0x7fc7b00ad7b0 0x7fc7b00d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.169+0000 7fc7b8330640 1 --2- 172.21.15.26:0/2444758445 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015adc0 0x7fc7b00b4370 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.169+0000 7fc7b8330640 1 -- 172.21.15.26:0/2444758445 >> 172.21.15.26:0/2444758445 conn(0x7fc7b006b540 msgr2=0x7fc7b006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.169+0000 7fc7b8330640 1 -- 172.21.15.26:0/2444758445 shutdown_connections 2024-04-24T23:23:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.169+0000 7fc7b8330640 1 -- 172.21.15.26:0/2444758445 wait complete. 2024-04-24T23:23:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.170+0000 7fc7b8330640 1 Processor -- start 2024-04-24T23:23:15.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.170+0000 7fc7b8330640 1 -- start start 2024-04-24T23:23:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.170+0000 7fc7b8330640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b00ad7b0 0x7fc7b00cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.170+0000 7fc7b8330640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b015adc0 0x7fc7b00adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.170+0000 7fc7b8330640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015e9c0 0x7fc7b009bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.170+0000 7fc7b8330640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b015b1a0 con 0x7fc7b015adc0 2024-04-24T23:23:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.170+0000 7fc7b8330640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b00b48b0 con 0x7fc7b00ad7b0 2024-04-24T23:23:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.170+0000 7fc7b8330640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b00d0580 con 0x7fc7b015e9c0 2024-04-24T23:23:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b58cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b015adc0 0x7fc7b00adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b58cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b015adc0 0x7fc7b00adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60404/0 (socket says 172.21.15.26:60404) 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b58cb640 1 -- 172.21.15.26:0/610785026 learned_addr learned my addr 172.21.15.26:0/610785026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b68cd640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015e9c0 0x7fc7b009bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b60cc640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b00ad7b0 0x7fc7b00cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b58cb640 1 -- 172.21.15.26:0/610785026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015e9c0 msgr2=0x7fc7b009bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b58cb640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015e9c0 0x7fc7b009bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b58cb640 1 -- 172.21.15.26:0/610785026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b00ad7b0 msgr2=0x7fc7b00cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b58cb640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b00ad7b0 0x7fc7b00cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b58cb640 1 -- 172.21.15.26:0/610785026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b00cdd10 con 0x7fc7b015adc0 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b60cc640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b00ad7b0 0x7fc7b00cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b58cb640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b015adc0 0x7fc7b00adec0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000ebe0 tx=0x7fc7a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7a6ffd640 1 -- 172.21.15.26:0/610785026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0017070 con 0x7fc7b015adc0 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7a6ffd640 1 -- 172.21.15.26:0/610785026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a000cc30 con 0x7fc7b015adc0 2024-04-24T23:23:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b00cdff0 con 0x7fc7b015adc0 2024-04-24T23:23:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7a6ffd640 1 -- 172.21.15.26:0/610785026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0013470 con 0x7fc7b015adc0 2024-04-24T23:23:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.171+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b00ce5b0 con 0x7fc7b015adc0 2024-04-24T23:23:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.172+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b0075610 con 0x7fc7b015adc0 2024-04-24T23:23:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.173+0000 7fc7a6ffd640 1 -- 172.21.15.26:0/610785026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc7a001a070 con 0x7fc7b015adc0 2024-04-24T23:23:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.173+0000 7fc7a6ffd640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc79406be30 0x7fc79406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.173+0000 7fc7a6ffd640 1 -- 172.21.15.26:0/610785026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc7a008b180 con 0x7fc7b015adc0 2024-04-24T23:23:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.173+0000 7fc7b60cc640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc79406be30 0x7fc79406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.174+0000 7fc7b60cc640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc79406be30 0x7fc79406e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8037000 tx=0x7fc7a8032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:15.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.178+0000 7fc7a6ffd640 1 -- 172.21.15.26:0/610785026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc7a0059b30 con 0x7fc7b015adc0 2024-04-24T23:23:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.400+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc7b00719f0 con 0x7fc7b015adc0 2024-04-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.843+0000 7fc7a6ffd640 1 -- 172.21.15.26:0/610785026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v34) ==== 105+0+0 (secure 0 0 0) 0x7fc7a0059430 con 0x7fc7b015adc0 2024-04-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc79406be30 msgr2=0x7fc79406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc79406be30 0x7fc79406e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8037000 tx=0x7fc7a8032000 comp rx=0 tx=0).stop 2024-04-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b015adc0 msgr2=0x7fc7b00adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b015adc0 0x7fc7b00adec0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000ebe0 tx=0x7fc7a000c520 comp rx=0 tx=0).stop 2024-04-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 shutdown_connections 2024-04-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc79406be30 0x7fc79406e2f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7b015e9c0 0x7fc7b009bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b015adc0 0x7fc7b00adec0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 --2- 172.21.15.26:0/610785026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7b00ad7b0 0x7fc7b00cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 >> 172.21.15.26:0/610785026 conn(0x7fc7b006b540 msgr2=0x7fc7b00b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 shutdown_connections 2024-04-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.846+0000 7fc7b8330640 1 -- 172.21.15.26:0/610785026 wait complete. 2024-04-24T23:23:15.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.966+0000 7f11adfe7640 1 Processor -- start 2024-04-24T23:23:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.966+0000 7f11adfe7640 1 -- start start 2024-04-24T23:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.967+0000 7f11adfe7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f11a80b3ac0 0x7f11a80b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.967+0000 7f11adfe7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11a80b62e0 0x7f11a80b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.967+0000 7f11adfe7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a80b5840 con 0x7f11a815e290 2024-04-24T23:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.967+0000 7f11adfe7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f11a80b5e50 con 0x7f11a80b62e0 2024-04-24T23:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.967+0000 7f11adfe7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f11a80cfb30 con 0x7f11a80b3ac0 2024-04-24T23:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.967+0000 7f11a7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11a80b62e0 0x7f11a80b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.967+0000 7f11a7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11a80b62e0 0x7f11a80b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60410/0 (socket says 172.21.15.26:60410) 2024-04-24T23:23:15.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.967+0000 7f11a7fff640 1 -- 172.21.15.26:0/3330958905 learned_addr learned my addr 172.21.15.26:0/3330958905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:15.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.967+0000 7f11a6ffd640 1 --2- 172.21.15.26:0/3330958905 >> v2:172.21.15.29:3300/0 conn(0x7f11a80b3ac0 0x7f11a80b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.968+0000 7f11a7fff640 1 -- 172.21.15.26:0/3330958905 >> v2:172.21.15.29:3300/0 conn(0x7f11a80b3ac0 msgr2=0x7f11a80b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:15.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.968+0000 7f11a7fff640 1 --2- 172.21.15.26:0/3330958905 >> v2:172.21.15.29:3300/0 conn(0x7f11a80b3ac0 0x7f11a80b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.968+0000 7f11a7fff640 1 -- 172.21.15.26:0/3330958905 >> v1:172.21.15.29:6789/0 conn(0x7f11a815e290 legacy=0x7f11a80d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.968+0000 7f11a7fff640 1 -- 172.21.15.26:0/3330958905 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a80cfe10 con 0x7f11a80b62e0 2024-04-24T23:23:15.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.968+0000 7f11a7fff640 1 --2- 172.21.15.26:0/3330958905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a80b62e0 0x7f11a80b6700 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f11a80cfa80 tx=0x7f11a002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2c7ac0ddb012a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:15.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11a67fc640 1 -- 172.21.15.26:0/3330958905 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0036070 con 0x7f11a80b62e0 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11a67fc640 1 -- 172.21.15.26:0/3330958905 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a0031600 con 0x7f11a80b62e0 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11a67fc640 1 -- 172.21.15.26:0/3330958905 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a00319d0 con 0x7f11a80b62e0 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11adfe7640 1 -- 172.21.15.26:0/3330958905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a80b62e0 msgr2=0x7f11a80b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11adfe7640 1 --2- 172.21.15.26:0/3330958905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a80b62e0 0x7f11a80b6700 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f11a80cfa80 tx=0x7f11a002a730 comp rx=0 tx=0).stop 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11adfe7640 1 -- 172.21.15.26:0/3330958905 shutdown_connections 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11adfe7640 1 --2- 172.21.15.26:0/3330958905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a80b62e0 0x7f11a80b6700 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11adfe7640 1 --2- 172.21.15.26:0/3330958905 >> v2:172.21.15.29:3300/0 conn(0x7f11a80b3ac0 0x7f11a80b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11adfe7640 1 -- 172.21.15.26:0/3330958905 >> 172.21.15.26:0/3330958905 conn(0x7f11a806b120 msgr2=0x7f11a806b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11adfe7640 1 -- 172.21.15.26:0/3330958905 shutdown_connections 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.969+0000 7f11adfe7640 1 -- 172.21.15.26:0/3330958905 wait complete. 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.970+0000 7f11adfe7640 1 Processor -- start 2024-04-24T23:23:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.970+0000 7f11adfe7640 1 -- start start 2024-04-24T23:23:15.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11adfe7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11a80b3ac0 0x7f11a80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11adfe7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f11a80b62e0 0x7f11a80c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11adfe7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a815e290 0x7f11a80c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11adfe7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11a815e670 con 0x7f11a815e290 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11adfe7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80b5ae0 con 0x7f11a80b3ac0 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11adfe7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f11a806dfd0 con 0x7f11a80b62e0 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a815e290 0x7f11a80c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a815e290 0x7f11a80c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60420/0 (socket says 172.21.15.26:60420) 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11a7fff640 1 -- 172.21.15.26:0/733801226 learned_addr learned my addr 172.21.15.26:0/733801226 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11a77fe640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11a80b3ac0 0x7f11a80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.971+0000 7f11a6ffd640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f11a80b62e0 0x7f11a80c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f11a7fff640 1 -- 172.21.15.26:0/733801226 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f11a80b62e0 msgr2=0x7f11a80c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f11a7fff640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f11a80b62e0 0x7f11a80c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f11a7fff640 1 -- 172.21.15.26:0/733801226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11a80b3ac0 msgr2=0x7f11a80c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f11a7fff640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11a80b3ac0 0x7f11a80c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f11a7fff640 1 -- 172.21.15.26:0/733801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a815fbb0 con 0x7f11a815e290 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f11a7fff640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a815e290 0x7f11a80c73f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009f40 tx=0x7f11a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f118ffff640 1 -- 172.21.15.26:0/733801226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a003f020 con 0x7f11a815e290 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f118ffff640 1 -- 172.21.15.26:0/733801226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a0037040 con 0x7f11a815e290 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a8150140 con 0x7f11a815e290 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f118ffff640 1 -- 172.21.15.26:0/733801226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0036070 con 0x7f11a815e290 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.972+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a80c98c0 con 0x7f11a815e290 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.973+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a8150530 con 0x7f11a815e290 2024-04-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.974+0000 7f118ffff640 1 -- 172.21.15.26:0/733801226 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f11a0042020 con 0x7f11a815e290 2024-04-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.974+0000 7f118ffff640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f119406be30 0x7f119406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.974+0000 7f118ffff640 1 -- 172.21.15.26:0/733801226 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f11a00ab130 con 0x7f11a815e290 2024-04-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.974+0000 7f11a77fe640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f119406be30 0x7f119406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.975+0000 7f11a77fe640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f119406be30 0x7f119406e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1198000f60 tx=0x7f1198009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:15.979+0000 7f118ffff640 1 -- 172.21.15.26:0/733801226 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f11a0049030 con 0x7f11a815e290 2024-04-24T23:23:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.202+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11a80694c0 con 0x7f11a815e290 2024-04-24T23:23:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.203+0000 7f118ffff640 1 -- 172.21.15.26:0/733801226 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5191 (secure 0 0 0) 0x7f11a0079490 con 0x7f11a815e290 2024-04-24T23:23:16.204 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:16.204 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:15.842998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5742,"mds_1":5329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:creating","state_seq":1,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-24T23:23:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 34 2024-04-24T23:23:16.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f119406be30 msgr2=0x7f119406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:16.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f119406be30 0x7f119406e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1198000f60 tx=0x7f1198009040 comp rx=0 tx=0).stop 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a815e290 msgr2=0x7f11a80c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a815e290 0x7f11a80c73f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009f40 tx=0x7f11a0004a40 comp rx=0 tx=0).stop 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 shutdown_connections 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f119406be30 0x7f119406e2f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a815e290 0x7f11a80c73f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f11a80b62e0 0x7f11a80c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 --2- 172.21.15.26:0/733801226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f11a80b3ac0 0x7f11a80c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 >> 172.21.15.26:0/733801226 conn(0x7f11a806b120 msgr2=0x7f11a80ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 shutdown_connections 2024-04-24T23:23:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:16.205+0000 7f11adfe7640 1 -- 172.21.15.26:0/733801226 wait complete. 2024-04-24T23:23:16.620 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:23:03.164134+0000', 'modified': '2024-04-24T23:23:15.842998+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5742, 'mds_1': 5329}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5329': {'gid': 5329, 'name': 'd', 'rank': 1, 'incarnation': 34, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.29:6834/243988267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 243988267}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 243988267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5742': {'gid': 5742, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6836/730964888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 730964888}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 730964888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5742, 'qdb_cluster': [5742]} 2024-04-24T23:23:16.620 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5329:up:creating 2024-04-24T23:23:17.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:17.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.720+0000 7fde2d102640 1 Processor -- start 2024-04-24T23:23:17.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.720+0000 7fde2d102640 1 -- start start 2024-04-24T23:23:17.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.721+0000 7fde2d102640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde280d4870 0x7fde280b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:17.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.721+0000 7fde2d102640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde280d32f0 con 0x7fde2815e570 2024-04-24T23:23:17.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.721+0000 7fde2d102640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde280d0270 con 0x7fde280b3130 2024-04-24T23:23:17.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.721+0000 7fde2d102640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fde280d1ab0 con 0x7fde280d4870 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.721+0000 7fde26d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fde2815e570 0x7fde280d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50858/0 (socket says 172.21.15.26:50858) 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.721+0000 7fde26d9d640 1 -- 172.21.15.26:0/2335132495 learned_addr learned my addr 172.21.15.26:0/2335132495 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2712483241 0 0) 0x7fde18001110 con 0x7fde2815e570 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde10003520 con 0x7fde2815e570 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde2659c640 1 --2- 172.21.15.26:0/2335132495 >> v2:172.21.15.29:3300/0 conn(0x7fde280d4870 0x7fde280b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 859165174 0 0) 0x7fde10003520 con 0x7fde2815e570 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 >> v2:172.21.15.29:3300/0 conn(0x7fde280d4870 msgr2=0x7fde280b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 --2- 172.21.15.26:0/2335132495 >> v2:172.21.15.29:3300/0 conn(0x7fde280d4870 0x7fde280b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 >> v1:172.21.15.29:6789/0 conn(0x7fde280b3130 legacy=0x7fde280b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde280d2d30 con 0x7fde2815e570 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fde18001840 con 0x7fde2815e570 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 >> v1:172.21.15.26:6789/0 conn(0x7fde2815e570 legacy=0x7fde280d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde100039b0 0x7fde10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:17.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde100051c0 0x7fde1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde280d1ab0 con 0x7fde10004800 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde1000c160 con 0x7fde100039b0 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fde1000c360 con 0x7fde100051c0 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde2659c640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde26d9d640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde100039b0 0x7fde10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.722+0000 7fde2759e640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde100051c0 0x7fde1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde2659c640 1 -- 172.21.15.26:0/2335132495 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde100051c0 msgr2=0x7fde1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde2659c640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde100051c0 0x7fde1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde2659c640 1 -- 172.21.15.26:0/2335132495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde100039b0 msgr2=0x7fde10003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde2659c640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde100039b0 0x7fde10003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde2659c640 1 -- 172.21.15.26:0/2335132495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde10003520 con 0x7fde10004800 2024-04-24T23:23:17.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde2759e640 1 -- 172.21.15.26:0/2335132495 reap_dead start 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde26d9d640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde100039b0 0x7fde10003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde2659c640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde10004c80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009ac0 tx=0x7fde10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d00e5d337f06f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c036070 con 0x7fde10004800 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde1c031400 con 0x7fde10004800 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.723+0000 7fde25d9b640 1 -- 172.21.15.26:0/2335132495 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c009690 con 0x7fde10004800 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.724+0000 7fde2d102640 1 -- 172.21.15.26:0/2335132495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 msgr2=0x7fde10004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.724+0000 7fde2d102640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde10004c80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009ac0 tx=0x7fde10005830 comp rx=0 tx=0).stop 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.724+0000 7fde2d102640 1 -- 172.21.15.26:0/2335132495 shutdown_connections 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.724+0000 7fde2d102640 1 --2- 172.21.15.26:0/2335132495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde10004c80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.724+0000 7fde2d102640 1 -- 172.21.15.26:0/2335132495 >> 172.21.15.26:0/2335132495 conn(0x7fde2808dbe0 msgr2=0x7fde2809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.724+0000 7fde2d102640 1 -- 172.21.15.26:0/2335132495 shutdown_connections 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.724+0000 7fde2d102640 1 -- 172.21.15.26:0/2335132495 wait complete. 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.726+0000 7fde2d102640 1 Processor -- start 2024-04-24T23:23:17.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.726+0000 7fde2d102640 1 -- start start 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2d102640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde280b9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2d102640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2814c830 0x7fde2814cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2d102640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde2814a7a0 0x7fde2814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2d102640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde280cfa70 con 0x7fde10004800 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2d102640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde280d2ae0 con 0x7fde2814a7a0 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2d102640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fde280cfd50 con 0x7fde2814c830 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde26d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde280b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde26d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde280b9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60440/0 (socket says 172.21.15.26:60440) 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde26d9d640 1 -- 172.21.15.26:0/4119919440 learned_addr learned my addr 172.21.15.26:0/4119919440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2659c640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2814c830 0x7fde2814cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2759e640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde2814a7a0 0x7fde2814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde26d9d640 1 -- 172.21.15.26:0/4119919440 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2814c830 msgr2=0x7fde2814cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde26d9d640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2814c830 0x7fde2814cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde26d9d640 1 -- 172.21.15.26:0/4119919440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde2814a7a0 msgr2=0x7fde2814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:17.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde26d9d640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde2814a7a0 0x7fde2814ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde26d9d640 1 -- 172.21.15.26:0/4119919440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2814b160 con 0x7fde10004800 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2659c640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2814c830 0x7fde2814cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde26d9d640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde280b9290 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fde18004fb0 tx=0x7fde180074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde0f7fe640 1 -- 172.21.15.26:0/4119919440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800c0e0 con 0x7fde10004800 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde0f7fe640 1 -- 172.21.15.26:0/4119919440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde18018070 con 0x7fde10004800 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2814b420 con 0x7fde10004800 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde0f7fe640 1 -- 172.21.15.26:0/4119919440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800cb90 con 0x7fde10004800 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.728+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2814b9e0 con 0x7fde10004800 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.730+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde280694c0 con 0x7fde10004800 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.730+0000 7fde0f7fe640 1 -- 172.21.15.26:0/4119919440 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde1801d460 con 0x7fde10004800 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.730+0000 7fde0f7fe640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1406be60 0x7fde1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.731+0000 7fde0f7fe640 1 -- 172.21.15.26:0/4119919440 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fde1808ae00 con 0x7fde10004800 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.731+0000 7fde2659c640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1406be60 0x7fde1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:17.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.731+0000 7fde2659c640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1406be60 0x7fde1406e320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fde1c033000 tx=0x7fde1c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:17.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.735+0000 7fde0f7fe640 1 -- 172.21.15.26:0/4119919440 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde280d32f0 con 0x7fde10004800 2024-04-24T23:23:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.956+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde280ced80 con 0x7fde10004800 2024-04-24T23:23:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.957+0000 7fde0f7fe640 1 -- 172.21.15.26:0/4119919440 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5213 (secure 0 0 0) 0x7fde180597b0 con 0x7fde10004800 2024-04-24T23:23:17.959 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:17.959 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:17.849716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5742,"mds_1":5329,"mds_2":5081},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5081":{"gid":5081,"name":"b","rank":2,"incarnation":36,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5081,5329,5742]},"id":3}]} 2024-04-24T23:23:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 37 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1406be60 msgr2=0x7fde1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1406be60 0x7fde1406e320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fde1c033000 tx=0x7fde1c008030 comp rx=0 tx=0).stop 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 msgr2=0x7fde280b9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde280b9290 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fde18004fb0 tx=0x7fde180074f0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 shutdown_connections 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1406be60 0x7fde1406e320 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde2814a7a0 0x7fde2814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2814c830 0x7fde2814cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 --2- 172.21.15.26:0/4119919440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde10004800 0x7fde280b9290 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 >> 172.21.15.26:0/4119919440 conn(0x7fde2808dbe0 msgr2=0x7fde280d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 shutdown_connections 2024-04-24T23:23:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:17.959+0000 7fde2d102640 1 -- 172.21.15.26:0/4119919440 wait complete. 2024-04-24T23:23:18.375 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:23:03.164134+0000', 'modified': '2024-04-24T23:23:17.849716+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5742, 'mds_1': 5329, 'mds_2': 5081}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5081': {'gid': 5081, 'name': 'b', 'rank': 2, 'incarnation': 36, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.29:6835/1578186344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 1578186344}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 1578186344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5329': {'gid': 5329, 'name': 'd', 'rank': 1, 'incarnation': 34, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.29:6834/243988267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 243988267}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 243988267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5742': {'gid': 5742, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6836/730964888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 730964888}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 730964888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5742, 'qdb_cluster': [5081, 5329, 5742]} 2024-04-24T23:23:18.375 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:23:18.375 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:23:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.475+0000 7fa29e928640 1 Processor -- start 2024-04-24T23:23:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.475+0000 7fa29e928640 1 -- start start 2024-04-24T23:23:18.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.476+0000 7fa29e928640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa29815e570 0x7fa2980d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.476+0000 7fa29e928640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2980b3130 0x7fa2980d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.476+0000 7fa29e928640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 0x7fa2980d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.476+0000 7fa29e928640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa2980d1ab0 con 0x7fa2980b3130 2024-04-24T23:23:18.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.476+0000 7fa29e928640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa2980d2ae0 con 0x7fa29815e570 2024-04-24T23:23:18.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.476+0000 7fa29e928640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa2980d32f0 con 0x7fa2980d22e0 2024-04-24T23:23:18.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.476+0000 7fa2977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2980b3130 0x7fa2980d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.476+0000 7fa2977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2980b3130 0x7fa2980d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60454/0 (socket says 172.21.15.26:60454) 2024-04-24T23:23:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.476+0000 7fa2977fe640 1 -- 172.21.15.26:0/3212379414 learned_addr learned my addr 172.21.15.26:0/3212379414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.477+0000 7fa29cec5640 1 --2- 172.21.15.26:0/3212379414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 0x7fa2980d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.477+0000 7fa297fff640 1 --2- 172.21.15.26:0/3212379414 >> v2:172.21.15.29:3300/0 conn(0x7fa29815e570 0x7fa2980d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.477+0000 7fa2977fe640 1 -- 172.21.15.26:0/3212379414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 msgr2=0x7fa2980d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.477+0000 7fa2977fe640 1 --2- 172.21.15.26:0/3212379414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 0x7fa2980d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.477+0000 7fa2977fe640 1 -- 172.21.15.26:0/3212379414 >> v2:172.21.15.29:3300/0 conn(0x7fa29815e570 msgr2=0x7fa2980d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.477+0000 7fa2977fe640 1 --2- 172.21.15.26:0/3212379414 >> v2:172.21.15.29:3300/0 conn(0x7fa29815e570 0x7fa2980d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.477+0000 7fa2977fe640 1 -- 172.21.15.26:0/3212379414 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2980d0270 con 0x7fa2980b3130 2024-04-24T23:23:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.477+0000 7fa2977fe640 1 --2- 172.21.15.26:0/3212379414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2980b3130 0x7fa2980d46a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa2980d52d0 tx=0x7fa28c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa7f7a17a489ead server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa296ffd640 1 -- 172.21.15.26:0/3212379414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c037070 con 0x7fa2980b3130 2024-04-24T23:23:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa296ffd640 1 -- 172.21.15.26:0/3212379414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa28c032440 con 0x7fa2980b3130 2024-04-24T23:23:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa296ffd640 1 -- 172.21.15.26:0/3212379414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c032810 con 0x7fa2980b3130 2024-04-24T23:23:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa29e928640 1 -- 172.21.15.26:0/3212379414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2980b3130 msgr2=0x7fa2980d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa29e928640 1 --2- 172.21.15.26:0/3212379414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2980b3130 0x7fa2980d46a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa2980d52d0 tx=0x7fa28c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa29e928640 1 -- 172.21.15.26:0/3212379414 shutdown_connections 2024-04-24T23:23:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa29e928640 1 --2- 172.21.15.26:0/3212379414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2980b3130 0x7fa2980d46a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa29e928640 1 --2- 172.21.15.26:0/3212379414 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 0x7fa2980d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa29e928640 1 --2- 172.21.15.26:0/3212379414 >> v2:172.21.15.29:3300/0 conn(0x7fa29815e570 0x7fa2980d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa29e928640 1 -- 172.21.15.26:0/3212379414 >> 172.21.15.26:0/3212379414 conn(0x7fa29808dbe0 msgr2=0x7fa29809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa29e928640 1 -- 172.21.15.26:0/3212379414 shutdown_connections 2024-04-24T23:23:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.478+0000 7fa29e928640 1 -- 172.21.15.26:0/3212379414 wait complete. 2024-04-24T23:23:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.479+0000 7fa29e928640 1 Processor -- start 2024-04-24T23:23:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.480+0000 7fa29e928640 1 -- start start 2024-04-24T23:23:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.480+0000 7fa29e928640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2980b3130 0x7fa29814b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.480+0000 7fa29e928640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 0x7fa29815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.480+0000 7fa29e928640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29815e570 0x7fa29815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.480+0000 7fa29e928640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa2980cfa70 con 0x7fa29815e570 2024-04-24T23:23:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.480+0000 7fa29e928640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa2980d3010 con 0x7fa2980b3130 2024-04-24T23:23:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.480+0000 7fa29e928640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa2980cfd50 con 0x7fa2980d22e0 2024-04-24T23:23:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.480+0000 7fa29cec5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29815e570 0x7fa29815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.480+0000 7fa29cec5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29815e570 0x7fa29815d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60456/0 (socket says 172.21.15.26:60456) 2024-04-24T23:23:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa29cec5640 1 -- 172.21.15.26:0/1753427903 learned_addr learned my addr 172.21.15.26:0/1753427903 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa297fff640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2980b3130 0x7fa29814b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa2977fe640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 0x7fa29815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa29cec5640 1 -- 172.21.15.26:0/1753427903 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 msgr2=0x7fa29815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa29cec5640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 0x7fa29815cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa29cec5640 1 -- 172.21.15.26:0/1753427903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2980b3130 msgr2=0x7fa29814b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa29cec5640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2980b3130 0x7fa29814b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa29cec5640 1 -- 172.21.15.26:0/1753427903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29814bd20 con 0x7fa29815e570 2024-04-24T23:23:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa29cec5640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29815e570 0x7fa29815d2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa290007e60 tx=0x7fa29000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa294ff9640 1 -- 172.21.15.26:0/1753427903 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290014620 con 0x7fa29815e570 2024-04-24T23:23:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa294ff9640 1 -- 172.21.15.26:0/1753427903 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa290019070 con 0x7fa29815e570 2024-04-24T23:23:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa29e928640 1 -- 172.21.15.26:0/1753427903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2981f5290 con 0x7fa29815e570 2024-04-24T23:23:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa294ff9640 1 -- 172.21.15.26:0/1753427903 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290014a20 con 0x7fa29815e570 2024-04-24T23:23:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.481+0000 7fa29e928640 1 -- 172.21.15.26:0/1753427903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2981f5850 con 0x7fa29815e570 2024-04-24T23:23:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.483+0000 7fa294ff9640 1 -- 172.21.15.26:0/1753427903 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa290014bc0 con 0x7fa29815e570 2024-04-24T23:23:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.483+0000 7fa294ff9640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa28406be30 0x7fa28406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.483+0000 7fa294ff9640 1 -- 172.21.15.26:0/1753427903 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fa29008b590 con 0x7fa29815e570 2024-04-24T23:23:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.483+0000 7fa297fff640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa28406be30 0x7fa28406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.484+0000 7fa297fff640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa28406be30 0x7fa28406e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa2880079a0 tx=0x7fa288008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.484+0000 7fa2a1630740 1 Processor -- start 2024-04-24T23:23:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.484+0000 7fa2a1630740 1 -- start start 2024-04-24T23:23:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee1de000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1de540 0x55c1ee1b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2a1630740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c1ee116f00 con 0x55c1ee1dec80 2024-04-24T23:23:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2a1630740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c1ee0bf750 con 0x55c1ee1fbb60 2024-04-24T23:23:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2a1630740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c1ee112ec0 con 0x55c1ee1de540 2024-04-24T23:23:18.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee1de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee1de000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60466/0 (socket says 172.21.15.26:60466) 2024-04-24T23:23:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2977fe640 1 -- 172.21.15.26:0/330324910 learned_addr learned my addr 172.21.15.26:0/330324910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa297fff640 1 --2- 172.21.15.26:0/330324910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa29cec5640 1 --2- 172.21.15.26:0/330324910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1de540 0x55c1ee1b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2977fe640 1 -- 172.21.15.26:0/330324910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1de540 msgr2=0x55c1ee1b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2977fe640 1 --2- 172.21.15.26:0/330324910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1de540 0x55c1ee1b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2977fe640 1 -- 172.21.15.26:0/330324910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1fbb60 msgr2=0x55c1ee1ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2977fe640 1 --2- 172.21.15.26:0/330324910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.485+0000 7fa2977fe640 1 -- 172.21.15.26:0/330324910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1ede2b2d0 con 0x55c1ee1dec80 2024-04-24T23:23:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2977fe640 1 --2- 172.21.15.26:0/330324910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee1de000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009960 tx=0x7fa28c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd37dc1147c560d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa27dffb640 1 -- 172.21.15.26:0/330324910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c037040 con 0x55c1ee1dec80 2024-04-24T23:23:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa27dffb640 1 -- 172.21.15.26:0/330324910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa28c0326c0 con 0x55c1ee1dec80 2024-04-24T23:23:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa27dffb640 1 -- 172.21.15.26:0/330324910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c032a70 con 0x55c1ee1dec80 2024-04-24T23:23:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2a1630740 1 -- 172.21.15.26:0/330324910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 msgr2=0x55c1ee1de000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2a1630740 1 --2- 172.21.15.26:0/330324910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee1de000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009960 tx=0x7fa28c02c000 comp rx=0 tx=0).stop 2024-04-24T23:23:18.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2a1630740 1 -- 172.21.15.26:0/330324910 shutdown_connections 2024-04-24T23:23:18.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2a1630740 1 --2- 172.21.15.26:0/330324910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1de540 0x55c1ee1b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2a1630740 1 --2- 172.21.15.26:0/330324910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee1de000 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2a1630740 1 --2- 172.21.15.26:0/330324910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2a1630740 1 -- 172.21.15.26:0/330324910 >> 172.21.15.26:0/330324910 conn(0x55c1ee1b0760 msgr2=0x55c1ee1f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2a1630740 1 -- 172.21.15.26:0/330324910 shutdown_connections 2024-04-24T23:23:18.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.486+0000 7fa2a1630740 1 -- 172.21.15.26:0/330324910 wait complete. 2024-04-24T23:23:18.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.487+0000 7fa2a1630740 1 -- start start 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.487+0000 7fa2a1630740 1 Processor -- start 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1dec80 0x55c1ee1f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 0x55c1ee1b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2a1630740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c1ee1b2910 con 0x55c1ee1fbb60 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2a1630740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c1ee0d2cf0 con 0x55c1ee1d5df0 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2a1630740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c1ee1b1d00 con 0x55c1ee1dec80 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2a1630740 10 client.0 _ll_register_callbacks cb 0x55c1ee0d26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60470/0 (socket says 172.21.15.26:60470) 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2977fe640 1 -- 172.21.15.26:0/1059873670 learned_addr learned my addr 172.21.15.26:0/1059873670 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa297fff640 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1dec80 0x55c1ee1f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa29cec5640 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 0x55c1ee1b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2977fe640 1 -- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1dec80 msgr2=0x55c1ee1f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2977fe640 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1dec80 0x55c1ee1f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2977fe640 1 -- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 msgr2=0x55c1ee1b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2977fe640 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 0x55c1ee1b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2977fe640 1 -- 172.21.15.26:0/1059873670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1edef56b0 con 0x55c1ee1fbb60 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa2977fe640 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1b2d10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa28c02a7d0 tx=0x7fa28c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65fecface02c069 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.488+0000 7fa276ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa276ffd640 1 -- 172.21.15.26:0/1059873670 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c037040 con 0x55c1ee1fbb60 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa276ffd640 1 -- 172.21.15.26:0/1059873670 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fa28c032a80 con 0x55c1ee1fbb60 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa2a1630740 1 -- 172.21.15.26:0/1059873670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28c0326c0 con 0x55c1ee1fbb60 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa276ffd640 1 -- 172.21.15.26:0/1059873670 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa28c032c80 con 0x55c1ee1fbb60 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa2a1630740 10 client.5955 fetch_fsmap learned FSMap version 37 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa2a1630740 1 -- 172.21.15.26:0/1059873670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55c1ee1e17c0 con 0x55c1ee1fbb60 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa276ffd640 1 -- 172.21.15.26:0/1059873670 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa28c0326c0 con 0x55c1ee1fbb60 2024-04-24T23:23:18.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa276ffd640 1 -- 172.21.15.26:0/1059873670 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 3534+0+0 (secure 0 0 0) 0x7fa28c054080 con 0x55c1ee1fbb60 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa2a1630740 10 client.5955 fetch_fsmap finished waiting for FSMap version 37 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa2a1630740 10 client.5955 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa2a1630740 20 client.5955 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa2a1630740 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55c1ee1f6e70 0x55c1ee1f7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa2a1630740 4 client.5955 mds_command: new command op to 5081 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa2a1630740 1 -- 172.21.15.26:0/1059873670 --> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1ee124440 con 0x55c1ee1f6e70 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.489+0000 7fa297fff640 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55c1ee1f6e70 0x55c1ee1f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.490+0000 7fa276ffd640 10 client.5955 ms_handle_connect on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.490+0000 7fa297fff640 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55c1ee1f6e70 0x55c1ee1f7250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ebfa3d7343e6d0d server_cookie=836b6e20f66b0ae3 in_seq=0 out_seq=0 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa276ffd640 1 -- 172.21.15.26:0/1059873670 <== mds.2 v2:172.21.15.29:6832/1578186344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1ee124440 con 0x55c1ee1f6e70 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa276ffd640 10 client.5955 handle_command_reply: tid=0 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 client.5955 shutdown 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 -- 172.21.15.26:0/1059873670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 msgr2=0x55c1ee1b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1b2d10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa28c02a7d0 tx=0x7fa28c004730 comp rx=0 tx=0).stop 2024-04-24T23:23:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 -- 172.21.15.26:0/1059873670 shutdown_connections 2024-04-24T23:23:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55c1ee1f6e70 0x55c1ee1f7250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 0x55c1ee1b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1b2d10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa276ffd640 0 client.5955 ms_handle_reset on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 --2- 172.21.15.26:0/1059873670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1dec80 0x55c1ee1f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 -- 172.21.15.26:0/1059873670 >> 172.21.15.26:0/1059873670 conn(0x55c1ee1de540 msgr2=0x55c1ee1f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 -- 172.21.15.26:0/1059873670 shutdown_connections 2024-04-24T23:23:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 1 -- 172.21.15.26:0/1059873670 wait complete. 2024-04-24T23:23:18.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.491+0000 7fa2a1630740 20 client.5955 trim_cache size 0 max 16384 2024-04-24T23:23:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.509+0000 7fa2a1630740 1 Processor -- start 2024-04-24T23:23:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.509+0000 7fa2a1630740 1 -- start start 2024-04-24T23:23:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1d5df0 0x55c1ee1b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1dec80 0x55c1ee1b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa2a1630740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa2981ff9a0 con 0x55c1ee1fbb60 2024-04-24T23:23:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa2a1630740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa298200070 con 0x55c1ee1dec80 2024-04-24T23:23:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa2a1630740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa2981feac0 con 0x55c1ee1d5df0 2024-04-24T23:23:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa29cec5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa29cec5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60472/0 (socket says 172.21.15.26:60472) 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa29cec5640 1 -- 172.21.15.26:0/704176665 learned_addr learned my addr 172.21.15.26:0/704176665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa297fff640 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1d5df0 0x55c1ee1b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa2977fe640 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1dec80 0x55c1ee1b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa29cec5640 1 -- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1d5df0 msgr2=0x55c1ee1b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa29cec5640 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1d5df0 0x55c1ee1b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa29cec5640 1 -- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1dec80 msgr2=0x55c1ee1b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa29cec5640 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1dec80 0x55c1ee1b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa29cec5640 1 -- 172.21.15.26:0/704176665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1ee1f7350 con 0x55c1ee1fbb60 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa297fff640 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1d5df0 0x55c1ee1b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa2977fe640 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1dec80 0x55c1ee1b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa29cec5640 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ad250 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa290089a60 tx=0x7fa290060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d87f8b4fdba50c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa27d7fa640 1 -- 172.21.15.26:0/704176665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900600b0 con 0x55c1ee1fbb60 2024-04-24T23:23:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa27d7fa640 1 -- 172.21.15.26:0/704176665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa290057bf0 con 0x55c1ee1fbb60 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.510+0000 7fa27d7fa640 1 -- 172.21.15.26:0/704176665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290061f90 con 0x55c1ee1fbb60 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 -- 172.21.15.26:0/704176665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 msgr2=0x55c1ee1ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ad250 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa290089a60 tx=0x7fa290060040 comp rx=0 tx=0).stop 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 -- 172.21.15.26:0/704176665 shutdown_connections 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1fbb60 0x55c1ee1ad250 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1dec80 0x55c1ee1b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 --2- 172.21.15.26:0/704176665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1d5df0 0x55c1ee1b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 -- 172.21.15.26:0/704176665 >> 172.21.15.26:0/704176665 conn(0x55c1ee1de540 msgr2=0x55c1ee1a9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 -- 172.21.15.26:0/704176665 shutdown_connections 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 -- 172.21.15.26:0/704176665 wait complete. 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.511+0000 7fa2a1630740 1 -- start start 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.512+0000 7fa2a1630740 1 Processor -- start 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.512+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 0x55c1ee21b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.512+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee21bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.512+0000 7fa2a1630740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1fbb60 0x55c1ee1df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.512+0000 7fa2a1630740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c1ee1b2910 con 0x55c1ee1dec80 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.512+0000 7fa2a1630740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c1ee0d2cf0 con 0x55c1ee1d5df0 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.512+0000 7fa2a1630740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c1ee1d9bb0 con 0x55c1ee1fbb60 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2a1630740 10 client.0 _ll_register_callbacks cb 0x7fa2900099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee21bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee21bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60474/0 (socket says 172.21.15.26:60474) 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2977fe640 1 -- 172.21.15.26:0/705101214 learned_addr learned my addr 172.21.15.26:0/705101214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa297fff640 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 0x55c1ee21b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa29cec5640 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1fbb60 0x55c1ee1df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2977fe640 1 -- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1fbb60 msgr2=0x55c1ee1df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2977fe640 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1fbb60 0x55c1ee1df8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2977fe640 1 -- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 msgr2=0x55c1ee21b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2977fe640 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 0x55c1ee21b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2977fe640 1 -- 172.21.15.26:0/705101214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1ee1fd1b0 con 0x55c1ee1dec80 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa29cec5640 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1fbb60 0x55c1ee1df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2977fe640 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee21bf10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa28c02a7d0 tx=0x7fa28c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb8c2d0191cd723 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa276ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa276ffd640 1 -- 172.21.15.26:0/705101214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c057030 con 0x55c1ee1dec80 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa276ffd640 1 -- 172.21.15.26:0/705101214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fa28c032a80 con 0x55c1ee1dec80 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2a1630740 1 -- 172.21.15.26:0/705101214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa290057bf0 con 0x55c1ee1dec80 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa276ffd640 1 -- 172.21.15.26:0/705101214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa28c037070 con 0x55c1ee1dec80 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2a1630740 10 client.5961 fetch_fsmap learned FSMap version 37 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.513+0000 7fa2a1630740 1 -- 172.21.15.26:0/705101214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55c1ee1b3230 con 0x55c1ee1dec80 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa276ffd640 1 -- 172.21.15.26:0/705101214 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa290057bf0 con 0x55c1ee1dec80 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa276ffd640 1 -- 172.21.15.26:0/705101214 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 3534+0+0 (secure 0 0 0) 0x7fa28c0038f0 con 0x55c1ee1dec80 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa2a1630740 10 client.5961 fetch_fsmap finished waiting for FSMap version 37 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa2a1630740 10 client.5961 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:23:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa2a1630740 20 client.5961 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa2a1630740 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55c1ee1d82e0 0x55c1ee1d86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa2a1630740 4 client.5961 mds_command: new command op to 5081 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa2a1630740 1 -- 172.21.15.26:0/705101214 --> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] -- command(tid 0: {"prefix": "status"}) -- 0x55c1ee1f0650 con 0x55c1ee1d82e0 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa297fff640 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55c1ee1d82e0 0x55c1ee1d86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa276ffd640 10 client.5961 ms_handle_connect on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.514+0000 7fa297fff640 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55c1ee1d82e0 0x55c1ee1d86c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=821c5bc39f29e204 server_cookie=41a26bce961f1f1d in_seq=0 out_seq=0 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa276ffd640 1 -- 172.21.15.26:0/705101214 <== mds.2 v2:172.21.15.29:6832/1578186344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55c1ee1f0650 con 0x55c1ee1d82e0 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa276ffd640 10 client.5961 handle_command_reply: tid=0 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 client.5961 shutdown 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 -- 172.21.15.26:0/705101214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 msgr2=0x55c1ee21bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee21bf10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa28c02a7d0 tx=0x7fa28c02e000 comp rx=0 tx=0).stop 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 -- 172.21.15.26:0/705101214 shutdown_connections 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55c1ee1d82e0 0x55c1ee1d86c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c1ee1fbb60 0x55c1ee1df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1ee1dec80 0x55c1ee21bf10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa276ffd640 0 client.5961 ms_handle_reset on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 --2- 172.21.15.26:0/705101214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1ee1d5df0 0x55c1ee21b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 -- 172.21.15.26:0/705101214 >> 172.21.15.26:0/705101214 conn(0x55c1ee1de540 msgr2=0x55c1ee1f6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 -- 172.21.15.26:0/705101214 shutdown_connections 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 1 -- 172.21.15.26:0/705101214 wait complete. 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa2a1630740 20 client.5961 trim_cache size 0 max 16384 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5081, 2024-04-24T23:23:18.533 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:23:18.534 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:23:18.534 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:23:18.534 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 1.665219936, 2024-04-24T23:23:18.534 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 37, 2024-04-24T23:23:18.534 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 44, 2024-04-24T23:23:18.534 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 44, 2024-04-24T23:23:18.534 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 19.703992027999998 2024-04-24T23:23:18.534 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 -- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa28406be30 msgr2=0x7fa28406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa28406be30 0x7fa28406e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa2880079a0 tx=0x7fa288008040 comp rx=0 tx=0).stop 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 -- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29815e570 msgr2=0x7fa29815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29815e570 0x7fa29815d2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa290007e60 tx=0x7fa29000e4b0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 -- 172.21.15.26:0/1753427903 shutdown_connections 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa28406be30 0x7fa28406e2f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa29815e570 0x7fa29815d2f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa2980d22e0 0x7fa29815cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 --2- 172.21.15.26:0/1753427903 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa2980b3130 0x7fa29814b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.515+0000 7fa29e928640 1 -- 172.21.15.26:0/1753427903 >> 172.21.15.26:0/1753427903 conn(0x7fa29808dbe0 msgr2=0x7fa29806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.516+0000 7fa29e928640 1 -- 172.21.15.26:0/1753427903 shutdown_connections 2024-04-24T23:23:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.516+0000 7fa29e928640 1 -- 172.21.15.26:0/1753427903 wait complete. 2024-04-24T23:23:18.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:23:18.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.637+0000 7f4c342f5640 1 Processor -- start 2024-04-24T23:23:18.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.637+0000 7f4c342f5640 1 -- start start 2024-04-24T23:23:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.638+0000 7f4c342f5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c2c15e570 0x7f4c2c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.638+0000 7f4c342f5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4c2c0b3130 0x7f4c2c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.638+0000 7f4c342f5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c2c0d32f0 con 0x7f4c2c0d22e0 2024-04-24T23:23:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.638+0000 7f4c342f5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4c2c0d1ab0 con 0x7f4c2c15e570 2024-04-24T23:23:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.638+0000 7f4c342f5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4c2c0d2ae0 con 0x7f4c2c0b3130 2024-04-24T23:23:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.638+0000 7f4c32091640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c2c15e570 0x7f4c2c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.638+0000 7f4c32091640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c2c15e570 0x7f4c2c0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60486/0 (socket says 172.21.15.26:60486) 2024-04-24T23:23:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.638+0000 7f4c32091640 1 -- 172.21.15.26:0/3047272120 learned_addr learned my addr 172.21.15.26:0/3047272120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.638+0000 7f4c31890640 1 --2- 172.21.15.26:0/3047272120 >> v2:172.21.15.29:3300/0 conn(0x7f4c2c0b3130 0x7f4c2c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.639+0000 7f4c32091640 1 -- 172.21.15.26:0/3047272120 >> v2:172.21.15.29:3300/0 conn(0x7f4c2c0b3130 msgr2=0x7f4c2c0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.639+0000 7f4c32091640 1 --2- 172.21.15.26:0/3047272120 >> v2:172.21.15.29:3300/0 conn(0x7f4c2c0b3130 0x7f4c2c0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.639+0000 7f4c32091640 1 -- 172.21.15.26:0/3047272120 >> v1:172.21.15.29:6789/0 conn(0x7f4c2c0d22e0 legacy=0x7f4c2c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.639+0000 7f4c32091640 1 -- 172.21.15.26:0/3047272120 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2c0d0270 con 0x7f4c2c15e570 2024-04-24T23:23:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.639+0000 7f4c31890640 1 --2- 172.21.15.26:0/3047272120 >> v2:172.21.15.29:3300/0 conn(0x7f4c2c0b3130 0x7f4c2c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:18.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.639+0000 7f4c32091640 1 --2- 172.21.15.26:0/3047272120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c15e570 0x7f4c2c0d4160 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c08cae0 tx=0x7f4c1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18308642d2bb72d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.639+0000 7f4c3108f640 1 -- 172.21.15.26:0/3047272120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c037070 con 0x7f4c2c15e570 2024-04-24T23:23:18.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c3108f640 1 -- 172.21.15.26:0/3047272120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1c032440 con 0x7f4c2c15e570 2024-04-24T23:23:18.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c3108f640 1 -- 172.21.15.26:0/3047272120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c032810 con 0x7f4c2c15e570 2024-04-24T23:23:18.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c342f5640 1 -- 172.21.15.26:0/3047272120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c15e570 msgr2=0x7f4c2c0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c342f5640 1 --2- 172.21.15.26:0/3047272120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c15e570 0x7f4c2c0d4160 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c08cae0 tx=0x7f4c1c02a270 comp rx=0 tx=0).stop 2024-04-24T23:23:18.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c342f5640 1 -- 172.21.15.26:0/3047272120 shutdown_connections 2024-04-24T23:23:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c342f5640 1 --2- 172.21.15.26:0/3047272120 >> v2:172.21.15.29:3300/0 conn(0x7f4c2c0b3130 0x7f4c2c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c342f5640 1 --2- 172.21.15.26:0/3047272120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c15e570 0x7f4c2c0d4160 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c342f5640 1 -- 172.21.15.26:0/3047272120 >> 172.21.15.26:0/3047272120 conn(0x7f4c2c08dbe0 msgr2=0x7f4c2c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c342f5640 1 -- 172.21.15.26:0/3047272120 shutdown_connections 2024-04-24T23:23:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.640+0000 7f4c342f5640 1 -- 172.21.15.26:0/3047272120 wait complete. 2024-04-24T23:23:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.641+0000 7f4c342f5640 1 Processor -- start 2024-04-24T23:23:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.641+0000 7f4c342f5640 1 -- start start 2024-04-24T23:23:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c342f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c0b3130 0x7f4c2c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c342f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c2c0d22e0 0x7f4c2c0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c342f5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15e570 0x7f4c2c0a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c342f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c0cfa70 con 0x7f4c2c0b3130 2024-04-24T23:23:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c342f5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c0d3010 con 0x7f4c2c15e570 2024-04-24T23:23:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c342f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2c0cfd50 con 0x7f4c2c0d22e0 2024-04-24T23:23:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c32091640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c0b3130 0x7f4c2c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c32091640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c0b3130 0x7f4c2c15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60500/0 (socket says 172.21.15.26:60500) 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c32091640 1 -- 172.21.15.26:0/3205095152 learned_addr learned my addr 172.21.15.26:0/3205095152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c31890640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c2c0d22e0 0x7f4c2c0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c32892640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15e570 0x7f4c2c0a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c32091640 1 -- 172.21.15.26:0/3205095152 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c2c0d22e0 msgr2=0x7f4c2c0cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c32091640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c2c0d22e0 0x7f4c2c0cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c32091640 1 -- 172.21.15.26:0/3205095152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15e570 msgr2=0x7f4c2c0a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c32091640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15e570 0x7f4c2c0a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.642+0000 7f4c32091640 1 -- 172.21.15.26:0/3205095152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2c09e250 con 0x7f4c2c0b3130 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.643+0000 7f4c32091640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c0b3130 0x7f4c2c15c550 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c000c00 tx=0x7f4c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.643+0000 7f4c22ffd640 1 -- 172.21.15.26:0/3205095152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c041020 con 0x7f4c2c0b3130 2024-04-24T23:23:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.643+0000 7f4c22ffd640 1 -- 172.21.15.26:0/3205095152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1c0306c0 con 0x7f4c2c0b3130 2024-04-24T23:23:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.643+0000 7f4c342f5640 1 -- 172.21.15.26:0/3205095152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2c09e530 con 0x7f4c2c0b3130 2024-04-24T23:23:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.643+0000 7f4c22ffd640 1 -- 172.21.15.26:0/3205095152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c037040 con 0x7f4c2c0b3130 2024-04-24T23:23:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.643+0000 7f4c342f5640 1 -- 172.21.15.26:0/3205095152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2c0cd460 con 0x7f4c2c0b3130 2024-04-24T23:23:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.645+0000 7f4c22ffd640 1 -- 172.21.15.26:0/3205095152 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c1c044020 con 0x7f4c2c0b3130 2024-04-24T23:23:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.645+0000 7f4c22ffd640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c1006be30 0x7f4c1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.646+0000 7f4c22ffd640 1 -- 172.21.15.26:0/3205095152 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f4c1c0ab6a0 con 0x7f4c2c0b3130 2024-04-24T23:23:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.646+0000 7f4c31890640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c1006be30 0x7f4c1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.646+0000 7f4c31890640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c1006be30 0x7f4c1006e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c1f7690 tx=0x7f4c28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.646+0000 7f4c37002740 1 Processor -- start 2024-04-24T23:23:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.646+0000 7f4c37002740 1 -- start start 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c37002740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a164f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c37002740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a46c80 0x55c826a46000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c37002740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46540 0x55c826a1b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c37002740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c82697ef00 con 0x55c826a46c80 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c37002740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c826927750 con 0x55c826a63b60 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c37002740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c82697aec0 con 0x55c826a46540 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c31890640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a46c80 0x55c826a46000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c31890640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a46c80 0x55c826a46000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60508/0 (socket says 172.21.15.26:60508) 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c31890640 1 -- 172.21.15.26:0/3831841854 learned_addr learned my addr 172.21.15.26:0/3831841854 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c32091640 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a164f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.647+0000 7f4c32892640 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46540 0x55c826a1b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c31890640 1 -- 172.21.15.26:0/3831841854 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46540 msgr2=0x55c826a1b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c31890640 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46540 0x55c826a1b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c31890640 1 -- 172.21.15.26:0/3831841854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 msgr2=0x55c826a164f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c31890640 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a164f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c31890640 1 -- 172.21.15.26:0/3831841854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8266932d0 con 0x55c826a46c80 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c32892640 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46540 0x55c826a1b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c31890640 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a46c80 0x55c826a46000 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4c280108f0 tx=0x7f4c28013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2255467c8fb8bc01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c0ffff640 1 -- 172.21.15.26:0/3831841854 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2801c050 con 0x55c826a46c80 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c0ffff640 1 -- 172.21.15.26:0/3831841854 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2801d040 con 0x55c826a46c80 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.648+0000 7f4c0ffff640 1 -- 172.21.15.26:0/3831841854 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800f040 con 0x55c826a46c80 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 -- 172.21.15.26:0/3831841854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a46c80 msgr2=0x55c826a46000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a46c80 0x55c826a46000 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4c280108f0 tx=0x7f4c28013000 comp rx=0 tx=0).stop 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 -- 172.21.15.26:0/3831841854 shutdown_connections 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46540 0x55c826a1b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a46c80 0x55c826a46000 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 --2- 172.21.15.26:0/3831841854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 -- 172.21.15.26:0/3831841854 >> 172.21.15.26:0/3831841854 conn(0x55c826a18760 msgr2=0x55c826a61000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 -- 172.21.15.26:0/3831841854 shutdown_connections 2024-04-24T23:23:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 -- 172.21.15.26:0/3831841854 wait complete. 2024-04-24T23:23:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 -- start start 2024-04-24T23:23:18.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.649+0000 7f4c37002740 1 Processor -- start 2024-04-24T23:23:18.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c37002740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 0x55c826a70e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c37002740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c37002740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a15250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c37002740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c826a1a910 con 0x55c826a73f00 2024-04-24T23:23:18.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c37002740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c82693acf0 con 0x55c826a63b60 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c37002740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c826a19d00 con 0x55c826a46c80 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c37002740 10 client.0 _ll_register_callbacks cb 0x55c82693a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c32892640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a15250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c32892640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a15250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60520/0 (socket says 172.21.15.26:60520) 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.650+0000 7f4c32892640 1 -- 172.21.15.26:0/388421443 learned_addr learned my addr 172.21.15.26:0/388421443 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c32091640 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 0x55c826a70e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c31890640 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c32892640 1 -- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 msgr2=0x55c826a70e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c32892640 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 0x55c826a70e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c32892640 1 -- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 msgr2=0x55c826a623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c32892640 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c32892640 1 -- 172.21.15.26:0/388421443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82675d6b0 con 0x55c826a73f00 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c32892640 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a15250 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005980 tx=0x7f4c24001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1440e38f12e232 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/388421443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24013570 con 0x55c826a73f00 2024-04-24T23:23:18.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/388421443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f4c24013710 con 0x55c826a73f00 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c37002740 1 -- 172.21.15.26:0/388421443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2801d040 con 0x55c826a73f00 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/388421443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4c24021050 con 0x55c826a73f00 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c37002740 10 client.5973 fetch_fsmap learned FSMap version 37 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c37002740 1 -- 172.21.15.26:0/388421443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55c826a57cc0 con 0x55c826a73f00 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.651+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/388421443 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2801d040 con 0x55c826a73f00 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/388421443 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 3534+0+0 (secure 0 0 0) 0x7f4c24024020 con 0x55c826a73f00 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c37002740 10 client.5973 fetch_fsmap finished waiting for FSMap version 37 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c37002740 10 client.5973 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c37002740 20 client.5973 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c37002740 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55c826a58430 0x55c826a74aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c37002740 4 client.5973 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c37002740 1 -- 172.21.15.26:0/388421443 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c82698c440 con 0x55c826a58430 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c32091640 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55c826a58430 0x55c826a74aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c32091640 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55c826a58430 0x55c826a74aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1025cc1a0293b50 server_cookie=f72ea952d4263aad in_seq=0 out_seq=0 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.652+0000 7f4c0d7fa640 10 client.5973 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.653+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/388421443 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c82698c440 con 0x55c826a58430 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.653+0000 7f4c0d7fa640 10 client.5973 handle_command_reply: tid=0 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.653+0000 7f4c37002740 1 client.5973 shutdown 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.653+0000 7f4c37002740 1 -- 172.21.15.26:0/388421443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 msgr2=0x55c826a15250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.653+0000 7f4c37002740 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a15250 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005980 tx=0x7f4c24001910 comp rx=0 tx=0).stop 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c37002740 1 -- 172.21.15.26:0/388421443 shutdown_connections 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c37002740 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55c826a58430 0x55c826a74aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c37002740 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a15250 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c37002740 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c0d7fa640 0 client.5973 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c37002740 1 --2- 172.21.15.26:0/388421443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 0x55c826a70e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c37002740 1 -- 172.21.15.26:0/388421443 >> 172.21.15.26:0/388421443 conn(0x55c826a46540 msgr2=0x55c826a43450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c37002740 1 -- 172.21.15.26:0/388421443 shutdown_connections 2024-04-24T23:23:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c37002740 1 -- 172.21.15.26:0/388421443 wait complete. 2024-04-24T23:23:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.654+0000 7f4c37002740 20 client.5973 trim_cache size 0 max 16384 2024-04-24T23:23:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.672+0000 7f4c37002740 1 Processor -- start 2024-04-24T23:23:18.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.672+0000 7f4c37002740 1 -- start start 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c37002740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a46c80 0x55c826a1bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c37002740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a63b60 0x55c826a83890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c37002740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c37002740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1ff9b0 con 0x55c826a73f00 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c37002740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c200040 con 0x55c826a46c80 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c37002740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2c0cb3d0 con 0x55c826a63b60 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32892640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32892640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60530/0 (socket says 172.21.15.26:60530) 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32892640 1 -- 172.21.15.26:0/2258815219 learned_addr learned my addr 172.21.15.26:0/2258815219 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c31890640 1 --2- 172.21.15.26:0/2258815219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a63b60 0x55c826a83890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32892640 1 -- 172.21.15.26:0/2258815219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a63b60 msgr2=0x55c826a83890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32091640 1 --2- 172.21.15.26:0/2258815219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a46c80 0x55c826a1bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32892640 1 --2- 172.21.15.26:0/2258815219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a63b60 0x55c826a83890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32892640 1 -- 172.21.15.26:0/2258815219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a46c80 msgr2=0x55c826a1bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32892640 1 --2- 172.21.15.26:0/2258815219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a46c80 0x55c826a1bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32892640 1 -- 172.21.15.26:0/2258815219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c826a198a0 con 0x55c826a73f00 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c32892640 1 --2- 172.21.15.26:0/2258815219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4c2401f040 tx=0x7f4c2400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312f72c27f716979 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c0f7fe640 1 -- 172.21.15.26:0/2258815219 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c240139b0 con 0x55c826a73f00 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c0f7fe640 1 -- 172.21.15.26:0/2258815219 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c24029070 con 0x55c826a73f00 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.673+0000 7f4c0f7fe640 1 -- 172.21.15.26:0/2258815219 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24013e90 con 0x55c826a73f00 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.674+0000 7f4c37002740 1 -- 172.21.15.26:0/2258815219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 msgr2=0x55c826a83dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.674+0000 7f4c37002740 1 --2- 172.21.15.26:0/2258815219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4c2401f040 tx=0x7f4c2400e000 comp rx=0 tx=0).stop 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.674+0000 7f4c37002740 1 -- 172.21.15.26:0/2258815219 shutdown_connections 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.674+0000 7f4c37002740 1 --2- 172.21.15.26:0/2258815219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.674+0000 7f4c37002740 1 --2- 172.21.15.26:0/2258815219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a63b60 0x55c826a83890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.674+0000 7f4c37002740 1 --2- 172.21.15.26:0/2258815219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a46c80 0x55c826a1bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.674+0000 7f4c37002740 1 -- 172.21.15.26:0/2258815219 >> 172.21.15.26:0/2258815219 conn(0x55c826a46540 msgr2=0x55c826a60420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.674+0000 7f4c37002740 1 -- 172.21.15.26:0/2258815219 shutdown_connections 2024-04-24T23:23:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.674+0000 7f4c37002740 1 -- 172.21.15.26:0/2258815219 wait complete. 2024-04-24T23:23:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.675+0000 7f4c37002740 1 -- start start 2024-04-24T23:23:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.675+0000 7f4c37002740 1 Processor -- start 2024-04-24T23:23:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c37002740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 0x55c826ac09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c37002740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a83890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c37002740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c37002740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c826a582b0 con 0x55c826a73f00 2024-04-24T23:23:18.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c37002740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c826a48440 con 0x55c826a63b60 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c37002740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c826a58900 con 0x55c826a46c80 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c37002740 10 client.0 _ll_register_callbacks cb 0x7f4c24005530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c32892640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c32892640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60532/0 (socket says 172.21.15.26:60532) 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c32892640 1 -- 172.21.15.26:0/1713493543 learned_addr learned my addr 172.21.15.26:0/1713493543 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c32091640 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 0x55c826ac09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c31890640 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a83890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c32892640 1 -- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 msgr2=0x55c826ac09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c32892640 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 0x55c826ac09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c32892640 1 -- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 msgr2=0x55c826a83890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c32892640 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a83890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.676+0000 7f4c32892640 1 -- 172.21.15.26:0/1713493543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c826a709b0 con 0x55c826a73f00 2024-04-24T23:23:18.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c32892640 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4c24000f80 tx=0x7f4c240014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6221476968fcda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/1713493543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24035060 con 0x55c826a73f00 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c37002740 1 -- 172.21.15.26:0/1713493543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c24029070 con 0x55c826a73f00 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/1713493543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f4c24013780 con 0x55c826a73f00 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/1713493543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4c2403b050 con 0x55c826a73f00 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c37002740 10 client.5979 fetch_fsmap learned FSMap version 37 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c37002740 1 -- 172.21.15.26:0/1713493543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55c826a84fd0 con 0x55c826a73f00 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/1713493543 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c24029070 con 0x55c826a73f00 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/1713493543 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 3534+0+0 (secure 0 0 0) 0x7f4c2403e020 con 0x55c826a73f00 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c37002740 10 client.5979 fetch_fsmap finished waiting for FSMap version 37 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c37002740 10 client.5979 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c37002740 20 client.5979 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c37002740 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55c826ad4c30 0x55c826a60bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c37002740 4 client.5979 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:23:18.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c37002740 1 -- 172.21.15.26:0/1713493543 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "status"}) -- 0x55c826986790 con 0x55c826ad4c30 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.677+0000 7f4c32091640 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55c826ad4c30 0x55c826a60bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c32091640 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55c826ad4c30 0x55c826a60bf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f00f1c4d1fdd5da2 server_cookie=2438094b24f4b921 in_seq=0 out_seq=0 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c0d7fa640 10 client.5979 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c0d7fa640 1 -- 172.21.15.26:0/1713493543 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55c826986790 con 0x55c826ad4c30 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c0d7fa640 10 client.5979 handle_command_reply: tid=0 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 client.5979 shutdown 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 -- 172.21.15.26:0/1713493543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 msgr2=0x55c826a83dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4c24000f80 tx=0x7f4c240014a0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 -- 172.21.15.26:0/1713493543 shutdown_connections 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55c826ad4c30 0x55c826a60bf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c826a73f00 0x55c826a83dd0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c826a63b60 0x55c826a83890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 --2- 172.21.15.26:0/1713493543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c826a46c80 0x55c826ac09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c0d7fa640 0 client.5979 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 -- 172.21.15.26:0/1713493543 >> 172.21.15.26:0/1713493543 conn(0x55c826a46540 msgr2=0x55c826a1bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 -- 172.21.15.26:0/1713493543 shutdown_connections 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 1 -- 172.21.15.26:0/1713493543 wait complete. 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.678+0000 7f4c37002740 20 client.5979 trim_cache size 0 max 16384 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5329, 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:23:18.685 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:23:18.686 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 2.8341837810000001, 2024-04-24T23:23:18.686 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 37, 2024-04-24T23:23:18.686 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 44, 2024-04-24T23:23:18.686 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 44, 2024-04-24T23:23:18.686 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 19.86757532 2024-04-24T23:23:18.686 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 -- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c1006be30 msgr2=0x7f4c1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c1006be30 0x7f4c1006e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c1f7690 tx=0x7f4c28008040 comp rx=0 tx=0).stop 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 -- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c0b3130 msgr2=0x7f4c2c15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c0b3130 0x7f4c2c15c550 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c000c00 tx=0x7f4c1c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 -- 172.21.15.26:0/3205095152 shutdown_connections 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4c1006be30 0x7f4c1006e2f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c2c15e570 0x7f4c2c0a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4c2c0d22e0 0x7f4c2c0cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 --2- 172.21.15.26:0/3205095152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c2c0b3130 0x7f4c2c15c550 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 -- 172.21.15.26:0/3205095152 >> 172.21.15.26:0/3205095152 conn(0x7f4c2c08dbe0 msgr2=0x7f4c2c0b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 -- 172.21.15.26:0/3205095152 shutdown_connections 2024-04-24T23:23:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.679+0000 7f4c342f5640 1 -- 172.21.15.26:0/3205095152 wait complete. 2024-04-24T23:23:18.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:23:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.791+0000 7f9aeb2fc640 1 Processor -- start 2024-04-24T23:23:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.791+0000 7f9aeb2fc640 1 -- start start 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9aeb2fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ae415e570 0x7f9ae40d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9aeb2fc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40b3130 0x7f9ae40d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9aeb2fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9ae40d22e0 0x7f9ae40d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9aeb2fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9ae40d1ab0 con 0x7f9ae415e570 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9aeb2fc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9ae40d2ae0 con 0x7f9ae40d22e0 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9aeb2fc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae40d32f0 con 0x7f9ae40b3130 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae9098640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ae415e570 0x7f9ae40d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae9098640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ae415e570 0x7f9ae40d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60534/0 (socket says 172.21.15.26:60534) 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae9098640 1 -- 172.21.15.26:0/461013962 learned_addr learned my addr 172.21.15.26:0/461013962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae9899640 1 --2- 172.21.15.26:0/461013962 >> v2:172.21.15.29:3300/0 conn(0x7f9ae40d22e0 0x7f9ae40d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae8897640 1 --2- 172.21.15.26:0/461013962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40b3130 0x7f9ae40d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae9098640 1 -- 172.21.15.26:0/461013962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40b3130 msgr2=0x7f9ae40d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae9098640 1 --2- 172.21.15.26:0/461013962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40b3130 0x7f9ae40d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae9098640 1 -- 172.21.15.26:0/461013962 >> v2:172.21.15.29:3300/0 conn(0x7f9ae40d22e0 msgr2=0x7f9ae40d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae9098640 1 --2- 172.21.15.26:0/461013962 >> v2:172.21.15.29:3300/0 conn(0x7f9ae40d22e0 0x7f9ae40d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae9098640 1 -- 172.21.15.26:0/461013962 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae40d0270 con 0x7f9ae415e570 2024-04-24T23:23:18.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.792+0000 7f9ae8897640 1 --2- 172.21.15.26:0/461013962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40b3130 0x7f9ae40d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.793+0000 7f9ae9098640 1 --2- 172.21.15.26:0/461013962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 0x7f9ae40d4160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40ba280 tx=0x7f9ad402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58c5f794f718d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9adbfff640 1 -- 172.21.15.26:0/461013962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4037070 con 0x7f9ae415e570 2024-04-24T23:23:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9adbfff640 1 -- 172.21.15.26:0/461013962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4032440 con 0x7f9ae415e570 2024-04-24T23:23:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9adbfff640 1 -- 172.21.15.26:0/461013962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4032810 con 0x7f9ae415e570 2024-04-24T23:23:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/461013962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 msgr2=0x7f9ae40d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/461013962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 0x7f9ae40d4160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40ba280 tx=0x7f9ad402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/461013962 shutdown_connections 2024-04-24T23:23:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/461013962 >> v2:172.21.15.29:3300/0 conn(0x7f9ae40d22e0 0x7f9ae40d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/461013962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40b3130 0x7f9ae40d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/461013962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 0x7f9ae40d4160 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/461013962 >> 172.21.15.26:0/461013962 conn(0x7f9ae408dbe0 msgr2=0x7f9ae409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/461013962 shutdown_connections 2024-04-24T23:23:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.794+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/461013962 wait complete. 2024-04-24T23:23:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.796+0000 7f9aeb2fc640 1 Processor -- start 2024-04-24T23:23:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.796+0000 7f9aeb2fc640 1 -- start start 2024-04-24T23:23:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.797+0000 7f9aeb2fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ae40b3130 0x7f9ae41f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.797+0000 7f9aeb2fc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40d22e0 0x7f9ae415e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.797+0000 7f9aeb2fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 0x7f9ae41525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.797+0000 7f9aeb2fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae40cfa70 con 0x7f9ae415e570 2024-04-24T23:23:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.797+0000 7f9aeb2fc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae40d3010 con 0x7f9ae40b3130 2024-04-24T23:23:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.797+0000 7f9aeb2fc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae40cfd50 con 0x7f9ae40d22e0 2024-04-24T23:23:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9899640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 0x7f9ae41525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9899640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 0x7f9ae41525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60548/0 (socket says 172.21.15.26:60548) 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9899640 1 -- 172.21.15.26:0/3302429321 learned_addr learned my addr 172.21.15.26:0/3302429321 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9098640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ae40b3130 0x7f9ae41f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae8897640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40d22e0 0x7f9ae415e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9899640 1 -- 172.21.15.26:0/3302429321 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40d22e0 msgr2=0x7f9ae415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9899640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40d22e0 0x7f9ae415e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9899640 1 -- 172.21.15.26:0/3302429321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ae40b3130 msgr2=0x7f9ae41f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9899640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ae40b3130 0x7f9ae41f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9899640 1 -- 172.21.15.26:0/3302429321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae415ee90 con 0x7f9ae415e570 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ae9899640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 0x7f9ae41525c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9adc007e40 tx=0x7f9adc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ad9ffb640 1 -- 172.21.15.26:0/3302429321 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc014620 con 0x7f9ae415e570 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ad9ffb640 1 -- 172.21.15.26:0/3302429321 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9adc019070 con 0x7f9ae415e570 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9ad9ffb640 1 -- 172.21.15.26:0/3302429321 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc014a20 con 0x7f9ae415e570 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.798+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/3302429321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae4161b20 con 0x7f9ae415e570 2024-04-24T23:23:18.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.799+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/3302429321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae40ccc50 con 0x7f9ae415e570 2024-04-24T23:23:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.800+0000 7f9ad9ffb640 1 -- 172.21.15.26:0/3302429321 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9adc014bc0 con 0x7f9ae415e570 2024-04-24T23:23:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.801+0000 7f9ad9ffb640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ac806be30 0x7f9ac806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.801+0000 7f9ad9ffb640 1 -- 172.21.15.26:0/3302429321 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9adc08b630 con 0x7f9ae415e570 2024-04-24T23:23:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.801+0000 7f9ae9098640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ac806be30 0x7f9ac806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.801+0000 7f9ae9098640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ac806be30 0x7f9ac806e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4034000 tx=0x7f9ad402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.802+0000 7f9aee009740 1 Processor -- start 2024-04-24T23:23:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.802+0000 7f9aee009740 1 -- start start 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9aee009740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162934f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9aee009740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162c3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9aee009740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3540 0x55b716298350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9aee009740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b7161fbf00 con 0x55b7162c3c80 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9aee009740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b7161a4750 con 0x55b7162e0b60 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9aee009740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b7161f7ec0 con 0x55b7162c3540 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae8897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae8897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60560/0 (socket says 172.21.15.26:60560) 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae9899640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3540 0x55b716298350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae8897640 1 -- 172.21.15.26:0/3457194133 learned_addr learned my addr 172.21.15.26:0/3457194133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae9098640 1 --2- 172.21.15.26:0/3457194133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae8897640 1 -- 172.21.15.26:0/3457194133 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3540 msgr2=0x55b716298350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae8897640 1 --2- 172.21.15.26:0/3457194133 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3540 0x55b716298350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae8897640 1 -- 172.21.15.26:0/3457194133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 msgr2=0x55b7162934f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae8897640 1 --2- 172.21.15.26:0/3457194133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162934f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae8897640 1 -- 172.21.15.26:0/3457194133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b715f102d0 con 0x55b7162c3c80 2024-04-24T23:23:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.803+0000 7f9ae8897640 1 --2- 172.21.15.26:0/3457194133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162c3000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009890 tx=0x7f9ae0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b02498c7a5eba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9ac6ffd640 1 -- 172.21.15.26:0/3457194133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0019020 con 0x55b7162c3c80 2024-04-24T23:23:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9ac6ffd640 1 -- 172.21.15.26:0/3457194133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae00042f0 con 0x55b7162c3c80 2024-04-24T23:23:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9ac6ffd640 1 -- 172.21.15.26:0/3457194133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000c070 con 0x55b7162c3c80 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9aee009740 1 -- 172.21.15.26:0/3457194133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 msgr2=0x55b7162c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9aee009740 1 --2- 172.21.15.26:0/3457194133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162c3000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009890 tx=0x7f9ae0009d50 comp rx=0 tx=0).stop 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9aee009740 1 -- 172.21.15.26:0/3457194133 shutdown_connections 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9aee009740 1 --2- 172.21.15.26:0/3457194133 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3540 0x55b716298350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9aee009740 1 --2- 172.21.15.26:0/3457194133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162c3000 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9aee009740 1 --2- 172.21.15.26:0/3457194133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162934f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9aee009740 1 -- 172.21.15.26:0/3457194133 >> 172.21.15.26:0/3457194133 conn(0x55b716295760 msgr2=0x55b7162de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9aee009740 1 -- 172.21.15.26:0/3457194133 shutdown_connections 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.804+0000 7f9aee009740 1 -- 172.21.15.26:0/3457194133 wait complete. 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.805+0000 7f9aee009740 1 -- start start 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.805+0000 7f9aee009740 1 Processor -- start 2024-04-24T23:23:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9aee009740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9aee009740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162dcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9aee009740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b716297d10 0x55b716298190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9aee009740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b716297910 con 0x55b7162c3c80 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9aee009740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b7161b7cf0 con 0x55b7162e0b60 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9aee009740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b716296d00 con 0x55b716297d10 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9aee009740 10 client.0 _ll_register_callbacks cb 0x55b7161b76e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9ae9098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9ae9098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162f0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60568/0 (socket says 172.21.15.26:60568) 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9ae9098640 1 -- 172.21.15.26:0/1392572984 learned_addr learned my addr 172.21.15.26:0/1392572984 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.806+0000 7f9ae8897640 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162dcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9ae9899640 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b716297d10 0x55b716298190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9ae9098640 1 -- 172.21.15.26:0/1392572984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b716297d10 msgr2=0x55b716298190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9ae9098640 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b716297d10 0x55b716298190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9ae9098640 1 -- 172.21.15.26:0/1392572984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 msgr2=0x55b7162dcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9ae9098640 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162dcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9ae9098640 1 -- 172.21.15.26:0/1392572984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b715fda6b0 con 0x55b7162c3c80 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9ae9098640 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162f0fe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009960 tx=0x7f9ad4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29c7d7db576432d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9abffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9abffff640 1 -- 172.21.15.26:0/1392572984 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad404e020 con 0x55b7162c3c80 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9abffff640 1 -- 172.21.15.26:0/1392572984 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9ad4049850 con 0x55b7162c3c80 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9aee009740 1 -- 172.21.15.26:0/1392572984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae00042f0 con 0x55b7162c3c80 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9abffff640 1 -- 172.21.15.26:0/1392572984 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9ad4058050 con 0x55b7162c3c80 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9aee009740 10 client.5991 fetch_fsmap learned FSMap version 37 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.807+0000 7f9aee009740 1 -- 172.21.15.26:0/1392572984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55b7162dc4f0 con 0x55b7162c3c80 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9abffff640 1 -- 172.21.15.26:0/1392572984 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae00042f0 con 0x55b7162c3c80 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9abffff640 1 -- 172.21.15.26:0/1392572984 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 3534+0+0 (secure 0 0 0) 0x7f9ad405b020 con 0x55b7162c3c80 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9aee009740 10 client.5991 fetch_fsmap finished waiting for FSMap version 37 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9aee009740 10 client.5991 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9aee009740 20 client.5991 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9aee009740 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b7162c4c40 0x55b7162f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9aee009740 4 client.5991 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9aee009740 1 -- 172.21.15.26:0/1392572984 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b716209440 con 0x55b7162c4c40 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9ae8897640 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b7162c4c40 0x55b7162f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9ae8897640 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b7162c4c40 0x55b7162f1cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa914af0275b3586 server_cookie=629df516d6150967 in_seq=0 out_seq=0 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.808+0000 7f9abffff640 10 client.5991 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9abffff640 1 -- 172.21.15.26:0/1392572984 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b716209440 con 0x55b7162c4c40 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9abffff640 10 client.5991 handle_command_reply: tid=0 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 client.5991 shutdown 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 -- 172.21.15.26:0/1392572984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 msgr2=0x55b7162f0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162f0fe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009960 tx=0x7f9ad4004730 comp rx=0 tx=0).stop 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 -- 172.21.15.26:0/1392572984 shutdown_connections 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b7162c4c40 0x55b7162f1cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b716297d10 0x55b716298190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9abffff640 0 client.5991 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 --2- 172.21.15.26:0/1392572984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b7162c3c80 0x55b7162f0fe0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 -- 172.21.15.26:0/1392572984 >> 172.21.15.26:0/1392572984 conn(0x55b7162c3540 msgr2=0x55b7162de890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 -- 172.21.15.26:0/1392572984 shutdown_connections 2024-04-24T23:23:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 1 -- 172.21.15.26:0/1392572984 wait complete. 2024-04-24T23:23:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.809+0000 7f9aee009740 20 client.5991 trim_cache size 0 max 16384 2024-04-24T23:23:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.828+0000 7f9aee009740 1 Processor -- start 2024-04-24T23:23:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.828+0000 7f9aee009740 1 -- start start 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.828+0000 7f9aee009740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716292250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.828+0000 7f9aee009740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 0x55b716292790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.828+0000 7f9aee009740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.828+0000 7f9aee009740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae40cc080 con 0x55b716297d10 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.828+0000 7f9aee009740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae41ffef0 con 0x55b7162e0b60 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.828+0000 7f9aee009740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae40ce4f0 con 0x55b7162c3c80 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716292250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716292250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60572/0 (socket says 172.21.15.26:60572) 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9098640 1 -- 172.21.15.26:0/1550025613 learned_addr learned my addr 172.21.15.26:0/1550025613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9899640 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae8897640 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 0x55b716292790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9098640 1 -- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 msgr2=0x55b716292790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9098640 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 0x55b716292790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9098640 1 -- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 msgr2=0x55b7162d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9098640 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9098640 1 -- 172.21.15.26:0/1550025613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b716291f70 con 0x55b716297d10 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9899640 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae8897640 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 0x55b716292790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ae9098640 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716292250 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4000be0 tx=0x7f9ad404a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c652395167ec9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ac67fc640 1 -- 172.21.15.26:0/1550025613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad404e070 con 0x55b716297d10 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ac67fc640 1 -- 172.21.15.26:0/1550025613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4049850 con 0x55b716297d10 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.829+0000 7f9ac67fc640 1 -- 172.21.15.26:0/1550025613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4037040 con 0x55b716297d10 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 -- 172.21.15.26:0/1550025613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 msgr2=0x55b716292250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716292250 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4000be0 tx=0x7f9ad404a000 comp rx=0 tx=0).stop 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 -- 172.21.15.26:0/1550025613 shutdown_connections 2024-04-24T23:23:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b7162d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 0x55b716292790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 --2- 172.21.15.26:0/1550025613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716292250 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 -- 172.21.15.26:0/1550025613 >> 172.21.15.26:0/1550025613 conn(0x55b7162c3540 msgr2=0x55b7162f0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 -- 172.21.15.26:0/1550025613 shutdown_connections 2024-04-24T23:23:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 -- 172.21.15.26:0/1550025613 wait complete. 2024-04-24T23:23:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 -- start start 2024-04-24T23:23:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.830+0000 7f9aee009740 1 Processor -- start 2024-04-24T23:23:18.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9aee009740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716301530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9aee009740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 0x55b716355490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9aee009740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b716357e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9aee009740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b7162dd9d0 con 0x55b716297d10 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9aee009740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b7162dcac0 con 0x55b7162e0b60 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9aee009740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b7162c4710 con 0x55b7162c3c80 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9aee009740 10 client.0 _ll_register_callbacks cb 0x7f9ad4032af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9ae9098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716301530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9ae9098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716301530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60576/0 (socket says 172.21.15.26:60576) 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.831+0000 7f9ae9098640 1 -- 172.21.15.26:0/2502603712 learned_addr learned my addr 172.21.15.26:0/2502603712 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9ae8897640 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 0x55b716355490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9ae9899640 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b716357e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9ae9098640 1 -- 172.21.15.26:0/2502603712 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 msgr2=0x55b716355490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9ae9098640 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 0x55b716355490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9ae9098640 1 -- 172.21.15.26:0/2502603712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 msgr2=0x55b716357e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9ae9098640 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b716357e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9ae9098640 1 -- 172.21.15.26:0/2502603712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7162d9ed0 con 0x55b716297d10 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9ae9098640 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716301530 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40090b0 tx=0x7f9ad4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651c35f8a9c7734a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9abffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9abffff640 1 -- 172.21.15.26:0/2502603712 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4037070 con 0x55b716297d10 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9abffff640 1 -- 172.21.15.26:0/2502603712 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9ad403c650 con 0x55b716297d10 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9aee009740 1 -- 172.21.15.26:0/2502603712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad4049850 con 0x55b716297d10 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9abffff640 1 -- 172.21.15.26:0/2502603712 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9ad4069050 con 0x55b716297d10 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9aee009740 10 client.5997 fetch_fsmap learned FSMap version 37 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9aee009740 1 -- 172.21.15.26:0/2502603712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55b7162df1d0 con 0x55b716297d10 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9abffff640 1 -- 172.21.15.26:0/2502603712 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4049850 con 0x55b716297d10 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9abffff640 1 -- 172.21.15.26:0/2502603712 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 3534+0+0 (secure 0 0 0) 0x7f9ad406c020 con 0x55b716297d10 2024-04-24T23:23:18.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.832+0000 7f9aee009740 10 client.5997 fetch_fsmap finished waiting for FSMap version 37 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 10 client.5997 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 20 client.5997 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b716292540 0x55b7162b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 4 client.5997 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 -- 172.21.15.26:0/2502603712 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "status"}) -- 0x55b716203790 con 0x55b716292540 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9ae8897640 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b716292540 0x55b7162b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9abffff640 10 client.5997 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9ae8897640 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b716292540 0x55b7162b2c40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdefebdc2d0ed342 server_cookie=97f488ff8eccbfc4 in_seq=0 out_seq=0 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9abffff640 1 -- 172.21.15.26:0/2502603712 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b716203790 con 0x55b716292540 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9abffff640 10 client.5997 handle_command_reply: tid=0 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 client.5997 shutdown 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 -- 172.21.15.26:0/2502603712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 msgr2=0x55b716301530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716301530 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40090b0 tx=0x7f9ad4009240 comp rx=0 tx=0).stop 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 -- 172.21.15.26:0/2502603712 shutdown_connections 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b716292540 0x55b7162b2c40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b7162e0b60 0x55b716357e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b7162c3c80 0x55b716355490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9abffff640 0 client.5997 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 --2- 172.21.15.26:0/2502603712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b716297d10 0x55b716301530 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 -- 172.21.15.26:0/2502603712 >> 172.21.15.26:0/2502603712 conn(0x55b7162c3540 msgr2=0x55b7162d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.833+0000 7f9aee009740 1 -- 172.21.15.26:0/2502603712 shutdown_connections 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aee009740 1 -- 172.21.15.26:0/2502603712 wait complete. 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aee009740 20 client.5997 trim_cache size 0 max 16384 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:23:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5742, 2024-04-24T23:23:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:23:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:23:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:23:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 15.036500861, 2024-04-24T23:23:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 37, 2024-04-24T23:23:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 44, 2024-04-24T23:23:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 44, 2024-04-24T23:23:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 20.043003723000002 2024-04-24T23:23:18.842 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ac806be30 msgr2=0x7f9ac806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ac806be30 0x7f9ac806e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4034000 tx=0x7f9ad402e000 comp rx=0 tx=0).stop 2024-04-24T23:23:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 msgr2=0x7f9ae41525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 0x7f9ae41525c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9adc007e40 tx=0x7f9adc00e4b0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/3302429321 shutdown_connections 2024-04-24T23:23:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9ac806be30 0x7f9ac806e2f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ae415e570 0x7f9ae41525c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9ae40d22e0 0x7f9ae415e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 --2- 172.21.15.26:0/3302429321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9ae40b3130 0x7f9ae41f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/3302429321 >> 172.21.15.26:0/3302429321 conn(0x7f9ae408dbe0 msgr2=0x7f9ae406cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/3302429321 shutdown_connections 2024-04-24T23:23:18.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.834+0000 7f9aeb2fc640 1 -- 172.21.15.26:0/3302429321 wait complete. 2024-04-24T23:23:18.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:18.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.945+0000 7f577d293640 1 Processor -- start 2024-04-24T23:23:18.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.945+0000 7f577d293640 1 -- start start 2024-04-24T23:23:18.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f577d293640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 0x7f577806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f577d293640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f577806d350 0x7f577815d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f577d293640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57780cd660 0x7f57780cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f577d293640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f57780cdff0 con 0x7f57780cd660 2024-04-24T23:23:18.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f577d293640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5778145cf0 con 0x7f577806d350 2024-04-24T23:23:18.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f577d293640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f577815d570 con 0x7f5778145910 2024-04-24T23:23:18.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f57777fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57780cd660 0x7f57780cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f57777fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57780cd660 0x7f57780cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60590/0 (socket says 172.21.15.26:60590) 2024-04-24T23:23:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f57777fe640 1 -- 172.21.15.26:0/3452362867 learned_addr learned my addr 172.21.15.26:0/3452362867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f5776ffd640 1 --2- 172.21.15.26:0/3452362867 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 0x7f577806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f57767fc640 1 --2- 172.21.15.26:0/3452362867 >> v2:172.21.15.29:3300/0 conn(0x7f577806d350 0x7f577815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f57777fe640 1 -- 172.21.15.26:0/3452362867 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 msgr2=0x7f577806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.946+0000 7f57777fe640 1 --2- 172.21.15.26:0/3452362867 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 0x7f577806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.947+0000 7f57777fe640 1 -- 172.21.15.26:0/3452362867 >> v2:172.21.15.29:3300/0 conn(0x7f577806d350 msgr2=0x7f577815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.947+0000 7f57777fe640 1 --2- 172.21.15.26:0/3452362867 >> v2:172.21.15.29:3300/0 conn(0x7f577806d350 0x7f577815d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.947+0000 7f57777fe640 1 -- 172.21.15.26:0/3452362867 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57780cfbc0 con 0x7f57780cd660 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.947+0000 7f57777fe640 1 --2- 172.21.15.26:0/3452362867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 0x7f57780cdab0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f57780b5210 tx=0x7f576802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5a922314fa3ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.947+0000 7f5775ffb640 1 -- 172.21.15.26:0/3452362867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768037070 con 0x7f57780cd660 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.947+0000 7f5775ffb640 1 -- 172.21.15.26:0/3452362867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5768032440 con 0x7f57780cd660 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.947+0000 7f5775ffb640 1 -- 172.21.15.26:0/3452362867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768032810 con 0x7f57780cd660 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.948+0000 7f577d293640 1 -- 172.21.15.26:0/3452362867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 msgr2=0x7f57780cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.948+0000 7f577d293640 1 --2- 172.21.15.26:0/3452362867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 0x7f57780cdab0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f57780b5210 tx=0x7f576802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.948+0000 7f577d293640 1 -- 172.21.15.26:0/3452362867 shutdown_connections 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.948+0000 7f577d293640 1 --2- 172.21.15.26:0/3452362867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 0x7f57780cdab0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.948+0000 7f577d293640 1 --2- 172.21.15.26:0/3452362867 >> v2:172.21.15.29:3300/0 conn(0x7f577806d350 0x7f577815d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.948+0000 7f577d293640 1 --2- 172.21.15.26:0/3452362867 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 0x7f577806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.948+0000 7f577d293640 1 -- 172.21.15.26:0/3452362867 >> 172.21.15.26:0/3452362867 conn(0x7f577806b300 msgr2=0x7f577806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.948+0000 7f577d293640 1 -- 172.21.15.26:0/3452362867 shutdown_connections 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.948+0000 7f577d293640 1 -- 172.21.15.26:0/3452362867 wait complete. 2024-04-24T23:23:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.949+0000 7f577d293640 1 Processor -- start 2024-04-24T23:23:18.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.949+0000 7f577d293640 1 -- start start 2024-04-24T23:23:18.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f577d293640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f577806d350 0x7f5778143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f577d293640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 0x7f57781f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f577d293640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 0x7f57781f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f577d293640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f577806ddb0 con 0x7f57780cd660 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f577d293640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f577806dad0 con 0x7f577806d350 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f577d293640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f577806e090 con 0x7f5778145910 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f57767fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 0x7f57781f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f57767fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 0x7f57781f0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60606/0 (socket says 172.21.15.26:60606) 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f57767fc640 1 -- 172.21.15.26:0/878022467 learned_addr learned my addr 172.21.15.26:0/878022467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f57777fe640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 0x7f57781f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f5776ffd640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f577806d350 0x7f5778143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f57767fc640 1 -- 172.21.15.26:0/878022467 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 msgr2=0x7f57781f1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.950+0000 7f57767fc640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 0x7f57781f1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.951+0000 7f57767fc640 1 -- 172.21.15.26:0/878022467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f577806d350 msgr2=0x7f5778143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.951+0000 7f57767fc640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f577806d350 0x7f5778143c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.951+0000 7f57767fc640 1 -- 172.21.15.26:0/878022467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57781f16b0 con 0x7f57780cd660 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.951+0000 7f57767fc640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 0x7f57781f0c30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f576000d800 tx=0x7f576000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.951+0000 7f576f7fe640 1 -- 172.21.15.26:0/878022467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760018020 con 0x7f57780cd660 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.951+0000 7f576f7fe640 1 -- 172.21.15.26:0/878022467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57600096a0 con 0x7f57780cd660 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.951+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57781f5d00 con 0x7f57780cd660 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.951+0000 7f576f7fe640 1 -- 172.21.15.26:0/878022467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760009a70 con 0x7f57780cd660 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.951+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57780ab050 con 0x7f57780cd660 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.952+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57780694c0 con 0x7f57780cd660 2024-04-24T23:23:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.952+0000 7f576f7fe640 1 -- 172.21.15.26:0/878022467 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f576001b020 con 0x7f57780cd660 2024-04-24T23:23:18.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.953+0000 7f576f7fe640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f575806be60 0x7f575806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:18.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.953+0000 7f576f7fe640 1 -- 172.21.15.26:0/878022467 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f576008af20 con 0x7f57780cd660 2024-04-24T23:23:18.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.953+0000 7f5776ffd640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f575806be60 0x7f575806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:18.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.953+0000 7f5776ffd640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f575806be60 0x7f575806e320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5770003aa0 tx=0x7f5770009830 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:18.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:18.957+0000 7f576f7fe640 1 -- 172.21.15.26:0/878022467 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5760022030 con 0x7f57780cd660 2024-04-24T23:23:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.178+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f577806c130 con 0x7f57780cd660 2024-04-24T23:23:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.179+0000 7f576f7fe640 1 -- 172.21.15.26:0/878022467 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5213 (secure 0 0 0) 0x7f5760059270 con 0x7f57780cd660 2024-04-24T23:23:19.181 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:19.181 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:17.849716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5742,"mds_1":5329,"mds_2":5081},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5081":{"gid":5081,"name":"b","rank":2,"incarnation":36,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5081,5329,5742]},"id":3}]} 2024-04-24T23:23:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 37 2024-04-24T23:23:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.180+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f575806be60 msgr2=0x7f575806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.180+0000 7f577d293640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f575806be60 0x7f575806e320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5770003aa0 tx=0x7f5770009830 comp rx=0 tx=0).stop 2024-04-24T23:23:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.180+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 msgr2=0x7f57781f0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.180+0000 7f577d293640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 0x7f57781f0c30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f576000d800 tx=0x7f576000dcc0 comp rx=0 tx=0).stop 2024-04-24T23:23:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.181+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 shutdown_connections 2024-04-24T23:23:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.181+0000 7f577d293640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f575806be60 0x7f575806e320 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.181+0000 7f577d293640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5778145910 0x7f57781f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.181+0000 7f577d293640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57780cd660 0x7f57781f0c30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.181+0000 7f577d293640 1 --2- 172.21.15.26:0/878022467 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f577806d350 0x7f5778143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.181+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 >> 172.21.15.26:0/878022467 conn(0x7f577806b300 msgr2=0x7f57780afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.181+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 shutdown_connections 2024-04-24T23:23:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:19.181+0000 7f577d293640 1 -- 172.21.15.26:0/878022467 wait complete. 2024-04-24T23:23:19.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:19.194 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:23:19.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:19.218 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:23:19.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:19.278 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-04-24T23:23:19.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:19.335 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-04-24T23:23:19.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:19.391 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-04-24T23:23:19.447 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:23:19.603 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:23:19.604 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:23:19.604 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:23:19.780 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:23:19.779+0000 7fc1e7675640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-24T23:23:19.781 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:23:19.779+0000 7fc1e7675640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-24T23:23:21.605 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:23:21.736 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:23:21.736 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:23:21.737 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:23:23.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:23:23.869 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:23:23.869 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:23:23.869 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:23:25.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:23:26.001 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:23:26.001 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:23:26.001 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:23:28.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:23:28.132 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:23:28.133 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:23:28.133 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:23:29.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:23:29.283 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:23:30.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:23:30.251 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-04-24T23:23:30.252 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:23:30.252 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:23:30.252 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:23:30.381 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:23:30.382 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:23:30.382 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:23:32.383 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:23:32.514 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:23:32.515 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:23:32.515 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:23:34.516 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:23:34.647 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-04-24T23:23:34.647 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:23:34.648 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:23:34.648 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-04-24T23:23:34.765 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:23:34.765 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:23:34.765 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:23:34.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-04-24T23:23:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.882+0000 7f2a7928c640 1 Processor -- start 2024-04-24T23:23:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.882+0000 7f2a7928c640 1 -- start start 2024-04-24T23:23:34.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.882+0000 7f2a7928c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7406e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.882+0000 7f2a7928c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a74071720 0x7f2a74071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.882+0000 7f2a7928c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7406b1c0 con 0x7f2a74158ae0 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.882+0000 7f2a7928c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f2a740720d0 con 0x7f2a74071720 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.882+0000 7f2a7928c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7406ec90 con 0x7f2a740d0ce0 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.883+0000 7f2a737fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a74071720 0x7f2a74071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.883+0000 7f2a72ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2a74158ae0 0x7f2a7415e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35490/0 (socket says 172.21.15.26:35490) 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.883+0000 7f2a72ffd640 1 -- 172.21.15.26:0/2640160386 learned_addr learned my addr 172.21.15.26:0/2640160386 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.883+0000 7f2a727fc640 1 --2- 172.21.15.26:0/2640160386 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.883+0000 7f2a737fe640 1 -- 172.21.15.26:0/2640160386 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 msgr2=0x7f2a7406e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.883+0000 7f2a737fe640 1 --2- 172.21.15.26:0/2640160386 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7406e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.883+0000 7f2a737fe640 1 -- 172.21.15.26:0/2640160386 >> v1:172.21.15.26:6789/0 conn(0x7f2a74158ae0 legacy=0x7f2a7415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.883+0000 7f2a71ffb640 1 -- 172.21.15.26:0/2640160386 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 700010423 0 0) 0x7f2a7406b1c0 con 0x7f2a74158ae0 2024-04-24T23:23:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.883+0000 7f2a737fe640 1 -- 172.21.15.26:0/2640160386 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7406ef70 con 0x7f2a74071720 2024-04-24T23:23:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.884+0000 7f2a737fe640 1 --2- 172.21.15.26:0/2640160386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a74071720 0x7f2a74071b90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009d90 tx=0x7f2a6c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb0a43bcf1cebac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.884+0000 7f2a71ffb640 1 -- 172.21.15.26:0/2640160386 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c031600 con 0x7f2a74071720 2024-04-24T23:23:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.884+0000 7f2a71ffb640 1 -- 172.21.15.26:0/2640160386 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c0317a0 con 0x7f2a74071720 2024-04-24T23:23:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.884+0000 7f2a71ffb640 1 -- 172.21.15.26:0/2640160386 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c031b70 con 0x7f2a74071720 2024-04-24T23:23:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.884+0000 7f2a7928c640 1 -- 172.21.15.26:0/2640160386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a74071720 msgr2=0x7f2a74071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.884+0000 7f2a7928c640 1 --2- 172.21.15.26:0/2640160386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a74071720 0x7f2a74071b90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009d90 tx=0x7f2a6c02a5d0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.885+0000 7f2a7928c640 1 -- 172.21.15.26:0/2640160386 shutdown_connections 2024-04-24T23:23:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.885+0000 7f2a7928c640 1 --2- 172.21.15.26:0/2640160386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a74071720 0x7f2a74071b90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.885+0000 7f2a7928c640 1 --2- 172.21.15.26:0/2640160386 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7406e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.885+0000 7f2a7928c640 1 -- 172.21.15.26:0/2640160386 >> 172.21.15.26:0/2640160386 conn(0x7f2a740b2970 msgr2=0x7f2a740d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.885+0000 7f2a7928c640 1 -- 172.21.15.26:0/2640160386 shutdown_connections 2024-04-24T23:23:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.885+0000 7f2a7928c640 1 -- 172.21.15.26:0/2640160386 wait complete. 2024-04-24T23:23:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.886+0000 7f2a7928c640 1 Processor -- start 2024-04-24T23:23:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.886+0000 7f2a7928c640 1 -- start start 2024-04-24T23:23:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a7928c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a74071720 0x7f2a7414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a7928c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a7928c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a74158ae0 0x7f2a74155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a7928c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7406f530 con 0x7f2a74158ae0 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a7928c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7415f060 con 0x7f2a74071720 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a7928c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2a740aff60 con 0x7f2a740d0ce0 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a74071720 0x7f2a7414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7415a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:48378/0 (socket says 172.21.15.26:48378) 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a74158ae0 0x7f2a74155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a727fc640 1 -- 172.21.15.26:0/1762774919 learned_addr learned my addr 172.21.15.26:0/1762774919 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a727fc640 1 -- 172.21.15.26:0/1762774919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a74071720 msgr2=0x7f2a7414c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a727fc640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a74071720 0x7f2a7414c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a727fc640 1 -- 172.21.15.26:0/1762774919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a74158ae0 msgr2=0x7f2a74155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a727fc640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a74158ae0 0x7f2a74155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.887+0000 7f2a727fc640 1 -- 172.21.15.26:0/1762774919 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7415a660 con 0x7f2a740d0ce0 2024-04-24T23:23:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.888+0000 7f2a727fc640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7415a120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9e0 tx=0x7f2a6800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.889+0000 7f2a5b7fe640 1 -- 172.21.15.26:0/1762774919 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68017070 con 0x7f2a740d0ce0 2024-04-24T23:23:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.889+0000 7f2a5b7fe640 1 -- 172.21.15.26:0/1762774919 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6800c9d0 con 0x7f2a740d0ce0 2024-04-24T23:23:34.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.889+0000 7f2a5b7fe640 1 -- 172.21.15.26:0/1762774919 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800cdc0 con 0x7f2a740d0ce0 2024-04-24T23:23:34.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.889+0000 7f2a7928c640 1 -- 172.21.15.26:0/1762774919 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a741573c0 con 0x7f2a740d0ce0 2024-04-24T23:23:34.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.889+0000 7f2a7928c640 1 -- 172.21.15.26:0/1762774919 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a74156b70 con 0x7f2a740d0ce0 2024-04-24T23:23:34.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.891+0000 7f2a5b7fe640 1 -- 172.21.15.26:0/1762774919 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a6801a070 con 0x7f2a740d0ce0 2024-04-24T23:23:34.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.891+0000 7f2a5b7fe640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a6006be30 0x7f2a6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.891+0000 7f2a5b7fe640 1 -- 172.21.15.26:0/1762774919 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f2a6808ae40 con 0x7f2a740d0ce0 2024-04-24T23:23:34.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.891+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a6006be30 0x7f2a6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.892+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a6006be30 0x7f2a6006e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a64007fd0 tx=0x7f2a64007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:34.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.892+0000 7f2a7bf97740 1 Processor -- start 2024-04-24T23:23:34.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.892+0000 7f2a7bf97740 1 -- start start 2024-04-24T23:23:34.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f28e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f2904b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28bca40 0x5648f2902cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5648f28206b0 con 0x5648f2904600 2024-04-24T23:23:34.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5648f27c8880 con 0x5648f28e6d60 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5648f281c670 con 0x5648f28bca40 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f2904b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f2904b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34658/0 (socket says 172.21.15.26:34658) 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a727fc640 1 -- 172.21.15.26:0/3419483079 learned_addr learned my addr 172.21.15.26:0/3419483079 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/3419483079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f28e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.893+0000 7f2a737fe640 1 --2- 172.21.15.26:0/3419483079 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28bca40 0x5648f2902cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.894+0000 7f2a727fc640 1 -- 172.21.15.26:0/3419483079 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28bca40 msgr2=0x5648f2902cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.894+0000 7f2a727fc640 1 --2- 172.21.15.26:0/3419483079 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28bca40 0x5648f2902cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.894+0000 7f2a727fc640 1 -- 172.21.15.26:0/3419483079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 msgr2=0x5648f28e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.894+0000 7f2a727fc640 1 --2- 172.21.15.26:0/3419483079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f28e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.894+0000 7f2a727fc640 1 -- 172.21.15.26:0/3419483079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f25fe7d0 con 0x5648f2904600 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.894+0000 7f2a727fc640 1 --2- 172.21.15.26:0/3419483079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f2904b50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2a68061740 tx=0x7f2a68058ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958b56f007c65650 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.894+0000 7f2a58ff9640 1 -- 172.21.15.26:0/3419483079 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68021040 con 0x5648f2904600 2024-04-24T23:23:34.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.894+0000 7f2a58ff9640 1 -- 172.21.15.26:0/3419483079 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a68057440 con 0x5648f2904600 2024-04-24T23:23:34.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.894+0000 7f2a58ff9640 1 -- 172.21.15.26:0/3419483079 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680557b0 con 0x5648f2904600 2024-04-24T23:23:34.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 -- 172.21.15.26:0/3419483079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 msgr2=0x5648f2904b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/3419483079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f2904b50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2a68061740 tx=0x7f2a68058ba0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 -- 172.21.15.26:0/3419483079 shutdown_connections 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/3419483079 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28bca40 0x5648f2902cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/3419483079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f2904b50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/3419483079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f28e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 -- 172.21.15.26:0/3419483079 >> 172.21.15.26:0/3419483079 conn(0x5648f28b9600 msgr2=0x5648f2901b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 -- 172.21.15.26:0/3419483079 shutdown_connections 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 -- 172.21.15.26:0/3419483079 wait complete. 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.895+0000 7f2a7bf97740 1 -- start start 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a7bf97740 1 Processor -- start 2024-04-24T23:23:34.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f28ea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28b6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f28e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5648f28bd7e0 con 0x5648f2904600 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5648f27dbf50 con 0x5648f28e6d60 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5648f28b7530 con 0x5648f28b6ab0 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a7bf97740 10 client.0 _ll_register_callbacks cb 0x5648f2817000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34664/0 (socket says 172.21.15.26:34664) 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.896+0000 7f2a727fc640 1 -- 172.21.15.26:0/674695179 learned_addr learned my addr 172.21.15.26:0/674695179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f28ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a737fe640 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f28e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a727fc640 1 -- 172.21.15.26:0/674695179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 msgr2=0x5648f28e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:34.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a727fc640 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f28e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a727fc640 1 -- 172.21.15.26:0/674695179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 msgr2=0x5648f28ea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a727fc640 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f28ea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a727fc640 1 -- 172.21.15.26:0/674695179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f25343f0 con 0x5648f2904600 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a727fc640 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28b6570 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2a6806f040 tx=0x7f2a68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95ac65e27c2d033 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a51ffb640 1 -- 172.21.15.26:0/674695179 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68021040 con 0x5648f2904600 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a51ffb640 1 -- 172.21.15.26:0/674695179 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f2a68058790 con 0x5648f2904600 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a7bf97740 1 -- 172.21.15.26:0/674695179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a68057440 con 0x5648f2904600 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a51ffb640 1 -- 172.21.15.26:0/674695179 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2a68054e90 con 0x5648f2904600 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a7bf97740 10 client.6018 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a7bf97740 1 -- 172.21.15.26:0/674695179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5648f24f40f0 con 0x5648f2904600 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.897+0000 7f2a51ffb640 1 -- 172.21.15.26:0/674695179 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a68057440 con 0x5648f2904600 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a51ffb640 1 -- 172.21.15.26:0/674695179 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7f2a68072070 con 0x5648f2904600 2024-04-24T23:23:34.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a7bf97740 10 client.6018 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a7bf97740 10 client.6018 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a7bf97740 20 client.6018 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5648f28e3dd0 0x5648f28e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a7bf97740 4 client.6018 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a7bf97740 1 -- 172.21.15.26:0/674695179 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648f282dbf0 con 0x5648f28e3dd0 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5648f28e3dd0 0x5648f28e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a51ffb640 10 client.6018 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.898+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5648f28e3dd0 0x5648f28e4240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cad84ed152e5439 server_cookie=1e4c9b1b7584492e in_seq=0 out_seq=0 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a51ffb640 1 -- 172.21.15.26:0/674695179 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648f282dbf0 con 0x5648f28e3dd0 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a51ffb640 10 client.6018 handle_command_reply: tid=0 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 client.6018 shutdown 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 -- 172.21.15.26:0/674695179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 msgr2=0x5648f28b6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28b6570 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2a6806f040 tx=0x7f2a68000bc0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 -- 172.21.15.26:0/674695179 shutdown_connections 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5648f28e3dd0 0x5648f28e4240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f28e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28b6570 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a51ffb640 0 client.6018 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/674695179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f28ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 -- 172.21.15.26:0/674695179 >> 172.21.15.26:0/674695179 conn(0x5648f28bca40 msgr2=0x5648f29050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 -- 172.21.15.26:0/674695179 shutdown_connections 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 1 -- 172.21.15.26:0/674695179 wait complete. 2024-04-24T23:23:34.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.899+0000 7f2a7bf97740 20 client.6018 trim_cache size 0 max 16384 2024-04-24T23:23:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.914+0000 7f2a7bf97740 1 Processor -- start 2024-04-24T23:23:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.914+0000 7f2a7bf97740 1 -- start start 2024-04-24T23:23:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f290f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f29255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5648f28e7d90 con 0x5648f2904600 2024-04-24T23:23:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74200830 con 0x5648f28e6d60 2024-04-24T23:23:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2a741ffaf0 con 0x5648f28b6ab0 2024-04-24T23:23:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28e6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34680/0 (socket says 172.21.15.26:34680) 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a737fe640 1 -- 172.21.15.26:0/2912256895 learned_addr learned my addr 172.21.15.26:0/2912256895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/2912256895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f290f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a727fc640 1 --2- 172.21.15.26:0/2912256895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f29255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a737fe640 1 -- 172.21.15.26:0/2912256895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 msgr2=0x5648f290f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a737fe640 1 --2- 172.21.15.26:0/2912256895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f290f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a737fe640 1 -- 172.21.15.26:0/2912256895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 msgr2=0x5648f29255b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a737fe640 1 --2- 172.21.15.26:0/2912256895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f29255b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.915+0000 7f2a737fe640 1 -- 172.21.15.26:0/2912256895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f28ea4f0 con 0x5648f2904600 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a737fe640 1 --2- 172.21.15.26:0/2912256895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28e6230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009d90 tx=0x7f2a6c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616c23507f5612f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a53fff640 1 -- 172.21.15.26:0/2912256895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c03c6a0 con 0x5648f2904600 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a53fff640 1 -- 172.21.15.26:0/2912256895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c03c840 con 0x5648f2904600 2024-04-24T23:23:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a53fff640 1 -- 172.21.15.26:0/2912256895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c03cbf0 con 0x5648f2904600 2024-04-24T23:23:34.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a7bf97740 1 -- 172.21.15.26:0/2912256895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 msgr2=0x5648f28e6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/2912256895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28e6230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009d90 tx=0x7f2a6c02e000 comp rx=0 tx=0).stop 2024-04-24T23:23:34.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a7bf97740 1 -- 172.21.15.26:0/2912256895 shutdown_connections 2024-04-24T23:23:34.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/2912256895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f28e6230 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/2912256895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f29255b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/2912256895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f290f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a7bf97740 1 -- 172.21.15.26:0/2912256895 >> 172.21.15.26:0/2912256895 conn(0x5648f28bca40 msgr2=0x5648f2903e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a7bf97740 1 -- 172.21.15.26:0/2912256895 shutdown_connections 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.916+0000 7f2a7bf97740 1 -- 172.21.15.26:0/2912256895 wait complete. 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.917+0000 7f2a7bf97740 1 -- start start 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.917+0000 7f2a7bf97740 1 Processor -- start 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f297a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f290ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a7bf97740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f290f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741f42f0 con 0x5648f2904600 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74200100 con 0x5648f28e6d60 2024-04-24T23:23:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a7bf97740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5648f29011e0 con 0x5648f28b6ab0 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a7bf97740 10 client.0 _ll_register_callbacks cb 0x7f2a64007380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f290f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f297a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f290ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f290f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34692/0 (socket says 172.21.15.26:34692) 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a737fe640 1 -- 172.21.15.26:0/1722565726 learned_addr learned my addr 172.21.15.26:0/1722565726 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a737fe640 1 -- 172.21.15.26:0/1722565726 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 msgr2=0x5648f297a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a737fe640 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f297a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a737fe640 1 -- 172.21.15.26:0/1722565726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 msgr2=0x5648f290ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a737fe640 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f290ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a737fe640 1 -- 172.21.15.26:0/1722565726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f2903bb0 con 0x5648f2904600 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f297a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a737fe640 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f290f320 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c049040 tx=0x7f2a6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73eaf224bbd4acd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.918+0000 7f2a51ffb640 1 -- 172.21.15.26:0/1722565726 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c03c6a0 con 0x5648f2904600 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a51ffb640 1 -- 172.21.15.26:0/1722565726 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f2a6c004040 con 0x5648f2904600 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a7bf97740 1 -- 172.21.15.26:0/1722565726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6c03c840 con 0x5648f2904600 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a51ffb640 1 -- 172.21.15.26:0/1722565726 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2a6c05a050 con 0x5648f2904600 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a7bf97740 10 client.6024 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a7bf97740 1 -- 172.21.15.26:0/1722565726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5648f28e7ff0 con 0x5648f2904600 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a51ffb640 1 -- 172.21.15.26:0/1722565726 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c03c840 con 0x5648f2904600 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a51ffb640 1 -- 172.21.15.26:0/1722565726 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7f2a6c05d020 con 0x5648f2904600 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a7bf97740 10 client.6024 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a7bf97740 10 client.6024 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a7bf97740 20 client.6024 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5648f290e6c0 0x5648f29049e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a7bf97740 4 client.6024 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:23:34.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a7bf97740 1 -- 172.21.15.26:0/1722565726 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648f2827f40 con 0x5648f290e6c0 2024-04-24T23:23:34.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5648f290e6c0 0x5648f29049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:34.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a51ffb640 10 client.6024 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:34.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.919+0000 7f2a72ffd640 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5648f290e6c0 0x5648f29049e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d387be2b9d7cc0 server_cookie=5f0f397e44d3c22c in_seq=0 out_seq=0 2024-04-24T23:23:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a51ffb640 1 -- 172.21.15.26:0/1722565726 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5648f2827f40 con 0x5648f290e6c0 2024-04-24T23:23:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a51ffb640 10 client.6024 handle_command_reply: tid=0 2024-04-24T23:23:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 client.6024 shutdown 2024-04-24T23:23:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 -- 172.21.15.26:0/1722565726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 msgr2=0x5648f290f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f290f320 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c049040 tx=0x7f2a6c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 -- 172.21.15.26:0/1722565726 shutdown_connections 2024-04-24T23:23:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5648f290e6c0 0x5648f29049e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5648f2904600 0x5648f290f320 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5648f28e6d60 0x5648f290ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 --2- 172.21.15.26:0/1722565726 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5648f28b6ab0 0x5648f297a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a51ffb640 0 client.6024 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:34.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 -- 172.21.15.26:0/1722565726 >> 172.21.15.26:0/1722565726 conn(0x5648f28bca40 msgr2=0x5648f28e6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:34.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.930+0000 7f2a7bf97740 1 -- 172.21.15.26:0/1722565726 shutdown_connections 2024-04-24T23:23:34.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7bf97740 1 -- 172.21.15.26:0/1722565726 wait complete. 2024-04-24T23:23:34.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7bf97740 20 client.6024 trim_cache size 0 max 16384 2024-04-24T23:23:34.933 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:34.934 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:23:34.934 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:23:34.934 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 -- 172.21.15.26:0/1762774919 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a6006be30 msgr2=0x7f2a6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a6006be30 0x7f2a6006e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a64007fd0 tx=0x7f2a64007de0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 -- 172.21.15.26:0/1762774919 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 msgr2=0x7f2a7415a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7415a120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9e0 tx=0x7f2a6800eea0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 -- 172.21.15.26:0/1762774919 shutdown_connections 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a6006be30 0x7f2a6006e2f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a74158ae0 0x7f2a74155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a740d0ce0 0x7f2a7415a120 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 --2- 172.21.15.26:0/1762774919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a74071720 0x7f2a7414c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 -- 172.21.15.26:0/1762774919 >> 172.21.15.26:0/1762774919 conn(0x7f2a740b2970 msgr2=0x7f2a7415e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 -- 172.21.15.26:0/1762774919 shutdown_connections 2024-04-24T23:23:34.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:34.931+0000 7f2a7928c640 1 -- 172.21.15.26:0/1762774919 wait complete. 2024-04-24T23:23:34.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 flush journal 2024-04-24T23:23:35.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.044+0000 7fd99a03f640 1 Processor -- start 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.044+0000 7fd99a03f640 1 -- start start 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.045+0000 7fd99a03f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd99415e370 0x7fd9940d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.045+0000 7fd99a03f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd99415e750 0x7fd9940d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.045+0000 7fd99a03f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9940d2120 con 0x7fd9940d4c10 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.045+0000 7fd99a03f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd9940d1510 con 0x7fd99415e370 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.045+0000 7fd99a03f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9940d1ed0 con 0x7fd99415e750 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.045+0000 7fd9937fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd99415e370 0x7fd9940d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.045+0000 7fd9937fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd99415e370 0x7fd9940d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34696/0 (socket says 172.21.15.26:34696) 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.045+0000 7fd9937fe640 1 -- 172.21.15.26:0/1806952539 learned_addr learned my addr 172.21.15.26:0/1806952539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.045+0000 7fd993fff640 1 --1- 172.21.15.26:0/1806952539 >> v1:172.21.15.26:6789/0 conn(0x7fd9940d4c10 0x7fd9940b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35492/0 (socket says 172.21.15.26:35492) 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.046+0000 7fd992ffd640 1 --2- 172.21.15.26:0/1806952539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd99415e750 0x7fd9940d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.046+0000 7fd9937fe640 1 -- 172.21.15.26:0/1806952539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd99415e750 msgr2=0x7fd9940d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.046+0000 7fd9937fe640 1 --2- 172.21.15.26:0/1806952539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd99415e750 0x7fd9940d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.046+0000 7fd9937fe640 1 -- 172.21.15.26:0/1806952539 >> v1:172.21.15.26:6789/0 conn(0x7fd9940d4c10 legacy=0x7fd9940b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.046+0000 7fd9927fc640 1 -- 172.21.15.26:0/1806952539 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083706565 0 0) 0x7fd9940d2120 con 0x7fd9940d4c10 2024-04-24T23:23:35.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.046+0000 7fd9937fe640 1 -- 172.21.15.26:0/1806952539 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9940b3740 con 0x7fd99415e370 2024-04-24T23:23:35.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.046+0000 7fd9937fe640 1 --2- 172.21.15.26:0/1806952539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd99415e370 0x7fd9940d2520 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd984009870 tx=0x7fd98402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6df4bbb60504398 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd9927fc640 1 -- 172.21.15.26:0/1806952539 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984037070 con 0x7fd99415e370 2024-04-24T23:23:35.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd9927fc640 1 -- 172.21.15.26:0/1806952539 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd984032440 con 0x7fd99415e370 2024-04-24T23:23:35.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd9927fc640 1 -- 172.21.15.26:0/1806952539 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984032810 con 0x7fd99415e370 2024-04-24T23:23:35.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd99a03f640 1 -- 172.21.15.26:0/1806952539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd99415e370 msgr2=0x7fd9940d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd99a03f640 1 --2- 172.21.15.26:0/1806952539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd99415e370 0x7fd9940d2520 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd984009870 tx=0x7fd98402a270 comp rx=0 tx=0).stop 2024-04-24T23:23:35.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd99a03f640 1 -- 172.21.15.26:0/1806952539 shutdown_connections 2024-04-24T23:23:35.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd99a03f640 1 --2- 172.21.15.26:0/1806952539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd99415e750 0x7fd9940d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd99a03f640 1 --2- 172.21.15.26:0/1806952539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd99415e370 0x7fd9940d2520 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd99a03f640 1 -- 172.21.15.26:0/1806952539 >> 172.21.15.26:0/1806952539 conn(0x7fd9940d40f0 msgr2=0x7fd9940d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd99a03f640 1 -- 172.21.15.26:0/1806952539 shutdown_connections 2024-04-24T23:23:35.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.047+0000 7fd99a03f640 1 -- 172.21.15.26:0/1806952539 wait complete. 2024-04-24T23:23:35.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.048+0000 7fd99a03f640 1 Processor -- start 2024-04-24T23:23:35.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd99a03f640 1 -- start start 2024-04-24T23:23:35.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd99a03f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9940d4c10 0x7fd994158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd99a03f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99415e370 0x7fd9941f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd99a03f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9941f78b0 0x7fd9940d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd99a03f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9940cfca0 con 0x7fd9940d4c10 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd99a03f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9940b3d00 con 0x7fd99415e370 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd99a03f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9940cfe20 con 0x7fd9941f78b0 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9940d4c10 0x7fd994158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9940d4c10 0x7fd994158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34698/0 (socket says 172.21.15.26:34698) 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.049+0000 7fd9937fe640 1 -- 172.21.15.26:0/2169522394 learned_addr learned my addr 172.21.15.26:0/2169522394 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd993fff640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9941f78b0 0x7fd9940d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd992ffd640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99415e370 0x7fd9941f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd9937fe640 1 -- 172.21.15.26:0/2169522394 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9941f78b0 msgr2=0x7fd9940d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd9937fe640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9941f78b0 0x7fd9940d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd9937fe640 1 -- 172.21.15.26:0/2169522394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99415e370 msgr2=0x7fd9941f7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd9937fe640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99415e370 0x7fd9941f7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd9937fe640 1 -- 172.21.15.26:0/2169522394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9941594a0 con 0x7fd9940d4c10 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd9937fe640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9940d4c10 0x7fd994158f60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd984009870 tx=0x7fd9840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd97bfff640 1 -- 172.21.15.26:0/2169522394 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984041020 con 0x7fd9940d4c10 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd97bfff640 1 -- 172.21.15.26:0/2169522394 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9840306c0 con 0x7fd9940d4c10 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd99a03f640 1 -- 172.21.15.26:0/2169522394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9941f7dc0 con 0x7fd9940d4c10 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd97bfff640 1 -- 172.21.15.26:0/2169522394 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984037040 con 0x7fd9940d4c10 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.050+0000 7fd99a03f640 1 -- 172.21.15.26:0/2169522394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99415fec0 con 0x7fd9940d4c10 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.052+0000 7fd97bfff640 1 -- 172.21.15.26:0/2169522394 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd984044020 con 0x7fd9940d4c10 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.052+0000 7fd97bfff640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98006be60 0x7fd98006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.052+0000 7fd97bfff640 1 -- 172.21.15.26:0/2169522394 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fd9840ab670 con 0x7fd9940d4c10 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.052+0000 7fd992ffd640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98006be60 0x7fd98006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.053+0000 7fd992ffd640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98006be60 0x7fd98006e320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd988006fd0 tx=0x7fd988008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.053+0000 7fd99ce2b740 1 Processor -- start 2024-04-24T23:23:35.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.053+0000 7fd99ce2b740 1 -- start start 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 0x55b0438e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b043904b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0438bca40 0x55b043902cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b0438206b0 con 0x55b043904600 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b0437c8880 con 0x55b0438bca40 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b04381c670 con 0x55b0438e6d60 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd992ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b043904b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd992ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b043904b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34700/0 (socket says 172.21.15.26:34700) 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd992ffd640 1 -- 172.21.15.26:0/1019090924 learned_addr learned my addr 172.21.15.26:0/1019090924 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd9937fe640 1 --2- 172.21.15.26:0/1019090924 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 0x55b0438e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd993fff640 1 --2- 172.21.15.26:0/1019090924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0438bca40 0x55b043902cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd992ffd640 1 -- 172.21.15.26:0/1019090924 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 msgr2=0x55b0438e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd992ffd640 1 --2- 172.21.15.26:0/1019090924 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 0x55b0438e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd992ffd640 1 -- 172.21.15.26:0/1019090924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0438bca40 msgr2=0x55b043902cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd992ffd640 1 --2- 172.21.15.26:0/1019090924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0438bca40 0x55b043902cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd992ffd640 1 -- 172.21.15.26:0/1019090924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0435fe7d0 con 0x55b043904600 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.054+0000 7fd992ffd640 1 --2- 172.21.15.26:0/1019090924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b043904b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd9940b7780 tx=0x7fd988012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54ef5d0d0a0cf2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd9797fa640 1 -- 172.21.15.26:0/1019090924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98801b050 con 0x55b043904600 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd9797fa640 1 -- 172.21.15.26:0/1019090924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd988025070 con 0x55b043904600 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd9797fa640 1 -- 172.21.15.26:0/1019090924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988014db0 con 0x55b043904600 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1019090924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 msgr2=0x55b043904b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1019090924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b043904b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd9940b7780 tx=0x7fd988012000 comp rx=0 tx=0).stop 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1019090924 shutdown_connections 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1019090924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b0438bca40 0x55b043902cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1019090924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b043904b50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1019090924 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 0x55b0438e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1019090924 >> 172.21.15.26:0/1019090924 conn(0x55b0438b9600 msgr2=0x55b043901b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1019090924 shutdown_connections 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.055+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1019090924 wait complete. 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.056+0000 7fd99ce2b740 1 -- start start 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.056+0000 7fd99ce2b740 1 Processor -- start 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 0x55b0438e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b0439075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043907b30 0x55b0438e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b0438bd7e0 con 0x55b043904600 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b0437dbf50 con 0x55b043907b30 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b0438b7530 con 0x55b0438e6d60 2024-04-24T23:23:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd99ce2b740 10 client.0 _ll_register_callbacks cb 0x55b043817000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd992ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b0439075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd992ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b0439075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34714/0 (socket says 172.21.15.26:34714) 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd992ffd640 1 -- 172.21.15.26:0/3095430590 learned_addr learned my addr 172.21.15.26:0/3095430590 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd9937fe640 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 0x55b0438e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd993fff640 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043907b30 0x55b0438e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd992ffd640 1 -- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 msgr2=0x55b0438e7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd992ffd640 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 0x55b0438e7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd992ffd640 1 -- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043907b30 msgr2=0x55b0438e9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd992ffd640 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043907b30 0x55b0438e9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd992ffd640 1 -- 172.21.15.26:0/3095430590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0435343f0 con 0x55b043904600 2024-04-24T23:23:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd992ffd640 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b0439075f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd988021000 tx=0x7fd988004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c621560655ff56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.057+0000 7fd9727fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd9727fc640 1 -- 172.21.15.26:0/3095430590 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988014e60 con 0x55b043904600 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd9727fc640 1 -- 172.21.15.26:0/3095430590 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fd988020c00 con 0x55b043904600 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd99ce2b740 1 -- 172.21.15.26:0/3095430590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd988025070 con 0x55b043904600 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd9727fc640 1 -- 172.21.15.26:0/3095430590 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd98803c050 con 0x55b043904600 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd99ce2b740 10 client.6039 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd99ce2b740 1 -- 172.21.15.26:0/3095430590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55b0434f40f0 con 0x55b043904600 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd9727fc640 1 -- 172.21.15.26:0/3095430590 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd988025070 con 0x55b043904600 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd9727fc640 1 -- 172.21.15.26:0/3095430590 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7fd98803f020 con 0x55b043904600 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd99ce2b740 10 client.6039 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd99ce2b740 10 client.6039 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd99ce2b740 20 client.6039 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b0438e2e40 0x55b0438ea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd99ce2b740 4 client.6039 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd99ce2b740 1 -- 172.21.15.26:0/3095430590 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b04382dbf0 con 0x55b0438e2e40 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.058+0000 7fd9937fe640 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b0438e2e40 0x55b0438ea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd9937fe640 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b0438e2e40 0x55b0438ea2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2008ed0753dfbd2e server_cookie=5a70f168702fa513 in_seq=0 out_seq=0 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd9727fc640 10 client.6039 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd9727fc640 1 -- 172.21.15.26:0/3095430590 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b04382dbf0 con 0x55b0438e2e40 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd9727fc640 10 client.6039 handle_command_reply: tid=0 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 client.6039 shutdown 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 -- 172.21.15.26:0/3095430590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 msgr2=0x55b0439075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b0439075f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd988021000 tx=0x7fd988004730 comp rx=0 tx=0).stop 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 -- 172.21.15.26:0/3095430590 shutdown_connections 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b0438e2e40 0x55b0438ea2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043907b30 0x55b0438e9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b043904600 0x55b0439075f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd9727fc640 0 client.6039 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/3095430590 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b0438e6d60 0x55b0438e7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 -- 172.21.15.26:0/3095430590 >> 172.21.15.26:0/3095430590 conn(0x55b0438bca40 msgr2=0x55b043905980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.059+0000 7fd99ce2b740 1 -- 172.21.15.26:0/3095430590 shutdown_connections 2024-04-24T23:23:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.060+0000 7fd99ce2b740 1 -- 172.21.15.26:0/3095430590 wait complete. 2024-04-24T23:23:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.060+0000 7fd99ce2b740 20 client.6039 trim_cache size 0 max 16384 2024-04-24T23:23:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.075+0000 7fd99ce2b740 1 Processor -- start 2024-04-24T23:23:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.075+0000 7fd99ce2b740 1 -- start start 2024-04-24T23:23:35.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.075+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438bce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.075+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043904600 0x55b0438bd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.075+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043906050 0x55b0438e0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.075+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd994200800 con 0x55b0438e6d60 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.075+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941ff9d0 con 0x55b043906050 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.075+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9942000d0 con 0x55b043904600 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438bce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34728/0 (socket says 172.21.15.26:34728) 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd9937fe640 1 -- 172.21.15.26:0/2576178600 learned_addr learned my addr 172.21.15.26:0/2576178600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd992ffd640 1 --2- 172.21.15.26:0/2576178600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043904600 0x55b0438bd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd993fff640 1 --2- 172.21.15.26:0/2576178600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043906050 0x55b0438e0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd9937fe640 1 -- 172.21.15.26:0/2576178600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043904600 msgr2=0x55b0438bd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd9937fe640 1 --2- 172.21.15.26:0/2576178600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043904600 0x55b0438bd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd9937fe640 1 -- 172.21.15.26:0/2576178600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043906050 msgr2=0x55b0438e0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd9937fe640 1 --2- 172.21.15.26:0/2576178600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043906050 0x55b0438e0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd9937fe640 1 -- 172.21.15.26:0/2576178600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0438b90b0 con 0x55b0438e6d60 2024-04-24T23:23:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd9937fe640 1 --2- 172.21.15.26:0/2576178600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438bce20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd984006da0 tx=0x7fd984008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6717c7a6780b9b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd978ff9640 1 -- 172.21.15.26:0/2576178600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9840824f0 con 0x55b0438e6d60 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd978ff9640 1 -- 172.21.15.26:0/2576178600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98407ee50 con 0x55b0438e6d60 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.076+0000 7fd978ff9640 1 -- 172.21.15.26:0/2576178600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984077c10 con 0x55b0438e6d60 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 -- 172.21.15.26:0/2576178600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 msgr2=0x55b0438bce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/2576178600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438bce20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd984006da0 tx=0x7fd984008040 comp rx=0 tx=0).stop 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 -- 172.21.15.26:0/2576178600 shutdown_connections 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/2576178600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043906050 0x55b0438e0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/2576178600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043904600 0x55b0438bd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/2576178600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438bce20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 -- 172.21.15.26:0/2576178600 >> 172.21.15.26:0/2576178600 conn(0x55b0438bca40 msgr2=0x55b043900310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 -- 172.21.15.26:0/2576178600 shutdown_connections 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 -- 172.21.15.26:0/2576178600 wait complete. 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.077+0000 7fd99ce2b740 1 -- start start 2024-04-24T23:23:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd99ce2b740 1 Processor -- start 2024-04-24T23:23:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043904600 0x55b0438bce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd99ce2b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043906050 0x55b0438bd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9940ce050 con 0x55b0438e6d60 2024-04-24T23:23:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b043901960 con 0x55b043904600 2024-04-24T23:23:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd99ce2b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b0438b71d0 con 0x55b043906050 2024-04-24T23:23:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd99ce2b740 10 client.0 _ll_register_callbacks cb 0x7fd98800f830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438e9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34744/0 (socket says 172.21.15.26:34744) 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd9937fe640 1 -- 172.21.15.26:0/1756928662 learned_addr learned my addr 172.21.15.26:0/1756928662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd992ffd640 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043904600 0x55b0438bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.078+0000 7fd993fff640 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043906050 0x55b0438bd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9937fe640 1 -- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043906050 msgr2=0x55b0438bd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9937fe640 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043906050 0x55b0438bd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9937fe640 1 -- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043904600 msgr2=0x55b0438bce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9937fe640 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043904600 0x55b0438bce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9937fe640 1 -- 172.21.15.26:0/1756928662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04391baa0 con 0x55b0438e6d60 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9937fe640 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438e9d60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd984079610 tx=0x7fd984080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c67d693d8e4985 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9727fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9727fc640 1 -- 172.21.15.26:0/1756928662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984075f80 con 0x55b0438e6d60 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9727fc640 1 -- 172.21.15.26:0/1756928662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fd984079b70 con 0x55b0438e6d60 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1756928662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98407ee50 con 0x55b0438e6d60 2024-04-24T23:23:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9727fc640 1 -- 172.21.15.26:0/1756928662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd98409f050 con 0x55b0438e6d60 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd99ce2b740 10 client.6045 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1756928662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55b043925aa0 con 0x55b0438e6d60 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9727fc640 1 -- 172.21.15.26:0/1756928662 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98407ee50 con 0x55b0438e6d60 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd9727fc640 1 -- 172.21.15.26:0/1756928662 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7fd9840a2020 con 0x55b0438e6d60 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd99ce2b740 10 client.6045 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd99ce2b740 10 client.6045 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd99ce2b740 20 client.6045 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b04397eb40 0x55b0438b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd99ce2b740 4 client.6045 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.079+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1756928662 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b043827f40 con 0x55b04397eb40 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.080+0000 7fd992ffd640 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b04397eb40 0x55b0438b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.080+0000 7fd992ffd640 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b04397eb40 0x55b0438b6260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65c548d95f52194 server_cookie=997f5ab70a3e46e4 in_seq=0 out_seq=0 2024-04-24T23:23:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.080+0000 7fd9727fc640 10 client.6045 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd9727fc640 1 -- 172.21.15.26:0/1756928662 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b043827f40 con 0x55b04397eb40 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd9727fc640 10 client.6045 handle_command_reply: tid=0 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 client.6045 shutdown 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1756928662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 msgr2=0x55b0438e9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438e9d60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd984079610 tx=0x7fd984080040 comp rx=0 tx=0).stop 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1756928662 shutdown_connections 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b04397eb40 0x55b0438b6260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b043906050 0x55b0438bd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b043904600 0x55b0438bce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd9727fc640 0 client.6045 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 --2- 172.21.15.26:0/1756928662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0438e6d60 0x55b0438e9d60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1756928662 >> 172.21.15.26:0/1756928662 conn(0x55b0438bca40 msgr2=0x55b043906450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1756928662 shutdown_connections 2024-04-24T23:23:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 1 -- 172.21.15.26:0/1756928662 wait complete. 2024-04-24T23:23:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.087+0000 7fd99ce2b740 20 client.6045 trim_cache size 0 max 16384 2024-04-24T23:23:35.090 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:35.090 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:23:35.090 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:23:35.090 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 -- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98006be60 msgr2=0x7fd98006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98006be60 0x7fd98006e320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd988006fd0 tx=0x7fd988008040 comp rx=0 tx=0).stop 2024-04-24T23:23:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 -- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9940d4c10 msgr2=0x7fd994158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9940d4c10 0x7fd994158f60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd984009870 tx=0x7fd9840046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 -- 172.21.15.26:0/2169522394 shutdown_connections 2024-04-24T23:23:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd98006be60 0x7fd98006e320 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9941f78b0 0x7fd9940d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd99415e370 0x7fd9941f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 --2- 172.21.15.26:0/2169522394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9940d4c10 0x7fd994158f60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 -- 172.21.15.26:0/2169522394 >> 172.21.15.26:0/2169522394 conn(0x7fd9940d40f0 msgr2=0x7fd99415c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 -- 172.21.15.26:0/2169522394 shutdown_connections 2024-04-24T23:23:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.088+0000 7fd99a03f640 1 -- 172.21.15.26:0/2169522394 wait complete. 2024-04-24T23:23:35.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 flush journal 2024-04-24T23:23:35.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.201+0000 7ff15f676640 1 Processor -- start 2024-04-24T23:23:35.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.201+0000 7ff15f676640 1 -- start start 2024-04-24T23:23:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.202+0000 7ff15f676640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580d0ce0 0x7ff15806e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.202+0000 7ff15f676640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff158071720 0x7ff158071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.202+0000 7ff15f676640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff15806b1c0 con 0x7ff158158ae0 2024-04-24T23:23:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.202+0000 7ff15f676640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff1580720d0 con 0x7ff158071720 2024-04-24T23:23:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.202+0000 7ff15f676640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff15806ec90 con 0x7ff1580d0ce0 2024-04-24T23:23:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.202+0000 7ff15dc13640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff158071720 0x7ff158071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.202+0000 7ff15dc13640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff158071720 0x7ff158071b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34754/0 (socket says 172.21.15.26:34754) 2024-04-24T23:23:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.202+0000 7ff15dc13640 1 -- 172.21.15.26:0/807175557 learned_addr learned my addr 172.21.15.26:0/807175557 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.202+0000 7ff15cc11640 1 --2- 172.21.15.26:0/807175557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580d0ce0 0x7ff15806e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.203+0000 7ff15dc13640 1 -- 172.21.15.26:0/807175557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580d0ce0 msgr2=0x7ff15806e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.203+0000 7ff15dc13640 1 --2- 172.21.15.26:0/807175557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580d0ce0 0x7ff15806e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.203+0000 7ff15dc13640 1 -- 172.21.15.26:0/807175557 >> v1:172.21.15.29:6789/0 conn(0x7ff158158ae0 legacy=0x7ff15815e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.203+0000 7ff15dc13640 1 -- 172.21.15.26:0/807175557 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15806ef70 con 0x7ff158071720 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.203+0000 7ff15dc13640 1 --2- 172.21.15.26:0/807175557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 0x7ff158071b90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff154009e50 tx=0x7ff15402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3999f6d06fcec827 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.203+0000 7ff14ffff640 1 -- 172.21.15.26:0/807175557 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154036070 con 0x7ff158071720 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.203+0000 7ff14ffff640 1 -- 172.21.15.26:0/807175557 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff154031600 con 0x7ff158071720 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.204+0000 7ff14ffff640 1 -- 172.21.15.26:0/807175557 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1540319d0 con 0x7ff158071720 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.204+0000 7ff15f676640 1 -- 172.21.15.26:0/807175557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 msgr2=0x7ff158071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.204+0000 7ff15f676640 1 --2- 172.21.15.26:0/807175557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 0x7ff158071b90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff154009e50 tx=0x7ff15402a680 comp rx=0 tx=0).stop 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.204+0000 7ff15f676640 1 -- 172.21.15.26:0/807175557 shutdown_connections 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.204+0000 7ff15f676640 1 --2- 172.21.15.26:0/807175557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 0x7ff158071b90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.204+0000 7ff15f676640 1 --2- 172.21.15.26:0/807175557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff1580d0ce0 0x7ff15806e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.204+0000 7ff15f676640 1 -- 172.21.15.26:0/807175557 >> 172.21.15.26:0/807175557 conn(0x7ff1580b2970 msgr2=0x7ff1580d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.204+0000 7ff15f676640 1 -- 172.21.15.26:0/807175557 shutdown_connections 2024-04-24T23:23:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.204+0000 7ff15f676640 1 -- 172.21.15.26:0/807175557 wait complete. 2024-04-24T23:23:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.205+0000 7ff15f676640 1 Processor -- start 2024-04-24T23:23:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.205+0000 7ff15f676640 1 -- start start 2024-04-24T23:23:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15f676640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 0x7ff1581470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15f676640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1580d0ce0 0x7ff1580b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15f676640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff158158ae0 0x7ff15815d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15f676640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff15806f530 con 0x7ff158071720 2024-04-24T23:23:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15f676640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff15815f060 con 0x7ff1580d0ce0 2024-04-24T23:23:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15f676640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff1580aff60 con 0x7ff158158ae0 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15d412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 0x7ff1581470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15d412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 0x7ff1581470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34764/0 (socket says 172.21.15.26:34764) 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15d412640 1 -- 172.21.15.26:0/31122433 learned_addr learned my addr 172.21.15.26:0/31122433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15dc13640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff158158ae0 0x7ff15815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.206+0000 7ff15cc11640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1580d0ce0 0x7ff1580b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff15d412640 1 -- 172.21.15.26:0/31122433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff158158ae0 msgr2=0x7ff15815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff15d412640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff158158ae0 0x7ff15815d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff15d412640 1 -- 172.21.15.26:0/31122433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1580d0ce0 msgr2=0x7ff1580b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff15d412640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1580d0ce0 0x7ff1580b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff15d412640 1 -- 172.21.15.26:0/31122433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1580b4750 con 0x7ff158071720 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff15d412640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 0x7ff1581470d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff15000c3f0 tx=0x7ff15000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff14dffb640 1 -- 172.21.15.26:0/31122433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1500052d0 con 0x7ff158071720 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff14dffb640 1 -- 172.21.15.26:0/31122433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff150005490 con 0x7ff158071720 2024-04-24T23:23:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff15f676640 1 -- 172.21.15.26:0/31122433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1580b4a30 con 0x7ff158071720 2024-04-24T23:23:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff14dffb640 1 -- 172.21.15.26:0/31122433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150007260 con 0x7ff158071720 2024-04-24T23:23:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.207+0000 7ff15f676640 1 -- 172.21.15.26:0/31122433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1580b4ff0 con 0x7ff158071720 2024-04-24T23:23:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.208+0000 7ff14dffb640 1 -- 172.21.15.26:0/31122433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff150007400 con 0x7ff158071720 2024-04-24T23:23:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.209+0000 7ff14dffb640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.209+0000 7ff14dffb640 1 -- 172.21.15.26:0/31122433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7ff15008a340 con 0x7ff158071720 2024-04-24T23:23:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.209+0000 7ff15cc11640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.210+0000 7ff15cc11640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff158158650 tx=0x7ff148009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.210+0000 7ff162385740 1 Processor -- start 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.210+0000 7ff162385740 1 -- start start 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff162385740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d9d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff162385740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 0x55ab47dbab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff162385740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47d72a40 0x55ab47db8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff162385740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ab47cd66b0 con 0x55ab47d9cd60 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff162385740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ab47c7e880 con 0x55ab47dba600 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff162385740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ab47cd2670 con 0x55ab47d72a40 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15d412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d9d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15d412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d9d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34770/0 (socket says 172.21.15.26:34770) 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15d412640 1 -- 172.21.15.26:0/1793773594 learned_addr learned my addr 172.21.15.26:0/1793773594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15dc13640 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47d72a40 0x55ab47db8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15cc11640 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 0x55ab47dbab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15d412640 1 -- 172.21.15.26:0/1793773594 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47d72a40 msgr2=0x55ab47db8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15d412640 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47d72a40 0x55ab47db8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15d412640 1 -- 172.21.15.26:0/1793773594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 msgr2=0x55ab47dbab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15d412640 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 0x55ab47dbab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15d412640 1 -- 172.21.15.26:0/1793773594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab47ab47d0 con 0x55ab47d9cd60 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.211+0000 7ff15cc11640 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 0x55ab47dbab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff15d412640 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d9d140 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff150060c00 tx=0x7ff15000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc7f3962375dff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff142ffd640 1 -- 172.21.15.26:0/1793773594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15008a0c0 con 0x55ab47d9cd60 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff142ffd640 1 -- 172.21.15.26:0/1793773594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff150056900 con 0x55ab47d9cd60 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff142ffd640 1 -- 172.21.15.26:0/1793773594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150060ca0 con 0x55ab47d9cd60 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff162385740 1 -- 172.21.15.26:0/1793773594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 msgr2=0x55ab47d9d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff162385740 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d9d140 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff150060c00 tx=0x7ff15000a000 comp rx=0 tx=0).stop 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff162385740 1 -- 172.21.15.26:0/1793773594 shutdown_connections 2024-04-24T23:23:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff162385740 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47d72a40 0x55ab47db8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff162385740 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 0x55ab47dbab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff162385740 1 --2- 172.21.15.26:0/1793773594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d9d140 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff162385740 1 -- 172.21.15.26:0/1793773594 >> 172.21.15.26:0/1793773594 conn(0x55ab47d6f600 msgr2=0x55ab47db7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff162385740 1 -- 172.21.15.26:0/1793773594 shutdown_connections 2024-04-24T23:23:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.212+0000 7ff162385740 1 -- 172.21.15.26:0/1793773594 wait complete. 2024-04-24T23:23:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.213+0000 7ff162385740 1 -- start start 2024-04-24T23:23:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.213+0000 7ff162385740 1 Processor -- start 2024-04-24T23:23:35.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff162385740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 0x55ab47d89dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff162385740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dba600 0x55ab47daf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff162385740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dafd20 0x55ab47d8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff162385740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ab47d737e0 con 0x55ab47dafd20 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff162385740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ab47c91f50 con 0x55ab47d9cd60 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff162385740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ab47d6d530 con 0x55ab47dba600 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff162385740 10 client.0 _ll_register_callbacks cb 0x55ab47ccd000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15dc13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dafd20 0x55ab47d8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15dc13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dafd20 0x55ab47d8c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34778/0 (socket says 172.21.15.26:34778) 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15dc13640 1 -- 172.21.15.26:0/3778576332 learned_addr learned my addr 172.21.15.26:0/3778576332 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15cc11640 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dba600 0x55ab47daf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15d412640 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 0x55ab47d89dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15dc13640 1 -- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dba600 msgr2=0x55ab47daf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15dc13640 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dba600 0x55ab47daf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15dc13640 1 -- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 msgr2=0x55ab47d89dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15dc13640 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 0x55ab47d89dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15dc13640 1 -- 172.21.15.26:0/3778576332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab479ea3f0 con 0x55ab47dafd20 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15cc11640 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dba600 0x55ab47daf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff15dc13640 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dafd20 0x55ab47d8c580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff15402ac70 tx=0x7ff154004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89970b3c1092884 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.214+0000 7ff137fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff137fff640 1 -- 172.21.15.26:0/3778576332 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154041030 con 0x55ab47dafd20 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff137fff640 1 -- 172.21.15.26:0/3778576332 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7ff1540319c0 con 0x55ab47dafd20 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff162385740 1 -- 172.21.15.26:0/3778576332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff150056900 con 0x55ab47dafd20 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff137fff640 1 -- 172.21.15.26:0/3778576332 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff15404d050 con 0x55ab47dafd20 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff162385740 10 client.6057 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff162385740 1 -- 172.21.15.26:0/3778576332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55ab479aa0f0 con 0x55ab47dafd20 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff137fff640 1 -- 172.21.15.26:0/3778576332 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff150056900 con 0x55ab47dafd20 2024-04-24T23:23:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff137fff640 1 -- 172.21.15.26:0/3778576332 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7ff154050020 con 0x55ab47dafd20 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff162385740 10 client.6057 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff162385740 10 client.6057 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff162385740 20 client.6057 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff162385740 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55ab47dcbca0 0x55ab47dcc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff162385740 4 client.6057 mds_command: new command op to 5081 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff162385740 1 -- 172.21.15.26:0/3778576332 --> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab47ce3bf0 con 0x55ab47dcbca0 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.215+0000 7ff15d412640 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55ab47dcbca0 0x55ab47dcc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff137fff640 10 client.6057 ms_handle_connect on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff15d412640 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55ab47dcbca0 0x55ab47dcc0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9a54c7c1fef51a4 server_cookie=2dcbd7ee8c6c28b7 in_seq=0 out_seq=0 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff137fff640 1 -- 172.21.15.26:0/3778576332 <== mds.2 v2:172.21.15.29:6832/1578186344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab47ce3bf0 con 0x55ab47dcbca0 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff137fff640 10 client.6057 handle_command_reply: tid=0 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff162385740 1 client.6057 shutdown 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff162385740 1 -- 172.21.15.26:0/3778576332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dafd20 msgr2=0x55ab47d8c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff162385740 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dafd20 0x55ab47d8c580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff15402ac70 tx=0x7ff154004ab0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff162385740 1 -- 172.21.15.26:0/3778576332 shutdown_connections 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff162385740 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55ab47dcbca0 0x55ab47dcc0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff162385740 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dafd20 0x55ab47d8c580 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff162385740 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dba600 0x55ab47daf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff137fff640 0 client.6057 ms_handle_reset on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff162385740 1 --2- 172.21.15.26:0/3778576332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 0x55ab47d89dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.216+0000 7ff162385740 1 -- 172.21.15.26:0/3778576332 >> 172.21.15.26:0/3778576332 conn(0x55ab47d72a40 msgr2=0x55ab47dbb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.217+0000 7ff162385740 1 -- 172.21.15.26:0/3778576332 shutdown_connections 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.217+0000 7ff162385740 1 -- 172.21.15.26:0/3778576332 wait complete. 2024-04-24T23:23:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.217+0000 7ff162385740 20 client.6057 trim_cache size 0 max 16384 2024-04-24T23:23:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.231+0000 7ff162385740 1 Processor -- start 2024-04-24T23:23:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.231+0000 7ff162385740 1 -- start start 2024-04-24T23:23:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff162385740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d73e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff162385740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 0x55ab47d9a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff162385740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 0x55ab47d9ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff162385740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1582008d0 con 0x55ab47d9cd60 2024-04-24T23:23:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff162385740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1581ffc50 con 0x55ab47dba600 2024-04-24T23:23:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff162385740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff1582001a0 con 0x55ab47dafd20 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff15d412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d73e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff15d412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d73e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34786/0 (socket says 172.21.15.26:34786) 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff15d412640 1 -- 172.21.15.26:0/1100737575 learned_addr learned my addr 172.21.15.26:0/1100737575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff15cc11640 1 --2- 172.21.15.26:0/1100737575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 0x55ab47d9a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.232+0000 7ff15dc13640 1 --2- 172.21.15.26:0/1100737575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 0x55ab47d9ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff15d412640 1 -- 172.21.15.26:0/1100737575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 msgr2=0x55ab47d9a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff15d412640 1 --2- 172.21.15.26:0/1100737575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 0x55ab47d9a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff15d412640 1 -- 172.21.15.26:0/1100737575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 msgr2=0x55ab47d9ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff15d412640 1 --2- 172.21.15.26:0/1100737575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 0x55ab47d9ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff15d412640 1 -- 172.21.15.26:0/1100737575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab47dafb50 con 0x55ab47d9cd60 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff15d412640 1 --2- 172.21.15.26:0/1100737575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d73e10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff1500587c0 tx=0x7ff15008a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747bb7ff7aecae15 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff1427fc640 1 -- 172.21.15.26:0/1100737575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150074070 con 0x55ab47d9cd60 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff1427fc640 1 -- 172.21.15.26:0/1100737575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15008a0c0 con 0x55ab47d9cd60 2024-04-24T23:23:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff1427fc640 1 -- 172.21.15.26:0/1100737575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1500883a0 con 0x55ab47d9cd60 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff162385740 1 -- 172.21.15.26:0/1100737575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 msgr2=0x55ab47d73e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff162385740 1 --2- 172.21.15.26:0/1100737575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d73e10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff1500587c0 tx=0x7ff15008a050 comp rx=0 tx=0).stop 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff162385740 1 -- 172.21.15.26:0/1100737575 shutdown_connections 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff162385740 1 --2- 172.21.15.26:0/1100737575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47dba600 0x55ab47d9ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff162385740 1 --2- 172.21.15.26:0/1100737575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 0x55ab47d9a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff162385740 1 --2- 172.21.15.26:0/1100737575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47d9cd60 0x55ab47d73e10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff162385740 1 -- 172.21.15.26:0/1100737575 >> 172.21.15.26:0/1100737575 conn(0x55ab47d72a40 msgr2=0x55ab47d72e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff162385740 1 -- 172.21.15.26:0/1100737575 shutdown_connections 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.233+0000 7ff162385740 1 -- 172.21.15.26:0/1100737575 wait complete. 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.234+0000 7ff162385740 1 -- start start 2024-04-24T23:23:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.234+0000 7ff162385740 1 Processor -- start 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff162385740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 0x55ab47dc6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff162385740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 0x55ab47d6c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff162385740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dba600 0x55ab47d99710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff162385740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff158147bb0 con 0x55ab47dba600 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff162385740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ab47d9a430 con 0x55ab47d9cd60 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff162385740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ab47d6cba0 con 0x55ab47dafd20 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff162385740 10 client.0 _ll_register_callbacks cb 0x7ff148009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15dc13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dba600 0x55ab47d99710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15dc13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dba600 0x55ab47d99710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34798/0 (socket says 172.21.15.26:34798) 2024-04-24T23:23:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15dc13640 1 -- 172.21.15.26:0/1023109784 learned_addr learned my addr 172.21.15.26:0/1023109784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15cc11640 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 0x55ab47d6c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15d412640 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 0x55ab47dc6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15dc13640 1 -- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 msgr2=0x55ab47d6c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15dc13640 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 0x55ab47d6c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15dc13640 1 -- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 msgr2=0x55ab47dc6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15dc13640 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 0x55ab47dc6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15dc13640 1 -- 172.21.15.26:0/1023109784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab47d71d50 con 0x55ab47dba600 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.235+0000 7ff15cc11640 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 0x55ab47d6c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff15dc13640 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dba600 0x55ab47d99710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff154003bc0 tx=0x7ff15402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f799b49539da83d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff137fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff137fff640 1 -- 172.21.15.26:0/1023109784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154041050 con 0x55ab47dba600 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff137fff640 1 -- 172.21.15.26:0/1023109784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7ff154004040 con 0x55ab47dba600 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff162385740 1 -- 172.21.15.26:0/1023109784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff15008a0c0 con 0x55ab47dba600 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff137fff640 1 -- 172.21.15.26:0/1023109784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff15405c050 con 0x55ab47dba600 2024-04-24T23:23:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff162385740 10 client.6063 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff162385740 1 -- 172.21.15.26:0/1023109784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55ab47d9a170 con 0x55ab47dba600 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff137fff640 1 -- 172.21.15.26:0/1023109784 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15008a0c0 con 0x55ab47dba600 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff137fff640 1 -- 172.21.15.26:0/1023109784 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7ff15405f020 con 0x55ab47dba600 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff162385740 10 client.6063 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff162385740 10 client.6063 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff162385740 20 client.6063 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff162385740 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55ab47d9af10 0x55ab47dbae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff162385740 4 client.6063 mds_command: new command op to 5081 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff162385740 1 -- 172.21.15.26:0/1023109784 --> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab47cddf40 con 0x55ab47d9af10 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.236+0000 7ff15d412640 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55ab47d9af10 0x55ab47dbae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.237+0000 7ff137fff640 10 client.6063 ms_handle_connect on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.237+0000 7ff15d412640 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55ab47d9af10 0x55ab47dbae20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ce6c2b230576776 server_cookie=15ae0d3cd5166286 in_seq=0 out_seq=0 2024-04-24T23:23:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff137fff640 1 -- 172.21.15.26:0/1023109784 <== mds.2 v2:172.21.15.29:6832/1578186344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab47cddf40 con 0x55ab47d9af10 2024-04-24T23:23:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff137fff640 10 client.6063 handle_command_reply: tid=0 2024-04-24T23:23:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff162385740 1 client.6063 shutdown 2024-04-24T23:23:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff162385740 1 -- 172.21.15.26:0/1023109784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dba600 msgr2=0x55ab47d99710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff162385740 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dba600 0x55ab47d99710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff154003bc0 tx=0x7ff15402f000 comp rx=0 tx=0).stop 2024-04-24T23:23:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff162385740 1 -- 172.21.15.26:0/1023109784 shutdown_connections 2024-04-24T23:23:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff162385740 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x55ab47d9af10 0x55ab47dbae20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff162385740 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ab47dba600 0x55ab47d99710 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff137fff640 0 client.6063 ms_handle_reset on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff162385740 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ab47dafd20 0x55ab47d6c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff162385740 1 --2- 172.21.15.26:0/1023109784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ab47d9cd60 0x55ab47dc6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.242+0000 7ff162385740 1 -- 172.21.15.26:0/1023109784 >> 172.21.15.26:0/1023109784 conn(0x55ab47d72a40 msgr2=0x55ab47d70910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff162385740 1 -- 172.21.15.26:0/1023109784 shutdown_connections 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff162385740 1 -- 172.21.15.26:0/1023109784 wait complete. 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff162385740 20 client.6063 trim_cache size 0 max 16384 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:23:35.245 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:23:35.246 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 -- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 msgr2=0x7ff13c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff158158650 tx=0x7ff148009190 comp rx=0 tx=0).stop 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 -- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 msgr2=0x7ff1581470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 0x7ff1581470d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff15000c3f0 tx=0x7ff15000c8b0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 -- 172.21.15.26:0/31122433 shutdown_connections 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff13c06be30 0x7ff13c06e2f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff158158ae0 0x7ff15815d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1580d0ce0 0x7ff1580b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 --2- 172.21.15.26:0/31122433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff158071720 0x7ff1581470d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 -- 172.21.15.26:0/31122433 >> 172.21.15.26:0/31122433 conn(0x7ff1580b2970 msgr2=0x7ff15815e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 -- 172.21.15.26:0/31122433 shutdown_connections 2024-04-24T23:23:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35.243+0000 7ff15f676640 1 -- 172.21.15.26:0/31122433 wait complete. 2024-04-24T23:23:35.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:35.257 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-04-24T23:23:35.279 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/ 2024-04-24T23:23:35.280 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4 2024-04-24T23:23:35.280 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5 2024-04-24T23:23:35.280 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6 2024-04-24T23:23:35.280 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7 2024-04-24T23:23:35.280 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-04-24T23:23:35.280 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:23:35.280 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:23:35.281 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:23:35.281 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:23:35.281 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-04-24T23:23:35.281 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:23:35.389 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627778 2024-04-24T23:23:35.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:23:35.399 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:23:35.399 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:23:35.399 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:23:35.399 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:23:35.399 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-04-24T23:23:35.399 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:23:35.506 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627779 2024-04-24T23:23:35.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:23:35.515 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:23:35.515 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:23:35.516 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:23:35.516 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:23:35.516 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-04-24T23:23:35.516 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:23:35.621 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627780 2024-04-24T23:23:35.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:23:35.631 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:23:35.631 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:23:35.631 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:23:35.631 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:23:35.631 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-04-24T23:23:35.631 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:23:35.738 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627781 2024-04-24T23:23:35.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:23:35.746 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:23:35.747 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:23:35.747 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:23:35.747 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:23:35.747 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-04-24T23:23:35.747 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:23:35.855 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627782 2024-04-24T23:23:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:23:35.864 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:23:35.864 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:23:35.864 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:23:35.864 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:23:35.865 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-04-24T23:23:35.865 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:23:35.971 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627783 2024-04-24T23:23:35.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:23:35.979 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-04-24T23:23:36.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.062+0000 7fb855a7ef40 1 Processor -- start 2024-04-24T23:23:36.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.062+0000 7fb855a7ef40 1 -- start start 2024-04-24T23:23:36.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb855a7ef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9bfffa90 0x55eb9c02abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb855a7ef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 0x55eb9c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb855a7ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9c044250 0x55eb9c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb855a7ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb9c0449b0 con 0x55eb9c044250 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb855a7ef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bfa4640 con 0x55eb9bfffa90 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb855a7ef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55eb9c044c90 con 0x55eb9bfffe70 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb853d80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9c044250 0x55eb9c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb853d80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9c044250 0x55eb9c0c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34808/0 (socket says 172.21.15.26:34808) 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb853d80640 1 -- 172.21.15.26:0/3435291944 learned_addr learned my addr 172.21.15.26:0/3435291944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb852d7e640 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 0x55eb9c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb85357f640 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9bfffa90 0x55eb9c02abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb853d80640 1 -- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 msgr2=0x55eb9c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb853d80640 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 0x55eb9c0c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb853d80640 1 -- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9bfffa90 msgr2=0x55eb9c02abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb853d80640 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9bfffa90 0x55eb9c02abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb853d80640 1 -- 172.21.15.26:0/3435291944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9c0d0490 con 0x55eb9c044250 2024-04-24T23:23:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb852d7e640 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 0x55eb9c0c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.063+0000 7fb85357f640 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9bfffa90 0x55eb9c02abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb853d80640 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9c044250 0x55eb9c0c5a80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00a4b0 tx=0x7fb84c050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a06ea8ad0447fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb851d7c640 1 -- 172.21.15.26:0/3435291944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c054040 con 0x55eb9c044250 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb851d7c640 1 -- 172.21.15.26:0/3435291944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84c050870 con 0x55eb9c044250 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb851d7c640 1 -- 172.21.15.26:0/3435291944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c050c40 con 0x55eb9c044250 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3435291944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9c044250 msgr2=0x55eb9c0c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb855a7ef40 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9c044250 0x55eb9c0c5a80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00a4b0 tx=0x7fb84c050000 comp rx=0 tx=0).stop 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3435291944 shutdown_connections 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb855a7ef40 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9c044250 0x55eb9c0c5a80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb855a7ef40 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 0x55eb9c0c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb855a7ef40 1 --2- 172.21.15.26:0/3435291944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9bfffa90 0x55eb9c02abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3435291944 >> 172.21.15.26:0/3435291944 conn(0x55eb9c027b90 msgr2=0x55eb9c0d4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3435291944 shutdown_connections 2024-04-24T23:23:36.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.064+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3435291944 wait complete. 2024-04-24T23:23:36.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.065+0000 7fb855a7ef40 1 Processor -- start 2024-04-24T23:23:36.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.065+0000 7fb855a7ef40 1 -- start start 2024-04-24T23:23:36.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb855a7ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9bfffa90 0x55eb9c0bff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb855a7ef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 0x55eb9c0c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb855a7ef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9c044250 0x55eb9c0c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb855a7ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb9c0c5030 con 0x55eb9bfffa90 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb855a7ef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55eb9c0cfbf0 con 0x55eb9c044250 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb855a7ef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55eb9c0273c0 con 0x55eb9bfffe70 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb85357f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9bfffa90 0x55eb9c0bff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb85357f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9bfffa90 0x55eb9c0bff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34824/0 (socket says 172.21.15.26:34824) 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb85357f640 1 -- 172.21.15.26:0/3439511893 learned_addr learned my addr 172.21.15.26:0/3439511893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb852d7e640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 0x55eb9c0c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb853d80640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9c044250 0x55eb9c0c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb85357f640 1 -- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 msgr2=0x55eb9c0c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb85357f640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55eb9bfffe70 0x55eb9c0c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb85357f640 1 -- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9c044250 msgr2=0x55eb9c0c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb85357f640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eb9c044250 0x55eb9c0c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb85357f640 1 -- 172.21.15.26:0/3439511893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9c0c4420 con 0x55eb9bfffa90 2024-04-24T23:23:36.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb85357f640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9bfffa90 0x55eb9c0bff40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb84400bf80 tx=0x7fb8440074b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb850d7a640 1 -- 172.21.15.26:0/3439511893 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84400b0d0 con 0x55eb9bfffa90 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb850d7a640 1 -- 172.21.15.26:0/3439511893 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84400b270 con 0x55eb9bfffa90 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb850d7a640 1 -- 172.21.15.26:0/3439511893 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844015660 con 0x55eb9bfffa90 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.066+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55eb9c0c0480 con 0x55eb9bfffa90 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.067+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9c0460a0 con 0x55eb9bfffa90 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.068+0000 7fb850d7a640 1 -- 172.21.15.26:0/3439511893 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb84401e440 con 0x55eb9bfffa90 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.068+0000 7fb850d7a640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb84006a810 0x7fb84006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.068+0000 7fb850d7a640 1 -- 172.21.15.26:0/3439511893 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fb8440892d0 con 0x55eb9bfffa90 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.068+0000 7fb852d7e640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb84006a810 0x7fb84006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.068+0000 7fb855a7ef40 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55eb9c0d5db0 0x55eb9c0d61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.069+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 --> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45) -- 0x55eb9c0d66f0 con 0x55eb9c0d5db0 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.069+0000 7fb853d80640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55eb9c0d5db0 0x55eb9c0d61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.069+0000 7fb852d7e640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb84006a810 0x7fb84006ccd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55eb9c0441b0 tx=0x7fb8480017a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.069+0000 7fb853d80640 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55eb9c0d5db0 0x55eb9c0d61b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb853d80640 1 -- 172.21.15.26:0/3439511893 <== osd.7 v2:172.21.15.29:6824/2604664408 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v45'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb84c056030 con 0x55eb9c0d5db0 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55eb9c0d5db0 msgr2=0x55eb9c0d61b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55eb9c0d5db0 0x55eb9c0d61b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb84006a810 msgr2=0x7fb84006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb84006a810 0x7fb84006ccd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55eb9c0441b0 tx=0x7fb8480017a0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9bfffa90 msgr2=0x55eb9c0bff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 --2- 172.21.15.26:0/3439511893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9bfffa90 0x55eb9c0bff40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb84400bf80 tx=0x7fb8440074b0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb853d80640 1 -- 172.21.15.26:0/3439511893 reap_dead start 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 shutdown_connections 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 >> 172.21.15.26:0/3439511893 conn(0x55eb9c027b90 msgr2=0x55eb9c0c1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 shutdown_connections 2024-04-24T23:23:36.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.073+0000 7fb855a7ef40 1 -- 172.21.15.26:0/3439511893 wait complete. 2024-04-24T23:23:36.077 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-04-24T23:23:36.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.118+0000 7f2f5f7d0f40 1 Processor -- start 2024-04-24T23:23:36.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.118+0000 7f2f5f7d0f40 1 -- start start 2024-04-24T23:23:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5f7d0f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 0x55bc61455ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5f7d0f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 0x55bc61466c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5f7d0f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc61467170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5f7d0f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bc613d89b0 con 0x55bc613d8250 2024-04-24T23:23:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5f7d0f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bc61338640 con 0x55bc61393a90 2024-04-24T23:23:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5f7d0f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55bc613d8c90 con 0x55bc61393e70 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5dad2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc61467170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5dad2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc61467170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34852/0 (socket says 172.21.15.26:34852) 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5dad2640 1 -- 172.21.15.26:0/3707887180 learned_addr learned my addr 172.21.15.26:0/3707887180 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5dad2640 1 -- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 msgr2=0x55bc61466c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5cad0640 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 0x55bc61466c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5d2d1640 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 0x55bc61455ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5dad2640 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 0x55bc61466c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5dad2640 1 -- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 msgr2=0x55bc61455ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5dad2640 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 0x55bc61455ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5dad2640 1 -- 172.21.15.26:0/3707887180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc61456530 con 0x55bc613d8250 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5cad0640 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 0x55bc61466c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5d2d1640 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 0x55bc61455ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f5dad2640 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc61467170 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2f50002850 tx=0x7f2f50002d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d724bd93ff5658 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.119+0000 7f2f577fe640 1 -- 172.21.15.26:0/3707887180 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f50016020 con 0x55bc613d8250 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f577fe640 1 -- 172.21.15.26:0/3707887180 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5000d9c0 con 0x55bc613d8250 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f577fe640 1 -- 172.21.15.26:0/3707887180 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5000ddb0 con 0x55bc613d8250 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/3707887180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 msgr2=0x55bc61467170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f5f7d0f40 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc61467170 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2f50002850 tx=0x7f2f50002d10 comp rx=0 tx=0).stop 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/3707887180 shutdown_connections 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f5f7d0f40 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc61467170 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f5f7d0f40 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 0x55bc61466c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f5f7d0f40 1 --2- 172.21.15.26:0/3707887180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 0x55bc61455ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/3707887180 >> 172.21.15.26:0/3707887180 conn(0x55bc613bbb90 msgr2=0x55bc613c0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/3707887180 shutdown_connections 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.120+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/3707887180 wait complete. 2024-04-24T23:23:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5f7d0f40 1 Processor -- start 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5f7d0f40 1 -- start start 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5f7d0f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 0x55bc6146ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5f7d0f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 0x55bc6146b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5f7d0f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc6145a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5f7d0f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bc614596f0 con 0x55bc613d8250 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5f7d0f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bc61465160 con 0x55bc61393a90 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5f7d0f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55bc614599d0 con 0x55bc61393e70 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5dad2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc6145a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5dad2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc6145a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34868/0 (socket says 172.21.15.26:34868) 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5dad2640 1 -- 172.21.15.26:0/4236240783 learned_addr learned my addr 172.21.15.26:0/4236240783 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5d2d1640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 0x55bc6146ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.121+0000 7f2f5cad0640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 0x55bc6146b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f5dad2640 1 -- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 msgr2=0x55bc6146b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f5dad2640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bc61393e70 0x55bc6146b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f5dad2640 1 -- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 msgr2=0x55bc6146ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f5dad2640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bc61393a90 0x55bc6146ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f5dad2640 1 -- 172.21.15.26:0/4236240783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc61452c40 con 0x55bc613d8250 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f5dad2640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc6145a280 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55bc614649c0 tx=0x7f2f500033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f567fc640 1 -- 172.21.15.26:0/4236240783 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f50016030 con 0x55bc613d8250 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f567fc640 1 -- 172.21.15.26:0/4236240783 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5001f070 con 0x55bc613d8250 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc61455790 con 0x55bc613d8250 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f567fc640 1 -- 172.21.15.26:0/4236240783 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5001b660 con 0x55bc613d8250 2024-04-24T23:23:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.122+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc61465c40 con 0x55bc613d8250 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.124+0000 7f2f567fc640 1 -- 172.21.15.26:0/4236240783 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f50019020 con 0x55bc613d8250 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.124+0000 7f2f567fc640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2f4006a840 0x7f2f4006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.124+0000 7f2f567fc640 1 -- 172.21.15.26:0/4236240783 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f2f5008b560 con 0x55bc613d8250 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.124+0000 7f2f5d2d1640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2f4006a840 0x7f2f4006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.124+0000 7f2f5f7d0f40 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55bc6146bd00 0x55bc61502e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.124+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 --> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] -- osd_op(unknown.0.0:1 6.12 6:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45) -- 0x55bc6146c450 con 0x55bc6146bd00 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.125+0000 7f2f5d2d1640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2f4006a840 0x7f2f4006cd00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f58057000 tx=0x7f2f58061040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.125+0000 7f2f5cad0640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55bc6146bd00 0x55bc61502e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.125+0000 7f2f5cad0640 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55bc6146bd00 0x55bc61502e20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.126+0000 7f2f5cad0640 1 -- 172.21.15.26:0/4236240783 <== osd.7 v2:172.21.15.29:6824/2604664408 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v45'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2f4800f6c0 con 0x55bc6146bd00 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.126+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55bc6146bd00 msgr2=0x55bc61502e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.126+0000 7f2f5f7d0f40 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55bc6146bd00 0x55bc61502e20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.127+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2f4006a840 msgr2=0x7f2f4006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.127+0000 7f2f5f7d0f40 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2f4006a840 0x7f2f4006cd00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f58057000 tx=0x7f2f58061040 comp rx=0 tx=0).stop 2024-04-24T23:23:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.127+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 msgr2=0x55bc6145a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.127+0000 7f2f5f7d0f40 1 --2- 172.21.15.26:0/4236240783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bc613d8250 0x55bc6145a280 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55bc614649c0 tx=0x7f2f500033a0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.127+0000 7f2f5dad2640 1 -- 172.21.15.26:0/4236240783 reap_dead start 2024-04-24T23:23:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.127+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 shutdown_connections 2024-04-24T23:23:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.127+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 >> 172.21.15.26:0/4236240783 conn(0x55bc613bbb90 msgr2=0x55bc61466b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.127+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 shutdown_connections 2024-04-24T23:23:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.127+0000 7f2f5f7d0f40 1 -- 172.21.15.26:0/4236240783 wait complete. 2024-04-24T23:23:36.132 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-04-24T23:23:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.213+0000 7f3d4755bf40 1 Processor -- start 2024-04-24T23:23:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.213+0000 7f3d4755bf40 1 -- start start 2024-04-24T23:23:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4755bf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1af3620 0x5614a1af2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4755bf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3b920 0x5614a1c59f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4755bf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5614a1b3bd00 0x5614a1c5a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4755bf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5614a1af11b0 con 0x5614a1b3b920 2024-04-24T23:23:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4755bf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5614a1a99640 con 0x5614a1af3620 2024-04-24T23:23:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4755bf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5614a1af5290 con 0x5614a1b3bd00 2024-04-24T23:23:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4485b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3b920 0x5614a1c59f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4485b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3b920 0x5614a1c59f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34888/0 (socket says 172.21.15.26:34888) 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4485b640 1 -- 172.21.15.26:0/195433400 learned_addr learned my addr 172.21.15.26:0/195433400 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4505c640 1 --2- 172.21.15.26:0/195433400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1af3620 0x5614a1af2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4585d640 1 --2- 172.21.15.26:0/195433400 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5614a1b3bd00 0x5614a1c5a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4485b640 1 -- 172.21.15.26:0/195433400 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5614a1b3bd00 msgr2=0x5614a1c5a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4485b640 1 --2- 172.21.15.26:0/195433400 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5614a1b3bd00 0x5614a1c5a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4485b640 1 -- 172.21.15.26:0/195433400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1af3620 msgr2=0x5614a1af2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4485b640 1 --2- 172.21.15.26:0/195433400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1af3620 0x5614a1af2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4485b640 1 -- 172.21.15.26:0/195433400 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a1bbd900 con 0x5614a1b3b920 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d4485b640 1 --2- 172.21.15.26:0/195433400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3b920 0x5614a1c59f60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000a770 tx=0x7f3d3000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a071bb8913da025e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d3f7fe640 1 -- 172.21.15.26:0/195433400 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30019070 con 0x5614a1b3b920 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d3f7fe640 1 -- 172.21.15.26:0/195433400 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d30007600 con 0x5614a1b3b920 2024-04-24T23:23:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.214+0000 7f3d3f7fe640 1 -- 172.21.15.26:0/195433400 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d300079d0 con 0x5614a1b3b920 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 -- 172.21.15.26:0/195433400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3b920 msgr2=0x5614a1c59f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 --2- 172.21.15.26:0/195433400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3b920 0x5614a1c59f60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000a770 tx=0x7f3d3000a860 comp rx=0 tx=0).stop 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 -- 172.21.15.26:0/195433400 shutdown_connections 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 --2- 172.21.15.26:0/195433400 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5614a1b3bd00 0x5614a1c5a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 --2- 172.21.15.26:0/195433400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3b920 0x5614a1c59f60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 --2- 172.21.15.26:0/195433400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1af3620 0x5614a1af2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 -- 172.21.15.26:0/195433400 >> 172.21.15.26:0/195433400 conn(0x5614a1b1ebb0 msgr2=0x5614a1bca130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 -- 172.21.15.26:0/195433400 shutdown_connections 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 -- 172.21.15.26:0/195433400 wait complete. 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.215+0000 7f3d4755bf40 1 Processor -- start 2024-04-24T23:23:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4755bf40 1 -- start start 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4755bf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5614a1af3620 0x5614a1bad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4755bf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1b3b920 0x5614a1bc5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4755bf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3bd00 0x5614a1bc60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4755bf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5614a1bc79e0 con 0x5614a1b3bd00 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4755bf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5614a1bc0f00 con 0x5614a1b3b920 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4755bf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5614a1bbd620 con 0x5614a1af3620 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4585d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3bd00 0x5614a1bc60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4585d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3bd00 0x5614a1bc60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34898/0 (socket says 172.21.15.26:34898) 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4585d640 1 -- 172.21.15.26:0/3047419937 learned_addr learned my addr 172.21.15.26:0/3047419937 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4505c640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5614a1af3620 0x5614a1bad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.216+0000 7f3d4485b640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1b3b920 0x5614a1bc5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d4585d640 1 -- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5614a1af3620 msgr2=0x5614a1bad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d4585d640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5614a1af3620 0x5614a1bad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d4585d640 1 -- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1b3b920 msgr2=0x5614a1bc5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d4585d640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1b3b920 0x5614a1bc5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d4585d640 1 -- 172.21.15.26:0/3047419937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a1bba860 con 0x5614a1b3bd00 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d4485b640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5614a1b3b920 0x5614a1bc5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d4585d640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3bd00 0x5614a1bc60a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800a5a0 tx=0x7f3d38002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d3e7fc640 1 -- 172.21.15.26:0/3047419937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d380538f0 con 0x5614a1b3bd00 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d3e7fc640 1 -- 172.21.15.26:0/3047419937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d38053a90 con 0x5614a1b3bd00 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5614a1bb98e0 con 0x5614a1b3bd00 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d3e7fc640 1 -- 172.21.15.26:0/3047419937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38053e60 con 0x5614a1b3bd00 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.217+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a1bae9d0 con 0x5614a1b3bd00 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.218+0000 7f3d3e7fc640 1 -- 172.21.15.26:0/3047419937 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d38072440 con 0x5614a1b3bd00 2024-04-24T23:23:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.219+0000 7f3d3e7fc640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d2806a840 0x7f3d2806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.219+0000 7f3d3e7fc640 1 -- 172.21.15.26:0/3047419937 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f3d380ce5a0 con 0x5614a1b3bd00 2024-04-24T23:23:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.219+0000 7f3d4505c640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d2806a840 0x7f3d2806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.219+0000 7f3d4755bf40 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x5614a1baf4b0 0x5614a1baf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.219+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 --> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] -- osd_op(unknown.0.0:1 6.5 6:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45) -- 0x5614a1bafdd0 con 0x5614a1baf4b0 2024-04-24T23:23:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.219+0000 7f3d4505c640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d2806a840 0x7f3d2806cd00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5614a1b1c0d0 tx=0x7f3d40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.219+0000 7f3d4485b640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x5614a1baf4b0 0x5614a1baf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.220+0000 7f3d4485b640 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x5614a1baf4b0 0x5614a1baf890 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4485b640 1 -- 172.21.15.26:0/3047419937 <== osd.7 v2:172.21.15.29:6824/2604664408 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v45'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d30007bd0 con 0x5614a1baf4b0 2024-04-24T23:23:36.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x5614a1baf4b0 msgr2=0x5614a1baf890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4755bf40 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x5614a1baf4b0 0x5614a1baf890 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d2806a840 msgr2=0x7f3d2806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4755bf40 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3d2806a840 0x7f3d2806cd00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5614a1b1c0d0 tx=0x7f3d40008040 comp rx=0 tx=0).stop 2024-04-24T23:23:36.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3bd00 msgr2=0x5614a1bc60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4755bf40 1 --2- 172.21.15.26:0/3047419937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614a1b3bd00 0x5614a1bc60a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800a5a0 tx=0x7f3d38002fb0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4585d640 1 -- 172.21.15.26:0/3047419937 reap_dead start 2024-04-24T23:23:36.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 shutdown_connections 2024-04-24T23:23:36.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.221+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 >> 172.21.15.26:0/3047419937 conn(0x5614a1b1ebb0 msgr2=0x5614a1bbdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.222+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 shutdown_connections 2024-04-24T23:23:36.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.222+0000 7f3d4755bf40 1 -- 172.21.15.26:0/3047419937 wait complete. 2024-04-24T23:23:36.227 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-04-24T23:23:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.307+0000 7fe12a892f40 1 Processor -- start 2024-04-24T23:23:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe12a892f40 1 -- start start 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe12a892f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55817668b710 0x5581767bfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe12a892f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 0x558176733bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe12a892f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581767307a0 0x558176731850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe12a892f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55817665d940 con 0x5581767307a0 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe12a892f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558176602640 con 0x5581766a4430 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe12a892f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5581766a4c20 con 0x55817668b710 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe128b96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581767307a0 0x558176731850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe128b96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581767307a0 0x558176731850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34916/0 (socket says 172.21.15.26:34916) 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe128b96640 1 -- 172.21.15.26:0/3647008571 learned_addr learned my addr 172.21.15.26:0/3647008571 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe128395640 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55817668b710 0x5581767bfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.308+0000 7fe127b94640 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 0x558176733bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe128b96640 1 -- 172.21.15.26:0/3647008571 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55817668b710 msgr2=0x5581767bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe128b96640 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55817668b710 0x5581767bfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe128b96640 1 -- 172.21.15.26:0/3647008571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 msgr2=0x558176733bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe128b96640 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 0x558176733bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe128b96640 1 -- 172.21.15.26:0/3647008571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55817672fa90 con 0x5581767307a0 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe127b94640 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 0x558176733bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe128b96640 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581767307a0 0x558176731850 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe11800a4b0 tx=0x7fe11800a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe47770c12c1b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe126b92640 1 -- 172.21.15.26:0/3647008571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118015070 con 0x5581767307a0 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe126b92640 1 -- 172.21.15.26:0/3647008571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11800e710 con 0x5581767307a0 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe126b92640 1 -- 172.21.15.26:0/3647008571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11800eae0 con 0x5581767307a0 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe12a892f40 1 -- 172.21.15.26:0/3647008571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581767307a0 msgr2=0x558176731850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.309+0000 7fe12a892f40 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581767307a0 0x558176731850 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe11800a4b0 tx=0x7fe11800a970 comp rx=0 tx=0).stop 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.310+0000 7fe12a892f40 1 -- 172.21.15.26:0/3647008571 shutdown_connections 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.310+0000 7fe12a892f40 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5581767307a0 0x558176731850 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.310+0000 7fe12a892f40 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 0x558176733bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.310+0000 7fe12a892f40 1 --2- 172.21.15.26:0/3647008571 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55817668b710 0x5581767bfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.310+0000 7fe12a892f40 1 -- 172.21.15.26:0/3647008571 >> 172.21.15.26:0/3647008571 conn(0x558176659400 msgr2=0x558176730b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.310+0000 7fe12a892f40 1 -- 172.21.15.26:0/3647008571 shutdown_connections 2024-04-24T23:23:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.310+0000 7fe12a892f40 1 -- 172.21.15.26:0/3647008571 wait complete. 2024-04-24T23:23:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.310+0000 7fe12a892f40 1 Processor -- start 2024-04-24T23:23:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.310+0000 7fe12a892f40 1 -- start start 2024-04-24T23:23:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe12a892f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55817668b710 0x55817672e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe12a892f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 0x558176731f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe12a892f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5581767307a0 0x558176733cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe12a892f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55817668b050 con 0x55817668b710 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe12a892f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55817672f840 con 0x5581766a4430 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe12a892f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55817668ace0 con 0x5581767307a0 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128395640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55817668b710 0x55817672e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128395640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55817668b710 0x55817672e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34926/0 (socket says 172.21.15.26:34926) 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128395640 1 -- 172.21.15.26:0/9498068 learned_addr learned my addr 172.21.15.26:0/9498068 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe127b94640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 0x558176731f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128b96640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5581767307a0 0x558176733cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128395640 1 -- 172.21.15.26:0/9498068 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5581767307a0 msgr2=0x558176733cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128395640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5581767307a0 0x558176733cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128395640 1 -- 172.21.15.26:0/9498068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 msgr2=0x558176731f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128395640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 0x558176731f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128395640 1 -- 172.21.15.26:0/9498068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558176687ea0 con 0x55817668b710 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe127b94640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5581766a4430 0x558176731f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128b96640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5581767307a0 0x558176733cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.311+0000 7fe128395640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55817668b710 0x55817672e390 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe120000c00 tx=0x7fe120003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.312+0000 7fe125b90640 1 -- 172.21.15.26:0/9498068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12006d070 con 0x55817668b710 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.312+0000 7fe125b90640 1 -- 172.21.15.26:0/9498068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe120052d80 con 0x55817668b710 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.312+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55817672d2f0 con 0x55817668b710 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.312+0000 7fe125b90640 1 -- 172.21.15.26:0/9498068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200686a0 con 0x55817668b710 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.312+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817672d8b0 con 0x55817668b710 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.313+0000 7fe125b90640 1 -- 172.21.15.26:0/9498068 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe120070070 con 0x55817668b710 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.313+0000 7fe125b90640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10806a810 0x7fe10806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.313+0000 7fe125b90640 1 -- 172.21.15.26:0/9498068 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fe1200ce190 con 0x55817668b710 2024-04-24T23:23:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.314+0000 7fe127b94640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10806a810 0x7fe10806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.314+0000 7fe12a892f40 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x55817665e8a0 0x55817669f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.314+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- osd_op(unknown.0.0:1 6.9 6:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45) -- 0x55817669f840 con 0x55817665e8a0 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.314+0000 7fe128b96640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x55817665e8a0 0x55817669f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.314+0000 7fe127b94640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10806a810 0x7fe10806ccd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55817672f380 tx=0x7fe110002660 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.314+0000 7fe128b96640 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x55817665e8a0 0x55817669f300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe128b96640 1 -- 172.21.15.26:0/9498068 <== osd.0 v2:172.21.15.26:6819/256768257 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v45'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe118018040 con 0x55817665e8a0 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x55817665e8a0 msgr2=0x55817669f300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x55817665e8a0 0x55817669f300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10806a810 msgr2=0x7fe10806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10806a810 0x7fe10806ccd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55817672f380 tx=0x7fe110002660 comp rx=0 tx=0).stop 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55817668b710 msgr2=0x55817672e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 --2- 172.21.15.26:0/9498068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55817668b710 0x55817672e390 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe120000c00 tx=0x7fe120003020 comp rx=0 tx=0).stop 2024-04-24T23:23:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe128b96640 1 -- 172.21.15.26:0/9498068 reap_dead start 2024-04-24T23:23:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 shutdown_connections 2024-04-24T23:23:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 >> 172.21.15.26:0/9498068 conn(0x558176659400 msgr2=0x5581766a3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 shutdown_connections 2024-04-24T23:23:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.316+0000 7fe12a892f40 1 -- 172.21.15.26:0/9498068 wait complete. 2024-04-24T23:23:36.322 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-04-24T23:23:36.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.404+0000 7f3294e10f40 1 Processor -- start 2024-04-24T23:23:36.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.404+0000 7f3294e10f40 1 -- start start 2024-04-24T23:23:36.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3294e10f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32800022d0 0x55a8be756430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3294e10f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a8be6b2220 0x55a8be75a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3294e10f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8be75ab00 0x55a8be754ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3294e10f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8be62a640 con 0x7f32800022d0 2024-04-24T23:23:36.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3294e10f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a8be62e680 con 0x55a8be75ab00 2024-04-24T23:23:36.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3294e10f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a8be6ca9b0 con 0x55a8be6b2220 2024-04-24T23:23:36.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3291ba1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32800022d0 0x55a8be756430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3291ba1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32800022d0 0x55a8be756430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34940/0 (socket says 172.21.15.26:34940) 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3291ba1640 1 -- 172.21.15.26:0/875790381 learned_addr learned my addr 172.21.15.26:0/875790381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3291ba1640 1 -- 172.21.15.26:0/875790381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a8be6b2220 msgr2=0x55a8be75a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f328bfff640 1 --2- 172.21.15.26:0/875790381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a8be6b2220 0x55a8be75a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3291ba1640 1 --2- 172.21.15.26:0/875790381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a8be6b2220 0x55a8be75a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f32923a2640 1 --2- 172.21.15.26:0/875790381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8be75ab00 0x55a8be754ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3291ba1640 1 -- 172.21.15.26:0/875790381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8be75ab00 msgr2=0x55a8be754ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3291ba1640 1 --2- 172.21.15.26:0/875790381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8be75ab00 0x55a8be754ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3291ba1640 1 -- 172.21.15.26:0/875790381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8be7e95c0 con 0x7f32800022d0 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3291ba1640 1 --2- 172.21.15.26:0/875790381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32800022d0 0x55a8be756430 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3284001cb0 tx=0x7f328404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4795609d219f9468 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.405+0000 7f3290b9f640 1 -- 172.21.15.26:0/875790381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284063070 con 0x7f32800022d0 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3290b9f640 1 -- 172.21.15.26:0/875790381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32840549a0 con 0x7f32800022d0 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3290b9f640 1 -- 172.21.15.26:0/875790381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284059040 con 0x7f32800022d0 2024-04-24T23:23:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3294e10f40 1 -- 172.21.15.26:0/875790381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32800022d0 msgr2=0x55a8be756430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3294e10f40 1 --2- 172.21.15.26:0/875790381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32800022d0 0x55a8be756430 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3284001cb0 tx=0x7f328404e000 comp rx=0 tx=0).stop 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3294e10f40 1 -- 172.21.15.26:0/875790381 shutdown_connections 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3294e10f40 1 --2- 172.21.15.26:0/875790381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8be75ab00 0x55a8be754ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3294e10f40 1 --2- 172.21.15.26:0/875790381 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a8be6b2220 0x55a8be75a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3294e10f40 1 --2- 172.21.15.26:0/875790381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32800022d0 0x55a8be756430 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3294e10f40 1 -- 172.21.15.26:0/875790381 >> 172.21.15.26:0/875790381 conn(0x55a8be6adb90 msgr2=0x55a8be6ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3294e10f40 1 -- 172.21.15.26:0/875790381 shutdown_connections 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.406+0000 7f3294e10f40 1 -- 172.21.15.26:0/875790381 wait complete. 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3294e10f40 1 Processor -- start 2024-04-24T23:23:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3294e10f40 1 -- start start 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3294e10f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8be6b2220 0x55a8be7597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3294e10f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8be75ab00 0x55a8be6ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3294e10f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f32800022d0 0x55a8be6cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3294e10f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8be756150 con 0x55a8be6b2220 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3294e10f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a8be755c20 con 0x55a8be75ab00 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3294e10f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a8be755e70 con 0x7f32800022d0 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3291ba1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8be6b2220 0x55a8be7597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3291ba1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8be6b2220 0x55a8be7597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34948/0 (socket says 172.21.15.26:34948) 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.407+0000 7f3291ba1640 1 -- 172.21.15.26:0/3075929738 learned_addr learned my addr 172.21.15.26:0/3075929738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f32923a2640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f32800022d0 0x55a8be6cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f328bfff640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8be75ab00 0x55a8be6ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f3291ba1640 1 -- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f32800022d0 msgr2=0x55a8be6cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f3291ba1640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f32800022d0 0x55a8be6cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f3291ba1640 1 -- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8be75ab00 msgr2=0x55a8be6ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f3291ba1640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8be75ab00 0x55a8be6ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f3291ba1640 1 -- 172.21.15.26:0/3075929738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8be758ff0 con 0x55a8be6b2220 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f3291ba1640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8be6b2220 0x55a8be7597d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3284001d90 tx=0x7f3284001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f328affd640 1 -- 172.21.15.26:0/3075929738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284063040 con 0x55a8be6b2220 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f328affd640 1 -- 172.21.15.26:0/3075929738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328405e410 con 0x55a8be6b2220 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a8be754640 con 0x55a8be6b2220 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f328affd640 1 -- 172.21.15.26:0/3075929738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328405e820 con 0x55a8be6b2220 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.408+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8be7592b0 con 0x55a8be6b2220 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.410+0000 7f328affd640 1 -- 172.21.15.26:0/3075929738 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f328406c3a0 con 0x55a8be6b2220 2024-04-24T23:23:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.410+0000 7f328affd640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f327406a810 0x7f327406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.410+0000 7f328affd640 1 -- 172.21.15.26:0/3075929738 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f32840cd650 con 0x55a8be6b2220 2024-04-24T23:23:36.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.410+0000 7f328bfff640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f327406a810 0x7f327406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.410+0000 7f3294e10f40 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55a8be747e20 0x55a8be748230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.410+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 --> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] -- osd_op(unknown.0.0:1 6.5 6:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45) -- 0x55a8be75ceb0 con 0x55a8be747e20 2024-04-24T23:23:36.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.410+0000 7f32923a2640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55a8be747e20 0x55a8be748230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.410+0000 7f328bfff640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f327406a810 0x7f327406ccd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55a8be6cb600 tx=0x7f327c000f80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.411+0000 7f32923a2640 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55a8be747e20 0x55a8be748230 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f32923a2640 1 -- 172.21.15.26:0/3075929738 <== osd.7 v2:172.21.15.29:6824/2604664408 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v45'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f328c012820 con 0x55a8be747e20 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55a8be747e20 msgr2=0x55a8be748230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x55a8be747e20 0x55a8be748230 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f327406a810 msgr2=0x7f327406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f327406a810 0x7f327406ccd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55a8be6cb600 tx=0x7f327c000f80 comp rx=0 tx=0).stop 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8be6b2220 msgr2=0x55a8be7597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 --2- 172.21.15.26:0/3075929738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8be6b2220 0x55a8be7597d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3284001d90 tx=0x7f3284001120 comp rx=0 tx=0).stop 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f32923a2640 1 -- 172.21.15.26:0/3075929738 reap_dead start 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 shutdown_connections 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 >> 172.21.15.26:0/3075929738 conn(0x55a8be6adb90 msgr2=0x55a8be6b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 shutdown_connections 2024-04-24T23:23:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.412+0000 7f3294e10f40 1 -- 172.21.15.26:0/3075929738 wait complete. 2024-04-24T23:23:36.418 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-04-24T23:23:36.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.499+0000 7ff611e5ef40 1 Processor -- start 2024-04-24T23:23:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.499+0000 7ff611e5ef40 1 -- start start 2024-04-24T23:23:36.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff611e5ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd75fe0 0x55f2bdd7b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff611e5ef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2bdd95920 0x55f2bde24100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff611e5ef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd95d00 0x55f2bde225b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff611e5ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2bdcf3640 con 0x55f2bdd75fe0 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff611e5ef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f2bdcf7680 con 0x55f2bdd95920 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff611e5ef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f2bdd4b1b0 con 0x55f2bdd95d00 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff60f95f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd75fe0 0x55f2bdd7b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff60f95f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd75fe0 0x55f2bdd7b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34962/0 (socket says 172.21.15.26:34962) 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff60f95f640 1 -- 172.21.15.26:0/3758376979 learned_addr learned my addr 172.21.15.26:0/3758376979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff60f15e640 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2bdd95920 0x55f2bde24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff610160640 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd95d00 0x55f2bde225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff60f95f640 1 -- 172.21.15.26:0/3758376979 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd95d00 msgr2=0x55f2bde225b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff60f95f640 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd95d00 0x55f2bde225b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff60f95f640 1 -- 172.21.15.26:0/3758376979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2bdd95920 msgr2=0x55f2bde24100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff60f95f640 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2bdd95920 0x55f2bde24100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff60f95f640 1 -- 172.21.15.26:0/3758376979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2bdd93220 con 0x55f2bdd75fe0 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.500+0000 7ff610160640 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd95d00 0x55f2bde225b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff60f95f640 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd75fe0 0x55f2bdd7b100 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff60000a520 tx=0x7ff60004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b451003b92a1160 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff60e15c640 1 -- 172.21.15.26:0/3758376979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600059040 con 0x55f2bdd75fe0 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff60e15c640 1 -- 172.21.15.26:0/3758376979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60000a020 con 0x55f2bdd75fe0 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff60e15c640 1 -- 172.21.15.26:0/3758376979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000546b0 con 0x55f2bdd75fe0 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff611e5ef40 1 -- 172.21.15.26:0/3758376979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd75fe0 msgr2=0x55f2bdd7b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff611e5ef40 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd75fe0 0x55f2bdd7b100 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff60000a520 tx=0x7ff60004e000 comp rx=0 tx=0).stop 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff611e5ef40 1 -- 172.21.15.26:0/3758376979 shutdown_connections 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff611e5ef40 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd95d00 0x55f2bde225b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff611e5ef40 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2bdd95920 0x55f2bde24100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff611e5ef40 1 --2- 172.21.15.26:0/3758376979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd75fe0 0x55f2bdd7b100 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff611e5ef40 1 -- 172.21.15.26:0/3758376979 >> 172.21.15.26:0/3758376979 conn(0x55f2bdd78bb0 msgr2=0x55f2bdd77700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff611e5ef40 1 -- 172.21.15.26:0/3758376979 shutdown_connections 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.501+0000 7ff611e5ef40 1 -- 172.21.15.26:0/3758376979 wait complete. 2024-04-24T23:23:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.502+0000 7ff611e5ef40 1 Processor -- start 2024-04-24T23:23:36.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.502+0000 7ff611e5ef40 1 -- start start 2024-04-24T23:23:36.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.502+0000 7ff611e5ef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd75fe0 0x55f2bdd77700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.502+0000 7ff611e5ef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2bdd95920 0x55f2bde22730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.502+0000 7ff611e5ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd95d00 0x55f2bde13c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.502+0000 7ff611e5ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2bde1ac30 con 0x55f2bdd95d00 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.502+0000 7ff611e5ef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f2bde16f40 con 0x55f2bdd95920 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.502+0000 7ff611e5ef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f2bde16c60 con 0x55f2bdd75fe0 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff610160640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd95d00 0x55f2bde13c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff610160640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd95d00 0x55f2bde13c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34964/0 (socket says 172.21.15.26:34964) 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff610160640 1 -- 172.21.15.26:0/1410945344 learned_addr learned my addr 172.21.15.26:0/1410945344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff60f15e640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2bdd95920 0x55f2bde22730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff60f95f640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd75fe0 0x55f2bdd77700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff610160640 1 -- 172.21.15.26:0/1410945344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd75fe0 msgr2=0x55f2bdd77700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff610160640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2bdd75fe0 0x55f2bdd77700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff610160640 1 -- 172.21.15.26:0/1410945344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2bdd95920 msgr2=0x55f2bde22730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff610160640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2bdd95920 0x55f2bde22730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff610160640 1 -- 172.21.15.26:0/1410945344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2bde21f10 con 0x55f2bdd95d00 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff610160640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd95d00 0x55f2bde13c30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c450 tx=0x7ff60800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff60d15a640 1 -- 172.21.15.26:0/1410945344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608017070 con 0x55f2bdd95d00 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff60d15a640 1 -- 172.21.15.26:0/1410945344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff608004540 con 0x55f2bdd95d00 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f2bde221f0 con 0x55f2bdd95d00 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff60d15a640 1 -- 172.21.15.26:0/1410945344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608004910 con 0x55f2bdd95d00 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.503+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2bde22c70 con 0x55f2bdd95d00 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.505+0000 7ff60d15a640 1 -- 172.21.15.26:0/1410945344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff60801a070 con 0x55f2bdd95d00 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.505+0000 7ff60d15a640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5fc06a810 0x7ff5fc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.505+0000 7ff60d15a640 1 -- 172.21.15.26:0/1410945344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7ff608089530 con 0x55f2bdd95d00 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.505+0000 7ff60f95f640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5fc06a810 0x7ff5fc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.505+0000 7ff611e5ef40 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55f2bde1fee0 0x55f2bde1be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.505+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 --> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] -- osd_op(unknown.0.0:1 6.1d 6:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45) -- 0x55f2bde1c350 con 0x55f2bde1fee0 2024-04-24T23:23:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.505+0000 7ff60f15e640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55f2bde1fee0 0x55f2bde1be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.506+0000 7ff60f95f640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5fc06a810 0x7ff5fc06ccd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff600060000 tx=0x7ff600051000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.506+0000 7ff60f15e640 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55f2bde1fee0 0x55f2bde1be10 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff60f15e640 1 -- 172.21.15.26:0/1410945344 <== osd.1 v2:172.21.15.26:6816/1500554534 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v45'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff60400e040 con 0x55f2bde1fee0 2024-04-24T23:23:36.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55f2bde1fee0 msgr2=0x55f2bde1be10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x55f2bde1fee0 0x55f2bde1be10 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5fc06a810 msgr2=0x7ff5fc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff5fc06a810 0x7ff5fc06ccd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff600060000 tx=0x7ff600051000 comp rx=0 tx=0).stop 2024-04-24T23:23:36.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd95d00 msgr2=0x55f2bde13c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 --2- 172.21.15.26:0/1410945344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2bdd95d00 0x55f2bde13c30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c450 tx=0x7ff60800c910 comp rx=0 tx=0).stop 2024-04-24T23:23:36.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff610160640 1 -- 172.21.15.26:0/1410945344 reap_dead start 2024-04-24T23:23:36.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 shutdown_connections 2024-04-24T23:23:36.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 >> 172.21.15.26:0/1410945344 conn(0x55f2bdd78bb0 msgr2=0x55f2bdd7b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 shutdown_connections 2024-04-24T23:23:36.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.509+0000 7ff611e5ef40 1 -- 172.21.15.26:0/1410945344 wait complete. 2024-04-24T23:23:36.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:36.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start /d1/d2/d3 recursive,force 2024-04-24T23:23:36.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.658+0000 7f9a49142640 1 Processor -- start 2024-04-24T23:23:36.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.658+0000 7f9a49142640 1 -- start start 2024-04-24T23:23:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.658+0000 7f9a49142640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9a4415f430 0x7f9a4406c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.658+0000 7f9a49142640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a4406c670 0x7f9a440b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.658+0000 7f9a49142640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4406ca90 con 0x7f9a44158a00 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.658+0000 7f9a49142640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9a44072610 con 0x7f9a4406c670 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.658+0000 7f9a49142640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9a440ae220 con 0x7f9a4415f430 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.659+0000 7f9a4359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a4406c670 0x7f9a440b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.659+0000 7f9a4359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a4406c670 0x7f9a440b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34974/0 (socket says 172.21.15.26:34974) 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.659+0000 7f9a4359e640 1 -- 172.21.15.26:0/4273069890 learned_addr learned my addr 172.21.15.26:0/4273069890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.659+0000 7f9a4259c640 1 --2- 172.21.15.26:0/4273069890 >> v2:172.21.15.29:3300/0 conn(0x7f9a4415f430 0x7f9a4406c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.659+0000 7f9a4359e640 1 -- 172.21.15.26:0/4273069890 >> v2:172.21.15.29:3300/0 conn(0x7f9a4415f430 msgr2=0x7f9a4406c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.659+0000 7f9a4359e640 1 --2- 172.21.15.26:0/4273069890 >> v2:172.21.15.29:3300/0 conn(0x7f9a4415f430 0x7f9a4406c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.659+0000 7f9a4359e640 1 -- 172.21.15.26:0/4273069890 >> v1:172.21.15.29:6789/0 conn(0x7f9a44158a00 legacy=0x7f9a440ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.659+0000 7f9a4359e640 1 -- 172.21.15.26:0/4273069890 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a440ae500 con 0x7f9a4406c670 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a4359e640 1 --2- 172.21.15.26:0/4273069890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 0x7f9a440b33c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9a44071670 tx=0x7f9a3c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e629ae065c1d1df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a41d9b640 1 -- 172.21.15.26:0/4273069890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c036070 con 0x7f9a4406c670 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a41d9b640 1 -- 172.21.15.26:0/4273069890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3c031600 con 0x7f9a4406c670 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a41d9b640 1 -- 172.21.15.26:0/4273069890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0319d0 con 0x7f9a4406c670 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a49142640 1 -- 172.21.15.26:0/4273069890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 msgr2=0x7f9a440b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a49142640 1 --2- 172.21.15.26:0/4273069890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 0x7f9a440b33c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9a44071670 tx=0x7f9a3c02a730 comp rx=0 tx=0).stop 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a49142640 1 -- 172.21.15.26:0/4273069890 shutdown_connections 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a49142640 1 --2- 172.21.15.26:0/4273069890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 0x7f9a440b33c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a49142640 1 --2- 172.21.15.26:0/4273069890 >> v2:172.21.15.29:3300/0 conn(0x7f9a4415f430 0x7f9a4406c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a49142640 1 -- 172.21.15.26:0/4273069890 >> 172.21.15.26:0/4273069890 conn(0x7f9a440716e0 msgr2=0x7f9a44071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a49142640 1 -- 172.21.15.26:0/4273069890 shutdown_connections 2024-04-24T23:23:36.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.660+0000 7f9a49142640 1 -- 172.21.15.26:0/4273069890 wait complete. 2024-04-24T23:23:36.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.662+0000 7f9a49142640 1 Processor -- start 2024-04-24T23:23:36.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.662+0000 7f9a49142640 1 -- start start 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a49142640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 0x7f9a4406f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a49142640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9a44158a00 0x7f9a44159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a49142640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a4415f430 0x7f9a4406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a49142640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9a440ae960 con 0x7f9a4406c670 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a49142640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9a440ae7e0 con 0x7f9a4415f430 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a49142640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9a440b3900 con 0x7f9a44158a00 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 0x7f9a4406f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 0x7f9a4406f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34978/0 (socket says 172.21.15.26:34978) 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a42d9d640 1 -- 172.21.15.26:0/1564504610 learned_addr learned my addr 172.21.15.26:0/1564504610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a4359e640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a4415f430 0x7f9a4406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a4259c640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9a44158a00 0x7f9a44159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a42d9d640 1 -- 172.21.15.26:0/1564504610 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9a44158a00 msgr2=0x7f9a44159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9a44158a00 0x7f9a44159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a42d9d640 1 -- 172.21.15.26:0/1564504610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a4415f430 msgr2=0x7f9a4406d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a4415f430 0x7f9a4406d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a42d9d640 1 -- 172.21.15.26:0/1564504610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4406df60 con 0x7f9a4406c670 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 0x7f9a4406f000 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400c400 tx=0x7f9a3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a2b7fe640 1 -- 172.21.15.26:0/1564504610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a340052d0 con 0x7f9a4406c670 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a2b7fe640 1 -- 172.21.15.26:0/1564504610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a34005490 con 0x7f9a4406c670 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a49142640 1 -- 172.21.15.26:0/1564504610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4406e240 con 0x7f9a4406c670 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.663+0000 7f9a2b7fe640 1 -- 172.21.15.26:0/1564504610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34007260 con 0x7f9a4406c670 2024-04-24T23:23:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.664+0000 7f9a49142640 1 -- 172.21.15.26:0/1564504610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4406e800 con 0x7f9a4406c670 2024-04-24T23:23:36.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.665+0000 7f9a2b7fe640 1 -- 172.21.15.26:0/1564504610 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a34007400 con 0x7f9a4406c670 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.665+0000 7f9a2b7fe640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9a3006be30 0x7f9a3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.666+0000 7f9a2b7fe640 1 -- 172.21.15.26:0/1564504610 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f9a3408a340 con 0x7f9a4406c670 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.666+0000 7f9a4259c640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9a3006be30 0x7f9a3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.666+0000 7f9a4259c640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9a3006be30 0x7f9a3006e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9a441ea2e0 tx=0x7f9a38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.666+0000 7f9a4be4d740 1 Processor -- start 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4be4d740 1 -- start start 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d16c60 0x55b544d321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d28680 0x55b544d17040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d13840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b544c4f490 con 0x55b544d28680 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b544bf7ae0 con 0x55b544d17580 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b544c4b450 con 0x55b544d16c60 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d28680 0x55b544d17040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d28680 0x55b544d17040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34984/0 (socket says 172.21.15.26:34984) 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4259c640 1 -- 172.21.15.26:0/828064658 learned_addr learned my addr 172.21.15.26:0/828064658 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/828064658 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d16c60 0x55b544d321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.667+0000 7f9a4359e640 1 --2- 172.21.15.26:0/828064658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d13840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.668+0000 7f9a4259c640 1 -- 172.21.15.26:0/828064658 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d16c60 msgr2=0x55b544d321f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.668+0000 7f9a4259c640 1 --2- 172.21.15.26:0/828064658 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d16c60 0x55b544d321f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.668+0000 7f9a4259c640 1 -- 172.21.15.26:0/828064658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 msgr2=0x55b544d13840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.668+0000 7f9a4259c640 1 --2- 172.21.15.26:0/828064658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d13840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.668+0000 7f9a4259c640 1 -- 172.21.15.26:0/828064658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b544a2ba50 con 0x55b544d28680 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.668+0000 7f9a4259c640 1 --2- 172.21.15.26:0/828064658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d28680 0x55b544d17040 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9a38010a30 tx=0x7f9a38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc0d2cad99d3f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.668+0000 7f9a28ff9640 1 -- 172.21.15.26:0/828064658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3801c050 con 0x55b544d28680 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.668+0000 7f9a28ff9640 1 -- 172.21.15.26:0/828064658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a38026070 con 0x55b544d28680 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.668+0000 7f9a28ff9640 1 -- 172.21.15.26:0/828064658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38016e10 con 0x55b544d28680 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 -- 172.21.15.26:0/828064658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d28680 msgr2=0x55b544d17040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/828064658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d28680 0x55b544d17040 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9a38010a30 tx=0x7f9a38013000 comp rx=0 tx=0).stop 2024-04-24T23:23:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 -- 172.21.15.26:0/828064658 shutdown_connections 2024-04-24T23:23:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/828064658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d13840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/828064658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d28680 0x55b544d17040 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/828064658 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d16c60 0x55b544d321f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 -- 172.21.15.26:0/828064658 >> 172.21.15.26:0/828064658 conn(0x55b544cb35e0 msgr2=0x55b544d31a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 -- 172.21.15.26:0/828064658 shutdown_connections 2024-04-24T23:23:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 -- 172.21.15.26:0/828064658 wait complete. 2024-04-24T23:23:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 -- start start 2024-04-24T23:23:36.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.669+0000 7f9a4be4d740 1 Processor -- start 2024-04-24T23:23:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d11070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d1a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544d30670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b544cedfa0 con 0x55b544d16c60 2024-04-24T23:23:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b544c0b060 con 0x55b544d17580 2024-04-24T23:23:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b544ce8820 con 0x55b544d28680 2024-04-24T23:23:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a4be4d740 10 client.0 _ll_register_callbacks cb 0x55b544c46970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:36.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d11070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d11070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34998/0 (socket says 172.21.15.26:34998) 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a42d9d640 1 -- 172.21.15.26:0/895871647 learned_addr learned my addr 172.21.15.26:0/895871647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a4259c640 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d1a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.670+0000 7f9a4359e640 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544d30670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a42d9d640 1 -- 172.21.15.26:0/895871647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 msgr2=0x55b544d30670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544d30670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a42d9d640 1 -- 172.21.15.26:0/895871647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 msgr2=0x55b544d1a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d1a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a42d9d640 1 -- 172.21.15.26:0/895871647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5449233e0 con 0x55b544d16c60 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d11070 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9a34000fb0 tx=0x7f9a3405c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2d3ae0343870d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a21ffb640 1 -- 172.21.15.26:0/895871647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34057aa0 con 0x55b544d16c60 2024-04-24T23:23:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a21ffb640 1 -- 172.21.15.26:0/895871647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f9a34057c40 con 0x55b544d16c60 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4be4d740 1 -- 172.21.15.26:0/895871647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a38026070 con 0x55b544d16c60 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a21ffb640 1 -- 172.21.15.26:0/895871647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9a3405dae0 con 0x55b544d16c60 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4be4d740 10 client.6135 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4be4d740 1 -- 172.21.15.26:0/895871647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55b544a8c1e0 con 0x55b544d16c60 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a21ffb640 1 -- 172.21.15.26:0/895871647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a38026070 con 0x55b544d16c60 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a21ffb640 1 -- 172.21.15.26:0/895871647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7f9a3405dc80 con 0x55b544d16c60 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4be4d740 10 client.6135 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4be4d740 10 client.6135 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4be4d740 20 client.6135 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b544d1aa20 0x55b544d31080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4be4d740 4 client.6135 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4be4d740 1 -- 172.21.15.26:0/895871647 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b544c5c9d0 con 0x55b544d1aa20 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.671+0000 7f9a4259c640 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b544d1aa20 0x55b544d31080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.672+0000 7f9a4259c640 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b544d1aa20 0x55b544d31080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af669a80b4bb2bc0 server_cookie=6139672c7a6502b0 in_seq=0 out_seq=0 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.672+0000 7f9a21ffb640 10 client.6135 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.672+0000 7f9a21ffb640 1 -- 172.21.15.26:0/895871647 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b544c5c9d0 con 0x55b544d1aa20 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.672+0000 7f9a21ffb640 10 client.6135 handle_command_reply: tid=0 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.672+0000 7f9a4be4d740 1 client.6135 shutdown 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.672+0000 7f9a4be4d740 1 -- 172.21.15.26:0/895871647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 msgr2=0x55b544d11070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.672+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d11070 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9a34000fb0 tx=0x7f9a3405c950 comp rx=0 tx=0).stop 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a4be4d740 1 -- 172.21.15.26:0/895871647 shutdown_connections 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b544d1aa20 0x55b544d31080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544d30670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d1a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/895871647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d11070 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a21ffb640 0 client.6135 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a4be4d740 1 -- 172.21.15.26:0/895871647 >> 172.21.15.26:0/895871647 conn(0x55b544cb35e0 msgr2=0x55b544d0ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a4be4d740 1 -- 172.21.15.26:0/895871647 shutdown_connections 2024-04-24T23:23:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a4be4d740 1 -- 172.21.15.26:0/895871647 wait complete. 2024-04-24T23:23:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.673+0000 7f9a4be4d740 20 client.6135 trim_cache size 0 max 16384 2024-04-24T23:23:36.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.690+0000 7f9a4be4d740 1 Processor -- start 2024-04-24T23:23:36.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a4be4d740 1 -- start start 2024-04-24T23:23:36.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d13af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d14030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544d44190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9a442009c0 con 0x55b544d16c60 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441ffbf0 con 0x55b544d17580 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9a44200290 con 0x55b544d28680 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d13af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d13af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35014/0 (socket says 172.21.15.26:35014) 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a42d9d640 1 -- 172.21.15.26:0/2598485520 learned_addr learned my addr 172.21.15.26:0/2598485520 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a4359e640 1 --2- 172.21.15.26:0/2598485520 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544d44190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.691+0000 7f9a4259c640 1 --2- 172.21.15.26:0/2598485520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d14030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a42d9d640 1 -- 172.21.15.26:0/2598485520 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 msgr2=0x55b544d44190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/2598485520 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544d44190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a42d9d640 1 -- 172.21.15.26:0/2598485520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 msgr2=0x55b544d14030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/2598485520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d14030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a42d9d640 1 -- 172.21.15.26:0/2598485520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5448fc4a0 con 0x55b544d16c60 2024-04-24T23:23:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/2598485520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d13af0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9a340012b0 tx=0x7f9a3408a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6dac016cb9d0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a23fff640 1 -- 172.21.15.26:0/2598485520 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34058420 con 0x55b544d16c60 2024-04-24T23:23:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a23fff640 1 -- 172.21.15.26:0/2598485520 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a34083070 con 0x55b544d16c60 2024-04-24T23:23:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a23fff640 1 -- 172.21.15.26:0/2598485520 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a340775a0 con 0x55b544d16c60 2024-04-24T23:23:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2598485520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 msgr2=0x55b544d13af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2598485520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d13af0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9a340012b0 tx=0x7f9a3408a050 comp rx=0 tx=0).stop 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2598485520 shutdown_connections 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2598485520 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544d44190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2598485520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d17580 0x55b544d14030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2598485520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d16c60 0x55b544d13af0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.692+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2598485520 >> 172.21.15.26:0/2598485520 conn(0x55b544cb35e0 msgr2=0x55b544ce2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.693+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2598485520 shutdown_connections 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.693+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2598485520 wait complete. 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.693+0000 7f9a4be4d740 1 -- start start 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.693+0000 7f9a4be4d740 1 Processor -- start 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d16c60 0x55b544d44190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d17580 0x55b544d446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4be4d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544daba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b544ceb000 con 0x55b544d17580 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441fece0 con 0x55b544d16c60 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4be4d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b544d110d0 con 0x55b544d28680 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4be4d740 10 client.0 _ll_register_callbacks cb 0x7f9a340580a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d17580 0x55b544d446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d17580 0x55b544d446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35022/0 (socket says 172.21.15.26:35022) 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4259c640 1 -- 172.21.15.26:0/2629912897 learned_addr learned my addr 172.21.15.26:0/2629912897 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d16c60 0x55b544d44190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4359e640 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544daba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4259c640 1 -- 172.21.15.26:0/2629912897 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 msgr2=0x55b544daba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4259c640 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544daba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4259c640 1 -- 172.21.15.26:0/2629912897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d16c60 msgr2=0x55b544d44190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4259c640 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d16c60 0x55b544d44190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4259c640 1 -- 172.21.15.26:0/2629912897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b544d54fb0 con 0x55b544d17580 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a4259c640 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d17580 0x55b544d446d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9a38023040 tx=0x7f9a38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4062e9f33c0be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.694+0000 7f9a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a21ffb640 1 -- 172.21.15.26:0/2629912897 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3801c040 con 0x55b544d17580 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a21ffb640 1 -- 172.21.15.26:0/2629912897 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f9a3800f040 con 0x55b544d17580 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2629912897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a34083070 con 0x55b544d17580 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a21ffb640 1 -- 172.21.15.26:0/2629912897 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9a3803f050 con 0x55b544d17580 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a4be4d740 10 client.6141 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2629912897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55b544d274c0 con 0x55b544d17580 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a21ffb640 1 -- 172.21.15.26:0/2629912897 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a34083070 con 0x55b544d17580 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a21ffb640 1 -- 172.21.15.26:0/2629912897 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7f9a38042020 con 0x55b544d17580 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a4be4d740 10 client.6141 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a4be4d740 10 client.6141 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-24T23:23:36.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a4be4d740 20 client.6141 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b544da5890 0x55b544dabfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a4be4d740 4 client.6141 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2629912897 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x55b544c56d20 con 0x55b544da5890 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b544da5890 0x55b544dabfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a21ffb640 10 client.6141 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.695+0000 7f9a42d9d640 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b544da5890 0x55b544dabfd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f132f29d4ae48393 server_cookie=759b654703c05c67 in_seq=0 out_seq=0 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a21ffb640 1 -- 172.21.15.26:0/2629912897 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b544c56d20 con 0x55b544da5890 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a21ffb640 10 client.6141 handle_command_reply: tid=0 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 client.6141 shutdown 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2629912897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d17580 msgr2=0x55b544d446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d17580 0x55b544d446d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9a38023040 tx=0x7f9a38004730 comp rx=0 tx=0).stop 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2629912897 shutdown_connections 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x55b544da5890 0x55b544dabfd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b544d28680 0x55b544daba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b544d17580 0x55b544d446d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a21ffb640 0 client.6141 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:36.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 --2- 172.21.15.26:0/2629912897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b544d16c60 0x55b544d44190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2629912897 >> 172.21.15.26:0/2629912897 conn(0x55b544cb35e0 msgr2=0x55b544cea540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2629912897 shutdown_connections 2024-04-24T23:23:36.705 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:23:36.696+0000 7fc1e366d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-04-24T23:23:19.212994+0000 1=0+1) n(v3 rc2024-04-24T23:23:19.443990+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5458=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5458 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55e60391c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.212994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.443990+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.212994+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-04-24T23:23:19.443990+0000 6=0+6)","error_str":""},"return_code":-61} 2024-04-24T23:23:36.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 1 -- 172.21.15.26:0/2629912897 wait complete. 2024-04-24T23:23:36.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.696+0000 7f9a4be4d740 20 client.6141 trim_cache size 0 max 16384 2024-04-24T23:23:36.706 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:36.706 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-04-24T23:23:36.706 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "2f83100e-c05d-4f54-a685-af48cef98182", 2024-04-24T23:23:36.706 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-04-24T23:23:36.706 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 -- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9a3006be30 msgr2=0x7f9a3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9a3006be30 0x7f9a3006e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9a441ea2e0 tx=0x7f9a38008040 comp rx=0 tx=0).stop 2024-04-24T23:23:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 -- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 msgr2=0x7f9a4406f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 0x7f9a4406f000 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400c400 tx=0x7f9a3400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 -- 172.21.15.26:0/1564504610 shutdown_connections 2024-04-24T23:23:36.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9a3006be30 0x7f9a3006e2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9a4415f430 0x7f9a4406d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9a44158a00 0x7f9a44159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 --2- 172.21.15.26:0/1564504610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4406c670 0x7f9a4406f000 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 -- 172.21.15.26:0/1564504610 >> 172.21.15.26:0/1564504610 conn(0x7f9a440716e0 msgr2=0x7f9a4415e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 -- 172.21.15.26:0/1564504610 shutdown_connections 2024-04-24T23:23:36.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.697+0000 7f9a49142640 1 -- 172.21.15.26:0/1564504610 wait complete. 2024-04-24T23:23:36.708 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:23:36.698+0000 7fc1e9e7a640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1) n(v4 rc2024-04-24T23:23:19.443990+0000 5=0+5) (inest sync->mix g=1,2) caps={5458=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55e603a53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.443990+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-04-24T23:23:19.443990+0000 5=0+5)","error_str":""},"return_code":-61} 2024-04-24T23:23:36.709 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:23:36.697+0000 7f1294097640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1) n(v2 rc2024-04-24T23:23:19.443990+0000 4=0+4) (inest lock) (ifile excl) caps={5458=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5458 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x56421e708580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.443990+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-04-24T23:23:19.443990+0000 4=0+4)","error_str":""},"return_code":-61} 2024-04-24T23:23:36.710 INFO:tasks.ceph.mds.b.smithi029.stderr:2024-04-24T23:23:36.698+0000 7fbb6668e640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1) n(v0 rc2024-04-24T23:23:19.214994+0000 2=0+2) (inest lock) (ifile excl) caps={5458=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5458 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55cbdf449080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.214994+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-04-24T23:23:19.214994+0000 2=0+2)","error_str":""},"return_code":-61} 2024-04-24T23:23:36.710 INFO:tasks.ceph.mds.b.smithi029.stderr:2024-04-24T23:23:36.698+0000 7fbb6668e640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5458=pAsLsXsFsx/AsLsXsFsx@2},l=5458 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55cbdf449600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-04-24T23:23:36.710 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:23:36.699+0000 7f129a8a4640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1) n(v1 rc2024-04-24T23:23:19.443990+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={5458=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x56421f65cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.214994+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-04-24T23:23:19.443990+0000 3=0+3)","error_str":""},"return_code":-61} 2024-04-24T23:23:36.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:36.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub abort 2024-04-24T23:23:36.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.812+0000 7f6dda04d640 1 Processor -- start 2024-04-24T23:23:36.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.813+0000 7f6dda04d640 1 -- start start 2024-04-24T23:23:36.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.813+0000 7f6dda04d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6dd4158ae0 0x7f6dd406e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.813+0000 7f6dda04d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd40d0ce0 0x7f6dd415e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.813+0000 7f6dda04d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd406b1c0 con 0x7f6dd415ee90 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.813+0000 7f6dda04d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd406ec90 con 0x7f6dd4158ae0 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.813+0000 7f6dda04d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd406ef70 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6dd4158ae0 0x7f6dd406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6dd4158ae0 0x7f6dd406e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35030/0 (socket says 172.21.15.26:35030) 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd37fe640 1 -- 172.21.15.26:0/2091115117 learned_addr learned my addr 172.21.15.26:0/2091115117 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd3fff640 1 --1- 172.21.15.26:0/2091115117 >> v1:172.21.15.26:6789/0 conn(0x7f6dd415ee90 0x7f6dd4071720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35514/0 (socket says 172.21.15.26:35514) 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/2091115117 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd40d0ce0 0x7f6dd415e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd37fe640 1 -- 172.21.15.26:0/2091115117 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd40d0ce0 msgr2=0x7f6dd415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/2091115117 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd40d0ce0 0x7f6dd415e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd37fe640 1 -- 172.21.15.26:0/2091115117 >> v1:172.21.15.26:6789/0 conn(0x7f6dd415ee90 legacy=0x7f6dd4071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd37fe640 1 -- 172.21.15.26:0/2091115117 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd406f250 con 0x7f6dd4158ae0 2024-04-24T23:23:36.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd27fc640 1 -- 172.21.15.26:0/2091115117 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234747590 0 0) 0x7f6dd406b1c0 con 0x7f6dd415ee90 2024-04-24T23:23:36.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.814+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/2091115117 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd40d0ce0 0x7f6dd415e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/2091115117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4158ae0 0x7f6dd406e620 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599ee14e3539aeb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dd27fc640 1 -- 172.21.15.26:0/2091115117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4037070 con 0x7f6dd4158ae0 2024-04-24T23:23:36.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dd27fc640 1 -- 172.21.15.26:0/2091115117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc4032440 con 0x7f6dd4158ae0 2024-04-24T23:23:36.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dd27fc640 1 -- 172.21.15.26:0/2091115117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4032810 con 0x7f6dd4158ae0 2024-04-24T23:23:36.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dda04d640 1 -- 172.21.15.26:0/2091115117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4158ae0 msgr2=0x7f6dd406e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dda04d640 1 --2- 172.21.15.26:0/2091115117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4158ae0 0x7f6dd406e620 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dda04d640 1 -- 172.21.15.26:0/2091115117 shutdown_connections 2024-04-24T23:23:36.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dda04d640 1 --2- 172.21.15.26:0/2091115117 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd40d0ce0 0x7f6dd415e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dda04d640 1 --2- 172.21.15.26:0/2091115117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd4158ae0 0x7f6dd406e620 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.815+0000 7f6dda04d640 1 -- 172.21.15.26:0/2091115117 >> 172.21.15.26:0/2091115117 conn(0x7f6dd40b2970 msgr2=0x7f6dd40d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.816+0000 7f6dda04d640 1 -- 172.21.15.26:0/2091115117 shutdown_connections 2024-04-24T23:23:36.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.816+0000 7f6dda04d640 1 -- 172.21.15.26:0/2091115117 wait complete. 2024-04-24T23:23:36.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.817+0000 7f6dda04d640 1 Processor -- start 2024-04-24T23:23:36.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.817+0000 7f6dda04d640 1 -- start start 2024-04-24T23:23:36.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dda04d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40d0ce0 0x7f6dd4147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dda04d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dd4158ae0 0x7f6dd4147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dda04d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd415ee90 0x7f6dd4159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dda04d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd40b00e0 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dda04d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd40aff60 con 0x7f6dd4158ae0 2024-04-24T23:23:36.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dda04d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd4071e30 con 0x7f6dd415ee90 2024-04-24T23:23:36.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40d0ce0 0x7f6dd4147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40d0ce0 0x7f6dd4147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35044/0 (socket says 172.21.15.26:35044) 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd37fe640 1 -- 172.21.15.26:0/3040903546 learned_addr learned my addr 172.21.15.26:0/3040903546 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd3fff640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd415ee90 0x7f6dd4159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dd4158ae0 0x7f6dd4147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd37fe640 1 -- 172.21.15.26:0/3040903546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd415ee90 msgr2=0x7f6dd4159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd415ee90 0x7f6dd4159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd37fe640 1 -- 172.21.15.26:0/3040903546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dd4158ae0 msgr2=0x7f6dd4147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dd4158ae0 0x7f6dd4147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd37fe640 1 -- 172.21.15.26:0/3040903546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd4147580 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dd4158ae0 0x7f6dd4147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40d0ce0 0x7f6dd4147040 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dbbfff640 1 -- 172.21.15.26:0/3040903546 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4037040 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dbbfff640 1 -- 172.21.15.26:0/3040903546 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc4004230 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dbbfff640 1 -- 172.21.15.26:0/3040903546 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4003610 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dda04d640 1 -- 172.21.15.26:0/3040903546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd414a900 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.818+0000 7f6dda04d640 1 -- 172.21.15.26:0/3040903546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd40adcb0 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.820+0000 7f6dbbfff640 1 -- 172.21.15.26:0/3040903546 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6dc402ad10 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.821+0000 7f6dbbfff640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6dc006be60 0x7f6dc006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.821+0000 7f6dbbfff640 1 -- 172.21.15.26:0/3040903546 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f6dc40aba60 con 0x7f6dd40d0ce0 2024-04-24T23:23:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.821+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6dc006be60 0x7f6dc006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.821+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6dc006be60 0x7f6dc006e320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40afb00 tx=0x7f6dc8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.822+0000 7f6ddc9fd740 1 Processor -- start 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.822+0000 7f6ddc9fd740 1 -- start start 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.822+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 0x5635ae3d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 0x5635ae402020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402560 0x5635ae3d7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5635ae33b330 con 0x5635ae402560 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5635ae2e3ab0 con 0x5635ae402ca0 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5635ae3372f0 con 0x5635ae41fb80 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402560 0x5635ae3d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402560 0x5635ae3d7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35052/0 (socket says 172.21.15.26:35052) 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd3fff640 1 -- 172.21.15.26:0/4193283405 learned_addr learned my addr 172.21.15.26:0/4193283405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/4193283405 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 0x5635ae3d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/4193283405 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 0x5635ae402020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd3fff640 1 -- 172.21.15.26:0/4193283405 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 msgr2=0x5635ae3d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd3fff640 1 --2- 172.21.15.26:0/4193283405 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 0x5635ae3d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd3fff640 1 -- 172.21.15.26:0/4193283405 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 msgr2=0x5635ae402020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd3fff640 1 --2- 172.21.15.26:0/4193283405 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 0x5635ae402020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd3fff640 1 -- 172.21.15.26:0/4193283405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ae1197d0 con 0x5635ae402560 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6dd3fff640 1 --2- 172.21.15.26:0/4193283405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402560 0x5635ae3d7370 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007450 tx=0x7f6dcc007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8ce191422718ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6db97fa640 1 -- 172.21.15.26:0/4193283405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc019070 con 0x5635ae402560 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6db97fa640 1 -- 172.21.15.26:0/4193283405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc001c50 con 0x5635ae402560 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.823+0000 7f6db97fa640 1 -- 172.21.15.26:0/4193283405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc002020 con 0x5635ae402560 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.824+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/4193283405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402560 msgr2=0x5635ae3d7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.824+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/4193283405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402560 0x5635ae3d7370 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007450 tx=0x7f6dcc007910 comp rx=0 tx=0).stop 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.824+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/4193283405 shutdown_connections 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.824+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/4193283405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402560 0x5635ae3d7370 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.824+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/4193283405 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 0x5635ae402020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.824+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/4193283405 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 0x5635ae3d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.824+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/4193283405 >> 172.21.15.26:0/4193283405 conn(0x5635ae3d4780 msgr2=0x5635ae41d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.824+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/4193283405 shutdown_connections 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.824+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/4193283405 wait complete. 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.825+0000 7f6ddc9fd740 1 -- start start 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.825+0000 7f6ddc9fd740 1 Processor -- start 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae402ca0 0x5635ae3f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae404970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 0x5635ae415810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5635ae3d6930 con 0x5635ae41fb80 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5635ae2f6f40 con 0x5635ae415390 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5635ae3d5d20 con 0x5635ae402ca0 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6ddc9fd740 10 client.0 _ll_register_callbacks cb 0x5635ae04ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:36.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae404970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae404970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35056/0 (socket says 172.21.15.26:35056) 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd2ffd640 1 -- 172.21.15.26:0/526709939 learned_addr learned my addr 172.21.15.26:0/526709939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae402ca0 0x5635ae3f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd3fff640 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 0x5635ae415810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd2ffd640 1 -- 172.21.15.26:0/526709939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae402ca0 msgr2=0x5635ae3f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae402ca0 0x5635ae3f0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd2ffd640 1 -- 172.21.15.26:0/526709939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 msgr2=0x5635ae415810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 0x5635ae415810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd2ffd640 1 -- 172.21.15.26:0/526709939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ae00f0f0 con 0x5635ae41fb80 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae404970 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80126c0 tx=0x7f6dc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3feaeb23b064adc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6db27fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6db27fc640 1 -- 172.21.15.26:0/526709939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8023070 con 0x5635ae41fb80 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6db27fc640 1 -- 172.21.15.26:0/526709939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f6dc801f5f0 con 0x5635ae41fb80 2024-04-24T23:23:36.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.826+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/526709939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dcc001c50 con 0x5635ae41fb80 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6db27fc640 1 -- 172.21.15.26:0/526709939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6dc8019040 con 0x5635ae41fb80 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6ddc9fd740 10 client.6156 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/526709939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5635ae179f60 con 0x5635ae41fb80 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6db27fc640 1 -- 172.21.15.26:0/526709939 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc001c50 con 0x5635ae41fb80 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6db27fc640 1 -- 172.21.15.26:0/526709939 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7f6dc8031080 con 0x5635ae41fb80 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6ddc9fd740 10 client.6156 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6ddc9fd740 10 client.6156 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6ddc9fd740 20 client.6156 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5635ae403660 0x5635ae403a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6ddc9fd740 4 client.6156 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/526709939 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635ae348870 con 0x5635ae403660 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5635ae403660 0x5635ae403a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5635ae403660 0x5635ae403a70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc11f596cde0b4e0 server_cookie=760ab5b79a172ede in_seq=0 out_seq=0 2024-04-24T23:23:36.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.827+0000 7f6db27fc640 10 client.6156 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6db27fc640 1 -- 172.21.15.26:0/526709939 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635ae348870 con 0x5635ae403660 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6db27fc640 10 client.6156 handle_command_reply: tid=0 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 client.6156 shutdown 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/526709939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 msgr2=0x5635ae404970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae404970 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80126c0 tx=0x7f6dc80046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/526709939 shutdown_connections 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5635ae403660 0x5635ae403a70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 0x5635ae415810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae404970 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/526709939 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae402ca0 0x5635ae3f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6db27fc640 0 client.6156 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/526709939 >> 172.21.15.26:0/526709939 conn(0x5635ae402560 msgr2=0x5635ae3d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/526709939 shutdown_connections 2024-04-24T23:23:36.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/526709939 wait complete. 2024-04-24T23:23:36.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.828+0000 7f6ddc9fd740 20 client.6156 trim_cache size 0 max 16384 2024-04-24T23:23:36.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.845+0000 7f6ddc9fd740 1 Processor -- start 2024-04-24T23:23:36.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.846+0000 7f6ddc9fd740 1 -- start start 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.846+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402ca0 0x5635ae3d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.846+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 0x5635ae3d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.846+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 0x5635ae3ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.846+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4200820 con 0x5635ae402ca0 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.846+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41610e0 con 0x5635ae415390 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.846+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd42000f0 con 0x5635ae41fb80 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.846+0000 7f6dd37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402ca0 0x5635ae3d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402ca0 0x5635ae3d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35066/0 (socket says 172.21.15.26:35066) 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd37fe640 1 -- 172.21.15.26:0/1396316758 learned_addr learned my addr 172.21.15.26:0/1396316758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd3fff640 1 --2- 172.21.15.26:0/1396316758 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 0x5635ae3ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/1396316758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 0x5635ae3d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd37fe640 1 -- 172.21.15.26:0/1396316758 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 msgr2=0x5635ae3ef9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/1396316758 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 0x5635ae3ef9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd37fe640 1 -- 172.21.15.26:0/1396316758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 msgr2=0x5635ae3d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/1396316758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 0x5635ae3d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd37fe640 1 -- 172.21.15.26:0/1396316758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635adfe81b0 con 0x5635ae402ca0 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/1396316758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402ca0 0x5635ae3d1fc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4035040 tx=0x7f6dc4048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad660717d52213fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6db8ff9640 1 -- 172.21.15.26:0/1396316758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40a7020 con 0x5635ae402ca0 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6db8ff9640 1 -- 172.21.15.26:0/1396316758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc4082940 con 0x5635ae402ca0 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.847+0000 7f6db8ff9640 1 -- 172.21.15.26:0/1396316758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4077e00 con 0x5635ae402ca0 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/1396316758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402ca0 msgr2=0x5635ae3d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/1396316758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402ca0 0x5635ae3d1fc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4035040 tx=0x7f6dc4048040 comp rx=0 tx=0).stop 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/1396316758 shutdown_connections 2024-04-24T23:23:36.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/1396316758 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae41fb80 0x5635ae3ef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/1396316758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae415390 0x5635ae3d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/1396316758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae402ca0 0x5635ae3d1fc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/1396316758 >> 172.21.15.26:0/1396316758 conn(0x5635ae402560 msgr2=0x5635ae3d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/1396316758 shutdown_connections 2024-04-24T23:23:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/1396316758 wait complete. 2024-04-24T23:23:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 -- start start 2024-04-24T23:23:36.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.848+0000 7f6ddc9fd740 1 Processor -- start 2024-04-24T23:23:36.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 0x5635ae4969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae415390 0x5635ae41c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6ddc9fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae3d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5635ae405ea0 con 0x5635ae41fb80 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd40af1c0 con 0x5635ae402ca0 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6ddc9fd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5635ae41f070 con 0x5635ae415390 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6ddc9fd740 10 client.0 _ll_register_callbacks cb 0x7f6dc80086b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae3d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae3d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35068/0 (socket says 172.21.15.26:35068) 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6dd3fff640 1 -- 172.21.15.26:0/2655234113 learned_addr learned my addr 172.21.15.26:0/2655234113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 0x5635ae4969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.849+0000 7f6dd2ffd640 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae415390 0x5635ae41c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6dd3fff640 1 -- 172.21.15.26:0/2655234113 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae415390 msgr2=0x5635ae41c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6dd3fff640 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae415390 0x5635ae41c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6dd3fff640 1 -- 172.21.15.26:0/2655234113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 msgr2=0x5635ae4969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6dd3fff640 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 0x5635ae4969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6dd3fff640 1 -- 172.21.15.26:0/2655234113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ae3ede70 con 0x5635ae41fb80 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6dd3fff640 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae3d1fc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007420 tx=0x7f6dcc007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d009079ac2cd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6db27fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6db27fc640 1 -- 172.21.15.26:0/2655234113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc026060 con 0x5635ae41fb80 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/2655234113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc4082940 con 0x5635ae41fb80 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6db27fc640 1 -- 172.21.15.26:0/2655234113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f6dcc014480 con 0x5635ae41fb80 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6db27fc640 1 -- 172.21.15.26:0/2655234113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6dcc02c050 con 0x5635ae41fb80 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6ddc9fd740 10 client.6162 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/2655234113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5635ae3ee080 con 0x5635ae41fb80 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6db27fc640 1 -- 172.21.15.26:0/2655234113 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc4082940 con 0x5635ae41fb80 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6db27fc640 1 -- 172.21.15.26:0/2655234113 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7f6dcc02f020 con 0x5635ae41fb80 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6ddc9fd740 10 client.6162 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6ddc9fd740 10 client.6162 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6ddc9fd740 20 client.6162 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5635ae3d2500 0x5635ae42f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6ddc9fd740 4 client.6162 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.850+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/2655234113 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x5635ae342bc0 con 0x5635ae3d2500 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5635ae3d2500 0x5635ae42f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6db27fc640 10 client.6162 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6dd37fe640 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5635ae3d2500 0x5635ae42f490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba63009209ebac94 server_cookie=82546da86086fbca in_seq=0 out_seq=0 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6db27fc640 1 -- 172.21.15.26:0/2655234113 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5635ae342bc0 con 0x5635ae3d2500 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6db27fc640 10 client.6162 handle_command_reply: tid=0 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 client.6162 shutdown 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/2655234113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 msgr2=0x5635ae3d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae3d1fc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007420 tx=0x7f6dcc007ec0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/2655234113 shutdown_connections 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x5635ae3d2500 0x5635ae42f490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635ae41fb80 0x5635ae3d1fc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635ae415390 0x5635ae41c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6db27fc640 0 client.6162 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 --2- 172.21.15.26:0/2655234113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5635ae402ca0 0x5635ae4969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/2655234113 >> 172.21.15.26:0/2655234113 conn(0x5635ae402560 msgr2=0x5635ae3d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/2655234113 shutdown_connections 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 1 -- 172.21.15.26:0/2655234113 wait complete. 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.851+0000 7f6ddc9fd740 20 client.6162 trim_cache size 0 max 16384 2024-04-24T23:23:36.859 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:36.860 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:23:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 -- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6dc006be60 msgr2=0x7f6dc006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6dc006be60 0x7f6dc006e320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40afb00 tx=0x7f6dc8008040 comp rx=0 tx=0).stop 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 -- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40d0ce0 msgr2=0x7f6dd4147040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40d0ce0 0x7f6dd4147040 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc40046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 -- 172.21.15.26:0/3040903546 shutdown_connections 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6dc006be60 0x7f6dc006e320 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6dd415ee90 0x7f6dd4159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dd4158ae0 0x7f6dd4147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 --2- 172.21.15.26:0/3040903546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd40d0ce0 0x7f6dd4147040 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 -- 172.21.15.26:0/3040903546 >> 172.21.15.26:0/3040903546 conn(0x7f6dd40b2970 msgr2=0x7f6dd406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 -- 172.21.15.26:0/3040903546 shutdown_connections 2024-04-24T23:23:36.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.852+0000 7f6dda04d640 1 -- 172.21.15.26:0/3040903546 wait complete. 2024-04-24T23:23:36.864 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:23:36.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-04-24T23:23:36.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.967+0000 7f59b25ed640 1 Processor -- start 2024-04-24T23:23:36.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.967+0000 7f59b25ed640 1 -- start start 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.968+0000 7f59b25ed640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac0b4ce0 0x7f59ac0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.968+0000 7f59b25ed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ac0aeb10 con 0x7f59ac15be40 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.968+0000 7f59b25ed640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ac0aedf0 con 0x7f59ac0d0750 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.968+0000 7f59b25ed640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac09df20 con 0x7f59ac0b4ce0 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.968+0000 7f59abfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f59ac15be40 0x7f59ac0d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35516/0 (socket says 172.21.15.26:35516) 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.968+0000 7f59abfff640 1 -- 172.21.15.26:0/2690405219 learned_addr learned my addr 172.21.15.26:0/2690405219 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.968+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac0b4ce0 0x7f59ac0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.968+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333066566 0 0) 0x7f59ac0aeb10 con 0x7f59ac15be40 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.968+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5994003520 con 0x7f59ac15be40 2024-04-24T23:23:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2014160426 0 0) 0x7f5994003520 con 0x7f59ac15be40 2024-04-24T23:23:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac0b4ce0 msgr2=0x7f59ac0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac0b4ce0 0x7f59ac0acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 >> v1:172.21.15.29:6789/0 conn(0x7f59ac0d0750 legacy=0x7f59ac15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac0af0d0 con 0x7f59ac15be40 2024-04-24T23:23:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f599c001670 con 0x7f59ac15be40 2024-04-24T23:23:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 >> v1:172.21.15.26:6789/0 conn(0x7f59ac15be40 legacy=0x7f59ac0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59940039e0 0x7f5994003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac0b4ce0 0x7f5994004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5994004d40 0x7f59940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f599400b9c0 con 0x7f59940039e0 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f599400bb40 con 0x7f5994004d40 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f599400bd40 con 0x7f59ac0b4ce0 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59abfff640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59940039e0 0x7f5994003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac0b4ce0 0x7f5994004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5994004d40 0x7f59940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59abfff640 1 -- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac0b4ce0 msgr2=0x7f5994004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59abfff640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac0b4ce0 0x7f5994004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59abfff640 1 -- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5994004d40 msgr2=0x7f59940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59abfff640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5994004d40 0x7f59940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59abfff640 1 -- 172.21.15.26:0/2690405219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5994003520 con 0x7f59940039e0 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac0b4ce0 0x7f5994004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.969+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5994004d40 0x7f59940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.970+0000 7f59abfff640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59940039e0 0x7f5994003dc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c240 tx=0x7f5994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e63dedd23eb8e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.970+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c007d10 con 0x7f59940039e0 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.970+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c005ad0 con 0x7f59940039e0 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.970+0000 7f59aaffd640 1 -- 172.21.15.26:0/2690405219 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c007070 con 0x7f59940039e0 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.970+0000 7f59b25ed640 1 -- 172.21.15.26:0/2690405219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59940039e0 msgr2=0x7f5994003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.970+0000 7f59b25ed640 1 --2- 172.21.15.26:0/2690405219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59940039e0 0x7f5994003dc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c240 tx=0x7f5994005830 comp rx=0 tx=0).stop 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.971+0000 7f59b0b8a640 1 -- 172.21.15.26:0/2690405219 reap_dead start 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.971+0000 7f59b25ed640 1 -- 172.21.15.26:0/2690405219 shutdown_connections 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.971+0000 7f59b25ed640 1 -- 172.21.15.26:0/2690405219 >> 172.21.15.26:0/2690405219 conn(0x7f59ac06b540 msgr2=0x7f59ac06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.971+0000 7f59b25ed640 1 -- 172.21.15.26:0/2690405219 shutdown_connections 2024-04-24T23:23:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.971+0000 7f59b25ed640 1 -- 172.21.15.26:0/2690405219 wait complete. 2024-04-24T23:23:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.972+0000 7f59b25ed640 1 Processor -- start 2024-04-24T23:23:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.972+0000 7f59b25ed640 1 -- start start 2024-04-24T23:23:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59b25ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59940039e0 0x7f59ac1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59b25ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59ac1535e0 0x7f59ac1f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59b25ed640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac1f7ae0 0x7f59ac09bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59b25ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac06cd10 con 0x7f59ac1535e0 2024-04-24T23:23:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59b25ed640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac06cb90 con 0x7f59940039e0 2024-04-24T23:23:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59b25ed640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac0acd30 con 0x7f59ac1f7ae0 2024-04-24T23:23:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59ac1535e0 0x7f59ac1f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59ab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59ac1535e0 0x7f59ac1f75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35098/0 (socket says 172.21.15.26:35098) 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59ab7fe640 1 -- 172.21.15.26:0/1630411299 learned_addr learned my addr 172.21.15.26:0/1630411299 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac1f7ae0 0x7f59ac09bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59ab7fe640 1 -- 172.21.15.26:0/1630411299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac1f7ae0 msgr2=0x7f59ac09bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59abfff640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59940039e0 0x7f59ac1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac1f7ae0 0x7f59ac09bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59ab7fe640 1 -- 172.21.15.26:0/1630411299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59940039e0 msgr2=0x7f59ac1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59940039e0 0x7f59ac1530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59ab7fe640 1 -- 172.21.15.26:0/1630411299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac153be0 con 0x7f59ac1535e0 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59ac1535e0 0x7f59ac1f75a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e880 tx=0x7f59a000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59a8ff9640 1 -- 172.21.15.26:0/1630411299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0001d30 con 0x7f59ac1535e0 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59a8ff9640 1 -- 172.21.15.26:0/1630411299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a0018070 con 0x7f59ac1535e0 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.973+0000 7f59b25ed640 1 -- 172.21.15.26:0/1630411299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac153ec0 con 0x7f59ac1535e0 2024-04-24T23:23:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.974+0000 7f59a8ff9640 1 -- 172.21.15.26:0/1630411299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a000c6c0 con 0x7f59ac1535e0 2024-04-24T23:23:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.974+0000 7f59b25ed640 1 -- 172.21.15.26:0/1630411299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac154480 con 0x7f59ac1535e0 2024-04-24T23:23:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.975+0000 7f59a8ff9640 1 -- 172.21.15.26:0/1630411299 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59a000c860 con 0x7f59ac1535e0 2024-04-24T23:23:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.975+0000 7f59a8ff9640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f599806bfd0 0x7f599806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.976+0000 7f59a8ff9640 1 -- 172.21.15.26:0/1630411299 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f59a008ac90 con 0x7f59ac1535e0 2024-04-24T23:23:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.976+0000 7f59abfff640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f599806bfd0 0x7f599806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.976+0000 7f59abfff640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f599806bfd0 0x7f599806e490 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f599c032000 tx=0x7f599c007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.977+0000 7f59b53cf740 1 Processor -- start 2024-04-24T23:23:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.977+0000 7f59b53cf740 1 -- start start 2024-04-24T23:23:36.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.977+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8386d60 0x558db8387140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.977+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db83a4600 0x558db83a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.977+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db835ca40 0x558db83a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.977+0000 7f59b53cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558db82c06b0 con 0x558db835ca40 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.977+0000 7f59b53cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558db8268880 con 0x558db8386d60 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.977+0000 7f59b53cf740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x558db82bc670 con 0x558db83a4600 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.977+0000 7f59b0b8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db835ca40 0x558db83a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59b0b8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db835ca40 0x558db83a2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35114/0 (socket says 172.21.15.26:35114) 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59b0b8a640 1 -- 172.21.15.26:0/1603251449 learned_addr learned my addr 172.21.15.26:0/1603251449 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db83a4600 0x558db83a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59abfff640 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8386d60 0x558db8387140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59b0b8a640 1 -- 172.21.15.26:0/1603251449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db83a4600 msgr2=0x558db83a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db83a4600 0x558db83a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59b0b8a640 1 -- 172.21.15.26:0/1603251449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8386d60 msgr2=0x558db8387140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8386d60 0x558db8387140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59b0b8a640 1 -- 172.21.15.26:0/1603251449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558db809e7d0 con 0x558db835ca40 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59abfff640 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8386d60 0x558db8387140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db835ca40 0x558db83a2cc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0d0c00 tx=0x7f59a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2930cf07e211350 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f5991ffb640 1 -- 172.21.15.26:0/1603251449 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a401b020 con 0x558db835ca40 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f5991ffb640 1 -- 172.21.15.26:0/1603251449 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a4017660 con 0x558db835ca40 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.978+0000 7f5991ffb640 1 -- 172.21.15.26:0/1603251449 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4017a10 con 0x558db835ca40 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 -- 172.21.15.26:0/1603251449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db835ca40 msgr2=0x558db83a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db835ca40 0x558db83a2cc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0d0c00 tx=0x7f59a4009750 comp rx=0 tx=0).stop 2024-04-24T23:23:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 -- 172.21.15.26:0/1603251449 shutdown_connections 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db835ca40 0x558db83a2cc0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db83a4600 0x558db83a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1603251449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8386d60 0x558db8387140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 -- 172.21.15.26:0/1603251449 >> 172.21.15.26:0/1603251449 conn(0x558db8359600 msgr2=0x558db83a1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 -- 172.21.15.26:0/1603251449 shutdown_connections 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 -- 172.21.15.26:0/1603251449 wait complete. 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.979+0000 7f59b53cf740 1 -- start start 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59b53cf740 1 Processor -- start 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8386d60 0x558db8377560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db83a4600 0x558db83a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8375a90 0x558db83a3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59b53cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558db835d7e0 con 0x558db8386d60 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59b53cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558db827bf50 con 0x558db83a4600 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59b53cf740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x558db8357530 con 0x558db8375a90 2024-04-24T23:23:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59b53cf740 10 client.0 _ll_register_callbacks cb 0x558db82b7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8386d60 0x558db8377560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8386d60 0x558db8377560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35120/0 (socket says 172.21.15.26:35120) 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.980+0000 7f59abfff640 1 -- 172.21.15.26:0/2464596011 learned_addr learned my addr 172.21.15.26:0/2464596011 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8375a90 0x558db83a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db83a4600 0x558db83a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59abfff640 1 -- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8375a90 msgr2=0x558db83a3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59abfff640 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8375a90 0x558db83a3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59abfff640 1 -- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db83a4600 msgr2=0x558db83a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59abfff640 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db83a4600 0x558db83a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59abfff640 1 -- 172.21.15.26:0/2464596011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558db7fd43f0 con 0x558db8386d60 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8375a90 0x558db83a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59abfff640 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8386d60 0x558db8377560 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f599c02ad60 tx=0x7f599c02af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712856c65a126532 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f598affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f598affd640 1 -- 172.21.15.26:0/2464596011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c050060 con 0x558db8386d60 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f598affd640 1 -- 172.21.15.26:0/2464596011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f599c004c70 con 0x558db8386d60 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59b53cf740 1 -- 172.21.15.26:0/2464596011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a4017660 con 0x558db8386d60 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f598affd640 1 -- 172.21.15.26:0/2464596011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f599c037070 con 0x558db8386d60 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59b53cf740 10 client.6174 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f59b53cf740 1 -- 172.21.15.26:0/2464596011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x558db7f940f0 con 0x558db8386d60 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.981+0000 7f598affd640 1 -- 172.21.15.26:0/2464596011 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a4017660 con 0x558db8386d60 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f598affd640 1 -- 172.21.15.26:0/2464596011 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7f599c059080 con 0x558db8386d60 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f59b53cf740 10 client.6174 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f59b53cf740 10 client.6174 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f59b53cf740 20 client.6174 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f59b53cf740 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x558db838a210 0x558db838a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f59b53cf740 4 client.6174 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f59b53cf740 1 -- 172.21.15.26:0/2464596011 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558db82cdbf0 con 0x558db838a210 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x558db838a210 0x558db838a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x558db838a210 0x558db838a610 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ab53150509e8af2 server_cookie=ed367d76f3f7eedb in_seq=0 out_seq=0 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.982+0000 7f598affd640 10 client.6174 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f598affd640 1 -- 172.21.15.26:0/2464596011 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558db82cdbf0 con 0x558db838a210 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f598affd640 10 client.6174 handle_command_reply: tid=0 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 client.6174 shutdown 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 -- 172.21.15.26:0/2464596011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8386d60 msgr2=0x558db8377560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8386d60 0x558db8377560 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f599c02ad60 tx=0x7f599c02af90 comp rx=0 tx=0).stop 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 -- 172.21.15.26:0/2464596011 shutdown_connections 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x558db838a210 0x558db838a610 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8375a90 0x558db83a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db83a4600 0x558db83a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f598affd640 0 client.6174 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 --2- 172.21.15.26:0/2464596011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8386d60 0x558db8377560 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 -- 172.21.15.26:0/2464596011 >> 172.21.15.26:0/2464596011 conn(0x558db835ca40 msgr2=0x558db83a50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 -- 172.21.15.26:0/2464596011 shutdown_connections 2024-04-24T23:23:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 1 -- 172.21.15.26:0/2464596011 wait complete. 2024-04-24T23:23:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:36.983+0000 7f59b53cf740 20 client.6174 trim_cache size 0 max 16384 2024-04-24T23:23:37.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.001+0000 7f59b53cf740 1 Processor -- start 2024-04-24T23:23:37.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.001+0000 7f59b53cf740 1 -- start start 2024-04-24T23:23:37.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8375a90 0x558db838ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8386d60 0x558db8385f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db83a4600 0x558db835d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59b53cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac200d60 con 0x558db8375a90 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59b53cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac1fff30 con 0x558db83a4600 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59b53cf740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac200630 con 0x558db8386d60 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8375a90 0x558db838ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8375a90 0x558db838ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35128/0 (socket says 172.21.15.26:35128) 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59abfff640 1 -- 172.21.15.26:0/3089405901 learned_addr learned my addr 172.21.15.26:0/3089405901 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59abfff640 1 -- 172.21.15.26:0/3089405901 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8386d60 msgr2=0x558db8385f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59abfff640 1 --2- 172.21.15.26:0/3089405901 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8386d60 0x558db8385f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59abfff640 1 -- 172.21.15.26:0/3089405901 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db83a4600 msgr2=0x558db835d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59abfff640 1 --2- 172.21.15.26:0/3089405901 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db83a4600 0x558db835d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59abfff640 1 -- 172.21.15.26:0/3089405901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558db8358320 con 0x558db8375a90 2024-04-24T23:23:37.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59abfff640 1 --2- 172.21.15.26:0/3089405901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8375a90 0x558db838ac70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f599c03e620 tx=0x7f599c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee7d9509339a5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:37.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59917fa640 1 -- 172.21.15.26:0/3089405901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c050020 con 0x558db8375a90 2024-04-24T23:23:37.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59917fa640 1 -- 172.21.15.26:0/3089405901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c038040 con 0x558db8375a90 2024-04-24T23:23:37.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.002+0000 7f59917fa640 1 -- 172.21.15.26:0/3089405901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c03e880 con 0x558db8375a90 2024-04-24T23:23:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.003+0000 7f59b53cf740 1 -- 172.21.15.26:0/3089405901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8375a90 msgr2=0x558db838ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.003+0000 7f59b53cf740 1 --2- 172.21.15.26:0/3089405901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8375a90 0x558db838ac70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f599c03e620 tx=0x7f599c03c000 comp rx=0 tx=0).stop 2024-04-24T23:23:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.003+0000 7f59b53cf740 1 -- 172.21.15.26:0/3089405901 shutdown_connections 2024-04-24T23:23:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.003+0000 7f59b53cf740 1 --2- 172.21.15.26:0/3089405901 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db83a4600 0x558db835d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.003+0000 7f59b53cf740 1 --2- 172.21.15.26:0/3089405901 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8386d60 0x558db8385f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.003+0000 7f59b53cf740 1 --2- 172.21.15.26:0/3089405901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db8375a90 0x558db838ac70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.003+0000 7f59b53cf740 1 -- 172.21.15.26:0/3089405901 >> 172.21.15.26:0/3089405901 conn(0x558db835ca40 msgr2=0x558db8356bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.003+0000 7f59b53cf740 1 -- 172.21.15.26:0/3089405901 shutdown_connections 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.003+0000 7f59b53cf740 1 -- 172.21.15.26:0/3089405901 wait complete. 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.004+0000 7f59b53cf740 1 -- start start 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.004+0000 7f59b53cf740 1 Processor -- start 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8375a90 0x558db841b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8386d60 0x558db835a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b53cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db83a4600 0x558db83b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b53cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558db82f4fc0 con 0x558db83a4600 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b53cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac0726b0 con 0x558db8375a90 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b53cf740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x558db83c5020 con 0x558db8386d60 2024-04-24T23:23:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b53cf740 10 client.0 _ll_register_callbacks cb 0x7f59940036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b0b8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db83a4600 0x558db83b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b0b8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db83a4600 0x558db83b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35130/0 (socket says 172.21.15.26:35130) 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b0b8a640 1 -- 172.21.15.26:0/1894643429 learned_addr learned my addr 172.21.15.26:0/1894643429 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59ab7fe640 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8386d60 0x558db835a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b0b8a640 1 -- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8386d60 msgr2=0x558db835a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8386d60 0x558db835a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b0b8a640 1 -- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8375a90 msgr2=0x558db841b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8375a90 0x558db841b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b0b8a640 1 -- 172.21.15.26:0/1894643429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558db835d0d0 con 0x558db83a4600 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b0b8a640 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db83a4600 0x558db83b1390 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f59a4019000 tx=0x7f59a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61da460afb2a392 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f598affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f598affd640 1 -- 172.21.15.26:0/1894643429 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a401b040 con 0x558db83a4600 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f598affd640 1 -- 172.21.15.26:0/1894643429 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f59a4017ba0 con 0x558db83a4600 2024-04-24T23:23:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b53cf740 1 -- 172.21.15.26:0/1894643429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f599c038040 con 0x558db83a4600 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f598affd640 1 -- 172.21.15.26:0/1894643429 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f59a4033050 con 0x558db83a4600 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.005+0000 7f59b53cf740 10 client.6180 fetch_fsmap learned FSMap version 39 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f59b53cf740 1 -- 172.21.15.26:0/1894643429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x558db841be90 con 0x558db83a4600 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f598affd640 1 -- 172.21.15.26:0/1894643429 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c038040 con 0x558db83a4600 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f598affd640 1 -- 172.21.15.26:0/1894643429 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 3546+0+0 (secure 0 0 0) 0x7f59a402b450 con 0x558db83a4600 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f59b53cf740 10 client.6180 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f59b53cf740 10 client.6180 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f59b53cf740 20 client.6180 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x558db83784f0 0x558db83b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f59b53cf740 4 client.6180 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f59b53cf740 1 -- 172.21.15.26:0/1894643429 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558db82c7f40 con 0x558db83784f0 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f59abfff640 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x558db83784f0 0x558db83b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f598affd640 10 client.6180 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.006+0000 7f59abfff640 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x558db83784f0 0x558db83b18d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b926b441d370ac30 server_cookie=7e954e517e3e2225 in_seq=0 out_seq=0 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f598affd640 1 -- 172.21.15.26:0/1894643429 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x558db82c7f40 con 0x558db83784f0 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f598affd640 10 client.6180 handle_command_reply: tid=0 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 client.6180 shutdown 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 -- 172.21.15.26:0/1894643429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db83a4600 msgr2=0x558db83b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db83a4600 0x558db83b1390 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f59a4019000 tx=0x7f59a4004ab0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 -- 172.21.15.26:0/1894643429 shutdown_connections 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x558db83784f0 0x558db83b18d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db83a4600 0x558db83b1390 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558db8386d60 0x558db835a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 --2- 172.21.15.26:0/1894643429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558db8375a90 0x558db841b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f598affd640 0 client.6180 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 -- 172.21.15.26:0/1894643429 >> 172.21.15.26:0/1894643429 conn(0x558db835ca40 msgr2=0x558db8386130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 -- 172.21.15.26:0/1894643429 shutdown_connections 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 1 -- 172.21.15.26:0/1894643429 wait complete. 2024-04-24T23:23:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b53cf740 20 client.6180 trim_cache size 0 max 16384 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout: "2f83100e-c05d-4f54-a685-af48cef98182": { 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "2f83100e-c05d-4f54-a685-af48cef98182", 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:37.014 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:37.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b25ed640 1 -- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f599806bfd0 msgr2=0x7f599806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.007+0000 7f59b25ed640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f599806bfd0 0x7f599806e490 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f599c032000 tx=0x7f599c007000 comp rx=0 tx=0).stop 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 -- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59ac1535e0 msgr2=0x7f59ac1f75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59ac1535e0 0x7f59ac1f75a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e880 tx=0x7f59a000ed40 comp rx=0 tx=0).stop 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 -- 172.21.15.26:0/1630411299 shutdown_connections 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f599806bfd0 0x7f599806e490 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59ac1f7ae0 0x7f59ac09bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59ac1535e0 0x7f59ac1f75a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 --2- 172.21.15.26:0/1630411299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59940039e0 0x7f59ac1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 -- 172.21.15.26:0/1630411299 >> 172.21.15.26:0/1630411299 conn(0x7f59ac06b540 msgr2=0x7f59ac0b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 -- 172.21.15.26:0/1630411299 shutdown_connections 2024-04-24T23:23:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:37.008+0000 7f59b25ed640 1 -- 172.21.15.26:0/1630411299 wait complete. 2024-04-24T23:23:39.907 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:23:39.906+0000 7f26a3749640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T23:23:42.020 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:23:42.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-04-24T23:23:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.123+0000 7fc111f42640 1 Processor -- start 2024-04-24T23:23:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.123+0000 7fc111f42640 1 -- start start 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc111f42640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc10c15e370 0x7fc10c0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc111f42640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c15e750 0x7fc10c0d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc111f42640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc10c0d4c10 0x7fc10c0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc111f42640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc10c0d1510 con 0x7fc10c15e370 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc111f42640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc10c0d1ed0 con 0x7fc10c0d4c10 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc111f42640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c0d2120 con 0x7fc10c15e750 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc10c15e370 0x7fc10c0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc10c15e370 0x7fc10c0d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35138/0 (socket says 172.21.15.26:35138) 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10b7fe640 1 -- 172.21.15.26:0/964449849 learned_addr learned my addr 172.21.15.26:0/964449849 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10affd640 1 --2- 172.21.15.26:0/964449849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c15e750 0x7fc10c0d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10bfff640 1 --2- 172.21.15.26:0/964449849 >> v2:172.21.15.29:3300/0 conn(0x7fc10c0d4c10 0x7fc10c0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10b7fe640 1 -- 172.21.15.26:0/964449849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c15e750 msgr2=0x7fc10c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/964449849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c15e750 0x7fc10c0d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10b7fe640 1 -- 172.21.15.26:0/964449849 >> v2:172.21.15.29:3300/0 conn(0x7fc10c0d4c10 msgr2=0x7fc10c0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/964449849 >> v2:172.21.15.29:3300/0 conn(0x7fc10c0d4c10 0x7fc10c0b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.124+0000 7fc10b7fe640 1 -- 172.21.15.26:0/964449849 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c0b3740 con 0x7fc10c15e370 2024-04-24T23:23:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.125+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/964449849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15e370 0x7fc10c0d2520 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc10c161f80 tx=0x7fc0fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41184bb8bfc69633 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.125+0000 7fc10a7fc640 1 -- 172.21.15.26:0/964449849 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc037070 con 0x7fc10c15e370 2024-04-24T23:23:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.125+0000 7fc10a7fc640 1 -- 172.21.15.26:0/964449849 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0fc032440 con 0x7fc10c15e370 2024-04-24T23:23:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.125+0000 7fc10a7fc640 1 -- 172.21.15.26:0/964449849 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc032810 con 0x7fc10c15e370 2024-04-24T23:23:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.126+0000 7fc111f42640 1 -- 172.21.15.26:0/964449849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15e370 msgr2=0x7fc10c0d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.126+0000 7fc111f42640 1 --2- 172.21.15.26:0/964449849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15e370 0x7fc10c0d2520 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc10c161f80 tx=0x7fc0fc02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.126+0000 7fc111f42640 1 -- 172.21.15.26:0/964449849 shutdown_connections 2024-04-24T23:23:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.126+0000 7fc111f42640 1 --2- 172.21.15.26:0/964449849 >> v2:172.21.15.29:3300/0 conn(0x7fc10c0d4c10 0x7fc10c0b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.126+0000 7fc111f42640 1 --2- 172.21.15.26:0/964449849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c15e750 0x7fc10c0d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.126+0000 7fc111f42640 1 --2- 172.21.15.26:0/964449849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15e370 0x7fc10c0d2520 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.126+0000 7fc111f42640 1 -- 172.21.15.26:0/964449849 >> 172.21.15.26:0/964449849 conn(0x7fc10c0d40f0 msgr2=0x7fc10c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.126+0000 7fc111f42640 1 -- 172.21.15.26:0/964449849 shutdown_connections 2024-04-24T23:23:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.126+0000 7fc111f42640 1 -- 172.21.15.26:0/964449849 wait complete. 2024-04-24T23:23:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.127+0000 7fc111f42640 1 Processor -- start 2024-04-24T23:23:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.127+0000 7fc111f42640 1 -- start start 2024-04-24T23:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc111f42640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c0d4c10 0x7fc10c0b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc111f42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c15e370 0x7fc10c15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc111f42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15f080 0x7fc10c1f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc111f42640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0b4760 con 0x7fc10c15f080 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc111f42640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0b3d00 con 0x7fc10c15e370 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc111f42640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c0b48e0 con 0x7fc10c0d4c10 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15f080 0x7fc10c1f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15f080 0x7fc10c1f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35154/0 (socket says 172.21.15.26:35154) 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10bfff640 1 -- 172.21.15.26:0/3399332615 learned_addr learned my addr 172.21.15.26:0/3399332615 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10affd640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c15e370 0x7fc10c15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c0d4c10 0x7fc10c0b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10bfff640 1 -- 172.21.15.26:0/3399332615 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c0d4c10 msgr2=0x7fc10c0b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10bfff640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c0d4c10 0x7fc10c0b2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10bfff640 1 -- 172.21.15.26:0/3399332615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c15e370 msgr2=0x7fc10c15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10bfff640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c15e370 0x7fc10c15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10bfff640 1 -- 172.21.15.26:0/3399332615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c160820 con 0x7fc10c15f080 2024-04-24T23:23:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.128+0000 7fc10affd640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c15e370 0x7fc10c15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.129+0000 7fc10bfff640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15f080 0x7fc10c1f2da0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc104007e90 tx=0x7fc10400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.129+0000 7fc0f3fff640 1 -- 172.21.15.26:0/3399332615 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104014420 con 0x7fc10c15f080 2024-04-24T23:23:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.129+0000 7fc0f3fff640 1 -- 172.21.15.26:0/3399332615 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc104019070 con 0x7fc10c15f080 2024-04-24T23:23:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.129+0000 7fc111f42640 1 -- 172.21.15.26:0/3399332615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c160ac0 con 0x7fc10c15f080 2024-04-24T23:23:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.129+0000 7fc0f3fff640 1 -- 172.21.15.26:0/3399332615 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104014820 con 0x7fc10c15f080 2024-04-24T23:23:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.129+0000 7fc111f42640 1 -- 172.21.15.26:0/3399332615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c161080 con 0x7fc10c15f080 2024-04-24T23:23:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.130+0000 7fc0f3fff640 1 -- 172.21.15.26:0/3399332615 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1040149c0 con 0x7fc10c15f080 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.131+0000 7fc0f3fff640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc0f806be60 0x7fc0f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.131+0000 7fc0f3fff640 1 -- 172.21.15.26:0/3399332615 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fc10408b700 con 0x7fc10c15f080 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.131+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc0f806be60 0x7fc0f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.131+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc0f806be60 0x7fc0f806e320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc034000 tx=0x7fc0fc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.131+0000 7fc114c4e740 1 Processor -- start 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.132+0000 7fc114c4e740 1 -- start start 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.132+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 0x55b3c495d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.132+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c497ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.132+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4932a40 0x55b3c4978cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.132+0000 7fc114c4e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b3c48966b0 con 0x55b3c497a600 2024-04-24T23:23:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.132+0000 7fc114c4e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b3c483e880 con 0x55b3c4932a40 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.132+0000 7fc114c4e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b3c4892670 con 0x55b3c495cd60 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c497ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c497ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35164/0 (socket says 172.21.15.26:35164) 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10affd640 1 -- 172.21.15.26:0/3018402781 learned_addr learned my addr 172.21.15.26:0/3018402781 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/3018402781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 0x55b3c495d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10bfff640 1 --2- 172.21.15.26:0/3018402781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4932a40 0x55b3c4978cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10affd640 1 -- 172.21.15.26:0/3018402781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 msgr2=0x55b3c495d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10affd640 1 --2- 172.21.15.26:0/3018402781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 0x55b3c495d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10affd640 1 -- 172.21.15.26:0/3018402781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4932a40 msgr2=0x55b3c4978cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10affd640 1 --2- 172.21.15.26:0/3018402781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4932a40 0x55b3c4978cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10affd640 1 -- 172.21.15.26:0/3018402781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3c46747d0 con 0x55b3c497a600 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc10affd640 1 --2- 172.21.15.26:0/3018402781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c497ab50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc10c15b9b0 tx=0x7fc100009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f681a28597bb1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc0f17fa640 1 -- 172.21.15.26:0/3018402781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100017020 con 0x55b3c497a600 2024-04-24T23:23:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc0f17fa640 1 -- 172.21.15.26:0/3018402781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10000e950 con 0x55b3c497a600 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.133+0000 7fc0f17fa640 1 -- 172.21.15.26:0/3018402781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10000f040 con 0x55b3c497a600 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.134+0000 7fc114c4e740 1 -- 172.21.15.26:0/3018402781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 msgr2=0x55b3c497ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.134+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3018402781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c497ab50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc10c15b9b0 tx=0x7fc100009d70 comp rx=0 tx=0).stop 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.134+0000 7fc114c4e740 1 -- 172.21.15.26:0/3018402781 shutdown_connections 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.134+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3018402781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4932a40 0x55b3c4978cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.134+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3018402781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c497ab50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.134+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3018402781 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 0x55b3c495d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.134+0000 7fc114c4e740 1 -- 172.21.15.26:0/3018402781 >> 172.21.15.26:0/3018402781 conn(0x55b3c492f600 msgr2=0x55b3c4977b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.134+0000 7fc114c4e740 1 -- 172.21.15.26:0/3018402781 shutdown_connections 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.134+0000 7fc114c4e740 1 -- 172.21.15.26:0/3018402781 wait complete. 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc114c4e740 1 -- start start 2024-04-24T23:23:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc114c4e740 1 Processor -- start 2024-04-24T23:23:42.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 0x55b3c4959700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c49706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 0x55b3c4960240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc114c4e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b3c49337e0 con 0x55b3c497a600 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc114c4e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b3c4851f50 con 0x55b3c4970c20 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc114c4e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b3c492d530 con 0x55b3c495cd60 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc114c4e740 10 client.0 _ll_register_callbacks cb 0x55b3c488d000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.135+0000 7fc10affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c49706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c49706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35168/0 (socket says 172.21.15.26:35168) 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10affd640 1 -- 172.21.15.26:0/1111987410 learned_addr learned my addr 172.21.15.26:0/1111987410 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10bfff640 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 0x55b3c4960240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 0x55b3c4959700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10affd640 1 -- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 msgr2=0x55b3c4959700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10affd640 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 0x55b3c4959700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10affd640 1 -- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 msgr2=0x55b3c4960240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10affd640 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 0x55b3c4960240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10affd640 1 -- 172.21.15.26:0/1111987410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3c45aa3f0 con 0x55b3c497a600 2024-04-24T23:23:42.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc10affd640 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c49706e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc100013640 tx=0x7fc100004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe9df15e6c3e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc0ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/1111987410 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100017070 con 0x55b3c497a600 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/1111987410 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fc100026b20 con 0x55b3c497a600 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc114c4e740 1 -- 172.21.15.26:0/1111987410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10000e950 con 0x55b3c497a600 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/1111987410 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc100009120 con 0x55b3c497a600 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc114c4e740 10 client.6192 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc114c4e740 1 -- 172.21.15.26:0/1111987410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x55b3c456a0f0 con 0x55b3c497a600 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.136+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/1111987410 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10000e950 con 0x55b3c497a600 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/1111987410 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7fc10002d050 con 0x55b3c497a600 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc114c4e740 10 client.6192 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc114c4e740 10 client.6192 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc114c4e740 20 client.6192 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc114c4e740 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b3c495ee60 0x55b3c495f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc114c4e740 4 client.6192 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:42.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc114c4e740 1 -- 172.21.15.26:0/1111987410 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3c48a3bf0 con 0x55b3c495ee60 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b3c495ee60 0x55b3c495f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b3c495ee60 0x55b3c495f270 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3faa085e696fcd2 server_cookie=3ae1745763b5277d in_seq=0 out_seq=0 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.137+0000 7fc0ea7fc640 10 client.6192 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/1111987410 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3c48a3bf0 con 0x55b3c495ee60 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc0ea7fc640 10 client.6192 handle_command_reply: tid=0 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 client.6192 shutdown 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 -- 172.21.15.26:0/1111987410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 msgr2=0x55b3c49706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c49706e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc100013640 tx=0x7fc100004730 comp rx=0 tx=0).stop 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 -- 172.21.15.26:0/1111987410 shutdown_connections 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b3c495ee60 0x55b3c495f270 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 0x55b3c4960240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c49706e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc0ea7fc640 0 client.6192 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 --2- 172.21.15.26:0/1111987410 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c495cd60 0x55b3c4959700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 -- 172.21.15.26:0/1111987410 >> 172.21.15.26:0/1111987410 conn(0x55b3c4932a40 msgr2=0x55b3c497b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:42.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 -- 172.21.15.26:0/1111987410 shutdown_connections 2024-04-24T23:23:42.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.138+0000 7fc114c4e740 1 -- 172.21.15.26:0/1111987410 wait complete. 2024-04-24T23:23:42.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.139+0000 7fc114c4e740 20 client.6192 trim_cache size 0 max 16384 2024-04-24T23:23:42.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.156+0000 7fc114c4e740 1 Processor -- start 2024-04-24T23:23:42.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.156+0000 7fc114c4e740 1 -- start start 2024-04-24T23:23:42.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c495cd60 0x55b3c492ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c4970c20 0x55b3c495c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c4960b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc114c4e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c2009d0 con 0x55b3c497a600 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc114c4e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1ffb70 con 0x55b3c495cd60 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc114c4e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c2002a0 con 0x55b3c4970c20 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c4960b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c4960b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35180/0 (socket says 172.21.15.26:35180) 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10bfff640 1 -- 172.21.15.26:0/290212628 learned_addr learned my addr 172.21.15.26:0/290212628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c495cd60 0x55b3c492ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10affd640 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c4970c20 0x55b3c495c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10bfff640 1 -- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c4970c20 msgr2=0x55b3c495c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10bfff640 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c4970c20 0x55b3c495c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10bfff640 1 -- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c495cd60 msgr2=0x55b3c492ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10bfff640 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c495cd60 0x55b3c492ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10bfff640 1 -- 172.21.15.26:0/290212628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3c498b2b0 con 0x55b3c497a600 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c495cd60 0x55b3c492ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10affd640 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c4970c20 0x55b3c495c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.157+0000 7fc10bfff640 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c4960b80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc104055dd0 tx=0x7fc104089ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ca6730dd50944e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc0f0ff9640 1 -- 172.21.15.26:0/290212628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104062550 con 0x55b3c497a600 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc0f0ff9640 1 -- 172.21.15.26:0/290212628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc104057a10 con 0x55b3c497a600 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc0f0ff9640 1 -- 172.21.15.26:0/290212628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104012d90 con 0x55b3c497a600 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc114c4e740 1 -- 172.21.15.26:0/290212628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 msgr2=0x55b3c4960b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc114c4e740 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c4960b80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc104055dd0 tx=0x7fc104089ac0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc114c4e740 1 -- 172.21.15.26:0/290212628 shutdown_connections 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc114c4e740 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c497a600 0x55b3c4960b80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc114c4e740 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c4970c20 0x55b3c495c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc114c4e740 1 --2- 172.21.15.26:0/290212628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c495cd60 0x55b3c492ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc114c4e740 1 -- 172.21.15.26:0/290212628 >> 172.21.15.26:0/290212628 conn(0x55b3c4932a40 msgr2=0x55b3c492e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc114c4e740 1 -- 172.21.15.26:0/290212628 shutdown_connections 2024-04-24T23:23:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.158+0000 7fc114c4e740 1 -- 172.21.15.26:0/290212628 wait complete. 2024-04-24T23:23:42.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.159+0000 7fc114c4e740 1 -- start start 2024-04-24T23:23:42.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.159+0000 7fc114c4e740 1 Processor -- start 2024-04-24T23:23:42.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c495cd60 0x55b3c49f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 0x55b3c4984280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc114c4e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c497a600 0x55b3c49847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc114c4e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b3c499c030 con 0x55b3c495cd60 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc114c4e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c14e260 con 0x55b3c4970c20 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc114c4e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b3c4988960 con 0x55b3c497a600 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc114c4e740 10 client.0 _ll_register_callbacks cb 0x7fc100009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c495cd60 0x55b3c49f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c495cd60 0x55b3c49f1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35194/0 (socket says 172.21.15.26:35194) 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10b7fe640 1 -- 172.21.15.26:0/3139917745 learned_addr learned my addr 172.21.15.26:0/3139917745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10bfff640 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c497a600 0x55b3c49847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10affd640 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 0x55b3c4984280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10b7fe640 1 -- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c497a600 msgr2=0x55b3c49847c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c497a600 0x55b3c49847c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10b7fe640 1 -- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 msgr2=0x55b3c4984280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 0x55b3c4984280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10b7fe640 1 -- 172.21.15.26:0/3139917745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3c4958eb0 con 0x55b3c495cd60 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc10b7fe640 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c495cd60 0x55b3c49f1760 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc035040 tx=0x7fc0fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34828e587c95283e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc0ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/3139917745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc037040 con 0x55b3c495cd60 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/3139917745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fc0fc04e3d0 con 0x55b3c495cd60 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.160+0000 7fc114c4e740 1 -- 172.21.15.26:0/3139917745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc104057a10 con 0x55b3c495cd60 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/3139917745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc0fc05b050 con 0x55b3c495cd60 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc114c4e740 10 client.6198 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc114c4e740 1 -- 172.21.15.26:0/3139917745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x55b3c497bab0 con 0x55b3c495cd60 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/3139917745 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc104057a10 con 0x55b3c495cd60 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/3139917745 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7fc0fc05e020 con 0x55b3c495cd60 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc114c4e740 10 client.6198 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc114c4e740 10 client.6198 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc114c4e740 20 client.6198 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b3c4978d60 0x55b3c4930780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc114c4e740 4 client.6198 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc114c4e740 1 -- 172.21.15.26:0/3139917745 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3c489df40 con 0x55b3c4978d60 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc10affd640 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b3c4978d60 0x55b3c4930780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.161+0000 7fc10affd640 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b3c4978d60 0x55b3c4930780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc1676da5d53ff60 server_cookie=6b082a1b1de9b2d6 in_seq=0 out_seq=0 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc0ea7fc640 10 client.6198 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc0ea7fc640 1 -- 172.21.15.26:0/3139917745 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55b3c489df40 con 0x55b3c4978d60 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc0ea7fc640 10 client.6198 handle_command_reply: tid=0 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 client.6198 shutdown 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 -- 172.21.15.26:0/3139917745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c495cd60 msgr2=0x55b3c49f1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c495cd60 0x55b3c49f1760 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc035040 tx=0x7fc0fc004730 comp rx=0 tx=0).stop 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 -- 172.21.15.26:0/3139917745 shutdown_connections 2024-04-24T23:23:42.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x55b3c4978d60 0x55b3c4930780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b3c497a600 0x55b3c49847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b3c4970c20 0x55b3c4984280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc0ea7fc640 0 client.6198 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 --2- 172.21.15.26:0/3139917745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3c495cd60 0x55b3c49f1760 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 -- 172.21.15.26:0/3139917745 >> 172.21.15.26:0/3139917745 conn(0x55b3c4932a40 msgr2=0x55b3c495fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 -- 172.21.15.26:0/3139917745 shutdown_connections 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 1 -- 172.21.15.26:0/3139917745 wait complete. 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc114c4e740 20 client.6198 trim_cache size 0 max 16384 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:42.169 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:23:42.170 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:23:42.170 INFO:teuthology.orchestra.run.smithi026.stdout: "2f83100e-c05d-4f54-a685-af48cef98182": { 2024-04-24T23:23:42.170 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:23:42.170 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "2f83100e-c05d-4f54-a685-af48cef98182", 2024-04-24T23:23:42.170 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:23:42.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:42.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:42.170 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.162+0000 7fc111f42640 1 -- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc0f806be60 msgr2=0x7fc0f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc0f806be60 0x7fc0f806e320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc034000 tx=0x7fc0fc02e000 comp rx=0 tx=0).stop 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 -- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15f080 msgr2=0x7fc10c1f2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15f080 0x7fc10c1f2da0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc104007e90 tx=0x7fc10400e4b0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 -- 172.21.15.26:0/3399332615 shutdown_connections 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc0f806be60 0x7fc0f806e320 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc10c15f080 0x7fc10c1f2da0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc10c15e370 0x7fc10c15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 --2- 172.21.15.26:0/3399332615 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc10c0d4c10 0x7fc10c0b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 -- 172.21.15.26:0/3399332615 >> 172.21.15.26:0/3399332615 conn(0x7fc10c0d40f0 msgr2=0x7fc10c0d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:42.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 -- 172.21.15.26:0/3399332615 shutdown_connections 2024-04-24T23:23:42.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:42.163+0000 7fc111f42640 1 -- 172.21.15.26:0/3399332615 wait complete. 2024-04-24T23:23:45.847 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:23:45.846+0000 7f26a3749640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T23:23:47.176 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:23:47.176 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-04-24T23:23:47.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.290+0000 7fe59395a640 1 Processor -- start 2024-04-24T23:23:47.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.290+0000 7fe59395a640 1 -- start start 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.291+0000 7fe59395a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe58c0d0750 0x7fe58c0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.291+0000 7fe59395a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58c0aeb10 con 0x7fe58c0b4ce0 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.291+0000 7fe59395a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58c0aedf0 con 0x7fe58c15be40 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.291+0000 7fe59395a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe58c09df20 con 0x7fe58c0d0750 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.291+0000 7fe590ef5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe58c0d0750 0x7fe58c0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.291+0000 7fe590ef5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe58c0d0750 0x7fe58c0acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54958/0 (socket says 172.21.15.26:54958) 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.291+0000 7fe591ef7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe58c0b4ce0 0x7fe58c15c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54178/0 (socket says 172.21.15.26:54178) 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.291+0000 7fe590ef5640 1 -- 172.21.15.26:0/1500694204 learned_addr learned my addr 172.21.15.26:0/1500694204 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.292+0000 7fe590ef5640 1 -- 172.21.15.26:0/1500694204 >> v1:172.21.15.29:6789/0 conn(0x7fe58c15be40 legacy=0x7fe58c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.292+0000 7fe590ef5640 1 -- 172.21.15.26:0/1500694204 >> v1:172.21.15.26:6789/0 conn(0x7fe58c0b4ce0 legacy=0x7fe58c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.292+0000 7fe583fff640 1 -- 172.21.15.26:0/1500694204 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52590294 0 0) 0x7fe58c0aeb10 con 0x7fe58c0b4ce0 2024-04-24T23:23:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.292+0000 7fe590ef5640 1 -- 172.21.15.26:0/1500694204 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c0af0d0 con 0x7fe58c0d0750 2024-04-24T23:23:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.292+0000 7fe590ef5640 1 --2- 172.21.15.26:0/1500694204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c0d0750 0x7fe58c0acd30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009870 tx=0x7fe57c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aed5fe640b10967 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe583fff640 1 -- 172.21.15.26:0/1500694204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c037070 con 0x7fe58c0d0750 2024-04-24T23:23:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe583fff640 1 -- 172.21.15.26:0/1500694204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57c02ae50 con 0x7fe58c0d0750 2024-04-24T23:23:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe583fff640 1 -- 172.21.15.26:0/1500694204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c032860 con 0x7fe58c0d0750 2024-04-24T23:23:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe59395a640 1 -- 172.21.15.26:0/1500694204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c0d0750 msgr2=0x7fe58c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe59395a640 1 --2- 172.21.15.26:0/1500694204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c0d0750 0x7fe58c0acd30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009870 tx=0x7fe57c02a300 comp rx=0 tx=0).stop 2024-04-24T23:23:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe59395a640 1 -- 172.21.15.26:0/1500694204 shutdown_connections 2024-04-24T23:23:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe59395a640 1 --2- 172.21.15.26:0/1500694204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c0d0750 0x7fe58c0acd30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe59395a640 1 -- 172.21.15.26:0/1500694204 >> 172.21.15.26:0/1500694204 conn(0x7fe58c06b540 msgr2=0x7fe58c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:47.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe59395a640 1 -- 172.21.15.26:0/1500694204 shutdown_connections 2024-04-24T23:23:47.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.293+0000 7fe59395a640 1 -- 172.21.15.26:0/1500694204 wait complete. 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.294+0000 7fe59395a640 1 Processor -- start 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe59395a640 1 -- start start 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe59395a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58c0b4ce0 0x7fe58c1f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe59395a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe58c0d0750 0x7fe58c157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe59395a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c15be40 0x7fe58c09b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe59395a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c06cd10 con 0x7fe58c15be40 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe59395a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c06cb90 con 0x7fe58c0b4ce0 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe59395a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c15c930 con 0x7fe58c0d0750 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe591ef7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c15be40 0x7fe58c09b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe591ef7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c15be40 0x7fe58c09b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54974/0 (socket says 172.21.15.26:54974) 2024-04-24T23:23:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe591ef7640 1 -- 172.21.15.26:0/3721088957 learned_addr learned my addr 172.21.15.26:0/3721088957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe5916f6640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58c0b4ce0 0x7fe58c1f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.295+0000 7fe590ef5640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe58c0d0750 0x7fe58c157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe591ef7640 1 -- 172.21.15.26:0/3721088957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe58c0d0750 msgr2=0x7fe58c157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe591ef7640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe58c0d0750 0x7fe58c157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe591ef7640 1 -- 172.21.15.26:0/3721088957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58c0b4ce0 msgr2=0x7fe58c1f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe591ef7640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58c0b4ce0 0x7fe58c1f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe591ef7640 1 -- 172.21.15.26:0/3721088957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c1f24d0 con 0x7fe58c15be40 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe590ef5640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe58c0d0750 0x7fe58c157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe591ef7640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c15be40 0x7fe58c09b230 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe58400c7a0 tx=0x7fe58400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe581ffb640 1 -- 172.21.15.26:0/3721088957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584018070 con 0x7fe58c15be40 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe581ffb640 1 -- 172.21.15.26:0/3721088957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58400a640 con 0x7fe58c15be40 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe59395a640 1 -- 172.21.15.26:0/3721088957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c1f27b0 con 0x7fe58c15be40 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe581ffb640 1 -- 172.21.15.26:0/3721088957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58400aab0 con 0x7fe58c15be40 2024-04-24T23:23:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.296+0000 7fe59395a640 1 -- 172.21.15.26:0/3721088957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c1f2170 con 0x7fe58c15be40 2024-04-24T23:23:47.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.298+0000 7fe581ffb640 1 -- 172.21.15.26:0/3721088957 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe58401b070 con 0x7fe58c15be40 2024-04-24T23:23:47.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.298+0000 7fe581ffb640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.298+0000 7fe581ffb640 1 -- 172.21.15.26:0/3721088957 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fe58408a430 con 0x7fe58c15be40 2024-04-24T23:23:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.298+0000 7fe5916f6640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.299+0000 7fe5916f6640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe588000f60 tx=0x7fe588009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.299+0000 7fe596664740 1 Processor -- start 2024-04-24T23:23:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.299+0000 7fe596664740 1 -- start start 2024-04-24T23:23:47.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe596664740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c1310 0x5610cf599970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe596664740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c16f0 0x5610cf5c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe596664740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf597910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe596664740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5610cf4fd6b0 con 0x5610cf5c40b0 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe596664740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5610cf4a5880 con 0x5610cf5c1310 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe596664740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5610cf4f9670 con 0x5610cf5c16f0 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe591ef7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf597910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe591ef7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf597910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54986/0 (socket says 172.21.15.26:54986) 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe591ef7640 1 -- 172.21.15.26:0/3119380954 learned_addr learned my addr 172.21.15.26:0/3119380954 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe590ef5640 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c16f0 0x5610cf5c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe5916f6640 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c1310 0x5610cf599970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.300+0000 7fe591ef7640 1 -- 172.21.15.26:0/3119380954 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c16f0 msgr2=0x5610cf5c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe591ef7640 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c16f0 0x5610cf5c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe591ef7640 1 -- 172.21.15.26:0/3119380954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c1310 msgr2=0x5610cf599970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe591ef7640 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c1310 0x5610cf599970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe591ef7640 1 -- 172.21.15.26:0/3119380954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cf2db7d0 con 0x5610cf5c40b0 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe5916f6640 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c1310 0x5610cf599970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe591ef7640 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf597910 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe5840888c0 tx=0x7fe5840581e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8860ad48e4d3d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe576ffd640 1 -- 172.21.15.26:0/3119380954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584056a50 con 0x5610cf5c40b0 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe576ffd640 1 -- 172.21.15.26:0/3119380954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe584054dc0 con 0x5610cf5c40b0 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe576ffd640 1 -- 172.21.15.26:0/3119380954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58400ae50 con 0x5610cf5c40b0 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe596664740 1 -- 172.21.15.26:0/3119380954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 msgr2=0x5610cf597910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.301+0000 7fe596664740 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf597910 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe5840888c0 tx=0x7fe5840581e0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.302+0000 7fe596664740 1 -- 172.21.15.26:0/3119380954 shutdown_connections 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.302+0000 7fe596664740 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf597910 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.302+0000 7fe596664740 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c16f0 0x5610cf5c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.302+0000 7fe596664740 1 --2- 172.21.15.26:0/3119380954 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c1310 0x5610cf599970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.302+0000 7fe596664740 1 -- 172.21.15.26:0/3119380954 >> 172.21.15.26:0/3119380954 conn(0x5610cf596600 msgr2=0x5610cf5deb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.302+0000 7fe596664740 1 -- 172.21.15.26:0/3119380954 shutdown_connections 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.302+0000 7fe596664740 1 -- 172.21.15.26:0/3119380954 wait complete. 2024-04-24T23:23:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.302+0000 7fe596664740 1 -- start start 2024-04-24T23:23:47.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.302+0000 7fe596664740 1 Processor -- start 2024-04-24T23:23:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe596664740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c16f0 0x5610cf5d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe596664740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 0x5610cf5f6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe596664740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5f7270 0x5610cf5f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe596664740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5610cf4b8f50 con 0x5610cf5f7270 2024-04-24T23:23:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe596664740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5610cf5dc870 con 0x5610cf5c16f0 2024-04-24T23:23:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe596664740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5610cf5ddca0 con 0x5610cf5c40b0 2024-04-24T23:23:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe596664740 10 client.0 _ll_register_callbacks cb 0x5610cf4f4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe591ef7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5f7270 0x5610cf5f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe591ef7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5f7270 0x5610cf5f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54988/0 (socket says 172.21.15.26:54988) 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe591ef7640 1 -- 172.21.15.26:0/1517272299 learned_addr learned my addr 172.21.15.26:0/1517272299 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe5916f6640 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c16f0 0x5610cf5d5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.303+0000 7fe590ef5640 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 0x5610cf5f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe591ef7640 1 -- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 msgr2=0x5610cf5f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe591ef7640 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 0x5610cf5f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe591ef7640 1 -- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c16f0 msgr2=0x5610cf5d5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe591ef7640 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c16f0 0x5610cf5d5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe591ef7640 1 -- 172.21.15.26:0/1517272299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610cf2113f0 con 0x5610cf5f7270 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe590ef5640 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 0x5610cf5f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe591ef7640 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5f7270 0x5610cf5f6ba0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe5840586e0 tx=0x7fe58405c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f6c04bb3bd92d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe56bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe56bfff640 1 -- 172.21.15.26:0/1517272299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584060ff0 con 0x5610cf5f7270 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe56bfff640 1 -- 172.21.15.26:0/1517272299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fe58406cb30 con 0x5610cf5f7270 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe596664740 1 -- 172.21.15.26:0/1517272299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe584054dc0 con 0x5610cf5f7270 2024-04-24T23:23:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe56bfff640 1 -- 172.21.15.26:0/1517272299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe58406cd30 con 0x5610cf5f7270 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe596664740 10 client.6213 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe596664740 1 -- 172.21.15.26:0/1517272299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x5610cf1d10f0 con 0x5610cf5f7270 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe56bfff640 1 -- 172.21.15.26:0/1517272299 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe584054dc0 con 0x5610cf5f7270 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe56bfff640 1 -- 172.21.15.26:0/1517272299 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7fe584057ba0 con 0x5610cf5f7270 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe596664740 10 client.6213 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.304+0000 7fe596664740 10 client.6213 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.305+0000 7fe596664740 20 client.6213 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.305+0000 7fe596664740 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x5610cf5dd140 0x5610cf5ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.305+0000 7fe596664740 4 client.6213 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.305+0000 7fe596664740 1 -- 172.21.15.26:0/1517272299 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610cf50abf0 con 0x5610cf5dd140 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.305+0000 7fe5916f6640 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x5610cf5dd140 0x5610cf5ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.305+0000 7fe5916f6640 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x5610cf5dd140 0x5610cf5ad830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=938feee0ceda92f2 server_cookie=4eebe26b050d8b7b in_seq=0 out_seq=0 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.305+0000 7fe56bfff640 10 client.6213 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe56bfff640 1 -- 172.21.15.26:0/1517272299 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610cf50abf0 con 0x5610cf5dd140 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe56bfff640 10 client.6213 handle_command_reply: tid=0 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 client.6213 shutdown 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 -- 172.21.15.26:0/1517272299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5f7270 msgr2=0x5610cf5f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5f7270 0x5610cf5f6ba0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe5840586e0 tx=0x7fe58405c000 comp rx=0 tx=0).stop 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 -- 172.21.15.26:0/1517272299 shutdown_connections 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x5610cf5dd140 0x5610cf5ad830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5f7270 0x5610cf5f6ba0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe56bfff640 0 client.6213 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 0x5610cf5f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 --2- 172.21.15.26:0/1517272299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c16f0 0x5610cf5d5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 -- 172.21.15.26:0/1517272299 >> 172.21.15.26:0/1517272299 conn(0x5610cf5c1310 msgr2=0x5610cf597910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:47.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 -- 172.21.15.26:0/1517272299 shutdown_connections 2024-04-24T23:23:47.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 1 -- 172.21.15.26:0/1517272299 wait complete. 2024-04-24T23:23:47.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.306+0000 7fe596664740 20 client.6213 trim_cache size 0 max 16384 2024-04-24T23:23:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.324+0000 7fe596664740 1 Processor -- start 2024-04-24T23:23:47.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.324+0000 7fe596664740 1 -- start start 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe596664740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c16f0 0x5610cf58fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe596664740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf5dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe596664740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5dd140 0x5610cf597b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe596664740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1f52f0 con 0x5610cf5c40b0 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe596664740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c072250 con 0x5610cf5c16f0 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe596664740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c072980 con 0x5610cf5dd140 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe590ef5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf5dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe590ef5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf5dc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54998/0 (socket says 172.21.15.26:54998) 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe590ef5640 1 -- 172.21.15.26:0/2748352142 learned_addr learned my addr 172.21.15.26:0/2748352142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe591ef7640 1 --2- 172.21.15.26:0/2748352142 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5dd140 0x5610cf597b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe590ef5640 1 -- 172.21.15.26:0/2748352142 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5dd140 msgr2=0x5610cf597b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe590ef5640 1 --2- 172.21.15.26:0/2748352142 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5dd140 0x5610cf597b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe590ef5640 1 -- 172.21.15.26:0/2748352142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c16f0 msgr2=0x5610cf58fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe590ef5640 1 --2- 172.21.15.26:0/2748352142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c16f0 0x5610cf58fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe590ef5640 1 -- 172.21.15.26:0/2748352142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cf5d6d20 con 0x5610cf5c40b0 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe590ef5640 1 --2- 172.21.15.26:0/2748352142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf5dc1b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe57c035040 tx=0x7fe57c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32b83c56f05a119 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe5767fc640 1 -- 172.21.15.26:0/2748352142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c037040 con 0x5610cf5c40b0 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.325+0000 7fe5767fc640 1 -- 172.21.15.26:0/2748352142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57c02ae50 con 0x5610cf5c40b0 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe5767fc640 1 -- 172.21.15.26:0/2748352142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c003610 con 0x5610cf5c40b0 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe596664740 1 -- 172.21.15.26:0/2748352142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 msgr2=0x5610cf5dc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe596664740 1 --2- 172.21.15.26:0/2748352142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf5dc1b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe57c035040 tx=0x7fe57c02c000 comp rx=0 tx=0).stop 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe596664740 1 -- 172.21.15.26:0/2748352142 shutdown_connections 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe596664740 1 --2- 172.21.15.26:0/2748352142 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5dd140 0x5610cf597b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe596664740 1 --2- 172.21.15.26:0/2748352142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c40b0 0x5610cf5dc1b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe596664740 1 --2- 172.21.15.26:0/2748352142 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5c16f0 0x5610cf58fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe596664740 1 -- 172.21.15.26:0/2748352142 >> 172.21.15.26:0/2748352142 conn(0x5610cf5c1310 msgr2=0x5610cf5d5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe596664740 1 -- 172.21.15.26:0/2748352142 shutdown_connections 2024-04-24T23:23:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.326+0000 7fe596664740 1 -- 172.21.15.26:0/2748352142 wait complete. 2024-04-24T23:23:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.327+0000 7fe596664740 1 -- start start 2024-04-24T23:23:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.327+0000 7fe596664740 1 Processor -- start 2024-04-24T23:23:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe596664740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c16f0 0x5610cf5e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe596664740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 0x5610cf597b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe596664740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5dd140 0x5610cf598040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe596664740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c14d220 con 0x5610cf5c16f0 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe596664740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5610cf5aaf90 con 0x5610cf5dd140 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe596664740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5610cf654ce0 con 0x5610cf5c40b0 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe596664740 10 client.0 _ll_register_callbacks cb 0x7fe584015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe5916f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c16f0 0x5610cf5e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe5916f6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c16f0 0x5610cf5e9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55012/0 (socket says 172.21.15.26:55012) 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe5916f6640 1 -- 172.21.15.26:0/2588290088 learned_addr learned my addr 172.21.15.26:0/2588290088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe590ef5640 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 0x5610cf597b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe591ef7640 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5dd140 0x5610cf598040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe5916f6640 1 -- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 msgr2=0x5610cf597b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe5916f6640 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 0x5610cf597b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe5916f6640 1 -- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5dd140 msgr2=0x5610cf598040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe5916f6640 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5dd140 0x5610cf598040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe5916f6640 1 -- 172.21.15.26:0/2588290088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610cf599fb0 con 0x5610cf5c16f0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe5916f6640 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c16f0 0x5610cf5e9f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe58800fcc0 tx=0x7fe588007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518b9aba828983a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe56bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe56bfff640 1 -- 172.21.15.26:0/2588290088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58801a040 con 0x5610cf5c16f0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.328+0000 7fe56bfff640 1 -- 172.21.15.26:0/2588290088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fe588014450 con 0x5610cf5c16f0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe596664740 1 -- 172.21.15.26:0/2588290088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57c02ae50 con 0x5610cf5c16f0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe56bfff640 1 -- 172.21.15.26:0/2588290088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe588037050 con 0x5610cf5c16f0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe596664740 10 client.6219 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe596664740 1 -- 172.21.15.26:0/2588290088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x5610cf654a20 con 0x5610cf5c16f0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe56bfff640 1 -- 172.21.15.26:0/2588290088 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57c02ae50 con 0x5610cf5c16f0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe56bfff640 1 -- 172.21.15.26:0/2588290088 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7fe58803a020 con 0x5610cf5c16f0 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe596664740 10 client.6219 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe596664740 10 client.6219 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe596664740 20 client.6219 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe596664740 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x5610cf5f7270 0x5610cf653bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe596664740 4 client.6219 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe596664740 1 -- 172.21.15.26:0/2588290088 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610cf504f40 con 0x5610cf5f7270 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.329+0000 7fe590ef5640 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x5610cf5f7270 0x5610cf653bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe56bfff640 10 client.6219 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe590ef5640 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x5610cf5f7270 0x5610cf653bd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f22b1b868f8f92eb server_cookie=99a15402f6a23aff in_seq=0 out_seq=0 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe56bfff640 1 -- 172.21.15.26:0/2588290088 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5610cf504f40 con 0x5610cf5f7270 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe56bfff640 10 client.6219 handle_command_reply: tid=0 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 client.6219 shutdown 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 -- 172.21.15.26:0/2588290088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c16f0 msgr2=0x5610cf5e9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c16f0 0x5610cf5e9f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe58800fcc0 tx=0x7fe588007e30 comp rx=0 tx=0).stop 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 -- 172.21.15.26:0/2588290088 shutdown_connections 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x5610cf5f7270 0x5610cf653bd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5610cf5dd140 0x5610cf598040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe56bfff640 0 client.6219 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5610cf5c40b0 0x5610cf597b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 --2- 172.21.15.26:0/2588290088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5610cf5c16f0 0x5610cf5e9f30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 -- 172.21.15.26:0/2588290088 >> 172.21.15.26:0/2588290088 conn(0x5610cf5c1310 msgr2=0x5610cf5dc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 -- 172.21.15.26:0/2588290088 shutdown_connections 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 1 -- 172.21.15.26:0/2588290088 wait complete. 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.330+0000 7fe596664740 20 client.6219 trim_cache size 0 max 16384 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:23:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: "2f83100e-c05d-4f54-a685-af48cef98182": { 2024-04-24T23:23:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:23:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "2f83100e-c05d-4f54-a685-af48cef98182", 2024-04-24T23:23:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:23:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:23:47.338 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 -- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 msgr2=0x7fe57006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe588000f60 tx=0x7fe588009040 comp rx=0 tx=0).stop 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 -- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c15be40 msgr2=0x7fe58c09b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c15be40 0x7fe58c09b230 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe58400c7a0 tx=0x7fe58400cc60 comp rx=0 tx=0).stop 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 -- 172.21.15.26:0/3721088957 shutdown_connections 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58c15be40 0x7fe58c09b230 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe58c0d0750 0x7fe58c157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 --2- 172.21.15.26:0/3721088957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58c0b4ce0 0x7fe58c1f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 -- 172.21.15.26:0/3721088957 >> 172.21.15.26:0/3721088957 conn(0x7fe58c06b540 msgr2=0x7fe58c15c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:47.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 -- 172.21.15.26:0/3721088957 shutdown_connections 2024-04-24T23:23:47.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:47.331+0000 7fe59395a640 1 -- 172.21.15.26:0/3721088957 wait complete. 2024-04-24T23:23:52.343 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:23:52.344 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-04-24T23:23:52.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.458+0000 7f485aecd640 1 Processor -- start 2024-04-24T23:23:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.458+0000 7f485aecd640 1 -- start start 2024-04-24T23:23:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.459+0000 7f485aecd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f48540b4ce0 0x7f48540acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.459+0000 7f485aecd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48540aeb10 con 0x7f485415be40 2024-04-24T23:23:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.459+0000 7f485aecd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48540aedf0 con 0x7f48540d0750 2024-04-24T23:23:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.459+0000 7f485aecd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f485409df20 con 0x7f48540b4ce0 2024-04-24T23:23:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.459+0000 7f4858c69640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f485415be40 0x7f48540d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54184/0 (socket says 172.21.15.26:54184) 2024-04-24T23:23:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.459+0000 7f4858c69640 1 -- 172.21.15.26:0/787652303 learned_addr learned my addr 172.21.15.26:0/787652303 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.459+0000 7f485946a640 1 --2- 172.21.15.26:0/787652303 >> v2:172.21.15.29:3300/0 conn(0x7f48540b4ce0 0x7f48540acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.460+0000 7f48537fe640 1 -- 172.21.15.26:0/787652303 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2579295370 0 0) 0x7f48540aeb10 con 0x7f485415be40 2024-04-24T23:23:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.460+0000 7f48537fe640 1 -- 172.21.15.26:0/787652303 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f483c003520 con 0x7f485415be40 2024-04-24T23:23:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.460+0000 7f48537fe640 1 -- 172.21.15.26:0/787652303 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1884567929 0 0) 0x7f48540aedf0 con 0x7f48540d0750 2024-04-24T23:23:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.460+0000 7f48537fe640 1 -- 172.21.15.26:0/787652303 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48540aeb10 con 0x7f48540d0750 2024-04-24T23:23:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.460+0000 7f485946a640 1 -- 172.21.15.26:0/787652303 >> v1:172.21.15.29:6789/0 conn(0x7f48540d0750 legacy=0x7f485415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.460+0000 7f48537fe640 1 -- 172.21.15.26:0/787652303 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 331258675 0 0) 0x7f483c003520 con 0x7f485415be40 2024-04-24T23:23:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.460+0000 7f485946a640 1 -- 172.21.15.26:0/787652303 >> v1:172.21.15.26:6789/0 conn(0x7f485415be40 legacy=0x7f48540d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.460+0000 7f485946a640 1 -- 172.21.15.26:0/787652303 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48540af0d0 con 0x7f48540b4ce0 2024-04-24T23:23:52.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.461+0000 7f485946a640 1 --2- 172.21.15.26:0/787652303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48540b4ce0 0x7f48540acd30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f484c009d90 tx=0x7f484c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50f66b598064e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.461+0000 7f48537fe640 1 -- 172.21.15.26:0/787652303 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844001670 con 0x7f48540b4ce0 2024-04-24T23:23:52.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.461+0000 7f48537fe640 1 -- 172.21.15.26:0/787652303 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484c007540 con 0x7f48540b4ce0 2024-04-24T23:23:52.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.461+0000 7f48537fe640 1 -- 172.21.15.26:0/787652303 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c007910 con 0x7f48540b4ce0 2024-04-24T23:23:52.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.461+0000 7f485aecd640 1 -- 172.21.15.26:0/787652303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48540b4ce0 msgr2=0x7f48540acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.461+0000 7f485aecd640 1 --2- 172.21.15.26:0/787652303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48540b4ce0 0x7f48540acd30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f484c009d90 tx=0x7f484c02a680 comp rx=0 tx=0).stop 2024-04-24T23:23:52.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.461+0000 7f485aecd640 1 -- 172.21.15.26:0/787652303 shutdown_connections 2024-04-24T23:23:52.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.461+0000 7f485aecd640 1 --2- 172.21.15.26:0/787652303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48540b4ce0 0x7f48540acd30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.461+0000 7f485aecd640 1 -- 172.21.15.26:0/787652303 >> 172.21.15.26:0/787652303 conn(0x7f485406b540 msgr2=0x7f485406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.462+0000 7f485aecd640 1 -- 172.21.15.26:0/787652303 shutdown_connections 2024-04-24T23:23:52.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.462+0000 7f485aecd640 1 -- 172.21.15.26:0/787652303 wait complete. 2024-04-24T23:23:52.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.463+0000 7f485aecd640 1 Processor -- start 2024-04-24T23:23:52.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.463+0000 7f485aecd640 1 -- start start 2024-04-24T23:23:52.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f485aecd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48540b4ce0 0x7f4854157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f485aecd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48540d0750 0x7f48540ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f485aecd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f485415be40 0x7f48540ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f485aecd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f485406cd10 con 0x7f48540b4ce0 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f485aecd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f485406cb90 con 0x7f48540d0750 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f485aecd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f485415c930 con 0x7f485415be40 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4858c69640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48540b4ce0 0x7f4854157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4858c69640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48540b4ce0 0x7f4854157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55020/0 (socket says 172.21.15.26:55020) 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4858c69640 1 -- 172.21.15.26:0/3071695358 learned_addr learned my addr 172.21.15.26:0/3071695358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f485946a640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f485415be40 0x7f48540ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4853fff640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48540d0750 0x7f48540ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4858c69640 1 -- 172.21.15.26:0/3071695358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f485415be40 msgr2=0x7f48540ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4858c69640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f485415be40 0x7f48540ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4858c69640 1 -- 172.21.15.26:0/3071695358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48540d0750 msgr2=0x7f48540ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4858c69640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48540d0750 0x7f48540ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4858c69640 1 -- 172.21.15.26:0/3071695358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48540c5840 con 0x7f48540b4ce0 2024-04-24T23:23:52.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.464+0000 7f4858c69640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48540b4ce0 0x7f4854157d50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f484400c4f0 tx=0x7f484400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.465+0000 7f48517fa640 1 -- 172.21.15.26:0/3071695358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844017070 con 0x7f48540b4ce0 2024-04-24T23:23:52.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.465+0000 7f48517fa640 1 -- 172.21.15.26:0/3071695358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4844005ce0 con 0x7f48540b4ce0 2024-04-24T23:23:52.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.465+0000 7f48517fa640 1 -- 172.21.15.26:0/3071695358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844007290 con 0x7f48540b4ce0 2024-04-24T23:23:52.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.465+0000 7f485aecd640 1 -- 172.21.15.26:0/3071695358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48540c5b20 con 0x7f48540b4ce0 2024-04-24T23:23:52.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.465+0000 7f485aecd640 1 -- 172.21.15.26:0/3071695358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485409bd30 con 0x7f48540b4ce0 2024-04-24T23:23:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.466+0000 7f48517fa640 1 -- 172.21.15.26:0/3071695358 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f484401a070 con 0x7f48540b4ce0 2024-04-24T23:23:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.467+0000 7f48517fa640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f484006be30 0x7f484006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.467+0000 7f48517fa640 1 -- 172.21.15.26:0/3071695358 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f484408a180 con 0x7f48540b4ce0 2024-04-24T23:23:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.467+0000 7f4853fff640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f484006be30 0x7f484006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.467+0000 7f4853fff640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f484006be30 0x7f484006e2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f48540c65b0 tx=0x7f4848009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.468+0000 7f485dbdb740 1 Processor -- start 2024-04-24T23:23:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.468+0000 7f485dbdb740 1 -- start start 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.468+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaedfe310 0x562aaedd6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.468+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaee00b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.468+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee010b0 0x562aaedd4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.468+0000 7f485dbdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562aaed3a6b0 con 0x562aaee010b0 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.468+0000 7f485dbdb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562aaece2880 con 0x562aaedfe6f0 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.468+0000 7f485dbdb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562aaed36670 con 0x562aaedfe310 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f485946a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee010b0 0x562aaedd4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f485946a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee010b0 0x562aaedd4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55022/0 (socket says 172.21.15.26:55022) 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f485946a640 1 -- 172.21.15.26:0/991123408 learned_addr learned my addr 172.21.15.26:0/991123408 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f4858c69640 1 --2- 172.21.15.26:0/991123408 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaedfe310 0x562aaedd6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f4853fff640 1 --2- 172.21.15.26:0/991123408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaee00b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f485946a640 1 -- 172.21.15.26:0/991123408 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaedfe310 msgr2=0x562aaedd6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f485946a640 1 --2- 172.21.15.26:0/991123408 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaedfe310 0x562aaedd6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f485946a640 1 -- 172.21.15.26:0/991123408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 msgr2=0x562aaee00b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f485946a640 1 --2- 172.21.15.26:0/991123408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaee00b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f485946a640 1 -- 172.21.15.26:0/991123408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aaeb187d0 con 0x562aaee010b0 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f485946a640 1 --2- 172.21.15.26:0/991123408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee010b0 0x562aaedd4910 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f484c009eb0 tx=0x7f484c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fec0c30c4a4b4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f483a7fc640 1 -- 172.21.15.26:0/991123408 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c007740 con 0x562aaee010b0 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f483a7fc640 1 -- 172.21.15.26:0/991123408 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484c0078e0 con 0x562aaee010b0 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.469+0000 7f483a7fc640 1 -- 172.21.15.26:0/991123408 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c007c90 con 0x562aaee010b0 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.470+0000 7f485dbdb740 1 -- 172.21.15.26:0/991123408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee010b0 msgr2=0x562aaedd4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.470+0000 7f485dbdb740 1 --2- 172.21.15.26:0/991123408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee010b0 0x562aaedd4910 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f484c009eb0 tx=0x7f484c02e000 comp rx=0 tx=0).stop 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.470+0000 7f485dbdb740 1 -- 172.21.15.26:0/991123408 shutdown_connections 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.470+0000 7f485dbdb740 1 --2- 172.21.15.26:0/991123408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee010b0 0x562aaedd4910 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.470+0000 7f485dbdb740 1 --2- 172.21.15.26:0/991123408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaee00b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.470+0000 7f485dbdb740 1 --2- 172.21.15.26:0/991123408 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaedfe310 0x562aaedd6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.470+0000 7f485dbdb740 1 -- 172.21.15.26:0/991123408 >> 172.21.15.26:0/991123408 conn(0x562aaedd3600 msgr2=0x562aaee1bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.470+0000 7f485dbdb740 1 -- 172.21.15.26:0/991123408 shutdown_connections 2024-04-24T23:23:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.470+0000 7f485dbdb740 1 -- 172.21.15.26:0/991123408 wait complete. 2024-04-24T23:23:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.471+0000 7f485dbdb740 1 -- start start 2024-04-24T23:23:52.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.471+0000 7f485dbdb740 1 Processor -- start 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaedfe6f0 0x562aaedef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee00040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaee00580 0x562aaee132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f485dbdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562aaecf5f50 con 0x562aaedfe6f0 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f485dbdb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562aaee19870 con 0x562aaee00580 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f485dbdb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562aaee00a60 con 0x562aaee010b0 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f485dbdb740 10 client.0 _ll_register_callbacks cb 0x562aaed31000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4858c69640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaedfe6f0 0x562aaedef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4858c69640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaedfe6f0 0x562aaedef520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55028/0 (socket says 172.21.15.26:55028) 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4858c69640 1 -- 172.21.15.26:0/2655216383 learned_addr learned my addr 172.21.15.26:0/2655216383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4853fff640 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee00040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f485946a640 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaee00580 0x562aaee132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4858c69640 1 -- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 msgr2=0x562aaee00040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4858c69640 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee00040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4858c69640 1 -- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaee00580 msgr2=0x562aaee132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4858c69640 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaee00580 0x562aaee132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4858c69640 1 -- 172.21.15.26:0/2655216383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aaea4e3f0 con 0x562aaedfe6f0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f4858c69640 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaedfe6f0 0x562aaedef520 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f484400ceb0 tx=0x7f484405c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45962cebf68f0f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f48337fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f48337fe640 1 -- 172.21.15.26:0/2655216383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844070020 con 0x562aaedfe6f0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f48337fe640 1 -- 172.21.15.26:0/2655216383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f4844060dd0 con 0x562aaedfe6f0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.472+0000 7f485dbdb740 1 -- 172.21.15.26:0/2655216383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f484c0078e0 con 0x562aaedfe6f0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f48337fe640 1 -- 172.21.15.26:0/2655216383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4844075050 con 0x562aaedfe6f0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f485dbdb740 10 client.6231 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f485dbdb740 1 -- 172.21.15.26:0/2655216383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x562aaea0e0f0 con 0x562aaedfe6f0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f48337fe640 1 -- 172.21.15.26:0/2655216383 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484c0078e0 con 0x562aaedfe6f0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f48337fe640 1 -- 172.21.15.26:0/2655216383 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7f48440772e0 con 0x562aaedfe6f0 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f485dbdb740 10 client.6231 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f485dbdb740 10 client.6231 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f485dbdb740 20 client.6231 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x562aaee34710 0x562aaee34b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f485dbdb740 4 client.6231 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f485dbdb740 1 -- 172.21.15.26:0/2655216383 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aaed47bf0 con 0x562aaee34710 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.473+0000 7f4853fff640 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x562aaee34710 0x562aaee34b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.474+0000 7f4853fff640 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x562aaee34710 0x562aaee34b60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2c65e2c2f8a8149 server_cookie=d2ef8a9d0d2fd13d in_seq=0 out_seq=0 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.474+0000 7f48337fe640 10 client.6231 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.474+0000 7f48337fe640 1 -- 172.21.15.26:0/2655216383 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562aaed47bf0 con 0x562aaee34710 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.474+0000 7f48337fe640 10 client.6231 handle_command_reply: tid=0 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.474+0000 7f485dbdb740 1 client.6231 shutdown 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.474+0000 7f485dbdb740 1 -- 172.21.15.26:0/2655216383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaedfe6f0 msgr2=0x562aaedef520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.474+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaedfe6f0 0x562aaedef520 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f484400ceb0 tx=0x7f484405c830 comp rx=0 tx=0).stop 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f485dbdb740 1 -- 172.21.15.26:0/2655216383 shutdown_connections 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x562aaee34710 0x562aaee34b60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaee00580 0x562aaee132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee00040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f48337fe640 0 client.6231 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2655216383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaedfe6f0 0x562aaedef520 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f485dbdb740 1 -- 172.21.15.26:0/2655216383 >> 172.21.15.26:0/2655216383 conn(0x562aaedfe310 msgr2=0x562aaee1eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f485dbdb740 1 -- 172.21.15.26:0/2655216383 shutdown_connections 2024-04-24T23:23:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f485dbdb740 1 -- 172.21.15.26:0/2655216383 wait complete. 2024-04-24T23:23:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.475+0000 7f485dbdb740 20 client.6231 trim_cache size 0 max 16384 2024-04-24T23:23:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.492+0000 7f485dbdb740 1 Processor -- start 2024-04-24T23:23:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f485dbdb740 1 -- start start 2024-04-24T23:23:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaedd5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaedfcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee13b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f485dbdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f48541fc1b0 con 0x562aaee00580 2024-04-24T23:23:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f485dbdb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f485409c980 con 0x562aaedfe6f0 2024-04-24T23:23:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f485dbdb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f485409d0b0 con 0x562aaee010b0 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f4853fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaedfcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f4853fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaedfcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55036/0 (socket says 172.21.15.26:55036) 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.493+0000 7f4853fff640 1 -- 172.21.15.26:0/2138402454 learned_addr learned my addr 172.21.15.26:0/2138402454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4858c69640 1 --2- 172.21.15.26:0/2138402454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaedd5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f485946a640 1 --2- 172.21.15.26:0/2138402454 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee13b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4853fff640 1 -- 172.21.15.26:0/2138402454 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 msgr2=0x562aaee13b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4853fff640 1 --2- 172.21.15.26:0/2138402454 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee13b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4853fff640 1 -- 172.21.15.26:0/2138402454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 msgr2=0x562aaedd5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4853fff640 1 --2- 172.21.15.26:0/2138402454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaedd5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4853fff640 1 -- 172.21.15.26:0/2138402454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aaedd8580 con 0x562aaee00580 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4853fff640 1 --2- 172.21.15.26:0/2138402454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaedfcc70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4848007fd0 tx=0x7f484800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46e4fbd1ab389e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4839ffb640 1 -- 172.21.15.26:0/2138402454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484800fe50 con 0x562aaee00580 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4839ffb640 1 -- 172.21.15.26:0/2138402454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484801b040 con 0x562aaee00580 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.494+0000 7f4839ffb640 1 -- 172.21.15.26:0/2138402454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484802ab50 con 0x562aaee00580 2024-04-24T23:23:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 -- 172.21.15.26:0/2138402454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 msgr2=0x562aaedfcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2138402454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaedfcc70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4848007fd0 tx=0x7f484800a000 comp rx=0 tx=0).stop 2024-04-24T23:23:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 -- 172.21.15.26:0/2138402454 shutdown_connections 2024-04-24T23:23:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2138402454 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee13b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2138402454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaedfcc70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 --2- 172.21.15.26:0/2138402454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaedd5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 -- 172.21.15.26:0/2138402454 >> 172.21.15.26:0/2138402454 conn(0x562aaedfe310 msgr2=0x562aaedffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 -- 172.21.15.26:0/2138402454 shutdown_connections 2024-04-24T23:23:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 -- 172.21.15.26:0/2138402454 wait complete. 2024-04-24T23:23:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 -- start start 2024-04-24T23:23:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.495+0000 7f485dbdb740 1 Processor -- start 2024-04-24T23:23:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaee1f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaee9e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f485dbdb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee9a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f485dbdb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4854072480 con 0x562aaee00580 2024-04-24T23:23:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f485dbdb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562aaee34d20 con 0x562aaedfe6f0 2024-04-24T23:23:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f485dbdb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562aaee9aeb0 con 0x562aaee010b0 2024-04-24T23:23:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f485dbdb740 10 client.0 _ll_register_callbacks cb 0x7f4844057eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f4853fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaee9e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f4853fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaee9e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55050/0 (socket says 172.21.15.26:55050) 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.496+0000 7f4853fff640 1 -- 172.21.15.26:0/1463008666 learned_addr learned my addr 172.21.15.26:0/1463008666 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f4858c69640 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaee1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f485946a640 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee9a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f4853fff640 1 -- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 msgr2=0x562aaee9a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f4853fff640 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee9a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f4853fff640 1 -- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 msgr2=0x562aaee1f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f4853fff640 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaee1f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f4853fff640 1 -- 172.21.15.26:0/1463008666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aaee43770 con 0x562aaee00580 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f4853fff640 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaee9e190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f484802c040 tx=0x7f4848001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882073c774a2e558 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f48337fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f48337fe640 1 -- 172.21.15.26:0/1463008666 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848038020 con 0x562aaee00580 2024-04-24T23:23:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f48337fe640 1 -- 172.21.15.26:0/1463008666 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f484802a4c0 con 0x562aaee00580 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f485dbdb740 1 -- 172.21.15.26:0/1463008666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f484801b040 con 0x562aaee00580 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f48337fe640 1 -- 172.21.15.26:0/1463008666 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4848042050 con 0x562aaee00580 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f485dbdb740 10 client.6237 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f485dbdb740 1 -- 172.21.15.26:0/1463008666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x562aaee9ac50 con 0x562aaee00580 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f48337fe640 1 -- 172.21.15.26:0/1463008666 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484801b040 con 0x562aaee00580 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f48337fe640 1 -- 172.21.15.26:0/1463008666 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7f4848045020 con 0x562aaee00580 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f485dbdb740 10 client.6237 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f485dbdb740 10 client.6237 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.497+0000 7f485dbdb740 20 client.6237 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f485dbdb740 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x562aaee9a7b0 0x562aaee907d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f485dbdb740 4 client.6237 mds_command: new command op to 5329 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f485dbdb740 1 -- 172.21.15.26:0/1463008666 --> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562aaedd1ed0 con 0x562aaee9a7b0 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f4858c69640 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x562aaee9a7b0 0x562aaee907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f4858c69640 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x562aaee9a7b0 0x562aaee907d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ed29243c9b8cd94 server_cookie=db2163ffedd65943 in_seq=0 out_seq=0 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f48337fe640 10 client.6237 ms_handle_connect on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f48337fe640 1 -- 172.21.15.26:0/1463008666 <== mds.1 v2:172.21.15.29:6833/243988267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562aaedd1ed0 con 0x562aaee9a7b0 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f48337fe640 10 client.6237 handle_command_reply: tid=0 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f485dbdb740 1 client.6237 shutdown 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f485dbdb740 1 -- 172.21.15.26:0/1463008666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 msgr2=0x562aaee9e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.498+0000 7f485dbdb740 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaee9e190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f484802c040 tx=0x7f4848001150 comp rx=0 tx=0).stop 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485dbdb740 1 -- 172.21.15.26:0/1463008666 shutdown_connections 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485dbdb740 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:6833/243988267,v1:172.21.15.29:6834/243988267] conn(0x562aaee9a7b0 0x562aaee907d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485dbdb740 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562aaee010b0 0x562aaee9a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485dbdb740 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aaee00580 0x562aaee9e190 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485dbdb740 1 --2- 172.21.15.26:0/1463008666 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aaedfe6f0 0x562aaee1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f48337fe640 0 client.6237 ms_handle_reset on v2:172.21.15.29:6833/243988267 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485dbdb740 1 -- 172.21.15.26:0/1463008666 >> 172.21.15.26:0/1463008666 conn(0x562aaedfe310 msgr2=0x562aaee022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485dbdb740 1 -- 172.21.15.26:0/1463008666 shutdown_connections 2024-04-24T23:23:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485dbdb740 1 -- 172.21.15.26:0/1463008666 wait complete. 2024-04-24T23:23:52.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485dbdb740 20 client.6237 trim_cache size 0 max 16384 2024-04-24T23:23:52.506 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:52.506 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-04-24T23:23:52.506 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-04-24T23:23:52.506 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:52.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 -- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f484006be30 msgr2=0x7f484006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f484006be30 0x7f484006e2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f48540c65b0 tx=0x7f4848009040 comp rx=0 tx=0).stop 2024-04-24T23:23:52.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 -- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48540b4ce0 msgr2=0x7f4854157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48540b4ce0 0x7f4854157d50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f484400c4f0 tx=0x7f484400c9b0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 -- 172.21.15.26:0/3071695358 shutdown_connections 2024-04-24T23:23:52.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f484006be30 0x7f484006e2f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f485415be40 0x7f48540ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48540d0750 0x7f48540ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 --2- 172.21.15.26:0/3071695358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48540b4ce0 0x7f4854157d50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 -- 172.21.15.26:0/3071695358 >> 172.21.15.26:0/3071695358 conn(0x7f485406b540 msgr2=0x7f485415e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 -- 172.21.15.26:0/3071695358 shutdown_connections 2024-04-24T23:23:52.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.499+0000 7f485aecd640 1 -- 172.21.15.26:0/3071695358 wait complete. 2024-04-24T23:23:52.511 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:23:52.512 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:23:52.512 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:2 scrub status 2024-04-24T23:23:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.613+0000 7f83c72fc640 1 Processor -- start 2024-04-24T23:23:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.613+0000 7f83c72fc640 1 -- start start 2024-04-24T23:23:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.614+0000 7f83c72fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83c015be40 0x7f83c00acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.614+0000 7f83c72fc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c00d0750 0x7f83c015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.614+0000 7f83c72fc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c00aeb10 con 0x7f83c00b4ce0 2024-04-24T23:23:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.614+0000 7f83c72fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f83c009df20 con 0x7f83c015be40 2024-04-24T23:23:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.614+0000 7f83c72fc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f83c00aedf0 con 0x7f83c00d0750 2024-04-24T23:23:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.614+0000 7f83c5098640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83c015be40 0x7f83c00acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.614+0000 7f83c5098640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83c015be40 0x7f83c00acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55062/0 (socket says 172.21.15.26:55062) 2024-04-24T23:23:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.614+0000 7f83c5098640 1 -- 172.21.15.26:0/1693882127 learned_addr learned my addr 172.21.15.26:0/1693882127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.615+0000 7f83c5899640 1 --2- 172.21.15.26:0/1693882127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c00d0750 0x7f83c015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.615+0000 7f83c5098640 1 -- 172.21.15.26:0/1693882127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c00d0750 msgr2=0x7f83c015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.615+0000 7f83c5098640 1 --2- 172.21.15.26:0/1693882127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c00d0750 0x7f83c015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.615+0000 7f83c5098640 1 -- 172.21.15.26:0/1693882127 >> v1:172.21.15.29:6789/0 conn(0x7f83c00b4ce0 legacy=0x7f83c00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.615+0000 7f83c5098640 1 -- 172.21.15.26:0/1693882127 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c00af0d0 con 0x7f83c015be40 2024-04-24T23:23:52.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.615+0000 7f83c5098640 1 --2- 172.21.15.26:0/1693882127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c015be40 0x7f83c00acd30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009870 tx=0x7f83b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3447fddaf3b72b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83bffff640 1 -- 172.21.15.26:0/1693882127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0037070 con 0x7f83c015be40 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83bffff640 1 -- 172.21.15.26:0/1693882127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b0032440 con 0x7f83c015be40 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83bffff640 1 -- 172.21.15.26:0/1693882127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0032810 con 0x7f83c015be40 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83c72fc640 1 -- 172.21.15.26:0/1693882127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c015be40 msgr2=0x7f83c00acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83c72fc640 1 --2- 172.21.15.26:0/1693882127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c015be40 0x7f83c00acd30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009870 tx=0x7f83b002a270 comp rx=0 tx=0).stop 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83c72fc640 1 -- 172.21.15.26:0/1693882127 shutdown_connections 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83c72fc640 1 --2- 172.21.15.26:0/1693882127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c00d0750 0x7f83c015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83c72fc640 1 --2- 172.21.15.26:0/1693882127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c015be40 0x7f83c00acd30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83c72fc640 1 -- 172.21.15.26:0/1693882127 >> 172.21.15.26:0/1693882127 conn(0x7f83c006b540 msgr2=0x7f83c006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83c72fc640 1 -- 172.21.15.26:0/1693882127 shutdown_connections 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.616+0000 7f83c72fc640 1 -- 172.21.15.26:0/1693882127 wait complete. 2024-04-24T23:23:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.617+0000 7f83c72fc640 1 Processor -- start 2024-04-24T23:23:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.617+0000 7f83c72fc640 1 -- start start 2024-04-24T23:23:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c72fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83c00b4ce0 0x7f83c009d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c72fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c00d0750 0x7f83c0070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c72fc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c015be40 0x7f83c00ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c72fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83c006cd10 con 0x7f83c00d0750 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c72fc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83c006cb90 con 0x7f83c00b4ce0 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c72fc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f83c015c760 con 0x7f83c015be40 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c4897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c00d0750 0x7f83c0070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c4897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c00d0750 0x7f83c0070e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55068/0 (socket says 172.21.15.26:55068) 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c4897640 1 -- 172.21.15.26:0/2377341186 learned_addr learned my addr 172.21.15.26:0/2377341186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c5899640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c015be40 0x7f83c00ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.618+0000 7f83c5098640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83c00b4ce0 0x7f83c009d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83c4897640 1 -- 172.21.15.26:0/2377341186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c015be40 msgr2=0x7f83c00ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83c4897640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c015be40 0x7f83c00ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83c4897640 1 -- 172.21.15.26:0/2377341186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83c00b4ce0 msgr2=0x7f83c009d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83c4897640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83c00b4ce0 0x7f83c009d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83c4897640 1 -- 172.21.15.26:0/2377341186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c0071350 con 0x7f83c00d0750 2024-04-24T23:23:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83c5098640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83c00b4ce0 0x7f83c009d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83c4897640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c00d0750 0x7f83c0070e10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c400 tx=0x7f83b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83bdffb640 1 -- 172.21.15.26:0/2377341186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40052d0 con 0x7f83c00d0750 2024-04-24T23:23:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83bdffb640 1 -- 172.21.15.26:0/2377341186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4005490 con 0x7f83c00d0750 2024-04-24T23:23:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83c72fc640 1 -- 172.21.15.26:0/2377341186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c0071620 con 0x7f83c00d0750 2024-04-24T23:23:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83bdffb640 1 -- 172.21.15.26:0/2377341186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4007260 con 0x7f83c00d0750 2024-04-24T23:23:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.619+0000 7f83c72fc640 1 -- 172.21.15.26:0/2377341186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c0071be0 con 0x7f83c00d0750 2024-04-24T23:23:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.621+0000 7f83bdffb640 1 -- 172.21.15.26:0/2377341186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83b4007400 con 0x7f83c00d0750 2024-04-24T23:23:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.621+0000 7f83bdffb640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83ac06be30 0x7f83ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.621+0000 7f83bdffb640 1 -- 172.21.15.26:0/2377341186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f83b408a480 con 0x7f83c00d0750 2024-04-24T23:23:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.621+0000 7f83c5098640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83ac06be30 0x7f83ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.622+0000 7f83c5098640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83ac06be30 0x7f83ac06e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f83b0034000 tx=0x7f83b002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.622+0000 7f83ca00a740 1 Processor -- start 2024-04-24T23:23:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.622+0000 7f83ca00a740 1 -- start start 2024-04-24T23:23:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561167482d60 0x561167483140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674a0600 0x5611674a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167458a40 0x56116749ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83ca00a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5611673bc6b0 con 0x561167482d60 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83ca00a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561167364880 con 0x5611674a0600 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83ca00a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5611673b8670 con 0x561167458a40 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83c5098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561167482d60 0x561167483140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83c5098640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561167482d60 0x561167483140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55084/0 (socket says 172.21.15.26:55084) 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83c5098640 1 -- 172.21.15.26:0/3887284585 learned_addr learned my addr 172.21.15.26:0/3887284585 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83c5098640 1 -- 172.21.15.26:0/3887284585 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167458a40 msgr2=0x56116749ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83c5098640 1 --2- 172.21.15.26:0/3887284585 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167458a40 0x56116749ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83c5098640 1 -- 172.21.15.26:0/3887284585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674a0600 msgr2=0x5611674a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83c5098640 1 --2- 172.21.15.26:0/3887284585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674a0600 0x5611674a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83c5098640 1 -- 172.21.15.26:0/3887284585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56116719a7d0 con 0x561167482d60 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83c5098640 1 --2- 172.21.15.26:0/3887284585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561167482d60 0x561167483140 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f83b0035040 tx=0x7f83b002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b506381f0286dcd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83a6ffd640 1 -- 172.21.15.26:0/3887284585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b004e070 con 0x561167482d60 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83a6ffd640 1 -- 172.21.15.26:0/3887284585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b003e460 con 0x561167482d60 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.623+0000 7f83a6ffd640 1 -- 172.21.15.26:0/3887284585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0037040 con 0x561167482d60 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.624+0000 7f83ca00a740 1 -- 172.21.15.26:0/3887284585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561167482d60 msgr2=0x561167483140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.624+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3887284585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561167482d60 0x561167483140 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f83b0035040 tx=0x7f83b002c000 comp rx=0 tx=0).stop 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.624+0000 7f83ca00a740 1 -- 172.21.15.26:0/3887284585 shutdown_connections 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.624+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3887284585 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167458a40 0x56116749ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.624+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3887284585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674a0600 0x5611674a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.624+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3887284585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561167482d60 0x561167483140 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.624+0000 7f83ca00a740 1 -- 172.21.15.26:0/3887284585 >> 172.21.15.26:0/3887284585 conn(0x561167455600 msgr2=0x56116749db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.624+0000 7f83ca00a740 1 -- 172.21.15.26:0/3887284585 shutdown_connections 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.624+0000 7f83ca00a740 1 -- 172.21.15.26:0/3887284585 wait complete. 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.625+0000 7f83ca00a740 1 -- start start 2024-04-24T23:23:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.625+0000 7f83ca00a740 1 Processor -- start 2024-04-24T23:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167482d60 0x561167475a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167485910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674b18f0 0x561167485e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5611674597e0 con 0x5611674a0600 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561167377f50 con 0x5611674b18f0 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561167453530 con 0x561167482d60 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 10 client.0 _ll_register_callbacks cb 0x5611673b3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83c4897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167485910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83c4897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167485910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55088/0 (socket says 172.21.15.26:55088) 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83c4897640 1 -- 172.21.15.26:0/2149743849 learned_addr learned my addr 172.21.15.26:0/2149743849 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83c4897640 1 -- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167482d60 msgr2=0x561167475a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83c4897640 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167482d60 0x561167475a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83c4897640 1 -- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674b18f0 msgr2=0x561167485e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83c4897640 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674b18f0 0x561167485e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83c4897640 1 -- 172.21.15.26:0/2149743849 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611670d03f0 con 0x5611674a0600 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83c4897640 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167485910 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f83b400cfd0 tx=0x7f83b4058650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be5a4ef13caa730 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f839ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f839ffff640 1 -- 172.21.15.26:0/2149743849 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4061230 con 0x5611674a0600 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f839ffff640 1 -- 172.21.15.26:0/2149743849 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f83b4060da0 con 0x5611674a0600 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 1 -- 172.21.15.26:0/2149743849 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b003e460 con 0x5611674a0600 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f839ffff640 1 -- 172.21.15.26:0/2149743849 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f83b4060fc0 con 0x5611674a0600 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 10 client.6249 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.626+0000 7f83ca00a740 1 -- 172.21.15.26:0/2149743849 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x5611670900f0 con 0x5611674a0600 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f839ffff640 1 -- 172.21.15.26:0/2149743849 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b003e460 con 0x5611674a0600 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f839ffff640 1 -- 172.21.15.26:0/2149743849 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7f83b405db90 con 0x5611674a0600 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f83ca00a740 10 client.6249 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f83ca00a740 10 client.6249 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f83ca00a740 20 client.6249 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f83ca00a740 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x5611674950f0 0x561167495500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f83ca00a740 4 client.6249 mds_command: new command op to 5081 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f83ca00a740 1 -- 172.21.15.26:0/2149743849 --> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611673c9bf0 con 0x5611674950f0 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f83c5098640 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x5611674950f0 0x561167495500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f83c5098640 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x5611674950f0 0x561167495500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29ca871cb7af51e7 server_cookie=28c6b12bf61967d8 in_seq=0 out_seq=0 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.627+0000 7f839ffff640 10 client.6249 ms_handle_connect on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f839ffff640 1 -- 172.21.15.26:0/2149743849 <== mds.2 v2:172.21.15.29:6832/1578186344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611673c9bf0 con 0x5611674950f0 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f839ffff640 10 client.6249 handle_command_reply: tid=0 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 client.6249 shutdown 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 -- 172.21.15.26:0/2149743849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 msgr2=0x561167485910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167485910 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f83b400cfd0 tx=0x7f83b4058650 comp rx=0 tx=0).stop 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 -- 172.21.15.26:0/2149743849 shutdown_connections 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x5611674950f0 0x561167495500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674b18f0 0x561167485e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f839ffff640 0 client.6249 ms_handle_reset on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167485910 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 --2- 172.21.15.26:0/2149743849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167482d60 0x561167475a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 -- 172.21.15.26:0/2149743849 >> 172.21.15.26:0/2149743849 conn(0x561167458a40 msgr2=0x5611674a1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 -- 172.21.15.26:0/2149743849 shutdown_connections 2024-04-24T23:23:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.628+0000 7f83ca00a740 1 -- 172.21.15.26:0/2149743849 wait complete. 2024-04-24T23:23:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.629+0000 7f83ca00a740 20 client.6249 trim_cache size 0 max 16384 2024-04-24T23:23:52.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.646+0000 7f83ca00a740 1 Processor -- start 2024-04-24T23:23:52.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.646+0000 7f83ca00a740 1 -- start start 2024-04-24T23:23:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561167482d60 0x561167459010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167493500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611674b18f0 0x561167493a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83ca00a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83c02005a0 con 0x5611674a0600 2024-04-24T23:23:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83ca00a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01ff760 con 0x561167482d60 2024-04-24T23:23:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83ca00a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f83c01ffe70 con 0x5611674b18f0 2024-04-24T23:23:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c4897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167493500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c4897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167493500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55094/0 (socket says 172.21.15.26:55094) 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c4897640 1 -- 172.21.15.26:0/1535652359 learned_addr learned my addr 172.21.15.26:0/1535652359 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c5899640 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611674b18f0 0x561167493a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c5098640 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561167482d60 0x561167459010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c4897640 1 -- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611674b18f0 msgr2=0x561167493a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c4897640 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611674b18f0 0x561167493a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c4897640 1 -- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561167482d60 msgr2=0x561167459010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c4897640 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561167482d60 0x561167459010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c4897640 1 -- 172.21.15.26:0/1535652359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561167456130 con 0x5611674a0600 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c5899640 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611674b18f0 0x561167493a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c5098640 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561167482d60 0x561167459010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83c4897640 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167493500 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f83b4058f00 tx=0x7f83b405a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db4612683db7acf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83a67fc640 1 -- 172.21.15.26:0/1535652359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b405db90 con 0x5611674a0600 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83a67fc640 1 -- 172.21.15.26:0/1535652359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4057b40 con 0x5611674a0600 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.647+0000 7f83a67fc640 1 -- 172.21.15.26:0/1535652359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b406cae0 con 0x5611674a0600 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.648+0000 7f83ca00a740 1 -- 172.21.15.26:0/1535652359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 msgr2=0x561167493500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.648+0000 7f83ca00a740 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167493500 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f83b4058f00 tx=0x7f83b405a000 comp rx=0 tx=0).stop 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.648+0000 7f83ca00a740 1 -- 172.21.15.26:0/1535652359 shutdown_connections 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.648+0000 7f83ca00a740 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611674b18f0 0x561167493a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.648+0000 7f83ca00a740 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167493500 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.648+0000 7f83ca00a740 1 --2- 172.21.15.26:0/1535652359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561167482d60 0x561167459010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.648+0000 7f83ca00a740 1 -- 172.21.15.26:0/1535652359 >> 172.21.15.26:0/1535652359 conn(0x561167458a40 msgr2=0x5611674c1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.648+0000 7f83ca00a740 1 -- 172.21.15.26:0/1535652359 shutdown_connections 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.648+0000 7f83ca00a740 1 -- 172.21.15.26:0/1535652359 wait complete. 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.649+0000 7f83ca00a740 1 -- start start 2024-04-24T23:23:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.649+0000 7f83ca00a740 1 Processor -- start 2024-04-24T23:23:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.649+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167482d60 0x5611674faf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.649+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167456780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.649+0000 7f83ca00a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674b18f0 0x561167493da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.649+0000 7f83ca00a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01f86f0 con 0x5611674a0600 2024-04-24T23:23:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.649+0000 7f83ca00a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561167486f00 con 0x5611674b18f0 2024-04-24T23:23:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.649+0000 7f83ca00a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5611674fb490 con 0x561167482d60 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83ca00a740 10 client.0 _ll_register_callbacks cb 0x7f83b4007790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c4897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167456780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c4897640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167456780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55110/0 (socket says 172.21.15.26:55110) 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c4897640 1 -- 172.21.15.26:0/3159312225 learned_addr learned my addr 172.21.15.26:0/3159312225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c5098640 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167482d60 0x5611674faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c4897640 1 -- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167482d60 msgr2=0x5611674faf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c4897640 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167482d60 0x5611674faf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c5899640 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674b18f0 0x561167493da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c4897640 1 -- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674b18f0 msgr2=0x561167493da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c4897640 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674b18f0 0x561167493da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c4897640 1 -- 172.21.15.26:0/3159312225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611674861e0 con 0x5611674a0600 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83c4897640 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167456780 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f83b4054ab0 tx=0x7f83b4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729671ac9ba9df25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f839ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:52.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f839ffff640 1 -- 172.21.15.26:0/3159312225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b406cae0 con 0x5611674a0600 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f839ffff640 1 -- 172.21.15.26:0/3159312225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f83b406cc80 con 0x5611674a0600 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83ca00a740 1 -- 172.21.15.26:0/3159312225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b4057b40 con 0x5611674a0600 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f839ffff640 1 -- 172.21.15.26:0/3159312225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f83b407e050 con 0x5611674a0600 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83ca00a740 10 client.6255 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f83ca00a740 1 -- 172.21.15.26:0/3159312225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x5611674861e0 con 0x5611674a0600 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.650+0000 7f839ffff640 1 -- 172.21.15.26:0/3159312225 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4057b40 con 0x5611674a0600 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f839ffff640 1 -- 172.21.15.26:0/3159312225 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7f83b4081020 con 0x5611674a0600 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f83ca00a740 10 client.6255 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f83ca00a740 10 client.6255 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f83ca00a740 20 client.6255 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x561167474d40 0x56116750d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f83ca00a740 4 client.6255 mds_command: new command op to 5081 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:23:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f83ca00a740 1 -- 172.21.15.26:0/3159312225 --> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5611673c3f40 con 0x561167474d40 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f83c5098640 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x561167474d40 0x56116750d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f839ffff640 10 client.6255 ms_handle_connect on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f83c5098640 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x561167474d40 0x56116750d8a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51eb753b2b0ff8a server_cookie=7570f324bf13f868 in_seq=0 out_seq=0 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f839ffff640 1 -- 172.21.15.26:0/3159312225 <== mds.2 v2:172.21.15.29:6832/1578186344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5611673c3f40 con 0x561167474d40 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f839ffff640 10 client.6255 handle_command_reply: tid=0 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.651+0000 7f83ca00a740 1 client.6255 shutdown 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 -- 172.21.15.26:0/3159312225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 msgr2=0x561167456780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167456780 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f83b4054ab0 tx=0x7f83b4000ed0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 -- 172.21.15.26:0/3159312225 shutdown_connections 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:6832/1578186344,v1:172.21.15.29:6835/1578186344] conn(0x561167474d40 0x56116750d8a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611674b18f0 0x561167493da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611674a0600 0x561167456780 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 --2- 172.21.15.26:0/3159312225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561167482d60 0x5611674faf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f839ffff640 0 client.6255 ms_handle_reset on v2:172.21.15.29:6832/1578186344 2024-04-24T23:23:52.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 -- 172.21.15.26:0/3159312225 >> 172.21.15.26:0/3159312225 conn(0x561167458a40 msgr2=0x5611674c1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 -- 172.21.15.26:0/3159312225 shutdown_connections 2024-04-24T23:23:52.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 1 -- 172.21.15.26:0/3159312225 wait complete. 2024-04-24T23:23:52.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83ca00a740 20 client.6255 trim_cache size 0 max 16384 2024-04-24T23:23:52.660 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:52.660 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-04-24T23:23:52.660 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-04-24T23:23:52.660 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:52.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 -- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83ac06be30 msgr2=0x7f83ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83ac06be30 0x7f83ac06e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f83b0034000 tx=0x7f83b002e000 comp rx=0 tx=0).stop 2024-04-24T23:23:52.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 -- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c00d0750 msgr2=0x7f83c0070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c00d0750 0x7f83c0070e10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c400 tx=0x7f83b400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 -- 172.21.15.26:0/2377341186 shutdown_connections 2024-04-24T23:23:52.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f83ac06be30 0x7f83ac06e2f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f83c015be40 0x7f83c00ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83c00d0750 0x7f83c0070e10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 --2- 172.21.15.26:0/2377341186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83c00b4ce0 0x7f83c009d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 -- 172.21.15.26:0/2377341186 >> 172.21.15.26:0/2377341186 conn(0x7f83c006b540 msgr2=0x7f83c00b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 -- 172.21.15.26:0/2377341186 shutdown_connections 2024-04-24T23:23:52.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.652+0000 7f83c72fc640 1 -- 172.21.15.26:0/2377341186 wait complete. 2024-04-24T23:23:52.664 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:23:52.664 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:23:52.665 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-24T23:23:52.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.766+0000 7f8d813b7640 1 Processor -- start 2024-04-24T23:23:52.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.766+0000 7f8d813b7640 1 -- start start 2024-04-24T23:23:52.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d813b7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c15e750 0x7f8d7c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d813b7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d7c0d2c30 0x7f8d7c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d813b7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d7c0d2120 con 0x7f8d7c15e370 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d813b7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8d7c0d1510 con 0x7f8d7c0d2c30 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d813b7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c0d1ed0 con 0x7f8d7c15e750 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d7b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d7c0d2c30 0x7f8d7c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d7b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d7c0d2c30 0x7f8d7c0cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55118/0 (socket says 172.21.15.26:55118) 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d7b7fe640 1 -- 172.21.15.26:0/206532541 learned_addr learned my addr 172.21.15.26:0/206532541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/206532541 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c15e750 0x7f8d7c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d7b7fe640 1 -- 172.21.15.26:0/206532541 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c15e750 msgr2=0x7f8d7c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d7b7fe640 1 --2- 172.21.15.26:0/206532541 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c15e750 0x7f8d7c0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d7b7fe640 1 -- 172.21.15.26:0/206532541 >> v1:172.21.15.29:6789/0 conn(0x7f8d7c15e370 legacy=0x7f8d7c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.767+0000 7f8d7b7fe640 1 -- 172.21.15.26:0/206532541 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c0b3740 con 0x7f8d7c0d2c30 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.768+0000 7f8d7b7fe640 1 --2- 172.21.15.26:0/206532541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c0d2c30 0x7f8d7c0cf1d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0d2050 tx=0x7f8d7402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa8a1aaf17babbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.768+0000 7f8d79ffb640 1 -- 172.21.15.26:0/206532541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74036070 con 0x7f8d7c0d2c30 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.768+0000 7f8d79ffb640 1 -- 172.21.15.26:0/206532541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d74031600 con 0x7f8d7c0d2c30 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.768+0000 7f8d79ffb640 1 -- 172.21.15.26:0/206532541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740319d0 con 0x7f8d7c0d2c30 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.769+0000 7f8d813b7640 1 -- 172.21.15.26:0/206532541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c0d2c30 msgr2=0x7f8d7c0cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.769+0000 7f8d813b7640 1 --2- 172.21.15.26:0/206532541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c0d2c30 0x7f8d7c0cf1d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0d2050 tx=0x7f8d7402a6c0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.769+0000 7f8d813b7640 1 -- 172.21.15.26:0/206532541 shutdown_connections 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.769+0000 7f8d813b7640 1 --2- 172.21.15.26:0/206532541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c0d2c30 0x7f8d7c0cf1d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.769+0000 7f8d813b7640 1 --2- 172.21.15.26:0/206532541 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c15e750 0x7f8d7c0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.769+0000 7f8d813b7640 1 -- 172.21.15.26:0/206532541 >> 172.21.15.26:0/206532541 conn(0x7f8d7c0d40f0 msgr2=0x7f8d7c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.769+0000 7f8d813b7640 1 -- 172.21.15.26:0/206532541 shutdown_connections 2024-04-24T23:23:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.769+0000 7f8d813b7640 1 -- 172.21.15.26:0/206532541 wait complete. 2024-04-24T23:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.770+0000 7f8d813b7640 1 Processor -- start 2024-04-24T23:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.770+0000 7f8d813b7640 1 -- start start 2024-04-24T23:23:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d813b7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c0d2c30 0x7f8d7c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d813b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c15e370 0x7f8d7c0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d813b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d7c0c8160 0x7f8d7c1f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d813b7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c0b5230 con 0x7f8d7c15e370 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d813b7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c0b3d00 con 0x7f8d7c0c8160 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d813b7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c0b53b0 con 0x7f8d7c0d2c30 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c15e370 0x7f8d7c0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c15e370 0x7f8d7c0c7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55122/0 (socket says 172.21.15.26:55122) 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/3548570660 learned_addr learned my addr 172.21.15.26:0/3548570660 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c0d2c30 0x7f8d7c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7b7fe640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d7c0c8160 0x7f8d7c1f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/3548570660 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c0d2c30 msgr2=0x7f8d7c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c0d2c30 0x7f8d7c0cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/3548570660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d7c0c8160 msgr2=0x7f8d7c1f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d7c0c8160 0x7f8d7c1f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/3548570660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c0caac0 con 0x7f8d7c15e370 2024-04-24T23:23:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c0d2c30 0x7f8d7c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c15e370 0x7f8d7c0c7c20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d5f0 tx=0x7f8d7000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.771+0000 7f8d637fe640 1 -- 172.21.15.26:0/3548570660 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700125f0 con 0x7f8d7c15e370 2024-04-24T23:23:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.772+0000 7f8d637fe640 1 -- 172.21.15.26:0/3548570660 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d70019070 con 0x7f8d7c15e370 2024-04-24T23:23:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.772+0000 7f8d813b7640 1 -- 172.21.15.26:0/3548570660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c0c8800 con 0x7f8d7c15e370 2024-04-24T23:23:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.772+0000 7f8d637fe640 1 -- 172.21.15.26:0/3548570660 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700129d0 con 0x7f8d7c15e370 2024-04-24T23:23:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.772+0000 7f8d813b7640 1 -- 172.21.15.26:0/3548570660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c0c8dc0 con 0x7f8d7c15e370 2024-04-24T23:23:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.773+0000 7f8d637fe640 1 -- 172.21.15.26:0/3548570660 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d70012b70 con 0x7f8d7c15e370 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.773+0000 7f8d637fe640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8d6806be30 0x7f8d6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.773+0000 7f8d637fe640 1 -- 172.21.15.26:0/3548570660 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f8d7008b250 con 0x7f8d7c15e370 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.773+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8d6806be30 0x7f8d6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.774+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8d6806be30 0x7f8d6806e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0d0480 tx=0x7f8d6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.774+0000 7f8d841a4740 1 Processor -- start 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.774+0000 7f8d841a4740 1 -- start start 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097afb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097b18600 0x556097b18b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097ad0a40 0x556097b16cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d841a4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556097a346b0 con 0x556097afad60 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d841a4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5560979dc880 con 0x556097ad0a40 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d841a4740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556097a30670 con 0x556097b18600 2024-04-24T23:23:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d7affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097afb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d7affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097afb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55130/0 (socket says 172.21.15.26:55130) 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d7affd640 1 -- 172.21.15.26:0/3021022862 learned_addr learned my addr 172.21.15.26:0/3021022862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/3021022862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097b18600 0x556097b18b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.775+0000 7f8d7b7fe640 1 --2- 172.21.15.26:0/3021022862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097ad0a40 0x556097b16cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.776+0000 7f8d7affd640 1 -- 172.21.15.26:0/3021022862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097b18600 msgr2=0x556097b18b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.776+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3021022862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097b18600 0x556097b18b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.776+0000 7f8d7affd640 1 -- 172.21.15.26:0/3021022862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097ad0a40 msgr2=0x556097b16cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.776+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3021022862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097ad0a40 0x556097b16cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.776+0000 7f8d7affd640 1 -- 172.21.15.26:0/3021022862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560978127d0 con 0x556097afad60 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.776+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3021022862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097afb140 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005420 tx=0x7f8d6c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46668dab5424c33d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.776+0000 7f8d60ff9640 1 -- 172.21.15.26:0/3021022862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c024070 con 0x556097afad60 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.776+0000 7f8d60ff9640 1 -- 172.21.15.26:0/3021022862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c014950 con 0x556097afad60 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.776+0000 7f8d60ff9640 1 -- 172.21.15.26:0/3021022862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c014d00 con 0x556097afad60 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 -- 172.21.15.26:0/3021022862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 msgr2=0x556097afb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3021022862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097afb140 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005420 tx=0x7f8d6c010000 comp rx=0 tx=0).stop 2024-04-24T23:23:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 -- 172.21.15.26:0/3021022862 shutdown_connections 2024-04-24T23:23:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3021022862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097ad0a40 0x556097b16cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3021022862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097b18600 0x556097b18b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3021022862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097afb140 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 -- 172.21.15.26:0/3021022862 >> 172.21.15.26:0/3021022862 conn(0x556097acd600 msgr2=0x556097b15b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 -- 172.21.15.26:0/3021022862 shutdown_connections 2024-04-24T23:23:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 -- 172.21.15.26:0/3021022862 wait complete. 2024-04-24T23:23:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 -- start start 2024-04-24T23:23:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.777+0000 7f8d841a4740 1 Processor -- start 2024-04-24T23:23:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 0x556097afe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b18600 0x556097b0cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b25640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d841a4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556097ad17e0 con 0x556097b0d1d0 2024-04-24T23:23:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d841a4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5560979eff50 con 0x556097b18600 2024-04-24T23:23:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d841a4740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556097acb530 con 0x556097afad60 2024-04-24T23:23:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d841a4740 10 client.0 _ll_register_callbacks cb 0x556097a2b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d7b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b25640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d7b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b25640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55138/0 (socket says 172.21.15.26:55138) 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d7b7fe640 1 -- 172.21.15.26:0/3197304144 learned_addr learned my addr 172.21.15.26:0/3197304144 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 0x556097afe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.778+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b18600 0x556097b0cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d7b7fe640 1 -- 172.21.15.26:0/3197304144 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 msgr2=0x556097afe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d7b7fe640 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 0x556097afe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d7b7fe640 1 -- 172.21.15.26:0/3197304144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b18600 msgr2=0x556097b0cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d7b7fe640 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b18600 0x556097b0cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d7b7fe640 1 -- 172.21.15.26:0/3197304144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560977483f0 con 0x556097b0d1d0 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d7b7fe640 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b25640 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009f40 tx=0x7f8d74004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1879597dabdfb32e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d59ffb640 1 -- 172.21.15.26:0/3197304144 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74042070 con 0x556097b0d1d0 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d59ffb640 1 -- 172.21.15.26:0/3197304144 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f8d74031d10 con 0x556097b0d1d0 2024-04-24T23:23:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d841a4740 1 -- 172.21.15.26:0/3197304144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d6c014950 con 0x556097b0d1d0 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d59ffb640 1 -- 172.21.15.26:0/3197304144 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8d74036040 con 0x556097b0d1d0 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d841a4740 10 client.6267 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d841a4740 1 -- 172.21.15.26:0/3197304144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x5560977080f0 con 0x556097b0d1d0 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d59ffb640 1 -- 172.21.15.26:0/3197304144 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c014950 con 0x556097b0d1d0 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d59ffb640 1 -- 172.21.15.26:0/3197304144 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7f8d7404f080 con 0x556097b0d1d0 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d841a4740 10 client.6267 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d841a4740 10 client.6267 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d841a4740 20 client.6267 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556097b29c70 0x556097b2a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d841a4740 4 client.6267 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.779+0000 7f8d841a4740 1 -- 172.21.15.26:0/3197304144 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556097a41bf0 con 0x556097b29c70 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.780+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556097b29c70 0x556097b2a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.780+0000 7f8d59ffb640 10 client.6267 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.780+0000 7f8d7affd640 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556097b29c70 0x556097b2a080 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223c354834abe53a server_cookie=b530753c2c025e7 in_seq=0 out_seq=0 2024-04-24T23:23:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.780+0000 7f8d59ffb640 1 -- 172.21.15.26:0/3197304144 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556097a41bf0 con 0x556097b29c70 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.780+0000 7f8d59ffb640 10 client.6267 handle_command_reply: tid=0 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.780+0000 7f8d841a4740 1 client.6267 shutdown 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 -- 172.21.15.26:0/3197304144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 msgr2=0x556097b25640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b25640 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009f40 tx=0x7f8d74004ab0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 -- 172.21.15.26:0/3197304144 shutdown_connections 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556097b29c70 0x556097b2a080 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b25640 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b18600 0x556097b0cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d59ffb640 0 client.6267 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 --2- 172.21.15.26:0/3197304144 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 0x556097afe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 -- 172.21.15.26:0/3197304144 >> 172.21.15.26:0/3197304144 conn(0x556097ad0a40 msgr2=0x556097b190f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 -- 172.21.15.26:0/3197304144 shutdown_connections 2024-04-24T23:23:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 1 -- 172.21.15.26:0/3197304144 wait complete. 2024-04-24T23:23:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.781+0000 7f8d841a4740 20 client.6267 trim_cache size 0 max 16384 2024-04-24T23:23:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.798+0000 7f8d841a4740 1 Processor -- start 2024-04-24T23:23:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.798+0000 7f8d841a4740 1 -- start start 2024-04-24T23:23:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097b15450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b0d1d0 0x556097b19190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097b18600 0x556097aca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d841a4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c200940 con 0x556097afad60 2024-04-24T23:23:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d841a4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1ffae0 con 0x556097b0d1d0 2024-04-24T23:23:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d841a4740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c200210 con 0x556097b18600 2024-04-24T23:23:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d7affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097b15450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d7affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097b15450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55146/0 (socket says 172.21.15.26:55146) 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d7affd640 1 -- 172.21.15.26:0/1087956378 learned_addr learned my addr 172.21.15.26:0/1087956378 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d7affd640 1 -- 172.21.15.26:0/1087956378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097b18600 msgr2=0x556097aca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d7affd640 1 --2- 172.21.15.26:0/1087956378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097b18600 0x556097aca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d7affd640 1 -- 172.21.15.26:0/1087956378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b0d1d0 msgr2=0x556097b19190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d7affd640 1 --2- 172.21.15.26:0/1087956378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b0d1d0 0x556097b19190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.799+0000 7f8d7affd640 1 -- 172.21.15.26:0/1087956378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556097ac72a0 con 0x556097afad60 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d7affd640 1 --2- 172.21.15.26:0/1087956378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097b15450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005620 tx=0x7f8d6c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9f132616cbc0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d5bfff640 1 -- 172.21.15.26:0/1087956378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c036070 con 0x556097afad60 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d5bfff640 1 -- 172.21.15.26:0/1087956378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c0149d0 con 0x556097afad60 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d5bfff640 1 -- 172.21.15.26:0/1087956378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c024040 con 0x556097afad60 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d841a4740 1 -- 172.21.15.26:0/1087956378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 msgr2=0x556097b15450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d841a4740 1 --2- 172.21.15.26:0/1087956378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097b15450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005620 tx=0x7f8d6c012000 comp rx=0 tx=0).stop 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d841a4740 1 -- 172.21.15.26:0/1087956378 shutdown_connections 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d841a4740 1 --2- 172.21.15.26:0/1087956378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097b18600 0x556097aca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d841a4740 1 --2- 172.21.15.26:0/1087956378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b0d1d0 0x556097b19190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d841a4740 1 --2- 172.21.15.26:0/1087956378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097afad60 0x556097b15450 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d841a4740 1 -- 172.21.15.26:0/1087956378 >> 172.21.15.26:0/1087956378 conn(0x556097ad0a40 msgr2=0x556097b39ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d841a4740 1 -- 172.21.15.26:0/1087956378 shutdown_connections 2024-04-24T23:23:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.800+0000 7f8d841a4740 1 -- 172.21.15.26:0/1087956378 wait complete. 2024-04-24T23:23:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.801+0000 7f8d841a4740 1 -- start start 2024-04-24T23:23:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.801+0000 7f8d841a4740 1 Processor -- start 2024-04-24T23:23:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 0x556097b17670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b19190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d841a4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b18600 0x556097b196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d841a4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556097b15c90 con 0x556097b0d1d0 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d841a4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1f4a40 con 0x556097b18600 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d841a4740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556097a01ec0 con 0x556097afad60 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d841a4740 10 client.0 _ll_register_callbacks cb 0x7f8d74034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b19190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b19190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55156/0 (socket says 172.21.15.26:55156) 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/4196949388 learned_addr learned my addr 172.21.15.26:0/4196949388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7affd640 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 0x556097b17670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/4196949388 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 msgr2=0x556097b17670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 0x556097b17670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/4196949388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b18600 msgr2=0x556097b196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b18600 0x556097b196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7a7fc640 1 -- 172.21.15.26:0/4196949388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556097b0bfc0 con 0x556097b0d1d0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d7a7fc640 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b19190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d5c0 tx=0x7f8d70059c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd3785269b4683d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d59ffb640 1 -- 172.21.15.26:0/4196949388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700620a0 con 0x556097b0d1d0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d59ffb640 1 -- 172.21.15.26:0/4196949388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f8d70089340 con 0x556097b0d1d0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d841a4740 1 -- 172.21.15.26:0/4196949388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d6c0149d0 con 0x556097b0d1d0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d59ffb640 1 -- 172.21.15.26:0/4196949388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8d70078050 con 0x556097b0d1d0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.802+0000 7f8d841a4740 10 client.6273 fetch_fsmap learned FSMap version 42 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d841a4740 1 -- 172.21.15.26:0/4196949388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x556097ad0040 con 0x556097b0d1d0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d59ffb640 1 -- 172.21.15.26:0/4196949388 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c0149d0 con 0x556097b0d1d0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d59ffb640 1 -- 172.21.15.26:0/4196949388 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 42) ==== 3550+0+0 (secure 0 0 0) 0x7f8d70089020 con 0x556097b0d1d0 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d841a4740 10 client.6273 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T23:23:52.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d841a4740 10 client.6273 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d841a4740 20 client.6273 populate_metadata read hostname 'smithi026' 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d841a4740 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556097ad0560 0x556097b28230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d841a4740 4 client.6273 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d841a4740 1 -- 172.21.15.26:0/4196949388 --> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556097a3bf40 con 0x556097ad0560 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d7affd640 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556097ad0560 0x556097b28230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d59ffb640 10 client.6273 ms_handle_connect on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d7affd640 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556097ad0560 0x556097b28230 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845d428afc04b07f server_cookie=a7e82bad5778e06d in_seq=0 out_seq=0 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d59ffb640 1 -- 172.21.15.26:0/4196949388 <== mds.0 v2:172.21.15.26:6834/730964888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556097a3bf40 con 0x556097ad0560 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d59ffb640 10 client.6273 handle_command_reply: tid=0 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.803+0000 7f8d841a4740 1 client.6273 shutdown 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 -- 172.21.15.26:0/4196949388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 msgr2=0x556097b19190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b19190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d5c0 tx=0x7f8d70059c50 comp rx=0 tx=0).stop 2024-04-24T23:23:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 -- 172.21.15.26:0/4196949388 shutdown_connections 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.26:6834/730964888,v1:172.21.15.26:6836/730964888] conn(0x556097ad0560 0x556097b28230 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556097b18600 0x556097b196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556097b0d1d0 0x556097b19190 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d59ffb640 0 client.6273 ms_handle_reset on v2:172.21.15.26:6834/730964888 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 --2- 172.21.15.26:0/4196949388 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556097afad60 0x556097b17670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 -- 172.21.15.26:0/4196949388 >> 172.21.15.26:0/4196949388 conn(0x556097ad0a40 msgr2=0x556097b39ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 -- 172.21.15.26:0/4196949388 shutdown_connections 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 1 -- 172.21.15.26:0/4196949388 wait complete. 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d841a4740 20 client.6273 trim_cache size 0 max 16384 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:23:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-04-24T23:23:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-04-24T23:23:52.812 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:23:52.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 -- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8d6806be30 msgr2=0x7f8d6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8d6806be30 0x7f8d6806e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0d0480 tx=0x7f8d6c009040 comp rx=0 tx=0).stop 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 -- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c15e370 msgr2=0x7f8d7c0c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c15e370 0x7f8d7c0c7c20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d5f0 tx=0x7f8d7000dab0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 -- 172.21.15.26:0/3548570660 shutdown_connections 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8d6806be30 0x7f8d6806e2f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8d7c0c8160 0x7f8d7c1f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d7c15e370 0x7f8d7c0c7c20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 --2- 172.21.15.26:0/3548570660 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8d7c0d2c30 0x7f8d7c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 -- 172.21.15.26:0/3548570660 >> 172.21.15.26:0/3548570660 conn(0x7f8d7c0d40f0 msgr2=0x7f8d7c0d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 -- 172.21.15.26:0/3548570660 shutdown_connections 2024-04-24T23:23:52.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.804+0000 7f8d813b7640 1 -- 172.21.15.26:0/3548570660 wait complete. 2024-04-24T23:23:52.816 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:23:52.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.916+0000 7fbff8c72640 1 Processor -- start 2024-04-24T23:23:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.916+0000 7fbff8c72640 1 -- start start 2024-04-24T23:23:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbff8c72640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbff40cdd70 0x7fbff406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbff8c72640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbff40cfbc0 con 0x7fbff406d350 2024-04-24T23:23:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbff8c72640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbff406d7f0 con 0x7fbff4145910 2024-04-24T23:23:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbff8c72640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbff4145cf0 con 0x7fbff40cdd70 2024-04-24T23:23:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbff2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbff40cdd70 0x7fbff406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbfebfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbff406d350 0x7fbff40cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54194/0 (socket says 172.21.15.26:54194) 2024-04-24T23:23:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbfebfff640 1 -- 172.21.15.26:0/2507283549 learned_addr learned my addr 172.21.15.26:0/2507283549 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbff2d9d640 1 -- 172.21.15.26:0/2507283549 >> v1:172.21.15.29:6789/0 conn(0x7fbff4145910 legacy=0x7fbff415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbff2d9d640 1 -- 172.21.15.26:0/2507283549 >> v1:172.21.15.26:6789/0 conn(0x7fbff406d350 legacy=0x7fbff40cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbff2d9d640 1 -- 172.21.15.26:0/2507283549 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff406dad0 con 0x7fbff40cdd70 2024-04-24T23:23:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.917+0000 7fbff1d9b640 1 -- 172.21.15.26:0/2507283549 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2853730316 0 0) 0x7fbff40cfbc0 con 0x7fbff406d350 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.918+0000 7fbff2d9d640 1 --2- 172.21.15.26:0/2507283549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 0x7fbff406e890 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009dc0 tx=0x7fbfec02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21db7a8f99298202 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.918+0000 7fbff1d9b640 1 -- 172.21.15.26:0/2507283549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec007740 con 0x7fbff40cdd70 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.918+0000 7fbff1d9b640 1 -- 172.21.15.26:0/2507283549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec0078e0 con 0x7fbff40cdd70 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.918+0000 7fbff1d9b640 1 -- 172.21.15.26:0/2507283549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec007cb0 con 0x7fbff40cdd70 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.919+0000 7fbff8c72640 1 -- 172.21.15.26:0/2507283549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 msgr2=0x7fbff406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.919+0000 7fbff8c72640 1 --2- 172.21.15.26:0/2507283549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 0x7fbff406e890 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009dc0 tx=0x7fbfec02a640 comp rx=0 tx=0).stop 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.919+0000 7fbff8c72640 1 -- 172.21.15.26:0/2507283549 shutdown_connections 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.919+0000 7fbff8c72640 1 --2- 172.21.15.26:0/2507283549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 0x7fbff406e890 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.919+0000 7fbff8c72640 1 -- 172.21.15.26:0/2507283549 >> 172.21.15.26:0/2507283549 conn(0x7fbff406b300 msgr2=0x7fbff406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.919+0000 7fbff8c72640 1 -- 172.21.15.26:0/2507283549 shutdown_connections 2024-04-24T23:23:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.919+0000 7fbff8c72640 1 -- 172.21.15.26:0/2507283549 wait complete. 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.920+0000 7fbff8c72640 1 Processor -- start 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.920+0000 7fbff8c72640 1 -- start start 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbff8c72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff406d350 0x7fbff413b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbff8c72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 0x7fbff40b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbff8c72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbff4145910 0x7fbff4158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbff8c72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbff406e370 con 0x7fbff40cdd70 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbff8c72640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbff406e090 con 0x7fbff406d350 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbff8c72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbff415d770 con 0x7fbff4145910 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbfebfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 0x7fbff40b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbfebfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 0x7fbff40b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55176/0 (socket says 172.21.15.26:55176) 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbfebfff640 1 -- 172.21.15.26:0/3408516393 learned_addr learned my addr 172.21.15.26:0/3408516393 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbff259c640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff406d350 0x7fbff413b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbff2d9d640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbff4145910 0x7fbff4158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbfebfff640 1 -- 172.21.15.26:0/3408516393 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbff4145910 msgr2=0x7fbff4158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbfebfff640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbff4145910 0x7fbff4158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbfebfff640 1 -- 172.21.15.26:0/3408516393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff406d350 msgr2=0x7fbff413b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbfebfff640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff406d350 0x7fbff413b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbfebfff640 1 -- 172.21.15.26:0/3408516393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff4159010 con 0x7fbff40cdd70 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.921+0000 7fbfebfff640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 0x7fbff40b2f80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00c4d0 tx=0x7fbfdc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.922+0000 7fbfeaffd640 1 -- 172.21.15.26:0/3408516393 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc018070 con 0x7fbff40cdd70 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.922+0000 7fbfeaffd640 1 -- 172.21.15.26:0/3408516393 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfdc007500 con 0x7fbff40cdd70 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.922+0000 7fbfeaffd640 1 -- 172.21.15.26:0/3408516393 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc004a20 con 0x7fbff40cdd70 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.922+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff40b4750 con 0x7fbff40cdd70 2024-04-24T23:23:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.922+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff40b3990 con 0x7fbff40cdd70 2024-04-24T23:23:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.923+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff40694c0 con 0x7fbff40cdd70 2024-04-24T23:23:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.923+0000 7fbfeaffd640 1 -- 172.21.15.26:0/3408516393 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbfdc01b070 con 0x7fbff40cdd70 2024-04-24T23:23:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.924+0000 7fbfeaffd640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfd406be30 0x7fbfd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.924+0000 7fbfeaffd640 1 -- 172.21.15.26:0/3408516393 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fbfdc08a1a0 con 0x7fbff40cdd70 2024-04-24T23:23:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.924+0000 7fbff259c640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfd406be30 0x7fbfd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.924+0000 7fbff259c640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfd406be30 0x7fbfd406e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4006fd0 tx=0x7fbfe4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:52.928+0000 7fbfeaffd640 1 -- 172.21.15.26:0/3408516393 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbfdc058b10 con 0x7fbff40cdd70 2024-04-24T23:23:53.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.149+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbff40726c0 con 0x7fbff40cdd70 2024-04-24T23:23:53.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.149+0000 7fbfeaffd640 1 -- 172.21.15.26:0/3408516393 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5222 (secure 0 0 0) 0x7fbfdc058410 con 0x7fbff40cdd70 2024-04-24T23:23:53.151 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:53.151 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:40.866214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5742,"mds_1":5329,"mds_2":5081},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5081":{"gid":5081,"name":"b","rank":2,"incarnation":36,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742,5329,5081]},"id":3}]} 2024-04-24T23:23:53.151 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 42 2024-04-24T23:23:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfd406be30 msgr2=0x7fbfd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfd406be30 0x7fbfd406e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4006fd0 tx=0x7fbfe4008040 comp rx=0 tx=0).stop 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 msgr2=0x7fbff40b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 0x7fbff40b2f80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00c4d0 tx=0x7fbfdc00c990 comp rx=0 tx=0).stop 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 shutdown_connections 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfd406be30 0x7fbfd406e2f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbff4145910 0x7fbff4158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbff40cdd70 0x7fbff40b2f80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 --2- 172.21.15.26:0/3408516393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbff406d350 0x7fbff413b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 >> 172.21.15.26:0/3408516393 conn(0x7fbff406b300 msgr2=0x7fbff406b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 shutdown_connections 2024-04-24T23:23:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.151+0000 7fbff8c72640 1 -- 172.21.15.26:0/3408516393 wait complete. 2024-04-24T23:23:53.164 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'd', 'b'] 2024-04-24T23:23:53.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:23:53.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.263+0000 7f31e39c1640 1 Processor -- start 2024-04-24T23:23:53.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.263+0000 7f31e39c1640 1 -- start start 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e39c1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31dc15be90 0x7f31dc15f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e39c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 0x7f31dc06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e39c1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f31dc162500 0x7f31dc162960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e39c1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f31dc15f850 con 0x7f31dc15be90 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e39c1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f31dc162000 con 0x7f31dc162500 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e39c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f31dc15c270 con 0x7f31dc06d760 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e175d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31dc15be90 0x7f31dc15f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e175d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31dc15be90 0x7f31dc15f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55188/0 (socket says 172.21.15.26:55188) 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e175d640 1 -- 172.21.15.26:0/1729873807 learned_addr learned my addr 172.21.15.26:0/1729873807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e0f5c640 1 --2- 172.21.15.26:0/1729873807 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 0x7f31dc06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.264+0000 7f31e1f5e640 1 --2- 172.21.15.26:0/1729873807 >> v2:172.21.15.29:3300/0 conn(0x7f31dc162500 0x7f31dc162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.265+0000 7f31e175d640 1 -- 172.21.15.26:0/1729873807 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 msgr2=0x7f31dc06dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.265+0000 7f31e175d640 1 --2- 172.21.15.26:0/1729873807 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 0x7f31dc06dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.265+0000 7f31e175d640 1 -- 172.21.15.26:0/1729873807 >> v2:172.21.15.29:3300/0 conn(0x7f31dc162500 msgr2=0x7f31dc162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.265+0000 7f31e175d640 1 --2- 172.21.15.26:0/1729873807 >> v2:172.21.15.29:3300/0 conn(0x7f31dc162500 0x7f31dc162960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.265+0000 7f31e175d640 1 -- 172.21.15.26:0/1729873807 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc071a70 con 0x7f31dc15be90 2024-04-24T23:23:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.265+0000 7f31e1f5e640 1 --2- 172.21.15.26:0/1729873807 >> v2:172.21.15.29:3300/0 conn(0x7f31dc162500 0x7f31dc162960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.265+0000 7f31e175d640 1 --2- 172.21.15.26:0/1729873807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 0x7f31dc15f310 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0ccdf0 tx=0x7f31d402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9fe3ce41cca05c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.265+0000 7f31d3fff640 1 -- 172.21.15.26:0/1729873807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4037070 con 0x7f31dc15be90 2024-04-24T23:23:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31d3fff640 1 -- 172.21.15.26:0/1729873807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d4032440 con 0x7f31dc15be90 2024-04-24T23:23:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31d3fff640 1 -- 172.21.15.26:0/1729873807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4032810 con 0x7f31dc15be90 2024-04-24T23:23:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31e39c1640 1 -- 172.21.15.26:0/1729873807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 msgr2=0x7f31dc15f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31e39c1640 1 --2- 172.21.15.26:0/1729873807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 0x7f31dc15f310 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0ccdf0 tx=0x7f31d402a140 comp rx=0 tx=0).stop 2024-04-24T23:23:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31e39c1640 1 -- 172.21.15.26:0/1729873807 shutdown_connections 2024-04-24T23:23:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31e39c1640 1 --2- 172.21.15.26:0/1729873807 >> v2:172.21.15.29:3300/0 conn(0x7f31dc162500 0x7f31dc162960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31e39c1640 1 --2- 172.21.15.26:0/1729873807 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 0x7f31dc06dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31e39c1640 1 --2- 172.21.15.26:0/1729873807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 0x7f31dc15f310 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31e39c1640 1 -- 172.21.15.26:0/1729873807 >> 172.21.15.26:0/1729873807 conn(0x7f31dc0ccff0 msgr2=0x7f31dc0cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31e39c1640 1 -- 172.21.15.26:0/1729873807 shutdown_connections 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.266+0000 7f31e39c1640 1 -- 172.21.15.26:0/1729873807 wait complete. 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.267+0000 7f31e39c1640 1 Processor -- start 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.267+0000 7f31e39c1640 1 -- start start 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e39c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 0x7f31dc155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e39c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 0x7f31dc157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e39c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31dc162500 0x7f31dc0cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e39c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc071740 con 0x7f31dc15be90 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e39c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc072030 con 0x7f31dc162500 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e39c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f31dc0b3650 con 0x7f31dc06d760 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e0f5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 0x7f31dc157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e0f5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 0x7f31dc157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55198/0 (socket says 172.21.15.26:55198) 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e0f5c640 1 -- 172.21.15.26:0/3686175484 learned_addr learned my addr 172.21.15.26:0/3686175484 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e175d640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 0x7f31dc155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e0f5c640 1 -- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 msgr2=0x7f31dc155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e1f5e640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31dc162500 0x7f31dc0cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.268+0000 7f31e0f5c640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 0x7f31dc155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31e0f5c640 1 -- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31dc162500 msgr2=0x7f31dc0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31e0f5c640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31dc162500 0x7f31dc0cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31e0f5c640 1 -- 172.21.15.26:0/3686175484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc155ea0 con 0x7f31dc15be90 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31e175d640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 0x7f31dc155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31e1f5e640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31dc162500 0x7f31dc0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31e0f5c640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 0x7f31dc157480 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00d800 tx=0x7f31cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31d1ffb640 1 -- 172.21.15.26:0/3686175484 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc018030 con 0x7f31dc15be90 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31d1ffb640 1 -- 172.21.15.26:0/3686175484 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31cc0096a0 con 0x7f31dc15be90 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31d1ffb640 1 -- 172.21.15.26:0/3686175484 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc009870 con 0x7f31dc15be90 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc156180 con 0x7f31dc15be90 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.269+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc156740 con 0x7f31dc15be90 2024-04-24T23:23:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.270+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc0694c0 con 0x7f31dc15be90 2024-04-24T23:23:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.270+0000 7f31d1ffb640 1 -- 172.21.15.26:0/3686175484 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31cc01b070 con 0x7f31dc15be90 2024-04-24T23:23:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.271+0000 7f31d1ffb640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f31c006be30 0x7f31c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.271+0000 7f31d1ffb640 1 -- 172.21.15.26:0/3686175484 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f31cc08b1a0 con 0x7f31dc15be90 2024-04-24T23:23:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.271+0000 7f31e175d640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f31c006be30 0x7f31c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.271+0000 7f31e175d640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f31c006be30 0x7f31c006e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f31d4034000 tx=0x7f31d402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:53.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.275+0000 7f31d1ffb640 1 -- 172.21.15.26:0/3686175484 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31cc059b00 con 0x7f31dc15be90 2024-04-24T23:23:53.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.526+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f31dc0728a0 con 0x7f31dc15be90 2024-04-24T23:23:53.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.527+0000 7f31d1ffb640 1 -- 172.21.15.26:0/3686175484 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1870 (secure 0 0 0) 0x7f31cc059400 con 0x7f31dc15be90 2024-04-24T23:23:53.528 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:53.528 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":202,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706268,"bytes_used":265592832,"bytes_avail":772828520448,"bytes_total":773094113280,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":42,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"b","status":"up:active","gid":5081},{"filesystem_id":3,"rank":1,"name":"d","status":"up:active","gid":5329},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5742}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-24T23:23:51.353931+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:23:53.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f31c006be30 msgr2=0x7f31c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:53.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f31c006be30 0x7f31c006e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f31d4034000 tx=0x7f31d402e000 comp rx=0 tx=0).stop 2024-04-24T23:23:53.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 msgr2=0x7f31dc157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 0x7f31dc157480 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00d800 tx=0x7f31cc00dcc0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 shutdown_connections 2024-04-24T23:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f31c006be30 0x7f31c006e2f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31dc162500 0x7f31dc0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31dc15be90 0x7f31dc157480 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 --2- 172.21.15.26:0/3686175484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31dc06d760 0x7f31dc155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 >> 172.21.15.26:0/3686175484 conn(0x7f31dc0ccff0 msgr2=0x7f31dc15d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 shutdown_connections 2024-04-24T23:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:53.529+0000 7f31e39c1640 1 -- 172.21.15.26:0/3686175484 wait complete. 2024-04-24T23:23:53.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]'} 2024-04-24T23:23:54.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.654+0000 7f422f730640 1 Processor -- start 2024-04-24T23:23:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.654+0000 7f422f730640 1 -- start start 2024-04-24T23:23:54.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422f730640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42280727b0 0x7f4228140da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:54.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422f730640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4228141430 con 0x7f422813e030 2024-04-24T23:23:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422f730640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f422806f5e0 con 0x7f422813dc50 2024-04-24T23:23:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422f730640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f42280d0760 con 0x7f42280727b0 2024-04-24T23:23:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422dccd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42280727b0 0x7f4228140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422cccb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f422813e030 0x7f4228140690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35116/0 (socket says 172.21.15.26:35116) 2024-04-24T23:23:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422cccb640 1 -- 172.21.15.26:0/496224547 learned_addr learned my addr 172.21.15.26:0/496224547 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422dccd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42280727b0 0x7f4228140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51084/0 (socket says 172.21.15.26:51084) 2024-04-24T23:23:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422dccd640 1 -- 172.21.15.26:0/496224547 >> v1:172.21.15.29:6789/0 conn(0x7f422813dc50 legacy=0x7f42280720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422dccd640 1 -- 172.21.15.26:0/496224547 >> v1:172.21.15.26:6789/0 conn(0x7f422813e030 legacy=0x7f4228140690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.655+0000 7f422dccd640 1 -- 172.21.15.26:0/496224547 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422806f8c0 con 0x7f42280727b0 2024-04-24T23:23:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.656+0000 7f422dccd640 1 --2- 172.21.15.26:0/496224547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42280727b0 0x7f4228140da0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4220009d90 tx=0x7f422002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da42bf9c3f5be91c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.656+0000 7f421ffff640 1 -- 172.21.15.26:0/496224547 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220007740 con 0x7f42280727b0 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.656+0000 7f421ffff640 1 -- 172.21.15.26:0/496224547 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42200078e0 con 0x7f42280727b0 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.656+0000 7f421ffff640 1 -- 172.21.15.26:0/496224547 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220007cb0 con 0x7f42280727b0 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.657+0000 7f422f730640 1 -- 172.21.15.26:0/496224547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42280727b0 msgr2=0x7f4228140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.657+0000 7f422f730640 1 --2- 172.21.15.26:0/496224547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42280727b0 0x7f4228140da0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4220009d90 tx=0x7f422002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.657+0000 7f422f730640 1 -- 172.21.15.26:0/496224547 shutdown_connections 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.657+0000 7f422f730640 1 --2- 172.21.15.26:0/496224547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42280727b0 0x7f4228140da0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.657+0000 7f422f730640 1 -- 172.21.15.26:0/496224547 >> 172.21.15.26:0/496224547 conn(0x7f422806e2c0 msgr2=0x7f422806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.657+0000 7f422f730640 1 -- 172.21.15.26:0/496224547 shutdown_connections 2024-04-24T23:23:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.657+0000 7f422f730640 1 -- 172.21.15.26:0/496224547 wait complete. 2024-04-24T23:23:54.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.658+0000 7f422f730640 1 Processor -- start 2024-04-24T23:23:54.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.658+0000 7f422f730640 1 -- start start 2024-04-24T23:23:54.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422f730640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f42280727b0 0x7f4228151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:54.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422f730640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422813dc50 0x7f422815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422f730640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f422813e030 0x7f42280b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422f730640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f42280703a0 con 0x7f422813e030 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422f730640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f422806fe80 con 0x7f422813dc50 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422f730640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4228070520 con 0x7f42280727b0 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422dccd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f422813e030 0x7f42280b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422dccd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f422813e030 0x7f42280b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51096/0 (socket says 172.21.15.26:51096) 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422dccd640 1 -- 172.21.15.26:0/2541305262 learned_addr learned my addr 172.21.15.26:0/2541305262 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422cccb640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422813dc50 0x7f422815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422d4cc640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f42280727b0 0x7f4228151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422dccd640 1 -- 172.21.15.26:0/2541305262 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f42280727b0 msgr2=0x7f4228151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422dccd640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f42280727b0 0x7f4228151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:54.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422dccd640 1 -- 172.21.15.26:0/2541305262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422813dc50 msgr2=0x7f422815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:54.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422dccd640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422813dc50 0x7f422815f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:54.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.659+0000 7f422dccd640 1 -- 172.21.15.26:0/2541305262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4228156780 con 0x7f422813e030 2024-04-24T23:23:54.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.660+0000 7f422dccd640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f422813e030 0x7f42280b4ff0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4220009eb0 tx=0x7f4220004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:54.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.660+0000 7f421dffb640 1 -- 172.21.15.26:0/2541305262 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220040070 con 0x7f422813e030 2024-04-24T23:23:54.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.660+0000 7f421dffb640 1 -- 172.21.15.26:0/2541305262 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4220031c40 con 0x7f422813e030 2024-04-24T23:23:54.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.660+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4228156a60 con 0x7f422813e030 2024-04-24T23:23:54.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.660+0000 7f421dffb640 1 -- 172.21.15.26:0/2541305262 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422003c420 con 0x7f422813e030 2024-04-24T23:23:54.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.660+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4228157020 con 0x7f422813e030 2024-04-24T23:23:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.661+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42280694c0 con 0x7f422813e030 2024-04-24T23:23:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.662+0000 7f421dffb640 1 -- 172.21.15.26:0/2541305262 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4220039040 con 0x7f422813e030 2024-04-24T23:23:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.662+0000 7f421dffb640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f420c06be30 0x7f420c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.662+0000 7f421dffb640 1 -- 172.21.15.26:0/2541305262 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f42200ac3c0 con 0x7f422813e030 2024-04-24T23:23:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.662+0000 7f422d4cc640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f420c06be30 0x7f420c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:54.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.663+0000 7f422d4cc640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f420c06be30 0x7f420c06e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4224004f60 tx=0x7f4224007e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:54.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.667+0000 7f421dffb640 1 -- 172.21.15.26:0/2541305262 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f422007a770 con 0x7f422813e030 2024-04-24T23:23:54.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.891+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4228157820 con 0x7f422813e030 2024-04-24T23:23:54.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.891+0000 7f421dffb640 1 -- 172.21.15.26:0/2541305262 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5222 (secure 0 0 0) 0x7f422007ea70 con 0x7f422813e030 2024-04-24T23:23:54.893 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:54.893 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:40.866214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5742,"mds_1":5329,"mds_2":5081},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5081":{"gid":5081,"name":"b","rank":2,"incarnation":36,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742,5329,5081]},"id":3}]} 2024-04-24T23:23:54.893 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 42 2024-04-24T23:23:54.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f420c06be30 msgr2=0x7f420c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:54.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f420c06be30 0x7f420c06e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4224004f60 tx=0x7f4224007e30 comp rx=0 tx=0).stop 2024-04-24T23:23:54.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f422813e030 msgr2=0x7f42280b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:54.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f422813e030 0x7f42280b4ff0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4220009eb0 tx=0x7f4220004a40 comp rx=0 tx=0).stop 2024-04-24T23:23:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 shutdown_connections 2024-04-24T23:23:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f420c06be30 0x7f420c06e2f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f422813e030 0x7f42280b4ff0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f422813dc50 0x7f422815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 --2- 172.21.15.26:0/2541305262 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f42280727b0 0x7f4228151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 >> 172.21.15.26:0/2541305262 conn(0x7f422806e2c0 msgr2=0x7f422815d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 shutdown_connections 2024-04-24T23:23:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:54.894+0000 7f422f730640 1 -- 172.21.15.26:0/2541305262 wait complete. 2024-04-24T23:23:54.906 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'd', 'b'] 2024-04-24T23:23:54.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:23:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.005+0000 7fd589464640 1 Processor -- start 2024-04-24T23:23:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.005+0000 7fd589464640 1 -- start start 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.006+0000 7fd589464640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd5840b24e0 0x7fd5841692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.006+0000 7fd589464640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd584162860 con 0x7fd584147e30 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.006+0000 7fd589464640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd584162b40 con 0x7fd5840b4100 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.006+0000 7fd589464640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd584162e20 con 0x7fd5840b24e0 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.006+0000 7fd582ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd584147e30 0x7fd584161f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35126/0 (socket says 172.21.15.26:35126) 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.006+0000 7fd582ffd640 1 -- 172.21.15.26:0/951345771 learned_addr learned my addr 172.21.15.26:0/951345771 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.006+0000 7fd5837fe640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd5840b24e0 0x7fd5841692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.006+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1822472851 0 0) 0x7fd584162860 con 0x7fd584147e30 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.006+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd570003520 con 0x7fd584147e30 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980315435 0 0) 0x7fd584162b40 con 0x7fd5840b4100 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd584162860 con 0x7fd5840b4100 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1745433457 0 0) 0x7fd570003520 con 0x7fd584147e30 2024-04-24T23:23:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd5840b24e0 msgr2=0x7fd5841692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd5840b24e0 0x7fd5841692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 >> v1:172.21.15.29:6789/0 conn(0x7fd5840b4100 legacy=0x7fd5840b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5840b0cd0 con 0x7fd584147e30 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fd574001670 con 0x7fd584147e30 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 >> v1:172.21.15.26:6789/0 conn(0x7fd584147e30 legacy=0x7fd584161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd570003b70 0x7fd570003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd5840b24e0 0x7fd570004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570004d40 0x7fd5700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd584162e20 con 0x7fd570004d40 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd57000bc00 con 0x7fd570003b70 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd57000be00 con 0x7fd5840b24e0 2024-04-24T23:23:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd582ffd640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd570003b70 0x7fd570003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5837fe640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570004d40 0x7fd5700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd57a7fc640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd5840b24e0 0x7fd570004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5837fe640 1 -- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd5840b24e0 msgr2=0x7fd570004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5837fe640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd5840b24e0 0x7fd570004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5837fe640 1 -- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd570003b70 msgr2=0x7fd570003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5837fe640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd570003b70 0x7fd570003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.007+0000 7fd5837fe640 1 -- 172.21.15.26:0/951345771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd570003520 con 0x7fd570004d40 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.008+0000 7fd5837fe640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570004d40 0x7fd5700051c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009f90 tx=0x7fd570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4827ca833822e37a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.008+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c009c10 con 0x7fd570004d40 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.008+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57c036070 con 0x7fd570004d40 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.008+0000 7fd5827fc640 1 -- 172.21.15.26:0/951345771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c031850 con 0x7fd570004d40 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.009+0000 7fd589464640 1 -- 172.21.15.26:0/951345771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570004d40 msgr2=0x7fd5700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.009+0000 7fd589464640 1 --2- 172.21.15.26:0/951345771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd570004d40 0x7fd5700051c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009f90 tx=0x7fd570005830 comp rx=0 tx=0).stop 2024-04-24T23:23:55.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.009+0000 7fd5837fe640 1 -- 172.21.15.26:0/951345771 reap_dead start 2024-04-24T23:23:55.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.009+0000 7fd589464640 1 -- 172.21.15.26:0/951345771 shutdown_connections 2024-04-24T23:23:55.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.009+0000 7fd589464640 1 -- 172.21.15.26:0/951345771 >> 172.21.15.26:0/951345771 conn(0x7fd58409aa00 msgr2=0x7fd5840b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:55.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.009+0000 7fd589464640 1 -- 172.21.15.26:0/951345771 shutdown_connections 2024-04-24T23:23:55.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.009+0000 7fd589464640 1 -- 172.21.15.26:0/951345771 wait complete. 2024-04-24T23:23:55.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.010+0000 7fd589464640 1 Processor -- start 2024-04-24T23:23:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.010+0000 7fd589464640 1 -- start start 2024-04-24T23:23:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd589464640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5841447e0 0x7fd5840cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd589464640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5841e86f0 0x7fd584146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd589464640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd584146ac0 0x7fd5840ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd589464640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5840b6190 con 0x7fd5841e86f0 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd589464640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd5840b6010 con 0x7fd5841447e0 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd589464640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd5840b6310 con 0x7fd584146ac0 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5841e86f0 0x7fd584146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5841e86f0 0x7fd584146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51108/0 (socket says 172.21.15.26:51108) 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57a7fc640 1 -- 172.21.15.26:0/2606021321 learned_addr learned my addr 172.21.15.26:0/2606021321 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57a7fc640 1 -- 172.21.15.26:0/2606021321 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd584146ac0 msgr2=0x7fd5840ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57a7fc640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd584146ac0 0x7fd5840ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57a7fc640 1 -- 172.21.15.26:0/2606021321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5841447e0 msgr2=0x7fd5840cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57a7fc640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5841447e0 0x7fd5840cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57a7fc640 1 -- 172.21.15.26:0/2606021321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5841e8f40 con 0x7fd5841e86f0 2024-04-24T23:23:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57a7fc640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5841e86f0 0x7fd584146580 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00f7a0 tx=0x7fd56c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57bfff640 1 -- 172.21.15.26:0/2606021321 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c01a070 con 0x7fd5841e86f0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.011+0000 7fd57bfff640 1 -- 172.21.15.26:0/2606021321 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c005110 con 0x7fd5841e86f0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.012+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5841e9220 con 0x7fd5841e86f0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.012+0000 7fd57bfff640 1 -- 172.21.15.26:0/2606021321 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c007050 con 0x7fd5841e86f0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.012+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5841e97e0 con 0x7fd5841e86f0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.012+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd584146ee0 con 0x7fd5841e86f0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.013+0000 7fd57bfff640 1 -- 172.21.15.26:0/2606021321 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd56c01d070 con 0x7fd5841e86f0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.014+0000 7fd57bfff640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd56406be60 0x7fd56406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.014+0000 7fd582ffd640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd56406be60 0x7fd56406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.014+0000 7fd57bfff640 1 -- 172.21.15.26:0/2606021321 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fd56c08a5e0 con 0x7fd5841e86f0 2024-04-24T23:23:55.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.014+0000 7fd582ffd640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd56406be60 0x7fd56406e320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd574001bc0 tx=0x7fd574001a70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:55.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.018+0000 7fd57bfff640 1 -- 172.21.15.26:0/2606021321 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd56c058f50 con 0x7fd5841e86f0 2024-04-24T23:23:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.272+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd5840694c0 con 0x7fd5841e86f0 2024-04-24T23:23:55.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.273+0000 7fd57bfff640 1 -- 172.21.15.26:0/2606021321 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1870 (secure 0 0 0) 0x7fd56c058850 con 0x7fd5841e86f0 2024-04-24T23:23:55.274 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:55.274 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":204,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706268,"bytes_used":265592832,"bytes_avail":772828520448,"bytes_total":773094113280,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":42,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"b","status":"up:active","gid":5081},{"filesystem_id":3,"rank":1,"name":"d","status":"up:active","gid":5329},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5742}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-24T23:23:51.353931+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:23:55.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd56406be60 msgr2=0x7fd56406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:55.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd56406be60 0x7fd56406e320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd574001bc0 tx=0x7fd574001a70 comp rx=0 tx=0).stop 2024-04-24T23:23:55.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5841e86f0 msgr2=0x7fd584146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:55.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5841e86f0 0x7fd584146580 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00f7a0 tx=0x7fd56c00fc60 comp rx=0 tx=0).stop 2024-04-24T23:23:55.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 shutdown_connections 2024-04-24T23:23:55.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd56406be60 0x7fd56406e320 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:55.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd584146ac0 0x7fd5840ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:55.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5841e86f0 0x7fd584146580 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:55.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 --2- 172.21.15.26:0/2606021321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5841447e0 0x7fd5840cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:55.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 >> 172.21.15.26:0/2606021321 conn(0x7fd58409aa00 msgr2=0x7fd584160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:55.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 shutdown_connections 2024-04-24T23:23:55.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:55.275+0000 7fd589464640 1 -- 172.21.15.26:0/2606021321 wait complete. 2024-04-24T23:23:55.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]'} 2024-04-24T23:23:56.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.401+0000 7f4549b4c640 1 Processor -- start 2024-04-24T23:23:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.401+0000 7f4549b4c640 1 -- start start 2024-04-24T23:23:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4549b4c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f454406f110 0x7f454415f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4549b4c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45440adf80 0x7f4544154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4549b4c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f454415f750 con 0x7f454406f4f0 2024-04-24T23:23:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4549b4c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f45440b0220 con 0x7f45440adf80 2024-04-24T23:23:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4549b4c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f45440af180 con 0x7f454406f110 2024-04-24T23:23:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4543fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45440adf80 0x7f4544154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4543fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45440adf80 0x7f4544154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51114/0 (socket says 172.21.15.26:51114) 2024-04-24T23:23:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4543fff640 1 -- 172.21.15.26:0/1002822191 learned_addr learned my addr 172.21.15.26:0/1002822191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4543fff640 1 -- 172.21.15.26:0/1002822191 >> v2:172.21.15.29:3300/0 conn(0x7f454406f110 msgr2=0x7f454415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4543fff640 1 --2- 172.21.15.26:0/1002822191 >> v2:172.21.15.29:3300/0 conn(0x7f454406f110 0x7f454415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4543fff640 1 -- 172.21.15.26:0/1002822191 >> v1:172.21.15.29:6789/0 conn(0x7f454406f4f0 legacy=0x7f45440ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.402+0000 7f4543fff640 1 -- 172.21.15.26:0/1002822191 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45440af460 con 0x7f45440adf80 2024-04-24T23:23:56.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.403+0000 7f4543fff640 1 --2- 172.21.15.26:0/1002822191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45440adf80 0x7f4544154a50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4544072f30 tx=0x7f453c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d394c6b34ab2b3e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:56.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.403+0000 7f45427fc640 1 -- 172.21.15.26:0/1002822191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c036070 con 0x7f45440adf80 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.403+0000 7f45427fc640 1 -- 172.21.15.26:0/1002822191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f453c031600 con 0x7f45440adf80 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.403+0000 7f45427fc640 1 -- 172.21.15.26:0/1002822191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c0319d0 con 0x7f45440adf80 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.404+0000 7f4549b4c640 1 -- 172.21.15.26:0/1002822191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45440adf80 msgr2=0x7f4544154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.404+0000 7f4549b4c640 1 --2- 172.21.15.26:0/1002822191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45440adf80 0x7f4544154a50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4544072f30 tx=0x7f453c02a660 comp rx=0 tx=0).stop 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.404+0000 7f4549b4c640 1 -- 172.21.15.26:0/1002822191 shutdown_connections 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.404+0000 7f4549b4c640 1 --2- 172.21.15.26:0/1002822191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45440adf80 0x7f4544154a50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.404+0000 7f4549b4c640 1 --2- 172.21.15.26:0/1002822191 >> v2:172.21.15.29:3300/0 conn(0x7f454406f110 0x7f454415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.404+0000 7f4549b4c640 1 -- 172.21.15.26:0/1002822191 >> 172.21.15.26:0/1002822191 conn(0x7f45440729a0 msgr2=0x7f454406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.404+0000 7f4549b4c640 1 -- 172.21.15.26:0/1002822191 shutdown_connections 2024-04-24T23:23:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.404+0000 7f4549b4c640 1 -- 172.21.15.26:0/1002822191 wait complete. 2024-04-24T23:23:56.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.405+0000 7f4549b4c640 1 Processor -- start 2024-04-24T23:23:56.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.405+0000 7f4549b4c640 1 -- start start 2024-04-24T23:23:56.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4549b4c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454406f110 0x7f4544156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4549b4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f454406f4f0 0x7f4544160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4549b4c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45440adf80 0x7f4544158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4549b4c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45440afd00 con 0x7f454406f4f0 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4549b4c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f45440afa20 con 0x7f454406f110 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4549b4c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f45440aee50 con 0x7f45440adf80 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f454406f4f0 0x7f4544160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4542ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f454406f4f0 0x7f4544160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51120/0 (socket says 172.21.15.26:51120) 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4542ffd640 1 -- 172.21.15.26:0/384036499 learned_addr learned my addr 172.21.15.26:0/384036499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4543fff640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45440adf80 0x7f4544158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f45437fe640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454406f110 0x7f4544156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4542ffd640 1 -- 172.21.15.26:0/384036499 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45440adf80 msgr2=0x7f4544158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4542ffd640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45440adf80 0x7f4544158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4542ffd640 1 -- 172.21.15.26:0/384036499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454406f110 msgr2=0x7f4544156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4542ffd640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454406f110 0x7f4544156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.406+0000 7f4542ffd640 1 -- 172.21.15.26:0/384036499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45441f7360 con 0x7f454406f4f0 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.407+0000 7f4542ffd640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f454406f4f0 0x7f4544160bf0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f453400c400 tx=0x7f453400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.407+0000 7f452bfff640 1 -- 172.21.15.26:0/384036499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534017070 con 0x7f454406f4f0 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.407+0000 7f452bfff640 1 -- 172.21.15.26:0/384036499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45340035f0 con 0x7f454406f4f0 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.407+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45441f42e0 con 0x7f454406f4f0 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.407+0000 7f452bfff640 1 -- 172.21.15.26:0/384036499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534003a20 con 0x7f454406f4f0 2024-04-24T23:23:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.407+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45441f4b80 con 0x7f454406f4f0 2024-04-24T23:23:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.408+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45441f7530 con 0x7f454406f4f0 2024-04-24T23:23:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.408+0000 7f452bfff640 1 -- 172.21.15.26:0/384036499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f453401a070 con 0x7f454406f4f0 2024-04-24T23:23:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.409+0000 7f452bfff640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453006be30 0x7f453006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.409+0000 7f452bfff640 1 -- 172.21.15.26:0/384036499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f453408a180 con 0x7f454406f4f0 2024-04-24T23:23:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.409+0000 7f45437fe640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453006be30 0x7f453006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.409+0000 7f45437fe640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453006be30 0x7f453006e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4538009890 tx=0x7f4538008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:56.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.413+0000 7f452bfff640 1 -- 172.21.15.26:0/384036499 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45340584a0 con 0x7f454406f4f0 2024-04-24T23:23:56.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.633+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45440694c0 con 0x7f454406f4f0 2024-04-24T23:23:56.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.633+0000 7f452bfff640 1 -- 172.21.15.26:0/384036499 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5222 (secure 0 0 0) 0x7f453405c7a0 con 0x7f454406f4f0 2024-04-24T23:23:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:56.635 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:40.866214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5742,"mds_1":5329,"mds_2":5081},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5081":{"gid":5081,"name":"b","rank":2,"incarnation":36,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742,5329,5081]},"id":3}]} 2024-04-24T23:23:56.635 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 42 2024-04-24T23:23:56.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453006be30 msgr2=0x7f453006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453006be30 0x7f453006e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4538009890 tx=0x7f4538008040 comp rx=0 tx=0).stop 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f454406f4f0 msgr2=0x7f4544160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f454406f4f0 0x7f4544160bf0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f453400c400 tx=0x7f453400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 shutdown_connections 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f453006be30 0x7f453006e2f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45440adf80 0x7f4544158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f454406f4f0 0x7f4544160bf0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 --2- 172.21.15.26:0/384036499 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454406f110 0x7f4544156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 >> 172.21.15.26:0/384036499 conn(0x7f45440729a0 msgr2=0x7f45440ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 shutdown_connections 2024-04-24T23:23:56.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.635+0000 7f4549b4c640 1 -- 172.21.15.26:0/384036499 wait complete. 2024-04-24T23:23:56.648 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'd', 'b'] 2024-04-24T23:23:56.648 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:23:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.748+0000 7f0b701ed640 1 Processor -- start 2024-04-24T23:23:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.748+0000 7f0b701ed640 1 -- start start 2024-04-24T23:23:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b701ed640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0b6806e010 0x7f0b680d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b701ed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b6806f740 con 0x7f0b6809b110 2024-04-24T23:23:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b701ed640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b6806fa20 con 0x7f0b6806dc30 2024-04-24T23:23:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b701ed640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0b6806fd00 con 0x7f0b6806e010 2024-04-24T23:23:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b6e78a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0b6806e010 0x7f0b680d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b6d788640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0b6809b110 0x7f0b6815dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35128/0 (socket says 172.21.15.26:35128) 2024-04-24T23:23:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b6e78a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0b6806e010 0x7f0b680d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51134/0 (socket says 172.21.15.26:51134) 2024-04-24T23:23:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b6d788640 1 -- 172.21.15.26:0/1571987564 learned_addr learned my addr 172.21.15.26:0/1571987564 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b6e78a640 1 -- 172.21.15.26:0/1571987564 >> v1:172.21.15.29:6789/0 conn(0x7f0b6806dc30 legacy=0x7f0b6809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b6e78a640 1 -- 172.21.15.26:0/1571987564 >> v1:172.21.15.26:6789/0 conn(0x7f0b6809b110 legacy=0x7f0b6815dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b6e78a640 1 -- 172.21.15.26:0/1571987564 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6806ffe0 con 0x7f0b6806e010 2024-04-24T23:23:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.749+0000 7f0b6cf87640 1 -- 172.21.15.26:0/1571987564 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 802768403 0 0) 0x7f0b6806f740 con 0x7f0b6809b110 2024-04-24T23:23:56.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.750+0000 7f0b6e78a640 1 --2- 172.21.15.26:0/1571987564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806e010 0x7f0b680d1180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009d90 tx=0x7f0b6002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6978670b19b2617 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:56.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.750+0000 7f0b6cf87640 1 -- 172.21.15.26:0/1571987564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60007740 con 0x7f0b6806e010 2024-04-24T23:23:56.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.750+0000 7f0b6cf87640 1 -- 172.21.15.26:0/1571987564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b600078e0 con 0x7f0b6806e010 2024-04-24T23:23:56.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.750+0000 7f0b6cf87640 1 -- 172.21.15.26:0/1571987564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60007cb0 con 0x7f0b6806e010 2024-04-24T23:23:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.751+0000 7f0b701ed640 1 -- 172.21.15.26:0/1571987564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806e010 msgr2=0x7f0b680d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.751+0000 7f0b701ed640 1 --2- 172.21.15.26:0/1571987564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806e010 0x7f0b680d1180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009d90 tx=0x7f0b6002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.751+0000 7f0b701ed640 1 -- 172.21.15.26:0/1571987564 shutdown_connections 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.751+0000 7f0b701ed640 1 --2- 172.21.15.26:0/1571987564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806e010 0x7f0b680d1180 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.751+0000 7f0b701ed640 1 -- 172.21.15.26:0/1571987564 >> 172.21.15.26:0/1571987564 conn(0x7f0b680b3a60 msgr2=0x7f0b680b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.751+0000 7f0b701ed640 1 -- 172.21.15.26:0/1571987564 shutdown_connections 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.751+0000 7f0b701ed640 1 -- 172.21.15.26:0/1571987564 wait complete. 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.752+0000 7f0b701ed640 1 Processor -- start 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.752+0000 7f0b701ed640 1 -- start start 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b701ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806dc30 0x7f0b681f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b701ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b6806e010 0x7f0b68159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b701ed640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0b6809b110 0x7f0b680b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b701ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6806d240 con 0x7f0b6806dc30 2024-04-24T23:23:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b701ed640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0b680d0570 con 0x7f0b6806e010 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b701ed640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6806d3c0 con 0x7f0b6809b110 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6df89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806dc30 0x7f0b681f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6df89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806dc30 0x7f0b681f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51140/0 (socket says 172.21.15.26:51140) 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6df89640 1 -- 172.21.15.26:0/2986920375 learned_addr learned my addr 172.21.15.26:0/2986920375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6e78a640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0b6809b110 0x7f0b680b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6d788640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b6806e010 0x7f0b68159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6df89640 1 -- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0b6809b110 msgr2=0x7f0b680b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6df89640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0b6809b110 0x7f0b680b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6df89640 1 -- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b6806e010 msgr2=0x7f0b68159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6df89640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b6806e010 0x7f0b68159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6df89640 1 -- 172.21.15.26:0/2986920375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b681f7710 con 0x7f0b6806dc30 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6e78a640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0b6809b110 0x7f0b680b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6d788640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b6806e010 0x7f0b68159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.753+0000 7f0b6df89640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806dc30 0x7f0b681f7030 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400c430 tx=0x7f0b6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.754+0000 7f0b5e7fc640 1 -- 172.21.15.26:0/2986920375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64011790 con 0x7f0b6806dc30 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.754+0000 7f0b5e7fc640 1 -- 172.21.15.26:0/2986920375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b64017070 con 0x7f0b6806dc30 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.754+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b681f79f0 con 0x7f0b6806dc30 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.754+0000 7f0b5e7fc640 1 -- 172.21.15.26:0/2986920375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64011bb0 con 0x7f0b6806dc30 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.754+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b680ca1c0 con 0x7f0b6806dc30 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.755+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b680ca3c0 con 0x7f0b6806dc30 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.755+0000 7f0b5e7fc640 1 -- 172.21.15.26:0/2986920375 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b6401c460 con 0x7f0b6806dc30 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.756+0000 7f0b5e7fc640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0b4c06be30 0x7f0b4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.756+0000 7f0b5e7fc640 1 -- 172.21.15.26:0/2986920375 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f0b6408acf0 con 0x7f0b6806dc30 2024-04-24T23:23:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.756+0000 7f0b6d788640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0b4c06be30 0x7f0b4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.756+0000 7f0b6d788640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0b4c06be30 0x7f0b4c06e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0b6815f7c0 tx=0x7f0b58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:56.760+0000 7f0b5e7fc640 1 -- 172.21.15.26:0/2986920375 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0b6408f070 con 0x7f0b6806dc30 2024-04-24T23:23:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.012+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f0b680694c0 con 0x7f0b6806dc30 2024-04-24T23:23:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.012+0000 7f0b5e7fc640 1 -- 172.21.15.26:0/2986920375 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1823 (secure 0 0 0) 0x7f0b640596f0 con 0x7f0b6806dc30 2024-04-24T23:23:57.014 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:57.014 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":205,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706268,"bytes_used":265592832,"bytes_avail":772828520448,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":42,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"b","status":"up:active","gid":5081},{"filesystem_id":3,"rank":1,"name":"d","status":"up:active","gid":5329},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5742}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-04-24T23:23:55.354941+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:23:57.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0b4c06be30 msgr2=0x7f0b4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0b4c06be30 0x7f0b4c06e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0b6815f7c0 tx=0x7f0b58009040 comp rx=0 tx=0).stop 2024-04-24T23:23:57.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806dc30 msgr2=0x7f0b681f7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806dc30 0x7f0b681f7030 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400c430 tx=0x7f0b6400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 shutdown_connections 2024-04-24T23:23:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0b4c06be30 0x7f0b4c06e2f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0b6809b110 0x7f0b680b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b6806e010 0x7f0b68159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 --2- 172.21.15.26:0/2986920375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b6806dc30 0x7f0b681f7030 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 >> 172.21.15.26:0/2986920375 conn(0x7f0b680b3a60 msgr2=0x7f0b680b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 shutdown_connections 2024-04-24T23:23:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.014+0000 7f0b701ed640 1 -- 172.21.15.26:0/2986920375 wait complete. 2024-04-24T23:23:57.026 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-24T23:23:57.027 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:23:57.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:23:57.060 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:23:57.108 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:23:57.108 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:23:57.109 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:23:57.109 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:23:57.124 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:23:57.124 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:23:57.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.321+0000 7fed1ab5a640 1 Processor -- start 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.321+0000 7fed1ab5a640 1 -- start start 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed1ab5a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fed1406f4f0 0x7fed140ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed1ab5a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed140addb0 0x7fed14154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed1ab5a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed140b0220 con 0x7fed1406f110 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed1ab5a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fed140af180 con 0x7fed1406f4f0 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed1ab5a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fed140af460 con 0x7fed140addb0 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed190f7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed140addb0 0x7fed14154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed13fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fed1406f4f0 0x7fed140ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed190f7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed140addb0 0x7fed14154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:60282/0 (socket says 172.21.15.26:60282) 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed190f7640 1 -- 172.21.15.26:0/326477066 learned_addr learned my addr 172.21.15.26:0/326477066 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.322+0000 7fed188f6640 1 --1- 172.21.15.26:0/326477066 >> v1:172.21.15.29:6789/0 conn(0x7fed1406f110 0x7fed1415f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:52024/0 (socket says 172.21.15.26:52024) 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.323+0000 7fed13fff640 1 -- 172.21.15.26:0/326477066 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed140addb0 msgr2=0x7fed14154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.323+0000 7fed13fff640 1 --2- 172.21.15.26:0/326477066 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed140addb0 0x7fed14154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.323+0000 7fed13fff640 1 -- 172.21.15.26:0/326477066 >> v1:172.21.15.29:6789/0 conn(0x7fed1406f110 legacy=0x7fed1415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.323+0000 7fed13fff640 1 -- 172.21.15.26:0/326477066 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed140af740 con 0x7fed1406f4f0 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.323+0000 7fed190f7640 1 --2- 172.21.15.26:0/326477066 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed140addb0 0x7fed14154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.323+0000 7fed13fff640 1 --2- 172.21.15.26:0/326477066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1406f4f0 0x7fed140ad870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fed04009d90 tx=0x7fed0402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ec79571075ef8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed137fe640 1 -- 172.21.15.26:0/326477066 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04007740 con 0x7fed1406f4f0 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed137fe640 1 -- 172.21.15.26:0/326477066 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed040078e0 con 0x7fed1406f4f0 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed137fe640 1 -- 172.21.15.26:0/326477066 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04007cb0 con 0x7fed1406f4f0 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed1ab5a640 1 -- 172.21.15.26:0/326477066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1406f4f0 msgr2=0x7fed140ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed1ab5a640 1 --2- 172.21.15.26:0/326477066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1406f4f0 0x7fed140ad870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fed04009d90 tx=0x7fed0402a660 comp rx=0 tx=0).stop 2024-04-24T23:23:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed1ab5a640 1 -- 172.21.15.26:0/326477066 shutdown_connections 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed1ab5a640 1 --2- 172.21.15.26:0/326477066 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed140addb0 0x7fed14154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed1ab5a640 1 --2- 172.21.15.26:0/326477066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed1406f4f0 0x7fed140ad870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed1ab5a640 1 -- 172.21.15.26:0/326477066 >> 172.21.15.26:0/326477066 conn(0x7fed140729a0 msgr2=0x7fed1406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed1ab5a640 1 -- 172.21.15.26:0/326477066 shutdown_connections 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.324+0000 7fed1ab5a640 1 -- 172.21.15.26:0/326477066 wait complete. 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.325+0000 7fed1ab5a640 1 Processor -- start 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.325+0000 7fed1ab5a640 1 -- start start 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed1ab5a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed1406f110 0x7fed14160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed1ab5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1406f4f0 0x7fed141571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed1ab5a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed140addb0 0x7fed14157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed1ab5a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed140aee50 con 0x7fed1406f4f0 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed1ab5a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fed140afd00 con 0x7fed140addb0 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed1ab5a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fed1406fb00 con 0x7fed1406f110 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1406f4f0 0x7fed141571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1406f4f0 0x7fed141571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51146/0 (socket says 172.21.15.26:51146) 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed13fff640 1 -- 172.21.15.26:0/3959468564 learned_addr learned my addr 172.21.15.26:0/3959468564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed188f6640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed1406f110 0x7fed14160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.326+0000 7fed190f7640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed140addb0 0x7fed14157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed13fff640 1 -- 172.21.15.26:0/3959468564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed1406f110 msgr2=0x7fed14160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed13fff640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed1406f110 0x7fed14160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed13fff640 1 -- 172.21.15.26:0/3959468564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed140addb0 msgr2=0x7fed14157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed13fff640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed140addb0 0x7fed14157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed13fff640 1 -- 172.21.15.26:0/3959468564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed14157c40 con 0x7fed1406f4f0 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed13fff640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1406f4f0 0x7fed141571c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fed04009d90 tx=0x7fed04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed117fa640 1 -- 172.21.15.26:0/3959468564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04040070 con 0x7fed1406f4f0 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed117fa640 1 -- 172.21.15.26:0/3959468564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed04031c40 con 0x7fed1406f4f0 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed14157ef0 con 0x7fed1406f4f0 2024-04-24T23:23:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed117fa640 1 -- 172.21.15.26:0/3959468564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04009450 con 0x7fed1406f4f0 2024-04-24T23:23:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.327+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed141584b0 con 0x7fed1406f4f0 2024-04-24T23:23:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.328+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed14158c70 con 0x7fed1406f4f0 2024-04-24T23:23:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.329+0000 7fed117fa640 1 -- 172.21.15.26:0/3959468564 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed04039040 con 0x7fed1406f4f0 2024-04-24T23:23:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.329+0000 7fed117fa640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fecf406be30 0x7fecf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.329+0000 7fed117fa640 1 -- 172.21.15.26:0/3959468564 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fed040ac590 con 0x7fed1406f4f0 2024-04-24T23:23:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.329+0000 7fed188f6640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fecf406be30 0x7fecf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.329+0000 7fed188f6640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fecf406be30 0x7fecf406e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fed0c007fd0 tx=0x7fed0c007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.333+0000 7fed117fa640 1 -- 172.21.15.26:0/3959468564 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fed14158c70 con 0x7fed1406f4f0 2024-04-24T23:23:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.518+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed1406c5b0 con 0x7fed1406f4f0 2024-04-24T23:23:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.519+0000 7fed117fa640 1 -- 172.21.15.26:0/3959468564 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16705 (secure 0 0 0) 0x7fed0407a8b0 con 0x7fed1406f4f0 2024-04-24T23:23:57.520 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:57.520 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":46,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:23:36.893433+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:23:01.453865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:23:02.136971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":4}]},{"pgid":"7.2c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:23:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fecf406be30 msgr2=0x7fecf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fecf406be30 0x7fecf406e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fed0c007fd0 tx=0x7fed0c007e80 comp rx=0 tx=0).stop 2024-04-24T23:23:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1406f4f0 msgr2=0x7fed141571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1406f4f0 0x7fed141571c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fed04009d90 tx=0x7fed04004a40 comp rx=0 tx=0).stop 2024-04-24T23:23:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 shutdown_connections 2024-04-24T23:23:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fecf406be30 0x7fecf406e2f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed140addb0 0x7fed14157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed1406f4f0 0x7fed141571c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 --2- 172.21.15.26:0/3959468564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed1406f110 0x7fed14160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 >> 172.21.15.26:0/3959468564 conn(0x7fed140729a0 msgr2=0x7fed140b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 shutdown_connections 2024-04-24T23:23:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.521+0000 7fed1ab5a640 1 -- 172.21.15.26:0/3959468564 wait complete. 2024-04-24T23:23:57.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:23:01.453865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:23:02.136971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:23:57.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:23:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.636+0000 7f65462ba640 1 Processor -- start 2024-04-24T23:23:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.636+0000 7f65462ba640 1 -- start start 2024-04-24T23:23:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f65462ba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 0x7f65400b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f65462ba640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f65400ad7b0 0x7f65400d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f65462ba640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f654015ef60 con 0x7f654015e9c0 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f65462ba640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f654015f240 con 0x7f65400ad7b0 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f65462ba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f654015f520 con 0x7f654015adc0 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f653f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f65400ad7b0 0x7f65400d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f653f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f65400ad7b0 0x7f65400d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51162/0 (socket says 172.21.15.26:51162) 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f653f7fe640 1 -- 172.21.15.26:0/567975761 learned_addr learned my addr 172.21.15.26:0/567975761 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f6544857640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f654015e9c0 0x7f654006dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35144/0 (socket says 172.21.15.26:35144) 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.637+0000 7f653ffff640 1 --2- 172.21.15.26:0/567975761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 0x7f65400b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.638+0000 7f653f7fe640 1 -- 172.21.15.26:0/567975761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 msgr2=0x7f65400b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.638+0000 7f653f7fe640 1 --2- 172.21.15.26:0/567975761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 0x7f65400b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.638+0000 7f653f7fe640 1 -- 172.21.15.26:0/567975761 >> v1:172.21.15.26:6789/0 conn(0x7f654015e9c0 legacy=0x7f654006dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.638+0000 7f653f7fe640 1 -- 172.21.15.26:0/567975761 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654015f800 con 0x7f65400ad7b0 2024-04-24T23:23:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.638+0000 7f653effd640 1 -- 172.21.15.26:0/567975761 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 482377261 0 0) 0x7f654015ef60 con 0x7f654015e9c0 2024-04-24T23:23:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.638+0000 7f653f7fe640 1 --2- 172.21.15.26:0/567975761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 0x7f65400d0040 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6530009870 tx=0x7f653002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1c53dd3f1b056d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f653effd640 1 -- 172.21.15.26:0/567975761 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530037070 con 0x7f65400ad7b0 2024-04-24T23:23:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f653effd640 1 -- 172.21.15.26:0/567975761 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6530032440 con 0x7f65400ad7b0 2024-04-24T23:23:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f653effd640 1 -- 172.21.15.26:0/567975761 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530032810 con 0x7f65400ad7b0 2024-04-24T23:23:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f65462ba640 1 -- 172.21.15.26:0/567975761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 msgr2=0x7f65400d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f65462ba640 1 --2- 172.21.15.26:0/567975761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 0x7f65400d0040 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6530009870 tx=0x7f653002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f65462ba640 1 -- 172.21.15.26:0/567975761 shutdown_connections 2024-04-24T23:23:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f65462ba640 1 --2- 172.21.15.26:0/567975761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 0x7f65400d0040 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f65462ba640 1 --2- 172.21.15.26:0/567975761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 0x7f65400b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f65462ba640 1 -- 172.21.15.26:0/567975761 >> 172.21.15.26:0/567975761 conn(0x7f654006b540 msgr2=0x7f654006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f65462ba640 1 -- 172.21.15.26:0/567975761 shutdown_connections 2024-04-24T23:23:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.639+0000 7f65462ba640 1 -- 172.21.15.26:0/567975761 wait complete. 2024-04-24T23:23:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.640+0000 7f65462ba640 1 Processor -- start 2024-04-24T23:23:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f65462ba640 1 -- start start 2024-04-24T23:23:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f65462ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 0x7f6540141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f65462ba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 0x7f65400b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f65462ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f654015e9c0 0x7f65400cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f65462ba640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f654015b1a0 con 0x7f65400ad7b0 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f65462ba640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f65400b48b0 con 0x7f654015e9c0 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f65462ba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f65400d0580 con 0x7f654015adc0 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f653ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 0x7f6540141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f653ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 0x7f6540141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51176/0 (socket says 172.21.15.26:51176) 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.641+0000 7f653ffff640 1 -- 172.21.15.26:0/3094971255 learned_addr learned my addr 172.21.15.26:0/3094971255 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653f7fe640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 0x7f65400b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653ffff640 1 -- 172.21.15.26:0/3094971255 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 msgr2=0x7f65400b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653ffff640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 0x7f65400b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653ffff640 1 -- 172.21.15.26:0/3094971255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f654015e9c0 msgr2=0x7f65400cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653ffff640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f654015e9c0 0x7f65400cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653ffff640 1 -- 172.21.15.26:0/3094971255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6540155500 con 0x7f65400ad7b0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653ffff640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 0x7f6540141fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f653400e9b0 tx=0x7f653400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653cff9640 1 -- 172.21.15.26:0/3094971255 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534017070 con 0x7f65400ad7b0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653cff9640 1 -- 172.21.15.26:0/3094971255 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653400c9a0 con 0x7f65400ad7b0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65400b0a00 con 0x7f65400ad7b0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f653cff9640 1 -- 172.21.15.26:0/3094971255 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400cd90 con 0x7f65400ad7b0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.642+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65400b0fc0 con 0x7f65400ad7b0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.643+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6540075610 con 0x7f65400ad7b0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.644+0000 7f653cff9640 1 -- 172.21.15.26:0/3094971255 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f653401a070 con 0x7f65400ad7b0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.644+0000 7f653cff9640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f652c06be30 0x7f652c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.644+0000 7f653cff9640 1 -- 172.21.15.26:0/3094971255 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f653408b210 con 0x7f65400ad7b0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.644+0000 7f653f7fe640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f652c06be30 0x7f652c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.645+0000 7f653f7fe640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f652c06be30 0x7f652c06e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6530034000 tx=0x7f653002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.648+0000 7f653cff9640 1 -- 172.21.15.26:0/3094971255 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6534059c10 con 0x7f65400ad7b0 2024-04-24T23:23:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.833+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65400719f0 con 0x7f65400ad7b0 2024-04-24T23:23:57.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.833+0000 7f653cff9640 1 -- 172.21.15.26:0/3094971255 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f6534059510 con 0x7f65400ad7b0 2024-04-24T23:23:57.835 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:23:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f652c06be30 msgr2=0x7f652c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f652c06be30 0x7f652c06e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6530034000 tx=0x7f653002e000 comp rx=0 tx=0).stop 2024-04-24T23:23:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 msgr2=0x7f6540141fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 0x7f6540141fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f653400e9b0 tx=0x7f653400ee70 comp rx=0 tx=0).stop 2024-04-24T23:23:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 shutdown_connections 2024-04-24T23:23:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f652c06be30 0x7f652c06e2f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f654015e9c0 0x7f65400cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f654015adc0 0x7f65400b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 --2- 172.21.15.26:0/3094971255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65400ad7b0 0x7f6540141fe0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.835+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 >> 172.21.15.26:0/3094971255 conn(0x7f654006b540 msgr2=0x7f654015dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.836+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 shutdown_connections 2024-04-24T23:23:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.836+0000 7f65462ba640 1 -- 172.21.15.26:0/3094971255 wait complete. 2024-04-24T23:23:57.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:23:57.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.951+0000 7f26d8f63640 1 Processor -- start 2024-04-24T23:23:57.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.951+0000 7f26d8f63640 1 -- start start 2024-04-24T23:23:57.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.952+0000 7f26d8f63640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26d40d0750 0x7f26d40acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.952+0000 7f26d8f63640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d40b4ce0 0x7f26d415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.952+0000 7f26d8f63640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d40aeb10 con 0x7f26d415be40 2024-04-24T23:23:57.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.952+0000 7f26d8f63640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f26d409df20 con 0x7f26d40d0750 2024-04-24T23:23:57.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.952+0000 7f26d8f63640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f26d40aedf0 con 0x7f26d40b4ce0 2024-04-24T23:23:57.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.952+0000 7f26d1d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26d40d0750 0x7f26d40acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.952+0000 7f26d1d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26d40d0750 0x7f26d40acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51186/0 (socket says 172.21.15.26:51186) 2024-04-24T23:23:57.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.952+0000 7f26d1d9b640 1 -- 172.21.15.26:0/2011618507 learned_addr learned my addr 172.21.15.26:0/2011618507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:57.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.952+0000 7f26d2d9d640 1 --2- 172.21.15.26:0/2011618507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d40b4ce0 0x7f26d415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.953+0000 7f26d1d9b640 1 -- 172.21.15.26:0/2011618507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d40b4ce0 msgr2=0x7f26d415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:57.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.953+0000 7f26d1d9b640 1 --2- 172.21.15.26:0/2011618507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d40b4ce0 0x7f26d415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.953+0000 7f26d1d9b640 1 -- 172.21.15.26:0/2011618507 >> v1:172.21.15.29:6789/0 conn(0x7f26d415be40 legacy=0x7f26d40d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.953+0000 7f26d1d9b640 1 -- 172.21.15.26:0/2011618507 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d40af0d0 con 0x7f26d40d0750 2024-04-24T23:23:57.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.953+0000 7f26d1d9b640 1 --2- 172.21.15.26:0/2011618507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40d0750 0x7f26d40acd30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009870 tx=0x7f26c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a36a139885ee10 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.953+0000 7f26d159a640 1 -- 172.21.15.26:0/2011618507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8037070 con 0x7f26d40d0750 2024-04-24T23:23:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d159a640 1 -- 172.21.15.26:0/2011618507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c8032440 con 0x7f26d40d0750 2024-04-24T23:23:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d159a640 1 -- 172.21.15.26:0/2011618507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8032810 con 0x7f26d40d0750 2024-04-24T23:23:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d8f63640 1 -- 172.21.15.26:0/2011618507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40d0750 msgr2=0x7f26d40acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d8f63640 1 --2- 172.21.15.26:0/2011618507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40d0750 0x7f26d40acd30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009870 tx=0x7f26c802a270 comp rx=0 tx=0).stop 2024-04-24T23:23:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d8f63640 1 -- 172.21.15.26:0/2011618507 shutdown_connections 2024-04-24T23:23:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d8f63640 1 --2- 172.21.15.26:0/2011618507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d40b4ce0 0x7f26d415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d8f63640 1 --2- 172.21.15.26:0/2011618507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40d0750 0x7f26d40acd30 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d8f63640 1 -- 172.21.15.26:0/2011618507 >> 172.21.15.26:0/2011618507 conn(0x7f26d406b540 msgr2=0x7f26d406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d8f63640 1 -- 172.21.15.26:0/2011618507 shutdown_connections 2024-04-24T23:23:57.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.954+0000 7f26d8f63640 1 -- 172.21.15.26:0/2011618507 wait complete. 2024-04-24T23:23:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.956+0000 7f26d8f63640 1 Processor -- start 2024-04-24T23:23:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.956+0000 7f26d8f63640 1 -- start start 2024-04-24T23:23:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d8f63640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40b4ce0 0x7f26d414a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d8f63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d40d0750 0x7f26d409ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d8f63640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d415be40 0x7f26d409bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d8f63640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26d406cd10 con 0x7f26d40b4ce0 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d8f63640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f26d406cb90 con 0x7f26d40d0750 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d8f63640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f26d415c760 con 0x7f26d415be40 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40b4ce0 0x7f26d414a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40b4ce0 0x7f26d414a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51194/0 (socket says 172.21.15.26:51194) 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d259c640 1 -- 172.21.15.26:0/3396448908 learned_addr learned my addr 172.21.15.26:0/3396448908 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d1d9b640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d40d0750 0x7f26d409ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d2d9d640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d415be40 0x7f26d409bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d259c640 1 -- 172.21.15.26:0/3396448908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d415be40 msgr2=0x7f26d409bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d259c640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d415be40 0x7f26d409bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d259c640 1 -- 172.21.15.26:0/3396448908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d40d0750 msgr2=0x7f26d409ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d259c640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d40d0750 0x7f26d409ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d259c640 1 -- 172.21.15.26:0/3396448908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d414ac90 con 0x7f26d40b4ce0 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d2d9d640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d415be40 0x7f26d409bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d259c640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40b4ce0 0x7f26d414a750 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c3c0 tx=0x7f26c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26c2ffd640 1 -- 172.21.15.26:0/3396448908 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40052d0 con 0x7f26d40b4ce0 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26c2ffd640 1 -- 172.21.15.26:0/3396448908 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4005490 con 0x7f26d40b4ce0 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d409c5c0 con 0x7f26d40b4ce0 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26c2ffd640 1 -- 172.21.15.26:0/3396448908 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4007260 con 0x7f26d40b4ce0 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.957+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d41e49a0 con 0x7f26d40b4ce0 2024-04-24T23:23:57.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.958+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d4075610 con 0x7f26d40b4ce0 2024-04-24T23:23:57.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.959+0000 7f26c2ffd640 1 -- 172.21.15.26:0/3396448908 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26c4007400 con 0x7f26d40b4ce0 2024-04-24T23:23:57.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.959+0000 7f26c2ffd640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f26b406be30 0x7f26b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.959+0000 7f26c2ffd640 1 -- 172.21.15.26:0/3396448908 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f26c408a240 con 0x7f26d40b4ce0 2024-04-24T23:23:57.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.959+0000 7f26d1d9b640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f26b406be30 0x7f26b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.960+0000 7f26d1d9b640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f26b406be30 0x7f26b406e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f26c8034000 tx=0x7f26c802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:57.963+0000 7f26c2ffd640 1 -- 172.21.15.26:0/3396448908 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f26c40585f0 con 0x7f26d40b4ce0 2024-04-24T23:23:58.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.147+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f26d41e4750 con 0x7f26d40b4ce0 2024-04-24T23:23:58.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.147+0000 7f26c2ffd640 1 -- 172.21.15.26:0/3396448908 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7f26c405c8f0 con 0x7f26d40b4ce0 2024-04-24T23:23:58.149 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:23:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f26b406be30 msgr2=0x7f26b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f26b406be30 0x7f26b406e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f26c8034000 tx=0x7f26c802e000 comp rx=0 tx=0).stop 2024-04-24T23:23:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40b4ce0 msgr2=0x7f26d414a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40b4ce0 0x7f26d414a750 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c3c0 tx=0x7f26c400c880 comp rx=0 tx=0).stop 2024-04-24T23:23:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 shutdown_connections 2024-04-24T23:23:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f26b406be30 0x7f26b406e2f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f26d415be40 0x7f26d409bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d40d0750 0x7f26d409ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 --2- 172.21.15.26:0/3396448908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26d40b4ce0 0x7f26d414a750 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.149+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 >> 172.21.15.26:0/3396448908 conn(0x7f26d406b540 msgr2=0x7f26d406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.150+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 shutdown_connections 2024-04-24T23:23:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.150+0000 7f26d8f63640 1 -- 172.21.15.26:0/3396448908 wait complete. 2024-04-24T23:23:58.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:23:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.264+0000 7fd7acdd5640 1 Processor -- start 2024-04-24T23:23:58.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.264+0000 7fd7acdd5640 1 -- start start 2024-04-24T23:23:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7acdd5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7a815adc0 0x7fd7a80b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7acdd5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a80ad7b0 0x7fd7a80d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7acdd5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7a815ef60 con 0x7fd7a815e9c0 2024-04-24T23:23:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7acdd5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a815f240 con 0x7fd7a815adc0 2024-04-24T23:23:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7acdd5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a815f520 con 0x7fd7a80ad7b0 2024-04-24T23:23:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7a659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7a815adc0 0x7fd7a80b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7a659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7a815adc0 0x7fd7a80b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51200/0 (socket says 172.21.15.26:51200) 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7a659c640 1 -- 172.21.15.26:0/3988442777 learned_addr learned my addr 172.21.15.26:0/3988442777 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd79ffff640 1 --2- 172.21.15.26:0/3988442777 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a80ad7b0 0x7fd7a80d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7a659c640 1 -- 172.21.15.26:0/3988442777 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a80ad7b0 msgr2=0x7fd7a80d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7a659c640 1 --2- 172.21.15.26:0/3988442777 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a80ad7b0 0x7fd7a80d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7a659c640 1 -- 172.21.15.26:0/3988442777 >> v1:172.21.15.29:6789/0 conn(0x7fd7a815e9c0 legacy=0x7fd7a806dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.265+0000 7fd7a659c640 1 -- 172.21.15.26:0/3988442777 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a815f800 con 0x7fd7a815adc0 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.266+0000 7fd7a659c640 1 --2- 172.21.15.26:0/3988442777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 0x7fd7a80b4370 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd798009870 tx=0x7fd79802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0435747a9243748 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.266+0000 7fd7a5d9b640 1 -- 172.21.15.26:0/3988442777 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798037070 con 0x7fd7a815adc0 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.266+0000 7fd7a5d9b640 1 -- 172.21.15.26:0/3988442777 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd798032440 con 0x7fd7a815adc0 2024-04-24T23:23:58.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.266+0000 7fd7a5d9b640 1 -- 172.21.15.26:0/3988442777 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798032810 con 0x7fd7a815adc0 2024-04-24T23:23:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.267+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3988442777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 msgr2=0x7fd7a80b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.267+0000 7fd7acdd5640 1 --2- 172.21.15.26:0/3988442777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 0x7fd7a80b4370 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd798009870 tx=0x7fd79802a180 comp rx=0 tx=0).stop 2024-04-24T23:23:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.267+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3988442777 shutdown_connections 2024-04-24T23:23:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.267+0000 7fd7acdd5640 1 --2- 172.21.15.26:0/3988442777 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a80ad7b0 0x7fd7a80d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.267+0000 7fd7acdd5640 1 --2- 172.21.15.26:0/3988442777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 0x7fd7a80b4370 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.267+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3988442777 >> 172.21.15.26:0/3988442777 conn(0x7fd7a806b540 msgr2=0x7fd7a806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.267+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3988442777 shutdown_connections 2024-04-24T23:23:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.267+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3988442777 wait complete. 2024-04-24T23:23:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.268+0000 7fd7acdd5640 1 Processor -- start 2024-04-24T23:23:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.268+0000 7fd7acdd5640 1 -- start start 2024-04-24T23:23:58.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7acdd5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a80ad7b0 0x7fd7a80ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7acdd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 0x7fd7a80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7acdd5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a815e9c0 0x7fd7a806ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7acdd5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a815b1a0 con 0x7fd7a815adc0 2024-04-24T23:23:58.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7acdd5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a80b48b0 con 0x7fd7a80ad7b0 2024-04-24T23:23:58.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7acdd5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a80d0580 con 0x7fd7a815e9c0 2024-04-24T23:23:58.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd79ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 0x7fd7a80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd79ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 0x7fd7a80cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51216/0 (socket says 172.21.15.26:51216) 2024-04-24T23:23:58.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd79ffff640 1 -- 172.21.15.26:0/3278553597 learned_addr learned my addr 172.21.15.26:0/3278553597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7a6d9d640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a815e9c0 0x7fd7a806ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7a659c640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a80ad7b0 0x7fd7a80ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd79ffff640 1 -- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a815e9c0 msgr2=0x7fd7a806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd79ffff640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a815e9c0 0x7fd7a806ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd79ffff640 1 -- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a80ad7b0 msgr2=0x7fd7a80ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd79ffff640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a80ad7b0 0x7fd7a80ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd79ffff640 1 -- 172.21.15.26:0/3278553597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a80ce410 con 0x7fd7a815adc0 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7a659c640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a80ad7b0 0x7fd7a80ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd7a6d9d640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a815e9c0 0x7fd7a806ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.269+0000 7fd79ffff640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 0x7fd7a80cdca0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd79000c7f0 tx=0x7fd79000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.270+0000 7fd79effd640 1 -- 172.21.15.26:0/3278553597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790015070 con 0x7fd7a815adc0 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.270+0000 7fd79effd640 1 -- 172.21.15.26:0/3278553597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd790004240 con 0x7fd7a815adc0 2024-04-24T23:23:58.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.270+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a80ce6f0 con 0x7fd7a815adc0 2024-04-24T23:23:58.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.270+0000 7fd79effd640 1 -- 172.21.15.26:0/3278553597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790003610 con 0x7fd7a815adc0 2024-04-24T23:23:58.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.270+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a80cecb0 con 0x7fd7a815adc0 2024-04-24T23:23:58.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.271+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a80d0040 con 0x7fd7a815adc0 2024-04-24T23:23:58.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.271+0000 7fd79effd640 1 -- 172.21.15.26:0/3278553597 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7900039c0 con 0x7fd7a815adc0 2024-04-24T23:23:58.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.272+0000 7fd79effd640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd78806be30 0x7fd78806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.272+0000 7fd79effd640 1 -- 172.21.15.26:0/3278553597 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fd79008b2e0 con 0x7fd7a815adc0 2024-04-24T23:23:58.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.272+0000 7fd7a659c640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd78806be30 0x7fd78806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.272+0000 7fd7a659c640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd78806be30 0x7fd78806e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd798034000 tx=0x7fd79802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.276+0000 7fd79effd640 1 -- 172.21.15.26:0/3278553597 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7a80d0040 con 0x7fd7a815adc0 2024-04-24T23:23:58.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.457+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd7a8075610 con 0x7fd7a815adc0 2024-04-24T23:23:58.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.457+0000 7fd79effd640 1 -- 172.21.15.26:0/3278553597 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7fd790059bc0 con 0x7fd7a815adc0 2024-04-24T23:23:58.459 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:23:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd78806be30 msgr2=0x7fd78806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd78806be30 0x7fd78806e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd798034000 tx=0x7fd79802e000 comp rx=0 tx=0).stop 2024-04-24T23:23:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 msgr2=0x7fd7a80cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 0x7fd7a80cdca0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd79000c7f0 tx=0x7fd79000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 shutdown_connections 2024-04-24T23:23:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd78806be30 0x7fd78806e2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7a815e9c0 0x7fd7a806ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a815adc0 0x7fd7a80cdca0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 --2- 172.21.15.26:0/3278553597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7a80ad7b0 0x7fd7a80ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 >> 172.21.15.26:0/3278553597 conn(0x7fd7a806b540 msgr2=0x7fd7a80b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 shutdown_connections 2024-04-24T23:23:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.459+0000 7fd7acdd5640 1 -- 172.21.15.26:0/3278553597 wait complete. 2024-04-24T23:23:58.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:23:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.572+0000 7fc7dfa09640 1 Processor -- start 2024-04-24T23:23:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.572+0000 7fc7dfa09640 1 -- start start 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.573+0000 7fc7dfa09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a480 0x7fc7d80b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.573+0000 7fc7dfa09640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7d815a860 0x7fc7d806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.573+0000 7fc7dfa09640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7d80b2e80 con 0x7fc7d80b4e30 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.573+0000 7fc7dfa09640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc7d80b3120 con 0x7fc7d815a860 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.573+0000 7fc7dfa09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d80b33c0 con 0x7fc7d815a480 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.573+0000 7fc7dd7a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a480 0x7fc7d80b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.573+0000 7fc7dcfa4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc7d815a860 0x7fc7d806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.573+0000 7fc7dd7a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a480 0x7fc7d80b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:60358/0 (socket says 172.21.15.26:60358) 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.573+0000 7fc7dd7a5640 1 -- 172.21.15.26:0/1219505186 learned_addr learned my addr 172.21.15.26:0/1219505186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.574+0000 7fc7dd7a5640 1 -- 172.21.15.26:0/1219505186 >> v2:172.21.15.29:3300/0 conn(0x7fc7d815a860 msgr2=0x7fc7d806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.574+0000 7fc7dd7a5640 1 --2- 172.21.15.26:0/1219505186 >> v2:172.21.15.29:3300/0 conn(0x7fc7d815a860 0x7fc7d806dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.574+0000 7fc7dd7a5640 1 -- 172.21.15.26:0/1219505186 >> v1:172.21.15.29:6789/0 conn(0x7fc7d80b4e30 legacy=0x7fc7d815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.574+0000 7fc7dd7a5640 1 -- 172.21.15.26:0/1219505186 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d80b36a0 con 0x7fc7d815a480 2024-04-24T23:23:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.574+0000 7fc7dd7a5640 1 --2- 172.21.15.26:0/1219505186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a480 0x7fc7d80b48f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009e50 tx=0x7fc7c802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a76f39d550c16f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7d7fff640 1 -- 172.21.15.26:0/1219505186 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8036070 con 0x7fc7d815a480 2024-04-24T23:23:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7d7fff640 1 -- 172.21.15.26:0/1219505186 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c8031600 con 0x7fc7d815a480 2024-04-24T23:23:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7d7fff640 1 -- 172.21.15.26:0/1219505186 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80319d0 con 0x7fc7d815a480 2024-04-24T23:23:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7dfa09640 1 -- 172.21.15.26:0/1219505186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a480 msgr2=0x7fc7d80b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7dfa09640 1 --2- 172.21.15.26:0/1219505186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a480 0x7fc7d80b48f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009e50 tx=0x7fc7c802a680 comp rx=0 tx=0).stop 2024-04-24T23:23:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7dfa09640 1 -- 172.21.15.26:0/1219505186 shutdown_connections 2024-04-24T23:23:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7dfa09640 1 --2- 172.21.15.26:0/1219505186 >> v2:172.21.15.29:3300/0 conn(0x7fc7d815a860 0x7fc7d806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7dfa09640 1 --2- 172.21.15.26:0/1219505186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a480 0x7fc7d80b48f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7dfa09640 1 -- 172.21.15.26:0/1219505186 >> 172.21.15.26:0/1219505186 conn(0x7fc7d806b300 msgr2=0x7fc7d806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7dfa09640 1 -- 172.21.15.26:0/1219505186 shutdown_connections 2024-04-24T23:23:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.575+0000 7fc7dfa09640 1 -- 172.21.15.26:0/1219505186 wait complete. 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.576+0000 7fc7dfa09640 1 Processor -- start 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.576+0000 7fc7dfa09640 1 -- start start 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dfa09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d80b4e30 0x7fc7d814b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dfa09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7d815a480 0x7fc7d8155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dfa09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a860 0x7fc7d81558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dfa09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8072080 con 0x7fc7d80b4e30 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dfa09640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d80b3c60 con 0x7fc7d815a480 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dfa09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d8072200 con 0x7fc7d815a860 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dd7a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d80b4e30 0x7fc7d814b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dd7a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d80b4e30 0x7fc7d814b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51232/0 (socket says 172.21.15.26:51232) 2024-04-24T23:23:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dd7a5640 1 -- 172.21.15.26:0/3167180364 learned_addr learned my addr 172.21.15.26:0/3167180364 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7ddfa6640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a860 0x7fc7d81558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.577+0000 7fc7dcfa4640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7d815a480 0x7fc7d8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7dd7a5640 1 -- 172.21.15.26:0/3167180364 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a860 msgr2=0x7fc7d81558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7dd7a5640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a860 0x7fc7d81558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7dd7a5640 1 -- 172.21.15.26:0/3167180364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7d815a480 msgr2=0x7fc7d8155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7dd7a5640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7d815a480 0x7fc7d8155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7dd7a5640 1 -- 172.21.15.26:0/3167180364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d814bb60 con 0x7fc7d80b4e30 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7dd7a5640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d80b4e30 0x7fc7d814b620 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8034040 tx=0x7fc7c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7d5ffb640 1 -- 172.21.15.26:0/3167180364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8036050 con 0x7fc7d80b4e30 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7d5ffb640 1 -- 172.21.15.26:0/3167180364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c8042070 con 0x7fc7d80b4e30 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d81672c0 con 0x7fc7d80b4e30 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7d5ffb640 1 -- 172.21.15.26:0/3167180364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8009510 con 0x7fc7d80b4e30 2024-04-24T23:23:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.578+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d815b100 con 0x7fc7d80b4e30 2024-04-24T23:23:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.579+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d80694c0 con 0x7fc7d80b4e30 2024-04-24T23:23:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.580+0000 7fc7d5ffb640 1 -- 172.21.15.26:0/3167180364 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc7c8007c70 con 0x7fc7d80b4e30 2024-04-24T23:23:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.580+0000 7fc7d5ffb640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc7c406be30 0x7fc7c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.580+0000 7fc7d5ffb640 1 -- 172.21.15.26:0/3167180364 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fc7c80abd00 con 0x7fc7d80b4e30 2024-04-24T23:23:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.580+0000 7fc7dcfa4640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc7c406be30 0x7fc7c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.581+0000 7fc7dcfa4640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc7c406be30 0x7fc7c406e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8166c80 tx=0x7fc7cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.584+0000 7fc7d5ffb640 1 -- 172.21.15.26:0/3167180364 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc7c8045360 con 0x7fc7d80b4e30 2024-04-24T23:23:58.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.781+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc7d814cd60 con 0x7fc7d80b4e30 2024-04-24T23:23:58.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.782+0000 7fc7d5ffb640 1 -- 172.21.15.26:0/3167180364 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7fc7c807a5f0 con 0x7fc7d80b4e30 2024-04-24T23:23:58.783 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:23:58.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc7c406be30 msgr2=0x7fc7c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc7c406be30 0x7fc7c406e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8166c80 tx=0x7fc7cc008040 comp rx=0 tx=0).stop 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d80b4e30 msgr2=0x7fc7d814b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d80b4e30 0x7fc7d814b620 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8034040 tx=0x7fc7c8004a40 comp rx=0 tx=0).stop 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 shutdown_connections 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc7c406be30 0x7fc7c406e2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7d815a860 0x7fc7d81558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7d815a480 0x7fc7d8155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 --2- 172.21.15.26:0/3167180364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d80b4e30 0x7fc7d814b620 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 >> 172.21.15.26:0/3167180364 conn(0x7fc7d806b300 msgr2=0x7fc7d80cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 shutdown_connections 2024-04-24T23:23:58.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:58.784+0000 7fc7dfa09640 1 -- 172.21.15.26:0/3167180364 wait complete. 2024-04-24T23:23:58.796 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:23:58.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:58.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:23:58.825 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:23:58.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:23:58.825 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:23:58.881 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:23:58.881 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:23:58.881 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:23:58.900 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:23:58.900 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:23:58.901 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:23:58.901 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:23:58.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:23:58.936 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:23:58.937 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:23:58.937 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:23:58.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:23:58.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:23:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.076+0000 7fe0f7e5d640 1 Processor -- start 2024-04-24T23:23:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.076+0000 7fe0f7e5d640 1 -- start start 2024-04-24T23:23:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f7e5d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe0f00b63e0 0x7fe0f0142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f7e5d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f006deb0 con 0x7fe0f0142310 2024-04-24T23:23:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f7e5d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f006e190 con 0x7fe0f0145890 2024-04-24T23:23:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f7e5d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f006dc60 con 0x7fe0f00b63e0 2024-04-24T23:23:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f5bf9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe0f0142310 0x7fe0f00b5cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35152/0 (socket says 172.21.15.26:35152) 2024-04-24T23:23:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f5bf9640 1 -- 172.21.15.26:0/3053079055 learned_addr learned my addr 172.21.15.26:0/3053079055 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1140510302 0 0) 0x7fe0f006deb0 con 0x7fe0f0142310 2024-04-24T23:23:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0d8003520 con 0x7fe0f0142310 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f53f8640 1 --2- 172.21.15.26:0/3053079055 >> v2:172.21.15.29:3300/0 conn(0x7fe0f00b63e0 0x7fe0f0142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 54962127 0 0) 0x7fe0d8003520 con 0x7fe0f0142310 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 >> v2:172.21.15.29:3300/0 conn(0x7fe0f00b63e0 msgr2=0x7fe0f0142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f4bf7640 1 --2- 172.21.15.26:0/3053079055 >> v2:172.21.15.29:3300/0 conn(0x7fe0f00b63e0 0x7fe0f0142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.077+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 >> v1:172.21.15.29:6789/0 conn(0x7fe0f0145890 legacy=0x7fe0f006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f006e470 con 0x7fe0f0142310 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fe0e80019f0 con 0x7fe0f0142310 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 >> v1:172.21.15.26:6789/0 conn(0x7fe0f0142310 legacy=0x7fe0f00b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f4bf7640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d80039b0 0x7fe0d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f4bf7640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0d8004800 0x7fe0d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f4bf7640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d80052e0 0x7fe0d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f006dc60 con 0x7fe0d80052e0 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d800c1b0 con 0x7fe0d80039b0 2024-04-24T23:23:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d800c3b0 con 0x7fe0d8004800 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f63fa640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d80052e0 0x7fe0d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f5bf9640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d80039b0 0x7fe0d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f53f8640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0d8004800 0x7fe0d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f63fa640 1 -- 172.21.15.26:0/3053079055 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0d8004800 msgr2=0x7fe0d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f63fa640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0d8004800 0x7fe0d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f63fa640 1 -- 172.21.15.26:0/3053079055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d80039b0 msgr2=0x7fe0d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f63fa640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d80039b0 0x7fe0d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.078+0000 7fe0f63fa640 1 -- 172.21.15.26:0/3053079055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d8003520 con 0x7fe0d80052e0 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.079+0000 7fe0f63fa640 1 -- 172.21.15.26:0/3053079055 reap_dead start 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.079+0000 7fe0f63fa640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d80052e0 0x7fe0d800b9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0003d80 tx=0x7fe0d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515d424cbde3eb51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.079+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0037070 con 0x7fe0d80052e0 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.079+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e002a9d0 con 0x7fe0d80052e0 2024-04-24T23:23:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.079+0000 7fe0f4bf7640 1 -- 172.21.15.26:0/3053079055 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0003980 con 0x7fe0d80052e0 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.079+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/3053079055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d80052e0 msgr2=0x7fe0d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.079+0000 7fe0f7e5d640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d80052e0 0x7fe0d800b9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0003d80 tx=0x7fe0d8005830 comp rx=0 tx=0).stop 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.080+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/3053079055 shutdown_connections 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.080+0000 7fe0f7e5d640 1 --2- 172.21.15.26:0/3053079055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d80052e0 0x7fe0d800b9c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.080+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/3053079055 >> 172.21.15.26:0/3053079055 conn(0x7fe0f00729a0 msgr2=0x7fe0f006b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.080+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/3053079055 shutdown_connections 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.080+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/3053079055 wait complete. 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.081+0000 7fe0f7e5d640 1 Processor -- start 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.081+0000 7fe0f7e5d640 1 -- start start 2024-04-24T23:23:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f7e5d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d80052e0 0x7fe0f0150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f7e5d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0f015b450 0x7fe0f00b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f7e5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0f00b5180 0x7fe0f014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f7e5d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f01426f0 con 0x7fe0f00b5180 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f7e5d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f006eac0 con 0x7fe0d80052e0 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f7e5d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f0142870 con 0x7fe0f015b450 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f63fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0f00b5180 0x7fe0f014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f63fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0f00b5180 0x7fe0f014f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51252/0 (socket says 172.21.15.26:51252) 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f63fa640 1 -- 172.21.15.26:0/1660792139 learned_addr learned my addr 172.21.15.26:0/1660792139 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f53f8640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0f015b450 0x7fe0f00b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f63fa640 1 -- 172.21.15.26:0/1660792139 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0f015b450 msgr2=0x7fe0f00b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f5bf9640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d80052e0 0x7fe0f0150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f63fa640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0f015b450 0x7fe0f00b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f63fa640 1 -- 172.21.15.26:0/1660792139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d80052e0 msgr2=0x7fe0f0150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f63fa640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d80052e0 0x7fe0f0150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f63fa640 1 -- 172.21.15.26:0/1660792139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f0137dd0 con 0x7fe0f00b5180 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f63fa640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0f00b5180 0x7fe0f014f580 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0003d80 tx=0x7fe0e00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0ee7fc640 1 -- 172.21.15.26:0/1660792139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0042070 con 0x7fe0f00b5180 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0ee7fc640 1 -- 172.21.15.26:0/1660792139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e0032a40 con 0x7fe0f00b5180 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f015b8b0 con 0x7fe0f00b5180 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0ee7fc640 1 -- 172.21.15.26:0/1660792139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0037040 con 0x7fe0f00b5180 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.082+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f0138350 con 0x7fe0f00b5180 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.083+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f00694c0 con 0x7fe0f00b5180 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.084+0000 7fe0ee7fc640 1 -- 172.21.15.26:0/1660792139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0e0045070 con 0x7fe0f00b5180 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.084+0000 7fe0ee7fc640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0dc06be60 0x7fe0dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.084+0000 7fe0ee7fc640 1 -- 172.21.15.26:0/1660792139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fe0e00a9b90 con 0x7fe0f00b5180 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.084+0000 7fe0f5bf9640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0dc06be60 0x7fe0dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.085+0000 7fe0f5bf9640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0dc06be60 0x7fe0dc06e320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8004b60 tx=0x7fe0e8006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.088+0000 7fe0ee7fc640 1 -- 172.21.15.26:0/1660792139 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe0e0077f40 con 0x7fe0f00b5180 2024-04-24T23:23:59.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.307+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0f00ce750 con 0x7fe0f00b5180 2024-04-24T23:23:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.308+0000 7fe0ee7fc640 1 -- 172.21.15.26:0/1660792139 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5222 (secure 0 0 0) 0x7fe0e007c240 con 0x7fe0f00b5180 2024-04-24T23:23:59.309 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:59.309 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:40.866214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5742,"mds_1":5329,"mds_2":5081},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5081":{"gid":5081,"name":"b","rank":2,"incarnation":36,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742,5329,5081]},"id":3}]} 2024-04-24T23:23:59.310 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 42 2024-04-24T23:23:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0dc06be60 msgr2=0x7fe0dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0dc06be60 0x7fe0dc06e320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8004b60 tx=0x7fe0e8006040 comp rx=0 tx=0).stop 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0f00b5180 msgr2=0x7fe0f014f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0f00b5180 0x7fe0f014f580 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0003d80 tx=0x7fe0e00036f0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 shutdown_connections 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe0dc06be60 0x7fe0dc06e320 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0f00b5180 0x7fe0f014f580 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe0f015b450 0x7fe0f00b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 --2- 172.21.15.26:0/1660792139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d80052e0 0x7fe0f0150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 >> 172.21.15.26:0/1660792139 conn(0x7fe0f00729a0 msgr2=0x7fe0f0144270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 shutdown_connections 2024-04-24T23:23:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.310+0000 7fe0f7e5d640 1 -- 172.21.15.26:0/1660792139 wait complete. 2024-04-24T23:23:59.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:23:59.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:23:59.368 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:23:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.424+0000 7f674082a640 1 Processor -- start 2024-04-24T23:23:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.424+0000 7f674082a640 1 -- start start 2024-04-24T23:23:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f674082a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f673c06d350 0x7f673c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f674082a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f673c0cd660 0x7f673c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f674082a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673c0cfbc0 con 0x7f673c145910 2024-04-24T23:23:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f674082a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f673c145cf0 con 0x7f673c0cd660 2024-04-24T23:23:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f674082a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f673c06d7f0 con 0x7f673c06d350 2024-04-24T23:23:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f673ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f673c0cd660 0x7f673c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f673ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f673c0cd660 0x7f673c0cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51262/0 (socket says 172.21.15.26:51262) 2024-04-24T23:23:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f673ad9d640 1 -- 172.21.15.26:0/2456398772 learned_addr learned my addr 172.21.15.26:0/2456398772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f673ad9d640 1 -- 172.21.15.26:0/2456398772 >> v2:172.21.15.29:3300/0 conn(0x7f673c06d350 msgr2=0x7f673c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f6731d9b640 1 --2- 172.21.15.26:0/2456398772 >> v2:172.21.15.29:3300/0 conn(0x7f673c06d350 0x7f673c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f673ad9d640 1 --2- 172.21.15.26:0/2456398772 >> v2:172.21.15.29:3300/0 conn(0x7f673c06d350 0x7f673c06e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f673ad9d640 1 -- 172.21.15.26:0/2456398772 >> v1:172.21.15.29:6789/0 conn(0x7f673c145910 legacy=0x7f673c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.425+0000 7f673ad9d640 1 -- 172.21.15.26:0/2456398772 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c06dad0 con 0x7f673c0cd660 2024-04-24T23:23:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f673ad9d640 1 --2- 172.21.15.26:0/2456398772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 0x7f673c0cdaf0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6734009e50 tx=0x7f673402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5c3f7645266ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f6739d9b640 1 -- 172.21.15.26:0/2456398772 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734036070 con 0x7f673c0cd660 2024-04-24T23:23:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f6739d9b640 1 -- 172.21.15.26:0/2456398772 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6734031600 con 0x7f673c0cd660 2024-04-24T23:23:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f6739d9b640 1 -- 172.21.15.26:0/2456398772 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673402ae60 con 0x7f673c0cd660 2024-04-24T23:23:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f674082a640 1 -- 172.21.15.26:0/2456398772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 msgr2=0x7f673c0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f674082a640 1 --2- 172.21.15.26:0/2456398772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 0x7f673c0cdaf0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6734009e50 tx=0x7f673402a680 comp rx=0 tx=0).stop 2024-04-24T23:23:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f674082a640 1 -- 172.21.15.26:0/2456398772 shutdown_connections 2024-04-24T23:23:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f674082a640 1 --2- 172.21.15.26:0/2456398772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 0x7f673c0cdaf0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f674082a640 1 --2- 172.21.15.26:0/2456398772 >> v2:172.21.15.29:3300/0 conn(0x7f673c06d350 0x7f673c06e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.426+0000 7f674082a640 1 -- 172.21.15.26:0/2456398772 >> 172.21.15.26:0/2456398772 conn(0x7f673c06b300 msgr2=0x7f673c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.427+0000 7f674082a640 1 -- 172.21.15.26:0/2456398772 shutdown_connections 2024-04-24T23:23:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.427+0000 7f674082a640 1 -- 172.21.15.26:0/2456398772 wait complete. 2024-04-24T23:23:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.428+0000 7f674082a640 1 Processor -- start 2024-04-24T23:23:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.428+0000 7f674082a640 1 -- start start 2024-04-24T23:23:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f674082a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f673c06d350 0x7f673c13e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f674082a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 0x7f673c13ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f674082a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f673c145910 0x7f673c0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f674082a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f673c06e370 con 0x7f673c0cd660 2024-04-24T23:23:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f674082a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f673c06e090 con 0x7f673c06d350 2024-04-24T23:23:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f674082a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f673c15d770 con 0x7f673c145910 2024-04-24T23:23:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f6731d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 0x7f673c13ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f6731d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 0x7f673c13ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51268/0 (socket says 172.21.15.26:51268) 2024-04-24T23:23:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f6731d9b640 1 -- 172.21.15.26:0/574036795 learned_addr learned my addr 172.21.15.26:0/574036795 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f673a59c640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f673c06d350 0x7f673c13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f673ad9d640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f673c145910 0x7f673c0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f6731d9b640 1 -- 172.21.15.26:0/574036795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f673c145910 msgr2=0x7f673c0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f6731d9b640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f673c145910 0x7f673c0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f6731d9b640 1 -- 172.21.15.26:0/574036795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f673c06d350 msgr2=0x7f673c13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f6731d9b640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f673c06d350 0x7f673c13e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f6731d9b640 1 -- 172.21.15.26:0/574036795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c1401f0 con 0x7f673c0cd660 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.429+0000 7f6731d9b640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 0x7f673c13ea20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f672400c820 tx=0x7f672400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.430+0000 7f67337fe640 1 -- 172.21.15.26:0/574036795 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724015070 con 0x7f673c0cd660 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.430+0000 7f67337fe640 1 -- 172.21.15.26:0/574036795 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6724004240 con 0x7f673c0cd660 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.430+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c0b3f00 con 0x7f673c0cd660 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.430+0000 7f67337fe640 1 -- 172.21.15.26:0/574036795 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672400ce10 con 0x7f673c0cd660 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.430+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c13f430 con 0x7f673c0cd660 2024-04-24T23:23:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.431+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673c0694c0 con 0x7f673c0cd660 2024-04-24T23:23:59.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.431+0000 7f67337fe640 1 -- 172.21.15.26:0/574036795 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67240077d0 con 0x7f673c0cd660 2024-04-24T23:23:59.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.432+0000 7f67337fe640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f671c06be30 0x7f671c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.432+0000 7f67337fe640 1 -- 172.21.15.26:0/574036795 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f672408ad30 con 0x7f673c0cd660 2024-04-24T23:23:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.432+0000 7f673a59c640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f671c06be30 0x7f671c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.432+0000 7f673a59c640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f671c06be30 0x7f671c06e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f672c006fd0 tx=0x7f672c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.436+0000 7f67337fe640 1 -- 172.21.15.26:0/574036795 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f67240596a0 con 0x7f673c0cd660 2024-04-24T23:23:59.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.621+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f673c144cd0 con 0x7f673c0cd660 2024-04-24T23:23:59.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.622+0000 7f67337fe640 1 -- 172.21.15.26:0/574036795 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16705 (secure 0 0 0) 0x7f6724058fa0 con 0x7f673c0cd660 2024-04-24T23:23:59.624 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:23:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":46,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:23:36.893433+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:23:01.453865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:23:02.136971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":4}]},{"pgid":"7.2c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:23:59.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f671c06be30 msgr2=0x7f671c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f671c06be30 0x7f671c06e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f672c006fd0 tx=0x7f672c008040 comp rx=0 tx=0).stop 2024-04-24T23:23:59.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 msgr2=0x7f673c13ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 0x7f673c13ea20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f672400c820 tx=0x7f672400cce0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 shutdown_connections 2024-04-24T23:23:59.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f671c06be30 0x7f671c06e2f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f673c145910 0x7f673c0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f673c0cd660 0x7f673c13ea20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 --2- 172.21.15.26:0/574036795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f673c06d350 0x7f673c13e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 >> 172.21.15.26:0/574036795 conn(0x7f673c06b300 msgr2=0x7f673c1443b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 shutdown_connections 2024-04-24T23:23:59.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.625+0000 7f674082a640 1 -- 172.21.15.26:0/574036795 wait complete. 2024-04-24T23:23:59.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:23:01.453865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:23:02.136971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:23:59.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:23:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.779+0000 7f38081ed640 1 Processor -- start 2024-04-24T23:23:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.780+0000 7f38081ed640 1 -- start start 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f38081ed640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f380015adc0 0x7f38000b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f38081ed640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380015ef60 con 0x7f38000d0750 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f38081ed640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380015f240 con 0x7f38000ad7b0 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f38081ed640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f380015f520 con 0x7f380015adc0 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f380678a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38000d0750 0x7f3800072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35156/0 (socket says 172.21.15.26:35156) 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f380678a640 1 -- 172.21.15.26:0/2345674964 learned_addr learned my addr 172.21.15.26:0/2345674964 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3805f89640 1 --2- 172.21.15.26:0/2345674964 >> v2:172.21.15.29:3300/0 conn(0x7f380015adc0 0x7f38000b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3673292494 0 0) 0x7f380015ef60 con 0x7f38000d0750 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37ec003520 con 0x7f38000d0750 2024-04-24T23:23:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3332579091 0 0) 0x7f37ec003520 con 0x7f38000d0750 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 >> v2:172.21.15.29:3300/0 conn(0x7f380015adc0 msgr2=0x7f38000b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 --2- 172.21.15.26:0/2345674964 >> v2:172.21.15.29:3300/0 conn(0x7f380015adc0 0x7f38000b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 >> v1:172.21.15.29:6789/0 conn(0x7f38000ad7b0 legacy=0x7f38000d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380015f800 con 0x7f38000d0750 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f37f80019f0 con 0x7f38000d0750 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 >> v1:172.21.15.26:6789/0 conn(0x7f38000d0750 legacy=0x7f3800072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f37ec0039b0 0x7f37ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec004800 0x7f37ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37ec0051c0 0x7f37ec00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f380015f520 con 0x7f37ec0051c0 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec00e370 con 0x7f37ec004800 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec00e570 con 0x7f37ec0039b0 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.781+0000 7f380678a640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37ec0051c0 0x7f37ec00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.782+0000 7f3805f89640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f37ec0039b0 0x7f37ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.782+0000 7f3805788640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec004800 0x7f37ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.782+0000 7f380678a640 1 -- 172.21.15.26:0/2345674964 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f37ec0039b0 msgr2=0x7f37ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.782+0000 7f380678a640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f37ec0039b0 0x7f37ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.782+0000 7f380678a640 1 -- 172.21.15.26:0/2345674964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec004800 msgr2=0x7f37ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.782+0000 7f380678a640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec004800 0x7f37ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.782+0000 7f380678a640 1 -- 172.21.15.26:0/2345674964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec003520 con 0x7f37ec0051c0 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.782+0000 7f380678a640 1 -- 172.21.15.26:0/2345674964 reap_dead start 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f380678a640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37ec0051c0 0x7f37ec00dbd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f37f800a810 tx=0x7f37ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bce49a7f967d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f800a460 con 0x7f37ec0051c0 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f802f070 con 0x7f37ec0051c0 2024-04-24T23:23:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f3804f87640 1 -- 172.21.15.26:0/2345674964 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f802bc90 con 0x7f37ec0051c0 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f38081ed640 1 -- 172.21.15.26:0/2345674964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37ec0051c0 msgr2=0x7f37ec00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f38081ed640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37ec0051c0 0x7f37ec00dbd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f37f800a810 tx=0x7f37ec005830 comp rx=0 tx=0).stop 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f38081ed640 1 -- 172.21.15.26:0/2345674964 shutdown_connections 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f38081ed640 1 --2- 172.21.15.26:0/2345674964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37ec0051c0 0x7f37ec00dbd0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f38081ed640 1 -- 172.21.15.26:0/2345674964 >> 172.21.15.26:0/2345674964 conn(0x7f380006b540 msgr2=0x7f380006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f38081ed640 1 -- 172.21.15.26:0/2345674964 shutdown_connections 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.783+0000 7f38081ed640 1 -- 172.21.15.26:0/2345674964 wait complete. 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.784+0000 7f38081ed640 1 Processor -- start 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f38081ed640 1 -- start start 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f38081ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0051c0 0x7f3800146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f38081ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38001f3e20 0x7f3800159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f38081ed640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38001f42a0 0x7f3800156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f38081ed640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f380015e9c0 con 0x7f38001f3e20 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f38081ed640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f380015b1a0 con 0x7f37ec0051c0 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f38081ed640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f380015eb40 con 0x7f38001f42a0 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f3805788640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38001f3e20 0x7f3800159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f3805788640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38001f3e20 0x7f3800159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51300/0 (socket says 172.21.15.26:51300) 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.785+0000 7f3805788640 1 -- 172.21.15.26:0/3492633962 learned_addr learned my addr 172.21.15.26:0/3492633962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f3805f89640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0051c0 0x7f3800146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f380678a640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38001f42a0 0x7f3800156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f3805788640 1 -- 172.21.15.26:0/3492633962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38001f42a0 msgr2=0x7f3800156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f3805788640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38001f42a0 0x7f3800156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f3805788640 1 -- 172.21.15.26:0/3492633962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0051c0 msgr2=0x7f3800146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f3805788640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0051c0 0x7f3800146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f3805788640 1 -- 172.21.15.26:0/3492633962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380015a310 con 0x7f38001f3e20 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f3805788640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38001f3e20 0x7f3800159870 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f37f000f5c0 tx=0x7f37f000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f37f67fc640 1 -- 172.21.15.26:0/3492633962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f00052e0 con 0x7f38001f3e20 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f37f67fc640 1 -- 172.21.15.26:0/3492633962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f0018070 con 0x7f38001f3e20 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380009bb80 con 0x7f38001f3e20 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f37f67fc640 1 -- 172.21.15.26:0/3492633962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0003830 con 0x7f38001f3e20 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.786+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380009c140 con 0x7f38001f3e20 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.787+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3800075610 con 0x7f38001f3e20 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.787+0000 7f37f67fc640 1 -- 172.21.15.26:0/3492633962 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37f00039d0 con 0x7f38001f3e20 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.788+0000 7f37f67fc640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e406be60 0x7f37e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.788+0000 7f3805f89640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e406be60 0x7f37e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.788+0000 7f37f67fc640 1 -- 172.21.15.26:0/3492633962 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f37f008ac00 con 0x7f38001f3e20 2024-04-24T23:23:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.788+0000 7f3805f89640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e406be60 0x7f37e406e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009a80 tx=0x7f37fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.792+0000 7f37f67fc640 1 -- 172.21.15.26:0/3492633962 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f37f0091080 con 0x7f38001f3e20 2024-04-24T23:23:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.978+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f38000719f0 con 0x7f38001f3e20 2024-04-24T23:23:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.978+0000 7f37f67fc640 1 -- 172.21.15.26:0/3492633962 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f37f0059600 con 0x7f38001f3e20 2024-04-24T23:23:59.979 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e406be60 msgr2=0x7f37e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e406be60 0x7f37e406e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009a80 tx=0x7f37fc008040 comp rx=0 tx=0).stop 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38001f3e20 msgr2=0x7f3800159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38001f3e20 0x7f3800159870 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f37f000f5c0 tx=0x7f37f000fa80 comp rx=0 tx=0).stop 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 shutdown_connections 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e406be60 0x7f37e406e320 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38001f42a0 0x7f3800156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38001f3e20 0x7f3800159870 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 --2- 172.21.15.26:0/3492633962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37ec0051c0 0x7f3800146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 >> 172.21.15.26:0/3492633962 conn(0x7f380006b540 msgr2=0x7f38000afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 shutdown_connections 2024-04-24T23:23:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:23:59.980+0000 7f38081ed640 1 -- 172.21.15.26:0/3492633962 wait complete. 2024-04-24T23:23:59.993 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:24:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.096+0000 7fe12eeba640 1 Processor -- start 2024-04-24T23:24:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.096+0000 7fe12eeba640 1 -- start start 2024-04-24T23:24:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.096+0000 7fe12eeba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1280d2c30 0x7fe1280cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.096+0000 7fe12eeba640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1280d2120 con 0x7fe12815e750 2024-04-24T23:24:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.096+0000 7fe12eeba640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1280b3740 con 0x7fe12815e370 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.096+0000 7fe12eeba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe1280d1510 con 0x7fe1280d2c30 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe127fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe12815e750 0x7fe1280cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35160/0 (socket says 172.21.15.26:35160) 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe127fff640 1 -- 172.21.15.26:0/949263890 learned_addr learned my addr 172.21.15.26:0/949263890 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45218958 0 0) 0x7fe1280d2120 con 0x7fe12815e750 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe12d457640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1280d2c30 0x7fe1280cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe118003520 con 0x7fe12815e750 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 197170745 0 0) 0x7fe118003520 con 0x7fe12815e750 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1280d2c30 msgr2=0x7fe1280cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1280d2c30 0x7fe1280cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 >> v1:172.21.15.29:6789/0 conn(0x7fe12815e370 legacy=0x7fe1280d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1280b3a20 con 0x7fe12815e750 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fe114001670 con 0x7fe12815e750 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 >> v1:172.21.15.26:6789/0 conn(0x7fe12815e750 legacy=0x7fe1280cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1180039e0 0x7fe118003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe118004800 0x7fe118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1280d2c30 0x7fe1180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe11800ba70 con 0x7fe1180039e0 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe11800bc20 con 0x7fe118004800 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe11800be20 con 0x7fe1280d2c30 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe12cc56640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1180039e0 0x7fe118003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe12d457640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1280d2c30 0x7fe1180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.097+0000 7fe127fff640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe118004800 0x7fe118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.098+0000 7fe12cc56640 1 -- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1280d2c30 msgr2=0x7fe1180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.098+0000 7fe12cc56640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1280d2c30 0x7fe1180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.098+0000 7fe12cc56640 1 -- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe118004800 msgr2=0x7fe118004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.098+0000 7fe12cc56640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe118004800 0x7fe118004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.098+0000 7fe12cc56640 1 -- 172.21.15.26:0/949263890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe118003520 con 0x7fe1180039e0 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.098+0000 7fe12cc56640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1180039e0 0x7fe118003e00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe11c001720 tx=0x7fe118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad86642fee20450f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.098+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c001370 con 0x7fe1180039e0 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.098+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c0052d0 con 0x7fe1180039e0 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.099+0000 7fe1277fe640 1 -- 172.21.15.26:0/949263890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c02ab90 con 0x7fe1180039e0 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.099+0000 7fe12eeba640 1 -- 172.21.15.26:0/949263890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1180039e0 msgr2=0x7fe118003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.099+0000 7fe12eeba640 1 --2- 172.21.15.26:0/949263890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1180039e0 0x7fe118003e00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe11c001720 tx=0x7fe118005830 comp rx=0 tx=0).stop 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.099+0000 7fe12d457640 1 -- 172.21.15.26:0/949263890 reap_dead start 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.099+0000 7fe12eeba640 1 -- 172.21.15.26:0/949263890 shutdown_connections 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.099+0000 7fe12eeba640 1 -- 172.21.15.26:0/949263890 >> 172.21.15.26:0/949263890 conn(0x7fe1280d40f0 msgr2=0x7fe1280d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.099+0000 7fe12eeba640 1 -- 172.21.15.26:0/949263890 shutdown_connections 2024-04-24T23:24:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.099+0000 7fe12eeba640 1 -- 172.21.15.26:0/949263890 wait complete. 2024-04-24T23:24:00.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.100+0000 7fe12eeba640 1 Processor -- start 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.100+0000 7fe12eeba640 1 -- start start 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.101+0000 7fe12eeba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1180039e0 0x7fe1280b98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.101+0000 7fe12eeba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1281f7440 0x7fe128160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.101+0000 7fe12eeba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1280cb8f0 0x7fe1280cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.101+0000 7fe12eeba640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280b5230 con 0x7fe1281f7440 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.101+0000 7fe12eeba640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280d1ed0 con 0x7fe1280cb8f0 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.101+0000 7fe12eeba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe1280b53b0 con 0x7fe1180039e0 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.101+0000 7fe127fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1281f7440 0x7fe128160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.101+0000 7fe127fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1281f7440 0x7fe128160f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51316/0 (socket says 172.21.15.26:51316) 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.101+0000 7fe127fff640 1 -- 172.21.15.26:0/1494552545 learned_addr learned my addr 172.21.15.26:0/1494552545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe12cc56640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1180039e0 0x7fe1280b98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe12d457640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1280cb8f0 0x7fe1280cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe127fff640 1 -- 172.21.15.26:0/1494552545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1180039e0 msgr2=0x7fe1280b98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe127fff640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1180039e0 0x7fe1280b98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe127fff640 1 -- 172.21.15.26:0/1494552545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1280cb8f0 msgr2=0x7fe1280cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe127fff640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1280cb8f0 0x7fe1280cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe127fff640 1 -- 172.21.15.26:0/1494552545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1281f7bb0 con 0x7fe1281f7440 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe12d457640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1280cb8f0 0x7fe1280cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe127fff640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1281f7440 0x7fe128160f80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe114000ff0 tx=0x7fe114007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe1257fa640 1 -- 172.21.15.26:0/1494552545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11400f780 con 0x7fe1281f7440 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe1257fa640 1 -- 172.21.15.26:0/1494552545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe114018070 con 0x7fe1281f7440 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe1257fa640 1 -- 172.21.15.26:0/1494552545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11400fb80 con 0x7fe1281f7440 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1281f7e90 con 0x7fe1281f7440 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.102+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1280cb3d0 con 0x7fe1281f7440 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.103+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1280cb5d0 con 0x7fe1281f7440 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.104+0000 7fe1257fa640 1 -- 172.21.15.26:0/1494552545 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe11400fd20 con 0x7fe1281f7440 2024-04-24T23:24:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.104+0000 7fe1257fa640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10c06be30 0x7fe10c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.105+0000 7fe12cc56640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10c06be30 0x7fe10c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.105+0000 7fe1257fa640 1 -- 172.21.15.26:0/1494552545 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fe11408ad70 con 0x7fe1281f7440 2024-04-24T23:24:00.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.105+0000 7fe12cc56640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10c06be30 0x7fe10c06e2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe11c032000 tx=0x7fe11c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:00.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.109+0000 7fe1257fa640 1 -- 172.21.15.26:0/1494552545 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1280cb5d0 con 0x7fe1281f7440 2024-04-24T23:24:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.294+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe128075610 con 0x7fe1281f7440 2024-04-24T23:24:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.294+0000 7fe1257fa640 1 -- 172.21.15.26:0/1494552545 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7fe1140596e0 con 0x7fe1281f7440 2024-04-24T23:24:00.296 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10c06be30 msgr2=0x7fe10c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10c06be30 0x7fe10c06e2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe11c032000 tx=0x7fe11c02e000 comp rx=0 tx=0).stop 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1281f7440 msgr2=0x7fe128160f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1281f7440 0x7fe128160f80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe114000ff0 tx=0x7fe114007430 comp rx=0 tx=0).stop 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 shutdown_connections 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe10c06be30 0x7fe10c06e2f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1280cb8f0 0x7fe1280cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1281f7440 0x7fe128160f80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 --2- 172.21.15.26:0/1494552545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1180039e0 0x7fe1280b98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 >> 172.21.15.26:0/1494552545 conn(0x7fe1280d40f0 msgr2=0x7fe1280cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 shutdown_connections 2024-04-24T23:24:00.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.296+0000 7fe12eeba640 1 -- 172.21.15.26:0/1494552545 wait complete. 2024-04-24T23:24:00.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:24:00.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.412+0000 7f7c966f5640 1 Processor -- start 2024-04-24T23:24:00.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.412+0000 7f7c966f5640 1 -- start start 2024-04-24T23:24:00.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c966f5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c900d0750 0x7f7c900acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c966f5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c900aeb10 con 0x7f7c9015be40 2024-04-24T23:24:00.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c966f5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c900aedf0 con 0x7f7c900b4ce0 2024-04-24T23:24:00.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c966f5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7c9009df20 con 0x7f7c900d0750 2024-04-24T23:24:00.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c8f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c900d0750 0x7f7c900acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c8ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7c9015be40 0x7f7c900d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35170/0 (socket says 172.21.15.26:35170) 2024-04-24T23:24:00.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c8f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c900d0750 0x7f7c900acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51324/0 (socket says 172.21.15.26:51324) 2024-04-24T23:24:00.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c8ffff640 1 -- 172.21.15.26:0/3357311299 learned_addr learned my addr 172.21.15.26:0/3357311299 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:00.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c8f7fe640 1 -- 172.21.15.26:0/3357311299 >> v1:172.21.15.29:6789/0 conn(0x7f7c900b4ce0 legacy=0x7f7c9015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:00.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c8f7fe640 1 -- 172.21.15.26:0/3357311299 >> v1:172.21.15.26:6789/0 conn(0x7f7c9015be40 legacy=0x7f7c900d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c8effd640 1 -- 172.21.15.26:0/3357311299 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2433461733 0 0) 0x7f7c900aeb10 con 0x7f7c9015be40 2024-04-24T23:24:00.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.413+0000 7f7c8f7fe640 1 -- 172.21.15.26:0/3357311299 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c900af0d0 con 0x7f7c900d0750 2024-04-24T23:24:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.414+0000 7f7c8f7fe640 1 --2- 172.21.15.26:0/3357311299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900d0750 0x7f7c900acd30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7c840097a0 tx=0x7f7c8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1e8d0d830475f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.414+0000 7f7c8effd640 1 -- 172.21.15.26:0/3357311299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84037070 con 0x7f7c900d0750 2024-04-24T23:24:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.414+0000 7f7c8effd640 1 -- 172.21.15.26:0/3357311299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c84032440 con 0x7f7c900d0750 2024-04-24T23:24:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.414+0000 7f7c8effd640 1 -- 172.21.15.26:0/3357311299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84032810 con 0x7f7c900d0750 2024-04-24T23:24:00.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.414+0000 7f7c966f5640 1 -- 172.21.15.26:0/3357311299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900d0750 msgr2=0x7f7c900acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.414+0000 7f7c966f5640 1 --2- 172.21.15.26:0/3357311299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900d0750 0x7f7c900acd30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7c840097a0 tx=0x7f7c8402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.415+0000 7f7c966f5640 1 -- 172.21.15.26:0/3357311299 shutdown_connections 2024-04-24T23:24:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.415+0000 7f7c966f5640 1 --2- 172.21.15.26:0/3357311299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900d0750 0x7f7c900acd30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.415+0000 7f7c966f5640 1 -- 172.21.15.26:0/3357311299 >> 172.21.15.26:0/3357311299 conn(0x7f7c9006b540 msgr2=0x7f7c9006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.415+0000 7f7c966f5640 1 -- 172.21.15.26:0/3357311299 shutdown_connections 2024-04-24T23:24:00.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.415+0000 7f7c966f5640 1 -- 172.21.15.26:0/3357311299 wait complete. 2024-04-24T23:24:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.416+0000 7f7c966f5640 1 Processor -- start 2024-04-24T23:24:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.416+0000 7f7c966f5640 1 -- start start 2024-04-24T23:24:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c966f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900b4ce0 0x7f7c90148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c966f5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c900d0750 0x7f7c901f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c966f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7c9015be40 0x7f7c90146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c966f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9006cd10 con 0x7f7c900b4ce0 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c966f5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9006cb90 con 0x7f7c900d0750 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c966f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9015c930 con 0x7f7c9015be40 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900b4ce0 0x7f7c90148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900b4ce0 0x7f7c90148260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51326/0 (socket says 172.21.15.26:51326) 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8ffff640 1 -- 172.21.15.26:0/550375587 learned_addr learned my addr 172.21.15.26:0/550375587 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8ffff640 1 -- 172.21.15.26:0/550375587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7c9015be40 msgr2=0x7f7c90146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8ffff640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7c9015be40 0x7f7c90146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8ffff640 1 -- 172.21.15.26:0/550375587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c900d0750 msgr2=0x7f7c901f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8ffff640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c900d0750 0x7f7c901f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8ffff640 1 -- 172.21.15.26:0/550375587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c901f1610 con 0x7f7c900b4ce0 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8ffff640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900b4ce0 0x7f7c90148260 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c430 tx=0x7f7c8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8cff9640 1 -- 172.21.15.26:0/550375587 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c800074e0 con 0x7f7c900b4ce0 2024-04-24T23:24:00.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8cff9640 1 -- 172.21.15.26:0/550375587 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c800047e0 con 0x7f7c900b4ce0 2024-04-24T23:24:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c901487a0 con 0x7f7c900b4ce0 2024-04-24T23:24:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c8cff9640 1 -- 172.21.15.26:0/550375587 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80004bd0 con 0x7f7c900b4ce0 2024-04-24T23:24:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.417+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c90148d50 con 0x7f7c900b4ce0 2024-04-24T23:24:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.418+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c90075610 con 0x7f7c900b4ce0 2024-04-24T23:24:00.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.419+0000 7f7c8cff9640 1 -- 172.21.15.26:0/550375587 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c80011070 con 0x7f7c900b4ce0 2024-04-24T23:24:00.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.419+0000 7f7c8cff9640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c7c06be30 0x7f7c7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.419+0000 7f7c8cff9640 1 -- 172.21.15.26:0/550375587 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f7c8008aa50 con 0x7f7c900b4ce0 2024-04-24T23:24:00.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.420+0000 7f7c8f7fe640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c7c06be30 0x7f7c7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.420+0000 7f7c8f7fe640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c7c06be30 0x7f7c7c06e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7c84034000 tx=0x7f7c8402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:00.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.424+0000 7f7c8cff9640 1 -- 172.21.15.26:0/550375587 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7c80059450 con 0x7f7c900b4ce0 2024-04-24T23:24:00.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.605+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c90074eb0 con 0x7f7c900b4ce0 2024-04-24T23:24:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.606+0000 7f7c8cff9640 1 -- 172.21.15.26:0/550375587 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7f7c80058d50 con 0x7f7c900b4ce0 2024-04-24T23:24:00.607 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c7c06be30 msgr2=0x7f7c7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c7c06be30 0x7f7c7c06e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7c84034000 tx=0x7f7c8402c000 comp rx=0 tx=0).stop 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900b4ce0 msgr2=0x7f7c90148260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900b4ce0 0x7f7c90148260 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c430 tx=0x7f7c8000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 shutdown_connections 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7c7c06be30 0x7f7c7c06e2f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7c9015be40 0x7f7c90146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c900d0750 0x7f7c901f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 --2- 172.21.15.26:0/550375587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c900b4ce0 0x7f7c90148260 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 >> 172.21.15.26:0/550375587 conn(0x7f7c9006b540 msgr2=0x7f7c9006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 shutdown_connections 2024-04-24T23:24:00.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.608+0000 7f7c966f5640 1 -- 172.21.15.26:0/550375587 wait complete. 2024-04-24T23:24:00.620 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.720+0000 7f1712516640 1 Processor -- start 2024-04-24T23:24:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.720+0000 7f1712516640 1 -- start start 2024-04-24T23:24:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.720+0000 7f1712516640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c0d4870 0x7f170c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.720+0000 7f1712516640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f170c0b3130 0x7f170c0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.720+0000 7f1712516640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170c0d32f0 con 0x7f170c15e570 2024-04-24T23:24:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.720+0000 7f1712516640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f170c0d1ab0 con 0x7f170c0b3130 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.720+0000 7f1712516640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f170c0d2ae0 con 0x7f170c0d4870 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.721+0000 7f1710ab3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f170c0b3130 0x7f170c0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.721+0000 7f1710ab3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f170c0b3130 0x7f170c0b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51330/0 (socket says 172.21.15.26:51330) 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.721+0000 7f1710ab3640 1 -- 172.21.15.26:0/3630420747 learned_addr learned my addr 172.21.15.26:0/3630420747 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.721+0000 7f170b7fe640 1 --2- 172.21.15.26:0/3630420747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c0d4870 0x7f170c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.721+0000 7f1710ab3640 1 -- 172.21.15.26:0/3630420747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c0d4870 msgr2=0x7f170c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.721+0000 7f1710ab3640 1 --2- 172.21.15.26:0/3630420747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c0d4870 0x7f170c0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.721+0000 7f1710ab3640 1 -- 172.21.15.26:0/3630420747 >> v1:172.21.15.29:6789/0 conn(0x7f170c15e570 legacy=0x7f170c0d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.721+0000 7f1710ab3640 1 -- 172.21.15.26:0/3630420747 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c0d0270 con 0x7f170c0b3130 2024-04-24T23:24:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.721+0000 7f170b7fe640 1 --2- 172.21.15.26:0/3630420747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c0d4870 0x7f170c0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f1710ab3640 1 --2- 172.21.15.26:0/3630420747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0b3130 0x7f170c0b4640 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1704009e50 tx=0x7f170402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48312925864565fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f170affd640 1 -- 172.21.15.26:0/3630420747 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704036070 con 0x7f170c0b3130 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f170affd640 1 -- 172.21.15.26:0/3630420747 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1704031600 con 0x7f170c0b3130 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f170affd640 1 -- 172.21.15.26:0/3630420747 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040319d0 con 0x7f170c0b3130 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f1712516640 1 -- 172.21.15.26:0/3630420747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0b3130 msgr2=0x7f170c0b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f1712516640 1 --2- 172.21.15.26:0/3630420747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0b3130 0x7f170c0b4640 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1704009e50 tx=0x7f170402a710 comp rx=0 tx=0).stop 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f1712516640 1 -- 172.21.15.26:0/3630420747 shutdown_connections 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f1712516640 1 --2- 172.21.15.26:0/3630420747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0b3130 0x7f170c0b4640 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f1712516640 1 --2- 172.21.15.26:0/3630420747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c0d4870 0x7f170c0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f1712516640 1 -- 172.21.15.26:0/3630420747 >> 172.21.15.26:0/3630420747 conn(0x7f170c08dbe0 msgr2=0x7f170c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f1712516640 1 -- 172.21.15.26:0/3630420747 shutdown_connections 2024-04-24T23:24:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.722+0000 7f1712516640 1 -- 172.21.15.26:0/3630420747 wait complete. 2024-04-24T23:24:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.724+0000 7f1712516640 1 Processor -- start 2024-04-24T23:24:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.724+0000 7f1712516640 1 -- start start 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.724+0000 7f1712516640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0b3130 0x7f170c0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.724+0000 7f1712516640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0d4870 0x7f170c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f1712516640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c15e570 0x7f170c161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f1712516640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f170c0cfa70 con 0x7f170c0d4870 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f1712516640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f170c0d3010 con 0x7f170c0b3130 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f1712516640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f170c0cfd50 con 0x7f170c15e570 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0d4870 0x7f170c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0d4870 0x7f170c1525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51334/0 (socket says 172.21.15.26:51334) 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f170b7fe640 1 -- 172.21.15.26:0/1994603568 learned_addr learned my addr 172.21.15.26:0/1994603568 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f170b7fe640 1 -- 172.21.15.26:0/1994603568 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c15e570 msgr2=0x7f170c161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f170b7fe640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c15e570 0x7f170c161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f170b7fe640 1 -- 172.21.15.26:0/1994603568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0b3130 msgr2=0x7f170c0cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f170b7fe640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0b3130 0x7f170c0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f170b7fe640 1 -- 172.21.15.26:0/1994603568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c1f7860 con 0x7f170c0d4870 2024-04-24T23:24:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f170b7fe640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0d4870 0x7f170c1525c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e7e0 tx=0x7f16fc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f1708ff9640 1 -- 172.21.15.26:0/1994603568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00f040 con 0x7f170c0d4870 2024-04-24T23:24:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f1708ff9640 1 -- 172.21.15.26:0/1994603568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc004030 con 0x7f170c0d4870 2024-04-24T23:24:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f1708ff9640 1 -- 172.21.15.26:0/1994603568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0043e0 con 0x7f170c0d4870 2024-04-24T23:24:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170c1f7b40 con 0x7f170c0d4870 2024-04-24T23:24:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.725+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f170c1f8100 con 0x7f170c0d4870 2024-04-24T23:24:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.726+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170c0694c0 con 0x7f170c0d4870 2024-04-24T23:24:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.727+0000 7f1708ff9640 1 -- 172.21.15.26:0/1994603568 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f16fc00c6d0 con 0x7f170c0d4870 2024-04-24T23:24:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.727+0000 7f1708ff9640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f16ec06be30 0x7f16ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.727+0000 7f1708ff9640 1 -- 172.21.15.26:0/1994603568 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f16fc08af50 con 0x7f170c0d4870 2024-04-24T23:24:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.727+0000 7f170bfff640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f16ec06be30 0x7f16ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.728+0000 7f170bfff640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f16ec06be30 0x7f16ec06e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16f4000f60 tx=0x7f16f40079d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:00.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.731+0000 7f1708ff9640 1 -- 172.21.15.26:0/1994603568 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f16fc017020 con 0x7f170c0d4870 2024-04-24T23:24:00.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.952+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f170c0caca0 con 0x7f170c0d4870 2024-04-24T23:24:00.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.953+0000 7f1708ff9640 1 -- 172.21.15.26:0/1994603568 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5222 (secure 0 0 0) 0x7f16fc059950 con 0x7f170c0d4870 2024-04-24T23:24:00.954 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:00.954 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:40.866214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5742,"mds_1":5329,"mds_2":5081},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5081":{"gid":5081,"name":"b","rank":2,"incarnation":36,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742,5329,5081]},"id":3}]} 2024-04-24T23:24:00.955 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 42 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f16ec06be30 msgr2=0x7f16ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f16ec06be30 0x7f16ec06e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16f4000f60 tx=0x7f16f40079d0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0d4870 msgr2=0x7f170c1525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0d4870 0x7f170c1525c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e7e0 tx=0x7f16fc00eca0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 shutdown_connections 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f16ec06be30 0x7f16ec06e2f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f170c15e570 0x7f170c161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f170c0d4870 0x7f170c1525c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 --2- 172.21.15.26:0/1994603568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f170c0b3130 0x7f170c0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 >> 172.21.15.26:0/1994603568 conn(0x7f170c08dbe0 msgr2=0x7f170c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:00.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 shutdown_connections 2024-04-24T23:24:00.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:00.955+0000 7f1712516640 1 -- 172.21.15.26:0/1994603568 wait complete. 2024-04-24T23:24:00.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.067+0000 7fcbbe25d640 1 Processor -- start 2024-04-24T23:24:01.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.067+0000 7fcbbe25d640 1 -- start start 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbbe25d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb80b4530 0x7fcbb80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbbe25d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcbb80b0030 0x7fcbb80b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbbe25d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb80b5700 con 0x7fcbb815e380 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbbe25d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb815e760 con 0x7fcbb80b0030 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbbe25d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb80725b0 con 0x7fcbb80b4530 2024-04-24T23:24:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcbb80b0030 0x7fcbb80b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb6ffd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb80b4530 0x7fcbb80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb6ffd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb80b4530 0x7fcbb80b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:60448/0 (socket says 172.21.15.26:60448) 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb7fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcbb80b0030 0x7fcbb80b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:34310/0 (socket says 172.21.15.26:34310) 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb77fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fcbb815e380 0x7fcbb8072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:52098/0 (socket says 172.21.15.26:52098) 2024-04-24T23:24:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb6ffd640 1 -- 172.21.15.26:0/4236725481 learned_addr learned my addr 172.21.15.26:0/4236725481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb7fff640 1 -- 172.21.15.26:0/4236725481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb80b4530 msgr2=0x7fcbb80b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb7fff640 1 --2- 172.21.15.26:0/4236725481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb80b4530 0x7fcbb80b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb7fff640 1 -- 172.21.15.26:0/4236725481 >> v1:172.21.15.29:6789/0 conn(0x7fcbb815e380 legacy=0x7fcbb8072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb7fff640 1 -- 172.21.15.26:0/4236725481 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb80b59e0 con 0x7fcbb80b0030 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.068+0000 7fcbb6ffd640 1 --2- 172.21.15.26:0/4236725481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb80b4530 0x7fcbb80b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.069+0000 7fcbb7fff640 1 --2- 172.21.15.26:0/4236725481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbb80b0030 0x7fcbb80b0460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009d90 tx=0x7fcbb002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a660fa87b88cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.069+0000 7fcbb67fc640 1 -- 172.21.15.26:0/4236725481 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0007740 con 0x7fcbb80b0030 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.069+0000 7fcbb67fc640 1 -- 172.21.15.26:0/4236725481 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb00078e0 con 0x7fcbb80b0030 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.069+0000 7fcbb67fc640 1 -- 172.21.15.26:0/4236725481 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0007cb0 con 0x7fcbb80b0030 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.070+0000 7fcbbe25d640 1 -- 172.21.15.26:0/4236725481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbb80b0030 msgr2=0x7fcbb80b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.070+0000 7fcbbe25d640 1 --2- 172.21.15.26:0/4236725481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbb80b0030 0x7fcbb80b0460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009d90 tx=0x7fcbb002a6f0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.070+0000 7fcbbe25d640 1 -- 172.21.15.26:0/4236725481 shutdown_connections 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.070+0000 7fcbbe25d640 1 --2- 172.21.15.26:0/4236725481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbb80b0030 0x7fcbb80b0460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.070+0000 7fcbbe25d640 1 --2- 172.21.15.26:0/4236725481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb80b4530 0x7fcbb80b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.070+0000 7fcbbe25d640 1 -- 172.21.15.26:0/4236725481 >> 172.21.15.26:0/4236725481 conn(0x7fcbb806e2c0 msgr2=0x7fcbb806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.070+0000 7fcbbe25d640 1 -- 172.21.15.26:0/4236725481 shutdown_connections 2024-04-24T23:24:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.070+0000 7fcbbe25d640 1 -- 172.21.15.26:0/4236725481 wait complete. 2024-04-24T23:24:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.071+0000 7fcbbe25d640 1 Processor -- start 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.071+0000 7fcbbe25d640 1 -- start start 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbbe25d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbb80b0030 0x7fcbb814a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbbe25d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbb80b4530 0x7fcbb81f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbbe25d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb815e380 0x7fcbb80cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbbe25d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb80b09a0 con 0x7fcbb80b0030 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbbe25d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb80b4970 con 0x7fcbb80b4530 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbbe25d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb80ae530 con 0x7fcbb815e380 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbb80b0030 0x7fcbb814a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbb80b0030 0x7fcbb814a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51350/0 (socket says 172.21.15.26:51350) 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb77fe640 1 -- 172.21.15.26:0/1742153802 learned_addr learned my addr 172.21.15.26:0/1742153802 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb7fff640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb815e380 0x7fcbb80cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb6ffd640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbb80b4530 0x7fcbb81f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb77fe640 1 -- 172.21.15.26:0/1742153802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb815e380 msgr2=0x7fcbb80cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb77fe640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb815e380 0x7fcbb80cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb77fe640 1 -- 172.21.15.26:0/1742153802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbb80b4530 msgr2=0x7fcbb81f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb77fe640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbb80b4530 0x7fcbb81f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.072+0000 7fcbb77fe640 1 -- 172.21.15.26:0/1742153802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb814cc40 con 0x7fcbb80b0030 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.073+0000 7fcbb77fe640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbb80b0030 0x7fcbb814a320 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcba800c3c0 tx=0x7fcba800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.073+0000 7fcb9ffff640 1 -- 172.21.15.26:0/1742153802 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba80074e0 con 0x7fcbb80b0030 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.073+0000 7fcb9ffff640 1 -- 172.21.15.26:0/1742153802 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba8007680 con 0x7fcbb80b0030 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.073+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb814cf20 con 0x7fcbb80b0030 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.073+0000 7fcb9ffff640 1 -- 172.21.15.26:0/1742153802 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8004a00 con 0x7fcbb80b0030 2024-04-24T23:24:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.073+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb814d4e0 con 0x7fcbb80b0030 2024-04-24T23:24:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.074+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb80694c0 con 0x7fcbb80b0030 2024-04-24T23:24:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.074+0000 7fcb9ffff640 1 -- 172.21.15.26:0/1742153802 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcba801a020 con 0x7fcbb80b0030 2024-04-24T23:24:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.075+0000 7fcb9ffff640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcba406be30 0x7fcba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.075+0000 7fcbb6ffd640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcba406be30 0x7fcba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.075+0000 7fcb9ffff640 1 -- 172.21.15.26:0/1742153802 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fcba808a440 con 0x7fcbb80b0030 2024-04-24T23:24:01.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.075+0000 7fcbb6ffd640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcba406be30 0x7fcba406e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80cd3c0 tx=0x7fcbac009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.079+0000 7fcb9ffff640 1 -- 172.21.15.26:0/1742153802 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcba80587f0 con 0x7fcbb80b0030 2024-04-24T23:24:01.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.263+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbb815d0c0 con 0x7fcbb80b0030 2024-04-24T23:24:01.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.264+0000 7fcb9ffff640 1 -- 172.21.15.26:0/1742153802 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16705 (secure 0 0 0) 0x7fcba805caf0 con 0x7fcbb80b0030 2024-04-24T23:24:01.266 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:01.266 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":46,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:23:36.893433+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:23:01.453865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:23:02.136971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":4}]},{"pgid":"7.2c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:24:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcba406be30 msgr2=0x7fcba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcba406be30 0x7fcba406e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80cd3c0 tx=0x7fcbac009240 comp rx=0 tx=0).stop 2024-04-24T23:24:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbb80b0030 msgr2=0x7fcbb814a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbb80b0030 0x7fcbb814a320 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcba800c3c0 tx=0x7fcba800c880 comp rx=0 tx=0).stop 2024-04-24T23:24:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 shutdown_connections 2024-04-24T23:24:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcba406be30 0x7fcba406e2f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcbb815e380 0x7fcbb80cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcbb80b4530 0x7fcbb81f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 --2- 172.21.15.26:0/1742153802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbb80b0030 0x7fcbb814a320 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 >> 172.21.15.26:0/1742153802 conn(0x7fcbb806e2c0 msgr2=0x7fcbb815d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.266+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 shutdown_connections 2024-04-24T23:24:01.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.267+0000 7fcbbe25d640 1 -- 172.21.15.26:0/1742153802 wait complete. 2024-04-24T23:24:01.279 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:24:01.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.379+0000 7f02ac1a5640 1 Processor -- start 2024-04-24T23:24:01.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.379+0000 7f02ac1a5640 1 -- start start 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02ac1a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a406f4f0 0x7f02a40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02ac1a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f02a40addb0 0x7f02a4154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02ac1a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a40b0220 con 0x7f02a406f110 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02ac1a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f02a40ae220 con 0x7f02a40addb0 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02ac1a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f02a40af180 con 0x7f02a406f4f0 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02a9f41640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f02a406f110 0x7f02a415f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35184/0 (socket says 172.21.15.26:35184) 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02aa742640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f02a40addb0 0x7f02a4154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02a9f41640 1 -- 172.21.15.26:0/789368799 learned_addr learned my addr 172.21.15.26:0/789368799 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02a9740640 1 --2- 172.21.15.26:0/789368799 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a406f4f0 0x7f02a40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02aa742640 1 -- 172.21.15.26:0/789368799 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a406f4f0 msgr2=0x7f02a40ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02aa742640 1 --2- 172.21.15.26:0/789368799 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a406f4f0 0x7f02a40ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02aa742640 1 -- 172.21.15.26:0/789368799 >> v1:172.21.15.26:6789/0 conn(0x7f02a406f110 legacy=0x7f02a415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.380+0000 7f02aa742640 1 -- 172.21.15.26:0/789368799 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a40af460 con 0x7f02a40addb0 2024-04-24T23:24:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.381+0000 7f02aa742640 1 --2- 172.21.15.26:0/789368799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a40addb0 0x7f02a4154a50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009d90 tx=0x7f02a002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e6b9243b71e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.381+0000 7f02a8f3f640 1 -- 172.21.15.26:0/789368799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0007740 con 0x7f02a40addb0 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.381+0000 7f02a8f3f640 1 -- 172.21.15.26:0/789368799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a00078e0 con 0x7f02a40addb0 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.381+0000 7f02a8f3f640 1 -- 172.21.15.26:0/789368799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0007cb0 con 0x7f02a40addb0 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.382+0000 7f02ac1a5640 1 -- 172.21.15.26:0/789368799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a40addb0 msgr2=0x7f02a4154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.382+0000 7f02ac1a5640 1 --2- 172.21.15.26:0/789368799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a40addb0 0x7f02a4154a50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009d90 tx=0x7f02a002a660 comp rx=0 tx=0).stop 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.382+0000 7f02ac1a5640 1 -- 172.21.15.26:0/789368799 shutdown_connections 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.382+0000 7f02ac1a5640 1 --2- 172.21.15.26:0/789368799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a40addb0 0x7f02a4154a50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.382+0000 7f02ac1a5640 1 --2- 172.21.15.26:0/789368799 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a406f4f0 0x7f02a40ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.382+0000 7f02ac1a5640 1 -- 172.21.15.26:0/789368799 >> 172.21.15.26:0/789368799 conn(0x7f02a40729a0 msgr2=0x7f02a406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.382+0000 7f02ac1a5640 1 -- 172.21.15.26:0/789368799 shutdown_connections 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.382+0000 7f02ac1a5640 1 -- 172.21.15.26:0/789368799 wait complete. 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.383+0000 7f02ac1a5640 1 Processor -- start 2024-04-24T23:24:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.383+0000 7f02ac1a5640 1 -- start start 2024-04-24T23:24:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02ac1a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a406f110 0x7f02a41552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02ac1a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a406f4f0 0x7f02a41f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02ac1a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a40addb0 0x7f02a41f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02ac1a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f02a40afd00 con 0x7f02a406f4f0 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02ac1a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02a40afa20 con 0x7f02a406f110 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02ac1a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f02a40aee50 con 0x7f02a40addb0 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9740640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a406f4f0 0x7f02a41f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9740640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a406f4f0 0x7f02a41f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51374/0 (socket says 172.21.15.26:51374) 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9740640 1 -- 172.21.15.26:0/355343028 learned_addr learned my addr 172.21.15.26:0/355343028 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02aa742640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a40addb0 0x7f02a41f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9f41640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a406f110 0x7f02a41552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9740640 1 -- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a40addb0 msgr2=0x7f02a41f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9740640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a40addb0 0x7f02a41f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9740640 1 -- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a406f110 msgr2=0x7f02a41552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9740640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a406f110 0x7f02a41552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9740640 1 -- 172.21.15.26:0/355343028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a4155950 con 0x7f02a406f4f0 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9f41640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a406f110 0x7f02a41552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02aa742640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a40addb0 0x7f02a41f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.384+0000 7f02a9740640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a406f4f0 0x7f02a41f7ef0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f029400d7d0 tx=0x7f029400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.385+0000 7f029a7fc640 1 -- 172.21.15.26:0/355343028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029400b6c0 con 0x7f02a406f4f0 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.385+0000 7f029a7fc640 1 -- 172.21.15.26:0/355343028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0294019070 con 0x7f02a406f4f0 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.385+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a415c430 con 0x7f02a406f4f0 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.385+0000 7f029a7fc640 1 -- 172.21.15.26:0/355343028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029400ba80 con 0x7f02a406f4f0 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.385+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a415c9f0 con 0x7f02a406f4f0 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.386+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a40694c0 con 0x7f02a406f4f0 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.386+0000 7f029a7fc640 1 -- 172.21.15.26:0/355343028 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f029400bc20 con 0x7f02a406f4f0 2024-04-24T23:24:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.386+0000 7f029a7fc640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f028806be30 0x7f028806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.387+0000 7f029a7fc640 1 -- 172.21.15.26:0/355343028 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f029408b330 con 0x7f02a406f4f0 2024-04-24T23:24:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.387+0000 7f02a9f41640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f028806be30 0x7f028806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.387+0000 7f02a9f41640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f028806be30 0x7f028806e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f029c000f60 tx=0x7f029c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.391+0000 7f029a7fc640 1 -- 172.21.15.26:0/355343028 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f029401c360 con 0x7f02a406f4f0 2024-04-24T23:24:01.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.612+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02a406c4b0 con 0x7f02a406f4f0 2024-04-24T23:24:01.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.612+0000 7f029a7fc640 1 -- 172.21.15.26:0/355343028 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5222 (secure 0 0 0) 0x7f0294059ca0 con 0x7f02a406f4f0 2024-04-24T23:24:01.613 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:01.614 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:40.866214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5742,"mds_1":5329,"mds_2":5081},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5081":{"gid":5081,"name":"b","rank":2,"incarnation":36,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742,5329,5081]},"id":3}]} 2024-04-24T23:24:01.614 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 42 2024-04-24T23:24:01.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f028806be30 msgr2=0x7f028806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f028806be30 0x7f028806e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f029c000f60 tx=0x7f029c009040 comp rx=0 tx=0).stop 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a406f4f0 msgr2=0x7f02a41f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a406f4f0 0x7f02a41f7ef0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f029400d7d0 tx=0x7f029400dc90 comp rx=0 tx=0).stop 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 shutdown_connections 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f028806be30 0x7f028806e2f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02a40addb0 0x7f02a41f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02a406f4f0 0x7f02a41f7ef0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 --2- 172.21.15.26:0/355343028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02a406f110 0x7f02a41552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 >> 172.21.15.26:0/355343028 conn(0x7f02a40729a0 msgr2=0x7f02a4151540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 shutdown_connections 2024-04-24T23:24:01.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.614+0000 7f02ac1a5640 1 -- 172.21.15.26:0/355343028 wait complete. 2024-04-24T23:24:01.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:01.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.726+0000 7f213a053640 1 Processor -- start 2024-04-24T23:24:01.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.726+0000 7f213a053640 1 -- start start 2024-04-24T23:24:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f213a053640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2134142310 0x7f2134142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f213a053640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2134145890 0x7f21340d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f213a053640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213406deb0 con 0x7f21340b5cd0 2024-04-24T23:24:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f213a053640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f213406dc60 con 0x7f2134142310 2024-04-24T23:24:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f213a053640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f213406eac0 con 0x7f2134145890 2024-04-24T23:24:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f2133fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21340b5cd0 0x7f213406ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35196/0 (socket says 172.21.15.26:35196) 2024-04-24T23:24:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f2133fff640 1 -- 172.21.15.26:0/2578213260 learned_addr learned my addr 172.21.15.26:0/2578213260 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f2132ffd640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2134145890 0x7f21340d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f21337fe640 1 --2- 172.21.15.26:0/2578213260 >> v2:172.21.15.29:3300/0 conn(0x7f2134142310 0x7f2134142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650469606 0 0) 0x7f213406deb0 con 0x7f21340b5cd0 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.727+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2120003520 con 0x7f21340b5cd0 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 6913418 0 0) 0x7f2120003520 con 0x7f21340b5cd0 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2134145890 msgr2=0x7f21340d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2134145890 0x7f21340d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 >> v2:172.21.15.29:3300/0 conn(0x7f2134142310 msgr2=0x7f2134142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 --2- 172.21.15.26:0/2578213260 >> v2:172.21.15.29:3300/0 conn(0x7f2134142310 0x7f2134142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213406e190 con 0x7f21340b5cd0 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f212c0019f0 con 0x7f21340b5cd0 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 >> v1:172.21.15.26:6789/0 conn(0x7f21340b5cd0 legacy=0x7f213406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2134145890 0x7f21200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2120003f30 0x7f2120004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2120004d40 0x7f21200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f213406dc60 con 0x7f2120003f30 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f213406eac0 con 0x7f2120004d40 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f212000ba10 con 0x7f2134145890 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f2132ffd640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2120003f30 0x7f2120004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f21337fe640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2134145890 0x7f21200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f2133fff640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2120004d40 0x7f21200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f2132ffd640 1 -- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2134145890 msgr2=0x7f21200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f2132ffd640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2134145890 0x7f21200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f2132ffd640 1 -- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2120004d40 msgr2=0x7f21200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f2132ffd640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2120004d40 0x7f21200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.728+0000 7f2132ffd640 1 -- 172.21.15.26:0/2578213260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2120003520 con 0x7f2120003f30 2024-04-24T23:24:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.729+0000 7f2132ffd640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2120003f30 0x7f2120004800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f211c009af0 tx=0x7f2120005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e1c8cceb064bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.729+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c036070 con 0x7f2120003f30 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.729+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c031400 con 0x7f2120003f30 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.729+0000 7f21327fc640 1 -- 172.21.15.26:0/2578213260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c009690 con 0x7f2120003f30 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.729+0000 7f213a053640 1 -- 172.21.15.26:0/2578213260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2120003f30 msgr2=0x7f2120004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.729+0000 7f213a053640 1 --2- 172.21.15.26:0/2578213260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2120003f30 0x7f2120004800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f211c009af0 tx=0x7f2120005830 comp rx=0 tx=0).stop 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.729+0000 7f2133fff640 1 -- 172.21.15.26:0/2578213260 reap_dead start 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.730+0000 7f213a053640 1 -- 172.21.15.26:0/2578213260 shutdown_connections 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.730+0000 7f213a053640 1 -- 172.21.15.26:0/2578213260 >> 172.21.15.26:0/2578213260 conn(0x7f21340729a0 msgr2=0x7f213406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.730+0000 7f213a053640 1 -- 172.21.15.26:0/2578213260 shutdown_connections 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.730+0000 7f213a053640 1 -- 172.21.15.26:0/2578213260 wait complete. 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.731+0000 7f213a053640 1 Processor -- start 2024-04-24T23:24:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.731+0000 7f213a053640 1 -- start start 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f213a053640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2120003f30 0x7f2134159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f213a053640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2134157730 0x7f2134159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f213a053640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f213413bee0 0x7f213413c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f213a053640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21341426f0 con 0x7f2134157730 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f213a053640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f213406e750 con 0x7f213413bee0 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f213a053640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2134142870 con 0x7f2120003f30 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2132ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2134157730 0x7f2134159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2132ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2134157730 0x7f2134159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51394/0 (socket says 172.21.15.26:51394) 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2132ffd640 1 -- 172.21.15.26:0/1451112987 learned_addr learned my addr 172.21.15.26:0/1451112987 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f21337fe640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2120003f30 0x7f2134159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2133fff640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f213413bee0 0x7f213413c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2132ffd640 1 -- 172.21.15.26:0/1451112987 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2120003f30 msgr2=0x7f2134159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2132ffd640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2120003f30 0x7f2134159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2132ffd640 1 -- 172.21.15.26:0/1451112987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f213413bee0 msgr2=0x7f213413c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2132ffd640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f213413bee0 0x7f213413c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2132ffd640 1 -- 172.21.15.26:0/1451112987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2134158110 con 0x7f2134157730 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2133fff640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f213413bee0 0x7f213413c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f2132ffd640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2134157730 0x7f2134159860 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f211c02abe0 tx=0x7f211c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f212bfff640 1 -- 172.21.15.26:0/1451112987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c040020 con 0x7f2134157730 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f212bfff640 1 -- 172.21.15.26:0/1451112987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c0038f0 con 0x7f2134157730 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21341583f0 con 0x7f2134157730 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f212bfff640 1 -- 172.21.15.26:0/1451112987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c036040 con 0x7f2134157730 2024-04-24T23:24:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.732+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21341589b0 con 0x7f2134157730 2024-04-24T23:24:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.734+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2134067720 con 0x7f2134157730 2024-04-24T23:24:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.734+0000 7f212bfff640 1 -- 172.21.15.26:0/1451112987 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f211c043020 con 0x7f2134157730 2024-04-24T23:24:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.735+0000 7f212bfff640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211406be30 0x7f211406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.735+0000 7f21337fe640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211406be30 0x7f211406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.735+0000 7f212bfff640 1 -- 172.21.15.26:0/1451112987 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f211c0aafd0 con 0x7f2134157730 2024-04-24T23:24:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.735+0000 7f21337fe640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211406be30 0x7f211406e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2124009a60 tx=0x7f2124008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.739+0000 7f212bfff640 1 -- 172.21.15.26:0/1451112987 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f211c04a070 con 0x7f2134157730 2024-04-24T23:24:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.958+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21340694c0 con 0x7f2134157730 2024-04-24T23:24:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.958+0000 7f212bfff640 1 -- 172.21.15.26:0/1451112987 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5222 (secure 0 0 0) 0x7f211c0798b0 con 0x7f2134157730 2024-04-24T23:24:01.960 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:01.960 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:23:03.164134+0000","modified":"2024-04-24T23:23:40.866214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5742,"mds_1":5329,"mds_2":5081},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5081":{"gid":5081,"name":"b","rank":2,"incarnation":36,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/1578186344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1578186344},{"type":"v1","addr":"172.21.15.29:6835","nonce":1578186344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5329":{"gid":5329,"name":"d","rank":1,"incarnation":34,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/243988267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":243988267},{"type":"v1","addr":"172.21.15.29:6834","nonce":243988267}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/730964888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":730964888},{"type":"v1","addr":"172.21.15.26:6836","nonce":730964888}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742,5329,5081]},"id":3}]} 2024-04-24T23:24:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 42 2024-04-24T23:24:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211406be30 msgr2=0x7f211406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211406be30 0x7f211406e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2124009a60 tx=0x7f2124008040 comp rx=0 tx=0).stop 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2134157730 msgr2=0x7f2134159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2134157730 0x7f2134159860 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f211c02abe0 tx=0x7f211c004730 comp rx=0 tx=0).stop 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 shutdown_connections 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f211406be30 0x7f211406e2f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f213413bee0 0x7f213413c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2134157730 0x7f2134159860 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 --2- 172.21.15.26:0/1451112987 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2120003f30 0x7f2134159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 >> 172.21.15.26:0/1451112987 conn(0x7f21340729a0 msgr2=0x7f21340b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 shutdown_connections 2024-04-24T23:24:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:01.960+0000 7f213a053640 1 -- 172.21.15.26:0/1451112987 wait complete. 2024-04-24T23:24:01.973 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:24:02.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.073+0000 7f4313a7b640 1 Processor -- start 2024-04-24T23:24:02.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.073+0000 7f4313a7b640 1 -- start start 2024-04-24T23:24:02.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4313a7b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c15a860 0x7f430c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4313a7b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f430c06e120 0x7f430c071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4313a7b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f430c0b2e80 con 0x7f430c15a480 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4313a7b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f430c06e550 con 0x7f430c06e120 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4313a7b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f430c0b3120 con 0x7f430c15a860 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4311817640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f430c15a480 0x7f430c0b48f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35202/0 (socket says 172.21.15.26:35202) 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4311817640 1 -- 172.21.15.26:0/2939818450 learned_addr learned my addr 172.21.15.26:0/2939818450 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4312018640 1 --2- 172.21.15.26:0/2939818450 >> v2:172.21.15.26:3300/0 conn(0x7f430c06e120 0x7f430c071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4311016640 1 --2- 172.21.15.26:0/2939818450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c15a860 0x7f430c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4310815640 1 -- 172.21.15.26:0/2939818450 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2442960253 0 0) 0x7f430c0b2e80 con 0x7f430c15a480 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4310815640 1 -- 172.21.15.26:0/2939818450 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42f8003520 con 0x7f430c15a480 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4312018640 1 -- 172.21.15.26:0/2939818450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c15a860 msgr2=0x7f430c06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4312018640 1 --2- 172.21.15.26:0/2939818450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c15a860 0x7f430c06dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4312018640 1 -- 172.21.15.26:0/2939818450 >> v1:172.21.15.26:6789/0 conn(0x7f430c15a480 legacy=0x7f430c0b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.074+0000 7f4312018640 1 -- 172.21.15.26:0/2939818450 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c0b33c0 con 0x7f430c06e120 2024-04-24T23:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4312018640 1 --2- 172.21.15.26:0/2939818450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c06e120 0x7f430c071070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4304009d90 tx=0x7f430402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafc4d50e8823d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4310815640 1 -- 172.21.15.26:0/2939818450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304031600 con 0x7f430c06e120 2024-04-24T23:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4310815640 1 -- 172.21.15.26:0/2939818450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43040317a0 con 0x7f430c06e120 2024-04-24T23:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4310815640 1 -- 172.21.15.26:0/2939818450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304031b70 con 0x7f430c06e120 2024-04-24T23:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4313a7b640 1 -- 172.21.15.26:0/2939818450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c06e120 msgr2=0x7f430c071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4313a7b640 1 --2- 172.21.15.26:0/2939818450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c06e120 0x7f430c071070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4304009d90 tx=0x7f430402a5d0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4313a7b640 1 -- 172.21.15.26:0/2939818450 shutdown_connections 2024-04-24T23:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4313a7b640 1 --2- 172.21.15.26:0/2939818450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c06e120 0x7f430c071070 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4313a7b640 1 --2- 172.21.15.26:0/2939818450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c15a860 0x7f430c06dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.075+0000 7f4313a7b640 1 -- 172.21.15.26:0/2939818450 >> 172.21.15.26:0/2939818450 conn(0x7f430c06b300 msgr2=0x7f430c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.076+0000 7f4313a7b640 1 -- 172.21.15.26:0/2939818450 shutdown_connections 2024-04-24T23:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.076+0000 7f4313a7b640 1 -- 172.21.15.26:0/2939818450 wait complete. 2024-04-24T23:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.077+0000 7f4313a7b640 1 Processor -- start 2024-04-24T23:24:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.077+0000 7f4313a7b640 1 -- start start 2024-04-24T23:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.077+0000 7f4313a7b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c06e120 0x7f430c13d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.077+0000 7f4313a7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c15a480 0x7f430c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.077+0000 7f4313a7b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430c15a860 0x7f430c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.077+0000 7f4313a7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f430c0b3c60 con 0x7f430c15a480 2024-04-24T23:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.077+0000 7f4313a7b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f430c0b3980 con 0x7f430c15a860 2024-04-24T23:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.077+0000 7f4313a7b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f430c0b5000 con 0x7f430c06e120 2024-04-24T23:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311016640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c15a480 0x7f430c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311016640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c15a480 0x7f430c155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51412/0 (socket says 172.21.15.26:51412) 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311016640 1 -- 172.21.15.26:0/1708311723 learned_addr learned my addr 172.21.15.26:0/1708311723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311817640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c06e120 0x7f430c13d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4312018640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430c15a860 0x7f430c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311016640 1 -- 172.21.15.26:0/1708311723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c06e120 msgr2=0x7f430c13d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311016640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c06e120 0x7f430c13d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311016640 1 -- 172.21.15.26:0/1708311723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430c15a860 msgr2=0x7f430c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311016640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430c15a860 0x7f430c06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311016640 1 -- 172.21.15.26:0/1708311723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c151210 con 0x7f430c15a480 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4312018640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430c15a860 0x7f430c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4311016640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c15a480 0x7f430c155df0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00e870 tx=0x7f42fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f43027fc640 1 -- 172.21.15.26:0/1708311723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc017070 con 0x7f430c15a480 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f43027fc640 1 -- 172.21.15.26:0/1708311723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42fc00c940 con 0x7f430c15a480 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430c0b0d50 con 0x7f430c15a480 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f43027fc640 1 -- 172.21.15.26:0/1708311723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc00cd30 con 0x7f430c15a480 2024-04-24T23:24:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.078+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430c0b1310 con 0x7f430c15a480 2024-04-24T23:24:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.079+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f430c0694c0 con 0x7f430c15a480 2024-04-24T23:24:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.080+0000 7f43027fc640 1 -- 172.21.15.26:0/1708311723 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f42fc01a070 con 0x7f430c15a480 2024-04-24T23:24:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.080+0000 7f43027fc640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f42f006be30 0x7f42f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.080+0000 7f43027fc640 1 -- 172.21.15.26:0/1708311723 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f42fc08b010 con 0x7f430c15a480 2024-04-24T23:24:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.080+0000 7f4311817640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f42f006be30 0x7f42f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.081+0000 7f4311817640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f42f006be30 0x7f42f006e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4308001200 tx=0x7f4308000bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.085+0000 7f43027fc640 1 -- 172.21.15.26:0/1708311723 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f42fc059330 con 0x7f430c15a480 2024-04-24T23:24:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.270+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f430c0b44a0 con 0x7f430c15a480 2024-04-24T23:24:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.271+0000 7f43027fc640 1 -- 172.21.15.26:0/1708311723 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16705 (secure 0 0 0) 0x7f42fc05d630 con 0x7f430c15a480 2024-04-24T23:24:02.273 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:02.273 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":46,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:23:36.893433+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:23:01.453865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T23:23:02.136971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":4}]},{"pgid":"7.2c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:24:02.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f42f006be30 msgr2=0x7f42f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f42f006be30 0x7f42f006e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4308001200 tx=0x7f4308000bc0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c15a480 msgr2=0x7f430c155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c15a480 0x7f430c155df0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00e870 tx=0x7f42fc00ed30 comp rx=0 tx=0).stop 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 shutdown_connections 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f42f006be30 0x7f42f006e2f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f430c15a860 0x7f430c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f430c15a480 0x7f430c155df0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 --2- 172.21.15.26:0/1708311723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f430c06e120 0x7f430c13d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 >> 172.21.15.26:0/1708311723 conn(0x7f430c06b300 msgr2=0x7f430c06b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 shutdown_connections 2024-04-24T23:24:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.273+0000 7f4313a7b640 1 -- 172.21.15.26:0/1708311723 wait complete. 2024-04-24T23:24:02.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:24:02.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.386+0000 7f7fb2091640 1 Processor -- start 2024-04-24T23:24:02.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.386+0000 7f7fb2091640 1 -- start start 2024-04-24T23:24:02.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.387+0000 7f7fb2091640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fac15e570 0x7f7fac0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.387+0000 7f7fb2091640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0b3130 0x7f7fac0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.387+0000 7f7fb2091640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fac0d32f0 con 0x7f7fac0d22e0 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.387+0000 7f7fb2091640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7fac0d1ab0 con 0x7f7fac15e570 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.387+0000 7f7fb2091640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7fac0d2ae0 con 0x7f7fac0b3130 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.387+0000 7f7fab7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fac15e570 0x7f7fac0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.387+0000 7f7fabfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7fac0d22e0 0x7f7fac0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35208/0 (socket says 172.21.15.26:35208) 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.387+0000 7f7fab7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fac15e570 0x7f7fac0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51422/0 (socket says 172.21.15.26:51422) 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.387+0000 7f7fabfff640 1 -- 172.21.15.26:0/1267189448 learned_addr learned my addr 172.21.15.26:0/1267189448 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.388+0000 7f7faaffd640 1 --2- 172.21.15.26:0/1267189448 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0b3130 0x7f7fac0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.388+0000 7f7fab7fe640 1 -- 172.21.15.26:0/1267189448 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0b3130 msgr2=0x7f7fac0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.388+0000 7f7fab7fe640 1 --2- 172.21.15.26:0/1267189448 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0b3130 0x7f7fac0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.388+0000 7f7fab7fe640 1 -- 172.21.15.26:0/1267189448 >> v1:172.21.15.26:6789/0 conn(0x7f7fac0d22e0 legacy=0x7f7fac0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.388+0000 7f7faa7fc640 1 -- 172.21.15.26:0/1267189448 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2630128647 0 0) 0x7f7fa4001490 con 0x7f7fac0d22e0 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.388+0000 7f7fab7fe640 1 -- 172.21.15.26:0/1267189448 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac0d0270 con 0x7f7fac15e570 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.388+0000 7f7faaffd640 1 --2- 172.21.15.26:0/1267189448 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0b3130 0x7f7fac0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7fab7fe640 1 --2- 172.21.15.26:0/1267189448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 0x7f7fac0d4160 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0097a0 tx=0x7f7f9c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b27db29dee054c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7faa7fc640 1 -- 172.21.15.26:0/1267189448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c037070 con 0x7f7fac15e570 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7faa7fc640 1 -- 172.21.15.26:0/1267189448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9c032440 con 0x7f7fac15e570 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7faa7fc640 1 -- 172.21.15.26:0/1267189448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c032810 con 0x7f7fac15e570 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7fb2091640 1 -- 172.21.15.26:0/1267189448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 msgr2=0x7f7fac0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7fb2091640 1 --2- 172.21.15.26:0/1267189448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 0x7f7fac0d4160 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0097a0 tx=0x7f7f9c02a180 comp rx=0 tx=0).stop 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7fb2091640 1 -- 172.21.15.26:0/1267189448 shutdown_connections 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7fb2091640 1 --2- 172.21.15.26:0/1267189448 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0b3130 0x7f7fac0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7fb2091640 1 --2- 172.21.15.26:0/1267189448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 0x7f7fac0d4160 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7fb2091640 1 -- 172.21.15.26:0/1267189448 >> 172.21.15.26:0/1267189448 conn(0x7f7fac08dbe0 msgr2=0x7f7fac09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7fb2091640 1 -- 172.21.15.26:0/1267189448 shutdown_connections 2024-04-24T23:24:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.389+0000 7f7fb2091640 1 -- 172.21.15.26:0/1267189448 wait complete. 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.390+0000 7f7fb2091640 1 Processor -- start 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.391+0000 7f7fb2091640 1 -- start start 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.391+0000 7f7fb2091640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fac0b3130 0x7f7fac0b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.391+0000 7f7fb2091640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0d22e0 0x7f7fac1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.391+0000 7f7fb2091640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 0x7f7fac0b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.391+0000 7f7fb2091640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac0cfa70 con 0x7f7fac15e570 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.391+0000 7f7fb2091640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac0d3010 con 0x7f7fac0b3130 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.391+0000 7f7fb2091640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7fac0cfd50 con 0x7f7fac0d22e0 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.391+0000 7f7fabfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 0x7f7fac0b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fabfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 0x7f7fac0b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51424/0 (socket says 172.21.15.26:51424) 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fabfff640 1 -- 172.21.15.26:0/3496902356 learned_addr learned my addr 172.21.15.26:0/3496902356 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:02.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7faaffd640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0d22e0 0x7f7fac1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fab7fe640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fac0b3130 0x7f7fac0b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fabfff640 1 -- 172.21.15.26:0/3496902356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0d22e0 msgr2=0x7f7fac1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fabfff640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0d22e0 0x7f7fac1565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fabfff640 1 -- 172.21.15.26:0/3496902356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fac0b3130 msgr2=0x7f7fac0b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fabfff640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fac0b3130 0x7f7fac0b6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fabfff640 1 -- 172.21.15.26:0/3496902356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac152ca0 con 0x7f7fac15e570 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fabfff640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 0x7f7fac0b8520 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400c7f0 tx=0x7f7fa400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7f93fff640 1 -- 172.21.15.26:0/3496902356 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4018070 con 0x7f7fac15e570 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7f93fff640 1 -- 172.21.15.26:0/3496902356 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa400a5e0 con 0x7f7fac15e570 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fac0cd3a0 con 0x7f7fac15e570 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7f93fff640 1 -- 172.21.15.26:0/3496902356 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400aa10 con 0x7f7fac15e570 2024-04-24T23:24:02.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.392+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fac0b7a40 con 0x7f7fac15e570 2024-04-24T23:24:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.393+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fac0694c0 con 0x7f7fac15e570 2024-04-24T23:24:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.394+0000 7f7f93fff640 1 -- 172.21.15.26:0/3496902356 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fa401b070 con 0x7f7fac15e570 2024-04-24T23:24:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.394+0000 7f7f93fff640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9806be60 0x7f7f9806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.394+0000 7f7fab7fe640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9806be60 0x7f7f9806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.394+0000 7f7f93fff640 1 -- 172.21.15.26:0/3496902356 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f7fa408a3b0 con 0x7f7fac15e570 2024-04-24T23:24:02.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.395+0000 7f7fab7fe640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9806be60 0x7f7f9806e320 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c034000 tx=0x7f7f9c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:02.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.399+0000 7f7f93fff640 1 -- 172.21.15.26:0/3496902356 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7fac0d32f0 con 0x7f7fac15e570 2024-04-24T23:24:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.621+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7fac0d3a60 con 0x7f7fac15e570 2024-04-24T23:24:02.622 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:24:02.621+0000 7f26a0f44640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:24:02.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.623+0000 7f7f93fff640 1 -- 172.21.15.26:0/3496902356 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7f7fa4058db0 con 0x7f7fac15e570 2024-04-24T23:24:02.624 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:24:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.625+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9806be60 msgr2=0x7f7f9806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.625+0000 7f7fb2091640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9806be60 0x7f7f9806e320 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c034000 tx=0x7f7f9c02e000 comp rx=0 tx=0).stop 2024-04-24T23:24:02.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.625+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 msgr2=0x7f7fac0b8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.625+0000 7f7fb2091640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 0x7f7fac0b8520 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400c7f0 tx=0x7f7fa400ccb0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.626+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 shutdown_connections 2024-04-24T23:24:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.626+0000 7f7fb2091640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9806be60 0x7f7f9806e320 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.626+0000 7f7fb2091640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fac15e570 0x7f7fac0b8520 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.626+0000 7f7fb2091640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fac0d22e0 0x7f7fac1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.626+0000 7f7fb2091640 1 --2- 172.21.15.26:0/3496902356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fac0b3130 0x7f7fac0b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.626+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 >> 172.21.15.26:0/3496902356 conn(0x7f7fac08dbe0 msgr2=0x7f7fac15d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.626+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 shutdown_connections 2024-04-24T23:24:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.626+0000 7f7fb2091640 1 -- 172.21.15.26:0/3496902356 wait complete. 2024-04-24T23:24:02.631 INFO:tasks.ceph.mds.a.smithi026.stderr: -7522> 2024-04-24T23:23:19.779+0000 7fc1e7675640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-24T23:24:02.631 INFO:tasks.ceph.mds.a.smithi026.stderr: -7520> 2024-04-24T23:23:19.779+0000 7fc1e7675640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-24T23:24:02.632 INFO:tasks.ceph.mds.b.smithi029.stderr: -1213> 2024-04-24T23:23:36.698+0000 7fbb6668e640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1) n(v0 rc2024-04-24T23:23:19.214994+0000 2=0+2) (inest lock) (ifile excl) caps={5458=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5458 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55cbdf449080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.214994+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-04-24T23:23:19.214994+0000 2=0+2)","error_str":""},"return_code":-61} 2024-04-24T23:24:02.632 INFO:tasks.ceph.mds.b.smithi029.stderr: -1192> 2024-04-24T23:23:36.698+0000 7fbb6668e640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5458=pAsLsXsFsx/AsLsXsFsx@2},l=5458 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55cbdf449600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-04-24T23:24:02.633 INFO:tasks.ceph.mds.d.smithi029.stderr: -1430> 2024-04-24T23:23:36.697+0000 7f1294097640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1) n(v2 rc2024-04-24T23:23:19.443990+0000 4=0+4) (inest lock) (ifile excl) caps={5458=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5458 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x56421e708580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.443990+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-04-24T23:23:19.443990+0000 4=0+4)","error_str":""},"return_code":-61} 2024-04-24T23:24:02.633 INFO:tasks.ceph.mds.d.smithi029.stderr: -1292> 2024-04-24T23:23:36.699+0000 7f129a8a4640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1) n(v1 rc2024-04-24T23:23:19.443990+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={5458=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x56421f65cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.214994+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.214994+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-04-24T23:23:19.443990+0000 3=0+3)","error_str":""},"return_code":-61} 2024-04-24T23:24:02.636 INFO:tasks.ceph.mds.a.smithi026.stderr: -2690> 2024-04-24T23:23:36.696+0000 7fc1e366d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-04-24T23:23:19.212994+0000 1=0+1) n(v3 rc2024-04-24T23:23:19.443990+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5458=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5458 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55e60391c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.212994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.443990+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.212994+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-04-24T23:23:19.443990+0000 6=0+6)","error_str":""},"return_code":-61} 2024-04-24T23:24:02.636 INFO:tasks.ceph.mds.a.smithi026.stderr: -2572> 2024-04-24T23:23:36.698+0000 7fc1e9e7a640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1) n(v4 rc2024-04-24T23:23:19.443990+0000 5=0+5) (inest sync->mix g=1,2) caps={5458=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55e603a53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:23:19.443990+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-04-24T23:23:19.213994+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-04-24T23:23:19.443990+0000 5=0+5)","error_str":""},"return_code":-61} 2024-04-24T23:24:02.638 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:24:02.662 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:24:02.663 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:24:02.663 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:24:02.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.741+0000 7f6e4dc37640 1 Processor -- start 2024-04-24T23:24:02.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.741+0000 7f6e4dc37640 1 -- start start 2024-04-24T23:24:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.742+0000 7f6e4dc37640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e4815c270 0x7f6e480cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.742+0000 7f6e4dc37640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e480cf050 con 0x7f6e480d0ab0 2024-04-24T23:24:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.742+0000 7f6e4dc37640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e480cdd60 con 0x7f6e4815c650 2024-04-24T23:24:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.742+0000 7f6e4dc37640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6e480cee00 con 0x7f6e4815c270 2024-04-24T23:24:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.742+0000 7f6e477fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e4815c270 0x7f6e480cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.742+0000 7f6e47fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e480d0ab0 0x7f6e480c5910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35224/0 (socket says 172.21.15.26:35224) 2024-04-24T23:24:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.742+0000 7f6e47fff640 1 -- 172.21.15.26:0/2257501954 learned_addr learned my addr 172.21.15.26:0/2257501954 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.743+0000 7f6e477fe640 1 -- 172.21.15.26:0/2257501954 >> v1:172.21.15.29:6789/0 conn(0x7f6e4815c650 legacy=0x7f6e480d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.743+0000 7f6e477fe640 1 -- 172.21.15.26:0/2257501954 >> v1:172.21.15.26:6789/0 conn(0x7f6e480d0ab0 legacy=0x7f6e480c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.743+0000 7f6e467fc640 1 -- 172.21.15.26:0/2257501954 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 706406848 0 0) 0x7f6e480cf050 con 0x7f6e480d0ab0 2024-04-24T23:24:02.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.743+0000 7f6e477fe640 1 -- 172.21.15.26:0/2257501954 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e480d2b30 con 0x7f6e4815c270 2024-04-24T23:24:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.743+0000 7f6e477fe640 1 --2- 172.21.15.26:0/2257501954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e4815c270 0x7f6e480cfd50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6e380097a0 tx=0x7f6e3802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f492b74dc7050f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.743+0000 7f6e467fc640 1 -- 172.21.15.26:0/2257501954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38037070 con 0x7f6e4815c270 2024-04-24T23:24:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.744+0000 7f6e467fc640 1 -- 172.21.15.26:0/2257501954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e38032440 con 0x7f6e4815c270 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.744+0000 7f6e467fc640 1 -- 172.21.15.26:0/2257501954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38032810 con 0x7f6e4815c270 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.744+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2257501954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e4815c270 msgr2=0x7f6e480cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.744+0000 7f6e4dc37640 1 --2- 172.21.15.26:0/2257501954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e4815c270 0x7f6e480cfd50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6e380097a0 tx=0x7f6e3802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.744+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2257501954 shutdown_connections 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.744+0000 7f6e4dc37640 1 --2- 172.21.15.26:0/2257501954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e4815c270 0x7f6e480cfd50 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.744+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2257501954 >> 172.21.15.26:0/2257501954 conn(0x7f6e480d1dc0 msgr2=0x7f6e480d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.744+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2257501954 shutdown_connections 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.744+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2257501954 wait complete. 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.745+0000 7f6e4dc37640 1 Processor -- start 2024-04-24T23:24:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.745+0000 7f6e4dc37640 1 -- start start 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e4dc37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480d0ab0 0x7f6e48159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e4dc37640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e4815c270 0x7f6e480cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e4dc37640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e4815c650 0x7f6e480b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e4dc37640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480b2950 con 0x7f6e480d0ab0 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e4dc37640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480d28e0 con 0x7f6e4815c650 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e4dc37640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6e480b2ad0 con 0x7f6e4815c270 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480d0ab0 0x7f6e48159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480d0ab0 0x7f6e48159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51464/0 (socket says 172.21.15.26:51464) 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e477fe640 1 -- 172.21.15.26:0/2187963338 learned_addr learned my addr 172.21.15.26:0/2187963338 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e47fff640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e4815c650 0x7f6e480b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.746+0000 7f6e46ffd640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e4815c270 0x7f6e480cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e477fe640 1 -- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e4815c270 msgr2=0x7f6e480cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e477fe640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e4815c270 0x7f6e480cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e477fe640 1 -- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e4815c650 msgr2=0x7f6e480b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e477fe640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e4815c650 0x7f6e480b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e477fe640 1 -- 172.21.15.26:0/2187963338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e480c9f80 con 0x7f6e480d0ab0 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e47fff640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e4815c650 0x7f6e480b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e46ffd640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e4815c270 0x7f6e480cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e477fe640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480d0ab0 0x7f6e48159e90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6e38000c00 tx=0x7f6e380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e2ffff640 1 -- 172.21.15.26:0/2187963338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38041070 con 0x7f6e480d0ab0 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e2ffff640 1 -- 172.21.15.26:0/2187963338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3803c930 con 0x7f6e480d0ab0 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e480ca260 con 0x7f6e480d0ab0 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e2ffff640 1 -- 172.21.15.26:0/2187963338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38037040 con 0x7f6e480d0ab0 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.747+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e480cab00 con 0x7f6e480d0ab0 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.748+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e480ca430 con 0x7f6e480d0ab0 2024-04-24T23:24:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.748+0000 7f6e2ffff640 1 -- 172.21.15.26:0/2187963338 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e38044070 con 0x7f6e480d0ab0 2024-04-24T23:24:02.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.749+0000 7f6e2ffff640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3406be30 0x7f6e3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:02.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.749+0000 7f6e2ffff640 1 -- 172.21.15.26:0/2187963338 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6025+0+0 (secure 0 0 0) 0x7f6e380abe10 con 0x7f6e480d0ab0 2024-04-24T23:24:02.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.749+0000 7f6e46ffd640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3406be30 0x7f6e3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:02.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.749+0000 7f6e46ffd640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3406be30 0x7f6e3406e2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6e480d00e0 tx=0x7f6e3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:02.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.753+0000 7f6e2ffff640 1 -- 172.21.15.26:0/2187963338 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e380af030 con 0x7f6e480d0ab0 2024-04-24T23:24:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:02.977+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6e480b4a10 con 0x7f6e480d0ab0 2024-04-24T23:24:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.624+0000 7f6e2ffff640 1 -- 172.21.15.26:0/2187963338 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) ==== 104+0+0 (secure 0 0 0) 0x7f6e480b4a10 con 0x7f6e480d0ab0 2024-04-24T23:24:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3406be30 msgr2=0x7f6e3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3406be30 0x7f6e3406e2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6e480d00e0 tx=0x7f6e3c009040 comp rx=0 tx=0).stop 2024-04-24T23:24:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480d0ab0 msgr2=0x7f6e48159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480d0ab0 0x7f6e48159e90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6e38000c00 tx=0x7f6e380046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 shutdown_connections 2024-04-24T23:24:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6e3406be30 0x7f6e3406e2f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6e4815c650 0x7f6e480b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6e4815c270 0x7f6e480cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 --2- 172.21.15.26:0/2187963338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480d0ab0 0x7f6e48159e90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 >> 172.21.15.26:0/2187963338 conn(0x7f6e480d1dc0 msgr2=0x7f6e480d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 shutdown_connections 2024-04-24T23:24:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.628+0000 7f6e4dc37640 1 -- 172.21.15.26:0/2187963338 wait complete. 2024-04-24T23:24:03.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:24:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.796+0000 7f65b27c1640 1 Processor -- start 2024-04-24T23:24:03.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.796+0000 7f65b27c1640 1 -- start start 2024-04-24T23:24:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b27c1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f65ac141160 0x7f65ac0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b27c1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f65ac0711a0 0x7f65ac071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b27c1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65ac0cf050 con 0x7f65ac15ea80 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b27c1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f65ac0cf330 con 0x7f65ac0711a0 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b27c1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f65ac0cf610 con 0x7f65ac141160 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b0d5e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f65ac0711a0 0x7f65ac071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b0d5e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f65ac0711a0 0x7f65ac071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39746/0 (socket says 172.21.15.26:39746) 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b0d5e640 1 -- 172.21.15.26:0/3865591079 learned_addr learned my addr 172.21.15.26:0/3865591079 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65abfff640 1 --1- 172.21.15.26:0/3865591079 >> v1:172.21.15.26:6789/0 conn(0x7f65ac15ea80 0x7f65ac15ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46654/0 (socket says 172.21.15.26:46654) 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65ab7fe640 1 --2- 172.21.15.26:0/3865591079 >> v2:172.21.15.29:3300/0 conn(0x7f65ac141160 0x7f65ac0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b0d5e640 1 -- 172.21.15.26:0/3865591079 >> v2:172.21.15.29:3300/0 conn(0x7f65ac141160 msgr2=0x7f65ac0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b0d5e640 1 --2- 172.21.15.26:0/3865591079 >> v2:172.21.15.29:3300/0 conn(0x7f65ac141160 0x7f65ac0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b0d5e640 1 -- 172.21.15.26:0/3865591079 >> v1:172.21.15.26:6789/0 conn(0x7f65ac15ea80 legacy=0x7f65ac15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65aaffd640 1 -- 172.21.15.26:0/3865591079 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1520408378 0 0) 0x7f65ac0cf050 con 0x7f65ac15ea80 2024-04-24T23:24:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.797+0000 7f65b0d5e640 1 -- 172.21.15.26:0/3865591079 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac0cf8f0 con 0x7f65ac0711a0 2024-04-24T23:24:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.798+0000 7f65ab7fe640 1 --2- 172.21.15.26:0/3865591079 >> v2:172.21.15.29:3300/0 conn(0x7f65ac141160 0x7f65ac0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.798+0000 7f65b0d5e640 1 --2- 172.21.15.26:0/3865591079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac0711a0 0x7f65ac071610 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f65ac06f180 tx=0x7f65a402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56877ab6572a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.798+0000 7f65aaffd640 1 -- 172.21.15.26:0/3865591079 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4036070 con 0x7f65ac0711a0 2024-04-24T23:24:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.798+0000 7f65aaffd640 1 -- 172.21.15.26:0/3865591079 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a4031600 con 0x7f65ac0711a0 2024-04-24T23:24:03.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.798+0000 7f65aaffd640 1 -- 172.21.15.26:0/3865591079 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40319d0 con 0x7f65ac0711a0 2024-04-24T23:24:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.799+0000 7f65b27c1640 1 -- 172.21.15.26:0/3865591079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac0711a0 msgr2=0x7f65ac071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.799+0000 7f65b27c1640 1 --2- 172.21.15.26:0/3865591079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac0711a0 0x7f65ac071610 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f65ac06f180 tx=0x7f65a402a730 comp rx=0 tx=0).stop 2024-04-24T23:24:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.799+0000 7f65b27c1640 1 -- 172.21.15.26:0/3865591079 shutdown_connections 2024-04-24T23:24:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.799+0000 7f65b27c1640 1 --2- 172.21.15.26:0/3865591079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac0711a0 0x7f65ac071610 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.799+0000 7f65b27c1640 1 --2- 172.21.15.26:0/3865591079 >> v2:172.21.15.29:3300/0 conn(0x7f65ac141160 0x7f65ac0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.799+0000 7f65b27c1640 1 -- 172.21.15.26:0/3865591079 >> 172.21.15.26:0/3865591079 conn(0x7f65ac06b300 msgr2=0x7f65ac06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.799+0000 7f65b27c1640 1 -- 172.21.15.26:0/3865591079 shutdown_connections 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.799+0000 7f65b27c1640 1 -- 172.21.15.26:0/3865591079 wait complete. 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.800+0000 7f65b27c1640 1 Processor -- start 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.800+0000 7f65b27c1640 1 -- start start 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65b27c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ac0711a0 0x7f65ac1f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65b27c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f65ac141160 0x7f65ac15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65b27c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac15ea80 0x7f65ac15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65b27c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac0cee00 con 0x7f65ac15ea80 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65b27c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac0cec80 con 0x7f65ac0711a0 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65b27c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f65ac13e090 con 0x7f65ac141160 2024-04-24T23:24:03.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65b0d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac15ea80 0x7f65ac15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65b0d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac15ea80 0x7f65ac15b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39750/0 (socket says 172.21.15.26:39750) 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65b0d5e640 1 -- 172.21.15.26:0/2552089351 learned_addr learned my addr 172.21.15.26:0/2552089351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65ab7fe640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f65ac141160 0x7f65ac15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.801+0000 7f65abfff640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ac0711a0 0x7f65ac1f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65b0d5e640 1 -- 172.21.15.26:0/2552089351 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f65ac141160 msgr2=0x7f65ac15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65b0d5e640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f65ac141160 0x7f65ac15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65b0d5e640 1 -- 172.21.15.26:0/2552089351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ac0711a0 msgr2=0x7f65ac1f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65b0d5e640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ac0711a0 0x7f65ac1f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65b0d5e640 1 -- 172.21.15.26:0/2552089351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac1f5520 con 0x7f65ac15ea80 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65b0d5e640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac15ea80 0x7f65ac15b4d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009f40 tx=0x7f65a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65a8ff9640 1 -- 172.21.15.26:0/2552089351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4041020 con 0x7f65ac15ea80 2024-04-24T23:24:03.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65a8ff9640 1 -- 172.21.15.26:0/2552089351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a4037040 con 0x7f65ac15ea80 2024-04-24T23:24:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac1f1fd0 con 0x7f65ac15ea80 2024-04-24T23:24:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65a8ff9640 1 -- 172.21.15.26:0/2552089351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4036050 con 0x7f65ac15ea80 2024-04-24T23:24:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.802+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac1f4f60 con 0x7f65ac15ea80 2024-04-24T23:24:03.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.803+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ac0694c0 con 0x7f65ac15ea80 2024-04-24T23:24:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.803+0000 7f65a8ff9640 1 -- 172.21.15.26:0/2552089351 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65a4007c50 con 0x7f65ac15ea80 2024-04-24T23:24:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.804+0000 7f65a8ff9640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f659806be30 0x7f659806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.804+0000 7f65a8ff9640 1 -- 172.21.15.26:0/2552089351 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6025+0+0 (secure 0 0 0) 0x7f65a40ab680 con 0x7f65ac15ea80 2024-04-24T23:24:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.804+0000 7f65abfff640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f659806be30 0x7f659806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.804+0000 7f65abfff640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f659806be30 0x7f659806e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f659c000f60 tx=0x7f659c007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.808+0000 7f65a8ff9640 1 -- 172.21.15.26:0/2552089351 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f65a404b060 con 0x7f65ac15ea80 2024-04-24T23:24:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:03.991+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f65ac1f21a0 con 0x7f65ac15ea80 2024-04-24T23:24:04.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.974+0000 7f65a8ff9640 1 -- 172.21.15.26:0/2552089351 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) ==== 181+0+0 (secure 0 0 0) 0x7f65ac1f21a0 con 0x7f65ac15ea80 2024-04-24T23:24:04.976 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:24:04.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.978+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f659806be30 msgr2=0x7f659806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:04.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.978+0000 7f65b27c1640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f659806be30 0x7f659806e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f659c000f60 tx=0x7f659c007b50 comp rx=0 tx=0).stop 2024-04-24T23:24:04.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.978+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac15ea80 msgr2=0x7f65ac15b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:04.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.978+0000 7f65b27c1640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac15ea80 0x7f65ac15b4d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009f40 tx=0x7f65a4004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:04.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.979+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 shutdown_connections 2024-04-24T23:24:04.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.979+0000 7f65b27c1640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f659806be30 0x7f659806e2f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.979+0000 7f65b27c1640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65ac15ea80 0x7f65ac15b4d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.979+0000 7f65b27c1640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f65ac141160 0x7f65ac15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.979+0000 7f65b27c1640 1 --2- 172.21.15.26:0/2552089351 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f65ac0711a0 0x7f65ac1f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.979+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 >> 172.21.15.26:0/2552089351 conn(0x7f65ac06b300 msgr2=0x7f65ac0b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:04.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.979+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 shutdown_connections 2024-04-24T23:24:04.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:04.979+0000 7f65b27c1640 1 -- 172.21.15.26:0/2552089351 wait complete. 2024-04-24T23:24:04.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:24:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.105+0000 7f9757bb7640 1 Processor -- start 2024-04-24T23:24:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.105+0000 7f9757bb7640 1 -- start start 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9757bb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97500d0c00 0x7f97500cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9757bb7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97500cfe20 con 0x7f975015c350 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9757bb7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97500d0100 con 0x7f975015c730 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9757bb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f97500cdf40 con 0x7f97500d0c00 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9755953640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f975015c350 0x7f97500d04f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46660/0 (socket says 172.21.15.26:46660) 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9755953640 1 -- 172.21.15.26:0/630531223 learned_addr learned my addr 172.21.15.26:0/630531223 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012786178 0 0) 0x7f97500cfe20 con 0x7f975015c350 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f973c003520 con 0x7f975015c350 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9755152640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97500d0c00 0x7f97500cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 961141451 0 0) 0x7f973c003520 con 0x7f975015c350 2024-04-24T23:24:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97500d0c00 msgr2=0x7f97500cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9754951640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97500d0c00 0x7f97500cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 >> v1:172.21.15.29:6789/0 conn(0x7f975015c730 legacy=0x7f97500c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97500b3220 con 0x7f975015c350 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.106+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f974c001670 con 0x7f975015c350 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 >> v1:172.21.15.26:6789/0 conn(0x7f975015c350 legacy=0x7f97500d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9754951640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f973c0039e0 0x7f973c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9754951640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c004800 0x7f973c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9754951640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97500d0c00 0x7f973c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97500cdf40 con 0x7f973c004800 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f973c00bb80 con 0x7f973c0039e0 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f973c00bd80 con 0x7f97500d0c00 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9755152640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c004800 0x7f973c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9755953640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f973c0039e0 0x7f973c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9756154640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97500d0c00 0x7f973c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9755152640 1 -- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97500d0c00 msgr2=0x7f973c005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9755152640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97500d0c00 0x7f973c005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9755152640 1 -- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f973c0039e0 msgr2=0x7f973c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9755152640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f973c0039e0 0x7f973c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.107+0000 7f9755152640 1 -- 172.21.15.26:0/630531223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c003520 con 0x7f973c004800 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.108+0000 7f9755152640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c004800 0x7f973c004c80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9740009a90 tx=0x7f973c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a645553e0aac15aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.108+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740036070 con 0x7f973c004800 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.108+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9740031400 con 0x7f973c004800 2024-04-24T23:24:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.108+0000 7f9754951640 1 -- 172.21.15.26:0/630531223 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740009690 con 0x7f973c004800 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.108+0000 7f9757bb7640 1 -- 172.21.15.26:0/630531223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c004800 msgr2=0x7f973c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.108+0000 7f9757bb7640 1 --2- 172.21.15.26:0/630531223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c004800 0x7f973c004c80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9740009a90 tx=0x7f973c005830 comp rx=0 tx=0).stop 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.108+0000 7f9756154640 1 -- 172.21.15.26:0/630531223 reap_dead start 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.108+0000 7f9757bb7640 1 -- 172.21.15.26:0/630531223 shutdown_connections 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.108+0000 7f9757bb7640 1 -- 172.21.15.26:0/630531223 >> 172.21.15.26:0/630531223 conn(0x7f97500d1f10 msgr2=0x7f97500d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.109+0000 7f9757bb7640 1 -- 172.21.15.26:0/630531223 shutdown_connections 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.109+0000 7f9757bb7640 1 -- 172.21.15.26:0/630531223 wait complete. 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.110+0000 7f9757bb7640 1 Processor -- start 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.110+0000 7f9757bb7640 1 -- start start 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.110+0000 7f9757bb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9750154990 0x7f97500ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.110+0000 7f9757bb7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97500cae90 0x7f975009e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.110+0000 7f9757bb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f975009e8f0 0x7f975009ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.110+0000 7f9757bb7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97500b2fd0 con 0x7f975009e8f0 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.110+0000 7f9757bb7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f97500cf1e0 con 0x7f97500cae90 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.110+0000 7f9757bb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f97500d2a30 con 0x7f9750154990 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9756154640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f975009e8f0 0x7f975009ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9756154640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f975009e8f0 0x7f975009ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39756/0 (socket says 172.21.15.26:39756) 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9756154640 1 -- 172.21.15.26:0/1990372469 learned_addr learned my addr 172.21.15.26:0/1990372469 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9755953640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9750154990 0x7f97500ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9755152640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97500cae90 0x7f975009e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9756154640 1 -- 172.21.15.26:0/1990372469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9750154990 msgr2=0x7f97500ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9756154640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9750154990 0x7f97500ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9756154640 1 -- 172.21.15.26:0/1990372469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97500cae90 msgr2=0x7f975009e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9756154640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97500cae90 0x7f975009e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9756154640 1 -- 172.21.15.26:0/1990372469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97500cb6c0 con 0x7f975009e8f0 2024-04-24T23:24:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9756154640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f975009e8f0 0x7f975009ed50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f975015cb10 tx=0x7f974800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f97467fc640 1 -- 172.21.15.26:0/1990372469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974801a070 con 0x7f975009e8f0 2024-04-24T23:24:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f97467fc640 1 -- 172.21.15.26:0/1990372469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9748006d60 con 0x7f975009e8f0 2024-04-24T23:24:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97500cb9a0 con 0x7f975009e8f0 2024-04-24T23:24:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f97467fc640 1 -- 172.21.15.26:0/1990372469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748003b90 con 0x7f975009e8f0 2024-04-24T23:24:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.111+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97500cbf60 con 0x7f975009e8f0 2024-04-24T23:24:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.112+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97500694c0 con 0x7f975009e8f0 2024-04-24T23:24:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.113+0000 7f97467fc640 1 -- 172.21.15.26:0/1990372469 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f974801d070 con 0x7f975009e8f0 2024-04-24T23:24:05.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.113+0000 7f97467fc640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f973406be60 0x7f973406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:05.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.113+0000 7f9755953640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f973406be60 0x7f973406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:05.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.113+0000 7f97467fc640 1 -- 172.21.15.26:0/1990372469 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5587+0+0 (secure 0 0 0) 0x7f974808a300 con 0x7f975009e8f0 2024-04-24T23:24:05.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.114+0000 7f9755953640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f973406be60 0x7f973406e320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f974c000c00 tx=0x7f974c000fe0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:05.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.118+0000 7f97467fc640 1 -- 172.21.15.26:0/1990372469 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9748058760 con 0x7f975009e8f0 2024-04-24T23:24:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.300+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9750074d60 con 0x7f975009e8f0 2024-04-24T23:24:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.979+0000 7f97467fc640 1 -- 172.21.15.26:0/1990372469 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) ==== 169+0+0 (secure 0 0 0) 0x7f974805ca60 con 0x7f975009e8f0 2024-04-24T23:24:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f973406be60 msgr2=0x7f973406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f973406be60 0x7f973406e320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f974c000c00 tx=0x7f974c000fe0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f975009e8f0 msgr2=0x7f975009ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f975009e8f0 0x7f975009ed50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f975015cb10 tx=0x7f974800fd40 comp rx=0 tx=0).stop 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 shutdown_connections 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f973406be60 0x7f973406e320 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f975009e8f0 0x7f975009ed50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97500cae90 0x7f975009e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 --2- 172.21.15.26:0/1990372469 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9750154990 0x7f97500ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 >> 172.21.15.26:0/1990372469 conn(0x7f97500d1f10 msgr2=0x7f97500704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.983+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 shutdown_connections 2024-04-24T23:24:05.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:05.984+0000 7f9757bb7640 1 -- 172.21.15.26:0/1990372469 wait complete. 2024-04-24T23:24:06.000 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:24:06.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.108+0000 7f060262c640 1 Processor -- start 2024-04-24T23:24:06.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.108+0000 7f060262c640 1 -- start start 2024-04-24T23:24:06.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f060262c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f05fc15e750 0x7f05fc0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f060262c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05fc0d2c30 0x7f05fc0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f060262c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05fc0d2120 con 0x7f05fc15e370 2024-04-24T23:24:06.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f060262c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f05fc0d1510 con 0x7f05fc15e750 2024-04-24T23:24:06.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f060262c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc0d1ed0 con 0x7f05fc0d2c30 2024-04-24T23:24:06.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05fbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f05fc15e370 0x7f05fc0d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46662/0 (socket says 172.21.15.26:46662) 2024-04-24T23:24:06.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05fbfff640 1 -- 172.21.15.26:0/2793181196 learned_addr learned my addr 172.21.15.26:0/2793181196 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:06.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1387722136 0 0) 0x7f05fc0d2120 con 0x7f05fc15e370 2024-04-24T23:24:06.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05fb7fe640 1 --2- 172.21.15.26:0/2793181196 >> v2:172.21.15.29:3300/0 conn(0x7f05fc15e750 0x7f05fc0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05e8003520 con 0x7f05fc15e370 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f0600bc9640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05fc0d2c30 0x7f05fc0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2788004001 0 0) 0x7f05e8003520 con 0x7f05fc15e370 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05fc0d2c30 msgr2=0x7f05fc0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05faffd640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05fc0d2c30 0x7f05fc0cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 >> v2:172.21.15.29:3300/0 conn(0x7f05fc15e750 msgr2=0x7f05fc0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05faffd640 1 --2- 172.21.15.26:0/2793181196 >> v2:172.21.15.29:3300/0 conn(0x7f05fc15e750 0x7f05fc0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc0b3740 con 0x7f05fc15e370 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.109+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f05e4001670 con 0x7f05fc15e370 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 >> v1:172.21.15.26:6789/0 conn(0x7f05fc15e370 legacy=0x7f05fc0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f05faffd640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05fc0d2c30 0x7f05e80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f05faffd640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e8003f30 0x7f05e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f05faffd640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e8004d40 0x7f05e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc0d1ed0 con 0x7f05e8004d40 2024-04-24T23:24:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05e800b9c0 con 0x7f05e8003f30 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f05e800bb40 con 0x7f05fc0d2c30 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f0600bc9640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e8004d40 0x7f05e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f05fb7fe640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e8003f30 0x7f05e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f05fbfff640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05fc0d2c30 0x7f05e80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f0600bc9640 1 -- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05fc0d2c30 msgr2=0x7f05e80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f0600bc9640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05fc0d2c30 0x7f05e80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f0600bc9640 1 -- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e8003f30 msgr2=0x7f05e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f0600bc9640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e8003f30 0x7f05e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.110+0000 7f0600bc9640 1 -- 172.21.15.26:0/2793181196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e8003520 con 0x7f05e8004d40 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.111+0000 7f0600bc9640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e8004d40 0x7f05e80051c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009fc0 tx=0x7f05e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77474834748f7f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.111+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4009c10 con 0x7f05e8004d40 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.111+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f4036070 con 0x7f05e8004d40 2024-04-24T23:24:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.111+0000 7f05faffd640 1 -- 172.21.15.26:0/2793181196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4031670 con 0x7f05e8004d40 2024-04-24T23:24:06.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.111+0000 7f060262c640 1 -- 172.21.15.26:0/2793181196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e8004d40 msgr2=0x7f05e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.111+0000 7f060262c640 1 --2- 172.21.15.26:0/2793181196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e8004d40 0x7f05e80051c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009fc0 tx=0x7f05e8005830 comp rx=0 tx=0).stop 2024-04-24T23:24:06.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.111+0000 7f0600bc9640 1 -- 172.21.15.26:0/2793181196 reap_dead start 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.112+0000 7f060262c640 1 -- 172.21.15.26:0/2793181196 shutdown_connections 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.112+0000 7f060262c640 1 -- 172.21.15.26:0/2793181196 >> 172.21.15.26:0/2793181196 conn(0x7f05fc0d40f0 msgr2=0x7f05fc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.112+0000 7f060262c640 1 -- 172.21.15.26:0/2793181196 shutdown_connections 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.112+0000 7f060262c640 1 -- 172.21.15.26:0/2793181196 wait complete. 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.113+0000 7f060262c640 1 Processor -- start 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.113+0000 7f060262c640 1 -- start start 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f060262c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05e8004d40 0x7f05fc1e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f060262c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc155c10 0x7f05fc1f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f060262c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05fc0c7c00 0x7f05fc0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f060262c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc0b5230 con 0x7f05fc155c10 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f060262c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc0b3d00 con 0x7f05fc0c7c00 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f060262c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc0b53b0 con 0x7f05e8004d40 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc155c10 0x7f05fc1f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc155c10 0x7f05fc1f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39768/0 (socket says 172.21.15.26:39768) 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fb7fe640 1 -- 172.21.15.26:0/3286223297 learned_addr learned my addr 172.21.15.26:0/3286223297 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f0600bc9640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05fc0c7c00 0x7f05fc0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fbfff640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05e8004d40 0x7f05fc1e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fb7fe640 1 -- 172.21.15.26:0/3286223297 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05e8004d40 msgr2=0x7f05fc1e6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fb7fe640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05e8004d40 0x7f05fc1e6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fb7fe640 1 -- 172.21.15.26:0/3286223297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05fc0c7c00 msgr2=0x7f05fc0c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fb7fe640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05fc0c7c00 0x7f05fc0c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fb7fe640 1 -- 172.21.15.26:0/3286223297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc1e7400 con 0x7f05fc155c10 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f0600bc9640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05fc0c7c00 0x7f05fc0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05fb7fe640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc155c10 0x7f05fc1f3370 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0119c0 tx=0x7f05ec011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.114+0000 7f05f8ff9640 1 -- 172.21.15.26:0/3286223297 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec00fb50 con 0x7f05fc155c10 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.115+0000 7f05f8ff9640 1 -- 172.21.15.26:0/3286223297 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05ec00fcf0 con 0x7f05fc155c10 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.115+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1e76e0 con 0x7f05fc155c10 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.115+0000 7f05f8ff9640 1 -- 172.21.15.26:0/3286223297 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec00fea0 con 0x7f05fc155c10 2024-04-24T23:24:06.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.115+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1e7ca0 con 0x7f05fc155c10 2024-04-24T23:24:06.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.116+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc0d4c10 con 0x7f05fc155c10 2024-04-24T23:24:06.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.116+0000 7f05f8ff9640 1 -- 172.21.15.26:0/3286223297 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05ec012040 con 0x7f05fc155c10 2024-04-24T23:24:06.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.116+0000 7f05f8ff9640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05dc06be30 0x7f05dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.116+0000 7f05f8ff9640 1 -- 172.21.15.26:0/3286223297 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5115+0+0 (secure 0 0 0) 0x7f05ec08b290 con 0x7f05fc155c10 2024-04-24T23:24:06.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.116+0000 7f05fbfff640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05dc06be30 0x7f05dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.117+0000 7f05fbfff640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05dc06be30 0x7f05dc06e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05e4000fb0 tx=0x7f05e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:06.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.121+0000 7f05f8ff9640 1 -- 172.21.15.26:0/3286223297 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f05fc0d4c10 con 0x7f05fc155c10 2024-04-24T23:24:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.306+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f05fc075610 con 0x7f05fc155c10 2024-04-24T23:24:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.309+0000 7f05f8ff9640 1 -- 172.21.15.26:0/3286223297 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f05ec019030 con 0x7f05fc155c10 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05dc06be30 msgr2=0x7f05dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05dc06be30 0x7f05dc06e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05e4000fb0 tx=0x7f05e4009040 comp rx=0 tx=0).stop 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc155c10 msgr2=0x7f05fc1f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc155c10 0x7f05fc1f3370 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0119c0 tx=0x7f05ec011e80 comp rx=0 tx=0).stop 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 shutdown_connections 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f05dc06be30 0x7f05dc06e2f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05fc0c7c00 0x7f05fc0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc155c10 0x7f05fc1f3370 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 --2- 172.21.15.26:0/3286223297 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05e8004d40 0x7f05fc1e6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.311+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 >> 172.21.15.26:0/3286223297 conn(0x7f05fc0d40f0 msgr2=0x7f05fc156760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.312+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 shutdown_connections 2024-04-24T23:24:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.312+0000 7f060262c640 1 -- 172.21.15.26:0/3286223297 wait complete. 2024-04-24T23:24:06.326 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-04-24T23:24:06.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.425+0000 7f20b5631640 1 Processor -- start 2024-04-24T23:24:06.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.426+0000 7f20b5631640 1 -- start start 2024-04-24T23:24:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.426+0000 7f20b5631640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20b0143c40 0x7f20b00cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.426+0000 7f20b5631640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b00cd0b0 con 0x7f20b00cc860 2024-04-24T23:24:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.426+0000 7f20b5631640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b0071240 con 0x7f20b015eb00 2024-04-24T23:24:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.426+0000 7f20b5631640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f20b0071b70 con 0x7f20b0143c40 2024-04-24T23:24:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.427+0000 7f20aeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20b0143c40 0x7f20b00cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.427+0000 7f20ae7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f20b00cc860 0x7f20b0072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46668/0 (socket says 172.21.15.26:46668) 2024-04-24T23:24:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.427+0000 7f20ae7fc640 1 -- 172.21.15.26:0/1153938976 learned_addr learned my addr 172.21.15.26:0/1153938976 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.427+0000 7f20aeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20b0143c40 0x7f20b00cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39784/0 (socket says 172.21.15.26:39784) 2024-04-24T23:24:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.427+0000 7f20aeffd640 1 -- 172.21.15.26:0/1153938976 >> v1:172.21.15.29:6789/0 conn(0x7f20b015eb00 legacy=0x7f20b015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.427+0000 7f20aeffd640 1 -- 172.21.15.26:0/1153938976 >> v1:172.21.15.26:6789/0 conn(0x7f20b00cc860 legacy=0x7f20b0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.427+0000 7f20aeffd640 1 -- 172.21.15.26:0/1153938976 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b0071520 con 0x7f20b0143c40 2024-04-24T23:24:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.427+0000 7f20adffb640 1 -- 172.21.15.26:0/1153938976 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844245436 0 0) 0x7f20b00cd0b0 con 0x7f20b00cc860 2024-04-24T23:24:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20aeffd640 1 --2- 172.21.15.26:0/1153938976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b0143c40 0x7f20b00cf870 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f20a00097a0 tx=0x7f20a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a32a7f2beaa219 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20adffb640 1 -- 172.21.15.26:0/1153938976 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0037070 con 0x7f20b0143c40 2024-04-24T23:24:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20adffb640 1 -- 172.21.15.26:0/1153938976 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a0032440 con 0x7f20b0143c40 2024-04-24T23:24:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20adffb640 1 -- 172.21.15.26:0/1153938976 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0032810 con 0x7f20b0143c40 2024-04-24T23:24:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20b5631640 1 -- 172.21.15.26:0/1153938976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b0143c40 msgr2=0x7f20b00cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20b5631640 1 --2- 172.21.15.26:0/1153938976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b0143c40 0x7f20b00cf870 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f20a00097a0 tx=0x7f20a002a230 comp rx=0 tx=0).stop 2024-04-24T23:24:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20b5631640 1 -- 172.21.15.26:0/1153938976 shutdown_connections 2024-04-24T23:24:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20b5631640 1 --2- 172.21.15.26:0/1153938976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b0143c40 0x7f20b00cf870 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20b5631640 1 -- 172.21.15.26:0/1153938976 >> 172.21.15.26:0/1153938976 conn(0x7f20b006b300 msgr2=0x7f20b006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:06.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20b5631640 1 -- 172.21.15.26:0/1153938976 shutdown_connections 2024-04-24T23:24:06.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.428+0000 7f20b5631640 1 -- 172.21.15.26:0/1153938976 wait complete. 2024-04-24T23:24:06.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20b5631640 1 Processor -- start 2024-04-24T23:24:06.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20b5631640 1 -- start start 2024-04-24T23:24:06.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20b5631640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b00cc860 0x7f20b015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20b5631640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20b0143c40 0x7f20b00aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20b5631640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20b015eb00 0x7f20b0142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20b5631640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20b00cccd0 con 0x7f20b00cc860 2024-04-24T23:24:06.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20b5631640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0144020 con 0x7f20b0143c40 2024-04-24T23:24:06.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20b5631640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f20b00cce50 con 0x7f20b015eb00 2024-04-24T23:24:06.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20aeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b00cc860 0x7f20b015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20aeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b00cc860 0x7f20b015a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39788/0 (socket says 172.21.15.26:39788) 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.430+0000 7f20aeffd640 1 -- 172.21.15.26:0/3049012063 learned_addr learned my addr 172.21.15.26:0/3049012063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20ae7fc640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20b0143c40 0x7f20b00aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20af7fe640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20b015eb00 0x7f20b0142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20aeffd640 1 -- 172.21.15.26:0/3049012063 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20b015eb00 msgr2=0x7f20b0142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20aeffd640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20b015eb00 0x7f20b0142ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20aeffd640 1 -- 172.21.15.26:0/3049012063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20b0143c40 msgr2=0x7f20b00aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20aeffd640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20b0143c40 0x7f20b00aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20aeffd640 1 -- 172.21.15.26:0/3049012063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b0141a40 con 0x7f20b00cc860 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20ae7fc640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20b0143c40 0x7f20b00aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20aeffd640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b00cc860 0x7f20b015a9f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f20a00098c0 tx=0x7f20a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20977fe640 1 -- 172.21.15.26:0/3049012063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0037040 con 0x7f20b00cc860 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20977fe640 1 -- 172.21.15.26:0/3049012063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a0042070 con 0x7f20b00cc860 2024-04-24T23:24:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20977fe640 1 -- 172.21.15.26:0/3049012063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0030a70 con 0x7f20b00cc860 2024-04-24T23:24:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b0141d20 con 0x7f20b00cc860 2024-04-24T23:24:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.431+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b01422e0 con 0x7f20b00cc860 2024-04-24T23:24:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.432+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b0075610 con 0x7f20b00cc860 2024-04-24T23:24:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.433+0000 7f20977fe640 1 -- 172.21.15.26:0/3049012063 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20a0030c10 con 0x7f20b00cc860 2024-04-24T23:24:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.433+0000 7f20977fe640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f209c06be60 0x7f209c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:06.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.433+0000 7f20977fe640 1 -- 172.21.15.26:0/3049012063 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5115+0+0 (secure 0 0 0) 0x7f20a00ab5d0 con 0x7f20b00cc860 2024-04-24T23:24:06.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.433+0000 7f20ae7fc640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f209c06be60 0x7f209c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:06.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.433+0000 7f20ae7fc640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f209c06be60 0x7f209c06e320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f20a4000f60 tx=0x7f20a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:06.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.437+0000 7f20977fe640 1 -- 172.21.15.26:0/3049012063 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f20b0075610 con 0x7f20b00cc860 2024-04-24T23:24:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.695+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) -- 0x7f20b00b4d20 con 0x7f20b00cc860 2024-04-24T23:24:06.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.980+0000 7f20977fe640 1 -- 172.21.15.26:0/3049012063 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v178) ==== 143+0+0 (secure 0 0 0) 0x7f20a0045360 con 0x7f20b00cc860 2024-04-24T23:24:06.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f209c06be60 msgr2=0x7f209c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f209c06be60 0x7f209c06e320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f20a4000f60 tx=0x7f20a4009040 comp rx=0 tx=0).stop 2024-04-24T23:24:06.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b00cc860 msgr2=0x7f20b015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:06.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b00cc860 0x7f20b015a9f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f20a00098c0 tx=0x7f20a00046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 shutdown_connections 2024-04-24T23:24:06.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f209c06be60 0x7f209c06e320 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20b015eb00 0x7f20b0142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20b0143c40 0x7f20b00aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 --2- 172.21.15.26:0/3049012063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b00cc860 0x7f20b015a9f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:06.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.983+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 >> 172.21.15.26:0/3049012063 conn(0x7f20b006b300 msgr2=0x7f20b00b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:06.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.984+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 shutdown_connections 2024-04-24T23:24:06.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:06.984+0000 7f20b5631640 1 -- 172.21.15.26:0/3049012063 wait complete. 2024-04-24T23:24:07.000 INFO:tasks.cephfs_test_runner:test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-04-24T23:24:07.000 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-04-24T23:24:07.000 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:24:07.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.101+0000 7f8f639ee640 1 Processor -- start 2024-04-24T23:24:07.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.102+0000 7f8f639ee640 1 -- start start 2024-04-24T23:24:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.102+0000 7f8f639ee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f5c0b3130 0x7f8f5c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.102+0000 7f8f639ee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f5c0d32f0 con 0x7f8f5c0d4870 2024-04-24T23:24:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.102+0000 7f8f639ee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f5c0d0270 con 0x7f8f5c15e570 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.102+0000 7f8f639ee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8f5c0d1ab0 con 0x7f8f5c0b3130 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f60f89640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8f5c0d4870 0x7f8f5c0b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46676/0 (socket says 172.21.15.26:46676) 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f60f89640 1 -- 172.21.15.26:0/579721214 learned_addr learned my addr 172.21.15.26:0/579721214 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f61f8b640 1 --2- 172.21.15.26:0/579721214 >> v2:172.21.15.29:3300/0 conn(0x7f8f5c0b3130 0x7f8f5c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3728774658 0 0) 0x7f8f4c001110 con 0x7f8f5c0d4870 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f48003520 con 0x7f8f5c0d4870 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3500898993 0 0) 0x7f8f48003520 con 0x7f8f5c0d4870 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 >> v2:172.21.15.29:3300/0 conn(0x7f8f5c0b3130 msgr2=0x7f8f5c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 --2- 172.21.15.26:0/579721214 >> v2:172.21.15.29:3300/0 conn(0x7f8f5c0b3130 0x7f8f5c0b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 >> v1:172.21.15.29:6789/0 conn(0x7f8f5c15e570 legacy=0x7f8f5c0d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5c0d2d30 con 0x7f8f5c0d4870 2024-04-24T23:24:07.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f8f4c001840 con 0x7f8f5c0d4870 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 >> v1:172.21.15.26:6789/0 conn(0x7f8f5c0d4870 legacy=0x7f8f5c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f48003a00 0x7f8f48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f48005210 0x7f8f4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c0d1ab0 con 0x7f8f48004800 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4800c160 con 0x7f8f48005210 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4800c360 con 0x7f8f48003a00 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f60f89640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f61f8b640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f48005210 0x7f8f4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.103+0000 7f8f6178a640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f48003a00 0x7f8f48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f60f89640 1 -- 172.21.15.26:0/579721214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f48003a00 msgr2=0x7f8f48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f60f89640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f48003a00 0x7f8f48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f60f89640 1 -- 172.21.15.26:0/579721214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f48005210 msgr2=0x7f8f4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f60f89640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f48005210 0x7f8f4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f60f89640 1 -- 172.21.15.26:0/579721214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f48003520 con 0x7f8f48004800 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f61f8b640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f48005210 0x7f8f4800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f61f8b640 1 -- 172.21.15.26:0/579721214 reap_dead start 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f60f89640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f48004c80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c001050 tx=0x7f8f48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8785e0eeaf0ae57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c036070 con 0x7f8f48004800 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c005af0 con 0x7f8f48004800 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.104+0000 7f8f53fff640 1 -- 172.21.15.26:0/579721214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c007e10 con 0x7f8f48004800 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.105+0000 7f8f639ee640 1 -- 172.21.15.26:0/579721214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 msgr2=0x7f8f48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.105+0000 7f8f639ee640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f48004c80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c001050 tx=0x7f8f48005830 comp rx=0 tx=0).stop 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.105+0000 7f8f639ee640 1 -- 172.21.15.26:0/579721214 shutdown_connections 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.105+0000 7f8f639ee640 1 --2- 172.21.15.26:0/579721214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f48004c80 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.105+0000 7f8f639ee640 1 -- 172.21.15.26:0/579721214 >> 172.21.15.26:0/579721214 conn(0x7f8f5c08dbe0 msgr2=0x7f8f5c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:07.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.105+0000 7f8f639ee640 1 -- 172.21.15.26:0/579721214 shutdown_connections 2024-04-24T23:24:07.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.105+0000 7f8f639ee640 1 -- 172.21.15.26:0/579721214 wait complete. 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.106+0000 7f8f639ee640 1 Processor -- start 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.106+0000 7f8f639ee640 1 -- start start 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f639ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f5c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f639ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f5c0b7a10 0x7f8f5c14f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f639ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f5c14f6c0 0x7f8f5c1f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f639ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c0cfa70 con 0x7f8f48004800 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f639ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c0d2ae0 con 0x7f8f5c14f6c0 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f639ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5c0cfd50 con 0x7f8f5c0b7a10 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f6178a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f5c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f6178a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f5c0b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39804/0 (socket says 172.21.15.26:39804) 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f6178a640 1 -- 172.21.15.26:0/1905500698 learned_addr learned my addr 172.21.15.26:0/1905500698 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:07.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f61f8b640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f5c14f6c0 0x7f8f5c1f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f60f89640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f5c0b7a10 0x7f8f5c14f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f6178a640 1 -- 172.21.15.26:0/1905500698 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f5c0b7a10 msgr2=0x7f8f5c14f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f6178a640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f5c0b7a10 0x7f8f5c14f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f6178a640 1 -- 172.21.15.26:0/1905500698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f5c14f6c0 msgr2=0x7f8f5c1f7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f6178a640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f5c14f6c0 0x7f8f5c1f7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f6178a640 1 -- 172.21.15.26:0/1905500698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5c14fb40 con 0x7f8f48004800 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.107+0000 7f8f61f8b640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f5c14f6c0 0x7f8f5c1f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.108+0000 7f8f6178a640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f5c0b4810 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400e4d0 tx=0x7f8f5400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.108+0000 7f8f51ffb640 1 -- 172.21.15.26:0/1905500698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f540053a0 con 0x7f8f48004800 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.108+0000 7f8f51ffb640 1 -- 172.21.15.26:0/1905500698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54018070 con 0x7f8f48004800 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.108+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5c1f8420 con 0x7f8f48004800 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.108+0000 7f8f51ffb640 1 -- 172.21.15.26:0/1905500698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54003810 con 0x7f8f48004800 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.108+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5c0b7210 con 0x7f8f48004800 2024-04-24T23:24:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.109+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f5c0694c0 con 0x7f8f48004800 2024-04-24T23:24:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.109+0000 7f8f51ffb640 1 -- 172.21.15.26:0/1905500698 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f540039b0 con 0x7f8f48004800 2024-04-24T23:24:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.110+0000 7f8f51ffb640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4006be30 0x7f8f4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.110+0000 7f8f51ffb640 1 -- 172.21.15.26:0/1905500698 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5115+0+0 (secure 0 0 0) 0x7f8f5408a950 con 0x7f8f48004800 2024-04-24T23:24:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.110+0000 7f8f60f89640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4006be30 0x7f8f4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.110+0000 7f8f60f89640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4006be30 0x7f8f4006e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c033000 tx=0x7f8f4c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.114+0000 7f8f51ffb640 1 -- 172.21.15.26:0/1905500698 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8f54059550 con 0x7f8f48004800 2024-04-24T23:24:07.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.297+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f5c06cb90 con 0x7f8f48004800 2024-04-24T23:24:07.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.298+0000 7f8f51ffb640 1 -- 172.21.15.26:0/1905500698 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+13216 (secure 0 0 0) 0x7f8f54058e50 con 0x7f8f48004800 2024-04-24T23:24:07.299 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:07.299 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:24:05.976092+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6836/730964888":"2024-04-25T23:24:02.622422+0000","172.21.15.26:6834/730964888":"2024-04-25T23:24:02.622422+0000","172.21.15.29:6835/1578186344":"2024-04-25T23:24:02.622405+0000","172.21.15.29:6832/1578186344":"2024-04-25T23:24:02.622405+0000","172.21.15.29:6834/243988267":"2024-04-25T23:24:02.622417+0000","172.21.15.29:6833/243988267":"2024-04-25T23:24:02.622417+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:24:07.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4006be30 msgr2=0x7f8f4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4006be30 0x7f8f4006e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c033000 tx=0x7f8f4c009030 comp rx=0 tx=0).stop 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 msgr2=0x7f8f5c0b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f5c0b4810 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400e4d0 tx=0x7f8f5400e990 comp rx=0 tx=0).stop 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 shutdown_connections 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4006be30 0x7f8f4006e2f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f5c14f6c0 0x7f8f5c1f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f5c0b7a10 0x7f8f5c14f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 --2- 172.21.15.26:0/1905500698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f48004800 0x7f8f5c0b4810 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 >> 172.21.15.26:0/1905500698 conn(0x7f8f5c08dbe0 msgr2=0x7f8f5c06d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 shutdown_connections 2024-04-24T23:24:07.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.300+0000 7f8f639ee640 1 -- 172.21.15.26:0/1905500698 wait complete. 2024-04-24T23:24:07.313 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:24:07.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:24:07.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.416+0000 7f75fa342640 1 Processor -- start 2024-04-24T23:24:07.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.417+0000 7f75fa342640 1 -- start start 2024-04-24T23:24:07.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.417+0000 7f75fa342640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f40d0ce0 0x7f75f406e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.417+0000 7f75fa342640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75f4071720 0x7f75f4071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.417+0000 7f75fa342640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75f406b1c0 con 0x7f75f4158ae0 2024-04-24T23:24:07.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.417+0000 7f75fa342640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f75f40720d0 con 0x7f75f4071720 2024-04-24T23:24:07.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.417+0000 7f75fa342640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f75f406ec90 con 0x7f75f40d0ce0 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f88df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75f4071720 0x7f75f4071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f75f4158ae0 0x7f75f415e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46690/0 (socket says 172.21.15.26:46690) 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f88df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75f4071720 0x7f75f4071b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39806/0 (socket says 172.21.15.26:39806) 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f3fff640 1 -- 172.21.15.26:0/3666375696 learned_addr learned my addr 172.21.15.26:0/3666375696 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f37fe640 1 --2- 172.21.15.26:0/3666375696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f40d0ce0 0x7f75f406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f88df640 1 -- 172.21.15.26:0/3666375696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f40d0ce0 msgr2=0x7f75f406e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f88df640 1 --2- 172.21.15.26:0/3666375696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f40d0ce0 0x7f75f406e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f88df640 1 -- 172.21.15.26:0/3666375696 >> v1:172.21.15.26:6789/0 conn(0x7f75f4158ae0 legacy=0x7f75f415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f2ffd640 1 -- 172.21.15.26:0/3666375696 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748565763 0 0) 0x7f75f406b1c0 con 0x7f75f4158ae0 2024-04-24T23:24:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.418+0000 7f75f88df640 1 -- 172.21.15.26:0/3666375696 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f406ef70 con 0x7f75f4071720 2024-04-24T23:24:07.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75f88df640 1 --2- 172.21.15.26:0/3666375696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4071720 0x7f75f4071b90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009d90 tx=0x7f75ec02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d614573c4dc191 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75f2ffd640 1 -- 172.21.15.26:0/3666375696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec031600 con 0x7f75f4071720 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75f2ffd640 1 -- 172.21.15.26:0/3666375696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec0317a0 con 0x7f75f4071720 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75f2ffd640 1 -- 172.21.15.26:0/3666375696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec031b70 con 0x7f75f4071720 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75fa342640 1 -- 172.21.15.26:0/3666375696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4071720 msgr2=0x7f75f4071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75fa342640 1 --2- 172.21.15.26:0/3666375696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4071720 0x7f75f4071b90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009d90 tx=0x7f75ec02a5d0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75fa342640 1 -- 172.21.15.26:0/3666375696 shutdown_connections 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75fa342640 1 --2- 172.21.15.26:0/3666375696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4071720 0x7f75f4071b90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75fa342640 1 --2- 172.21.15.26:0/3666375696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f40d0ce0 0x7f75f406e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75fa342640 1 -- 172.21.15.26:0/3666375696 >> 172.21.15.26:0/3666375696 conn(0x7f75f40b2970 msgr2=0x7f75f40d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75fa342640 1 -- 172.21.15.26:0/3666375696 shutdown_connections 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.419+0000 7f75fa342640 1 -- 172.21.15.26:0/3666375696 wait complete. 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.420+0000 7f75fa342640 1 Processor -- start 2024-04-24T23:24:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.421+0000 7f75fa342640 1 -- start start 2024-04-24T23:24:07.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.421+0000 7f75fa342640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f4071720 0x7f75f41575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.421+0000 7f75fa342640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75f40d0ce0 0x7f75f40b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.421+0000 7f75fa342640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4158ae0 0x7f75f40b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.421+0000 7f75fa342640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75f406f530 con 0x7f75f4158ae0 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.421+0000 7f75fa342640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f75f415f060 con 0x7f75f40d0ce0 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.421+0000 7f75fa342640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f75f40aff60 con 0x7f75f4071720 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.421+0000 7f75f88df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4158ae0 0x7f75f40b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f88df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4158ae0 0x7f75f40b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39814/0 (socket says 172.21.15.26:39814) 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f88df640 1 -- 172.21.15.26:0/3715231252 learned_addr learned my addr 172.21.15.26:0/3715231252 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f3fff640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f4071720 0x7f75f41575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f37fe640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75f40d0ce0 0x7f75f40b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f88df640 1 -- 172.21.15.26:0/3715231252 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f4071720 msgr2=0x7f75f41575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f88df640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f4071720 0x7f75f41575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f88df640 1 -- 172.21.15.26:0/3715231252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75f40d0ce0 msgr2=0x7f75f40b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f88df640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75f40d0ce0 0x7f75f40b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f88df640 1 -- 172.21.15.26:0/3715231252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f40b5180 con 0x7f75f4158ae0 2024-04-24T23:24:07.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f88df640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4158ae0 0x7f75f40b4c40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009d90 tx=0x7f75ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:07.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f0ff9640 1 -- 172.21.15.26:0/3715231252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec03f070 con 0x7f75f4158ae0 2024-04-24T23:24:07.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f0ff9640 1 -- 172.21.15.26:0/3715231252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec030b60 con 0x7f75f4158ae0 2024-04-24T23:24:07.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f40b3ba0 con 0x7f75f4158ae0 2024-04-24T23:24:07.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75f0ff9640 1 -- 172.21.15.26:0/3715231252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec009450 con 0x7f75f4158ae0 2024-04-24T23:24:07.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.422+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f40b4160 con 0x7f75f4158ae0 2024-04-24T23:24:07.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.423+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f4075610 con 0x7f75f4158ae0 2024-04-24T23:24:07.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.424+0000 7f75f0ff9640 1 -- 172.21.15.26:0/3715231252 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75ec03b410 con 0x7f75f4158ae0 2024-04-24T23:24:07.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.424+0000 7f75f0ff9640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75e006be30 0x7f75e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.424+0000 7f75f0ff9640 1 -- 172.21.15.26:0/3715231252 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5115+0+0 (secure 0 0 0) 0x7f75ec0abdb0 con 0x7f75f4158ae0 2024-04-24T23:24:07.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.424+0000 7f75f3fff640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75e006be30 0x7f75e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.425+0000 7f75f3fff640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75e006be30 0x7f75e006e2f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f75e4000f60 tx=0x7f75e4007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:07.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.429+0000 7f75f0ff9640 1 -- 172.21.15.26:0/3715231252 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75ec0b0070 con 0x7f75f4158ae0 2024-04-24T23:24:07.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.610+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75f406e940 con 0x7f75f4158ae0 2024-04-24T23:24:07.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.610+0000 7f75f0ff9640 1 -- 172.21.15.26:0/3715231252 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f75ec07a2c0 con 0x7f75f4158ae0 2024-04-24T23:24:07.611 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:24:07.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75e006be30 msgr2=0x7f75e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75e006be30 0x7f75e006e2f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f75e4000f60 tx=0x7f75e4007d50 comp rx=0 tx=0).stop 2024-04-24T23:24:07.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4158ae0 msgr2=0x7f75f40b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4158ae0 0x7f75f40b4c40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009d90 tx=0x7f75ec004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:07.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 shutdown_connections 2024-04-24T23:24:07.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75e006be30 0x7f75e006e2f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75f4158ae0 0x7f75f40b4c40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75f40d0ce0 0x7f75f40b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 --2- 172.21.15.26:0/3715231252 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75f4071720 0x7f75f41575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 >> 172.21.15.26:0/3715231252 conn(0x7f75f40b2970 msgr2=0x7f75f415e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:07.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 shutdown_connections 2024-04-24T23:24:07.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.612+0000 7f75fa342640 1 -- 172.21.15.26:0/3715231252 wait complete. 2024-04-24T23:24:07.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-04-24T23:24:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.724+0000 7f050f088640 1 Processor -- start 2024-04-24T23:24:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.724+0000 7f050f088640 1 -- start start 2024-04-24T23:24:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.724+0000 7f050f088640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f05080cf870 0x7f0508072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.724+0000 7f050f088640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f050815eb00 0x7f050815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.724+0000 7f050f088640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f050815f450 con 0x7f0508143c40 2024-04-24T23:24:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.724+0000 7f050f088640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0508071b70 con 0x7f050815eb00 2024-04-24T23:24:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.724+0000 7f050f088640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f05080cd0b0 con 0x7f05080cf870 2024-04-24T23:24:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.725+0000 7f050d625640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f050815eb00 0x7f050815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.725+0000 7f050d625640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f050815eb00 0x7f050815ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39824/0 (socket says 172.21.15.26:39824) 2024-04-24T23:24:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.725+0000 7f050d625640 1 -- 172.21.15.26:0/2982448526 learned_addr learned my addr 172.21.15.26:0/2982448526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.725+0000 7f0507fff640 1 --2- 172.21.15.26:0/2982448526 >> v2:172.21.15.29:3300/0 conn(0x7f05080cf870 0x7f0508072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.725+0000 7f050d625640 1 -- 172.21.15.26:0/2982448526 >> v2:172.21.15.29:3300/0 conn(0x7f05080cf870 msgr2=0x7f0508072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.725+0000 7f050d625640 1 --2- 172.21.15.26:0/2982448526 >> v2:172.21.15.29:3300/0 conn(0x7f05080cf870 0x7f0508072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.725+0000 7f050d625640 1 -- 172.21.15.26:0/2982448526 >> v1:172.21.15.29:6789/0 conn(0x7f0508143c40 legacy=0x7f05080cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.725+0000 7f050d625640 1 -- 172.21.15.26:0/2982448526 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0508071240 con 0x7f050815eb00 2024-04-24T23:24:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f050d625640 1 --2- 172.21.15.26:0/2982448526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 0x7f050815ef10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f050806db50 tx=0x7f050002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4069eb80d26ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f05077fe640 1 -- 172.21.15.26:0/2982448526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500036070 con 0x7f050815eb00 2024-04-24T23:24:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f05077fe640 1 -- 172.21.15.26:0/2982448526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0500031600 con 0x7f050815eb00 2024-04-24T23:24:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f05077fe640 1 -- 172.21.15.26:0/2982448526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05000319d0 con 0x7f050815eb00 2024-04-24T23:24:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f050f088640 1 -- 172.21.15.26:0/2982448526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 msgr2=0x7f050815ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f050f088640 1 --2- 172.21.15.26:0/2982448526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 0x7f050815ef10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f050806db50 tx=0x7f050002a730 comp rx=0 tx=0).stop 2024-04-24T23:24:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f050f088640 1 -- 172.21.15.26:0/2982448526 shutdown_connections 2024-04-24T23:24:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f050f088640 1 --2- 172.21.15.26:0/2982448526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 0x7f050815ef10 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f050f088640 1 --2- 172.21.15.26:0/2982448526 >> v2:172.21.15.29:3300/0 conn(0x7f05080cf870 0x7f0508072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f050f088640 1 -- 172.21.15.26:0/2982448526 >> 172.21.15.26:0/2982448526 conn(0x7f050806b300 msgr2=0x7f050806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f050f088640 1 -- 172.21.15.26:0/2982448526 shutdown_connections 2024-04-24T23:24:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.726+0000 7f050f088640 1 -- 172.21.15.26:0/2982448526 wait complete. 2024-04-24T23:24:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.728+0000 7f050f088640 1 Processor -- start 2024-04-24T23:24:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.728+0000 7f050f088640 1 -- start start 2024-04-24T23:24:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.728+0000 7f050f088640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05080cf870 0x7f0508134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050f088640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0508143c40 0x7f05080b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050f088640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 0x7f05080b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050f088640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0508144020 con 0x7f050815eb00 2024-04-24T23:24:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050f088640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0508071800 con 0x7f05080cf870 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050f088640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0508072540 con 0x7f0508143c40 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050d625640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 0x7f05080b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050d625640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 0x7f05080b2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39826/0 (socket says 172.21.15.26:39826) 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050d625640 1 -- 172.21.15.26:0/4095511791 learned_addr learned my addr 172.21.15.26:0/4095511791 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f0507fff640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0508143c40 0x7f05080b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050ce24640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05080cf870 0x7f0508134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050d625640 1 -- 172.21.15.26:0/4095511791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0508143c40 msgr2=0x7f05080b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050d625640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0508143c40 0x7f05080b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050d625640 1 -- 172.21.15.26:0/4095511791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05080cf870 msgr2=0x7f0508134730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050d625640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05080cf870 0x7f0508134730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050d625640 1 -- 172.21.15.26:0/4095511791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050815c130 con 0x7f050815eb00 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f0507fff640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0508143c40 0x7f05080b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050d625640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 0x7f05080b2020 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0500009f40 tx=0x7f0500004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:07.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f05057fa640 1 -- 172.21.15.26:0/4095511791 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500036050 con 0x7f050815eb00 2024-04-24T23:24:07.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f05057fa640 1 -- 172.21.15.26:0/4095511791 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0500042070 con 0x7f050815eb00 2024-04-24T23:24:07.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050815c410 con 0x7f050815eb00 2024-04-24T23:24:07.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f05057fa640 1 -- 172.21.15.26:0/4095511791 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500009510 con 0x7f050815eb00 2024-04-24T23:24:07.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.729+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050815c9d0 con 0x7f050815eb00 2024-04-24T23:24:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.731+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05080694c0 con 0x7f050815eb00 2024-04-24T23:24:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.731+0000 7f05057fa640 1 -- 172.21.15.26:0/4095511791 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0500007bf0 con 0x7f050815eb00 2024-04-24T23:24:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.731+0000 7f05057fa640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f04ec06be30 0x7f04ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.731+0000 7f05057fa640 1 -- 172.21.15.26:0/4095511791 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5115+0+0 (secure 0 0 0) 0x7f05000aba10 con 0x7f050815eb00 2024-04-24T23:24:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.731+0000 7f050ce24640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f04ec06be30 0x7f04ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.732+0000 7f050ce24640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f04ec06be30 0x7f04ec06e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0015a0 tx=0x7f04fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.736+0000 7f05057fa640 1 -- 172.21.15.26:0/4095511791 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f050007a6a0 con 0x7f050815eb00 2024-04-24T23:24:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:07.996+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) -- 0x7f05080b4d20 con 0x7f050815eb00 2024-04-24T23:24:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.986+0000 7f05057fa640 1 -- 172.21.15.26:0/4095511791 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v180) ==== 145+0+0 (secure 0 0 0) 0x7f0500079fa0 con 0x7f050815eb00 2024-04-24T23:24:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.989+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f04ec06be30 msgr2=0x7f04ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.989+0000 7f050f088640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f04ec06be30 0x7f04ec06e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0015a0 tx=0x7f04fc009040 comp rx=0 tx=0).stop 2024-04-24T23:24:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.989+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 msgr2=0x7f05080b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.989+0000 7f050f088640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 0x7f05080b2020 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0500009f40 tx=0x7f0500004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.990+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 shutdown_connections 2024-04-24T23:24:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.990+0000 7f050f088640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f04ec06be30 0x7f04ec06e2f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.990+0000 7f050f088640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f050815eb00 0x7f05080b2020 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.990+0000 7f050f088640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0508143c40 0x7f05080b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.990+0000 7f050f088640 1 --2- 172.21.15.26:0/4095511791 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05080cf870 0x7f0508134730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.990+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 >> 172.21.15.26:0/4095511791 conn(0x7f050806b300 msgr2=0x7f05080adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.990+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 shutdown_connections 2024-04-24T23:24:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:08.990+0000 7f050f088640 1 -- 172.21.15.26:0/4095511791 wait complete. 2024-04-24T23:24:09.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:24:09.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.114+0000 7f468e15a640 1 Processor -- start 2024-04-24T23:24:09.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.114+0000 7f468e15a640 1 -- start start 2024-04-24T23:24:09.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f468e15a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f46880ad7b0 0x7f46880d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f468e15a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468815ef60 con 0x7f468815e9c0 2024-04-24T23:24:09.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f468e15a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468815f240 con 0x7f468815adc0 2024-04-24T23:24:09.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f468e15a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f468815f520 con 0x7f46880ad7b0 2024-04-24T23:24:09.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f4687fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f468815e9c0 0x7f4688072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46694/0 (socket says 172.21.15.26:46694) 2024-04-24T23:24:09.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f4687fff640 1 -- 172.21.15.26:0/3091496904 learned_addr learned my addr 172.21.15.26:0/3091496904 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:09.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f4686ffd640 1 --2- 172.21.15.26:0/3091496904 >> v2:172.21.15.29:3300/0 conn(0x7f46880ad7b0 0x7f46880d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f46867fc640 1 -- 172.21.15.26:0/3091496904 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1402643392 0 0) 0x7f468815ef60 con 0x7f468815e9c0 2024-04-24T23:24:09.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f46867fc640 1 -- 172.21.15.26:0/3091496904 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4670005730 con 0x7f468815e9c0 2024-04-24T23:24:09.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f4686ffd640 1 -- 172.21.15.26:0/3091496904 >> v1:172.21.15.29:6789/0 conn(0x7f468815adc0 legacy=0x7f46880b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f4686ffd640 1 -- 172.21.15.26:0/3091496904 >> v1:172.21.15.26:6789/0 conn(0x7f468815e9c0 legacy=0x7f4688072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.115+0000 7f4686ffd640 1 -- 172.21.15.26:0/3091496904 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468815f800 con 0x7f46880ad7b0 2024-04-24T23:24:09.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.116+0000 7f4686ffd640 1 --2- 172.21.15.26:0/3091496904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46880ad7b0 0x7f46880d0040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f467c0097d0 tx=0x7f467c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e253c7b14b6b0674 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.116+0000 7f46867fc640 1 -- 172.21.15.26:0/3091496904 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c037070 con 0x7f46880ad7b0 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.116+0000 7f46867fc640 1 -- 172.21.15.26:0/3091496904 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c02ae40 con 0x7f46880ad7b0 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.116+0000 7f46867fc640 1 -- 172.21.15.26:0/3091496904 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c032640 con 0x7f46880ad7b0 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.117+0000 7f468e15a640 1 -- 172.21.15.26:0/3091496904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46880ad7b0 msgr2=0x7f46880d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.117+0000 7f468e15a640 1 --2- 172.21.15.26:0/3091496904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46880ad7b0 0x7f46880d0040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f467c0097d0 tx=0x7f467c02a130 comp rx=0 tx=0).stop 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.117+0000 7f468e15a640 1 -- 172.21.15.26:0/3091496904 shutdown_connections 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.117+0000 7f468e15a640 1 --2- 172.21.15.26:0/3091496904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46880ad7b0 0x7f46880d0040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.117+0000 7f468e15a640 1 -- 172.21.15.26:0/3091496904 >> 172.21.15.26:0/3091496904 conn(0x7f468806b540 msgr2=0x7f468806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.117+0000 7f468e15a640 1 -- 172.21.15.26:0/3091496904 shutdown_connections 2024-04-24T23:24:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.117+0000 7f468e15a640 1 -- 172.21.15.26:0/3091496904 wait complete. 2024-04-24T23:24:09.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.118+0000 7f468e15a640 1 Processor -- start 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.118+0000 7f468e15a640 1 -- start start 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f468e15a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46880ad7b0 0x7f46881f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f468e15a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468815adc0 0x7f46880cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f468e15a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f468815e9c0 0x7f468809ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f468e15a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f46880d0580 con 0x7f468815adc0 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f468e15a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f468815b1a0 con 0x7f46880ad7b0 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f468e15a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f46880d0700 con 0x7f468815e9c0 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f4686ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468815adc0 0x7f46880cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f4686ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468815adc0 0x7f46880cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39836/0 (socket says 172.21.15.26:39836) 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f4686ffd640 1 -- 172.21.15.26:0/2458018038 learned_addr learned my addr 172.21.15.26:0/2458018038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f46877fe640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46880ad7b0 0x7f46881f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.119+0000 7f4687fff640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f468815e9c0 0x7f468809ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f4686ffd640 1 -- 172.21.15.26:0/2458018038 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f468815e9c0 msgr2=0x7f468809ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f4686ffd640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f468815e9c0 0x7f468809ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f4686ffd640 1 -- 172.21.15.26:0/2458018038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46880ad7b0 msgr2=0x7f46881f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f4686ffd640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46880ad7b0 0x7f46881f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f4686ffd640 1 -- 172.21.15.26:0/2458018038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46881f7d00 con 0x7f468815adc0 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f4686ffd640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468815adc0 0x7f46880cb6d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f467c035040 tx=0x7f467c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f466ffff640 1 -- 172.21.15.26:0/2458018038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c037040 con 0x7f468815adc0 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f466ffff640 1 -- 172.21.15.26:0/2458018038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c004030 con 0x7f468815adc0 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46881f7fe0 con 0x7f468815adc0 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f466ffff640 1 -- 172.21.15.26:0/2458018038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c004400 con 0x7f468815adc0 2024-04-24T23:24:09.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.120+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46881f4290 con 0x7f468815adc0 2024-04-24T23:24:09.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.121+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4688075610 con 0x7f468815adc0 2024-04-24T23:24:09.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.121+0000 7f466ffff640 1 -- 172.21.15.26:0/2458018038 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f467c003610 con 0x7f468815adc0 2024-04-24T23:24:09.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.122+0000 7f466ffff640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f467406be30 0x7f467406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.122+0000 7f466ffff640 1 -- 172.21.15.26:0/2458018038 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5115+0+0 (secure 0 0 0) 0x7f467c0ab8a0 con 0x7f468815adc0 2024-04-24T23:24:09.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.122+0000 7f46877fe640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f467406be30 0x7f467406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.122+0000 7f46877fe640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f467406be30 0x7f467406e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f468806d200 tx=0x7f4678009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:09.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.126+0000 7f466ffff640 1 -- 172.21.15.26:0/2458018038 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f467c079f70 con 0x7f468815adc0 2024-04-24T23:24:09.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.308+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f468815e730 con 0x7f468815adc0 2024-04-24T23:24:09.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.311+0000 7f466ffff640 1 -- 172.21.15.26:0/2458018038 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f467c07e270 con 0x7f468815adc0 2024-04-24T23:24:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f467406be30 msgr2=0x7f467406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f467406be30 0x7f467406e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f468806d200 tx=0x7f4678009040 comp rx=0 tx=0).stop 2024-04-24T23:24:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468815adc0 msgr2=0x7f46880cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468815adc0 0x7f46880cb6d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f467c035040 tx=0x7f467c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 shutdown_connections 2024-04-24T23:24:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f467406be30 0x7f467406e2f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f468815e9c0 0x7f468809ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468815adc0 0x7f46880cb6d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 --2- 172.21.15.26:0/2458018038 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46880ad7b0 0x7f46881f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 >> 172.21.15.26:0/2458018038 conn(0x7f468806b540 msgr2=0x7f468806d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 shutdown_connections 2024-04-24T23:24:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.313+0000 7f468e15a640 1 -- 172.21.15.26:0/2458018038 wait complete. 2024-04-24T23:24:09.325 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:24:09.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:24:09.361 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:24:09.403 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:24:09.424 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:24:09.425 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:24:09.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:24:09.445 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:24:09.445 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:24:09.464 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:24:09.465 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:24:09.465 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:24:09.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:24:09.504 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:24:09.504 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:24:09.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:24:09.535 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:24:09.536 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.656+0000 7f0179200640 1 Processor -- start 2024-04-24T23:24:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.656+0000 7f0179200640 1 -- start start 2024-04-24T23:24:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.657+0000 7f0179200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01741623c0 0x7f017406d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.657+0000 7f0179200640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0174162850 0x7f01740b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.657+0000 7f0179200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01740b55b0 con 0x7f017415e200 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.657+0000 7f0179200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f01740b5890 con 0x7f01741623c0 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.657+0000 7f0179200640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f017406b1c0 con 0x7f0174162850 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.657+0000 7f017259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01741623c0 0x7f017406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.657+0000 7f017259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01741623c0 0x7f017406d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39850/0 (socket says 172.21.15.26:39850) 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.657+0000 7f017259c640 1 -- 172.21.15.26:0/1773395113 learned_addr learned my addr 172.21.15.26:0/1773395113 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.657+0000 7f0172d9d640 1 --1- 172.21.15.26:0/1773395113 >> v1:172.21.15.26:6789/0 conn(0x7f017415e200 0x7f0174161cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46696/0 (socket says 172.21.15.26:46696) 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.658+0000 7f017359e640 1 --2- 172.21.15.26:0/1773395113 >> v2:172.21.15.29:3300/0 conn(0x7f0174162850 0x7f01740b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.658+0000 7f017259c640 1 -- 172.21.15.26:0/1773395113 >> v2:172.21.15.29:3300/0 conn(0x7f0174162850 msgr2=0x7f01740b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.658+0000 7f017259c640 1 --2- 172.21.15.26:0/1773395113 >> v2:172.21.15.29:3300/0 conn(0x7f0174162850 0x7f01740b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.658+0000 7f017259c640 1 -- 172.21.15.26:0/1773395113 >> v1:172.21.15.26:6789/0 conn(0x7f017415e200 legacy=0x7f0174161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.658+0000 7f017259c640 1 -- 172.21.15.26:0/1773395113 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017406b4a0 con 0x7f01741623c0 2024-04-24T23:24:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.658+0000 7f0171d9b640 1 -- 172.21.15.26:0/1773395113 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2285967156 0 0) 0x7f01740b55b0 con 0x7f017415e200 2024-04-24T23:24:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.658+0000 7f017259c640 1 --2- 172.21.15.26:0/1773395113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01741623c0 0x7f017406d470 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0174071400 tx=0x7f015c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c30b8aaf4c8fda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0171d9b640 1 -- 172.21.15.26:0/1773395113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c037070 con 0x7f01741623c0 2024-04-24T23:24:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0171d9b640 1 -- 172.21.15.26:0/1773395113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c032440 con 0x7f01741623c0 2024-04-24T23:24:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0171d9b640 1 -- 172.21.15.26:0/1773395113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c032810 con 0x7f01741623c0 2024-04-24T23:24:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0179200640 1 -- 172.21.15.26:0/1773395113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01741623c0 msgr2=0x7f017406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0179200640 1 --2- 172.21.15.26:0/1773395113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01741623c0 0x7f017406d470 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0174071400 tx=0x7f015c02a270 comp rx=0 tx=0).stop 2024-04-24T23:24:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0179200640 1 -- 172.21.15.26:0/1773395113 shutdown_connections 2024-04-24T23:24:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0179200640 1 --2- 172.21.15.26:0/1773395113 >> v2:172.21.15.29:3300/0 conn(0x7f0174162850 0x7f01740b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0179200640 1 --2- 172.21.15.26:0/1773395113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01741623c0 0x7f017406d470 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0179200640 1 -- 172.21.15.26:0/1773395113 >> 172.21.15.26:0/1773395113 conn(0x7f01740d0400 msgr2=0x7f01740d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0179200640 1 -- 172.21.15.26:0/1773395113 shutdown_connections 2024-04-24T23:24:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.659+0000 7f0179200640 1 -- 172.21.15.26:0/1773395113 wait complete. 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f0179200640 1 Processor -- start 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f0179200640 1 -- start start 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f0179200640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f017415e200 0x7f01740cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f0179200640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01741623c0 0x7f0174146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f0179200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0174162850 0x7f01741f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f0179200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f01740b5370 con 0x7f0174162850 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f0179200640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f01740b51f0 con 0x7f01741623c0 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f0179200640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f017406bca0 con 0x7f017415e200 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f017359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0174162850 0x7f01741f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f017359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0174162850 0x7f01741f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39856/0 (socket says 172.21.15.26:39856) 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.661+0000 7f017359e640 1 -- 172.21.15.26:0/3002926312 learned_addr learned my addr 172.21.15.26:0/3002926312 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f017259c640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01741623c0 0x7f0174146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f0172d9d640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f017415e200 0x7f01740cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f017359e640 1 -- 172.21.15.26:0/3002926312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f017415e200 msgr2=0x7f01740cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f017359e640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f017415e200 0x7f01740cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f017359e640 1 -- 172.21.15.26:0/3002926312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01741623c0 msgr2=0x7f0174146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f017359e640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01741623c0 0x7f0174146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f017359e640 1 -- 172.21.15.26:0/3002926312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01741f51c0 con 0x7f0174162850 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f017359e640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0174162850 0x7f01741f4c80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f016400d5d0 tx=0x7f016400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f016b7fe640 1 -- 172.21.15.26:0/3002926312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640125f0 con 0x7f0174162850 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f016b7fe640 1 -- 172.21.15.26:0/3002926312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164019070 con 0x7f0174162850 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f016b7fe640 1 -- 172.21.15.26:0/3002926312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640129f0 con 0x7f0174162850 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01741f5490 con 0x7f0174162850 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.662+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741f5a50 con 0x7f0174162850 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.663+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0174075610 con 0x7f0174162850 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.664+0000 7f016b7fe640 1 -- 172.21.15.26:0/3002926312 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0164012b90 con 0x7f0174162850 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.664+0000 7f016b7fe640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.664+0000 7f016b7fe640 1 -- 172.21.15.26:0/3002926312 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5115+0+0 (secure 0 0 0) 0x7f016408afa0 con 0x7f0174162850 2024-04-24T23:24:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.664+0000 7f0172d9d640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.665+0000 7f0172d9d640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f015406be30 0x7f015406e2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f016c0012e0 tx=0x7f016c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.668+0000 7f016b7fe640 1 -- 172.21.15.26:0/3002926312 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0164059b90 con 0x7f0174162850 2024-04-24T23:24:09.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.889+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01740d01e0 con 0x7f0174162850 2024-04-24T23:24:09.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.889+0000 7f016b7fe640 1 -- 172.21.15.26:0/3002926312 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+3866 (secure 0 0 0) 0x7f0164059490 con 0x7f0174162850 2024-04-24T23:24:09.892 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:09.892 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3504703199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":3504703199},{"type":"v1","addr":"172.21.15.26:6837","nonce":3504703199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1697945696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":1697945696},{"type":"v1","addr":"172.21.15.29:6835","nonce":1697945696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/605411536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":605411536},{"type":"v1","addr":"172.21.15.29:6834","nonce":605411536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6444,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/3814026276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3814026276},{"type":"v1","addr":"172.21.15.26:6836","nonce":3814026276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[]} 2024-04-24T23:24:09.892 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 44 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f015406be30 msgr2=0x7f015406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f015406be30 0x7f015406e2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f016c0012e0 tx=0x7f016c009040 comp rx=0 tx=0).stop 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0174162850 msgr2=0x7f01741f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0174162850 0x7f01741f4c80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f016400d5d0 tx=0x7f016400da90 comp rx=0 tx=0).stop 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 shutdown_connections 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0174162850 0x7f01741f4c80 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f01741623c0 0x7f0174146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 --2- 172.21.15.26:0/3002926312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f017415e200 0x7f01740cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:09.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 >> 172.21.15.26:0/3002926312 conn(0x7f01740d0400 msgr2=0x7f017406bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 shutdown_connections 2024-04-24T23:24:09.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:09.891+0000 7f0179200640 1 -- 172.21.15.26:0/3002926312 wait complete. 2024-04-24T23:24:09.903 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:24:09.904 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:24:09.904 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:24:09.904 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:24:09.904 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:24:09.904 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:24:09.904 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:24:09.904 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:24:09.905 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:24:09.905 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:24:09.905 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:24:09.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:24:09.905 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:24:09.905 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:24:09.905 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:24:09.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:24:09.914 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:24:09.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:24:09.914 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:24:09.915 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:24:09.917 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:24:09.917 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:24:09.918 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:24:09.918 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:24:09.957 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:24:09.957 DEBUG:teuthology.parallel:result is None 2024-04-24T23:24:09.958 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:24:09.959 DEBUG:teuthology.parallel:result is None 2024-04-24T23:24:09.962 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:24:09.962 DEBUG:teuthology.parallel:result is None 2024-04-24T23:24:09.963 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:24:09.964 DEBUG:teuthology.parallel:result is None 2024-04-24T23:24:09.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:24:10.033 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:24:10.033 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:24:10.049 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:24:10.049 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:24:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.069+0000 7f3779a58640 1 Processor -- start 2024-04-24T23:24:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.069+0000 7f3779a58640 1 -- start start 2024-04-24T23:24:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3779a58640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f377415e570 0x7f37740d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3779a58640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f37740b3130 0x7f37740d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3779a58640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37740d32f0 con 0x7f37740d22e0 2024-04-24T23:24:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3779a58640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f37740d1ab0 con 0x7f377415e570 2024-04-24T23:24:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3779a58640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f37740d2ae0 con 0x7f37740b3130 2024-04-24T23:24:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3772ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f377415e570 0x7f37740d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3772ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f377415e570 0x7f37740d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39904/0 (socket says 172.21.15.26:39904) 2024-04-24T23:24:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3772ffd640 1 -- 172.21.15.26:0/3252428690 learned_addr learned my addr 172.21.15.26:0/3252428690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f37727fc640 1 --2- 172.21.15.26:0/3252428690 >> v2:172.21.15.29:3300/0 conn(0x7f37740b3130 0x7f37740d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3772ffd640 1 -- 172.21.15.26:0/3252428690 >> v2:172.21.15.29:3300/0 conn(0x7f37740b3130 msgr2=0x7f37740d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3772ffd640 1 --2- 172.21.15.26:0/3252428690 >> v2:172.21.15.29:3300/0 conn(0x7f37740b3130 0x7f37740d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3772ffd640 1 -- 172.21.15.26:0/3252428690 >> v1:172.21.15.26:6789/0 conn(0x7f37740d22e0 legacy=0x7f37740d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3772ffd640 1 -- 172.21.15.26:0/3252428690 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37740d0270 con 0x7f377415e570 2024-04-24T23:24:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f3771ffb640 1 -- 172.21.15.26:0/3252428690 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434716017 0 0) 0x7f376c001490 con 0x7f37740d22e0 2024-04-24T23:24:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.070+0000 7f37727fc640 1 --2- 172.21.15.26:0/3252428690 >> v2:172.21.15.29:3300/0 conn(0x7f37740b3130 0x7f37740d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.071+0000 7f3772ffd640 1 --2- 172.21.15.26:0/3252428690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 0x7f37740d4160 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f377408cae0 tx=0x7f376402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f06866b4eac3f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.071+0000 7f3771ffb640 1 -- 172.21.15.26:0/3252428690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764037070 con 0x7f377415e570 2024-04-24T23:24:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.071+0000 7f3771ffb640 1 -- 172.21.15.26:0/3252428690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3764032440 con 0x7f377415e570 2024-04-24T23:24:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.071+0000 7f3771ffb640 1 -- 172.21.15.26:0/3252428690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764032810 con 0x7f377415e570 2024-04-24T23:24:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.072+0000 7f3779a58640 1 -- 172.21.15.26:0/3252428690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 msgr2=0x7f37740d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.072+0000 7f3779a58640 1 --2- 172.21.15.26:0/3252428690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 0x7f37740d4160 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f377408cae0 tx=0x7f376402a270 comp rx=0 tx=0).stop 2024-04-24T23:24:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.072+0000 7f3779a58640 1 -- 172.21.15.26:0/3252428690 shutdown_connections 2024-04-24T23:24:10.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.072+0000 7f3779a58640 1 --2- 172.21.15.26:0/3252428690 >> v2:172.21.15.29:3300/0 conn(0x7f37740b3130 0x7f37740d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:10.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.072+0000 7f3779a58640 1 --2- 172.21.15.26:0/3252428690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 0x7f37740d4160 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:10.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.072+0000 7f3779a58640 1 -- 172.21.15.26:0/3252428690 >> 172.21.15.26:0/3252428690 conn(0x7f377408dbe0 msgr2=0x7f377409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:10.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.072+0000 7f3779a58640 1 -- 172.21.15.26:0/3252428690 shutdown_connections 2024-04-24T23:24:10.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.072+0000 7f3779a58640 1 -- 172.21.15.26:0/3252428690 wait complete. 2024-04-24T23:24:10.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.073+0000 7f3779a58640 1 Processor -- start 2024-04-24T23:24:10.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.073+0000 7f3779a58640 1 -- start start 2024-04-24T23:24:10.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f3779a58640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f37740b3130 0x7f37740a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:10.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f3779a58640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37740d22e0 0x7f37740cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f3779a58640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 0x7f37740cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f3779a58640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37740cfa70 con 0x7f377415e570 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f3779a58640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f37740d3010 con 0x7f37740d22e0 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f3779a58640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f37740cfd50 con 0x7f37740b3130 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 0x7f37740cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37737fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 0x7f37740cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39914/0 (socket says 172.21.15.26:39914) 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37737fe640 1 -- 172.21.15.26:0/4058185777 learned_addr learned my addr 172.21.15.26:0/4058185777 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37727fc640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37740d22e0 0x7f37740cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f3772ffd640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f37740b3130 0x7f37740a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37737fe640 1 -- 172.21.15.26:0/4058185777 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f37740b3130 msgr2=0x7f37740a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37737fe640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f37740b3130 0x7f37740a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37737fe640 1 -- 172.21.15.26:0/4058185777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37740d22e0 msgr2=0x7f37740cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37737fe640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37740d22e0 0x7f37740cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37737fe640 1 -- 172.21.15.26:0/4058185777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37740cc150 con 0x7f377415e570 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f37737fe640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 0x7f37740cba40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c7c0 tx=0x7f376c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.074+0000 7f375bfff640 1 -- 172.21.15.26:0/4058185777 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c003da0 con 0x7f377415e570 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.075+0000 7f375bfff640 1 -- 172.21.15.26:0/4058185777 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376c005070 con 0x7f377415e570 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.075+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37740cc430 con 0x7f377415e570 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.075+0000 7f375bfff640 1 -- 172.21.15.26:0/4058185777 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c005460 con 0x7f377415e570 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.075+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37740cc9f0 con 0x7f377415e570 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.076+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37740cf5c0 con 0x7f377415e570 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.076+0000 7f375bfff640 1 -- 172.21.15.26:0/4058185777 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f376c005600 con 0x7f377415e570 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.077+0000 7f375bfff640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f376006be30 0x7f376006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.077+0000 7f375bfff640 1 -- 172.21.15.26:0/4058185777 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5115+0+0 (secure 0 0 0) 0x7f376c08aa50 con 0x7f377415e570 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.077+0000 7f3772ffd640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f376006be30 0x7f376006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:10.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.077+0000 7f3772ffd640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f376006be30 0x7f376006e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3764034000 tx=0x7f376402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.081+0000 7f375bfff640 1 -- 172.21.15.26:0/4058185777 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f37740cf5c0 con 0x7f377415e570 2024-04-24T23:24:10.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:10.276+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f37740694c0 con 0x7f377415e570 2024-04-24T23:24:11.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.323+0000 7f375bfff640 1 -- 172.21.15.26:0/4058185777 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7f37740d32f0 con 0x7f377415e570 2024-04-24T23:24:11.324 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-04-24T23:24:11.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f376006be30 msgr2=0x7f376006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f376006be30 0x7f376006e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3764034000 tx=0x7f376402e000 comp rx=0 tx=0).stop 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 msgr2=0x7f37740cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 0x7f37740cba40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c7c0 tx=0x7f376c00cc80 comp rx=0 tx=0).stop 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 shutdown_connections 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f376006be30 0x7f376006e2f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377415e570 0x7f37740cba40 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f37740d22e0 0x7f37740cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 --2- 172.21.15.26:0/4058185777 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f37740b3130 0x7f37740a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 >> 172.21.15.26:0/4058185777 conn(0x7f377408dbe0 msgr2=0x7f37740d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 shutdown_connections 2024-04-24T23:24:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.326+0000 7f3779a58640 1 -- 172.21.15.26:0/4058185777 wait complete. 2024-04-24T23:24:11.341 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:24:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.447+0000 7f466b0a5640 1 Processor -- start 2024-04-24T23:24:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.447+0000 7f466b0a5640 1 -- start start 2024-04-24T23:24:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.448+0000 7f466b0a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f46640cdd70 0x7f466406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.448+0000 7f466b0a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46640cfbc0 con 0x7f466406d350 2024-04-24T23:24:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.448+0000 7f466b0a5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f466406d7f0 con 0x7f4664145910 2024-04-24T23:24:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.448+0000 7f466b0a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4664145cf0 con 0x7f46640cdd70 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.448+0000 7f4663fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f466406d350 0x7f46640cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46720/0 (socket says 172.21.15.26:46720) 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.448+0000 7f4663fff640 1 -- 172.21.15.26:0/504168786 learned_addr learned my addr 172.21.15.26:0/504168786 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.449+0000 7f4669642640 1 --2- 172.21.15.26:0/504168786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f46640cdd70 0x7f466406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.449+0000 7f46637fe640 1 -- 172.21.15.26:0/504168786 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084843349 0 0) 0x7f46640cfbc0 con 0x7f466406d350 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.449+0000 7f46637fe640 1 -- 172.21.15.26:0/504168786 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f464c003520 con 0x7f466406d350 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.449+0000 7f46637fe640 1 -- 172.21.15.26:0/504168786 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3885378671 0 0) 0x7f466406d7f0 con 0x7f4664145910 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.449+0000 7f46637fe640 1 -- 172.21.15.26:0/504168786 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46640cfbc0 con 0x7f4664145910 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.449+0000 7f4669642640 1 -- 172.21.15.26:0/504168786 >> v1:172.21.15.29:6789/0 conn(0x7f4664145910 legacy=0x7f466415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.449+0000 7f4669642640 1 -- 172.21.15.26:0/504168786 >> v1:172.21.15.26:6789/0 conn(0x7f466406d350 legacy=0x7f46640cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.449+0000 7f4669642640 1 -- 172.21.15.26:0/504168786 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466406dad0 con 0x7f46640cdd70 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.450+0000 7f4669642640 1 --2- 172.21.15.26:0/504168786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f46640cdd70 0x7f466406e890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4654009800 tx=0x7f465402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70d73790bfa6fee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.450+0000 7f46637fe640 1 -- 172.21.15.26:0/504168786 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654037070 con 0x7f46640cdd70 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.450+0000 7f46637fe640 1 -- 172.21.15.26:0/504168786 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4654032440 con 0x7f46640cdd70 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.450+0000 7f46637fe640 1 -- 172.21.15.26:0/504168786 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46540327f0 con 0x7f46640cdd70 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.450+0000 7f466b0a5640 1 -- 172.21.15.26:0/504168786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f46640cdd70 msgr2=0x7f466406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.450+0000 7f466b0a5640 1 --2- 172.21.15.26:0/504168786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f46640cdd70 0x7f466406e890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4654009800 tx=0x7f465402a230 comp rx=0 tx=0).stop 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.451+0000 7f466b0a5640 1 -- 172.21.15.26:0/504168786 shutdown_connections 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.451+0000 7f466b0a5640 1 --2- 172.21.15.26:0/504168786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f46640cdd70 0x7f466406e890 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.451+0000 7f466b0a5640 1 -- 172.21.15.26:0/504168786 >> 172.21.15.26:0/504168786 conn(0x7f466406b300 msgr2=0x7f466406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.451+0000 7f466b0a5640 1 -- 172.21.15.26:0/504168786 shutdown_connections 2024-04-24T23:24:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.451+0000 7f466b0a5640 1 -- 172.21.15.26:0/504168786 wait complete. 2024-04-24T23:24:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.452+0000 7f466b0a5640 1 Processor -- start 2024-04-24T23:24:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.452+0000 7f466b0a5640 1 -- start start 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f466b0a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f466406d350 0x7f466413b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f466b0a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46640cdd70 0x7f4664139520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f466b0a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4664145910 0x7f466413f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f466b0a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f466406e370 con 0x7f46640cdd70 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f466b0a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f466406e090 con 0x7f4664145910 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f466b0a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f466415d770 con 0x7f466406d350 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4663fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46640cdd70 0x7f4664139520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4663fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46640cdd70 0x7f4664139520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39920/0 (socket says 172.21.15.26:39920) 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4663fff640 1 -- 172.21.15.26:0/398918405 learned_addr learned my addr 172.21.15.26:0/398918405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4668e41640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f466406d350 0x7f466413b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4669642640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4664145910 0x7f466413f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4663fff640 1 -- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f466406d350 msgr2=0x7f466413b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4663fff640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f466406d350 0x7f466413b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4663fff640 1 -- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4664145910 msgr2=0x7f466413f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4663fff640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4664145910 0x7f466413f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4663fff640 1 -- 172.21.15.26:0/398918405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466413eec0 con 0x7f46640cdd70 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4668e41640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f466406d350 0x7f466413b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4669642640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4664145910 0x7f466413f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f4663fff640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46640cdd70 0x7f4664139520 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f465800c4d0 tx=0x7f465800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f46617fa640 1 -- 172.21.15.26:0/398918405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658017070 con 0x7f46640cdd70 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f46617fa640 1 -- 172.21.15.26:0/398918405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4658005ce0 con 0x7f46640cdd70 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466413a940 con 0x7f46640cdd70 2024-04-24T23:24:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f46617fa640 1 -- 172.21.15.26:0/398918405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658007270 con 0x7f46640cdd70 2024-04-24T23:24:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.453+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466413af00 con 0x7f46640cdd70 2024-04-24T23:24:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.454+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46640694c0 con 0x7f46640cdd70 2024-04-24T23:24:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.455+0000 7f46617fa640 1 -- 172.21.15.26:0/398918405 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f465801a070 con 0x7f46640cdd70 2024-04-24T23:24:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.455+0000 7f46617fa640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f465006be60 0x7f465006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.456+0000 7f4668e41640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f465006be60 0x7f465006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.456+0000 7f46617fa640 1 -- 172.21.15.26:0/398918405 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f4658089ea0 con 0x7f46640cdd70 2024-04-24T23:24:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.456+0000 7f4668e41640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f465006be60 0x7f465006e320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f465c007fd0 tx=0x7f465c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:11.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.460+0000 7f46617fa640 1 -- 172.21.15.26:0/398918405 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f46580584c0 con 0x7f46640cdd70 2024-04-24T23:24:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.716+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f466415c970 con 0x7f46640cdd70 2024-04-24T23:24:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.716+0000 7f46617fa640 1 -- 172.21.15.26:0/398918405 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f465805c7c0 con 0x7f46640cdd70 2024-04-24T23:24:11.717 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmg5ZUFBAAkapM9La3f+wSyLczFkm/0w==", 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:24:11.718 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPS3cHxAASacBQN+95Ds6N5668Je+DA==", 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.719 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmStpLGxAAz7molt8venundBs+CcFSbg==", 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylml0TBJhAAXQkjExbw1EXc90XlcAEBBw==", 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjkylmQt74DBAAZLk9JTH3qTKRouUawqNURw==", 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.721 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCmkylmi6IJCxAAWBnNl2y0XIUW/YoAIt6VIQ==", 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCpkylmRcgOCRAA79pdcH0OlB2VROLSRE4dcw==", 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.722 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCskylm8SopBxAArvUph95+zD5/0LmNM8WPIA==", 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCwkylmbHipDxAAVUI9OpiMFiM/HFo03fEzSA==", 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCzkylmDm4CDhAA1T/X+ywXEUCL96OGyi6S9Q==", 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:24:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC2kylmwWyVDBAAJsFFLltURqVqGLNjK9OCJQ==", 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.725 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5kylm/nLXChAA6kk0UI2hchkN2lhPO1sXHw==", 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ==", 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:24:11.726 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylm/BVuMxAAAdEb6dd3Q5eMc+I3oszOyg==", 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmjnJGMBAAvwAmiHrkGkDFTeL/WZbdWw==", 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPh3qOBAAa4cbQeF96mdpgNVb/l3xPg==", 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.728 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCgkylm1ZfsMxAAExymnmhW44TcXzRblp95UA==", 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.729 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmq+8yERAAMKjK+5f2Qr35ral8L2qJMw==", 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmqYXWDBAAnuAn2h4Gb9qSFBoAZX73qQ==", 2024-04-24T23:24:11.730 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:24:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:24:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:24:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:24:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:11.731 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-24T23:24:11.731 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:11.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f465006be60 msgr2=0x7f465006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f465006be60 0x7f465006e320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f465c007fd0 tx=0x7f465c009040 comp rx=0 tx=0).stop 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46640cdd70 msgr2=0x7f4664139520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46640cdd70 0x7f4664139520 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f465800c4d0 tx=0x7f465800c990 comp rx=0 tx=0).stop 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 shutdown_connections 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f465006be60 0x7f465006e320 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4664145910 0x7f466413f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46640cdd70 0x7f4664139520 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 --2- 172.21.15.26:0/398918405 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f466406d350 0x7f466413b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.718+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 >> 172.21.15.26:0/398918405 conn(0x7f466406b300 msgr2=0x7f466406b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.719+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 shutdown_connections 2024-04-24T23:24:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.719+0000 7f466b0a5640 1 -- 172.21.15.26:0/398918405 wait complete. 2024-04-24T23:24:11.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:24:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.834+0000 7fedbd7a9640 1 Processor -- start 2024-04-24T23:24:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.834+0000 7fedbd7a9640 1 -- start start 2024-04-24T23:24:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedbd7a9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fedb8143c40 0x7fedb80cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedbd7a9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 0x7fedb815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedbd7a9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb80cd0b0 con 0x7fedb80cc860 2024-04-24T23:24:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedbd7a9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fedb8071b70 con 0x7fedb8143c40 2024-04-24T23:24:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedbd7a9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fedb815f470 con 0x7fedb815eb00 2024-04-24T23:24:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedb6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fedb8143c40 0x7fedb80cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedb6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fedb8143c40 0x7fedb80cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39930/0 (socket says 172.21.15.26:39930) 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedb6ffd640 1 -- 172.21.15.26:0/2951929324 learned_addr learned my addr 172.21.15.26:0/2951929324 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedb77fe640 1 --2- 172.21.15.26:0/2951929324 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 0x7fedb815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedb6ffd640 1 -- 172.21.15.26:0/2951929324 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 msgr2=0x7fedb815ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedb6ffd640 1 --2- 172.21.15.26:0/2951929324 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 0x7fedb815ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedb6ffd640 1 -- 172.21.15.26:0/2951929324 >> v1:172.21.15.29:6789/0 conn(0x7fedb80cc860 legacy=0x7fedb8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedb6ffd640 1 -- 172.21.15.26:0/2951929324 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb8071240 con 0x7fedb8143c40 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.835+0000 7fedb77fe640 1 --2- 172.21.15.26:0/2951929324 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 0x7fedb815ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.836+0000 7fedb6ffd640 1 --2- 172.21.15.26:0/2951929324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 0x7fedb80cf870 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7feda800bb10 tx=0x7feda802c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7466aa192eb9f784 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.836+0000 7fedb67fc640 1 -- 172.21.15.26:0/2951929324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8032790 con 0x7fedb8143c40 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.836+0000 7fedb67fc640 1 -- 172.21.15.26:0/2951929324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda8038070 con 0x7fedb8143c40 2024-04-24T23:24:11.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.836+0000 7fedb67fc640 1 -- 172.21.15.26:0/2951929324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8032b50 con 0x7fedb8143c40 2024-04-24T23:24:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.837+0000 7fedbd7a9640 1 -- 172.21.15.26:0/2951929324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 msgr2=0x7fedb80cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.837+0000 7fedbd7a9640 1 --2- 172.21.15.26:0/2951929324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 0x7fedb80cf870 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7feda800bb10 tx=0x7feda802c8d0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.837+0000 7fedbd7a9640 1 -- 172.21.15.26:0/2951929324 shutdown_connections 2024-04-24T23:24:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.837+0000 7fedbd7a9640 1 --2- 172.21.15.26:0/2951929324 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 0x7fedb815ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.837+0000 7fedbd7a9640 1 --2- 172.21.15.26:0/2951929324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 0x7fedb80cf870 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.837+0000 7fedbd7a9640 1 -- 172.21.15.26:0/2951929324 >> 172.21.15.26:0/2951929324 conn(0x7fedb806b300 msgr2=0x7fedb806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.837+0000 7fedbd7a9640 1 -- 172.21.15.26:0/2951929324 shutdown_connections 2024-04-24T23:24:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.837+0000 7fedbd7a9640 1 -- 172.21.15.26:0/2951929324 wait complete. 2024-04-24T23:24:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.838+0000 7fedbd7a9640 1 Processor -- start 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.838+0000 7fedbd7a9640 1 -- start start 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedbd7a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb80cc860 0x7fedb81f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedbd7a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 0x7fedb80b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedbd7a9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 0x7fedb80b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedbd7a9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fedb8144020 con 0x7fedb8143c40 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedbd7a9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fedb8071800 con 0x7fedb80cc860 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedbd7a9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fedb80cccd0 con 0x7fedb815eb00 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 0x7fedb80b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 0x7fedb80b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39946/0 (socket says 172.21.15.26:39946) 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaffff640 1 -- 172.21.15.26:0/3487313295 learned_addr learned my addr 172.21.15.26:0/3487313295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaffff640 1 -- 172.21.15.26:0/3487313295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 msgr2=0x7fedb80b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedb6ffd640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb80cc860 0x7fedb81f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaffff640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 0x7fedb80b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaffff640 1 -- 172.21.15.26:0/3487313295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb80cc860 msgr2=0x7fedb81f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaffff640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb80cc860 0x7fedb81f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaffff640 1 -- 172.21.15.26:0/3487313295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb8139190 con 0x7fedb8143c40 2024-04-24T23:24:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaffff640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 0x7fedb80b1010 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7feda000c460 tx=0x7feda000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaf7fe640 1 -- 172.21.15.26:0/3487313295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0016020 con 0x7fedb8143c40 2024-04-24T23:24:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaf7fe640 1 -- 172.21.15.26:0/3487313295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda0012870 con 0x7fedb8143c40 2024-04-24T23:24:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedaf7fe640 1 -- 172.21.15.26:0/3487313295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0012c60 con 0x7fedb8143c40 2024-04-24T23:24:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb8139470 con 0x7fedb8143c40 2024-04-24T23:24:11.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.839+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb8139a30 con 0x7fedb8143c40 2024-04-24T23:24:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.840+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb8139c30 con 0x7fedb8143c40 2024-04-24T23:24:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.841+0000 7fedaf7fe640 1 -- 172.21.15.26:0/3487313295 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feda0019020 con 0x7fedb8143c40 2024-04-24T23:24:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.841+0000 7fedaf7fe640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed9806be30 0x7fed9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.841+0000 7fedaf7fe640 1 -- 172.21.15.26:0/3487313295 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7feda008aa80 con 0x7fedb8143c40 2024-04-24T23:24:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.841+0000 7fedb6ffd640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed9806be30 0x7fed9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.842+0000 7fedb6ffd640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed9806be30 0x7fed9806e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feda8037000 tx=0x7feda8032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:11.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:11.846+0000 7fedaf7fe640 1 -- 172.21.15.26:0/3487313295 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fedb8139c30 con 0x7fedb8143c40 2024-04-24T23:24:12.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.032+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedb80694c0 con 0x7fedb8143c40 2024-04-24T23:24:12.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.033+0000 7fedaf7fe640 1 -- 172.21.15.26:0/3487313295 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+12831 (secure 0 0 0) 0x7feda0020020 con 0x7fedb8143c40 2024-04-24T23:24:12.034 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:12.034 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":54,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:24:11.322716+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:24:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed9806be30 msgr2=0x7fed9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed9806be30 0x7fed9806e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feda8037000 tx=0x7feda8032000 comp rx=0 tx=0).stop 2024-04-24T23:24:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 msgr2=0x7fedb80b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 0x7fedb80b1010 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7feda000c460 tx=0x7feda000c920 comp rx=0 tx=0).stop 2024-04-24T23:24:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 shutdown_connections 2024-04-24T23:24:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed9806be30 0x7fed9806e2f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fedb815eb00 0x7fedb80b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedb8143c40 0x7fedb80b1010 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 --2- 172.21.15.26:0/3487313295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fedb80cc860 0x7fedb81f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 >> 172.21.15.26:0/3487313295 conn(0x7fedb806b300 msgr2=0x7fedb813f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 shutdown_connections 2024-04-24T23:24:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.035+0000 7fedbd7a9640 1 -- 172.21.15.26:0/3487313295 wait complete. 2024-04-24T23:24:12.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:24:12.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:24:12.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.150+0000 7f4d937a9640 1 Processor -- start 2024-04-24T23:24:12.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.150+0000 7f4d937a9640 1 -- start start 2024-04-24T23:24:12.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d937a9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d937a9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4d8c150e60 0x7f4d8c1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d937a9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c070f30 con 0x7f4d8c15e500 2024-04-24T23:24:12.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d937a9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4d8c06dcf0 con 0x7f4d8c150e60 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d937a9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8c06dfd0 con 0x7f4d8c15e120 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d91d46640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4d8c150e60 0x7f4d8c1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d91d46640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4d8c150e60 0x7f4d8c1512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39960/0 (socket says 172.21.15.26:39960) 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d91d46640 1 -- 172.21.15.26:0/1044772044 learned_addr learned my addr 172.21.15.26:0/1044772044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d90d44640 1 --1- 172.21.15.26:0/1044772044 >> v1:172.21.15.26:6789/0 conn(0x7f4d8c15e500 0x7f4d8c0b37e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46724/0 (socket says 172.21.15.26:46724) 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d91545640 1 --2- 172.21.15.26:0/1044772044 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d91d46640 1 -- 172.21.15.26:0/1044772044 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 msgr2=0x7f4d8c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d91d46640 1 --2- 172.21.15.26:0/1044772044 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d91d46640 1 -- 172.21.15.26:0/1044772044 >> v1:172.21.15.26:6789/0 conn(0x7f4d8c15e500 legacy=0x7f4d8c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.152+0000 7f4d91d46640 1 -- 172.21.15.26:0/1044772044 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c06e2b0 con 0x7f4d8c150e60 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.151+0000 7f4d8bfff640 1 -- 172.21.15.26:0/1044772044 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 713528634 0 0) 0x7f4d8c070f30 con 0x7f4d8c15e500 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.152+0000 7f4d91545640 1 --2- 172.21.15.26:0/1044772044 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.152+0000 7f4d91d46640 1 --2- 172.21.15.26:0/1044772044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 0x7f4d8c1512d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009e50 tx=0x7f4d7c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fbe5cf5569f6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:12.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.152+0000 7f4d8bfff640 1 -- 172.21.15.26:0/1044772044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c036070 con 0x7f4d8c150e60 2024-04-24T23:24:12.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.152+0000 7f4d8bfff640 1 -- 172.21.15.26:0/1044772044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c031600 con 0x7f4d8c150e60 2024-04-24T23:24:12.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.152+0000 7f4d8bfff640 1 -- 172.21.15.26:0/1044772044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0319d0 con 0x7f4d8c150e60 2024-04-24T23:24:12.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.153+0000 7f4d937a9640 1 -- 172.21.15.26:0/1044772044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 msgr2=0x7f4d8c1512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.153+0000 7f4d937a9640 1 --2- 172.21.15.26:0/1044772044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 0x7f4d8c1512d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009e50 tx=0x7f4d7c02a710 comp rx=0 tx=0).stop 2024-04-24T23:24:12.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.153+0000 7f4d937a9640 1 -- 172.21.15.26:0/1044772044 shutdown_connections 2024-04-24T23:24:12.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.153+0000 7f4d937a9640 1 --2- 172.21.15.26:0/1044772044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 0x7f4d8c1512d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.153+0000 7f4d937a9640 1 --2- 172.21.15.26:0/1044772044 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.153+0000 7f4d937a9640 1 -- 172.21.15.26:0/1044772044 >> 172.21.15.26:0/1044772044 conn(0x7f4d8c06b540 msgr2=0x7f4d8c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.153+0000 7f4d937a9640 1 -- 172.21.15.26:0/1044772044 shutdown_connections 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.153+0000 7f4d937a9640 1 -- 172.21.15.26:0/1044772044 wait complete. 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.154+0000 7f4d937a9640 1 Processor -- start 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.154+0000 7f4d937a9640 1 -- start start 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d937a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 0x7f4d8c148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d937a9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c09c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d937a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d8c15e500 0x7f4d8c147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d937a9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c15e940 con 0x7f4d8c150e60 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d937a9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c0b3ef0 con 0x7f4d8c15e500 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d937a9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8c09ca60 con 0x7f4d8c15e120 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d91545640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 0x7f4d8c148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d91545640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 0x7f4d8c148000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39972/0 (socket says 172.21.15.26:39972) 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d91545640 1 -- 172.21.15.26:0/368031689 learned_addr learned my addr 172.21.15.26:0/368031689 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d90d44640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c09c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d91d46640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d8c15e500 0x7f4d8c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d91545640 1 -- 172.21.15.26:0/368031689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 msgr2=0x7f4d8c09c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d91545640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c09c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d91545640 1 -- 172.21.15.26:0/368031689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d8c15e500 msgr2=0x7f4d8c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d91545640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d8c15e500 0x7f4d8c147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d91545640 1 -- 172.21.15.26:0/368031689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c09cc80 con 0x7f4d8c150e60 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.155+0000 7f4d90d44640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c09c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.156+0000 7f4d91545640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 0x7f4d8c148000 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c7f0 tx=0x7f4d8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.156+0000 7f4d89ffb640 1 -- 172.21.15.26:0/368031689 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84015070 con 0x7f4d8c150e60 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.156+0000 7f4d89ffb640 1 -- 172.21.15.26:0/368031689 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d84007a30 con 0x7f4d8c150e60 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.156+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8c09cf60 con 0x7f4d8c150e60 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.156+0000 7f4d89ffb640 1 -- 172.21.15.26:0/368031689 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84007e00 con 0x7f4d8c150e60 2024-04-24T23:24:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.156+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8c1f5110 con 0x7f4d8c150e60 2024-04-24T23:24:12.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.157+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8c075610 con 0x7f4d8c150e60 2024-04-24T23:24:12.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.157+0000 7f4d89ffb640 1 -- 172.21.15.26:0/368031689 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d84018070 con 0x7f4d8c150e60 2024-04-24T23:24:12.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.158+0000 7f4d89ffb640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7806be30 0x7f4d7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.158+0000 7f4d89ffb640 1 -- 172.21.15.26:0/368031689 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f4d8408aba0 con 0x7f4d8c150e60 2024-04-24T23:24:12.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.158+0000 7f4d90d44640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7806be30 0x7f4d7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.158+0000 7f4d90d44640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7806be30 0x7f4d7806e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0aed50 tx=0x7f4d80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.162+0000 7f4d89ffb640 1 -- 172.21.15.26:0/368031689 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4d84059800 con 0x7f4d8c150e60 2024-04-24T23:24:12.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.343+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d8c06d820 con 0x7f4d8c150e60 2024-04-24T23:24:12.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.344+0000 7f4d89ffb640 1 -- 172.21.15.26:0/368031689 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f4d84059100 con 0x7f4d8c150e60 2024-04-24T23:24:12.345 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:24:12.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7806be30 msgr2=0x7f4d7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7806be30 0x7f4d7806e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0aed50 tx=0x7f4d80009040 comp rx=0 tx=0).stop 2024-04-24T23:24:12.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 msgr2=0x7f4d8c148000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 0x7f4d8c148000 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c7f0 tx=0x7f4d8400cbf0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 shutdown_connections 2024-04-24T23:24:12.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7806be30 0x7f4d7806e2f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d8c15e500 0x7f4d8c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d8c15e120 0x7f4d8c09c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 --2- 172.21.15.26:0/368031689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d8c150e60 0x7f4d8c148000 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 >> 172.21.15.26:0/368031689 conn(0x7f4d8c06b540 msgr2=0x7f4d8c0ace70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:12.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 shutdown_connections 2024-04-24T23:24:12.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.346+0000 7f4d937a9640 1 -- 172.21.15.26:0/368031689 wait complete. 2024-04-24T23:24:12.358 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:24:12.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.458+0000 7fc331afc640 1 Processor -- start 2024-04-24T23:24:12.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.458+0000 7fc331afc640 1 -- start start 2024-04-24T23:24:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc331afc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc32c143c40 0x7fc32c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc331afc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc32c0cc860 0x7fc32c072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc331afc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc32c0cd0b0 con 0x7fc32c15eb00 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc331afc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc32c071b70 con 0x7fc32c0cc860 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc331afc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc32c071240 con 0x7fc32c143c40 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc32affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc32c0cc860 0x7fc32c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc32affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc32c0cc860 0x7fc32c072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39978/0 (socket says 172.21.15.26:39978) 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc32affd640 1 -- 172.21.15.26:0/3352401055 learned_addr learned my addr 172.21.15.26:0/3352401055 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc32b7fe640 1 --2- 172.21.15.26:0/3352401055 >> v2:172.21.15.29:3300/0 conn(0x7fc32c143c40 0x7fc32c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc32affd640 1 -- 172.21.15.26:0/3352401055 >> v2:172.21.15.29:3300/0 conn(0x7fc32c143c40 msgr2=0x7fc32c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc32affd640 1 --2- 172.21.15.26:0/3352401055 >> v2:172.21.15.29:3300/0 conn(0x7fc32c143c40 0x7fc32c0cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc32affd640 1 -- 172.21.15.26:0/3352401055 >> v1:172.21.15.29:6789/0 conn(0x7fc32c15eb00 legacy=0x7fc32c15ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:12.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.459+0000 7fc32affd640 1 -- 172.21.15.26:0/3352401055 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c071520 con 0x7fc32c0cc860 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.460+0000 7fc32affd640 1 --2- 172.21.15.26:0/3352401055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c0cc860 0x7fc32c072000 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc32c06ecf0 tx=0x7fc31c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bab83f5041c4aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.460+0000 7fc32a7fc640 1 -- 172.21.15.26:0/3352401055 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c037070 con 0x7fc32c0cc860 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.460+0000 7fc32a7fc640 1 -- 172.21.15.26:0/3352401055 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c032440 con 0x7fc32c0cc860 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.460+0000 7fc32a7fc640 1 -- 172.21.15.26:0/3352401055 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c032810 con 0x7fc32c0cc860 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.461+0000 7fc331afc640 1 -- 172.21.15.26:0/3352401055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c0cc860 msgr2=0x7fc32c072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.461+0000 7fc331afc640 1 --2- 172.21.15.26:0/3352401055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c0cc860 0x7fc32c072000 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc32c06ecf0 tx=0x7fc31c02a230 comp rx=0 tx=0).stop 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.461+0000 7fc331afc640 1 -- 172.21.15.26:0/3352401055 shutdown_connections 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.461+0000 7fc331afc640 1 --2- 172.21.15.26:0/3352401055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c0cc860 0x7fc32c072000 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.461+0000 7fc331afc640 1 --2- 172.21.15.26:0/3352401055 >> v2:172.21.15.29:3300/0 conn(0x7fc32c143c40 0x7fc32c0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.461+0000 7fc331afc640 1 -- 172.21.15.26:0/3352401055 >> 172.21.15.26:0/3352401055 conn(0x7fc32c06b300 msgr2=0x7fc32c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.461+0000 7fc331afc640 1 -- 172.21.15.26:0/3352401055 shutdown_connections 2024-04-24T23:24:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.461+0000 7fc331afc640 1 -- 172.21.15.26:0/3352401055 wait complete. 2024-04-24T23:24:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.462+0000 7fc331afc640 1 Processor -- start 2024-04-24T23:24:12.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.462+0000 7fc331afc640 1 -- start start 2024-04-24T23:24:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc331afc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc32c0cc860 0x7fc32c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc331afc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c143c40 0x7fc32c0adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc331afc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c15eb00 0x7fc32c0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc331afc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c072540 con 0x7fc32c143c40 2024-04-24T23:24:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc331afc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c144020 con 0x7fc32c15eb00 2024-04-24T23:24:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc331afc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c0aaf30 con 0x7fc32c0cc860 2024-04-24T23:24:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c143c40 0x7fc32c0adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c143c40 0x7fc32c0adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39980/0 (socket says 172.21.15.26:39980) 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32affd640 1 -- 172.21.15.26:0/3772296358 learned_addr learned my addr 172.21.15.26:0/3772296358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32b7fe640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc32c0cc860 0x7fc32c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32bfff640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c15eb00 0x7fc32c0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32affd640 1 -- 172.21.15.26:0/3772296358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc32c0cc860 msgr2=0x7fc32c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32affd640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc32c0cc860 0x7fc32c0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32affd640 1 -- 172.21.15.26:0/3772296358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c15eb00 msgr2=0x7fc32c0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32affd640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c15eb00 0x7fc32c0ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32affd640 1 -- 172.21.15.26:0/3772296358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c0aac50 con 0x7fc32c143c40 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.463+0000 7fc32affd640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c143c40 0x7fc32c0adcf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc31c000c00 tx=0x7fc31c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.464+0000 7fc313fff640 1 -- 172.21.15.26:0/3772296358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c041020 con 0x7fc32c143c40 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.464+0000 7fc313fff640 1 -- 172.21.15.26:0/3772296358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c030970 con 0x7fc32c143c40 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.464+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c130470 con 0x7fc32c143c40 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.464+0000 7fc313fff640 1 -- 172.21.15.26:0/3772296358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c037040 con 0x7fc32c143c40 2024-04-24T23:24:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.464+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c130a30 con 0x7fc32c143c40 2024-04-24T23:24:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.465+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c0694c0 con 0x7fc32c143c40 2024-04-24T23:24:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.465+0000 7fc313fff640 1 -- 172.21.15.26:0/3772296358 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc31c044020 con 0x7fc32c143c40 2024-04-24T23:24:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.465+0000 7fc313fff640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc31806be60 0x7fc31806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.466+0000 7fc313fff640 1 -- 172.21.15.26:0/3772296358 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fc31c0ab410 con 0x7fc32c143c40 2024-04-24T23:24:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.466+0000 7fc32b7fe640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc31806be60 0x7fc31806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.466+0000 7fc32b7fe640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc31806be60 0x7fc31806e320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc32000aac0 tx=0x7fc320008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.470+0000 7fc313fff640 1 -- 172.21.15.26:0/3772296358 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc31c04b030 con 0x7fc32c143c40 2024-04-24T23:24:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.655+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc32c06d4a0 con 0x7fc32c143c40 2024-04-24T23:24:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.655+0000 7fc313fff640 1 -- 172.21.15.26:0/3772296358 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7fc31c079ac0 con 0x7fc32c143c40 2024-04-24T23:24:12.656 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:12.656 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:24:12.656 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:24:12.656 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-24T23:24:12.657 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-24T23:24:12.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:12.657 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc31806be60 msgr2=0x7fc31806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc31806be60 0x7fc31806e320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc32000aac0 tx=0x7fc320008040 comp rx=0 tx=0).stop 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c143c40 msgr2=0x7fc32c0adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c143c40 0x7fc32c0adcf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc31c000c00 tx=0x7fc31c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 shutdown_connections 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc31806be60 0x7fc31806e320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc32c15eb00 0x7fc32c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32c143c40 0x7fc32c0adcf0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 --2- 172.21.15.26:0/3772296358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc32c0cc860 0x7fc32c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 >> 172.21.15.26:0/3772296358 conn(0x7fc32c06b300 msgr2=0x7fc32c15b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 shutdown_connections 2024-04-24T23:24:12.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.657+0000 7fc331afc640 1 -- 172.21.15.26:0/3772296358 wait complete. 2024-04-24T23:24:12.670 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:24:12.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:24:12.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.773+0000 7f1667f1b640 1 Processor -- start 2024-04-24T23:24:12.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.774+0000 7f1667f1b640 1 -- start start 2024-04-24T23:24:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.774+0000 7f1667f1b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f166015ea80 0x7f1660141160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.774+0000 7f1667f1b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f16600ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.774+0000 7f1667f1b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16600cf050 con 0x7f166015b490 2024-04-24T23:24:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.774+0000 7f1667f1b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f16600cf330 con 0x7f166015ea80 2024-04-24T23:24:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.774+0000 7f1667f1b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f16600cf610 con 0x7f166015ee60 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f1665cb7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f166015ea80 0x7f1660141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f16664b8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f166015b490 0x7f166015b920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46734/0 (socket says 172.21.15.26:46734) 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f1665cb7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f166015ea80 0x7f1660141160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39990/0 (socket says 172.21.15.26:39990) 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f16664b8640 1 -- 172.21.15.26:0/1245207100 learned_addr learned my addr 172.21.15.26:0/1245207100 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f16654b6640 1 --2- 172.21.15.26:0/1245207100 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f16600ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f1665cb7640 1 -- 172.21.15.26:0/1245207100 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 msgr2=0x7f16600ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f1665cb7640 1 --2- 172.21.15.26:0/1245207100 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f16600ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f1665cb7640 1 -- 172.21.15.26:0/1245207100 >> v1:172.21.15.26:6789/0 conn(0x7f166015b490 legacy=0x7f166015b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f1665cb7640 1 -- 172.21.15.26:0/1245207100 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16600cf8f0 con 0x7f166015ea80 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f1664cb5640 1 -- 172.21.15.26:0/1245207100 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565914933 0 0) 0x7f16600cf050 con 0x7f166015b490 2024-04-24T23:24:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.775+0000 7f16654b6640 1 --2- 172.21.15.26:0/1245207100 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f16600ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1665cb7640 1 --2- 172.21.15.26:0/1245207100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015ea80 0x7f1660141160 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f165c0097a0 tx=0x7f165c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd161b2ab84e4eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1664cb5640 1 -- 172.21.15.26:0/1245207100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c037070 con 0x7f166015ea80 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1664cb5640 1 -- 172.21.15.26:0/1245207100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c032440 con 0x7f166015ea80 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1664cb5640 1 -- 172.21.15.26:0/1245207100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c032810 con 0x7f166015ea80 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1667f1b640 1 -- 172.21.15.26:0/1245207100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015ea80 msgr2=0x7f1660141160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1667f1b640 1 --2- 172.21.15.26:0/1245207100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015ea80 0x7f1660141160 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f165c0097a0 tx=0x7f165c02a180 comp rx=0 tx=0).stop 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1667f1b640 1 -- 172.21.15.26:0/1245207100 shutdown_connections 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1667f1b640 1 --2- 172.21.15.26:0/1245207100 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f16600ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1667f1b640 1 --2- 172.21.15.26:0/1245207100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015ea80 0x7f1660141160 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1667f1b640 1 -- 172.21.15.26:0/1245207100 >> 172.21.15.26:0/1245207100 conn(0x7f166006b300 msgr2=0x7f166006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:12.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1667f1b640 1 -- 172.21.15.26:0/1245207100 shutdown_connections 2024-04-24T23:24:12.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.776+0000 7f1667f1b640 1 -- 172.21.15.26:0/1245207100 wait complete. 2024-04-24T23:24:12.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1667f1b640 1 Processor -- start 2024-04-24T23:24:12.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1667f1b640 1 -- start start 2024-04-24T23:24:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1667f1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015b490 0x7f1660154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1667f1b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f166015ea80 0x7f166015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1667f1b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f1660158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1667f1b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f166015f2a0 con 0x7f166015b490 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1667f1b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f16600cec80 con 0x7f166015ea80 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1667f1b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f166015c760 con 0x7f166015ee60 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1665cb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015b490 0x7f1660154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1665cb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015b490 0x7f1660154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40002/0 (socket says 172.21.15.26:40002) 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.778+0000 7f1665cb7640 1 -- 172.21.15.26:0/2565373358 learned_addr learned my addr 172.21.15.26:0/2565373358 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f16654b6640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f166015ea80 0x7f166015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f16664b8640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f1660158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f1665cb7640 1 -- 172.21.15.26:0/2565373358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 msgr2=0x7f1660158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f1665cb7640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f1660158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f1665cb7640 1 -- 172.21.15.26:0/2565373358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f166015ea80 msgr2=0x7f166015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f1665cb7640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f166015ea80 0x7f166015d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f1665cb7640 1 -- 172.21.15.26:0/2565373358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166015ca40 con 0x7f166015b490 2024-04-24T23:24:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f16664b8640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f1660158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f1665cb7640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015b490 0x7f1660154a70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f165c0097a0 tx=0x7f165c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f16567fc640 1 -- 172.21.15.26:0/2565373358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c037050 con 0x7f166015b490 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f16567fc640 1 -- 172.21.15.26:0/2565373358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c038040 con 0x7f166015b490 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166015cd20 con 0x7f166015b490 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f16567fc640 1 -- 172.21.15.26:0/2565373358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c004470 con 0x7f166015b490 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.779+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1660154fb0 con 0x7f166015b490 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.780+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16600694c0 con 0x7f166015b490 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.781+0000 7f16567fc640 1 -- 172.21.15.26:0/2565373358 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f165c030900 con 0x7f166015b490 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.781+0000 7f16567fc640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f164406be30 0x7f164406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.781+0000 7f16654b6640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f164406be30 0x7f164406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.781+0000 7f16567fc640 1 -- 172.21.15.26:0/2565373358 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f165c0ab610 con 0x7f166015b490 2024-04-24T23:24:12.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.782+0000 7f16654b6640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f164406be30 0x7f164406e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f166015e410 tx=0x7f1650008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:12.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.786+0000 7f16567fc640 1 -- 172.21.15.26:0/2565373358 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f165c07a3a0 con 0x7f166015b490 2024-04-24T23:24:12.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:12.970+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f166006d9c0 con 0x7f166015b490 2024-04-24T23:24:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.326+0000 7f16567fc640 1 -- 172.21.15.26:0/2565373358 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7f165c079ca0 con 0x7f166015b490 2024-04-24T23:24:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-24T23:24:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.328+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f164406be30 msgr2=0x7f164406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.328+0000 7f1667f1b640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f164406be30 0x7f164406e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f166015e410 tx=0x7f1650008040 comp rx=0 tx=0).stop 2024-04-24T23:24:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015b490 msgr2=0x7f1660154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015b490 0x7f1660154a70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f165c0097a0 tx=0x7f165c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 shutdown_connections 2024-04-24T23:24:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f164406be30 0x7f164406e2f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f166015ee60 0x7f1660158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:13.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f166015ea80 0x7f166015d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:13.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 --2- 172.21.15.26:0/2565373358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f166015b490 0x7f1660154a70 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:13.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 >> 172.21.15.26:0/2565373358 conn(0x7f166006b300 msgr2=0x7f16600721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:13.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 shutdown_connections 2024-04-24T23:24:13.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.329+0000 7f1667f1b640 1 -- 172.21.15.26:0/2565373358 wait complete. 2024-04-24T23:24:13.344 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:24:13.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.452+0000 7f791f464640 1 Processor -- start 2024-04-24T23:24:13.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.452+0000 7f791f464640 1 -- start start 2024-04-24T23:24:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.452+0000 7f791f464640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f791815e370 0x7f79180d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.453+0000 7f791f464640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79180d2120 con 0x7f79180d4c10 2024-04-24T23:24:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.453+0000 7f791f464640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79180b3740 con 0x7f791815e750 2024-04-24T23:24:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.453+0000 7f791f464640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f79180d1510 con 0x7f791815e370 2024-04-24T23:24:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.453+0000 7f791d200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f791815e370 0x7f79180d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.453+0000 7f791d200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f791815e370 0x7f79180d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40012/0 (socket says 172.21.15.26:40012) 2024-04-24T23:24:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.453+0000 7f791d200640 1 -- 172.21.15.26:0/1468170872 learned_addr learned my addr 172.21.15.26:0/1468170872 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:13.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.453+0000 7f791d200640 1 -- 172.21.15.26:0/1468170872 >> v1:172.21.15.29:6789/0 conn(0x7f791815e750 legacy=0x7f79180b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.453+0000 7f791d200640 1 -- 172.21.15.26:0/1468170872 >> v1:172.21.15.26:6789/0 conn(0x7f79180d4c10 legacy=0x7f79180b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.453+0000 7f791d200640 1 -- 172.21.15.26:0/1468170872 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79180b3a20 con 0x7f791815e370 2024-04-24T23:24:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.454+0000 7f791d200640 1 --2- 172.21.15.26:0/1468170872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f791815e370 0x7f79180d2520 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7910009e50 tx=0x7f791002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8f77d00bc96741 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.454+0000 7f7917fff640 1 -- 172.21.15.26:0/1468170872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910036070 con 0x7f791815e370 2024-04-24T23:24:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.454+0000 7f7917fff640 1 -- 172.21.15.26:0/1468170872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7910031600 con 0x7f791815e370 2024-04-24T23:24:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.454+0000 7f7917fff640 1 -- 172.21.15.26:0/1468170872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100319d0 con 0x7f791815e370 2024-04-24T23:24:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.455+0000 7f791f464640 1 -- 172.21.15.26:0/1468170872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f791815e370 msgr2=0x7f79180d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.455+0000 7f791f464640 1 --2- 172.21.15.26:0/1468170872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f791815e370 0x7f79180d2520 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7910009e50 tx=0x7f791002a760 comp rx=0 tx=0).stop 2024-04-24T23:24:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.455+0000 7f791f464640 1 -- 172.21.15.26:0/1468170872 shutdown_connections 2024-04-24T23:24:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.455+0000 7f791f464640 1 --2- 172.21.15.26:0/1468170872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f791815e370 0x7f79180d2520 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.455+0000 7f791f464640 1 -- 172.21.15.26:0/1468170872 >> 172.21.15.26:0/1468170872 conn(0x7f79180d40f0 msgr2=0x7f79180d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:13.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.455+0000 7f791f464640 1 -- 172.21.15.26:0/1468170872 shutdown_connections 2024-04-24T23:24:13.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.455+0000 7f791f464640 1 -- 172.21.15.26:0/1468170872 wait complete. 2024-04-24T23:24:13.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.456+0000 7f791f464640 1 Processor -- start 2024-04-24T23:24:13.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.456+0000 7f791f464640 1 -- start start 2024-04-24T23:24:13.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791f464640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79180d4c10 0x7f79180c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:13.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791f464640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f791815e370 0x7f79180cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:13.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791f464640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79181f70d0 0x7f791815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:13.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791f464640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79180cfca0 con 0x7f79180d4c10 2024-04-24T23:24:13.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791f464640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f79180d1ed0 con 0x7f791815e370 2024-04-24T23:24:13.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791f464640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f79180cfe20 con 0x7f79181f70d0 2024-04-24T23:24:13.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79180d4c10 0x7f79180c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:13.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79180d4c10 0x7f79180c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40028/0 (socket says 172.21.15.26:40028) 2024-04-24T23:24:13.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791d200640 1 -- 172.21.15.26:0/2713801387 learned_addr learned my addr 172.21.15.26:0/2713801387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791d200640 1 -- 172.21.15.26:0/2713801387 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79181f70d0 msgr2=0x7f791815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791d200640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79181f70d0 0x7f791815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791d200640 1 -- 172.21.15.26:0/2713801387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f791815e370 msgr2=0x7f79180cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791d200640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f791815e370 0x7f79180cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791d200640 1 -- 172.21.15.26:0/2713801387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79181f7580 con 0x7f79180d4c10 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791d200640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79180d4c10 0x7f79180c8650 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7910009e50 tx=0x7f7910004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f7915ffb640 1 -- 172.21.15.26:0/2713801387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910036050 con 0x7f79180d4c10 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f7915ffb640 1 -- 172.21.15.26:0/2713801387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7910042070 con 0x7f79180d4c10 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f7915ffb640 1 -- 172.21.15.26:0/2713801387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910009510 con 0x7f79180d4c10 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.457+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79180cb910 con 0x7f79180d4c10 2024-04-24T23:24:13.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.458+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79180cbed0 con 0x7f79180d4c10 2024-04-24T23:24:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.459+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791806d5c0 con 0x7f79180d4c10 2024-04-24T23:24:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.459+0000 7f7915ffb640 1 -- 172.21.15.26:0/2713801387 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7910007c50 con 0x7f79180d4c10 2024-04-24T23:24:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.460+0000 7f7915ffb640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f790406be30 0x7f790406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.460+0000 7f7915ffb640 1 -- 172.21.15.26:0/2713801387 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7f79100abb10 con 0x7f79180d4c10 2024-04-24T23:24:13.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.460+0000 7f791c9ff640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f790406be30 0x7f790406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:13.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.460+0000 7f791c9ff640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f790406be30 0x7f790406e2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f790c006fd0 tx=0x7f790c006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:13.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.465+0000 7f7915ffb640 1 -- 172.21.15.26:0/2713801387 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f791806d5c0 con 0x7f79180d4c10 2024-04-24T23:24:13.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:13.647+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7918075610 con 0x7f79180d4c10 2024-04-24T23:24:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.331+0000 7f7915ffb640 1 -- 172.21.15.26:0/2713801387 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7f791007a0e0 con 0x7f79180d4c10 2024-04-24T23:24:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-24T23:24:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.335+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f790406be30 msgr2=0x7f790406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.335+0000 7f791f464640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f790406be30 0x7f790406e2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f790c006fd0 tx=0x7f790c006bc0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.335+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79180d4c10 msgr2=0x7f79180c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.335+0000 7f791f464640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79180d4c10 0x7f79180c8650 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7910009e50 tx=0x7f7910004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.340+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 shutdown_connections 2024-04-24T23:24:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.340+0000 7f791f464640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f790406be30 0x7f790406e2f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.340+0000 7f791f464640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79181f70d0 0x7f791815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.340+0000 7f791f464640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f791815e370 0x7f79180cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.340+0000 7f791f464640 1 --2- 172.21.15.26:0/2713801387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79180d4c10 0x7f79180c8650 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.340+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 >> 172.21.15.26:0/2713801387 conn(0x7f79180d40f0 msgr2=0x7f79180b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.342+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 shutdown_connections 2024-04-24T23:24:14.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.342+0000 7f791f464640 1 -- 172.21.15.26:0/2713801387 wait complete. 2024-04-24T23:24:14.366 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:24:14.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.473+0000 7feb59615640 1 Processor -- start 2024-04-24T23:24:14.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.473+0000 7feb59615640 1 -- start start 2024-04-24T23:24:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb59615640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb540d18b0 0x7feb5406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb59615640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb5415e920 con 0x7feb5406b660 2024-04-24T23:24:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb59615640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb5415eba0 con 0x7feb540cc9b0 2024-04-24T23:24:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb59615640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7feb5406bab0 con 0x7feb540d18b0 2024-04-24T23:24:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb537fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feb5406b660 0x7feb54157fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58790/0 (socket says 172.21.15.26:58790) 2024-04-24T23:24:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb537fe640 1 -- 172.21.15.26:0/1983026708 learned_addr learned my addr 172.21.15.26:0/1983026708 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb4bfff640 1 --2- 172.21.15.26:0/1983026708 >> v2:172.21.15.29:3300/0 conn(0x7feb540d18b0 0x7feb5406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358664093 0 0) 0x7feb4c001490 con 0x7feb5406b660 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb40003520 con 0x7feb5406b660 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2750524037 0 0) 0x7feb5415eba0 con 0x7feb540cc9b0 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb4c001490 con 0x7feb540cc9b0 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1429595941 0 0) 0x7feb40003520 con 0x7feb5406b660 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 >> v2:172.21.15.29:3300/0 conn(0x7feb540d18b0 msgr2=0x7feb5406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 --2- 172.21.15.26:0/1983026708 >> v2:172.21.15.29:3300/0 conn(0x7feb540d18b0 0x7feb5406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 >> v1:172.21.15.29:6789/0 conn(0x7feb540cc9b0 legacy=0x7feb540d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5415ee80 con 0x7feb5406b660 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.474+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7feb4c001bc0 con 0x7feb5406b660 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 >> v1:172.21.15.26:6789/0 conn(0x7feb5406b660 legacy=0x7feb54157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb527fc640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb40003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb527fc640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb40004800 0x7feb40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb527fc640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400051c0 0x7feb4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feb5406bab0 con 0x7feb40003b40 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feb4000c040 con 0x7feb400051c0 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feb4000c240 con 0x7feb40004800 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb52ffd640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb40003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb537fe640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400051c0 0x7feb4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb4bfff640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb40004800 0x7feb40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb52ffd640 1 -- 172.21.15.26:0/1983026708 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb40004800 msgr2=0x7feb40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb52ffd640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb40004800 0x7feb40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb52ffd640 1 -- 172.21.15.26:0/1983026708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400051c0 msgr2=0x7feb4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb52ffd640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb400051c0 0x7feb4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb52ffd640 1 -- 172.21.15.26:0/1983026708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb40003520 con 0x7feb40003b40 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.475+0000 7feb537fe640 1 -- 172.21.15.26:0/1983026708 reap_dead start 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.476+0000 7feb52ffd640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb40003f40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feb440018d0 tx=0x7feb40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6156d76194c93a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.476+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440014f0 con 0x7feb40003b40 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.476+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb440052d0 con 0x7feb40003b40 2024-04-24T23:24:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.476+0000 7feb527fc640 1 -- 172.21.15.26:0/1983026708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4402ac70 con 0x7feb40003b40 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.476+0000 7feb59615640 1 -- 172.21.15.26:0/1983026708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 msgr2=0x7feb40003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.476+0000 7feb59615640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb40003f40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feb440018d0 tx=0x7feb40005830 comp rx=0 tx=0).stop 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.476+0000 7feb59615640 1 -- 172.21.15.26:0/1983026708 shutdown_connections 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.477+0000 7feb59615640 1 --2- 172.21.15.26:0/1983026708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb40003f40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.477+0000 7feb59615640 1 -- 172.21.15.26:0/1983026708 >> 172.21.15.26:0/1983026708 conn(0x7feb540d0d20 msgr2=0x7feb540d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.477+0000 7feb59615640 1 -- 172.21.15.26:0/1983026708 shutdown_connections 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.477+0000 7feb59615640 1 -- 172.21.15.26:0/1983026708 wait complete. 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.478+0000 7feb59615640 1 Processor -- start 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.478+0000 7feb59615640 1 -- start start 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.478+0000 7feb59615640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb54147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.478+0000 7feb59615640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5415c510 0x7feb5415da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.478+0000 7feb59615640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb5415a0b0 0x7feb54157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.478+0000 7feb59615640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feb540cec20 con 0x7feb40003b40 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.478+0000 7feb59615640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feb5415f440 con 0x7feb5415c510 2024-04-24T23:24:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb59615640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feb5406b120 con 0x7feb5415a0b0 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb54147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb54147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52100/0 (socket says 172.21.15.26:52100) 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb52ffd640 1 -- 172.21.15.26:0/2332304998 learned_addr learned my addr 172.21.15.26:0/2332304998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb4bfff640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5415c510 0x7feb5415da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb537fe640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb5415a0b0 0x7feb54157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb52ffd640 1 -- 172.21.15.26:0/2332304998 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb5415a0b0 msgr2=0x7feb54157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb52ffd640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb5415a0b0 0x7feb54157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb52ffd640 1 -- 172.21.15.26:0/2332304998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5415c510 msgr2=0x7feb5415da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb52ffd640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5415c510 0x7feb5415da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb52ffd640 1 -- 172.21.15.26:0/2332304998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5415c9f0 con 0x7feb40003b40 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb4bfff640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5415c510 0x7feb5415da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb52ffd640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb54147350 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feb44000c00 tx=0x7feb440012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb4b7fe640 1 -- 172.21.15.26:0/2332304998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44037040 con 0x7feb40003b40 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb4b7fe640 1 -- 172.21.15.26:0/2332304998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb44006e50 con 0x7feb40003b40 2024-04-24T23:24:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5415ccd0 con 0x7feb40003b40 2024-04-24T23:24:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb4b7fe640 1 -- 172.21.15.26:0/2332304998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440014f0 con 0x7feb40003b40 2024-04-24T23:24:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.479+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb5415d290 con 0x7feb40003b40 2024-04-24T23:24:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.480+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb54075610 con 0x7feb40003b40 2024-04-24T23:24:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.481+0000 7feb4b7fe640 1 -- 172.21.15.26:0/2332304998 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb44044020 con 0x7feb40003b40 2024-04-24T23:24:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.481+0000 7feb4b7fe640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3406be60 0x7feb3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.482+0000 7feb4bfff640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3406be60 0x7feb3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.482+0000 7feb4b7fe640 1 -- 172.21.15.26:0/2332304998 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7feb440ab5e0 con 0x7feb40003b40 2024-04-24T23:24:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.482+0000 7feb4bfff640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3406be60 0x7feb3406e320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feb540b6cd0 tx=0x7feb3c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.486+0000 7feb4b7fe640 1 -- 172.21.15.26:0/2332304998 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feb4407a160 con 0x7feb40003b40 2024-04-24T23:24:14.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:14.711+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7feb5415bf70 con 0x7feb40003b40 2024-04-24T23:24:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.333+0000 7feb4b7fe640 1 -- 172.21.15.26:0/2332304998 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v46) ==== 172+0+0 (secure 0 0 0) 0x7feb44079a60 con 0x7feb40003b40 2024-04-24T23:24:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 8 and data pool 9 2024-04-24T23:24:15.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3406be60 msgr2=0x7feb3406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3406be60 0x7feb3406e320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feb540b6cd0 tx=0x7feb3c008040 comp rx=0 tx=0).stop 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 msgr2=0x7feb54147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb54147350 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feb44000c00 tx=0x7feb440012d0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 shutdown_connections 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb3406be60 0x7feb3406e320 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb5415a0b0 0x7feb54157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb5415c510 0x7feb5415da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 --2- 172.21.15.26:0/2332304998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb40003b40 0x7feb54147350 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 >> 172.21.15.26:0/2332304998 conn(0x7feb540d0d20 msgr2=0x7feb5415fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 shutdown_connections 2024-04-24T23:24:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.335+0000 7feb59615640 1 -- 172.21.15.26:0/2332304998 wait complete. 2024-04-24T23:24:15.352 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:24:15.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.456+0000 7f89cfe8c640 1 Processor -- start 2024-04-24T23:24:15.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.456+0000 7f89cfe8c640 1 -- start start 2024-04-24T23:24:15.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89cfe8c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c8145910 0x7f89c806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89cfe8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89c80cd660 0x7f89c80cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89cfe8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c80cfbc0 con 0x7f89c806d350 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89cfe8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f89c80cdff0 con 0x7f89c80cd660 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89cfe8c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f89c8145cf0 con 0x7f89c8145910 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89ce429640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89c80cd660 0x7f89c80cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89ce429640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89c80cd660 0x7f89c80cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52108/0 (socket says 172.21.15.26:52108) 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89ce429640 1 -- 172.21.15.26:0/947241587 learned_addr learned my addr 172.21.15.26:0/947241587 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89cd427640 1 --1- 172.21.15.26:0/947241587 >> v1:172.21.15.26:6789/0 conn(0x7f89c806d350 0x7f89c815d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58796/0 (socket says 172.21.15.26:58796) 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89ce429640 1 -- 172.21.15.26:0/947241587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c8145910 msgr2=0x7f89c806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89cdc28640 1 --2- 172.21.15.26:0/947241587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c8145910 0x7f89c806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89ccc26640 1 -- 172.21.15.26:0/947241587 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2783971614 0 0) 0x7f89c80cfbc0 con 0x7f89c806d350 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89ce429640 1 --2- 172.21.15.26:0/947241587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c8145910 0x7f89c806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89ce429640 1 -- 172.21.15.26:0/947241587 >> v1:172.21.15.26:6789/0 conn(0x7f89c806d350 legacy=0x7f89c815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89ce429640 1 -- 172.21.15.26:0/947241587 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c806d7f0 con 0x7f89c80cd660 2024-04-24T23:24:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.457+0000 7f89cdc28640 1 --2- 172.21.15.26:0/947241587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c8145910 0x7f89c806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89ce429640 1 --2- 172.21.15.26:0/947241587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c80cd660 0x7f89c80cdab0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009e50 tx=0x7f89c002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c302c03ac73d842 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89ccc26640 1 -- 172.21.15.26:0/947241587 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0036070 con 0x7f89c80cd660 2024-04-24T23:24:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89ccc26640 1 -- 172.21.15.26:0/947241587 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c0031600 con 0x7f89c80cd660 2024-04-24T23:24:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89ccc26640 1 -- 172.21.15.26:0/947241587 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c00319d0 con 0x7f89c80cd660 2024-04-24T23:24:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89cfe8c640 1 -- 172.21.15.26:0/947241587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c80cd660 msgr2=0x7f89c80cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89cfe8c640 1 --2- 172.21.15.26:0/947241587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c80cd660 0x7f89c80cdab0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009e50 tx=0x7f89c002a730 comp rx=0 tx=0).stop 2024-04-24T23:24:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89cfe8c640 1 -- 172.21.15.26:0/947241587 shutdown_connections 2024-04-24T23:24:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89cfe8c640 1 --2- 172.21.15.26:0/947241587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c80cd660 0x7f89c80cdab0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89cfe8c640 1 --2- 172.21.15.26:0/947241587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c8145910 0x7f89c806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.458+0000 7f89cfe8c640 1 -- 172.21.15.26:0/947241587 >> 172.21.15.26:0/947241587 conn(0x7f89c806b300 msgr2=0x7f89c806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.459+0000 7f89cfe8c640 1 -- 172.21.15.26:0/947241587 shutdown_connections 2024-04-24T23:24:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.459+0000 7f89cfe8c640 1 -- 172.21.15.26:0/947241587 wait complete. 2024-04-24T23:24:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.460+0000 7f89cfe8c640 1 Processor -- start 2024-04-24T23:24:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.460+0000 7f89cfe8c640 1 -- start start 2024-04-24T23:24:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.460+0000 7f89cfe8c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89c806d350 0x7f89c80b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.460+0000 7f89cfe8c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c80cd660 0x7f89c8139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.460+0000 7f89cfe8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c8145910 0x7f89c80b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.460+0000 7f89cfe8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f89c806e090 con 0x7f89c8145910 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.460+0000 7f89cfe8c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f89c806ddb0 con 0x7f89c806d350 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.460+0000 7f89cfe8c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f89c806e370 con 0x7f89c80cd660 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89ce429640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c8145910 0x7f89c80b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89ce429640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c8145910 0x7f89c80b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52112/0 (socket says 172.21.15.26:52112) 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89ce429640 1 -- 172.21.15.26:0/19687320 learned_addr learned my addr 172.21.15.26:0/19687320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89cd427640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c80cd660 0x7f89c8139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89ce429640 1 -- 172.21.15.26:0/19687320 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c80cd660 msgr2=0x7f89c8139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89cdc28640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89c806d350 0x7f89c80b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89ce429640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c80cd660 0x7f89c8139770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89ce429640 1 -- 172.21.15.26:0/19687320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89c806d350 msgr2=0x7f89c80b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89ce429640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89c806d350 0x7f89c80b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89ce429640 1 -- 172.21.15.26:0/19687320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c80b3b60 con 0x7f89c8145910 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89ce429640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c8145910 0x7f89c80b30a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f89c0034040 tx=0x7f89c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89be7fc640 1 -- 172.21.15.26:0/19687320 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0036050 con 0x7f89c8145910 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89be7fc640 1 -- 172.21.15.26:0/19687320 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c0042070 con 0x7f89c8145910 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c80b3e40 con 0x7f89c8145910 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89be7fc640 1 -- 172.21.15.26:0/19687320 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0009510 con 0x7f89c8145910 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.461+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c80b4400 con 0x7f89c8145910 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.462+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c80694c0 con 0x7f89c8145910 2024-04-24T23:24:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.463+0000 7f89be7fc640 1 -- 172.21.15.26:0/19687320 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89c0007bf0 con 0x7f89c8145910 2024-04-24T23:24:15.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.463+0000 7f89be7fc640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89ac06be60 0x7f89ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.463+0000 7f89be7fc640 1 -- 172.21.15.26:0/19687320 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f89c00abc60 con 0x7f89c8145910 2024-04-24T23:24:15.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.463+0000 7f89cdc28640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89ac06be60 0x7f89ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.463+0000 7f89cdc28640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89ac06be60 0x7f89ac06e320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b840 tx=0x7f89c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:15.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.467+0000 7f89be7fc640 1 -- 172.21.15.26:0/19687320 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f89c0045360 con 0x7f89c8145910 2024-04-24T23:24:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.652+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89c8144f20 con 0x7f89c8145910 2024-04-24T23:24:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.653+0000 7f89be7fc640 1 -- 172.21.15.26:0/19687320 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16618 (secure 0 0 0) 0x7f89c007a6a0 con 0x7f89c8145910 2024-04-24T23:24:15.655 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:15.655 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:24:14.712931+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:24:13.001912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:24:13.678528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89ac06be60 msgr2=0x7f89ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89ac06be60 0x7f89ac06e320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b840 tx=0x7f89c4008040 comp rx=0 tx=0).stop 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c8145910 msgr2=0x7f89c80b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c8145910 0x7f89c80b30a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f89c0034040 tx=0x7f89c0004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 shutdown_connections 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89ac06be60 0x7f89ac06e320 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c8145910 0x7f89c80b30a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f89c80cd660 0x7f89c8139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 --2- 172.21.15.26:0/19687320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89c806d350 0x7f89c80b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.655+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 >> 172.21.15.26:0/19687320 conn(0x7f89c806b300 msgr2=0x7f89c80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.656+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 shutdown_connections 2024-04-24T23:24:15.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.656+0000 7f89cfe8c640 1 -- 172.21.15.26:0/19687320 wait complete. 2024-04-24T23:24:15.668 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:24:15.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.767+0000 7f69e9b9e640 1 Processor -- start 2024-04-24T23:24:15.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.767+0000 7f69e9b9e640 1 -- start start 2024-04-24T23:24:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e9b9e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69e415e380 0x7f69e40b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e9b9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69e40b4fd0 0x7f69e4072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e9b9e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e40b5700 con 0x7f69e40b0030 2024-04-24T23:24:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e9b9e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f69e415e760 con 0x7f69e415e380 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e9b9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f69e40725b0 con 0x7f69e40b4fd0 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69e415e380 0x7f69e40b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69e415e380 0x7f69e40b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52114/0 (socket says 172.21.15.26:52114) 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f69e40b0030 0x7f69e40ae530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58804/0 (socket says 172.21.15.26:58804) 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e37fe640 1 -- 172.21.15.26:0/2286997255 learned_addr learned my addr 172.21.15.26:0/2286997255 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e2ffd640 1 --2- 172.21.15.26:0/2286997255 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69e40b4fd0 0x7f69e4072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e37fe640 1 -- 172.21.15.26:0/2286997255 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69e40b4fd0 msgr2=0x7f69e4072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e37fe640 1 --2- 172.21.15.26:0/2286997255 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69e40b4fd0 0x7f69e4072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e37fe640 1 -- 172.21.15.26:0/2286997255 >> v1:172.21.15.26:6789/0 conn(0x7f69e40b0030 legacy=0x7f69e40ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e27fc640 1 -- 172.21.15.26:0/2286997255 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2259667927 0 0) 0x7f69e40b5700 con 0x7f69e40b0030 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.768+0000 7f69e37fe640 1 -- 172.21.15.26:0/2286997255 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e40b59e0 con 0x7f69e415e380 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.769+0000 7f69e37fe640 1 --2- 172.21.15.26:0/2286997255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e415e380 0x7f69e40b4530 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009870 tx=0x7f69d402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6314f7765eb297fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.769+0000 7f69e27fc640 1 -- 172.21.15.26:0/2286997255 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4037070 con 0x7f69e415e380 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.769+0000 7f69e27fc640 1 -- 172.21.15.26:0/2286997255 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d402ae50 con 0x7f69e415e380 2024-04-24T23:24:15.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.769+0000 7f69e27fc640 1 -- 172.21.15.26:0/2286997255 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4032860 con 0x7f69e415e380 2024-04-24T23:24:15.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.770+0000 7f69e9b9e640 1 -- 172.21.15.26:0/2286997255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e415e380 msgr2=0x7f69e40b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.770+0000 7f69e9b9e640 1 --2- 172.21.15.26:0/2286997255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e415e380 0x7f69e40b4530 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009870 tx=0x7f69d402a300 comp rx=0 tx=0).stop 2024-04-24T23:24:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.770+0000 7f69e9b9e640 1 -- 172.21.15.26:0/2286997255 shutdown_connections 2024-04-24T23:24:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.770+0000 7f69e9b9e640 1 --2- 172.21.15.26:0/2286997255 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69e40b4fd0 0x7f69e4072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.770+0000 7f69e9b9e640 1 --2- 172.21.15.26:0/2286997255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e415e380 0x7f69e40b4530 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.770+0000 7f69e9b9e640 1 -- 172.21.15.26:0/2286997255 >> 172.21.15.26:0/2286997255 conn(0x7f69e406e2c0 msgr2=0x7f69e406e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.770+0000 7f69e9b9e640 1 -- 172.21.15.26:0/2286997255 shutdown_connections 2024-04-24T23:24:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.770+0000 7f69e9b9e640 1 -- 172.21.15.26:0/2286997255 wait complete. 2024-04-24T23:24:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.771+0000 7f69e9b9e640 1 Processor -- start 2024-04-24T23:24:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.771+0000 7f69e9b9e640 1 -- start start 2024-04-24T23:24:15.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e9b9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e40b0030 0x7f69e4157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e9b9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69e40b4fd0 0x7f69e40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e9b9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69e415e380 0x7f69e41f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e9b9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4072f80 con 0x7f69e40b0030 2024-04-24T23:24:15.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e9b9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40b5410 con 0x7f69e40b4fd0 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e9b9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f69e40aec40 con 0x7f69e415e380 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e40b0030 0x7f69e4157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e40b0030 0x7f69e4157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52130/0 (socket says 172.21.15.26:52130) 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e37fe640 1 -- 172.21.15.26:0/3079908830 learned_addr learned my addr 172.21.15.26:0/3079908830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e37fe640 1 -- 172.21.15.26:0/3079908830 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69e415e380 msgr2=0x7f69e41f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e37fe640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69e415e380 0x7f69e41f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e37fe640 1 -- 172.21.15.26:0/3079908830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69e40b4fd0 msgr2=0x7f69e40d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e37fe640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69e40b4fd0 0x7f69e40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.772+0000 7f69e37fe640 1 -- 172.21.15.26:0/3079908830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e40b27e0 con 0x7f69e40b0030 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.773+0000 7f69e37fe640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e40b0030 0x7f69e4157a80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.773+0000 7f69cbfff640 1 -- 172.21.15.26:0/3079908830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4041020 con 0x7f69e40b0030 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.773+0000 7f69cbfff640 1 -- 172.21.15.26:0/3079908830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d403c670 con 0x7f69e40b0030 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.773+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e40b2ac0 con 0x7f69e40b0030 2024-04-24T23:24:15.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.773+0000 7f69cbfff640 1 -- 172.21.15.26:0/3079908830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4037040 con 0x7f69e40b0030 2024-04-24T23:24:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.773+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e40b3080 con 0x7f69e40b0030 2024-04-24T23:24:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.774+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e40694c0 con 0x7f69e40b0030 2024-04-24T23:24:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.774+0000 7f69cbfff640 1 -- 172.21.15.26:0/3079908830 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f69d4044020 con 0x7f69e40b0030 2024-04-24T23:24:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.775+0000 7f69cbfff640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f69d006be30 0x7f69d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.775+0000 7f69cbfff640 1 -- 172.21.15.26:0/3079908830 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f69d40ab480 con 0x7f69e40b0030 2024-04-24T23:24:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.775+0000 7f69e2ffd640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f69d006be30 0x7f69d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.775+0000 7f69e2ffd640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f69d006be30 0x7f69d006e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f69e415add0 tx=0x7f69d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:15.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.779+0000 7f69cbfff640 1 -- 172.21.15.26:0/3079908830 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f69d404b030 con 0x7f69e40b0030 2024-04-24T23:24:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.961+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69e40b3ec0 con 0x7f69e40b0030 2024-04-24T23:24:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.962+0000 7f69cbfff640 1 -- 172.21.15.26:0/3079908830 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16618 (secure 0 0 0) 0x7f69d4079ec0 con 0x7f69e40b0030 2024-04-24T23:24:15.964 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:15.964 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:24:14.712931+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:24:13.001912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:24:13.678528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:24:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f69d006be30 msgr2=0x7f69d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f69d006be30 0x7f69d006e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f69e415add0 tx=0x7f69d8008040 comp rx=0 tx=0).stop 2024-04-24T23:24:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e40b0030 msgr2=0x7f69e4157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e40b0030 0x7f69e4157a80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69d40046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 shutdown_connections 2024-04-24T23:24:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f69d006be30 0x7f69d006e2f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69e415e380 0x7f69e41f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69e40b4fd0 0x7f69e40d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 --2- 172.21.15.26:0/3079908830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e40b0030 0x7f69e4157a80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 >> 172.21.15.26:0/3079908830 conn(0x7f69e406e2c0 msgr2=0x7f69e41612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 shutdown_connections 2024-04-24T23:24:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:15.965+0000 7f69e9b9e640 1 -- 172.21.15.26:0/3079908830 wait complete. 2024-04-24T23:24:15.977 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:24:16.000 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:24:15.998+0000 7f75ab619640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:24:16.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.083+0000 7f85528e1640 1 Processor -- start 2024-04-24T23:24:16.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.083+0000 7f85528e1640 1 -- start start 2024-04-24T23:24:16.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f85528e1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f854c158ae0 0x7f854c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f85528e1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f854c071720 0x7f854c071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f85528e1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f854c06b1c0 con 0x7f854c0d0ce0 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f85528e1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f854c072070 con 0x7f854c158ae0 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f85528e1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f854c06ec90 con 0x7f854c071720 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f854bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f854c158ae0 0x7f854c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f854b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f854c0d0ce0 0x7f854c15e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58814/0 (socket says 172.21.15.26:58814) 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f854bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f854c158ae0 0x7f854c06e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52136/0 (socket says 172.21.15.26:52136) 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f854b7fe640 1 -- 172.21.15.26:0/623141330 learned_addr learned my addr 172.21.15.26:0/623141330 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f854bfff640 1 -- 172.21.15.26:0/623141330 >> v2:172.21.15.29:3300/0 conn(0x7f854c071720 msgr2=0x7f854c071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f8550e7e640 1 --2- 172.21.15.26:0/623141330 >> v2:172.21.15.29:3300/0 conn(0x7f854c071720 0x7f854c071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f854affd640 1 -- 172.21.15.26:0/623141330 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 883092420 0 0) 0x7f854c06b1c0 con 0x7f854c0d0ce0 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f854bfff640 1 --2- 172.21.15.26:0/623141330 >> v2:172.21.15.29:3300/0 conn(0x7f854c071720 0x7f854c071b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:16.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f854bfff640 1 -- 172.21.15.26:0/623141330 >> v1:172.21.15.26:6789/0 conn(0x7f854c0d0ce0 legacy=0x7f854c15e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f854bfff640 1 -- 172.21.15.26:0/623141330 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c06ef70 con 0x7f854c158ae0 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.084+0000 7f8550e7e640 1 --2- 172.21.15.26:0/623141330 >> v2:172.21.15.29:3300/0 conn(0x7f854c071720 0x7f854c071b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.085+0000 7f854bfff640 1 --2- 172.21.15.26:0/623141330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c158ae0 0x7f854c06e620 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f854c15faa0 tx=0x7f854002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8ad231e7b32a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.085+0000 7f854affd640 1 -- 172.21.15.26:0/623141330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540037070 con 0x7f854c158ae0 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.085+0000 7f854affd640 1 -- 172.21.15.26:0/623141330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8540032440 con 0x7f854c158ae0 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.085+0000 7f854affd640 1 -- 172.21.15.26:0/623141330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854002ae60 con 0x7f854c158ae0 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.086+0000 7f85528e1640 1 -- 172.21.15.26:0/623141330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c158ae0 msgr2=0x7f854c06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.086+0000 7f85528e1640 1 --2- 172.21.15.26:0/623141330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c158ae0 0x7f854c06e620 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f854c15faa0 tx=0x7f854002a110 comp rx=0 tx=0).stop 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.086+0000 7f85528e1640 1 -- 172.21.15.26:0/623141330 shutdown_connections 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.086+0000 7f85528e1640 1 --2- 172.21.15.26:0/623141330 >> v2:172.21.15.29:3300/0 conn(0x7f854c071720 0x7f854c071b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.086+0000 7f85528e1640 1 --2- 172.21.15.26:0/623141330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c158ae0 0x7f854c06e620 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.086+0000 7f85528e1640 1 -- 172.21.15.26:0/623141330 >> 172.21.15.26:0/623141330 conn(0x7f854c0b2970 msgr2=0x7f854c0d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.086+0000 7f85528e1640 1 -- 172.21.15.26:0/623141330 shutdown_connections 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.086+0000 7f85528e1640 1 -- 172.21.15.26:0/623141330 wait complete. 2024-04-24T23:24:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.087+0000 7f85528e1640 1 Processor -- start 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.087+0000 7f85528e1640 1 -- start start 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f85528e1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f854c071720 0x7f854c1f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f85528e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c0d0ce0 0x7f854c14a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f85528e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f854c158ae0 0x7f854c0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f85528e1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f854c06f530 con 0x7f854c0d0ce0 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f85528e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f854c15f060 con 0x7f854c158ae0 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f85528e1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f854c0aff60 con 0x7f854c071720 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f854b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c0d0ce0 0x7f854c14a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f854b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c0d0ce0 0x7f854c14a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52142/0 (socket says 172.21.15.26:52142) 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f854b7fe640 1 -- 172.21.15.26:0/1346995436 learned_addr learned my addr 172.21.15.26:0/1346995436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f854b7fe640 1 -- 172.21.15.26:0/1346995436 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f854c071720 msgr2=0x7f854c1f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f854b7fe640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f854c071720 0x7f854c1f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:16.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f854b7fe640 1 -- 172.21.15.26:0/1346995436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f854c158ae0 msgr2=0x7f854c0aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f8550e7e640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f854c158ae0 0x7f854c0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f854b7fe640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f854c158ae0 0x7f854c0aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.088+0000 7f854b7fe640 1 -- 172.21.15.26:0/1346995436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c0adac0 con 0x7f854c0d0ce0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.089+0000 7f854b7fe640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c0d0ce0 0x7f854c14a000 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c4d0 tx=0x7f853c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.089+0000 7f8548ff9640 1 -- 172.21.15.26:0/1346995436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c018070 con 0x7f854c0d0ce0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.089+0000 7f8548ff9640 1 -- 172.21.15.26:0/1346995436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853c007500 con 0x7f854c0d0ce0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.089+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c0adda0 con 0x7f854c0d0ce0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.089+0000 7f8548ff9640 1 -- 172.21.15.26:0/1346995436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c004a20 con 0x7f854c0d0ce0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.089+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c0ae360 con 0x7f854c0d0ce0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.090+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854c075610 con 0x7f854c0d0ce0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.090+0000 7f8548ff9640 1 -- 172.21.15.26:0/1346995436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f853c01b070 con 0x7f854c0d0ce0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.091+0000 7f8548ff9640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f853806be30 0x7f853806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.091+0000 7f8548ff9640 1 -- 172.21.15.26:0/1346995436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f853c08a1e0 con 0x7f854c0d0ce0 2024-04-24T23:24:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.091+0000 7f854bfff640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f853806be30 0x7f853806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:16.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.091+0000 7f854bfff640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f853806be30 0x7f853806e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8540034000 tx=0x7f854002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:16.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.095+0000 7f8548ff9640 1 -- 172.21.15.26:0/1346995436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f853c058b90 con 0x7f854c0d0ce0 2024-04-24T23:24:16.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:16.316+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f854c067580 con 0x7f854c0d0ce0 2024-04-24T23:24:17.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.016+0000 7f8548ff9640 1 -- 172.21.15.26:0/1346995436 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) ==== 118+0+0 (secure 0 0 0) 0x7f853c058490 con 0x7f854c0d0ce0 2024-04-24T23:24:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f853806be30 msgr2=0x7f853806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f853806be30 0x7f853806e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8540034000 tx=0x7f854002c000 comp rx=0 tx=0).stop 2024-04-24T23:24:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c0d0ce0 msgr2=0x7f854c14a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c0d0ce0 0x7f854c14a000 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c4d0 tx=0x7f853c00c990 comp rx=0 tx=0).stop 2024-04-24T23:24:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 shutdown_connections 2024-04-24T23:24:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f853806be30 0x7f853806e2f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f854c158ae0 0x7f854c0aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f854c0d0ce0 0x7f854c14a000 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 --2- 172.21.15.26:0/1346995436 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f854c071720 0x7f854c1f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 >> 172.21.15.26:0/1346995436 conn(0x7f854c0b2970 msgr2=0x7f854c15e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:17.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.018+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 shutdown_connections 2024-04-24T23:24:17.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.019+0000 7f85528e1640 1 -- 172.21.15.26:0/1346995436 wait complete. 2024-04-24T23:24:17.032 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:24:17.032 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:24:17.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.137+0000 7f7253bb7640 1 Processor -- start 2024-04-24T23:24:17.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.137+0000 7f7253bb7640 1 -- start start 2024-04-24T23:24:17.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.138+0000 7f7253bb7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f724c15e370 0x7f724c0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.138+0000 7f7253bb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c0d4c10 0x7f724c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.138+0000 7f7253bb7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f724c0d2120 con 0x7f724c15e750 2024-04-24T23:24:17.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.138+0000 7f7253bb7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f724c0d1510 con 0x7f724c15e370 2024-04-24T23:24:17.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.138+0000 7f7253bb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f724c0d1ed0 con 0x7f724c0d4c10 2024-04-24T23:24:17.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7251953640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f724c15e370 0x7f724c0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7252154640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c0d4c10 0x7f724c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7251152640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f724c15e750 0x7f724c0b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:43930/0 (socket says 172.21.15.26:43930) 2024-04-24T23:24:17.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7251953640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f724c15e370 0x7f724c0d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:39580/0 (socket says 172.21.15.26:39580) 2024-04-24T23:24:17.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7251152640 1 -- 172.21.15.26:0/179711505 learned_addr learned my addr 172.21.15.26:0/179711505 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:17.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7252154640 1 -- 172.21.15.26:0/179711505 >> v2:172.21.15.29:3300/0 conn(0x7f724c15e370 msgr2=0x7f724c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:17.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7252154640 1 --2- 172.21.15.26:0/179711505 >> v2:172.21.15.29:3300/0 conn(0x7f724c15e370 0x7f724c0d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7252154640 1 -- 172.21.15.26:0/179711505 >> v1:172.21.15.29:6789/0 conn(0x7f724c15e750 legacy=0x7f724c0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7252154640 1 -- 172.21.15.26:0/179711505 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c0b3740 con 0x7f724c0d4c10 2024-04-24T23:24:17.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.139+0000 7f7251953640 1 --2- 172.21.15.26:0/179711505 >> v2:172.21.15.29:3300/0 conn(0x7f724c15e370 0x7f724c0d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:17.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.140+0000 7f7252154640 1 --2- 172.21.15.26:0/179711505 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c0d4c10 0x7f724c0cfca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72440097a0 tx=0x7f724402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e96452d44d662ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.140+0000 7f7250951640 1 -- 172.21.15.26:0/179711505 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244037070 con 0x7f724c0d4c10 2024-04-24T23:24:17.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.140+0000 7f7250951640 1 -- 172.21.15.26:0/179711505 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7244032440 con 0x7f724c0d4c10 2024-04-24T23:24:17.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.140+0000 7f7250951640 1 -- 172.21.15.26:0/179711505 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244032810 con 0x7f724c0d4c10 2024-04-24T23:24:17.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.140+0000 7f7253bb7640 1 -- 172.21.15.26:0/179711505 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c0d4c10 msgr2=0x7f724c0cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.140+0000 7f7253bb7640 1 --2- 172.21.15.26:0/179711505 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c0d4c10 0x7f724c0cfca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72440097a0 tx=0x7f724402a180 comp rx=0 tx=0).stop 2024-04-24T23:24:17.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.141+0000 7f7253bb7640 1 -- 172.21.15.26:0/179711505 shutdown_connections 2024-04-24T23:24:17.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.141+0000 7f7253bb7640 1 --2- 172.21.15.26:0/179711505 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c0d4c10 0x7f724c0cfca0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.141+0000 7f7253bb7640 1 --2- 172.21.15.26:0/179711505 >> v2:172.21.15.29:3300/0 conn(0x7f724c15e370 0x7f724c0d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.141+0000 7f7253bb7640 1 -- 172.21.15.26:0/179711505 >> 172.21.15.26:0/179711505 conn(0x7f724c0d40f0 msgr2=0x7f724c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:17.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.141+0000 7f7253bb7640 1 -- 172.21.15.26:0/179711505 shutdown_connections 2024-04-24T23:24:17.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.141+0000 7f7253bb7640 1 -- 172.21.15.26:0/179711505 wait complete. 2024-04-24T23:24:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.142+0000 7f7253bb7640 1 Processor -- start 2024-04-24T23:24:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.142+0000 7f7253bb7640 1 -- start start 2024-04-24T23:24:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7253bb7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c0d4c10 0x7f724c0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7253bb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c15e370 0x7f724c147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7253bb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f724c1f3a20 0x7f724c06bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7253bb7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f724c0d2a60 con 0x7f724c1f3a20 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7253bb7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f724c0b3d00 con 0x7f724c0d4c10 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7253bb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f724c0d2be0 con 0x7f724c15e370 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7252154640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f724c1f3a20 0x7f724c06bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7252154640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f724c1f3a20 0x7f724c06bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52144/0 (socket says 172.21.15.26:52144) 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7252154640 1 -- 172.21.15.26:0/3897275098 learned_addr learned my addr 172.21.15.26:0/3897275098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7251953640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c0d4c10 0x7f724c0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7251152640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c15e370 0x7f724c147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7252154640 1 -- 172.21.15.26:0/3897275098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c15e370 msgr2=0x7f724c147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7252154640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c15e370 0x7f724c147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7252154640 1 -- 172.21.15.26:0/3897275098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c0d4c10 msgr2=0x7f724c0ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7252154640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c0d4c10 0x7f724c0ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7252154640 1 -- 172.21.15.26:0/3897275098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c147b90 con 0x7f724c1f3a20 2024-04-24T23:24:17.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7251953640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c0d4c10 0x7f724c0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:17.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f7252154640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f724c1f3a20 0x7f724c06bdb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f72440098c0 tx=0x7f72440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.143+0000 7f724a7fc640 1 -- 172.21.15.26:0/3897275098 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244037040 con 0x7f724c1f3a20 2024-04-24T23:24:17.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.144+0000 7f724a7fc640 1 -- 172.21.15.26:0/3897275098 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7244042070 con 0x7f724c1f3a20 2024-04-24T23:24:17.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.144+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c0c9830 con 0x7f724c1f3a20 2024-04-24T23:24:17.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.144+0000 7f724a7fc640 1 -- 172.21.15.26:0/3897275098 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72440308a0 con 0x7f724c1f3a20 2024-04-24T23:24:17.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.144+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c0c9df0 con 0x7f724c1f3a20 2024-04-24T23:24:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.145+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724c075610 con 0x7f724c1f3a20 2024-04-24T23:24:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.145+0000 7f724a7fc640 1 -- 172.21.15.26:0/3897275098 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7244030a40 con 0x7f724c1f3a20 2024-04-24T23:24:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.145+0000 7f724a7fc640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f723806be30 0x7f723806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.146+0000 7f724a7fc640 1 -- 172.21.15.26:0/3897275098 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f72440ab770 con 0x7f724c1f3a20 2024-04-24T23:24:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.146+0000 7f7251953640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f723806be30 0x7f723806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.146+0000 7f7251953640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f723806be30 0x7f723806e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f723c007520 tx=0x7f723c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.150+0000 7f724a7fc640 1 -- 172.21.15.26:0/3897275098 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f724c147b90 con 0x7f724c1f3a20 2024-04-24T23:24:17.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:17.371+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f724c0b5380 con 0x7f724c1f3a20 2024-04-24T23:24:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.020+0000 7f724a7fc640 1 -- 172.21.15.26:0/3897275098 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f724407a120 con 0x7f724c1f3a20 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f723806be30 msgr2=0x7f723806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f723806be30 0x7f723806e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f723c007520 tx=0x7f723c00b040 comp rx=0 tx=0).stop 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f724c1f3a20 msgr2=0x7f724c06bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f724c1f3a20 0x7f724c06bdb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f72440098c0 tx=0x7f72440046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 shutdown_connections 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f723806be30 0x7f723806e2f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f724c1f3a20 0x7f724c06bdb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f724c15e370 0x7f724c147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 --2- 172.21.15.26:0/3897275098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f724c0d4c10 0x7f724c0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 >> 172.21.15.26:0/3897275098 conn(0x7f724c0d40f0 msgr2=0x7f724c15d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:18.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.023+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 shutdown_connections 2024-04-24T23:24:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.024+0000 7f7253bb7640 1 -- 172.21.15.26:0/3897275098 wait complete. 2024-04-24T23:24:18.038 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:24:18.038 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:18.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.145+0000 7fb8e2b70640 1 Processor -- start 2024-04-24T23:24:18.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.145+0000 7fb8e2b70640 1 -- start start 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8e2b70640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb8dc143c40 0x7fb8dc0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8e2b70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8e2b70640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8dc0cd0b0 con 0x7fb8dc15eb00 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8e2b70640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc071b70 con 0x7fb8dc143c40 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8e2b70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc071240 con 0x7fb8dc0cc860 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8e110d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb8dc15eb00 0x7fb8dc0b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58826/0 (socket says 172.21.15.26:58826) 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8e110d640 1 -- 172.21.15.26:0/2074390942 learned_addr learned my addr 172.21.15.26:0/2074390942 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8dbfff640 1 --2- 172.21.15.26:0/2074390942 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8e090c640 1 --2- 172.21.15.26:0/2074390942 >> v2:172.21.15.29:3300/0 conn(0x7fb8dc143c40 0x7fb8dc0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8db7fe640 1 -- 172.21.15.26:0/2074390942 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1952245539 0 0) 0x7fb8dc0cd0b0 con 0x7fb8dc15eb00 2024-04-24T23:24:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8db7fe640 1 -- 172.21.15.26:0/2074390942 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8cc005730 con 0x7fb8dc15eb00 2024-04-24T23:24:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.146+0000 7fb8dbfff640 1 -- 172.21.15.26:0/2074390942 >> v2:172.21.15.29:3300/0 conn(0x7fb8dc143c40 msgr2=0x7fb8dc0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.147+0000 7fb8dbfff640 1 --2- 172.21.15.26:0/2074390942 >> v2:172.21.15.29:3300/0 conn(0x7fb8dc143c40 0x7fb8dc0cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.147+0000 7fb8dbfff640 1 -- 172.21.15.26:0/2074390942 >> v1:172.21.15.26:6789/0 conn(0x7fb8dc15eb00 legacy=0x7fb8dc0b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.147+0000 7fb8dbfff640 1 -- 172.21.15.26:0/2074390942 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc071520 con 0x7fb8dc0cc860 2024-04-24T23:24:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.147+0000 7fb8e090c640 1 --2- 172.21.15.26:0/2074390942 >> v2:172.21.15.29:3300/0 conn(0x7fb8dc143c40 0x7fb8dc0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.147+0000 7fb8dbfff640 1 --2- 172.21.15.26:0/2074390942 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc072000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80097a0 tx=0x7fb8c802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81dd7a19dfb570 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.147+0000 7fb8db7fe640 1 -- 172.21.15.26:0/2074390942 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8037070 con 0x7fb8dc0cc860 2024-04-24T23:24:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8db7fe640 1 -- 172.21.15.26:0/2074390942 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c8032440 con 0x7fb8dc0cc860 2024-04-24T23:24:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8db7fe640 1 -- 172.21.15.26:0/2074390942 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8032810 con 0x7fb8dc0cc860 2024-04-24T23:24:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2074390942 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 msgr2=0x7fb8dc072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8e2b70640 1 --2- 172.21.15.26:0/2074390942 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc072000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80097a0 tx=0x7fb8c802a180 comp rx=0 tx=0).stop 2024-04-24T23:24:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2074390942 shutdown_connections 2024-04-24T23:24:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8e2b70640 1 --2- 172.21.15.26:0/2074390942 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc072000 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8e2b70640 1 --2- 172.21.15.26:0/2074390942 >> v2:172.21.15.29:3300/0 conn(0x7fb8dc143c40 0x7fb8dc0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2074390942 >> 172.21.15.26:0/2074390942 conn(0x7fb8dc06b300 msgr2=0x7fb8dc06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2074390942 shutdown_connections 2024-04-24T23:24:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.148+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2074390942 wait complete. 2024-04-24T23:24:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.149+0000 7fb8e2b70640 1 Processor -- start 2024-04-24T23:24:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.149+0000 7fb8e2b70640 1 -- start start 2024-04-24T23:24:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e2b70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc0c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e2b70640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8dc143c40 0x7fb8dc15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e2b70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc15eb00 0x7fb8dc0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e2b70640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc072540 con 0x7fb8dc15eb00 2024-04-24T23:24:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e2b70640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc144020 con 0x7fb8dc143c40 2024-04-24T23:24:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e2b70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc0c6c90 con 0x7fb8dc0cc860 2024-04-24T23:24:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e110d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc15eb00 0x7fb8dc0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e110d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc15eb00 0x7fb8dc0c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52160/0 (socket says 172.21.15.26:52160) 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e110d640 1 -- 172.21.15.26:0/2636477647 learned_addr learned my addr 172.21.15.26:0/2636477647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8dbfff640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8dc143c40 0x7fb8dc15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.150+0000 7fb8e090c640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc0c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8e110d640 1 -- 172.21.15.26:0/2636477647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 msgr2=0x7fb8dc0c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8e110d640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc0c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8e110d640 1 -- 172.21.15.26:0/2636477647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8dc143c40 msgr2=0x7fb8dc15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8e110d640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8dc143c40 0x7fb8dc15a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8e110d640 1 -- 172.21.15.26:0/2636477647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc0c6f70 con 0x7fb8dc15eb00 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8e090c640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc0c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8e110d640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc15eb00 0x7fb8dc0c7d30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400cad0 tx=0x7fb8d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8d97fa640 1 -- 172.21.15.26:0/2636477647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4017070 con 0x7fb8dc15eb00 2024-04-24T23:24:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8d97fa640 1 -- 172.21.15.26:0/2636477647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d400ac30 con 0x7fb8dc15eb00 2024-04-24T23:24:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc0c7250 con 0x7fb8dc15eb00 2024-04-24T23:24:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8d97fa640 1 -- 172.21.15.26:0/2636477647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4012400 con 0x7fb8dc15eb00 2024-04-24T23:24:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.151+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc0c7810 con 0x7fb8dc15eb00 2024-04-24T23:24:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.152+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8dc0694c0 con 0x7fb8dc15eb00 2024-04-24T23:24:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.152+0000 7fb8d97fa640 1 -- 172.21.15.26:0/2636477647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8d401a070 con 0x7fb8dc15eb00 2024-04-24T23:24:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.153+0000 7fb8d97fa640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8c006be30 0x7fb8c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.153+0000 7fb8d97fa640 1 -- 172.21.15.26:0/2636477647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb8d408ac60 con 0x7fb8dc15eb00 2024-04-24T23:24:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.153+0000 7fb8e090c640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8c006be30 0x7fb8c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.153+0000 7fb8e090c640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8c006be30 0x7fb8c006e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009ba0 tx=0x7fb8d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.157+0000 7fb8d97fa640 1 -- 172.21.15.26:0/2636477647 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb8d4058eb0 con 0x7fb8dc15eb00 2024-04-24T23:24:18.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.380+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8dc0b4d20 con 0x7fb8dc15eb00 2024-04-24T23:24:18.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.380+0000 7fb8d97fa640 1 -- 172.21.15.26:0/2636477647 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5181 (secure 0 0 0) 0x7fb8d405d1b0 con 0x7fb8dc15eb00 2024-04-24T23:24:18.382 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:18.382 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1519104007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1519104007},{"type":"v1","addr":"172.21.15.26:6836","nonce":1519104007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2743842996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2743842996},{"type":"v1","addr":"172.21.15.26:6837","nonce":2743842996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:24:14.713249+0000","modified":"2024-04-24T23:24:17.372849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2096523178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2096523178},{"type":"v1","addr":"172.21.15.29:6835","nonce":2096523178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":4}]} 2024-04-24T23:24:18.382 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2024-04-24T23:24:18.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8c006be30 msgr2=0x7fb8c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8c006be30 0x7fb8c006e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009ba0 tx=0x7fb8d0008040 comp rx=0 tx=0).stop 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc15eb00 msgr2=0x7fb8dc0c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc15eb00 0x7fb8dc0c7d30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400cad0 tx=0x7fb8d400cf90 comp rx=0 tx=0).stop 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 shutdown_connections 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8c006be30 0x7fb8c006e2f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8dc15eb00 0x7fb8dc0c7d30 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8dc143c40 0x7fb8dc15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 --2- 172.21.15.26:0/2636477647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8dc0cc860 0x7fb8dc0c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 >> 172.21.15.26:0/2636477647 conn(0x7fb8dc06b300 msgr2=0x7fb8dc0cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 shutdown_connections 2024-04-24T23:24:18.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.382+0000 7fb8e2b70640 1 -- 172.21.15.26:0/2636477647 wait complete. 2024-04-24T23:24:18.394 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:24:18.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.494+0000 7f130f3e7640 1 Processor -- start 2024-04-24T23:24:18.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.494+0000 7f130f3e7640 1 -- start start 2024-04-24T23:24:18.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130f3e7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1308143c40 0x7f13080cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130f3e7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 0x7f130815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130f3e7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13080cd0b0 con 0x7f13080cc860 2024-04-24T23:24:18.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130f3e7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1308071b70 con 0x7f1308143c40 2024-04-24T23:24:18.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130f3e7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f130815f470 con 0x7f130815eb00 2024-04-24T23:24:18.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130d183640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1308143c40 0x7f13080cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130d183640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1308143c40 0x7f13080cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52176/0 (socket says 172.21.15.26:52176) 2024-04-24T23:24:18.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130d183640 1 -- 172.21.15.26:0/2490732448 learned_addr learned my addr 172.21.15.26:0/2490732448 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:18.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130d183640 1 -- 172.21.15.26:0/2490732448 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 msgr2=0x7f130815ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:18.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130d183640 1 --2- 172.21.15.26:0/2490732448 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 0x7f130815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130d183640 1 -- 172.21.15.26:0/2490732448 >> v1:172.21.15.29:6789/0 conn(0x7f13080cc860 legacy=0x7f1308072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:18.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.495+0000 7f130d183640 1 -- 172.21.15.26:0/2490732448 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1308071240 con 0x7f1308143c40 2024-04-24T23:24:18.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.496+0000 7f130d183640 1 --2- 172.21.15.26:0/2490732448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 0x7f13080cf870 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1300009870 tx=0x7f130002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3f7feef2d3ae55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.496+0000 7f12fffff640 1 -- 172.21.15.26:0/2490732448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300037070 con 0x7f1308143c40 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.496+0000 7f12fffff640 1 -- 172.21.15.26:0/2490732448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1300032440 con 0x7f1308143c40 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.496+0000 7f12fffff640 1 -- 172.21.15.26:0/2490732448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300032810 con 0x7f1308143c40 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.497+0000 7f130f3e7640 1 -- 172.21.15.26:0/2490732448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 msgr2=0x7f13080cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.497+0000 7f130f3e7640 1 --2- 172.21.15.26:0/2490732448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 0x7f13080cf870 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1300009870 tx=0x7f130002a210 comp rx=0 tx=0).stop 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.497+0000 7f130f3e7640 1 -- 172.21.15.26:0/2490732448 shutdown_connections 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.497+0000 7f130f3e7640 1 --2- 172.21.15.26:0/2490732448 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 0x7f130815ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.497+0000 7f130f3e7640 1 --2- 172.21.15.26:0/2490732448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 0x7f13080cf870 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.497+0000 7f130f3e7640 1 -- 172.21.15.26:0/2490732448 >> 172.21.15.26:0/2490732448 conn(0x7f130806b300 msgr2=0x7f130806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.497+0000 7f130f3e7640 1 -- 172.21.15.26:0/2490732448 shutdown_connections 2024-04-24T23:24:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.497+0000 7f130f3e7640 1 -- 172.21.15.26:0/2490732448 wait complete. 2024-04-24T23:24:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.498+0000 7f130f3e7640 1 Processor -- start 2024-04-24T23:24:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.498+0000 7f130f3e7640 1 -- start start 2024-04-24T23:24:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130f3e7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13080cc860 0x7f13081423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130f3e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 0x7f13081f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130f3e7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 0x7f13081e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130f3e7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1308144020 con 0x7f1308143c40 2024-04-24T23:24:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130f3e7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1308071800 con 0x7f13080cc860 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130f3e7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f13080cccd0 con 0x7f130815eb00 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130c982640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 0x7f13081f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130c982640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 0x7f13081f47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52182/0 (socket says 172.21.15.26:52182) 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130c982640 1 -- 172.21.15.26:0/2827399785 learned_addr learned my addr 172.21.15.26:0/2827399785 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130d183640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13080cc860 0x7f13081423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130d984640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 0x7f13081e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130c982640 1 -- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 msgr2=0x7f13081e6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130c982640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 0x7f13081e6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130c982640 1 -- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13080cc860 msgr2=0x7f13081423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130c982640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13080cc860 0x7f13081423a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130c982640 1 -- 172.21.15.26:0/2827399785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13080a95e0 con 0x7f1308143c40 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130d183640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13080cc860 0x7f13081423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130d984640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 0x7f13081e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.499+0000 7f130c982640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 0x7f13081f47e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e760 tx=0x7f12f800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.500+0000 7f12fdffb640 1 -- 172.21.15.26:0/2827399785 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80041f0 con 0x7f1308143c40 2024-04-24T23:24:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.500+0000 7f12fdffb640 1 -- 172.21.15.26:0/2827399785 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f80043b0 con 0x7f1308143c40 2024-04-24T23:24:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.500+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13081f5230 con 0x7f1308143c40 2024-04-24T23:24:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.500+0000 7f12fdffb640 1 -- 172.21.15.26:0/2827399785 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80116b0 con 0x7f1308143c40 2024-04-24T23:24:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.500+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1308142e50 con 0x7f1308143c40 2024-04-24T23:24:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.501+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13080694c0 con 0x7f1308143c40 2024-04-24T23:24:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.501+0000 7f12fdffb640 1 -- 172.21.15.26:0/2827399785 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12f8011850 con 0x7f1308143c40 2024-04-24T23:24:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.501+0000 7f12fdffb640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f12ec06be60 0x7f12ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.502+0000 7f12fdffb640 1 -- 172.21.15.26:0/2827399785 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f12f808d570 con 0x7f1308143c40 2024-04-24T23:24:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.502+0000 7f130d183640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f12ec06be60 0x7f12ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.502+0000 7f130d183640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f12ec06be60 0x7f12ec06e320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1300034000 tx=0x7f130002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.506+0000 7f12fdffb640 1 -- 172.21.15.26:0/2827399785 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f12f8016030 con 0x7f1308143c40 2024-04-24T23:24:18.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.689+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13080b4d20 con 0x7f1308143c40 2024-04-24T23:24:18.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.690+0000 7f12fdffb640 1 -- 172.21.15.26:0/2827399785 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7f12f808be20 con 0x7f1308143c40 2024-04-24T23:24:18.692 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:18.692 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:24:16.006495+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:24:13.001912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:24:13.678528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:24:18.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f12ec06be60 msgr2=0x7f12ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f12ec06be60 0x7f12ec06e320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1300034000 tx=0x7f130002e000 comp rx=0 tx=0).stop 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 msgr2=0x7f13081f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 0x7f13081f47e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e760 tx=0x7f12f800ec20 comp rx=0 tx=0).stop 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 shutdown_connections 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f12ec06be60 0x7f12ec06e320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130815eb00 0x7f13081e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1308143c40 0x7f13081f47e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 --2- 172.21.15.26:0/2827399785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f13080cc860 0x7f13081423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 >> 172.21.15.26:0/2827399785 conn(0x7f130806b300 msgr2=0x7f13080b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 shutdown_connections 2024-04-24T23:24:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.692+0000 7f130f3e7640 1 -- 172.21.15.26:0/2827399785 wait complete. 2024-04-24T23:24:18.704 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:24:18.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:24:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.804+0000 7fa577129640 1 Processor -- start 2024-04-24T23:24:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.804+0000 7fa577129640 1 -- start start 2024-04-24T23:24:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa577129640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa570145910 0x7fa57006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa577129640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 0x7fa57015d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa577129640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57015d570 con 0x7fa5700cd660 2024-04-24T23:24:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa577129640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa570145cf0 con 0x7fa570145910 2024-04-24T23:24:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa577129640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa5700cfbc0 con 0x7fa57006d350 2024-04-24T23:24:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa574ec5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa570145910 0x7fa57006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa5756c6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa5700cd660 0x7fa57015d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58836/0 (socket says 172.21.15.26:58836) 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa5756c6640 1 -- 172.21.15.26:0/189187040 learned_addr learned my addr 172.21.15.26:0/189187040 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa574ec5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa570145910 0x7fa57006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52196/0 (socket says 172.21.15.26:52196) 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa56ffff640 1 --2- 172.21.15.26:0/189187040 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 0x7fa57015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa574ec5640 1 -- 172.21.15.26:0/189187040 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 msgr2=0x7fa57015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa574ec5640 1 --2- 172.21.15.26:0/189187040 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 0x7fa57015d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa574ec5640 1 -- 172.21.15.26:0/189187040 >> v1:172.21.15.26:6789/0 conn(0x7fa5700cd660 legacy=0x7fa57015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa574ec5640 1 -- 172.21.15.26:0/189187040 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57006d7f0 con 0x7fa570145910 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.805+0000 7fa56f7fe640 1 -- 172.21.15.26:0/189187040 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3245060397 0 0) 0x7fa568001490 con 0x7fa5700cd660 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.806+0000 7fa574ec5640 1 --2- 172.21.15.26:0/189187040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa570145910 0x7fa57006e890 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa5640097a0 tx=0x7fa56402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf3d849862b7747 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.806+0000 7fa56f7fe640 1 -- 172.21.15.26:0/189187040 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564037070 con 0x7fa570145910 2024-04-24T23:24:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.806+0000 7fa56f7fe640 1 -- 172.21.15.26:0/189187040 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa564032440 con 0x7fa570145910 2024-04-24T23:24:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.806+0000 7fa56f7fe640 1 -- 172.21.15.26:0/189187040 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564032810 con 0x7fa570145910 2024-04-24T23:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.807+0000 7fa577129640 1 -- 172.21.15.26:0/189187040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa570145910 msgr2=0x7fa57006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.807+0000 7fa577129640 1 --2- 172.21.15.26:0/189187040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa570145910 0x7fa57006e890 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa5640097a0 tx=0x7fa56402a180 comp rx=0 tx=0).stop 2024-04-24T23:24:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.807+0000 7fa577129640 1 -- 172.21.15.26:0/189187040 shutdown_connections 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.807+0000 7fa577129640 1 --2- 172.21.15.26:0/189187040 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 0x7fa57015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.807+0000 7fa577129640 1 --2- 172.21.15.26:0/189187040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa570145910 0x7fa57006e890 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.807+0000 7fa577129640 1 -- 172.21.15.26:0/189187040 >> 172.21.15.26:0/189187040 conn(0x7fa57006b300 msgr2=0x7fa57006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.807+0000 7fa577129640 1 -- 172.21.15.26:0/189187040 shutdown_connections 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.807+0000 7fa577129640 1 -- 172.21.15.26:0/189187040 wait complete. 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.808+0000 7fa577129640 1 Processor -- start 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.808+0000 7fa577129640 1 -- start start 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa577129640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 0x7fa5701557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa577129640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5700cd660 0x7fa5701e5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa577129640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570145910 0x7fa570144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa577129640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa57006e090 con 0x7fa5700cd660 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa577129640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa57006ddb0 con 0x7fa570145910 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa577129640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa57006e370 con 0x7fa57006d350 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa56ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5700cd660 0x7fa5701e5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa56ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5700cd660 0x7fa5701e5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52210/0 (socket says 172.21.15.26:52210) 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa56ffff640 1 -- 172.21.15.26:0/1055392552 learned_addr learned my addr 172.21.15.26:0/1055392552 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa574ec5640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 0x7fa5701557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa5756c6640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570145910 0x7fa570144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa56ffff640 1 -- 172.21.15.26:0/1055392552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 msgr2=0x7fa5701557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa56ffff640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 0x7fa5701557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa56ffff640 1 -- 172.21.15.26:0/1055392552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570145910 msgr2=0x7fa570144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa56ffff640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570145910 0x7fa570144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa56ffff640 1 -- 172.21.15.26:0/1055392552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa570158d30 con 0x7fa5700cd660 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.809+0000 7fa56ffff640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5700cd660 0x7fa5701e5170 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00d830 tx=0x7fa55c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.810+0000 7fa56d7fa640 1 -- 172.21.15.26:0/1055392552 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00b6c0 con 0x7fa5700cd660 2024-04-24T23:24:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.810+0000 7fa56d7fa640 1 -- 172.21.15.26:0/1055392552 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c019070 con 0x7fa5700cd660 2024-04-24T23:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.810+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5701e4690 con 0x7fa5700cd660 2024-04-24T23:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.810+0000 7fa56d7fa640 1 -- 172.21.15.26:0/1055392552 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00ba80 con 0x7fa5700cd660 2024-04-24T23:24:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.810+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa570155d20 con 0x7fa5700cd660 2024-04-24T23:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.811+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5700694c0 con 0x7fa5700cd660 2024-04-24T23:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.811+0000 7fa56d7fa640 1 -- 172.21.15.26:0/1055392552 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa55c01e460 con 0x7fa5700cd660 2024-04-24T23:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.811+0000 7fa56d7fa640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa55406be30 0x7fa55406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.812+0000 7fa56d7fa640 1 -- 172.21.15.26:0/1055392552 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa55c08b4b0 con 0x7fa5700cd660 2024-04-24T23:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.812+0000 7fa574ec5640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa55406be30 0x7fa55406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.812+0000 7fa574ec5640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa55406be30 0x7fa55406e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa564034000 tx=0x7fa56402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.816+0000 7fa56d7fa640 1 -- 172.21.15.26:0/1055392552 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa55c059ef0 con 0x7fa5700cd660 2024-04-24T23:24:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:18.995+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa57006e830 con 0x7fa55406be30 2024-04-24T23:24:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.001+0000 7fa56d7fa640 1 -- 172.21.15.26:0/1055392552 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7fa57006e830 con 0x7fa55406be30 2024-04-24T23:24:19.005 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:19.007 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:24:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.004+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa55406be30 msgr2=0x7fa55406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.004+0000 7fa577129640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa55406be30 0x7fa55406e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa564034000 tx=0x7fa56402e000 comp rx=0 tx=0).stop 2024-04-24T23:24:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.004+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5700cd660 msgr2=0x7fa5701e5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.004+0000 7fa577129640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5700cd660 0x7fa5701e5170 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00d830 tx=0x7fa55c00dcf0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.005+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 shutdown_connections 2024-04-24T23:24:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.005+0000 7fa577129640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa55406be30 0x7fa55406e2f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.005+0000 7fa577129640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa570145910 0x7fa570144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.005+0000 7fa577129640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5700cd660 0x7fa5701e5170 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.005+0000 7fa577129640 1 --2- 172.21.15.26:0/1055392552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa57006d350 0x7fa5701557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.005+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 >> 172.21.15.26:0/1055392552 conn(0x7fa57006b300 msgr2=0x7fa5700afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.005+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 shutdown_connections 2024-04-24T23:24:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.005+0000 7fa577129640 1 -- 172.21.15.26:0/1055392552 wait complete. 2024-04-24T23:24:19.017 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-04-24T23:24:17.358870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260264,"kb_used_data":6616,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754714456,"statfs":{"total":773094113280,"available":772827602944,"internally_reserved":0,"allocated":6774784,"data_stored":3504204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031288"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946097+0000","last_change":"2024-04-24T23:24:15.027361+0000","last_active":"2024-04-24T23:24:16.946097+0000","last_peered":"2024-04-24T23:24:16.946097+0000","last_clean":"2024-04-24T23:24:16.946097+0000","last_became_active":"2024-04-24T23:24:15.027085+0000","last_became_peered":"2024-04-24T23:24:15.027085+0000","last_unstale":"2024-04-24T23:24:16.946097+0000","last_undegraded":"2024-04-24T23:24:16.946097+0000","last_fullsized":"2024-04-24T23:24:16.946097+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:25.631396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013154+0000","last_change":"2024-04-24T23:24:15.025229+0000","last_active":"2024-04-24T23:24:16.013154+0000","last_peered":"2024-04-24T23:24:16.013154+0000","last_clean":"2024-04-24T23:24:16.013154+0000","last_became_active":"2024-04-24T23:24:15.024965+0000","last_became_peered":"2024-04-24T23:24:15.024965+0000","last_unstale":"2024-04-24T23:24:16.013154+0000","last_undegraded":"2024-04-24T23:24:16.013154+0000","last_fullsized":"2024-04-24T23:24:16.013154+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:22.388242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947006+0000","last_change":"2024-04-24T23:24:15.031938+0000","last_active":"2024-04-24T23:24:16.947006+0000","last_peered":"2024-04-24T23:24:16.947006+0000","last_clean":"2024-04-24T23:24:16.947006+0000","last_became_active":"2024-04-24T23:24:15.031640+0000","last_became_peered":"2024-04-24T23:24:15.031640+0000","last_unstale":"2024-04-24T23:24:16.947006+0000","last_undegraded":"2024-04-24T23:24:16.947006+0000","last_fullsized":"2024-04-24T23:24:16.947006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:05.395174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947924+0000","last_change":"2024-04-24T23:24:15.016297+0000","last_active":"2024-04-24T23:24:16.947924+0000","last_peered":"2024-04-24T23:24:16.947924+0000","last_clean":"2024-04-24T23:24:16.947924+0000","last_became_active":"2024-04-24T23:24:15.016007+0000","last_became_peered":"2024-04-24T23:24:15.016007+0000","last_unstale":"2024-04-24T23:24:16.947924+0000","last_undegraded":"2024-04-24T23:24:16.947924+0000","last_fullsized":"2024-04-24T23:24:16.947924+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:00.047511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946006+0000","last_change":"2024-04-24T23:24:15.014827+0000","last_active":"2024-04-24T23:24:16.946006+0000","last_peered":"2024-04-24T23:24:16.946006+0000","last_clean":"2024-04-24T23:24:16.946006+0000","last_became_active":"2024-04-24T23:24:15.014544+0000","last_became_peered":"2024-04-24T23:24:15.014544+0000","last_unstale":"2024-04-24T23:24:16.946006+0000","last_undegraded":"2024-04-24T23:24:16.946006+0000","last_fullsized":"2024-04-24T23:24:16.946006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:40.327365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340369+0000","last_change":"2024-04-24T23:24:15.021059+0000","last_active":"2024-04-24T23:24:16.340369+0000","last_peered":"2024-04-24T23:24:16.340369+0000","last_clean":"2024-04-24T23:24:16.340369+0000","last_became_active":"2024-04-24T23:24:15.020782+0000","last_became_peered":"2024-04-24T23:24:15.020782+0000","last_unstale":"2024-04-24T23:24:16.340369+0000","last_undegraded":"2024-04-24T23:24:16.340369+0000","last_fullsized":"2024-04-24T23:24:16.340369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:38.948032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947446+0000","last_change":"2024-04-24T23:24:15.020545+0000","last_active":"2024-04-24T23:24:16.947446+0000","last_peered":"2024-04-24T23:24:16.947446+0000","last_clean":"2024-04-24T23:24:16.947446+0000","last_became_active":"2024-04-24T23:24:15.020260+0000","last_became_peered":"2024-04-24T23:24:15.020260+0000","last_unstale":"2024-04-24T23:24:16.947446+0000","last_undegraded":"2024-04-24T23:24:16.947446+0000","last_fullsized":"2024-04-24T23:24:16.947446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:30.640853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.012965+0000","last_change":"2024-04-24T23:24:15.020232+0000","last_active":"2024-04-24T23:24:16.012965+0000","last_peered":"2024-04-24T23:24:16.012965+0000","last_clean":"2024-04-24T23:24:16.012965+0000","last_became_active":"2024-04-24T23:24:15.019978+0000","last_became_peered":"2024-04-24T23:24:15.019978+0000","last_unstale":"2024-04-24T23:24:16.012965+0000","last_undegraded":"2024-04-24T23:24:16.012965+0000","last_fullsized":"2024-04-24T23:24:16.012965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:17.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946800+0000","last_change":"2024-04-24T23:24:15.023502+0000","last_active":"2024-04-24T23:24:16.946800+0000","last_peered":"2024-04-24T23:24:16.946800+0000","last_clean":"2024-04-24T23:24:16.946800+0000","last_became_active":"2024-04-24T23:24:15.023129+0000","last_became_peered":"2024-04-24T23:24:15.023129+0000","last_unstale":"2024-04-24T23:24:16.946800+0000","last_undegraded":"2024-04-24T23:24:16.946800+0000","last_fullsized":"2024-04-24T23:24:16.946800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:54.068906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742269+0000","last_change":"2024-04-24T23:24:15.020536+0000","last_active":"2024-04-24T23:24:16.742269+0000","last_peered":"2024-04-24T23:24:16.742269+0000","last_clean":"2024-04-24T23:24:16.742269+0000","last_became_active":"2024-04-24T23:24:15.020188+0000","last_became_peered":"2024-04-24T23:24:15.020188+0000","last_unstale":"2024-04-24T23:24:16.742269+0000","last_undegraded":"2024-04-24T23:24:16.742269+0000","last_fullsized":"2024-04-24T23:24:16.742269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:52.883280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742318+0000","last_change":"2024-04-24T23:24:15.026685+0000","last_active":"2024-04-24T23:24:16.742318+0000","last_peered":"2024-04-24T23:24:16.742318+0000","last_clean":"2024-04-24T23:24:16.742318+0000","last_became_active":"2024-04-24T23:24:15.026305+0000","last_became_peered":"2024-04-24T23:24:15.026305+0000","last_unstale":"2024-04-24T23:24:16.742318+0000","last_undegraded":"2024-04-24T23:24:16.742318+0000","last_fullsized":"2024-04-24T23:24:16.742318+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:46.459683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948014+0000","last_change":"2024-04-24T23:24:15.022404+0000","last_active":"2024-04-24T23:24:16.948014+0000","last_peered":"2024-04-24T23:24:16.948014+0000","last_clean":"2024-04-24T23:24:16.948014+0000","last_became_active":"2024-04-24T23:24:15.021971+0000","last_became_peered":"2024-04-24T23:24:15.021971+0000","last_unstale":"2024-04-24T23:24:16.948014+0000","last_undegraded":"2024-04-24T23:24:16.948014+0000","last_fullsized":"2024-04-24T23:24:16.948014+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:48.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014401+0000","last_change":"2024-04-24T23:24:15.022380+0000","last_active":"2024-04-24T23:24:16.014401+0000","last_peered":"2024-04-24T23:24:16.014401+0000","last_clean":"2024-04-24T23:24:16.014401+0000","last_became_active":"2024-04-24T23:24:15.021893+0000","last_became_peered":"2024-04-24T23:24:15.021893+0000","last_unstale":"2024-04-24T23:24:16.014401+0000","last_undegraded":"2024-04-24T23:24:16.014401+0000","last_fullsized":"2024-04-24T23:24:16.014401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:41.911420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947036+0000","last_change":"2024-04-24T23:24:15.016624+0000","last_active":"2024-04-24T23:24:16.947036+0000","last_peered":"2024-04-24T23:24:16.947036+0000","last_clean":"2024-04-24T23:24:16.947036+0000","last_became_active":"2024-04-24T23:24:15.016288+0000","last_became_peered":"2024-04-24T23:24:15.016288+0000","last_unstale":"2024-04-24T23:24:16.947036+0000","last_undegraded":"2024-04-24T23:24:16.947036+0000","last_fullsized":"2024-04-24T23:24:16.947036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:59.371935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947326+0000","last_change":"2024-04-24T23:24:15.019696+0000","last_active":"2024-04-24T23:24:16.947326+0000","last_peered":"2024-04-24T23:24:16.947326+0000","last_clean":"2024-04-24T23:24:16.947326+0000","last_became_active":"2024-04-24T23:24:15.019340+0000","last_became_peered":"2024-04-24T23:24:15.019340+0000","last_unstale":"2024-04-24T23:24:16.947326+0000","last_undegraded":"2024-04-24T23:24:16.947326+0000","last_fullsized":"2024-04-24T23:24:16.947326+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:51.461001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340295+0000","last_change":"2024-04-24T23:24:15.028629+0000","last_active":"2024-04-24T23:24:16.340295+0000","last_peered":"2024-04-24T23:24:16.340295+0000","last_clean":"2024-04-24T23:24:16.340295+0000","last_became_active":"2024-04-24T23:24:15.028379+0000","last_became_peered":"2024-04-24T23:24:15.028379+0000","last_unstale":"2024-04-24T23:24:16.340295+0000","last_undegraded":"2024-04-24T23:24:16.340295+0000","last_fullsized":"2024-04-24T23:24:16.340295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:37.044137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013570+0000","last_change":"2024-04-24T23:24:15.024054+0000","last_active":"2024-04-24T23:24:16.013570+0000","last_peered":"2024-04-24T23:24:16.013570+0000","last_clean":"2024-04-24T23:24:16.013570+0000","last_became_active":"2024-04-24T23:24:15.023695+0000","last_became_peered":"2024-04-24T23:24:15.023695+0000","last_unstale":"2024-04-24T23:24:16.013570+0000","last_undegraded":"2024-04-24T23:24:16.013570+0000","last_fullsized":"2024-04-24T23:24:16.013570+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:46.265731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947213+0000","last_change":"2024-04-24T23:24:15.031020+0000","last_active":"2024-04-24T23:24:16.947213+0000","last_peered":"2024-04-24T23:24:16.947213+0000","last_clean":"2024-04-24T23:24:16.947213+0000","last_became_active":"2024-04-24T23:24:15.030712+0000","last_became_peered":"2024-04-24T23:24:15.030712+0000","last_unstale":"2024-04-24T23:24:16.947213+0000","last_undegraded":"2024-04-24T23:24:16.947213+0000","last_fullsized":"2024-04-24T23:24:16.947213+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:00.047511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946148+0000","last_change":"2024-04-24T23:24:15.026223+0000","last_active":"2024-04-24T23:24:16.946148+0000","last_peered":"2024-04-24T23:24:16.946148+0000","last_clean":"2024-04-24T23:24:16.946148+0000","last_became_active":"2024-04-24T23:24:15.025958+0000","last_became_peered":"2024-04-24T23:24:15.025958+0000","last_unstale":"2024-04-24T23:24:16.946148+0000","last_undegraded":"2024-04-24T23:24:16.946148+0000","last_fullsized":"2024-04-24T23:24:16.946148+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:53.718925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013197+0000","last_change":"2024-04-24T23:24:15.022765+0000","last_active":"2024-04-24T23:24:16.013197+0000","last_peered":"2024-04-24T23:24:16.013197+0000","last_clean":"2024-04-24T23:24:16.013197+0000","last_became_active":"2024-04-24T23:24:15.022500+0000","last_became_peered":"2024-04-24T23:24:15.022500+0000","last_unstale":"2024-04-24T23:24:16.013197+0000","last_undegraded":"2024-04-24T23:24:16.013197+0000","last_fullsized":"2024-04-24T23:24:16.013197+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:26.158823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946799+0000","last_change":"2024-04-24T23:24:15.030749+0000","last_active":"2024-04-24T23:24:16.946799+0000","last_peered":"2024-04-24T23:24:16.946799+0000","last_clean":"2024-04-24T23:24:16.946799+0000","last_became_active":"2024-04-24T23:24:15.030449+0000","last_became_peered":"2024-04-24T23:24:15.030449+0000","last_unstale":"2024-04-24T23:24:16.946799+0000","last_undegraded":"2024-04-24T23:24:16.946799+0000","last_fullsized":"2024-04-24T23:24:16.946799+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:15.488825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013079+0000","last_change":"2024-04-24T23:24:15.023839+0000","last_active":"2024-04-24T23:24:16.013079+0000","last_peered":"2024-04-24T23:24:16.013079+0000","last_clean":"2024-04-24T23:24:16.013079+0000","last_became_active":"2024-04-24T23:24:15.022784+0000","last_became_peered":"2024-04-24T23:24:15.022784+0000","last_unstale":"2024-04-24T23:24:16.013079+0000","last_undegraded":"2024-04-24T23:24:16.013079+0000","last_fullsized":"2024-04-24T23:24:16.013079+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:19.460584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947819+0000","last_change":"2024-04-24T23:24:15.028126+0000","last_active":"2024-04-24T23:24:16.947819+0000","last_peered":"2024-04-24T23:24:16.947819+0000","last_clean":"2024-04-24T23:24:16.947819+0000","last_became_active":"2024-04-24T23:24:15.027807+0000","last_became_peered":"2024-04-24T23:24:15.027807+0000","last_unstale":"2024-04-24T23:24:16.947819+0000","last_undegraded":"2024-04-24T23:24:16.947819+0000","last_fullsized":"2024-04-24T23:24:16.947819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:37.893931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014182+0000","last_change":"2024-04-24T23:24:15.022857+0000","last_active":"2024-04-24T23:24:16.014182+0000","last_peered":"2024-04-24T23:24:16.014182+0000","last_clean":"2024-04-24T23:24:16.014182+0000","last_became_active":"2024-04-24T23:24:15.021730+0000","last_became_peered":"2024-04-24T23:24:15.021730+0000","last_unstale":"2024-04-24T23:24:16.014182+0000","last_undegraded":"2024-04-24T23:24:16.014182+0000","last_fullsized":"2024-04-24T23:24:16.014182+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:58.886412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946185+0000","last_change":"2024-04-24T23:24:15.019092+0000","last_active":"2024-04-24T23:24:16.946185+0000","last_peered":"2024-04-24T23:24:16.946185+0000","last_clean":"2024-04-24T23:24:16.946185+0000","last_became_active":"2024-04-24T23:24:15.018795+0000","last_became_peered":"2024-04-24T23:24:15.018795+0000","last_unstale":"2024-04-24T23:24:16.946185+0000","last_undegraded":"2024-04-24T23:24:16.946185+0000","last_fullsized":"2024-04-24T23:24:16.946185+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:16.562294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946909+0000","last_change":"2024-04-24T23:24:15.019719+0000","last_active":"2024-04-24T23:24:16.946909+0000","last_peered":"2024-04-24T23:24:16.946909+0000","last_clean":"2024-04-24T23:24:16.946909+0000","last_became_active":"2024-04-24T23:24:15.018541+0000","last_became_peered":"2024-04-24T23:24:15.018541+0000","last_unstale":"2024-04-24T23:24:16.946909+0000","last_undegraded":"2024-04-24T23:24:16.946909+0000","last_fullsized":"2024-04-24T23:24:16.946909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:28.322115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340518+0000","last_change":"2024-04-24T23:24:15.028983+0000","last_active":"2024-04-24T23:24:16.340518+0000","last_peered":"2024-04-24T23:24:16.340518+0000","last_clean":"2024-04-24T23:24:16.340518+0000","last_became_active":"2024-04-24T23:24:15.028701+0000","last_became_peered":"2024-04-24T23:24:15.028701+0000","last_unstale":"2024-04-24T23:24:16.340518+0000","last_undegraded":"2024-04-24T23:24:16.340518+0000","last_fullsized":"2024-04-24T23:24:16.340518+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:05.395174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946155+0000","last_change":"2024-04-24T23:24:15.018572+0000","last_active":"2024-04-24T23:24:16.946155+0000","last_peered":"2024-04-24T23:24:16.946155+0000","last_clean":"2024-04-24T23:24:16.946155+0000","last_became_active":"2024-04-24T23:24:15.017843+0000","last_became_peered":"2024-04-24T23:24:15.017843+0000","last_unstale":"2024-04-24T23:24:16.946155+0000","last_undegraded":"2024-04-24T23:24:16.946155+0000","last_fullsized":"2024-04-24T23:24:16.946155+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:34.764580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013640+0000","last_change":"2024-04-24T23:24:15.025709+0000","last_active":"2024-04-24T23:24:16.013640+0000","last_peered":"2024-04-24T23:24:16.013640+0000","last_clean":"2024-04-24T23:24:16.013640+0000","last_became_active":"2024-04-24T23:24:15.025419+0000","last_became_peered":"2024-04-24T23:24:15.025419+0000","last_unstale":"2024-04-24T23:24:16.013640+0000","last_undegraded":"2024-04-24T23:24:16.013640+0000","last_fullsized":"2024-04-24T23:24:16.013640+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:34.926701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946753+0000","last_change":"2024-04-24T23:24:15.031653+0000","last_active":"2024-04-24T23:24:16.946753+0000","last_peered":"2024-04-24T23:24:16.946753+0000","last_clean":"2024-04-24T23:24:16.946753+0000","last_became_active":"2024-04-24T23:24:15.031339+0000","last_became_peered":"2024-04-24T23:24:15.031339+0000","last_unstale":"2024-04-24T23:24:16.946753+0000","last_undegraded":"2024-04-24T23:24:16.946753+0000","last_fullsized":"2024-04-24T23:24:16.946753+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:38.948032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.945914+0000","last_change":"2024-04-24T23:24:15.018383+0000","last_active":"2024-04-24T23:24:16.945914+0000","last_peered":"2024-04-24T23:24:16.945914+0000","last_clean":"2024-04-24T23:24:16.945914+0000","last_became_active":"2024-04-24T23:24:15.018092+0000","last_became_peered":"2024-04-24T23:24:15.018092+0000","last_unstale":"2024-04-24T23:24:16.945914+0000","last_undegraded":"2024-04-24T23:24:16.945914+0000","last_fullsized":"2024-04-24T23:24:16.945914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:34.472583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340538+0000","last_change":"2024-04-24T23:24:15.023226+0000","last_active":"2024-04-24T23:24:16.340538+0000","last_peered":"2024-04-24T23:24:16.340538+0000","last_clean":"2024-04-24T23:24:16.340538+0000","last_became_active":"2024-04-24T23:24:15.022917+0000","last_became_peered":"2024-04-24T23:24:15.022917+0000","last_unstale":"2024-04-24T23:24:16.340538+0000","last_undegraded":"2024-04-24T23:24:16.340538+0000","last_fullsized":"2024-04-24T23:24:16.340538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:00.047511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340728+0000","last_change":"2024-04-24T23:24:15.020912+0000","last_active":"2024-04-24T23:24:16.340728+0000","last_peered":"2024-04-24T23:24:16.340728+0000","last_clean":"2024-04-24T23:24:16.340728+0000","last_became_active":"2024-04-24T23:24:15.020292+0000","last_became_peered":"2024-04-24T23:24:15.020292+0000","last_unstale":"2024-04-24T23:24:16.340728+0000","last_undegraded":"2024-04-24T23:24:16.340728+0000","last_fullsized":"2024-04-24T23:24:16.340728+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:59.338397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340743+0000","last_change":"2024-04-24T23:24:14.346692+0000","last_active":"2024-04-24T23:24:16.340743+0000","last_peered":"2024-04-24T23:24:16.340743+0000","last_clean":"2024-04-24T23:24:16.340743+0000","last_became_active":"2024-04-24T23:24:14.346432+0000","last_became_peered":"2024-04-24T23:24:14.346432+0000","last_unstale":"2024-04-24T23:24:16.340743+0000","last_undegraded":"2024-04-24T23:24:16.340743+0000","last_fullsized":"2024-04-24T23:24:16.340743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:45.457469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742333+0000","last_change":"2024-04-24T23:24:15.021139+0000","last_active":"2024-04-24T23:24:16.742333+0000","last_peered":"2024-04-24T23:24:16.742333+0000","last_clean":"2024-04-24T23:24:16.742333+0000","last_became_active":"2024-04-24T23:24:15.020885+0000","last_became_peered":"2024-04-24T23:24:15.020885+0000","last_unstale":"2024-04-24T23:24:16.742333+0000","last_undegraded":"2024-04-24T23:24:16.742333+0000","last_fullsized":"2024-04-24T23:24:16.742333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:37.396907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013267+0000","last_change":"2024-04-24T23:24:14.353647+0000","last_active":"2024-04-24T23:24:16.013267+0000","last_peered":"2024-04-24T23:24:16.013267+0000","last_clean":"2024-04-24T23:24:16.013267+0000","last_became_active":"2024-04-24T23:24:14.353378+0000","last_became_peered":"2024-04-24T23:24:14.353378+0000","last_unstale":"2024-04-24T23:24:16.013267+0000","last_undegraded":"2024-04-24T23:24:16.013267+0000","last_fullsized":"2024-04-24T23:24:16.013267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:59.045297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340517+0000","last_change":"2024-04-24T23:24:15.026093+0000","last_active":"2024-04-24T23:24:16.340517+0000","last_peered":"2024-04-24T23:24:16.340517+0000","last_clean":"2024-04-24T23:24:16.340517+0000","last_became_active":"2024-04-24T23:24:15.025804+0000","last_became_peered":"2024-04-24T23:24:15.025804+0000","last_unstale":"2024-04-24T23:24:16.340517+0000","last_undegraded":"2024-04-24T23:24:16.340517+0000","last_fullsized":"2024-04-24T23:24:16.340517+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:54.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340682+0000","last_change":"2024-04-24T23:24:14.345945+0000","last_active":"2024-04-24T23:24:16.340682+0000","last_peered":"2024-04-24T23:24:16.340682+0000","last_clean":"2024-04-24T23:24:16.340682+0000","last_became_active":"2024-04-24T23:24:14.345628+0000","last_became_peered":"2024-04-24T23:24:14.345628+0000","last_unstale":"2024-04-24T23:24:16.340682+0000","last_undegraded":"2024-04-24T23:24:16.340682+0000","last_fullsized":"2024-04-24T23:24:16.340682+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:53.066692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340593+0000","last_change":"2024-04-24T23:24:15.028874+0000","last_active":"2024-04-24T23:24:16.340593+0000","last_peered":"2024-04-24T23:24:16.340593+0000","last_clean":"2024-04-24T23:24:16.340593+0000","last_became_active":"2024-04-24T23:24:15.028372+0000","last_became_peered":"2024-04-24T23:24:15.028372+0000","last_unstale":"2024-04-24T23:24:16.340593+0000","last_undegraded":"2024-04-24T23:24:16.340593+0000","last_fullsized":"2024-04-24T23:24:16.340593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:32.978848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948057+0000","last_change":"2024-04-24T23:24:14.348475+0000","last_active":"2024-04-24T23:24:16.948057+0000","last_peered":"2024-04-24T23:24:16.948057+0000","last_clean":"2024-04-24T23:24:16.948057+0000","last_became_active":"2024-04-24T23:24:14.348135+0000","last_became_peered":"2024-04-24T23:24:14.348135+0000","last_unstale":"2024-04-24T23:24:16.948057+0000","last_undegraded":"2024-04-24T23:24:16.948057+0000","last_fullsized":"2024-04-24T23:24:16.948057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:38.017768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013393+0000","last_change":"2024-04-24T23:24:15.023284+0000","last_active":"2024-04-24T23:24:16.013393+0000","last_peered":"2024-04-24T23:24:16.013393+0000","last_clean":"2024-04-24T23:24:16.013393+0000","last_became_active":"2024-04-24T23:24:15.023018+0000","last_became_peered":"2024-04-24T23:24:15.023018+0000","last_unstale":"2024-04-24T23:24:16.013393+0000","last_undegraded":"2024-04-24T23:24:16.013393+0000","last_fullsized":"2024-04-24T23:24:16.013393+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:54.414987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947779+0000","last_change":"2024-04-24T23:24:14.348570+0000","last_active":"2024-04-24T23:24:16.947779+0000","last_peered":"2024-04-24T23:24:16.947779+0000","last_clean":"2024-04-24T23:24:16.947779+0000","last_became_active":"2024-04-24T23:24:14.348405+0000","last_became_peered":"2024-04-24T23:24:14.348405+0000","last_unstale":"2024-04-24T23:24:16.947779+0000","last_undegraded":"2024-04-24T23:24:16.947779+0000","last_fullsized":"2024-04-24T23:24:16.947779+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:06.580007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013349+0000","last_change":"2024-04-24T23:24:15.025333+0000","last_active":"2024-04-24T23:24:16.013349+0000","last_peered":"2024-04-24T23:24:16.013349+0000","last_clean":"2024-04-24T23:24:16.013349+0000","last_became_active":"2024-04-24T23:24:15.025041+0000","last_became_peered":"2024-04-24T23:24:15.025041+0000","last_unstale":"2024-04-24T23:24:16.013349+0000","last_undegraded":"2024-04-24T23:24:16.013349+0000","last_fullsized":"2024-04-24T23:24:16.013349+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:40.153193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340558+0000","last_change":"2024-04-24T23:24:14.344502+0000","last_active":"2024-04-24T23:24:16.340558+0000","last_peered":"2024-04-24T23:24:16.340558+0000","last_clean":"2024-04-24T23:24:16.340558+0000","last_became_active":"2024-04-24T23:24:14.342934+0000","last_became_peered":"2024-04-24T23:24:14.342934+0000","last_unstale":"2024-04-24T23:24:16.340558+0000","last_undegraded":"2024-04-24T23:24:16.340558+0000","last_fullsized":"2024-04-24T23:24:16.340558+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:07.174115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014148+0000","last_change":"2024-04-24T23:24:15.024649+0000","last_active":"2024-04-24T23:24:16.014148+0000","last_peered":"2024-04-24T23:24:16.014148+0000","last_clean":"2024-04-24T23:24:16.014148+0000","last_became_active":"2024-04-24T23:24:15.024348+0000","last_became_peered":"2024-04-24T23:24:15.024348+0000","last_unstale":"2024-04-24T23:24:16.014148+0000","last_undegraded":"2024-04-24T23:24:16.014148+0000","last_fullsized":"2024-04-24T23:24:16.014148+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:30.219114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946609+0000","last_change":"2024-04-24T23:24:14.342658+0000","last_active":"2024-04-24T23:24:16.946609+0000","last_peered":"2024-04-24T23:24:16.946609+0000","last_clean":"2024-04-24T23:24:16.946609+0000","last_became_active":"2024-04-24T23:24:14.342244+0000","last_became_peered":"2024-04-24T23:24:14.342244+0000","last_unstale":"2024-04-24T23:24:16.946609+0000","last_undegraded":"2024-04-24T23:24:16.946609+0000","last_fullsized":"2024-04-24T23:24:16.946609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:53.066692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013626+0000","last_change":"2024-04-24T23:24:15.021387+0000","last_active":"2024-04-24T23:24:16.013626+0000","last_peered":"2024-04-24T23:24:16.013626+0000","last_clean":"2024-04-24T23:24:16.013626+0000","last_became_active":"2024-04-24T23:24:15.021103+0000","last_became_peered":"2024-04-24T23:24:15.021103+0000","last_unstale":"2024-04-24T23:24:16.013626+0000","last_undegraded":"2024-04-24T23:24:16.013626+0000","last_fullsized":"2024-04-24T23:24:16.013626+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:31.220641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948069+0000","last_change":"2024-04-24T23:24:14.351852+0000","last_active":"2024-04-24T23:24:16.948069+0000","last_peered":"2024-04-24T23:24:16.948069+0000","last_clean":"2024-04-24T23:24:16.948069+0000","last_became_active":"2024-04-24T23:24:14.351524+0000","last_became_peered":"2024-04-24T23:24:14.351524+0000","last_unstale":"2024-04-24T23:24:16.948069+0000","last_undegraded":"2024-04-24T23:24:16.948069+0000","last_fullsized":"2024-04-24T23:24:16.948069+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:40.909206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947540+0000","last_change":"2024-04-24T23:24:15.019788+0000","last_active":"2024-04-24T23:24:16.947540+0000","last_peered":"2024-04-24T23:24:16.947540+0000","last_clean":"2024-04-24T23:24:16.947540+0000","last_became_active":"2024-04-24T23:24:15.019494+0000","last_became_peered":"2024-04-24T23:24:15.019494+0000","last_unstale":"2024-04-24T23:24:16.947540+0000","last_undegraded":"2024-04-24T23:24:16.947540+0000","last_fullsized":"2024-04-24T23:24:16.947540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:39.019982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947478+0000","last_change":"2024-04-24T23:24:14.349239+0000","last_active":"2024-04-24T23:24:16.947478+0000","last_peered":"2024-04-24T23:24:16.947478+0000","last_clean":"2024-04-24T23:24:16.947478+0000","last_became_active":"2024-04-24T23:24:14.348950+0000","last_became_peered":"2024-04-24T23:24:14.348950+0000","last_unstale":"2024-04-24T23:24:16.947478+0000","last_undegraded":"2024-04-24T23:24:16.947478+0000","last_fullsized":"2024-04-24T23:24:16.947478+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:50.458787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947357+0000","last_change":"2024-04-24T23:24:15.033131+0000","last_active":"2024-04-24T23:24:16.947357+0000","last_peered":"2024-04-24T23:24:16.947357+0000","last_clean":"2024-04-24T23:24:16.947357+0000","last_became_active":"2024-04-24T23:24:15.032669+0000","last_became_peered":"2024-04-24T23:24:15.032669+0000","last_unstale":"2024-04-24T23:24:16.947357+0000","last_undegraded":"2024-04-24T23:24:16.947357+0000","last_fullsized":"2024-04-24T23:24:16.947357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:59.338397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013270+0000","last_change":"2024-04-24T23:24:14.351685+0000","last_active":"2024-04-24T23:24:16.013270+0000","last_peered":"2024-04-24T23:24:16.013270+0000","last_clean":"2024-04-24T23:24:16.013270+0000","last_became_active":"2024-04-24T23:24:14.351284+0000","last_became_peered":"2024-04-24T23:24:14.351284+0000","last_unstale":"2024-04-24T23:24:16.013270+0000","last_undegraded":"2024-04-24T23:24:16.013270+0000","last_fullsized":"2024-04-24T23:24:16.013270+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:47.091701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742191+0000","last_change":"2024-04-24T23:24:15.020567+0000","last_active":"2024-04-24T23:24:16.742191+0000","last_peered":"2024-04-24T23:24:16.742191+0000","last_clean":"2024-04-24T23:24:16.742191+0000","last_became_active":"2024-04-24T23:24:15.019774+0000","last_became_peered":"2024-04-24T23:24:15.019774+0000","last_unstale":"2024-04-24T23:24:16.742191+0000","last_undegraded":"2024-04-24T23:24:16.742191+0000","last_fullsized":"2024-04-24T23:24:16.742191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:29.985321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947567+0000","last_change":"2024-04-24T23:24:14.354115+0000","last_active":"2024-04-24T23:24:16.947567+0000","last_peered":"2024-04-24T23:24:16.947567+0000","last_clean":"2024-04-24T23:24:16.947567+0000","last_became_active":"2024-04-24T23:24:14.353809+0000","last_became_peered":"2024-04-24T23:24:14.353809+0000","last_unstale":"2024-04-24T23:24:16.947567+0000","last_undegraded":"2024-04-24T23:24:16.947567+0000","last_fullsized":"2024-04-24T23:24:16.947567+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:58.369720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947223+0000","last_change":"2024-04-24T23:24:15.030348+0000","last_active":"2024-04-24T23:24:16.947223+0000","last_peered":"2024-04-24T23:24:16.947223+0000","last_clean":"2024-04-24T23:24:16.947223+0000","last_became_active":"2024-04-24T23:24:15.030020+0000","last_became_peered":"2024-04-24T23:24:15.030020+0000","last_unstale":"2024-04-24T23:24:16.947223+0000","last_undegraded":"2024-04-24T23:24:16.947223+0000","last_fullsized":"2024-04-24T23:24:16.947223+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:48.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948381+0000","last_change":"2024-04-24T23:24:14.344826+0000","last_active":"2024-04-24T23:24:16.948381+0000","last_peered":"2024-04-24T23:24:16.948381+0000","last_clean":"2024-04-24T23:24:16.948381+0000","last_became_active":"2024-04-24T23:24:14.344520+0000","last_became_peered":"2024-04-24T23:24:14.344520+0000","last_unstale":"2024-04-24T23:24:16.948381+0000","last_undegraded":"2024-04-24T23:24:16.948381+0000","last_fullsized":"2024-04-24T23:24:16.948381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:40.975800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947410+0000","last_change":"2024-04-24T23:24:15.020602+0000","last_active":"2024-04-24T23:24:16.947410+0000","last_peered":"2024-04-24T23:24:16.947410+0000","last_clean":"2024-04-24T23:24:16.947410+0000","last_became_active":"2024-04-24T23:24:15.020222+0000","last_became_peered":"2024-04-24T23:24:15.020222+0000","last_unstale":"2024-04-24T23:24:16.947410+0000","last_undegraded":"2024-04-24T23:24:16.947410+0000","last_fullsized":"2024-04-24T23:24:16.947410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:15.488825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013462+0000","last_change":"2024-04-24T23:24:14.354341+0000","last_active":"2024-04-24T23:24:16.013462+0000","last_peered":"2024-04-24T23:24:16.013462+0000","last_clean":"2024-04-24T23:24:16.013462+0000","last_became_active":"2024-04-24T23:24:14.354041+0000","last_became_peered":"2024-04-24T23:24:14.354041+0000","last_unstale":"2024-04-24T23:24:16.013462+0000","last_undegraded":"2024-04-24T23:24:16.013462+0000","last_fullsized":"2024-04-24T23:24:16.013462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:58.336183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947543+0000","last_change":"2024-04-24T23:24:15.029748+0000","last_active":"2024-04-24T23:24:16.947543+0000","last_peered":"2024-04-24T23:24:16.947543+0000","last_clean":"2024-04-24T23:24:16.947543+0000","last_became_active":"2024-04-24T23:24:15.029017+0000","last_became_peered":"2024-04-24T23:24:15.029017+0000","last_unstale":"2024-04-24T23:24:16.947543+0000","last_undegraded":"2024-04-24T23:24:16.947543+0000","last_fullsized":"2024-04-24T23:24:16.947543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:03.751530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014179+0000","last_change":"2024-04-24T23:24:14.351000+0000","last_active":"2024-04-24T23:24:16.014179+0000","last_peered":"2024-04-24T23:24:16.014179+0000","last_clean":"2024-04-24T23:24:16.014179+0000","last_became_active":"2024-04-24T23:24:14.350461+0000","last_became_peered":"2024-04-24T23:24:14.350461+0000","last_unstale":"2024-04-24T23:24:16.014179+0000","last_undegraded":"2024-04-24T23:24:16.014179+0000","last_fullsized":"2024-04-24T23:24:16.014179+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:08.694848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948086+0000","last_change":"2024-04-24T23:24:15.022598+0000","last_active":"2024-04-24T23:24:16.948086+0000","last_peered":"2024-04-24T23:24:16.948086+0000","last_clean":"2024-04-24T23:24:16.948086+0000","last_became_active":"2024-04-24T23:24:15.022128+0000","last_became_peered":"2024-04-24T23:24:15.022128+0000","last_unstale":"2024-04-24T23:24:16.948086+0000","last_undegraded":"2024-04-24T23:24:16.948086+0000","last_fullsized":"2024-04-24T23:24:16.948086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:19.516771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946722+0000","last_change":"2024-04-24T23:24:14.348951+0000","last_active":"2024-04-24T23:24:16.946722+0000","last_peered":"2024-04-24T23:24:16.946722+0000","last_clean":"2024-04-24T23:24:16.946722+0000","last_became_active":"2024-04-24T23:24:14.348576+0000","last_became_peered":"2024-04-24T23:24:14.348576+0000","last_unstale":"2024-04-24T23:24:16.946722+0000","last_undegraded":"2024-04-24T23:24:16.946722+0000","last_fullsized":"2024-04-24T23:24:16.946722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:50.458787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947831+0000","last_change":"2024-04-24T23:24:15.028932+0000","last_active":"2024-04-24T23:24:16.947831+0000","last_peered":"2024-04-24T23:24:16.947831+0000","last_clean":"2024-04-24T23:24:16.947831+0000","last_became_active":"2024-04-24T23:24:15.028553+0000","last_became_peered":"2024-04-24T23:24:15.028553+0000","last_unstale":"2024-04-24T23:24:16.947831+0000","last_undegraded":"2024-04-24T23:24:16.947831+0000","last_fullsized":"2024-04-24T23:24:16.947831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:21.065862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947489+0000","last_change":"2024-04-24T23:24:14.349655+0000","last_active":"2024-04-24T23:24:16.947489+0000","last_peered":"2024-04-24T23:24:16.947489+0000","last_clean":"2024-04-24T23:24:16.947489+0000","last_became_active":"2024-04-24T23:24:14.349352+0000","last_became_peered":"2024-04-24T23:24:14.349352+0000","last_unstale":"2024-04-24T23:24:16.947489+0000","last_undegraded":"2024-04-24T23:24:16.947489+0000","last_fullsized":"2024-04-24T23:24:16.947489+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:29.638639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947223+0000","last_change":"2024-04-24T23:24:15.031725+0000","last_active":"2024-04-24T23:24:16.947223+0000","last_peered":"2024-04-24T23:24:16.947223+0000","last_clean":"2024-04-24T23:24:16.947223+0000","last_became_active":"2024-04-24T23:24:15.031391+0000","last_became_peered":"2024-04-24T23:24:15.031391+0000","last_unstale":"2024-04-24T23:24:16.947223+0000","last_undegraded":"2024-04-24T23:24:16.947223+0000","last_fullsized":"2024-04-24T23:24:16.947223+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:32.978848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013629+0000","last_change":"2024-04-24T23:24:14.355432+0000","last_active":"2024-04-24T23:24:16.013629+0000","last_peered":"2024-04-24T23:24:16.013629+0000","last_clean":"2024-04-24T23:24:16.013629+0000","last_became_active":"2024-04-24T23:24:14.355133+0000","last_became_peered":"2024-04-24T23:24:14.355133+0000","last_unstale":"2024-04-24T23:24:16.013629+0000","last_undegraded":"2024-04-24T23:24:16.013629+0000","last_fullsized":"2024-04-24T23:24:16.013629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:31.976634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947986+0000","last_change":"2024-04-24T23:24:15.021549+0000","last_active":"2024-04-24T23:24:16.947986+0000","last_peered":"2024-04-24T23:24:16.947986+0000","last_clean":"2024-04-24T23:24:16.947986+0000","last_became_active":"2024-04-24T23:24:15.021040+0000","last_became_peered":"2024-04-24T23:24:15.021040+0000","last_unstale":"2024-04-24T23:24:16.947986+0000","last_undegraded":"2024-04-24T23:24:16.947986+0000","last_fullsized":"2024-04-24T23:24:16.947986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:32.978848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340695+0000","last_change":"2024-04-24T23:24:14.350265+0000","last_active":"2024-04-24T23:24:16.340695+0000","last_peered":"2024-04-24T23:24:16.340695+0000","last_clean":"2024-04-24T23:24:16.340695+0000","last_became_active":"2024-04-24T23:24:14.349999+0000","last_became_peered":"2024-04-24T23:24:14.349999+0000","last_unstale":"2024-04-24T23:24:16.340695+0000","last_undegraded":"2024-04-24T23:24:16.340695+0000","last_fullsized":"2024-04-24T23:24:16.340695+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:29.638639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013333+0000","last_change":"2024-04-24T23:24:15.022211+0000","last_active":"2024-04-24T23:24:16.013333+0000","last_peered":"2024-04-24T23:24:16.013333+0000","last_clean":"2024-04-24T23:24:16.013333+0000","last_became_active":"2024-04-24T23:24:15.021891+0000","last_became_peered":"2024-04-24T23:24:15.021891+0000","last_unstale":"2024-04-24T23:24:16.013333+0000","last_undegraded":"2024-04-24T23:24:16.013333+0000","last_fullsized":"2024-04-24T23:24:16.013333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:45.104652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947871+0000","last_change":"2024-04-24T23:24:14.351480+0000","last_active":"2024-04-24T23:24:16.947871+0000","last_peered":"2024-04-24T23:24:16.947871+0000","last_clean":"2024-04-24T23:24:16.947871+0000","last_became_active":"2024-04-24T23:24:14.351008+0000","last_became_peered":"2024-04-24T23:24:14.351008+0000","last_unstale":"2024-04-24T23:24:16.947871+0000","last_undegraded":"2024-04-24T23:24:16.947871+0000","last_fullsized":"2024-04-24T23:24:16.947871+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:29.638639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947449+0000","last_change":"2024-04-24T23:24:15.029615+0000","last_active":"2024-04-24T23:24:16.947449+0000","last_peered":"2024-04-24T23:24:16.947449+0000","last_clean":"2024-04-24T23:24:16.947449+0000","last_became_active":"2024-04-24T23:24:15.029322+0000","last_became_peered":"2024-04-24T23:24:15.029322+0000","last_unstale":"2024-04-24T23:24:16.947449+0000","last_undegraded":"2024-04-24T23:24:16.947449+0000","last_fullsized":"2024-04-24T23:24:16.947449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:00.629136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340603+0000","last_change":"2024-04-24T23:24:14.353896+0000","last_active":"2024-04-24T23:24:16.340603+0000","last_peered":"2024-04-24T23:24:16.340603+0000","last_clean":"2024-04-24T23:24:16.340603+0000","last_became_active":"2024-04-24T23:24:14.353507+0000","last_became_peered":"2024-04-24T23:24:14.353507+0000","last_unstale":"2024-04-24T23:24:16.340603+0000","last_undegraded":"2024-04-24T23:24:16.340603+0000","last_fullsized":"2024-04-24T23:24:16.340603+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:36.394693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947585+0000","last_change":"2024-04-24T23:24:15.017614+0000","last_active":"2024-04-24T23:24:16.947585+0000","last_peered":"2024-04-24T23:24:16.947585+0000","last_clean":"2024-04-24T23:24:16.947585+0000","last_became_active":"2024-04-24T23:24:15.017246+0000","last_became_peered":"2024-04-24T23:24:15.017246+0000","last_unstale":"2024-04-24T23:24:16.947585+0000","last_undegraded":"2024-04-24T23:24:16.947585+0000","last_fullsized":"2024-04-24T23:24:16.947585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:05.395174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013693+0000","last_change":"2024-04-24T23:24:14.351937+0000","last_active":"2024-04-24T23:24:16.013693+0000","last_peered":"2024-04-24T23:24:16.013693+0000","last_clean":"2024-04-24T23:24:16.013693+0000","last_became_active":"2024-04-24T23:24:14.351663+0000","last_became_peered":"2024-04-24T23:24:14.351663+0000","last_unstale":"2024-04-24T23:24:16.013693+0000","last_undegraded":"2024-04-24T23:24:16.013693+0000","last_fullsized":"2024-04-24T23:24:16.013693+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:59.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340506+0000","last_change":"2024-04-24T23:24:15.023108+0000","last_active":"2024-04-24T23:24:16.340506+0000","last_peered":"2024-04-24T23:24:16.340506+0000","last_clean":"2024-04-24T23:24:16.340506+0000","last_became_active":"2024-04-24T23:24:15.022770+0000","last_became_peered":"2024-04-24T23:24:15.022770+0000","last_unstale":"2024-04-24T23:24:16.340506+0000","last_undegraded":"2024-04-24T23:24:16.340506+0000","last_fullsized":"2024-04-24T23:24:16.340506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:15.488825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742094+0000","last_change":"2024-04-24T23:24:14.348980+0000","last_active":"2024-04-24T23:24:16.742094+0000","last_peered":"2024-04-24T23:24:16.742094+0000","last_clean":"2024-04-24T23:24:16.742094+0000","last_became_active":"2024-04-24T23:24:14.348640+0000","last_became_peered":"2024-04-24T23:24:14.348640+0000","last_unstale":"2024-04-24T23:24:16.742094+0000","last_undegraded":"2024-04-24T23:24:16.742094+0000","last_fullsized":"2024-04-24T23:24:16.742094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:03.381663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340333+0000","last_change":"2024-04-24T23:24:15.019877+0000","last_active":"2024-04-24T23:24:16.340333+0000","last_peered":"2024-04-24T23:24:16.340333+0000","last_clean":"2024-04-24T23:24:16.340333+0000","last_became_active":"2024-04-24T23:24:15.019621+0000","last_became_peered":"2024-04-24T23:24:15.019621+0000","last_unstale":"2024-04-24T23:24:16.340333+0000","last_undegraded":"2024-04-24T23:24:16.340333+0000","last_fullsized":"2024-04-24T23:24:16.340333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:28.322115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946532+0000","last_change":"2024-04-24T23:24:14.351171+0000","last_active":"2024-04-24T23:24:16.946532+0000","last_peered":"2024-04-24T23:24:16.946532+0000","last_clean":"2024-04-24T23:24:16.946532+0000","last_became_active":"2024-04-24T23:24:14.350864+0000","last_became_peered":"2024-04-24T23:24:14.350864+0000","last_unstale":"2024-04-24T23:24:16.946532+0000","last_undegraded":"2024-04-24T23:24:16.946532+0000","last_fullsized":"2024-04-24T23:24:16.946532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:58.369720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947083+0000","last_change":"2024-04-24T23:24:15.017232+0000","last_active":"2024-04-24T23:24:16.947083+0000","last_peered":"2024-04-24T23:24:16.947083+0000","last_clean":"2024-04-24T23:24:16.947083+0000","last_became_active":"2024-04-24T23:24:15.015671+0000","last_became_peered":"2024-04-24T23:24:15.015671+0000","last_unstale":"2024-04-24T23:24:16.947083+0000","last_undegraded":"2024-04-24T23:24:16.947083+0000","last_fullsized":"2024-04-24T23:24:16.947083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:38.948032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742349+0000","last_change":"2024-04-24T23:24:14.352609+0000","last_active":"2024-04-24T23:24:16.742349+0000","last_peered":"2024-04-24T23:24:16.742349+0000","last_clean":"2024-04-24T23:24:16.742349+0000","last_became_active":"2024-04-24T23:24:14.352333+0000","last_became_peered":"2024-04-24T23:24:14.352333+0000","last_unstale":"2024-04-24T23:24:16.742349+0000","last_undegraded":"2024-04-24T23:24:16.742349+0000","last_fullsized":"2024-04-24T23:24:16.742349+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:04.346009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947641+0000","last_change":"2024-04-24T23:24:15.022268+0000","last_active":"2024-04-24T23:24:16.947641+0000","last_peered":"2024-04-24T23:24:16.947641+0000","last_clean":"2024-04-24T23:24:16.947641+0000","last_became_active":"2024-04-24T23:24:15.021914+0000","last_became_peered":"2024-04-24T23:24:15.021914+0000","last_unstale":"2024-04-24T23:24:16.947641+0000","last_undegraded":"2024-04-24T23:24:16.947641+0000","last_fullsized":"2024-04-24T23:24:16.947641+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:54.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.342426+0000","last_change":"2024-04-24T23:24:14.344848+0000","last_active":"2024-04-24T23:24:16.342426+0000","last_peered":"2024-04-24T23:24:16.342426+0000","last_clean":"2024-04-24T23:24:16.342426+0000","last_became_active":"2024-04-24T23:24:14.342883+0000","last_became_peered":"2024-04-24T23:24:14.342883+0000","last_unstale":"2024-04-24T23:24:16.342426+0000","last_undegraded":"2024-04-24T23:24:16.342426+0000","last_fullsized":"2024-04-24T23:24:16.342426+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:58.369720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742321+0000","last_change":"2024-04-24T23:24:15.017512+0000","last_active":"2024-04-24T23:24:16.742321+0000","last_peered":"2024-04-24T23:24:16.742321+0000","last_clean":"2024-04-24T23:24:16.742321+0000","last_became_active":"2024-04-24T23:24:15.016928+0000","last_became_peered":"2024-04-24T23:24:15.016928+0000","last_unstale":"2024-04-24T23:24:16.742321+0000","last_undegraded":"2024-04-24T23:24:16.742321+0000","last_fullsized":"2024-04-24T23:24:16.742321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:34.764452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014232+0000","last_change":"2024-04-24T23:24:14.346908+0000","last_active":"2024-04-24T23:24:16.014232+0000","last_peered":"2024-04-24T23:24:16.014232+0000","last_clean":"2024-04-24T23:24:16.014232+0000","last_became_active":"2024-04-24T23:24:14.346613+0000","last_became_peered":"2024-04-24T23:24:14.346613+0000","last_unstale":"2024-04-24T23:24:16.014232+0000","last_undegraded":"2024-04-24T23:24:16.014232+0000","last_fullsized":"2024-04-24T23:24:16.014232+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:32:20.621642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340678+0000","last_change":"2024-04-24T23:24:15.028653+0000","last_active":"2024-04-24T23:24:16.340678+0000","last_peered":"2024-04-24T23:24:16.340678+0000","last_clean":"2024-04-24T23:24:16.340678+0000","last_became_active":"2024-04-24T23:24:15.028351+0000","last_became_peered":"2024-04-24T23:24:15.028351+0000","last_unstale":"2024-04-24T23:24:16.340678+0000","last_undegraded":"2024-04-24T23:24:16.340678+0000","last_fullsized":"2024-04-24T23:24:16.340678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:48.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948124+0000","last_change":"2024-04-24T23:24:14.342803+0000","last_active":"2024-04-24T23:24:16.948124+0000","last_peered":"2024-04-24T23:24:16.948124+0000","last_clean":"2024-04-24T23:24:16.948124+0000","last_became_active":"2024-04-24T23:24:14.342503+0000","last_became_peered":"2024-04-24T23:24:14.342503+0000","last_unstale":"2024-04-24T23:24:16.948124+0000","last_undegraded":"2024-04-24T23:24:16.948124+0000","last_fullsized":"2024-04-24T23:24:16.948124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:18.514557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013246+0000","last_change":"2024-04-24T23:24:15.025137+0000","last_active":"2024-04-24T23:24:16.013246+0000","last_peered":"2024-04-24T23:24:16.013246+0000","last_clean":"2024-04-24T23:24:16.013246+0000","last_became_active":"2024-04-24T23:24:15.024794+0000","last_became_peered":"2024-04-24T23:24:15.024794+0000","last_unstale":"2024-04-24T23:24:16.013246+0000","last_undegraded":"2024-04-24T23:24:16.013246+0000","last_fullsized":"2024-04-24T23:24:16.013246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:29.794681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.741990+0000","last_change":"2024-04-24T23:24:14.343856+0000","last_active":"2024-04-24T23:24:16.741990+0000","last_peered":"2024-04-24T23:24:16.741990+0000","last_clean":"2024-04-24T23:24:16.741990+0000","last_became_active":"2024-04-24T23:24:14.343447+0000","last_became_peered":"2024-04-24T23:24:14.343447+0000","last_unstale":"2024-04-24T23:24:16.741990+0000","last_undegraded":"2024-04-24T23:24:16.741990+0000","last_fullsized":"2024-04-24T23:24:16.741990+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:32:20.621642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947019+0000","last_change":"2024-04-24T23:24:15.028471+0000","last_active":"2024-04-24T23:24:16.947019+0000","last_peered":"2024-04-24T23:24:16.947019+0000","last_clean":"2024-04-24T23:24:16.947019+0000","last_became_active":"2024-04-24T23:24:15.027874+0000","last_became_peered":"2024-04-24T23:24:15.027874+0000","last_unstale":"2024-04-24T23:24:16.947019+0000","last_undegraded":"2024-04-24T23:24:16.947019+0000","last_fullsized":"2024-04-24T23:24:16.947019+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:54.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947139+0000","last_change":"2024-04-24T23:24:14.349759+0000","last_active":"2024-04-24T23:24:16.947139+0000","last_peered":"2024-04-24T23:24:16.947139+0000","last_clean":"2024-04-24T23:24:16.947139+0000","last_became_active":"2024-04-24T23:24:14.349471+0000","last_became_peered":"2024-04-24T23:24:14.349471+0000","last_unstale":"2024-04-24T23:24:16.947139+0000","last_undegraded":"2024-04-24T23:24:16.947139+0000","last_fullsized":"2024-04-24T23:24:16.947139+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:12.616378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.741994+0000","last_change":"2024-04-24T23:24:15.020528+0000","last_active":"2024-04-24T23:24:16.741994+0000","last_peered":"2024-04-24T23:24:16.741994+0000","last_clean":"2024-04-24T23:24:16.741994+0000","last_became_active":"2024-04-24T23:24:15.019686+0000","last_became_peered":"2024-04-24T23:24:15.019686+0000","last_unstale":"2024-04-24T23:24:16.741994+0000","last_undegraded":"2024-04-24T23:24:16.741994+0000","last_fullsized":"2024-04-24T23:24:16.741994+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:41.978014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742160+0000","last_change":"2024-04-24T23:24:14.347743+0000","last_active":"2024-04-24T23:24:16.742160+0000","last_peered":"2024-04-24T23:24:16.742160+0000","last_clean":"2024-04-24T23:24:16.742160+0000","last_became_active":"2024-04-24T23:24:14.347461+0000","last_became_peered":"2024-04-24T23:24:14.347461+0000","last_unstale":"2024-04-24T23:24:16.742160+0000","last_undegraded":"2024-04-24T23:24:16.742160+0000","last_fullsized":"2024-04-24T23:24:16.742160+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:08.694848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013101+0000","last_change":"2024-04-24T23:24:15.024467+0000","last_active":"2024-04-24T23:24:16.013101+0000","last_peered":"2024-04-24T23:24:16.013101+0000","last_clean":"2024-04-24T23:24:16.013101+0000","last_became_active":"2024-04-24T23:24:15.023770+0000","last_became_peered":"2024-04-24T23:24:15.023770+0000","last_unstale":"2024-04-24T23:24:16.013101+0000","last_undegraded":"2024-04-24T23:24:16.013101+0000","last_fullsized":"2024-04-24T23:24:16.013101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:56.194872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013148+0000","last_change":"2024-04-24T23:24:14.342175+0000","last_active":"2024-04-24T23:24:16.013148+0000","last_peered":"2024-04-24T23:24:16.013148+0000","last_clean":"2024-04-24T23:24:16.013148+0000","last_became_active":"2024-04-24T23:24:14.341845+0000","last_became_peered":"2024-04-24T23:24:14.341845+0000","last_unstale":"2024-04-24T23:24:16.013148+0000","last_undegraded":"2024-04-24T23:24:16.013148+0000","last_fullsized":"2024-04-24T23:24:16.013148+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:53.837226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.012994+0000","last_change":"2024-04-24T23:24:15.019470+0000","last_active":"2024-04-24T23:24:16.012994+0000","last_peered":"2024-04-24T23:24:16.012994+0000","last_clean":"2024-04-24T23:24:16.012994+0000","last_became_active":"2024-04-24T23:24:15.019184+0000","last_became_peered":"2024-04-24T23:24:15.019184+0000","last_unstale":"2024-04-24T23:24:16.012994+0000","last_undegraded":"2024-04-24T23:24:16.012994+0000","last_fullsized":"2024-04-24T23:24:16.012994+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:49.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014235+0000","last_change":"2024-04-24T23:24:14.343131+0000","last_active":"2024-04-24T23:24:16.014235+0000","last_peered":"2024-04-24T23:24:16.014235+0000","last_clean":"2024-04-24T23:24:16.014235+0000","last_became_active":"2024-04-24T23:24:14.342828+0000","last_became_peered":"2024-04-24T23:24:14.342828+0000","last_unstale":"2024-04-24T23:24:16.014235+0000","last_undegraded":"2024-04-24T23:24:16.014235+0000","last_fullsized":"2024-04-24T23:24:16.014235+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:03.381663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947610+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:24:16.947610+0000","last_peered":"2024-04-24T23:24:16.947610+0000","last_clean":"2024-04-24T23:24:16.947610+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:24:16.947610+0000","last_undegraded":"2024-04-24T23:24:16.947610+0000","last_fullsized":"2024-04-24T23:24:16.947610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":877209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":696320,"data_stored":292981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":287333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":696320,"data_stored":288293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":293037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36024,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335816,"statfs":{"total":96636764160,"available":96599875584,"internally_reserved":0,"allocated":692224,"data_stored":286737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":700416,"data_stored":293071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1306624,"data_stored":885543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:24:19.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:24:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.156+0000 7f4d02c7d640 1 Processor -- start 2024-04-24T23:24:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.156+0000 7f4d02c7d640 1 -- start start 2024-04-24T23:24:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.157+0000 7f4d02c7d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4cfc15e290 0x7f4cfc0d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.157+0000 7f4d02c7d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4cfc0b72f0 0x7f4cfc0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.157+0000 7f4d02c7d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cfc0b5840 con 0x7f4cfc0b3ac0 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.157+0000 7f4d02c7d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4cfc0b5e50 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.157+0000 7f4d02c7d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4cfc0cfb30 con 0x7f4cfc15e290 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.157+0000 7f4d0121a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4cfc0b72f0 0x7f4cfc0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.157+0000 7f4d0121a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4cfc0b72f0 0x7f4cfc0b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52214/0 (socket says 172.21.15.26:52214) 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.157+0000 7f4d0121a640 1 -- 172.21.15.26:0/3576396514 learned_addr learned my addr 172.21.15.26:0/3576396514 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.157+0000 7f4d00a19640 1 --2- 172.21.15.26:0/3576396514 >> v2:172.21.15.29:3300/0 conn(0x7f4cfc15e290 0x7f4cfc0d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.158+0000 7f4d0121a640 1 -- 172.21.15.26:0/3576396514 >> v2:172.21.15.29:3300/0 conn(0x7f4cfc15e290 msgr2=0x7f4cfc0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.158+0000 7f4d0121a640 1 --2- 172.21.15.26:0/3576396514 >> v2:172.21.15.29:3300/0 conn(0x7f4cfc15e290 0x7f4cfc0d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.158+0000 7f4d0121a640 1 -- 172.21.15.26:0/3576396514 >> v1:172.21.15.26:6789/0 conn(0x7f4cfc0b3ac0 legacy=0x7f4cfc06c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.158+0000 7f4d0121a640 1 -- 172.21.15.26:0/3576396514 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc0cfe10 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.158+0000 7f4cfb7fe640 1 -- 172.21.15.26:0/3576396514 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1988840022 0 0) 0x7f4cfc0b5840 con 0x7f4cfc0b3ac0 2024-04-24T23:24:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.158+0000 7f4d0121a640 1 --2- 172.21.15.26:0/3576396514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 0x7f4cfc0b3f50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0b49e0 tx=0x7f4cec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74366fe473000500 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4cfb7fe640 1 -- 172.21.15.26:0/3576396514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec037070 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4cfb7fe640 1 -- 172.21.15.26:0/3576396514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cec032440 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4cfb7fe640 1 -- 172.21.15.26:0/3576396514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec032810 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4d02c7d640 1 -- 172.21.15.26:0/3576396514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 msgr2=0x7f4cfc0b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4d02c7d640 1 --2- 172.21.15.26:0/3576396514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 0x7f4cfc0b3f50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0b49e0 tx=0x7f4cec02a270 comp rx=0 tx=0).stop 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4d02c7d640 1 -- 172.21.15.26:0/3576396514 shutdown_connections 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4d02c7d640 1 --2- 172.21.15.26:0/3576396514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 0x7f4cfc0b3f50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4d02c7d640 1 --2- 172.21.15.26:0/3576396514 >> v2:172.21.15.29:3300/0 conn(0x7f4cfc15e290 0x7f4cfc0d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4d02c7d640 1 -- 172.21.15.26:0/3576396514 >> 172.21.15.26:0/3576396514 conn(0x7f4cfc06b120 msgr2=0x7f4cfc06b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4d02c7d640 1 -- 172.21.15.26:0/3576396514 shutdown_connections 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.159+0000 7f4d02c7d640 1 -- 172.21.15.26:0/3576396514 wait complete. 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.160+0000 7f4d02c7d640 1 Processor -- start 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.160+0000 7f4d02c7d640 1 -- start start 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4d02c7d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cfc0b3ac0 0x7f4cfc14bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4d02c7d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 0x7f4cfc1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4d02c7d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cfc15e290 0x7f4cfc160f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4d02c7d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc15e670 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4d02c7d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc0b5ae0 con 0x7f4cfc0b3ac0 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4d02c7d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4cfc0d1c70 con 0x7f4cfc15e290 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4cfbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 0x7f4cfc1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4cfbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 0x7f4cfc1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52224/0 (socket says 172.21.15.26:52224) 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4cfbfff640 1 -- 172.21.15.26:0/279208496 learned_addr learned my addr 172.21.15.26:0/279208496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4d00a19640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cfc0b3ac0 0x7f4cfc14bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.161+0000 7f4d0121a640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cfc15e290 0x7f4cfc160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4cfbfff640 1 -- 172.21.15.26:0/279208496 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cfc15e290 msgr2=0x7f4cfc160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4cfbfff640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cfc15e290 0x7f4cfc160f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4cfbfff640 1 -- 172.21.15.26:0/279208496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cfc0b3ac0 msgr2=0x7f4cfc14bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4cfbfff640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cfc0b3ac0 0x7f4cfc14bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4cfbfff640 1 -- 172.21.15.26:0/279208496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc15d9f0 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4cfbfff640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 0x7f4cfc1609c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c520 tx=0x7f4ce400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4cf97fa640 1 -- 172.21.15.26:0/279208496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4017070 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4cf97fa640 1 -- 172.21.15.26:0/279208496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce40052d0 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cfc0c8440 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4cf97fa640 1 -- 172.21.15.26:0/279208496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4012870 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.162+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cfc0c8a00 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.163+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cfc0694c0 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.164+0000 7f4cf97fa640 1 -- 172.21.15.26:0/279208496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ce401a070 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.164+0000 7f4cf97fa640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4cdc06be30 0x7f4cdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.164+0000 7f4cf97fa640 1 -- 172.21.15.26:0/279208496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4ce408a1f0 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.164+0000 7f4d00a19640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4cdc06be30 0x7f4cdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.164+0000 7f4d00a19640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4cdc06be30 0x7f4cdc06e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4011340 tx=0x7f4cf400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.168+0000 7f4cf97fa640 1 -- 172.21.15.26:0/279208496 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ce40584d0 con 0x7f4cfc0b72f0 2024-04-24T23:24:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.349+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4cfc0b3640 con 0x7f4cdc06be30 2024-04-24T23:24:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.355+0000 7f4cf97fa640 1 -- 172.21.15.26:0/279208496 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f4cfc0b3640 con 0x7f4cdc06be30 2024-04-24T23:24:19.359 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:24:19.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4cdc06be30 msgr2=0x7f4cdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4cdc06be30 0x7f4cdc06e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4011340 tx=0x7f4cf400b040 comp rx=0 tx=0).stop 2024-04-24T23:24:19.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 msgr2=0x7f4cfc1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 0x7f4cfc1609c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c520 tx=0x7f4ce400c9e0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 shutdown_connections 2024-04-24T23:24:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4cdc06be30 0x7f4cdc06e2f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4cfc15e290 0x7f4cfc160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0b72f0 0x7f4cfc1609c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 --2- 172.21.15.26:0/279208496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cfc0b3ac0 0x7f4cfc14bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 >> 172.21.15.26:0/279208496 conn(0x7f4cfc06b120 msgr2=0x7f4cfc06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 shutdown_connections 2024-04-24T23:24:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.358+0000 7f4d02c7d640 1 -- 172.21.15.26:0/279208496 wait complete. 2024-04-24T23:24:19.371 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-04-24T23:24:17.358870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260264,"kb_used_data":6616,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754714456,"statfs":{"total":773094113280,"available":772827602944,"internally_reserved":0,"allocated":6774784,"data_stored":3504204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031288"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946097+0000","last_change":"2024-04-24T23:24:15.027361+0000","last_active":"2024-04-24T23:24:16.946097+0000","last_peered":"2024-04-24T23:24:16.946097+0000","last_clean":"2024-04-24T23:24:16.946097+0000","last_became_active":"2024-04-24T23:24:15.027085+0000","last_became_peered":"2024-04-24T23:24:15.027085+0000","last_unstale":"2024-04-24T23:24:16.946097+0000","last_undegraded":"2024-04-24T23:24:16.946097+0000","last_fullsized":"2024-04-24T23:24:16.946097+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:25.631396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013154+0000","last_change":"2024-04-24T23:24:15.025229+0000","last_active":"2024-04-24T23:24:16.013154+0000","last_peered":"2024-04-24T23:24:16.013154+0000","last_clean":"2024-04-24T23:24:16.013154+0000","last_became_active":"2024-04-24T23:24:15.024965+0000","last_became_peered":"2024-04-24T23:24:15.024965+0000","last_unstale":"2024-04-24T23:24:16.013154+0000","last_undegraded":"2024-04-24T23:24:16.013154+0000","last_fullsized":"2024-04-24T23:24:16.013154+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:22.388242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947006+0000","last_change":"2024-04-24T23:24:15.031938+0000","last_active":"2024-04-24T23:24:16.947006+0000","last_peered":"2024-04-24T23:24:16.947006+0000","last_clean":"2024-04-24T23:24:16.947006+0000","last_became_active":"2024-04-24T23:24:15.031640+0000","last_became_peered":"2024-04-24T23:24:15.031640+0000","last_unstale":"2024-04-24T23:24:16.947006+0000","last_undegraded":"2024-04-24T23:24:16.947006+0000","last_fullsized":"2024-04-24T23:24:16.947006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:05.395174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947924+0000","last_change":"2024-04-24T23:24:15.016297+0000","last_active":"2024-04-24T23:24:16.947924+0000","last_peered":"2024-04-24T23:24:16.947924+0000","last_clean":"2024-04-24T23:24:16.947924+0000","last_became_active":"2024-04-24T23:24:15.016007+0000","last_became_peered":"2024-04-24T23:24:15.016007+0000","last_unstale":"2024-04-24T23:24:16.947924+0000","last_undegraded":"2024-04-24T23:24:16.947924+0000","last_fullsized":"2024-04-24T23:24:16.947924+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:00.047511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946006+0000","last_change":"2024-04-24T23:24:15.014827+0000","last_active":"2024-04-24T23:24:16.946006+0000","last_peered":"2024-04-24T23:24:16.946006+0000","last_clean":"2024-04-24T23:24:16.946006+0000","last_became_active":"2024-04-24T23:24:15.014544+0000","last_became_peered":"2024-04-24T23:24:15.014544+0000","last_unstale":"2024-04-24T23:24:16.946006+0000","last_undegraded":"2024-04-24T23:24:16.946006+0000","last_fullsized":"2024-04-24T23:24:16.946006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:40.327365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340369+0000","last_change":"2024-04-24T23:24:15.021059+0000","last_active":"2024-04-24T23:24:16.340369+0000","last_peered":"2024-04-24T23:24:16.340369+0000","last_clean":"2024-04-24T23:24:16.340369+0000","last_became_active":"2024-04-24T23:24:15.020782+0000","last_became_peered":"2024-04-24T23:24:15.020782+0000","last_unstale":"2024-04-24T23:24:16.340369+0000","last_undegraded":"2024-04-24T23:24:16.340369+0000","last_fullsized":"2024-04-24T23:24:16.340369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:38.948032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947446+0000","last_change":"2024-04-24T23:24:15.020545+0000","last_active":"2024-04-24T23:24:16.947446+0000","last_peered":"2024-04-24T23:24:16.947446+0000","last_clean":"2024-04-24T23:24:16.947446+0000","last_became_active":"2024-04-24T23:24:15.020260+0000","last_became_peered":"2024-04-24T23:24:15.020260+0000","last_unstale":"2024-04-24T23:24:16.947446+0000","last_undegraded":"2024-04-24T23:24:16.947446+0000","last_fullsized":"2024-04-24T23:24:16.947446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:30.640853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.012965+0000","last_change":"2024-04-24T23:24:15.020232+0000","last_active":"2024-04-24T23:24:16.012965+0000","last_peered":"2024-04-24T23:24:16.012965+0000","last_clean":"2024-04-24T23:24:16.012965+0000","last_became_active":"2024-04-24T23:24:15.019978+0000","last_became_peered":"2024-04-24T23:24:15.019978+0000","last_unstale":"2024-04-24T23:24:16.012965+0000","last_undegraded":"2024-04-24T23:24:16.012965+0000","last_fullsized":"2024-04-24T23:24:16.012965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:17.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946800+0000","last_change":"2024-04-24T23:24:15.023502+0000","last_active":"2024-04-24T23:24:16.946800+0000","last_peered":"2024-04-24T23:24:16.946800+0000","last_clean":"2024-04-24T23:24:16.946800+0000","last_became_active":"2024-04-24T23:24:15.023129+0000","last_became_peered":"2024-04-24T23:24:15.023129+0000","last_unstale":"2024-04-24T23:24:16.946800+0000","last_undegraded":"2024-04-24T23:24:16.946800+0000","last_fullsized":"2024-04-24T23:24:16.946800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:54.068906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742269+0000","last_change":"2024-04-24T23:24:15.020536+0000","last_active":"2024-04-24T23:24:16.742269+0000","last_peered":"2024-04-24T23:24:16.742269+0000","last_clean":"2024-04-24T23:24:16.742269+0000","last_became_active":"2024-04-24T23:24:15.020188+0000","last_became_peered":"2024-04-24T23:24:15.020188+0000","last_unstale":"2024-04-24T23:24:16.742269+0000","last_undegraded":"2024-04-24T23:24:16.742269+0000","last_fullsized":"2024-04-24T23:24:16.742269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:52.883280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742318+0000","last_change":"2024-04-24T23:24:15.026685+0000","last_active":"2024-04-24T23:24:16.742318+0000","last_peered":"2024-04-24T23:24:16.742318+0000","last_clean":"2024-04-24T23:24:16.742318+0000","last_became_active":"2024-04-24T23:24:15.026305+0000","last_became_peered":"2024-04-24T23:24:15.026305+0000","last_unstale":"2024-04-24T23:24:16.742318+0000","last_undegraded":"2024-04-24T23:24:16.742318+0000","last_fullsized":"2024-04-24T23:24:16.742318+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:46.459683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948014+0000","last_change":"2024-04-24T23:24:15.022404+0000","last_active":"2024-04-24T23:24:16.948014+0000","last_peered":"2024-04-24T23:24:16.948014+0000","last_clean":"2024-04-24T23:24:16.948014+0000","last_became_active":"2024-04-24T23:24:15.021971+0000","last_became_peered":"2024-04-24T23:24:15.021971+0000","last_unstale":"2024-04-24T23:24:16.948014+0000","last_undegraded":"2024-04-24T23:24:16.948014+0000","last_fullsized":"2024-04-24T23:24:16.948014+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:48.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014401+0000","last_change":"2024-04-24T23:24:15.022380+0000","last_active":"2024-04-24T23:24:16.014401+0000","last_peered":"2024-04-24T23:24:16.014401+0000","last_clean":"2024-04-24T23:24:16.014401+0000","last_became_active":"2024-04-24T23:24:15.021893+0000","last_became_peered":"2024-04-24T23:24:15.021893+0000","last_unstale":"2024-04-24T23:24:16.014401+0000","last_undegraded":"2024-04-24T23:24:16.014401+0000","last_fullsized":"2024-04-24T23:24:16.014401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:41.911420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947036+0000","last_change":"2024-04-24T23:24:15.016624+0000","last_active":"2024-04-24T23:24:16.947036+0000","last_peered":"2024-04-24T23:24:16.947036+0000","last_clean":"2024-04-24T23:24:16.947036+0000","last_became_active":"2024-04-24T23:24:15.016288+0000","last_became_peered":"2024-04-24T23:24:15.016288+0000","last_unstale":"2024-04-24T23:24:16.947036+0000","last_undegraded":"2024-04-24T23:24:16.947036+0000","last_fullsized":"2024-04-24T23:24:16.947036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:59.371935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947326+0000","last_change":"2024-04-24T23:24:15.019696+0000","last_active":"2024-04-24T23:24:16.947326+0000","last_peered":"2024-04-24T23:24:16.947326+0000","last_clean":"2024-04-24T23:24:16.947326+0000","last_became_active":"2024-04-24T23:24:15.019340+0000","last_became_peered":"2024-04-24T23:24:15.019340+0000","last_unstale":"2024-04-24T23:24:16.947326+0000","last_undegraded":"2024-04-24T23:24:16.947326+0000","last_fullsized":"2024-04-24T23:24:16.947326+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:51.461001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340295+0000","last_change":"2024-04-24T23:24:15.028629+0000","last_active":"2024-04-24T23:24:16.340295+0000","last_peered":"2024-04-24T23:24:16.340295+0000","last_clean":"2024-04-24T23:24:16.340295+0000","last_became_active":"2024-04-24T23:24:15.028379+0000","last_became_peered":"2024-04-24T23:24:15.028379+0000","last_unstale":"2024-04-24T23:24:16.340295+0000","last_undegraded":"2024-04-24T23:24:16.340295+0000","last_fullsized":"2024-04-24T23:24:16.340295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:37.044137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013570+0000","last_change":"2024-04-24T23:24:15.024054+0000","last_active":"2024-04-24T23:24:16.013570+0000","last_peered":"2024-04-24T23:24:16.013570+0000","last_clean":"2024-04-24T23:24:16.013570+0000","last_became_active":"2024-04-24T23:24:15.023695+0000","last_became_peered":"2024-04-24T23:24:15.023695+0000","last_unstale":"2024-04-24T23:24:16.013570+0000","last_undegraded":"2024-04-24T23:24:16.013570+0000","last_fullsized":"2024-04-24T23:24:16.013570+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:46.265731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947213+0000","last_change":"2024-04-24T23:24:15.031020+0000","last_active":"2024-04-24T23:24:16.947213+0000","last_peered":"2024-04-24T23:24:16.947213+0000","last_clean":"2024-04-24T23:24:16.947213+0000","last_became_active":"2024-04-24T23:24:15.030712+0000","last_became_peered":"2024-04-24T23:24:15.030712+0000","last_unstale":"2024-04-24T23:24:16.947213+0000","last_undegraded":"2024-04-24T23:24:16.947213+0000","last_fullsized":"2024-04-24T23:24:16.947213+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:00.047511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946148+0000","last_change":"2024-04-24T23:24:15.026223+0000","last_active":"2024-04-24T23:24:16.946148+0000","last_peered":"2024-04-24T23:24:16.946148+0000","last_clean":"2024-04-24T23:24:16.946148+0000","last_became_active":"2024-04-24T23:24:15.025958+0000","last_became_peered":"2024-04-24T23:24:15.025958+0000","last_unstale":"2024-04-24T23:24:16.946148+0000","last_undegraded":"2024-04-24T23:24:16.946148+0000","last_fullsized":"2024-04-24T23:24:16.946148+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:53.718925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013197+0000","last_change":"2024-04-24T23:24:15.022765+0000","last_active":"2024-04-24T23:24:16.013197+0000","last_peered":"2024-04-24T23:24:16.013197+0000","last_clean":"2024-04-24T23:24:16.013197+0000","last_became_active":"2024-04-24T23:24:15.022500+0000","last_became_peered":"2024-04-24T23:24:15.022500+0000","last_unstale":"2024-04-24T23:24:16.013197+0000","last_undegraded":"2024-04-24T23:24:16.013197+0000","last_fullsized":"2024-04-24T23:24:16.013197+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:26.158823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946799+0000","last_change":"2024-04-24T23:24:15.030749+0000","last_active":"2024-04-24T23:24:16.946799+0000","last_peered":"2024-04-24T23:24:16.946799+0000","last_clean":"2024-04-24T23:24:16.946799+0000","last_became_active":"2024-04-24T23:24:15.030449+0000","last_became_peered":"2024-04-24T23:24:15.030449+0000","last_unstale":"2024-04-24T23:24:16.946799+0000","last_undegraded":"2024-04-24T23:24:16.946799+0000","last_fullsized":"2024-04-24T23:24:16.946799+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:15.488825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013079+0000","last_change":"2024-04-24T23:24:15.023839+0000","last_active":"2024-04-24T23:24:16.013079+0000","last_peered":"2024-04-24T23:24:16.013079+0000","last_clean":"2024-04-24T23:24:16.013079+0000","last_became_active":"2024-04-24T23:24:15.022784+0000","last_became_peered":"2024-04-24T23:24:15.022784+0000","last_unstale":"2024-04-24T23:24:16.013079+0000","last_undegraded":"2024-04-24T23:24:16.013079+0000","last_fullsized":"2024-04-24T23:24:16.013079+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:19.460584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947819+0000","last_change":"2024-04-24T23:24:15.028126+0000","last_active":"2024-04-24T23:24:16.947819+0000","last_peered":"2024-04-24T23:24:16.947819+0000","last_clean":"2024-04-24T23:24:16.947819+0000","last_became_active":"2024-04-24T23:24:15.027807+0000","last_became_peered":"2024-04-24T23:24:15.027807+0000","last_unstale":"2024-04-24T23:24:16.947819+0000","last_undegraded":"2024-04-24T23:24:16.947819+0000","last_fullsized":"2024-04-24T23:24:16.947819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:37.893931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014182+0000","last_change":"2024-04-24T23:24:15.022857+0000","last_active":"2024-04-24T23:24:16.014182+0000","last_peered":"2024-04-24T23:24:16.014182+0000","last_clean":"2024-04-24T23:24:16.014182+0000","last_became_active":"2024-04-24T23:24:15.021730+0000","last_became_peered":"2024-04-24T23:24:15.021730+0000","last_unstale":"2024-04-24T23:24:16.014182+0000","last_undegraded":"2024-04-24T23:24:16.014182+0000","last_fullsized":"2024-04-24T23:24:16.014182+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:58.886412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946185+0000","last_change":"2024-04-24T23:24:15.019092+0000","last_active":"2024-04-24T23:24:16.946185+0000","last_peered":"2024-04-24T23:24:16.946185+0000","last_clean":"2024-04-24T23:24:16.946185+0000","last_became_active":"2024-04-24T23:24:15.018795+0000","last_became_peered":"2024-04-24T23:24:15.018795+0000","last_unstale":"2024-04-24T23:24:16.946185+0000","last_undegraded":"2024-04-24T23:24:16.946185+0000","last_fullsized":"2024-04-24T23:24:16.946185+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:16.562294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946909+0000","last_change":"2024-04-24T23:24:15.019719+0000","last_active":"2024-04-24T23:24:16.946909+0000","last_peered":"2024-04-24T23:24:16.946909+0000","last_clean":"2024-04-24T23:24:16.946909+0000","last_became_active":"2024-04-24T23:24:15.018541+0000","last_became_peered":"2024-04-24T23:24:15.018541+0000","last_unstale":"2024-04-24T23:24:16.946909+0000","last_undegraded":"2024-04-24T23:24:16.946909+0000","last_fullsized":"2024-04-24T23:24:16.946909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:28.322115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340518+0000","last_change":"2024-04-24T23:24:15.028983+0000","last_active":"2024-04-24T23:24:16.340518+0000","last_peered":"2024-04-24T23:24:16.340518+0000","last_clean":"2024-04-24T23:24:16.340518+0000","last_became_active":"2024-04-24T23:24:15.028701+0000","last_became_peered":"2024-04-24T23:24:15.028701+0000","last_unstale":"2024-04-24T23:24:16.340518+0000","last_undegraded":"2024-04-24T23:24:16.340518+0000","last_fullsized":"2024-04-24T23:24:16.340518+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:05.395174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946155+0000","last_change":"2024-04-24T23:24:15.018572+0000","last_active":"2024-04-24T23:24:16.946155+0000","last_peered":"2024-04-24T23:24:16.946155+0000","last_clean":"2024-04-24T23:24:16.946155+0000","last_became_active":"2024-04-24T23:24:15.017843+0000","last_became_peered":"2024-04-24T23:24:15.017843+0000","last_unstale":"2024-04-24T23:24:16.946155+0000","last_undegraded":"2024-04-24T23:24:16.946155+0000","last_fullsized":"2024-04-24T23:24:16.946155+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:34.764580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013640+0000","last_change":"2024-04-24T23:24:15.025709+0000","last_active":"2024-04-24T23:24:16.013640+0000","last_peered":"2024-04-24T23:24:16.013640+0000","last_clean":"2024-04-24T23:24:16.013640+0000","last_became_active":"2024-04-24T23:24:15.025419+0000","last_became_peered":"2024-04-24T23:24:15.025419+0000","last_unstale":"2024-04-24T23:24:16.013640+0000","last_undegraded":"2024-04-24T23:24:16.013640+0000","last_fullsized":"2024-04-24T23:24:16.013640+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:34.926701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946753+0000","last_change":"2024-04-24T23:24:15.031653+0000","last_active":"2024-04-24T23:24:16.946753+0000","last_peered":"2024-04-24T23:24:16.946753+0000","last_clean":"2024-04-24T23:24:16.946753+0000","last_became_active":"2024-04-24T23:24:15.031339+0000","last_became_peered":"2024-04-24T23:24:15.031339+0000","last_unstale":"2024-04-24T23:24:16.946753+0000","last_undegraded":"2024-04-24T23:24:16.946753+0000","last_fullsized":"2024-04-24T23:24:16.946753+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:38.948032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.945914+0000","last_change":"2024-04-24T23:24:15.018383+0000","last_active":"2024-04-24T23:24:16.945914+0000","last_peered":"2024-04-24T23:24:16.945914+0000","last_clean":"2024-04-24T23:24:16.945914+0000","last_became_active":"2024-04-24T23:24:15.018092+0000","last_became_peered":"2024-04-24T23:24:15.018092+0000","last_unstale":"2024-04-24T23:24:16.945914+0000","last_undegraded":"2024-04-24T23:24:16.945914+0000","last_fullsized":"2024-04-24T23:24:16.945914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:34.472583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340538+0000","last_change":"2024-04-24T23:24:15.023226+0000","last_active":"2024-04-24T23:24:16.340538+0000","last_peered":"2024-04-24T23:24:16.340538+0000","last_clean":"2024-04-24T23:24:16.340538+0000","last_became_active":"2024-04-24T23:24:15.022917+0000","last_became_peered":"2024-04-24T23:24:15.022917+0000","last_unstale":"2024-04-24T23:24:16.340538+0000","last_undegraded":"2024-04-24T23:24:16.340538+0000","last_fullsized":"2024-04-24T23:24:16.340538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:00.047511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340728+0000","last_change":"2024-04-24T23:24:15.020912+0000","last_active":"2024-04-24T23:24:16.340728+0000","last_peered":"2024-04-24T23:24:16.340728+0000","last_clean":"2024-04-24T23:24:16.340728+0000","last_became_active":"2024-04-24T23:24:15.020292+0000","last_became_peered":"2024-04-24T23:24:15.020292+0000","last_unstale":"2024-04-24T23:24:16.340728+0000","last_undegraded":"2024-04-24T23:24:16.340728+0000","last_fullsized":"2024-04-24T23:24:16.340728+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:59.338397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340743+0000","last_change":"2024-04-24T23:24:14.346692+0000","last_active":"2024-04-24T23:24:16.340743+0000","last_peered":"2024-04-24T23:24:16.340743+0000","last_clean":"2024-04-24T23:24:16.340743+0000","last_became_active":"2024-04-24T23:24:14.346432+0000","last_became_peered":"2024-04-24T23:24:14.346432+0000","last_unstale":"2024-04-24T23:24:16.340743+0000","last_undegraded":"2024-04-24T23:24:16.340743+0000","last_fullsized":"2024-04-24T23:24:16.340743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:45.457469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742333+0000","last_change":"2024-04-24T23:24:15.021139+0000","last_active":"2024-04-24T23:24:16.742333+0000","last_peered":"2024-04-24T23:24:16.742333+0000","last_clean":"2024-04-24T23:24:16.742333+0000","last_became_active":"2024-04-24T23:24:15.020885+0000","last_became_peered":"2024-04-24T23:24:15.020885+0000","last_unstale":"2024-04-24T23:24:16.742333+0000","last_undegraded":"2024-04-24T23:24:16.742333+0000","last_fullsized":"2024-04-24T23:24:16.742333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:37.396907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013267+0000","last_change":"2024-04-24T23:24:14.353647+0000","last_active":"2024-04-24T23:24:16.013267+0000","last_peered":"2024-04-24T23:24:16.013267+0000","last_clean":"2024-04-24T23:24:16.013267+0000","last_became_active":"2024-04-24T23:24:14.353378+0000","last_became_peered":"2024-04-24T23:24:14.353378+0000","last_unstale":"2024-04-24T23:24:16.013267+0000","last_undegraded":"2024-04-24T23:24:16.013267+0000","last_fullsized":"2024-04-24T23:24:16.013267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:59.045297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340517+0000","last_change":"2024-04-24T23:24:15.026093+0000","last_active":"2024-04-24T23:24:16.340517+0000","last_peered":"2024-04-24T23:24:16.340517+0000","last_clean":"2024-04-24T23:24:16.340517+0000","last_became_active":"2024-04-24T23:24:15.025804+0000","last_became_peered":"2024-04-24T23:24:15.025804+0000","last_unstale":"2024-04-24T23:24:16.340517+0000","last_undegraded":"2024-04-24T23:24:16.340517+0000","last_fullsized":"2024-04-24T23:24:16.340517+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:54.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340682+0000","last_change":"2024-04-24T23:24:14.345945+0000","last_active":"2024-04-24T23:24:16.340682+0000","last_peered":"2024-04-24T23:24:16.340682+0000","last_clean":"2024-04-24T23:24:16.340682+0000","last_became_active":"2024-04-24T23:24:14.345628+0000","last_became_peered":"2024-04-24T23:24:14.345628+0000","last_unstale":"2024-04-24T23:24:16.340682+0000","last_undegraded":"2024-04-24T23:24:16.340682+0000","last_fullsized":"2024-04-24T23:24:16.340682+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:53.066692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340593+0000","last_change":"2024-04-24T23:24:15.028874+0000","last_active":"2024-04-24T23:24:16.340593+0000","last_peered":"2024-04-24T23:24:16.340593+0000","last_clean":"2024-04-24T23:24:16.340593+0000","last_became_active":"2024-04-24T23:24:15.028372+0000","last_became_peered":"2024-04-24T23:24:15.028372+0000","last_unstale":"2024-04-24T23:24:16.340593+0000","last_undegraded":"2024-04-24T23:24:16.340593+0000","last_fullsized":"2024-04-24T23:24:16.340593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:32.978848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948057+0000","last_change":"2024-04-24T23:24:14.348475+0000","last_active":"2024-04-24T23:24:16.948057+0000","last_peered":"2024-04-24T23:24:16.948057+0000","last_clean":"2024-04-24T23:24:16.948057+0000","last_became_active":"2024-04-24T23:24:14.348135+0000","last_became_peered":"2024-04-24T23:24:14.348135+0000","last_unstale":"2024-04-24T23:24:16.948057+0000","last_undegraded":"2024-04-24T23:24:16.948057+0000","last_fullsized":"2024-04-24T23:24:16.948057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:38.017768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013393+0000","last_change":"2024-04-24T23:24:15.023284+0000","last_active":"2024-04-24T23:24:16.013393+0000","last_peered":"2024-04-24T23:24:16.013393+0000","last_clean":"2024-04-24T23:24:16.013393+0000","last_became_active":"2024-04-24T23:24:15.023018+0000","last_became_peered":"2024-04-24T23:24:15.023018+0000","last_unstale":"2024-04-24T23:24:16.013393+0000","last_undegraded":"2024-04-24T23:24:16.013393+0000","last_fullsized":"2024-04-24T23:24:16.013393+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:54.414987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947779+0000","last_change":"2024-04-24T23:24:14.348570+0000","last_active":"2024-04-24T23:24:16.947779+0000","last_peered":"2024-04-24T23:24:16.947779+0000","last_clean":"2024-04-24T23:24:16.947779+0000","last_became_active":"2024-04-24T23:24:14.348405+0000","last_became_peered":"2024-04-24T23:24:14.348405+0000","last_unstale":"2024-04-24T23:24:16.947779+0000","last_undegraded":"2024-04-24T23:24:16.947779+0000","last_fullsized":"2024-04-24T23:24:16.947779+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:06.580007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013349+0000","last_change":"2024-04-24T23:24:15.025333+0000","last_active":"2024-04-24T23:24:16.013349+0000","last_peered":"2024-04-24T23:24:16.013349+0000","last_clean":"2024-04-24T23:24:16.013349+0000","last_became_active":"2024-04-24T23:24:15.025041+0000","last_became_peered":"2024-04-24T23:24:15.025041+0000","last_unstale":"2024-04-24T23:24:16.013349+0000","last_undegraded":"2024-04-24T23:24:16.013349+0000","last_fullsized":"2024-04-24T23:24:16.013349+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:40.153193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340558+0000","last_change":"2024-04-24T23:24:14.344502+0000","last_active":"2024-04-24T23:24:16.340558+0000","last_peered":"2024-04-24T23:24:16.340558+0000","last_clean":"2024-04-24T23:24:16.340558+0000","last_became_active":"2024-04-24T23:24:14.342934+0000","last_became_peered":"2024-04-24T23:24:14.342934+0000","last_unstale":"2024-04-24T23:24:16.340558+0000","last_undegraded":"2024-04-24T23:24:16.340558+0000","last_fullsized":"2024-04-24T23:24:16.340558+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:07.174115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014148+0000","last_change":"2024-04-24T23:24:15.024649+0000","last_active":"2024-04-24T23:24:16.014148+0000","last_peered":"2024-04-24T23:24:16.014148+0000","last_clean":"2024-04-24T23:24:16.014148+0000","last_became_active":"2024-04-24T23:24:15.024348+0000","last_became_peered":"2024-04-24T23:24:15.024348+0000","last_unstale":"2024-04-24T23:24:16.014148+0000","last_undegraded":"2024-04-24T23:24:16.014148+0000","last_fullsized":"2024-04-24T23:24:16.014148+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:30.219114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946609+0000","last_change":"2024-04-24T23:24:14.342658+0000","last_active":"2024-04-24T23:24:16.946609+0000","last_peered":"2024-04-24T23:24:16.946609+0000","last_clean":"2024-04-24T23:24:16.946609+0000","last_became_active":"2024-04-24T23:24:14.342244+0000","last_became_peered":"2024-04-24T23:24:14.342244+0000","last_unstale":"2024-04-24T23:24:16.946609+0000","last_undegraded":"2024-04-24T23:24:16.946609+0000","last_fullsized":"2024-04-24T23:24:16.946609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:53.066692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013626+0000","last_change":"2024-04-24T23:24:15.021387+0000","last_active":"2024-04-24T23:24:16.013626+0000","last_peered":"2024-04-24T23:24:16.013626+0000","last_clean":"2024-04-24T23:24:16.013626+0000","last_became_active":"2024-04-24T23:24:15.021103+0000","last_became_peered":"2024-04-24T23:24:15.021103+0000","last_unstale":"2024-04-24T23:24:16.013626+0000","last_undegraded":"2024-04-24T23:24:16.013626+0000","last_fullsized":"2024-04-24T23:24:16.013626+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:31.220641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948069+0000","last_change":"2024-04-24T23:24:14.351852+0000","last_active":"2024-04-24T23:24:16.948069+0000","last_peered":"2024-04-24T23:24:16.948069+0000","last_clean":"2024-04-24T23:24:16.948069+0000","last_became_active":"2024-04-24T23:24:14.351524+0000","last_became_peered":"2024-04-24T23:24:14.351524+0000","last_unstale":"2024-04-24T23:24:16.948069+0000","last_undegraded":"2024-04-24T23:24:16.948069+0000","last_fullsized":"2024-04-24T23:24:16.948069+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:40.909206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947540+0000","last_change":"2024-04-24T23:24:15.019788+0000","last_active":"2024-04-24T23:24:16.947540+0000","last_peered":"2024-04-24T23:24:16.947540+0000","last_clean":"2024-04-24T23:24:16.947540+0000","last_became_active":"2024-04-24T23:24:15.019494+0000","last_became_peered":"2024-04-24T23:24:15.019494+0000","last_unstale":"2024-04-24T23:24:16.947540+0000","last_undegraded":"2024-04-24T23:24:16.947540+0000","last_fullsized":"2024-04-24T23:24:16.947540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:39.019982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947478+0000","last_change":"2024-04-24T23:24:14.349239+0000","last_active":"2024-04-24T23:24:16.947478+0000","last_peered":"2024-04-24T23:24:16.947478+0000","last_clean":"2024-04-24T23:24:16.947478+0000","last_became_active":"2024-04-24T23:24:14.348950+0000","last_became_peered":"2024-04-24T23:24:14.348950+0000","last_unstale":"2024-04-24T23:24:16.947478+0000","last_undegraded":"2024-04-24T23:24:16.947478+0000","last_fullsized":"2024-04-24T23:24:16.947478+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:50.458787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947357+0000","last_change":"2024-04-24T23:24:15.033131+0000","last_active":"2024-04-24T23:24:16.947357+0000","last_peered":"2024-04-24T23:24:16.947357+0000","last_clean":"2024-04-24T23:24:16.947357+0000","last_became_active":"2024-04-24T23:24:15.032669+0000","last_became_peered":"2024-04-24T23:24:15.032669+0000","last_unstale":"2024-04-24T23:24:16.947357+0000","last_undegraded":"2024-04-24T23:24:16.947357+0000","last_fullsized":"2024-04-24T23:24:16.947357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:59.338397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013270+0000","last_change":"2024-04-24T23:24:14.351685+0000","last_active":"2024-04-24T23:24:16.013270+0000","last_peered":"2024-04-24T23:24:16.013270+0000","last_clean":"2024-04-24T23:24:16.013270+0000","last_became_active":"2024-04-24T23:24:14.351284+0000","last_became_peered":"2024-04-24T23:24:14.351284+0000","last_unstale":"2024-04-24T23:24:16.013270+0000","last_undegraded":"2024-04-24T23:24:16.013270+0000","last_fullsized":"2024-04-24T23:24:16.013270+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:47.091701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742191+0000","last_change":"2024-04-24T23:24:15.020567+0000","last_active":"2024-04-24T23:24:16.742191+0000","last_peered":"2024-04-24T23:24:16.742191+0000","last_clean":"2024-04-24T23:24:16.742191+0000","last_became_active":"2024-04-24T23:24:15.019774+0000","last_became_peered":"2024-04-24T23:24:15.019774+0000","last_unstale":"2024-04-24T23:24:16.742191+0000","last_undegraded":"2024-04-24T23:24:16.742191+0000","last_fullsized":"2024-04-24T23:24:16.742191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:29.985321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947567+0000","last_change":"2024-04-24T23:24:14.354115+0000","last_active":"2024-04-24T23:24:16.947567+0000","last_peered":"2024-04-24T23:24:16.947567+0000","last_clean":"2024-04-24T23:24:16.947567+0000","last_became_active":"2024-04-24T23:24:14.353809+0000","last_became_peered":"2024-04-24T23:24:14.353809+0000","last_unstale":"2024-04-24T23:24:16.947567+0000","last_undegraded":"2024-04-24T23:24:16.947567+0000","last_fullsized":"2024-04-24T23:24:16.947567+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:58.369720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947223+0000","last_change":"2024-04-24T23:24:15.030348+0000","last_active":"2024-04-24T23:24:16.947223+0000","last_peered":"2024-04-24T23:24:16.947223+0000","last_clean":"2024-04-24T23:24:16.947223+0000","last_became_active":"2024-04-24T23:24:15.030020+0000","last_became_peered":"2024-04-24T23:24:15.030020+0000","last_unstale":"2024-04-24T23:24:16.947223+0000","last_undegraded":"2024-04-24T23:24:16.947223+0000","last_fullsized":"2024-04-24T23:24:16.947223+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:48.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948381+0000","last_change":"2024-04-24T23:24:14.344826+0000","last_active":"2024-04-24T23:24:16.948381+0000","last_peered":"2024-04-24T23:24:16.948381+0000","last_clean":"2024-04-24T23:24:16.948381+0000","last_became_active":"2024-04-24T23:24:14.344520+0000","last_became_peered":"2024-04-24T23:24:14.344520+0000","last_unstale":"2024-04-24T23:24:16.948381+0000","last_undegraded":"2024-04-24T23:24:16.948381+0000","last_fullsized":"2024-04-24T23:24:16.948381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:40.975800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947410+0000","last_change":"2024-04-24T23:24:15.020602+0000","last_active":"2024-04-24T23:24:16.947410+0000","last_peered":"2024-04-24T23:24:16.947410+0000","last_clean":"2024-04-24T23:24:16.947410+0000","last_became_active":"2024-04-24T23:24:15.020222+0000","last_became_peered":"2024-04-24T23:24:15.020222+0000","last_unstale":"2024-04-24T23:24:16.947410+0000","last_undegraded":"2024-04-24T23:24:16.947410+0000","last_fullsized":"2024-04-24T23:24:16.947410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:15.488825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013462+0000","last_change":"2024-04-24T23:24:14.354341+0000","last_active":"2024-04-24T23:24:16.013462+0000","last_peered":"2024-04-24T23:24:16.013462+0000","last_clean":"2024-04-24T23:24:16.013462+0000","last_became_active":"2024-04-24T23:24:14.354041+0000","last_became_peered":"2024-04-24T23:24:14.354041+0000","last_unstale":"2024-04-24T23:24:16.013462+0000","last_undegraded":"2024-04-24T23:24:16.013462+0000","last_fullsized":"2024-04-24T23:24:16.013462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:58.336183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947543+0000","last_change":"2024-04-24T23:24:15.029748+0000","last_active":"2024-04-24T23:24:16.947543+0000","last_peered":"2024-04-24T23:24:16.947543+0000","last_clean":"2024-04-24T23:24:16.947543+0000","last_became_active":"2024-04-24T23:24:15.029017+0000","last_became_peered":"2024-04-24T23:24:15.029017+0000","last_unstale":"2024-04-24T23:24:16.947543+0000","last_undegraded":"2024-04-24T23:24:16.947543+0000","last_fullsized":"2024-04-24T23:24:16.947543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:03.751530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014179+0000","last_change":"2024-04-24T23:24:14.351000+0000","last_active":"2024-04-24T23:24:16.014179+0000","last_peered":"2024-04-24T23:24:16.014179+0000","last_clean":"2024-04-24T23:24:16.014179+0000","last_became_active":"2024-04-24T23:24:14.350461+0000","last_became_peered":"2024-04-24T23:24:14.350461+0000","last_unstale":"2024-04-24T23:24:16.014179+0000","last_undegraded":"2024-04-24T23:24:16.014179+0000","last_fullsized":"2024-04-24T23:24:16.014179+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:08.694848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948086+0000","last_change":"2024-04-24T23:24:15.022598+0000","last_active":"2024-04-24T23:24:16.948086+0000","last_peered":"2024-04-24T23:24:16.948086+0000","last_clean":"2024-04-24T23:24:16.948086+0000","last_became_active":"2024-04-24T23:24:15.022128+0000","last_became_peered":"2024-04-24T23:24:15.022128+0000","last_unstale":"2024-04-24T23:24:16.948086+0000","last_undegraded":"2024-04-24T23:24:16.948086+0000","last_fullsized":"2024-04-24T23:24:16.948086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:19.516771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946722+0000","last_change":"2024-04-24T23:24:14.348951+0000","last_active":"2024-04-24T23:24:16.946722+0000","last_peered":"2024-04-24T23:24:16.946722+0000","last_clean":"2024-04-24T23:24:16.946722+0000","last_became_active":"2024-04-24T23:24:14.348576+0000","last_became_peered":"2024-04-24T23:24:14.348576+0000","last_unstale":"2024-04-24T23:24:16.946722+0000","last_undegraded":"2024-04-24T23:24:16.946722+0000","last_fullsized":"2024-04-24T23:24:16.946722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:50.458787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947831+0000","last_change":"2024-04-24T23:24:15.028932+0000","last_active":"2024-04-24T23:24:16.947831+0000","last_peered":"2024-04-24T23:24:16.947831+0000","last_clean":"2024-04-24T23:24:16.947831+0000","last_became_active":"2024-04-24T23:24:15.028553+0000","last_became_peered":"2024-04-24T23:24:15.028553+0000","last_unstale":"2024-04-24T23:24:16.947831+0000","last_undegraded":"2024-04-24T23:24:16.947831+0000","last_fullsized":"2024-04-24T23:24:16.947831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:21.065862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947489+0000","last_change":"2024-04-24T23:24:14.349655+0000","last_active":"2024-04-24T23:24:16.947489+0000","last_peered":"2024-04-24T23:24:16.947489+0000","last_clean":"2024-04-24T23:24:16.947489+0000","last_became_active":"2024-04-24T23:24:14.349352+0000","last_became_peered":"2024-04-24T23:24:14.349352+0000","last_unstale":"2024-04-24T23:24:16.947489+0000","last_undegraded":"2024-04-24T23:24:16.947489+0000","last_fullsized":"2024-04-24T23:24:16.947489+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:29.638639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947223+0000","last_change":"2024-04-24T23:24:15.031725+0000","last_active":"2024-04-24T23:24:16.947223+0000","last_peered":"2024-04-24T23:24:16.947223+0000","last_clean":"2024-04-24T23:24:16.947223+0000","last_became_active":"2024-04-24T23:24:15.031391+0000","last_became_peered":"2024-04-24T23:24:15.031391+0000","last_unstale":"2024-04-24T23:24:16.947223+0000","last_undegraded":"2024-04-24T23:24:16.947223+0000","last_fullsized":"2024-04-24T23:24:16.947223+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:32.978848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013629+0000","last_change":"2024-04-24T23:24:14.355432+0000","last_active":"2024-04-24T23:24:16.013629+0000","last_peered":"2024-04-24T23:24:16.013629+0000","last_clean":"2024-04-24T23:24:16.013629+0000","last_became_active":"2024-04-24T23:24:14.355133+0000","last_became_peered":"2024-04-24T23:24:14.355133+0000","last_unstale":"2024-04-24T23:24:16.013629+0000","last_undegraded":"2024-04-24T23:24:16.013629+0000","last_fullsized":"2024-04-24T23:24:16.013629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:31.976634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947986+0000","last_change":"2024-04-24T23:24:15.021549+0000","last_active":"2024-04-24T23:24:16.947986+0000","last_peered":"2024-04-24T23:24:16.947986+0000","last_clean":"2024-04-24T23:24:16.947986+0000","last_became_active":"2024-04-24T23:24:15.021040+0000","last_became_peered":"2024-04-24T23:24:15.021040+0000","last_unstale":"2024-04-24T23:24:16.947986+0000","last_undegraded":"2024-04-24T23:24:16.947986+0000","last_fullsized":"2024-04-24T23:24:16.947986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:32.978848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340695+0000","last_change":"2024-04-24T23:24:14.350265+0000","last_active":"2024-04-24T23:24:16.340695+0000","last_peered":"2024-04-24T23:24:16.340695+0000","last_clean":"2024-04-24T23:24:16.340695+0000","last_became_active":"2024-04-24T23:24:14.349999+0000","last_became_peered":"2024-04-24T23:24:14.349999+0000","last_unstale":"2024-04-24T23:24:16.340695+0000","last_undegraded":"2024-04-24T23:24:16.340695+0000","last_fullsized":"2024-04-24T23:24:16.340695+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:29.638639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013333+0000","last_change":"2024-04-24T23:24:15.022211+0000","last_active":"2024-04-24T23:24:16.013333+0000","last_peered":"2024-04-24T23:24:16.013333+0000","last_clean":"2024-04-24T23:24:16.013333+0000","last_became_active":"2024-04-24T23:24:15.021891+0000","last_became_peered":"2024-04-24T23:24:15.021891+0000","last_unstale":"2024-04-24T23:24:16.013333+0000","last_undegraded":"2024-04-24T23:24:16.013333+0000","last_fullsized":"2024-04-24T23:24:16.013333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:45.104652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947871+0000","last_change":"2024-04-24T23:24:14.351480+0000","last_active":"2024-04-24T23:24:16.947871+0000","last_peered":"2024-04-24T23:24:16.947871+0000","last_clean":"2024-04-24T23:24:16.947871+0000","last_became_active":"2024-04-24T23:24:14.351008+0000","last_became_peered":"2024-04-24T23:24:14.351008+0000","last_unstale":"2024-04-24T23:24:16.947871+0000","last_undegraded":"2024-04-24T23:24:16.947871+0000","last_fullsized":"2024-04-24T23:24:16.947871+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:29.638639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947449+0000","last_change":"2024-04-24T23:24:15.029615+0000","last_active":"2024-04-24T23:24:16.947449+0000","last_peered":"2024-04-24T23:24:16.947449+0000","last_clean":"2024-04-24T23:24:16.947449+0000","last_became_active":"2024-04-24T23:24:15.029322+0000","last_became_peered":"2024-04-24T23:24:15.029322+0000","last_unstale":"2024-04-24T23:24:16.947449+0000","last_undegraded":"2024-04-24T23:24:16.947449+0000","last_fullsized":"2024-04-24T23:24:16.947449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:00.629136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340603+0000","last_change":"2024-04-24T23:24:14.353896+0000","last_active":"2024-04-24T23:24:16.340603+0000","last_peered":"2024-04-24T23:24:16.340603+0000","last_clean":"2024-04-24T23:24:16.340603+0000","last_became_active":"2024-04-24T23:24:14.353507+0000","last_became_peered":"2024-04-24T23:24:14.353507+0000","last_unstale":"2024-04-24T23:24:16.340603+0000","last_undegraded":"2024-04-24T23:24:16.340603+0000","last_fullsized":"2024-04-24T23:24:16.340603+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:36.394693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947585+0000","last_change":"2024-04-24T23:24:15.017614+0000","last_active":"2024-04-24T23:24:16.947585+0000","last_peered":"2024-04-24T23:24:16.947585+0000","last_clean":"2024-04-24T23:24:16.947585+0000","last_became_active":"2024-04-24T23:24:15.017246+0000","last_became_peered":"2024-04-24T23:24:15.017246+0000","last_unstale":"2024-04-24T23:24:16.947585+0000","last_undegraded":"2024-04-24T23:24:16.947585+0000","last_fullsized":"2024-04-24T23:24:16.947585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:05.395174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013693+0000","last_change":"2024-04-24T23:24:14.351937+0000","last_active":"2024-04-24T23:24:16.013693+0000","last_peered":"2024-04-24T23:24:16.013693+0000","last_clean":"2024-04-24T23:24:16.013693+0000","last_became_active":"2024-04-24T23:24:14.351663+0000","last_became_peered":"2024-04-24T23:24:14.351663+0000","last_unstale":"2024-04-24T23:24:16.013693+0000","last_undegraded":"2024-04-24T23:24:16.013693+0000","last_fullsized":"2024-04-24T23:24:16.013693+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:59.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340506+0000","last_change":"2024-04-24T23:24:15.023108+0000","last_active":"2024-04-24T23:24:16.340506+0000","last_peered":"2024-04-24T23:24:16.340506+0000","last_clean":"2024-04-24T23:24:16.340506+0000","last_became_active":"2024-04-24T23:24:15.022770+0000","last_became_peered":"2024-04-24T23:24:15.022770+0000","last_unstale":"2024-04-24T23:24:16.340506+0000","last_undegraded":"2024-04-24T23:24:16.340506+0000","last_fullsized":"2024-04-24T23:24:16.340506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:15.488825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742094+0000","last_change":"2024-04-24T23:24:14.348980+0000","last_active":"2024-04-24T23:24:16.742094+0000","last_peered":"2024-04-24T23:24:16.742094+0000","last_clean":"2024-04-24T23:24:16.742094+0000","last_became_active":"2024-04-24T23:24:14.348640+0000","last_became_peered":"2024-04-24T23:24:14.348640+0000","last_unstale":"2024-04-24T23:24:16.742094+0000","last_undegraded":"2024-04-24T23:24:16.742094+0000","last_fullsized":"2024-04-24T23:24:16.742094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:03.381663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340333+0000","last_change":"2024-04-24T23:24:15.019877+0000","last_active":"2024-04-24T23:24:16.340333+0000","last_peered":"2024-04-24T23:24:16.340333+0000","last_clean":"2024-04-24T23:24:16.340333+0000","last_became_active":"2024-04-24T23:24:15.019621+0000","last_became_peered":"2024-04-24T23:24:15.019621+0000","last_unstale":"2024-04-24T23:24:16.340333+0000","last_undegraded":"2024-04-24T23:24:16.340333+0000","last_fullsized":"2024-04-24T23:24:16.340333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:28.322115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.946532+0000","last_change":"2024-04-24T23:24:14.351171+0000","last_active":"2024-04-24T23:24:16.946532+0000","last_peered":"2024-04-24T23:24:16.946532+0000","last_clean":"2024-04-24T23:24:16.946532+0000","last_became_active":"2024-04-24T23:24:14.350864+0000","last_became_peered":"2024-04-24T23:24:14.350864+0000","last_unstale":"2024-04-24T23:24:16.946532+0000","last_undegraded":"2024-04-24T23:24:16.946532+0000","last_fullsized":"2024-04-24T23:24:16.946532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:58.369720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947083+0000","last_change":"2024-04-24T23:24:15.017232+0000","last_active":"2024-04-24T23:24:16.947083+0000","last_peered":"2024-04-24T23:24:16.947083+0000","last_clean":"2024-04-24T23:24:16.947083+0000","last_became_active":"2024-04-24T23:24:15.015671+0000","last_became_peered":"2024-04-24T23:24:15.015671+0000","last_unstale":"2024-04-24T23:24:16.947083+0000","last_undegraded":"2024-04-24T23:24:16.947083+0000","last_fullsized":"2024-04-24T23:24:16.947083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:38.948032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742349+0000","last_change":"2024-04-24T23:24:14.352609+0000","last_active":"2024-04-24T23:24:16.742349+0000","last_peered":"2024-04-24T23:24:16.742349+0000","last_clean":"2024-04-24T23:24:16.742349+0000","last_became_active":"2024-04-24T23:24:14.352333+0000","last_became_peered":"2024-04-24T23:24:14.352333+0000","last_unstale":"2024-04-24T23:24:16.742349+0000","last_undegraded":"2024-04-24T23:24:16.742349+0000","last_fullsized":"2024-04-24T23:24:16.742349+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:04.346009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947641+0000","last_change":"2024-04-24T23:24:15.022268+0000","last_active":"2024-04-24T23:24:16.947641+0000","last_peered":"2024-04-24T23:24:16.947641+0000","last_clean":"2024-04-24T23:24:16.947641+0000","last_became_active":"2024-04-24T23:24:15.021914+0000","last_became_peered":"2024-04-24T23:24:15.021914+0000","last_unstale":"2024-04-24T23:24:16.947641+0000","last_undegraded":"2024-04-24T23:24:16.947641+0000","last_fullsized":"2024-04-24T23:24:16.947641+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:54.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.342426+0000","last_change":"2024-04-24T23:24:14.344848+0000","last_active":"2024-04-24T23:24:16.342426+0000","last_peered":"2024-04-24T23:24:16.342426+0000","last_clean":"2024-04-24T23:24:16.342426+0000","last_became_active":"2024-04-24T23:24:14.342883+0000","last_became_peered":"2024-04-24T23:24:14.342883+0000","last_unstale":"2024-04-24T23:24:16.342426+0000","last_undegraded":"2024-04-24T23:24:16.342426+0000","last_fullsized":"2024-04-24T23:24:16.342426+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:58.369720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742321+0000","last_change":"2024-04-24T23:24:15.017512+0000","last_active":"2024-04-24T23:24:16.742321+0000","last_peered":"2024-04-24T23:24:16.742321+0000","last_clean":"2024-04-24T23:24:16.742321+0000","last_became_active":"2024-04-24T23:24:15.016928+0000","last_became_peered":"2024-04-24T23:24:15.016928+0000","last_unstale":"2024-04-24T23:24:16.742321+0000","last_undegraded":"2024-04-24T23:24:16.742321+0000","last_fullsized":"2024-04-24T23:24:16.742321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:34.764452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014232+0000","last_change":"2024-04-24T23:24:14.346908+0000","last_active":"2024-04-24T23:24:16.014232+0000","last_peered":"2024-04-24T23:24:16.014232+0000","last_clean":"2024-04-24T23:24:16.014232+0000","last_became_active":"2024-04-24T23:24:14.346613+0000","last_became_peered":"2024-04-24T23:24:14.346613+0000","last_unstale":"2024-04-24T23:24:16.014232+0000","last_undegraded":"2024-04-24T23:24:16.014232+0000","last_fullsized":"2024-04-24T23:24:16.014232+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:32:20.621642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.340678+0000","last_change":"2024-04-24T23:24:15.028653+0000","last_active":"2024-04-24T23:24:16.340678+0000","last_peered":"2024-04-24T23:24:16.340678+0000","last_clean":"2024-04-24T23:24:16.340678+0000","last_became_active":"2024-04-24T23:24:15.028351+0000","last_became_peered":"2024-04-24T23:24:15.028351+0000","last_unstale":"2024-04-24T23:24:16.340678+0000","last_undegraded":"2024-04-24T23:24:16.340678+0000","last_fullsized":"2024-04-24T23:24:16.340678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:48.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.948124+0000","last_change":"2024-04-24T23:24:14.342803+0000","last_active":"2024-04-24T23:24:16.948124+0000","last_peered":"2024-04-24T23:24:16.948124+0000","last_clean":"2024-04-24T23:24:16.948124+0000","last_became_active":"2024-04-24T23:24:14.342503+0000","last_became_peered":"2024-04-24T23:24:14.342503+0000","last_unstale":"2024-04-24T23:24:16.948124+0000","last_undegraded":"2024-04-24T23:24:16.948124+0000","last_fullsized":"2024-04-24T23:24:16.948124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:18.514557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013246+0000","last_change":"2024-04-24T23:24:15.025137+0000","last_active":"2024-04-24T23:24:16.013246+0000","last_peered":"2024-04-24T23:24:16.013246+0000","last_clean":"2024-04-24T23:24:16.013246+0000","last_became_active":"2024-04-24T23:24:15.024794+0000","last_became_peered":"2024-04-24T23:24:15.024794+0000","last_unstale":"2024-04-24T23:24:16.013246+0000","last_undegraded":"2024-04-24T23:24:16.013246+0000","last_fullsized":"2024-04-24T23:24:16.013246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:29.794681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.741990+0000","last_change":"2024-04-24T23:24:14.343856+0000","last_active":"2024-04-24T23:24:16.741990+0000","last_peered":"2024-04-24T23:24:16.741990+0000","last_clean":"2024-04-24T23:24:16.741990+0000","last_became_active":"2024-04-24T23:24:14.343447+0000","last_became_peered":"2024-04-24T23:24:14.343447+0000","last_unstale":"2024-04-24T23:24:16.741990+0000","last_undegraded":"2024-04-24T23:24:16.741990+0000","last_fullsized":"2024-04-24T23:24:16.741990+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:32:20.621642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947019+0000","last_change":"2024-04-24T23:24:15.028471+0000","last_active":"2024-04-24T23:24:16.947019+0000","last_peered":"2024-04-24T23:24:16.947019+0000","last_clean":"2024-04-24T23:24:16.947019+0000","last_became_active":"2024-04-24T23:24:15.027874+0000","last_became_peered":"2024-04-24T23:24:15.027874+0000","last_unstale":"2024-04-24T23:24:16.947019+0000","last_undegraded":"2024-04-24T23:24:16.947019+0000","last_fullsized":"2024-04-24T23:24:16.947019+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:54.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947139+0000","last_change":"2024-04-24T23:24:14.349759+0000","last_active":"2024-04-24T23:24:16.947139+0000","last_peered":"2024-04-24T23:24:16.947139+0000","last_clean":"2024-04-24T23:24:16.947139+0000","last_became_active":"2024-04-24T23:24:14.349471+0000","last_became_peered":"2024-04-24T23:24:14.349471+0000","last_unstale":"2024-04-24T23:24:16.947139+0000","last_undegraded":"2024-04-24T23:24:16.947139+0000","last_fullsized":"2024-04-24T23:24:16.947139+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:12.616378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.741994+0000","last_change":"2024-04-24T23:24:15.020528+0000","last_active":"2024-04-24T23:24:16.741994+0000","last_peered":"2024-04-24T23:24:16.741994+0000","last_clean":"2024-04-24T23:24:16.741994+0000","last_became_active":"2024-04-24T23:24:15.019686+0000","last_became_peered":"2024-04-24T23:24:15.019686+0000","last_unstale":"2024-04-24T23:24:16.741994+0000","last_undegraded":"2024-04-24T23:24:16.741994+0000","last_fullsized":"2024-04-24T23:24:16.741994+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:41.978014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.742160+0000","last_change":"2024-04-24T23:24:14.347743+0000","last_active":"2024-04-24T23:24:16.742160+0000","last_peered":"2024-04-24T23:24:16.742160+0000","last_clean":"2024-04-24T23:24:16.742160+0000","last_became_active":"2024-04-24T23:24:14.347461+0000","last_became_peered":"2024-04-24T23:24:14.347461+0000","last_unstale":"2024-04-24T23:24:16.742160+0000","last_undegraded":"2024-04-24T23:24:16.742160+0000","last_fullsized":"2024-04-24T23:24:16.742160+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:08.694848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013101+0000","last_change":"2024-04-24T23:24:15.024467+0000","last_active":"2024-04-24T23:24:16.013101+0000","last_peered":"2024-04-24T23:24:16.013101+0000","last_clean":"2024-04-24T23:24:16.013101+0000","last_became_active":"2024-04-24T23:24:15.023770+0000","last_became_peered":"2024-04-24T23:24:15.023770+0000","last_unstale":"2024-04-24T23:24:16.013101+0000","last_undegraded":"2024-04-24T23:24:16.013101+0000","last_fullsized":"2024-04-24T23:24:16.013101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:56.194872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.013148+0000","last_change":"2024-04-24T23:24:14.342175+0000","last_active":"2024-04-24T23:24:16.013148+0000","last_peered":"2024-04-24T23:24:16.013148+0000","last_clean":"2024-04-24T23:24:16.013148+0000","last_became_active":"2024-04-24T23:24:14.341845+0000","last_became_peered":"2024-04-24T23:24:14.341845+0000","last_unstale":"2024-04-24T23:24:16.013148+0000","last_undegraded":"2024-04-24T23:24:16.013148+0000","last_fullsized":"2024-04-24T23:24:16.013148+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:53.837226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.012994+0000","last_change":"2024-04-24T23:24:15.019470+0000","last_active":"2024-04-24T23:24:16.012994+0000","last_peered":"2024-04-24T23:24:16.012994+0000","last_clean":"2024-04-24T23:24:16.012994+0000","last_became_active":"2024-04-24T23:24:15.019184+0000","last_became_peered":"2024-04-24T23:24:15.019184+0000","last_unstale":"2024-04-24T23:24:16.012994+0000","last_undegraded":"2024-04-24T23:24:16.012994+0000","last_fullsized":"2024-04-24T23:24:16.012994+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:14.327726+0000","last_clean_scrub_stamp":"2024-04-24T23:24:14.327726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:49.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.014235+0000","last_change":"2024-04-24T23:24:14.343131+0000","last_active":"2024-04-24T23:24:16.014235+0000","last_peered":"2024-04-24T23:24:16.014235+0000","last_clean":"2024-04-24T23:24:16.014235+0000","last_became_active":"2024-04-24T23:24:14.342828+0000","last_became_peered":"2024-04-24T23:24:14.342828+0000","last_unstale":"2024-04-24T23:24:16.014235+0000","last_undegraded":"2024-04-24T23:24:16.014235+0000","last_fullsized":"2024-04-24T23:24:16.014235+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:24:13.325512+0000","last_clean_scrub_stamp":"2024-04-24T23:24:13.325512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:03.381663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-24T23:24:16.947610+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:24:16.947610+0000","last_peered":"2024-04-24T23:24:16.947610+0000","last_clean":"2024-04-24T23:24:16.947610+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:24:16.947610+0000","last_undegraded":"2024-04-24T23:24:16.947610+0000","last_fullsized":"2024-04-24T23:24:16.947610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":877209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":696320,"data_stored":292981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":287333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":696320,"data_stored":288293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":293037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36024,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335816,"statfs":{"total":96636764160,"available":96599875584,"internally_reserved":0,"allocated":692224,"data_stored":286737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":700416,"data_stored":293071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1306624,"data_stored":885543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:24:19.374 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:24:19.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:24:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.518+0000 7fee88363640 1 Processor -- start 2024-04-24T23:24:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.518+0000 7fee88363640 1 -- start start 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.519+0000 7fee88363640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee800ad7b0 0x7fee800d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.519+0000 7fee88363640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 0x7fee800d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.519+0000 7fee88363640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee8015ef60 con 0x7fee8015adc0 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.519+0000 7fee88363640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fee8015f240 con 0x7fee800ad7b0 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.519+0000 7fee88363640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fee8015f520 con 0x7fee8015e9c0 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.519+0000 7fee858fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee800ad7b0 0x7fee800d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.519+0000 7fee858fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee800ad7b0 0x7fee800d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52234/0 (socket says 172.21.15.26:52234) 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.519+0000 7fee858fe640 1 -- 172.21.15.26:0/1317549978 learned_addr learned my addr 172.21.15.26:0/1317549978 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.520+0000 7fee86900640 1 --2- 172.21.15.26:0/1317549978 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 0x7fee800d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.520+0000 7fee858fe640 1 -- 172.21.15.26:0/1317549978 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 msgr2=0x7fee800d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.520+0000 7fee858fe640 1 --2- 172.21.15.26:0/1317549978 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 0x7fee800d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.520+0000 7fee858fe640 1 -- 172.21.15.26:0/1317549978 >> v1:172.21.15.29:6789/0 conn(0x7fee8015adc0 legacy=0x7fee800b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.520+0000 7fee858fe640 1 -- 172.21.15.26:0/1317549978 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8015f800 con 0x7fee800ad7b0 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.520+0000 7fee858fe640 1 --2- 172.21.15.26:0/1317549978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee800ad7b0 0x7fee800d0040 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fee7400bb40 tx=0x7fee7402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13a4a478edfd8be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee850fd640 1 -- 172.21.15.26:0/1317549978 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74032790 con 0x7fee800ad7b0 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee850fd640 1 -- 172.21.15.26:0/1317549978 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee74038070 con 0x7fee800ad7b0 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee850fd640 1 -- 172.21.15.26:0/1317549978 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74032b50 con 0x7fee800ad7b0 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee88363640 1 -- 172.21.15.26:0/1317549978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee800ad7b0 msgr2=0x7fee800d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee88363640 1 --2- 172.21.15.26:0/1317549978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee800ad7b0 0x7fee800d0040 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fee7400bb40 tx=0x7fee7402c8d0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee88363640 1 -- 172.21.15.26:0/1317549978 shutdown_connections 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee88363640 1 --2- 172.21.15.26:0/1317549978 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 0x7fee800d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee88363640 1 --2- 172.21.15.26:0/1317549978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee800ad7b0 0x7fee800d0040 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee88363640 1 -- 172.21.15.26:0/1317549978 >> 172.21.15.26:0/1317549978 conn(0x7fee8006b540 msgr2=0x7fee8006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee88363640 1 -- 172.21.15.26:0/1317549978 shutdown_connections 2024-04-24T23:24:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.521+0000 7fee88363640 1 -- 172.21.15.26:0/1317549978 wait complete. 2024-04-24T23:24:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.522+0000 7fee88363640 1 Processor -- start 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.522+0000 7fee88363640 1 -- start start 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee88363640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee800ad7b0 0x7fee800cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee88363640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee8015adc0 0x7fee80154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee88363640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 0x7fee8006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee88363640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee8006cb90 con 0x7fee8015adc0 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee88363640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fee8015b1a0 con 0x7fee800ad7b0 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee88363640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fee8006cd10 con 0x7fee8015e9c0 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee858fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee8015adc0 0x7fee80154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee858fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee8015adc0 0x7fee80154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52250/0 (socket says 172.21.15.26:52250) 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee858fe640 1 -- 172.21.15.26:0/2316481426 learned_addr learned my addr 172.21.15.26:0/2316481426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee860ff640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee800ad7b0 0x7fee800cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.523+0000 7fee86900640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 0x7fee8006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee858fe640 1 -- 172.21.15.26:0/2316481426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 msgr2=0x7fee8006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee858fe640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 0x7fee8006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee858fe640 1 -- 172.21.15.26:0/2316481426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee800ad7b0 msgr2=0x7fee800cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee858fe640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee800ad7b0 0x7fee800cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee858fe640 1 -- 172.21.15.26:0/2316481426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee801547a0 con 0x7fee8015adc0 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee86900640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 0x7fee8006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee858fe640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee8015adc0 0x7fee80154260 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fee7400bc60 tx=0x7fee74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee7effd640 1 -- 172.21.15.26:0/2316481426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74043070 con 0x7fee8015adc0 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee7effd640 1 -- 172.21.15.26:0/2316481426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7403c640 con 0x7fee8015adc0 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee7effd640 1 -- 172.21.15.26:0/2316481426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7403ca30 con 0x7fee8015adc0 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8009cd40 con 0x7fee8015adc0 2024-04-24T23:24:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.524+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8009d300 con 0x7fee8015adc0 2024-04-24T23:24:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.525+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee80075610 con 0x7fee8015adc0 2024-04-24T23:24:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.525+0000 7fee7effd640 1 -- 172.21.15.26:0/2316481426 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee74046070 con 0x7fee8015adc0 2024-04-24T23:24:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.526+0000 7fee7effd640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee6c06be30 0x7fee6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.526+0000 7fee7effd640 1 -- 172.21.15.26:0/2316481426 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fee740ab3c0 con 0x7fee8015adc0 2024-04-24T23:24:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.526+0000 7fee860ff640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee6c06be30 0x7fee6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.526+0000 7fee860ff640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee6c06be30 0x7fee6c06e2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fee78001780 tx=0x7fee78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.530+0000 7fee7effd640 1 -- 172.21.15.26:0/2316481426 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fee74079720 con 0x7fee8015adc0 2024-04-24T23:24:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.789+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fee8015e730 con 0x7fee8015adc0 2024-04-24T23:24:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.791+0000 7fee7effd640 1 -- 172.21.15.26:0/2316481426 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fee7407da20 con 0x7fee8015adc0 2024-04-24T23:24:19.792 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-24T23:24:19.792 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ== 2024-04-24T23:24:19.792 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-04-24T23:24:19.793 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-04-24T23:24:19.793 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:24:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-04-24T23:24:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee6c06be30 msgr2=0x7fee6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee6c06be30 0x7fee6c06e2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fee78001780 tx=0x7fee78009040 comp rx=0 tx=0).stop 2024-04-24T23:24:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee8015adc0 msgr2=0x7fee80154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee8015adc0 0x7fee80154260 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fee7400bc60 tx=0x7fee74004730 comp rx=0 tx=0).stop 2024-04-24T23:24:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 shutdown_connections 2024-04-24T23:24:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fee6c06be30 0x7fee6c06e2f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fee8015e9c0 0x7fee8006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee8015adc0 0x7fee80154260 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 --2- 172.21.15.26:0/2316481426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee800ad7b0 0x7fee800cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.793+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 >> 172.21.15.26:0/2316481426 conn(0x7fee8006b540 msgr2=0x7fee80158440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.794+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 shutdown_connections 2024-04-24T23:24:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.794+0000 7fee88363640 1 -- 172.21.15.26:0/2316481426 wait complete. 2024-04-24T23:24:19.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:19.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.905+0000 7f78c6c58640 1 Processor -- start 2024-04-24T23:24:19.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.906+0000 7f78c6c58640 1 -- start start 2024-04-24T23:24:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.906+0000 7f78c6c58640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c015e570 0x7f78c00d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.906+0000 7f78c6c58640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c00d32f0 con 0x7f78c00d22e0 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.906+0000 7f78c6c58640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c00d1600 con 0x7f78c00b3130 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.906+0000 7f78c6c58640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00d1ab0 con 0x7f78c015e570 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.906+0000 7f78c51f5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f78c00d22e0 0x7f78c00d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58852/0 (socket says 172.21.15.26:58852) 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78c51f5640 1 -- 172.21.15.26:0/3237346292 learned_addr learned my addr 172.21.15.26:0/3237346292 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78c49f4640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c015e570 0x7f78c00d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1023604844 0 0) 0x7f78b8001490 con 0x7f78c00d22e0 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78a8003520 con 0x7f78c00d22e0 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 335113253 0 0) 0x7f78a8003520 con 0x7f78c00d22e0 2024-04-24T23:24:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c015e570 msgr2=0x7f78c00d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c015e570 0x7f78c00d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 >> v1:172.21.15.29:6789/0 conn(0x7f78c00b3130 legacy=0x7f78c00d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c00d0270 con 0x7f78c00d22e0 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f78b8001bc0 con 0x7f78c00d22e0 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 >> v1:172.21.15.26:6789/0 conn(0x7f78c00d22e0 legacy=0x7f78c00d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78a8004800 0x7f78a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c015e570 0x7f78a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00d1ab0 con 0x7f78a80039e0 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f78a800ba70 con 0x7f78a8004800 2024-04-24T23:24:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f78a800bc70 con 0x7f78c015e570 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78c49f4640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78bffff640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78a8004800 0x7f78a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.907+0000 7f78c51f5640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c015e570 0x7f78a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78c49f4640 1 -- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c015e570 msgr2=0x7f78a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78c49f4640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c015e570 0x7f78a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78c49f4640 1 -- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78a8004800 msgr2=0x7f78a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78c49f4640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78a8004800 0x7f78a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78c49f4640 1 -- 172.21.15.26:0/3237346292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a8003520 con 0x7f78a80039e0 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78c51f5640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c015e570 0x7f78a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78c49f4640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78a8003e00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009a90 tx=0x7f78a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5b94175941350f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0036070 con 0x7f78a80039e0 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b0031400 con 0x7f78a80039e0 2024-04-24T23:24:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.908+0000 7f78bf7fe640 1 -- 172.21.15.26:0/3237346292 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0009690 con 0x7f78a80039e0 2024-04-24T23:24:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.909+0000 7f78c6c58640 1 -- 172.21.15.26:0/3237346292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 msgr2=0x7f78a8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.909+0000 7f78c6c58640 1 --2- 172.21.15.26:0/3237346292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78a8003e00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009a90 tx=0x7f78a8005830 comp rx=0 tx=0).stop 2024-04-24T23:24:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.909+0000 7f78c51f5640 1 -- 172.21.15.26:0/3237346292 reap_dead start 2024-04-24T23:24:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.909+0000 7f78c6c58640 1 -- 172.21.15.26:0/3237346292 shutdown_connections 2024-04-24T23:24:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.909+0000 7f78c6c58640 1 -- 172.21.15.26:0/3237346292 >> 172.21.15.26:0/3237346292 conn(0x7f78c008dbe0 msgr2=0x7f78c009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.909+0000 7f78c6c58640 1 -- 172.21.15.26:0/3237346292 shutdown_connections 2024-04-24T23:24:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.909+0000 7f78c6c58640 1 -- 172.21.15.26:0/3237346292 wait complete. 2024-04-24T23:24:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.910+0000 7f78c6c58640 1 Processor -- start 2024-04-24T23:24:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.910+0000 7f78c6c58640 1 -- start start 2024-04-24T23:24:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c6c58640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78c00b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c6c58640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c0158bd0 0x7f78c0157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c6c58640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78c0158610 0x7f78c00cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c6c58640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00d3010 con 0x7f78a80039e0 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c6c58640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00d2ae0 con 0x7f78c0158610 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c6c58640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00cfa70 con 0x7f78c0158bd0 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c49f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78c00b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c49f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78c00b8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52276/0 (socket says 172.21.15.26:52276) 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c49f4640 1 -- 172.21.15.26:0/1449401333 learned_addr learned my addr 172.21.15.26:0/1449401333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c51f5640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78c0158610 0x7f78c00cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c49f4640 1 -- 172.21.15.26:0/1449401333 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c0158bd0 msgr2=0x7f78c0157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c49f4640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c0158bd0 0x7f78c0157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c49f4640 1 -- 172.21.15.26:0/1449401333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78c0158610 msgr2=0x7f78c00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c49f4640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78c0158610 0x7f78c00cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.911+0000 7f78c49f4640 1 -- 172.21.15.26:0/1449401333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c00b7190 con 0x7f78a80039e0 2024-04-24T23:24:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.912+0000 7f78c49f4640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78c00b8230 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009bb0 tx=0x7f78b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.912+0000 7f78bd7fa640 1 -- 172.21.15.26:0/1449401333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0040020 con 0x7f78a80039e0 2024-04-24T23:24:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.912+0000 7f78bd7fa640 1 -- 172.21.15.26:0/1449401333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b0003bc0 con 0x7f78a80039e0 2024-04-24T23:24:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.912+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c00b7470 con 0x7f78a80039e0 2024-04-24T23:24:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.912+0000 7f78bd7fa640 1 -- 172.21.15.26:0/1449401333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0036040 con 0x7f78a80039e0 2024-04-24T23:24:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.912+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c00b7a30 con 0x7f78a80039e0 2024-04-24T23:24:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.913+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c00694c0 con 0x7f78a80039e0 2024-04-24T23:24:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.913+0000 7f78bd7fa640 1 -- 172.21.15.26:0/1449401333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78b0043020 con 0x7f78a80039e0 2024-04-24T23:24:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.914+0000 7f78bd7fa640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f78ac06be30 0x7f78ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.914+0000 7f78bd7fa640 1 -- 172.21.15.26:0/1449401333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f78b00ab140 con 0x7f78a80039e0 2024-04-24T23:24:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.914+0000 7f78bffff640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f78ac06be30 0x7f78ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.914+0000 7f78bffff640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f78ac06be30 0x7f78ac06e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f78c00b2930 tx=0x7f78b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:19.918+0000 7f78bd7fa640 1 -- 172.21.15.26:0/1449401333 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f78b004a030 con 0x7f78a80039e0 2024-04-24T23:24:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.143+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78c00b2680 con 0x7f78a80039e0 2024-04-24T23:24:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.143+0000 7f78bd7fa640 1 -- 172.21.15.26:0/1449401333 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5181 (secure 0 0 0) 0x7f78b00044e0 con 0x7f78a80039e0 2024-04-24T23:24:20.145 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:20.145 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1519104007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1519104007},{"type":"v1","addr":"172.21.15.26:6836","nonce":1519104007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2743842996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2743842996},{"type":"v1","addr":"172.21.15.26:6837","nonce":2743842996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:24:14.713249+0000","modified":"2024-04-24T23:24:17.372849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2096523178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2096523178},{"type":"v1","addr":"172.21.15.29:6835","nonce":2096523178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":4}]} 2024-04-24T23:24:20.145 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2024-04-24T23:24:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.145+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f78ac06be30 msgr2=0x7f78ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.145+0000 7f78c6c58640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f78ac06be30 0x7f78ac06e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f78c00b2930 tx=0x7f78b4009040 comp rx=0 tx=0).stop 2024-04-24T23:24:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.145+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 msgr2=0x7f78c00b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.145+0000 7f78c6c58640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78c00b8230 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009bb0 tx=0x7f78b0004730 comp rx=0 tx=0).stop 2024-04-24T23:24:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.145+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 shutdown_connections 2024-04-24T23:24:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.145+0000 7f78c6c58640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f78ac06be30 0x7f78ac06e2f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.145+0000 7f78c6c58640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78c0158610 0x7f78c00cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.145+0000 7f78c6c58640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f78c0158bd0 0x7f78c0157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.145+0000 7f78c6c58640 1 --2- 172.21.15.26:0/1449401333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78a80039e0 0x7f78c00b8230 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.146+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 >> 172.21.15.26:0/1449401333 conn(0x7f78c008dbe0 msgr2=0x7f78c00b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.146+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 shutdown_connections 2024-04-24T23:24:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.146+0000 7f78c6c58640 1 -- 172.21.15.26:0/1449401333 wait complete. 2024-04-24T23:24:20.560 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:24:14.713249+0000', 'modified': '2024-04-24T23:24:17.372849+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6546}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6546': {'gid': 6546, 'name': 'd', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/2096523178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 2096523178}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 2096523178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6546, 'qdb_cluster': [6546]} 2024-04-24T23:24:20.560 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:24:20.561 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:24:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.668+0000 7f384de1c640 1 Processor -- start 2024-04-24T23:24:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.668+0000 7f384de1c640 1 -- start start 2024-04-24T23:24:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f384de1c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f384806d350 0x7f384806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f384de1c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38480cd660 0x7f38480cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f384de1c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38480cfbc0 con 0x7f3848145910 2024-04-24T23:24:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f384de1c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3848145cf0 con 0x7f384806d350 2024-04-24T23:24:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f384de1c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f384806d7f0 con 0x7f38480cd660 2024-04-24T23:24:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f3846ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f384806d350 0x7f384806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f3846ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f384806d350 0x7f384806e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52282/0 (socket says 172.21.15.26:52282) 2024-04-24T23:24:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f38477fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3848145910 0x7f384815d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58868/0 (socket says 172.21.15.26:58868) 2024-04-24T23:24:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f3846ffd640 1 -- 172.21.15.26:0/658290861 learned_addr learned my addr 172.21.15.26:0/658290861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f3847fff640 1 --2- 172.21.15.26:0/658290861 >> v2:172.21.15.29:3300/0 conn(0x7f38480cd660 0x7f38480cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f3846ffd640 1 -- 172.21.15.26:0/658290861 >> v2:172.21.15.29:3300/0 conn(0x7f38480cd660 msgr2=0x7f38480cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f3846ffd640 1 --2- 172.21.15.26:0/658290861 >> v2:172.21.15.29:3300/0 conn(0x7f38480cd660 0x7f38480cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f3846ffd640 1 -- 172.21.15.26:0/658290861 >> v1:172.21.15.26:6789/0 conn(0x7f3848145910 legacy=0x7f384815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f3846ffd640 1 -- 172.21.15.26:0/658290861 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384806dad0 con 0x7f384806d350 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f38467fc640 1 -- 172.21.15.26:0/658290861 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4175916329 0 0) 0x7f38480cfbc0 con 0x7f3848145910 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.669+0000 7f3847fff640 1 --2- 172.21.15.26:0/658290861 >> v2:172.21.15.29:3300/0 conn(0x7f38480cd660 0x7f38480cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.670+0000 7f3846ffd640 1 --2- 172.21.15.26:0/658290861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 0x7f384806e890 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f383c009870 tx=0x7f383c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7979fc9c8988df96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.670+0000 7f38467fc640 1 -- 172.21.15.26:0/658290861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c037070 con 0x7f384806d350 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.670+0000 7f38467fc640 1 -- 172.21.15.26:0/658290861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c032440 con 0x7f384806d350 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.670+0000 7f38467fc640 1 -- 172.21.15.26:0/658290861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c032810 con 0x7f384806d350 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.671+0000 7f384de1c640 1 -- 172.21.15.26:0/658290861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 msgr2=0x7f384806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.671+0000 7f384de1c640 1 --2- 172.21.15.26:0/658290861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 0x7f384806e890 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f383c009870 tx=0x7f383c02a210 comp rx=0 tx=0).stop 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.671+0000 7f384de1c640 1 -- 172.21.15.26:0/658290861 shutdown_connections 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.671+0000 7f384de1c640 1 --2- 172.21.15.26:0/658290861 >> v2:172.21.15.29:3300/0 conn(0x7f38480cd660 0x7f38480cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.671+0000 7f384de1c640 1 --2- 172.21.15.26:0/658290861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 0x7f384806e890 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.671+0000 7f384de1c640 1 -- 172.21.15.26:0/658290861 >> 172.21.15.26:0/658290861 conn(0x7f384806b300 msgr2=0x7f384806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.671+0000 7f384de1c640 1 -- 172.21.15.26:0/658290861 shutdown_connections 2024-04-24T23:24:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.671+0000 7f384de1c640 1 -- 172.21.15.26:0/658290861 wait complete. 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.672+0000 7f384de1c640 1 Processor -- start 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.672+0000 7f384de1c640 1 -- start start 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f384de1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 0x7f384813ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f384de1c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38480cd660 0x7f38481586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f384de1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3848145910 0x7f384813c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f384de1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f384806e370 con 0x7f384806d350 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f384de1c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f384806e090 con 0x7f3848145910 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f384de1c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f384815d770 con 0x7f38480cd660 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f38477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 0x7f384813ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f38477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 0x7f384813ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52284/0 (socket says 172.21.15.26:52284) 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f38477fe640 1 -- 172.21.15.26:0/791015971 learned_addr learned my addr 172.21.15.26:0/791015971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:20.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f3847fff640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3848145910 0x7f384813c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f3846ffd640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38480cd660 0x7f38481586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f38477fe640 1 -- 172.21.15.26:0/791015971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38480cd660 msgr2=0x7f38481586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f38477fe640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38480cd660 0x7f38481586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f38477fe640 1 -- 172.21.15.26:0/791015971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3848145910 msgr2=0x7f384813c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f38477fe640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3848145910 0x7f384813c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f38477fe640 1 -- 172.21.15.26:0/791015971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384813f5c0 con 0x7f384806d350 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f3846ffd640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38480cd660 0x7f38481586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.673+0000 7f38477fe640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 0x7f384813ee70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f383800c460 tx=0x7f383800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.674+0000 7f382ffff640 1 -- 172.21.15.26:0/791015971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838018070 con 0x7f384806d350 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.674+0000 7f382ffff640 1 -- 172.21.15.26:0/791015971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3838007500 con 0x7f384806d350 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.674+0000 7f384de1c640 1 -- 172.21.15.26:0/791015971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384813f8a0 con 0x7f384806d350 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.674+0000 7f382ffff640 1 -- 172.21.15.26:0/791015971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838004a20 con 0x7f384806d350 2024-04-24T23:24:20.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.674+0000 7f384de1c640 1 -- 172.21.15.26:0/791015971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384813fe60 con 0x7f384806d350 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.675+0000 7f382ffff640 1 -- 172.21.15.26:0/791015971 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f383801b070 con 0x7f384806d350 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.676+0000 7f382ffff640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f383406be30 0x7f383406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.676+0000 7f382ffff640 1 -- 172.21.15.26:0/791015971 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f383808a110 con 0x7f384806d350 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.676+0000 7f3846ffd640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f383406be30 0x7f383406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.676+0000 7f3846ffd640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f383406be30 0x7f383406e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f383c034000 tx=0x7f383c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.677+0000 7f3850c06740 1 Processor -- start 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.677+0000 7f3850c06740 1 -- start start 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.677+0000 7f3850c06740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c5594334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.677+0000 7f3850c06740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559463000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.677+0000 7f3850c06740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463540 0x55c559438350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3850c06740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c55939bf00 con 0x55c559463c80 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3850c06740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c559344750 con 0x55c559480b60 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3850c06740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c559397ec0 con 0x55c559463540 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3846ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559463000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3846ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559463000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52298/0 (socket says 172.21.15.26:52298) 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3846ffd640 1 -- 172.21.15.26:0/1688370683 learned_addr learned my addr 172.21.15.26:0/1688370683 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3847fff640 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463540 0x55c559438350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3846ffd640 1 -- 172.21.15.26:0/1688370683 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463540 msgr2=0x55c559438350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3846ffd640 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463540 0x55c559438350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f38477fe640 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c5594334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3846ffd640 1 -- 172.21.15.26:0/1688370683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 msgr2=0x55c5594334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3846ffd640 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c5594334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3846ffd640 1 -- 172.21.15.26:0/1688370683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5590b02d0 con 0x55c559463c80 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3847fff640 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463540 0x55c559438350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f3846ffd640 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559463000 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f383c009960 tx=0x7f383c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479c9034d10ddcba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f382d7fa640 1 -- 172.21.15.26:0/1688370683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c04b070 con 0x55c559463c80 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f382d7fa640 1 -- 172.21.15.26:0/1688370683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c03c460 con 0x55c559463c80 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.678+0000 7f382d7fa640 1 -- 172.21.15.26:0/1688370683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c037040 con 0x55c559463c80 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.679+0000 7f3850c06740 1 -- 172.21.15.26:0/1688370683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 msgr2=0x55c559463000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.679+0000 7f3850c06740 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559463000 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f383c009960 tx=0x7f383c02c000 comp rx=0 tx=0).stop 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.679+0000 7f3850c06740 1 -- 172.21.15.26:0/1688370683 shutdown_connections 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.679+0000 7f3850c06740 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463540 0x55c559438350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.679+0000 7f3850c06740 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559463000 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.679+0000 7f3850c06740 1 --2- 172.21.15.26:0/1688370683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c5594334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.679+0000 7f3850c06740 1 -- 172.21.15.26:0/1688370683 >> 172.21.15.26:0/1688370683 conn(0x55c559435760 msgr2=0x55c55947e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.679+0000 7f3850c06740 1 -- 172.21.15.26:0/1688370683 shutdown_connections 2024-04-24T23:24:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.679+0000 7f3850c06740 1 -- 172.21.15.26:0/1688370683 wait complete. 2024-04-24T23:24:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.680+0000 7f3850c06740 1 -- start start 2024-04-24T23:24:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.680+0000 7f3850c06740 1 Processor -- start 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3850c06740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559474da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3850c06740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c55945eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3850c06740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 0x55c559466900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3850c06740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c559437910 con 0x55c559463c80 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3850c06740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c559357cf0 con 0x55c559480b60 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3850c06740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c559436d00 con 0x55c559466480 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3850c06740 10 client.0 _ll_register_callbacks cb 0x55c5593576e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559474da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559474da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52308/0 (socket says 172.21.15.26:52308) 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38477fe640 1 -- 172.21.15.26:0/627138320 learned_addr learned my addr 172.21.15.26:0/627138320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3847fff640 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 0x55c559466900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3846ffd640 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c55945eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38477fe640 1 -- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 msgr2=0x55c559466900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38477fe640 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 0x55c559466900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38477fe640 1 -- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 msgr2=0x55c55945eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38477fe640 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c55945eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38477fe640 1 -- 172.21.15.26:0/627138320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c55917a6b0 con 0x55c559463c80 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3846ffd640 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c55945eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38477fe640 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559474da0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f383800ce20 tx=0x7f3838058ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e36504406c766e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38267fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38267fc640 1 -- 172.21.15.26:0/627138320 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838004dc0 con 0x55c559463c80 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f38267fc640 1 -- 172.21.15.26:0/627138320 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3838057810 con 0x55c559463c80 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.681+0000 7f3850c06740 1 -- 172.21.15.26:0/627138320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383c03c460 con 0x55c559463c80 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f38267fc640 1 -- 172.21.15.26:0/627138320 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3838060fa0 con 0x55c559463c80 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f3850c06740 10 client.6693 fetch_fsmap learned FSMap version 49 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f3850c06740 1 -- 172.21.15.26:0/627138320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55c5594614f0 con 0x55c559463c80 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f38267fc640 1 -- 172.21.15.26:0/627138320 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c03c460 con 0x55c559463c80 2024-04-24T23:24:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f38267fc640 1 -- 172.21.15.26:0/627138320 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f3838070170 con 0x55c559463c80 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f3850c06740 10 client.6693 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f3850c06740 10 client.6693 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f3850c06740 20 client.6693 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f3850c06740 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55c5594919e0 0x55c559491dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f3850c06740 4 client.6693 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f3850c06740 1 -- 172.21.15.26:0/627138320 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5593a9440 con 0x55c5594919e0 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.682+0000 7f3846ffd640 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55c5594919e0 0x55c559491dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.683+0000 7f38267fc640 10 client.6693 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.683+0000 7f3846ffd640 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55c5594919e0 0x55c559491dc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1ac8ae086f3b96c server_cookie=a0f4f4ac60057002 in_seq=0 out_seq=0 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.683+0000 7f38267fc640 1 -- 172.21.15.26:0/627138320 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5593a9440 con 0x55c5594919e0 2024-04-24T23:24:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.683+0000 7f38267fc640 10 client.6693 handle_command_reply: tid=0 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.683+0000 7f3850c06740 1 client.6693 shutdown 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.683+0000 7f3850c06740 1 -- 172.21.15.26:0/627138320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 msgr2=0x55c559474da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.683+0000 7f3850c06740 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559474da0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f383800ce20 tx=0x7f3838058ac0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f3850c06740 1 -- 172.21.15.26:0/627138320 shutdown_connections 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f3850c06740 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55c5594919e0 0x55c559491dc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f3850c06740 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 0x55c559466900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f3850c06740 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c55945eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f3850c06740 1 --2- 172.21.15.26:0/627138320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c559474da0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f38267fc640 0 client.6693 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f3850c06740 1 -- 172.21.15.26:0/627138320 >> 172.21.15.26:0/627138320 conn(0x55c559463540 msgr2=0x55c55947e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f3850c06740 1 -- 172.21.15.26:0/627138320 shutdown_connections 2024-04-24T23:24:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f3850c06740 1 -- 172.21.15.26:0/627138320 wait complete. 2024-04-24T23:24:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.684+0000 7f3850c06740 20 client.6693 trim_cache size 0 max 16384 2024-04-24T23:24:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.702+0000 7f3850c06740 1 Processor -- start 2024-04-24T23:24:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.702+0000 7f3850c06740 1 -- start start 2024-04-24T23:24:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3850c06740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463c80 0x55c55947b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3850c06740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559466480 0x55c5594646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3850c06740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c559464c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3850c06740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38481fe160 con 0x55c559466480 2024-04-24T23:24:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3850c06740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38481fd1f0 con 0x55c559480b60 2024-04-24T23:24:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3850c06740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3848142a90 con 0x55c559463c80 2024-04-24T23:24:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3846ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559466480 0x55c5594646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3846ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559466480 0x55c5594646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52324/0 (socket says 172.21.15.26:52324) 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3846ffd640 1 -- 172.21.15.26:0/1789371874 learned_addr learned my addr 172.21.15.26:0/1789371874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3847fff640 1 --2- 172.21.15.26:0/1789371874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c559464c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f38477fe640 1 --2- 172.21.15.26:0/1789371874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463c80 0x55c55947b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3846ffd640 1 -- 172.21.15.26:0/1789371874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463c80 msgr2=0x55c55947b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3846ffd640 1 --2- 172.21.15.26:0/1789371874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463c80 0x55c55947b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3846ffd640 1 -- 172.21.15.26:0/1789371874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 msgr2=0x55c559464c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3846ffd640 1 --2- 172.21.15.26:0/1789371874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c559464c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.703+0000 7f3846ffd640 1 -- 172.21.15.26:0/1789371874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c559464450 con 0x55c559466480 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3846ffd640 1 --2- 172.21.15.26:0/1789371874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559466480 0x55c5594646f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f383c049040 tx=0x7f383c03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd47e5070111daa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f382cff9640 1 -- 172.21.15.26:0/1789371874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c055070 con 0x55c559466480 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f382cff9640 1 -- 172.21.15.26:0/1789371874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c0503a0 con 0x55c559466480 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f382cff9640 1 -- 172.21.15.26:0/1789371874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c04b070 con 0x55c559466480 2024-04-24T23:24:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3850c06740 1 -- 172.21.15.26:0/1789371874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559466480 msgr2=0x55c5594646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3850c06740 1 --2- 172.21.15.26:0/1789371874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559466480 0x55c5594646f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f383c049040 tx=0x7f383c03a000 comp rx=0 tx=0).stop 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3850c06740 1 -- 172.21.15.26:0/1789371874 shutdown_connections 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3850c06740 1 --2- 172.21.15.26:0/1789371874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c559464c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3850c06740 1 --2- 172.21.15.26:0/1789371874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559466480 0x55c5594646f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3850c06740 1 --2- 172.21.15.26:0/1789371874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559463c80 0x55c55947b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3850c06740 1 -- 172.21.15.26:0/1789371874 >> 172.21.15.26:0/1789371874 conn(0x55c559463540 msgr2=0x55c55942e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3850c06740 1 -- 172.21.15.26:0/1789371874 shutdown_connections 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.704+0000 7f3850c06740 1 -- 172.21.15.26:0/1789371874 wait complete. 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.705+0000 7f3850c06740 1 -- start start 2024-04-24T23:24:20.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.705+0000 7f3850c06740 1 Processor -- start 2024-04-24T23:24:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3850c06740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c5594f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3850c06740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 0x55c55947b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3850c06740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c5594646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3850c06740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c559476bc0 con 0x55c559463c80 2024-04-24T23:24:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3850c06740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38481430e0 con 0x55c559480b60 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3850c06740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c559462210 con 0x55c559466480 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3850c06740 10 client.0 _ll_register_callbacks cb 0x7f3838057fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c5594f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c5594f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52330/0 (socket says 172.21.15.26:52330) 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38477fe640 1 -- 172.21.15.26:0/1770486845 learned_addr learned my addr 172.21.15.26:0/1770486845 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3847fff640 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c5594646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3846ffd640 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 0x55c55947b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38477fe640 1 -- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 msgr2=0x55c55947b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38477fe640 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 0x55c55947b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38477fe640 1 -- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 msgr2=0x55c5594646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38477fe640 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c5594646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38477fe640 1 -- 172.21.15.26:0/1770486845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5594a2320 con 0x55c559463c80 2024-04-24T23:24:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f3847fff640 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c5594646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38477fe640 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c5594f2de0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3838058c70 tx=0x7f3838088510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790f3fcf4c1164e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38267fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38267fc640 1 -- 172.21.15.26:0/1770486845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838004bc0 con 0x55c559463c80 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.706+0000 7f38267fc640 1 -- 172.21.15.26:0/1770486845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3838004d60 con 0x55c559463c80 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3850c06740 1 -- 172.21.15.26:0/1770486845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383c0503a0 con 0x55c559463c80 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f38267fc640 1 -- 172.21.15.26:0/1770486845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3838088320 con 0x55c559463c80 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3850c06740 10 client.6699 fetch_fsmap learned FSMap version 49 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3850c06740 1 -- 172.21.15.26:0/1770486845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55c5594b83e0 con 0x55c559463c80 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f38267fc640 1 -- 172.21.15.26:0/1770486845 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c0503a0 con 0x55c559463c80 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f38267fc640 1 -- 172.21.15.26:0/1770486845 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f3838054110 con 0x55c559463c80 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3850c06740 10 client.6699 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3850c06740 10 client.6699 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:24:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3850c06740 20 client.6699 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3850c06740 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55c559464de0 0x55c5594ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3850c06740 4 client.6699 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3850c06740 1 -- 172.21.15.26:0/1770486845 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "status"}) -- 0x55c559434bb0 con 0x55c559464de0 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.707+0000 7f3846ffd640 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55c559464de0 0x55c5594ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f38267fc640 10 client.6699 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3846ffd640 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55c559464de0 0x55c5594ffa20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c560e3db229e4941 server_cookie=a778583f7df4b9d4 in_seq=0 out_seq=0 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f38267fc640 1 -- 172.21.15.26:0/1770486845 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55c559434bb0 con 0x55c559464de0 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f38267fc640 10 client.6699 handle_command_reply: tid=0 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 client.6699 shutdown 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 -- 172.21.15.26:0/1770486845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 msgr2=0x55c5594f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c5594f2de0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3838058c70 tx=0x7f3838088510 comp rx=0 tx=0).stop 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 -- 172.21.15.26:0/1770486845 shutdown_connections 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55c559464de0 0x55c5594ffa20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c559480b60 0x55c5594646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c559466480 0x55c55947b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 --2- 172.21.15.26:0/1770486845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c559463c80 0x55c5594f2de0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f38267fc640 0 client.6699 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 -- 172.21.15.26:0/1770486845 >> 172.21.15.26:0/1770486845 conn(0x55c559463540 msgr2=0x55c559431f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 -- 172.21.15.26:0/1770486845 shutdown_connections 2024-04-24T23:24:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 1 -- 172.21.15.26:0/1770486845 wait complete. 2024-04-24T23:24:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f3850c06740 20 client.6699 trim_cache size 0 max 16384 2024-04-24T23:24:20.716 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:20.716 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:24:20.716 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6546, 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.3745143149999999, 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 49, 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 58, 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 58, 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.658081138 2024-04-24T23:24:20.717 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f384de1c640 1 -- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f383406be30 msgr2=0x7f383406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.708+0000 7f384de1c640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f383406be30 0x7f383406e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f383c034000 tx=0x7f383c02e000 comp rx=0 tx=0).stop 2024-04-24T23:24:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 -- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 msgr2=0x7f384813ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 0x7f384813ee70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f383800c460 tx=0x7f383800c920 comp rx=0 tx=0).stop 2024-04-24T23:24:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 -- 172.21.15.26:0/791015971 shutdown_connections 2024-04-24T23:24:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f383406be30 0x7f383406e2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3848145910 0x7f384813c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38480cd660 0x7f38481586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 --2- 172.21.15.26:0/791015971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f384806d350 0x7f384813ee70 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 -- 172.21.15.26:0/791015971 >> 172.21.15.26:0/791015971 conn(0x7f384806b300 msgr2=0x7f384815cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 -- 172.21.15.26:0/791015971 shutdown_connections 2024-04-24T23:24:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.709+0000 7f384de1c640 1 -- 172.21.15.26:0/791015971 wait complete. 2024-04-24T23:24:20.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:24:20.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.820+0000 7f2c3c729640 1 Processor -- start 2024-04-24T23:24:20.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.820+0000 7f2c3c729640 1 -- start start 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.821+0000 7f2c3c729640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2c34145910 0x7f2c3406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.821+0000 7f2c3c729640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.821+0000 7f2c3c729640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c340cfbc0 con 0x7f2c3406d350 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.821+0000 7f2c3c729640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2c34145cf0 con 0x7f2c34145910 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.821+0000 7f2c3c729640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3406d7f0 con 0x7f2c340cd660 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.821+0000 7f2c3acc6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.821+0000 7f2c3a4c5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2c34145910 0x7f2c3406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.821+0000 7f2c3acc6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:51168/0 (socket says 172.21.15.26:51168) 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.821+0000 7f2c3acc6640 1 -- 172.21.15.26:0/339071481 learned_addr learned my addr 172.21.15.26:0/339071481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.822+0000 7f2c3acc6640 1 -- 172.21.15.26:0/339071481 >> v2:172.21.15.29:3300/0 conn(0x7f2c34145910 msgr2=0x7f2c3406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.822+0000 7f2c3acc6640 1 --2- 172.21.15.26:0/339071481 >> v2:172.21.15.29:3300/0 conn(0x7f2c34145910 0x7f2c3406e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.822+0000 7f2c3acc6640 1 -- 172.21.15.26:0/339071481 >> v1:172.21.15.29:6789/0 conn(0x7f2c3406d350 legacy=0x7f2c3415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.822+0000 7f2c3acc6640 1 -- 172.21.15.26:0/339071481 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3406dad0 con 0x7f2c340cd660 2024-04-24T23:24:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.822+0000 7f2c3a4c5640 1 --2- 172.21.15.26:0/339071481 >> v2:172.21.15.29:3300/0 conn(0x7f2c34145910 0x7f2c3406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.822+0000 7f2c3acc6640 1 --2- 172.21.15.26:0/339071481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340cdaf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009e80 tx=0x7f2c3002a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efbed8e6f62cafd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c394c3640 1 -- 172.21.15.26:0/339071481 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30036070 con 0x7f2c340cd660 2024-04-24T23:24:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c394c3640 1 -- 172.21.15.26:0/339071481 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c30031600 con 0x7f2c340cd660 2024-04-24T23:24:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c394c3640 1 -- 172.21.15.26:0/339071481 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300319d0 con 0x7f2c340cd660 2024-04-24T23:24:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c3c729640 1 -- 172.21.15.26:0/339071481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 msgr2=0x7f2c340cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c3c729640 1 --2- 172.21.15.26:0/339071481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340cdaf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009e80 tx=0x7f2c3002a750 comp rx=0 tx=0).stop 2024-04-24T23:24:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c3c729640 1 -- 172.21.15.26:0/339071481 shutdown_connections 2024-04-24T23:24:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c3c729640 1 --2- 172.21.15.26:0/339071481 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340cdaf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c3c729640 1 --2- 172.21.15.26:0/339071481 >> v2:172.21.15.29:3300/0 conn(0x7f2c34145910 0x7f2c3406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c3c729640 1 -- 172.21.15.26:0/339071481 >> 172.21.15.26:0/339071481 conn(0x7f2c3406b300 msgr2=0x7f2c3406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c3c729640 1 -- 172.21.15.26:0/339071481 shutdown_connections 2024-04-24T23:24:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.823+0000 7f2c3c729640 1 -- 172.21.15.26:0/339071481 wait complete. 2024-04-24T23:24:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.824+0000 7f2c3c729640 1 Processor -- start 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3c729640 1 -- start start 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3c729640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c3406d350 0x7f2c340ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3c729640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3c729640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c34145910 0x7f2c3413f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3c729640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3406e370 con 0x7f2c3406d350 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3c729640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3406e090 con 0x7f2c34145910 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3c729640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3415d770 con 0x7f2c340cd660 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3a4c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c3406d350 0x7f2c340ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3a4c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c3406d350 0x7f2c340ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52334/0 (socket says 172.21.15.26:52334) 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3a4c5640 1 -- 172.21.15.26:0/4064213237 learned_addr learned my addr 172.21.15.26:0/4064213237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c39cc4640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.825+0000 7f2c3acc6640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c34145910 0x7f2c3413f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c3a4c5640 1 -- 172.21.15.26:0/4064213237 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 msgr2=0x7f2c340b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c3a4c5640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c3a4c5640 1 -- 172.21.15.26:0/4064213237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c34145910 msgr2=0x7f2c3413f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c3a4c5640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c34145910 0x7f2c3413f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c3a4c5640 1 -- 172.21.15.26:0/4064213237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c34140160 con 0x7f2c3406d350 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c3a4c5640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c3406d350 0x7f2c340ae350 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00e990 tx=0x7f2c2c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c2affd640 1 -- 172.21.15.26:0/4064213237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c017070 con 0x7f2c3406d350 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c2affd640 1 -- 172.21.15.26:0/4064213237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c00c9e0 con 0x7f2c3406d350 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c34140440 con 0x7f2c3406d350 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c2affd640 1 -- 172.21.15.26:0/4064213237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00cdb0 con 0x7f2c3406d350 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.826+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c340adb50 con 0x7f2c3406d350 2024-04-24T23:24:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.827+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c340694c0 con 0x7f2c3406d350 2024-04-24T23:24:20.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.828+0000 7f2c2affd640 1 -- 172.21.15.26:0/4064213237 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c2c01a070 con 0x7f2c3406d350 2024-04-24T23:24:20.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.828+0000 7f2c2affd640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c1806be30 0x7f2c1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:20.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.828+0000 7f2c2affd640 1 -- 172.21.15.26:0/4064213237 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2c2c08aca0 con 0x7f2c3406d350 2024-04-24T23:24:20.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.828+0000 7f2c39cc4640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c1806be30 0x7f2c1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:20.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.829+0000 7f2c39cc4640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c1806be30 0x7f2c1806e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2c34159b70 tx=0x7f2c24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:20.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:20.832+0000 7f2c2affd640 1 -- 172.21.15.26:0/4064213237 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c2c059090 con 0x7f2c3406d350 2024-04-24T23:24:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.056+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2c3415f2e0 con 0x7f2c3406d350 2024-04-24T23:24:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.056+0000 7f2c2affd640 1 -- 172.21.15.26:0/4064213237 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7f2c2c05d390 con 0x7f2c3406d350 2024-04-24T23:24:21.057 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:24:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c1806be30 msgr2=0x7f2c1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c1806be30 0x7f2c1806e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2c34159b70 tx=0x7f2c24009040 comp rx=0 tx=0).stop 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c3406d350 msgr2=0x7f2c340ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c3406d350 0x7f2c340ae350 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00e990 tx=0x7f2c2c00ee50 comp rx=0 tx=0).stop 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 shutdown_connections 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c1806be30 0x7f2c1806e2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c34145910 0x7f2c3413f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c340cd660 0x7f2c340b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 --2- 172.21.15.26:0/4064213237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c3406d350 0x7f2c340ae350 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 >> 172.21.15.26:0/4064213237 conn(0x7f2c3406b300 msgr2=0x7f2c34144990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 shutdown_connections 2024-04-24T23:24:21.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.058+0000 7f2c3c729640 1 -- 172.21.15.26:0/4064213237 wait complete. 2024-04-24T23:24:21.070 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:24:21.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:24:21.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-24T23:24:21.070 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:24:21.070 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:24:21.071 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:24:21.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:24:21.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:24:21.071 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:24:21.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:21.071 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-24T23:24:21.086 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-24T23:24:21.087 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42532sec preferred_lft 42532sec 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ccf2:1eff:fe9b:4749/64 scope link 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c2:ea:e0:db:00:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:24:21.088 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c0ea:e0ff:fedb:e0/64 scope link 2024-04-24T23:24:21.089 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:24:21.089 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:24:21.089 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:24:21.089 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::1c82:d0ff:fefa:5b3f/64 scope link 2024-04-24T23:24:21.089 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:24:21.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:21.089 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:24:21.150 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-24T23:24:21.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:21.151 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-24T23:24:21.206 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-24T23:24:21.206 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:24:21.206 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:24:21.207 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:24:21.207 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:24:21.207 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:24:21.207 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:24:21.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:24:21.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:24:21.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:24:21.208 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:24:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:24:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:24:21.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:21.333 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:24:21.349 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:24:21.349 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:24:21.349 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:24:21.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:21.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-24T23:24:21.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:24:21.413 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:24:21.469 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:24:21.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:24:21.470 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:24:21.525 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:24:21.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:21.526 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:24:21.607 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:24:21.607 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:24:21.607 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:24:21.615 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:24:21.616 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:24:21.616 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:24:21.616 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:24:21.616 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:24:21.622 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:24:21.622 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:24:21.622 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:24:21.622 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:24:21.622 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:24:21.622 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:24:21.622 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:24:21.622 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:24:21.623 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:24:21.623 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:24:21.623 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:24:21.623 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:24:21.623 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:24:21.624 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:24:21.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:21.625 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:24:21.690 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:24:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:24:21.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:24:21.792 INFO:teuthology.orchestra.run.smithi026.stdout:client5609 2024-04-24T23:24:21.793 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:24:21.793 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:24:21.809 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T23:24:21.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.951+0000 7fc5503ce640 1 Processor -- start 2024-04-24T23:24:21.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.951+0000 7fc5503ce640 1 -- start start 2024-04-24T23:24:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc5503ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5480b4ce0 0x7fc5480acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc5503ce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5480aeb10 con 0x7fc54815be40 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc5503ce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5480aedf0 con 0x7fc5480d0750 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc5503ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc54809df20 con 0x7fc5480b4ce0 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc54e96b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5480b4ce0 0x7fc5480acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc54e16a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc54815be40 0x7fc5480d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58878/0 (socket says 172.21.15.26:58878) 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc54e16a640 1 -- 172.21.15.26:0/2625073418 learned_addr learned my addr 172.21.15.26:0/2625073418 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc54e96b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5480b4ce0 0x7fc5480acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52348/0 (socket says 172.21.15.26:52348) 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc54e96b640 1 -- 172.21.15.26:0/2625073418 >> v1:172.21.15.29:6789/0 conn(0x7fc5480d0750 legacy=0x7fc54815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc54e96b640 1 -- 172.21.15.26:0/2625073418 >> v1:172.21.15.26:6789/0 conn(0x7fc54815be40 legacy=0x7fc5480d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc54e96b640 1 -- 172.21.15.26:0/2625073418 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5480af0d0 con 0x7fc5480b4ce0 2024-04-24T23:24:21.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.952+0000 7fc54d168640 1 -- 172.21.15.26:0/2625073418 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3205266582 0 0) 0x7fc5480aeb10 con 0x7fc54815be40 2024-04-24T23:24:21.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.953+0000 7fc54e96b640 1 --2- 172.21.15.26:0/2625073418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5480b4ce0 0x7fc5480acd30 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc538009d90 tx=0x7fc53802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1cedb3ce00e50e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.953+0000 7fc54d168640 1 -- 172.21.15.26:0/2625073418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538007740 con 0x7fc5480b4ce0 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.953+0000 7fc54d168640 1 -- 172.21.15.26:0/2625073418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5380078e0 con 0x7fc5480b4ce0 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.953+0000 7fc54d168640 1 -- 172.21.15.26:0/2625073418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538007cb0 con 0x7fc5480b4ce0 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.954+0000 7fc5503ce640 1 -- 172.21.15.26:0/2625073418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5480b4ce0 msgr2=0x7fc5480acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.954+0000 7fc5503ce640 1 --2- 172.21.15.26:0/2625073418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5480b4ce0 0x7fc5480acd30 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc538009d90 tx=0x7fc53802a6f0 comp rx=0 tx=0).stop 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.954+0000 7fc5503ce640 1 -- 172.21.15.26:0/2625073418 shutdown_connections 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.954+0000 7fc5503ce640 1 --2- 172.21.15.26:0/2625073418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5480b4ce0 0x7fc5480acd30 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.954+0000 7fc5503ce640 1 -- 172.21.15.26:0/2625073418 >> 172.21.15.26:0/2625073418 conn(0x7fc54806b540 msgr2=0x7fc54806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.954+0000 7fc5503ce640 1 -- 172.21.15.26:0/2625073418 shutdown_connections 2024-04-24T23:24:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.954+0000 7fc5503ce640 1 -- 172.21.15.26:0/2625073418 wait complete. 2024-04-24T23:24:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.955+0000 7fc5503ce640 1 Processor -- start 2024-04-24T23:24:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.955+0000 7fc5503ce640 1 -- start start 2024-04-24T23:24:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc5503ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5480b4ce0 0x7fc548142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc5503ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc5480d0750 0x7fc5481e4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc5503ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54815be40 0x7fc5481e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc5503ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc54806cd10 con 0x7fc54815be40 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc5503ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc54806cb90 con 0x7fc5480b4ce0 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc5503ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc54815c930 con 0x7fc5480d0750 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e96b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54815be40 0x7fc5481e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e96b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54815be40 0x7fc5481e4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52358/0 (socket says 172.21.15.26:52358) 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e96b640 1 -- 172.21.15.26:0/509928411 learned_addr learned my addr 172.21.15.26:0/509928411 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e16a640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5480b4ce0 0x7fc548142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e96b640 1 -- 172.21.15.26:0/509928411 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc5480d0750 msgr2=0x7fc5481e4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e96b640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc5480d0750 0x7fc5481e4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e96b640 1 -- 172.21.15.26:0/509928411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5480b4ce0 msgr2=0x7fc548142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e96b640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5480b4ce0 0x7fc548142760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e96b640 1 -- 172.21.15.26:0/509928411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc548142ca0 con 0x7fc54815be40 2024-04-24T23:24:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.956+0000 7fc54e16a640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5480b4ce0 0x7fc548142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.957+0000 7fc54e96b640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54815be40 0x7fc5481e4c90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc538009eb0 tx=0x7fc538004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.957+0000 7fc546ffd640 1 -- 172.21.15.26:0/509928411 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538040070 con 0x7fc54815be40 2024-04-24T23:24:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.957+0000 7fc546ffd640 1 -- 172.21.15.26:0/509928411 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc538031c90 con 0x7fc54815be40 2024-04-24T23:24:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.957+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc548143200 con 0x7fc54815be40 2024-04-24T23:24:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.957+0000 7fc546ffd640 1 -- 172.21.15.26:0/509928411 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53803c620 con 0x7fc54815be40 2024-04-24T23:24:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.957+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481437c0 con 0x7fc54815be40 2024-04-24T23:24:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.958+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54809dd40 con 0x7fc54815be40 2024-04-24T23:24:21.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.958+0000 7fc546ffd640 1 -- 172.21.15.26:0/509928411 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc538005d10 con 0x7fc54815be40 2024-04-24T23:24:21.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.958+0000 7fc546ffd640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53406be60 0x7fc53406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:21.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.959+0000 7fc546ffd640 1 -- 172.21.15.26:0/509928411 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc5380ac180 con 0x7fc54815be40 2024-04-24T23:24:21.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.959+0000 7fc54e16a640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53406be60 0x7fc53406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:21.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.959+0000 7fc54e16a640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53406be60 0x7fc53406e320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc540000f60 tx=0x7fc540009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:21.963+0000 7fc546ffd640 1 -- 172.21.15.26:0/509928411 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc54809dd40 con 0x7fc54815be40 2024-04-24T23:24:22.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:22.186+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc548075610 con 0x7fc54815be40 2024-04-24T23:24:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.032+0000 7fc546ffd640 1 -- 172.21.15.26:0/509928411 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v50) ==== 105+0+0 (secure 0 0 0) 0x7fc53804b070 con 0x7fc54815be40 2024-04-24T23:24:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53406be60 msgr2=0x7fc53406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53406be60 0x7fc53406e320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc540000f60 tx=0x7fc540009040 comp rx=0 tx=0).stop 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54815be40 msgr2=0x7fc5481e4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54815be40 0x7fc5481e4c90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc538009eb0 tx=0x7fc538004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 shutdown_connections 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53406be60 0x7fc53406e320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54815be40 0x7fc5481e4c90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc5480d0750 0x7fc5481e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 --2- 172.21.15.26:0/509928411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5480b4ce0 0x7fc548142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 >> 172.21.15.26:0/509928411 conn(0x7fc54806b540 msgr2=0x7fc5480b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 shutdown_connections 2024-04-24T23:24:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.035+0000 7fc5503ce640 1 -- 172.21.15.26:0/509928411 wait complete. 2024-04-24T23:24:23.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:23.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.160+0000 7fc2e14f5640 1 Processor -- start 2024-04-24T23:24:23.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.160+0000 7fc2e14f5640 1 -- start start 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2e14f5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc2dc143c40 0x7fc2dc0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2e14f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc15eb00 0x7fc2dc15ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2e14f5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2dc0cd0b0 con 0x7fc2dc0cc860 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2e14f5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc2dc071b70 con 0x7fc2dc143c40 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2e14f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc2dc15f470 con 0x7fc2dc15eb00 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2da7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc2dc0cc860 0x7fc2dc072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58882/0 (socket says 172.21.15.26:58882) 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2da7fc640 1 -- 172.21.15.26:0/2229231995 learned_addr learned my addr 172.21.15.26:0/2229231995 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc15eb00 0x7fc2dc15ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2daffd640 1 --2- 172.21.15.26:0/2229231995 >> v2:172.21.15.29:3300/0 conn(0x7fc2dc143c40 0x7fc2dc0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 989302287 0 0) 0x7fc2dc0cd0b0 con 0x7fc2dc0cc860 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2c4003520 con 0x7fc2dc0cc860 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.161+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1414356521 0 0) 0x7fc2c4003520 con 0x7fc2dc0cc860 2024-04-24T23:24:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc15eb00 msgr2=0x7fc2dc15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc15eb00 0x7fc2dc15ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 >> v2:172.21.15.29:3300/0 conn(0x7fc2dc143c40 msgr2=0x7fc2dc0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 --2- 172.21.15.26:0/2229231995 >> v2:172.21.15.29:3300/0 conn(0x7fc2dc143c40 0x7fc2dc0cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2dc071240 con 0x7fc2dc0cc860 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fc2d0001670 con 0x7fc2dc0cc860 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 >> v1:172.21.15.26:6789/0 conn(0x7fc2dc0cc860 legacy=0x7fc2dc072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2c40039b0 0x7fc2c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc15eb00 0x7fc2c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c4004d40 0x7fc2c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc071b70 con 0x7fc2c40039b0 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc15f470 con 0x7fc2c4004d40 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c400b9c0 con 0x7fc2dc15eb00 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2daffd640 1 --2- 172.21.15.26:0/2229231995 >> v2:172.21.15.29:3300/0 conn(0x7fc2dc143c40 0x7fc2dc0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2da7fc640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc15eb00 0x7fc2c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2daffd640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2c40039b0 0x7fc2c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c4004d40 0x7fc2c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2daffd640 1 -- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc15eb00 msgr2=0x7fc2c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2daffd640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc15eb00 0x7fc2c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2daffd640 1 -- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c4004d40 msgr2=0x7fc2c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2daffd640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c4004d40 0x7fc2c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2daffd640 1 -- 172.21.15.26:0/2229231995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c4003520 con 0x7fc2c40039b0 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.162+0000 7fc2da7fc640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc15eb00 0x7fc2c4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.163+0000 7fc2daffd640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2c40039b0 0x7fc2c4003dd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009c70 tx=0x7fc2c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441656d39ce4a013 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.163+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4009840 con 0x7fc2c40039b0 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.163+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d4004020 con 0x7fc2c40039b0 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.163+0000 7fc2d9ffb640 1 -- 172.21.15.26:0/2229231995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40043d0 con 0x7fc2c40039b0 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.164+0000 7fc2e14f5640 1 -- 172.21.15.26:0/2229231995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2c40039b0 msgr2=0x7fc2c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.164+0000 7fc2e14f5640 1 --2- 172.21.15.26:0/2229231995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2c40039b0 0x7fc2c4003dd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009c70 tx=0x7fc2c4005830 comp rx=0 tx=0).stop 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.164+0000 7fc2db7fe640 1 -- 172.21.15.26:0/2229231995 reap_dead start 2024-04-24T23:24:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.164+0000 7fc2e14f5640 1 -- 172.21.15.26:0/2229231995 shutdown_connections 2024-04-24T23:24:23.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.164+0000 7fc2e14f5640 1 -- 172.21.15.26:0/2229231995 >> 172.21.15.26:0/2229231995 conn(0x7fc2dc06b300 msgr2=0x7fc2dc06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:23.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.164+0000 7fc2e14f5640 1 -- 172.21.15.26:0/2229231995 shutdown_connections 2024-04-24T23:24:23.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.164+0000 7fc2e14f5640 1 -- 172.21.15.26:0/2229231995 wait complete. 2024-04-24T23:24:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.165+0000 7fc2e14f5640 1 Processor -- start 2024-04-24T23:24:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.165+0000 7fc2e14f5640 1 -- start start 2024-04-24T23:24:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2e14f5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40039b0 0x7fc2dc15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2e14f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc132e00 0x7fc2dc140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2e14f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc140fe0 0x7fc2dc0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2e14f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc144020 con 0x7fc2dc132e00 2024-04-24T23:24:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2e14f5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc071800 con 0x7fc2c40039b0 2024-04-24T23:24:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2e14f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc2dc0cccd0 con 0x7fc2dc140fe0 2024-04-24T23:24:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2da7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc132e00 0x7fc2dc140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2da7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc132e00 0x7fc2dc140aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52372/0 (socket says 172.21.15.26:52372) 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2da7fc640 1 -- 172.21.15.26:0/3297375388 learned_addr learned my addr 172.21.15.26:0/3297375388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc140fe0 0x7fc2dc0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2daffd640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40039b0 0x7fc2dc15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2da7fc640 1 -- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc140fe0 msgr2=0x7fc2dc0ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2da7fc640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc140fe0 0x7fc2dc0ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2da7fc640 1 -- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40039b0 msgr2=0x7fc2dc15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2da7fc640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40039b0 0x7fc2dc15a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2da7fc640 1 -- 172.21.15.26:0/3297375388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2dc133280 con 0x7fc2dc132e00 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2daffd640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40039b0 0x7fc2dc15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc140fe0 0x7fc2dc0ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.166+0000 7fc2da7fc640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc132e00 0x7fc2dc140aa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0007340 tx=0x7fc2d0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.167+0000 7fc2c37fe640 1 -- 172.21.15.26:0/3297375388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000fac0 con 0x7fc2dc132e00 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.167+0000 7fc2c37fe640 1 -- 172.21.15.26:0/3297375388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d0018070 con 0x7fc2dc132e00 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.167+0000 7fc2c37fe640 1 -- 172.21.15.26:0/3297375388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000fe60 con 0x7fc2dc132e00 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.167+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2dc141520 con 0x7fc2dc132e00 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.167+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2dc0adeb0 con 0x7fc2dc132e00 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.168+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2dc0694c0 con 0x7fc2dc132e00 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.168+0000 7fc2c37fe640 1 -- 172.21.15.26:0/3297375388 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2d001d460 con 0x7fc2dc132e00 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.169+0000 7fc2c37fe640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2c806be30 0x7fc2c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.169+0000 7fc2c37fe640 1 -- 172.21.15.26:0/3297375388 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc2d008b190 con 0x7fc2dc132e00 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.169+0000 7fc2daffd640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2c806be30 0x7fc2c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.169+0000 7fc2daffd640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2c806be30 0x7fc2c806e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4032000 tx=0x7fc2d4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.173+0000 7fc2c37fe640 1 -- 172.21.15.26:0/3297375388 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc2dc133280 con 0x7fc2dc132e00 2024-04-24T23:24:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.396+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2dc0b4d20 con 0x7fc2dc132e00 2024-04-24T23:24:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.397+0000 7fc2c37fe640 1 -- 172.21.15.26:0/3297375388 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5197 (secure 0 0 0) 0x7fc2d0059aa0 con 0x7fc2dc132e00 2024-04-24T23:24:23.398 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:23.399 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":51,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1519104007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1519104007},{"type":"v1","addr":"172.21.15.26:6836","nonce":1519104007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:24:14.713249+0000","modified":"2024-04-24T23:24:23.036317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6546,"mds_1":6534},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"a","rank":1,"incarnation":51,"state":"up:creating","state_seq":1,"addr":"172.21.15.26:6837/2743842996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2743842996},{"type":"v1","addr":"172.21.15.26:6837","nonce":2743842996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6546":{"gid":6546,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2096523178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2096523178},{"type":"v1","addr":"172.21.15.29:6835","nonce":2096523178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":4}]} 2024-04-24T23:24:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 51 2024-04-24T23:24:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2c806be30 msgr2=0x7fc2c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2c806be30 0x7fc2c806e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4032000 tx=0x7fc2d4008030 comp rx=0 tx=0).stop 2024-04-24T23:24:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc132e00 msgr2=0x7fc2dc140aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc132e00 0x7fc2dc140aa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0007340 tx=0x7fc2d0007800 comp rx=0 tx=0).stop 2024-04-24T23:24:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 shutdown_connections 2024-04-24T23:24:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc2c806be30 0x7fc2c806e2f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2dc140fe0 0x7fc2dc0ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc132e00 0x7fc2dc140aa0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 --2- 172.21.15.26:0/3297375388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2c40039b0 0x7fc2dc15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 >> 172.21.15.26:0/3297375388 conn(0x7fc2dc06b300 msgr2=0x7fc2dc0b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 shutdown_connections 2024-04-24T23:24:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:23.399+0000 7fc2e14f5640 1 -- 172.21.15.26:0/3297375388 wait complete. 2024-04-24T23:24:23.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:24:14.713249+0000', 'modified': '2024-04-24T23:24:23.036317+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6546, 'mds_1': 6534}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6534': {'gid': 6534, 'name': 'a', 'rank': 1, 'incarnation': 51, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.26:6837/2743842996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6835', 'nonce': 2743842996}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 2743842996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6546': {'gid': 6546, 'name': 'd', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/2096523178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 2096523178}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 2096523178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6546, 'qdb_cluster': [6546]} 2024-04-24T23:24:23.814 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6534:up:creating 2024-04-24T23:24:24.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:24.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.926+0000 7f733907d640 1 Processor -- start 2024-04-24T23:24:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.926+0000 7f733907d640 1 -- start start 2024-04-24T23:24:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.927+0000 7f733907d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f73340b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.927+0000 7f733907d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f733415e5a0 0x7f73340ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.927+0000 7f733907d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73340733c0 con 0x7f73340af440 2024-04-24T23:24:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.927+0000 7f733907d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f73340add00 con 0x7f733415e5a0 2024-04-24T23:24:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.927+0000 7f733907d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f73340adfe0 con 0x7f733415e1c0 2024-04-24T23:24:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.927+0000 7f7332d9d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f73340b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:24.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.927+0000 7f732bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f733415e5a0 0x7f73340ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:24.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.927+0000 7f733359e640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f73340af440 0x7f73340afb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:58500/0 (socket says 172.21.15.26:58500) 2024-04-24T23:24:24.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.927+0000 7f733359e640 1 -- 172.21.15.26:0/1584193831 learned_addr learned my addr 172.21.15.26:0/1584193831 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:24.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.928+0000 7f732bfff640 1 -- 172.21.15.26:0/1584193831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 msgr2=0x7f73340b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:24.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.928+0000 7f732bfff640 1 --2- 172.21.15.26:0/1584193831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f73340b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:24.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.928+0000 7f732bfff640 1 -- 172.21.15.26:0/1584193831 >> v1:172.21.15.29:6789/0 conn(0x7f73340af440 legacy=0x7f73340afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:24.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.928+0000 7f733259c640 1 -- 172.21.15.26:0/1584193831 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913247146 0 0) 0x7f73340733c0 con 0x7f73340af440 2024-04-24T23:24:24.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.928+0000 7f732bfff640 1 -- 172.21.15.26:0/1584193831 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73340ae2c0 con 0x7f733415e5a0 2024-04-24T23:24:24.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.928+0000 7f7332d9d640 1 --2- 172.21.15.26:0/1584193831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f73340b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f732bfff640 1 --2- 172.21.15.26:0/1584193831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733415e5a0 0x7f73340ae7e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097a0 tx=0x7f731c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff4b83877028c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733259c640 1 -- 172.21.15.26:0/1584193831 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c037070 con 0x7f733415e5a0 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733259c640 1 -- 172.21.15.26:0/1584193831 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c02ae50 con 0x7f733415e5a0 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733259c640 1 -- 172.21.15.26:0/1584193831 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c032660 con 0x7f733415e5a0 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733907d640 1 -- 172.21.15.26:0/1584193831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733415e5a0 msgr2=0x7f73340ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733907d640 1 --2- 172.21.15.26:0/1584193831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733415e5a0 0x7f73340ae7e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097a0 tx=0x7f731c02a100 comp rx=0 tx=0).stop 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733907d640 1 -- 172.21.15.26:0/1584193831 shutdown_connections 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733907d640 1 --2- 172.21.15.26:0/1584193831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f733415e5a0 0x7f73340ae7e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733907d640 1 --2- 172.21.15.26:0/1584193831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f73340b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733907d640 1 -- 172.21.15.26:0/1584193831 >> 172.21.15.26:0/1584193831 conn(0x7f7334072790 msgr2=0x7f7334072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733907d640 1 -- 172.21.15.26:0/1584193831 shutdown_connections 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.929+0000 7f733907d640 1 -- 172.21.15.26:0/1584193831 wait complete. 2024-04-24T23:24:24.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.931+0000 7f733907d640 1 Processor -- start 2024-04-24T23:24:24.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.931+0000 7f733907d640 1 -- start start 2024-04-24T23:24:24.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.931+0000 7f733907d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73340af440 0x7f73340cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:24.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.931+0000 7f733907d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f7334151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:24.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.931+0000 7f733907d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733415e5a0 0x7f73341606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:24.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.931+0000 7f733907d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f73340b03b0 con 0x7f733415e5a0 2024-04-24T23:24:24.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.931+0000 7f733907d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f73340b0230 con 0x7f73340af440 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.931+0000 7f733907d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f73340aed20 con 0x7f733415e1c0 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f733359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733415e5a0 0x7f73341606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732bfff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f7334151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f7332d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73340af440 0x7f73340cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732bfff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f7334151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:49850/0 (socket says 172.21.15.26:49850) 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f733359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733415e5a0 0x7f73341606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33692/0 (socket says 172.21.15.26:33692) 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732bfff640 1 -- 172.21.15.26:0/3024794203 learned_addr learned my addr 172.21.15.26:0/3024794203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732bfff640 1 -- 172.21.15.26:0/3024794203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73340af440 msgr2=0x7f73340cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732bfff640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73340af440 0x7f73340cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732bfff640 1 -- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733415e5a0 msgr2=0x7f73341606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732bfff640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733415e5a0 0x7f73341606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:24.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732bfff640 1 -- 172.21.15.26:0/3024794203 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73340cbbe0 con 0x7f733415e1c0 2024-04-24T23:24:24.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f733359e640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733415e5a0 0x7f73341606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:24.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732bfff640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f7334151640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f731c000c00 tx=0x7f731c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:24.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732b7fe640 1 -- 172.21.15.26:0/3024794203 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c037040 con 0x7f733415e1c0 2024-04-24T23:24:24.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732b7fe640 1 -- 172.21.15.26:0/3024794203 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c004030 con 0x7f733415e1c0 2024-04-24T23:24:24.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.932+0000 7f732b7fe640 1 -- 172.21.15.26:0/3024794203 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c004400 con 0x7f733415e1c0 2024-04-24T23:24:24.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.933+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73340cbec0 con 0x7f733415e1c0 2024-04-24T23:24:24.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.933+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f733415b8e0 con 0x7f733415e1c0 2024-04-24T23:24:24.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.934+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733406d6e0 con 0x7f733415e1c0 2024-04-24T23:24:24.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.934+0000 7f732b7fe640 1 -- 172.21.15.26:0/3024794203 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f731c003610 con 0x7f733415e1c0 2024-04-24T23:24:24.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.934+0000 7f732b7fe640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f731406be30 0x7f731406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:24.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.935+0000 7f732b7fe640 1 -- 172.21.15.26:0/3024794203 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f731c0abb30 con 0x7f733415e1c0 2024-04-24T23:24:24.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.935+0000 7f7332d9d640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f731406be30 0x7f731406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:24.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.935+0000 7f7332d9d640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f731406be30 0x7f731406e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7324009560 tx=0x7f7324009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:24.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:24.939+0000 7f732b7fe640 1 -- 172.21.15.26:0/3024794203 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f733406d6e0 con 0x7f733415e1c0 2024-04-24T23:24:25.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.166+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73340694c0 con 0x7f733415e1c0 2024-04-24T23:24:25.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.166+0000 7f732b7fe640 1 -- 172.21.15.26:0/3024794203 <== mon.2 v2:172.21.15.29:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5219 (secure 0 0 0) 0x7f731c07a040 con 0x7f733415e1c0 2024-04-24T23:24:25.169 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:25.169 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:24:14.713249+0000","modified":"2024-04-24T23:24:25.043181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6546,"mds_1":6534,"mds_2":6531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6531":{"gid":6531,"name":"c","rank":2,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6836/1519104007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1519104007},{"type":"v1","addr":"172.21.15.26:6836","nonce":1519104007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6534":{"gid":6534,"name":"a","rank":1,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6837/2743842996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2743842996},{"type":"v1","addr":"172.21.15.26:6837","nonce":2743842996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6546":{"gid":6546,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2096523178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2096523178},{"type":"v1","addr":"172.21.15.29:6835","nonce":2096523178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6531,6534,6546]},"id":4}]} 2024-04-24T23:24:25.169 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 54 2024-04-24T23:24:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f731406be30 msgr2=0x7f731406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f731406be30 0x7f731406e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7324009560 tx=0x7f7324009190 comp rx=0 tx=0).stop 2024-04-24T23:24:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 msgr2=0x7f7334151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f7334151640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f731c000c00 tx=0x7f731c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 shutdown_connections 2024-04-24T23:24:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f731406be30 0x7f731406e2f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733415e5a0 0x7f73341606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f733415e1c0 0x7f7334151640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 --2- 172.21.15.26:0/3024794203 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73340af440 0x7f73340cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 >> 172.21.15.26:0/3024794203 conn(0x7f7334072790 msgr2=0x7f73340b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 shutdown_connections 2024-04-24T23:24:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.169+0000 7f733907d640 1 -- 172.21.15.26:0/3024794203 wait complete. 2024-04-24T23:24:25.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:24:14.713249+0000', 'modified': '2024-04-24T23:24:25.043181+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6546, 'mds_1': 6534, 'mds_2': 6531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6531': {'gid': 6531, 'name': 'c', 'rank': 2, 'incarnation': 53, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6836/1519104007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1519104007}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 1519104007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6534': {'gid': 6534, 'name': 'a', 'rank': 1, 'incarnation': 51, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6837/2743842996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6835', 'nonce': 2743842996}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 2743842996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6546': {'gid': 6546, 'name': 'd', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/2096523178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 2096523178}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 2096523178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6546, 'qdb_cluster': [6531, 6534, 6546]} 2024-04-24T23:24:25.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:24:25.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:24:25.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.693+0000 7fb8f664d640 1 Processor -- start 2024-04-24T23:24:25.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.694+0000 7fb8f664d640 1 -- start start 2024-04-24T23:24:25.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.694+0000 7fb8f664d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb8f00d4870 0x7fb8f00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.694+0000 7fb8f664d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00b3130 0x7fb8f00b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.694+0000 7fb8f664d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f00d32f0 con 0x7fb8f015e570 2024-04-24T23:24:25.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.694+0000 7fb8f664d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb8f00d1ab0 con 0x7fb8f00d4870 2024-04-24T23:24:25.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.694+0000 7fb8f664d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f00d2ae0 con 0x7fb8f00b3130 2024-04-24T23:24:25.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.694+0000 7fb8effff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fb8f015e570 0x7fb8f00d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:58516/0 (socket says 172.21.15.26:58516) 2024-04-24T23:24:25.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.694+0000 7fb8f4bea640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00b3130 0x7fb8f00b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.694+0000 7fb8effff640 1 -- 172.21.15.26:0/3497426832 learned_addr learned my addr 172.21.15.26:0/3497426832 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:25.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.695+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/3497426832 >> v2:172.21.15.29:3300/0 conn(0x7fb8f00d4870 0x7fb8f00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.695+0000 7fb8eeffd640 1 -- 172.21.15.26:0/3497426832 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867940533 0 0) 0x7fb8e4001110 con 0x7fb8f015e570 2024-04-24T23:24:25.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.695+0000 7fb8eeffd640 1 -- 172.21.15.26:0/3497426832 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8d8003520 con 0x7fb8f015e570 2024-04-24T23:24:25.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.695+0000 7fb8ef7fe640 1 -- 172.21.15.26:0/3497426832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00b3130 msgr2=0x7fb8f00b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.695+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/3497426832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00b3130 0x7fb8f00b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.695+0000 7fb8ef7fe640 1 -- 172.21.15.26:0/3497426832 >> v1:172.21.15.29:6789/0 conn(0x7fb8f015e570 legacy=0x7fb8f00d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.695+0000 7fb8ef7fe640 1 -- 172.21.15.26:0/3497426832 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f00d0270 con 0x7fb8f00d4870 2024-04-24T23:24:25.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.695+0000 7fb8f4bea640 1 --2- 172.21.15.26:0/3497426832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00b3130 0x7fb8f00b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:25.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/3497426832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8f00d4870 0x7fb8f00b4100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00097a0 tx=0x7fb8e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=252cde90b4e0463 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8eeffd640 1 -- 172.21.15.26:0/3497426832 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0037070 con 0x7fb8f00d4870 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8eeffd640 1 -- 172.21.15.26:0/3497426832 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e0032440 con 0x7fb8f00d4870 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8eeffd640 1 -- 172.21.15.26:0/3497426832 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0032810 con 0x7fb8f00d4870 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8f664d640 1 -- 172.21.15.26:0/3497426832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8f00d4870 msgr2=0x7fb8f00b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8f664d640 1 --2- 172.21.15.26:0/3497426832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8f00d4870 0x7fb8f00b4100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00097a0 tx=0x7fb8e002a180 comp rx=0 tx=0).stop 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8f664d640 1 -- 172.21.15.26:0/3497426832 shutdown_connections 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8f664d640 1 --2- 172.21.15.26:0/3497426832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00b3130 0x7fb8f00b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8f664d640 1 --2- 172.21.15.26:0/3497426832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8f00d4870 0x7fb8f00b4100 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8f664d640 1 -- 172.21.15.26:0/3497426832 >> 172.21.15.26:0/3497426832 conn(0x7fb8f008dbe0 msgr2=0x7fb8f009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8f664d640 1 -- 172.21.15.26:0/3497426832 shutdown_connections 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.696+0000 7fb8f664d640 1 -- 172.21.15.26:0/3497426832 wait complete. 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.698+0000 7fb8f664d640 1 Processor -- start 2024-04-24T23:24:25.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.698+0000 7fb8f664d640 1 -- start start 2024-04-24T23:24:25.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.698+0000 7fb8f664d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00b3130 0x7fb8f01f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.698+0000 7fb8f664d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00d4870 0x7fb8f00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.698+0000 7fb8f664d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8f015e570 0x7fb8f01f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.698+0000 7fb8f664d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f00cfa70 con 0x7fb8f00b3130 2024-04-24T23:24:25.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.698+0000 7fb8f664d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f00d3010 con 0x7fb8f015e570 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.698+0000 7fb8f664d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f00cfd50 con 0x7fb8f00d4870 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.698+0000 7fb8effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00b3130 0x7fb8f01f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00b3130 0x7fb8f01f8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33700/0 (socket says 172.21.15.26:33700) 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8effff640 1 -- 172.21.15.26:0/4204079550 learned_addr learned my addr 172.21.15.26:0/4204079550 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00d4870 0x7fb8f00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8f4bea640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8f015e570 0x7fb8f01f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8effff640 1 -- 172.21.15.26:0/4204079550 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00d4870 msgr2=0x7fb8f00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8effff640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00d4870 0x7fb8f00b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8effff640 1 -- 172.21.15.26:0/4204079550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8f015e570 msgr2=0x7fb8f01f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8effff640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8f015e570 0x7fb8f01f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8effff640 1 -- 172.21.15.26:0/4204079550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f01f52e0 con 0x7fb8f00b3130 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00d4870 0x7fb8f00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8effff640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00b3130 0x7fb8f01f8770 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400c700 tx=0x7fb8e400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8ecff9640 1 -- 172.21.15.26:0/4204079550 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4017070 con 0x7fb8f00b3130 2024-04-24T23:24:25.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8ecff9640 1 -- 172.21.15.26:0/4204079550 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e4005ce0 con 0x7fb8f00b3130 2024-04-24T23:24:25.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8f664d640 1 -- 172.21.15.26:0/4204079550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f01f55c0 con 0x7fb8f00b3130 2024-04-24T23:24:25.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8ecff9640 1 -- 172.21.15.26:0/4204079550 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4007310 con 0x7fb8f00b3130 2024-04-24T23:24:25.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.699+0000 7fb8f664d640 1 -- 172.21.15.26:0/4204079550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f01f8cb0 con 0x7fb8f00b3130 2024-04-24T23:24:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.701+0000 7fb8ecff9640 1 -- 172.21.15.26:0/4204079550 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8e401a070 con 0x7fb8f00b3130 2024-04-24T23:24:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.701+0000 7fb8ecff9640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8dc06be30 0x7fb8dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.701+0000 7fb8ecff9640 1 -- 172.21.15.26:0/4204079550 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb8e408a120 con 0x7fb8f00b3130 2024-04-24T23:24:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.701+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8dc06be30 0x7fb8dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.702+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8dc06be30 0x7fb8dc06e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0034000 tx=0x7fb8e002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.702+0000 7fb8f9438740 1 Processor -- start 2024-04-24T23:24:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.702+0000 7fb8f9438740 1 -- start start 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64fbb60 0x55a7f64ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 0x55a7f64de000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64de540 0x55a7f64b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8f9438740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a7f6416f00 con 0x55a7f64fbb60 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8f9438740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a7f63bf750 con 0x55a7f64dec80 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8f9438740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a7f6412ec0 con 0x55a7f64de540 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64fbb60 0x55a7f64ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64fbb60 0x55a7f64ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33714/0 (socket says 172.21.15.26:33714) 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8effff640 1 -- 172.21.15.26:0/2368781378 learned_addr learned my addr 172.21.15.26:0/2368781378 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/2368781378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 0x55a7f64de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8f4bea640 1 --2- 172.21.15.26:0/2368781378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64de540 0x55a7f64b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8effff640 1 -- 172.21.15.26:0/2368781378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64de540 msgr2=0x55a7f64b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8effff640 1 --2- 172.21.15.26:0/2368781378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64de540 0x55a7f64b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8effff640 1 -- 172.21.15.26:0/2368781378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 msgr2=0x55a7f64de000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8effff640 1 --2- 172.21.15.26:0/2368781378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 0x55a7f64de000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8effff640 1 -- 172.21.15.26:0/2368781378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f612b2d0 con 0x55a7f64fbb60 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.703+0000 7fb8effff640 1 --2- 172.21.15.26:0/2368781378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64fbb60 0x55a7f64ae4f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40885c0 tx=0x7fb8e4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5ed0d34c44f871 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8d5ffb640 1 -- 172.21.15.26:0/2368781378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4056700 con 0x55a7f64fbb60 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8d5ffb640 1 -- 172.21.15.26:0/2368781378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e4054a70 con 0x55a7f64fbb60 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8d5ffb640 1 -- 172.21.15.26:0/2368781378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40076f0 con 0x55a7f64fbb60 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8f9438740 1 -- 172.21.15.26:0/2368781378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64fbb60 msgr2=0x55a7f64ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2368781378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64fbb60 0x55a7f64ae4f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40885c0 tx=0x7fb8e4007f90 comp rx=0 tx=0).stop 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8f9438740 1 -- 172.21.15.26:0/2368781378 shutdown_connections 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2368781378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64de540 0x55a7f64b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2368781378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 0x55a7f64de000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2368781378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64fbb60 0x55a7f64ae4f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8f9438740 1 -- 172.21.15.26:0/2368781378 >> 172.21.15.26:0/2368781378 conn(0x55a7f64b0760 msgr2=0x55a7f64f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8f9438740 1 -- 172.21.15.26:0/2368781378 shutdown_connections 2024-04-24T23:24:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.704+0000 7fb8f9438740 1 -- 172.21.15.26:0/2368781378 wait complete. 2024-04-24T23:24:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.705+0000 7fb8f9438740 1 -- start start 2024-04-24T23:24:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.705+0000 7fb8f9438740 1 Processor -- start 2024-04-24T23:24:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 0x55a7f64f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 0x55a7f64efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64fa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f9438740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a7f64b2910 con 0x55a7f64d9980 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f9438740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a7f63d2cf0 con 0x55a7f64fbb60 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f9438740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a7f64b1d00 con 0x55a7f64dec80 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f9438740 10 client.0 _ll_register_callbacks cb 0x55a7f63d26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f4bea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f4bea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64fa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33730/0 (socket says 172.21.15.26:33730) 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f4bea640 1 -- 172.21.15.26:0/2359335797 learned_addr learned my addr 172.21.15.26:0/2359335797 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8effff640 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 0x55a7f64f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 0x55a7f64efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f4bea640 1 -- 172.21.15.26:0/2359335797 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 msgr2=0x55a7f64f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f4bea640 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 0x55a7f64f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f4bea640 1 -- 172.21.15.26:0/2359335797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 msgr2=0x55a7f64efcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f4bea640 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 0x55a7f64efcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f4bea640 1 -- 172.21.15.26:0/2359335797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f61f56b0 con 0x55a7f64d9980 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8f4bea640 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64fa130 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8014da0 tx=0x7fb8e8012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2176ccca77233603 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.706+0000 7fb8ceffd640 1 -- 172.21.15.26:0/2359335797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e801e620 con 0x55a7f64d9980 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8ceffd640 1 -- 172.21.15.26:0/2359335797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb8e801e7c0 con 0x55a7f64d9980 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8f9438740 1 -- 172.21.15.26:0/2359335797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e4054a70 con 0x55a7f64d9980 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8ceffd640 1 -- 172.21.15.26:0/2359335797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb8e801e9c0 con 0x55a7f64d9980 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8f9438740 10 client.6729 fetch_fsmap learned FSMap version 54 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8f9438740 1 -- 172.21.15.26:0/2359335797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55a7f64fae50 con 0x55a7f64d9980 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8ceffd640 1 -- 172.21.15.26:0/2359335797 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e4054a70 con 0x55a7f64d9980 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8ceffd640 1 -- 172.21.15.26:0/2359335797 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7fb8e802b080 con 0x55a7f64d9980 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8f9438740 10 client.6729 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T23:24:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8f9438740 10 client.6729 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8f9438740 20 client.6729 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55a7f64ad3d0 0x55a7f6509840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8f9438740 4 client.6729 mds_command: new command op to 6531 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8f9438740 1 -- 172.21.15.26:0/2359335797 --> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7f6424440 con 0x55a7f64ad3d0 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8effff640 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55a7f64ad3d0 0x55a7f6509840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8ceffd640 10 client.6729 ms_handle_connect on v2:172.21.15.26:6834/1519104007 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.707+0000 7fb8effff640 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55a7f64ad3d0 0x55a7f6509840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13860ba83cb6ede3 server_cookie=265fb2310fd93fd3 in_seq=0 out_seq=0 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8ceffd640 1 -- 172.21.15.26:0/2359335797 <== mds.2 v2:172.21.15.26:6834/1519104007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7f6424440 con 0x55a7f64ad3d0 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8ceffd640 10 client.6729 handle_command_reply: tid=0 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 client.6729 shutdown 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 -- 172.21.15.26:0/2359335797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 msgr2=0x55a7f64fa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64fa130 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8014da0 tx=0x7fb8e8012670 comp rx=0 tx=0).stop 2024-04-24T23:24:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 -- 172.21.15.26:0/2359335797 shutdown_connections 2024-04-24T23:24:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55a7f64ad3d0 0x55a7f6509840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64fa130 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 0x55a7f64efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8ceffd640 0 client.6729 ms_handle_reset on v2:172.21.15.26:6834/1519104007 2024-04-24T23:24:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 --2- 172.21.15.26:0/2359335797 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 0x55a7f64f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 -- 172.21.15.26:0/2359335797 >> 172.21.15.26:0/2359335797 conn(0x55a7f64de540 msgr2=0x55a7f64f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 -- 172.21.15.26:0/2359335797 shutdown_connections 2024-04-24T23:24:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 1 -- 172.21.15.26:0/2359335797 wait complete. 2024-04-24T23:24:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.708+0000 7fb8f9438740 20 client.6729 trim_cache size 0 max 16384 2024-04-24T23:24:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8f9438740 1 Processor -- start 2024-04-24T23:24:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8f9438740 1 -- start start 2024-04-24T23:24:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 0x55a7f64f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 0x55a7f64f1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8f9438740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f00b7810 con 0x55a7f64d9980 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8f9438740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f00a1250 con 0x55a7f64fbb60 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8f9438740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f009fa00 con 0x55a7f64dec80 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33736/0 (socket says 172.21.15.26:33736) 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8effff640 1 -- 172.21.15.26:0/1700043714 learned_addr learned my addr 172.21.15.26:0/1700043714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.727+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/1700043714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 0x55a7f64f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8f4bea640 1 --2- 172.21.15.26:0/1700043714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 0x55a7f64f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8effff640 1 -- 172.21.15.26:0/1700043714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 msgr2=0x55a7f64f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8effff640 1 --2- 172.21.15.26:0/1700043714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 0x55a7f64f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8effff640 1 -- 172.21.15.26:0/1700043714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 msgr2=0x55a7f64f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8effff640 1 --2- 172.21.15.26:0/1700043714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 0x55a7f64f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8effff640 1 -- 172.21.15.26:0/1700043714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f64d9790 con 0x55a7f64d9980 2024-04-24T23:24:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8effff640 1 --2- 172.21.15.26:0/1700043714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64b35c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4058390 tx=0x7fb8e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace36436d605db6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8d57fa640 1 -- 172.21.15.26:0/1700043714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4056700 con 0x55a7f64d9980 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8d57fa640 1 -- 172.21.15.26:0/1700043714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e4070130 con 0x55a7f64d9980 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.728+0000 7fb8d57fa640 1 -- 172.21.15.26:0/1700043714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40074b0 con 0x55a7f64d9980 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 -- 172.21.15.26:0/1700043714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 msgr2=0x55a7f64b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 --2- 172.21.15.26:0/1700043714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64b35c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4058390 tx=0x7fb8e4014000 comp rx=0 tx=0).stop 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 -- 172.21.15.26:0/1700043714 shutdown_connections 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 --2- 172.21.15.26:0/1700043714 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64fbb60 0x55a7f64f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 --2- 172.21.15.26:0/1700043714 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64dec80 0x55a7f64f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 --2- 172.21.15.26:0/1700043714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f64b35c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 -- 172.21.15.26:0/1700043714 >> 172.21.15.26:0/1700043714 conn(0x55a7f64de540 msgr2=0x55a7f64da9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 -- 172.21.15.26:0/1700043714 shutdown_connections 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 -- 172.21.15.26:0/1700043714 wait complete. 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 -- start start 2024-04-24T23:24:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.729+0000 7fb8f9438740 1 Processor -- start 2024-04-24T23:24:25.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f656d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 0x55a7f65130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8f9438740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64fbb60 0x55a7f64d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8f9438740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a7f64ad250 con 0x55a7f64d9980 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8f9438740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a7f64e1200 con 0x55a7f64dec80 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8f9438740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a7f64ad850 con 0x55a7f64fbb60 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8f9438740 10 client.0 _ll_register_callbacks cb 0x7fb8e80149e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f656d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f656d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33748/0 (socket says 172.21.15.26:33748) 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8effff640 1 -- 172.21.15.26:0/3712954103 learned_addr learned my addr 172.21.15.26:0/3712954103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8f4bea640 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64fbb60 0x55a7f64d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.730+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 0x55a7f65130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8effff640 1 -- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64fbb60 msgr2=0x55a7f64d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8effff640 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64fbb60 0x55a7f64d7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8effff640 1 -- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 msgr2=0x55a7f65130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8effff640 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 0x55a7f65130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8effff640 1 -- 172.21.15.26:0/3712954103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f64dff40 con 0x55a7f64d9980 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8f4bea640 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64fbb60 0x55a7f64d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 0x55a7f65130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8effff640 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f656d340 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4011000 tx=0x7fb8e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683d8c2c2086bbe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8ceffd640 1 -- 172.21.15.26:0/3712954103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4054a70 con 0x55a7f64d9980 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8ceffd640 1 -- 172.21.15.26:0/3712954103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb8e40702e0 con 0x55a7f64d9980 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8f9438740 1 -- 172.21.15.26:0/3712954103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e4070130 con 0x55a7f64d9980 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8ceffd640 1 -- 172.21.15.26:0/3712954103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb8e40882b0 con 0x55a7f64d9980 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8f9438740 10 client.6735 fetch_fsmap learned FSMap version 54 2024-04-24T23:24:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8f9438740 1 -- 172.21.15.26:0/3712954103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55a7f64d8770 con 0x55a7f64d9980 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8ceffd640 1 -- 172.21.15.26:0/3712954103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e4070130 con 0x55a7f64d9980 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8ceffd640 1 -- 172.21.15.26:0/3712954103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7fb8e40074b0 con 0x55a7f64d9980 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8f9438740 10 client.6735 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8f9438740 10 client.6735 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8f9438740 20 client.6735 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.731+0000 7fb8f9438740 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55a7f64df170 0x55a7f64df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 4 client.6735 mds_command: new command op to 6531 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 -- 172.21.15.26:0/3712954103 --> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] -- command(tid 0: {"prefix": "status"}) -- 0x55a7f641e790 con 0x55a7f64df170 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55a7f64df170 0x55a7f64df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8ef7fe640 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55a7f64df170 0x55a7f64df550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1fa68a345edd32cb server_cookie=d1947aa38fb0853a in_seq=0 out_seq=0 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8ceffd640 10 client.6735 ms_handle_connect on v2:172.21.15.26:6834/1519104007 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8ceffd640 1 -- 172.21.15.26:0/3712954103 <== mds.2 v2:172.21.15.26:6834/1519104007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55a7f641e790 con 0x55a7f64df170 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8ceffd640 10 client.6735 handle_command_reply: tid=0 2024-04-24T23:24:25.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 client.6735 shutdown 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 -- 172.21.15.26:0/3712954103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 msgr2=0x55a7f656d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f656d340 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4011000 tx=0x7fb8e4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 -- 172.21.15.26:0/3712954103 shutdown_connections 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55a7f64df170 0x55a7f64df550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a7f64fbb60 0x55a7f64d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f64dec80 0x55a7f65130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8ceffd640 0 client.6735 ms_handle_reset on v2:172.21.15.26:6834/1519104007 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 --2- 172.21.15.26:0/3712954103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f64d9980 0x55a7f656d340 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 -- 172.21.15.26:0/3712954103 >> 172.21.15.26:0/3712954103 conn(0x55a7f64de540 msgr2=0x55a7f64f8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.732+0000 7fb8f9438740 1 -- 172.21.15.26:0/3712954103 shutdown_connections 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f9438740 1 -- 172.21.15.26:0/3712954103 wait complete. 2024-04-24T23:24:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f9438740 20 client.6735 trim_cache size 0 max 16384 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6531, 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 1.690616444, 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 54, 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 58, 2024-04-24T23:24:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 58, 2024-04-24T23:24:25.746 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.699609697 2024-04-24T23:24:25.746 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 -- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8dc06be30 msgr2=0x7fb8dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8dc06be30 0x7fb8dc06e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0034000 tx=0x7fb8e002e000 comp rx=0 tx=0).stop 2024-04-24T23:24:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 -- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00b3130 msgr2=0x7fb8f01f8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00b3130 0x7fb8f01f8770 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400c700 tx=0x7fb8e400cb00 comp rx=0 tx=0).stop 2024-04-24T23:24:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 -- 172.21.15.26:0/4204079550 shutdown_connections 2024-04-24T23:24:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb8dc06be30 0x7fb8dc06e2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb8f015e570 0x7fb8f01f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb8f00d4870 0x7fb8f00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 --2- 172.21.15.26:0/4204079550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00b3130 0x7fb8f01f8770 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 -- 172.21.15.26:0/4204079550 >> 172.21.15.26:0/4204079550 conn(0x7fb8f008dbe0 msgr2=0x7fb8f00cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 -- 172.21.15.26:0/4204079550 shutdown_connections 2024-04-24T23:24:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.733+0000 7fb8f664d640 1 -- 172.21.15.26:0/4204079550 wait complete. 2024-04-24T23:24:25.748 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:24:25.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.848+0000 7f7686e22640 1 Processor -- start 2024-04-24T23:24:25.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.848+0000 7f7686e22640 1 -- start start 2024-04-24T23:24:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.848+0000 7f7686e22640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76800cf870 0x7f7680072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.848+0000 7f7686e22640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f768015eb00 0x7f768015ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.848+0000 7f7686e22640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76800cd0b0 con 0x7f7680143c40 2024-04-24T23:24:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.848+0000 7f7686e22640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7680071b70 con 0x7f76800cf870 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.848+0000 7f7686e22640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f768015f470 con 0x7f768015eb00 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.848+0000 7f767ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76800cf870 0x7f7680072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f7684bbe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7680143c40 0x7f76800cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36224/0 (socket says 172.21.15.26:36224) 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f767ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76800cf870 0x7f7680072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33756/0 (socket says 172.21.15.26:33756) 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f7684bbe640 1 -- 172.21.15.26:0/2905301815 learned_addr learned my addr 172.21.15.26:0/2905301815 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f76853bf640 1 --2- 172.21.15.26:0/2905301815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f768015eb00 0x7f768015ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f767ffff640 1 -- 172.21.15.26:0/2905301815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f768015eb00 msgr2=0x7f768015ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f767ffff640 1 --2- 172.21.15.26:0/2905301815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f768015eb00 0x7f768015ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f767ffff640 1 -- 172.21.15.26:0/2905301815 >> v1:172.21.15.26:6789/0 conn(0x7f7680143c40 legacy=0x7f76800cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f767ffff640 1 -- 172.21.15.26:0/2905301815 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7680071240 con 0x7f76800cf870 2024-04-24T23:24:25.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f767f7fe640 1 -- 172.21.15.26:0/2905301815 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914485992 0 0) 0x7f76800cd0b0 con 0x7f7680143c40 2024-04-24T23:24:25.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.849+0000 7f76853bf640 1 --2- 172.21.15.26:0/2905301815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f768015eb00 0x7f768015ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:25.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f767ffff640 1 --2- 172.21.15.26:0/2905301815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 0x7f7680072000 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f766c00ba40 tx=0x7f766c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e42ad0c1e01921 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f767f7fe640 1 -- 172.21.15.26:0/2905301815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c032750 con 0x7f76800cf870 2024-04-24T23:24:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f767f7fe640 1 -- 172.21.15.26:0/2905301815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c030050 con 0x7f76800cf870 2024-04-24T23:24:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f767f7fe640 1 -- 172.21.15.26:0/2905301815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c032b10 con 0x7f76800cf870 2024-04-24T23:24:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f7686e22640 1 -- 172.21.15.26:0/2905301815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 msgr2=0x7f7680072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f7686e22640 1 --2- 172.21.15.26:0/2905301815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 0x7f7680072000 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f766c00ba40 tx=0x7f766c02c850 comp rx=0 tx=0).stop 2024-04-24T23:24:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f7686e22640 1 -- 172.21.15.26:0/2905301815 shutdown_connections 2024-04-24T23:24:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f7686e22640 1 --2- 172.21.15.26:0/2905301815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f768015eb00 0x7f768015ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f7686e22640 1 --2- 172.21.15.26:0/2905301815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 0x7f7680072000 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f7686e22640 1 -- 172.21.15.26:0/2905301815 >> 172.21.15.26:0/2905301815 conn(0x7f768006b300 msgr2=0x7f768006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f7686e22640 1 -- 172.21.15.26:0/2905301815 shutdown_connections 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.850+0000 7f7686e22640 1 -- 172.21.15.26:0/2905301815 wait complete. 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.851+0000 7f7686e22640 1 Processor -- start 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.851+0000 7f7686e22640 1 -- start start 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.852+0000 7f7686e22640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 0x7f7680142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.852+0000 7f7686e22640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7680143c40 0x7f76800abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.852+0000 7f7686e22640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f768015eb00 0x7f76801f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.852+0000 7f7686e22640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7680144020 con 0x7f76800cf870 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.852+0000 7f7686e22640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7680071800 con 0x7f768015eb00 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.852+0000 7f7686e22640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7680072540 con 0x7f7680143c40 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.852+0000 7f7684bbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 0x7f7680142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.852+0000 7f7684bbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 0x7f7680142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33762/0 (socket says 172.21.15.26:33762) 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.852+0000 7f7684bbe640 1 -- 172.21.15.26:0/2936404327 learned_addr learned my addr 172.21.15.26:0/2936404327 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f76853bf640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f768015eb00 0x7f76801f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f767ffff640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7680143c40 0x7f76800abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f7684bbe640 1 -- 172.21.15.26:0/2936404327 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7680143c40 msgr2=0x7f76800abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f7684bbe640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7680143c40 0x7f76800abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f7684bbe640 1 -- 172.21.15.26:0/2936404327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f768015eb00 msgr2=0x7f76801f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f7684bbe640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f768015eb00 0x7f76801f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f7684bbe640 1 -- 172.21.15.26:0/2936404327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7680140ae0 con 0x7f76800cf870 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f7684bbe640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 0x7f7680142da0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f767400c500 tx=0x7f767400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f767d7fa640 1 -- 172.21.15.26:0/2936404327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674018070 con 0x7f76800cf870 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f767d7fa640 1 -- 172.21.15.26:0/2936404327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7674007500 con 0x7f76800cf870 2024-04-24T23:24:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f7686e22640 1 -- 172.21.15.26:0/2936404327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7680140d60 con 0x7f76800cf870 2024-04-24T23:24:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f767d7fa640 1 -- 172.21.15.26:0/2936404327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674004a20 con 0x7f76800cf870 2024-04-24T23:24:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.853+0000 7f7686e22640 1 -- 172.21.15.26:0/2936404327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7680141320 con 0x7f76800cf870 2024-04-24T23:24:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.855+0000 7f767d7fa640 1 -- 172.21.15.26:0/2936404327 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f767401b070 con 0x7f76800cf870 2024-04-24T23:24:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.855+0000 7f767d7fa640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f766406be30 0x7f766406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.855+0000 7f767ffff640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f766406be30 0x7f766406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.855+0000 7f767d7fa640 1 -- 172.21.15.26:0/2936404327 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f767408a020 con 0x7f76800cf870 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.856+0000 7f767ffff640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f766406be30 0x7f766406e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f766c037000 tx=0x7f766c032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.856+0000 7f7689c05740 1 Processor -- start 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.856+0000 7f7689c05740 1 -- start start 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f7689c05740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f7689c05740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f7689c05740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ce540 0x55ee482a3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f7689c05740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee48206f00 con 0x55ee482ebb60 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f7689c05740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ee481af750 con 0x55ee482cec80 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f7689c05740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ee48202ec0 con 0x55ee482ce540 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f7684bbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f7684bbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33772/0 (socket says 172.21.15.26:33772) 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f767ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f7684bbe640 1 -- 172.21.15.26:0/3205747741 learned_addr learned my addr 172.21.15.26:0/3205747741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.857+0000 7f76853bf640 1 --2- 172.21.15.26:0/3205747741 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ce540 0x55ee482a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f7684bbe640 1 -- 172.21.15.26:0/3205747741 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ce540 msgr2=0x55ee482a3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f7684bbe640 1 --2- 172.21.15.26:0/3205747741 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ce540 0x55ee482a3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f7684bbe640 1 -- 172.21.15.26:0/3205747741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 msgr2=0x55ee482ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f7684bbe640 1 --2- 172.21.15.26:0/3205747741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f7684bbe640 1 -- 172.21.15.26:0/3205747741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee47f1b2d0 con 0x55ee482ebb60 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f767ffff640 1 --2- 172.21.15.26:0/3205747741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f7684bbe640 1 --2- 172.21.15.26:0/3205747741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829e4f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7674088550 tx=0x7f76740584d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67b1ed1005f47e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f766a7fc640 1 -- 172.21.15.26:0/3205747741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674056710 con 0x55ee482ebb60 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f766a7fc640 1 -- 172.21.15.26:0/3205747741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7674054a80 con 0x55ee482ebb60 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f766a7fc640 1 -- 172.21.15.26:0/3205747741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674070370 con 0x55ee482ebb60 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f7689c05740 1 -- 172.21.15.26:0/3205747741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 msgr2=0x55ee4829e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.858+0000 7f7689c05740 1 --2- 172.21.15.26:0/3205747741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829e4f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7674088550 tx=0x7f76740584d0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.859+0000 7f7689c05740 1 -- 172.21.15.26:0/3205747741 shutdown_connections 2024-04-24T23:24:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.859+0000 7f7689c05740 1 --2- 172.21.15.26:0/3205747741 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ce540 0x55ee482a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.859+0000 7f7689c05740 1 --2- 172.21.15.26:0/3205747741 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.859+0000 7f7689c05740 1 --2- 172.21.15.26:0/3205747741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829e4f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.859+0000 7f7689c05740 1 -- 172.21.15.26:0/3205747741 >> 172.21.15.26:0/3205747741 conn(0x55ee482a0760 msgr2=0x55ee482e9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.859+0000 7f7689c05740 1 -- 172.21.15.26:0/3205747741 shutdown_connections 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.859+0000 7f7689c05740 1 -- 172.21.15.26:0/3205747741 wait complete. 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.859+0000 7f7689c05740 1 -- start start 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.859+0000 7f7689c05740 1 Processor -- start 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f7689c05740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f7689c05740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ebb60 0x55ee4829d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f7689c05740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482d1120 0x55ee482e0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f7689c05740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee482a2910 con 0x55ee482d1120 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f7689c05740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ee481c2cf0 con 0x55ee482cec80 2024-04-24T23:24:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f7689c05740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ee482a1d00 con 0x55ee482ebb60 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f7689c05740 10 client.0 _ll_register_callbacks cb 0x55ee481c26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f76853bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482d1120 0x55ee482e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f7684bbe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f76853bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482d1120 0x55ee482e0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33774/0 (socket says 172.21.15.26:33774) 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f767ffff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ebb60 0x55ee4829d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.860+0000 7f76853bf640 1 -- 172.21.15.26:0/2907973839 learned_addr learned my addr 172.21.15.26:0/2907973839 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7684bbe640 1 -- 172.21.15.26:0/2907973839 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ebb60 msgr2=0x55ee4829d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7684bbe640 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ebb60 0x55ee4829d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7684bbe640 1 -- 172.21.15.26:0/2907973839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482d1120 msgr2=0x55ee482e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7684bbe640 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482d1120 0x55ee482e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7684bbe640 1 -- 172.21.15.26:0/2907973839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee47fe56b0 con 0x55ee482cec80 2024-04-24T23:24:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f767ffff640 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ebb60 0x55ee4829d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7684bbe640 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482f90b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7674058110 tx=0x7f7674000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86cbe47f4d0a32d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f765f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f765f7fe640 1 -- 172.21.15.26:0/2907973839 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674004bc0 con 0x55ee482cec80 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f765f7fe640 1 -- 172.21.15.26:0/2907973839 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7674004d60 con 0x55ee482cec80 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7689c05740 1 -- 172.21.15.26:0/2907973839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7674054a80 con 0x55ee482cec80 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f765f7fe640 1 -- 172.21.15.26:0/2907973839 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7674057860 con 0x55ee482cec80 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7689c05740 10 client.5998 fetch_fsmap learned FSMap version 54 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7689c05740 1 -- 172.21.15.26:0/2907973839 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55ee482e8660 con 0x55ee482cec80 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f765f7fe640 1 -- 172.21.15.26:0/2907973839 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7674054a80 con 0x55ee482cec80 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f765f7fe640 1 -- 172.21.15.26:0/2907973839 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7f7674057a00 con 0x55ee482cec80 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7689c05740 10 client.5998 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7689c05740 10 client.5998 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:24:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7689c05740 20 client.5998 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7689c05740 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x55ee482d1540 0x55ee482a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7689c05740 4 client.5998 mds_command: new command op to 6534 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.861+0000 7f7689c05740 1 -- 172.21.15.26:0/2907973839 --> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee48214440 con 0x55ee482d1540 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.862+0000 7f767ffff640 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x55ee482d1540 0x55ee482a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.862+0000 7f765f7fe640 10 client.5998 ms_handle_connect on v2:172.21.15.26:6835/2743842996 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.862+0000 7f767ffff640 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x55ee482d1540 0x55ee482a2d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b6864a197acbd11 server_cookie=a66437fde1a19fa4 in_seq=0 out_seq=0 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.862+0000 7f765f7fe640 1 -- 172.21.15.26:0/2907973839 <== mds.1 v2:172.21.15.26:6835/2743842996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee48214440 con 0x55ee482d1540 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.862+0000 7f765f7fe640 10 client.5998 handle_command_reply: tid=0 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 client.5998 shutdown 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 -- 172.21.15.26:0/2907973839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 msgr2=0x55ee482f90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482f90b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7674058110 tx=0x7f7674000ed0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 -- 172.21.15.26:0/2907973839 shutdown_connections 2024-04-24T23:24:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x55ee482d1540 0x55ee482a2d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482d1120 0x55ee482e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482ebb60 0x55ee4829d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 --2- 172.21.15.26:0/2907973839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482cec80 0x55ee482f90b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f765f7fe640 0 client.5998 ms_handle_reset on v2:172.21.15.26:6835/2743842996 2024-04-24T23:24:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 -- 172.21.15.26:0/2907973839 >> 172.21.15.26:0/2907973839 conn(0x55ee482ce540 msgr2=0x55ee482eb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 -- 172.21.15.26:0/2907973839 shutdown_connections 2024-04-24T23:24:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 1 -- 172.21.15.26:0/2907973839 wait complete. 2024-04-24T23:24:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.863+0000 7f7689c05740 20 client.5998 trim_cache size 0 max 16384 2024-04-24T23:24:25.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.881+0000 7f7689c05740 1 Processor -- start 2024-04-24T23:24:25.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.881+0000 7f7689c05740 1 -- start start 2024-04-24T23:24:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f7689c05740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 0x55ee482e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f7689c05740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f7689c05740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f7689c05740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76801fb400 con 0x55ee482ebb60 2024-04-24T23:24:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f7689c05740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f76801fbad0 con 0x55ee482d1120 2024-04-24T23:24:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f7689c05740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f76800ae100 con 0x55ee482cec80 2024-04-24T23:24:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f76853bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f76853bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33778/0 (socket says 172.21.15.26:33778) 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f76853bf640 1 -- 172.21.15.26:0/88287439 learned_addr learned my addr 172.21.15.26:0/88287439 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f7684bbe640 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 0x55ee482e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f767ffff640 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f76853bf640 1 -- 172.21.15.26:0/88287439 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 msgr2=0x55ee482e8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f76853bf640 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 0x55ee482e8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f76853bf640 1 -- 172.21.15.26:0/88287439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 msgr2=0x55ee4829d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f76853bf640 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f76853bf640 1 -- 172.21.15.26:0/88287439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee482cb190 con 0x55ee482ebb60 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.882+0000 7f767ffff640 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f76853bf640 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7678015860 tx=0x7f7678009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cca84efd4ad3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7669ffb640 1 -- 172.21.15.26:0/88287439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678024070 con 0x55ee482ebb60 2024-04-24T23:24:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7669ffb640 1 -- 172.21.15.26:0/88287439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7678017da0 con 0x55ee482ebb60 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7669ffb640 1 -- 172.21.15.26:0/88287439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767801f670 con 0x55ee482ebb60 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7689c05740 1 -- 172.21.15.26:0/88287439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 msgr2=0x55ee4829d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7689c05740 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7678015860 tx=0x7f7678009750 comp rx=0 tx=0).stop 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7689c05740 1 -- 172.21.15.26:0/88287439 shutdown_connections 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7689c05740 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7689c05740 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7689c05740 1 --2- 172.21.15.26:0/88287439 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 0x55ee482e8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7689c05740 1 -- 172.21.15.26:0/88287439 >> 172.21.15.26:0/88287439 conn(0x55ee482ce540 msgr2=0x55ee482c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7689c05740 1 -- 172.21.15.26:0/88287439 shutdown_connections 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.883+0000 7f7689c05740 1 -- 172.21.15.26:0/88287439 wait complete. 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.884+0000 7f7689c05740 1 -- start start 2024-04-24T23:24:25.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.884+0000 7f7689c05740 1 Processor -- start 2024-04-24T23:24:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f7689c05740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 0x55ee48364950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f7689c05740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f7689c05740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f7689c05740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee481c2cf0 con 0x55ee482ebb60 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f7689c05740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ee482a1d00 con 0x55ee482d1120 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f7689c05740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ee482a2910 con 0x55ee482cec80 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f7689c05740 10 client.0 _ll_register_callbacks cb 0x7f7678009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f76853bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f76853bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33792/0 (socket says 172.21.15.26:33792) 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f76853bf640 1 -- 172.21.15.26:0/3254477291 learned_addr learned my addr 172.21.15.26:0/3254477291 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f767ffff640 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f7684bbe640 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 0x55ee48364950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f76853bf640 1 -- 172.21.15.26:0/3254477291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 msgr2=0x55ee48364950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f76853bf640 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 0x55ee48364950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f76853bf640 1 -- 172.21.15.26:0/3254477291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 msgr2=0x55ee4829d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f76853bf640 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f76853bf640 1 -- 172.21.15.26:0/3254477291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee482d24d0 con 0x55ee482ebb60 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f767ffff640 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f76853bf640 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7678022040 tx=0x7f7678004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162bdeb6dba65e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.885+0000 7f765f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f765f7fe640 1 -- 172.21.15.26:0/3254477291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767802e070 con 0x55ee482ebb60 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7689c05740 1 -- 172.21.15.26:0/3254477291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7678017da0 con 0x55ee482ebb60 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f765f7fe640 1 -- 172.21.15.26:0/3254477291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f767801f920 con 0x55ee482ebb60 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f765f7fe640 1 -- 172.21.15.26:0/3254477291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7678024040 con 0x55ee482ebb60 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7689c05740 10 client.6756 fetch_fsmap learned FSMap version 54 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7689c05740 1 -- 172.21.15.26:0/3254477291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55ee482a2740 con 0x55ee482ebb60 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f765f7fe640 1 -- 172.21.15.26:0/3254477291 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7678017da0 con 0x55ee482ebb60 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f765f7fe640 1 -- 172.21.15.26:0/3254477291 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7f7678029460 con 0x55ee482ebb60 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7689c05740 10 client.6756 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T23:24:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7689c05740 10 client.6756 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7689c05740 20 client.6756 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7689c05740 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x55ee482cfa60 0x55ee4836a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7689c05740 4 client.6756 mds_command: new command op to 6534 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7689c05740 1 -- 172.21.15.26:0/3254477291 --> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] -- command(tid 0: {"prefix": "status"}) -- 0x55ee4820e790 con 0x55ee482cfa60 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7684bbe640 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x55ee482cfa60 0x55ee4836a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f7684bbe640 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x55ee482cfa60 0x55ee4836a9a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62d134469e3add7c server_cookie=7760e93d195f6554 in_seq=0 out_seq=0 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.886+0000 7f765f7fe640 10 client.6756 ms_handle_connect on v2:172.21.15.26:6835/2743842996 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f765f7fe640 1 -- 172.21.15.26:0/3254477291 <== mds.1 v2:172.21.15.26:6835/2743842996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ee4820e790 con 0x55ee482cfa60 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f765f7fe640 10 client.6756 handle_command_reply: tid=0 2024-04-24T23:24:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 client.6756 shutdown 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 -- 172.21.15.26:0/3254477291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 msgr2=0x55ee4829d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7678022040 tx=0x7f7678004ab0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 -- 172.21.15.26:0/3254477291 shutdown_connections 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x55ee482cfa60 0x55ee4836a9a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee482ebb60 0x55ee4829d9a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ee482d1120 0x55ee4829d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f765f7fe640 0 client.6756 ms_handle_reset on v2:172.21.15.26:6835/2743842996 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 --2- 172.21.15.26:0/3254477291 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ee482cec80 0x55ee48364950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 -- 172.21.15.26:0/3254477291 >> 172.21.15.26:0/3254477291 conn(0x55ee482ce540 msgr2=0x55ee482ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 -- 172.21.15.26:0/3254477291 shutdown_connections 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 1 -- 172.21.15.26:0/3254477291 wait complete. 2024-04-24T23:24:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7689c05740 20 client.6756 trim_cache size 0 max 16384 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6534, 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 2.8506050919999999, 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 54, 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 58, 2024-04-24T23:24:25.897 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 58, 2024-04-24T23:24:25.898 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.853500819000001 2024-04-24T23:24:25.898 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7686e22640 1 -- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f766406be30 msgr2=0x7f766406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7686e22640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f766406be30 0x7f766406e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f766c037000 tx=0x7f766c032000 comp rx=0 tx=0).stop 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7686e22640 1 -- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 msgr2=0x7f7680142da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.887+0000 7f7686e22640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 0x7f7680142da0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f767400c500 tx=0x7f767400c900 comp rx=0 tx=0).stop 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.888+0000 7f7686e22640 1 -- 172.21.15.26:0/2936404327 shutdown_connections 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.888+0000 7f7686e22640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f766406be30 0x7f766406e2f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.888+0000 7f7686e22640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f768015eb00 0x7f76801f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.888+0000 7f7686e22640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7680143c40 0x7f76800abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.888+0000 7f7686e22640 1 --2- 172.21.15.26:0/2936404327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76800cf870 0x7f7680142da0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.888+0000 7f7686e22640 1 -- 172.21.15.26:0/2936404327 >> 172.21.15.26:0/2936404327 conn(0x7f768006b300 msgr2=0x7f76800cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.888+0000 7f7686e22640 1 -- 172.21.15.26:0/2936404327 shutdown_connections 2024-04-24T23:24:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:25.888+0000 7f7686e22640 1 -- 172.21.15.26:0/2936404327 wait complete. 2024-04-24T23:24:25.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:24:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.000+0000 7ff74f9c1640 1 Processor -- start 2024-04-24T23:24:26.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.000+0000 7ff74f9c1640 1 -- start start 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74f9c1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff748143c40 0x7ff7480cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74f9c1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff74815eb00 0x7ff74815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74f9c1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff74815f450 con 0x7ff7480cc860 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74f9c1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff748071b70 con 0x7ff748143c40 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74f9c1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff7480cd0b0 con 0x7ff74815eb00 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74d75d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff748143c40 0x7ff7480cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74d75d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff748143c40 0x7ff7480cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33800/0 (socket says 172.21.15.26:33800) 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74d75d640 1 -- 172.21.15.26:0/2267360474 learned_addr learned my addr 172.21.15.26:0/2267360474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74df5e640 1 --2- 172.21.15.26:0/2267360474 >> v2:172.21.15.29:3300/0 conn(0x7ff74815eb00 0x7ff74815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74d75d640 1 -- 172.21.15.26:0/2267360474 >> v2:172.21.15.29:3300/0 conn(0x7ff74815eb00 msgr2=0x7ff74815ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74d75d640 1 --2- 172.21.15.26:0/2267360474 >> v2:172.21.15.29:3300/0 conn(0x7ff74815eb00 0x7ff74815ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74d75d640 1 -- 172.21.15.26:0/2267360474 >> v1:172.21.15.26:6789/0 conn(0x7ff7480cc860 legacy=0x7ff748072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74d75d640 1 -- 172.21.15.26:0/2267360474 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff748071240 con 0x7ff748143c40 2024-04-24T23:24:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff73ffff640 1 -- 172.21.15.26:0/2267360474 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3724086436 0 0) 0x7ff738001110 con 0x7ff7480cc860 2024-04-24T23:24:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74df5e640 1 --2- 172.21.15.26:0/2267360474 >> v2:172.21.15.29:3300/0 conn(0x7ff74815eb00 0x7ff74815ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff74d75d640 1 --2- 172.21.15.26:0/2267360474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 0x7ff7480cf870 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff748144d10 tx=0x7ff74402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9c00f53798b933 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff73ffff640 1 -- 172.21.15.26:0/2267360474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744037070 con 0x7ff748143c40 2024-04-24T23:24:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff73ffff640 1 -- 172.21.15.26:0/2267360474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff744032440 con 0x7ff748143c40 2024-04-24T23:24:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.002+0000 7ff73ffff640 1 -- 172.21.15.26:0/2267360474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744032810 con 0x7ff748143c40 2024-04-24T23:24:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.003+0000 7ff74f9c1640 1 -- 172.21.15.26:0/2267360474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 msgr2=0x7ff7480cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.003+0000 7ff74f9c1640 1 --2- 172.21.15.26:0/2267360474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 0x7ff7480cf870 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff748144d10 tx=0x7ff74402a270 comp rx=0 tx=0).stop 2024-04-24T23:24:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.003+0000 7ff74f9c1640 1 -- 172.21.15.26:0/2267360474 shutdown_connections 2024-04-24T23:24:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.003+0000 7ff74f9c1640 1 --2- 172.21.15.26:0/2267360474 >> v2:172.21.15.29:3300/0 conn(0x7ff74815eb00 0x7ff74815ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.003+0000 7ff74f9c1640 1 --2- 172.21.15.26:0/2267360474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 0x7ff7480cf870 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.003+0000 7ff74f9c1640 1 -- 172.21.15.26:0/2267360474 >> 172.21.15.26:0/2267360474 conn(0x7ff74806b300 msgr2=0x7ff74806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.003+0000 7ff74f9c1640 1 -- 172.21.15.26:0/2267360474 shutdown_connections 2024-04-24T23:24:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.003+0000 7ff74f9c1640 1 -- 172.21.15.26:0/2267360474 wait complete. 2024-04-24T23:24:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74f9c1640 1 Processor -- start 2024-04-24T23:24:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74f9c1640 1 -- start start 2024-04-24T23:24:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74f9c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff7480cc860 0x7ff74815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74f9c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 0x7ff7481429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74f9c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74815eb00 0x7ff7480aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74f9c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff748144020 con 0x7ff748143c40 2024-04-24T23:24:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74f9c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff748071800 con 0x7ff74815eb00 2024-04-24T23:24:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74f9c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff7480cccd0 con 0x7ff7480cc860 2024-04-24T23:24:26.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74cf5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 0x7ff7481429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74cf5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 0x7ff7481429b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33806/0 (socket says 172.21.15.26:33806) 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74cf5c640 1 -- 172.21.15.26:0/1418629260 learned_addr learned my addr 172.21.15.26:0/1418629260 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74df5e640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74815eb00 0x7ff7480aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74d75d640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff7480cc860 0x7ff74815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74cf5c640 1 -- 172.21.15.26:0/1418629260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff7480cc860 msgr2=0x7ff74815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff7480cc860 0x7ff74815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74cf5c640 1 -- 172.21.15.26:0/1418629260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74815eb00 msgr2=0x7ff7480aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74815eb00 0x7ff7480aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74cf5c640 1 -- 172.21.15.26:0/1418629260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7480adbe0 con 0x7ff748143c40 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.005+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 0x7ff7481429b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c750 tx=0x7ff73800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.006+0000 7ff73dffb640 1 -- 172.21.15.26:0/1418629260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738017070 con 0x7ff748143c40 2024-04-24T23:24:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.006+0000 7ff73dffb640 1 -- 172.21.15.26:0/1418629260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7380052b0 con 0x7ff748143c40 2024-04-24T23:24:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.006+0000 7ff74f9c1640 1 -- 172.21.15.26:0/1418629260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7480adec0 con 0x7ff748143c40 2024-04-24T23:24:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.006+0000 7ff73dffb640 1 -- 172.21.15.26:0/1418629260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73800a880 con 0x7ff748143c40 2024-04-24T23:24:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.006+0000 7ff74f9c1640 1 -- 172.21.15.26:0/1418629260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7480ae480 con 0x7ff748143c40 2024-04-24T23:24:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.007+0000 7ff73dffb640 1 -- 172.21.15.26:0/1418629260 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff73801a070 con 0x7ff748143c40 2024-04-24T23:24:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.008+0000 7ff73dffb640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff72c06be30 0x7ff72c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.008+0000 7ff73dffb640 1 -- 172.21.15.26:0/1418629260 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff73808a090 con 0x7ff748143c40 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.008+0000 7ff74d75d640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff72c06be30 0x7ff72c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.008+0000 7ff74d75d640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff72c06be30 0x7ff72c06e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff744034000 tx=0x7ff74402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.009+0000 7ff7527a9740 1 Processor -- start 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.009+0000 7ff7527a9740 1 -- start start 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.009+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ad4b60 0x565312a874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.009+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ab7c80 0x565312ab7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.009+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ab7540 0x565312a8c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.009+0000 7ff7527a9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5653129eff00 con 0x565312ad4b60 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.009+0000 7ff7527a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x565312998750 con 0x565312ab7c80 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.009+0000 7ff7527a9740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5653129ebec0 con 0x565312ab7540 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74d75d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ad4b60 0x565312a874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74cf5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ab7c80 0x565312ab7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74d75d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ad4b60 0x565312a874f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33808/0 (socket says 172.21.15.26:33808) 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74d75d640 1 -- 172.21.15.26:0/1302592614 learned_addr learned my addr 172.21.15.26:0/1302592614 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74df5e640 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ab7540 0x565312a8c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74d75d640 1 -- 172.21.15.26:0/1302592614 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ab7540 msgr2=0x565312a8c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74d75d640 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ab7540 0x565312a8c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74d75d640 1 -- 172.21.15.26:0/1302592614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ab7c80 msgr2=0x565312ab7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74d75d640 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ab7c80 0x565312ab7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74d75d640 1 -- 172.21.15.26:0/1302592614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653127042d0 con 0x565312ad4b60 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ab7c80 0x565312ab7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74df5e640 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ab7540 0x565312a8c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff74d75d640 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ad4b60 0x565312a874f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff74403e040 tx=0x7ff74402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a143bca6495e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff732ffd640 1 -- 172.21.15.26:0/1302592614 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744037040 con 0x565312ad4b60 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff732ffd640 1 -- 172.21.15.26:0/1302592614 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74404e070 con 0x565312ad4b60 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.010+0000 7ff732ffd640 1 -- 172.21.15.26:0/1302592614 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744032db0 con 0x565312ad4b60 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 -- 172.21.15.26:0/1302592614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ad4b60 msgr2=0x565312a874f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ad4b60 0x565312a874f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff74403e040 tx=0x7ff74402c000 comp rx=0 tx=0).stop 2024-04-24T23:24:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 -- 172.21.15.26:0/1302592614 shutdown_connections 2024-04-24T23:24:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ab7540 0x565312a8c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ab7c80 0x565312ab7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1302592614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ad4b60 0x565312a874f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 -- 172.21.15.26:0/1302592614 >> 172.21.15.26:0/1302592614 conn(0x565312a89760 msgr2=0x565312ad2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 -- 172.21.15.26:0/1302592614 shutdown_connections 2024-04-24T23:24:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 -- 172.21.15.26:0/1302592614 wait complete. 2024-04-24T23:24:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.011+0000 7ff7527a9740 1 -- start start 2024-04-24T23:24:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.012+0000 7ff7527a9740 1 Processor -- start 2024-04-24T23:24:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.012+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312ae5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.012+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ad4b60 0x565312a86250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.012+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 0x565312a86790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.012+0000 7ff7527a9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565312a8b910 con 0x565312ab7c80 2024-04-24T23:24:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.012+0000 7ff7527a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5653129abcf0 con 0x565312ad4b60 2024-04-24T23:24:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.012+0000 7ff7527a9740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x565312a8ad00 con 0x565312aa78b0 2024-04-24T23:24:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff7527a9740 10 client.0 _ll_register_callbacks cb 0x5653129ab6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:26.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74d75d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312ae5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74d75d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312ae5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33812/0 (socket says 172.21.15.26:33812) 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74d75d640 1 -- 172.21.15.26:0/1124260486 learned_addr learned my addr 172.21.15.26:0/1124260486 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ad4b60 0x565312a86250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74df5e640 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 0x565312a86790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74d75d640 1 -- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 msgr2=0x565312a86790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74d75d640 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 0x565312a86790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74d75d640 1 -- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ad4b60 msgr2=0x565312a86250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74d75d640 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ad4b60 0x565312a86250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74d75d640 1 -- 172.21.15.26:0/1124260486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653127ce6b0 con 0x565312ab7c80 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff74d75d640 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312ae5fc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff7440090b0 tx=0x7ff744004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad70dcc3639a9a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff727fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff727fff640 1 -- 172.21.15.26:0/1124260486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744058070 con 0x565312ab7c80 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff727fff640 1 -- 172.21.15.26:0/1124260486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff744049890 con 0x565312ab7c80 2024-04-24T23:24:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff7527a9740 1 -- 172.21.15.26:0/1124260486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff74404e070 con 0x565312ab7c80 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff727fff640 1 -- 172.21.15.26:0/1124260486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff744063050 con 0x565312ab7c80 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff7527a9740 10 client.6771 fetch_fsmap learned FSMap version 54 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.013+0000 7ff7527a9740 1 -- 172.21.15.26:0/1124260486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x565312aba6f0 con 0x565312ab7c80 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff727fff640 1 -- 172.21.15.26:0/1124260486 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74404e070 con 0x565312ab7c80 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff727fff640 1 -- 172.21.15.26:0/1124260486 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7ff744066020 con 0x565312ab7c80 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff7527a9740 10 client.6771 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff7527a9740 10 client.6771 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff7527a9740 20 client.6771 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x565312ac92c0 0x565312ac96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff7527a9740 4 client.6771 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff7527a9740 1 -- 172.21.15.26:0/1124260486 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653129fd440 con 0x565312ac92c0 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x565312ac92c0 0x565312ac96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x565312ac92c0 0x565312ac96a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ebb12e89cc320b7 server_cookie=d08553dc007737f9 in_seq=0 out_seq=0 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.014+0000 7ff727fff640 10 client.6771 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff727fff640 1 -- 172.21.15.26:0/1124260486 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653129fd440 con 0x565312ac92c0 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff727fff640 10 client.6771 handle_command_reply: tid=0 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 client.6771 shutdown 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 -- 172.21.15.26:0/1124260486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 msgr2=0x565312ae5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312ae5fc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff7440090b0 tx=0x7ff744004730 comp rx=0 tx=0).stop 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 -- 172.21.15.26:0/1124260486 shutdown_connections 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x565312ac92c0 0x565312ac96a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 0x565312a86790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff727fff640 0 client.6771 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ad4b60 0x565312a86250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 --2- 172.21.15.26:0/1124260486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312ae5fc0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 -- 172.21.15.26:0/1124260486 >> 172.21.15.26:0/1124260486 conn(0x565312ab7540 msgr2=0x565312ab4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 -- 172.21.15.26:0/1124260486 shutdown_connections 2024-04-24T23:24:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 1 -- 172.21.15.26:0/1124260486 wait complete. 2024-04-24T23:24:26.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.015+0000 7ff7527a9740 20 client.6771 trim_cache size 0 max 16384 2024-04-24T23:24:26.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.034+0000 7ff7527a9740 1 Processor -- start 2024-04-24T23:24:26.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.034+0000 7ff7527a9740 1 -- start start 2024-04-24T23:24:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.034+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 0x565312ab2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.034+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312a8cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.034+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ac92c0 0x565312a8d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.034+0000 7ff7527a9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481ff990 con 0x565312ab7c80 2024-04-24T23:24:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.034+0000 7ff7527a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff748200060 con 0x565312ac92c0 2024-04-24T23:24:26.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.034+0000 7ff7527a9740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff74806ddd0 con 0x565312aa78b0 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74cf5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312a8cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74cf5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312a8cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33814/0 (socket says 172.21.15.26:33814) 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74cf5c640 1 -- 172.21.15.26:0/475759347 learned_addr learned my addr 172.21.15.26:0/475759347 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74d75d640 1 --2- 172.21.15.26:0/475759347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 0x565312ab2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74df5e640 1 --2- 172.21.15.26:0/475759347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ac92c0 0x565312a8d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74cf5c640 1 -- 172.21.15.26:0/475759347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 msgr2=0x565312ab2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/475759347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 0x565312ab2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74cf5c640 1 -- 172.21.15.26:0/475759347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ac92c0 msgr2=0x565312a8d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/475759347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ac92c0 0x565312a8d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74cf5c640 1 -- 172.21.15.26:0/475759347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565312aba9d0 con 0x565312ab7c80 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/475759347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312a8cb30 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff73800a430 tx=0x7ff738088480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d974eedfb2307fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff7327fc640 1 -- 172.21.15.26:0/475759347 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7380566e0 con 0x565312ab7c80 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff7327fc640 1 -- 172.21.15.26:0/475759347 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff738077300 con 0x565312ab7c80 2024-04-24T23:24:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.035+0000 7ff7327fc640 1 -- 172.21.15.26:0/475759347 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738054a50 con 0x565312ab7c80 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 -- 172.21.15.26:0/475759347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 msgr2=0x565312a8cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 --2- 172.21.15.26:0/475759347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312a8cb30 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff73800a430 tx=0x7ff738088480 comp rx=0 tx=0).stop 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 -- 172.21.15.26:0/475759347 shutdown_connections 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 --2- 172.21.15.26:0/475759347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312ac92c0 0x565312a8d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 --2- 172.21.15.26:0/475759347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312a8cb30 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 --2- 172.21.15.26:0/475759347 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312aa78b0 0x565312ab2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 -- 172.21.15.26:0/475759347 >> 172.21.15.26:0/475759347 conn(0x565312ab7540 msgr2=0x565312a82690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 -- 172.21.15.26:0/475759347 shutdown_connections 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 -- 172.21.15.26:0/475759347 wait complete. 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 -- start start 2024-04-24T23:24:26.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.036+0000 7ff7527a9740 1 Processor -- start 2024-04-24T23:24:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312aa78b0 0x565312b45e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312af4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff7527a9740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ac92c0 0x565312ae5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff7527a9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565312aca1a0 con 0x565312ab7c80 2024-04-24T23:24:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff7527a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x565312a8b910 con 0x565312aa78b0 2024-04-24T23:24:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff7527a9740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x565312ab8760 con 0x565312ac92c0 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff7527a9740 10 client.0 _ll_register_callbacks cb 0x7ff7440096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff74cf5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312af4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff74d75d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312aa78b0 0x565312b45e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff74cf5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312af4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33816/0 (socket says 172.21.15.26:33816) 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff74d75d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312aa78b0 0x565312b45e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:38668/0 (socket says 172.21.15.26:38668) 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff74cf5c640 1 -- 172.21.15.26:0/4126665349 learned_addr learned my addr 172.21.15.26:0/4126665349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.037+0000 7ff74df5e640 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ac92c0 0x565312ae5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff74cf5c640 1 -- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ac92c0 msgr2=0x565312ae5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ac92c0 0x565312ae5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff74cf5c640 1 -- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312aa78b0 msgr2=0x565312b45e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312aa78b0 0x565312b45e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff74cf5c640 1 -- 172.21.15.26:0/4126665349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565312b4ca50 con 0x565312ab7c80 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff74d75d640 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312aa78b0 0x565312b45e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff74cf5c640 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312af4a90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff738058370 tx=0x7ff73801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d99dd63cac48aec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff727fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff727fff640 1 -- 172.21.15.26:0/4126665349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738088580 con 0x565312ab7c80 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff727fff640 1 -- 172.21.15.26:0/4126665349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff738070190 con 0x565312ab7c80 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff7527a9740 1 -- 172.21.15.26:0/4126665349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff738077300 con 0x565312ab7c80 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff727fff640 1 -- 172.21.15.26:0/4126665349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff738088720 con 0x565312ab7c80 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff7527a9740 10 client.6777 fetch_fsmap learned FSMap version 54 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff7527a9740 1 -- 172.21.15.26:0/4126665349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x565312ab96c0 con 0x565312ab7c80 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff727fff640 1 -- 172.21.15.26:0/4126665349 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff738077300 con 0x565312ab7c80 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff727fff640 1 -- 172.21.15.26:0/4126665349 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7ff738060a80 con 0x565312ab7c80 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff7527a9740 10 client.6777 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff7527a9740 10 client.6777 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff7527a9740 20 client.6777 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff7527a9740 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x565312ad4b60 0x565312a86470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff7527a9740 4 client.6777 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:24:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.038+0000 7ff7527a9740 1 -- 172.21.15.26:0/4126665349 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "status"}) -- 0x565312a88bb0 con 0x565312ad4b60 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff74d75d640 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x565312ad4b60 0x565312a86470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff727fff640 10 client.6777 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff74d75d640 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x565312ad4b60 0x565312a86470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28d7d81141f5d51b server_cookie=8c57cd517f75f38 in_seq=0 out_seq=0 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff727fff640 1 -- 172.21.15.26:0/4126665349 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x565312a88bb0 con 0x565312ad4b60 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff727fff640 10 client.6777 handle_command_reply: tid=0 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 client.6777 shutdown 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 -- 172.21.15.26:0/4126665349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 msgr2=0x565312af4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312af4a90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff738058370 tx=0x7ff73801e040 comp rx=0 tx=0).stop 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 -- 172.21.15.26:0/4126665349 shutdown_connections 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x565312ad4b60 0x565312a86470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565312ac92c0 0x565312ae5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565312ab7c80 0x565312af4a90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff727fff640 0 client.6777 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 --2- 172.21.15.26:0/4126665349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565312aa78b0 0x565312b45e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 -- 172.21.15.26:0/4126665349 >> 172.21.15.26:0/4126665349 conn(0x565312ab7540 msgr2=0x565312abae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 -- 172.21.15.26:0/4126665349 shutdown_connections 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 1 -- 172.21.15.26:0/4126665349 wait complete. 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.039+0000 7ff7527a9740 20 client.6777 trim_cache size 0 max 16384 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6546, 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 10.706817081000001, 2024-04-24T23:24:26.048 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 54, 2024-04-24T23:24:26.049 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 58, 2024-04-24T23:24:26.049 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 58, 2024-04-24T23:24:26.049 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.990382577 2024-04-24T23:24:26.049 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:26.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 -- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff72c06be30 msgr2=0x7ff72c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff72c06be30 0x7ff72c06e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff744034000 tx=0x7ff74402e000 comp rx=0 tx=0).stop 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 -- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 msgr2=0x7ff7481429b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 0x7ff7481429b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c750 tx=0x7ff73800cb50 comp rx=0 tx=0).stop 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 -- 172.21.15.26:0/1418629260 shutdown_connections 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff72c06be30 0x7ff72c06e2f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff74815eb00 0x7ff7480aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748143c40 0x7ff7481429b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 --2- 172.21.15.26:0/1418629260 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff7480cc860 0x7ff74815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 -- 172.21.15.26:0/1418629260 >> 172.21.15.26:0/1418629260 conn(0x7ff74806b300 msgr2=0x7ff7480b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 -- 172.21.15.26:0/1418629260 shutdown_connections 2024-04-24T23:24:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.040+0000 7ff74f9c1640 1 -- 172.21.15.26:0/1418629260 wait complete. 2024-04-24T23:24:26.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:24:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.152+0000 7f8648e4d640 1 Processor -- start 2024-04-24T23:24:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.152+0000 7f8648e4d640 1 -- start start 2024-04-24T23:24:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f8648e4d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8644143c40 0x7f86440cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f8648e4d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86440cd0b0 con 0x7f86440cc860 2024-04-24T23:24:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f8648e4d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8644071240 con 0x7f864415eb00 2024-04-24T23:24:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f8648e4d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8644071b70 con 0x7f8644143c40 2024-04-24T23:24:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f864259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8644143c40 0x7f86440cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f8641d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86440cc860 0x7f8644072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36248/0 (socket says 172.21.15.26:36248) 2024-04-24T23:24:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f864259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8644143c40 0x7f86440cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33826/0 (socket says 172.21.15.26:33826) 2024-04-24T23:24:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f8641d9b640 1 -- 172.21.15.26:0/909494504 learned_addr learned my addr 172.21.15.26:0/909494504 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f864259c640 1 -- 172.21.15.26:0/909494504 >> v1:172.21.15.29:6789/0 conn(0x7f864415eb00 legacy=0x7f864415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f864259c640 1 -- 172.21.15.26:0/909494504 >> v1:172.21.15.26:6789/0 conn(0x7f86440cc860 legacy=0x7f8644072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f864159a640 1 -- 172.21.15.26:0/909494504 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1450535030 0 0) 0x7f86440cd0b0 con 0x7f86440cc860 2024-04-24T23:24:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.153+0000 7f864259c640 1 -- 172.21.15.26:0/909494504 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8644071520 con 0x7f8644143c40 2024-04-24T23:24:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.154+0000 7f864259c640 1 --2- 172.21.15.26:0/909494504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8644143c40 0x7f86440cf870 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8634009d90 tx=0x7f863402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325aa477f05da9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.154+0000 7f864159a640 1 -- 172.21.15.26:0/909494504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634007740 con 0x7f8644143c40 2024-04-24T23:24:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.154+0000 7f864159a640 1 -- 172.21.15.26:0/909494504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86340078e0 con 0x7f8644143c40 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.154+0000 7f864159a640 1 -- 172.21.15.26:0/909494504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634007cb0 con 0x7f8644143c40 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.155+0000 7f8648e4d640 1 -- 172.21.15.26:0/909494504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8644143c40 msgr2=0x7f86440cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.155+0000 7f8648e4d640 1 --2- 172.21.15.26:0/909494504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8644143c40 0x7f86440cf870 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8634009d90 tx=0x7f863402a6f0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.155+0000 7f8648e4d640 1 -- 172.21.15.26:0/909494504 shutdown_connections 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.155+0000 7f8648e4d640 1 --2- 172.21.15.26:0/909494504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8644143c40 0x7f86440cf870 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.155+0000 7f8648e4d640 1 -- 172.21.15.26:0/909494504 >> 172.21.15.26:0/909494504 conn(0x7f864406b300 msgr2=0x7f864406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.155+0000 7f8648e4d640 1 -- 172.21.15.26:0/909494504 shutdown_connections 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.155+0000 7f8648e4d640 1 -- 172.21.15.26:0/909494504 wait complete. 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.156+0000 7f8648e4d640 1 Processor -- start 2024-04-24T23:24:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.156+0000 7f8648e4d640 1 -- start start 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8648e4d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86440cc860 0x7f86440b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8648e4d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8644143c40 0x7f86440b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8648e4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864415eb00 0x7f86441f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8648e4d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86440cccd0 con 0x7f86440cc860 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8648e4d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8644144020 con 0x7f864415eb00 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8648e4d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f86440cce50 con 0x7f8644143c40 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f864259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86440cc860 0x7f86440b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f864259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86440cc860 0x7f86440b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33834/0 (socket says 172.21.15.26:33834) 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f864259c640 1 -- 172.21.15.26:0/2043342129 learned_addr learned my addr 172.21.15.26:0/2043342129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8642d9d640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864415eb00 0x7f86441f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f864259c640 1 -- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8644143c40 msgr2=0x7f86440b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8641d9b640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8644143c40 0x7f86440b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f864259c640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8644143c40 0x7f86440b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f864259c640 1 -- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864415eb00 msgr2=0x7f86441f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f864259c640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864415eb00 0x7f86441f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f864259c640 1 -- 172.21.15.26:0/2043342129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864413c270 con 0x7f86440cc860 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8642d9d640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864415eb00 0x7f86441f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.157+0000 7f8641d9b640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8644143c40 0x7f86440b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.158+0000 7f864259c640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86440cc860 0x7f86440b3a20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8634000f80 tx=0x7f8634004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.158+0000 7f863affd640 1 -- 172.21.15.26:0/2043342129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634040070 con 0x7f86440cc860 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.158+0000 7f863affd640 1 -- 172.21.15.26:0/2043342129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8634031d00 con 0x7f86440cc860 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.158+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864413c550 con 0x7f86440cc860 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.158+0000 7f863affd640 1 -- 172.21.15.26:0/2043342129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634003b90 con 0x7f86440cc860 2024-04-24T23:24:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.158+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864413cb10 con 0x7f86440cc860 2024-04-24T23:24:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.159+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86440694c0 con 0x7f86440cc860 2024-04-24T23:24:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.159+0000 7f863affd640 1 -- 172.21.15.26:0/2043342129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8634039040 con 0x7f86440cc860 2024-04-24T23:24:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.160+0000 7f863affd640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f862406be30 0x7f862406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.160+0000 7f863affd640 1 -- 172.21.15.26:0/2043342129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f86340ac140 con 0x7f86440cc860 2024-04-24T23:24:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.160+0000 7f8641d9b640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f862406be30 0x7f862406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.160+0000 7f8641d9b640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f862406be30 0x7f862406e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f864413d9d0 tx=0x7f862c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:26.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.164+0000 7f863affd640 1 -- 172.21.15.26:0/2043342129 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f863407ab80 con 0x7f86440cc860 2024-04-24T23:24:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.384+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86440b4d20 con 0x7f86440cc860 2024-04-24T23:24:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.384+0000 7f863affd640 1 -- 172.21.15.26:0/2043342129 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5219 (secure 0 0 0) 0x7f863407a480 con 0x7f86440cc860 2024-04-24T23:24:26.387 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:24:26.387 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:24:14.713249+0000","modified":"2024-04-24T23:24:25.043181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6546,"mds_1":6534,"mds_2":6531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6531":{"gid":6531,"name":"c","rank":2,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6836/1519104007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1519104007},{"type":"v1","addr":"172.21.15.26:6836","nonce":1519104007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6534":{"gid":6534,"name":"a","rank":1,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6837/2743842996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2743842996},{"type":"v1","addr":"172.21.15.26:6837","nonce":2743842996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6546":{"gid":6546,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2096523178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2096523178},{"type":"v1","addr":"172.21.15.29:6835","nonce":2096523178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6531,6534,6546]},"id":4}]} 2024-04-24T23:24:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 54 2024-04-24T23:24:26.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f862406be30 msgr2=0x7f862406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f862406be30 0x7f862406e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f864413d9d0 tx=0x7f862c009040 comp rx=0 tx=0).stop 2024-04-24T23:24:26.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86440cc860 msgr2=0x7f86440b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:26.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86440cc860 0x7f86440b3a20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8634000f80 tx=0x7f8634004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:26.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 shutdown_connections 2024-04-24T23:24:26.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f862406be30 0x7f862406e2f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864415eb00 0x7f86441f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8644143c40 0x7f86440b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 --2- 172.21.15.26:0/2043342129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86440cc860 0x7f86440b3a20 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 >> 172.21.15.26:0/2043342129 conn(0x7f864406b300 msgr2=0x7f864415e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 shutdown_connections 2024-04-24T23:24:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:26.387+0000 7f8648e4d640 1 -- 172.21.15.26:0/2043342129 wait complete. 2024-04-24T23:24:26.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:26.400 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:24:26.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:26.425 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:24:26.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:26.485 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-04-24T23:24:26.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:26.541 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-04-24T23:24:26.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:26.597 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-04-24T23:24:26.652 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:24:26.775 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:24:26.776 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:24:26.776 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:24:28.777 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:24:28.906 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:24:28.906 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:24:28.906 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:24:29.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:24:29.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:24:30.907 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:24:31.040 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:24:31.041 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:24:31.041 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:24:33.042 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:24:33.171 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:24:33.171 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:24:33.171 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:24:35.172 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:24:35.302 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"exportbound":0,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"15","version":"15","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2024-04-24T23:24:35.303 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "exportbound": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "15", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "15" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:24:35.303 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:24:37.304 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:24:37.434 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"15","version":"15","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2024-04-24T23:24:37.435 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "15", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "15" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:24:37.435 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:24:37.436 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:24:37.566 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"0","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"subtree":1,"importbound":0,"sticky":0},"nref":1}}] 2024-04-24T23:24:37.566 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "importbound": 0, "sticky": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:24:37.567 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:24:37.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:24:37.684 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:24:37.685 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:24:37.685 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:24:37.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-04-24T23:24:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.787+0000 7f8a145ab640 1 Processor -- start 2024-04-24T23:24:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.787+0000 7f8a145ab640 1 -- start start 2024-04-24T23:24:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.788+0000 7f8a145ab640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8a0c0ce640 0x7f8a0c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.788+0000 7f8a145ab640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8a0c0d28e0 0x7f8a0c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.788+0000 7f8a145ab640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0c06d750 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.788+0000 7f8a145ab640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0c0d1110 con 0x7f8a0c0ce640 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.788+0000 7f8a145ab640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0c0cc410 con 0x7f8a0c0d28e0 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.788+0000 7f8a12347640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8a0c0ce640 0x7f8a0c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.788+0000 7f8a12b48640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8a0c06c4d0 0x7f8a0c06c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55138/0 (socket says 172.21.15.26:55138) 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.788+0000 7f8a12b48640 1 -- 172.21.15.26:0/7996025 learned_addr learned my addr 172.21.15.26:0/7996025 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.788+0000 7f8a11b46640 1 --2- 172.21.15.26:0/7996025 >> v2:172.21.15.29:3300/0 conn(0x7f8a0c0d28e0 0x7f8a0c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.789+0000 7f8a12347640 1 -- 172.21.15.26:0/7996025 >> v2:172.21.15.29:3300/0 conn(0x7f8a0c0d28e0 msgr2=0x7f8a0c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.789+0000 7f8a12347640 1 --2- 172.21.15.26:0/7996025 >> v2:172.21.15.29:3300/0 conn(0x7f8a0c0d28e0 0x7f8a0c0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.789+0000 7f8a12347640 1 -- 172.21.15.26:0/7996025 >> v1:172.21.15.26:6789/0 conn(0x7f8a0c06c4d0 legacy=0x7f8a0c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.789+0000 7f8a11345640 1 -- 172.21.15.26:0/7996025 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432511072 0 0) 0x7f8a0c06d750 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.789+0000 7f8a12347640 1 -- 172.21.15.26:0/7996025 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c0b1720 con 0x7f8a0c0ce640 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.789+0000 7f8a12347640 1 --2- 172.21.15.26:0/7996025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c0ce640 0x7f8a0c0d00f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0b7e70 tx=0x7f8a0402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e2501d766d0ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.789+0000 7f8a11345640 1 -- 172.21.15.26:0/7996025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04037070 con 0x7f8a0c0ce640 2024-04-24T23:24:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a11345640 1 -- 172.21.15.26:0/7996025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04032440 con 0x7f8a0c0ce640 2024-04-24T23:24:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a11345640 1 -- 172.21.15.26:0/7996025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04032810 con 0x7f8a0c0ce640 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a145ab640 1 -- 172.21.15.26:0/7996025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c0ce640 msgr2=0x7f8a0c0d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a145ab640 1 --2- 172.21.15.26:0/7996025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c0ce640 0x7f8a0c0d00f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0b7e70 tx=0x7f8a0402a230 comp rx=0 tx=0).stop 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a145ab640 1 -- 172.21.15.26:0/7996025 shutdown_connections 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a145ab640 1 --2- 172.21.15.26:0/7996025 >> v2:172.21.15.29:3300/0 conn(0x7f8a0c0d28e0 0x7f8a0c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a145ab640 1 --2- 172.21.15.26:0/7996025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c0ce640 0x7f8a0c0d00f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a145ab640 1 -- 172.21.15.26:0/7996025 >> 172.21.15.26:0/7996025 conn(0x7f8a0c0d1dc0 msgr2=0x7f8a0c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a145ab640 1 -- 172.21.15.26:0/7996025 shutdown_connections 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.790+0000 7f8a145ab640 1 -- 172.21.15.26:0/7996025 wait complete. 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.791+0000 7f8a145ab640 1 Processor -- start 2024-04-24T23:24:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.791+0000 7f8a145ab640 1 -- start start 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a145ab640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c06c4d0 0x7f8a0c154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a145ab640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a0c0ce640 0x7f8a0c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a145ab640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0c0d28e0 0x7f8a0c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a145ab640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0cc660 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a145ab640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0d1360 con 0x7f8a0c0d28e0 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a145ab640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c0cc940 con 0x7f8a0c0ce640 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a12347640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c06c4d0 0x7f8a0c154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a12347640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c06c4d0 0x7f8a0c154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39368/0 (socket says 172.21.15.26:39368) 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a12347640 1 -- 172.21.15.26:0/2675671513 learned_addr learned my addr 172.21.15.26:0/2675671513 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a12b48640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0c0d28e0 0x7f8a0c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.792+0000 7f8a11b46640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a0c0ce640 0x7f8a0c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a12347640 1 -- 172.21.15.26:0/2675671513 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a0c0ce640 msgr2=0x7f8a0c0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a12347640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a0c0ce640 0x7f8a0c0cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a12347640 1 -- 172.21.15.26:0/2675671513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0c0d28e0 msgr2=0x7f8a0c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a12347640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0c0d28e0 0x7f8a0c0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a12347640 1 -- 172.21.15.26:0/2675671513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c0b6040 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a12b48640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0c0d28e0 0x7f8a0c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a12347640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c06c4d0 0x7f8a0c154bf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009890 tx=0x7f8a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a0affd640 1 -- 172.21.15.26:0/2675671513 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04037040 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a0affd640 1 -- 172.21.15.26:0/2675671513 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04042070 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a0affd640 1 -- 172.21.15.26:0/2675671513 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04030bd0 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a145ab640 1 -- 172.21.15.26:0/2675671513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c0b6320 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.793+0000 7f8a145ab640 1 -- 172.21.15.26:0/2675671513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c0b68e0 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.795+0000 7f8a0affd640 1 -- 172.21.15.26:0/2675671513 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a04047460 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.795+0000 7f8a0affd640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89f806be30 0x7f89f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.795+0000 7f8a0affd640 1 -- 172.21.15.26:0/2675671513 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8a040aba30 con 0x7f8a0c06c4d0 2024-04-24T23:24:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.795+0000 7f8a11b46640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89f806be30 0x7f89f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.795+0000 7f8a11b46640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89f806be30 0x7f89f806e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a00006fd0 tx=0x7f8a00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.796+0000 7f8a1738b740 1 Processor -- start 2024-04-24T23:24:37.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.796+0000 7f8a1738b740 1 -- start start 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 0x5645d8fc3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fe0600 0x5645d8fe0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8f98a40 0x5645d8fdecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a1738b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645d8efc6b0 con 0x5645d8f98a40 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a1738b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5645d8ea4880 con 0x5645d8fc2d60 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a1738b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5645d8ef8670 con 0x5645d8fe0600 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12b48640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8f98a40 0x5645d8fdecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12b48640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8f98a40 0x5645d8fdecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39376/0 (socket says 172.21.15.26:39376) 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12b48640 1 -- 172.21.15.26:0/832502106 learned_addr learned my addr 172.21.15.26:0/832502106 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12347640 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 0x5645d8fc3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a11b46640 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fe0600 0x5645d8fe0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12b48640 1 -- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fe0600 msgr2=0x5645d8fe0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12b48640 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fe0600 0x5645d8fe0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12b48640 1 -- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 msgr2=0x5645d8fc3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12b48640 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 0x5645d8fc3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12b48640 1 -- 172.21.15.26:0/832502106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d8cda7d0 con 0x5645d8f98a40 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a11b46640 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fe0600 0x5645d8fe0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.797+0000 7f8a12347640 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 0x5645d8fc3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.798+0000 7f8a12b48640 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8f98a40 0x5645d8fdecc0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0073d0 tx=0x7f89fc007890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48690eeab134a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.798+0000 7f89f3fff640 1 -- 172.21.15.26:0/832502106 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc019070 con 0x5645d8f98a40 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.798+0000 7f89f3fff640 1 -- 172.21.15.26:0/832502106 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc001e20 con 0x5645d8f98a40 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.798+0000 7f89f3fff640 1 -- 172.21.15.26:0/832502106 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc014480 con 0x5645d8f98a40 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.798+0000 7f8a1738b740 1 -- 172.21.15.26:0/832502106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8f98a40 msgr2=0x5645d8fdecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.798+0000 7f8a1738b740 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8f98a40 0x5645d8fdecc0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0073d0 tx=0x7f89fc007890 comp rx=0 tx=0).stop 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.799+0000 7f8a1738b740 1 -- 172.21.15.26:0/832502106 shutdown_connections 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.799+0000 7f8a1738b740 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8f98a40 0x5645d8fdecc0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.799+0000 7f8a1738b740 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fe0600 0x5645d8fe0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.799+0000 7f8a1738b740 1 --2- 172.21.15.26:0/832502106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 0x5645d8fc3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.799+0000 7f8a1738b740 1 -- 172.21.15.26:0/832502106 >> 172.21.15.26:0/832502106 conn(0x5645d8f95600 msgr2=0x5645d8fddb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.799+0000 7f8a1738b740 1 -- 172.21.15.26:0/832502106 shutdown_connections 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.799+0000 7f8a1738b740 1 -- 172.21.15.26:0/832502106 wait complete. 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.799+0000 7f8a1738b740 1 -- start start 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.799+0000 7f8a1738b740 1 Processor -- start 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 0x5645d8fd6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8ff1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 0x5645d8fee940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a1738b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645d8f997e0 con 0x5645d8fe0600 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a1738b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5645d8eb7f50 con 0x5645d8ff16b0 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a1738b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5645d8f93530 con 0x5645d8fc2d60 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a1738b740 10 client.0 _ll_register_callbacks cb 0x5645d8ef3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a11b46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8ff1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a11b46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8ff1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39378/0 (socket says 172.21.15.26:39378) 2024-04-24T23:24:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a11b46640 1 -- 172.21.15.26:0/2596961686 learned_addr learned my addr 172.21.15.26:0/2596961686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a12347640 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 0x5645d8fd6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.800+0000 7f8a12b48640 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 0x5645d8fee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a11b46640 1 -- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 msgr2=0x5645d8fd6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a11b46640 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 0x5645d8fd6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a11b46640 1 -- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 msgr2=0x5645d8fee940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a11b46640 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 0x5645d8fee940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a11b46640 1 -- 172.21.15.26:0/2596961686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d8c103f0 con 0x5645d8fe0600 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a11b46640 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8ff1170 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8a00010010 tx=0x7f8a00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31421b11112150b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f89f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f89f17fa640 1 -- 172.21.15.26:0/2596961686 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00025020 con 0x5645d8fe0600 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f89f17fa640 1 -- 172.21.15.26:0/2596961686 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8a00021830 con 0x5645d8fe0600 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a1738b740 1 -- 172.21.15.26:0/2596961686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89fc001e20 con 0x5645d8fe0600 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f89f17fa640 1 -- 172.21.15.26:0/2596961686 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8a0002f050 con 0x5645d8fe0600 2024-04-24T23:24:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a1738b740 10 client.6801 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a1738b740 1 -- 172.21.15.26:0/2596961686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x5645d8bd00f0 con 0x5645d8fe0600 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f89f17fa640 1 -- 172.21.15.26:0/2596961686 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc001e20 con 0x5645d8fe0600 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f89f17fa640 1 -- 172.21.15.26:0/2596961686 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7f8a00032020 con 0x5645d8fe0600 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a1738b740 10 client.6801 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a1738b740 10 client.6801 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a1738b740 20 client.6801 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a1738b740 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x5645d8ff9f00 0x5645d8ffc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a1738b740 4 client.6801 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.801+0000 7f8a1738b740 1 -- 172.21.15.26:0/2596961686 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645d8f09bf0 con 0x5645d8ff9f00 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.802+0000 7f8a12347640 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x5645d8ff9f00 0x5645d8ffc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.802+0000 7f8a12347640 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x5645d8ff9f00 0x5645d8ffc2f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa708a1edb52ae2 server_cookie=400fd5671ec8ad15 in_seq=0 out_seq=0 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.802+0000 7f89f17fa640 10 client.6801 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f89f17fa640 1 -- 172.21.15.26:0/2596961686 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645d8f09bf0 con 0x5645d8ff9f00 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f89f17fa640 10 client.6801 handle_command_reply: tid=0 2024-04-24T23:24:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 client.6801 shutdown 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 -- 172.21.15.26:0/2596961686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 msgr2=0x5645d8ff1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8ff1170 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8a00010010 tx=0x7f8a00004730 comp rx=0 tx=0).stop 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 -- 172.21.15.26:0/2596961686 shutdown_connections 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x5645d8ff9f00 0x5645d8ffc2f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 0x5645d8fee940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8ff1170 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f89f17fa640 0 client.6801 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 --2- 172.21.15.26:0/2596961686 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 0x5645d8fd6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 -- 172.21.15.26:0/2596961686 >> 172.21.15.26:0/2596961686 conn(0x5645d8f98a40 msgr2=0x5645d8fc1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 -- 172.21.15.26:0/2596961686 shutdown_connections 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 1 -- 172.21.15.26:0/2596961686 wait complete. 2024-04-24T23:24:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.803+0000 7f8a1738b740 20 client.6801 trim_cache size 0 max 16384 2024-04-24T23:24:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.818+0000 7f8a1738b740 1 Processor -- start 2024-04-24T23:24:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.818+0000 7f8a1738b740 1 -- start start 2024-04-24T23:24:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 0x5645d8fc5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fdc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8ff16b0 0x5645d8fdc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a1738b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c200550 con 0x5645d8fe0600 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a1738b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1ff770 con 0x5645d8fc2d60 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a1738b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c1ffe20 con 0x5645d8ff16b0 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a11b46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fdc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a11b46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fdc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39392/0 (socket says 172.21.15.26:39392) 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a11b46640 1 -- 172.21.15.26:0/4079600891 learned_addr learned my addr 172.21.15.26:0/4079600891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a12347640 1 --2- 172.21.15.26:0/4079600891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 0x5645d8fc5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a12b48640 1 --2- 172.21.15.26:0/4079600891 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8ff16b0 0x5645d8fdc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a11b46640 1 -- 172.21.15.26:0/4079600891 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8ff16b0 msgr2=0x5645d8fdc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a11b46640 1 --2- 172.21.15.26:0/4079600891 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8ff16b0 0x5645d8fdc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a11b46640 1 -- 172.21.15.26:0/4079600891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 msgr2=0x5645d8fc5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a11b46640 1 --2- 172.21.15.26:0/4079600891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 0x5645d8fc5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a11b46640 1 -- 172.21.15.26:0/4079600891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d8fc2b70 con 0x5645d8fe0600 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f8a11b46640 1 --2- 172.21.15.26:0/4079600891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fdc0b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8a00021bc0 tx=0x7f8a00020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4feb26956eb9004a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f89f37fe640 1 -- 172.21.15.26:0/4079600891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00025050 con 0x5645d8fe0600 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f89f37fe640 1 -- 172.21.15.26:0/4079600891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0001c040 con 0x5645d8fe0600 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.819+0000 7f89f37fe640 1 -- 172.21.15.26:0/4079600891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0001b070 con 0x5645d8fe0600 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 -- 172.21.15.26:0/4079600891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 msgr2=0x5645d8fdc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 --2- 172.21.15.26:0/4079600891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fdc0b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8a00021bc0 tx=0x7f8a00020000 comp rx=0 tx=0).stop 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 -- 172.21.15.26:0/4079600891 shutdown_connections 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 --2- 172.21.15.26:0/4079600891 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8ff16b0 0x5645d8fdc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 --2- 172.21.15.26:0/4079600891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fdc0b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 --2- 172.21.15.26:0/4079600891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8fc2d60 0x5645d8fc5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 -- 172.21.15.26:0/4079600891 >> 172.21.15.26:0/4079600891 conn(0x5645d8f98a40 msgr2=0x5645d8fd7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 -- 172.21.15.26:0/4079600891 shutdown_connections 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 -- 172.21.15.26:0/4079600891 wait complete. 2024-04-24T23:24:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.820+0000 7f8a1738b740 1 -- start start 2024-04-24T23:24:37.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.821+0000 7f8a1738b740 1 Processor -- start 2024-04-24T23:24:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.821+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 0x5645d9051ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.821+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fd3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.821+0000 7f8a1738b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 0x5645d9057eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.821+0000 7f8a1738b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645d8fc46a0 con 0x5645d8fe0600 2024-04-24T23:24:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.821+0000 7f8a1738b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0c92a0 con 0x5645d8ff16b0 2024-04-24T23:24:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.821+0000 7f8a1738b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5645d8fc4d50 con 0x5645d8fc2d60 2024-04-24T23:24:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a1738b740 10 client.0 _ll_register_callbacks cb 0x7f8a0000fcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a11b46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fd3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a11b46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fd3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39406/0 (socket says 172.21.15.26:39406) 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a11b46640 1 -- 172.21.15.26:0/3492575188 learned_addr learned my addr 172.21.15.26:0/3492575188 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a12b48640 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 0x5645d9057eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a12347640 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 0x5645d9051ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a11b46640 1 -- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 msgr2=0x5645d9051ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a11b46640 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 0x5645d9051ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a11b46640 1 -- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 msgr2=0x5645d9057eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a11b46640 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 0x5645d9057eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a11b46640 1 -- 172.21.15.26:0/3492575188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d8fbf000 con 0x5645d8fe0600 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a11b46640 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fd3500 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8a00010010 tx=0x7f8a00014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0721ca7e5aa0956 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f89f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f89f17fa640 1 -- 172.21.15.26:0/3492575188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00025070 con 0x5645d8fe0600 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f89f17fa640 1 -- 172.21.15.26:0/3492575188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8a00005da0 con 0x5645d8fe0600 2024-04-24T23:24:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a1738b740 1 -- 172.21.15.26:0/3492575188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a0001c040 con 0x5645d8fe0600 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f89f17fa640 1 -- 172.21.15.26:0/3492575188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8a0004a050 con 0x5645d8fe0600 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a1738b740 10 client.6807 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.822+0000 7f8a1738b740 1 -- 172.21.15.26:0/3492575188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x5645d8fbea40 con 0x5645d8fe0600 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f89f17fa640 1 -- 172.21.15.26:0/3492575188 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0001c040 con 0x5645d8fe0600 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f89f17fa640 1 -- 172.21.15.26:0/3492575188 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7f8a0004d020 con 0x5645d8fe0600 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f8a1738b740 10 client.6807 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f8a1738b740 10 client.6807 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f8a1738b740 20 client.6807 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f8a1738b740 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x5645d90584c0 0x5645d9000310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f8a1738b740 4 client.6807 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f8a1738b740 1 -- 172.21.15.26:0/3492575188 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645d8f03f40 con 0x5645d90584c0 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f8a12347640 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x5645d90584c0 0x5645d9000310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f89f17fa640 10 client.6807 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.823+0000 7f8a12347640 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x5645d90584c0 0x5645d9000310 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c90f0dfed289fb4 server_cookie=8f453f30595c7406 in_seq=0 out_seq=0 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f89f17fa640 1 -- 172.21.15.26:0/3492575188 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645d8f03f40 con 0x5645d90584c0 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f89f17fa640 10 client.6807 handle_command_reply: tid=0 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 client.6807 shutdown 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 -- 172.21.15.26:0/3492575188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 msgr2=0x5645d8fd3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fd3500 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8a00010010 tx=0x7f8a00014000 comp rx=0 tx=0).stop 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 -- 172.21.15.26:0/3492575188 shutdown_connections 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x5645d90584c0 0x5645d9000310 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645d8ff16b0 0x5645d9057eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645d8fe0600 0x5645d8fd3500 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f89f17fa640 0 client.6807 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 --2- 172.21.15.26:0/3492575188 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645d8fc2d60 0x5645d9051ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 -- 172.21.15.26:0/3492575188 >> 172.21.15.26:0/3492575188 conn(0x5645d8f98a40 msgr2=0x5645d8f98e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 -- 172.21.15.26:0/3492575188 shutdown_connections 2024-04-24T23:24:37.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 1 -- 172.21.15.26:0/3492575188 wait complete. 2024-04-24T23:24:37.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.835+0000 7f8a1738b740 20 client.6807 trim_cache size 0 max 16384 2024-04-24T23:24:37.838 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:24:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:24:37.838 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 -- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89f806be30 msgr2=0x7f89f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89f806be30 0x7f89f806e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a00006fd0 tx=0x7f8a00008040 comp rx=0 tx=0).stop 2024-04-24T23:24:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 -- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c06c4d0 msgr2=0x7f8a0c154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c06c4d0 0x7f8a0c154bf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009890 tx=0x7f8a040046c0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 -- 172.21.15.26:0/2675671513 shutdown_connections 2024-04-24T23:24:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f89f806be30 0x7f89f806e2f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a0c0d28e0 0x7f8a0c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a0c0ce640 0x7f8a0c0cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 --2- 172.21.15.26:0/2675671513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a0c06c4d0 0x7f8a0c154bf0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 -- 172.21.15.26:0/2675671513 >> 172.21.15.26:0/2675671513 conn(0x7f8a0c0d1dc0 msgr2=0x7f8a0c0cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 -- 172.21.15.26:0/2675671513 shutdown_connections 2024-04-24T23:24:37.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.836+0000 7f8a145ab640 1 -- 172.21.15.26:0/2675671513 wait complete. 2024-04-24T23:24:37.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 flush journal 2024-04-24T23:24:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.949+0000 7f81f78f5640 1 Processor -- start 2024-04-24T23:24:37.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.949+0000 7f81f78f5640 1 -- start start 2024-04-24T23:24:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.950+0000 7f81f78f5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f81f015e370 0x7f81f00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.950+0000 7f81f78f5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81f00d4c10 0x7f81f00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.950+0000 7f81f78f5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81f00d2120 con 0x7f81f015e750 2024-04-24T23:24:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.950+0000 7f81f78f5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f81f00d1510 con 0x7f81f00d4c10 2024-04-24T23:24:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.950+0000 7f81f78f5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f81f00d1ed0 con 0x7f81f015e370 2024-04-24T23:24:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.950+0000 7f81f5e92640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81f00d4c10 0x7f81f00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.950+0000 7f81f5e92640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81f00d4c10 0x7f81f00cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39416/0 (socket says 172.21.15.26:39416) 2024-04-24T23:24:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.950+0000 7f81f5e92640 1 -- 172.21.15.26:0/3227454132 learned_addr learned my addr 172.21.15.26:0/3227454132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.951+0000 7f81f5691640 1 --2- 172.21.15.26:0/3227454132 >> v2:172.21.15.29:3300/0 conn(0x7f81f015e370 0x7f81f00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.951+0000 7f81f5e92640 1 -- 172.21.15.26:0/3227454132 >> v2:172.21.15.29:3300/0 conn(0x7f81f015e370 msgr2=0x7f81f00d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.951+0000 7f81f5e92640 1 --2- 172.21.15.26:0/3227454132 >> v2:172.21.15.29:3300/0 conn(0x7f81f015e370 0x7f81f00d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.951+0000 7f81f5e92640 1 -- 172.21.15.26:0/3227454132 >> v1:172.21.15.29:6789/0 conn(0x7f81f015e750 legacy=0x7f81f00b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.951+0000 7f81f5e92640 1 -- 172.21.15.26:0/3227454132 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f00b3740 con 0x7f81f00d4c10 2024-04-24T23:24:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.951+0000 7f81f5e92640 1 --2- 172.21.15.26:0/3227454132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 0x7f81f00cfca0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f81f008c7e0 tx=0x7f81e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8f33f73cfd1d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81effff640 1 -- 172.21.15.26:0/3227454132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0037070 con 0x7f81f00d4c10 2024-04-24T23:24:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81effff640 1 -- 172.21.15.26:0/3227454132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e0032440 con 0x7f81f00d4c10 2024-04-24T23:24:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81effff640 1 -- 172.21.15.26:0/3227454132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0032810 con 0x7f81f00d4c10 2024-04-24T23:24:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81f78f5640 1 -- 172.21.15.26:0/3227454132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 msgr2=0x7f81f00cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81f78f5640 1 --2- 172.21.15.26:0/3227454132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 0x7f81f00cfca0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f81f008c7e0 tx=0x7f81e002a230 comp rx=0 tx=0).stop 2024-04-24T23:24:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81f78f5640 1 -- 172.21.15.26:0/3227454132 shutdown_connections 2024-04-24T23:24:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81f78f5640 1 --2- 172.21.15.26:0/3227454132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 0x7f81f00cfca0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81f78f5640 1 --2- 172.21.15.26:0/3227454132 >> v2:172.21.15.29:3300/0 conn(0x7f81f015e370 0x7f81f00d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81f78f5640 1 -- 172.21.15.26:0/3227454132 >> 172.21.15.26:0/3227454132 conn(0x7f81f00d40f0 msgr2=0x7f81f00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81f78f5640 1 -- 172.21.15.26:0/3227454132 shutdown_connections 2024-04-24T23:24:37.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.952+0000 7f81f78f5640 1 -- 172.21.15.26:0/3227454132 wait complete. 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.953+0000 7f81f78f5640 1 Processor -- start 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f78f5640 1 -- start start 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f78f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 0x7f81f00cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f78f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f81f015e370 0x7f81f00ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f78f5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f00cc120 0x7f81f00c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f78f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81f00d2a60 con 0x7f81f00d4c10 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f78f5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81f00b3d00 con 0x7f81f00cc120 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f78f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f81f00d2be0 con 0x7f81f015e370 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f5691640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 0x7f81f00cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f5691640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 0x7f81f00cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39428/0 (socket says 172.21.15.26:39428) 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.954+0000 7f81f5691640 1 -- 172.21.15.26:0/1505550944 learned_addr learned my addr 172.21.15.26:0/1505550944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f5e92640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f00cc120 0x7f81f00c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f81f015e370 0x7f81f00ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f5691640 1 -- 172.21.15.26:0/1505550944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f81f015e370 msgr2=0x7f81f00ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f5691640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f81f015e370 0x7f81f00ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f5691640 1 -- 172.21.15.26:0/1505550944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f00cc120 msgr2=0x7f81f00c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f5691640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f00cc120 0x7f81f00c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f5691640 1 -- 172.21.15.26:0/1505550944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f00cd130 con 0x7f81f00d4c10 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f81f015e370 0x7f81f00ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f5691640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 0x7f81f00cc5c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f81e800c7f0 tx=0x7f81e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81edffb640 1 -- 172.21.15.26:0/1505550944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8015070 con 0x7f81f00d4c10 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81edffb640 1 -- 172.21.15.26:0/1505550944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e8004240 con 0x7f81f00d4c10 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f78f5640 1 -- 172.21.15.26:0/1505550944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f00cb360 con 0x7f81f00d4c10 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81edffb640 1 -- 172.21.15.26:0/1505550944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8007620 con 0x7f81f00d4c10 2024-04-24T23:24:37.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.955+0000 7f81f78f5640 1 -- 172.21.15.26:0/1505550944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f00cb920 con 0x7f81f00d4c10 2024-04-24T23:24:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.957+0000 7f81edffb640 1 -- 172.21.15.26:0/1505550944 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81e80079d0 con 0x7f81f00d4c10 2024-04-24T23:24:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.957+0000 7f81edffb640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f81dc06be30 0x7f81dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.957+0000 7f81edffb640 1 -- 172.21.15.26:0/1505550944 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f81e808b2f0 con 0x7f81f00d4c10 2024-04-24T23:24:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.957+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f81dc06be30 0x7f81dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.957+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f81dc06be30 0x7f81dc06e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f81f00b9ab0 tx=0x7f81e40073c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.958+0000 7f81fa601740 1 Processor -- start 2024-04-24T23:24:37.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.958+0000 7f81fa601740 1 -- start start 2024-04-24T23:24:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81fa601740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad918bd60 0x556ad918c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81fa601740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 0x556ad91a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81fa601740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad9161a40 0x556ad91a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81fa601740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556ad90c56b0 con 0x556ad9161a40 2024-04-24T23:24:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81fa601740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ad906d880 con 0x556ad91a9600 2024-04-24T23:24:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81fa601740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556ad90c1670 con 0x556ad918bd60 2024-04-24T23:24:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f5e92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad9161a40 0x556ad91a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f5e92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad9161a40 0x556ad91a7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39444/0 (socket says 172.21.15.26:39444) 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f5e92640 1 -- 172.21.15.26:0/2405353700 learned_addr learned my addr 172.21.15.26:0/2405353700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f5e92640 1 -- 172.21.15.26:0/2405353700 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad918bd60 msgr2=0x556ad918c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f4e90640 1 --2- 172.21.15.26:0/2405353700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 0x556ad91a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f5691640 1 --2- 172.21.15.26:0/2405353700 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad918bd60 0x556ad918c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f5e92640 1 --2- 172.21.15.26:0/2405353700 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad918bd60 0x556ad918c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f5e92640 1 -- 172.21.15.26:0/2405353700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 msgr2=0x556ad91a9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f5e92640 1 --2- 172.21.15.26:0/2405353700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 0x556ad91a9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.959+0000 7f81f5e92640 1 -- 172.21.15.26:0/2405353700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad8ea37d0 con 0x556ad9161a40 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81f5e92640 1 --2- 172.21.15.26:0/2405353700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad9161a40 0x556ad91a7cc0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f81e00090b0 tx=0x7f81e002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675ccecd5ea86086 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81d6ffd640 1 -- 172.21.15.26:0/2405353700 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0042070 con 0x556ad9161a40 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81d6ffd640 1 -- 172.21.15.26:0/2405353700 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e00326c0 con 0x556ad9161a40 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81d6ffd640 1 -- 172.21.15.26:0/2405353700 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0037040 con 0x556ad9161a40 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81fa601740 1 -- 172.21.15.26:0/2405353700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad9161a40 msgr2=0x556ad91a7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81fa601740 1 --2- 172.21.15.26:0/2405353700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad9161a40 0x556ad91a7cc0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f81e00090b0 tx=0x7f81e002c000 comp rx=0 tx=0).stop 2024-04-24T23:24:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81fa601740 1 -- 172.21.15.26:0/2405353700 shutdown_connections 2024-04-24T23:24:37.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81fa601740 1 --2- 172.21.15.26:0/2405353700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad9161a40 0x556ad91a7cc0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81fa601740 1 --2- 172.21.15.26:0/2405353700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 0x556ad91a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81fa601740 1 --2- 172.21.15.26:0/2405353700 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad918bd60 0x556ad918c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81fa601740 1 -- 172.21.15.26:0/2405353700 >> 172.21.15.26:0/2405353700 conn(0x556ad915e600 msgr2=0x556ad91a6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81fa601740 1 -- 172.21.15.26:0/2405353700 shutdown_connections 2024-04-24T23:24:37.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.960+0000 7f81fa601740 1 -- 172.21.15.26:0/2405353700 wait complete. 2024-04-24T23:24:37.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.961+0000 7f81fa601740 1 -- start start 2024-04-24T23:24:37.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.961+0000 7f81fa601740 1 Processor -- start 2024-04-24T23:24:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81fa601740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad91886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81fa601740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 0x556ad919f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81fa601740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad919ff30 0x556ad91a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81fa601740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556ad91627e0 con 0x556ad918bd60 2024-04-24T23:24:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81fa601740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ad9080f50 con 0x556ad919ff30 2024-04-24T23:24:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81fa601740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556ad915c530 con 0x556ad91a9600 2024-04-24T23:24:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5691640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad91886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81fa601740 10 client.0 _ll_register_callbacks cb 0x556ad90bc000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5691640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad91886e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39446/0 (socket says 172.21.15.26:39446) 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5691640 1 -- 172.21.15.26:0/364764595 learned_addr learned my addr 172.21.15.26:0/364764595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5e92640 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad919ff30 0x556ad91a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f4e90640 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 0x556ad919f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5691640 1 -- 172.21.15.26:0/364764595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 msgr2=0x556ad919f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5691640 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 0x556ad919f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5691640 1 -- 172.21.15.26:0/364764595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad919ff30 msgr2=0x556ad91a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5691640 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad919ff30 0x556ad91a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5691640 1 -- 172.21.15.26:0/364764595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad8dd93f0 con 0x556ad918bd60 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81f5691640 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad91886e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f81e8089840 tx=0x7f81e80595d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733b37441c678419 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.962+0000 7f81cffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81cffff640 1 -- 172.21.15.26:0/364764595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e801e070 con 0x556ad918bd60 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81cffff640 1 -- 172.21.15.26:0/364764595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f81e8007cc0 con 0x556ad918bd60 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81fa601740 1 -- 172.21.15.26:0/364764595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e00326c0 con 0x556ad918bd60 2024-04-24T23:24:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81cffff640 1 -- 172.21.15.26:0/364764595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f81e8062180 con 0x556ad918bd60 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81fa601740 10 client.6819 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81fa601740 1 -- 172.21.15.26:0/364764595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x556ad8d990f0 con 0x556ad918bd60 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81cffff640 1 -- 172.21.15.26:0/364764595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e00326c0 con 0x556ad918bd60 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81cffff640 1 -- 172.21.15.26:0/364764595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7f81e8061cf0 con 0x556ad918bd60 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81fa601740 10 client.6819 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81fa601740 10 client.6819 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81fa601740 20 client.6819 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81fa601740 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x556ad918e1c0 0x556ad918e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81fa601740 4 client.6819 mds_command: new command op to 6534 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81fa601740 1 -- 172.21.15.26:0/364764595 --> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad90d2bf0 con 0x556ad918e1c0 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.963+0000 7f81f4e90640 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x556ad918e1c0 0x556ad918e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81cffff640 10 client.6819 ms_handle_connect on v2:172.21.15.26:6835/2743842996 2024-04-24T23:24:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81f4e90640 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x556ad918e1c0 0x556ad918e5a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2d82bd9b0efad99 server_cookie=cdf6e431e026f036 in_seq=0 out_seq=0 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81cffff640 1 -- 172.21.15.26:0/364764595 <== mds.1 v2:172.21.15.26:6835/2743842996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ad90d2bf0 con 0x556ad918e1c0 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81cffff640 10 client.6819 handle_command_reply: tid=0 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 client.6819 shutdown 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 -- 172.21.15.26:0/364764595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 msgr2=0x556ad91886e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad91886e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f81e8089840 tx=0x7f81e80595d0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 -- 172.21.15.26:0/364764595 shutdown_connections 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x556ad918e1c0 0x556ad918e5a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad919ff30 0x556ad91a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 0x556ad919f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81cffff640 0 client.6819 ms_handle_reset on v2:172.21.15.26:6835/2743842996 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 --2- 172.21.15.26:0/364764595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad91886e0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 -- 172.21.15.26:0/364764595 >> 172.21.15.26:0/364764595 conn(0x556ad9161a40 msgr2=0x556ad91aa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.964+0000 7f81fa601740 1 -- 172.21.15.26:0/364764595 shutdown_connections 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.965+0000 7f81fa601740 1 -- 172.21.15.26:0/364764595 wait complete. 2024-04-24T23:24:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.965+0000 7f81fa601740 20 client.6819 trim_cache size 0 max 16384 2024-04-24T23:24:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.979+0000 7f81fa601740 1 Processor -- start 2024-04-24T23:24:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81fa601740 1 -- start start 2024-04-24T23:24:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81fa601740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad918bd60 0x556ad91a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81fa601740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad919ff30 0x556ad918ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81fa601740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 0x556ad918cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81fa601740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81f0200780 con 0x556ad919ff30 2024-04-24T23:24:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81fa601740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01ff980 con 0x556ad918bd60 2024-04-24T23:24:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81fa601740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f81f0200050 con 0x556ad91a9600 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81f4e90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad919ff30 0x556ad918ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81f4e90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad919ff30 0x556ad918ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39458/0 (socket says 172.21.15.26:39458) 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81f4e90640 1 -- 172.21.15.26:0/1234181372 learned_addr learned my addr 172.21.15.26:0/1234181372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.980+0000 7f81f5e92640 1 --2- 172.21.15.26:0/1234181372 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 0x556ad918cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81f5691640 1 --2- 172.21.15.26:0/1234181372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad918bd60 0x556ad91a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81f4e90640 1 -- 172.21.15.26:0/1234181372 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 msgr2=0x556ad918cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1234181372 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 0x556ad918cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81f4e90640 1 -- 172.21.15.26:0/1234181372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad918bd60 msgr2=0x556ad91a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1234181372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad918bd60 0x556ad91a7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81f4e90640 1 -- 172.21.15.26:0/1234181372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad91a67c0 con 0x556ad919ff30 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1234181372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad919ff30 0x556ad918ca20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x556ad91ba610 tx=0x7f81e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3315e61bf9158fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81d67fc640 1 -- 172.21.15.26:0/1234181372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e400fad0 con 0x556ad919ff30 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81d67fc640 1 -- 172.21.15.26:0/1234181372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e402a070 con 0x556ad919ff30 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81d67fc640 1 -- 172.21.15.26:0/1234181372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4024310 con 0x556ad919ff30 2024-04-24T23:24:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.981+0000 7f81fa601740 1 -- 172.21.15.26:0/1234181372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad919ff30 msgr2=0x556ad918ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 --2- 172.21.15.26:0/1234181372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad919ff30 0x556ad918ca20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x556ad91ba610 tx=0x7f81e4012000 comp rx=0 tx=0).stop 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 -- 172.21.15.26:0/1234181372 shutdown_connections 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 --2- 172.21.15.26:0/1234181372 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad91a9600 0x556ad918cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 --2- 172.21.15.26:0/1234181372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad919ff30 0x556ad918ca20 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 --2- 172.21.15.26:0/1234181372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad918bd60 0x556ad91a7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 -- 172.21.15.26:0/1234181372 >> 172.21.15.26:0/1234181372 conn(0x556ad9161a40 msgr2=0x556ad91b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 -- 172.21.15.26:0/1234181372 shutdown_connections 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 -- 172.21.15.26:0/1234181372 wait complete. 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 -- start start 2024-04-24T23:24:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.982+0000 7f81fa601740 1 Processor -- start 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81fa601740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad9220790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81fa601740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad919ff30 0x556ad915b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81fa601740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 0x556ad91c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81fa601740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556ad91a78d0 con 0x556ad918bd60 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81fa601740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ad9163350 con 0x556ad91a9600 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81fa601740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556ad915ee50 con 0x556ad919ff30 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81fa601740 10 client.0 _ll_register_callbacks cb 0x7f81e80557b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81f5691640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad9220790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81f5691640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad9220790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39470/0 (socket says 172.21.15.26:39470) 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81f5691640 1 -- 172.21.15.26:0/1510832556 learned_addr learned my addr 172.21.15.26:0/1510832556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad919ff30 0x556ad915b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.983+0000 7f81f5e92640 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 0x556ad91c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81f5691640 1 -- 172.21.15.26:0/1510832556 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad919ff30 msgr2=0x556ad915b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81f5691640 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad919ff30 0x556ad915b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81f5691640 1 -- 172.21.15.26:0/1510832556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 msgr2=0x556ad91c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81f5691640 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 0x556ad91c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81f5691640 1 -- 172.21.15.26:0/1510832556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad9223e20 con 0x556ad918bd60 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81f5e92640 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 0x556ad91c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81f5691640 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad9220790 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f81e8061c50 tx=0x7f81e801b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3757e26dfaac65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81cffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81cffff640 1 -- 172.21.15.26:0/1510832556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8007cc0 con 0x556ad918bd60 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81cffff640 1 -- 172.21.15.26:0/1510832556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f81e8058a00 con 0x556ad918bd60 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81fa601740 1 -- 172.21.15.26:0/1510832556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e402a070 con 0x556ad918bd60 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81cffff640 1 -- 172.21.15.26:0/1510832556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f81e8007e60 con 0x556ad918bd60 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81fa601740 10 client.6825 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81fa601740 1 -- 172.21.15.26:0/1510832556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x556ad92215f0 con 0x556ad918bd60 2024-04-24T23:24:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81cffff640 1 -- 172.21.15.26:0/1510832556 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e402a070 con 0x556ad918bd60 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81cffff640 1 -- 172.21.15.26:0/1510832556 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7f81e8078350 con 0x556ad918bd60 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81fa601740 10 client.6825 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81fa601740 10 client.6825 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81fa601740 20 client.6825 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81fa601740 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x556ad9224590 0x556ad91ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81fa601740 4 client.6825 mds_command: new command op to 6534 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81fa601740 1 -- 172.21.15.26:0/1510832556 --> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ad90ccf40 con 0x556ad9224590 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.984+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x556ad9224590 0x556ad91ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.985+0000 7f81cffff640 10 client.6825 ms_handle_connect on v2:172.21.15.26:6835/2743842996 2024-04-24T23:24:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.985+0000 7f81f4e90640 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x556ad9224590 0x556ad91ca4d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed7075852ea8cd22 server_cookie=968e1edfbee3f2ba in_seq=0 out_seq=0 2024-04-24T23:24:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.991+0000 7f81cffff640 1 -- 172.21.15.26:0/1510832556 <== mds.1 v2:172.21.15.26:6835/2743842996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ad90ccf40 con 0x556ad9224590 2024-04-24T23:24:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81cffff640 10 client.6825 handle_command_reply: tid=0 2024-04-24T23:24:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 client.6825 shutdown 2024-04-24T23:24:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 -- 172.21.15.26:0/1510832556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 msgr2=0x556ad9220790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad9220790 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f81e8061c50 tx=0x7f81e801b040 comp rx=0 tx=0).stop 2024-04-24T23:24:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 -- 172.21.15.26:0/1510832556 shutdown_connections 2024-04-24T23:24:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x556ad9224590 0x556ad91ca4d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ad91a9600 0x556ad91c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ad919ff30 0x556ad915b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81cffff640 0 client.6825 ms_handle_reset on v2:172.21.15.26:6835/2743842996 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 --2- 172.21.15.26:0/1510832556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ad918bd60 0x556ad9220790 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 -- 172.21.15.26:0/1510832556 >> 172.21.15.26:0/1510832556 conn(0x556ad9161a40 msgr2=0x556ad9187a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 -- 172.21.15.26:0/1510832556 shutdown_connections 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 1 -- 172.21.15.26:0/1510832556 wait complete. 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.992+0000 7f81fa601740 20 client.6825 trim_cache size 0 max 16384 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:24:37.994 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:24:37.995 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:37.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 -- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f81dc06be30 msgr2=0x7f81dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f81dc06be30 0x7f81dc06e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f81f00b9ab0 tx=0x7f81e40073c0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 -- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 msgr2=0x7f81f00cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 0x7f81f00cc5c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f81e800c7f0 tx=0x7f81e800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 -- 172.21.15.26:0/1505550944 shutdown_connections 2024-04-24T23:24:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f81dc06be30 0x7f81dc06e2f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f00cc120 0x7f81f00c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f81f015e370 0x7f81f00ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 --2- 172.21.15.26:0/1505550944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f00d4c10 0x7f81f00cc5c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 -- 172.21.15.26:0/1505550944 >> 172.21.15.26:0/1505550944 conn(0x7f81f00d40f0 msgr2=0x7f81f00d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 -- 172.21.15.26:0/1505550944 shutdown_connections 2024-04-24T23:24:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:37.993+0000 7f81f78f5640 1 -- 172.21.15.26:0/1505550944 wait complete. 2024-04-24T23:24:38.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 flush journal 2024-04-24T23:24:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.107+0000 7f1bc5790640 1 Processor -- start 2024-04-24T23:24:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.107+0000 7f1bc5790640 1 -- start start 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.108+0000 7f1bc5790640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1bc015e370 0x7f1bc00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.108+0000 7f1bc5790640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1bc00d4c10 0x7f1bc00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.108+0000 7f1bc5790640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc00d2120 con 0x7f1bc015e750 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.108+0000 7f1bc5790640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc00d1510 con 0x7f1bc015e370 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.108+0000 7f1bc5790640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc00d1ed0 con 0x7f1bc00d4c10 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.108+0000 7f1bbeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1bc015e370 0x7f1bc00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.109+0000 7f1bbeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1bc015e370 0x7f1bc00d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39472/0 (socket says 172.21.15.26:39472) 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.109+0000 7f1bbeffd640 1 -- 172.21.15.26:0/2551258246 learned_addr learned my addr 172.21.15.26:0/2551258246 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.109+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/2551258246 >> v2:172.21.15.29:3300/0 conn(0x7f1bc00d4c10 0x7f1bc00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.109+0000 7f1bbeffd640 1 -- 172.21.15.26:0/2551258246 >> v2:172.21.15.29:3300/0 conn(0x7f1bc00d4c10 msgr2=0x7f1bc00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.109+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/2551258246 >> v2:172.21.15.29:3300/0 conn(0x7f1bc00d4c10 0x7f1bc00cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.109+0000 7f1bbeffd640 1 -- 172.21.15.26:0/2551258246 >> v1:172.21.15.29:6789/0 conn(0x7f1bc015e750 legacy=0x7f1bc00b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.109+0000 7f1bbeffd640 1 -- 172.21.15.26:0/2551258246 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc00b3740 con 0x7f1bc015e370 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/2551258246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc015e370 0x7f1bc00d2520 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1bc008d9a0 tx=0x7f1bb802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603e9d9ddcc2d2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bbdffb640 1 -- 172.21.15.26:0/2551258246 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8037070 con 0x7f1bc015e370 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bbdffb640 1 -- 172.21.15.26:0/2551258246 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb8032440 con 0x7f1bc015e370 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bbdffb640 1 -- 172.21.15.26:0/2551258246 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8032810 con 0x7f1bc015e370 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bc5790640 1 -- 172.21.15.26:0/2551258246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc015e370 msgr2=0x7f1bc00d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bc5790640 1 --2- 172.21.15.26:0/2551258246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc015e370 0x7f1bc00d2520 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1bc008d9a0 tx=0x7f1bb802a230 comp rx=0 tx=0).stop 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bc5790640 1 -- 172.21.15.26:0/2551258246 shutdown_connections 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bc5790640 1 --2- 172.21.15.26:0/2551258246 >> v2:172.21.15.29:3300/0 conn(0x7f1bc00d4c10 0x7f1bc00cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bc5790640 1 --2- 172.21.15.26:0/2551258246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc015e370 0x7f1bc00d2520 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bc5790640 1 -- 172.21.15.26:0/2551258246 >> 172.21.15.26:0/2551258246 conn(0x7f1bc00d40f0 msgr2=0x7f1bc00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bc5790640 1 -- 172.21.15.26:0/2551258246 shutdown_connections 2024-04-24T23:24:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.110+0000 7f1bc5790640 1 -- 172.21.15.26:0/2551258246 wait complete. 2024-04-24T23:24:38.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.112+0000 7f1bc5790640 1 Processor -- start 2024-04-24T23:24:38.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.112+0000 7f1bc5790640 1 -- start start 2024-04-24T23:24:38.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.112+0000 7f1bc5790640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1bc00d4c10 0x7f1bc00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.112+0000 7f1bc5790640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc015e370 0x7f1bc00c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.112+0000 7f1bc5790640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc00cdb10 0x7f1bc0156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.112+0000 7f1bc5790640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc00d2a60 con 0x7f1bc00cdb10 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.112+0000 7f1bc5790640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc00b3d00 con 0x7f1bc015e370 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.112+0000 7f1bc5790640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc00d2be0 con 0x7f1bc00d4c10 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.112+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc00cdb10 0x7f1bc0156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc00cdb10 0x7f1bc0156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39488/0 (socket says 172.21.15.26:39488) 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/4089456283 learned_addr learned my addr 172.21.15.26:0/4089456283 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbe7fc640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc015e370 0x7f1bc00c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1bc00d4c10 0x7f1bc00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/4089456283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1bc00d4c10 msgr2=0x7f1bc00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1bc00d4c10 0x7f1bc00cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/4089456283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc015e370 msgr2=0x7f1bc00c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc015e370 0x7f1bc00c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/4089456283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc00cc840 con 0x7f1bc00cdb10 2024-04-24T23:24:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc00cdb10 0x7f1bc0156740 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000cd70 tx=0x7f1bb000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1ba77fe640 1 -- 172.21.15.26:0/4089456283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0003b90 con 0x7f1bc00cdb10 2024-04-24T23:24:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1ba77fe640 1 -- 172.21.15.26:0/4089456283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb0003d30 con 0x7f1bc00cdb10 2024-04-24T23:24:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1ba77fe640 1 -- 172.21.15.26:0/4089456283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0004240 con 0x7f1bc00cdb10 2024-04-24T23:24:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bc5790640 1 -- 172.21.15.26:0/4089456283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc00ccb20 con 0x7f1bc00cdb10 2024-04-24T23:24:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.113+0000 7f1bc5790640 1 -- 172.21.15.26:0/4089456283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc00cd0e0 con 0x7f1bc00cdb10 2024-04-24T23:24:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.115+0000 7f1ba77fe640 1 -- 172.21.15.26:0/4089456283 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1bb00043e0 con 0x7f1bc00cdb10 2024-04-24T23:24:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.115+0000 7f1ba77fe640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1bac06be30 0x7f1bac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.115+0000 7f1ba77fe640 1 -- 172.21.15.26:0/4089456283 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1bb008b7d0 con 0x7f1bc00cdb10 2024-04-24T23:24:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.115+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1bac06be30 0x7f1bac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.116+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1bac06be30 0x7f1bac06e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8034000 tx=0x7f1bb802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.116+0000 7f1bc857c740 1 Processor -- start 2024-04-24T23:24:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.116+0000 7f1bc857c740 1 -- start start 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d043d60 0x55cb9d044140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d061600 0x55cb9d061b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d019a40 0x55cb9d05fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bc857c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cb9cf7d6b0 con 0x55cb9d019a40 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bc857c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cb9cf25880 con 0x55cb9d043d60 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bc857c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55cb9cf79670 con 0x55cb9d061600 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d019a40 0x55cb9d05fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d019a40 0x55cb9d05fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39490/0 (socket says 172.21.15.26:39490) 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/3677171105 learned_addr learned my addr 172.21.15.26:0/3677171105 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/3677171105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d061600 msgr2=0x55cb9d061b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/3677171105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d061600 0x55cb9d061b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/3677171105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d043d60 msgr2=0x55cb9d044140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/3677171105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d043d60 0x55cb9d044140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/3677171105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb9cd5b7d0 con 0x55cb9d019a40 2024-04-24T23:24:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/3677171105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d019a40 0x55cb9d05fcc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0059b50 tx=0x7f1bb0059ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3996a0de39dd1a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.117+0000 7f1ba4ff9640 1 -- 172.21.15.26:0/3677171105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0088070 con 0x55cb9d019a40 2024-04-24T23:24:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1ba4ff9640 1 -- 172.21.15.26:0/3677171105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb00626f0 con 0x55cb9d019a40 2024-04-24T23:24:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1ba4ff9640 1 -- 172.21.15.26:0/3677171105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0071030 con 0x55cb9d019a40 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1bc857c740 1 -- 172.21.15.26:0/3677171105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d019a40 msgr2=0x55cb9d05fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3677171105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d019a40 0x55cb9d05fcc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0059b50 tx=0x7f1bb0059ac0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1bc857c740 1 -- 172.21.15.26:0/3677171105 shutdown_connections 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3677171105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d019a40 0x55cb9d05fcc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3677171105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d061600 0x55cb9d061b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3677171105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d043d60 0x55cb9d044140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1bc857c740 1 -- 172.21.15.26:0/3677171105 >> 172.21.15.26:0/3677171105 conn(0x55cb9d016600 msgr2=0x55cb9d05eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1bc857c740 1 -- 172.21.15.26:0/3677171105 shutdown_connections 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.118+0000 7f1bc857c740 1 -- 172.21.15.26:0/3677171105 wait complete. 2024-04-24T23:24:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.119+0000 7f1bc857c740 1 -- start start 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.119+0000 7f1bc857c740 1 Processor -- start 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 0x55cb9d040340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 0x55cb9d046700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d05ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bc857c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cb9d01a7e0 con 0x55cb9d05faf0 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bc857c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cb9cf38f50 con 0x55cb9d061600 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bc857c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55cb9d014530 con 0x55cb9d043d60 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bc857c740 10 client.0 _ll_register_callbacks cb 0x55cb9cf74000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d05ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d05ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39500/0 (socket says 172.21.15.26:39500) 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/532861161 learned_addr learned my addr 172.21.15.26:0/532861161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbe7fc640 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 0x55cb9d046700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 0x55cb9d040340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/532861161 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 msgr2=0x55cb9d040340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 0x55cb9d040340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/532861161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 msgr2=0x55cb9d046700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 0x55cb9d046700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/532861161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb9cc913f0 con 0x55cb9d05faf0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d05ff30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1bb005c010 tx=0x7f1bb005c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6850a4476b9616c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1b9dffb640 1 -- 172.21.15.26:0/532861161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0076060 con 0x55cb9d05faf0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1b9dffb640 1 -- 172.21.15.26:0/532861161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1bb0062260 con 0x55cb9d05faf0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.120+0000 7f1bc857c740 1 -- 172.21.15.26:0/532861161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bb00626f0 con 0x55cb9d05faf0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1b9dffb640 1 -- 172.21.15.26:0/532861161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1bb0088070 con 0x55cb9d05faf0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bc857c740 10 client.6837 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bc857c740 1 -- 172.21.15.26:0/532861161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x55cb9cc510f0 con 0x55cb9d05faf0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1b9dffb640 1 -- 172.21.15.26:0/532861161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb00626f0 con 0x55cb9d05faf0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1b9dffb640 1 -- 172.21.15.26:0/532861161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7f1bb0059010 con 0x55cb9d05faf0 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bc857c740 10 client.6837 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bc857c740 10 client.6837 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.c 2024-04-24T23:24:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bc857c740 20 client.6837 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bc857c740 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55cb9d056210 0x55cb9d056620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bc857c740 4 client.6837 mds_command: new command op to 6531 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bc857c740 1 -- 172.21.15.26:0/532861161 --> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb9cf8abf0 con 0x55cb9d056210 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55cb9d056210 0x55cb9d056620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55cb9d056210 0x55cb9d056620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f83eeae9f4bccf server_cookie=4c06537f368f744a in_seq=0 out_seq=0 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.121+0000 7f1b9dffb640 10 client.6837 ms_handle_connect on v2:172.21.15.26:6834/1519104007 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1b9dffb640 1 -- 172.21.15.26:0/532861161 <== mds.2 v2:172.21.15.26:6834/1519104007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb9cf8abf0 con 0x55cb9d056210 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1b9dffb640 10 client.6837 handle_command_reply: tid=0 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 client.6837 shutdown 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 -- 172.21.15.26:0/532861161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 msgr2=0x55cb9d05ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d05ff30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1bb005c010 tx=0x7f1bb005c040 comp rx=0 tx=0).stop 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 -- 172.21.15.26:0/532861161 shutdown_connections 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55cb9d056210 0x55cb9d056620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d05ff30 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 0x55cb9d046700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 --2- 172.21.15.26:0/532861161 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 0x55cb9d040340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1b9dffb640 0 client.6837 ms_handle_reset on v2:172.21.15.26:6834/1519104007 2024-04-24T23:24:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 -- 172.21.15.26:0/532861161 >> 172.21.15.26:0/532861161 conn(0x55cb9d019a40 msgr2=0x55cb9d0620f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 -- 172.21.15.26:0/532861161 shutdown_connections 2024-04-24T23:24:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 1 -- 172.21.15.26:0/532861161 wait complete. 2024-04-24T23:24:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.122+0000 7f1bc857c740 20 client.6837 trim_cache size 0 max 16384 2024-04-24T23:24:38.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.137+0000 7f1bc857c740 1 Processor -- start 2024-04-24T23:24:38.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.137+0000 7f1bc857c740 1 -- start start 2024-04-24T23:24:38.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 0x55cb9d0429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d042f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 0x55cb9d0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bc857c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc02008a0 con 0x55cb9d05faf0 2024-04-24T23:24:38.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bc857c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc01ffb70 con 0x55cb9d061600 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bc857c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc0200170 con 0x55cb9d043d60 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbe7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d042f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbe7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d042f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39512/0 (socket says 172.21.15.26:39512) 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbe7fc640 1 -- 172.21.15.26:0/3718182888 learned_addr learned my addr 172.21.15.26:0/3718182888 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 0x55cb9d0429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 0x55cb9d0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbe7fc640 1 -- 172.21.15.26:0/3718182888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 msgr2=0x55cb9d0429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbe7fc640 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 0x55cb9d0429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbe7fc640 1 -- 172.21.15.26:0/3718182888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 msgr2=0x55cb9d0712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbe7fc640 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 0x55cb9d0712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbe7fc640 1 -- 172.21.15.26:0/3718182888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb9d006090 con 0x55cb9d05faf0 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.138+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 0x55cb9d0712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bbe7fc640 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d042f00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009990 tx=0x7f1bb4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9dba7170039f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1b9ffff640 1 -- 172.21.15.26:0/3718182888 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4019070 con 0x55cb9d05faf0 2024-04-24T23:24:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1b9ffff640 1 -- 172.21.15.26:0/3718182888 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb4011a60 con 0x55cb9d05faf0 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1b9ffff640 1 -- 172.21.15.26:0/3718182888 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4011e50 con 0x55cb9d05faf0 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bc857c740 1 -- 172.21.15.26:0/3718182888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 msgr2=0x55cb9d042f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d042f00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009990 tx=0x7f1bb4009e50 comp rx=0 tx=0).stop 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bc857c740 1 -- 172.21.15.26:0/3718182888 shutdown_connections 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d061600 0x55cb9d0712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d05faf0 0x55cb9d042f00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3718182888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d043d60 0x55cb9d0429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bc857c740 1 -- 172.21.15.26:0/3718182888 >> 172.21.15.26:0/3718182888 conn(0x55cb9d019a40 msgr2=0x55cb9d040150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bc857c740 1 -- 172.21.15.26:0/3718182888 shutdown_connections 2024-04-24T23:24:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.139+0000 7f1bc857c740 1 -- 172.21.15.26:0/3718182888 wait complete. 2024-04-24T23:24:38.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.140+0000 7f1bc857c740 1 -- start start 2024-04-24T23:24:38.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.140+0000 7f1bc857c740 1 Processor -- start 2024-04-24T23:24:38.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d043d60 0x55cb9d013dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d05faf0 0x55cb9d071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bc857c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d061600 0x55cb9d0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bc857c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cb9d060210 con 0x55cb9d061600 2024-04-24T23:24:38.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bc857c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc01f7830 con 0x55cb9d043d60 2024-04-24T23:24:38.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bc857c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55cb9d0dc8d0 con 0x55cb9d05faf0 2024-04-24T23:24:38.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bc857c740 10 client.0 _ll_register_callbacks cb 0x7f1bb0086070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d061600 0x55cb9d0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d061600 0x55cb9d0ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39528/0 (socket says 172.21.15.26:39528) 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/3866508186 learned_addr learned my addr 172.21.15.26:0/3866508186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d043d60 0x55cb9d013dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbe7fc640 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d05faf0 0x55cb9d071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/3866508186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d05faf0 msgr2=0x55cb9d071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d05faf0 0x55cb9d071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/3866508186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d043d60 msgr2=0x55cb9d013dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d043d60 0x55cb9d013dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbf7fe640 1 -- 172.21.15.26:0/3866508186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb9d072060 con 0x55cb9d061600 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bbf7fe640 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d061600 0x55cb9d0ce8b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00598c0 tx=0x7f1bb0007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e35c36abea2b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1b9dffb640 1 -- 172.21.15.26:0/3866508186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0076070 con 0x55cb9d061600 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1b9dffb640 1 -- 172.21.15.26:0/3866508186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1bb00600b0 con 0x55cb9d061600 2024-04-24T23:24:38.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1bc857c740 1 -- 172.21.15.26:0/3866508186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bb4011a60 con 0x55cb9d061600 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.141+0000 7f1b9dffb640 1 -- 172.21.15.26:0/3866508186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1bb006a050 con 0x55cb9d061600 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bc857c740 10 client.6843 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bc857c740 1 -- 172.21.15.26:0/3866508186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x55cb9d072060 con 0x55cb9d061600 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1b9dffb640 1 -- 172.21.15.26:0/3866508186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb4011a60 con 0x55cb9d061600 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1b9dffb640 1 -- 172.21.15.26:0/3866508186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7f1bb006d020 con 0x55cb9d061600 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bc857c740 10 client.6843 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bc857c740 10 client.6843 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.c 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bc857c740 20 client.6843 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55cb9d013260 0x55cb9d0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bc857c740 4 client.6843 mds_command: new command op to 6531 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bc857c740 1 -- 172.21.15.26:0/3866508186 --> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb9d015420 con 0x55cb9d013260 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55cb9d013260 0x55cb9d0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1b9dffb640 10 client.6843 ms_handle_connect on v2:172.21.15.26:6834/1519104007 2024-04-24T23:24:38.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.142+0000 7f1bbeffd640 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55cb9d013260 0x55cb9d0cedf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf948d2a0530b18e server_cookie=fc204aa7678d1fac in_seq=0 out_seq=0 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1b9dffb640 1 -- 172.21.15.26:0/3866508186 <== mds.2 v2:172.21.15.26:6834/1519104007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb9d015420 con 0x55cb9d013260 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1b9dffb640 10 client.6843 handle_command_reply: tid=0 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 client.6843 shutdown 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 -- 172.21.15.26:0/3866508186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d061600 msgr2=0x55cb9d0ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d061600 0x55cb9d0ce8b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00598c0 tx=0x7f1bb0007a10 comp rx=0 tx=0).stop 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 -- 172.21.15.26:0/3866508186 shutdown_connections 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x55cb9d013260 0x55cb9d0cedf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb9d061600 0x55cb9d0ce8b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55cb9d05faf0 0x55cb9d071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1b9dffb640 0 client.6843 ms_handle_reset on v2:172.21.15.26:6834/1519104007 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 --2- 172.21.15.26:0/3866508186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cb9d043d60 0x55cb9d013dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 -- 172.21.15.26:0/3866508186 >> 172.21.15.26:0/3866508186 conn(0x55cb9d019a40 msgr2=0x55cb9d06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 -- 172.21.15.26:0/3866508186 shutdown_connections 2024-04-24T23:24:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 1 -- 172.21.15.26:0/3866508186 wait complete. 2024-04-24T23:24:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.150+0000 7f1bc857c740 20 client.6843 trim_cache size 0 max 16384 2024-04-24T23:24:38.153 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:38.153 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:24:38.153 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:24:38.153 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 -- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1bac06be30 msgr2=0x7f1bac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1bac06be30 0x7f1bac06e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8034000 tx=0x7f1bb802e000 comp rx=0 tx=0).stop 2024-04-24T23:24:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 -- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc00cdb10 msgr2=0x7f1bc0156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc00cdb10 0x7f1bc0156740 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000cd70 tx=0x7f1bb000a660 comp rx=0 tx=0).stop 2024-04-24T23:24:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 -- 172.21.15.26:0/4089456283 shutdown_connections 2024-04-24T23:24:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1bac06be30 0x7f1bac06e2f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bc00cdb10 0x7f1bc0156740 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1bc015e370 0x7f1bc00c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 --2- 172.21.15.26:0/4089456283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1bc00d4c10 0x7f1bc00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 -- 172.21.15.26:0/4089456283 >> 172.21.15.26:0/4089456283 conn(0x7f1bc00d40f0 msgr2=0x7f1bc006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:38.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 -- 172.21.15.26:0/4089456283 shutdown_connections 2024-04-24T23:24:38.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.151+0000 7f1bc5790640 1 -- 172.21.15.26:0/4089456283 wait complete. 2024-04-24T23:24:38.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:38.163 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-04-24T23:24:38.184 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/ 2024-04-24T23:24:38.184 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4 2024-04-24T23:24:38.184 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5 2024-04-24T23:24:38.184 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6 2024-04-24T23:24:38.184 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7 2024-04-24T23:24:38.184 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-04-24T23:24:38.185 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:24:38.185 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:24:38.185 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:24:38.185 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:24:38.185 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-04-24T23:24:38.185 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:24:38.292 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627778 2024-04-24T23:24:38.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:24:38.300 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:24:38.301 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:24:38.301 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:24:38.301 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:24:38.301 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-04-24T23:24:38.301 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:24:38.408 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627779 2024-04-24T23:24:38.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:24:38.417 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:24:38.417 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:24:38.417 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:24:38.417 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:24:38.417 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-04-24T23:24:38.418 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:24:38.524 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627780 2024-04-24T23:24:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:24:38.533 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:24:38.533 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:24:38.533 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:24:38.534 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:24:38.534 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-04-24T23:24:38.534 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:24:38.640 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627781 2024-04-24T23:24:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:24:38.649 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:24:38.649 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:24:38.650 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:24:38.650 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:24:38.650 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-04-24T23:24:38.650 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:24:38.756 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627782 2024-04-24T23:24:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:24:38.766 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:24:38.766 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:24:38.766 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:24:38.766 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:24:38.766 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-04-24T23:24:38.766 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:24:38.873 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627783 2024-04-24T23:24:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:24:38.881 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-04-24T23:24:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.964+0000 7ff5b7d20f40 1 Processor -- start 2024-04-24T23:24:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.964+0000 7ff5b7d20f40 1 -- start start 2024-04-24T23:24:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b7d20f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5a00022a0 0x562f61d5a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b7d20f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d69f50 0x562f61d653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b7d20f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562f61d65900 0x562f61cbe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b7d20f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562f61d6bc10 con 0x562f61d69f50 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b7d20f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562f61c3a640 con 0x562f61d65900 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b7d20f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562f61cdb210 con 0x7ff5a00022a0 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b4705640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d69f50 0x562f61d653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b4705640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d69f50 0x562f61d653c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39544/0 (socket says 172.21.15.26:39544) 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b4705640 1 -- 172.21.15.26:0/3960050075 learned_addr learned my addr 172.21.15.26:0/3960050075 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b4705640 1 -- 172.21.15.26:0/3960050075 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5a00022a0 msgr2=0x562f61d5a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b4705640 1 --2- 172.21.15.26:0/3960050075 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5a00022a0 0x562f61d5a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b4705640 1 -- 172.21.15.26:0/3960050075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562f61d65900 msgr2=0x562f61cbe940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b4705640 1 --2- 172.21.15.26:0/3960050075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562f61d65900 0x562f61cbe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b4705640 1 -- 172.21.15.26:0/3960050075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f61d5ad80 con 0x562f61d69f50 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b4705640 1 --2- 172.21.15.26:0/3960050075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d69f50 0x562f61d653c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c780 tx=0x7ff59c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff483c33c1e452a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b3703640 1 -- 172.21.15.26:0/3960050075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c012070 con 0x562f61d69f50 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b3703640 1 -- 172.21.15.26:0/3960050075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59c00ce30 con 0x562f61d69f50 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.965+0000 7ff5b3703640 1 -- 172.21.15.26:0/3960050075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c005030 con 0x562f61d69f50 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/3960050075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d69f50 msgr2=0x562f61d653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 --2- 172.21.15.26:0/3960050075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d69f50 0x562f61d653c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c780 tx=0x7ff59c00cc40 comp rx=0 tx=0).stop 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/3960050075 shutdown_connections 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 --2- 172.21.15.26:0/3960050075 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562f61d65900 0x562f61cbe940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 --2- 172.21.15.26:0/3960050075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d69f50 0x562f61d653c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 --2- 172.21.15.26:0/3960050075 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff5a00022a0 0x562f61d5a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/3960050075 >> 172.21.15.26:0/3960050075 conn(0x562f61cbfbb0 msgr2=0x562f61d687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/3960050075 shutdown_connections 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/3960050075 wait complete. 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 Processor -- start 2024-04-24T23:24:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.966+0000 7ff5b7d20f40 1 -- start start 2024-04-24T23:24:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b7d20f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d65900 0x562f61d68f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b7d20f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562f61d69f50 0x562f61d56e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b7d20f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5a00022a0 0x562f61d573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b7d20f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562f61d68420 con 0x562f61d65900 2024-04-24T23:24:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b7d20f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562f61d618a0 con 0x7ff5a00022a0 2024-04-24T23:24:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b7d20f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562f61d61c30 con 0x562f61d69f50 2024-04-24T23:24:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b4f06640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d65900 0x562f61d68f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b4f06640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d65900 0x562f61d68f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39556/0 (socket says 172.21.15.26:39556) 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b4f06640 1 -- 172.21.15.26:0/1110639786 learned_addr learned my addr 172.21.15.26:0/1110639786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b5707640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5a00022a0 0x562f61d573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.967+0000 7ff5b4705640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562f61d69f50 0x562f61d56e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b4f06640 1 -- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562f61d69f50 msgr2=0x562f61d56e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b4f06640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562f61d69f50 0x562f61d56e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b4f06640 1 -- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5a00022a0 msgr2=0x562f61d573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b4f06640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5a00022a0 0x562f61d573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b4f06640 1 -- 172.21.15.26:0/1110639786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f61d62430 con 0x562f61d65900 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b4705640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562f61d69f50 0x562f61d56e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b5707640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff5a00022a0 0x562f61d573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b4f06640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d65900 0x562f61d68f00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009760 tx=0x7ff5ac005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b2701640 1 -- 172.21.15.26:0/1110639786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac04f860 con 0x562f61d65900 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b2701640 1 -- 172.21.15.26:0/1110639786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac04fa20 con 0x562f61d65900 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f61d62710 con 0x562f61d65900 2024-04-24T23:24:38.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b2701640 1 -- 172.21.15.26:0/1110639786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac04fe10 con 0x562f61d65900 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.968+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f61d62cd0 con 0x562f61d65900 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.969+0000 7ff5b2701640 1 -- 172.21.15.26:0/1110639786 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5ac062030 con 0x562f61d65900 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.970+0000 7ff5b2701640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff59406a810 0x7ff59406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.970+0000 7ff5b2701640 1 -- 172.21.15.26:0/1110639786 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff5ac0cc9d0 con 0x562f61d65900 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.970+0000 7ff5b4705640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff59406a810 0x7ff59406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.970+0000 7ff5b7d20f40 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x562f61d4e8a0 0x562f61d639e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.970+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- osd_op(unknown.0.0:1 8.12 8:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) -- 0x562f61d63f20 con 0x562f61d4e8a0 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.970+0000 7ff5b4705640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff59406a810 0x7ff59406ccd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c780 tx=0x7ff59c006000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.970+0000 7ff5b5707640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x562f61d4e8a0 0x562f61d639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.971+0000 7ff5b5707640 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x562f61d4e8a0 0x562f61d639e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b5707640 1 -- 172.21.15.26:0/1110639786 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v58'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff5a4011040 con 0x562f61d4e8a0 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x562f61d4e8a0 msgr2=0x562f61d639e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x562f61d4e8a0 0x562f61d639e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff59406a810 msgr2=0x7ff59406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff59406a810 0x7ff59406ccd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c780 tx=0x7ff59c006000 comp rx=0 tx=0).stop 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d65900 msgr2=0x562f61d68f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 --2- 172.21.15.26:0/1110639786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562f61d65900 0x562f61d68f00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009760 tx=0x7ff5ac005f50 comp rx=0 tx=0).stop 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b5707640 1 -- 172.21.15.26:0/1110639786 reap_dead start 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 shutdown_connections 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 >> 172.21.15.26:0/1110639786 conn(0x562f61cbfbb0 msgr2=0x562f61d687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:38.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 shutdown_connections 2024-04-24T23:24:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:38.973+0000 7ff5b7d20f40 1 -- 172.21.15.26:0/1110639786 wait complete. 2024-04-24T23:24:38.977 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-04-24T23:24:39.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.060+0000 7fae72ef6f40 1 Processor -- start 2024-04-24T23:24:39.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.060+0000 7fae72ef6f40 1 -- start start 2024-04-24T23:24:39.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae72ef6f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f08a25e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae72ef6f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f08ab2c50 0x560f08a24670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae72ef6f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f089e5960 0x560f08a24bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae72ef6f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560f08989640 con 0x560f089e5580 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae72ef6f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560f0898d680 con 0x560f08ab2c50 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae72ef6f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560f08a23170 con 0x560f089e5960 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae709f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f08a25e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae709f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f08a25e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39562/0 (socket says 172.21.15.26:39562) 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae709f9640 1 -- 172.21.15.26:0/848575020 learned_addr learned my addr 172.21.15.26:0/848575020 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae709f9640 1 -- 172.21.15.26:0/848575020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f089e5960 msgr2=0x560f08a24bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae709f9640 1 --2- 172.21.15.26:0/848575020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f089e5960 0x560f08a24bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae709f9640 1 -- 172.21.15.26:0/848575020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f08ab2c50 msgr2=0x560f08a24670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae709f9640 1 --2- 172.21.15.26:0/848575020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f08ab2c50 0x560f08a24670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae709f9640 1 -- 172.21.15.26:0/848575020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f08ab28c0 con 0x560f089e5580 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae709f9640 1 --2- 172.21.15.26:0/848575020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f08a25e40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fae64002fb0 tx=0x7fae6400e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c96699c3b85fbb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae6affd640 1 -- 172.21.15.26:0/848575020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64001ac0 con 0x560f089e5580 2024-04-24T23:24:39.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae6affd640 1 -- 172.21.15.26:0/848575020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae64001c60 con 0x560f089e5580 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.061+0000 7fae6affd640 1 -- 172.21.15.26:0/848575020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64002250 con 0x560f089e5580 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 -- 172.21.15.26:0/848575020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 msgr2=0x560f08a25e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 --2- 172.21.15.26:0/848575020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f08a25e40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fae64002fb0 tx=0x7fae6400e760 comp rx=0 tx=0).stop 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 -- 172.21.15.26:0/848575020 shutdown_connections 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 --2- 172.21.15.26:0/848575020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f089e5960 0x560f08a24bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 --2- 172.21.15.26:0/848575020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f08ab2c50 0x560f08a24670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 --2- 172.21.15.26:0/848575020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f08a25e40 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 -- 172.21.15.26:0/848575020 >> 172.21.15.26:0/848575020 conn(0x560f089e0400 msgr2=0x560f08a23870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 -- 172.21.15.26:0/848575020 shutdown_connections 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 -- 172.21.15.26:0/848575020 wait complete. 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 Processor -- start 2024-04-24T23:24:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.062+0000 7fae72ef6f40 1 -- start start 2024-04-24T23:24:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae72ef6f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f089e31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae72ef6f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f089e5960 0x560f08a24b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae72ef6f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f08ab2c50 0x560f08a2a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae72ef6f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560f08ab61e0 con 0x560f089e5580 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae72ef6f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560f08ab64c0 con 0x560f08ab2c50 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae72ef6f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560f08a233a0 con 0x560f089e5960 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae709f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f089e31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae709f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f089e31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39568/0 (socket says 172.21.15.26:39568) 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae709f9640 1 -- 172.21.15.26:0/3898288971 learned_addr learned my addr 172.21.15.26:0/3898288971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae6bfff640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f089e5960 0x560f08a24b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.063+0000 7fae711fa640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f08ab2c50 0x560f08a2a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae709f9640 1 -- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f089e5960 msgr2=0x560f08a24b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae709f9640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560f089e5960 0x560f08a24b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae709f9640 1 -- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f08ab2c50 msgr2=0x560f08a2a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae709f9640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560f08ab2c50 0x560f08a2a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae709f9640 1 -- 172.21.15.26:0/3898288971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f08a26170 con 0x560f089e5580 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae709f9640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f089e31a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fae64002940 tx=0x7fae64001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae69ffb640 1 -- 172.21.15.26:0/3898288971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64001ac0 con 0x560f089e5580 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae69ffb640 1 -- 172.21.15.26:0/3898288971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae64000ed0 con 0x560f089e5580 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae69ffb640 1 -- 172.21.15.26:0/3898288971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64011aa0 con 0x560f089e5580 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f089e26c0 con 0x560f089e5580 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.064+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f089e2c80 con 0x560f089e5580 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.065+0000 7fae69ffb640 1 -- 172.21.15.26:0/3898288971 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae64011c40 con 0x560f089e5580 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.066+0000 7fae69ffb640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae5406a840 0x7fae5406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.066+0000 7fae69ffb640 1 -- 172.21.15.26:0/3898288971 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fae6408b350 con 0x560f089e5580 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.066+0000 7fae6bfff640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae5406a840 0x7fae5406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.066+0000 7fae72ef6f40 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x560f08aacb40 0x560f089e1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.066+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- osd_op(unknown.0.0:1 8.12 8:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) -- 0x560f089e1a50 con 0x560f08aacb40 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.066+0000 7fae6bfff640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae5406a840 0x7fae5406cd00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x560f08aad460 tx=0x7fae5c001110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.066+0000 7fae711fa640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x560f08aacb40 0x560f089e1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.067+0000 7fae711fa640 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x560f08aacb40 0x560f089e1510 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae711fa640 1 -- 172.21.15.26:0/3898288971 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v58'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fae6c005040 con 0x560f08aacb40 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x560f08aacb40 msgr2=0x560f089e1510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x560f08aacb40 0x560f089e1510 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae5406a840 msgr2=0x7fae5406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae5406a840 0x7fae5406cd00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x560f08aad460 tx=0x7fae5c001110 comp rx=0 tx=0).stop 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 msgr2=0x560f089e31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 --2- 172.21.15.26:0/3898288971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f089e5580 0x560f089e31a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fae64002940 tx=0x7fae64001190 comp rx=0 tx=0).stop 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae711fa640 1 -- 172.21.15.26:0/3898288971 reap_dead start 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 shutdown_connections 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 >> 172.21.15.26:0/3898288971 conn(0x560f089e0400 msgr2=0x560f08a23a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 shutdown_connections 2024-04-24T23:24:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.068+0000 7fae72ef6f40 1 -- 172.21.15.26:0/3898288971 wait complete. 2024-04-24T23:24:39.072 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-04-24T23:24:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639b14df40 1 Processor -- start 2024-04-24T23:24:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639b14df40 1 -- start start 2024-04-24T23:24:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639b14df40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56165aa54a90 0x56165ab16ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639b14df40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54e70 0x56165ab291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639b14df40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56165aa99250 0x56165ab296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639b14df40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56165a9f9640 con 0x56165aa54e70 2024-04-24T23:24:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639b14df40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56165a9fd680 con 0x56165aa99250 2024-04-24T23:24:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639b14df40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56165aa999b0 con 0x56165aa54a90 2024-04-24T23:24:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639844d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54e70 0x56165ab291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639844d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54e70 0x56165ab291b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39574/0 (socket says 172.21.15.26:39574) 2024-04-24T23:24:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639844d640 1 -- 172.21.15.26:0/807504384 learned_addr learned my addr 172.21.15.26:0/807504384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f6398c4e640 1 --2- 172.21.15.26:0/807504384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56165aa54a90 0x56165ab16ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.114+0000 7f639944f640 1 --2- 172.21.15.26:0/807504384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56165aa99250 0x56165ab296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639844d640 1 -- 172.21.15.26:0/807504384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56165aa54a90 msgr2=0x56165ab16ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639844d640 1 --2- 172.21.15.26:0/807504384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56165aa54a90 0x56165ab16ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639844d640 1 -- 172.21.15.26:0/807504384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56165aa99250 msgr2=0x56165ab296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639844d640 1 --2- 172.21.15.26:0/807504384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56165aa99250 0x56165ab296f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639844d640 1 -- 172.21.15.26:0/807504384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165ab1fe80 con 0x56165aa54e70 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639844d640 1 --2- 172.21.15.26:0/807504384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54e70 0x56165ab291b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6380001ea0 tx=0x7f638000b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc83c14a8a23760 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639744b640 1 -- 172.21.15.26:0/807504384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380008070 con 0x56165aa54e70 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639744b640 1 -- 172.21.15.26:0/807504384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638000d500 con 0x56165aa54e70 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639744b640 1 -- 172.21.15.26:0/807504384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638000d910 con 0x56165aa54e70 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639b14df40 1 -- 172.21.15.26:0/807504384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54e70 msgr2=0x56165ab291b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639b14df40 1 --2- 172.21.15.26:0/807504384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54e70 0x56165ab291b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6380001ea0 tx=0x7f638000b6a0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639b14df40 1 -- 172.21.15.26:0/807504384 shutdown_connections 2024-04-24T23:24:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639b14df40 1 --2- 172.21.15.26:0/807504384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56165aa99250 0x56165ab296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639b14df40 1 --2- 172.21.15.26:0/807504384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54e70 0x56165ab291b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639b14df40 1 --2- 172.21.15.26:0/807504384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56165aa54a90 0x56165ab16ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.115+0000 7f639b14df40 1 -- 172.21.15.26:0/807504384 >> 172.21.15.26:0/807504384 conn(0x56165aa7cb90 msgr2=0x56165ab281c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.116+0000 7f639b14df40 1 -- 172.21.15.26:0/807504384 shutdown_connections 2024-04-24T23:24:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.116+0000 7f639b14df40 1 -- 172.21.15.26:0/807504384 wait complete. 2024-04-24T23:24:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.116+0000 7f639b14df40 1 Processor -- start 2024-04-24T23:24:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.116+0000 7f639b14df40 1 -- start start 2024-04-24T23:24:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f639b14df40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54a90 0x56165ab2d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f639b14df40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56165aa54e70 0x56165abbfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f639b14df40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56165aa99250 0x56165ab1b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f639b14df40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56165ab1aaf0 con 0x56165aa54a90 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f639b14df40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56165ab25d60 con 0x56165aa54e70 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f639b14df40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56165ab25b10 con 0x56165aa99250 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f6398c4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54a90 0x56165ab2d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f6398c4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54a90 0x56165ab2d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39578/0 (socket says 172.21.15.26:39578) 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f6398c4e640 1 -- 172.21.15.26:0/1482125815 learned_addr learned my addr 172.21.15.26:0/1482125815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f639944f640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56165aa99250 0x56165ab1b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f639844d640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56165aa54e70 0x56165abbfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f6398c4e640 1 -- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56165aa99250 msgr2=0x56165ab1b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f6398c4e640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56165aa99250 0x56165ab1b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f6398c4e640 1 -- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56165aa54e70 msgr2=0x56165abbfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f6398c4e640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56165aa54e70 0x56165abbfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f6398c4e640 1 -- 172.21.15.26:0/1482125815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165ab2db50 con 0x56165aa54a90 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.117+0000 7f6398c4e640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54a90 0x56165ab2d610 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6390005400 tx=0x7f6390003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.118+0000 7f6396449640 1 -- 172.21.15.26:0/1482125815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639006c070 con 0x56165aa54a90 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.118+0000 7f6396449640 1 -- 172.21.15.26:0/1482125815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6390052d50 con 0x56165aa54a90 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.118+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56165ab1d3a0 con 0x56165aa54a90 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.118+0000 7f6396449640 1 -- 172.21.15.26:0/1482125815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390056070 con 0x56165aa54a90 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.118+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165ab1d960 con 0x56165aa54a90 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.119+0000 7f6396449640 1 -- 172.21.15.26:0/1482125815 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f639006f070 con 0x56165aa54a90 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.119+0000 7f6396449640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637806a810 0x7f637806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.119+0000 7f6396449640 1 -- 172.21.15.26:0/1482125815 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f63900ce140 con 0x56165aa54a90 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.120+0000 7f639844d640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637806a810 0x7f637806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.120+0000 7f639b14df40 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56165ab1b7a0 0x56165ab27300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.120+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- osd_op(unknown.0.0:1 8.5 8:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) -- 0x56165abc0350 con 0x56165ab1b7a0 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.120+0000 7f639844d640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637806a810 0x7f637806ccd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6380000fb0 tx=0x7f6380009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.120+0000 7f639944f640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56165ab1b7a0 0x56165ab27300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.120+0000 7f639944f640 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56165ab1b7a0 0x56165ab27300 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639944f640 1 -- 172.21.15.26:0/1482125815 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v58'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6388002720 con 0x56165ab1b7a0 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56165ab1b7a0 msgr2=0x56165ab27300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56165ab1b7a0 0x56165ab27300 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637806a810 msgr2=0x7f637806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f637806a810 0x7f637806ccd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6380000fb0 tx=0x7f6380009000 comp rx=0 tx=0).stop 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54a90 msgr2=0x56165ab2d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 --2- 172.21.15.26:0/1482125815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56165aa54a90 0x56165ab2d610 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6390005400 tx=0x7f6390003020 comp rx=0 tx=0).stop 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639944f640 1 -- 172.21.15.26:0/1482125815 reap_dead start 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 shutdown_connections 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 >> 172.21.15.26:0/1482125815 conn(0x56165aa7cb90 msgr2=0x56165ab1edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 shutdown_connections 2024-04-24T23:24:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.122+0000 7f639b14df40 1 -- 172.21.15.26:0/1482125815 wait complete. 2024-04-24T23:24:39.126 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-04-24T23:24:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.208+0000 7f880abfef40 1 Processor -- start 2024-04-24T23:24:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.208+0000 7f880abfef40 1 -- start start 2024-04-24T23:24:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.208+0000 7f880abfef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561d4e12e540 0x561d4e12ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.208+0000 7f880abfef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e147430 0x561d4e101580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.208+0000 7f880abfef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e1d37a0 0x561d4e1d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.208+0000 7f880abfef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561d4e147c20 con 0x561d4e147430 2024-04-24T23:24:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.208+0000 7f880abfef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561d4e100940 con 0x561d4e1d37a0 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.208+0000 7f880abfef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561d4e100b90 con 0x561d4e12e540 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8807f00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e147430 0x561d4e101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8807f00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e147430 0x561d4e101580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39602/0 (socket says 172.21.15.26:39602) 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8807f00640 1 -- 172.21.15.26:0/2112976982 learned_addr learned my addr 172.21.15.26:0/2112976982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8808701640 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561d4e12e540 0x561d4e12ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8808f02640 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e1d37a0 0x561d4e1d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8807f00640 1 -- 172.21.15.26:0/2112976982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561d4e12e540 msgr2=0x561d4e12ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8807f00640 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561d4e12e540 0x561d4e12ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8807f00640 1 -- 172.21.15.26:0/2112976982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e1d37a0 msgr2=0x561d4e1d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8807f00640 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e1d37a0 0x561d4e1d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8807f00640 1 -- 172.21.15.26:0/2112976982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4e02acb0 con 0x561d4e147430 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8808f02640 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e1d37a0 0x561d4e1d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8807f00640 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e147430 0x561d4e101580 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f87f000a7a0 tx=0x7f87f000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4145aa018425816 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8806efe640 1 -- 172.21.15.26:0/2112976982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0019070 con 0x561d4e147430 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8806efe640 1 -- 172.21.15.26:0/2112976982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f0007600 con 0x561d4e147430 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.209+0000 7f8806efe640 1 -- 172.21.15.26:0/2112976982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00079d0 con 0x561d4e147430 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 -- 172.21.15.26:0/2112976982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e147430 msgr2=0x561d4e101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e147430 0x561d4e101580 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f87f000a7a0 tx=0x7f87f000a890 comp rx=0 tx=0).stop 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 -- 172.21.15.26:0/2112976982 shutdown_connections 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e1d37a0 0x561d4e1d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e147430 0x561d4e101580 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 --2- 172.21.15.26:0/2112976982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561d4e12e540 0x561d4e12ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 -- 172.21.15.26:0/2112976982 >> 172.21.15.26:0/2112976982 conn(0x561d4e0fc400 msgr2=0x561d4e0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 -- 172.21.15.26:0/2112976982 shutdown_connections 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 -- 172.21.15.26:0/2112976982 wait complete. 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 Processor -- start 2024-04-24T23:24:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.210+0000 7f880abfef40 1 -- start start 2024-04-24T23:24:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f880abfef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e12e540 0x561d4e1ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f880abfef40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e147430 0x561d4e1c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f880abfef40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561d4e1d37a0 0x561d4e1443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f880abfef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561d4e12e300 con 0x561d4e12e540 2024-04-24T23:24:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f880abfef40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561d4e128ad0 con 0x561d4e147430 2024-04-24T23:24:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f880abfef40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561d4e129360 con 0x561d4e1d37a0 2024-04-24T23:24:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f8808701640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e12e540 0x561d4e1ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f8808701640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e12e540 0x561d4e1ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39606/0 (socket says 172.21.15.26:39606) 2024-04-24T23:24:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f8808701640 1 -- 172.21.15.26:0/17143271 learned_addr learned my addr 172.21.15.26:0/17143271 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f8808f02640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561d4e1d37a0 0x561d4e1443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.211+0000 7f8807f00640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e147430 0x561d4e1c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f8808701640 1 -- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561d4e1d37a0 msgr2=0x561d4e1443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f8808701640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561d4e1d37a0 0x561d4e1443e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f8808701640 1 -- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e147430 msgr2=0x561d4e1c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f8808701640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561d4e147430 0x561d4e1c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f8808701640 1 -- 172.21.15.26:0/17143271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4e1ccff0 con 0x561d4e12e540 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f8808701640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e12e540 0x561d4e1ce820 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f87f800b480 tx=0x7f87f800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f8805efc640 1 -- 172.21.15.26:0/17143271 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8016020 con 0x561d4e12e540 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f8805efc640 1 -- 172.21.15.26:0/17143271 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f8012660 con 0x561d4e12e540 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d4e1d1790 con 0x561d4e12e540 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f8805efc640 1 -- 172.21.15.26:0/17143271 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8012a30 con 0x561d4e12e540 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.212+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4e1d1d50 con 0x561d4e12e540 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.213+0000 7f8805efc640 1 -- 172.21.15.26:0/17143271 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87f8019020 con 0x561d4e12e540 2024-04-24T23:24:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.214+0000 7f8805efc640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87e806a840 0x7f87e806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.214+0000 7f8805efc640 1 -- 172.21.15.26:0/17143271 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f87f8089eb0 con 0x561d4e12e540 2024-04-24T23:24:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.214+0000 7f8807f00640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87e806a840 0x7f87e806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.214+0000 7f880abfef40 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x561d4e1c9bf0 0x561d4e1c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.214+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- osd_op(unknown.0.0:1 8.9 8:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) -- 0x561d4e1ca530 con 0x561d4e1c9bf0 2024-04-24T23:24:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.214+0000 7f8807f00640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87e806a840 0x7f87e806cd00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x561d4e0ff6f0 tx=0x7f87f000c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.214+0000 7f8808f02640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x561d4e1c9bf0 0x561d4e1c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.215+0000 7f8808f02640 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x561d4e1c9bf0 0x561d4e1c9ff0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f8808f02640 1 -- 172.21.15.26:0/17143271 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v58'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f880005c040 con 0x561d4e1c9bf0 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x561d4e1c9bf0 msgr2=0x561d4e1c9ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x561d4e1c9bf0 0x561d4e1c9ff0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87e806a840 msgr2=0x7f87e806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87e806a840 0x7f87e806cd00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x561d4e0ff6f0 tx=0x7f87f000c000 comp rx=0 tx=0).stop 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e12e540 msgr2=0x561d4e1ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 --2- 172.21.15.26:0/17143271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561d4e12e540 0x561d4e1ce820 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f87f800b480 tx=0x7f87f800b940 comp rx=0 tx=0).stop 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f8808f02640 1 -- 172.21.15.26:0/17143271 reap_dead start 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 shutdown_connections 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 >> 172.21.15.26:0/17143271 conn(0x561d4e0fc400 msgr2=0x561d4e0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 shutdown_connections 2024-04-24T23:24:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.216+0000 7f880abfef40 1 -- 172.21.15.26:0/17143271 wait complete. 2024-04-24T23:24:39.220 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-04-24T23:24:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.302+0000 7fecb0139f40 1 Processor -- start 2024-04-24T23:24:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.302+0000 7fecb0139f40 1 -- start start 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecb0139f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c469664870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecb0139f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c46966c1a0 0x55c469674650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecb0139f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c469674b90 0x55c4696761b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecb0139f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c469546640 con 0x55c469663930 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecb0139f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c46954a680 con 0x55c469674b90 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecb0139f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c4695e69b0 con 0x55c46966c1a0 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecadc3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c469664870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecadc3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c469664870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39626/0 (socket says 172.21.15.26:39626) 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecadc3a640 1 -- 172.21.15.26:0/1942594506 learned_addr learned my addr 172.21.15.26:0/1942594506 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecae43b640 1 --2- 172.21.15.26:0/1942594506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c469674b90 0x55c4696761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecad439640 1 --2- 172.21.15.26:0/1942594506 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c46966c1a0 0x55c469674650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecadc3a640 1 -- 172.21.15.26:0/1942594506 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c46966c1a0 msgr2=0x55c469674650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecadc3a640 1 --2- 172.21.15.26:0/1942594506 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c46966c1a0 0x55c469674650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecadc3a640 1 -- 172.21.15.26:0/1942594506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c469674b90 msgr2=0x55c4696761b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecadc3a640 1 --2- 172.21.15.26:0/1942594506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c469674b90 0x55c4696761b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecadc3a640 1 -- 172.21.15.26:0/1942594506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c469663d10 con 0x55c469663930 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.303+0000 7fecadc3a640 1 --2- 172.21.15.26:0/1942594506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c469664870 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7feca80096b0 tx=0x7feca8009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56287e69a09a77f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7feca7fff640 1 -- 172.21.15.26:0/1942594506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca800e960 con 0x55c469663930 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7feca7fff640 1 -- 172.21.15.26:0/1942594506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8013030 con 0x55c469663930 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7feca7fff640 1 -- 172.21.15.26:0/1942594506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca800ed40 con 0x55c469663930 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7fecb0139f40 1 -- 172.21.15.26:0/1942594506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 msgr2=0x55c469664870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7fecb0139f40 1 --2- 172.21.15.26:0/1942594506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c469664870 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7feca80096b0 tx=0x7feca8009b70 comp rx=0 tx=0).stop 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7fecb0139f40 1 -- 172.21.15.26:0/1942594506 shutdown_connections 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7fecb0139f40 1 --2- 172.21.15.26:0/1942594506 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c469674b90 0x55c4696761b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7fecb0139f40 1 --2- 172.21.15.26:0/1942594506 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c46966c1a0 0x55c469674650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7fecb0139f40 1 --2- 172.21.15.26:0/1942594506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c469664870 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7fecb0139f40 1 -- 172.21.15.26:0/1942594506 >> 172.21.15.26:0/1942594506 conn(0x55c4695c9b90 msgr2=0x55c4695ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7fecb0139f40 1 -- 172.21.15.26:0/1942594506 shutdown_connections 2024-04-24T23:24:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.304+0000 7fecb0139f40 1 -- 172.21.15.26:0/1942594506 wait complete. 2024-04-24T23:24:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.305+0000 7fecb0139f40 1 Processor -- start 2024-04-24T23:24:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.305+0000 7fecb0139f40 1 -- start start 2024-04-24T23:24:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.305+0000 7fecb0139f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c4695ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.305+0000 7fecb0139f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c46966c1a0 0x55c46965a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.305+0000 7fecb0139f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c469674b90 0x55c4696761b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.305+0000 7fecb0139f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c469665380 con 0x55c469663930 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.305+0000 7fecb0139f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c469665130 con 0x55c46966c1a0 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.305+0000 7fecb0139f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c4695e60c0 con 0x55c469674b90 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecadc3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c4695ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecadc3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c4695ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39628/0 (socket says 172.21.15.26:39628) 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecadc3a640 1 -- 172.21.15.26:0/2194635735 learned_addr learned my addr 172.21.15.26:0/2194635735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecad439640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c46966c1a0 0x55c46965a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecae43b640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c469674b90 0x55c4696761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecadc3a640 1 -- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c469674b90 msgr2=0x55c4696761b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecadc3a640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c469674b90 0x55c4696761b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecadc3a640 1 -- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c46966c1a0 msgr2=0x55c46965a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecadc3a640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c46966c1a0 0x55c46965a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecadc3a640 1 -- 172.21.15.26:0/2194635735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c469661cb0 con 0x55c469663930 2024-04-24T23:24:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecadc3a640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c4695ceb30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7feca8013000 tx=0x7feca8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7feca6ffd640 1 -- 172.21.15.26:0/2194635735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca800f860 con 0x55c469663930 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7feca6ffd640 1 -- 172.21.15.26:0/2194635735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca800fa00 con 0x55c469663930 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c469669700 con 0x55c469663930 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7feca6ffd640 1 -- 172.21.15.26:0/2194635735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca800fe30 con 0x55c469663930 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.306+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c469669ca0 con 0x55c469663930 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.308+0000 7feca6ffd640 1 -- 172.21.15.26:0/2194635735 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feca8023070 con 0x55c469663930 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.308+0000 7feca6ffd640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec9006a810 0x7fec9006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.308+0000 7feca6ffd640 1 -- 172.21.15.26:0/2194635735 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7feca808c3a0 con 0x55c469663930 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.308+0000 7fecad439640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec9006a810 0x7fec9006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.308+0000 7fecb0139f40 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55c46966a780 0x55c4696786f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.308+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- osd_op(unknown.0.0:1 8.5 8:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) -- 0x55c469674650 con 0x55c46966a780 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.309+0000 7fecae43b640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55c46966a780 0x55c4696786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.309+0000 7fecad439640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec9006a810 0x7fec9006ccd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55c4695cf150 tx=0x7fec9800c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.309+0000 7fecae43b640 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55c46966a780 0x55c4696786f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecae43b640 1 -- 172.21.15.26:0/2194635735 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v58'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feca0003030 con 0x55c46966a780 2024-04-24T23:24:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55c46966a780 msgr2=0x55c4696786f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecb0139f40 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55c46966a780 0x55c4696786f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec9006a810 msgr2=0x7fec9006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecb0139f40 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec9006a810 0x7fec9006ccd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55c4695cf150 tx=0x7fec9800c040 comp rx=0 tx=0).stop 2024-04-24T23:24:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 msgr2=0x55c4695ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecb0139f40 1 --2- 172.21.15.26:0/2194635735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c469663930 0x55c4695ceb30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7feca8013000 tx=0x7feca8001a40 comp rx=0 tx=0).stop 2024-04-24T23:24:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecae43b640 1 -- 172.21.15.26:0/2194635735 reap_dead start 2024-04-24T23:24:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 shutdown_connections 2024-04-24T23:24:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.310+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 >> 172.21.15.26:0/2194635735 conn(0x55c4695c9b90 msgr2=0x55c469664670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.311+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 shutdown_connections 2024-04-24T23:24:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.311+0000 7fecb0139f40 1 -- 172.21.15.26:0/2194635735 wait complete. 2024-04-24T23:24:39.315 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-04-24T23:24:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.396+0000 7f8e3c80df40 1 Processor -- start 2024-04-24T23:24:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.396+0000 7f8e3c80df40 1 -- start start 2024-04-24T23:24:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e3c80df40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d00f8540 0x5571d0186f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e3c80df40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e3c80df40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571d019d7a0 0x5571d01a0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e3c80df40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5571d006f640 con 0x5571d0113560 2024-04-24T23:24:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e3c80df40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5571d0073680 con 0x5571d019d7a0 2024-04-24T23:24:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e3c80df40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5571d00ca940 con 0x5571d00f8540 2024-04-24T23:24:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e38e7f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e38e7f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39642/0 (socket says 172.21.15.26:39642) 2024-04-24T23:24:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e38e7f640 1 -- 172.21.15.26:0/2103415696 learned_addr learned my addr 172.21.15.26:0/2103415696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e39e81640 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571d019d7a0 0x5571d01a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e39680640 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d00f8540 0x5571d0186f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e38e7f640 1 -- 172.21.15.26:0/2103415696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d00f8540 msgr2=0x5571d0186f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e38e7f640 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d00f8540 0x5571d0186f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e38e7f640 1 -- 172.21.15.26:0/2103415696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571d019d7a0 msgr2=0x5571d01a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e38e7f640 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571d019d7a0 0x5571d01a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e38e7f640 1 -- 172.21.15.26:0/2103415696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d01876f0 con 0x5571d0113560 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e39680640 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d00f8540 0x5571d0186f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.397+0000 7f8e38e7f640 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d019eda0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8e30057040 tx=0x7f8e3004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6cd91e9e97324 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e2b7fe640 1 -- 172.21.15.26:0/2103415696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30059040 con 0x5571d0113560 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e2b7fe640 1 -- 172.21.15.26:0/2103415696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e30064070 con 0x5571d0113560 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e2b7fe640 1 -- 172.21.15.26:0/2103415696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30054bf0 con 0x5571d0113560 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2103415696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 msgr2=0x5571d019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e3c80df40 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d019eda0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8e30057040 tx=0x7f8e3004e000 comp rx=0 tx=0).stop 2024-04-24T23:24:39.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2103415696 shutdown_connections 2024-04-24T23:24:39.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e3c80df40 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571d019d7a0 0x5571d01a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e3c80df40 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d019eda0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e3c80df40 1 --2- 172.21.15.26:0/2103415696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d00f8540 0x5571d0186f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2103415696 >> 172.21.15.26:0/2103415696 conn(0x5571d00c6400 msgr2=0x5571d00c67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2103415696 shutdown_connections 2024-04-24T23:24:39.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.398+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2103415696 wait complete. 2024-04-24T23:24:39.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.399+0000 7f8e3c80df40 1 Processor -- start 2024-04-24T23:24:39.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.399+0000 7f8e3c80df40 1 -- start start 2024-04-24T23:24:39.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.399+0000 7f8e3c80df40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571d00f8540 0x5571d0181930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.399+0000 7f8e3c80df40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d01a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.399+0000 7f8e3c80df40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d019d7a0 0x5571d01a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.399+0000 7f8e3c80df40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5571d01874a0 con 0x5571d0113560 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.399+0000 7f8e3c80df40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5571d0187990 con 0x5571d00f8540 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.399+0000 7f8e3c80df40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5571d019dd20 con 0x5571d019d7a0 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e38e7f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d01a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e38e7f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d01a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39644/0 (socket says 172.21.15.26:39644) 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e38e7f640 1 -- 172.21.15.26:0/2378758100 learned_addr learned my addr 172.21.15.26:0/2378758100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e39680640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571d00f8540 0x5571d0181930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e39e81640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d019d7a0 0x5571d01a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e38e7f640 1 -- 172.21.15.26:0/2378758100 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d019d7a0 msgr2=0x5571d01a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e38e7f640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5571d019d7a0 0x5571d01a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e38e7f640 1 -- 172.21.15.26:0/2378758100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571d00f8540 msgr2=0x5571d0181930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e38e7f640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571d00f8540 0x5571d0181930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e38e7f640 1 -- 172.21.15.26:0/2378758100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d0198590 con 0x5571d0113560 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e38e7f640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d01a0740 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8e30025bc0 tx=0x7f8e30002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e2a7fc640 1 -- 172.21.15.26:0/2378758100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30059040 con 0x5571d0113560 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e2a7fc640 1 -- 172.21.15.26:0/2378758100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e30054c20 con 0x5571d0113560 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5571d00c8b90 con 0x5571d0113560 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e2a7fc640 1 -- 172.21.15.26:0/2378758100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3005f470 con 0x5571d0113560 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.400+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d00c9150 con 0x5571d0113560 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.402+0000 7f8e2a7fc640 1 -- 172.21.15.26:0/2378758100 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e3005f610 con 0x5571d0113560 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.402+0000 7f8e2a7fc640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8e1c06a810 0x7f8e1c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.402+0000 7f8e2a7fc640 1 -- 172.21.15.26:0/2378758100 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8e300cecf0 con 0x5571d0113560 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.402+0000 7f8e39680640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8e1c06a810 0x7f8e1c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.402+0000 7f8e3c80df40 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5571d022a8b0 0x5571d022acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.402+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- osd_op(unknown.0.0:1 8.1d 8:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) -- 0x5571d022b200 con 0x5571d022a8b0 2024-04-24T23:24:39.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.402+0000 7f8e39e81640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5571d022a8b0 0x5571d022acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.403+0000 7f8e39680640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8e1c06a810 0x7f8e1c06ccd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c004290 tx=0x7f8e2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.403+0000 7f8e39e81640 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5571d022a8b0 0x5571d022acc0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e39e81640 1 -- 172.21.15.26:0/2378758100 <== osd.0 v2:172.21.15.26:6819/256768257 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v58'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8e3400e3e0 con 0x5571d022a8b0 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5571d022a8b0 msgr2=0x5571d022acc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5571d022a8b0 0x5571d022acc0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8e1c06a810 msgr2=0x7f8e1c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8e1c06a810 0x7f8e1c06ccd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c004290 tx=0x7f8e2c008040 comp rx=0 tx=0).stop 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 msgr2=0x5571d01a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 --2- 172.21.15.26:0/2378758100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d0113560 0x5571d01a0740 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8e30025bc0 tx=0x7f8e30002fb0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e39e81640 1 -- 172.21.15.26:0/2378758100 reap_dead start 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 shutdown_connections 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 >> 172.21.15.26:0/2378758100 conn(0x5571d00c6400 msgr2=0x5571d00c67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 shutdown_connections 2024-04-24T23:24:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.404+0000 7f8e3c80df40 1 -- 172.21.15.26:0/2378758100 wait complete. 2024-04-24T23:24:39.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:39.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start /d1/d2/d3 recursive,force 2024-04-24T23:24:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.553+0000 7fef9fb09640 1 Processor -- start 2024-04-24T23:24:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.553+0000 7fef9fb09640 1 -- start start 2024-04-24T23:24:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.554+0000 7fef9fb09640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fef9815e710 0x7fef980d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.554+0000 7fef9fb09640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef980d27b0 con 0x7fef9815e330 2024-04-24T23:24:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.554+0000 7fef9fb09640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef980cef00 con 0x7fef980d4630 2024-04-24T23:24:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.554+0000 7fef9fb09640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fef980cf1e0 con 0x7fef9815e710 2024-04-24T23:24:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.554+0000 7fef9d8a5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fef9815e330 0x7fef980d3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55206/0 (socket says 172.21.15.26:55206) 2024-04-24T23:24:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.554+0000 7fef9d8a5640 1 -- 172.21.15.26:0/2835644121 learned_addr learned my addr 172.21.15.26:0/2835644121 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/2835644121 >> v2:172.21.15.29:3300/0 conn(0x7fef9815e710 0x7fef980d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1413260631 0 0) 0x7fef980d27b0 con 0x7fef9815e330 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef80003520 con 0x7fef9815e330 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2415905741 0 0) 0x7fef80003520 con 0x7fef9815e330 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 >> v2:172.21.15.29:3300/0 conn(0x7fef9815e710 msgr2=0x7fef980d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 --2- 172.21.15.26:0/2835644121 >> v2:172.21.15.29:3300/0 conn(0x7fef9815e710 0x7fef980d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 >> v1:172.21.15.29:6789/0 conn(0x7fef980d4630 legacy=0x7fef980b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef980cf4c0 con 0x7fef9815e330 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fef880019f0 con 0x7fef9815e330 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 >> v1:172.21.15.26:6789/0 conn(0x7fef9815e330 legacy=0x7fef980d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fef800039b0 0x7fef80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef800043b0 0x7fef80004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef80005080 0x7fef8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef980cf1e0 con 0x7fef80005080 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef8000bfe0 con 0x7fef800043b0 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fef8000c1e0 con 0x7fef800039b0 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef80005080 0x7fef8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef800043b0 0x7fef80004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fef800039b0 0x7fef80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9e0a6640 1 -- 172.21.15.26:0/2835644121 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fef800039b0 msgr2=0x7fef80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fef800039b0 0x7fef80003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9e0a6640 1 -- 172.21.15.26:0/2835644121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef800043b0 msgr2=0x7fef80004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef800043b0 0x7fef80004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.555+0000 7fef9e0a6640 1 -- 172.21.15.26:0/2835644121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef80003520 con 0x7fef80005080 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.556+0000 7fef9e0a6640 1 -- 172.21.15.26:0/2835644121 reap_dead start 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.556+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef80005080 0x7fef8000b9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fef900018e0 tx=0x7fef80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac862b1106d4e412 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.556+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900014e0 con 0x7fef80005080 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.556+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef90005310 con 0x7fef80005080 2024-04-24T23:24:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.556+0000 7fef9c8a3640 1 -- 172.21.15.26:0/2835644121 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9002ac30 con 0x7fef80005080 2024-04-24T23:24:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.557+0000 7fef9fb09640 1 -- 172.21.15.26:0/2835644121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef80005080 msgr2=0x7fef8000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.557+0000 7fef9fb09640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef80005080 0x7fef8000b9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fef900018e0 tx=0x7fef80005830 comp rx=0 tx=0).stop 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.557+0000 7fef9fb09640 1 -- 172.21.15.26:0/2835644121 shutdown_connections 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.557+0000 7fef9fb09640 1 --2- 172.21.15.26:0/2835644121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef80005080 0x7fef8000b9c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.557+0000 7fef9fb09640 1 -- 172.21.15.26:0/2835644121 >> 172.21.15.26:0/2835644121 conn(0x7fef9808dbc0 msgr2=0x7fef9809d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.557+0000 7fef9fb09640 1 -- 172.21.15.26:0/2835644121 shutdown_connections 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.557+0000 7fef9fb09640 1 -- 172.21.15.26:0/2835644121 wait complete. 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.558+0000 7fef9fb09640 1 Processor -- start 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.558+0000 7fef9fb09640 1 -- start start 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9fb09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef80005080 0x7fef980b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9fb09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef980c92d0 0x7fef98154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9fb09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fef980ca8c0 0x7fef980b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9fb09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef980d2230 con 0x7fef980c92d0 2024-04-24T23:24:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9fb09640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef980d20b0 con 0x7fef80005080 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9fb09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fef980d23b0 con 0x7fef980ca8c0 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9d0a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef980c92d0 0x7fef98154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9d0a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef980c92d0 0x7fef98154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39658/0 (socket says 172.21.15.26:39658) 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9d0a4640 1 -- 172.21.15.26:0/1265147445 learned_addr learned my addr 172.21.15.26:0/1265147445 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef80005080 0x7fef980b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fef980ca8c0 0x7fef980b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9d0a4640 1 -- 172.21.15.26:0/1265147445 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fef980ca8c0 msgr2=0x7fef980b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fef980ca8c0 0x7fef980b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9d0a4640 1 -- 172.21.15.26:0/1265147445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef80005080 msgr2=0x7fef980b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef80005080 0x7fef980b8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.559+0000 7fef9d0a4640 1 -- 172.21.15.26:0/1265147445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef980c9820 con 0x7fef980c92d0 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.560+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef980c92d0 0x7fef98154fe0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fef8c011910 tx=0x7fef8c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.560+0000 7fef967fc640 1 -- 172.21.15.26:0/1265147445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00faf0 con 0x7fef980c92d0 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.560+0000 7fef967fc640 1 -- 172.21.15.26:0/1265147445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c00fc90 con 0x7fef980c92d0 2024-04-24T23:24:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.560+0000 7fef967fc640 1 -- 172.21.15.26:0/1265147445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c014440 con 0x7fef980c92d0 2024-04-24T23:24:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.560+0000 7fef9fb09640 1 -- 172.21.15.26:0/1265147445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef980c9b00 con 0x7fef980c92d0 2024-04-24T23:24:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.560+0000 7fef9fb09640 1 -- 172.21.15.26:0/1265147445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef980ca0c0 con 0x7fef980c92d0 2024-04-24T23:24:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.561+0000 7fef967fc640 1 -- 172.21.15.26:0/1265147445 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef8c012040 con 0x7fef980c92d0 2024-04-24T23:24:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.562+0000 7fef967fc640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef8406be60 0x7fef8406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.562+0000 7fef967fc640 1 -- 172.21.15.26:0/1265147445 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fef8c08b460 con 0x7fef980c92d0 2024-04-24T23:24:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.562+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef8406be60 0x7fef8406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.562+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef8406be60 0x7fef8406e320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fef88002230 tx=0x7fef88006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.563+0000 7fefa2813740 1 Processor -- start 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.563+0000 7fefa2813740 1 -- start start 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.563+0000 7fefa2813740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.563+0000 7fefa2813740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 0x562c13c68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.563+0000 7fefa2813740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 0x562c13c64840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.563+0000 7fefa2813740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562c13ba0490 con 0x562c13c67c60 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.563+0000 7fefa2813740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562c13b48ae0 con 0x562c13c68580 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.563+0000 7fefa2813740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562c13b9c450 con 0x562c13c79680 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.563+0000 7fef9d8a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9d8a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c831f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39674/0 (socket says 172.21.15.26:39674) 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9d8a5640 1 -- 172.21.15.26:0/2528785488 learned_addr learned my addr 172.21.15.26:0/2528785488 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/2528785488 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 0x562c13c68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/2528785488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 0x562c13c64840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9d8a5640 1 -- 172.21.15.26:0/2528785488 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 msgr2=0x562c13c68040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/2528785488 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 0x562c13c68040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9d8a5640 1 -- 172.21.15.26:0/2528785488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 msgr2=0x562c13c64840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/2528785488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 0x562c13c64840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9d8a5640 1 -- 172.21.15.26:0/2528785488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1397ca50 con 0x562c13c67c60 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/2528785488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c831f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fef88015430 tx=0x7fef88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e573de129fdb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef7f7fe640 1 -- 172.21.15.26:0/2528785488 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef880145e0 con 0x562c13c67c60 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef7f7fe640 1 -- 172.21.15.26:0/2528785488 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef88026070 con 0x562c13c67c60 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.564+0000 7fef7f7fe640 1 -- 172.21.15.26:0/2528785488 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88014980 con 0x562c13c67c60 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 -- 172.21.15.26:0/2528785488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 msgr2=0x562c13c831f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 --2- 172.21.15.26:0/2528785488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c831f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fef88015430 tx=0x7fef88014000 comp rx=0 tx=0).stop 2024-04-24T23:24:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 -- 172.21.15.26:0/2528785488 shutdown_connections 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 --2- 172.21.15.26:0/2528785488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 0x562c13c64840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 --2- 172.21.15.26:0/2528785488 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 0x562c13c68040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 --2- 172.21.15.26:0/2528785488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c831f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 -- 172.21.15.26:0/2528785488 >> 172.21.15.26:0/2528785488 conn(0x562c13c045e0 msgr2=0x562c13c82a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 -- 172.21.15.26:0/2528785488 shutdown_connections 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 -- 172.21.15.26:0/2528785488 wait complete. 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.565+0000 7fefa2813740 1 -- start start 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.566+0000 7fefa2813740 1 Processor -- start 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.566+0000 7fefa2813740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c67c60 0x562c13c93640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.566+0000 7fefa2813740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c68580 0x562c13c84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.566+0000 7fefa2813740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c79680 0x562c13c6a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.566+0000 7fefa2813740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562c13c3efa0 con 0x562c13c79680 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.566+0000 7fefa2813740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562c13b5c060 con 0x562c13c67c60 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.566+0000 7fefa2813740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562c13c39820 con 0x562c13c68580 2024-04-24T23:24:39.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.566+0000 7fefa2813740 10 client.0 _ll_register_callbacks cb 0x562c13b97970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9e0a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c79680 0x562c13c6a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9e0a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c79680 0x562c13c6a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39686/0 (socket says 172.21.15.26:39686) 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9e0a6640 1 -- 172.21.15.26:0/2958205449 learned_addr learned my addr 172.21.15.26:0/2958205449 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c67c60 0x562c13c93640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c68580 0x562c13c84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9e0a6640 1 -- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c68580 msgr2=0x562c13c84d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c68580 0x562c13c84d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9e0a6640 1 -- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c67c60 msgr2=0x562c13c93640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c67c60 0x562c13c93640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9e0a6640 1 -- 172.21.15.26:0/2958205449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c138743e0 con 0x562c13c79680 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c79680 0x562c13c6a0e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fef9002a700 tx=0x7fef90001320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b064979ff6ad7660 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef7cff9640 1 -- 172.21.15.26:0/2958205449 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90045060 con 0x562c13c79680 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef7cff9640 1 -- 172.21.15.26:0/2958205449 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fef900052e0 con 0x562c13c79680 2024-04-24T23:24:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fefa2813740 1 -- 172.21.15.26:0/2958205449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef88026070 con 0x562c13c79680 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fef7cff9640 1 -- 172.21.15.26:0/2958205449 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fef9004b050 con 0x562c13c79680 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fefa2813740 10 client.6912 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.567+0000 7fefa2813740 1 -- 172.21.15.26:0/2958205449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x562c139dd1e0 con 0x562c13c79680 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fef7cff9640 1 -- 172.21.15.26:0/2958205449 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef88026070 con 0x562c13c79680 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fef7cff9640 1 -- 172.21.15.26:0/2958205449 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7fef9003e070 con 0x562c13c79680 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fefa2813740 10 client.6912 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fefa2813740 10 client.6912 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fefa2813740 20 client.6912 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fefa2813740 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x562c13c7b800 0x562c13c5ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fefa2813740 4 client.6912 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fefa2813740 1 -- 172.21.15.26:0/2958205449 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c13bad9d0 con 0x562c13c7b800 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x562c13c7b800 0x562c13c5ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fef7cff9640 10 client.6912 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.568+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x562c13c7b800 0x562c13c5ca10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165cc615702e9b4e server_cookie=c63da36758724c9b in_seq=0 out_seq=0 2024-04-24T23:24:39.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fef7cff9640 1 -- 172.21.15.26:0/2958205449 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c13bad9d0 con 0x562c13c7b800 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fef7cff9640 10 client.6912 handle_command_reply: tid=0 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 client.6912 shutdown 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 -- 172.21.15.26:0/2958205449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c79680 msgr2=0x562c13c6a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c79680 0x562c13c6a0e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fef9002a700 tx=0x7fef90001320 comp rx=0 tx=0).stop 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 -- 172.21.15.26:0/2958205449 shutdown_connections 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x562c13c7b800 0x562c13c5ca10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c79680 0x562c13c6a0e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c68580 0x562c13c84d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fef7cff9640 0 client.6912 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 --2- 172.21.15.26:0/2958205449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c67c60 0x562c13c93640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 -- 172.21.15.26:0/2958205449 >> 172.21.15.26:0/2958205449 conn(0x562c13c045e0 msgr2=0x562c13c3b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 -- 172.21.15.26:0/2958205449 shutdown_connections 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 1 -- 172.21.15.26:0/2958205449 wait complete. 2024-04-24T23:24:39.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.569+0000 7fefa2813740 20 client.6912 trim_cache size 0 max 16384 2024-04-24T23:24:39.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.587+0000 7fefa2813740 1 Processor -- start 2024-04-24T23:24:39.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.587+0000 7fefa2813740 1 -- start start 2024-04-24T23:24:39.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fefa2813740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c83e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fefa2813740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 0x562c13c3b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fefa2813740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 0x562c13c3bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fefa2813740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef98200e20 con 0x562c13c67c60 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fefa2813740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef981f7dc0 con 0x562c13c68580 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fefa2813740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fef982006f0 con 0x562c13c79680 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d8a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c83e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d8a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c83e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39702/0 (socket says 172.21.15.26:39702) 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d8a5640 1 -- 172.21.15.26:0/4082186108 learned_addr learned my addr 172.21.15.26:0/4082186108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 0x562c13c3b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 0x562c13c3bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d8a5640 1 -- 172.21.15.26:0/4082186108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 msgr2=0x562c13c3bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 0x562c13c3bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d8a5640 1 -- 172.21.15.26:0/4082186108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 msgr2=0x562c13c3b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 0x562c13c3b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d8a5640 1 -- 172.21.15.26:0/4082186108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1384d4a0 con 0x562c13c67c60 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.588+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 0x562c13c3b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c83e80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fef88023040 tx=0x7fef88016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fed45fba58f576 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fef7effd640 1 -- 172.21.15.26:0/4082186108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88014c20 con 0x562c13c67c60 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fef7effd640 1 -- 172.21.15.26:0/4082186108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef88026040 con 0x562c13c67c60 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fef7effd640 1 -- 172.21.15.26:0/4082186108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88037470 con 0x562c13c67c60 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fefa2813740 1 -- 172.21.15.26:0/4082186108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 msgr2=0x562c13c83e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fefa2813740 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c83e80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fef88023040 tx=0x7fef88016000 comp rx=0 tx=0).stop 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fefa2813740 1 -- 172.21.15.26:0/4082186108 shutdown_connections 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fefa2813740 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c79680 0x562c13c3bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fefa2813740 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c68580 0x562c13c3b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fefa2813740 1 --2- 172.21.15.26:0/4082186108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c67c60 0x562c13c83e80 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fefa2813740 1 -- 172.21.15.26:0/4082186108 >> 172.21.15.26:0/4082186108 conn(0x562c13c045e0 msgr2=0x562c13c6ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fefa2813740 1 -- 172.21.15.26:0/4082186108 shutdown_connections 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.589+0000 7fefa2813740 1 -- 172.21.15.26:0/4082186108 wait complete. 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.590+0000 7fefa2813740 1 -- start start 2024-04-24T23:24:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.590+0000 7fefa2813740 1 Processor -- start 2024-04-24T23:24:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fefa2813740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c67c60 0x562c13cf3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fefa2813740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c68580 0x562c13c82060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fefa2813740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c79680 0x562c13c5de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fefa2813740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562c13c90040 con 0x562c13c68580 2024-04-24T23:24:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fefa2813740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fef981f3940 con 0x562c13c79680 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fefa2813740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562c13c93160 con 0x562c13c67c60 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fefa2813740 10 client.0 _ll_register_callbacks cb 0x7fef80004f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d0a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c68580 0x562c13c82060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d0a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c68580 0x562c13c82060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39706/0 (socket says 172.21.15.26:39706) 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d0a4640 1 -- 172.21.15.26:0/3504960475 learned_addr learned my addr 172.21.15.26:0/3504960475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c67c60 0x562c13cf3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9e0a6640 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c79680 0x562c13c5de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d0a4640 1 -- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c67c60 msgr2=0x562c13cf3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c67c60 0x562c13cf3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d0a4640 1 -- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c79680 msgr2=0x562c13c5de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c79680 0x562c13c5de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d0a4640 1 -- 172.21.15.26:0/3504960475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c13c3d840 con 0x562c13c68580 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef9d0a4640 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c68580 0x562c13c82060 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fef8c059730 tx=0x7fef8c059f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749b74e7c1c442eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.591+0000 7fef7cff9640 1 -- 172.21.15.26:0/3504960475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c08b2b0 con 0x562c13c68580 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fef7cff9640 1 -- 172.21.15.26:0/3504960475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fef8c012330 con 0x562c13c68580 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fefa2813740 1 -- 172.21.15.26:0/3504960475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef88026040 con 0x562c13c68580 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fef7cff9640 1 -- 172.21.15.26:0/3504960475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fef8c012550 con 0x562c13c68580 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fefa2813740 10 client.6918 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fefa2813740 1 -- 172.21.15.26:0/3504960475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x562c13ca7b30 con 0x562c13c68580 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fef7cff9640 1 -- 172.21.15.26:0/3504960475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef88026040 con 0x562c13c68580 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fef7cff9640 1 -- 172.21.15.26:0/3504960475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7fef8c05ece0 con 0x562c13c68580 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fefa2813740 10 client.6918 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fefa2813740 10 client.6918 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fefa2813740 20 client.6918 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fefa2813740 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x562c13c78bf0 0x562c13c5e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fefa2813740 4 client.6918 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fefa2813740 1 -- 172.21.15.26:0/3504960475 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x562c13ba7d20 con 0x562c13c78bf0 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.592+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x562c13c78bf0 0x562c13c5e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fef7cff9640 10 client.6918 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fef9d8a5640 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x562c13c78bf0 0x562c13c5e3a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc0140457c56f34 server_cookie=5c0ac8ed9ff143cf in_seq=0 out_seq=0 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fef7cff9640 1 -- 172.21.15.26:0/3504960475 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c13ba7d20 con 0x562c13c78bf0 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fef7cff9640 10 client.6918 handle_command_reply: tid=0 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fefa2813740 1 client.6918 shutdown 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fefa2813740 1 -- 172.21.15.26:0/3504960475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c68580 msgr2=0x562c13c82060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fefa2813740 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c68580 0x562c13c82060 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fef8c059730 tx=0x7fef8c059f60 comp rx=0 tx=0).stop 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fefa2813740 1 -- 172.21.15.26:0/3504960475 shutdown_connections 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fefa2813740 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x562c13c78bf0 0x562c13c5e3a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fefa2813740 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562c13c79680 0x562c13c5de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fefa2813740 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c13c68580 0x562c13c82060 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fef7cff9640 0 client.6918 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fefa2813740 1 --2- 172.21.15.26:0/3504960475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562c13c67c60 0x562c13cf3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.593+0000 7fefa2813740 1 -- 172.21.15.26:0/3504960475 >> 172.21.15.26:0/3504960475 conn(0x562c13c045e0 msgr2=0x562c13c84ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fefa2813740 1 -- 172.21.15.26:0/3504960475 shutdown_connections 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fefa2813740 1 -- 172.21.15.26:0/3504960475 wait complete. 2024-04-24T23:24:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fefa2813740 20 client.6918 trim_cache size 0 max 16384 2024-04-24T23:24:39.601 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:24:39.594+0000 7f6efcc97640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1) n(v2 rc2024-04-24T23:24:26.649950+0000 4=0+4) (inest lock) (ifile excl) caps={5609=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5609 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55e807c71700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.649950+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-04-24T23:24:26.649950+0000 4=0+4)","error_str":""},"return_code":-61} 2024-04-24T23:24:39.601 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:39.601 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-04-24T23:24:39.601 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "16c6d4aa-6336-44ae-b490-7190c570fe18", 2024-04-24T23:24:39.601 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-04-24T23:24:39.601 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:39.602 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fede1dba640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-04-24T23:24:26.421954+0000 1=0+1) n(v0 rc2024-04-24T23:24:26.421954+0000 2=0+2) (inest lock) (ifile excl) caps={5609=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5609 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55af1122a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.421954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.421954+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.421954+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-04-24T23:24:26.421954+0000 2=0+2)","error_str":""},"return_code":-61} 2024-04-24T23:24:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fef9fb09640 1 -- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef8406be60 msgr2=0x7fef8406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fef9fb09640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef8406be60 0x7fef8406e320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fef88002230 tx=0x7fef88006040 comp rx=0 tx=0).stop 2024-04-24T23:24:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fef9fb09640 1 -- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef980c92d0 msgr2=0x7fef98154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fef9fb09640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef980c92d0 0x7fef98154fe0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fef8c011910 tx=0x7fef8c011dd0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fef9fb09640 1 -- 172.21.15.26:0/1265147445 shutdown_connections 2024-04-24T23:24:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.594+0000 7fef9fb09640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef8406be60 0x7fef8406e320 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.595+0000 7fef9fb09640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fef980ca8c0 0x7fef980b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.595+0000 7fef9fb09640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef980c92d0 0x7fef98154fe0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.595+0000 7fef9fb09640 1 --2- 172.21.15.26:0/1265147445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fef80005080 0x7fef980b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.595+0000 7fef9fb09640 1 -- 172.21.15.26:0/1265147445 >> 172.21.15.26:0/1265147445 conn(0x7fef9808dbc0 msgr2=0x7fef980b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.603 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-04-24T23:24:39.595+0000 7fede1dba640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5609=pAsLsXsFsx/AsLsXsFsx@3},l=5609 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55af11fca000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-04-24T23:24:39.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.595+0000 7fef9fb09640 1 -- 172.21.15.26:0/1265147445 shutdown_connections 2024-04-24T23:24:39.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.595+0000 7fef9fb09640 1 -- 172.21.15.26:0/1265147445 wait complete. 2024-04-24T23:24:39.604 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:24:39.595+0000 7f6f034a4640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1) n(v1 rc2024-04-24T23:24:26.649950+0000 3=0+3) (inest sync->mix g=0,2) caps={5609=pAsLsXsFs/pAsLsXsFsxcral@3} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 export_pin=2 0x55e806d14b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.421954+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-04-24T23:24:26.649950+0000 3=0+3)","error_str":""},"return_code":-61} 2024-04-24T23:24:39.604 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:24:39.594+0000 7f75a7611640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v18 ap=1 f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1) n(v3 rc2024-04-24T23:24:26.649950+0000 6=0+6)/n(v2 rc2024-04-24T23:24:26.649950+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5609=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5609 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=1 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x558b81427600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000002:[<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.649950+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-04-24T23:24:26.649950+0000 6=0+6)","error_str":""},"return_code":-61} 2024-04-24T23:24:39.605 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:24:39.595+0000 7f75ade1e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v18 ap=2 f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1) n(v4 rc2024-04-24T23:24:26.649950+0000 5=0+5)/n(v3 rc2024-04-24T23:24:26.649950+0000 5=0+5) (inest sync->mix g=1,2) caps={5609=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=1 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x558b81427b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000003:[<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.649950+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-04-24T23:24:26.649950+0000 5=0+5)","error_str":""},"return_code":-61} 2024-04-24T23:24:39.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:24:39.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-24T23:24:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.709+0000 7f9919963640 1 Processor -- start 2024-04-24T23:24:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.709+0000 7f9919963640 1 -- start start 2024-04-24T23:24:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9919963640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f991415c270 0x7f99140cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9919963640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99140cf050 con 0x7f991415c650 2024-04-24T23:24:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9919963640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99140cdd60 con 0x7f99140d0ab0 2024-04-24T23:24:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9919963640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f99140cee00 con 0x7f991415c270 2024-04-24T23:24:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9912ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f991415c270 0x7f99140cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9912ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f991415c270 0x7f99140cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39722/0 (socket says 172.21.15.26:39722) 2024-04-24T23:24:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9912ffd640 1 -- 172.21.15.26:0/1256878508 learned_addr learned my addr 172.21.15.26:0/1256878508 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f99127fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f991415c650 0x7f99140d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55218/0 (socket says 172.21.15.26:55218) 2024-04-24T23:24:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9912ffd640 1 -- 172.21.15.26:0/1256878508 >> v1:172.21.15.29:6789/0 conn(0x7f99140d0ab0 legacy=0x7f99140c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9912ffd640 1 -- 172.21.15.26:0/1256878508 >> v1:172.21.15.26:6789/0 conn(0x7f991415c650 legacy=0x7f99140d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9912ffd640 1 -- 172.21.15.26:0/1256878508 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99140d2b30 con 0x7f991415c270 2024-04-24T23:24:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.710+0000 7f9911ffb640 1 -- 172.21.15.26:0/1256878508 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2013831469 0 0) 0x7f99140cf050 con 0x7f991415c650 2024-04-24T23:24:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.711+0000 7f9912ffd640 1 --2- 172.21.15.26:0/1256878508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c270 0x7f99140cfd50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9904009870 tx=0x7f990402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f06d164751f5607 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.711+0000 7f9911ffb640 1 -- 172.21.15.26:0/1256878508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904037070 con 0x7f991415c270 2024-04-24T23:24:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.711+0000 7f9911ffb640 1 -- 172.21.15.26:0/1256878508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904032440 con 0x7f991415c270 2024-04-24T23:24:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.711+0000 7f9911ffb640 1 -- 172.21.15.26:0/1256878508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904032810 con 0x7f991415c270 2024-04-24T23:24:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.712+0000 7f9919963640 1 -- 172.21.15.26:0/1256878508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c270 msgr2=0x7f99140cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.712+0000 7f9919963640 1 --2- 172.21.15.26:0/1256878508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c270 0x7f99140cfd50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9904009870 tx=0x7f990402a270 comp rx=0 tx=0).stop 2024-04-24T23:24:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.712+0000 7f9919963640 1 -- 172.21.15.26:0/1256878508 shutdown_connections 2024-04-24T23:24:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.712+0000 7f9919963640 1 --2- 172.21.15.26:0/1256878508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c270 0x7f99140cfd50 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.712+0000 7f9919963640 1 -- 172.21.15.26:0/1256878508 >> 172.21.15.26:0/1256878508 conn(0x7f99140d1dc0 msgr2=0x7f99140d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.712+0000 7f9919963640 1 -- 172.21.15.26:0/1256878508 shutdown_connections 2024-04-24T23:24:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.712+0000 7f9919963640 1 -- 172.21.15.26:0/1256878508 wait complete. 2024-04-24T23:24:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.713+0000 7f9919963640 1 Processor -- start 2024-04-24T23:24:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.713+0000 7f9919963640 1 -- start start 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f9919963640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99140d0ab0 0x7f99141549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f9919963640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f991415c270 0x7f991406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f9919963640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c650 0x7f991406dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f9919963640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f99140b2950 con 0x7f991415c650 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f9919963640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f99140d28e0 con 0x7f99140d0ab0 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f9919963640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f99140b2ad0 con 0x7f991415c270 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c650 0x7f991406dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c650 0x7f991406dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39726/0 (socket says 172.21.15.26:39726) 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99137fe640 1 -- 172.21.15.26:0/2452891602 learned_addr learned my addr 172.21.15.26:0/2452891602 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f9912ffd640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99140d0ab0 0x7f99141549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99127fc640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f991415c270 0x7f991406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99137fe640 1 -- 172.21.15.26:0/2452891602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f991415c270 msgr2=0x7f991406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99137fe640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f991415c270 0x7f991406d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99137fe640 1 -- 172.21.15.26:0/2452891602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99140d0ab0 msgr2=0x7f99141549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99137fe640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99140d0ab0 0x7f99141549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99137fe640 1 -- 172.21.15.26:0/2452891602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991406c3d0 con 0x7f991415c650 2024-04-24T23:24:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99127fc640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f991415c270 0x7f991406d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.714+0000 7f99137fe640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c650 0x7f991406dde0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f990c00c760 tx=0x7f990c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.715+0000 7f98fbfff640 1 -- 172.21.15.26:0/2452891602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c003b70 con 0x7f991415c650 2024-04-24T23:24:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.715+0000 7f98fbfff640 1 -- 172.21.15.26:0/2452891602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c003d10 con 0x7f991415c650 2024-04-24T23:24:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.715+0000 7f9919963640 1 -- 172.21.15.26:0/2452891602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991406c6b0 con 0x7f991415c650 2024-04-24T23:24:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.715+0000 7f98fbfff640 1 -- 172.21.15.26:0/2452891602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c00aa90 con 0x7f991415c650 2024-04-24T23:24:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.715+0000 7f9919963640 1 -- 172.21.15.26:0/2452891602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991406cc70 con 0x7f991415c650 2024-04-24T23:24:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.716+0000 7f98fbfff640 1 -- 172.21.15.26:0/2452891602 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f990c00ac30 con 0x7f991415c650 2024-04-24T23:24:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.717+0000 7f98fbfff640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f990006be30 0x7f990006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.717+0000 7f9912ffd640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f990006be30 0x7f990006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.717+0000 7f98fbfff640 1 -- 172.21.15.26:0/2452891602 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f990c08ac40 con 0x7f991415c650 2024-04-24T23:24:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.717+0000 7f9912ffd640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f990006be30 0x7f990006e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9904034000 tx=0x7f990402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.718+0000 7f991c678740 1 Processor -- start 2024-04-24T23:24:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.718+0000 7f991c678740 1 -- start start 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f991c678740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f991c678740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f991c678740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17a4a40 0x556ca17eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f991c678740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556ca17086b0 con 0x556ca17ced60 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f991c678740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ca16b0880 con 0x556ca17ec600 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f991c678740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556ca1704670 con 0x556ca17a4a40 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f9912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f9912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39740/0 (socket says 172.21.15.26:39740) 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f9912ffd640 1 -- 172.21.15.26:0/252115563 learned_addr learned my addr 172.21.15.26:0/252115563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f99127fc640 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f99137fe640 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17a4a40 0x556ca17eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f9912ffd640 1 -- 172.21.15.26:0/252115563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17a4a40 msgr2=0x556ca17eacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f9912ffd640 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17a4a40 0x556ca17eacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f9912ffd640 1 -- 172.21.15.26:0/252115563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 msgr2=0x556ca17ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f9912ffd640 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17ecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f9912ffd640 1 -- 172.21.15.26:0/252115563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca14e67d0 con 0x556ca17ced60 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f99127fc640 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f9912ffd640 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17cf140 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9904035040 tx=0x7f9904008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ccc00bbd77bba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f98f97fa640 1 -- 172.21.15.26:0/252115563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990404e070 con 0x556ca17ced60 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f98f97fa640 1 -- 172.21.15.26:0/252115563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904032d70 con 0x556ca17ced60 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.719+0000 7f98f97fa640 1 -- 172.21.15.26:0/252115563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904037040 con 0x556ca17ced60 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.720+0000 7f991c678740 1 -- 172.21.15.26:0/252115563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 msgr2=0x556ca17cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.720+0000 7f991c678740 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17cf140 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9904035040 tx=0x7f9904008030 comp rx=0 tx=0).stop 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.720+0000 7f991c678740 1 -- 172.21.15.26:0/252115563 shutdown_connections 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.720+0000 7f991c678740 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17a4a40 0x556ca17eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.720+0000 7f991c678740 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.720+0000 7f991c678740 1 --2- 172.21.15.26:0/252115563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17cf140 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.720+0000 7f991c678740 1 -- 172.21.15.26:0/252115563 >> 172.21.15.26:0/252115563 conn(0x556ca17a1600 msgr2=0x556ca17e9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.720+0000 7f991c678740 1 -- 172.21.15.26:0/252115563 shutdown_connections 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.720+0000 7f991c678740 1 -- 172.21.15.26:0/252115563 wait complete. 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.721+0000 7f991c678740 1 -- start start 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.721+0000 7f991c678740 1 Processor -- start 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f991c678740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f991c678740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ec600 0x556ca17fdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f991c678740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca179e260 0x556ca179e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f991c678740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556ca17a57e0 con 0x556ca17ced60 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f991c678740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ca16c3f50 con 0x556ca179e260 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f991c678740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556ca179f530 con 0x556ca17ec600 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f991c678740 10 client.0 _ll_register_callbacks cb 0x556ca16ff000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f9912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f9912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39744/0 (socket says 172.21.15.26:39744) 2024-04-24T23:24:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f9912ffd640 1 -- 172.21.15.26:0/833498566 learned_addr learned my addr 172.21.15.26:0/833498566 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f9912ffd640 1 -- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ec600 msgr2=0x556ca17fdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f9912ffd640 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ec600 0x556ca17fdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f9912ffd640 1 -- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca179e260 msgr2=0x556ca179e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f9912ffd640 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca179e260 0x556ca179e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f9912ffd640 1 -- 172.21.15.26:0/833498566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca141c3f0 con 0x556ca17ced60 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f9912ffd640 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17ca610 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f990402a2b0 tx=0x7f9904004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d60d3b8aa565545 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f98f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f98f27fc640 1 -- 172.21.15.26:0/833498566 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904057060 con 0x556ca17ced60 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f98f27fc640 1 -- 172.21.15.26:0/833498566 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9904003610 con 0x556ca17ced60 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f991c678740 1 -- 172.21.15.26:0/833498566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9904032d70 con 0x556ca17ced60 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.722+0000 7f98f27fc640 1 -- 172.21.15.26:0/833498566 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9904037030 con 0x556ca17ced60 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f991c678740 10 client.6933 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f991c678740 1 -- 172.21.15.26:0/833498566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x556ca13dc0f0 con 0x556ca17ced60 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f98f27fc640 1 -- 172.21.15.26:0/833498566 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904032d70 con 0x556ca17ced60 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f98f27fc640 1 -- 172.21.15.26:0/833498566 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7f9904003810 con 0x556ca17ced60 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f991c678740 10 client.6933 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f991c678740 10 client.6933 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f991c678740 20 client.6933 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f991c678740 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x556ca17d01d0 0x556ca17d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f991c678740 4 client.6933 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f991c678740 1 -- 172.21.15.26:0/833498566 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ca1715bf0 con 0x556ca17d01d0 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f99127fc640 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x556ca17d01d0 0x556ca17d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f99127fc640 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x556ca17d01d0 0x556ca17d0640 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232d61b5e41ec658 server_cookie=ad41bdd539279d7a in_seq=0 out_seq=0 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.723+0000 7f98f27fc640 10 client.6933 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f98f27fc640 1 -- 172.21.15.26:0/833498566 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ca1715bf0 con 0x556ca17d01d0 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f98f27fc640 10 client.6933 handle_command_reply: tid=0 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 client.6933 shutdown 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 -- 172.21.15.26:0/833498566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 msgr2=0x556ca17ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17ca610 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f990402a2b0 tx=0x7f9904004730 comp rx=0 tx=0).stop 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 -- 172.21.15.26:0/833498566 shutdown_connections 2024-04-24T23:24:39.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x556ca17d01d0 0x556ca17d0640 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca179e260 0x556ca179e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ec600 0x556ca17fdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 --2- 172.21.15.26:0/833498566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca17ced60 0x556ca17ca610 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f98f27fc640 0 client.6933 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 -- 172.21.15.26:0/833498566 >> 172.21.15.26:0/833498566 conn(0x556ca17a4a40 msgr2=0x556ca17ed0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 -- 172.21.15.26:0/833498566 shutdown_connections 2024-04-24T23:24:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 1 -- 172.21.15.26:0/833498566 wait complete. 2024-04-24T23:24:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.724+0000 7f991c678740 20 client.6933 trim_cache size 0 max 16384 2024-04-24T23:24:39.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.742+0000 7f991c678740 1 Processor -- start 2024-04-24T23:24:39.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.742+0000 7f991c678740 1 -- start start 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f991c678740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f991c678740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ced60 0x556ca17e9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f991c678740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ec600 0x556ca17ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f991c678740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9914200800 con 0x556ca179e260 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f991c678740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f99141ffa00 con 0x556ca17ced60 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f991c678740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f99142000d0 con 0x556ca17ec600 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f9912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f9912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17a52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39754/0 (socket says 172.21.15.26:39754) 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f9912ffd640 1 -- 172.21.15.26:0/3626367549 learned_addr learned my addr 172.21.15.26:0/3626367549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f9912ffd640 1 -- 172.21.15.26:0/3626367549 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ec600 msgr2=0x556ca17ea1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f9912ffd640 1 --2- 172.21.15.26:0/3626367549 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ec600 0x556ca17ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f9912ffd640 1 -- 172.21.15.26:0/3626367549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ced60 msgr2=0x556ca17e9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f9912ffd640 1 --2- 172.21.15.26:0/3626367549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ced60 0x556ca17e9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f9912ffd640 1 -- 172.21.15.26:0/3626367549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca17cda00 con 0x556ca179e260 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.743+0000 7f9912ffd640 1 --2- 172.21.15.26:0/3626367549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17a52d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9904003d40 tx=0x7f990400b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea47b1df68c28b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f98f8ff9640 1 -- 172.21.15.26:0/3626367549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990404e040 con 0x556ca179e260 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f98f8ff9640 1 -- 172.21.15.26:0/3626367549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904032d70 con 0x556ca179e260 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f98f8ff9640 1 -- 172.21.15.26:0/3626367549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904003810 con 0x556ca179e260 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f991c678740 1 -- 172.21.15.26:0/3626367549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 msgr2=0x556ca17a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f991c678740 1 --2- 172.21.15.26:0/3626367549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17a52d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9904003d40 tx=0x7f990400b050 comp rx=0 tx=0).stop 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f991c678740 1 -- 172.21.15.26:0/3626367549 shutdown_connections 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f991c678740 1 --2- 172.21.15.26:0/3626367549 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ec600 0x556ca17ea1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f991c678740 1 --2- 172.21.15.26:0/3626367549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ced60 0x556ca17e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f991c678740 1 --2- 172.21.15.26:0/3626367549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17a52d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f991c678740 1 -- 172.21.15.26:0/3626367549 >> 172.21.15.26:0/3626367549 conn(0x556ca17a4a40 msgr2=0x556ca17ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f991c678740 1 -- 172.21.15.26:0/3626367549 shutdown_connections 2024-04-24T23:24:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.744+0000 7f991c678740 1 -- 172.21.15.26:0/3626367549 wait complete. 2024-04-24T23:24:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.745+0000 7f991c678740 1 -- start start 2024-04-24T23:24:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.745+0000 7f991c678740 1 Processor -- start 2024-04-24T23:24:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f991c678740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f991c678740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ced60 0x556ca18626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f991c678740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17df500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f991c678740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f99140b5590 con 0x556ca179e260 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f991c678740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556ca17dfde0 con 0x556ca17ec600 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f991c678740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556ca17ece00 con 0x556ca17ced60 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f991c678740 10 client.0 _ll_register_callbacks cb 0x7f990404c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f9912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f9912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17e89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39764/0 (socket says 172.21.15.26:39764) 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f9912ffd640 1 -- 172.21.15.26:0/4231660971 learned_addr learned my addr 172.21.15.26:0/4231660971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f99137fe640 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17df500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f99127fc640 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ced60 0x556ca18626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f9912ffd640 1 -- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ced60 msgr2=0x556ca18626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f9912ffd640 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ced60 0x556ca18626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f9912ffd640 1 -- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 msgr2=0x556ca17df500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f9912ffd640 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17df500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f9912ffd640 1 -- 172.21.15.26:0/4231660971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca1862c00 con 0x556ca179e260 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f99137fe640 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17df500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f9912ffd640 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17e89d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9904003c40 tx=0x7f990403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402a379ca6aa7127 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f98f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f98f27fc640 1 -- 172.21.15.26:0/4231660971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990404e050 con 0x556ca179e260 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f98f27fc640 1 -- 172.21.15.26:0/4231660971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9904003810 con 0x556ca179e260 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.746+0000 7f991c678740 1 -- 172.21.15.26:0/4231660971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9904032d70 con 0x556ca179e260 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f98f27fc640 1 -- 172.21.15.26:0/4231660971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f990406f050 con 0x556ca179e260 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f991c678740 10 client.6939 fetch_fsmap learned FSMap version 55 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f991c678740 1 -- 172.21.15.26:0/4231660971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x556ca17d0930 con 0x556ca179e260 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f98f27fc640 1 -- 172.21.15.26:0/4231660971 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904032d70 con 0x556ca179e260 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f98f27fc640 1 -- 172.21.15.26:0/4231660971 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 3542+0+0 (secure 0 0 0) 0x7f9904072020 con 0x556ca179e260 2024-04-24T23:24:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f991c678740 10 client.6939 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f991c678740 10 client.6939 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f991c678740 20 client.6939 populate_metadata read hostname 'smithi026' 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f991c678740 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x556ca17a4060 0x556ca17a4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f991c678740 4 client.6939 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f991c678740 1 -- 172.21.15.26:0/4231660971 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556ca170ff40 con 0x556ca17a4060 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.747+0000 7f99127fc640 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x556ca17a4060 0x556ca17a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f99127fc640 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x556ca17a4060 0x556ca17a4440 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a48f3174d96c64 server_cookie=a150a54365e3c106 in_seq=0 out_seq=0 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f98f27fc640 10 client.6939 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f98f27fc640 1 -- 172.21.15.26:0/4231660971 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x556ca170ff40 con 0x556ca17a4060 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f98f27fc640 10 client.6939 handle_command_reply: tid=0 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 client.6939 shutdown 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 -- 172.21.15.26:0/4231660971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 msgr2=0x556ca17e89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17e89d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9904003c40 tx=0x7f990403e040 comp rx=0 tx=0).stop 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 -- 172.21.15.26:0/4231660971 shutdown_connections 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x556ca17a4060 0x556ca17a4440 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ca17ec600 0x556ca17df500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f98f27fc640 0 client.6939 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556ca17ced60 0x556ca18626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 --2- 172.21.15.26:0/4231660971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ca179e260 0x556ca17e89d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 -- 172.21.15.26:0/4231660971 >> 172.21.15.26:0/4231660971 conn(0x556ca17a4a40 msgr2=0x556ca17a52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 -- 172.21.15.26:0/4231660971 shutdown_connections 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 1 -- 172.21.15.26:0/4231660971 wait complete. 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f991c678740 20 client.6939 trim_cache size 0 max 16384 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:24:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:24:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:24:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "16c6d4aa-6336-44ae-b490-7190c570fe18": { 2024-04-24T23:24:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:24:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "16c6d4aa-6336-44ae-b490-7190c570fe18", 2024-04-24T23:24:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:24:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:24:39.756 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f9919963640 1 -- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f990006be30 msgr2=0x7f990006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f9919963640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f990006be30 0x7f990006e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9904034000 tx=0x7f990402e000 comp rx=0 tx=0).stop 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f9919963640 1 -- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c650 msgr2=0x7f991406dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.748+0000 7f9919963640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c650 0x7f991406dde0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f990c00c760 tx=0x7f990c00cc20 comp rx=0 tx=0).stop 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.749+0000 7f9919963640 1 -- 172.21.15.26:0/2452891602 shutdown_connections 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.749+0000 7f9919963640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f990006be30 0x7f990006e2f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.749+0000 7f9919963640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f991415c650 0x7f991406dde0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.749+0000 7f9919963640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f991415c270 0x7f991406d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.749+0000 7f9919963640 1 --2- 172.21.15.26:0/2452891602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99140d0ab0 0x7f99141549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.749+0000 7f9919963640 1 -- 172.21.15.26:0/2452891602 >> 172.21.15.26:0/2452891602 conn(0x7f99140d1dc0 msgr2=0x7f99140c58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.749+0000 7f9919963640 1 -- 172.21.15.26:0/2452891602 shutdown_connections 2024-04-24T23:24:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:24:39.749+0000 7f9919963640 1 -- 172.21.15.26:0/2452891602 wait complete. 2024-04-24T23:24:39.760 INFO:tasks.cephfs.filesystem:scrub status for tag:16c6d4aa-6336-44ae-b490-7190c570fe18 - {'path': '/d1/d2/d3', 'tag': '16c6d4aa-6336-44ae-b490-7190c570fe18', 'options': 'recursive,force'} 2024-04-24T23:24:40.075 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:24:40.074+0000 7f26a3749640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T23:24:45.855 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:24:45.854+0000 7f26a3749640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T23:24:59.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:24:59.508 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:25:09.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:09.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-24T23:25:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.877+0000 7f917888c640 1 Processor -- start 2024-04-24T23:25:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.877+0000 7f917888c640 1 -- start start 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.878+0000 7f917888c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9174145b30 0x7f9174071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.878+0000 7f917888c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f917406b120 0x7f917406b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.878+0000 7f917888c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91740ae180 con 0x7f91740add40 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.878+0000 7f917888c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f91740cec20 con 0x7f917406b120 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.878+0000 7f917888c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9174072520 con 0x7f9174145b30 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.878+0000 7f9172d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f917406b120 0x7f917406b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.878+0000 7f9172d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f917406b120 0x7f917406b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46298/0 (socket says 172.21.15.26:46298) 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.878+0000 7f9172d9d640 1 -- 172.21.15.26:0/1695159747 learned_addr learned my addr 172.21.15.26:0/1695159747 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.878+0000 7f917259c640 1 --2- 172.21.15.26:0/1695159747 >> v2:172.21.15.29:3300/0 conn(0x7f9174145b30 0x7f9174071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.879+0000 7f9172d9d640 1 -- 172.21.15.26:0/1695159747 >> v2:172.21.15.29:3300/0 conn(0x7f9174145b30 msgr2=0x7f9174071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.879+0000 7f9172d9d640 1 --2- 172.21.15.26:0/1695159747 >> v2:172.21.15.29:3300/0 conn(0x7f9174145b30 0x7f9174071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.879+0000 7f9172d9d640 1 -- 172.21.15.26:0/1695159747 >> v1:172.21.15.29:6789/0 conn(0x7f91740add40 legacy=0x7f91740d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.879+0000 7f9172d9d640 1 -- 172.21.15.26:0/1695159747 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9174072800 con 0x7f917406b120 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.879+0000 7f9172d9d640 1 --2- 172.21.15.26:0/1695159747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f917406b120 0x7f917406b5d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f917404fae0 tx=0x7f916402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e55dfcd565f72b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.879+0000 7f917159a640 1 -- 172.21.15.26:0/1695159747 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164036070 con 0x7f917406b120 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917159a640 1 -- 172.21.15.26:0/1695159747 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9164031600 con 0x7f917406b120 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917159a640 1 -- 172.21.15.26:0/1695159747 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91640319d0 con 0x7f917406b120 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917888c640 1 -- 172.21.15.26:0/1695159747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f917406b120 msgr2=0x7f917406b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917888c640 1 --2- 172.21.15.26:0/1695159747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f917406b120 0x7f917406b5d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f917404fae0 tx=0x7f916402a730 comp rx=0 tx=0).stop 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917888c640 1 -- 172.21.15.26:0/1695159747 shutdown_connections 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917888c640 1 --2- 172.21.15.26:0/1695159747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f917406b120 0x7f917406b5d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917888c640 1 --2- 172.21.15.26:0/1695159747 >> v2:172.21.15.29:3300/0 conn(0x7f9174145b30 0x7f9174071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917888c640 1 -- 172.21.15.26:0/1695159747 >> 172.21.15.26:0/1695159747 conn(0x7f91740d0d20 msgr2=0x7f91740d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917888c640 1 -- 172.21.15.26:0/1695159747 shutdown_connections 2024-04-24T23:25:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.880+0000 7f917888c640 1 -- 172.21.15.26:0/1695159747 wait complete. 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.882+0000 7f917888c640 1 Processor -- start 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.882+0000 7f917888c640 1 -- start start 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.882+0000 7f917888c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f917406b120 0x7f917409e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.882+0000 7f917888c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91740add40 0x7f9174141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.882+0000 7f917888c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174145b30 0x7f9174141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.882+0000 7f917888c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f91740cf0b0 con 0x7f91740add40 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.882+0000 7f917888c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9174072dc0 con 0x7f9174145b30 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.882+0000 7f917888c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f91740cf230 con 0x7f917406b120 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.882+0000 7f9171d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91740add40 0x7f9174141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f9171d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91740add40 0x7f9174141310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46300/0 (socket says 172.21.15.26:46300) 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f9171d9b640 1 -- 172.21.15.26:0/4254415988 learned_addr learned my addr 172.21.15.26:0/4254415988 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f917259c640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f917406b120 0x7f917409e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f9172d9d640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174145b30 0x7f9174141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f9171d9b640 1 -- 172.21.15.26:0/4254415988 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f917406b120 msgr2=0x7f917409e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f9171d9b640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f917406b120 0x7f917409e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f9171d9b640 1 -- 172.21.15.26:0/4254415988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174145b30 msgr2=0x7f9174141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f9171d9b640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174145b30 0x7f9174141850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f9171d9b640 1 -- 172.21.15.26:0/4254415988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91740b5fe0 con 0x7f91740add40 2024-04-24T23:25:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f9171d9b640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91740add40 0x7f9174141310 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c440 tx=0x7f915c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f916affd640 1 -- 172.21.15.26:0/4254415988 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c0052d0 con 0x7f91740add40 2024-04-24T23:25:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f916affd640 1 -- 172.21.15.26:0/4254415988 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915c005470 con 0x7f91740add40 2024-04-24T23:25:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f917888c640 1 -- 172.21.15.26:0/4254415988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917409b970 con 0x7f91740add40 2024-04-24T23:25:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f916affd640 1 -- 172.21.15.26:0/4254415988 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c007260 con 0x7f91740add40 2024-04-24T23:25:09.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.883+0000 7f917888c640 1 -- 172.21.15.26:0/4254415988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f917409bf30 con 0x7f91740add40 2024-04-24T23:25:09.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.885+0000 7f916affd640 1 -- 172.21.15.26:0/4254415988 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f915c007400 con 0x7f91740add40 2024-04-24T23:25:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.886+0000 7f916affd640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f915406be30 0x7f915406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.886+0000 7f916affd640 1 -- 172.21.15.26:0/4254415988 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f915c08a2c0 con 0x7f91740add40 2024-04-24T23:25:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.886+0000 7f917259c640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f915406be30 0x7f915406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.886+0000 7f917259c640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f915406be30 0x7f915406e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f916c009aa0 tx=0x7f916c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:09.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.887+0000 7f917b59a740 1 Processor -- start 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.887+0000 7f917b59a740 1 -- start start 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917b59a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86838310 0x55db86810970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917b59a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db8683ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917b59a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8680e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917b59a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55db867746b0 con 0x55db86838310 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917b59a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55db8671c880 con 0x55db8683b0b0 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917b59a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55db86770670 con 0x55db868386f0 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86838310 0x55db86810970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86838310 0x55db86810970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46312/0 (socket says 172.21.15.26:46312) 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917259c640 1 -- 172.21.15.26:0/1534375310 learned_addr learned my addr 172.21.15.26:0/1534375310 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f9172d9d640 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8680e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f9171d9b640 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db8683ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917259c640 1 -- 172.21.15.26:0/1534375310 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 msgr2=0x55db8683ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917259c640 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db8683ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917259c640 1 -- 172.21.15.26:0/1534375310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 msgr2=0x55db8680e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917259c640 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8680e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917259c640 1 -- 172.21.15.26:0/1534375310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db865527d0 con 0x55db86838310 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f9171d9b640 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db8683ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f917259c640 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86838310 0x55db86810970 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f916c004470 tx=0x7f916c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3602d8d311c96df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f9153fff640 1 -- 172.21.15.26:0/1534375310 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c01b070 con 0x55db86838310 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f9153fff640 1 -- 172.21.15.26:0/1534375310 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f916c027070 con 0x55db86838310 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.888+0000 7f9153fff640 1 -- 172.21.15.26:0/1534375310 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c003a20 con 0x55db86838310 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.889+0000 7f917b59a740 1 -- 172.21.15.26:0/1534375310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86838310 msgr2=0x55db86810970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.889+0000 7f917b59a740 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86838310 0x55db86810970 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f916c004470 tx=0x7f916c014000 comp rx=0 tx=0).stop 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.889+0000 7f917b59a740 1 -- 172.21.15.26:0/1534375310 shutdown_connections 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.889+0000 7f917b59a740 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8680e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.889+0000 7f917b59a740 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db8683ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.889+0000 7f917b59a740 1 --2- 172.21.15.26:0/1534375310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86838310 0x55db86810970 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.889+0000 7f917b59a740 1 -- 172.21.15.26:0/1534375310 >> 172.21.15.26:0/1534375310 conn(0x55db8680d600 msgr2=0x55db86855b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.889+0000 7f917b59a740 1 -- 172.21.15.26:0/1534375310 shutdown_connections 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.889+0000 7f917b59a740 1 -- 172.21.15.26:0/1534375310 wait complete. 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.890+0000 7f917b59a740 1 -- start start 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.890+0000 7f917b59a740 1 Processor -- start 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f917b59a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db86836820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f917b59a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8684ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f917b59a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86836ed0 0x55db8684d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f917b59a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55db8672ff50 con 0x55db86836ed0 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f917b59a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55db86853870 con 0x55db8683b0b0 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f917b59a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55db86837c10 con 0x55db868386f0 2024-04-24T23:25:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f917b59a740 10 client.0 _ll_register_callbacks cb 0x55db8676b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9172d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86836ed0 0x55db8684d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9172d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86836ed0 0x55db8684d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46324/0 (socket says 172.21.15.26:46324) 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9172d9d640 1 -- 172.21.15.26:0/4006210906 learned_addr learned my addr 172.21.15.26:0/4006210906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9171d9b640 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8684ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f917259c640 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db86836820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9172d9d640 1 -- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 msgr2=0x55db86836820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9172d9d640 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db86836820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9172d9d640 1 -- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 msgr2=0x55db8684ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9172d9d640 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8684ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9172d9d640 1 -- 172.21.15.26:0/4006210906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db864883f0 con 0x55db86836ed0 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f917259c640 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db86836820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f9172d9d640 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86836ed0 0x55db8684d350 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9164009f40 tx=0x7f9164004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ca3abbf483e6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.891+0000 7f91517fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f91517fa640 1 -- 172.21.15.26:0/4006210906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164044070 con 0x55db86836ed0 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f91517fa640 1 -- 172.21.15.26:0/4006210906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f916403e470 con 0x55db86836ed0 2024-04-24T23:25:09.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917b59a740 1 -- 172.21.15.26:0/4006210906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f916c027070 con 0x55db86836ed0 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f91517fa640 1 -- 172.21.15.26:0/4006210906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f916404f050 con 0x55db86836ed0 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917b59a740 10 client.6951 fetch_fsmap learned FSMap version 58 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917b59a740 1 -- 172.21.15.26:0/4006210906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x55db864480f0 con 0x55db86836ed0 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f91517fa640 1 -- 172.21.15.26:0/4006210906 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f916c027070 con 0x55db86836ed0 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f91517fa640 1 -- 172.21.15.26:0/4006210906 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 58) ==== 3542+0+0 (secure 0 0 0) 0x7f9164052020 con 0x55db86836ed0 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917b59a740 10 client.6951 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917b59a740 10 client.6951 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917b59a740 20 client.6951 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917b59a740 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55db8684f190 0x55db86855630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917b59a740 4 client.6951 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917b59a740 1 -- 172.21.15.26:0/4006210906 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db86781bf0 con 0x55db8684f190 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.892+0000 7f917259c640 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55db8684f190 0x55db86855630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.893+0000 7f917259c640 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55db8684f190 0x55db86855630 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dd52926b3af47b7 server_cookie=6c0cc44444213ebb in_seq=0 out_seq=0 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.893+0000 7f91517fa640 10 client.6951 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.893+0000 7f91517fa640 1 -- 172.21.15.26:0/4006210906 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db86781bf0 con 0x55db8684f190 2024-04-24T23:25:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f91517fa640 10 client.6951 handle_command_reply: tid=0 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 client.6951 shutdown 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 -- 172.21.15.26:0/4006210906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86836ed0 msgr2=0x55db8684d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86836ed0 0x55db8684d350 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9164009f40 tx=0x7f9164004ab0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 -- 172.21.15.26:0/4006210906 shutdown_connections 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55db8684f190 0x55db86855630 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db86836ed0 0x55db8684d350 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8684ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 --2- 172.21.15.26:0/4006210906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db868386f0 0x55db86836820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 -- 172.21.15.26:0/4006210906 >> 172.21.15.26:0/4006210906 conn(0x55db86838310 msgr2=0x55db86858ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f91517fa640 0 client.6951 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 -- 172.21.15.26:0/4006210906 shutdown_connections 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 1 -- 172.21.15.26:0/4006210906 wait complete. 2024-04-24T23:25:09.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.894+0000 7f917b59a740 20 client.6951 trim_cache size 0 max 16384 2024-04-24T23:25:09.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.912+0000 7f917b59a740 1 Processor -- start 2024-04-24T23:25:09.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.912+0000 7f917b59a740 1 -- start start 2024-04-24T23:25:09.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f917b59a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db86836ed0 0x55db8680fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f917b59a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db86810fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f917b59a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db8683b0b0 0x55db86853a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f917b59a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f917406fce0 con 0x55db868386f0 2024-04-24T23:25:09.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f917b59a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f917406eee0 con 0x55db86836ed0 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f917b59a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f917406f5b0 con 0x55db8683b0b0 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f9171d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db86810fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f9171d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db86810fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46332/0 (socket says 172.21.15.26:46332) 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f9171d9b640 1 -- 172.21.15.26:0/686552290 learned_addr learned my addr 172.21.15.26:0/686552290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f917259c640 1 --2- 172.21.15.26:0/686552290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db86836ed0 0x55db8680fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f9171d9b640 1 -- 172.21.15.26:0/686552290 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db8683b0b0 msgr2=0x55db86853a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f9171d9b640 1 --2- 172.21.15.26:0/686552290 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db8683b0b0 0x55db86853a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f9171d9b640 1 -- 172.21.15.26:0/686552290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db86836ed0 msgr2=0x55db8680fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f9171d9b640 1 --2- 172.21.15.26:0/686552290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db86836ed0 0x55db8680fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f9171d9b640 1 -- 172.21.15.26:0/686552290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db86855830 con 0x55db868386f0 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f917259c640 1 --2- 172.21.15.26:0/686552290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db86836ed0 0x55db8680fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f9171d9b640 1 --2- 172.21.15.26:0/686552290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db86810fb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f915c000fb0 tx=0x7f915c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c55996bbe415e68 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f91537fe640 1 -- 172.21.15.26:0/686552290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c061150 con 0x55db868386f0 2024-04-24T23:25:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f91537fe640 1 -- 172.21.15.26:0/686552290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915c070070 con 0x55db868386f0 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.913+0000 7f91537fe640 1 -- 172.21.15.26:0/686552290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c056920 con 0x55db868386f0 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 -- 172.21.15.26:0/686552290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 msgr2=0x55db86810fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 --2- 172.21.15.26:0/686552290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db86810fb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f915c000fb0 tx=0x7f915c000ed0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 -- 172.21.15.26:0/686552290 shutdown_connections 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 --2- 172.21.15.26:0/686552290 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db8683b0b0 0x55db86853a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 --2- 172.21.15.26:0/686552290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db86810fb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 --2- 172.21.15.26:0/686552290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db86836ed0 0x55db8680fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 -- 172.21.15.26:0/686552290 >> 172.21.15.26:0/686552290 conn(0x55db86838310 msgr2=0x55db86858ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 -- 172.21.15.26:0/686552290 shutdown_connections 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 -- 172.21.15.26:0/686552290 wait complete. 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.914+0000 7f917b59a740 1 -- start start 2024-04-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.915+0000 7f917b59a740 1 Processor -- start 2024-04-24T23:25:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.915+0000 7f917b59a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db86836ed0 0x55db868d4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.915+0000 7f917b59a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db8680f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.915+0000 7f917b59a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8684d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.915+0000 7f917b59a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9174152080 con 0x55db868386f0 2024-04-24T23:25:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.915+0000 7f917b59a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55db868d4660 con 0x55db8683b0b0 2024-04-24T23:25:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.915+0000 7f917b59a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55db868d4ac0 con 0x55db86836ed0 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.915+0000 7f917b59a740 10 client.0 _ll_register_callbacks cb 0x7f916c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9171d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db8680f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9171d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db8680f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46336/0 (socket says 172.21.15.26:46336) 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9171d9b640 1 -- 172.21.15.26:0/1574394713 learned_addr learned my addr 172.21.15.26:0/1574394713 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f917259c640 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db86836ed0 0x55db868d4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9172d9d640 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8684d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9171d9b640 1 -- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db86836ed0 msgr2=0x55db868d4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9171d9b640 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db86836ed0 0x55db868d4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9171d9b640 1 -- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 msgr2=0x55db8684d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9171d9b640 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8684d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9171d9b640 1 -- 172.21.15.26:0/1574394713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8680f710 con 0x55db868386f0 2024-04-24T23:25:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f917259c640 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db86836ed0 0x55db868d4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9172d9d640 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8684d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f9171d9b640 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db8680f1d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f915c054a80 tx=0x7f915c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194aa3da816c8841 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f91517fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f91517fa640 1 -- 172.21.15.26:0/1574394713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c054c90 con 0x55db868386f0 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f91517fa640 1 -- 172.21.15.26:0/1574394713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f915c077360 con 0x55db868386f0 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f917b59a740 1 -- 172.21.15.26:0/1574394713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f915c070070 con 0x55db868386f0 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f91517fa640 1 -- 172.21.15.26:0/1574394713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f915c054e30 con 0x55db868386f0 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f917b59a740 10 client.6957 fetch_fsmap learned FSMap version 58 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f917b59a740 1 -- 172.21.15.26:0/1574394713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x55db868d4800 con 0x55db868386f0 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.916+0000 7f91517fa640 1 -- 172.21.15.26:0/1574394713 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915c070070 con 0x55db868386f0 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f91517fa640 1 -- 172.21.15.26:0/1574394713 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 58) ==== 3542+0+0 (secure 0 0 0) 0x7f915c086020 con 0x55db868386f0 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f917b59a740 10 client.6957 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f917b59a740 10 client.6957 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f917b59a740 20 client.6957 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:09.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f917b59a740 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55db8684d820 0x55db868c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f917b59a740 4 client.6957 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f917b59a740 1 -- 172.21.15.26:0/1574394713 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db8677bf40 con 0x55db8684d820 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f917259c640 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55db8684d820 0x55db868c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f91517fa640 10 client.6957 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f917259c640 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55db8684d820 0x55db868c3c70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15fc4dc976bcf0f3 server_cookie=babbdd1b1c75c92f in_seq=0 out_seq=0 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f91517fa640 1 -- 172.21.15.26:0/1574394713 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55db8677bf40 con 0x55db8684d820 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f91517fa640 10 client.6957 handle_command_reply: tid=0 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.917+0000 7f917b59a740 1 client.6957 shutdown 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 -- 172.21.15.26:0/1574394713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 msgr2=0x55db8680f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db8680f1d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f915c054a80 tx=0x7f915c00a000 comp rx=0 tx=0).stop 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 -- 172.21.15.26:0/1574394713 shutdown_connections 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55db8684d820 0x55db868c3c70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55db8683b0b0 0x55db8684d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f91517fa640 0 client.6957 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:25:09.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55db868386f0 0x55db8680f1d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 --2- 172.21.15.26:0/1574394713 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55db86836ed0 0x55db868d4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 -- 172.21.15.26:0/1574394713 >> 172.21.15.26:0/1574394713 conn(0x55db86838310 msgr2=0x55db8680f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 -- 172.21.15.26:0/1574394713 shutdown_connections 2024-04-24T23:25:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 1 -- 172.21.15.26:0/1574394713 wait complete. 2024-04-24T23:25:09.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917b59a740 20 client.6957 trim_cache size 0 max 16384 2024-04-24T23:25:09.925 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:25:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2024-04-24T23:25:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2024-04-24T23:25:09.925 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:25:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 -- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f915406be30 msgr2=0x7f915406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f915406be30 0x7f915406e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f916c009aa0 tx=0x7f916c009190 comp rx=0 tx=0).stop 2024-04-24T23:25:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 -- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91740add40 msgr2=0x7f9174141310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91740add40 0x7f9174141310 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c440 tx=0x7f915c001e70 comp rx=0 tx=0).stop 2024-04-24T23:25:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 -- 172.21.15.26:0/4254415988 shutdown_connections 2024-04-24T23:25:09.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f915406be30 0x7f915406e2f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174145b30 0x7f9174141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91740add40 0x7f9174141310 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 --2- 172.21.15.26:0/4254415988 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f917406b120 0x7f917409e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.918+0000 7f917888c640 1 -- 172.21.15.26:0/4254415988 >> 172.21.15.26:0/4254415988 conn(0x7f91740d0d20 msgr2=0x7f91740b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.919+0000 7f917888c640 1 -- 172.21.15.26:0/4254415988 shutdown_connections 2024-04-24T23:25:09.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:09.919+0000 7f917888c640 1 -- 172.21.15.26:0/4254415988 wait complete. 2024-04-24T23:25:09.930 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:25:09.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 damage ls 2024-04-24T23:25:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.032+0000 7f8be6b63640 1 Processor -- start 2024-04-24T23:25:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.032+0000 7f8be6b63640 1 -- start start 2024-04-24T23:25:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.033+0000 7f8be6b63640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8be00d4b60 0x7f8be00b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.033+0000 7f8be6b63640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be00cfe80 con 0x7f8be00d0680 2024-04-24T23:25:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.033+0000 7f8be6b63640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be00b70e0 con 0x7f8be00d1170 2024-04-24T23:25:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.033+0000 7f8be6b63640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8be00cfc30 con 0x7f8be00d4b60 2024-04-24T23:25:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.033+0000 7f8be48ff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8be00d0680 0x7f8be00d0a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41464/0 (socket says 172.21.15.26:41464) 2024-04-24T23:25:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.033+0000 7f8be48ff640 1 -- 172.21.15.26:0/843147501 learned_addr learned my addr 172.21.15.26:0/843147501 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.033+0000 7f8bdffff640 1 --2- 172.21.15.26:0/843147501 >> v2:172.21.15.29:3300/0 conn(0x7f8be00d4b60 0x7f8be00b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.034+0000 7f8bdf7fe640 1 -- 172.21.15.26:0/843147501 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2558567477 0 0) 0x7f8be00cfe80 con 0x7f8be00d0680 2024-04-24T23:25:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.034+0000 7f8bdf7fe640 1 -- 172.21.15.26:0/843147501 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bc8003520 con 0x7f8be00d0680 2024-04-24T23:25:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.034+0000 7f8bdffff640 1 -- 172.21.15.26:0/843147501 >> v1:172.21.15.29:6789/0 conn(0x7f8be00d1170 legacy=0x7f8be00b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.034+0000 7f8bdf7fe640 1 -- 172.21.15.26:0/843147501 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4067822087 0 0) 0x7f8be00b70e0 con 0x7f8be00d1170 2024-04-24T23:25:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.034+0000 7f8bdffff640 1 -- 172.21.15.26:0/843147501 >> v1:172.21.15.26:6789/0 conn(0x7f8be00d0680 legacy=0x7f8be00d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.034+0000 7f8bdffff640 1 -- 172.21.15.26:0/843147501 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be015ea40 con 0x7f8be00d4b60 2024-04-24T23:25:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8bdffff640 1 --2- 172.21.15.26:0/843147501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8be00d4b60 0x7f8be00b78e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40097a0 tx=0x7f8bd402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c5dd135152ded40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8bdf7fe640 1 -- 172.21.15.26:0/843147501 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4037070 con 0x7f8be00d4b60 2024-04-24T23:25:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8bdf7fe640 1 -- 172.21.15.26:0/843147501 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd4032440 con 0x7f8be00d4b60 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8bdf7fe640 1 -- 172.21.15.26:0/843147501 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4032810 con 0x7f8be00d4b60 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8be6b63640 1 -- 172.21.15.26:0/843147501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8be00d4b60 msgr2=0x7f8be00b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8be6b63640 1 --2- 172.21.15.26:0/843147501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8be00d4b60 0x7f8be00b78e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40097a0 tx=0x7f8bd402a230 comp rx=0 tx=0).stop 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8be6b63640 1 -- 172.21.15.26:0/843147501 shutdown_connections 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8be6b63640 1 --2- 172.21.15.26:0/843147501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8be00d4b60 0x7f8be00b78e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8be6b63640 1 -- 172.21.15.26:0/843147501 >> 172.21.15.26:0/843147501 conn(0x7f8be00d4040 msgr2=0x7f8be00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8be6b63640 1 -- 172.21.15.26:0/843147501 shutdown_connections 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.035+0000 7f8be6b63640 1 -- 172.21.15.26:0/843147501 wait complete. 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.036+0000 7f8be6b63640 1 Processor -- start 2024-04-24T23:25:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.036+0000 7f8be6b63640 1 -- start start 2024-04-24T23:25:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.037+0000 7f8be6b63640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8be00d0680 0x7f8be0153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.037+0000 7f8be6b63640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8be00d1170 0x7f8be01f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.037+0000 7f8be6b63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8be00d4b60 0x7f8be01f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.037+0000 7f8be6b63640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00b6e90 con 0x7f8be00d0680 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.037+0000 7f8be6b63640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00ceea0 con 0x7f8be00d4b60 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.037+0000 7f8be6b63640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00b8530 con 0x7f8be00d1170 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.037+0000 7f8be48ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8be00d0680 0x7f8be0153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be48ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8be00d0680 0x7f8be0153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46340/0 (socket says 172.21.15.26:46340) 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be48ff640 1 -- 172.21.15.26:0/849194021 learned_addr learned my addr 172.21.15.26:0/849194021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8bdffff640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8be00d1170 0x7f8be01f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be5100640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8be00d4b60 0x7f8be01f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be48ff640 1 -- 172.21.15.26:0/849194021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8be00d1170 msgr2=0x7f8be01f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be48ff640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8be00d1170 0x7f8be01f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be48ff640 1 -- 172.21.15.26:0/849194021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8be00d4b60 msgr2=0x7f8be01f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be48ff640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8be00d4b60 0x7f8be01f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be48ff640 1 -- 172.21.15.26:0/849194021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be0152f80 con 0x7f8be00d0680 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be48ff640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8be00d0680 0x7f8be0153a40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800cb40 tx=0x7f8bd800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8bdd7fa640 1 -- 172.21.15.26:0/849194021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8017070 con 0x7f8be00d0680 2024-04-24T23:25:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8bdd7fa640 1 -- 172.21.15.26:0/849194021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd800aca0 con 0x7f8be00d0680 2024-04-24T23:25:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be6b63640 1 -- 172.21.15.26:0/849194021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be01f7a80 con 0x7f8be00d0680 2024-04-24T23:25:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8bdd7fa640 1 -- 172.21.15.26:0/849194021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8012620 con 0x7f8be00d0680 2024-04-24T23:25:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.038+0000 7f8be6b63640 1 -- 172.21.15.26:0/849194021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be0153240 con 0x7f8be00d0680 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.040+0000 7f8bdd7fa640 1 -- 172.21.15.26:0/849194021 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bd801a070 con 0x7f8be00d0680 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.040+0000 7f8bdd7fa640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bcc06be30 0x7f8bcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.040+0000 7f8bdd7fa640 1 -- 172.21.15.26:0/849194021 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8bd808ac20 con 0x7f8be00d0680 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.040+0000 7f8bdffff640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bcc06be30 0x7f8bcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.041+0000 7f8bdffff640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bcc06be30 0x7f8bcc06e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4034000 tx=0x7f8bd402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.041+0000 7f8be987b740 1 Processor -- start 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.041+0000 7f8be987b740 1 -- start start 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be987b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b95926e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be987b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 0x55b95929ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be987b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929e330 0x55b959273140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be987b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b9591d7670 con 0x55b9592bb950 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be987b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b95917f970 con 0x55b95929e330 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be987b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b9591d3630 con 0x55b95929ea70 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be48ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b95926e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be48ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b95926e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46356/0 (socket says 172.21.15.26:46356) 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be48ff640 1 -- 172.21.15.26:0/3273017947 learned_addr learned my addr 172.21.15.26:0/3273017947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be48ff640 1 -- 172.21.15.26:0/3273017947 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 msgr2=0x55b95929ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8bdffff640 1 --2- 172.21.15.26:0/3273017947 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 0x55b95929ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be5100640 1 --2- 172.21.15.26:0/3273017947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929e330 0x55b959273140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be48ff640 1 --2- 172.21.15.26:0/3273017947 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 0x55b95929ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be48ff640 1 -- 172.21.15.26:0/3273017947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929e330 msgr2=0x55b959273140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be48ff640 1 --2- 172.21.15.26:0/3273017947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929e330 0x55b959273140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.042+0000 7f8be48ff640 1 -- 172.21.15.26:0/3273017947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9591a3400 con 0x55b9592bb950 2024-04-24T23:25:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be48ff640 1 --2- 172.21.15.26:0/3273017947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b95926e2e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80890f0 tx=0x7f8bd8058a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff798a058ac6cfed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8bc67fc640 1 -- 172.21.15.26:0/3273017947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8087070 con 0x55b9592bb950 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8bc67fc640 1 -- 172.21.15.26:0/3273017947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd8057280 con 0x55b9592bb950 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8bc67fc640 1 -- 172.21.15.26:0/3273017947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8061620 con 0x55b9592bb950 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be987b740 1 -- 172.21.15.26:0/3273017947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 msgr2=0x55b95926e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be987b740 1 --2- 172.21.15.26:0/3273017947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b95926e2e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80890f0 tx=0x7f8bd8058a10 comp rx=0 tx=0).stop 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be987b740 1 -- 172.21.15.26:0/3273017947 shutdown_connections 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be987b740 1 --2- 172.21.15.26:0/3273017947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929e330 0x55b959273140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be987b740 1 --2- 172.21.15.26:0/3273017947 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 0x55b95929ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be987b740 1 --2- 172.21.15.26:0/3273017947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b95926e2e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be987b740 1 -- 172.21.15.26:0/3273017947 >> 172.21.15.26:0/3273017947 conn(0x55b959270550 msgr2=0x55b9592b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be987b740 1 -- 172.21.15.26:0/3273017947 shutdown_connections 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.043+0000 7f8be987b740 1 -- 172.21.15.26:0/3273017947 wait complete. 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.044+0000 7f8be987b740 1 -- start start 2024-04-24T23:25:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.044+0000 7f8be987b740 1 Processor -- start 2024-04-24T23:25:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8be987b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 0x55b95928b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8be987b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b9592a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8be987b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 0x55b95928c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8be987b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b959272700 con 0x55b9592bb950 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8be987b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b959193050 con 0x55b9592a0020 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8be987b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b959271af0 con 0x55b95929ea70 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8be987b740 10 client.0 _ll_register_callbacks cb 0x55b958eeabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8bdffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b9592a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8bdffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b9592a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46364/0 (socket says 172.21.15.26:46364) 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8bdffff640 1 -- 172.21.15.26:0/2322294493 learned_addr learned my addr 172.21.15.26:0/2322294493 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8be48ff640 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 0x55b95928b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8be5100640 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 0x55b95928c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8bdffff640 1 -- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 msgr2=0x55b95928b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8bdffff640 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 0x55b95928b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8bdffff640 1 -- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 msgr2=0x55b95928c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8bdffff640 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 0x55b95928c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.045+0000 7f8bdffff640 1 -- 172.21.15.26:0/2322294493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b959164dc0 con 0x55b9592bb950 2024-04-24T23:25:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8bdffff640 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b9592a1ca0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8bd402a820 tx=0x7f8bd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0bf93e3203d49f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8bbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/2322294493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4037040 con 0x55b9592bb950 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/2322294493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8bd4049670 con 0x55b9592bb950 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be987b740 1 -- 172.21.15.26:0/2322294493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd8057280 con 0x55b9592bb950 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/2322294493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8bd4049870 con 0x55b9592bb950 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be987b740 10 client.6969 fetch_fsmap learned FSMap version 58 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be987b740 1 -- 172.21.15.26:0/2322294493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x55b958eeb3f0 con 0x55b9592bb950 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/2322294493 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd8057280 con 0x55b9592bb950 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/2322294493 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 58) ==== 3542+0+0 (secure 0 0 0) 0x7f8bd4049a10 con 0x55b9592bb950 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be987b740 10 client.6969 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be987b740 10 client.6969 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be987b740 20 client.6969 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be987b740 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55b95928c590 0x55b9592732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be987b740 4 client.6969 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be987b740 1 -- 172.21.15.26:0/2322294493 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9591e4bb0 con 0x55b95928c590 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.046+0000 7f8be48ff640 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55b95928c590 0x55b9592732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.047+0000 7f8be48ff640 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55b95928c590 0x55b9592732a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb6e88745dea7e7 server_cookie=306cf449c081305d in_seq=0 out_seq=0 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.047+0000 7f8bbf7fe640 10 client.6969 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.047+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/2322294493 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9591e4bb0 con 0x55b95928c590 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.047+0000 7f8bbf7fe640 10 client.6969 handle_command_reply: tid=0 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.047+0000 7f8be987b740 1 client.6969 shutdown 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.047+0000 7f8be987b740 1 -- 172.21.15.26:0/2322294493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 msgr2=0x55b9592a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b9592a1ca0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8bd402a820 tx=0x7f8bd4004730 comp rx=0 tx=0).stop 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 1 -- 172.21.15.26:0/2322294493 shutdown_connections 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55b95928c590 0x55b9592732a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 0x55b95928c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592bb950 0x55b9592a1ca0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8bbf7fe640 0 client.6969 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:25:10.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 1 --2- 172.21.15.26:0/2322294493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b95929ea70 0x55b95928b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 1 -- 172.21.15.26:0/2322294493 >> 172.21.15.26:0/2322294493 conn(0x55b95929e330 msgr2=0x55b9592baf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 1 -- 172.21.15.26:0/2322294493 shutdown_connections 2024-04-24T23:25:10.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 1 -- 172.21.15.26:0/2322294493 wait complete. 2024-04-24T23:25:10.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.048+0000 7f8be987b740 20 client.6969 trim_cache size 0 max 16384 2024-04-24T23:25:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.061+0000 7f8be987b740 1 Processor -- start 2024-04-24T23:25:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.061+0000 7f8be987b740 1 -- start start 2024-04-24T23:25:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8be987b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929ea70 0x55b9592d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8be987b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592a0020 0x55b95926f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8be987b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 0x55b959273760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8be987b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0200470 con 0x55b9592a0020 2024-04-24T23:25:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8be987b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01ff670 con 0x55b95929ea70 2024-04-24T23:25:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8be987b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8be01ffd40 con 0x55b9592bb950 2024-04-24T23:25:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8bdffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592a0020 0x55b95926f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8bdffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592a0020 0x55b95926f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46366/0 (socket says 172.21.15.26:46366) 2024-04-24T23:25:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8bdffff640 1 -- 172.21.15.26:0/222921037 learned_addr learned my addr 172.21.15.26:0/222921037 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8be5100640 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 0x55b959273760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8be48ff640 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929ea70 0x55b9592d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8bdffff640 1 -- 172.21.15.26:0/222921037 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 msgr2=0x55b959273760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8bdffff640 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 0x55b959273760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8bdffff640 1 -- 172.21.15.26:0/222921037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929ea70 msgr2=0x55b9592d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8bdffff640 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929ea70 0x55b9592d26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8bdffff640 1 -- 172.21.15.26:0/222921037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b958fb57d0 con 0x55b9592a0020 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.062+0000 7f8be48ff640 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929ea70 0x55b9592d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8bdffff640 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592a0020 0x55b95926f020 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4033fd0 tx=0x7f8bd4046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a20ffbc7068c509 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8bc5ffb640 1 -- 172.21.15.26:0/222921037 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd405f070 con 0x55b9592a0020 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8bc5ffb640 1 -- 172.21.15.26:0/222921037 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd404e070 con 0x55b9592a0020 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8bc5ffb640 1 -- 172.21.15.26:0/222921037 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4037030 con 0x55b9592a0020 2024-04-24T23:25:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8be987b740 1 -- 172.21.15.26:0/222921037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592a0020 msgr2=0x55b95926f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8be987b740 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592a0020 0x55b95926f020 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4033fd0 tx=0x7f8bd4046000 comp rx=0 tx=0).stop 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8be987b740 1 -- 172.21.15.26:0/222921037 shutdown_connections 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8be987b740 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 0x55b959273760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8be987b740 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9592a0020 0x55b95926f020 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8be987b740 1 --2- 172.21.15.26:0/222921037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b95929ea70 0x55b9592d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8be987b740 1 -- 172.21.15.26:0/222921037 >> 172.21.15.26:0/222921037 conn(0x55b95929e330 msgr2=0x55b95926d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8be987b740 1 -- 172.21.15.26:0/222921037 shutdown_connections 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.063+0000 7f8be987b740 1 -- 172.21.15.26:0/222921037 wait complete. 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.064+0000 7f8be987b740 1 -- start start 2024-04-24T23:25:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.064+0000 7f8be987b740 1 Processor -- start 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be987b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b95929ea70 0x55b95929a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be987b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 0x55b9593320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be987b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 0x55b9592b9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be987b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b95926daf0 con 0x55b95929ea70 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be987b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00cd710 con 0x55b9592a0020 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be987b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b9592a1c80 con 0x55b9592bb950 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be987b740 10 client.0 _ll_register_callbacks cb 0x7f8bc80036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be48ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b95929ea70 0x55b95929a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be48ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b95929ea70 0x55b95929a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46372/0 (socket says 172.21.15.26:46372) 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be48ff640 1 -- 172.21.15.26:0/351256890 learned_addr learned my addr 172.21.15.26:0/351256890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be5100640 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 0x55b9592b9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8bdffff640 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 0x55b9593320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be48ff640 1 -- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 msgr2=0x55b9592b9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be48ff640 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 0x55b9592b9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be48ff640 1 -- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 msgr2=0x55b9593320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be48ff640 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 0x55b9593320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be48ff640 1 -- 172.21.15.26:0/351256890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b959274110 con 0x55b95929ea70 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8be48ff640 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b95929ea70 0x55b95929a180 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8061580 tx=0x7f8bd801e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fa2927db5fc93f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8bbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.065+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/351256890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8087070 con 0x55b95929ea70 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/351256890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8bd8061620 con 0x55b95929ea70 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8be987b740 1 -- 172.21.15.26:0/351256890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd404e070 con 0x55b95929ea70 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/351256890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8bd8061820 con 0x55b95929ea70 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8be987b740 10 client.6975 fetch_fsmap learned FSMap version 58 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8be987b740 1 -- 172.21.15.26:0/351256890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x55b95929b400 con 0x55b95929ea70 2024-04-24T23:25:10.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/351256890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd404e070 con 0x55b95929ea70 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/351256890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 58) ==== 3542+0+0 (secure 0 0 0) 0x7f8bd80583d0 con 0x55b95929ea70 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8be987b740 10 client.6975 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8be987b740 10 client.6975 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8be987b740 20 client.6975 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8be987b740 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55b9592d26e0 0x55b9592d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8be987b740 4 client.6975 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8be987b740 1 -- 172.21.15.26:0/351256890 --> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9591def00 con 0x55b9592d26e0 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.066+0000 7f8bdffff640 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55b9592d26e0 0x55b9592d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8bbf7fe640 10 client.6975 ms_handle_connect on v2:172.21.15.29:6833/2096523178 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8bdffff640 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55b9592d26e0 0x55b9592d2af0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc3115b78468dbb server_cookie=e6b1abee1a5a3b91 in_seq=0 out_seq=0 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8bbf7fe640 1 -- 172.21.15.26:0/351256890 <== mds.0 v2:172.21.15.29:6833/2096523178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+272 (crc 0 0 0) 0x55b9591def00 con 0x55b9592d26e0 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8bbf7fe640 10 client.6975 handle_command_reply: tid=0 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 client.6975 shutdown 2024-04-24T23:25:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 -- 172.21.15.26:0/351256890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b95929ea70 msgr2=0x55b95929a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b95929ea70 0x55b95929a180 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8061580 tx=0x7f8bd801e070 comp rx=0 tx=0).stop 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 -- 172.21.15.26:0/351256890 shutdown_connections 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:6833/2096523178,v1:172.21.15.29:6835/2096523178] conn(0x55b9592d26e0 0x55b9592d2af0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b9592bb950 0x55b9592b9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8bbf7fe640 0 client.6975 ms_handle_reset on v2:172.21.15.29:6833/2096523178 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9592a0020 0x55b9593320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 --2- 172.21.15.26:0/351256890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b95929ea70 0x55b95929a180 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 -- 172.21.15.26:0/351256890 >> 172.21.15.26:0/351256890 conn(0x55b95929e330 msgr2=0x55b95926f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 -- 172.21.15.26:0/351256890 shutdown_connections 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 1 -- 172.21.15.26:0/351256890 wait complete. 2024-04-24T23:25:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.067+0000 7f8be987b740 20 client.6975 trim_cache size 0 max 16384 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3517365630, 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511627778, 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3" 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3834335946, 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511627779, 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3/d4" 2024-04-24T23:25:10.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:10.076 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:25:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 -- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bcc06be30 msgr2=0x7f8bcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bcc06be30 0x7f8bcc06e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4034000 tx=0x7f8bd402e000 comp rx=0 tx=0).stop 2024-04-24T23:25:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 -- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8be00d0680 msgr2=0x7f8be0153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8be00d0680 0x7f8be0153a40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800cb40 tx=0x7f8bd800a060 comp rx=0 tx=0).stop 2024-04-24T23:25:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 -- 172.21.15.26:0/849194021 shutdown_connections 2024-04-24T23:25:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bcc06be30 0x7f8bcc06e2f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8be00d4b60 0x7f8be01f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8be00d1170 0x7f8be01f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 --2- 172.21.15.26:0/849194021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8be00d0680 0x7f8be0153a40 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 -- 172.21.15.26:0/849194021 >> 172.21.15.26:0/849194021 conn(0x7f8be00d4040 msgr2=0x7f8be00b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 -- 172.21.15.26:0/849194021 shutdown_connections 2024-04-24T23:25:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.068+0000 7f8be6b63640 1 -- 172.21.15.26:0/849194021 wait complete. 2024-04-24T23:25:10.080 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 damage ls 2024-04-24T23:25:10.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.181+0000 7f1bac3dc640 1 Processor -- start 2024-04-24T23:25:10.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.181+0000 7f1bac3dc640 1 -- start start 2024-04-24T23:25:10.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.182+0000 7f1bac3dc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba415e370 0x7f1ba40d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.182+0000 7f1bac3dc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ba40d4c10 0x7f1ba40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.182+0000 7f1bac3dc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba40d2120 con 0x7f1ba415e750 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.182+0000 7f1bac3dc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba40d1510 con 0x7f1ba40d4c10 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.182+0000 7f1bac3dc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba40d1ed0 con 0x7f1ba415e370 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.182+0000 7f1baa979640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ba40d4c10 0x7f1ba40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.182+0000 7f1baa979640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ba40d4c10 0x7f1ba40cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46378/0 (socket says 172.21.15.26:46378) 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.182+0000 7f1baa979640 1 -- 172.21.15.26:0/4294721442 learned_addr learned my addr 172.21.15.26:0/4294721442 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.182+0000 7f1ba9977640 1 --1- 172.21.15.26:0/4294721442 >> v1:172.21.15.26:6789/0 conn(0x7f1ba415e750 0x7f1ba40b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41476/0 (socket says 172.21.15.26:41476) 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.183+0000 7f1baa178640 1 --2- 172.21.15.26:0/4294721442 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba415e370 0x7f1ba40d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.183+0000 7f1baa979640 1 -- 172.21.15.26:0/4294721442 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba415e370 msgr2=0x7f1ba40d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.183+0000 7f1baa979640 1 --2- 172.21.15.26:0/4294721442 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba415e370 0x7f1ba40d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.183+0000 7f1baa979640 1 -- 172.21.15.26:0/4294721442 >> v1:172.21.15.26:6789/0 conn(0x7f1ba415e750 legacy=0x7f1ba40b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.183+0000 7f1baa979640 1 -- 172.21.15.26:0/4294721442 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba40b3740 con 0x7f1ba40d4c10 2024-04-24T23:25:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.183+0000 7f1ba9176640 1 -- 172.21.15.26:0/4294721442 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4243652335 0 0) 0x7f1ba40d2120 con 0x7f1ba415e750 2024-04-24T23:25:10.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1baa979640 1 --2- 172.21.15.26:0/4294721442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40d4c10 0x7f1ba40cfca0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009e50 tx=0x7f1b9402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4b356f0816e914 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1ba9176640 1 -- 172.21.15.26:0/4294721442 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94036070 con 0x7f1ba40d4c10 2024-04-24T23:25:10.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1ba9176640 1 -- 172.21.15.26:0/4294721442 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b94031600 con 0x7f1ba40d4c10 2024-04-24T23:25:10.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1ba9176640 1 -- 172.21.15.26:0/4294721442 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940319d0 con 0x7f1ba40d4c10 2024-04-24T23:25:10.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1bac3dc640 1 -- 172.21.15.26:0/4294721442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40d4c10 msgr2=0x7f1ba40cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1bac3dc640 1 --2- 172.21.15.26:0/4294721442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40d4c10 0x7f1ba40cfca0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009e50 tx=0x7f1b9402a710 comp rx=0 tx=0).stop 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1bac3dc640 1 -- 172.21.15.26:0/4294721442 shutdown_connections 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1bac3dc640 1 --2- 172.21.15.26:0/4294721442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40d4c10 0x7f1ba40cfca0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1bac3dc640 1 --2- 172.21.15.26:0/4294721442 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba415e370 0x7f1ba40d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1bac3dc640 1 -- 172.21.15.26:0/4294721442 >> 172.21.15.26:0/4294721442 conn(0x7f1ba40d40f0 msgr2=0x7f1ba40d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1bac3dc640 1 -- 172.21.15.26:0/4294721442 shutdown_connections 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.184+0000 7f1bac3dc640 1 -- 172.21.15.26:0/4294721442 wait complete. 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.186+0000 7f1bac3dc640 1 Processor -- start 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.186+0000 7f1bac3dc640 1 -- start start 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.186+0000 7f1bac3dc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba40d4c10 0x7f1ba415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.186+0000 7f1bac3dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba415e370 0x7f1ba41f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.186+0000 7f1bac3dc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ba41f7a20 0x7f1ba41f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.186+0000 7f1bac3dc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba40d2a60 con 0x7f1ba415e370 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.186+0000 7f1bac3dc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba40b3d00 con 0x7f1ba41f7a20 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.186+0000 7f1bac3dc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba40d2be0 con 0x7f1ba40d4c10 2024-04-24T23:25:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.186+0000 7f1ba9977640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba415e370 0x7f1ba41f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba9977640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba415e370 0x7f1ba41f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46392/0 (socket says 172.21.15.26:46392) 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba9977640 1 -- 172.21.15.26:0/252628234 learned_addr learned my addr 172.21.15.26:0/252628234 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1baa178640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba40d4c10 0x7f1ba415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1baa979640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ba41f7a20 0x7f1ba41f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba9977640 1 -- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba40d4c10 msgr2=0x7f1ba415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba9977640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba40d4c10 0x7f1ba415a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba9977640 1 -- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ba41f7a20 msgr2=0x7f1ba41f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba9977640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ba41f7a20 0x7f1ba41f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba9977640 1 -- 172.21.15.26:0/252628234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba40cb260 con 0x7f1ba415e370 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1baa178640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba40d4c10 0x7f1ba415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1baa979640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ba41f7a20 0x7f1ba41f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba9977640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba415e370 0x7f1ba41f74e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c430 tx=0x7f1b9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba2ffd640 1 -- 172.21.15.26:0/252628234 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b980074e0 con 0x7f1ba415e370 2024-04-24T23:25:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba2ffd640 1 -- 172.21.15.26:0/252628234 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b980047e0 con 0x7f1ba415e370 2024-04-24T23:25:10.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1ba2ffd640 1 -- 172.21.15.26:0/252628234 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98004bd0 con 0x7f1ba415e370 2024-04-24T23:25:10.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1bac3dc640 1 -- 172.21.15.26:0/252628234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba40cb540 con 0x7f1ba415e370 2024-04-24T23:25:10.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.187+0000 7f1bac3dc640 1 -- 172.21.15.26:0/252628234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba40cbde0 con 0x7f1ba415e370 2024-04-24T23:25:10.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.189+0000 7f1ba2ffd640 1 -- 172.21.15.26:0/252628234 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b98011070 con 0x7f1ba415e370 2024-04-24T23:25:10.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.189+0000 7f1ba2ffd640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b9006be30 0x7f1b9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.189+0000 7f1ba2ffd640 1 -- 172.21.15.26:0/252628234 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1b9808ab00 con 0x7f1ba415e370 2024-04-24T23:25:10.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.189+0000 7f1baa178640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b9006be30 0x7f1b9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.190+0000 7f1baa178640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b9006be30 0x7f1b9006e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c006fd0 tx=0x7f1b9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.190+0000 7f1baf1ca740 1 Processor -- start 2024-04-24T23:25:10.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.190+0000 7f1baf1ca740 1 -- start start 2024-04-24T23:25:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 0x563f8742c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 0x563f8745bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745c330 0x563f87431140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563f87395670 con 0x563f8745c330 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563f8733d970 con 0x563f8745ca70 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563f87391630 con 0x563f87479950 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baa979640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745c330 0x563f87431140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baa979640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745c330 0x563f87431140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46404/0 (socket says 172.21.15.26:46404) 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baa979640 1 -- 172.21.15.26:0/1112547531 learned_addr learned my addr 172.21.15.26:0/1112547531 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1baa178640 1 --2- 172.21.15.26:0/1112547531 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 0x563f8742c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.191+0000 7f1ba9977640 1 --2- 172.21.15.26:0/1112547531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 0x563f8745bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.192+0000 7f1baa979640 1 -- 172.21.15.26:0/1112547531 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 msgr2=0x563f8742c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.192+0000 7f1baa979640 1 --2- 172.21.15.26:0/1112547531 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 0x563f8742c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.192+0000 7f1baa979640 1 -- 172.21.15.26:0/1112547531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 msgr2=0x563f8745bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.192+0000 7f1baa979640 1 --2- 172.21.15.26:0/1112547531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 0x563f8745bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.192+0000 7f1baa979640 1 -- 172.21.15.26:0/1112547531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f87361400 con 0x563f8745c330 2024-04-24T23:25:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.192+0000 7f1baa979640 1 --2- 172.21.15.26:0/1112547531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745c330 0x563f87431140 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402ad00 tx=0x7f1b9402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b89000b10b8e20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.192+0000 7f1b8bfff640 1 -- 172.21.15.26:0/1112547531 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94036040 con 0x563f8745c330 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.192+0000 7f1b8bfff640 1 -- 172.21.15.26:0/1112547531 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b94042070 con 0x563f8745c330 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.192+0000 7f1b8bfff640 1 -- 172.21.15.26:0/1112547531 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9403c870 con 0x563f8745c330 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.193+0000 7f1baf1ca740 1 -- 172.21.15.26:0/1112547531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745c330 msgr2=0x563f87431140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.193+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/1112547531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745c330 0x563f87431140 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402ad00 tx=0x7f1b9402e000 comp rx=0 tx=0).stop 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.193+0000 7f1baf1ca740 1 -- 172.21.15.26:0/1112547531 shutdown_connections 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.193+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/1112547531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745c330 0x563f87431140 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.193+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/1112547531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 0x563f8745bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.193+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/1112547531 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 0x563f8742c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.193+0000 7f1baf1ca740 1 -- 172.21.15.26:0/1112547531 >> 172.21.15.26:0/1112547531 conn(0x563f8742e550 msgr2=0x563f87476df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.193+0000 7f1baf1ca740 1 -- 172.21.15.26:0/1112547531 shutdown_connections 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.193+0000 7f1baf1ca740 1 -- 172.21.15.26:0/1112547531 wait complete. 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.194+0000 7f1baf1ca740 1 -- start start 2024-04-24T23:25:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.194+0000 7f1baf1ca740 1 Processor -- start 2024-04-24T23:25:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f8745ca70 0x563f8744c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f8745d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745d5a0 0x563f87478350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563f87430700 con 0x563f87479950 2024-04-24T23:25:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563f87351050 con 0x563f8745d5a0 2024-04-24T23:25:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563f8742faf0 con 0x563f8745ca70 2024-04-24T23:25:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1ba9977640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f8745d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 10 client.0 _ll_register_callbacks cb 0x563f870a8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1ba9977640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f8745d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46418/0 (socket says 172.21.15.26:46418) 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1ba9977640 1 -- 172.21.15.26:0/2281680971 learned_addr learned my addr 172.21.15.26:0/2281680971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1ba9977640 1 -- 172.21.15.26:0/2281680971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f8745ca70 msgr2=0x563f8744c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1ba9977640 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f8745ca70 0x563f8744c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1ba9977640 1 -- 172.21.15.26:0/2281680971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745d5a0 msgr2=0x563f87478350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1ba9977640 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745d5a0 0x563f87478350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1ba9977640 1 -- 172.21.15.26:0/2281680971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f87322dc0 con 0x563f87479950 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1ba9977640 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f8745d060 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1b98088fd0 tx=0x7f1b9805d160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf091453de09ad6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1b897fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1b897fa640 1 -- 172.21.15.26:0/2281680971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98087060 con 0x563f87479950 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1b897fa640 1 -- 172.21.15.26:0/2281680971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1b98055420 con 0x563f87479950 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 1 -- 172.21.15.26:0/2281680971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b94042070 con 0x563f87479950 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1b897fa640 1 -- 172.21.15.26:0/2281680971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1b98058200 con 0x563f87479950 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 10 client.6990 fetch_fsmap learned FSMap version 58 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1baf1ca740 1 -- 172.21.15.26:0/2281680971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x563f870a93f0 con 0x563f87479950 2024-04-24T23:25:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.195+0000 7f1b897fa640 1 -- 172.21.15.26:0/2281680971 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b94042070 con 0x563f87479950 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1b897fa640 1 -- 172.21.15.26:0/2281680971 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 58) ==== 3542+0+0 (secure 0 0 0) 0x7f1b9805f050 con 0x563f87479950 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1baf1ca740 10 client.6990 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1baf1ca740 10 client.6990 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1baf1ca740 20 client.6990 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x563f8748a580 0x563f8748aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1baf1ca740 4 client.6990 mds_command: new command op to 6534 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1baf1ca740 1 -- 172.21.15.26:0/2281680971 --> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f873a2bb0 con 0x563f8748a580 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1baa178640 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x563f8748a580 0x563f8748aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1b897fa640 10 client.6990 ms_handle_connect on v2:172.21.15.26:6835/2743842996 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.196+0000 7f1baa178640 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x563f8748a580 0x563f8748aa00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d4f9b4e357c6fe9 server_cookie=319c2a7fc0e47f30 in_seq=0 out_seq=0 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1b897fa640 1 -- 172.21.15.26:0/2281680971 <== mds.1 v2:172.21.15.26:6835/2743842996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f873a2bb0 con 0x563f8748a580 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1b897fa640 10 client.6990 handle_command_reply: tid=0 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 client.6990 shutdown 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 -- 172.21.15.26:0/2281680971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 msgr2=0x563f8745d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f8745d060 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1b98088fd0 tx=0x7f1b9805d160 comp rx=0 tx=0).stop 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 -- 172.21.15.26:0/2281680971 shutdown_connections 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x563f8748a580 0x563f8748aa00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745d5a0 0x563f87478350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f8745d060 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/2281680971 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f8745ca70 0x563f8744c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1b897fa640 0 client.6990 ms_handle_reset on v2:172.21.15.26:6835/2743842996 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 -- 172.21.15.26:0/2281680971 >> 172.21.15.26:0/2281680971 conn(0x563f8745c330 msgr2=0x563f8747a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 -- 172.21.15.26:0/2281680971 shutdown_connections 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 1 -- 172.21.15.26:0/2281680971 wait complete. 2024-04-24T23:25:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.197+0000 7f1baf1ca740 20 client.6990 trim_cache size 0 max 16384 2024-04-24T23:25:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.211+0000 7f1baf1ca740 1 Processor -- start 2024-04-24T23:25:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.211+0000 7f1baf1ca740 1 -- start start 2024-04-24T23:25:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.211+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745ca70 0x563f87476de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.211+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745d5a0 0x563f8749a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.211+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 0x563f8749aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.211+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4200560 con 0x563f8745ca70 2024-04-24T23:25:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.211+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41ff760 con 0x563f8745d5a0 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.211+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba41ffe30 con 0x563f87479950 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa178640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745ca70 0x563f87476de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa178640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745ca70 0x563f87476de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46424/0 (socket says 172.21.15.26:46424) 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa178640 1 -- 172.21.15.26:0/4186506522 learned_addr learned my addr 172.21.15.26:0/4186506522 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1ba9977640 1 --2- 172.21.15.26:0/4186506522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745d5a0 0x563f8749a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa979640 1 --2- 172.21.15.26:0/4186506522 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 0x563f8749aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa178640 1 -- 172.21.15.26:0/4186506522 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 msgr2=0x563f8749aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa178640 1 --2- 172.21.15.26:0/4186506522 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 0x563f8749aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa178640 1 -- 172.21.15.26:0/4186506522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745d5a0 msgr2=0x563f8749a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa178640 1 --2- 172.21.15.26:0/4186506522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745d5a0 0x563f8749a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa178640 1 -- 172.21.15.26:0/4186506522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8742f690 con 0x563f8745ca70 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1baa178640 1 --2- 172.21.15.26:0/4186506522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745ca70 0x563f87476de0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c004580 tx=0x7f1b9c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb79782139b5d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1b8b7fe640 1 -- 172.21.15.26:0/4186506522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c029070 con 0x563f8745ca70 2024-04-24T23:25:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1b8b7fe640 1 -- 172.21.15.26:0/4186506522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9c0253a0 con 0x563f8745ca70 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.212+0000 7f1b8b7fe640 1 -- 172.21.15.26:0/4186506522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c00f040 con 0x563f8745ca70 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 -- 172.21.15.26:0/4186506522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745ca70 msgr2=0x563f87476de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/4186506522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745ca70 0x563f87476de0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c004580 tx=0x7f1b9c011000 comp rx=0 tx=0).stop 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 -- 172.21.15.26:0/4186506522 shutdown_connections 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/4186506522 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f87479950 0x563f8749aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/4186506522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745d5a0 0x563f8749a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/4186506522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8745ca70 0x563f87476de0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 -- 172.21.15.26:0/4186506522 >> 172.21.15.26:0/4186506522 conn(0x563f8745c330 msgr2=0x563f8744c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 -- 172.21.15.26:0/4186506522 shutdown_connections 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 -- 172.21.15.26:0/4186506522 wait complete. 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.213+0000 7f1baf1ca740 1 -- start start 2024-04-24T23:25:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.214+0000 7f1baf1ca740 1 Processor -- start 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.214+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 0x563f87489a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.214+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f8745d5a0 0x563f8742b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.214+0000 7f1baf1ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f87475050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.214+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563f87430510 con 0x563f87479950 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.214+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41fe7a0 con 0x563f8745ca70 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.214+0000 7f1baf1ca740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563f87490d90 con 0x563f8745d5a0 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.214+0000 7f1baf1ca740 10 client.0 _ll_register_callbacks cb 0x7f1b980589e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa979640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f87475050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa979640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f87475050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46440/0 (socket says 172.21.15.26:46440) 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa979640 1 -- 172.21.15.26:0/725275692 learned_addr learned my addr 172.21.15.26:0/725275692 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa178640 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 0x563f87489a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1ba9977640 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f8745d5a0 0x563f8742b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa979640 1 -- 172.21.15.26:0/725275692 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f8745d5a0 msgr2=0x563f8742b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa979640 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f8745d5a0 0x563f8742b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa979640 1 -- 172.21.15.26:0/725275692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 msgr2=0x563f87489a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa979640 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 0x563f87489a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa979640 1 -- 172.21.15.26:0/725275692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f871737d0 con 0x563f87479950 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baa979640 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f87475050 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402a750 tx=0x7f1b94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4aab82435b7506 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1b897fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1b897fa640 1 -- 172.21.15.26:0/725275692 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94036040 con 0x563f87479950 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1b897fa640 1 -- 172.21.15.26:0/725275692 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1b94004250 con 0x563f87479950 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baf1ca740 1 -- 172.21.15.26:0/725275692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9c0253a0 con 0x563f87479950 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1b897fa640 1 -- 172.21.15.26:0/725275692 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1b94055050 con 0x563f87479950 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baf1ca740 10 client.6996 fetch_fsmap learned FSMap version 58 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.215+0000 7f1baf1ca740 1 -- 172.21.15.26:0/725275692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x563f874891f0 con 0x563f87479950 2024-04-24T23:25:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1b897fa640 1 -- 172.21.15.26:0/725275692 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9c0253a0 con 0x563f87479950 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1b897fa640 1 -- 172.21.15.26:0/725275692 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 58) ==== 3542+0+0 (secure 0 0 0) 0x7f1b94058020 con 0x563f87479950 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 10 client.6996 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 10 client.6996 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 20 client.6996 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x563f874767f0 0x563f87475590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 4 client.6996 mds_command: new command op to 6534 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 1 -- 172.21.15.26:0/725275692 --> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f8739cf00 con 0x563f874767f0 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baa178640 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x563f874767f0 0x563f87475590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baa178640 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x563f874767f0 0x563f87475590 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb52943887bb22e1 server_cookie=54cf07fbf4efa06f in_seq=0 out_seq=0 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1b897fa640 10 client.6996 ms_handle_connect on v2:172.21.15.26:6835/2743842996 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1b897fa640 1 -- 172.21.15.26:0/725275692 <== mds.1 v2:172.21.15.26:6835/2743842996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+284 (crc 0 0 0) 0x563f8739cf00 con 0x563f874767f0 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1b897fa640 10 client.6996 handle_command_reply: tid=0 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 1 client.6996 shutdown 2024-04-24T23:25:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 1 -- 172.21.15.26:0/725275692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 msgr2=0x563f87475050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f87475050 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402a750 tx=0x7f1b94004a40 comp rx=0 tx=0).stop 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 1 -- 172.21.15.26:0/725275692 shutdown_connections 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.216+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.26:6835/2743842996,v1:172.21.15.26:6837/2743842996] conn(0x563f874767f0 0x563f87475590 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f87479950 0x563f87475050 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563f8745d5a0 0x563f8742b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1baf1ca740 1 --2- 172.21.15.26:0/725275692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563f8745ca70 0x563f87489a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1b897fa640 0 client.6996 ms_handle_reset on v2:172.21.15.26:6835/2743842996 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1baf1ca740 1 -- 172.21.15.26:0/725275692 >> 172.21.15.26:0/725275692 conn(0x563f8745c330 msgr2=0x563f8742fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1baf1ca740 1 -- 172.21.15.26:0/725275692 shutdown_connections 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1baf1ca740 1 -- 172.21.15.26:0/725275692 wait complete. 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1baf1ca740 20 client.6996 trim_cache size 0 max 16384 2024-04-24T23:25:10.223 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 1789404374, 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511627780, 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3/d4/d5" 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2086059590, 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511627781, 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3/d4/d5/d6" 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:10.224 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:25:10.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 -- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b9006be30 msgr2=0x7f1b9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b9006be30 0x7f1b9006e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c006fd0 tx=0x7f1b9c008040 comp rx=0 tx=0).stop 2024-04-24T23:25:10.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 -- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba415e370 msgr2=0x7f1ba41f74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba415e370 0x7f1ba41f74e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c430 tx=0x7f1b9800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 -- 172.21.15.26:0/252628234 shutdown_connections 2024-04-24T23:25:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b9006be30 0x7f1b9006e2f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ba41f7a20 0x7f1ba41f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba415e370 0x7f1ba41f74e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 --2- 172.21.15.26:0/252628234 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ba40d4c10 0x7f1ba415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 -- 172.21.15.26:0/252628234 >> 172.21.15.26:0/252628234 conn(0x7f1ba40d40f0 msgr2=0x7f1ba40d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 -- 172.21.15.26:0/252628234 shutdown_connections 2024-04-24T23:25:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.217+0000 7f1bac3dc640 1 -- 172.21.15.26:0/252628234 wait complete. 2024-04-24T23:25:10.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 damage ls 2024-04-24T23:25:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.331+0000 7fa849ebc640 1 Processor -- start 2024-04-24T23:25:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.331+0000 7fa849ebc640 1 -- start start 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.331+0000 7fa849ebc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8440d0750 0x7fa844165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.331+0000 7fa849ebc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa84415ef60 con 0x7fa84415adc0 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.331+0000 7fa849ebc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa84415f240 con 0x7fa8440ad7b0 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.331+0000 7fa849ebc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa84415f520 con 0x7fa8440d0750 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.332+0000 7fa843fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8440d0750 0x7fa844165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.332+0000 7fa8437fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa84415adc0 0x7fa8440b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41482/0 (socket says 172.21.15.26:41482) 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.332+0000 7fa843fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8440d0750 0x7fa844165d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46446/0 (socket says 172.21.15.26:46446) 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.332+0000 7fa8437fe640 1 -- 172.21.15.26:0/2273567845 learned_addr learned my addr 172.21.15.26:0/2273567845 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.332+0000 7fa843fff640 1 -- 172.21.15.26:0/2273567845 >> v1:172.21.15.29:6789/0 conn(0x7fa8440ad7b0 legacy=0x7fa8440d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.332+0000 7fa843fff640 1 -- 172.21.15.26:0/2273567845 >> v1:172.21.15.26:6789/0 conn(0x7fa84415adc0 legacy=0x7fa8440b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.332+0000 7fa843fff640 1 -- 172.21.15.26:0/2273567845 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84415f800 con 0x7fa8440d0750 2024-04-24T23:25:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.332+0000 7fa8427fc640 1 -- 172.21.15.26:0/2273567845 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601183956 0 0) 0x7fa84415ef60 con 0x7fa84415adc0 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa843fff640 1 --2- 172.21.15.26:0/2273567845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 0x7fa844165d50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009dc0 tx=0x7fa83c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3dbf5a4cc8758f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa8427fc640 1 -- 172.21.15.26:0/2273567845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007740 con 0x7fa8440d0750 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa8427fc640 1 -- 172.21.15.26:0/2273567845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83c0078e0 con 0x7fa8440d0750 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa8427fc640 1 -- 172.21.15.26:0/2273567845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007cb0 con 0x7fa8440d0750 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa849ebc640 1 -- 172.21.15.26:0/2273567845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 msgr2=0x7fa844165d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa849ebc640 1 --2- 172.21.15.26:0/2273567845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 0x7fa844165d50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009dc0 tx=0x7fa83c02a640 comp rx=0 tx=0).stop 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa849ebc640 1 -- 172.21.15.26:0/2273567845 shutdown_connections 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa849ebc640 1 --2- 172.21.15.26:0/2273567845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 0x7fa844165d50 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa849ebc640 1 -- 172.21.15.26:0/2273567845 >> 172.21.15.26:0/2273567845 conn(0x7fa84406b540 msgr2=0x7fa84406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa849ebc640 1 -- 172.21.15.26:0/2273567845 shutdown_connections 2024-04-24T23:25:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.333+0000 7fa849ebc640 1 -- 172.21.15.26:0/2273567845 wait complete. 2024-04-24T23:25:10.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.334+0000 7fa849ebc640 1 Processor -- start 2024-04-24T23:25:10.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.335+0000 7fa849ebc640 1 -- start start 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.335+0000 7fa849ebc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa8440ad7b0 0x7fa8441476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.335+0000 7fa849ebc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 0x7fa8440af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.335+0000 7fa849ebc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84415adc0 0x7fa8440af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.335+0000 7fa849ebc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa84415e9c0 con 0x7fa8440d0750 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.335+0000 7fa849ebc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa84415b1a0 con 0x7fa84415adc0 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.335+0000 7fa849ebc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa84415eb40 con 0x7fa8440ad7b0 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa842ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 0x7fa8440af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa842ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 0x7fa8440af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46448/0 (socket says 172.21.15.26:46448) 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa842ffd640 1 -- 172.21.15.26:0/3990626753 learned_addr learned my addr 172.21.15.26:0/3990626753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa8437fe640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa8440ad7b0 0x7fa8441476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa843fff640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84415adc0 0x7fa8440af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa842ffd640 1 -- 172.21.15.26:0/3990626753 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa8440ad7b0 msgr2=0x7fa8441476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa842ffd640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa8440ad7b0 0x7fa8441476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa842ffd640 1 -- 172.21.15.26:0/3990626753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84415adc0 msgr2=0x7fa8440af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa842ffd640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84415adc0 0x7fa8440af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa842ffd640 1 -- 172.21.15.26:0/3990626753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa844147bf0 con 0x7fa8440d0750 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa843fff640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84415adc0 0x7fa8440af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa842ffd640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 0x7fa8440af040 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c400 tx=0x7fa83800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa82bfff640 1 -- 172.21.15.26:0/3990626753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838011790 con 0x7fa8440d0750 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa82bfff640 1 -- 172.21.15.26:0/3990626753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa838017070 con 0x7fa8440d0750 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa849ebc640 1 -- 172.21.15.26:0/3990626753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84409ccc0 con 0x7fa8440d0750 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa82bfff640 1 -- 172.21.15.26:0/3990626753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838011bb0 con 0x7fa8440d0750 2024-04-24T23:25:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.336+0000 7fa849ebc640 1 -- 172.21.15.26:0/3990626753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84409d280 con 0x7fa8440d0750 2024-04-24T23:25:10.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.338+0000 7fa82bfff640 1 -- 172.21.15.26:0/3990626753 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa83801c460 con 0x7fa8440d0750 2024-04-24T23:25:10.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.338+0000 7fa82bfff640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa83006be60 0x7fa83006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.338+0000 7fa82bfff640 1 -- 172.21.15.26:0/3990626753 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa83808ac20 con 0x7fa8440d0750 2024-04-24T23:25:10.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.338+0000 7fa8437fe640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa83006be60 0x7fa83006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.339+0000 7fa8437fe640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa83006be60 0x7fa83006e320 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa834000f60 tx=0x7fa834009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.339+0000 7fa84cbc5740 1 Processor -- start 2024-04-24T23:25:10.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.339+0000 7fa84cbc5740 1 -- start start 2024-04-24T23:25:10.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 0x555b3da662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3da95df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96330 0x555b3da6b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b3d9cf670 con 0x555b3da96330 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b3d977970 con 0x555b3da96a70 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555b3d9cb630 con 0x555b3dab3950 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa843fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96330 0x555b3da6b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa843fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96330 0x555b3da6b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46450/0 (socket says 172.21.15.26:46450) 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa843fff640 1 -- 172.21.15.26:0/585842352 learned_addr learned my addr 172.21.15.26:0/585842352 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa842ffd640 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3da95df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa8437fe640 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 0x555b3da662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa843fff640 1 -- 172.21.15.26:0/585842352 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 msgr2=0x555b3da662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa843fff640 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 0x555b3da662e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa843fff640 1 -- 172.21.15.26:0/585842352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 msgr2=0x555b3da95df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa843fff640 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3da95df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa843fff640 1 -- 172.21.15.26:0/585842352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3d99b400 con 0x555b3da96330 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa842ffd640 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3da95df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.340+0000 7fa843fff640 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96330 0x555b3da6b140 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009eb0 tx=0x7fa83c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e854966673e799 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.341+0000 7fa8297fa640 1 -- 172.21.15.26:0/585842352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004040 con 0x555b3da96330 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.341+0000 7fa8297fa640 1 -- 172.21.15.26:0/585842352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83c0041e0 con 0x555b3da96330 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.341+0000 7fa8297fa640 1 -- 172.21.15.26:0/585842352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004590 con 0x555b3da96330 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.341+0000 7fa84cbc5740 1 -- 172.21.15.26:0/585842352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96330 msgr2=0x555b3da6b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.341+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96330 0x555b3da6b140 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009eb0 tx=0x7fa83c02e000 comp rx=0 tx=0).stop 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.342+0000 7fa84cbc5740 1 -- 172.21.15.26:0/585842352 shutdown_connections 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.342+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96330 0x555b3da6b140 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.342+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3da95df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.342+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/585842352 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 0x555b3da662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.342+0000 7fa84cbc5740 1 -- 172.21.15.26:0/585842352 >> 172.21.15.26:0/585842352 conn(0x555b3da68550 msgr2=0x555b3dab0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.342+0000 7fa84cbc5740 1 -- 172.21.15.26:0/585842352 shutdown_connections 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.342+0000 7fa84cbc5740 1 -- 172.21.15.26:0/585842352 wait complete. 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.342+0000 7fa84cbc5740 1 -- start start 2024-04-24T23:25:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.342+0000 7fa84cbc5740 1 Processor -- start 2024-04-24T23:25:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3dab23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 0x555b3dab2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da99db0 0x555b3da83f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b3da6a700 con 0x555b3da99db0 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b3d98b050 con 0x555b3da96a70 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555b3da69af0 con 0x555b3dab3950 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa84cbc5740 10 client.0 _ll_register_callbacks cb 0x555b3d6e2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa843fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da99db0 0x555b3da83f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa843fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da99db0 0x555b3da83f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46462/0 (socket says 172.21.15.26:46462) 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa843fff640 1 -- 172.21.15.26:0/2376008628 learned_addr learned my addr 172.21.15.26:0/2376008628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa8437fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3dab23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa842ffd640 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 0x555b3dab2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa843fff640 1 -- 172.21.15.26:0/2376008628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 msgr2=0x555b3dab2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa843fff640 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 0x555b3dab2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa843fff640 1 -- 172.21.15.26:0/2376008628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 msgr2=0x555b3dab23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa843fff640 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3dab23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.343+0000 7fa843fff640 1 -- 172.21.15.26:0/2376008628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3d95cdc0 con 0x555b3da99db0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa843fff640 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da99db0 0x555b3da83f30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa83c02a680 tx=0x7fa83c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87b5fd455b6efe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa8227fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa8227fc640 1 -- 172.21.15.26:0/2376008628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c009a30 con 0x555b3da99db0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa8227fc640 1 -- 172.21.15.26:0/2376008628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa83c051b20 con 0x555b3da99db0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2376008628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa83c0041e0 con 0x555b3da99db0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa8227fc640 1 -- 172.21.15.26:0/2376008628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa83c004850 con 0x555b3da99db0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa84cbc5740 10 client.7011 fetch_fsmap learned FSMap version 58 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2376008628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x555b3d6e33f0 con 0x555b3da99db0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa8227fc640 1 -- 172.21.15.26:0/2376008628 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83c0041e0 con 0x555b3da99db0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa8227fc640 1 -- 172.21.15.26:0/2376008628 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 58) ==== 3542+0+0 (secure 0 0 0) 0x7fa83c007540 con 0x555b3da99db0 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa84cbc5740 10 client.7011 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa84cbc5740 10 client.7011 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.c 2024-04-24T23:25:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa84cbc5740 20 client.7011 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x555b3da83820 0x555b3da84470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa84cbc5740 4 client.7011 mds_command: new command op to 6531 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2376008628 --> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b3d9dcbb0 con 0x555b3da83820 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.344+0000 7fa8437fe640 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x555b3da83820 0x555b3da84470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.345+0000 7fa8227fc640 10 client.7011 ms_handle_connect on v2:172.21.15.26:6834/1519104007 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.345+0000 7fa8437fe640 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x555b3da83820 0x555b3da84470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c11c15c13143635c server_cookie=29b88b487e9f0a8d in_seq=0 out_seq=0 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.345+0000 7fa8227fc640 1 -- 172.21.15.26:0/2376008628 <== mds.2 v2:172.21.15.26:6834/1519104007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b3d9dcbb0 con 0x555b3da83820 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.345+0000 7fa8227fc640 10 client.7011 handle_command_reply: tid=0 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.345+0000 7fa84cbc5740 1 client.7011 shutdown 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2376008628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da99db0 msgr2=0x555b3da83f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da99db0 0x555b3da83f30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa83c02a680 tx=0x7fa83c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2376008628 shutdown_connections 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x555b3da83820 0x555b3da84470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da99db0 0x555b3da83f30 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa8227fc640 0 client.7011 ms_handle_reset on v2:172.21.15.26:6834/1519104007 2024-04-24T23:25:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3dab3950 0x555b3dab2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2376008628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3dab23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2376008628 >> 172.21.15.26:0/2376008628 conn(0x555b3da96330 msgr2=0x555b3dab4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2376008628 shutdown_connections 2024-04-24T23:25:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2376008628 wait complete. 2024-04-24T23:25:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.346+0000 7fa84cbc5740 20 client.7011 trim_cache size 0 max 16384 2024-04-24T23:25:10.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.360+0000 7fa84cbc5740 1 Processor -- start 2024-04-24T23:25:10.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.360+0000 7fa84cbc5740 1 -- start start 2024-04-24T23:25:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.361+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da83820 0x555b3da843f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.361+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96a70 0x555b3daa84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.361+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 0x555b3da6a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.361+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441fe3c0 con 0x555b3da96a70 2024-04-24T23:25:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.361+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441fd5c0 con 0x555b3da83820 2024-04-24T23:25:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.361+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa8441fdc90 con 0x555b3da99db0 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.361+0000 7fa842ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96a70 0x555b3daa84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.361+0000 7fa842ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96a70 0x555b3daa84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46466/0 (socket says 172.21.15.26:46466) 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.361+0000 7fa842ffd640 1 -- 172.21.15.26:0/3503196273 learned_addr learned my addr 172.21.15.26:0/3503196273 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa8437fe640 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da83820 0x555b3da843f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa843fff640 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 0x555b3da6a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa842ffd640 1 -- 172.21.15.26:0/3503196273 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 msgr2=0x555b3da6a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa842ffd640 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 0x555b3da6a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa842ffd640 1 -- 172.21.15.26:0/3503196273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da83820 msgr2=0x555b3da843f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa842ffd640 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da83820 0x555b3da843f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa842ffd640 1 -- 172.21.15.26:0/3503196273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3d7ad7d0 con 0x555b3da96a70 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa8437fe640 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da83820 0x555b3da843f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa842ffd640 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96a70 0x555b3daa84a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa838007fd0 tx=0x7fa83801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f21e02875813f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa828ff9640 1 -- 172.21.15.26:0/3503196273 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838057260 con 0x555b3da96a70 2024-04-24T23:25:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa828ff9640 1 -- 172.21.15.26:0/3503196273 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa838011d50 con 0x555b3da96a70 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.362+0000 7fa828ff9640 1 -- 172.21.15.26:0/3503196273 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8380616b0 con 0x555b3da96a70 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 -- 172.21.15.26:0/3503196273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96a70 msgr2=0x555b3daa84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96a70 0x555b3daa84a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa838007fd0 tx=0x7fa83801e040 comp rx=0 tx=0).stop 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 -- 172.21.15.26:0/3503196273 shutdown_connections 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 0x555b3da6a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da96a70 0x555b3daa84a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/3503196273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da83820 0x555b3da843f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 -- 172.21.15.26:0/3503196273 >> 172.21.15.26:0/3503196273 conn(0x555b3da96330 msgr2=0x555b3dab02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 -- 172.21.15.26:0/3503196273 shutdown_connections 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 -- 172.21.15.26:0/3503196273 wait complete. 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 -- start start 2024-04-24T23:25:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.363+0000 7fa84cbc5740 1 Processor -- start 2024-04-24T23:25:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da83820 0x555b3da6a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3da81c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa84cbc5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 0x555b3da821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b3d98b050 con 0x555b3da83820 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441f65e0 con 0x555b3da96a70 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa84cbc5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555b3da6a960 con 0x555b3da99db0 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa84cbc5740 10 client.0 _ll_register_callbacks cb 0x7fa83c003990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa8437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da83820 0x555b3da6a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa8437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da83820 0x555b3da6a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46476/0 (socket says 172.21.15.26:46476) 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa8437fe640 1 -- 172.21.15.26:0/2659711652 learned_addr learned my addr 172.21.15.26:0/2659711652 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa842ffd640 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3da81c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.364+0000 7fa843fff640 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 0x555b3da821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8437fe640 1 -- 172.21.15.26:0/2659711652 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 msgr2=0x555b3da821a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8437fe640 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 0x555b3da821a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8437fe640 1 -- 172.21.15.26:0/2659711652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 msgr2=0x555b3da81c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8437fe640 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3da81c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8437fe640 1 -- 172.21.15.26:0/2659711652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3da84340 con 0x555b3da83820 2024-04-24T23:25:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8437fe640 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da83820 0x555b3da6a420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa834001e80 tx=0x7fa834000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d681ad345d3620c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8227fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8227fc640 1 -- 172.21.15.26:0/2659711652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340265c0 con 0x555b3da83820 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8227fc640 1 -- 172.21.15.26:0/2659711652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa834026760 con 0x555b3da83820 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2659711652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa838011d50 con 0x555b3da83820 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8227fc640 1 -- 172.21.15.26:0/2659711652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa834035050 con 0x555b3da83820 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa84cbc5740 10 client.7017 fetch_fsmap learned FSMap version 58 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2659711652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x555b3db31ea0 con 0x555b3da83820 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8227fc640 1 -- 172.21.15.26:0/2659711652 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa838011d50 con 0x555b3da83820 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa8227fc640 1 -- 172.21.15.26:0/2659711652 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 58) ==== 3542+0+0 (secure 0 0 0) 0x7fa834038020 con 0x555b3da83820 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa84cbc5740 10 client.7017 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa84cbc5740 10 client.7017 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.c 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa84cbc5740 20 client.7017 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x555b3dab3950 0x555b3dab4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa84cbc5740 4 client.7017 mds_command: new command op to 6531 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.365+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2659711652 --> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b3daa98e0 con 0x555b3dab3950 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa842ffd640 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x555b3dab3950 0x555b3dab4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa8227fc640 10 client.7017 ms_handle_connect on v2:172.21.15.26:6834/1519104007 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa842ffd640 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x555b3dab3950 0x555b3dab4860 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=751cfe77309a756b server_cookie=9c1439edf77ff470 in_seq=0 out_seq=0 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa8227fc640 1 -- 172.21.15.26:0/2659711652 <== mds.2 v2:172.21.15.26:6834/1519104007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+296 (crc 0 0 0) 0x555b3daa98e0 con 0x555b3dab3950 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa8227fc640 10 client.7017 handle_command_reply: tid=0 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 client.7017 shutdown 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2659711652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da83820 msgr2=0x555b3da6a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da83820 0x555b3da6a420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa834001e80 tx=0x7fa834000ed0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2659711652 shutdown_connections 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.26:6834/1519104007,v1:172.21.15.26:6836/1519104007] conn(0x555b3dab3950 0x555b3dab4860 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b3da99db0 0x555b3da821a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b3da96a70 0x555b3da81c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 --2- 172.21.15.26:0/2659711652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b3da83820 0x555b3da6a420 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa8227fc640 0 client.7017 ms_handle_reset on v2:172.21.15.26:6834/1519104007 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2659711652 >> 172.21.15.26:0/2659711652 conn(0x555b3da96330 msgr2=0x555b3dac3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2659711652 shutdown_connections 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.366+0000 7fa84cbc5740 1 -- 172.21.15.26:0/2659711652 wait complete. 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa84cbc5740 20 client.7017 trim_cache size 0 max 16384 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3045719451, 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511627783, 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7/d8" 2024-04-24T23:25:10.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:10.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:10.375 INFO:teuthology.orchestra.run.smithi026.stdout: "damage_type": "backtrace", 2024-04-24T23:25:10.375 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4177155588, 2024-04-24T23:25:10.375 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511627782, 2024-04-24T23:25:10.375 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7" 2024-04-24T23:25:10.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:10.375 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 -- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa83006be60 msgr2=0x7fa83006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa83006be60 0x7fa83006e320 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa834000f60 tx=0x7fa834009040 comp rx=0 tx=0).stop 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 -- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 msgr2=0x7fa8440af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 0x7fa8440af040 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c400 tx=0x7fa83800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 -- 172.21.15.26:0/3990626753 shutdown_connections 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa83006be60 0x7fa83006e320 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa84415adc0 0x7fa8440af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440d0750 0x7fa8440af040 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 --2- 172.21.15.26:0/3990626753 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa8440ad7b0 0x7fa8441476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 -- 172.21.15.26:0/3990626753 >> 172.21.15.26:0/3990626753 conn(0x7fa84406b540 msgr2=0x7fa84406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 -- 172.21.15.26:0/3990626753 shutdown_connections 2024-04-24T23:25:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.367+0000 7fa849ebc640 1 -- 172.21.15.26:0/3990626753 wait complete. 2024-04-24T23:25:10.379 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:25:10.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:25:10.416 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:25:10.460 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:25:10.460 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:25:10.461 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:25:10.461 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:10.477 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:25:10.477 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:10.532 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:25:10.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.673+0000 7fe0a2df9640 1 Processor -- start 2024-04-24T23:25:10.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.673+0000 7fe0a2df9640 1 -- start start 2024-04-24T23:25:10.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.673+0000 7fe0a2df9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe09c143c40 0x7fe09c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.673+0000 7fe0a2df9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c15eb00 0x7fe09c15ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.673+0000 7fe0a2df9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe09c0cd0b0 con 0x7fe09c0cc860 2024-04-24T23:25:10.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.673+0000 7fe0a2df9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe09c071b70 con 0x7fe09c143c40 2024-04-24T23:25:10.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.673+0000 7fe0a2df9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe09c15f470 con 0x7fe09c15eb00 2024-04-24T23:25:10.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.674+0000 7fe0a0b95640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe09c143c40 0x7fe09c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.674+0000 7fe0a1396640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c15eb00 0x7fe09c15ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.674+0000 7fe0a0b95640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe09c143c40 0x7fe09c0cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:35846/0 (socket says 172.21.15.26:35846) 2024-04-24T23:25:10.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.674+0000 7fe0a0b95640 1 -- 172.21.15.26:0/4076421237 learned_addr learned my addr 172.21.15.26:0/4076421237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.674+0000 7fe09bfff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fe09c0cc860 0x7fe09c072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:49776/0 (socket says 172.21.15.26:49776) 2024-04-24T23:25:10.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.674+0000 7fe0a0b95640 1 -- 172.21.15.26:0/4076421237 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c15eb00 msgr2=0x7fe09c15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.674+0000 7fe0a0b95640 1 --2- 172.21.15.26:0/4076421237 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c15eb00 0x7fe09c15ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.674+0000 7fe0a0b95640 1 -- 172.21.15.26:0/4076421237 >> v1:172.21.15.29:6789/0 conn(0x7fe09c0cc860 legacy=0x7fe09c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.674+0000 7fe0a0b95640 1 -- 172.21.15.26:0/4076421237 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c071240 con 0x7fe09c143c40 2024-04-24T23:25:10.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.675+0000 7fe0a0b95640 1 --2- 172.21.15.26:0/4076421237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe09c143c40 0x7fe09c0cf870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe090009870 tx=0x7fe09002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b4b373d210436fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.675+0000 7fe09b7fe640 1 -- 172.21.15.26:0/4076421237 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090037070 con 0x7fe09c143c40 2024-04-24T23:25:10.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.675+0000 7fe09b7fe640 1 -- 172.21.15.26:0/4076421237 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe090032440 con 0x7fe09c143c40 2024-04-24T23:25:10.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.675+0000 7fe09b7fe640 1 -- 172.21.15.26:0/4076421237 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090032810 con 0x7fe09c143c40 2024-04-24T23:25:10.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.676+0000 7fe0a2df9640 1 -- 172.21.15.26:0/4076421237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe09c143c40 msgr2=0x7fe09c0cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.676+0000 7fe0a2df9640 1 --2- 172.21.15.26:0/4076421237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe09c143c40 0x7fe09c0cf870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe090009870 tx=0x7fe09002a250 comp rx=0 tx=0).stop 2024-04-24T23:25:10.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.676+0000 7fe0a2df9640 1 -- 172.21.15.26:0/4076421237 shutdown_connections 2024-04-24T23:25:10.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.676+0000 7fe0a2df9640 1 --2- 172.21.15.26:0/4076421237 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c15eb00 0x7fe09c15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.676+0000 7fe0a2df9640 1 --2- 172.21.15.26:0/4076421237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe09c143c40 0x7fe09c0cf870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.676+0000 7fe0a2df9640 1 -- 172.21.15.26:0/4076421237 >> 172.21.15.26:0/4076421237 conn(0x7fe09c06b300 msgr2=0x7fe09c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.676+0000 7fe0a2df9640 1 -- 172.21.15.26:0/4076421237 shutdown_connections 2024-04-24T23:25:10.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.676+0000 7fe0a2df9640 1 -- 172.21.15.26:0/4076421237 wait complete. 2024-04-24T23:25:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.677+0000 7fe0a2df9640 1 Processor -- start 2024-04-24T23:25:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.677+0000 7fe0a2df9640 1 -- start start 2024-04-24T23:25:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a2df9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c0cc860 0x7fe09c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a2df9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe09c143c40 0x7fe09c0aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a2df9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c15eb00 0x7fe09c13a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a2df9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c144020 con 0x7fe09c15eb00 2024-04-24T23:25:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a2df9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c071800 con 0x7fe09c143c40 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a2df9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe09c0cccd0 con 0x7fe09c0cc860 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a1396640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c15eb00 0x7fe09c13a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a1396640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c15eb00 0x7fe09c13a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46484/0 (socket says 172.21.15.26:46484) 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a1396640 1 -- 172.21.15.26:0/2354448996 learned_addr learned my addr 172.21.15.26:0/2354448996 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe09bfff640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe09c143c40 0x7fe09c0aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a0b95640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c0cc860 0x7fe09c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a1396640 1 -- 172.21.15.26:0/2354448996 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c0cc860 msgr2=0x7fe09c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a1396640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c0cc860 0x7fe09c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a1396640 1 -- 172.21.15.26:0/2354448996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe09c143c40 msgr2=0x7fe09c0aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a1396640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe09c143c40 0x7fe09c0aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.678+0000 7fe0a1396640 1 -- 172.21.15.26:0/2354448996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c0b0850 con 0x7fe09c15eb00 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.679+0000 7fe0a1396640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c15eb00 0x7fe09c13a3c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe09400efc0 tx=0x7fe09400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.679+0000 7fe0997fa640 1 -- 172.21.15.26:0/2354448996 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094018020 con 0x7fe09c15eb00 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.679+0000 7fe0997fa640 1 -- 172.21.15.26:0/2354448996 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe094014480 con 0x7fe09c15eb00 2024-04-24T23:25:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.679+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c0b0b30 con 0x7fe09c15eb00 2024-04-24T23:25:10.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.679+0000 7fe0997fa640 1 -- 172.21.15.26:0/2354448996 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094014870 con 0x7fe09c15eb00 2024-04-24T23:25:10.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.679+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c0b13d0 con 0x7fe09c15eb00 2024-04-24T23:25:10.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.680+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe09c0694c0 con 0x7fe09c15eb00 2024-04-24T23:25:10.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.680+0000 7fe0997fa640 1 -- 172.21.15.26:0/2354448996 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe094005d10 con 0x7fe09c15eb00 2024-04-24T23:25:10.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.681+0000 7fe0997fa640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe08006be30 0x7fe08006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.681+0000 7fe0997fa640 1 -- 172.21.15.26:0/2354448996 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe09408b7e0 con 0x7fe09c15eb00 2024-04-24T23:25:10.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.681+0000 7fe0a0b95640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe08006be30 0x7fe08006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.681+0000 7fe0a0b95640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe08006be30 0x7fe08006e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe090034000 tx=0x7fe09002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.685+0000 7fe0997fa640 1 -- 172.21.15.26:0/2354448996 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe094022070 con 0x7fe09c15eb00 2024-04-24T23:25:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.872+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe09c0b4d20 con 0x7fe09c15eb00 2024-04-24T23:25:10.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.873+0000 7fe0997fa640 1 -- 172.21.15.26:0/2354448996 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7fe094010040 con 0x7fe09c15eb00 2024-04-24T23:25:10.874 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:10.875 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:24:16.006495+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:24:13.001912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:24:13.678528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:25:10.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe08006be30 msgr2=0x7fe08006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe08006be30 0x7fe08006e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe090034000 tx=0x7fe09002e000 comp rx=0 tx=0).stop 2024-04-24T23:25:10.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c15eb00 msgr2=0x7fe09c13a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c15eb00 0x7fe09c13a3c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe09400efc0 tx=0x7fe09400c490 comp rx=0 tx=0).stop 2024-04-24T23:25:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 shutdown_connections 2024-04-24T23:25:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe08006be30 0x7fe08006e2f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe09c15eb00 0x7fe09c13a3c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe09c143c40 0x7fe09c0aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 --2- 172.21.15.26:0/2354448996 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe09c0cc860 0x7fe09c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 >> 172.21.15.26:0/2354448996 conn(0x7fe09c06b300 msgr2=0x7fe09c0b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 shutdown_connections 2024-04-24T23:25:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.875+0000 7fe0a2df9640 1 -- 172.21.15.26:0/2354448996 wait complete. 2024-04-24T23:25:10.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:24:13.001912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:24:13.678528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:25:10.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:25:10.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.990+0000 7ff4b2607640 1 Processor -- start 2024-04-24T23:25:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.991+0000 7ff4b2607640 1 -- start start 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.991+0000 7ff4b2607640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac15adc0 0x7ff4ac0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.991+0000 7ff4b2607640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff4ac0d0750 0x7ff4ac072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.991+0000 7ff4b2607640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ac15ef60 con 0x7ff4ac0ad7b0 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.991+0000 7ff4b2607640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ac15f240 con 0x7ff4ac0d0750 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.991+0000 7ff4b2607640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac15f520 con 0x7ff4ac15adc0 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4ab7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff4ac0ad7b0 0x7ff4ac0d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41486/0 (socket says 172.21.15.26:41486) 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/2123963616 learned_addr learned my addr 172.21.15.26:0/2123963616 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4b0ba4640 1 --2- 172.21.15.26:0/2123963616 >> v2:172.21.15.29:3300/0 conn(0x7ff4ac0d0750 0x7ff4ac072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 642364369 0 0) 0x7ff4ac15ef60 con 0x7ff4ac0ad7b0 2024-04-24T23:25:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4abfff640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac15adc0 0x7ff4ac0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff494005730 con 0x7ff4ac0ad7b0 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3461219988 0 0) 0x7ff494005730 con 0x7ff4ac0ad7b0 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac15adc0 msgr2=0x7ff4ac0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac15adc0 0x7ff4ac0b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 >> v2:172.21.15.29:3300/0 conn(0x7ff4ac0d0750 msgr2=0x7ff4ac072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/2123963616 >> v2:172.21.15.29:3300/0 conn(0x7ff4ac0d0750 0x7ff4ac072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac15f800 con 0x7ff4ac0ad7b0 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7ff4a0001670 con 0x7ff4ac0ad7b0 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 >> v1:172.21.15.26:6789/0 conn(0x7ff4ac0ad7b0 legacy=0x7ff4ac0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac15adc0 0x7ff494005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff494006140 0x7ff494006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff494006f50 0x7ff4940073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac15f240 con 0x7ff494006140 2024-04-24T23:25:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac15f520 con 0x7ff494006f50 2024-04-24T23:25:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff49400dc20 con 0x7ff4ac15adc0 2024-04-24T23:25:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff494006140 0x7ff494006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4abfff640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac15adc0 0x7ff494005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.992+0000 7ff4b0ba4640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff494006f50 0x7ff4940073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.993+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac15adc0 msgr2=0x7ff494005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.993+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac15adc0 0x7ff494005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.993+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff494006f50 msgr2=0x7ff4940073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.993+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff494006f50 0x7ff4940073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.993+0000 7ff4ab7fe640 1 -- 172.21.15.26:0/2123963616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff494005730 con 0x7ff494006140 2024-04-24T23:25:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.993+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff494006140 0x7ff494006a10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c270 tx=0x7ff494007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c587b0a708a1e061 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.993+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0007d30 con 0x7ff494006140 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.993+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a0005ad0 con 0x7ff494006140 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.993+0000 7ff4aaffd640 1 -- 172.21.15.26:0/2123963616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0007070 con 0x7ff494006140 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.994+0000 7ff4b2607640 1 -- 172.21.15.26:0/2123963616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff494006140 msgr2=0x7ff494006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.994+0000 7ff4b2607640 1 --2- 172.21.15.26:0/2123963616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff494006140 0x7ff494006a10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c270 tx=0x7ff494007a40 comp rx=0 tx=0).stop 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.994+0000 7ff4b0ba4640 1 -- 172.21.15.26:0/2123963616 reap_dead start 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.994+0000 7ff4b2607640 1 -- 172.21.15.26:0/2123963616 shutdown_connections 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.994+0000 7ff4b2607640 1 -- 172.21.15.26:0/2123963616 >> 172.21.15.26:0/2123963616 conn(0x7ff4ac06b540 msgr2=0x7ff4ac06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.994+0000 7ff4b2607640 1 -- 172.21.15.26:0/2123963616 shutdown_connections 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.994+0000 7ff4b2607640 1 -- 172.21.15.26:0/2123963616 wait complete. 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.995+0000 7ff4b2607640 1 Processor -- start 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.995+0000 7ff4b2607640 1 -- start start 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4b2607640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac153450 0x7ff4ac09bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4b2607640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4ac09c260 0x7ff4ac1463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4b2607640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac153830 0x7ff4ac1468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4b2607640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac15e9c0 con 0x7ff4ac153830 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4b2607640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac15b1a0 con 0x7ff4ac09c260 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4b2607640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac15eb40 con 0x7ff4ac153450 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4b0ba4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac153830 0x7ff4ac1468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4b0ba4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac153830 0x7ff4ac1468e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46500/0 (socket says 172.21.15.26:46500) 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4b0ba4640 1 -- 172.21.15.26:0/282161894 learned_addr learned my addr 172.21.15.26:0/282161894 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4ab7fe640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4ac09c260 0x7ff4ac1463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.996+0000 7ff4abfff640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac153450 0x7ff4ac09bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4b0ba4640 1 -- 172.21.15.26:0/282161894 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac153450 msgr2=0x7ff4ac09bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4b0ba4640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac153450 0x7ff4ac09bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4b0ba4640 1 -- 172.21.15.26:0/282161894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4ac09c260 msgr2=0x7ff4ac1463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4b0ba4640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4ac09c260 0x7ff4ac1463a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4b0ba4640 1 -- 172.21.15.26:0/282161894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac09c7c0 con 0x7ff4ac153830 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4b0ba4640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac153830 0x7ff4ac1468e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac06f210 tx=0x7ff4a400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4a8ff9640 1 -- 172.21.15.26:0/282161894 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400fce0 con 0x7ff4ac153830 2024-04-24T23:25:11.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4a8ff9640 1 -- 172.21.15.26:0/282161894 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a401d070 con 0x7ff4ac153830 2024-04-24T23:25:11.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac09caa0 con 0x7ff4ac153830 2024-04-24T23:25:11.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4a8ff9640 1 -- 172.21.15.26:0/282161894 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4015670 con 0x7ff4ac153830 2024-04-24T23:25:11.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.997+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac09d060 con 0x7ff4ac153830 2024-04-24T23:25:11.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.998+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac075610 con 0x7ff4ac153830 2024-04-24T23:25:11.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.999+0000 7ff4a8ff9640 1 -- 172.21.15.26:0/282161894 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4a4015810 con 0x7ff4ac153830 2024-04-24T23:25:11.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.999+0000 7ff4a8ff9640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff49806be60 0x7ff49806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.999+0000 7ff4a8ff9640 1 -- 172.21.15.26:0/282161894 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff4a408b6f0 con 0x7ff4ac153830 2024-04-24T23:25:11.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:10.999+0000 7ff4abfff640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff49806be60 0x7ff49806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.000+0000 7ff4abfff640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff49806be60 0x7ff49806e320 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009ae0 tx=0x7ff49c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.004+0000 7ff4a8ff9640 1 -- 172.21.15.26:0/282161894 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff4a4010040 con 0x7ff4ac153830 2024-04-24T23:25:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.188+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4ac15e730 con 0x7ff4ac153830 2024-04-24T23:25:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.188+0000 7ff4a8ff9640 1 -- 172.21.15.26:0/282161894 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7ff4a405a010 con 0x7ff4ac153830 2024-04-24T23:25:11.190 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:25:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff49806be60 msgr2=0x7ff49806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff49806be60 0x7ff49806e320 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009ae0 tx=0x7ff49c009190 comp rx=0 tx=0).stop 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac153830 msgr2=0x7ff4ac1468e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac153830 0x7ff4ac1468e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac06f210 tx=0x7ff4a400f1b0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 shutdown_connections 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff49806be60 0x7ff49806e320 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac153830 0x7ff4ac1468e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4ac09c260 0x7ff4ac1463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 --2- 172.21.15.26:0/282161894 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff4ac153450 0x7ff4ac09bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 >> 172.21.15.26:0/282161894 conn(0x7ff4ac06b540 msgr2=0x7ff4ac0adc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 shutdown_connections 2024-04-24T23:25:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.190+0000 7ff4b2607640 1 -- 172.21.15.26:0/282161894 wait complete. 2024-04-24T23:25:11.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:25:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.305+0000 7fd75689a640 1 Processor -- start 2024-04-24T23:25:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.305+0000 7fd75689a640 1 -- start start 2024-04-24T23:25:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd75689a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd75015adc0 0x7fd7500b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd75689a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 0x7fd750072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd75689a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd75015ef60 con 0x7fd7500ad7b0 2024-04-24T23:25:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd75689a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd75015f240 con 0x7fd75015adc0 2024-04-24T23:25:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd75689a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd75015f520 con 0x7fd7500d0750 2024-04-24T23:25:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd74ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd75015adc0 0x7fd7500b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd74f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd7500ad7b0 0x7fd7500d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41490/0 (socket says 172.21.15.26:41490) 2024-04-24T23:25:11.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd74f7fe640 1 -- 172.21.15.26:0/782611299 learned_addr learned my addr 172.21.15.26:0/782611299 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd754e37640 1 --2- 172.21.15.26:0/782611299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 0x7fd750072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd74ffff640 1 -- 172.21.15.26:0/782611299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 msgr2=0x7fd750072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd74ffff640 1 --2- 172.21.15.26:0/782611299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 0x7fd750072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd74ffff640 1 -- 172.21.15.26:0/782611299 >> v1:172.21.15.26:6789/0 conn(0x7fd7500ad7b0 legacy=0x7fd7500d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd74ffff640 1 -- 172.21.15.26:0/782611299 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75015f800 con 0x7fd75015adc0 2024-04-24T23:25:11.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.306+0000 7fd74effd640 1 -- 172.21.15.26:0/782611299 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435042346 0 0) 0x7fd75015ef60 con 0x7fd7500ad7b0 2024-04-24T23:25:11.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.307+0000 7fd74ffff640 1 --2- 172.21.15.26:0/782611299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd75015adc0 0x7fd7500b4370 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd7400097d0 tx=0x7fd74002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da3e5100f991f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.307+0000 7fd74effd640 1 -- 172.21.15.26:0/782611299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740037070 con 0x7fd75015adc0 2024-04-24T23:25:11.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.307+0000 7fd74effd640 1 -- 172.21.15.26:0/782611299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74002ae40 con 0x7fd75015adc0 2024-04-24T23:25:11.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.307+0000 7fd74effd640 1 -- 172.21.15.26:0/782611299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740032660 con 0x7fd75015adc0 2024-04-24T23:25:11.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.307+0000 7fd75689a640 1 -- 172.21.15.26:0/782611299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd75015adc0 msgr2=0x7fd7500b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.307+0000 7fd75689a640 1 --2- 172.21.15.26:0/782611299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd75015adc0 0x7fd7500b4370 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd7400097d0 tx=0x7fd74002a130 comp rx=0 tx=0).stop 2024-04-24T23:25:11.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.308+0000 7fd75689a640 1 -- 172.21.15.26:0/782611299 shutdown_connections 2024-04-24T23:25:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.308+0000 7fd75689a640 1 --2- 172.21.15.26:0/782611299 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 0x7fd750072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.308+0000 7fd75689a640 1 --2- 172.21.15.26:0/782611299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd75015adc0 0x7fd7500b4370 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.308+0000 7fd75689a640 1 -- 172.21.15.26:0/782611299 >> 172.21.15.26:0/782611299 conn(0x7fd75006b540 msgr2=0x7fd75006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.308+0000 7fd75689a640 1 -- 172.21.15.26:0/782611299 shutdown_connections 2024-04-24T23:25:11.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.308+0000 7fd75689a640 1 -- 172.21.15.26:0/782611299 wait complete. 2024-04-24T23:25:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.309+0000 7fd75689a640 1 Processor -- start 2024-04-24T23:25:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.309+0000 7fd75689a640 1 -- start start 2024-04-24T23:25:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd75689a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7500ad7b0 0x7fd750149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd75689a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 0x7fd75009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd75689a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd75015adc0 0x7fd7500b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd75689a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd75015b1a0 con 0x7fd7500ad7b0 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd75689a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7500b48b0 con 0x7fd75015adc0 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd75689a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd75015e9c0 con 0x7fd7500d0750 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7500ad7b0 0x7fd750149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7500ad7b0 0x7fd750149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46530/0 (socket says 172.21.15.26:46530) 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74ffff640 1 -- 172.21.15.26:0/479227824 learned_addr learned my addr 172.21.15.26:0/479227824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74f7fe640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 0x7fd75009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd754e37640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd75015adc0 0x7fd7500b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74ffff640 1 -- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 msgr2=0x7fd75009ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74ffff640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 0x7fd75009ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74ffff640 1 -- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd75015adc0 msgr2=0x7fd7500b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74ffff640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd75015adc0 0x7fd7500b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74ffff640 1 -- 172.21.15.26:0/479227824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75009c660 con 0x7fd7500ad7b0 2024-04-24T23:25:11.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd754e37640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd75015adc0 0x7fd7500b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:11.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74f7fe640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 0x7fd75009ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:11.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74ffff640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7500ad7b0 0x7fd750149860 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd740000c00 tx=0x7fd7400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.310+0000 7fd74cff9640 1 -- 172.21.15.26:0/479227824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740042070 con 0x7fd7500ad7b0 2024-04-24T23:25:11.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.311+0000 7fd74cff9640 1 -- 172.21.15.26:0/479227824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74003e670 con 0x7fd7500ad7b0 2024-04-24T23:25:11.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.311+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75009c940 con 0x7fd7500ad7b0 2024-04-24T23:25:11.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.311+0000 7fd74cff9640 1 -- 172.21.15.26:0/479227824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740037040 con 0x7fd7500ad7b0 2024-04-24T23:25:11.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.311+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75009cf00 con 0x7fd7500ad7b0 2024-04-24T23:25:11.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.311+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd750075610 con 0x7fd7500ad7b0 2024-04-24T23:25:11.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.312+0000 7fd74cff9640 1 -- 172.21.15.26:0/479227824 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd740045070 con 0x7fd7500ad7b0 2024-04-24T23:25:11.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.312+0000 7fd74cff9640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd73c06be30 0x7fd73c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.312+0000 7fd74cff9640 1 -- 172.21.15.26:0/479227824 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd7400ab390 con 0x7fd7500ad7b0 2024-04-24T23:25:11.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.313+0000 7fd74f7fe640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd73c06be30 0x7fd73c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.313+0000 7fd74f7fe640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd73c06be30 0x7fd73c06e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd75006ed60 tx=0x7fd744007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.317+0000 7fd74cff9640 1 -- 172.21.15.26:0/479227824 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd740079dd0 con 0x7fd7500ad7b0 2024-04-24T23:25:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.499+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd7500719f0 con 0x7fd7500ad7b0 2024-04-24T23:25:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.499+0000 7fd74cff9640 1 -- 172.21.15.26:0/479227824 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7fd7400796d0 con 0x7fd7500ad7b0 2024-04-24T23:25:11.500 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:25:11.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd73c06be30 msgr2=0x7fd73c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd73c06be30 0x7fd73c06e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd75006ed60 tx=0x7fd744007c10 comp rx=0 tx=0).stop 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7500ad7b0 msgr2=0x7fd750149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7500ad7b0 0x7fd750149860 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd740000c00 tx=0x7fd7400046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 shutdown_connections 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd73c06be30 0x7fd73c06e2f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd75015adc0 0x7fd7500b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7500d0750 0x7fd75009ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 --2- 172.21.15.26:0/479227824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7500ad7b0 0x7fd750149860 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 >> 172.21.15.26:0/479227824 conn(0x7fd75006b540 msgr2=0x7fd7500d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 shutdown_connections 2024-04-24T23:25:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.501+0000 7fd75689a640 1 -- 172.21.15.26:0/479227824 wait complete. 2024-04-24T23:25:11.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:25:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.616+0000 7f2031ebd640 1 Processor -- start 2024-04-24T23:25:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.616+0000 7f2031ebd640 1 -- start start 2024-04-24T23:25:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.616+0000 7f2031ebd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f202c15e120 0x7f202c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.616+0000 7f2031ebd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f202c150e60 0x7f202c1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.616+0000 7f2031ebd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f202c070f30 con 0x7f202c15e500 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f2031ebd640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f202c06dcf0 con 0x7f202c150e60 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f2031ebd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f202c06dfd0 con 0x7f202c15e120 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f202bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f202c150e60 0x7f202c1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f202affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f202c15e500 0x7f202c0b37e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41494/0 (socket says 172.21.15.26:41494) 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f202affd640 1 -- 172.21.15.26:0/1618089014 learned_addr learned my addr 172.21.15.26:0/1618089014 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f202b7fe640 1 --2- 172.21.15.26:0/1618089014 >> v2:172.21.15.29:3300/0 conn(0x7f202c15e120 0x7f202c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f202bfff640 1 -- 172.21.15.26:0/1618089014 >> v2:172.21.15.29:3300/0 conn(0x7f202c15e120 msgr2=0x7f202c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f202bfff640 1 --2- 172.21.15.26:0/1618089014 >> v2:172.21.15.29:3300/0 conn(0x7f202c15e120 0x7f202c06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f202bfff640 1 -- 172.21.15.26:0/1618089014 >> v1:172.21.15.26:6789/0 conn(0x7f202c15e500 legacy=0x7f202c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f202a7fc640 1 -- 172.21.15.26:0/1618089014 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2193096138 0 0) 0x7f202c070f30 con 0x7f202c15e500 2024-04-24T23:25:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.617+0000 7f202bfff640 1 -- 172.21.15.26:0/1618089014 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c06e2b0 con 0x7f202c150e60 2024-04-24T23:25:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f202bfff640 1 --2- 172.21.15.26:0/1618089014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c150e60 0x7f202c1512b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f202c06efc0 tx=0x7f202402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e58c265b122c444 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f202a7fc640 1 -- 172.21.15.26:0/1618089014 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024007740 con 0x7f202c150e60 2024-04-24T23:25:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f202a7fc640 1 -- 172.21.15.26:0/1618089014 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20240078e0 con 0x7f202c150e60 2024-04-24T23:25:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f202a7fc640 1 -- 172.21.15.26:0/1618089014 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024007cb0 con 0x7f202c150e60 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f2031ebd640 1 -- 172.21.15.26:0/1618089014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c150e60 msgr2=0x7f202c1512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f2031ebd640 1 --2- 172.21.15.26:0/1618089014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c150e60 0x7f202c1512b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f202c06efc0 tx=0x7f202402a680 comp rx=0 tx=0).stop 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f2031ebd640 1 -- 172.21.15.26:0/1618089014 shutdown_connections 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f2031ebd640 1 --2- 172.21.15.26:0/1618089014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c150e60 0x7f202c1512b0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f2031ebd640 1 --2- 172.21.15.26:0/1618089014 >> v2:172.21.15.29:3300/0 conn(0x7f202c15e120 0x7f202c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f2031ebd640 1 -- 172.21.15.26:0/1618089014 >> 172.21.15.26:0/1618089014 conn(0x7f202c06b540 msgr2=0x7f202c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f2031ebd640 1 -- 172.21.15.26:0/1618089014 shutdown_connections 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.618+0000 7f2031ebd640 1 -- 172.21.15.26:0/1618089014 wait complete. 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.620+0000 7f2031ebd640 1 Processor -- start 2024-04-24T23:25:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.620+0000 7f2031ebd640 1 -- start start 2024-04-24T23:25:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.620+0000 7f2031ebd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f202c150e60 0x7f202c1492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.620+0000 7f2031ebd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202c15e120 0x7f202c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f2031ebd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c15e500 0x7f202c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f2031ebd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f202c0b3ef0 con 0x7f202c15e500 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f2031ebd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f202c15e970 con 0x7f202c15e120 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f2031ebd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f202c0cc1c0 con 0x7f202c150e60 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c15e500 0x7f202c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c15e500 0x7f202c15a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46538/0 (socket says 172.21.15.26:46538) 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202bfff640 1 -- 172.21.15.26:0/3166413915 learned_addr learned my addr 172.21.15.26:0/3166413915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202affd640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202c15e120 0x7f202c0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202b7fe640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f202c150e60 0x7f202c1492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202bfff640 1 -- 172.21.15.26:0/3166413915 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f202c150e60 msgr2=0x7f202c1492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202bfff640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f202c150e60 0x7f202c1492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202bfff640 1 -- 172.21.15.26:0/3166413915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202c15e120 msgr2=0x7f202c0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202bfff640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202c15e120 0x7f202c0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202bfff640 1 -- 172.21.15.26:0/3166413915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c0cc460 con 0x7f202c15e500 2024-04-24T23:25:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f202bfff640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c15e500 0x7f202c15a140 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f202402ac70 tx=0x7f2024004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f2013fff640 1 -- 172.21.15.26:0/3166413915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024040070 con 0x7f202c15e500 2024-04-24T23:25:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f2013fff640 1 -- 172.21.15.26:0/3166413915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2024031bc0 con 0x7f202c15e500 2024-04-24T23:25:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202c0cc740 con 0x7f202c15e500 2024-04-24T23:25:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.621+0000 7f2013fff640 1 -- 172.21.15.26:0/3166413915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024045460 con 0x7f202c15e500 2024-04-24T23:25:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.622+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202c0ccd00 con 0x7f202c15e500 2024-04-24T23:25:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.623+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202c0ccf00 con 0x7f202c15e500 2024-04-24T23:25:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.623+0000 7f2013fff640 1 -- 172.21.15.26:0/3166413915 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2024039040 con 0x7f202c15e500 2024-04-24T23:25:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.623+0000 7f2013fff640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f201806be30 0x7f201806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.623+0000 7f2013fff640 1 -- 172.21.15.26:0/3166413915 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f20240ac650 con 0x7f202c15e500 2024-04-24T23:25:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.624+0000 7f202b7fe640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f201806be30 0x7f201806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.624+0000 7f202b7fe640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f201806be30 0x7f201806e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f201c006fd0 tx=0x7f201c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.628+0000 7f2013fff640 1 -- 172.21.15.26:0/3166413915 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f202c0ccf00 con 0x7f202c15e500 2024-04-24T23:25:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.808+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f202c075610 con 0x7f202c15e500 2024-04-24T23:25:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.808+0000 7f2013fff640 1 -- 172.21.15.26:0/3166413915 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7f202407aad0 con 0x7f202c15e500 2024-04-24T23:25:11.810 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f201806be30 msgr2=0x7f201806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f201806be30 0x7f201806e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f201c006fd0 tx=0x7f201c008040 comp rx=0 tx=0).stop 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c15e500 msgr2=0x7f202c15a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c15e500 0x7f202c15a140 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f202402ac70 tx=0x7f2024004a40 comp rx=0 tx=0).stop 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 shutdown_connections 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f201806be30 0x7f201806e2f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f202c15e500 0x7f202c15a140 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f202c15e120 0x7f202c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 --2- 172.21.15.26:0/3166413915 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f202c150e60 0x7f202c1492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.810+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 >> 172.21.15.26:0/3166413915 conn(0x7f202c06b540 msgr2=0x7f202c0d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:11.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.811+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 shutdown_connections 2024-04-24T23:25:11.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.811+0000 7f2031ebd640 1 -- 172.21.15.26:0/3166413915 wait complete. 2024-04-24T23:25:11.822 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:25:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.923+0000 7fd0781ce640 1 Processor -- start 2024-04-24T23:25:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.923+0000 7fd0781ce640 1 -- start start 2024-04-24T23:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.924+0000 7fd0781ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd0700cf870 0x7fd070072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.924+0000 7fd0781ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd07015eb00 0x7fd07015ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.924+0000 7fd0781ce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0700cd0b0 con 0x7fd070143c40 2024-04-24T23:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.924+0000 7fd0781ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd070071b70 con 0x7fd0700cf870 2024-04-24T23:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.924+0000 7fd0781ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd07015f470 con 0x7fd07015eb00 2024-04-24T23:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.924+0000 7fd075769640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd0700cf870 0x7fd070072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.924+0000 7fd075769640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd0700cf870 0x7fd070072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46542/0 (socket says 172.21.15.26:46542) 2024-04-24T23:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.924+0000 7fd075769640 1 -- 172.21.15.26:0/3961918365 learned_addr learned my addr 172.21.15.26:0/3961918365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.924+0000 7fd07676b640 1 --2- 172.21.15.26:0/3961918365 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd07015eb00 0x7fd07015ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.925+0000 7fd075769640 1 -- 172.21.15.26:0/3961918365 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd07015eb00 msgr2=0x7fd07015ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.925+0000 7fd075769640 1 --2- 172.21.15.26:0/3961918365 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd07015eb00 0x7fd07015ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.925+0000 7fd075769640 1 -- 172.21.15.26:0/3961918365 >> v1:172.21.15.29:6789/0 conn(0x7fd070143c40 legacy=0x7fd0700cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.925+0000 7fd075769640 1 -- 172.21.15.26:0/3961918365 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd070071240 con 0x7fd0700cf870 2024-04-24T23:25:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.925+0000 7fd075769640 1 --2- 172.21.15.26:0/3961918365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 0x7fd070072000 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd06400bb10 tx=0x7fd06402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b9d6e698b262bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.925+0000 7fd074f68640 1 -- 172.21.15.26:0/3961918365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064036020 con 0x7fd0700cf870 2024-04-24T23:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.925+0000 7fd074f68640 1 -- 172.21.15.26:0/3961918365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd064032820 con 0x7fd0700cf870 2024-04-24T23:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.925+0000 7fd074f68640 1 -- 172.21.15.26:0/3961918365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064032bf0 con 0x7fd0700cf870 2024-04-24T23:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.926+0000 7fd0781ce640 1 -- 172.21.15.26:0/3961918365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 msgr2=0x7fd070072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.926+0000 7fd0781ce640 1 --2- 172.21.15.26:0/3961918365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 0x7fd070072000 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd06400bb10 tx=0x7fd06402c920 comp rx=0 tx=0).stop 2024-04-24T23:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.926+0000 7fd0781ce640 1 -- 172.21.15.26:0/3961918365 shutdown_connections 2024-04-24T23:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.926+0000 7fd0781ce640 1 --2- 172.21.15.26:0/3961918365 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd07015eb00 0x7fd07015ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.926+0000 7fd0781ce640 1 --2- 172.21.15.26:0/3961918365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 0x7fd070072000 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.926+0000 7fd0781ce640 1 -- 172.21.15.26:0/3961918365 >> 172.21.15.26:0/3961918365 conn(0x7fd07006b300 msgr2=0x7fd07006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.926+0000 7fd0781ce640 1 -- 172.21.15.26:0/3961918365 shutdown_connections 2024-04-24T23:25:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.926+0000 7fd0781ce640 1 -- 172.21.15.26:0/3961918365 wait complete. 2024-04-24T23:25:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.927+0000 7fd0781ce640 1 Processor -- start 2024-04-24T23:25:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.927+0000 7fd0781ce640 1 -- start start 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd0781ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 0x7fd07014d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd0781ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd070143c40 0x7fd07015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd0781ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07015eb00 0x7fd0700b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd0781ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd070144020 con 0x7fd0700cf870 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd0781ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd070071800 con 0x7fd07015eb00 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd0781ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd070072540 con 0x7fd070143c40 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd075f6a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 0x7fd07014d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd075f6a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 0x7fd07014d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46546/0 (socket says 172.21.15.26:46546) 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd075f6a640 1 -- 172.21.15.26:0/3594507450 learned_addr learned my addr 172.21.15.26:0/3594507450 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd07676b640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07015eb00 0x7fd0700b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd075f6a640 1 -- 172.21.15.26:0/3594507450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd070143c40 msgr2=0x7fd07015ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd075f6a640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd070143c40 0x7fd07015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd075f6a640 1 -- 172.21.15.26:0/3594507450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07015eb00 msgr2=0x7fd0700b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd075f6a640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07015eb00 0x7fd0700b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.928+0000 7fd075f6a640 1 -- 172.21.15.26:0/3594507450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0701e5e50 con 0x7fd0700cf870 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.929+0000 7fd075f6a640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 0x7fd07014d890 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd06000c790 tx=0x7fd06000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.929+0000 7fd06e7fc640 1 -- 172.21.15.26:0/3594507450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060016020 con 0x7fd0700cf870 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.929+0000 7fd06e7fc640 1 -- 172.21.15.26:0/3594507450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd060006d50 con 0x7fd0700cf870 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.929+0000 7fd06e7fc640 1 -- 172.21.15.26:0/3594507450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060003bb0 con 0x7fd0700cf870 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.929+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0701e4df0 con 0x7fd0700cf870 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.929+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0701e5370 con 0x7fd0700cf870 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.930+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0700694c0 con 0x7fd0700cf870 2024-04-24T23:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.930+0000 7fd06e7fc640 1 -- 172.21.15.26:0/3594507450 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd06000a850 con 0x7fd0700cf870 2024-04-24T23:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.931+0000 7fd06e7fc640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd05c06be30 0x7fd05c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.931+0000 7fd075769640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd05c06be30 0x7fd05c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.931+0000 7fd06e7fc640 1 -- 172.21.15.26:0/3594507450 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd06008abe0 con 0x7fd0700cf870 2024-04-24T23:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.931+0000 7fd075769640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd05c06be30 0x7fd05c06e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd064037000 tx=0x7fd064031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:11.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:11.935+0000 7fd06e7fc640 1 -- 172.21.15.26:0/3594507450 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd060020020 con 0x7fd0700cf870 2024-04-24T23:25:12.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.132+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd0701e5a70 con 0x7fd0700cf870 2024-04-24T23:25:12.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.133+0000 7fd06e7fc640 1 -- 172.21.15.26:0/3594507450 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7fd060019310 con 0x7fd0700cf870 2024-04-24T23:25:12.134 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:25:12.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd05c06be30 msgr2=0x7fd05c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd05c06be30 0x7fd05c06e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd064037000 tx=0x7fd064031000 comp rx=0 tx=0).stop 2024-04-24T23:25:12.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 msgr2=0x7fd07014d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 0x7fd07014d890 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd06000c790 tx=0x7fd06000cc50 comp rx=0 tx=0).stop 2024-04-24T23:25:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 shutdown_connections 2024-04-24T23:25:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd05c06be30 0x7fd05c06e2f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd07015eb00 0x7fd0700b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd070143c40 0x7fd07015ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 --2- 172.21.15.26:0/3594507450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0700cf870 0x7fd07014d890 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 >> 172.21.15.26:0/3594507450 conn(0x7fd07006b300 msgr2=0x7fd07013eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 shutdown_connections 2024-04-24T23:25:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.135+0000 7fd0781ce640 1 -- 172.21.15.26:0/3594507450 wait complete. 2024-04-24T23:25:12.147 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:25:12.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:12.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:25:12.176 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:25:12.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:12.176 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:25:12.231 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:25:12.231 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:25:12.231 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:12.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:25:12.250 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:25:12.250 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:25:12.250 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:12.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:25:12.286 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:25:12.286 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:25:12.286 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:12.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:25:12.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.426+0000 7f1d57e7d640 1 Processor -- start 2024-04-24T23:25:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.426+0000 7f1d57e7d640 1 -- start start 2024-04-24T23:25:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d57e7d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1d500adf80 0x7f1d50154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d57e7d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d500b0220 con 0x7f1d5006f4f0 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d57e7d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d500af180 con 0x7f1d5006f110 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d57e7d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1d500af460 con 0x7f1d500adf80 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d55418640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1d5006f4f0 0x7f1d500ad870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41510/0 (socket says 172.21.15.26:41510) 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d55418640 1 -- 172.21.15.26:0/2803881888 learned_addr learned my addr 172.21.15.26:0/2803881888 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3630138955 0 0) 0x7f1d500b0220 con 0x7f1d5006f4f0 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d3c003520 con 0x7f1d5006f4f0 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d5641a640 1 --2- 172.21.15.26:0/2803881888 >> v2:172.21.15.29:3300/0 conn(0x7f1d500adf80 0x7f1d50154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 808098417 0 0) 0x7f1d3c003520 con 0x7f1d5006f4f0 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 >> v2:172.21.15.29:3300/0 conn(0x7f1d500adf80 msgr2=0x7f1d50154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 --2- 172.21.15.26:0/2803881888 >> v2:172.21.15.29:3300/0 conn(0x7f1d500adf80 0x7f1d50154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 >> v1:172.21.15.29:6789/0 conn(0x7f1d5006f110 legacy=0x7f1d5015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d500af740 con 0x7f1d5006f4f0 2024-04-24T23:25:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f1d40001670 con 0x7f1d5006f4f0 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 >> v1:172.21.15.26:6789/0 conn(0x7f1d5006f4f0 legacy=0x7f1d500ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d3c0039b0 0x7f1d3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c004800 0x7f1d3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1d3c0051c0 0x7f1d3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d500af460 con 0x7f1d3c0039b0 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c00c160 con 0x7f1d3c004800 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c00c360 con 0x7f1d3c0051c0 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.427+0000 7f1d55c19640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d3c0039b0 0x7f1d3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.428+0000 7f1d55418640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c004800 0x7f1d3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.428+0000 7f1d5641a640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1d3c0051c0 0x7f1d3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.428+0000 7f1d55c19640 1 -- 172.21.15.26:0/2803881888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1d3c0051c0 msgr2=0x7f1d3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.428+0000 7f1d55c19640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1d3c0051c0 0x7f1d3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.428+0000 7f1d55c19640 1 -- 172.21.15.26:0/2803881888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c004800 msgr2=0x7f1d3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.428+0000 7f1d55c19640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d3c004800 0x7f1d3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.428+0000 7f1d5641a640 1 -- 172.21.15.26:0/2803881888 reap_dead start 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.428+0000 7f1d55c19640 1 -- 172.21.15.26:0/2803881888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c003520 con 0x7f1d3c0039b0 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d55c19640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d3c0039b0 0x7f1d3c003dd0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c001750 tx=0x7f1d3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4baf9099b7bfad1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c001370 con 0x7f1d3c0039b0 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4c0052d0 con 0x7f1d3c0039b0 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d54c17640 1 -- 172.21.15.26:0/2803881888 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c02abc0 con 0x7f1d3c0039b0 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2803881888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d3c0039b0 msgr2=0x7f1d3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d57e7d640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d3c0039b0 0x7f1d3c003dd0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c001750 tx=0x7f1d3c005830 comp rx=0 tx=0).stop 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2803881888 shutdown_connections 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d57e7d640 1 --2- 172.21.15.26:0/2803881888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d3c0039b0 0x7f1d3c003dd0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2803881888 >> 172.21.15.26:0/2803881888 conn(0x7f1d500729a0 msgr2=0x7f1d5006b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2803881888 shutdown_connections 2024-04-24T23:25:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.429+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2803881888 wait complete. 2024-04-24T23:25:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.430+0000 7f1d57e7d640 1 Processor -- start 2024-04-24T23:25:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d57e7d640 1 -- start start 2024-04-24T23:25:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d57e7d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1d3c0039b0 0x7f1d50161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d57e7d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d5014fea0 0x7f1d50156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d57e7d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d50156770 0x7f1d50156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d57e7d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d500aee50 con 0x7f1d50156770 2024-04-24T23:25:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d57e7d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d500afd00 con 0x7f1d5014fea0 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d57e7d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5006fb00 con 0x7f1d3c0039b0 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d5641a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d50156770 0x7f1d50156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d5641a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d50156770 0x7f1d50156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46560/0 (socket says 172.21.15.26:46560) 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d5641a640 1 -- 172.21.15.26:0/2244073685 learned_addr learned my addr 172.21.15.26:0/2244073685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d55418640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d5014fea0 0x7f1d50156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.431+0000 7f1d55c19640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1d3c0039b0 0x7f1d50161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d5641a640 1 -- 172.21.15.26:0/2244073685 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1d3c0039b0 msgr2=0x7f1d50161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d5641a640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1d3c0039b0 0x7f1d50161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d5641a640 1 -- 172.21.15.26:0/2244073685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d5014fea0 msgr2=0x7f1d50156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d5641a640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d5014fea0 0x7f1d50156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d5641a640 1 -- 172.21.15.26:0/2244073685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d50158160 con 0x7f1d50156770 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d5641a640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d50156770 0x7f1d50156bf0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1d500730a0 tx=0x7f1d480115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d467fc640 1 -- 172.21.15.26:0/2244073685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48011de0 con 0x7f1d50156770 2024-04-24T23:25:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d467fc640 1 -- 172.21.15.26:0/2244073685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4801b070 con 0x7f1d50156770 2024-04-24T23:25:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d50158440 con 0x7f1d50156770 2024-04-24T23:25:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d467fc640 1 -- 172.21.15.26:0/2244073685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48003990 con 0x7f1d50156770 2024-04-24T23:25:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.432+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d50158a00 con 0x7f1d50156770 2024-04-24T23:25:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.433+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d5006c460 con 0x7f1d50156770 2024-04-24T23:25:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.434+0000 7f1d467fc640 1 -- 172.21.15.26:0/2244073685 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d48003b30 con 0x7f1d50156770 2024-04-24T23:25:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.434+0000 7f1d467fc640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1d3406be30 0x7f1d3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.434+0000 7f1d467fc640 1 -- 172.21.15.26:0/2244073685 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1d4808b570 con 0x7f1d50156770 2024-04-24T23:25:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.434+0000 7f1d55c19640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1d3406be30 0x7f1d3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.435+0000 7f1d55c19640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1d3406be30 0x7f1d3406e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c032000 tx=0x7f1d4c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.438+0000 7f1d467fc640 1 -- 172.21.15.26:0/2244073685 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1d5006c460 con 0x7f1d50156770 2024-04-24T23:25:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.660+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d500694c0 con 0x7f1d50156770 2024-04-24T23:25:12.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.661+0000 7f1d467fc640 1 -- 172.21.15.26:0/2244073685 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5224 (secure 0 0 0) 0x7f1d48003e20 con 0x7f1d50156770 2024-04-24T23:25:12.662 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:12.662 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:24:14.713249+0000","modified":"2024-04-24T23:24:43.349267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6546,"mds_1":6534,"mds_2":6531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6531":{"gid":6531,"name":"c","rank":2,"incarnation":53,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6836/1519104007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1519104007},{"type":"v1","addr":"172.21.15.26:6836","nonce":1519104007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6534":{"gid":6534,"name":"a","rank":1,"incarnation":51,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6837/2743842996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2743842996},{"type":"v1","addr":"172.21.15.26:6837","nonce":2743842996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6546":{"gid":6546,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":10,"addr":"172.21.15.29:6835/2096523178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2096523178},{"type":"v1","addr":"172.21.15.29:6835","nonce":2096523178}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6531,6534,6546]},"id":4}]} 2024-04-24T23:25:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 58 2024-04-24T23:25:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1d3406be30 msgr2=0x7f1d3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1d3406be30 0x7f1d3406e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c032000 tx=0x7f1d4c02e000 comp rx=0 tx=0).stop 2024-04-24T23:25:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d50156770 msgr2=0x7f1d50156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d50156770 0x7f1d50156bf0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1d500730a0 tx=0x7f1d480115a0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 shutdown_connections 2024-04-24T23:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1d3406be30 0x7f1d3406e2f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d50156770 0x7f1d50156bf0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d5014fea0 0x7f1d50156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 --2- 172.21.15.26:0/2244073685 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1d3c0039b0 0x7f1d50161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 >> 172.21.15.26:0/2244073685 conn(0x7f1d500729a0 msgr2=0x7f1d500ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 shutdown_connections 2024-04-24T23:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.663+0000 7f1d57e7d640 1 -- 172.21.15.26:0/2244073685 wait complete. 2024-04-24T23:25:12.675 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:25:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.774+0000 7fb01a3d5640 1 Processor -- start 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.774+0000 7fb01a3d5640 1 -- start start 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.775+0000 7fb01a3d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0140d2ea0 0x7fb0140d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.775+0000 7fb01a3d5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0140b3fc0 con 0x7fb0141618b0 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.775+0000 7fb01a3d5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0140b42a0 con 0x7fb0140d43a0 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.775+0000 7fb01a3d5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb014161c90 con 0x7fb0140d2ea0 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.775+0000 7fb0137fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0140d2ea0 0x7fb0140d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.775+0000 7fb0137fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0140d2ea0 0x7fb0140d3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46564/0 (socket says 172.21.15.26:46564) 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.775+0000 7fb013fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb0141618b0 0x7fb0140d2790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41518/0 (socket says 172.21.15.26:41518) 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.775+0000 7fb0137fe640 1 -- 172.21.15.26:0/957919915 learned_addr learned my addr 172.21.15.26:0/957919915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.776+0000 7fb0137fe640 1 -- 172.21.15.26:0/957919915 >> v1:172.21.15.29:6789/0 conn(0x7fb0140d43a0 legacy=0x7fb0140cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.776+0000 7fb0137fe640 1 -- 172.21.15.26:0/957919915 >> v1:172.21.15.26:6789/0 conn(0x7fb0141618b0 legacy=0x7fb0140d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.776+0000 7fb0137fe640 1 -- 172.21.15.26:0/957919915 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0140b4580 con 0x7fb0140d2ea0 2024-04-24T23:25:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.776+0000 7fb012ffd640 1 -- 172.21.15.26:0/957919915 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3802487272 0 0) 0x7fb0140b3fc0 con 0x7fb0141618b0 2024-04-24T23:25:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.776+0000 7fb0137fe640 1 --2- 172.21.15.26:0/957919915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0140d2ea0 0x7fb0140d3e60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb008009870 tx=0x7fb00802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e621c0514b2ec89b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb012ffd640 1 -- 172.21.15.26:0/957919915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008037070 con 0x7fb0140d2ea0 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb012ffd640 1 -- 172.21.15.26:0/957919915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb008032440 con 0x7fb0140d2ea0 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb012ffd640 1 -- 172.21.15.26:0/957919915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008032810 con 0x7fb0140d2ea0 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb01a3d5640 1 -- 172.21.15.26:0/957919915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0140d2ea0 msgr2=0x7fb0140d3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb01a3d5640 1 --2- 172.21.15.26:0/957919915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0140d2ea0 0x7fb0140d3e60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb008009870 tx=0x7fb00802a2a0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb01a3d5640 1 -- 172.21.15.26:0/957919915 shutdown_connections 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb01a3d5640 1 --2- 172.21.15.26:0/957919915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0140d2ea0 0x7fb0140d3e60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb01a3d5640 1 -- 172.21.15.26:0/957919915 >> 172.21.15.26:0/957919915 conn(0x7fb01408dbd0 msgr2=0x7fb01409d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb01a3d5640 1 -- 172.21.15.26:0/957919915 shutdown_connections 2024-04-24T23:25:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.777+0000 7fb01a3d5640 1 -- 172.21.15.26:0/957919915 wait complete. 2024-04-24T23:25:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.778+0000 7fb01a3d5640 1 Processor -- start 2024-04-24T23:25:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb01a3d5640 1 -- start start 2024-04-24T23:25:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb01a3d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0140d2ea0 0x7fb0141f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb01a3d5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0140d43a0 0x7fb0140a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb01a3d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141618b0 0x7fb01415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb01a3d5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140d47d0 con 0x7fb0141618b0 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb01a3d5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140b4bd0 con 0x7fb0140d2ea0 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb01a3d5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb0140b5060 con 0x7fb0140d43a0 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb018972640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141618b0 0x7fb01415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb018972640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141618b0 0x7fb01415ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46580/0 (socket says 172.21.15.26:46580) 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.779+0000 7fb018972640 1 -- 172.21.15.26:0/2108280959 learned_addr learned my addr 172.21.15.26:0/2108280959 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb013fff640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0140d2ea0 0x7fb0141f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb0137fe640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0140d43a0 0x7fb0140a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb018972640 1 -- 172.21.15.26:0/2108280959 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0140d43a0 msgr2=0x7fb0140a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb018972640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0140d43a0 0x7fb0140a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb018972640 1 -- 172.21.15.26:0/2108280959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0140d2ea0 msgr2=0x7fb0141f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb018972640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0140d2ea0 0x7fb0141f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb018972640 1 -- 172.21.15.26:0/2108280959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01409e7f0 con 0x7fb0141618b0 2024-04-24T23:25:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb018972640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141618b0 0x7fb01415ee10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c780 tx=0x7fb00c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb010ff9640 1 -- 172.21.15.26:0/2108280959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c004b60 con 0x7fb0141618b0 2024-04-24T23:25:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb010ff9640 1 -- 172.21.15.26:0/2108280959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb00c004d00 con 0x7fb0141618b0 2024-04-24T23:25:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01409ead0 con 0x7fb0141618b0 2024-04-24T23:25:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb010ff9640 1 -- 172.21.15.26:0/2108280959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00aa90 con 0x7fb0141618b0 2024-04-24T23:25:12.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.780+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01409f090 con 0x7fb0141618b0 2024-04-24T23:25:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.781+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0140694c0 con 0x7fb0141618b0 2024-04-24T23:25:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.782+0000 7fb010ff9640 1 -- 172.21.15.26:0/2108280959 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb00c00ac30 con 0x7fb0141618b0 2024-04-24T23:25:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.782+0000 7fb010ff9640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb00006be30 0x7fb00006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.782+0000 7fb010ff9640 1 -- 172.21.15.26:0/2108280959 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb00c08b0d0 con 0x7fb0141618b0 2024-04-24T23:25:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.782+0000 7fb013fff640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb00006be30 0x7fb00006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:12.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.782+0000 7fb013fff640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb00006be30 0x7fb00006e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb004000f60 tx=0x7fb004009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:12.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.786+0000 7fb010ff9640 1 -- 172.21.15.26:0/2108280959 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb00c020070 con 0x7fb0141618b0 2024-04-24T23:25:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.973+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0140d1350 con 0x7fb0141618b0 2024-04-24T23:25:12.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.974+0000 7fb010ff9640 1 -- 172.21.15.26:0/2108280959 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7fb00c059b10 con 0x7fb0141618b0 2024-04-24T23:25:12.975 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:12.975 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:24:16.006495+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:24:13.001912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:24:13.678528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:25:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb00006be30 msgr2=0x7fb00006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb00006be30 0x7fb00006e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb004000f60 tx=0x7fb004009040 comp rx=0 tx=0).stop 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141618b0 msgr2=0x7fb01415ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141618b0 0x7fb01415ee10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c780 tx=0x7fb00c00cc40 comp rx=0 tx=0).stop 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 shutdown_connections 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb00006be30 0x7fb00006e2f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141618b0 0x7fb01415ee10 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0140d43a0 0x7fb0140a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 --2- 172.21.15.26:0/2108280959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0140d2ea0 0x7fb0141f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 >> 172.21.15.26:0/2108280959 conn(0x7fb01408dbd0 msgr2=0x7fb0140c9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 shutdown_connections 2024-04-24T23:25:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:12.976+0000 7fb01a3d5640 1 -- 172.21.15.26:0/2108280959 wait complete. 2024-04-24T23:25:12.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:24:13.001912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:24:13.678528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:25:12.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:25:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.092+0000 7f071f163640 1 Processor -- start 2024-04-24T23:25:13.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.092+0000 7f071f163640 1 -- start start 2024-04-24T23:25:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.092+0000 7f071f163640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f071815c300 0x7f07180cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.092+0000 7f071f163640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f07180cceb0 0x7f07180ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.092+0000 7f071f163640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07180cad40 con 0x7f07180cca30 2024-04-24T23:25:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.092+0000 7f071f163640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f071806ef10 con 0x7f071815c300 2024-04-24T23:25:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.092+0000 7f071f163640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f07180caa60 con 0x7f07180cceb0 2024-04-24T23:25:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.093+0000 7f071ceff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f071815c300 0x7f07180cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.093+0000 7f071ceff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f071815c300 0x7f07180cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46582/0 (socket says 172.21.15.26:46582) 2024-04-24T23:25:13.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.093+0000 7f071ceff640 1 -- 172.21.15.26:0/2571783759 learned_addr learned my addr 172.21.15.26:0/2571783759 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:13.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.093+0000 7f071d700640 1 --2- 172.21.15.26:0/2571783759 >> v2:172.21.15.29:3300/0 conn(0x7f07180cceb0 0x7f07180ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.093+0000 7f071ceff640 1 -- 172.21.15.26:0/2571783759 >> v2:172.21.15.29:3300/0 conn(0x7f07180cceb0 msgr2=0x7f07180ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:13.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.093+0000 7f071ceff640 1 --2- 172.21.15.26:0/2571783759 >> v2:172.21.15.29:3300/0 conn(0x7f07180cceb0 0x7f07180ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.093+0000 7f071ceff640 1 -- 172.21.15.26:0/2571783759 >> v1:172.21.15.29:6789/0 conn(0x7f07180cca30 legacy=0x7f07180cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:13.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.093+0000 7f071ceff640 1 -- 172.21.15.26:0/2571783759 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071806d420 con 0x7f071815c300 2024-04-24T23:25:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f071ceff640 1 --2- 172.21.15.26:0/2571783759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071815c300 0x7f07180cc190 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0718073670 tx=0x7f070802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce69846ba9466c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f07177fe640 1 -- 172.21.15.26:0/2571783759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708037070 con 0x7f071815c300 2024-04-24T23:25:13.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f07177fe640 1 -- 172.21.15.26:0/2571783759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0708032440 con 0x7f071815c300 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f07177fe640 1 -- 172.21.15.26:0/2571783759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708032810 con 0x7f071815c300 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f071f163640 1 -- 172.21.15.26:0/2571783759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071815c300 msgr2=0x7f07180cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f071f163640 1 --2- 172.21.15.26:0/2571783759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071815c300 0x7f07180cc190 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0718073670 tx=0x7f070802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f071f163640 1 -- 172.21.15.26:0/2571783759 shutdown_connections 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f071f163640 1 --2- 172.21.15.26:0/2571783759 >> v2:172.21.15.29:3300/0 conn(0x7f07180cceb0 0x7f07180ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f071f163640 1 --2- 172.21.15.26:0/2571783759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071815c300 0x7f07180cc190 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f071f163640 1 -- 172.21.15.26:0/2571783759 >> 172.21.15.26:0/2571783759 conn(0x7f07180b1c80 msgr2=0x7f07180b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f071f163640 1 -- 172.21.15.26:0/2571783759 shutdown_connections 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.094+0000 7f071f163640 1 -- 172.21.15.26:0/2571783759 wait complete. 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.095+0000 7f071f163640 1 Processor -- start 2024-04-24T23:25:13.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.096+0000 7f071f163640 1 -- start start 2024-04-24T23:25:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.096+0000 7f071f163640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07180cca30 0x7f071815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.096+0000 7f071f163640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07180cceb0 0x7f07181f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.096+0000 7f071f163640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f071815c300 0x7f07181f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.096+0000 7f071f163640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07180cd340 con 0x7f07180cca30 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.096+0000 7f071f163640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f071806d9e0 con 0x7f07180cceb0 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.096+0000 7f071f163640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f071806df00 con 0x7f071815c300 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.096+0000 7f071ceff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07180cca30 0x7f071815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071ceff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07180cca30 0x7f071815dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46594/0 (socket says 172.21.15.26:46594) 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071ceff640 1 -- 172.21.15.26:0/1220485228 learned_addr learned my addr 172.21.15.26:0/1220485228 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f0717fff640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07180cceb0 0x7f07181f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071d700640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f071815c300 0x7f07181f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071ceff640 1 -- 172.21.15.26:0/1220485228 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f071815c300 msgr2=0x7f07181f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071ceff640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f071815c300 0x7f07181f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071ceff640 1 -- 172.21.15.26:0/1220485228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07180cceb0 msgr2=0x7f07181f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071ceff640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07180cceb0 0x7f07181f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071ceff640 1 -- 172.21.15.26:0/1220485228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07181f53c0 con 0x7f07180cca30 2024-04-24T23:25:13.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071ceff640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07180cca30 0x7f071815dfd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0708000c00 tx=0x7f07080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:13.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f07157fa640 1 -- 172.21.15.26:0/1220485228 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708041020 con 0x7f07180cca30 2024-04-24T23:25:13.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f07157fa640 1 -- 172.21.15.26:0/1220485228 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07080306c0 con 0x7f07180cca30 2024-04-24T23:25:13.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071815e690 con 0x7f07180cca30 2024-04-24T23:25:13.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f07157fa640 1 -- 172.21.15.26:0/1220485228 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708037040 con 0x7f07180cca30 2024-04-24T23:25:13.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.097+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071815ec50 con 0x7f07180cca30 2024-04-24T23:25:13.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.098+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0718075610 con 0x7f07180cca30 2024-04-24T23:25:13.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.099+0000 7f07157fa640 1 -- 172.21.15.26:0/1220485228 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0708044020 con 0x7f07180cca30 2024-04-24T23:25:13.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.099+0000 7f07157fa640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f070406be30 0x7f070406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.099+0000 7f0717fff640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f070406be30 0x7f070406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.099+0000 7f07157fa640 1 -- 172.21.15.26:0/1220485228 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f07080ab610 con 0x7f07180cca30 2024-04-24T23:25:13.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.100+0000 7f0717fff640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f070406be30 0x7f070406e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f070c006fd0 tx=0x7f070c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:13.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.104+0000 7f07157fa640 1 -- 172.21.15.26:0/1220485228 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f070804b030 con 0x7f07180cca30 2024-04-24T23:25:13.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.284+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f071806e540 con 0x7f07180cca30 2024-04-24T23:25:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.285+0000 7f07157fa640 1 -- 172.21.15.26:0/1220485228 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f070807a050 con 0x7f07180cca30 2024-04-24T23:25:13.286 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:25:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.286+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f070406be30 msgr2=0x7f070406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.286+0000 7f071f163640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f070406be30 0x7f070406e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f070c006fd0 tx=0x7f070c008040 comp rx=0 tx=0).stop 2024-04-24T23:25:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.286+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07180cca30 msgr2=0x7f071815dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.286+0000 7f071f163640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07180cca30 0x7f071815dfd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0708000c00 tx=0x7f07080046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.287+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 shutdown_connections 2024-04-24T23:25:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.287+0000 7f071f163640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f070406be30 0x7f070406e2f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.287+0000 7f071f163640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f071815c300 0x7f07181f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.287+0000 7f071f163640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07180cceb0 0x7f07181f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.287+0000 7f071f163640 1 --2- 172.21.15.26:0/1220485228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07180cca30 0x7f071815dfd0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.287+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 >> 172.21.15.26:0/1220485228 conn(0x7f07180b1c80 msgr2=0x7f07180b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:13.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.287+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 shutdown_connections 2024-04-24T23:25:13.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.287+0000 7f071f163640 1 -- 172.21.15.26:0/1220485228 wait complete. 2024-04-24T23:25:13.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:25:13.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.401+0000 7f578cc1c640 1 Processor -- start 2024-04-24T23:25:13.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.401+0000 7f578cc1c640 1 -- start start 2024-04-24T23:25:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.402+0000 7f578cc1c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57880d0750 0x7f57880acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.402+0000 7f578cc1c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57880b4ce0 0x7f578815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.402+0000 7f578cc1c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57880aeb10 con 0x7f578815be40 2024-04-24T23:25:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.402+0000 7f578cc1c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f578809df20 con 0x7f57880b4ce0 2024-04-24T23:25:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.402+0000 7f578cc1c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f57880aedf0 con 0x7f57880d0750 2024-04-24T23:25:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f5786d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57880b4ce0 0x7f578815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f578659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f578815be40 0x7f57880d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41530/0 (socket says 172.21.15.26:41530) 2024-04-24T23:25:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f5786d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57880b4ce0 0x7f578815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46608/0 (socket says 172.21.15.26:46608) 2024-04-24T23:25:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f578659c640 1 -- 172.21.15.26:0/2954370132 learned_addr learned my addr 172.21.15.26:0/2954370132 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:13.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f5785d9b640 1 --2- 172.21.15.26:0/2954370132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57880d0750 0x7f57880acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f5786d9d640 1 -- 172.21.15.26:0/2954370132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57880d0750 msgr2=0x7f57880acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:13.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f5786d9d640 1 --2- 172.21.15.26:0/2954370132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57880d0750 0x7f57880acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f5786d9d640 1 -- 172.21.15.26:0/2954370132 >> v1:172.21.15.26:6789/0 conn(0x7f578815be40 legacy=0x7f57880d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f5786d9d640 1 -- 172.21.15.26:0/2954370132 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57880af0d0 con 0x7f57880b4ce0 2024-04-24T23:25:13.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.403+0000 7f578559a640 1 -- 172.21.15.26:0/2954370132 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2954143870 0 0) 0x7f57880aeb10 con 0x7f578815be40 2024-04-24T23:25:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f5786d9d640 1 --2- 172.21.15.26:0/2954370132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880b4ce0 0x7f578815c220 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f57800097a0 tx=0x7f578002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3297cbb4cd1f3884 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578559a640 1 -- 172.21.15.26:0/2954370132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780037070 con 0x7f57880b4ce0 2024-04-24T23:25:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578559a640 1 -- 172.21.15.26:0/2954370132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5780032440 con 0x7f57880b4ce0 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578559a640 1 -- 172.21.15.26:0/2954370132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780032810 con 0x7f57880b4ce0 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578cc1c640 1 -- 172.21.15.26:0/2954370132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880b4ce0 msgr2=0x7f578815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578cc1c640 1 --2- 172.21.15.26:0/2954370132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880b4ce0 0x7f578815c220 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f57800097a0 tx=0x7f578002a180 comp rx=0 tx=0).stop 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578cc1c640 1 -- 172.21.15.26:0/2954370132 shutdown_connections 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578cc1c640 1 --2- 172.21.15.26:0/2954370132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880b4ce0 0x7f578815c220 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578cc1c640 1 --2- 172.21.15.26:0/2954370132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57880d0750 0x7f57880acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578cc1c640 1 -- 172.21.15.26:0/2954370132 >> 172.21.15.26:0/2954370132 conn(0x7f578806b540 msgr2=0x7f578806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578cc1c640 1 -- 172.21.15.26:0/2954370132 shutdown_connections 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.404+0000 7f578cc1c640 1 -- 172.21.15.26:0/2954370132 wait complete. 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.406+0000 7f578cc1c640 1 Processor -- start 2024-04-24T23:25:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.406+0000 7f578cc1c640 1 -- start start 2024-04-24T23:25:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.406+0000 7f578cc1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57880b4ce0 0x7f57880cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.406+0000 7f578cc1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880d0750 0x7f578815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.406+0000 7f578cc1c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f578815be40 0x7f57880c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.406+0000 7f578cc1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f578806cd10 con 0x7f57880d0750 2024-04-24T23:25:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.406+0000 7f578cc1c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f578806cb90 con 0x7f57880b4ce0 2024-04-24T23:25:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.406+0000 7f578cc1c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f578815c760 con 0x7f578815be40 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5785d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880d0750 0x7f578815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5785d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880d0750 0x7f578815cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46622/0 (socket says 172.21.15.26:46622) 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5785d9b640 1 -- 172.21.15.26:0/2065948421 learned_addr learned my addr 172.21.15.26:0/2065948421 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f578659c640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57880b4ce0 0x7f57880cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5786d9d640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f578815be40 0x7f57880c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5785d9b640 1 -- 172.21.15.26:0/2065948421 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f578815be40 msgr2=0x7f57880c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5785d9b640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f578815be40 0x7f57880c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5785d9b640 1 -- 172.21.15.26:0/2065948421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57880b4ce0 msgr2=0x7f57880cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5785d9b640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57880b4ce0 0x7f57880cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5785d9b640 1 -- 172.21.15.26:0/2065948421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578815d660 con 0x7f57880d0750 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5785d9b640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880d0750 0x7f578815cba0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c7f0 tx=0x7f577c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5776ffd640 1 -- 172.21.15.26:0/2065948421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c015070 con 0x7f57880d0750 2024-04-24T23:25:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5776ffd640 1 -- 172.21.15.26:0/2065948421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c004240 con 0x7f57880d0750 2024-04-24T23:25:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578815d940 con 0x7f57880d0750 2024-04-24T23:25:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f5776ffd640 1 -- 172.21.15.26:0/2065948421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c007620 con 0x7f57880d0750 2024-04-24T23:25:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.407+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578815e1e0 con 0x7f57880d0750 2024-04-24T23:25:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.408+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f578815db10 con 0x7f57880d0750 2024-04-24T23:25:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.409+0000 7f5776ffd640 1 -- 172.21.15.26:0/2065948421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f577c0079d0 con 0x7f57880d0750 2024-04-24T23:25:13.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.409+0000 7f5776ffd640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f576806be30 0x7f576806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.409+0000 7f5776ffd640 1 -- 172.21.15.26:0/2065948421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f577c08ae20 con 0x7f57880d0750 2024-04-24T23:25:13.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.409+0000 7f578659c640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f576806be30 0x7f576806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.410+0000 7f578659c640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f576806be30 0x7f576806e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57780012e0 tx=0x7f5778001630 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:13.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.413+0000 7f5776ffd640 1 -- 172.21.15.26:0/2065948421 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f577c059210 con 0x7f57880d0750 2024-04-24T23:25:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.597+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f578815e3b0 con 0x7f57880d0750 2024-04-24T23:25:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.598+0000 7f5776ffd640 1 -- 172.21.15.26:0/2065948421 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7f577c05d510 con 0x7f57880d0750 2024-04-24T23:25:13.599 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:25:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f576806be30 msgr2=0x7f576806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f576806be30 0x7f576806e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57780012e0 tx=0x7f5778001630 comp rx=0 tx=0).stop 2024-04-24T23:25:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880d0750 msgr2=0x7f578815cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880d0750 0x7f578815cba0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c7f0 tx=0x7f577c00ccb0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 shutdown_connections 2024-04-24T23:25:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f576806be30 0x7f576806e2f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f578815be40 0x7f57880c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880d0750 0x7f578815cba0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 --2- 172.21.15.26:0/2065948421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57880b4ce0 0x7f57880cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 >> 172.21.15.26:0/2065948421 conn(0x7f578806b540 msgr2=0x7f57880d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 shutdown_connections 2024-04-24T23:25:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.600+0000 7f578cc1c640 1 -- 172.21.15.26:0/2065948421 wait complete. 2024-04-24T23:25:13.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:25:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.715+0000 7f6d76258640 1 Processor -- start 2024-04-24T23:25:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.715+0000 7f6d76258640 1 -- start start 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d76258640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d700b4ce0 0x7f6d700acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d76258640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d700aeb10 con 0x7f6d700d0750 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d76258640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d700aedf0 con 0x7f6d7015be40 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d76258640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6d7009df20 con 0x7f6d700b4ce0 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6d700d0750 0x7f6d7015c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42728/0 (socket says 172.21.15.26:42728) 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6effd640 1 -- 172.21.15.26:0/3274500654 learned_addr learned my addr 172.21.15.26:0/3274500654 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6ffff640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d700b4ce0 0x7f6d700acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049246381 0 0) 0x7f6d700aeb10 con 0x7f6d700d0750 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d58003520 con 0x7f6d700d0750 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2373352145 0 0) 0x7f6d58003520 con 0x7f6d700d0750 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d700b4ce0 msgr2=0x7f6d700acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:13.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d700b4ce0 0x7f6d700acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 >> v1:172.21.15.29:6789/0 conn(0x7f6d7015be40 legacy=0x7f6d700d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d700af0d0 con 0x7f6d700d0750 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f6d60001670 con 0x7f6d700d0750 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 >> v1:172.21.15.26:6789/0 conn(0x7f6d700d0750 legacy=0x7f6d7015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d58004390 0x7f6d58004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d700b4ce0 0x7f6d580050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5800b9c0 con 0x7f6d580039e0 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5800bb40 con 0x7f6d58004390 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.716+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5800bd40 con 0x7f6d700b4ce0 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.717+0000 7f6d6f7fe640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.717+0000 7f6d6effd640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d58004390 0x7f6d58004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.717+0000 7f6d6ffff640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d700b4ce0 0x7f6d580050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.717+0000 7f6d6f7fe640 1 -- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d700b4ce0 msgr2=0x7f6d580050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.717+0000 7f6d6f7fe640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d700b4ce0 0x7f6d580050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.717+0000 7f6d6f7fe640 1 -- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d58004390 msgr2=0x7f6d58004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.717+0000 7f6d6f7fe640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d58004390 0x7f6d58004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.717+0000 7f6d6f7fe640 1 -- 172.21.15.26:0/3274500654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d58003520 con 0x7f6d580039e0 2024-04-24T23:25:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.717+0000 7f6d6effd640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d58004390 0x7f6d58004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d6f7fe640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d58003dc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6d64001910 tx=0x7f6d58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745b645ae6c752e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64037070 con 0x7f6d580039e0 2024-04-24T23:25:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d6402ac60 con 0x7f6d580039e0 2024-04-24T23:25:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d6e7fc640 1 -- 172.21.15.26:0/3274500654 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640014e0 con 0x7f6d580039e0 2024-04-24T23:25:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d76258640 1 -- 172.21.15.26:0/3274500654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 msgr2=0x7f6d58003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d76258640 1 --2- 172.21.15.26:0/3274500654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d58003dc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6d64001910 tx=0x7f6d58005830 comp rx=0 tx=0).stop 2024-04-24T23:25:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d6ffff640 1 -- 172.21.15.26:0/3274500654 reap_dead start 2024-04-24T23:25:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d76258640 1 -- 172.21.15.26:0/3274500654 shutdown_connections 2024-04-24T23:25:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d76258640 1 -- 172.21.15.26:0/3274500654 >> 172.21.15.26:0/3274500654 conn(0x7f6d7006b540 msgr2=0x7f6d7006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d76258640 1 -- 172.21.15.26:0/3274500654 shutdown_connections 2024-04-24T23:25:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.718+0000 7f6d76258640 1 -- 172.21.15.26:0/3274500654 wait complete. 2024-04-24T23:25:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.719+0000 7f6d76258640 1 Processor -- start 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d76258640 1 -- start start 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d76258640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d70070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d76258640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d70071350 0x7f6d70152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d76258640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d70153040 0x7f6d701f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d76258640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7006cd10 con 0x7f6d580039e0 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d76258640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7006cb90 con 0x7f6d70071350 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d76258640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6d700acd30 con 0x7f6d70153040 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d70070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d6f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d70070e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53650/0 (socket says 172.21.15.26:53650) 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.720+0000 7f6d6f7fe640 1 -- 172.21.15.26:0/916333543 learned_addr learned my addr 172.21.15.26:0/916333543 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d6ffff640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d70153040 0x7f6d701f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d6effd640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d70071350 0x7f6d70152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d6f7fe640 1 -- 172.21.15.26:0/916333543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d70153040 msgr2=0x7f6d701f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d6f7fe640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d70153040 0x7f6d701f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d6f7fe640 1 -- 172.21.15.26:0/916333543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d70071350 msgr2=0x7f6d70152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d6f7fe640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d70071350 0x7f6d70152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d6f7fe640 1 -- 172.21.15.26:0/916333543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d70071890 con 0x7f6d580039e0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d6effd640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d70071350 0x7f6d70152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d6f7fe640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d70070e10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6d6402a750 tx=0x7f6d640012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d57fff640 1 -- 172.21.15.26:0/916333543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64037040 con 0x7f6d580039e0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d57fff640 1 -- 172.21.15.26:0/916333543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d640052e0 con 0x7f6d580039e0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d57fff640 1 -- 172.21.15.26:0/916333543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64007240 con 0x7f6d580039e0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d70071b70 con 0x7f6d580039e0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.721+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d70072130 con 0x7f6d580039e0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.722+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d70075610 con 0x7f6d580039e0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.723+0000 7f6d57fff640 1 -- 172.21.15.26:0/916333543 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d6403a070 con 0x7f6d580039e0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.723+0000 7f6d57fff640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5c06be60 0x7f6d5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.723+0000 7f6d57fff640 1 -- 172.21.15.26:0/916333543 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6d640aad60 con 0x7f6d580039e0 2024-04-24T23:25:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.723+0000 7f6d6effd640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5c06be60 0x7f6d5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.724+0000 7f6d6effd640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5c06be60 0x7f6d5c06e320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6d70072ab0 tx=0x7f6d60001de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.727+0000 7f6d57fff640 1 -- 172.21.15.26:0/916333543 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d64079710 con 0x7f6d580039e0 2024-04-24T23:25:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.909+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6d70158090 con 0x7f6d580039e0 2024-04-24T23:25:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.909+0000 7f6d57fff640 1 -- 172.21.15.26:0/916333543 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7f6d64079010 con 0x7f6d580039e0 2024-04-24T23:25:13.910 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:25:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5c06be60 msgr2=0x7f6d5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5c06be60 0x7f6d5c06e320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6d70072ab0 tx=0x7f6d60001de0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 msgr2=0x7f6d70070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d70070e10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6d6402a750 tx=0x7f6d640012b0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 shutdown_connections 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d5c06be60 0x7f6d5c06e320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d70153040 0x7f6d701f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d70071350 0x7f6d70152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 --2- 172.21.15.26:0/916333543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d580039e0 0x7f6d70070e10 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 >> 172.21.15.26:0/916333543 conn(0x7f6d7006b540 msgr2=0x7f6d7015a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 shutdown_connections 2024-04-24T23:25:13.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:13.911+0000 7f6d76258640 1 -- 172.21.15.26:0/916333543 wait complete. 2024-04-24T23:25:13.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.022+0000 7fc9a9163640 1 Processor -- start 2024-04-24T23:25:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.022+0000 7fc9a9163640 1 -- start start 2024-04-24T23:25:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a9163640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9a415e570 0x7fc9a40d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a9163640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a40d32f0 con 0x7fc9a40d22e0 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a9163640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a40d1600 con 0x7fc9a40b3130 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a9163640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc9a40d1ab0 con 0x7fc9a415e570 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a359e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc9a40d22e0 0x7fc9a40d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42730/0 (socket says 172.21.15.26:42730) 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a359e640 1 -- 172.21.15.26:0/566672946 learned_addr learned my addr 172.21.15.26:0/566672946 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a2d9d640 1 --2- 172.21.15.26:0/566672946 >> v2:172.21.15.29:3300/0 conn(0x7fc9a415e570 0x7fc9a40d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 675368280 0 0) 0x7fc99c001490 con 0x7fc9a40d22e0 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc98c003520 con 0x7fc9a40d22e0 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2927757645 0 0) 0x7fc98c003520 con 0x7fc9a40d22e0 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 >> v2:172.21.15.29:3300/0 conn(0x7fc9a415e570 msgr2=0x7fc9a40d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a1d9b640 1 --2- 172.21.15.26:0/566672946 >> v2:172.21.15.29:3300/0 conn(0x7fc9a415e570 0x7fc9a40d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 >> v1:172.21.15.29:6789/0 conn(0x7fc9a40b3130 legacy=0x7fc9a40d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.023+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a40d0270 con 0x7fc9a40d22e0 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fc99c001bc0 con 0x7fc9a40d22e0 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 >> v1:172.21.15.26:6789/0 conn(0x7fc9a40d22e0 legacy=0x7fc9a40d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a1d9b640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc98c0039b0 0x7fc98c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a1d9b640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc98c004800 0x7fc98c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a1d9b640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc98c0051c0 0x7fc98c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a40d1ab0 con 0x7fc98c0051c0 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c00c060 con 0x7fc98c0039b0 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc98c00c260 con 0x7fc98c004800 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a359e640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc98c0051c0 0x7fc98c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a2d9d640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc98c0039b0 0x7fc98c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a259c640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc98c004800 0x7fc98c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a359e640 1 -- 172.21.15.26:0/566672946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc98c004800 msgr2=0x7fc98c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a359e640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc98c004800 0x7fc98c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a359e640 1 -- 172.21.15.26:0/566672946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc98c0039b0 msgr2=0x7fc98c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a359e640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc98c0039b0 0x7fc98c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.024+0000 7fc9a359e640 1 -- 172.21.15.26:0/566672946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98c003520 con 0x7fc98c0051c0 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a359e640 1 -- 172.21.15.26:0/566672946 reap_dead start 2024-04-24T23:25:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a359e640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc98c0051c0 0x7fc98c00b9c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00aa30 tx=0x7fc98c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9bee41f002078 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c00a650 con 0x7fc98c0051c0 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc99c02f070 con 0x7fc98c0051c0 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a1d9b640 1 -- 172.21.15.26:0/566672946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c034470 con 0x7fc98c0051c0 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a9163640 1 -- 172.21.15.26:0/566672946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc98c0051c0 msgr2=0x7fc98c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a9163640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc98c0051c0 0x7fc98c00b9c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00aa30 tx=0x7fc98c005830 comp rx=0 tx=0).stop 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a9163640 1 -- 172.21.15.26:0/566672946 shutdown_connections 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a9163640 1 --2- 172.21.15.26:0/566672946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc98c0051c0 0x7fc98c00b9c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a9163640 1 -- 172.21.15.26:0/566672946 >> 172.21.15.26:0/566672946 conn(0x7fc9a408dbe0 msgr2=0x7fc9a409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.025+0000 7fc9a9163640 1 -- 172.21.15.26:0/566672946 shutdown_connections 2024-04-24T23:25:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.026+0000 7fc9a9163640 1 -- 172.21.15.26:0/566672946 wait complete. 2024-04-24T23:25:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.027+0000 7fc9a9163640 1 Processor -- start 2024-04-24T23:25:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.027+0000 7fc9a9163640 1 -- start start 2024-04-24T23:25:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a9163640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc98c0051c0 0x7fc9a409e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a9163640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9a4160420 0x7fc9a4160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a9163640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a40a0d50 0x7fc9a40a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a9163640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a40d3010 con 0x7fc9a40a0d50 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a9163640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a40d2ae0 con 0x7fc9a4160420 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a9163640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a40cfa70 con 0x7fc98c0051c0 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a40a0d50 0x7fc9a40a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a40a0d50 0x7fc9a40a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53670/0 (socket says 172.21.15.26:53670) 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a359e640 1 -- 172.21.15.26:0/351139472 learned_addr learned my addr 172.21.15.26:0/351139472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a259c640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9a4160420 0x7fc9a4160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a2d9d640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc98c0051c0 0x7fc9a409e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a359e640 1 -- 172.21.15.26:0/351139472 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc98c0051c0 msgr2=0x7fc9a409e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a359e640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc98c0051c0 0x7fc9a409e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a359e640 1 -- 172.21.15.26:0/351139472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9a4160420 msgr2=0x7fc9a4160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a359e640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9a4160420 0x7fc9a4160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a359e640 1 -- 172.21.15.26:0/351139472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a4161190 con 0x7fc9a40a0d50 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.028+0000 7fc9a359e640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a40a0d50 0x7fc9a40a11d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00ab50 tx=0x7fc9a40d0440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.029+0000 7fc98b7fe640 1 -- 172.21.15.26:0/351139472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c00a650 con 0x7fc9a40a0d50 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.029+0000 7fc98b7fe640 1 -- 172.21.15.26:0/351139472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc99c001490 con 0x7fc9a40a0d50 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.029+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a40a0270 con 0x7fc9a40a0d50 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.029+0000 7fc98b7fe640 1 -- 172.21.15.26:0/351139472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c02be60 con 0x7fc9a40a0d50 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.029+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a40a0830 con 0x7fc9a40a0d50 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.029+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a40694c0 con 0x7fc9a40a0d50 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.030+0000 7fc98b7fe640 1 -- 172.21.15.26:0/351139472 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc99c034470 con 0x7fc9a40a0d50 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.030+0000 7fc98b7fe640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc99006be30 0x7fc99006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.031+0000 7fc98b7fe640 1 -- 172.21.15.26:0/351139472 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc99c0ab3d0 con 0x7fc9a40a0d50 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.031+0000 7fc9a2d9d640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc99006be30 0x7fc99006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.031+0000 7fc9a2d9d640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc99006be30 0x7fc99006e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc998009980 tx=0x7fc998008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:14.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.035+0000 7fc98b7fe640 1 -- 172.21.15.26:0/351139472 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc9a40d32f0 con 0x7fc9a40a0d50 2024-04-24T23:25:14.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.255+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9a40a0a00 con 0x7fc9a40a0d50 2024-04-24T23:25:14.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.255+0000 7fc98b7fe640 1 -- 172.21.15.26:0/351139472 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5224 (secure 0 0 0) 0x7fc99c079cf0 con 0x7fc9a40a0d50 2024-04-24T23:25:14.256 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:14.257 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:24:14.713249+0000","modified":"2024-04-24T23:24:43.349267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6546,"mds_1":6534,"mds_2":6531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6531":{"gid":6531,"name":"c","rank":2,"incarnation":53,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6836/1519104007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1519104007},{"type":"v1","addr":"172.21.15.26:6836","nonce":1519104007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6534":{"gid":6534,"name":"a","rank":1,"incarnation":51,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6837/2743842996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2743842996},{"type":"v1","addr":"172.21.15.26:6837","nonce":2743842996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6546":{"gid":6546,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":10,"addr":"172.21.15.29:6835/2096523178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2096523178},{"type":"v1","addr":"172.21.15.29:6835","nonce":2096523178}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6531,6534,6546]},"id":4}]} 2024-04-24T23:25:14.257 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 58 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc99006be30 msgr2=0x7fc99006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc99006be30 0x7fc99006e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc998009980 tx=0x7fc998008040 comp rx=0 tx=0).stop 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a40a0d50 msgr2=0x7fc9a40a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a40a0d50 0x7fc9a40a11d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00ab50 tx=0x7fc9a40d0440 comp rx=0 tx=0).stop 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 shutdown_connections 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc99006be30 0x7fc99006e2f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a40a0d50 0x7fc9a40a11d0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc9a4160420 0x7fc9a4160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 --2- 172.21.15.26:0/351139472 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc98c0051c0 0x7fc9a409e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 >> 172.21.15.26:0/351139472 conn(0x7fc9a408dbe0 msgr2=0x7fc9a40b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 shutdown_connections 2024-04-24T23:25:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.257+0000 7fc9a9163640 1 -- 172.21.15.26:0/351139472 wait complete. 2024-04-24T23:25:14.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:25:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.369+0000 7efda2807640 1 Processor -- start 2024-04-24T23:25:14.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.369+0000 7efda2807640 1 -- start start 2024-04-24T23:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.370+0000 7efda2807640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd9c15e570 0x7efd9c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.370+0000 7efda2807640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c0b3130 0x7efd9c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.370+0000 7efda2807640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd9c0d32f0 con 0x7efd9c0d22e0 2024-04-24T23:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.370+0000 7efda2807640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7efd9c0d1ab0 con 0x7efd9c15e570 2024-04-24T23:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.370+0000 7efda2807640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efd9c0d2ae0 con 0x7efd9c0b3130 2024-04-24T23:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.370+0000 7efd9bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd9c15e570 0x7efd9c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.370+0000 7efd9bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd9c15e570 0x7efd9c0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53676/0 (socket says 172.21.15.26:53676) 2024-04-24T23:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.371+0000 7efd9bfff640 1 -- 172.21.15.26:0/1673500453 learned_addr learned my addr 172.21.15.26:0/1673500453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.371+0000 7efd9b7fe640 1 --2- 172.21.15.26:0/1673500453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c0b3130 0x7efd9c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.371+0000 7efd9bfff640 1 -- 172.21.15.26:0/1673500453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c0b3130 msgr2=0x7efd9c0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.371+0000 7efd9bfff640 1 --2- 172.21.15.26:0/1673500453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c0b3130 0x7efd9c0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.371+0000 7efd9bfff640 1 -- 172.21.15.26:0/1673500453 >> v1:172.21.15.29:6789/0 conn(0x7efd9c0d22e0 legacy=0x7efd9c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.371+0000 7efd9bfff640 1 -- 172.21.15.26:0/1673500453 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c0d0270 con 0x7efd9c15e570 2024-04-24T23:25:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.371+0000 7efd9bfff640 1 --2- 172.21.15.26:0/1673500453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c15e570 0x7efd9c0d4160 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009870 tx=0x7efd8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ef4252bc1b652f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efd9affd640 1 -- 172.21.15.26:0/1673500453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c037070 con 0x7efd9c15e570 2024-04-24T23:25:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efd9affd640 1 -- 172.21.15.26:0/1673500453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c032440 con 0x7efd9c15e570 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efd9affd640 1 -- 172.21.15.26:0/1673500453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c032810 con 0x7efd9c15e570 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efda2807640 1 -- 172.21.15.26:0/1673500453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c15e570 msgr2=0x7efd9c0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efda2807640 1 --2- 172.21.15.26:0/1673500453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c15e570 0x7efd9c0d4160 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009870 tx=0x7efd8c02a270 comp rx=0 tx=0).stop 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efda2807640 1 -- 172.21.15.26:0/1673500453 shutdown_connections 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efda2807640 1 --2- 172.21.15.26:0/1673500453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c0b3130 0x7efd9c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efda2807640 1 --2- 172.21.15.26:0/1673500453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c15e570 0x7efd9c0d4160 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efda2807640 1 -- 172.21.15.26:0/1673500453 >> 172.21.15.26:0/1673500453 conn(0x7efd9c08dbe0 msgr2=0x7efd9c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efda2807640 1 -- 172.21.15.26:0/1673500453 shutdown_connections 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.372+0000 7efda2807640 1 -- 172.21.15.26:0/1673500453 wait complete. 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.373+0000 7efda2807640 1 Processor -- start 2024-04-24T23:25:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.374+0000 7efda2807640 1 -- start start 2024-04-24T23:25:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.374+0000 7efda2807640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd9c0b3130 0x7efd9c157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.374+0000 7efda2807640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c0d22e0 0x7efd9c0b9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.374+0000 7efda2807640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c15e570 0x7efd9c0b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.374+0000 7efda2807640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c0cfa70 con 0x7efd9c0d22e0 2024-04-24T23:25:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.374+0000 7efda2807640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c0d3010 con 0x7efd9c0b3130 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.374+0000 7efda2807640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efd9c0cfd50 con 0x7efd9c15e570 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.374+0000 7efd9b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c0d22e0 0x7efd9c0b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd9b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c0d22e0 0x7efd9c0b9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53684/0 (socket says 172.21.15.26:53684) 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd9b7fe640 1 -- 172.21.15.26:0/346421707 learned_addr learned my addr 172.21.15.26:0/346421707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd9bfff640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd9c0b3130 0x7efd9c157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efda0da4640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c15e570 0x7efd9c0b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd9b7fe640 1 -- 172.21.15.26:0/346421707 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c15e570 msgr2=0x7efd9c0b98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd9b7fe640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c15e570 0x7efd9c0b98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd9b7fe640 1 -- 172.21.15.26:0/346421707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd9c0b3130 msgr2=0x7efd9c157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd9b7fe640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd9c0b3130 0x7efd9c157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd9b7fe640 1 -- 172.21.15.26:0/346421707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c15c2e0 con 0x7efd9c0d22e0 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd9b7fe640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c0d22e0 0x7efd9c0b9390 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efd9000e9e0 tx=0x7efd9000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd98ff9640 1 -- 172.21.15.26:0/346421707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90017070 con 0x7efd9c0d22e0 2024-04-24T23:25:14.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd98ff9640 1 -- 172.21.15.26:0/346421707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9000c9d0 con 0x7efd9c0d22e0 2024-04-24T23:25:14.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9c0b85d0 con 0x7efd9c0d22e0 2024-04-24T23:25:14.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efd98ff9640 1 -- 172.21.15.26:0/346421707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9000cdc0 con 0x7efd9c0d22e0 2024-04-24T23:25:14.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.375+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9c0b8b90 con 0x7efd9c0d22e0 2024-04-24T23:25:14.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.376+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9c0672e0 con 0x7efd9c0d22e0 2024-04-24T23:25:14.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.377+0000 7efd98ff9640 1 -- 172.21.15.26:0/346421707 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd9001a070 con 0x7efd9c0d22e0 2024-04-24T23:25:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.377+0000 7efd98ff9640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd8806be30 0x7efd8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.377+0000 7efd9bfff640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd8806be30 0x7efd8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.377+0000 7efd98ff9640 1 -- 172.21.15.26:0/346421707 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7efd9008ad50 con 0x7efd9c0d22e0 2024-04-24T23:25:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.378+0000 7efd9bfff640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd8806be30 0x7efd8806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efd8c034000 tx=0x7efd8c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.381+0000 7efd98ff9640 1 -- 172.21.15.26:0/346421707 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd9c0672e0 con 0x7efd9c0d22e0 2024-04-24T23:25:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.570+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd9c0694c0 con 0x7efd9c0d22e0 2024-04-24T23:25:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.571+0000 7efd98ff9640 1 -- 172.21.15.26:0/346421707 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7efd9008e030 con 0x7efd9c0d22e0 2024-04-24T23:25:14.572 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:14.572 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:24:16.006495+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:24:13.001912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:24:13.678528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:25:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd8806be30 msgr2=0x7efd8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd8806be30 0x7efd8806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efd8c034000 tx=0x7efd8c02e000 comp rx=0 tx=0).stop 2024-04-24T23:25:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c0d22e0 msgr2=0x7efd9c0b9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c0d22e0 0x7efd9c0b9390 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efd9000e9e0 tx=0x7efd9000eea0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 shutdown_connections 2024-04-24T23:25:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd8806be30 0x7efd8806e2f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd9c15e570 0x7efd9c0b98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd9c0d22e0 0x7efd9c0b9390 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 --2- 172.21.15.26:0/346421707 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd9c0b3130 0x7efd9c157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 >> 172.21.15.26:0/346421707 conn(0x7efd9c08dbe0 msgr2=0x7efd9c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 shutdown_connections 2024-04-24T23:25:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.573+0000 7efda2807640 1 -- 172.21.15.26:0/346421707 wait complete. 2024-04-24T23:25:14.585 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:25:14.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.685+0000 7fbcb90bd640 1 Processor -- start 2024-04-24T23:25:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.685+0000 7fbcb90bd640 1 -- start start 2024-04-24T23:25:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.685+0000 7fbcb90bd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbcb4141010 0x7fbcb4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.685+0000 7fbcb90bd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbcb406ea50 0x7fbcb41622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.685+0000 7fbcb90bd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb4073250 con 0x7fbcb4161e20 2024-04-24T23:25:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.685+0000 7fbcb90bd640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb4071f90 con 0x7fbcb406ea50 2024-04-24T23:25:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.685+0000 7fbcb90bd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb406f0b0 con 0x7fbcb4141010 2024-04-24T23:25:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.686+0000 7fbcb359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbcb406ea50 0x7fbcb41622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.686+0000 7fbcb359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbcb406ea50 0x7fbcb41622d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53690/0 (socket says 172.21.15.26:53690) 2024-04-24T23:25:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.686+0000 7fbcb359e640 1 -- 172.21.15.26:0/959852524 learned_addr learned my addr 172.21.15.26:0/959852524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.686+0000 7fbcb2d9d640 1 --2- 172.21.15.26:0/959852524 >> v2:172.21.15.29:3300/0 conn(0x7fbcb4141010 0x7fbcb4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.686+0000 7fbcb359e640 1 -- 172.21.15.26:0/959852524 >> v2:172.21.15.29:3300/0 conn(0x7fbcb4141010 msgr2=0x7fbcb4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.686+0000 7fbcb359e640 1 --2- 172.21.15.26:0/959852524 >> v2:172.21.15.29:3300/0 conn(0x7fbcb4141010 0x7fbcb4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.686+0000 7fbcb359e640 1 -- 172.21.15.26:0/959852524 >> v1:172.21.15.29:6789/0 conn(0x7fbcb4161e20 legacy=0x7fbcb406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.686+0000 7fbcb359e640 1 -- 172.21.15.26:0/959852524 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb4072230 con 0x7fbcb406ea50 2024-04-24T23:25:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb359e640 1 --2- 172.21.15.26:0/959852524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 0x7fbcb41622d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009e50 tx=0x7fbcac02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5d0765b9afbf5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb1d9b640 1 -- 172.21.15.26:0/959852524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac036070 con 0x7fbcb406ea50 2024-04-24T23:25:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb1d9b640 1 -- 172.21.15.26:0/959852524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcac031600 con 0x7fbcb406ea50 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb1d9b640 1 -- 172.21.15.26:0/959852524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0319d0 con 0x7fbcb406ea50 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb90bd640 1 -- 172.21.15.26:0/959852524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 msgr2=0x7fbcb41622d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb90bd640 1 --2- 172.21.15.26:0/959852524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 0x7fbcb41622d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009e50 tx=0x7fbcac02a760 comp rx=0 tx=0).stop 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb90bd640 1 -- 172.21.15.26:0/959852524 shutdown_connections 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb90bd640 1 --2- 172.21.15.26:0/959852524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 0x7fbcb41622d0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb90bd640 1 --2- 172.21.15.26:0/959852524 >> v2:172.21.15.29:3300/0 conn(0x7fbcb4141010 0x7fbcb4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb90bd640 1 -- 172.21.15.26:0/959852524 >> 172.21.15.26:0/959852524 conn(0x7fbcb406b300 msgr2=0x7fbcb406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb90bd640 1 -- 172.21.15.26:0/959852524 shutdown_connections 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.687+0000 7fbcb90bd640 1 -- 172.21.15.26:0/959852524 wait complete. 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.689+0000 7fbcb90bd640 1 Processor -- start 2024-04-24T23:25:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.689+0000 7fbcb90bd640 1 -- start start 2024-04-24T23:25:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.689+0000 7fbcb90bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 0x7fbcb415e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.689+0000 7fbcb90bd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbcb4141010 0x7fbcb413c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.689+0000 7fbcb90bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb4161e20 0x7fbcb4138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.689+0000 7fbcb90bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb41413f0 con 0x7fbcb406ea50 2024-04-24T23:25:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.689+0000 7fbcb90bd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb40727f0 con 0x7fbcb4161e20 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.689+0000 7fbcb90bd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb4141570 con 0x7fbcb4141010 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 0x7fbcb415e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 0x7fbcb415e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53698/0 (socket says 172.21.15.26:53698) 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb2d9d640 1 -- 172.21.15.26:0/216214328 learned_addr learned my addr 172.21.15.26:0/216214328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb359e640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb4161e20 0x7fbcb4138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb259c640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbcb4141010 0x7fbcb413c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb2d9d640 1 -- 172.21.15.26:0/216214328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbcb4141010 msgr2=0x7fbcb413c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb2d9d640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbcb4141010 0x7fbcb413c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb2d9d640 1 -- 172.21.15.26:0/216214328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb4161e20 msgr2=0x7fbcb4138040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb2d9d640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb4161e20 0x7fbcb4138040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb2d9d640 1 -- 172.21.15.26:0/216214328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb415edc0 con 0x7fbcb406ea50 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb2d9d640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 0x7fbcb415e880 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d800 tx=0x7fbca400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbc9b7fe640 1 -- 172.21.15.26:0/216214328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca400b6c0 con 0x7fbcb406ea50 2024-04-24T23:25:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbc9b7fe640 1 -- 172.21.15.26:0/216214328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca4019070 con 0x7fbcb406ea50 2024-04-24T23:25:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb413c4d0 con 0x7fbcb406ea50 2024-04-24T23:25:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbc9b7fe640 1 -- 172.21.15.26:0/216214328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca400ba80 con 0x7fbcb406ea50 2024-04-24T23:25:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.690+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb413bf10 con 0x7fbcb406ea50 2024-04-24T23:25:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.691+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb40694c0 con 0x7fbcb406ea50 2024-04-24T23:25:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.692+0000 7fbc9b7fe640 1 -- 172.21.15.26:0/216214328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbca400bc20 con 0x7fbcb406ea50 2024-04-24T23:25:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.692+0000 7fbc9b7fe640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbca006be60 0x7fbca006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.692+0000 7fbc9b7fe640 1 -- 172.21.15.26:0/216214328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fbca408b120 con 0x7fbcb406ea50 2024-04-24T23:25:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.692+0000 7fbcb259c640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbca006be60 0x7fbca006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.693+0000 7fbcb259c640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbca006be60 0x7fbca006e320 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4161a70 tx=0x7fbca8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:14.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.697+0000 7fbc9b7fe640 1 -- 172.21.15.26:0/216214328 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbca4059510 con 0x7fbcb406ea50 2024-04-24T23:25:14.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.918+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcb4147d70 con 0x7fbcb406ea50 2024-04-24T23:25:14.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.918+0000 7fbc9b7fe640 1 -- 172.21.15.26:0/216214328 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5224 (secure 0 0 0) 0x7fbca405d810 con 0x7fbcb406ea50 2024-04-24T23:25:14.920 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:14.920 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:24:14.713249+0000","modified":"2024-04-24T23:24:43.349267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6546,"mds_1":6534,"mds_2":6531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6531":{"gid":6531,"name":"c","rank":2,"incarnation":53,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6836/1519104007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1519104007},{"type":"v1","addr":"172.21.15.26:6836","nonce":1519104007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6534":{"gid":6534,"name":"a","rank":1,"incarnation":51,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6837/2743842996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2743842996},{"type":"v1","addr":"172.21.15.26:6837","nonce":2743842996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6546":{"gid":6546,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":10,"addr":"172.21.15.29:6835/2096523178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2096523178},{"type":"v1","addr":"172.21.15.29:6835","nonce":2096523178}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6531,6534,6546]},"id":4}]} 2024-04-24T23:25:14.920 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 58 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbca006be60 msgr2=0x7fbca006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbca006be60 0x7fbca006e320 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4161a70 tx=0x7fbca8009040 comp rx=0 tx=0).stop 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 msgr2=0x7fbcb415e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 0x7fbcb415e880 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d800 tx=0x7fbca400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 shutdown_connections 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbca006be60 0x7fbca006e320 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb4161e20 0x7fbcb4138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbcb4141010 0x7fbcb413c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 --2- 172.21.15.26:0/216214328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb406ea50 0x7fbcb415e880 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 >> 172.21.15.26:0/216214328 conn(0x7fbcb406b300 msgr2=0x7fbcb415d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.920+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 shutdown_connections 2024-04-24T23:25:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:14.921+0000 7fbcb90bd640 1 -- 172.21.15.26:0/216214328 wait complete. 2024-04-24T23:25:14.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:15.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.032+0000 7f6421a6b640 1 Processor -- start 2024-04-24T23:25:15.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.032+0000 7f6421a6b640 1 -- start start 2024-04-24T23:25:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f6421a6b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f641c141010 0x7f641c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f6421a6b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f641c06ea50 0x7f641c1622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f6421a6b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641c073250 con 0x7f641c161e20 2024-04-24T23:25:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f6421a6b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f641c071f90 con 0x7f641c141010 2024-04-24T23:25:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f6421a6b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f641c06f0b0 con 0x7f641c06ea50 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f641c141010 0x7f641c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f641c141010 0x7f641c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53702/0 (socket says 172.21.15.26:53702) 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f641c161e20 0x7f641c06e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42740/0 (socket says 172.21.15.26:42740) 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641b7fe640 1 -- 172.21.15.26:0/2386332514 learned_addr learned my addr 172.21.15.26:0/2386332514 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641bfff640 1 --2- 172.21.15.26:0/2386332514 >> v2:172.21.15.29:3300/0 conn(0x7f641c06ea50 0x7f641c1622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641b7fe640 1 -- 172.21.15.26:0/2386332514 >> v2:172.21.15.29:3300/0 conn(0x7f641c06ea50 msgr2=0x7f641c1622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641b7fe640 1 --2- 172.21.15.26:0/2386332514 >> v2:172.21.15.29:3300/0 conn(0x7f641c06ea50 0x7f641c1622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641b7fe640 1 -- 172.21.15.26:0/2386332514 >> v1:172.21.15.26:6789/0 conn(0x7f641c161e20 legacy=0x7f641c06e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641a7fc640 1 -- 172.21.15.26:0/2386332514 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272239236 0 0) 0x7f641c073250 con 0x7f641c161e20 2024-04-24T23:25:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.033+0000 7f641b7fe640 1 -- 172.21.15.26:0/2386332514 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c072230 con 0x7f641c141010 2024-04-24T23:25:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.034+0000 7f641b7fe640 1 --2- 172.21.15.26:0/2386332514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 0x7f641c072d10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f640c009870 tx=0x7f640c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaf51c2eb87ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.034+0000 7f641a7fc640 1 -- 172.21.15.26:0/2386332514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c037070 con 0x7f641c141010 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.034+0000 7f641a7fc640 1 -- 172.21.15.26:0/2386332514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640c032440 con 0x7f641c141010 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.034+0000 7f641a7fc640 1 -- 172.21.15.26:0/2386332514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c032810 con 0x7f641c141010 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.034+0000 7f6421a6b640 1 -- 172.21.15.26:0/2386332514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 msgr2=0x7f641c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.034+0000 7f6421a6b640 1 --2- 172.21.15.26:0/2386332514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 0x7f641c072d10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f640c009870 tx=0x7f640c02a2a0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.035+0000 7f6421a6b640 1 -- 172.21.15.26:0/2386332514 shutdown_connections 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.035+0000 7f6421a6b640 1 --2- 172.21.15.26:0/2386332514 >> v2:172.21.15.29:3300/0 conn(0x7f641c06ea50 0x7f641c1622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.035+0000 7f6421a6b640 1 --2- 172.21.15.26:0/2386332514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 0x7f641c072d10 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.035+0000 7f6421a6b640 1 -- 172.21.15.26:0/2386332514 >> 172.21.15.26:0/2386332514 conn(0x7f641c06b300 msgr2=0x7f641c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.035+0000 7f6421a6b640 1 -- 172.21.15.26:0/2386332514 shutdown_connections 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.035+0000 7f6421a6b640 1 -- 172.21.15.26:0/2386332514 wait complete. 2024-04-24T23:25:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.036+0000 7f6421a6b640 1 Processor -- start 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.036+0000 7f6421a6b640 1 -- start start 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f6421a6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f641c06ea50 0x7f641c13db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f6421a6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 0x7f641c12ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f6421a6b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f641c161e20 0x7f641c1304e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f6421a6b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1413f0 con 0x7f641c141010 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f6421a6b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f641c0727f0 con 0x7f641c06ea50 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f6421a6b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f641c141570 con 0x7f641c161e20 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 0x7f641c12ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 0x7f641c12ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53718/0 (socket says 172.21.15.26:53718) 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641affd640 1 -- 172.21.15.26:0/3075872871 learned_addr learned my addr 172.21.15.26:0/3075872871 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641b7fe640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f641c06ea50 0x7f641c13db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641bfff640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f641c161e20 0x7f641c1304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641affd640 1 -- 172.21.15.26:0/3075872871 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f641c161e20 msgr2=0x7f641c1304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641affd640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f641c161e20 0x7f641c1304e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641affd640 1 -- 172.21.15.26:0/3075872871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f641c06ea50 msgr2=0x7f641c13db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641affd640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f641c06ea50 0x7f641c13db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641affd640 1 -- 172.21.15.26:0/3075872871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c160fe0 con 0x7f641c141010 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641b7fe640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f641c06ea50 0x7f641c13db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f641affd640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 0x7f641c12ffa0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f641000c4d0 tx=0x7f641000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f6403fff640 1 -- 172.21.15.26:0/3075872871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410018070 con 0x7f641c141010 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f6403fff640 1 -- 172.21.15.26:0/3075872871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6410007500 con 0x7f641c141010 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.037+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c1612c0 con 0x7f641c141010 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.038+0000 7f6403fff640 1 -- 172.21.15.26:0/3075872871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410004a20 con 0x7f641c141010 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.038+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c130d90 con 0x7f641c141010 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.039+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641c1617e0 con 0x7f641c141010 2024-04-24T23:25:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.039+0000 7f6403fff640 1 -- 172.21.15.26:0/3075872871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f641001b070 con 0x7f641c141010 2024-04-24T23:25:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.039+0000 7f6403fff640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f640806be30 0x7f640806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.039+0000 7f6403fff640 1 -- 172.21.15.26:0/3075872871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f641008a290 con 0x7f641c141010 2024-04-24T23:25:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.039+0000 7f641b7fe640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f640806be30 0x7f640806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.040+0000 7f641b7fe640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f640806be30 0x7f640806e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f640c034000 tx=0x7f640c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.044+0000 7f6403fff640 1 -- 172.21.15.26:0/3075872871 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6410058c40 con 0x7f641c141010 2024-04-24T23:25:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.266+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f641c0694c0 con 0x7f641c141010 2024-04-24T23:25:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.266+0000 7f6403fff640 1 -- 172.21.15.26:0/3075872871 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5224 (secure 0 0 0) 0x7f6410058540 con 0x7f641c141010 2024-04-24T23:25:15.267 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:15.267 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:24:14.713249+0000","modified":"2024-04-24T23:24:43.349267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6546,"mds_1":6534,"mds_2":6531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6531":{"gid":6531,"name":"c","rank":2,"incarnation":53,"state":"up:active","state_seq":9,"addr":"172.21.15.26:6836/1519104007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1519104007},{"type":"v1","addr":"172.21.15.26:6836","nonce":1519104007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6534":{"gid":6534,"name":"a","rank":1,"incarnation":51,"state":"up:active","state_seq":10,"addr":"172.21.15.26:6837/2743842996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2743842996},{"type":"v1","addr":"172.21.15.26:6837","nonce":2743842996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6546":{"gid":6546,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":10,"addr":"172.21.15.29:6835/2096523178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2096523178},{"type":"v1","addr":"172.21.15.29:6835","nonce":2096523178}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6531,6534,6546]},"id":4}]} 2024-04-24T23:25:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 58 2024-04-24T23:25:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f640806be30 msgr2=0x7f640806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f640806be30 0x7f640806e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f640c034000 tx=0x7f640c02e000 comp rx=0 tx=0).stop 2024-04-24T23:25:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 msgr2=0x7f641c12ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 0x7f641c12ffa0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f641000c4d0 tx=0x7f641000c990 comp rx=0 tx=0).stop 2024-04-24T23:25:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 shutdown_connections 2024-04-24T23:25:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f640806be30 0x7f640806e2f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f641c161e20 0x7f641c1304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c141010 0x7f641c12ffa0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 --2- 172.21.15.26:0/3075872871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f641c06ea50 0x7f641c13db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 >> 172.21.15.26:0/3075872871 conn(0x7f641c06b300 msgr2=0x7f641c140770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 shutdown_connections 2024-04-24T23:25:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.268+0000 7f6421a6b640 1 -- 172.21.15.26:0/3075872871 wait complete. 2024-04-24T23:25:15.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.380+0000 7f57deed4640 1 Processor -- start 2024-04-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.380+0000 7f57deed4640 1 -- start start 2024-04-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.381+0000 7f57deed4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f57d80cdd70 0x7f57d806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.381+0000 7f57deed4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d80cfbc0 con 0x7f57d806d350 2024-04-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.381+0000 7f57deed4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d806d7f0 con 0x7f57d8145910 2024-04-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.381+0000 7f57deed4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f57d8145cf0 con 0x7f57d80cdd70 2024-04-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.381+0000 7f57d7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f57d806d350 0x7f57d80cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42748/0 (socket says 172.21.15.26:42748) 2024-04-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.381+0000 7f57d7fff640 1 -- 172.21.15.26:0/1998663326 learned_addr learned my addr 172.21.15.26:0/1998663326 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.381+0000 7f57dd471640 1 --2- 172.21.15.26:0/1998663326 >> v2:172.21.15.29:3300/0 conn(0x7f57d80cdd70 0x7f57d806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.381+0000 7f57d77fe640 1 -- 172.21.15.26:0/1998663326 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956358841 0 0) 0x7f57d80cfbc0 con 0x7f57d806d350 2024-04-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.381+0000 7f57d77fe640 1 -- 172.21.15.26:0/1998663326 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57c4003520 con 0x7f57d806d350 2024-04-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.382+0000 7f57d77fe640 1 -- 172.21.15.26:0/1998663326 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1321985537 0 0) 0x7f57c4003520 con 0x7f57d806d350 2024-04-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.382+0000 7f57dd471640 1 -- 172.21.15.26:0/1998663326 >> v1:172.21.15.29:6789/0 conn(0x7f57d8145910 legacy=0x7f57d815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.382+0000 7f57dd471640 1 -- 172.21.15.26:0/1998663326 >> v1:172.21.15.26:6789/0 conn(0x7f57d806d350 legacy=0x7f57d80cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.382+0000 7f57dd471640 1 -- 172.21.15.26:0/1998663326 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d806dad0 con 0x7f57d80cdd70 2024-04-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.382+0000 7f57dd471640 1 --2- 172.21.15.26:0/1998663326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57d80cdd70 0x7f57d806e890 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f57c80097d0 tx=0x7f57c802a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec6857e23b25d1f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57d77fe640 1 -- 172.21.15.26:0/1998663326 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8037070 con 0x7f57d80cdd70 2024-04-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57d77fe640 1 -- 172.21.15.26:0/1998663326 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c802abf0 con 0x7f57d80cdd70 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57d77fe640 1 -- 172.21.15.26:0/1998663326 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0001670 con 0x7f57d80cdd70 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57deed4640 1 -- 172.21.15.26:0/1998663326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57d80cdd70 msgr2=0x7f57d806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57deed4640 1 --2- 172.21.15.26:0/1998663326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57d80cdd70 0x7f57d806e890 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f57c80097d0 tx=0x7f57c802a160 comp rx=0 tx=0).stop 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57deed4640 1 -- 172.21.15.26:0/1998663326 shutdown_connections 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57deed4640 1 --2- 172.21.15.26:0/1998663326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57d80cdd70 0x7f57d806e890 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57deed4640 1 -- 172.21.15.26:0/1998663326 >> 172.21.15.26:0/1998663326 conn(0x7f57d806b300 msgr2=0x7f57d806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57deed4640 1 -- 172.21.15.26:0/1998663326 shutdown_connections 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.383+0000 7f57deed4640 1 -- 172.21.15.26:0/1998663326 wait complete. 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.384+0000 7f57deed4640 1 Processor -- start 2024-04-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.384+0000 7f57deed4640 1 -- start start 2024-04-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.385+0000 7f57deed4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d806d350 0x7f57d814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.385+0000 7f57deed4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57d80cdd70 0x7f57d813e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.385+0000 7f57deed4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57d8145910 0x7f57d8141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.385+0000 7f57deed4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f57d806e370 con 0x7f57d806d350 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.385+0000 7f57deed4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f57d806e090 con 0x7f57d8145910 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.385+0000 7f57deed4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f57d815d770 con 0x7f57d80cdd70 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.385+0000 7f57dcc70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d806d350 0x7f57d814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.385+0000 7f57dcc70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d806d350 0x7f57d814f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53722/0 (socket says 172.21.15.26:53722) 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.385+0000 7f57dcc70640 1 -- 172.21.15.26:0/4059637634 learned_addr learned my addr 172.21.15.26:0/4059637634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57dd471640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57d8145910 0x7f57d8141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57d7fff640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57d80cdd70 0x7f57d813e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57dcc70640 1 -- 172.21.15.26:0/4059637634 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57d80cdd70 msgr2=0x7f57d813e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57dcc70640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57d80cdd70 0x7f57d813e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57dcc70640 1 -- 172.21.15.26:0/4059637634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57d8145910 msgr2=0x7f57d8141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57dcc70640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57d8145910 0x7f57d8141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57dcc70640 1 -- 172.21.15.26:0/4059637634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d813d660 con 0x7f57d806d350 2024-04-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57d7fff640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57d80cdd70 0x7f57d813e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57dcc70640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d806d350 0x7f57d814f6a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f57d000c730 tx=0x7f57d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57d57fa640 1 -- 172.21.15.26:0/4059637634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0003b70 con 0x7f57d806d350 2024-04-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57d57fa640 1 -- 172.21.15.26:0/4059637634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d0003d30 con 0x7f57d806d350 2024-04-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d813d940 con 0x7f57d806d350 2024-04-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57d57fa640 1 -- 172.21.15.26:0/4059637634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d000aa90 con 0x7f57d806d350 2024-04-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.386+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d813df00 con 0x7f57d806d350 2024-04-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.387+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d80694c0 con 0x7f57d806d350 2024-04-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.388+0000 7f57d57fa640 1 -- 172.21.15.26:0/4059637634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57d000ac30 con 0x7f57d806d350 2024-04-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.388+0000 7f57d57fa640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57b806be30 0x7f57b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.388+0000 7f57d7fff640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57b806be30 0x7f57b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.388+0000 7f57d57fa640 1 -- 172.21.15.26:0/4059637634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f57d008ac70 con 0x7f57d806d350 2024-04-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.389+0000 7f57d7fff640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57b806be30 0x7f57b806e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f57d813bf20 tx=0x7f57c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.393+0000 7f57d57fa640 1 -- 172.21.15.26:0/4059637634 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f57d00596b0 con 0x7f57d806d350 2024-04-24T23:25:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.577+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57d8073040 con 0x7f57d806d350 2024-04-24T23:25:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.578+0000 7f57d57fa640 1 -- 172.21.15.26:0/4059637634 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7f57d0058fb0 con 0x7f57d806d350 2024-04-24T23:25:15.580 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:15.580 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":58,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:24:16.006495+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:24:13.001912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T23:24:13.678528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:25:15.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57b806be30 msgr2=0x7f57b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57b806be30 0x7f57b806e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f57d813bf20 tx=0x7f57c0009040 comp rx=0 tx=0).stop 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d806d350 msgr2=0x7f57d814f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d806d350 0x7f57d814f6a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f57d000c730 tx=0x7f57d000cbf0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 shutdown_connections 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f57b806be30 0x7f57b806e2f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f57d8145910 0x7f57d8141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f57d80cdd70 0x7f57d813e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 --2- 172.21.15.26:0/4059637634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d806d350 0x7f57d814f6a0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 >> 172.21.15.26:0/4059637634 conn(0x7f57d806b300 msgr2=0x7f57d8144400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 shutdown_connections 2024-04-24T23:25:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.580+0000 7f57deed4640 1 -- 172.21.15.26:0/4059637634 wait complete. 2024-04-24T23:25:15.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.693+0000 7f2a6fb09640 1 Processor -- start 2024-04-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.693+0000 7f2a6fb09640 1 -- start start 2024-04-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6fb09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a6815a480 0x7f2a680b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6fb09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a6815a860 0x7f2a6806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6fb09640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2a680b4e30 0x7f2a6806e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6fb09640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f2a680b2e80 con 0x7f2a6815a860 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6fb09640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2a680b3120 con 0x7f2a680b4e30 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6fb09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2a680b33c0 con 0x7f2a6815a480 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6d0a4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a6815a860 0x7f2a6806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6d0a4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a6815a860 0x7f2a6806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53738/0 (socket says 172.21.15.26:53738) 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6d0a4640 1 -- 172.21.15.26:0/2786173948 learned_addr learned my addr 172.21.15.26:0/2786173948 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6e0a6640 1 --2- 172.21.15.26:0/2786173948 >> v2:172.21.15.29:3300/0 conn(0x7f2a680b4e30 0x7f2a6806e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.694+0000 7f2a6d8a5640 1 --2- 172.21.15.26:0/2786173948 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a6815a480 0x7f2a680b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.695+0000 7f2a6d0a4640 1 -- 172.21.15.26:0/2786173948 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a6815a480 msgr2=0x7f2a680b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.695+0000 7f2a6d0a4640 1 --2- 172.21.15.26:0/2786173948 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a6815a480 0x7f2a680b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.695+0000 7f2a6d0a4640 1 -- 172.21.15.26:0/2786173948 >> v2:172.21.15.29:3300/0 conn(0x7f2a680b4e30 msgr2=0x7f2a6806e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.695+0000 7f2a6d0a4640 1 --2- 172.21.15.26:0/2786173948 >> v2:172.21.15.29:3300/0 conn(0x7f2a680b4e30 0x7f2a6806e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.695+0000 7f2a6d0a4640 1 -- 172.21.15.26:0/2786173948 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a680b36a0 con 0x7f2a6815a860 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.695+0000 7f2a6d0a4640 1 --2- 172.21.15.26:0/2786173948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a860 0x7f2a6806dbe0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2a680733a0 tx=0x7f2a5c02c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48f7ac25d4e6e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.695+0000 7f2a6c8a3640 1 -- 172.21.15.26:0/2786173948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c032700 con 0x7f2a6815a860 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.695+0000 7f2a6c8a3640 1 -- 172.21.15.26:0/2786173948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a5c0328a0 con 0x7f2a6815a860 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.695+0000 7f2a6c8a3640 1 -- 172.21.15.26:0/2786173948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c032c70 con 0x7f2a6815a860 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.696+0000 7f2a6fb09640 1 -- 172.21.15.26:0/2786173948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a860 msgr2=0x7f2a6806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.696+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/2786173948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a860 0x7f2a6806dbe0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2a680733a0 tx=0x7f2a5c02c800 comp rx=0 tx=0).stop 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.696+0000 7f2a6fb09640 1 -- 172.21.15.26:0/2786173948 shutdown_connections 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.696+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/2786173948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a860 0x7f2a6806dbe0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.696+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/2786173948 >> v2:172.21.15.29:3300/0 conn(0x7f2a680b4e30 0x7f2a6806e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.696+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/2786173948 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a6815a480 0x7f2a680b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.696+0000 7f2a6fb09640 1 -- 172.21.15.26:0/2786173948 >> 172.21.15.26:0/2786173948 conn(0x7f2a6806b300 msgr2=0x7f2a6806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.696+0000 7f2a6fb09640 1 -- 172.21.15.26:0/2786173948 shutdown_connections 2024-04-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.696+0000 7f2a6fb09640 1 -- 172.21.15.26:0/2786173948 wait complete. 2024-04-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.697+0000 7f2a6fb09640 1 Processor -- start 2024-04-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.697+0000 7f2a6fb09640 1 -- start start 2024-04-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6fb09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a680b4e30 0x7f2a6815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6fb09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a480 0x7f2a680ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6fb09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a6815a860 0x7f2a680addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6fb09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6806d350 con 0x7f2a6815a480 2024-04-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6fb09640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2a680b3c60 con 0x7f2a6815a860 2024-04-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6fb09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6806d4d0 con 0x7f2a680b4e30 2024-04-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6d0a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a480 0x7f2a680ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6d0a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a480 0x7f2a680ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53754/0 (socket says 172.21.15.26:53754) 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6d0a4640 1 -- 172.21.15.26:0/3013202575 learned_addr learned my addr 172.21.15.26:0/3013202575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6e0a6640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a6815a860 0x7f2a680addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6d8a5640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a680b4e30 0x7f2a6815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6d0a4640 1 -- 172.21.15.26:0/3013202575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a680b4e30 msgr2=0x7f2a6815bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6d0a4640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a680b4e30 0x7f2a6815bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6d0a4640 1 -- 172.21.15.26:0/3013202575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a6815a860 msgr2=0x7f2a680addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.698+0000 7f2a6d0a4640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a6815a860 0x7f2a680addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.699+0000 7f2a6d0a4640 1 -- 172.21.15.26:0/3013202575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6815c9d0 con 0x7f2a6815a480 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.699+0000 7f2a6d0a4640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a480 0x7f2a680ad870 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c02c840 tx=0x7f2a5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.699+0000 7f2a667fc640 1 -- 172.21.15.26:0/3013202575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c041020 con 0x7f2a6815a480 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.699+0000 7f2a667fc640 1 -- 172.21.15.26:0/3013202575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a5c034bd0 con 0x7f2a6815a480 2024-04-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.699+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6815ccb0 con 0x7f2a6815a480 2024-04-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.699+0000 7f2a667fc640 1 -- 172.21.15.26:0/3013202575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c032e40 con 0x7f2a6815a480 2024-04-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.699+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6815d270 con 0x7f2a6815a480 2024-04-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.700+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a680694c0 con 0x7f2a6815a480 2024-04-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.700+0000 7f2a667fc640 1 -- 172.21.15.26:0/3013202575 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a5c044020 con 0x7f2a6815a480 2024-04-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.701+0000 7f2a667fc640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a5406be30 0x7f2a5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.701+0000 7f2a667fc640 1 -- 172.21.15.26:0/3013202575 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2a5c0ab630 con 0x7f2a6815a480 2024-04-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.701+0000 7f2a6d8a5640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a5406be30 0x7f2a5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.701+0000 7f2a6d8a5640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a5406be30 0x7f2a5406e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009a10 tx=0x7f2a60008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.705+0000 7f2a667fc640 1 -- 172.21.15.26:0/3013202575 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a5c04b030 con 0x7f2a6815a480 2024-04-24T23:25:15.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.926+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2a68153b40 con 0x7f2a6815a480 2024-04-24T23:25:15.928 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:25:15.927+0000 7f26a0f44640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:25:15.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.928+0000 7f2a667fc640 1 -- 172.21.15.26:0/3013202575 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7f2a5c031070 con 0x7f2a6815a480 2024-04-24T23:25:15.929 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:25:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a5406be30 msgr2=0x7f2a5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a5406be30 0x7f2a5406e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009a10 tx=0x7f2a60008040 comp rx=0 tx=0).stop 2024-04-24T23:25:15.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a480 msgr2=0x7f2a680ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:15.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a480 0x7f2a680ad870 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c02c840 tx=0x7f2a5c004730 comp rx=0 tx=0).stop 2024-04-24T23:25:15.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 shutdown_connections 2024-04-24T23:25:15.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2a5406be30 0x7f2a5406e2f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2a6815a860 0x7f2a680addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a6815a480 0x7f2a680ad870 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 --2- 172.21.15.26:0/3013202575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2a680b4e30 0x7f2a6815bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:15.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 >> 172.21.15.26:0/3013202575 conn(0x7f2a6806b300 msgr2=0x7f2a6814b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:15.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 shutdown_connections 2024-04-24T23:25:15.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:15.932+0000 7f2a6fb09640 1 -- 172.21.15.26:0/3013202575 wait complete. 2024-04-24T23:25:15.937 INFO:tasks.ceph.mds.a.smithi026.stderr: -1623> 2024-04-24T23:24:39.594+0000 7f6efcc97640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1) n(v2 rc2024-04-24T23:24:26.649950+0000 4=0+4) (inest lock) (ifile excl) caps={5609=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5609 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55e807c71700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.649950+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-04-24T23:24:26.649950+0000 4=0+4)","error_str":""},"return_code":-61} 2024-04-24T23:25:15.937 INFO:tasks.ceph.mds.a.smithi026.stderr: -1484> 2024-04-24T23:24:39.595+0000 7f6f034a4640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1) n(v1 rc2024-04-24T23:24:26.649950+0000 3=0+3) (inest sync->mix g=0,2) caps={5609=pAsLsXsFs/pAsLsXsFsxcral@3} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 export_pin=2 0x55e806d14b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.421954+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.420954+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-04-24T23:24:26.649950+0000 3=0+3)","error_str":""},"return_code":-61} 2024-04-24T23:25:15.938 INFO:tasks.ceph.mds.c.smithi026.stderr: -1536> 2024-04-24T23:24:39.594+0000 7fede1dba640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-04-24T23:24:26.421954+0000 1=0+1) n(v0 rc2024-04-24T23:24:26.421954+0000 2=0+2) (inest lock) (ifile excl) caps={5609=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5609 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55af1122a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.421954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.421954+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.421954+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-04-24T23:24:26.421954+0000 2=0+2)","error_str":""},"return_code":-61} 2024-04-24T23:25:15.938 INFO:tasks.ceph.mds.c.smithi026.stderr: -1514> 2024-04-24T23:24:39.595+0000 7fede1dba640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5609=pAsLsXsFsx/AsLsXsFsx@3},l=5609 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55af11fca000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-04-24T23:25:15.943 INFO:tasks.ceph.mds.d.smithi029.stderr: -3716> 2024-04-24T23:24:39.594+0000 7f75a7611640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v18 ap=1 f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1) n(v3 rc2024-04-24T23:24:26.649950+0000 6=0+6)/n(v2 rc2024-04-24T23:24:26.649950+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5609=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5609 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=1 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x558b81427600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000002:[<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.649950+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-04-24T23:24:26.649950+0000 6=0+6)","error_str":""},"return_code":-61} 2024-04-24T23:25:15.943 INFO:tasks.ceph.mds.d.smithi029.stderr: -3594> 2024-04-24T23:24:39.595+0000 7f75ade1e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v18 ap=2 f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1) n(v4 rc2024-04-24T23:24:26.649950+0000 5=0+5)/n(v3 rc2024-04-24T23:24:26.649950+0000 5=0+5) (inest sync->mix g=1,2) caps={5609=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=1 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x558b81427b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000003:[<0x10000000002/d4 v18>,<0x10000000001/d3 v18>,<0x10000000000/d2 v18>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:24:26.649950+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-04-24T23:24:26.419954+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-04-24T23:24:26.649950+0000 5=0+5)","error_str":""},"return_code":-61} 2024-04-24T23:25:15.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:25:15.965 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:25:15.966 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:25:15.973 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.053+0000 7f79ad800640 1 Processor -- start 2024-04-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.053+0000 7f79ad800640 1 -- start start 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.053+0000 7f79ad800640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f79a80d0940 0x7f79a80d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.053+0000 7f79ad800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a80d2f80 con 0x7f79a815e160 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.053+0000 7f79ad800640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a80b6f40 con 0x7f79a80b4af0 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.053+0000 7f79ad800640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f79a80b7220 con 0x7f79a80d0940 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f79a815e160 0x7f79a80d2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42778/0 (socket says 172.21.15.26:42778) 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a6ffd640 1 -- 172.21.15.26:0/2747177296 learned_addr learned my addr 172.21.15.26:0/2747177296 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a67fc640 1 --2- 172.21.15.26:0/2747177296 >> v2:172.21.15.29:3300/0 conn(0x7f79a80d0940 0x7f79a80d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a5ffb640 1 -- 172.21.15.26:0/2747177296 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 43192269 0 0) 0x7f79a80d2f80 con 0x7f79a815e160 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a5ffb640 1 -- 172.21.15.26:0/2747177296 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7994003520 con 0x7f79a815e160 2024-04-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a5ffb640 1 -- 172.21.15.26:0/2747177296 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956819354 0 0) 0x7f79a80b6f40 con 0x7f79a80b4af0 2024-04-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a5ffb640 1 -- 172.21.15.26:0/2747177296 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79a80d2f80 con 0x7f79a80b4af0 2024-04-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a67fc640 1 -- 172.21.15.26:0/2747177296 >> v1:172.21.15.29:6789/0 conn(0x7f79a80b4af0 legacy=0x7f79a80b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a67fc640 1 -- 172.21.15.26:0/2747177296 >> v1:172.21.15.26:6789/0 conn(0x7f79a815e160 legacy=0x7f79a80d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.054+0000 7f79a67fc640 1 -- 172.21.15.26:0/2747177296 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a80b7500 con 0x7f79a80d0940 2024-04-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.055+0000 7f79a67fc640 1 --2- 172.21.15.26:0/2747177296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a80d0940 0x7f79a80d0dc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f79980097a0 tx=0x7f799802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec9b8825fbae0ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.055+0000 7f79a5ffb640 1 -- 172.21.15.26:0/2747177296 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998037070 con 0x7f79a80d0940 2024-04-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.055+0000 7f79a5ffb640 1 -- 172.21.15.26:0/2747177296 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7998032440 con 0x7f79a80d0940 2024-04-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.055+0000 7f79a5ffb640 1 -- 172.21.15.26:0/2747177296 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998032810 con 0x7f79a80d0940 2024-04-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.055+0000 7f79ad800640 1 -- 172.21.15.26:0/2747177296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a80d0940 msgr2=0x7f79a80d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.056+0000 7f79ad800640 1 --2- 172.21.15.26:0/2747177296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a80d0940 0x7f79a80d0dc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f79980097a0 tx=0x7f799802a230 comp rx=0 tx=0).stop 2024-04-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.056+0000 7f79ad800640 1 -- 172.21.15.26:0/2747177296 shutdown_connections 2024-04-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.056+0000 7f79ad800640 1 --2- 172.21.15.26:0/2747177296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a80d0940 0x7f79a80d0dc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.056+0000 7f79ad800640 1 -- 172.21.15.26:0/2747177296 >> 172.21.15.26:0/2747177296 conn(0x7f79a809d1c0 msgr2=0x7f79a80d3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.056+0000 7f79ad800640 1 -- 172.21.15.26:0/2747177296 shutdown_connections 2024-04-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.056+0000 7f79ad800640 1 -- 172.21.15.26:0/2747177296 wait complete. 2024-04-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.057+0000 7f79ad800640 1 Processor -- start 2024-04-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.057+0000 7f79ad800640 1 -- start start 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79ad800640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79a80b4af0 0x7f79a8157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79ad800640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a80d0940 0x7f79a81603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79ad800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a815e160 0x7f79a8160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79ad800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79a815e540 con 0x7f79a815e160 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79ad800640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f79a80d2c40 con 0x7f79a80d0940 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79ad800640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f79a80cfe30 con 0x7f79a80b4af0 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a815e160 0x7f79a8160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a815e160 0x7f79a8160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53798/0 (socket says 172.21.15.26:53798) 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a77fe640 1 -- 172.21.15.26:0/772556215 learned_addr learned my addr 172.21.15.26:0/772556215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a67fc640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a80d0940 0x7f79a81603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a6ffd640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79a80b4af0 0x7f79a8157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a77fe640 1 -- 172.21.15.26:0/772556215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79a80b4af0 msgr2=0x7f79a8157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a77fe640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79a80b4af0 0x7f79a8157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a77fe640 1 -- 172.21.15.26:0/772556215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a80d0940 msgr2=0x7f79a81603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a77fe640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a80d0940 0x7f79a81603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.058+0000 7f79a77fe640 1 -- 172.21.15.26:0/772556215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a8158520 con 0x7f79a815e160 2024-04-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.059+0000 7f79a77fe640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a815e160 0x7f79a8160900 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c790 tx=0x7f79a000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.059+0000 7f799f7fe640 1 -- 172.21.15.26:0/772556215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0003b70 con 0x7f79a815e160 2024-04-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.059+0000 7f799f7fe640 1 -- 172.21.15.26:0/772556215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a0003d10 con 0x7f79a815e160 2024-04-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.059+0000 7f799f7fe640 1 -- 172.21.15.26:0/772556215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a000aa50 con 0x7f79a815e160 2024-04-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.059+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a80c8940 con 0x7f79a815e160 2024-04-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.059+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a80c8f00 con 0x7f79a815e160 2024-04-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.060+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a8075610 con 0x7f79a815e160 2024-04-24T23:25:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.060+0000 7f799f7fe640 1 -- 172.21.15.26:0/772556215 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79a000abf0 con 0x7f79a815e160 2024-04-24T23:25:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.061+0000 7f799f7fe640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f798806be60 0x7f798806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.061+0000 7f799f7fe640 1 -- 172.21.15.26:0/772556215 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5967+0+0 (secure 0 0 0) 0x7f79a008b0d0 con 0x7f79a815e160 2024-04-24T23:25:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.061+0000 7f79a6ffd640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f798806be60 0x7f798806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.061+0000 7f79a6ffd640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f798806be60 0x7f798806e320 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f79900011e0 tx=0x7f7990009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:16.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.065+0000 7f799f7fe640 1 -- 172.21.15.26:0/772556215 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79a0059a60 con 0x7f79a815e160 2024-04-24T23:25:16.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.286+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f79a80b6160 con 0x7f79a815e160 2024-04-24T23:25:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.930+0000 7f799f7fe640 1 -- 172.21.15.26:0/772556215 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f79a0059360 con 0x7f79a815e160 2024-04-24T23:25:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f798806be60 msgr2=0x7f798806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f798806be60 0x7f798806e320 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f79900011e0 tx=0x7f7990009040 comp rx=0 tx=0).stop 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a815e160 msgr2=0x7f79a8160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a815e160 0x7f79a8160900 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c790 tx=0x7f79a000cc50 comp rx=0 tx=0).stop 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 shutdown_connections 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f798806be60 0x7f798806e320 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a815e160 0x7f79a8160900 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79a80d0940 0x7f79a81603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 --2- 172.21.15.26:0/772556215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f79a80b4af0 0x7f79a8157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 >> 172.21.15.26:0/772556215 conn(0x7f79a809d1c0 msgr2=0x7f79a80ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 shutdown_connections 2024-04-24T23:25:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:16.933+0000 7f79ad800640 1 -- 172.21.15.26:0/772556215 wait complete. 2024-04-24T23:25:16.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:25:17.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.102+0000 7f0e2a68c640 1 Processor -- start 2024-04-24T23:25:17.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.102+0000 7f0e2a68c640 1 -- start start 2024-04-24T23:25:17.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e2a68c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0e240d1210 0x7f0e2415db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:17.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e2a68c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 0x7f0e2406c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e2a68c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2406f650 con 0x7f0e2409aa00 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e2a68c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0e240706f0 con 0x7f0e240d1210 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e2a68c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0e240d1650 con 0x7f0e2415e0a0 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e23fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0e2409aa00 0x7f0e2409ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42782/0 (socket says 172.21.15.26:42782) 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e23fff640 1 -- 172.21.15.26:0/3904633132 learned_addr learned my addr 172.21.15.26:0/3904633132 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e237fe640 1 --2- 172.21.15.26:0/3904633132 >> v2:172.21.15.29:3300/0 conn(0x7f0e240d1210 0x7f0e2415db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e28c29640 1 --2- 172.21.15.26:0/3904633132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 0x7f0e2406c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e22ffd640 1 -- 172.21.15.26:0/3904633132 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573894253 0 0) 0x7f0e2406f650 con 0x7f0e2409aa00 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.103+0000 7f0e22ffd640 1 -- 172.21.15.26:0/3904633132 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e10003520 con 0x7f0e2409aa00 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.104+0000 7f0e28c29640 1 -- 172.21.15.26:0/3904633132 >> v2:172.21.15.29:3300/0 conn(0x7f0e240d1210 msgr2=0x7f0e2415db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.104+0000 7f0e28c29640 1 --2- 172.21.15.26:0/3904633132 >> v2:172.21.15.29:3300/0 conn(0x7f0e240d1210 0x7f0e2415db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.104+0000 7f0e22ffd640 1 -- 172.21.15.26:0/3904633132 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 430004456 0 0) 0x7f0e10003520 con 0x7f0e2409aa00 2024-04-24T23:25:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.104+0000 7f0e28c29640 1 -- 172.21.15.26:0/3904633132 >> v1:172.21.15.26:6789/0 conn(0x7f0e2409aa00 legacy=0x7f0e2409ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.104+0000 7f0e28c29640 1 -- 172.21.15.26:0/3904633132 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2406f930 con 0x7f0e2415e0a0 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.104+0000 7f0e28c29640 1 --2- 172.21.15.26:0/3904633132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 0x7f0e2406c230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009d90 tx=0x7f0e1c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acdfead78de8e74f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e22ffd640 1 -- 172.21.15.26:0/3904633132 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c001670 con 0x7f0e2415e0a0 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e22ffd640 1 -- 172.21.15.26:0/3904633132 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e1c007540 con 0x7f0e2415e0a0 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e22ffd640 1 -- 172.21.15.26:0/3904633132 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c007910 con 0x7f0e2415e0a0 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3904633132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 msgr2=0x7f0e2406c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e2a68c640 1 --2- 172.21.15.26:0/3904633132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 0x7f0e2406c230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009d90 tx=0x7f0e1c02a5f0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3904633132 shutdown_connections 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e2a68c640 1 --2- 172.21.15.26:0/3904633132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 0x7f0e2406c230 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e2a68c640 1 --2- 172.21.15.26:0/3904633132 >> v2:172.21.15.29:3300/0 conn(0x7f0e240d1210 0x7f0e2415db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3904633132 >> 172.21.15.26:0/3904633132 conn(0x7f0e240b39e0 msgr2=0x7f0e240b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:17.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3904633132 shutdown_connections 2024-04-24T23:25:17.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.105+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3904633132 wait complete. 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.106+0000 7f0e2a68c640 1 Processor -- start 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e2a68c640 1 -- start start 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e2a68c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e2409aa00 0x7f0e240c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e2a68c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e240d1210 0x7f0e240ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e2a68c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 0x7f0e240ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e2a68c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2406fef0 con 0x7f0e240d1210 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e2a68c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0e24070260 con 0x7f0e2409aa00 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e2a68c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2406e090 con 0x7f0e2415e0a0 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e240d1210 0x7f0e240ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e240d1210 0x7f0e240ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53804/0 (socket says 172.21.15.26:53804) 2024-04-24T23:25:17.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e237fe640 1 -- 172.21.15.26:0/3106556641 learned_addr learned my addr 172.21.15.26:0/3106556641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.107+0000 7f0e23fff640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e2409aa00 0x7f0e240c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e28c29640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 0x7f0e240ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e237fe640 1 -- 172.21.15.26:0/3106556641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 msgr2=0x7f0e240ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e237fe640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 0x7f0e240ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e237fe640 1 -- 172.21.15.26:0/3106556641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e2409aa00 msgr2=0x7f0e240c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e237fe640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e2409aa00 0x7f0e240c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e237fe640 1 -- 172.21.15.26:0/3106556641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e240cab80 con 0x7f0e240d1210 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e237fe640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e240d1210 0x7f0e240ca010 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400e870 tx=0x7f0e1400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e20ff9640 1 -- 172.21.15.26:0/3106556641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14017070 con 0x7f0e240d1210 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e20ff9640 1 -- 172.21.15.26:0/3106556641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e1400c940 con 0x7f0e240d1210 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e240cae60 con 0x7f0e240d1210 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e20ff9640 1 -- 172.21.15.26:0/3106556641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400cd30 con 0x7f0e240d1210 2024-04-24T23:25:17.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.108+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e240cb700 con 0x7f0e240d1210 2024-04-24T23:25:17.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.109+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e240694c0 con 0x7f0e240d1210 2024-04-24T23:25:17.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.110+0000 7f0e20ff9640 1 -- 172.21.15.26:0/3106556641 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e1401a070 con 0x7f0e240d1210 2024-04-24T23:25:17.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.110+0000 7f0e20ff9640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0e0406be30 0x7f0e0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:17.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.110+0000 7f0e20ff9640 1 -- 172.21.15.26:0/3106556641 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5967+0+0 (secure 0 0 0) 0x7f0e1408b120 con 0x7f0e240d1210 2024-04-24T23:25:17.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.110+0000 7f0e23fff640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0e0406be30 0x7f0e0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:17.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.111+0000 7f0e23fff640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0e0406be30 0x7f0e0406e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005880 tx=0x7f0e0c0057f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:17.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.114+0000 7f0e20ff9640 1 -- 172.21.15.26:0/3106556641 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e14059380 con 0x7f0e240d1210 2024-04-24T23:25:17.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.298+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0e240cb030 con 0x7f0e240d1210 2024-04-24T23:25:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.934+0000 7f0e20ff9640 1 -- 172.21.15.26:0/3106556641 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7f0e1405d680 con 0x7f0e240d1210 2024-04-24T23:25:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:25:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0e0406be30 msgr2=0x7f0e0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0e0406be30 0x7f0e0406e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005880 tx=0x7f0e0c0057f0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e240d1210 msgr2=0x7f0e240ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e240d1210 0x7f0e240ca010 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400e870 tx=0x7f0e1400ed30 comp rx=0 tx=0).stop 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 shutdown_connections 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0e0406be30 0x7f0e0406e2f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0e2415e0a0 0x7f0e240ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e240d1210 0x7f0e240ca010 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 --2- 172.21.15.26:0/3106556641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e2409aa00 0x7f0e240c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 >> 172.21.15.26:0/3106556641 conn(0x7f0e240b39e0 msgr2=0x7f0e2406c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 shutdown_connections 2024-04-24T23:25:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:17.938+0000 7f0e2a68c640 1 -- 172.21.15.26:0/3106556641 wait complete. 2024-04-24T23:25:17.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:25:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.064+0000 7feda3ff4640 1 Processor -- start 2024-04-24T23:25:18.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.064+0000 7feda3ff4640 1 -- start start 2024-04-24T23:25:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.065+0000 7feda3ff4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c158a00 0x7fed9c15f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.065+0000 7feda3ff4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed9c0ae220 con 0x7fed9c0ad870 2024-04-24T23:25:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.065+0000 7feda3ff4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed9c0ae500 con 0x7fed9c06c840 2024-04-24T23:25:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.065+0000 7feda3ff4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fed9c072610 con 0x7fed9c158a00 2024-04-24T23:25:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.065+0000 7feda158f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fed9c0ad870 0x7fed9c06c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42788/0 (socket says 172.21.15.26:42788) 2024-04-24T23:25:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.065+0000 7feda158f640 1 -- 172.21.15.26:0/673074855 learned_addr learned my addr 172.21.15.26:0/673074855 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.065+0000 7feda1d90640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c158a00 0x7fed9c15f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 842109204 0 0) 0x7fed9c0ae220 con 0x7fed9c0ad870 2024-04-24T23:25:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed84003520 con 0x7fed9c0ad870 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425641857 0 0) 0x7fed9c0ae500 con 0x7fed9c06c840 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed9c0ae220 con 0x7fed9c06c840 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 851791255 0 0) 0x7fed84003520 con 0x7fed9c0ad870 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c158a00 msgr2=0x7fed9c15f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c158a00 0x7fed9c15f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 >> v1:172.21.15.29:6789/0 conn(0x7fed9c06c840 legacy=0x7fed9c0b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c0722a0 con 0x7fed9c0ad870 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fed90001670 con 0x7fed9c0ad870 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 >> v1:172.21.15.26:6789/0 conn(0x7fed9c0ad870 legacy=0x7fed9c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c158a00 0x7fed84003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed840040d0 0x7fed84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed84004d40 0x7fed840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c072610 con 0x7fed84004d40 2024-04-24T23:25:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fed8400baa0 con 0x7fed840040d0 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fed8400bca0 con 0x7fed9c158a00 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda2591640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed84004d40 0x7fed840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda1d90640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c158a00 0x7fed84003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda158f640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed840040d0 0x7fed84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda2591640 1 -- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c158a00 msgr2=0x7fed84003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda2591640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c158a00 0x7fed84003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda2591640 1 -- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed840040d0 msgr2=0x7fed84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda2591640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed840040d0 0x7fed84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda2591640 1 -- 172.21.15.26:0/673074855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed84003520 con 0x7fed84004d40 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.066+0000 7feda1d90640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c158a00 0x7fed84003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.067+0000 7feda2591640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed84004d40 0x7fed840051c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fed9c15f990 tx=0x7fed84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1fd2fa682cf36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.067+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c001bf0 con 0x7fed84004d40 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.067+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8c003b70 con 0x7fed84004d40 2024-04-24T23:25:18.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.067+0000 7feda0d8e640 1 -- 172.21.15.26:0/673074855 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c0050d0 con 0x7fed84004d40 2024-04-24T23:25:18.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.068+0000 7feda3ff4640 1 -- 172.21.15.26:0/673074855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed84004d40 msgr2=0x7fed840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:18.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.068+0000 7feda3ff4640 1 --2- 172.21.15.26:0/673074855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed84004d40 0x7fed840051c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fed9c15f990 tx=0x7fed84005830 comp rx=0 tx=0).stop 2024-04-24T23:25:18.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.068+0000 7feda2591640 1 -- 172.21.15.26:0/673074855 reap_dead start 2024-04-24T23:25:18.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.068+0000 7feda3ff4640 1 -- 172.21.15.26:0/673074855 shutdown_connections 2024-04-24T23:25:18.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.068+0000 7feda3ff4640 1 -- 172.21.15.26:0/673074855 >> 172.21.15.26:0/673074855 conn(0x7fed9c0716e0 msgr2=0x7fed9c071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:18.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.068+0000 7feda3ff4640 1 -- 172.21.15.26:0/673074855 shutdown_connections 2024-04-24T23:25:18.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.068+0000 7feda3ff4640 1 -- 172.21.15.26:0/673074855 wait complete. 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.069+0000 7feda3ff4640 1 Processor -- start 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.069+0000 7feda3ff4640 1 -- start start 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda3ff4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed9c13efc0 0x7fed9c06d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda3ff4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed9c1f5000 0x7fed9c13f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda3ff4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c13f8e0 0x7fed9c1f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda3ff4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c0ae960 con 0x7fed9c13efc0 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda3ff4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c0ae7e0 con 0x7fed9c1f5000 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda3ff4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fed9c15f430 con 0x7fed9c13f8e0 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda1d90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed9c13efc0 0x7fed9c06d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda1d90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed9c13efc0 0x7fed9c06d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53822/0 (socket says 172.21.15.26:53822) 2024-04-24T23:25:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda1d90640 1 -- 172.21.15.26:0/2448170750 learned_addr learned my addr 172.21.15.26:0/2448170750 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda1d90640 1 -- 172.21.15.26:0/2448170750 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c13f8e0 msgr2=0x7fed9c1f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda1d90640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c13f8e0 0x7fed9c1f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda1d90640 1 -- 172.21.15.26:0/2448170750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed9c1f5000 msgr2=0x7fed9c13f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda158f640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed9c1f5000 0x7fed9c13f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda1d90640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed9c1f5000 0x7fed9c13f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda1d90640 1 -- 172.21.15.26:0/2448170750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c153100 con 0x7fed9c13efc0 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.070+0000 7feda1d90640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed9c13efc0 0x7fed9c06d850 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fed94011650 tx=0x7fed94011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.071+0000 7fed9a7fc640 1 -- 172.21.15.26:0/2448170750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9400f9c0 con 0x7fed9c13efc0 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.071+0000 7fed9a7fc640 1 -- 172.21.15.26:0/2448170750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9400fb60 con 0x7fed9c13efc0 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.071+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c151400 con 0x7fed9c13efc0 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.071+0000 7fed9a7fc640 1 -- 172.21.15.26:0/2448170750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94015460 con 0x7fed9c13efc0 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.071+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c1533a0 con 0x7fed9c13efc0 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.072+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9c0694c0 con 0x7fed9c13efc0 2024-04-24T23:25:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.072+0000 7fed9a7fc640 1 -- 172.21.15.26:0/2448170750 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed94015600 con 0x7fed9c13efc0 2024-04-24T23:25:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.073+0000 7fed9a7fc640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.073+0000 7fed9a7fc640 1 -- 172.21.15.26:0/2448170750 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5529+0+0 (secure 0 0 0) 0x7fed9408b630 con 0x7fed9c13efc0 2024-04-24T23:25:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.073+0000 7feda158f640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.073+0000 7feda158f640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fed90001050 tx=0x7fed90009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.077+0000 7fed9a7fc640 1 -- 172.21.15.26:0/2448170750 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fed94019030 con 0x7fed9c13efc0 2024-04-24T23:25:18.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.262+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed9c13fd30 con 0x7fed9c13efc0 2024-04-24T23:25:18.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.936+0000 7fed9a7fc640 1 -- 172.21.15.26:0/2448170750 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7fed9405a120 con 0x7fed9c13efc0 2024-04-24T23:25:18.937 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-24T23:25:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 msgr2=0x7fed8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fed90001050 tx=0x7fed90009040 comp rx=0 tx=0).stop 2024-04-24T23:25:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed9c13efc0 msgr2=0x7fed9c06d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed9c13efc0 0x7fed9c06d850 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fed94011650 tx=0x7fed94011b10 comp rx=0 tx=0).stop 2024-04-24T23:25:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 shutdown_connections 2024-04-24T23:25:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fed8806be60 0x7fed8806e320 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fed9c13f8e0 0x7fed9c1f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed9c1f5000 0x7fed9c13f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 --2- 172.21.15.26:0/2448170750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed9c13efc0 0x7fed9c06d850 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 >> 172.21.15.26:0/2448170750 conn(0x7fed9c0716e0 msgr2=0x7fed9c15e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 shutdown_connections 2024-04-24T23:25:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:18.940+0000 7feda3ff4640 1 -- 172.21.15.26:0/2448170750 wait complete. 2024-04-24T23:25:18.957 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.066+0000 7fca3a1c8640 1 Processor -- start 2024-04-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.066+0000 7fca3a1c8640 1 -- start start 2024-04-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca3a1c8640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca340ad7b0 0x7fca340d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca3a1c8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fca3415e9c0 0x7fca340d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca3a1c8640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3415ef60 con 0x7fca3415adc0 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca3a1c8640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fca3415f240 con 0x7fca3415e9c0 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca3a1c8640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fca3415f520 con 0x7fca340ad7b0 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca33fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fca3415e9c0 0x7fca340d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca337fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fca3415adc0 0x7fca340b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42800/0 (socket says 172.21.15.26:42800) 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca337fe640 1 -- 172.21.15.26:0/517445602 learned_addr learned my addr 172.21.15.26:0/517445602 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca32ffd640 1 --2- 172.21.15.26:0/517445602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca340ad7b0 0x7fca340d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca33fff640 1 -- 172.21.15.26:0/517445602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca340ad7b0 msgr2=0x7fca340d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca33fff640 1 --2- 172.21.15.26:0/517445602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca340ad7b0 0x7fca340d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca33fff640 1 -- 172.21.15.26:0/517445602 >> v1:172.21.15.26:6789/0 conn(0x7fca3415adc0 legacy=0x7fca340b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca33fff640 1 -- 172.21.15.26:0/517445602 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3415f800 con 0x7fca3415e9c0 2024-04-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.067+0000 7fca327fc640 1 -- 172.21.15.26:0/517445602 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1340107858 0 0) 0x7fca3415ef60 con 0x7fca3415adc0 2024-04-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.068+0000 7fca33fff640 1 --2- 172.21.15.26:0/517445602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 0x7fca340d0580 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009d90 tx=0x7fca2c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5367e83e481374a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.068+0000 7fca327fc640 1 -- 172.21.15.26:0/517445602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c031600 con 0x7fca3415e9c0 2024-04-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.068+0000 7fca327fc640 1 -- 172.21.15.26:0/517445602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca2c0317a0 con 0x7fca3415e9c0 2024-04-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.068+0000 7fca327fc640 1 -- 172.21.15.26:0/517445602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c031b70 con 0x7fca3415e9c0 2024-04-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.069+0000 7fca3a1c8640 1 -- 172.21.15.26:0/517445602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 msgr2=0x7fca340d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.069+0000 7fca3a1c8640 1 --2- 172.21.15.26:0/517445602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 0x7fca340d0580 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009d90 tx=0x7fca2c02a5d0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.069+0000 7fca3a1c8640 1 -- 172.21.15.26:0/517445602 shutdown_connections 2024-04-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.069+0000 7fca3a1c8640 1 --2- 172.21.15.26:0/517445602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 0x7fca340d0580 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.069+0000 7fca3a1c8640 1 --2- 172.21.15.26:0/517445602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca340ad7b0 0x7fca340d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.069+0000 7fca3a1c8640 1 -- 172.21.15.26:0/517445602 >> 172.21.15.26:0/517445602 conn(0x7fca3406b540 msgr2=0x7fca3406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.069+0000 7fca3a1c8640 1 -- 172.21.15.26:0/517445602 shutdown_connections 2024-04-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.069+0000 7fca3a1c8640 1 -- 172.21.15.26:0/517445602 wait complete. 2024-04-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.070+0000 7fca3a1c8640 1 Processor -- start 2024-04-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.070+0000 7fca3a1c8640 1 -- start start 2024-04-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca3a1c8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca340ad7b0 0x7fca341f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca3a1c8640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca3415adc0 0x7fca341f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca3a1c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 0x7fca3409c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca3a1c8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fca3406cb90 con 0x7fca3415e9c0 2024-04-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca3a1c8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fca3415b1a0 con 0x7fca340ad7b0 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca3a1c8640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fca3406cd10 con 0x7fca3415adc0 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca33fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 0x7fca3409c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca33fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 0x7fca3409c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53826/0 (socket says 172.21.15.26:53826) 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca33fff640 1 -- 172.21.15.26:0/1086528782 learned_addr learned my addr 172.21.15.26:0/1086528782 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca32ffd640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca3415adc0 0x7fca341f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca337fe640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca340ad7b0 0x7fca341f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca33fff640 1 -- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca3415adc0 msgr2=0x7fca341f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca33fff640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca3415adc0 0x7fca341f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca33fff640 1 -- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca340ad7b0 msgr2=0x7fca341f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca33fff640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca340ad7b0 0x7fca341f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca33fff640 1 -- 172.21.15.26:0/1086528782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3409bd80 con 0x7fca3415e9c0 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca32ffd640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca3415adc0 0x7fca341f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.071+0000 7fca337fe640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca340ad7b0 0x7fca341f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.072+0000 7fca33fff640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 0x7fca3409c5f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009eb0 tx=0x7fca2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.072+0000 7fca1bfff640 1 -- 172.21.15.26:0/1086528782 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c03f070 con 0x7fca3415e9c0 2024-04-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.072+0000 7fca1bfff640 1 -- 172.21.15.26:0/1086528782 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca2c030b60 con 0x7fca3415e9c0 2024-04-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.072+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3409c060 con 0x7fca3415e9c0 2024-04-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.072+0000 7fca1bfff640 1 -- 172.21.15.26:0/1086528782 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c009450 con 0x7fca3415e9c0 2024-04-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.072+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3415b3c0 con 0x7fca3415e9c0 2024-04-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.073+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca34075610 con 0x7fca3415e9c0 2024-04-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.073+0000 7fca1bfff640 1 -- 172.21.15.26:0/1086528782 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca2c03b410 con 0x7fca3415e9c0 2024-04-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.074+0000 7fca1bfff640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca2006be60 0x7fca2006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.074+0000 7fca1bfff640 1 -- 172.21.15.26:0/1086528782 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7fca2c0ac230 con 0x7fca3415e9c0 2024-04-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.074+0000 7fca337fe640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca2006be60 0x7fca2006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.074+0000 7fca337fe640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca2006be60 0x7fca2006e320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fca24000f60 tx=0x7fca24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:19.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.078+0000 7fca1bfff640 1 -- 172.21.15.26:0/1086528782 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fca2c07ae20 con 0x7fca3415e9c0 2024-04-24T23:25:19.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.273+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fca3415e730 con 0x7fca3415e9c0 2024-04-24T23:25:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.276+0000 7fca1bfff640 1 -- 172.21.15.26:0/1086528782 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fca2c07a720 con 0x7fca3415e9c0 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca2006be60 msgr2=0x7fca2006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca2006be60 0x7fca2006e320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fca24000f60 tx=0x7fca24009040 comp rx=0 tx=0).stop 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 msgr2=0x7fca3409c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 0x7fca3409c5f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009eb0 tx=0x7fca2c004a40 comp rx=0 tx=0).stop 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 shutdown_connections 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca2006be60 0x7fca2006e320 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3415e9c0 0x7fca3409c5f0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca3415adc0 0x7fca341f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 --2- 172.21.15.26:0/1086528782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca340ad7b0 0x7fca341f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 >> 172.21.15.26:0/1086528782 conn(0x7fca3406b540 msgr2=0x7fca3414e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 shutdown_connections 2024-04-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.278+0000 7fca3a1c8640 1 -- 172.21.15.26:0/1086528782 wait complete. 2024-04-24T23:25:19.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-04-24T23:25:19.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.391+0000 7f5989ff4640 1 Processor -- start 2024-04-24T23:25:19.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.391+0000 7f5989ff4640 1 -- start start 2024-04-24T23:25:19.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.391+0000 7f5989ff4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59840d4870 0x7f59840b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.391+0000 7f5989ff4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59840d32f0 con 0x7f59840b3130 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.391+0000 7f5989ff4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59840d0270 con 0x7f598415e570 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.391+0000 7f5989ff4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f59840d1ab0 con 0x7f59840d4870 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f5983fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f59840b3130 0x7f59840b4640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42804/0 (socket says 172.21.15.26:42804) 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f5983fff640 1 -- 172.21.15.26:0/1126278833 learned_addr learned my addr 172.21.15.26:0/1126278833 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f5982ffd640 1 --2- 172.21.15.26:0/1126278833 >> v2:172.21.15.29:3300/0 conn(0x7f59840d4870 0x7f59840b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2476978725 0 0) 0x7f597c001490 con 0x7f59840b3130 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f596c003520 con 0x7f59840b3130 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2639320969 0 0) 0x7f596c003520 con 0x7f59840b3130 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 >> v2:172.21.15.29:3300/0 conn(0x7f59840d4870 msgr2=0x7f59840b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 --2- 172.21.15.26:0/1126278833 >> v2:172.21.15.29:3300/0 conn(0x7f59840d4870 0x7f59840b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 >> v1:172.21.15.29:6789/0 conn(0x7f598415e570 legacy=0x7f59840d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59840d2d30 con 0x7f59840b3130 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f597c001bc0 con 0x7f59840b3130 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 >> v1:172.21.15.26:6789/0 conn(0x7f59840b3130 legacy=0x7f59840b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f596c0039b0 0x7f596c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f596c0043b0 0x7f596c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596c005120 0x7f596c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59840d1ab0 con 0x7f596c005120 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f596c00c060 con 0x7f596c0043b0 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f596c00c260 con 0x7f596c0039b0 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f5983fff640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596c005120 0x7f596c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f59837fe640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f596c0039b0 0x7f596c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.392+0000 7f5982ffd640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f596c0043b0 0x7f596c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f5983fff640 1 -- 172.21.15.26:0/1126278833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f596c0039b0 msgr2=0x7f596c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f5983fff640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f596c0039b0 0x7f596c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f5983fff640 1 -- 172.21.15.26:0/1126278833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f596c0043b0 msgr2=0x7f596c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f5983fff640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f596c0043b0 0x7f596c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f5983fff640 1 -- 172.21.15.26:0/1126278833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c003520 con 0x7f596c005120 2024-04-24T23:25:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f5983fff640 1 -- 172.21.15.26:0/1126278833 reap_dead start 2024-04-24T23:25:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f5983fff640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596c005120 0x7f596c00b9c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f597c00aa50 tx=0x7f596c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f372c4b999db95f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c00a670 con 0x7f596c005120 2024-04-24T23:25:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597c02f070 con 0x7f596c005120 2024-04-24T23:25:19.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.393+0000 7f59827fc640 1 -- 172.21.15.26:0/1126278833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c034470 con 0x7f596c005120 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.394+0000 7f5989ff4640 1 -- 172.21.15.26:0/1126278833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596c005120 msgr2=0x7f596c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.394+0000 7f5989ff4640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596c005120 0x7f596c00b9c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f597c00aa50 tx=0x7f596c005830 comp rx=0 tx=0).stop 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.394+0000 7f5989ff4640 1 -- 172.21.15.26:0/1126278833 shutdown_connections 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.394+0000 7f5989ff4640 1 --2- 172.21.15.26:0/1126278833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596c005120 0x7f596c00b9c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.394+0000 7f5989ff4640 1 -- 172.21.15.26:0/1126278833 >> 172.21.15.26:0/1126278833 conn(0x7f598408dbe0 msgr2=0x7f598409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.394+0000 7f5989ff4640 1 -- 172.21.15.26:0/1126278833 shutdown_connections 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.394+0000 7f5989ff4640 1 -- 172.21.15.26:0/1126278833 wait complete. 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.395+0000 7f5989ff4640 1 Processor -- start 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.395+0000 7f5989ff4640 1 -- start start 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f5989ff4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f596c005120 0x7f59841525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f5989ff4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5984159650 0x7f5984159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f5989ff4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f598415b810 0x7f598415bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f5989ff4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59840cfa70 con 0x7f598415b810 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f5989ff4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59840d2ae0 con 0x7f5984159650 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f5989ff4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59840cfd50 con 0x7f596c005120 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f5983fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f598415b810 0x7f598415bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f5983fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f598415b810 0x7f598415bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53836/0 (socket says 172.21.15.26:53836) 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f5983fff640 1 -- 172.21.15.26:0/1529833878 learned_addr learned my addr 172.21.15.26:0/1529833878 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.396+0000 7f59837fe640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f596c005120 0x7f59841525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f5982ffd640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5984159650 0x7f5984159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f5983fff640 1 -- 172.21.15.26:0/1529833878 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f596c005120 msgr2=0x7f59841525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f5983fff640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f596c005120 0x7f59841525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f5983fff640 1 -- 172.21.15.26:0/1529833878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5984159650 msgr2=0x7f5984159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f5983fff640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5984159650 0x7f5984159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f5983fff640 1 -- 172.21.15.26:0/1529833878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598415c1d0 con 0x7f598415b810 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f5983fff640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f598415b810 0x7f598415bc90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f597c00ab70 tx=0x7f59840d2f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f596bfff640 1 -- 172.21.15.26:0/1529833878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c001b90 con 0x7f598415b810 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f596bfff640 1 -- 172.21.15.26:0/1529833878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597c004b60 con 0x7f598415b810 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59841f76a0 con 0x7f598415b810 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f596bfff640 1 -- 172.21.15.26:0/1529833878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c040480 con 0x7f598415b810 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.397+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59841f7c60 con 0x7f598415b810 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.398+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59840694c0 con 0x7f598415b810 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.399+0000 7f596bfff640 1 -- 172.21.15.26:0/1529833878 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f597c034470 con 0x7f598415b810 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.399+0000 7f596bfff640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f597006be30 0x7f597006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.399+0000 7f596bfff640 1 -- 172.21.15.26:0/1529833878 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f597c0ab020 con 0x7f598415b810 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.399+0000 7f59837fe640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f597006be30 0x7f597006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.399+0000 7f59837fe640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f597006be30 0x7f597006e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5974003640 tx=0x7f5974009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:19.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.403+0000 7f596bfff640 1 -- 172.21.15.26:0/1529833878 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f59840d32f0 con 0x7f598415b810 2024-04-24T23:25:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.660+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) -- 0x7f59840cf080 con 0x7f598415b810 2024-04-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.942+0000 7f596bfff640 1 -- 172.21.15.26:0/1529833878 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v234) ==== 142+0+0 (secure 0 0 0) 0x7f597c079cb0 con 0x7f598415b810 2024-04-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f597006be30 msgr2=0x7f597006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f597006be30 0x7f597006e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5974003640 tx=0x7f5974009040 comp rx=0 tx=0).stop 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f598415b810 msgr2=0x7f598415bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f598415b810 0x7f598415bc90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f597c00ab70 tx=0x7f59840d2f00 comp rx=0 tx=0).stop 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 shutdown_connections 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f597006be30 0x7f597006e2f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f598415b810 0x7f598415bc90 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5984159650 0x7f5984159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 --2- 172.21.15.26:0/1529833878 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f596c005120 0x7f59841525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 >> 172.21.15.26:0/1529833878 conn(0x7f598408dbe0 msgr2=0x7f598406d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 shutdown_connections 2024-04-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:19.944+0000 7f5989ff4640 1 -- 172.21.15.26:0/1529833878 wait complete. 2024-04-24T23:25:19.957 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-04-24T23:25:19.958 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-04-24T23:25:19.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:25:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.058+0000 7f282694c640 1 Processor -- start 2024-04-24T23:25:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.058+0000 7f282694c640 1 -- start start 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.059+0000 7f282694c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2820143c40 0x7f28200cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.059+0000 7f282694c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28200cc860 0x7f2820072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.059+0000 7f282694c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28200cd0b0 con 0x7f282015eb00 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.059+0000 7f282694c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2820071b70 con 0x7f2820143c40 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.059+0000 7f282694c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2820071240 con 0x7f28200cc860 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.060+0000 7f281f7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28200cc860 0x7f2820072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.060+0000 7f281ffff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2820143c40 0x7f28200cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.060+0000 7f281f7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28200cc860 0x7f2820072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:33666/0 (socket says 172.21.15.26:33666) 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.060+0000 7f281f7fe640 1 -- 172.21.15.26:0/3195167322 learned_addr learned my addr 172.21.15.26:0/3195167322 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.060+0000 7f2824ee9640 1 --1- 172.21.15.26:0/3195167322 >> v1:172.21.15.29:6789/0 conn(0x7f282015eb00 0x7f28200b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:46612/0 (socket says 172.21.15.26:46612) 2024-04-24T23:25:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.060+0000 7f281f7fe640 1 -- 172.21.15.26:0/3195167322 >> v2:172.21.15.29:3300/0 conn(0x7f2820143c40 msgr2=0x7f28200cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.060+0000 7f281f7fe640 1 --2- 172.21.15.26:0/3195167322 >> v2:172.21.15.29:3300/0 conn(0x7f2820143c40 0x7f28200cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.060+0000 7f281f7fe640 1 -- 172.21.15.26:0/3195167322 >> v1:172.21.15.29:6789/0 conn(0x7f282015eb00 legacy=0x7f28200b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.060+0000 7f281f7fe640 1 -- 172.21.15.26:0/3195167322 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2820071520 con 0x7f28200cc860 2024-04-24T23:25:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f281f7fe640 1 --2- 172.21.15.26:0/3195167322 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28200cc860 0x7f2820072000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2810009870 tx=0x7f281002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c06c2426f5d834e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f281effd640 1 -- 172.21.15.26:0/3195167322 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810037070 con 0x7f28200cc860 2024-04-24T23:25:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f281effd640 1 -- 172.21.15.26:0/3195167322 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2810032440 con 0x7f28200cc860 2024-04-24T23:25:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f281effd640 1 -- 172.21.15.26:0/3195167322 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810032810 con 0x7f28200cc860 2024-04-24T23:25:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f282694c640 1 -- 172.21.15.26:0/3195167322 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28200cc860 msgr2=0x7f2820072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f282694c640 1 --2- 172.21.15.26:0/3195167322 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28200cc860 0x7f2820072000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2810009870 tx=0x7f281002a270 comp rx=0 tx=0).stop 2024-04-24T23:25:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f282694c640 1 -- 172.21.15.26:0/3195167322 shutdown_connections 2024-04-24T23:25:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f282694c640 1 --2- 172.21.15.26:0/3195167322 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f28200cc860 0x7f2820072000 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f282694c640 1 --2- 172.21.15.26:0/3195167322 >> v2:172.21.15.29:3300/0 conn(0x7f2820143c40 0x7f28200cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.061+0000 7f282694c640 1 -- 172.21.15.26:0/3195167322 >> 172.21.15.26:0/3195167322 conn(0x7f282006b300 msgr2=0x7f282006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.062+0000 7f282694c640 1 -- 172.21.15.26:0/3195167322 shutdown_connections 2024-04-24T23:25:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.062+0000 7f282694c640 1 -- 172.21.15.26:0/3195167322 wait complete. 2024-04-24T23:25:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.063+0000 7f282694c640 1 Processor -- start 2024-04-24T23:25:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.063+0000 7f282694c640 1 -- start start 2024-04-24T23:25:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.063+0000 7f282694c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28200cc860 0x7f2820168980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.063+0000 7f282694c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2820143c40 0x7f282015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.063+0000 7f282694c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f282015eb00 0x7f2820141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.063+0000 7f282694c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2820072540 con 0x7f28200cc860 2024-04-24T23:25:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.063+0000 7f282694c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2820144020 con 0x7f282015eb00 2024-04-24T23:25:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.063+0000 7f282694c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f282013de60 con 0x7f2820143c40 2024-04-24T23:25:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28200cc860 0x7f2820168980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28200cc860 0x7f2820168980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53838/0 (socket says 172.21.15.26:53838) 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281ffff640 1 -- 172.21.15.26:0/3162753027 learned_addr learned my addr 172.21.15.26:0/3162753027 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f2824ee9640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f282015eb00 0x7f2820141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281f7fe640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2820143c40 0x7f282015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281ffff640 1 -- 172.21.15.26:0/3162753027 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2820143c40 msgr2=0x7f282015a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281ffff640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2820143c40 0x7f282015a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281ffff640 1 -- 172.21.15.26:0/3162753027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f282015eb00 msgr2=0x7f2820141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281ffff640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f282015eb00 0x7f2820141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281ffff640 1 -- 172.21.15.26:0/3162753027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282015ae30 con 0x7f28200cc860 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281ffff640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28200cc860 0x7f2820168980 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9b0 tx=0x7f280800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281cff9640 1 -- 172.21.15.26:0/3162753027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808017070 con 0x7f28200cc860 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281cff9640 1 -- 172.21.15.26:0/3162753027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280800c9a0 con 0x7f28200cc860 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2820163860 con 0x7f28200cc860 2024-04-24T23:25:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f281cff9640 1 -- 172.21.15.26:0/3162753027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280800cd90 con 0x7f28200cc860 2024-04-24T23:25:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.064+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2820163e20 con 0x7f28200cc860 2024-04-24T23:25:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.065+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28200694c0 con 0x7f28200cc860 2024-04-24T23:25:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.066+0000 7f281cff9640 1 -- 172.21.15.26:0/3162753027 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f280801a070 con 0x7f28200cc860 2024-04-24T23:25:20.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.066+0000 7f281cff9640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f280006be30 0x7f280006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.066+0000 7f281cff9640 1 -- 172.21.15.26:0/3162753027 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f280808ab50 con 0x7f28200cc860 2024-04-24T23:25:20.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.066+0000 7f281f7fe640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f280006be30 0x7f280006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.067+0000 7f281f7fe640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f280006be30 0x7f280006e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2810034000 tx=0x7f281002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:20.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.070+0000 7f281cff9640 1 -- 172.21.15.26:0/3162753027 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f282015ae30 con 0x7f28200cc860 2024-04-24T23:25:20.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.255+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28200b4d20 con 0x7f28200cc860 2024-04-24T23:25:20.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.255+0000 7f281cff9640 1 -- 172.21.15.26:0/3162753027 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+13220 (secure 0 0 0) 0x7f2808059100 con 0x7f28200cc860 2024-04-24T23:25:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:20.257 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":61,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:25:18.935372+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6837/2743842996":"2024-04-25T23:25:15.928255+0000","172.21.15.26:6835/2743842996":"2024-04-25T23:25:15.928255+0000","172.21.15.29:6835/2096523178":"2024-04-25T23:25:15.928259+0000","172.21.15.29:6833/2096523178":"2024-04-25T23:25:15.928259+0000","172.21.15.26:6836/1519104007":"2024-04-25T23:25:15.928243+0000","172.21.15.26:6834/1519104007":"2024-04-25T23:25:15.928243+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:25:20.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f280006be30 msgr2=0x7f280006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f280006be30 0x7f280006e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2810034000 tx=0x7f281002e000 comp rx=0 tx=0).stop 2024-04-24T23:25:20.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28200cc860 msgr2=0x7f2820168980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28200cc860 0x7f2820168980 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9b0 tx=0x7f280800ee70 comp rx=0 tx=0).stop 2024-04-24T23:25:20.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 shutdown_connections 2024-04-24T23:25:20.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f280006be30 0x7f280006e2f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f282015eb00 0x7f2820141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2820143c40 0x7f282015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 --2- 172.21.15.26:0/3162753027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28200cc860 0x7f2820168980 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 >> 172.21.15.26:0/3162753027 conn(0x7f282006b300 msgr2=0x7f28200b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:20.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 shutdown_connections 2024-04-24T23:25:20.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.257+0000 7f282694c640 1 -- 172.21.15.26:0/3162753027 wait complete. 2024-04-24T23:25:20.269 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:25:20.270 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:25:20.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.372+0000 7f87e0e72640 1 Processor -- start 2024-04-24T23:25:20.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.372+0000 7f87e0e72640 1 -- start start 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.373+0000 7f87e0e72640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87dc15e750 0x7f87dc0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.373+0000 7f87e0e72640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f87dc0d2c30 0x7f87dc0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.373+0000 7f87e0e72640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87dc0d2120 con 0x7f87dc15e370 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.373+0000 7f87e0e72640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f87dc0d1510 con 0x7f87dc15e750 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.373+0000 7f87e0e72640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f87dc0d1ed0 con 0x7f87dc0d2c30 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.373+0000 7f87d9d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87dc15e750 0x7f87dc0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.373+0000 7f87da59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87dc15e370 0x7f87dc0d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42810/0 (socket says 172.21.15.26:42810) 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.374+0000 7f87d9d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87dc15e750 0x7f87dc0cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53848/0 (socket says 172.21.15.26:53848) 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.374+0000 7f87da59c640 1 -- 172.21.15.26:0/1151835517 learned_addr learned my addr 172.21.15.26:0/1151835517 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.374+0000 7f87dad9d640 1 --2- 172.21.15.26:0/1151835517 >> v2:172.21.15.29:3300/0 conn(0x7f87dc0d2c30 0x7f87dc0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.374+0000 7f87d9d9b640 1 -- 172.21.15.26:0/1151835517 >> v2:172.21.15.29:3300/0 conn(0x7f87dc0d2c30 msgr2=0x7f87dc0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.374+0000 7f87d9d9b640 1 --2- 172.21.15.26:0/1151835517 >> v2:172.21.15.29:3300/0 conn(0x7f87dc0d2c30 0x7f87dc0cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.374+0000 7f87d9d9b640 1 -- 172.21.15.26:0/1151835517 >> v1:172.21.15.26:6789/0 conn(0x7f87dc15e370 legacy=0x7f87dc0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.374+0000 7f87d9d9b640 1 -- 172.21.15.26:0/1151835517 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc0b3740 con 0x7f87dc15e750 2024-04-24T23:25:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.374+0000 7f87d959a640 1 -- 172.21.15.26:0/1151835517 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751692042 0 0) 0x7f87dc0d2120 con 0x7f87dc15e370 2024-04-24T23:25:20.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87d9d9b640 1 --2- 172.21.15.26:0/1151835517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15e750 0x7f87dc0cec90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f87dc08d9a0 tx=0x7f87c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894658442d82a802 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:20.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87d959a640 1 -- 172.21.15.26:0/1151835517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4037070 con 0x7f87dc15e750 2024-04-24T23:25:20.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87d959a640 1 -- 172.21.15.26:0/1151835517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c4032440 con 0x7f87dc15e750 2024-04-24T23:25:20.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87d959a640 1 -- 172.21.15.26:0/1151835517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4032810 con 0x7f87dc15e750 2024-04-24T23:25:20.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87e0e72640 1 -- 172.21.15.26:0/1151835517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15e750 msgr2=0x7f87dc0cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87e0e72640 1 --2- 172.21.15.26:0/1151835517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15e750 0x7f87dc0cec90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f87dc08d9a0 tx=0x7f87c402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87e0e72640 1 -- 172.21.15.26:0/1151835517 shutdown_connections 2024-04-24T23:25:20.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87e0e72640 1 --2- 172.21.15.26:0/1151835517 >> v2:172.21.15.29:3300/0 conn(0x7f87dc0d2c30 0x7f87dc0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87e0e72640 1 --2- 172.21.15.26:0/1151835517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15e750 0x7f87dc0cec90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87e0e72640 1 -- 172.21.15.26:0/1151835517 >> 172.21.15.26:0/1151835517 conn(0x7f87dc0d40f0 msgr2=0x7f87dc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:20.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87e0e72640 1 -- 172.21.15.26:0/1151835517 shutdown_connections 2024-04-24T23:25:20.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.375+0000 7f87e0e72640 1 -- 172.21.15.26:0/1151835517 wait complete. 2024-04-24T23:25:20.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.377+0000 7f87e0e72640 1 Processor -- start 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.377+0000 7f87e0e72640 1 -- start start 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.377+0000 7f87e0e72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87dc0d2c30 0x7f87dc15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.377+0000 7f87e0e72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87dc15e370 0x7f87dc0b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.377+0000 7f87e0e72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc0b81b0 0x7f87dc0b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.377+0000 7f87e0e72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc0b5230 con 0x7f87dc0b81b0 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.377+0000 7f87e0e72640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc0b3d00 con 0x7f87dc15e370 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.377+0000 7f87e0e72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f87dc0b53b0 con 0x7f87dc0d2c30 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87dad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc0b81b0 0x7f87dc0b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87dad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc0b81b0 0x7f87dc0b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53862/0 (socket says 172.21.15.26:53862) 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87dad9d640 1 -- 172.21.15.26:0/3065011141 learned_addr learned my addr 172.21.15.26:0/3065011141 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87d9d9b640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87dc15e370 0x7f87dc0b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87da59c640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87dc0d2c30 0x7f87dc15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87dad9d640 1 -- 172.21.15.26:0/3065011141 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87dc0d2c30 msgr2=0x7f87dc15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87dad9d640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87dc0d2c30 0x7f87dc15b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87dad9d640 1 -- 172.21.15.26:0/3065011141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87dc15e370 msgr2=0x7f87dc0b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87dad9d640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87dc15e370 0x7f87dc0b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87dad9d640 1 -- 172.21.15.26:0/3065011141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc0cd1a0 con 0x7f87dc0b81b0 2024-04-24T23:25:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87dad9d640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc0b81b0 0x7f87dc0b16b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00cc30 tx=0x7f87cc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87d2ffd640 1 -- 172.21.15.26:0/3065011141 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00ce40 con 0x7f87dc0b81b0 2024-04-24T23:25:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87d2ffd640 1 -- 172.21.15.26:0/3065011141 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc003990 con 0x7f87dc0b81b0 2024-04-24T23:25:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc0cd420 con 0x7f87dc0b81b0 2024-04-24T23:25:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87d2ffd640 1 -- 172.21.15.26:0/3065011141 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc003d40 con 0x7f87dc0b81b0 2024-04-24T23:25:20.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.378+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc0cd9e0 con 0x7f87dc0b81b0 2024-04-24T23:25:20.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.379+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87dc075610 con 0x7f87dc0b81b0 2024-04-24T23:25:20.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.380+0000 7f87d2ffd640 1 -- 172.21.15.26:0/3065011141 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87cc003ee0 con 0x7f87dc0b81b0 2024-04-24T23:25:20.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.380+0000 7f87d2ffd640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87bc06be30 0x7f87bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.380+0000 7f87da59c640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87bc06be30 0x7f87bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.380+0000 7f87d2ffd640 1 -- 172.21.15.26:0/3065011141 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f87cc08b020 con 0x7f87dc0b81b0 2024-04-24T23:25:20.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.381+0000 7f87da59c640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87bc06be30 0x7f87bc06e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f87d4007fd0 tx=0x7f87d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:20.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.385+0000 7f87d2ffd640 1 -- 172.21.15.26:0/3065011141 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f87dc0cd1a0 con 0x7f87dc0b81b0 2024-04-24T23:25:20.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.568+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87dc0b6a40 con 0x7f87dc0b81b0 2024-04-24T23:25:20.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.568+0000 7f87d2ffd640 1 -- 172.21.15.26:0/3065011141 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7f87cc014030 con 0x7f87dc0b81b0 2024-04-24T23:25:20.569 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:25:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87bc06be30 msgr2=0x7f87bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87bc06be30 0x7f87bc06e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f87d4007fd0 tx=0x7f87d4009040 comp rx=0 tx=0).stop 2024-04-24T23:25:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc0b81b0 msgr2=0x7f87dc0b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc0b81b0 0x7f87dc0b16b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00cc30 tx=0x7f87cc00a5c0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 shutdown_connections 2024-04-24T23:25:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87bc06be30 0x7f87bc06e2f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc0b81b0 0x7f87dc0b16b0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87dc15e370 0x7f87dc0b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 --2- 172.21.15.26:0/3065011141 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87dc0d2c30 0x7f87dc15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 >> 172.21.15.26:0/3065011141 conn(0x7f87dc0d40f0 msgr2=0x7f87dc15d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 shutdown_connections 2024-04-24T23:25:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.570+0000 7f87e0e72640 1 -- 172.21.15.26:0/3065011141 wait complete. 2024-04-24T23:25:20.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-04-24T23:25:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.681+0000 7f18a0e9e640 1 Processor -- start 2024-04-24T23:25:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.681+0000 7f18a0e9e640 1 -- start start 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.682+0000 7f18a0e9e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f189c15a480 0x7f189c0b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.682+0000 7f18a0e9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c15a860 0x7f189c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.682+0000 7f18a0e9e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189c0b2e80 con 0x7f189c0b4e30 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.682+0000 7f18a0e9e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f189c0b3120 con 0x7f189c15a480 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.682+0000 7f18a0e9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f189c0b33c0 con 0x7f189c15a860 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f189ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f189c0b4e30 0x7f189c15e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42826/0 (socket says 172.21.15.26:42826) 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f189ad9d640 1 -- 172.21.15.26:0/1917251795 learned_addr learned my addr 172.21.15.26:0/1917251795 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f1899d9b640 1 --2- 172.21.15.26:0/1917251795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c15a860 0x7f189c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f189a59c640 1 --2- 172.21.15.26:0/1917251795 >> v2:172.21.15.29:3300/0 conn(0x7f189c15a480 0x7f189c0b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f189959a640 1 -- 172.21.15.26:0/1917251795 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50786825 0 0) 0x7f189c0b2e80 con 0x7f189c0b4e30 2024-04-24T23:25:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f189959a640 1 -- 172.21.15.26:0/1917251795 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1884003520 con 0x7f189c0b4e30 2024-04-24T23:25:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f189a59c640 1 -- 172.21.15.26:0/1917251795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c15a860 msgr2=0x7f189c06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f189a59c640 1 --2- 172.21.15.26:0/1917251795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c15a860 0x7f189c06dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f189a59c640 1 -- 172.21.15.26:0/1917251795 >> v1:172.21.15.26:6789/0 conn(0x7f189c0b4e30 legacy=0x7f189c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.683+0000 7f189a59c640 1 -- 172.21.15.26:0/1917251795 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c0b36a0 con 0x7f189c15a480 2024-04-24T23:25:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.684+0000 7f189a59c640 1 --2- 172.21.15.26:0/1917251795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189c15a480 0x7f189c0b48f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f18940097a0 tx=0x7f189402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e98adf4597a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.684+0000 7f189959a640 1 -- 172.21.15.26:0/1917251795 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894037070 con 0x7f189c15a480 2024-04-24T23:25:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.684+0000 7f189959a640 1 -- 172.21.15.26:0/1917251795 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189402ae50 con 0x7f189c15a480 2024-04-24T23:25:20.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.684+0000 7f189959a640 1 -- 172.21.15.26:0/1917251795 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894032640 con 0x7f189c15a480 2024-04-24T23:25:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.685+0000 7f18a0e9e640 1 -- 172.21.15.26:0/1917251795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189c15a480 msgr2=0x7f189c0b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.685+0000 7f18a0e9e640 1 --2- 172.21.15.26:0/1917251795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189c15a480 0x7f189c0b48f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f18940097a0 tx=0x7f189402a100 comp rx=0 tx=0).stop 2024-04-24T23:25:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.685+0000 7f18a0e9e640 1 -- 172.21.15.26:0/1917251795 shutdown_connections 2024-04-24T23:25:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.685+0000 7f18a0e9e640 1 --2- 172.21.15.26:0/1917251795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c15a860 0x7f189c06dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.685+0000 7f18a0e9e640 1 --2- 172.21.15.26:0/1917251795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189c15a480 0x7f189c0b48f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.685+0000 7f18a0e9e640 1 -- 172.21.15.26:0/1917251795 >> 172.21.15.26:0/1917251795 conn(0x7f189c06b300 msgr2=0x7f189c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.685+0000 7f18a0e9e640 1 -- 172.21.15.26:0/1917251795 shutdown_connections 2024-04-24T23:25:20.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.685+0000 7f18a0e9e640 1 -- 172.21.15.26:0/1917251795 wait complete. 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.686+0000 7f18a0e9e640 1 Processor -- start 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.686+0000 7f18a0e9e640 1 -- start start 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f18a0e9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c0b4e30 0x7f189c1f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f18a0e9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c15a480 0x7f189c13b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f18a0e9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189c15a860 0x7f189c14afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f18a0e9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f189c072080 con 0x7f189c15a480 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f18a0e9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f189c0b3c60 con 0x7f189c15a860 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f18a0e9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f189c072200 con 0x7f189c0b4e30 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f1899d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c15a480 0x7f189c13b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f1899d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c15a480 0x7f189c13b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53866/0 (socket says 172.21.15.26:53866) 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f1899d9b640 1 -- 172.21.15.26:0/3819879813 learned_addr learned my addr 172.21.15.26:0/3819879813 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f189a59c640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c0b4e30 0x7f189c1f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f189ad9d640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189c15a860 0x7f189c14afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f1899d9b640 1 -- 172.21.15.26:0/3819879813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c0b4e30 msgr2=0x7f189c1f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f1899d9b640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c0b4e30 0x7f189c1f3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f1899d9b640 1 -- 172.21.15.26:0/3819879813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189c15a860 msgr2=0x7f189c14afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f1899d9b640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189c15a860 0x7f189c14afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f1899d9b640 1 -- 172.21.15.26:0/3819879813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c1f5450 con 0x7f189c15a480 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.687+0000 7f1899d9b640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c15a480 0x7f189c13b570 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f189000e910 tx=0x7f189000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.688+0000 7f188affd640 1 -- 172.21.15.26:0/3819879813 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890016070 con 0x7f189c15a480 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.688+0000 7f188affd640 1 -- 172.21.15.26:0/3819879813 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189000cc40 con 0x7f189c15a480 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.688+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1f5730 con 0x7f189c15a480 2024-04-24T23:25:20.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.688+0000 7f188affd640 1 -- 172.21.15.26:0/3819879813 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000b040 con 0x7f189c15a480 2024-04-24T23:25:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.688+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1f5cf0 con 0x7f189c15a480 2024-04-24T23:25:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.689+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c0694c0 con 0x7f189c15a480 2024-04-24T23:25:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.689+0000 7f188affd640 1 -- 172.21.15.26:0/3819879813 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1890019070 con 0x7f189c15a480 2024-04-24T23:25:20.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.690+0000 7f188affd640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f187c06be30 0x7f187c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.690+0000 7f189a59c640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f187c06be30 0x7f187c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.690+0000 7f188affd640 1 -- 172.21.15.26:0/3819879813 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f189008ab40 con 0x7f189c15a480 2024-04-24T23:25:20.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.690+0000 7f189a59c640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f187c06be30 0x7f187c06e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1894034000 tx=0x7f189402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.694+0000 7f188affd640 1 -- 172.21.15.26:0/3819879813 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18900597d0 con 0x7f189c15a480 2024-04-24T23:25:20.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:20.946+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) -- 0x7f189c0cf640 con 0x7f189c15a480 2024-04-24T23:25:21.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.948+0000 7f188affd640 1 -- 172.21.15.26:0/3819879813 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v236) ==== 144+0+0 (secure 0 0 0) 0x7f18900590d0 con 0x7f189c15a480 2024-04-24T23:25:21.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f187c06be30 msgr2=0x7f187c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:21.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f187c06be30 0x7f187c06e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1894034000 tx=0x7f189402e000 comp rx=0 tx=0).stop 2024-04-24T23:25:21.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c15a480 msgr2=0x7f189c13b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c15a480 0x7f189c13b570 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f189000e910 tx=0x7f189000edd0 comp rx=0 tx=0).stop 2024-04-24T23:25:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 shutdown_connections 2024-04-24T23:25:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f187c06be30 0x7f187c06e2f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f189c15a860 0x7f189c14afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c15a480 0x7f189c13b570 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 --2- 172.21.15.26:0/3819879813 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f189c0b4e30 0x7f189c1f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 >> 172.21.15.26:0/3819879813 conn(0x7f189c06b300 msgr2=0x7f189c0cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 shutdown_connections 2024-04-24T23:25:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:21.951+0000 7f18a0e9e640 1 -- 172.21.15.26:0/3819879813 wait complete. 2024-04-24T23:25:21.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:25:22.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.075+0000 7fa6c8516640 1 Processor -- start 2024-04-24T23:25:22.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.075+0000 7fa6c8516640 1 -- start start 2024-04-24T23:25:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c8516640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa6c0158ae0 0x7fa6c006e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c8516640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00d0ce0 0x7fa6c015e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c8516640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c006b1c0 con 0x7fa6c015ee90 2024-04-24T23:25:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c8516640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c006ec90 con 0x7fa6c0158ae0 2024-04-24T23:25:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c8516640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c006ef70 con 0x7fa6c00d0ce0 2024-04-24T23:25:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c6ab3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa6c015ee90 0x7fa6c0071720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42834/0 (socket says 172.21.15.26:42834) 2024-04-24T23:25:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c6ab3640 1 -- 172.21.15.26:0/966301637 learned_addr learned my addr 172.21.15.26:0/966301637 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c5ab1640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00d0ce0 0x7fa6c015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c62b2640 1 --2- 172.21.15.26:0/966301637 >> v2:172.21.15.29:3300/0 conn(0x7fa6c0158ae0 0x7fa6c006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.076+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2128261608 0 0) 0x7fa6c006b1c0 con 0x7fa6c015ee90 2024-04-24T23:25:22.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6ac003520 con 0x7fa6c015ee90 2024-04-24T23:25:22.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3837570747 0 0) 0x7fa6ac003520 con 0x7fa6c015ee90 2024-04-24T23:25:22.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00d0ce0 msgr2=0x7fa6c015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:22.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00d0ce0 0x7fa6c015e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 >> v2:172.21.15.29:3300/0 conn(0x7fa6c0158ae0 msgr2=0x7fa6c006e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 --2- 172.21.15.26:0/966301637 >> v2:172.21.15.29:3300/0 conn(0x7fa6c0158ae0 0x7fa6c006e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c006f250 con 0x7fa6c015ee90 2024-04-24T23:25:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fa6bc0019f0 con 0x7fa6c015ee90 2024-04-24T23:25:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 >> v1:172.21.15.26:6789/0 conn(0x7fa6c015ee90 legacy=0x7fa6c0071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00d0ce0 0x7fa6ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6ac004d40 0x7fa6ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c006ec90 con 0x7fa6ac0039b0 2024-04-24T23:25:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c006ef70 con 0x7fa6ac004d40 2024-04-24T23:25:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac00b9c0 con 0x7fa6c00d0ce0 2024-04-24T23:25:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c62b2640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c5ab1640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00d0ce0 0x7fa6ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c6ab3640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6ac004d40 0x7fa6ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c62b2640 1 -- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00d0ce0 msgr2=0x7fa6ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c62b2640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00d0ce0 0x7fa6ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c62b2640 1 -- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6ac004d40 msgr2=0x7fa6ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c62b2640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6ac004d40 0x7fa6ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.077+0000 7fa6c62b2640 1 -- 172.21.15.26:0/966301637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac003520 con 0x7fa6ac0039b0 2024-04-24T23:25:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.078+0000 7fa6c62b2640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6ac003dd0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009ac0 tx=0x7fa6ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909423b6191ad474 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.078+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8036070 con 0x7fa6ac0039b0 2024-04-24T23:25:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.078+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b8031400 con 0x7fa6ac0039b0 2024-04-24T23:25:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.078+0000 7fa6c52b0640 1 -- 172.21.15.26:0/966301637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8009690 con 0x7fa6ac0039b0 2024-04-24T23:25:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.079+0000 7fa6c8516640 1 -- 172.21.15.26:0/966301637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 msgr2=0x7fa6ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.079+0000 7fa6c8516640 1 --2- 172.21.15.26:0/966301637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6ac003dd0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009ac0 tx=0x7fa6ac005830 comp rx=0 tx=0).stop 2024-04-24T23:25:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.079+0000 7fa6c6ab3640 1 -- 172.21.15.26:0/966301637 reap_dead start 2024-04-24T23:25:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.079+0000 7fa6c8516640 1 -- 172.21.15.26:0/966301637 shutdown_connections 2024-04-24T23:25:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.079+0000 7fa6c8516640 1 -- 172.21.15.26:0/966301637 >> 172.21.15.26:0/966301637 conn(0x7fa6c00b2970 msgr2=0x7fa6c00d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.079+0000 7fa6c8516640 1 -- 172.21.15.26:0/966301637 shutdown_connections 2024-04-24T23:25:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.079+0000 7fa6c8516640 1 -- 172.21.15.26:0/966301637 wait complete. 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.080+0000 7fa6c8516640 1 Processor -- start 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.080+0000 7fa6c8516640 1 -- start start 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c8516640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6c00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c8516640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00b3780 0x7fa6c01f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c8516640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c01f4aa0 0x7fa6c01f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c8516640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c00b00e0 con 0x7fa6ac0039b0 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c8516640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c00aff60 con 0x7fa6c01f4aa0 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c8516640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c0071e30 con 0x7fa6c00b3780 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c62b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6c00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c62b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6c00b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53886/0 (socket says 172.21.15.26:53886) 2024-04-24T23:25:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c62b2640 1 -- 172.21.15.26:0/2018857664 learned_addr learned my addr 172.21.15.26:0/2018857664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c5ab1640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00b3780 0x7fa6c01f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c6ab3640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c01f4aa0 0x7fa6c01f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c62b2640 1 -- 172.21.15.26:0/2018857664 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00b3780 msgr2=0x7fa6c01f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c62b2640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00b3780 0x7fa6c01f4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c62b2640 1 -- 172.21.15.26:0/2018857664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c01f4aa0 msgr2=0x7fa6c01f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c62b2640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c01f4aa0 0x7fa6c01f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c62b2640 1 -- 172.21.15.26:0/2018857664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c00aeb20 con 0x7fa6ac0039b0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.081+0000 7fa6c62b2640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6c00b3240 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8034040 tx=0x7fa6b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.082+0000 7fa6b6ffd640 1 -- 172.21.15.26:0/2018857664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8040020 con 0x7fa6ac0039b0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.082+0000 7fa6b6ffd640 1 -- 172.21.15.26:0/2018857664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b8003950 con 0x7fa6ac0039b0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.082+0000 7fa6b6ffd640 1 -- 172.21.15.26:0/2018857664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8036040 con 0x7fa6ac0039b0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.082+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c00b3c20 con 0x7fa6ac0039b0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.082+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c00b41e0 con 0x7fa6ac0039b0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.083+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c0075610 con 0x7fa6ac0039b0 2024-04-24T23:25:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.083+0000 7fa6b6ffd640 1 -- 172.21.15.26:0/2018857664 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6b8043020 con 0x7fa6ac0039b0 2024-04-24T23:25:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.083+0000 7fa6b6ffd640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6a406be30 0x7fa6a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.084+0000 7fa6b6ffd640 1 -- 172.21.15.26:0/2018857664 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7fa6b80aad90 con 0x7fa6ac0039b0 2024-04-24T23:25:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.084+0000 7fa6c5ab1640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6a406be30 0x7fa6a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.084+0000 7fa6c5ab1640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6a406be30 0x7fa6a406e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00af420 tx=0x7fa6b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.088+0000 7fa6b6ffd640 1 -- 172.21.15.26:0/2018857664 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa6b804a070 con 0x7fa6ac0039b0 2024-04-24T23:25:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.275+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa6c0067580 con 0x7fa6ac0039b0 2024-04-24T23:25:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.278+0000 7fa6b6ffd640 1 -- 172.21.15.26:0/2018857664 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fa6b80798f0 con 0x7fa6ac0039b0 2024-04-24T23:25:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.280+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6a406be30 msgr2=0x7fa6a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.280+0000 7fa6c8516640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6a406be30 0x7fa6a406e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00af420 tx=0x7fa6b0008040 comp rx=0 tx=0).stop 2024-04-24T23:25:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 msgr2=0x7fa6c00b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6c00b3240 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8034040 tx=0x7fa6b8004730 comp rx=0 tx=0).stop 2024-04-24T23:25:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 shutdown_connections 2024-04-24T23:25:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6a406be30 0x7fa6a406e2f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6c01f4aa0 0x7fa6c01f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6c00b3780 0x7fa6c01f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 --2- 172.21.15.26:0/2018857664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ac0039b0 0x7fa6c00b3240 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 >> 172.21.15.26:0/2018857664 conn(0x7fa6c00b2970 msgr2=0x7fa6c015cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 shutdown_connections 2024-04-24T23:25:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.281+0000 7fa6c8516640 1 -- 172.21.15.26:0/2018857664 wait complete. 2024-04-24T23:25:22.292 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:25:22.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:25:22.329 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:25:22.371 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:25:22.371 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:25:22.372 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:22.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:25:22.388 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:25:22.388 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:22.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:25:22.429 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:25:22.430 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:22.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:25:22.445 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:25:22.445 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:22.486 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:25:22.486 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:25:22.487 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.586+0000 7f8f8ae31640 1 Processor -- start 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.586+0000 7f8f8ae31640 1 -- start start 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.587+0000 7f8f8ae31640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f8406d350 0x7f8f8406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.587+0000 7f8f8ae31640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f840cd660 0x7f8f840cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.587+0000 7f8f8ae31640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f840cfbc0 con 0x7f8f84145910 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.587+0000 7f8f8ae31640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8f84145cf0 con 0x7f8f840cd660 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.587+0000 7f8f8ae31640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8406d7f0 con 0x7f8f8406d350 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.587+0000 7f8f893ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f840cd660 0x7f8f840cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.587+0000 7f8f893ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f840cd660 0x7f8f840cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53900/0 (socket says 172.21.15.26:53900) 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.587+0000 7f8f893ce640 1 -- 172.21.15.26:0/4119309435 learned_addr learned my addr 172.21.15.26:0/4119309435 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.588+0000 7f8f83fff640 1 --2- 172.21.15.26:0/4119309435 >> v2:172.21.15.29:3300/0 conn(0x7f8f8406d350 0x7f8f8406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.588+0000 7f8f893ce640 1 -- 172.21.15.26:0/4119309435 >> v2:172.21.15.29:3300/0 conn(0x7f8f8406d350 msgr2=0x7f8f8406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.588+0000 7f8f893ce640 1 --2- 172.21.15.26:0/4119309435 >> v2:172.21.15.29:3300/0 conn(0x7f8f8406d350 0x7f8f8406e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.588+0000 7f8f893ce640 1 -- 172.21.15.26:0/4119309435 >> v1:172.21.15.29:6789/0 conn(0x7f8f84145910 legacy=0x7f8f8415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.588+0000 7f8f893ce640 1 -- 172.21.15.26:0/4119309435 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8406dad0 con 0x7f8f840cd660 2024-04-24T23:25:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.588+0000 7f8f893ce640 1 --2- 172.21.15.26:0/4119309435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f840cd660 0x7f8f840cdaf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009e50 tx=0x7f8f7c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcf307e37b83621 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.588+0000 7f8f837fe640 1 -- 172.21.15.26:0/4119309435 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c036070 con 0x7f8f840cd660 2024-04-24T23:25:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.588+0000 7f8f837fe640 1 -- 172.21.15.26:0/4119309435 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c031600 con 0x7f8f840cd660 2024-04-24T23:25:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.588+0000 7f8f837fe640 1 -- 172.21.15.26:0/4119309435 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c02ae60 con 0x7f8f840cd660 2024-04-24T23:25:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.589+0000 7f8f8ae31640 1 -- 172.21.15.26:0/4119309435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f840cd660 msgr2=0x7f8f840cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.589+0000 7f8f8ae31640 1 --2- 172.21.15.26:0/4119309435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f840cd660 0x7f8f840cdaf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009e50 tx=0x7f8f7c02a680 comp rx=0 tx=0).stop 2024-04-24T23:25:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.589+0000 7f8f8ae31640 1 -- 172.21.15.26:0/4119309435 shutdown_connections 2024-04-24T23:25:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.589+0000 7f8f8ae31640 1 --2- 172.21.15.26:0/4119309435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f840cd660 0x7f8f840cdaf0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.589+0000 7f8f8ae31640 1 --2- 172.21.15.26:0/4119309435 >> v2:172.21.15.29:3300/0 conn(0x7f8f8406d350 0x7f8f8406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.589+0000 7f8f8ae31640 1 -- 172.21.15.26:0/4119309435 >> 172.21.15.26:0/4119309435 conn(0x7f8f8406b300 msgr2=0x7f8f8406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.589+0000 7f8f8ae31640 1 -- 172.21.15.26:0/4119309435 shutdown_connections 2024-04-24T23:25:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.589+0000 7f8f8ae31640 1 -- 172.21.15.26:0/4119309435 wait complete. 2024-04-24T23:25:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.590+0000 7f8f8ae31640 1 Processor -- start 2024-04-24T23:25:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.590+0000 7f8f8ae31640 1 -- start start 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f8ae31640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f8406d350 0x7f8f8413e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f8ae31640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f840cd660 0x7f8f8413ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f8ae31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f84145910 0x7f8f840ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f8ae31640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8406e370 con 0x7f8f84145910 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f8ae31640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8406e090 con 0x7f8f8406d350 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f8ae31640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8f8415d770 con 0x7f8f840cd660 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f893ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f84145910 0x7f8f840ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f893ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f84145910 0x7f8f840ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53908/0 (socket says 172.21.15.26:53908) 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f893ce640 1 -- 172.21.15.26:0/1261430433 learned_addr learned my addr 172.21.15.26:0/1261430433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f88bcd640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f8406d350 0x7f8f8413e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.591+0000 7f8f83fff640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f840cd660 0x7f8f8413ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f893ce640 1 -- 172.21.15.26:0/1261430433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f840cd660 msgr2=0x7f8f8413ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f893ce640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f840cd660 0x7f8f8413ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f893ce640 1 -- 172.21.15.26:0/1261430433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f8406d350 msgr2=0x7f8f8413e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f893ce640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f8406d350 0x7f8f8413e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f893ce640 1 -- 172.21.15.26:0/1261430433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f841401f0 con 0x7f8f84145910 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f893ce640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f84145910 0x7f8f840ad870 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009f70 tx=0x7f8f7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f817fa640 1 -- 172.21.15.26:0/1261430433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c036050 con 0x7f8f84145910 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f817fa640 1 -- 172.21.15.26:0/1261430433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c042070 con 0x7f8f84145910 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f840b3f00 con 0x7f8f84145910 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f817fa640 1 -- 172.21.15.26:0/1261430433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c009510 con 0x7f8f84145910 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.592+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8413f430 con 0x7f8f84145910 2024-04-24T23:25:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.593+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f840694c0 con 0x7f8f84145910 2024-04-24T23:25:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.593+0000 7f8f817fa640 1 -- 172.21.15.26:0/1261430433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f7c0079a0 con 0x7f8f84145910 2024-04-24T23:25:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.594+0000 7f8f817fa640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f7006be30 0x7f8f7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.594+0000 7f8f817fa640 1 -- 172.21.15.26:0/1261430433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f8f7c0ab8c0 con 0x7f8f84145910 2024-04-24T23:25:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.594+0000 7f8f88bcd640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f7006be30 0x7f8f7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.594+0000 7f8f88bcd640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f7006be30 0x7f8f7006e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8f840b11b0 tx=0x7f8f74009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:22.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.598+0000 7f8f817fa640 1 -- 172.21.15.26:0/1261430433 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8f7c079e70 con 0x7f8f84145910 2024-04-24T23:25:22.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.817+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f84144cd0 con 0x7f8f84145910 2024-04-24T23:25:22.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.818+0000 7f8f817fa640 1 -- 172.21.15.26:0/1261430433 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+3869 (secure 0 0 0) 0x7f8f7c07e170 con 0x7f8f84145910 2024-04-24T23:25:22.819 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:22.820 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2623563190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2623563190},{"type":"v1","addr":"172.21.15.29:6834","nonce":2623563190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3745438402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3745438402},{"type":"v1","addr":"172.21.15.26:6835","nonce":3745438402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6289,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2441359214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2441359214},{"type":"v1","addr":"172.21.15.29:6835","nonce":2441359214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7140,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2795554897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2795554897},{"type":"v1","addr":"172.21.15.26:6837","nonce":2795554897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[]} 2024-04-24T23:25:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 60 2024-04-24T23:25:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f7006be30 msgr2=0x7f8f7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f7006be30 0x7f8f7006e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8f840b11b0 tx=0x7f8f74009060 comp rx=0 tx=0).stop 2024-04-24T23:25:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f84145910 msgr2=0x7f8f840ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f84145910 0x7f8f840ad870 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009f70 tx=0x7f8f7c004a40 comp rx=0 tx=0).stop 2024-04-24T23:25:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 shutdown_connections 2024-04-24T23:25:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f7006be30 0x7f8f7006e2f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f84145910 0x7f8f840ad870 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f840cd660 0x7f8f8413ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 --2- 172.21.15.26:0/1261430433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f8406d350 0x7f8f8413e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 >> 172.21.15.26:0/1261430433 conn(0x7f8f8406b300 msgr2=0x7f8f841443b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 shutdown_connections 2024-04-24T23:25:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.820+0000 7f8f8ae31640 1 -- 172.21.15.26:0/1261430433 wait complete. 2024-04-24T23:25:22.833 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:25:22.833 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:25:22.833 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:25:22.833 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:25:22.833 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:25:22.833 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:25:22.833 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:25:22.833 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:25:22.834 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:25:22.834 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:25:22.834 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:25:22.834 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:25:22.834 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:25:22.834 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:25:22.834 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:25:22.835 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:25:22.843 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:25:22.843 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:25:22.843 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:25:22.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:25:22.847 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:25:22.847 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:25:22.847 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:25:22.847 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:25:22.885 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:25:22.885 DEBUG:teuthology.parallel:result is None 2024-04-24T23:25:22.886 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:25:22.886 DEBUG:teuthology.parallel:result is None 2024-04-24T23:25:22.890 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:25:22.890 DEBUG:teuthology.parallel:result is None 2024-04-24T23:25:22.891 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:25:22.891 DEBUG:teuthology.parallel:result is None 2024-04-24T23:25:22.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:25:22.962 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:25:22.962 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:25:22.976 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:25:22.977 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:25:22.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.996+0000 7fa4966f5640 1 Processor -- start 2024-04-24T23:25:22.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.996+0000 7fa4966f5640 1 -- start start 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa4966f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa49015e570 0x7fa4900d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa4966f5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4900d32f0 con 0x7fa4900d22e0 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa4966f5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4900d1600 con 0x7fa4900b3130 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa4966f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa4900d1ab0 con 0x7fa49015e570 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa494c92640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa4900d22e0 0x7fa4900d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42860/0 (socket says 172.21.15.26:42860) 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa494c92640 1 -- 172.21.15.26:0/2326014914 learned_addr learned my addr 172.21.15.26:0/2326014914 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa48ffff640 1 --2- 172.21.15.26:0/2326014914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa49015e570 0x7fa4900d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa48effd640 1 -- 172.21.15.26:0/2326014914 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 273833551 0 0) 0x7fa488001490 con 0x7fa4900d22e0 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa48effd640 1 -- 172.21.15.26:0/2326014914 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa47c003520 con 0x7fa4900d22e0 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa48effd640 1 -- 172.21.15.26:0/2326014914 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1371898643 0 0) 0x7fa4900d1600 con 0x7fa4900b3130 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.997+0000 7fa48effd640 1 -- 172.21.15.26:0/2326014914 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa488001490 con 0x7fa4900b3130 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.998+0000 7fa48ffff640 1 -- 172.21.15.26:0/2326014914 >> v1:172.21.15.29:6789/0 conn(0x7fa4900b3130 legacy=0x7fa4900d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.998+0000 7fa48ffff640 1 -- 172.21.15.26:0/2326014914 >> v1:172.21.15.26:6789/0 conn(0x7fa4900d22e0 legacy=0x7fa4900d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:22.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.998+0000 7fa48ffff640 1 -- 172.21.15.26:0/2326014914 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4900d0270 con 0x7fa49015e570 2024-04-24T23:25:23.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.998+0000 7fa48ffff640 1 --2- 172.21.15.26:0/2326014914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa49015e570 0x7fa4900d4160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa4800097d0 tx=0x7fa48002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b1ef02fac00910 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa48effd640 1 -- 172.21.15.26:0/2326014914 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480037070 con 0x7fa49015e570 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa48effd640 1 -- 172.21.15.26:0/2326014914 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa480032440 con 0x7fa49015e570 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa48effd640 1 -- 172.21.15.26:0/2326014914 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4800327f0 con 0x7fa49015e570 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa4966f5640 1 -- 172.21.15.26:0/2326014914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa49015e570 msgr2=0x7fa4900d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa4966f5640 1 --2- 172.21.15.26:0/2326014914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa49015e570 0x7fa4900d4160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa4800097d0 tx=0x7fa48002a210 comp rx=0 tx=0).stop 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa4966f5640 1 -- 172.21.15.26:0/2326014914 shutdown_connections 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa4966f5640 1 --2- 172.21.15.26:0/2326014914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa49015e570 0x7fa4900d4160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa4966f5640 1 -- 172.21.15.26:0/2326014914 >> 172.21.15.26:0/2326014914 conn(0x7fa49008dbe0 msgr2=0x7fa49009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa4966f5640 1 -- 172.21.15.26:0/2326014914 shutdown_connections 2024-04-24T23:25:23.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:22.999+0000 7fa4966f5640 1 -- 172.21.15.26:0/2326014914 wait complete. 2024-04-24T23:25:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.000+0000 7fa4966f5640 1 Processor -- start 2024-04-24T23:25:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.000+0000 7fa4966f5640 1 -- start start 2024-04-24T23:25:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa4966f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4900b3130 0x7fa49014f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa4966f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa4900d22e0 0x7fa4900cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa4966f5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa49015e570 0x7fa4900cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa4966f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa4900d3010 con 0x7fa4900b3130 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa4966f5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4900d2ae0 con 0x7fa49015e570 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa4966f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa4900cfa70 con 0x7fa4900d22e0 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa48ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4900b3130 0x7fa49014f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa48ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4900b3130 0x7fa49014f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53946/0 (socket says 172.21.15.26:53946) 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa48ffff640 1 -- 172.21.15.26:0/2650392647 learned_addr learned my addr 172.21.15.26:0/2650392647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa494c92640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa49015e570 0x7fa4900cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.001+0000 7fa48f7fe640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa4900d22e0 0x7fa4900cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa48ffff640 1 -- 172.21.15.26:0/2650392647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa4900d22e0 msgr2=0x7fa4900cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa48ffff640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa4900d22e0 0x7fa4900cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa48ffff640 1 -- 172.21.15.26:0/2650392647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa49015e570 msgr2=0x7fa4900cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa48ffff640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa49015e570 0x7fa4900cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa48ffff640 1 -- 172.21.15.26:0/2650392647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49009df70 con 0x7fa4900b3130 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa48ffff640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4900b3130 0x7fa49014f4f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa480035040 tx=0x7fa4800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa48cff9640 1 -- 172.21.15.26:0/2650392647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480041020 con 0x7fa4900b3130 2024-04-24T23:25:23.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa48cff9640 1 -- 172.21.15.26:0/2650392647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48003d6e0 con 0x7fa4900b3130 2024-04-24T23:25:23.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4900ccbe0 con 0x7fa4900b3130 2024-04-24T23:25:23.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa48cff9640 1 -- 172.21.15.26:0/2650392647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480037040 con 0x7fa4900b3130 2024-04-24T23:25:23.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.002+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa490150370 con 0x7fa4900b3130 2024-04-24T23:25:23.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.003+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4900694c0 con 0x7fa4900b3130 2024-04-24T23:25:23.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.003+0000 7fa48cff9640 1 -- 172.21.15.26:0/2650392647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa480044020 con 0x7fa4900b3130 2024-04-24T23:25:23.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.004+0000 7fa48cff9640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa47006be30 0x7fa47006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.004+0000 7fa48cff9640 1 -- 172.21.15.26:0/2650392647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7fa4800ab230 con 0x7fa4900b3130 2024-04-24T23:25:23.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.004+0000 7fa48f7fe640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa47006be30 0x7fa47006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.004+0000 7fa48f7fe640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa47006be30 0x7fa47006e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa478007fd0 tx=0x7fa478009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:23.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.008+0000 7fa48cff9640 1 -- 172.21.15.26:0/2650392647 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa48004b030 con 0x7fa4900b3130 2024-04-24T23:25:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.202+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa49006c180 con 0x7fa4900b3130 2024-04-24T23:25:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.958+0000 7fa48cff9640 1 -- 172.21.15.26:0/2650392647 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7fa49009df70 con 0x7fa4900b3130 2024-04-24T23:25:23.959 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa47006be30 msgr2=0x7fa47006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa47006be30 0x7fa47006e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa478007fd0 tx=0x7fa478009040 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4900b3130 msgr2=0x7fa49014f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4900b3130 0x7fa49014f4f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa480035040 tx=0x7fa4800046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 shutdown_connections 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa47006be30 0x7fa47006e2f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa49015e570 0x7fa4900cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa4900d22e0 0x7fa4900cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 --2- 172.21.15.26:0/2650392647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4900b3130 0x7fa49014f4f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 >> 172.21.15.26:0/2650392647 conn(0x7fa49008dbe0 msgr2=0x7fa4900d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 shutdown_connections 2024-04-24T23:25:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:23.962+0000 7fa4966f5640 1 -- 172.21.15.26:0/2650392647 wait complete. 2024-04-24T23:25:23.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:25:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.084+0000 7fb11385d640 1 Processor -- start 2024-04-24T23:25:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.084+0000 7fb11385d640 1 -- start start 2024-04-24T23:25:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb11385d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb10c0cf870 0x7fb10c072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb11385d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb10c15eb00 0x7fb10c15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb11385d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10c15f450 con 0x7fb10c143c40 2024-04-24T23:25:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb11385d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb10c071b70 con 0x7fb10c15eb00 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb11385d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb10c0cd0b0 con 0x7fb10c0cf870 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb111dfa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb10c15eb00 0x7fb10c15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb111dfa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb10c15eb00 0x7fb10c15ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58758/0 (socket says 172.21.15.26:58758) 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb111dfa640 1 -- 172.21.15.26:0/980246278 learned_addr learned my addr 172.21.15.26:0/980246278 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb110df8640 1 --2- 172.21.15.26:0/980246278 >> v2:172.21.15.29:3300/0 conn(0x7fb10c0cf870 0x7fb10c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb111dfa640 1 -- 172.21.15.26:0/980246278 >> v2:172.21.15.29:3300/0 conn(0x7fb10c0cf870 msgr2=0x7fb10c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb111dfa640 1 --2- 172.21.15.26:0/980246278 >> v2:172.21.15.29:3300/0 conn(0x7fb10c0cf870 0x7fb10c072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb111dfa640 1 -- 172.21.15.26:0/980246278 >> v1:172.21.15.29:6789/0 conn(0x7fb10c143c40 legacy=0x7fb10c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb111dfa640 1 -- 172.21.15.26:0/980246278 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c071240 con 0x7fb10c15eb00 2024-04-24T23:25:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.085+0000 7fb110df8640 1 --2- 172.21.15.26:0/980246278 >> v2:172.21.15.29:3300/0 conn(0x7fb10c0cf870 0x7fb10c072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.086+0000 7fb111dfa640 1 --2- 172.21.15.26:0/980246278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c15eb00 0x7fb10c15ef10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb10c06db50 tx=0x7fb10402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a1f7928082e95c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.086+0000 7fb103fff640 1 -- 172.21.15.26:0/980246278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10400b2e0 con 0x7fb10c15eb00 2024-04-24T23:25:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.086+0000 7fb103fff640 1 -- 172.21.15.26:0/980246278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb104037070 con 0x7fb10c15eb00 2024-04-24T23:25:24.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.086+0000 7fb103fff640 1 -- 172.21.15.26:0/980246278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10400b480 con 0x7fb10c15eb00 2024-04-24T23:25:24.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.087+0000 7fb11385d640 1 -- 172.21.15.26:0/980246278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c15eb00 msgr2=0x7fb10c15ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.087+0000 7fb11385d640 1 --2- 172.21.15.26:0/980246278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c15eb00 0x7fb10c15ef10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb10c06db50 tx=0x7fb10402c940 comp rx=0 tx=0).stop 2024-04-24T23:25:24.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.087+0000 7fb11385d640 1 -- 172.21.15.26:0/980246278 shutdown_connections 2024-04-24T23:25:24.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.087+0000 7fb11385d640 1 --2- 172.21.15.26:0/980246278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c15eb00 0x7fb10c15ef10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.087+0000 7fb11385d640 1 --2- 172.21.15.26:0/980246278 >> v2:172.21.15.29:3300/0 conn(0x7fb10c0cf870 0x7fb10c072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.087+0000 7fb11385d640 1 -- 172.21.15.26:0/980246278 >> 172.21.15.26:0/980246278 conn(0x7fb10c06b300 msgr2=0x7fb10c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:24.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.087+0000 7fb11385d640 1 -- 172.21.15.26:0/980246278 shutdown_connections 2024-04-24T23:25:24.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.087+0000 7fb11385d640 1 -- 172.21.15.26:0/980246278 wait complete. 2024-04-24T23:25:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.088+0000 7fb11385d640 1 Processor -- start 2024-04-24T23:25:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.088+0000 7fb11385d640 1 -- start start 2024-04-24T23:25:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb11385d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0cf870 0x7fb10c1e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb11385d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb10c143c40 0x7fb10c1e4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb11385d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb10c15eb00 0x7fb10c1e4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb11385d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c144020 con 0x7fb10c0cf870 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb11385d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c071800 con 0x7fb10c143c40 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb11385d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb10c072540 con 0x7fb10c15eb00 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb1115f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0cf870 0x7fb10c1e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb1115f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0cf870 0x7fb10c1e5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58766/0 (socket says 172.21.15.26:58766) 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb1115f9640 1 -- 172.21.15.26:0/2020020832 learned_addr learned my addr 172.21.15.26:0/2020020832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb111dfa640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb10c15eb00 0x7fb10c1e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb110df8640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb10c143c40 0x7fb10c1e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb1115f9640 1 -- 172.21.15.26:0/2020020832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb10c15eb00 msgr2=0x7fb10c1e4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb1115f9640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb10c15eb00 0x7fb10c1e4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb1115f9640 1 -- 172.21.15.26:0/2020020832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb10c143c40 msgr2=0x7fb10c1e4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb1115f9640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb10c143c40 0x7fb10c1e4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb1115f9640 1 -- 172.21.15.26:0/2020020832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c13b270 con 0x7fb10c0cf870 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.089+0000 7fb1115f9640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0cf870 0x7fb10c1e5ab0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c400 tx=0x7fb10800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.090+0000 7fb101ffb640 1 -- 172.21.15.26:0/2020020832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1080052d0 con 0x7fb10c0cf870 2024-04-24T23:25:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.090+0000 7fb101ffb640 1 -- 172.21.15.26:0/2020020832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb108005490 con 0x7fb10c0cf870 2024-04-24T23:25:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.090+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10c15bc50 con 0x7fb10c0cf870 2024-04-24T23:25:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.090+0000 7fb101ffb640 1 -- 172.21.15.26:0/2020020832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108007260 con 0x7fb10c0cf870 2024-04-24T23:25:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.090+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10c15c1d0 con 0x7fb10c0cf870 2024-04-24T23:25:24.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.091+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10c0694c0 con 0x7fb10c0cf870 2024-04-24T23:25:24.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.091+0000 7fb101ffb640 1 -- 172.21.15.26:0/2020020832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb108007400 con 0x7fb10c0cf870 2024-04-24T23:25:24.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.091+0000 7fb101ffb640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0f006be30 0x7fb0f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.092+0000 7fb101ffb640 1 -- 172.21.15.26:0/2020020832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7fb108089ff0 con 0x7fb10c0cf870 2024-04-24T23:25:24.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.092+0000 7fb110df8640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0f006be30 0x7fb0f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.092+0000 7fb110df8640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0f006be30 0x7fb0f006e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb10c14e3d0 tx=0x7fb0fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.096+0000 7fb101ffb640 1 -- 172.21.15.26:0/2020020832 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb10c15f450 con 0x7fb10c0cf870 2024-04-24T23:25:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.345+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb10c0b4d20 con 0x7fb10c0cf870 2024-04-24T23:25:24.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.345+0000 7fb101ffb640 1 -- 172.21.15.26:0/2020020832 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7fb1080586a0 con 0x7fb10c0cf870 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmg5ZUFBAAkapM9La3f+wSyLczFkm/0w==", 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.347 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPS3cHxAASacBQN+95Ds6N5668Je+DA==", 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmStpLGxAAz7molt8venundBs+CcFSbg==", 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:25:24.348 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylml0TBJhAAXQkjExbw1EXc90XlcAEBBw==", 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjkylmQt74DBAAZLk9JTH3qTKRouUawqNURw==", 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCmkylmi6IJCxAAWBnNl2y0XIUW/YoAIt6VIQ==", 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:25:24.350 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCpkylmRcgOCRAA79pdcH0OlB2VROLSRE4dcw==", 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCskylm8SopBxAArvUph95+zD5/0LmNM8WPIA==", 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.351 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCwkylmbHipDxAAVUI9OpiMFiM/HFo03fEzSA==", 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-04-24T23:25:24.352 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCzkylmDm4CDhAA1T/X+ywXEUCL96OGyi6S9Q==", 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC2kylmwWyVDBAAJsFFLltURqVqGLNjK9OCJQ==", 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:25:24.353 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5kylm/nLXChAA6kk0UI2hchkN2lhPO1sXHw==", 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ==", 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-04-24T23:25:24.355 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylm/BVuMxAAAdEb6dd3Q5eMc+I3oszOyg==", 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmjnJGMBAAvwAmiHrkGkDFTeL/WZbdWw==", 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:25:24.356 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPh3qOBAAa4cbQeF96mdpgNVb/l3xPg==", 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCgkylm1ZfsMxAAExymnmhW44TcXzRblp95UA==", 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmq+8yERAAMKjK+5f2Qr35ral8L2qJMw==", 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.358 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmqYXWDBAAnuAn2h4Gb9qSFBoAZX73qQ==", 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-24T23:25:24.359 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0f006be30 msgr2=0x7fb0f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0f006be30 0x7fb0f006e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb10c14e3d0 tx=0x7fb0fc008040 comp rx=0 tx=0).stop 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0cf870 msgr2=0x7fb10c1e5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0cf870 0x7fb10c1e5ab0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c400 tx=0x7fb10800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 shutdown_connections 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0f006be30 0x7fb0f006e2f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb10c15eb00 0x7fb10c1e4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb10c143c40 0x7fb10c1e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 --2- 172.21.15.26:0/2020020832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0cf870 0x7fb10c1e5ab0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.347+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 >> 172.21.15.26:0/2020020832 conn(0x7fb10c06b300 msgr2=0x7fb10c06b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.348+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 shutdown_connections 2024-04-24T23:25:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.348+0000 7fb11385d640 1 -- 172.21.15.26:0/2020020832 wait complete. 2024-04-24T23:25:24.362 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:25:24.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.462+0000 7f4d5148c640 1 Processor -- start 2024-04-24T23:25:24.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.462+0000 7f4d5148c640 1 -- start start 2024-04-24T23:25:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d5148c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d4c0cdd70 0x7f4d4c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d5148c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4c0cfbc0 con 0x7f4d4c06d350 2024-04-24T23:25:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d5148c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4c06d7f0 con 0x7f4d4c145910 2024-04-24T23:25:24.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d5148c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4c145cf0 con 0x7f4d4c0cdd70 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d4a7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4d4c06d350 0x7f4d4c0cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32810/0 (socket says 172.21.15.26:32810) 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d4a7fc640 1 -- 172.21.15.26:0/2734199751 learned_addr learned my addr 172.21.15.26:0/2734199751 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d4b7fe640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d4c0cdd70 0x7f4d4c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498877006 0 0) 0x7f4d4c0cfbc0 con 0x7f4d4c06d350 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d38003520 con 0x7f4d4c06d350 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3142915573 0 0) 0x7f4d38003520 con 0x7f4d4c06d350 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.463+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d4c0cdd70 msgr2=0x7f4d4c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d4c0cdd70 0x7f4d4c06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 >> v1:172.21.15.29:6789/0 conn(0x7f4d4c145910 legacy=0x7f4d4c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c06dad0 con 0x7f4d4c06d350 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f4d34001670 con 0x7f4d4c06d350 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 >> v1:172.21.15.26:6789/0 conn(0x7f4d4c06d350 legacy=0x7f4d4c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d4c0cdd70 0x7f4d380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d38003f20 0x7f4d38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d38004d40 0x7f4d380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c145cf0 con 0x7f4d38004d40 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3800ba20 con 0x7f4d38003f20 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4d3800bc20 con 0x7f4d4c0cdd70 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d4b7fe640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d38004d40 0x7f4d380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d4affd640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d4c0cdd70 0x7f4d380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d4a7fc640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d38003f20 0x7f4d38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d4b7fe640 1 -- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d4c0cdd70 msgr2=0x7f4d380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d4b7fe640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d4c0cdd70 0x7f4d380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d4b7fe640 1 -- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d38003f20 msgr2=0x7f4d38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d4b7fe640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d38003f20 0x7f4d38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.464+0000 7f4d4b7fe640 1 -- 172.21.15.26:0/2734199751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d38003520 con 0x7f4d38004d40 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d4b7fe640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d38004d40 0x7f4d380051c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009f90 tx=0x7f4d38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b759cb2df57e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c009c10 con 0x7f4d38004d40 2024-04-24T23:25:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c036070 con 0x7f4d38004d40 2024-04-24T23:25:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d49ffb640 1 -- 172.21.15.26:0/2734199751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c031850 con 0x7f4d38004d40 2024-04-24T23:25:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d5148c640 1 -- 172.21.15.26:0/2734199751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d38004d40 msgr2=0x7f4d380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d5148c640 1 --2- 172.21.15.26:0/2734199751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d38004d40 0x7f4d380051c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009f90 tx=0x7f4d38005830 comp rx=0 tx=0).stop 2024-04-24T23:25:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d4b7fe640 1 -- 172.21.15.26:0/2734199751 reap_dead start 2024-04-24T23:25:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d5148c640 1 -- 172.21.15.26:0/2734199751 shutdown_connections 2024-04-24T23:25:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d5148c640 1 -- 172.21.15.26:0/2734199751 >> 172.21.15.26:0/2734199751 conn(0x7f4d4c06b300 msgr2=0x7f4d4c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d5148c640 1 -- 172.21.15.26:0/2734199751 shutdown_connections 2024-04-24T23:25:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.465+0000 7f4d5148c640 1 -- 172.21.15.26:0/2734199751 wait complete. 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.467+0000 7f4d5148c640 1 Processor -- start 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.467+0000 7f4d5148c640 1 -- start start 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.467+0000 7f4d5148c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d38004d40 0x7f4d4c1f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.467+0000 7f4d5148c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d4c1e60c0 0x7f4d4c1e6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.467+0000 7f4d5148c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d4c0ad8e0 0x7f4d4c0add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.467+0000 7f4d5148c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c06e370 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.467+0000 7f4d5148c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c06e090 con 0x7f4d4c0ad8e0 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.467+0000 7f4d5148c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4c06e890 con 0x7f4d38004d40 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d4c1e60c0 0x7f4d4c1e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d4c1e60c0 0x7f4d4c1e6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58792/0 (socket says 172.21.15.26:58792) 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4a7fc640 1 -- 172.21.15.26:0/2623065628 learned_addr learned my addr 172.21.15.26:0/2623065628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4affd640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d38004d40 0x7f4d4c1f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4b7fe640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d4c0ad8e0 0x7f4d4c0add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4a7fc640 1 -- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d38004d40 msgr2=0x7f4d4c1f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4a7fc640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d38004d40 0x7f4d4c1f6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4a7fc640 1 -- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d4c0ad8e0 msgr2=0x7f4d4c0add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4a7fc640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d4c0ad8e0 0x7f4d4c0add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4a7fc640 1 -- 172.21.15.26:0/2623065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c0ae650 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4affd640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d38004d40 0x7f4d4c1f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4b7fe640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d4c0ad8e0 0x7f4d4c0add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d4a7fc640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d4c1e60c0 0x7f4d4c1e6540 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400e7f0 tx=0x7f4d3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d437fe640 1 -- 172.21.15.26:0/2623065628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400c870 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d437fe640 1 -- 172.21.15.26:0/2623065628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d34018070 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c0ae930 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d437fe640 1 -- 172.21.15.26:0/2623065628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400cc70 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.468+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c0aeef0 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.469+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4c0694c0 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.470+0000 7f4d437fe640 1 -- 172.21.15.26:0/2623065628 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d34013420 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.470+0000 7f4d437fe640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d2c06be60 0x7f4d2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.470+0000 7f4d437fe640 1 -- 172.21.15.26:0/2623065628 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f4d3408a6d0 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.470+0000 7f4d4affd640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d2c06be60 0x7f4d2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.471+0000 7f4d4affd640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d2c06be60 0x7f4d2c06e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c15ee90 tx=0x7f4d44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.474+0000 7f4d437fe640 1 -- 172.21.15.26:0/2623065628 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4d340593d0 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.657+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d4c06b920 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.658+0000 7f4d437fe640 1 -- 172.21.15.26:0/2623065628 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+12831 (secure 0 0 0) 0x7f4d34058cd0 con 0x7f4d4c1e60c0 2024-04-24T23:25:24.659 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:24.659 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:25:23.966323+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.659+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d2c06be60 msgr2=0x7f4d2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.659+0000 7f4d5148c640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d2c06be60 0x7f4d2c06e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c15ee90 tx=0x7f4d44009040 comp rx=0 tx=0).stop 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.659+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d4c1e60c0 msgr2=0x7f4d4c1e6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.659+0000 7f4d5148c640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d4c1e60c0 0x7f4d4c1e6540 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400e7f0 tx=0x7f4d3400ecb0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.660+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 shutdown_connections 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.660+0000 7f4d5148c640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d2c06be60 0x7f4d2c06e320 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.660+0000 7f4d5148c640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d4c0ad8e0 0x7f4d4c0add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.660+0000 7f4d5148c640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d4c1e60c0 0x7f4d4c1e6540 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.660+0000 7f4d5148c640 1 --2- 172.21.15.26:0/2623065628 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4d38004d40 0x7f4d4c1f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.660+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 >> 172.21.15.26:0/2623065628 conn(0x7f4d4c06b300 msgr2=0x7f4d4c0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.660+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 shutdown_connections 2024-04-24T23:25:24.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.660+0000 7f4d5148c640 1 -- 172.21.15.26:0/2623065628 wait complete. 2024-04-24T23:25:24.672 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:25:24.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:25:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.775+0000 7f476e129640 1 Processor -- start 2024-04-24T23:25:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.775+0000 7f476e129640 1 -- start start 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.775+0000 7f476e129640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47680d0750 0x7f47680acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.775+0000 7f476e129640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47680aeb10 con 0x7f47680b4ce0 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.775+0000 7f476e129640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47680aedf0 con 0x7f476815be40 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.775+0000 7f476e129640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f476809df20 con 0x7f47680d0750 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.776+0000 7f4766ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47680d0750 0x7f47680acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.776+0000 7f4767fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f47680b4ce0 0x7f476815c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32814/0 (socket says 172.21.15.26:32814) 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.776+0000 7f4767fff640 1 -- 172.21.15.26:0/1502331390 learned_addr learned my addr 172.21.15.26:0/1502331390 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.776+0000 7f4766ffd640 1 -- 172.21.15.26:0/1502331390 >> v1:172.21.15.29:6789/0 conn(0x7f476815be40 legacy=0x7f47680d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.776+0000 7f4766ffd640 1 -- 172.21.15.26:0/1502331390 >> v1:172.21.15.26:6789/0 conn(0x7f47680b4ce0 legacy=0x7f476815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.776+0000 7f4766ffd640 1 -- 172.21.15.26:0/1502331390 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47680af0d0 con 0x7f47680d0750 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f4766ffd640 1 --2- 172.21.15.26:0/1502331390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 0x7f47680acd30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f475c0097a0 tx=0x7f475c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67baad6c82ed3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f47667fc640 1 -- 172.21.15.26:0/1502331390 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c037070 con 0x7f47680d0750 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f47667fc640 1 -- 172.21.15.26:0/1502331390 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475c032440 con 0x7f47680d0750 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f47667fc640 1 -- 172.21.15.26:0/1502331390 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c032810 con 0x7f47680d0750 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f476e129640 1 -- 172.21.15.26:0/1502331390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 msgr2=0x7f47680acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f476e129640 1 --2- 172.21.15.26:0/1502331390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 0x7f47680acd30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f475c0097a0 tx=0x7f475c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f476e129640 1 -- 172.21.15.26:0/1502331390 shutdown_connections 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f476e129640 1 --2- 172.21.15.26:0/1502331390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 0x7f47680acd30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f476e129640 1 -- 172.21.15.26:0/1502331390 >> 172.21.15.26:0/1502331390 conn(0x7f476806b540 msgr2=0x7f476806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f476e129640 1 -- 172.21.15.26:0/1502331390 shutdown_connections 2024-04-24T23:25:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.777+0000 7f476e129640 1 -- 172.21.15.26:0/1502331390 wait complete. 2024-04-24T23:25:24.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.778+0000 7f476e129640 1 Processor -- start 2024-04-24T23:25:24.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.779+0000 7f476e129640 1 -- start start 2024-04-24T23:25:24.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.779+0000 7f476e129640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47680b4ce0 0x7f4768154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.779+0000 7f476e129640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 0x7f476809c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.779+0000 7f476e129640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f476815be40 0x7f476809cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.779+0000 7f476e129640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f476806cd10 con 0x7f47680d0750 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.779+0000 7f476e129640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f476806cb90 con 0x7f47680b4ce0 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.779+0000 7f476e129640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f476815c930 con 0x7f476815be40 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.779+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 0x7f476809c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.779+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 0x7f476809c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58812/0 (socket says 172.21.15.26:58812) 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f4766ffd640 1 -- 172.21.15.26:0/395597030 learned_addr learned my addr 172.21.15.26:0/395597030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f4767fff640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f476815be40 0x7f476809cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f47677fe640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47680b4ce0 0x7f4768154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f4766ffd640 1 -- 172.21.15.26:0/395597030 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f476815be40 msgr2=0x7f476809cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f4766ffd640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f476815be40 0x7f476809cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f4766ffd640 1 -- 172.21.15.26:0/395597030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47680b4ce0 msgr2=0x7f4768154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f4766ffd640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47680b4ce0 0x7f4768154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f4766ffd640 1 -- 172.21.15.26:0/395597030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4768071bf0 con 0x7f47680d0750 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f4766ffd640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 0x7f476809c800 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f475c0098c0 tx=0x7f475c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f474ffff640 1 -- 172.21.15.26:0/395597030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c037040 con 0x7f47680d0750 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f474ffff640 1 -- 172.21.15.26:0/395597030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475c042070 con 0x7f47680d0750 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4768071ed0 con 0x7f47680d0750 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f474ffff640 1 -- 172.21.15.26:0/395597030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c030a70 con 0x7f47680d0750 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.780+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4768072490 con 0x7f47680d0750 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.781+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476806ce90 con 0x7f47680d0750 2024-04-24T23:25:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.781+0000 7f474ffff640 1 -- 172.21.15.26:0/395597030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f475c030c10 con 0x7f47680d0750 2024-04-24T23:25:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.782+0000 7f474ffff640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f475406be30 0x7f475406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.782+0000 7f474ffff640 1 -- 172.21.15.26:0/395597030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f475c0ab610 con 0x7f47680d0750 2024-04-24T23:25:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.782+0000 7f47677fe640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f475406be30 0x7f475406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.782+0000 7f47677fe640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f475406be30 0x7f475406e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4758006fd0 tx=0x7f4758008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.786+0000 7f474ffff640 1 -- 172.21.15.26:0/395597030 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f476806ce90 con 0x7f47680d0750 2024-04-24T23:25:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.969+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4768072860 con 0x7f47680d0750 2024-04-24T23:25:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.969+0000 7f474ffff640 1 -- 172.21.15.26:0/395597030 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f475c07a310 con 0x7f47680d0750 2024-04-24T23:25:24.970 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:25:24.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f475406be30 msgr2=0x7f475406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f475406be30 0x7f475406e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4758006fd0 tx=0x7f4758008040 comp rx=0 tx=0).stop 2024-04-24T23:25:24.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 msgr2=0x7f476809c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 0x7f476809c800 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f475c0098c0 tx=0x7f475c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 shutdown_connections 2024-04-24T23:25:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f475406be30 0x7f475406e2f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f476815be40 0x7f476809cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47680d0750 0x7f476809c800 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 --2- 172.21.15.26:0/395597030 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47680b4ce0 0x7f4768154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 >> 172.21.15.26:0/395597030 conn(0x7f476806b540 msgr2=0x7f476806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 shutdown_connections 2024-04-24T23:25:24.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:24.971+0000 7f476e129640 1 -- 172.21.15.26:0/395597030 wait complete. 2024-04-24T23:25:24.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:25:25.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.083+0000 7fbe74fb7640 1 Processor -- start 2024-04-24T23:25:25.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.083+0000 7fbe74fb7640 1 -- start start 2024-04-24T23:25:25.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.084+0000 7fbe74fb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7015e290 0x7fbe700d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.084+0000 7fbe74fb7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbe700b3ac0 0x7fbe700b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.084+0000 7fbe74fb7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe7006c230 0x7fbe7006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.084+0000 7fbe74fb7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbe700b4460 con 0x7fbe7006c230 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.084+0000 7fbe74fb7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbe700b5e50 con 0x7fbe700b3ac0 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.084+0000 7fbe74fb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbe700b5840 con 0x7fbe7015e290 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.084+0000 7fbe6ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe7006c230 0x7fbe7006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.084+0000 7fbe6ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe7006c230 0x7fbe7006c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58818/0 (socket says 172.21.15.26:58818) 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.084+0000 7fbe6ed9d640 1 -- 172.21.15.26:0/3107103288 learned_addr learned my addr 172.21.15.26:0/3107103288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.085+0000 7fbe6dd9b640 1 --2- 172.21.15.26:0/3107103288 >> v2:172.21.15.29:3300/0 conn(0x7fbe700b3ac0 0x7fbe700b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.085+0000 7fbe6e59c640 1 --2- 172.21.15.26:0/3107103288 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7015e290 0x7fbe700d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.085+0000 7fbe6ed9d640 1 -- 172.21.15.26:0/3107103288 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7015e290 msgr2=0x7fbe700d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.085+0000 7fbe6ed9d640 1 --2- 172.21.15.26:0/3107103288 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7015e290 0x7fbe700d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.085+0000 7fbe6ed9d640 1 -- 172.21.15.26:0/3107103288 >> v2:172.21.15.29:3300/0 conn(0x7fbe700b3ac0 msgr2=0x7fbe700b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.085+0000 7fbe6ed9d640 1 --2- 172.21.15.26:0/3107103288 >> v2:172.21.15.29:3300/0 conn(0x7fbe700b3ac0 0x7fbe700b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.085+0000 7fbe6ed9d640 1 -- 172.21.15.26:0/3107103288 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe700cfb30 con 0x7fbe7006c230 2024-04-24T23:25:25.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.085+0000 7fbe6ed9d640 1 --2- 172.21.15.26:0/3107103288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7006c230 0x7fbe7006c6a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fbe7006bde0 tx=0x7fbe6002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a10b93d37a1b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:25.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.085+0000 7fbe6d59a640 1 -- 172.21.15.26:0/3107103288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60036070 con 0x7fbe7006c230 2024-04-24T23:25:25.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe6d59a640 1 -- 172.21.15.26:0/3107103288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe60031600 con 0x7fbe7006c230 2024-04-24T23:25:25.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe6d59a640 1 -- 172.21.15.26:0/3107103288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600319d0 con 0x7fbe7006c230 2024-04-24T23:25:25.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe74fb7640 1 -- 172.21.15.26:0/3107103288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7006c230 msgr2=0x7fbe7006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/3107103288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7006c230 0x7fbe7006c6a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fbe7006bde0 tx=0x7fbe6002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe74fb7640 1 -- 172.21.15.26:0/3107103288 shutdown_connections 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/3107103288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7006c230 0x7fbe7006c6a0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/3107103288 >> v2:172.21.15.29:3300/0 conn(0x7fbe700b3ac0 0x7fbe700b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/3107103288 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7015e290 0x7fbe700d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe74fb7640 1 -- 172.21.15.26:0/3107103288 >> 172.21.15.26:0/3107103288 conn(0x7fbe7006b120 msgr2=0x7fbe7006b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe74fb7640 1 -- 172.21.15.26:0/3107103288 shutdown_connections 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.086+0000 7fbe74fb7640 1 -- 172.21.15.26:0/3107103288 wait complete. 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.087+0000 7fbe74fb7640 1 Processor -- start 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe74fb7640 1 -- start start 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe74fb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7006c230 0x7fbe7014e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe74fb7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe700b3ac0 0x7fbe7015e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe74fb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7015e290 0x7fbe7015b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe74fb7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe700b5ae0 con 0x7fbe7015e290 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe74fb7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbe700d00f0 con 0x7fbe700b3ac0 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe74fb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7015e670 con 0x7fbe7006c230 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe6ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7015e290 0x7fbe7015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe6ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7015e290 0x7fbe7015b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58834/0 (socket says 172.21.15.26:58834) 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe6ed9d640 1 -- 172.21.15.26:0/1097957871 learned_addr learned my addr 172.21.15.26:0/1097957871 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe6e59c640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7006c230 0x7fbe7014e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.088+0000 7fbe6dd9b640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe700b3ac0 0x7fbe7015e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe6ed9d640 1 -- 172.21.15.26:0/1097957871 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7006c230 msgr2=0x7fbe7014e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe6ed9d640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7006c230 0x7fbe7014e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe6ed9d640 1 -- 172.21.15.26:0/1097957871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe700b3ac0 msgr2=0x7fbe7015e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe6ed9d640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe700b3ac0 0x7fbe7015e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe6ed9d640 1 -- 172.21.15.26:0/1097957871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7015f2c0 con 0x7fbe7015e290 2024-04-24T23:25:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe6ed9d640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7015e290 0x7fbe7015b210 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009f40 tx=0x7fbe60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe5effd640 1 -- 172.21.15.26:0/1097957871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6003f020 con 0x7fbe7015e290 2024-04-24T23:25:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe5effd640 1 -- 172.21.15.26:0/1097957871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe60037040 con 0x7fbe7015e290 2024-04-24T23:25:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7015f5a0 con 0x7fbe7015e290 2024-04-24T23:25:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe5effd640 1 -- 172.21.15.26:0/1097957871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60036070 con 0x7fbe7015e290 2024-04-24T23:25:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.089+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7015fb60 con 0x7fbe7015e290 2024-04-24T23:25:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.090+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe700694c0 con 0x7fbe7015e290 2024-04-24T23:25:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.091+0000 7fbe5effd640 1 -- 172.21.15.26:0/1097957871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe60042020 con 0x7fbe7015e290 2024-04-24T23:25:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.091+0000 7fbe5effd640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe5006be30 0x7fbe5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.091+0000 7fbe5effd640 1 -- 172.21.15.26:0/1097957871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7fbe600aadf0 con 0x7fbe7015e290 2024-04-24T23:25:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.091+0000 7fbe6e59c640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe5006be30 0x7fbe5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.091+0000 7fbe6e59c640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe5006be30 0x7fbe5006e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe7015be50 tx=0x7fbe68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:25.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.095+0000 7fbe5effd640 1 -- 172.21.15.26:0/1097957871 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe60049030 con 0x7fbe7015e290 2024-04-24T23:25:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.280+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbe700cf4a0 con 0x7fbe7015e290 2024-04-24T23:25:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.281+0000 7fbe5effd640 1 -- 172.21.15.26:0/1097957871 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) ==== 84+0+65 (secure 0 0 0) 0x7fbe60079530 con 0x7fbe7015e290 2024-04-24T23:25:25.282 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:25.282 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:25:25.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:25:25.283 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-24T23:25:25.283 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-24T23:25:25.283 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:25:25.283 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:25:25.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe5006be30 msgr2=0x7fbe5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe5006be30 0x7fbe5006e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe7015be50 tx=0x7fbe68008040 comp rx=0 tx=0).stop 2024-04-24T23:25:25.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7015e290 msgr2=0x7fbe7015b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7015e290 0x7fbe7015b210 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009f40 tx=0x7fbe60004a40 comp rx=0 tx=0).stop 2024-04-24T23:25:25.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 shutdown_connections 2024-04-24T23:25:25.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe5006be30 0x7fbe5006e2f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe7015e290 0x7fbe7015b210 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe700b3ac0 0x7fbe7015e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 --2- 172.21.15.26:0/1097957871 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe7006c230 0x7fbe7014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 >> 172.21.15.26:0/1097957871 conn(0x7fbe7006b120 msgr2=0x7fbe7006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:25.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 shutdown_connections 2024-04-24T23:25:25.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.283+0000 7fbe74fb7640 1 -- 172.21.15.26:0/1097957871 wait complete. 2024-04-24T23:25:25.295 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:25:25.295 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:25:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.400+0000 7f36353a5640 1 Processor -- start 2024-04-24T23:25:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.400+0000 7f36353a5640 1 -- start start 2024-04-24T23:25:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f36353a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3630146f90 0x7f36300b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f36353a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f363006b830 0x7f36300b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f36353a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36300b7010 con 0x7f3630147370 2024-04-24T23:25:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f36353a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f36300b0ba0 con 0x7f3630146f90 2024-04-24T23:25:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f36353a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f36300b0df0 con 0x7f363006b830 2024-04-24T23:25:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f362effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3630146f90 0x7f36300b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f36267fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3630147370 0x7f363006b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32830/0 (socket says 172.21.15.26:32830) 2024-04-24T23:25:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f36267fc640 1 -- 172.21.15.26:0/1259317453 learned_addr learned my addr 172.21.15.26:0/1259317453 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f362effd640 1 -- 172.21.15.26:0/1259317453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f363006b830 msgr2=0x7f36300b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f362effd640 1 --2- 172.21.15.26:0/1259317453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f363006b830 0x7f36300b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f362effd640 1 -- 172.21.15.26:0/1259317453 >> v1:172.21.15.26:6789/0 conn(0x7f3630147370 legacy=0x7f363006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.401+0000 7f362effd640 1 -- 172.21.15.26:0/1259317453 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36300b10d0 con 0x7f3630146f90 2024-04-24T23:25:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f362effd640 1 --2- 172.21.15.26:0/1259317453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630146f90 0x7f36300b4090 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f36200097a0 tx=0x7f362002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e1426e041ee79f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f362e7fc640 1 -- 172.21.15.26:0/1259317453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620037070 con 0x7f3630146f90 2024-04-24T23:25:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f362e7fc640 1 -- 172.21.15.26:0/1259317453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3620032440 con 0x7f3630146f90 2024-04-24T23:25:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f362e7fc640 1 -- 172.21.15.26:0/1259317453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362002ae60 con 0x7f3630146f90 2024-04-24T23:25:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f36353a5640 1 -- 172.21.15.26:0/1259317453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630146f90 msgr2=0x7f36300b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f36353a5640 1 --2- 172.21.15.26:0/1259317453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630146f90 0x7f36300b4090 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f36200097a0 tx=0x7f362002a110 comp rx=0 tx=0).stop 2024-04-24T23:25:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f36353a5640 1 -- 172.21.15.26:0/1259317453 shutdown_connections 2024-04-24T23:25:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f36353a5640 1 --2- 172.21.15.26:0/1259317453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f363006b830 0x7f36300b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f36353a5640 1 --2- 172.21.15.26:0/1259317453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630146f90 0x7f36300b4090 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.402+0000 7f36353a5640 1 -- 172.21.15.26:0/1259317453 >> 172.21.15.26:0/1259317453 conn(0x7f363009aa00 msgr2=0x7f36300b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.403+0000 7f36353a5640 1 -- 172.21.15.26:0/1259317453 shutdown_connections 2024-04-24T23:25:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.403+0000 7f36353a5640 1 -- 172.21.15.26:0/1259317453 wait complete. 2024-04-24T23:25:25.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.404+0000 7f36353a5640 1 Processor -- start 2024-04-24T23:25:25.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.404+0000 7f36353a5640 1 -- start start 2024-04-24T23:25:25.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f36353a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f363006b830 0x7f3630143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f36353a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3630146f90 0x7f363009b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f36353a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630147370 0x7f36301f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f36353a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3630147890 con 0x7f3630147370 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f36353a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f36300b6cc0 con 0x7f3630146f90 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f36353a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3630147a10 con 0x7f363006b830 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630147370 0x7f36301f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630147370 0x7f36301f33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58856/0 (socket says 172.21.15.26:58856) 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362f7fe640 1 -- 172.21.15.26:0/3190760803 learned_addr learned my addr 172.21.15.26:0/3190760803 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362effd640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f363006b830 0x7f3630143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f36267fc640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3630146f90 0x7f363009b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362f7fe640 1 -- 172.21.15.26:0/3190760803 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f363006b830 msgr2=0x7f3630143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362f7fe640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f363006b830 0x7f3630143f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362f7fe640 1 -- 172.21.15.26:0/3190760803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3630146f90 msgr2=0x7f363009b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362f7fe640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3630146f90 0x7f363009b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362f7fe640 1 -- 172.21.15.26:0/3190760803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363009d7f0 con 0x7f3630147370 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f362f7fe640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630147370 0x7f36301f33f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f362800b9e0 tx=0x7f362800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:25.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f3627fff640 1 -- 172.21.15.26:0/3190760803 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628019070 con 0x7f3630147370 2024-04-24T23:25:25.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f3627fff640 1 -- 172.21.15.26:0/3190760803 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362800cb20 con 0x7f3630147370 2024-04-24T23:25:25.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363009dad0 con 0x7f3630147370 2024-04-24T23:25:25.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f3627fff640 1 -- 172.21.15.26:0/3190760803 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628014440 con 0x7f3630147370 2024-04-24T23:25:25.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.405+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363009d130 con 0x7f3630147370 2024-04-24T23:25:25.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.406+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36300694c0 con 0x7f3630147370 2024-04-24T23:25:25.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.407+0000 7f3627fff640 1 -- 172.21.15.26:0/3190760803 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f362801c070 con 0x7f3630147370 2024-04-24T23:25:25.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.407+0000 7f3627fff640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f361006be30 0x7f361006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:25.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.407+0000 7f3627fff640 1 -- 172.21.15.26:0/3190760803 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f362808a8a0 con 0x7f3630147370 2024-04-24T23:25:25.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.407+0000 7f362effd640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f361006be30 0x7f361006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:25.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.408+0000 7f362effd640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f361006be30 0x7f361006e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3620034000 tx=0x7f362002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:25.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.412+0000 7f3627fff640 1 -- 172.21.15.26:0/3190760803 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3628059500 con 0x7f3630147370 2024-04-24T23:25:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.595+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f363006fda0 con 0x7f3630147370 2024-04-24T23:25:25.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.960+0000 7f3627fff640 1 -- 172.21.15.26:0/3190760803 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) ==== 138+0+0 (secure 0 0 0) 0x7f3628058e00 con 0x7f3630147370 2024-04-24T23:25:25.962 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-24T23:25:25.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f361006be30 msgr2=0x7f361006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f361006be30 0x7f361006e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3620034000 tx=0x7f362002e000 comp rx=0 tx=0).stop 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630147370 msgr2=0x7f36301f33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630147370 0x7f36301f33f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f362800b9e0 tx=0x7f362800bea0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 shutdown_connections 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f361006be30 0x7f361006e2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3630147370 0x7f36301f33f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3630146f90 0x7f363009b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 --2- 172.21.15.26:0/3190760803 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f363006b830 0x7f3630143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.963+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 >> 172.21.15.26:0/3190760803 conn(0x7f363009aa00 msgr2=0x7f36300b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.964+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 shutdown_connections 2024-04-24T23:25:25.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:25.964+0000 7f36353a5640 1 -- 172.21.15.26:0/3190760803 wait complete. 2024-04-24T23:25:25.980 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:25:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.089+0000 7fbe23fdc640 1 Processor -- start 2024-04-24T23:25:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.089+0000 7fbe23fdc640 1 -- start start 2024-04-24T23:25:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe23fdc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0ce640 0x7fbe1c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe23fdc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe1c15da30 0x7fbe1c15deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe23fdc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe1c06d750 con 0x7fbe1c0d28e0 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe23fdc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbe1c0d1110 con 0x7fbe1c15da30 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe23fdc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1c0cc410 con 0x7fbe1c0ce640 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe22579640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe1c15da30 0x7fbe1c15deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe22579640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe1c15da30 0x7fbe1c15deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58870/0 (socket says 172.21.15.26:58870) 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe21577640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbe1c0d28e0 0x7fbe1c0cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32832/0 (socket says 172.21.15.26:32832) 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe22579640 1 -- 172.21.15.26:0/1535273482 learned_addr learned my addr 172.21.15.26:0/1535273482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe21d78640 1 --2- 172.21.15.26:0/1535273482 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0ce640 0x7fbe1c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe22579640 1 -- 172.21.15.26:0/1535273482 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0ce640 msgr2=0x7fbe1c0d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe22579640 1 --2- 172.21.15.26:0/1535273482 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0ce640 0x7fbe1c0d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe22579640 1 -- 172.21.15.26:0/1535273482 >> v1:172.21.15.26:6789/0 conn(0x7fbe1c0d28e0 legacy=0x7fbe1c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe22579640 1 -- 172.21.15.26:0/1535273482 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c0b1720 con 0x7fbe1c15da30 2024-04-24T23:25:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.090+0000 7fbe20d76640 1 -- 172.21.15.26:0/1535273482 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1747582598 0 0) 0x7fbe1c06d750 con 0x7fbe1c0d28e0 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.091+0000 7fbe22579640 1 --2- 172.21.15.26:0/1535273482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c15da30 0x7fbe1c15deb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009e50 tx=0x7fbe1402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af541d74b965e235 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.091+0000 7fbe20d76640 1 -- 172.21.15.26:0/1535273482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14036070 con 0x7fbe1c15da30 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.091+0000 7fbe20d76640 1 -- 172.21.15.26:0/1535273482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe14031600 con 0x7fbe1c15da30 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.091+0000 7fbe20d76640 1 -- 172.21.15.26:0/1535273482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe140319d0 con 0x7fbe1c15da30 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.092+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1535273482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c15da30 msgr2=0x7fbe1c15deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.092+0000 7fbe23fdc640 1 --2- 172.21.15.26:0/1535273482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c15da30 0x7fbe1c15deb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009e50 tx=0x7fbe1402a7a0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.092+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1535273482 shutdown_connections 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.092+0000 7fbe23fdc640 1 --2- 172.21.15.26:0/1535273482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c15da30 0x7fbe1c15deb0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.092+0000 7fbe23fdc640 1 --2- 172.21.15.26:0/1535273482 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0ce640 0x7fbe1c0d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.092+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1535273482 >> 172.21.15.26:0/1535273482 conn(0x7fbe1c0d1dc0 msgr2=0x7fbe1c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.092+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1535273482 shutdown_connections 2024-04-24T23:25:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.092+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1535273482 wait complete. 2024-04-24T23:25:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.093+0000 7fbe23fdc640 1 Processor -- start 2024-04-24T23:25:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.093+0000 7fbe23fdc640 1 -- start start 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe23fdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c0ce640 0x7fbe1c0b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe23fdc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0d28e0 0x7fbe1c1e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe23fdc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe1c15da30 0x7fbe1c1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe23fdc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c0cc940 con 0x7fbe1c0ce640 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe23fdc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c0cc660 con 0x7fbe1c15da30 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe23fdc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1c0ccc20 con 0x7fbe1c0d28e0 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21d78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c0ce640 0x7fbe1c0b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21d78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c0ce640 0x7fbe1c0b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58874/0 (socket says 172.21.15.26:58874) 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21d78640 1 -- 172.21.15.26:0/1578106905 learned_addr learned my addr 172.21.15.26:0/1578106905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe22579640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe1c15da30 0x7fbe1c1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21577640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0d28e0 0x7fbe1c1e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21d78640 1 -- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0d28e0 msgr2=0x7fbe1c1e2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21d78640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0d28e0 0x7fbe1c1e2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21d78640 1 -- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe1c15da30 msgr2=0x7fbe1c1511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21d78640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe1c15da30 0x7fbe1c1511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21d78640 1 -- 172.21.15.26:0/1578106905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c070e60 con 0x7fbe1c0ce640 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21577640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0d28e0 0x7fbe1c1e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe22579640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe1c15da30 0x7fbe1c1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe21d78640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c0ce640 0x7fbe1c0b6430 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800d800 tx=0x7fbe1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe127fc640 1 -- 172.21.15.26:0/1578106905 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800b6c0 con 0x7fbe1c0ce640 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.094+0000 7fbe127fc640 1 -- 172.21.15.26:0/1578106905 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe18019070 con 0x7fbe1c0ce640 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.095+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c1e5540 con 0x7fbe1c0ce640 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.095+0000 7fbe127fc640 1 -- 172.21.15.26:0/1578106905 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800ba80 con 0x7fbe1c0ce640 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.095+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c1598d0 con 0x7fbe1c0ce640 2024-04-24T23:25:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.096+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe1c075610 con 0x7fbe1c0ce640 2024-04-24T23:25:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.096+0000 7fbe127fc640 1 -- 172.21.15.26:0/1578106905 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe1800bc20 con 0x7fbe1c0ce640 2024-04-24T23:25:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.097+0000 7fbe127fc640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe0006be60 0x7fbe0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.097+0000 7fbe127fc640 1 -- 172.21.15.26:0/1578106905 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5227+0+0 (secure 0 0 0) 0x7fbe1808b100 con 0x7fbe1c0ce640 2024-04-24T23:25:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.097+0000 7fbe21577640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe0006be60 0x7fbe0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.097+0000 7fbe21577640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe0006be60 0x7fbe0006e320 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c155150 tx=0x7fbe0c007c50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.101+0000 7fbe127fc640 1 -- 172.21.15.26:0/1578106905 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe1801c360 con 0x7fbe1c0ce640 2024-04-24T23:25:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.287+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbe1c0b1500 con 0x7fbe1c0ce640 2024-04-24T23:25:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.963+0000 7fbe127fc640 1 -- 172.21.15.26:0/1578106905 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) ==== 170+0+0 (secure 0 0 0) 0x7fbe18059c80 con 0x7fbe1c0ce640 2024-04-24T23:25:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-24T23:25:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe0006be60 msgr2=0x7fbe0006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe0006be60 0x7fbe0006e320 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c155150 tx=0x7fbe0c007c50 comp rx=0 tx=0).stop 2024-04-24T23:25:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c0ce640 msgr2=0x7fbe1c0b6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c0ce640 0x7fbe1c0b6430 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800d800 tx=0x7fbe1800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 shutdown_connections 2024-04-24T23:25:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbe0006be60 0x7fbe0006e320 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe1c15da30 0x7fbe1c1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbe1c0d28e0 0x7fbe1c1e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 --2- 172.21.15.26:0/1578106905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe1c0ce640 0x7fbe1c0b6430 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 >> 172.21.15.26:0/1578106905 conn(0x7fbe1c0d1dc0 msgr2=0x7fbe1c15d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 shutdown_connections 2024-04-24T23:25:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:26.966+0000 7fbe23fdc640 1 -- 172.21.15.26:0/1578106905 wait complete. 2024-04-24T23:25:26.991 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:25:27.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.100+0000 7f49f491b640 1 Processor -- start 2024-04-24T23:25:27.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.100+0000 7f49f491b640 1 -- start start 2024-04-24T23:25:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49f491b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49f00d18b0 0x7f49f006b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49f491b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f49f006b660 0x7f49f0157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49f491b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f015e920 con 0x7f49f00cc9b0 2024-04-24T23:25:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49f491b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f49f006bab0 con 0x7f49f00d18b0 2024-04-24T23:25:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49f491b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f49f015eba0 con 0x7f49f006b660 2024-04-24T23:25:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49edd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49f00d18b0 0x7f49f006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49edd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49f00d18b0 0x7f49f006b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58886/0 (socket says 172.21.15.26:58886) 2024-04-24T23:25:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49edd9b640 1 -- 172.21.15.26:0/628574394 learned_addr learned my addr 172.21.15.26:0/628574394 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49eed9d640 1 --2- 172.21.15.26:0/628574394 >> v2:172.21.15.29:3300/0 conn(0x7f49f006b660 0x7f49f0157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49edd9b640 1 -- 172.21.15.26:0/628574394 >> v2:172.21.15.29:3300/0 conn(0x7f49f006b660 msgr2=0x7f49f0157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49edd9b640 1 --2- 172.21.15.26:0/628574394 >> v2:172.21.15.29:3300/0 conn(0x7f49f006b660 0x7f49f0157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49edd9b640 1 -- 172.21.15.26:0/628574394 >> v1:172.21.15.29:6789/0 conn(0x7f49f00cc9b0 legacy=0x7f49f00d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:27.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.101+0000 7f49edd9b640 1 -- 172.21.15.26:0/628574394 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f015ee80 con 0x7f49f00d18b0 2024-04-24T23:25:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.102+0000 7f49edd9b640 1 --2- 172.21.15.26:0/628574394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f00d18b0 0x7f49f006b120 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f49f009eea0 tx=0x7f49e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e791a58bf7d43e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.102+0000 7f49ed59a640 1 -- 172.21.15.26:0/628574394 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4037070 con 0x7f49f00d18b0 2024-04-24T23:25:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.102+0000 7f49ed59a640 1 -- 172.21.15.26:0/628574394 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e4032440 con 0x7f49f00d18b0 2024-04-24T23:25:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.102+0000 7f49ed59a640 1 -- 172.21.15.26:0/628574394 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4032810 con 0x7f49f00d18b0 2024-04-24T23:25:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.102+0000 7f49f491b640 1 -- 172.21.15.26:0/628574394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f00d18b0 msgr2=0x7f49f006b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:27.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.102+0000 7f49f491b640 1 --2- 172.21.15.26:0/628574394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f00d18b0 0x7f49f006b120 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f49f009eea0 tx=0x7f49e402a270 comp rx=0 tx=0).stop 2024-04-24T23:25:27.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.103+0000 7f49f491b640 1 -- 172.21.15.26:0/628574394 shutdown_connections 2024-04-24T23:25:27.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.103+0000 7f49f491b640 1 --2- 172.21.15.26:0/628574394 >> v2:172.21.15.29:3300/0 conn(0x7f49f006b660 0x7f49f0157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:27.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.103+0000 7f49f491b640 1 --2- 172.21.15.26:0/628574394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f00d18b0 0x7f49f006b120 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:27.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.103+0000 7f49f491b640 1 -- 172.21.15.26:0/628574394 >> 172.21.15.26:0/628574394 conn(0x7f49f00d0d20 msgr2=0x7f49f00d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:27.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.103+0000 7f49f491b640 1 -- 172.21.15.26:0/628574394 shutdown_connections 2024-04-24T23:25:27.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.103+0000 7f49f491b640 1 -- 172.21.15.26:0/628574394 wait complete. 2024-04-24T23:25:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.104+0000 7f49f491b640 1 Processor -- start 2024-04-24T23:25:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.104+0000 7f49f491b640 1 -- start start 2024-04-24T23:25:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49f491b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f006b660 0x7f49f015af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49f491b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49f00cc9b0 0x7f49f014e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49f491b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f49f00d18b0 0x7f49f00b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49f491b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f49f00cec20 con 0x7f49f006b660 2024-04-24T23:25:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49f491b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f49f015f440 con 0x7f49f00cc9b0 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49f491b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f49f0158500 con 0x7f49f00d18b0 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49ee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f006b660 0x7f49f015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49ee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f006b660 0x7f49f015af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58898/0 (socket says 172.21.15.26:58898) 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49ee59c640 1 -- 172.21.15.26:0/3750177412 learned_addr learned my addr 172.21.15.26:0/3750177412 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49edd9b640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49f00cc9b0 0x7f49f014e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49eed9d640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f49f00d18b0 0x7f49f00b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49ee59c640 1 -- 172.21.15.26:0/3750177412 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f49f00d18b0 msgr2=0x7f49f00b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49ee59c640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f49f00d18b0 0x7f49f00b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49ee59c640 1 -- 172.21.15.26:0/3750177412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49f00cc9b0 msgr2=0x7f49f014e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49ee59c640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49f00cc9b0 0x7f49f014e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49ee59c640 1 -- 172.21.15.26:0/3750177412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f015b490 con 0x7f49f006b660 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49ee59c640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f006b660 0x7f49f015af50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f49e800c400 tx=0x7f49e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.105+0000 7f49deffd640 1 -- 172.21.15.26:0/3750177412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80052d0 con 0x7f49f006b660 2024-04-24T23:25:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.106+0000 7f49deffd640 1 -- 172.21.15.26:0/3750177412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e8005490 con 0x7f49f006b660 2024-04-24T23:25:27.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.106+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f015b730 con 0x7f49f006b660 2024-04-24T23:25:27.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.106+0000 7f49deffd640 1 -- 172.21.15.26:0/3750177412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8007260 con 0x7f49f006b660 2024-04-24T23:25:27.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.106+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f00b3f10 con 0x7f49f006b660 2024-04-24T23:25:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.107+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f00b3a20 con 0x7f49f006b660 2024-04-24T23:25:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.107+0000 7f49deffd640 1 -- 172.21.15.26:0/3750177412 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49e8007400 con 0x7f49f006b660 2024-04-24T23:25:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.107+0000 7f49deffd640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f49d006be60 0x7f49d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.108+0000 7f49deffd640 1 -- 172.21.15.26:0/3750177412 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5609+0+0 (secure 0 0 0) 0x7f49e808a290 con 0x7f49f006b660 2024-04-24T23:25:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.108+0000 7f49edd9b640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f49d006be60 0x7f49d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.108+0000 7f49edd9b640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f49d006be60 0x7f49d006e320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f49e4034000 tx=0x7f49e402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:27.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.112+0000 7f49deffd640 1 -- 172.21.15.26:0/3750177412 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f49f015e920 con 0x7f49f006b660 2024-04-24T23:25:27.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.339+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f49f0072fd0 con 0x7f49f006b660 2024-04-24T23:25:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 10 and data pool 11 2024-04-24T23:25:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:27.995+0000 7f49deffd640 1 -- 172.21.15.26:0/3750177412 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v63) ==== 174+0+0 (secure 0 0 0) 0x7f49e8058650 con 0x7f49f006b660 2024-04-24T23:25:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.000+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f49d006be60 msgr2=0x7f49d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.000+0000 7f49f491b640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f49d006be60 0x7f49d006e320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f49e4034000 tx=0x7f49e402e000 comp rx=0 tx=0).stop 2024-04-24T23:25:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.000+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f006b660 msgr2=0x7f49f015af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.000+0000 7f49f491b640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f006b660 0x7f49f015af50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f49e800c400 tx=0x7f49e800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.002+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 shutdown_connections 2024-04-24T23:25:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.002+0000 7f49f491b640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f49d006be60 0x7f49d006e320 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.002+0000 7f49f491b640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f49f00d18b0 0x7f49f00b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.002+0000 7f49f491b640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49f00cc9b0 0x7f49f014e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.002+0000 7f49f491b640 1 --2- 172.21.15.26:0/3750177412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49f006b660 0x7f49f015af50 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.002+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 >> 172.21.15.26:0/3750177412 conn(0x7f49f00d0d20 msgr2=0x7f49f00b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.004+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 shutdown_connections 2024-04-24T23:25:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.005+0000 7f49f491b640 1 -- 172.21.15.26:0/3750177412 wait complete. 2024-04-24T23:25:28.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:25:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.123+0000 7fda74d48640 1 Processor -- start 2024-04-24T23:25:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.123+0000 7fda74d48640 1 -- start start 2024-04-24T23:25:28.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda74d48640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fda7015e380 0x7fda700b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda74d48640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 0x7fda700b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda74d48640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda700b0980 con 0x7fda700b4fd0 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda74d48640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fda7015e760 con 0x7fda7015e380 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda74d48640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fda700725b0 con 0x7fda700b0030 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda67fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fda700b4fd0 0x7fda70072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32834/0 (socket says 172.21.15.26:32834) 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda67fff640 1 -- 172.21.15.26:0/3114856931 learned_addr learned my addr 172.21.15.26:0/3114856931 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6ed9d640 1 --2- 172.21.15.26:0/3114856931 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 0x7fda700b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6e59c640 1 --2- 172.21.15.26:0/3114856931 >> v2:172.21.15.29:3300/0 conn(0x7fda7015e380 0x7fda700b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6dd9b640 1 -- 172.21.15.26:0/3114856931 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3608957243 0 0) 0x7fda58001110 con 0x7fda700b4fd0 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6dd9b640 1 -- 172.21.15.26:0/3114856931 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda5c003520 con 0x7fda700b4fd0 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6dd9b640 1 -- 172.21.15.26:0/3114856931 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3916978231 0 0) 0x7fda5c003520 con 0x7fda700b4fd0 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6e59c640 1 -- 172.21.15.26:0/3114856931 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 msgr2=0x7fda700b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6e59c640 1 --2- 172.21.15.26:0/3114856931 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 0x7fda700b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6e59c640 1 -- 172.21.15.26:0/3114856931 >> v1:172.21.15.26:6789/0 conn(0x7fda700b4fd0 legacy=0x7fda70072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6e59c640 1 -- 172.21.15.26:0/3114856931 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda700b5440 con 0x7fda7015e380 2024-04-24T23:25:28.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.124+0000 7fda6ed9d640 1 --2- 172.21.15.26:0/3114856931 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 0x7fda700b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:28.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.125+0000 7fda6e59c640 1 --2- 172.21.15.26:0/3114856931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda7015e380 0x7fda700b4530 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda600097a0 tx=0x7fda6002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ba65a621e58f32d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:28.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.125+0000 7fda6dd9b640 1 -- 172.21.15.26:0/3114856931 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60037070 con 0x7fda7015e380 2024-04-24T23:25:28.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.125+0000 7fda6dd9b640 1 -- 172.21.15.26:0/3114856931 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6002ac30 con 0x7fda7015e380 2024-04-24T23:25:28.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.125+0000 7fda6dd9b640 1 -- 172.21.15.26:0/3114856931 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58001840 con 0x7fda7015e380 2024-04-24T23:25:28.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.126+0000 7fda74d48640 1 -- 172.21.15.26:0/3114856931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda7015e380 msgr2=0x7fda700b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.126+0000 7fda74d48640 1 --2- 172.21.15.26:0/3114856931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda7015e380 0x7fda700b4530 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda600097a0 tx=0x7fda6002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.126+0000 7fda74d48640 1 -- 172.21.15.26:0/3114856931 shutdown_connections 2024-04-24T23:25:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.126+0000 7fda74d48640 1 --2- 172.21.15.26:0/3114856931 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 0x7fda700b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.126+0000 7fda74d48640 1 --2- 172.21.15.26:0/3114856931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda7015e380 0x7fda700b4530 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.126+0000 7fda74d48640 1 -- 172.21.15.26:0/3114856931 >> 172.21.15.26:0/3114856931 conn(0x7fda7006e2c0 msgr2=0x7fda7006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.126+0000 7fda74d48640 1 -- 172.21.15.26:0/3114856931 shutdown_connections 2024-04-24T23:25:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.126+0000 7fda74d48640 1 -- 172.21.15.26:0/3114856931 wait complete. 2024-04-24T23:25:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.127+0000 7fda74d48640 1 Processor -- start 2024-04-24T23:25:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.127+0000 7fda74d48640 1 -- start start 2024-04-24T23:25:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda74d48640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 0x7fda701f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda74d48640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda700b4fd0 0x7fda701f1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda74d48640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda7015e380 0x7fda700c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda74d48640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda70072240 con 0x7fda7015e380 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda74d48640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda700b59e0 con 0x7fda700b4fd0 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda74d48640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fda700ae530 con 0x7fda700b0030 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda6ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda7015e380 0x7fda700c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda6ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda7015e380 0x7fda700c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58912/0 (socket says 172.21.15.26:58912) 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda6ed9d640 1 -- 172.21.15.26:0/1348681661 learned_addr learned my addr 172.21.15.26:0/1348681661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda67fff640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda700b4fd0 0x7fda701f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda6e59c640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 0x7fda701f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda6ed9d640 1 -- 172.21.15.26:0/1348681661 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 msgr2=0x7fda701f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda6ed9d640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 0x7fda701f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda6ed9d640 1 -- 172.21.15.26:0/1348681661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda700b4fd0 msgr2=0x7fda701f1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda6ed9d640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda700b4fd0 0x7fda701f1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.128+0000 7fda6ed9d640 1 -- 172.21.15.26:0/1348681661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda701f4e90 con 0x7fda7015e380 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.129+0000 7fda6ed9d640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda7015e380 0x7fda700c68f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fda6800efc0 tx=0x7fda68009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.129+0000 7fda66ffd640 1 -- 172.21.15.26:0/1348681661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68018030 con 0x7fda7015e380 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.129+0000 7fda66ffd640 1 -- 172.21.15.26:0/1348681661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6800cde0 con 0x7fda7015e380 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.129+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda700c64f0 con 0x7fda7015e380 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.129+0000 7fda66ffd640 1 -- 172.21.15.26:0/1348681661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68003bb0 con 0x7fda7015e380 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.129+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda701f56a0 con 0x7fda7015e380 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.130+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda700694c0 con 0x7fda7015e380 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.130+0000 7fda66ffd640 1 -- 172.21.15.26:0/1348681661 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda68005d10 con 0x7fda7015e380 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.131+0000 7fda66ffd640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda5006be30 0x7fda5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.131+0000 7fda66ffd640 1 -- 172.21.15.26:0/1348681661 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fda6808b900 con 0x7fda7015e380 2024-04-24T23:25:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.131+0000 7fda6e59c640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda5006be30 0x7fda5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.131+0000 7fda6e59c640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda5006be30 0x7fda5006e2f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fda60034000 tx=0x7fda6002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:28.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.135+0000 7fda66ffd640 1 -- 172.21.15.26:0/1348681661 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fda68022070 con 0x7fda7015e380 2024-04-24T23:25:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.320+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda700c66c0 con 0x7fda7015e380 2024-04-24T23:25:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.321+0000 7fda66ffd640 1 -- 172.21.15.26:0/1348681661 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16621 (secure 0 0 0) 0x7fda701f4e90 con 0x7fda7015e380 2024-04-24T23:25:28.323 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:28.324 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":68,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:25:27.340513+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:25:25.626812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:25:26.319330+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:25:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda5006be30 msgr2=0x7fda5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda5006be30 0x7fda5006e2f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fda60034000 tx=0x7fda6002e000 comp rx=0 tx=0).stop 2024-04-24T23:25:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda7015e380 msgr2=0x7fda700c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda7015e380 0x7fda700c68f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fda6800efc0 tx=0x7fda68009f90 comp rx=0 tx=0).stop 2024-04-24T23:25:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 shutdown_connections 2024-04-24T23:25:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda5006be30 0x7fda5006e2f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda7015e380 0x7fda700c68f0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda700b4fd0 0x7fda701f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 --2- 172.21.15.26:0/1348681661 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda700b0030 0x7fda701f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 >> 172.21.15.26:0/1348681661 conn(0x7fda7006e2c0 msgr2=0x7fda7006b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 shutdown_connections 2024-04-24T23:25:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.323+0000 7fda74d48640 1 -- 172.21.15.26:0/1348681661 wait complete. 2024-04-24T23:25:28.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:25:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.435+0000 7f9f693c1640 1 Processor -- start 2024-04-24T23:25:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.435+0000 7f9f693c1640 1 -- start start 2024-04-24T23:25:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.436+0000 7f9f693c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640b3ac0 0x7f9f640b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.436+0000 7f9f693c1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f640b5840 con 0x7f9f640b62e0 2024-04-24T23:25:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.436+0000 7f9f693c1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f640cfb30 con 0x7f9f6415e290 2024-04-24T23:25:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.436+0000 7f9f693c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9f640b4460 con 0x7f9f640b3ac0 2024-04-24T23:25:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.436+0000 7f9f637fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9f640b62e0 0x7f9f640b6750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32846/0 (socket says 172.21.15.26:32846) 2024-04-24T23:25:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.436+0000 7f9f637fe640 1 -- 172.21.15.26:0/3893421675 learned_addr learned my addr 172.21.15.26:0/3893421675 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f627fc640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640b3ac0 0x7f9f640b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1837344598 0 0) 0x7f9f640b5840 con 0x7f9f640b62e0 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f4c003520 con 0x7f9f640b62e0 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1220142540 0 0) 0x7f9f4c003520 con 0x7f9f640b62e0 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640b3ac0 msgr2=0x7f9f640b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640b3ac0 0x7f9f640b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 >> v1:172.21.15.29:6789/0 conn(0x7f9f6415e290 legacy=0x7f9f640d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f640cfe10 con 0x7f9f640b62e0 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f9f54001a10 con 0x7f9f640b62e0 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 >> v1:172.21.15.26:6789/0 conn(0x7f9f640b62e0 legacy=0x7f9f640b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f4c0039e0 0x7f9f4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640b3ac0 0x7f9f4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c00ba70 con 0x7f9f4c004800 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c00bbf0 con 0x7f9f4c0039e0 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4c00bdf0 con 0x7f9f640b3ac0 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f627fc640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f62ffd640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f4c0039e0 0x7f9f4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f637fe640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640b3ac0 0x7f9f4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f627fc640 1 -- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640b3ac0 msgr2=0x7f9f4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f627fc640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640b3ac0 0x7f9f4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f627fc640 1 -- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f4c0039e0 msgr2=0x7f9f4c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f627fc640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f4c0039e0 0x7f9f4c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f627fc640 1 -- 172.21.15.26:0/3893421675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c003520 con 0x7f9f4c004800 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.437+0000 7f9f62ffd640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f4c0039e0 0x7f9f4c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.438+0000 7f9f627fc640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f4c004c80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009a90 tx=0x7f9f4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f569d4b9987637fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.438+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58036070 con 0x7f9f4c004800 2024-04-24T23:25:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.438+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f58031400 con 0x7f9f4c004800 2024-04-24T23:25:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.438+0000 7f9f61ffb640 1 -- 172.21.15.26:0/3893421675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58009690 con 0x7f9f4c004800 2024-04-24T23:25:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.439+0000 7f9f693c1640 1 -- 172.21.15.26:0/3893421675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 msgr2=0x7f9f4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.439+0000 7f9f693c1640 1 --2- 172.21.15.26:0/3893421675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f4c004c80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009a90 tx=0x7f9f4c005830 comp rx=0 tx=0).stop 2024-04-24T23:25:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.439+0000 7f9f637fe640 1 -- 172.21.15.26:0/3893421675 reap_dead start 2024-04-24T23:25:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.439+0000 7f9f693c1640 1 -- 172.21.15.26:0/3893421675 shutdown_connections 2024-04-24T23:25:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.439+0000 7f9f693c1640 1 -- 172.21.15.26:0/3893421675 >> 172.21.15.26:0/3893421675 conn(0x7f9f6406b120 msgr2=0x7f9f6406b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.439+0000 7f9f693c1640 1 -- 172.21.15.26:0/3893421675 shutdown_connections 2024-04-24T23:25:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.439+0000 7f9f693c1640 1 -- 172.21.15.26:0/3893421675 wait complete. 2024-04-24T23:25:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.440+0000 7f9f693c1640 1 Processor -- start 2024-04-24T23:25:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.440+0000 7f9f693c1640 1 -- start start 2024-04-24T23:25:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f693c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f6415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f693c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f640d16f0 0x7f9f640d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f693c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640caf50 0x7f9f640cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f693c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640b5ae0 con 0x7f9f4c004800 2024-04-24T23:25:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f693c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640b5e50 con 0x7f9f640d16f0 2024-04-24T23:25:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f693c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6415e670 con 0x7f9f640caf50 2024-04-24T23:25:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f6415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f6415a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58934/0 (socket says 172.21.15.26:58934) 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f62ffd640 1 -- 172.21.15.26:0/1815528024 learned_addr learned my addr 172.21.15.26:0/1815528024 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f637fe640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640caf50 0x7f9f640cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f627fc640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f640d16f0 0x7f9f640d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f62ffd640 1 -- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640caf50 msgr2=0x7f9f640cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f62ffd640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640caf50 0x7f9f640cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f62ffd640 1 -- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f640d16f0 msgr2=0x7f9f640d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f62ffd640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f640d16f0 0x7f9f640d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f62ffd640 1 -- 172.21.15.26:0/1815528024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6415e980 con 0x7f9f4c004800 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f637fe640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640caf50 0x7f9f640cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f627fc640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f640d16f0 0x7f9f640d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.441+0000 7f9f62ffd640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f6415a910 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00e600 tx=0x7f9f5c00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.442+0000 7f9f4b7fe640 1 -- 172.21.15.26:0/1815528024 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0052e0 con 0x7f9f4c004800 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.442+0000 7f9f4b7fe640 1 -- 172.21.15.26:0/1815528024 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c018070 con 0x7f9f4c004800 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.442+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6415ae50 con 0x7f9f4c004800 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.442+0000 7f9f4b7fe640 1 -- 172.21.15.26:0/1815528024 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c003810 con 0x7f9f4c004800 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.442+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6415ebf0 con 0x7f9f4c004800 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.443+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f640694c0 con 0x7f9f4c004800 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.443+0000 7f9f4b7fe640 1 -- 172.21.15.26:0/1815528024 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f5c0039b0 con 0x7f9f4c004800 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.444+0000 7f9f4b7fe640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9f5006be60 0x7f9f5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.444+0000 7f9f4b7fe640 1 -- 172.21.15.26:0/1815528024 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f9f5c08ab50 con 0x7f9f4c004800 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.444+0000 7f9f627fc640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9f5006be60 0x7f9f5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.444+0000 7f9f627fc640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9f5006be60 0x7f9f5006e320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9f58033000 tx=0x7f9f58008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.448+0000 7f9f4b7fe640 1 -- 172.21.15.26:0/1815528024 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9f5c059500 con 0x7f9f4c004800 2024-04-24T23:25:28.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.633+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f640b7b10 con 0x7f9f4c004800 2024-04-24T23:25:28.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.635+0000 7f9f4b7fe640 1 -- 172.21.15.26:0/1815528024 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16621 (secure 0 0 0) 0x7f9f5c058e00 con 0x7f9f4c004800 2024-04-24T23:25:28.637 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:28.637 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":68,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:25:27.340513+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:25:25.626812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:25:26.319330+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:25:28.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9f5006be60 msgr2=0x7f9f5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9f5006be60 0x7f9f5006e320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9f58033000 tx=0x7f9f58008030 comp rx=0 tx=0).stop 2024-04-24T23:25:28.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 msgr2=0x7f9f6415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f6415a910 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00e600 tx=0x7f9f5c00eac0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 shutdown_connections 2024-04-24T23:25:28.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9f5006be60 0x7f9f5006e320 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9f640caf50 0x7f9f640cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f640d16f0 0x7f9f640d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 --2- 172.21.15.26:0/1815528024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f4c004800 0x7f9f6415a910 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 >> 172.21.15.26:0/1815528024 conn(0x7f9f6406b120 msgr2=0x7f9f6406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:28.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 shutdown_connections 2024-04-24T23:25:28.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.637+0000 7f9f693c1640 1 -- 172.21.15.26:0/1815528024 wait complete. 2024-04-24T23:25:28.650 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:25:28.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.752+0000 7f9b177e7640 1 Processor -- start 2024-04-24T23:25:28.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.752+0000 7f9b177e7640 1 -- start start 2024-04-24T23:25:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b177e7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9b1015adc0 0x7f9b100b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b177e7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b100ad7b0 0x7f9b100d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b177e7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1015ef60 con 0x7f9b1015e9c0 2024-04-24T23:25:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b177e7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1015f240 con 0x7f9b100ad7b0 2024-04-24T23:25:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b177e7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1015f520 con 0x7f9b1015adc0 2024-04-24T23:25:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b14d82640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b100ad7b0 0x7f9b100d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b14d82640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b100ad7b0 0x7f9b100d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58944/0 (socket says 172.21.15.26:58944) 2024-04-24T23:25:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b14d82640 1 -- 172.21.15.26:0/3444604862 learned_addr learned my addr 172.21.15.26:0/3444604862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b15583640 1 --2- 172.21.15.26:0/3444604862 >> v2:172.21.15.29:3300/0 conn(0x7f9b1015adc0 0x7f9b100b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b14d82640 1 -- 172.21.15.26:0/3444604862 >> v2:172.21.15.29:3300/0 conn(0x7f9b1015adc0 msgr2=0x7f9b100b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b14d82640 1 --2- 172.21.15.26:0/3444604862 >> v2:172.21.15.29:3300/0 conn(0x7f9b1015adc0 0x7f9b100b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b14d82640 1 -- 172.21.15.26:0/3444604862 >> v1:172.21.15.29:6789/0 conn(0x7f9b1015e9c0 legacy=0x7f9b1006dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b14d82640 1 -- 172.21.15.26:0/3444604862 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1015f800 con 0x7f9b100ad7b0 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.753+0000 7f9b15583640 1 --2- 172.21.15.26:0/3444604862 >> v2:172.21.15.29:3300/0 conn(0x7f9b1015adc0 0x7f9b100b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.754+0000 7f9b14d82640 1 --2- 172.21.15.26:0/3444604862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b100ad7b0 0x7f9b100d0040 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009870 tx=0x7f9b0002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7c78a9e3bd300 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.754+0000 7f9b07fff640 1 -- 172.21.15.26:0/3444604862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00037070 con 0x7f9b100ad7b0 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.754+0000 7f9b07fff640 1 -- 172.21.15.26:0/3444604862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0002ae40 con 0x7f9b100ad7b0 2024-04-24T23:25:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.754+0000 7f9b07fff640 1 -- 172.21.15.26:0/3444604862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00032660 con 0x7f9b100ad7b0 2024-04-24T23:25:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.755+0000 7f9b177e7640 1 -- 172.21.15.26:0/3444604862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b100ad7b0 msgr2=0x7f9b100d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.755+0000 7f9b177e7640 1 --2- 172.21.15.26:0/3444604862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b100ad7b0 0x7f9b100d0040 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009870 tx=0x7f9b0002a130 comp rx=0 tx=0).stop 2024-04-24T23:25:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.755+0000 7f9b177e7640 1 -- 172.21.15.26:0/3444604862 shutdown_connections 2024-04-24T23:25:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.755+0000 7f9b177e7640 1 --2- 172.21.15.26:0/3444604862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b100ad7b0 0x7f9b100d0040 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.755+0000 7f9b177e7640 1 --2- 172.21.15.26:0/3444604862 >> v2:172.21.15.29:3300/0 conn(0x7f9b1015adc0 0x7f9b100b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.755+0000 7f9b177e7640 1 -- 172.21.15.26:0/3444604862 >> 172.21.15.26:0/3444604862 conn(0x7f9b1006b540 msgr2=0x7f9b1006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.755+0000 7f9b177e7640 1 -- 172.21.15.26:0/3444604862 shutdown_connections 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.755+0000 7f9b177e7640 1 -- 172.21.15.26:0/3444604862 wait complete. 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.756+0000 7f9b177e7640 1 Processor -- start 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.756+0000 7f9b177e7640 1 -- start start 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b177e7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b100ad7b0 0x7f9b1014e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b177e7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b1015adc0 0x7f9b10155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b177e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b1015e9c0 0x7f9b100b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b177e7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1015b1a0 con 0x7f9b1015e9c0 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b177e7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9b100b48b0 con 0x7f9b100ad7b0 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b177e7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9b100d0580 con 0x7f9b1015adc0 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15d84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b1015e9c0 0x7f9b100b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15d84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b1015e9c0 0x7f9b100b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58958/0 (socket says 172.21.15.26:58958) 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15d84640 1 -- 172.21.15.26:0/2127359775 learned_addr learned my addr 172.21.15.26:0/2127359775 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b14d82640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b1015adc0 0x7f9b10155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15583640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b100ad7b0 0x7f9b1014e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15d84640 1 -- 172.21.15.26:0/2127359775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b1015adc0 msgr2=0x7f9b10155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15d84640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b1015adc0 0x7f9b10155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15d84640 1 -- 172.21.15.26:0/2127359775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b100ad7b0 msgr2=0x7f9b1014e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15d84640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b100ad7b0 0x7f9b1014e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15d84640 1 -- 172.21.15.26:0/2127359775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1009ca10 con 0x7f9b1015e9c0 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.757+0000 7f9b15d84640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b1015e9c0 0x7f9b100b1bc0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b970 tx=0x7f9b0800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.758+0000 7f9b05ffb640 1 -- 172.21.15.26:0/2127359775 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08019070 con 0x7f9b1015e9c0 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.758+0000 7f9b05ffb640 1 -- 172.21.15.26:0/2127359775 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0800cb40 con 0x7f9b1015e9c0 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.758+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1009ccf0 con 0x7f9b1015e9c0 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.758+0000 7f9b05ffb640 1 -- 172.21.15.26:0/2127359775 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08014440 con 0x7f9b1015e9c0 2024-04-24T23:25:28.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.758+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1009d2b0 con 0x7f9b1015e9c0 2024-04-24T23:25:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.759+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b10075610 con 0x7f9b1015e9c0 2024-04-24T23:25:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.759+0000 7f9b05ffb640 1 -- 172.21.15.26:0/2127359775 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b0801c070 con 0x7f9b1015e9c0 2024-04-24T23:25:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.759+0000 7f9b05ffb640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9af406be30 0x7f9af406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.759+0000 7f9b05ffb640 1 -- 172.21.15.26:0/2127359775 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f9b0808aa60 con 0x7f9b1015e9c0 2024-04-24T23:25:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.760+0000 7f9b15583640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9af406be30 0x7f9af406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.760+0000 7f9b15583640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9af406be30 0x7f9af406e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00a990 tx=0x7f9b0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:28.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.764+0000 7f9b05ffb640 1 -- 172.21.15.26:0/2127359775 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9b080594a0 con 0x7f9b1015e9c0 2024-04-24T23:25:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:28.994+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9b1015e730 con 0x7f9b1015e9c0 2024-04-24T23:25:29.571 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:25:29.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:25:29.962 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-04-24T23:25:29.960+0000 7f59be5ed640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:25:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:29.999+0000 7f9b05ffb640 1 -- 172.21.15.26:0/2127359775 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7f9b08058da0 con 0x7f9b1015e9c0 2024-04-24T23:25:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9af406be30 msgr2=0x7f9af406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9af406be30 0x7f9af406e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00a990 tx=0x7f9b0c008040 comp rx=0 tx=0).stop 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b1015e9c0 msgr2=0x7f9b100b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b1015e9c0 0x7f9b100b1bc0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b970 tx=0x7f9b0800be30 comp rx=0 tx=0).stop 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 shutdown_connections 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9af406be30 0x7f9af406e2f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b1015e9c0 0x7f9b100b1bc0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b1015adc0 0x7f9b10155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 --2- 172.21.15.26:0/2127359775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b100ad7b0 0x7f9b1014e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 >> 172.21.15.26:0/2127359775 conn(0x7f9b1006b540 msgr2=0x7f9b1015dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 shutdown_connections 2024-04-24T23:25:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.002+0000 7f9b177e7640 1 -- 172.21.15.26:0/2127359775 wait complete. 2024-04-24T23:25:30.018 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:25:30.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:25:30.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.128+0000 7f518a763640 1 Processor -- start 2024-04-24T23:25:30.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.128+0000 7f518a763640 1 -- start start 2024-04-24T23:25:30.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f518a763640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51840cd120 0x7f51840af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:30.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f518a763640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51840c5350 0x7f518406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:30.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f518a763640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f518406f0a0 con 0x7f51840c4ef0 2024-04-24T23:25:30.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f518a763640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f51840c69e0 con 0x7f51840c5350 2024-04-24T23:25:30.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f518a763640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f51840c6ef0 con 0x7f51840cd120 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f51837fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f51840c4ef0 0x7f518406e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32856/0 (socket says 172.21.15.26:32856) 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f51837fe640 1 -- 172.21.15.26:0/604838404 learned_addr learned my addr 172.21.15.26:0/604838404 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5188d00640 1 --2- 172.21.15.26:0/604838404 >> v2:172.21.15.29:3300/0 conn(0x7f51840c5350 0x7f518406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5183fff640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51840cd120 0x7f51840af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089381236 0 0) 0x7f518406f0a0 con 0x7f51840c4ef0 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5170003520 con 0x7f51840c4ef0 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2450160947 0 0) 0x7f5170003520 con 0x7f51840c4ef0 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51840cd120 msgr2=0x7f51840af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51840cd120 0x7f51840af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 >> v2:172.21.15.29:3300/0 conn(0x7f51840c5350 msgr2=0x7f518406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 --2- 172.21.15.26:0/604838404 >> v2:172.21.15.29:3300/0 conn(0x7f51840c5350 0x7f518406d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51840711e0 con 0x7f51840c4ef0 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f516c001670 con 0x7f51840c4ef0 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 >> v1:172.21.15.26:6789/0 conn(0x7f51840c4ef0 legacy=0x7f518406e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51700039b0 0x7f5170003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5170004800 0x7f5170004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51840cd120 0x7f51700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f51840c6ef0 con 0x7f5170004800 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f517000b9c0 con 0x7f51700039b0 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f517000bbc0 con 0x7f51840cd120 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f51837fe640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5170004800 0x7f5170004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5183fff640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51700039b0 0x7f5170003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5188d00640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51840cd120 0x7f51700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f51837fe640 1 -- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51840cd120 msgr2=0x7f51700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f51837fe640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51840cd120 0x7f51700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f51837fe640 1 -- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51700039b0 msgr2=0x7f5170003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f51837fe640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51700039b0 0x7f5170003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f51837fe640 1 -- 172.21.15.26:0/604838404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5170003520 con 0x7f5170004800 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.129+0000 7f5188d00640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51840cd120 0x7f51700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.130+0000 7f51837fe640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5170004800 0x7f5170004c80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f516c001050 tx=0x7f5170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99d50c507d1e9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.130+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c02e070 con 0x7f5170004800 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.130+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516c005af0 con 0x7f5170004800 2024-04-24T23:25:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.130+0000 7f5182ffd640 1 -- 172.21.15.26:0/604838404 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c007d70 con 0x7f5170004800 2024-04-24T23:25:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.130+0000 7f518a763640 1 -- 172.21.15.26:0/604838404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5170004800 msgr2=0x7f5170004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.130+0000 7f518a763640 1 --2- 172.21.15.26:0/604838404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5170004800 0x7f5170004c80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f516c001050 tx=0x7f5170005830 comp rx=0 tx=0).stop 2024-04-24T23:25:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.130+0000 7f5188d00640 1 -- 172.21.15.26:0/604838404 reap_dead start 2024-04-24T23:25:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.131+0000 7f518a763640 1 -- 172.21.15.26:0/604838404 shutdown_connections 2024-04-24T23:25:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.131+0000 7f518a763640 1 -- 172.21.15.26:0/604838404 >> 172.21.15.26:0/604838404 conn(0x7f51840b1c80 msgr2=0x7f51840b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.131+0000 7f518a763640 1 -- 172.21.15.26:0/604838404 shutdown_connections 2024-04-24T23:25:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.131+0000 7f518a763640 1 -- 172.21.15.26:0/604838404 wait complete. 2024-04-24T23:25:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.132+0000 7f518a763640 1 Processor -- start 2024-04-24T23:25:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.132+0000 7f518a763640 1 -- start start 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f518a763640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5170004800 0x7f5184159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f518a763640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f518414bdf0 0x7f51840b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f518a763640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51840b4200 0x7f51840b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f518a763640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f51840cfd10 con 0x7f51840b4200 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f518a763640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f51840cfa30 con 0x7f5170004800 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f518a763640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f51840cee80 con 0x7f518414bdf0 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5188d00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51840b4200 0x7f51840b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5188d00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51840b4200 0x7f51840b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58986/0 (socket says 172.21.15.26:58986) 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5188d00640 1 -- 172.21.15.26:0/101142809 learned_addr learned my addr 172.21.15.26:0/101142809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5183fff640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5170004800 0x7f5184159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f51837fe640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f518414bdf0 0x7f51840b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5188d00640 1 -- 172.21.15.26:0/101142809 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f518414bdf0 msgr2=0x7f51840b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5188d00640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f518414bdf0 0x7f51840b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5188d00640 1 -- 172.21.15.26:0/101142809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5170004800 msgr2=0x7f5184159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5188d00640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5170004800 0x7f5184159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5188d00640 1 -- 172.21.15.26:0/101142809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518409b760 con 0x7f51840b4200 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f51837fe640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f518414bdf0 0x7f51840b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5188d00640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51840b4200 0x7f51840b4680 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f51840c6520 tx=0x7f517c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5180ff9640 1 -- 172.21.15.26:0/101142809 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c01a030 con 0x7f51840b4200 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.133+0000 7f5180ff9640 1 -- 172.21.15.26:0/101142809 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c010da0 con 0x7f51840b4200 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.134+0000 7f5180ff9640 1 -- 172.21.15.26:0/101142809 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c003bd0 con 0x7f51840b4200 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.134+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518409ba40 con 0x7f51840b4200 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.134+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518414c270 con 0x7f51840b4200 2024-04-24T23:25:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.134+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5184067a40 con 0x7f51840b4200 2024-04-24T23:25:30.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.135+0000 7f5180ff9640 1 -- 172.21.15.26:0/101142809 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f517c01d070 con 0x7f51840b4200 2024-04-24T23:25:30.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.135+0000 7f5180ff9640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f516406be60 0x7f516406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:30.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.135+0000 7f5180ff9640 1 -- 172.21.15.26:0/101142809 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f517c08b850 con 0x7f51840b4200 2024-04-24T23:25:30.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.135+0000 7f5183fff640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f516406be60 0x7f516406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:30.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.136+0000 7f5183fff640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f516406be60 0x7f516406e320 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5174009a60 tx=0x7f5174009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:30.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.140+0000 7f5180ff9640 1 -- 172.21.15.26:0/101142809 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5184067a40 con 0x7f51840b4200 2024-04-24T23:25:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:30.359+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f518415dea0 con 0x7f51840b4200 2024-04-24T23:25:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.000+0000 7f5180ff9640 1 -- 172.21.15.26:0/101142809 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7f518415dea0 con 0x7f51840b4200 2024-04-24T23:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f516406be60 msgr2=0x7f516406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f516406be60 0x7f516406e320 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5174009a60 tx=0x7f5174009190 comp rx=0 tx=0).stop 2024-04-24T23:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51840b4200 msgr2=0x7f51840b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51840b4200 0x7f51840b4680 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f51840c6520 tx=0x7f517c010590 comp rx=0 tx=0).stop 2024-04-24T23:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 shutdown_connections 2024-04-24T23:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f516406be60 0x7f516406e320 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51840b4200 0x7f51840b4680 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f518414bdf0 0x7f51840b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 --2- 172.21.15.26:0/101142809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5170004800 0x7f5184159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 >> 172.21.15.26:0/101142809 conn(0x7f51840b1c80 msgr2=0x7f518406d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 shutdown_connections 2024-04-24T23:25:31.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.003+0000 7f518a763640 1 -- 172.21.15.26:0/101142809 wait complete. 2024-04-24T23:25:31.018 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:25:31.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:31.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.123+0000 7f8a4731b640 1 Processor -- start 2024-04-24T23:25:31.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.123+0000 7f8a4731b640 1 -- start start 2024-04-24T23:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.123+0000 7f8a4731b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a400b4530 0x7f8a400b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.123+0000 7f8a4731b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8a400b0030 0x7f8a400b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.123+0000 7f8a4731b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a400b5700 con 0x7f8a4015e380 2024-04-24T23:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.123+0000 7f8a4731b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8a4015e760 con 0x7f8a400b0030 2024-04-24T23:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.123+0000 7f8a4731b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8a400725b0 con 0x7f8a400b4530 2024-04-24T23:25:31.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a450b7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8a4015e380 0x7f8a40072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32858/0 (socket says 172.21.15.26:32858) 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a450b7640 1 -- 172.21.15.26:0/2591157505 learned_addr learned my addr 172.21.15.26:0/2591157505 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a458b8640 1 --2- 172.21.15.26:0/2591157505 >> v2:172.21.15.29:3300/0 conn(0x7f8a400b0030 0x7f8a400b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a448b6640 1 --2- 172.21.15.26:0/2591157505 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a400b4530 0x7f8a400b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a37fff640 1 -- 172.21.15.26:0/2591157505 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1618099216 0 0) 0x7f8a400b5700 con 0x7f8a4015e380 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a37fff640 1 -- 172.21.15.26:0/2591157505 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a2c003520 con 0x7f8a4015e380 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a448b6640 1 -- 172.21.15.26:0/2591157505 >> v2:172.21.15.29:3300/0 conn(0x7f8a400b0030 msgr2=0x7f8a400b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a448b6640 1 --2- 172.21.15.26:0/2591157505 >> v2:172.21.15.29:3300/0 conn(0x7f8a400b0030 0x7f8a400b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a448b6640 1 -- 172.21.15.26:0/2591157505 >> v1:172.21.15.26:6789/0 conn(0x7f8a4015e380 legacy=0x7f8a40072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a448b6640 1 -- 172.21.15.26:0/2591157505 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a400b59e0 con 0x7f8a400b4530 2024-04-24T23:25:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.124+0000 7f8a458b8640 1 --2- 172.21.15.26:0/2591157505 >> v2:172.21.15.29:3300/0 conn(0x7f8a400b0030 0x7f8a400b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.125+0000 7f8a448b6640 1 --2- 172.21.15.26:0/2591157505 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a400b4530 0x7f8a400b4fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a300097a0 tx=0x7f8a3002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68a49d2afc042303 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.125+0000 7f8a37fff640 1 -- 172.21.15.26:0/2591157505 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30037070 con 0x7f8a400b4530 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.125+0000 7f8a37fff640 1 -- 172.21.15.26:0/2591157505 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a30032440 con 0x7f8a400b4530 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.125+0000 7f8a37fff640 1 -- 172.21.15.26:0/2591157505 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300327f0 con 0x7f8a400b4530 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.126+0000 7f8a4731b640 1 -- 172.21.15.26:0/2591157505 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a400b4530 msgr2=0x7f8a400b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.126+0000 7f8a4731b640 1 --2- 172.21.15.26:0/2591157505 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a400b4530 0x7f8a400b4fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a300097a0 tx=0x7f8a3002a210 comp rx=0 tx=0).stop 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.126+0000 7f8a4731b640 1 -- 172.21.15.26:0/2591157505 shutdown_connections 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.126+0000 7f8a4731b640 1 --2- 172.21.15.26:0/2591157505 >> v2:172.21.15.29:3300/0 conn(0x7f8a400b0030 0x7f8a400b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.126+0000 7f8a4731b640 1 --2- 172.21.15.26:0/2591157505 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a400b4530 0x7f8a400b4fd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.126+0000 7f8a4731b640 1 -- 172.21.15.26:0/2591157505 >> 172.21.15.26:0/2591157505 conn(0x7f8a4006e2c0 msgr2=0x7f8a4006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.126+0000 7f8a4731b640 1 -- 172.21.15.26:0/2591157505 shutdown_connections 2024-04-24T23:25:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.126+0000 7f8a4731b640 1 -- 172.21.15.26:0/2591157505 wait complete. 2024-04-24T23:25:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.127+0000 7f8a4731b640 1 Processor -- start 2024-04-24T23:25:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.127+0000 7f8a4731b640 1 -- start start 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a4731b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a400b0030 0x7f8a400cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a4731b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a400b4530 0x7f8a40149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a4731b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a4015e380 0x7f8a400d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a4731b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400b09a0 con 0x7f8a400b4530 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a4731b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400b4970 con 0x7f8a400b0030 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a4731b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8a400ae530 con 0x7f8a4015e380 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a448b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a400b4530 0x7f8a40149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a448b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a400b4530 0x7f8a40149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58990/0 (socket says 172.21.15.26:58990) 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a448b6640 1 -- 172.21.15.26:0/1911199225 learned_addr learned my addr 172.21.15.26:0/1911199225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a458b8640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a4015e380 0x7f8a400d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a450b7640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a400b0030 0x7f8a400cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a448b6640 1 -- 172.21.15.26:0/1911199225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a4015e380 msgr2=0x7f8a400d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a448b6640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a4015e380 0x7f8a400d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a448b6640 1 -- 172.21.15.26:0/1911199225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a400b0030 msgr2=0x7f8a400cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a448b6640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a400b0030 0x7f8a400cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a448b6640 1 -- 172.21.15.26:0/1911199225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a400d0af0 con 0x7f8a400b4530 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a450b7640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a400b0030 0x7f8a400cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.128+0000 7f8a448b6640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a400b4530 0x7f8a40149ac0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8a3002a250 tx=0x7f8a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.129+0000 7f8a35ffb640 1 -- 172.21.15.26:0/1911199225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30037040 con 0x7f8a400b4530 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.129+0000 7f8a35ffb640 1 -- 172.21.15.26:0/1911199225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a30042070 con 0x7f8a400b4530 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.129+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a400b2110 con 0x7f8a400b4530 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.129+0000 7f8a35ffb640 1 -- 172.21.15.26:0/1911199225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3003b870 con 0x7f8a400b4530 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.129+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a400b26d0 con 0x7f8a400b4530 2024-04-24T23:25:31.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.130+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a400694c0 con 0x7f8a400b4530 2024-04-24T23:25:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.130+0000 7f8a35ffb640 1 -- 172.21.15.26:0/1911199225 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a3003ba10 con 0x7f8a400b4530 2024-04-24T23:25:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.131+0000 7f8a35ffb640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8a2406be30 0x7f8a2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.131+0000 7f8a35ffb640 1 -- 172.21.15.26:0/1911199225 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f8a300ab740 con 0x7f8a400b4530 2024-04-24T23:25:31.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.131+0000 7f8a450b7640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8a2406be30 0x7f8a2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.131+0000 7f8a450b7640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8a2406be30 0x7f8a2406e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c007fd0 tx=0x7f8a3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:31.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.136+0000 7f8a35ffb640 1 -- 172.21.15.26:0/1911199225 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a3007a0f0 con 0x7f8a400b4530 2024-04-24T23:25:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.357+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a4015c3c0 con 0x7f8a400b4530 2024-04-24T23:25:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.357+0000 7f8a35ffb640 1 -- 172.21.15.26:0/1911199225 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7f8a300799f0 con 0x7f8a400b4530 2024-04-24T23:25:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:31.359 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:25:30.362268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[7227]},"id":5}]} 2024-04-24T23:25:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 67 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8a2406be30 msgr2=0x7f8a2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8a2406be30 0x7f8a2406e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c007fd0 tx=0x7f8a3c009040 comp rx=0 tx=0).stop 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a400b4530 msgr2=0x7f8a40149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a400b4530 0x7f8a40149ac0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8a3002a250 tx=0x7f8a300046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 shutdown_connections 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8a2406be30 0x7f8a2406e2f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8a4015e380 0x7f8a400d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a400b4530 0x7f8a40149ac0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 --2- 172.21.15.26:0/1911199225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a400b0030 0x7f8a400cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 >> 172.21.15.26:0/1911199225 conn(0x7f8a4006e2c0 msgr2=0x7f8a4006b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:31.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.359+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 shutdown_connections 2024-04-24T23:25:31.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.360+0000 7f8a4731b640 1 -- 172.21.15.26:0/1911199225 wait complete. 2024-04-24T23:25:31.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:25:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.471+0000 7f154ef77640 1 Processor -- start 2024-04-24T23:25:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.471+0000 7f154ef77640 1 -- start start 2024-04-24T23:25:31.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.472+0000 7f154ef77640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1548143c40 0x7f15480cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.472+0000 7f154ef77640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f154815eb00 0x7f154815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.472+0000 7f154ef77640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154815f450 con 0x7f15480cc860 2024-04-24T23:25:31.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.472+0000 7f154ef77640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1548071b70 con 0x7f154815eb00 2024-04-24T23:25:31.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.472+0000 7f154ef77640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f15480cd0b0 con 0x7f1548143c40 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.472+0000 7f154d514640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f154815eb00 0x7f154815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.472+0000 7f154d514640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f154815eb00 0x7f154815ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59004/0 (socket says 172.21.15.26:59004) 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.472+0000 7f154d514640 1 -- 172.21.15.26:0/3400657855 learned_addr learned my addr 172.21.15.26:0/3400657855 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.472+0000 7f154cd13640 1 --2- 172.21.15.26:0/3400657855 >> v2:172.21.15.29:3300/0 conn(0x7f1548143c40 0x7f15480cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.473+0000 7f154d514640 1 -- 172.21.15.26:0/3400657855 >> v2:172.21.15.29:3300/0 conn(0x7f1548143c40 msgr2=0x7f15480cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.473+0000 7f154d514640 1 --2- 172.21.15.26:0/3400657855 >> v2:172.21.15.29:3300/0 conn(0x7f1548143c40 0x7f15480cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.473+0000 7f154d514640 1 -- 172.21.15.26:0/3400657855 >> v1:172.21.15.29:6789/0 conn(0x7f15480cc860 legacy=0x7f1548072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.473+0000 7f154d514640 1 -- 172.21.15.26:0/3400657855 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1548071240 con 0x7f154815eb00 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.473+0000 7f154cd13640 1 --2- 172.21.15.26:0/3400657855 >> v2:172.21.15.29:3300/0 conn(0x7f1548143c40 0x7f15480cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.473+0000 7f154d514640 1 --2- 172.21.15.26:0/3400657855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154815eb00 0x7f154815ef10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f154806ecf0 tx=0x7f154002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d1cd3d28ae44fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.473+0000 7f15477fe640 1 -- 172.21.15.26:0/3400657855 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540036070 con 0x7f154815eb00 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.473+0000 7f15477fe640 1 -- 172.21.15.26:0/3400657855 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1540031600 con 0x7f154815eb00 2024-04-24T23:25:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.473+0000 7f15477fe640 1 -- 172.21.15.26:0/3400657855 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15400319d0 con 0x7f154815eb00 2024-04-24T23:25:31.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.474+0000 7f154ef77640 1 -- 172.21.15.26:0/3400657855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154815eb00 msgr2=0x7f154815ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.474+0000 7f154ef77640 1 --2- 172.21.15.26:0/3400657855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154815eb00 0x7f154815ef10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f154806ecf0 tx=0x7f154002a730 comp rx=0 tx=0).stop 2024-04-24T23:25:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.474+0000 7f154ef77640 1 -- 172.21.15.26:0/3400657855 shutdown_connections 2024-04-24T23:25:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.474+0000 7f154ef77640 1 --2- 172.21.15.26:0/3400657855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154815eb00 0x7f154815ef10 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.474+0000 7f154ef77640 1 --2- 172.21.15.26:0/3400657855 >> v2:172.21.15.29:3300/0 conn(0x7f1548143c40 0x7f15480cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.474+0000 7f154ef77640 1 -- 172.21.15.26:0/3400657855 >> 172.21.15.26:0/3400657855 conn(0x7f154806b300 msgr2=0x7f154806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.474+0000 7f154ef77640 1 -- 172.21.15.26:0/3400657855 shutdown_connections 2024-04-24T23:25:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.474+0000 7f154ef77640 1 -- 172.21.15.26:0/3400657855 wait complete. 2024-04-24T23:25:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.475+0000 7f154ef77640 1 Processor -- start 2024-04-24T23:25:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.475+0000 7f154ef77640 1 -- start start 2024-04-24T23:25:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154ef77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480cc860 0x7f15481395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154ef77640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1548143c40 0x7f154813fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154ef77640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154815eb00 0x7f15481400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154ef77640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1548144020 con 0x7f15480cc860 2024-04-24T23:25:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154ef77640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1548071800 con 0x7f154815eb00 2024-04-24T23:25:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154ef77640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f15480cccd0 con 0x7f1548143c40 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154cd13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480cc860 0x7f15481395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154cd13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480cc860 0x7f15481395e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59020/0 (socket says 172.21.15.26:59020) 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154cd13640 1 -- 172.21.15.26:0/3273020125 learned_addr learned my addr 172.21.15.26:0/3273020125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f1547fff640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1548143c40 0x7f154813fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.476+0000 7f154d514640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154815eb00 0x7f15481400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f154cd13640 1 -- 172.21.15.26:0/3273020125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1548143c40 msgr2=0x7f154813fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f154cd13640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1548143c40 0x7f154813fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f154cd13640 1 -- 172.21.15.26:0/3273020125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154815eb00 msgr2=0x7f15481400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f154cd13640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154815eb00 0x7f15481400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f154cd13640 1 -- 172.21.15.26:0/3273020125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15480b04b0 con 0x7f15480cc860 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f154d514640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154815eb00 0x7f15481400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f154cd13640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480cc860 0x7f15481395e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e810 tx=0x7f153c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f15457fa640 1 -- 172.21.15.26:0/3273020125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c00f040 con 0x7f15480cc860 2024-04-24T23:25:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f15457fa640 1 -- 172.21.15.26:0/3273020125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153c004030 con 0x7f15480cc860 2024-04-24T23:25:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f15457fa640 1 -- 172.21.15.26:0/3273020125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c004400 con 0x7f15480cc860 2024-04-24T23:25:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15480b0730 con 0x7f15480cc860 2024-04-24T23:25:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.477+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15480b0cf0 con 0x7f15480cc860 2024-04-24T23:25:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.478+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15480694c0 con 0x7f15480cc860 2024-04-24T23:25:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.479+0000 7f15457fa640 1 -- 172.21.15.26:0/3273020125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f153c00c6d0 con 0x7f15480cc860 2024-04-24T23:25:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.479+0000 7f15457fa640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f152c06be30 0x7f152c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.479+0000 7f15457fa640 1 -- 172.21.15.26:0/3273020125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f153c08aba0 con 0x7f15480cc860 2024-04-24T23:25:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.479+0000 7f1547fff640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f152c06be30 0x7f152c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.480+0000 7f1547fff640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f152c06be30 0x7f152c06e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f154813ef40 tx=0x7f1534009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.483+0000 7f15457fa640 1 -- 172.21.15.26:0/3273020125 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15480b04b0 con 0x7f15480cc860 2024-04-24T23:25:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.666+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15480b13a0 con 0x7f15480cc860 2024-04-24T23:25:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.667+0000 7f15457fa640 1 -- 172.21.15.26:0/3273020125 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16608 (secure 0 0 0) 0x7f153c017020 con 0x7f15480cc860 2024-04-24T23:25:31.668 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:31.668 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":69,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:25:28.989129+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:25:25.626812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:25:26.319330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:25:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f152c06be30 msgr2=0x7f152c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f152c06be30 0x7f152c06e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f154813ef40 tx=0x7f1534009040 comp rx=0 tx=0).stop 2024-04-24T23:25:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480cc860 msgr2=0x7f15481395e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480cc860 0x7f15481395e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e810 tx=0x7f153c00ecd0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 shutdown_connections 2024-04-24T23:25:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f152c06be30 0x7f152c06e2f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f154815eb00 0x7f15481400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1548143c40 0x7f154813fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 --2- 172.21.15.26:0/3273020125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480cc860 0x7f15481395e0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 >> 172.21.15.26:0/3273020125 conn(0x7f154806b300 msgr2=0x7f15480b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 shutdown_connections 2024-04-24T23:25:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.669+0000 7f154ef77640 1 -- 172.21.15.26:0/3273020125 wait complete. 2024-04-24T23:25:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:25:31.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:25:31.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.781+0000 7fe9828b3640 1 Processor -- start 2024-04-24T23:25:31.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.781+0000 7fe9828b3640 1 -- start start 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.781+0000 7fe9828b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe97c15e570 0x7fe97c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.781+0000 7fe9828b3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe97c0d22e0 0x7fe97c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.781+0000 7fe9828b3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe97c0d32f0 con 0x7fe97c0b3130 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.781+0000 7fe9828b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe97c0d1ab0 con 0x7fe97c15e570 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.781+0000 7fe9828b3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe97c0d2ae0 con 0x7fe97c0d22e0 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.782+0000 7fe97bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe97c15e570 0x7fe97c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.782+0000 7fe97bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe97c15e570 0x7fe97c0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59032/0 (socket says 172.21.15.26:59032) 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.782+0000 7fe97bfff640 1 -- 172.21.15.26:0/2324230059 learned_addr learned my addr 172.21.15.26:0/2324230059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.782+0000 7fe980e50640 1 --2- 172.21.15.26:0/2324230059 >> v2:172.21.15.29:3300/0 conn(0x7fe97c0d22e0 0x7fe97c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.782+0000 7fe97bfff640 1 -- 172.21.15.26:0/2324230059 >> v2:172.21.15.29:3300/0 conn(0x7fe97c0d22e0 msgr2=0x7fe97c0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.782+0000 7fe97bfff640 1 --2- 172.21.15.26:0/2324230059 >> v2:172.21.15.29:3300/0 conn(0x7fe97c0d22e0 0x7fe97c0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.782+0000 7fe97bfff640 1 -- 172.21.15.26:0/2324230059 >> v1:172.21.15.29:6789/0 conn(0x7fe97c0b3130 legacy=0x7fe97c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.782+0000 7fe97bfff640 1 -- 172.21.15.26:0/2324230059 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c0d0270 con 0x7fe97c15e570 2024-04-24T23:25:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.782+0000 7fe980e50640 1 --2- 172.21.15.26:0/2324230059 >> v2:172.21.15.29:3300/0 conn(0x7fe97c0d22e0 0x7fe97c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe97bfff640 1 --2- 172.21.15.26:0/2324230059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 0x7fe97c0d4160 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0b6d60 tx=0x7fe96c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a6bbaa403e094 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe97affd640 1 -- 172.21.15.26:0/2324230059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c037070 con 0x7fe97c15e570 2024-04-24T23:25:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe97affd640 1 -- 172.21.15.26:0/2324230059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96c032440 con 0x7fe97c15e570 2024-04-24T23:25:31.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe97affd640 1 -- 172.21.15.26:0/2324230059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c032810 con 0x7fe97c15e570 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe9828b3640 1 -- 172.21.15.26:0/2324230059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 msgr2=0x7fe97c0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe9828b3640 1 --2- 172.21.15.26:0/2324230059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 0x7fe97c0d4160 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0b6d60 tx=0x7fe96c02a230 comp rx=0 tx=0).stop 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe9828b3640 1 -- 172.21.15.26:0/2324230059 shutdown_connections 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe9828b3640 1 --2- 172.21.15.26:0/2324230059 >> v2:172.21.15.29:3300/0 conn(0x7fe97c0d22e0 0x7fe97c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe9828b3640 1 --2- 172.21.15.26:0/2324230059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 0x7fe97c0d4160 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe9828b3640 1 -- 172.21.15.26:0/2324230059 >> 172.21.15.26:0/2324230059 conn(0x7fe97c08dbe0 msgr2=0x7fe97c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe9828b3640 1 -- 172.21.15.26:0/2324230059 shutdown_connections 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.783+0000 7fe9828b3640 1 -- 172.21.15.26:0/2324230059 wait complete. 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.785+0000 7fe9828b3640 1 Processor -- start 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.785+0000 7fe9828b3640 1 -- start start 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.785+0000 7fe9828b3640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97c0b3130 0x7fe97c09e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.785+0000 7fe9828b3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe97c0d22e0 0x7fe97c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.785+0000 7fe9828b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 0x7fe97c1f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.785+0000 7fe9828b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c0cfa70 con 0x7fe97c15e570 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.785+0000 7fe9828b3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c0d3010 con 0x7fe97c0d22e0 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.785+0000 7fe9828b3640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe97c0cfd50 con 0x7fe97c0b3130 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe980e50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 0x7fe97c1f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe980e50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 0x7fe97c1f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59036/0 (socket says 172.21.15.26:59036) 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe980e50640 1 -- 172.21.15.26:0/1682152862 learned_addr learned my addr 172.21.15.26:0/1682152862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe97bfff640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97c0b3130 0x7fe97c09e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe97b7fe640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe97c0d22e0 0x7fe97c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe980e50640 1 -- 172.21.15.26:0/1682152862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97c0b3130 msgr2=0x7fe97c09e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe980e50640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97c0b3130 0x7fe97c09e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe980e50640 1 -- 172.21.15.26:0/1682152862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe97c0d22e0 msgr2=0x7fe97c155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe980e50640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe97c0d22e0 0x7fe97c155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe980e50640 1 -- 172.21.15.26:0/1682152862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c1f8010 con 0x7fe97c15e570 2024-04-24T23:25:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe980e50640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 0x7fe97c1f7ad0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe97400cef0 tx=0x7fe97400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe978ff9640 1 -- 172.21.15.26:0/1682152862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97401a070 con 0x7fe97c15e570 2024-04-24T23:25:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe978ff9640 1 -- 172.21.15.26:0/1682152862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97400d860 con 0x7fe97c15e570 2024-04-24T23:25:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c1f82b0 con 0x7fe97c15e570 2024-04-24T23:25:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe978ff9640 1 -- 172.21.15.26:0/1682152862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97400dc50 con 0x7fe97c15e570 2024-04-24T23:25:31.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.786+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c1f8870 con 0x7fe97c15e570 2024-04-24T23:25:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.787+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97c0694c0 con 0x7fe97c15e570 2024-04-24T23:25:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.788+0000 7fe978ff9640 1 -- 172.21.15.26:0/1682152862 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe97401d070 con 0x7fe97c15e570 2024-04-24T23:25:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.788+0000 7fe978ff9640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe96806be30 0x7fe96806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.788+0000 7fe978ff9640 1 -- 172.21.15.26:0/1682152862 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fe97408b600 con 0x7fe97c15e570 2024-04-24T23:25:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.788+0000 7fe97bfff640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe96806be30 0x7fe96806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:31.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.789+0000 7fe97bfff640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe96806be30 0x7fe96806e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe96c034000 tx=0x7fe96c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:31.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.792+0000 7fe978ff9640 1 -- 172.21.15.26:0/1682152862 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe974059ab0 con 0x7fe97c15e570 2024-04-24T23:25:31.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.970+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe97c15dd70 con 0x7fe96806be30 2024-04-24T23:25:31.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.976+0000 7fe978ff9640 1 -- 172.21.15.26:0/1682152862 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246752 (secure 0 0 0) 0x7fe97c15dd70 con 0x7fe96806be30 2024-04-24T23:25:31.980 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:31.981 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:25:31.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe96806be30 msgr2=0x7fe96806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe96806be30 0x7fe96806e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe96c034000 tx=0x7fe96c02e000 comp rx=0 tx=0).stop 2024-04-24T23:25:31.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 msgr2=0x7fe97c1f7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:31.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 0x7fe97c1f7ad0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe97400cef0 tx=0x7fe97400a710 comp rx=0 tx=0).stop 2024-04-24T23:25:31.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 shutdown_connections 2024-04-24T23:25:31.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe96806be30 0x7fe96806e2f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe97c15e570 0x7fe97c1f7ad0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe97c0d22e0 0x7fe97c155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 --2- 172.21.15.26:0/1682152862 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97c0b3130 0x7fe97c09e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:31.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 >> 172.21.15.26:0/1682152862 conn(0x7fe97c08dbe0 msgr2=0x7fe97c0b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:31.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 shutdown_connections 2024-04-24T23:25:31.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:31.979+0000 7fe9828b3640 1 -- 172.21.15.26:0/1682152862 wait complete. 2024-04-24T23:25:31.991 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-04-24T23:25:31.376080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265256,"kb_used_data":7480,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754709464,"statfs":{"total":773094113280,"available":772822491136,"internally_reserved":0,"allocated":7659520,"data_stored":3908484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.413779"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972124+0000","last_change":"2024-04-24T23:25:26.972124+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972124+0000","last_undegraded":"2024-04-24T23:25:26.972124+0000","last_fullsized":"2024-04-24T23:25:26.972124+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973940+0000","last_change":"2024-04-24T23:25:26.973940+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973940+0000","last_undegraded":"2024-04-24T23:25:26.973940+0000","last_fullsized":"2024-04-24T23:25:26.973940+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971708+0000","last_change":"2024-04-24T23:25:26.971708+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971708+0000","last_undegraded":"2024-04-24T23:25:26.971708+0000","last_fullsized":"2024-04-24T23:25:26.971708+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.977999+0000","last_change":"2024-04-24T23:25:26.977999+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.977999+0000","last_undegraded":"2024-04-24T23:25:26.977999+0000","last_fullsized":"2024-04-24T23:25:26.977999+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974555+0000","last_change":"2024-04-24T23:25:26.974555+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974555+0000","last_undegraded":"2024-04-24T23:25:26.974555+0000","last_fullsized":"2024-04-24T23:25:26.974555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971249+0000","last_change":"2024-04-24T23:25:26.971249+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971249+0000","last_undegraded":"2024-04-24T23:25:26.971249+0000","last_fullsized":"2024-04-24T23:25:26.971249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968229+0000","last_change":"2024-04-24T23:25:26.968229+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968229+0000","last_undegraded":"2024-04-24T23:25:26.968229+0000","last_fullsized":"2024-04-24T23:25:26.968229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.976683+0000","last_change":"2024-04-24T23:25:26.976683+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.976683+0000","last_undegraded":"2024-04-24T23:25:26.976683+0000","last_fullsized":"2024-04-24T23:25:26.976683+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973701+0000","last_change":"2024-04-24T23:25:26.973701+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973701+0000","last_undegraded":"2024-04-24T23:25:26.973701+0000","last_fullsized":"2024-04-24T23:25:26.973701+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975677+0000","last_change":"2024-04-24T23:25:26.975677+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975677+0000","last_undegraded":"2024-04-24T23:25:26.975677+0000","last_fullsized":"2024-04-24T23:25:26.975677+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972646+0000","last_change":"2024-04-24T23:25:26.972646+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972646+0000","last_undegraded":"2024-04-24T23:25:26.972646+0000","last_fullsized":"2024-04-24T23:25:26.972646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.977553+0000","last_change":"2024-04-24T23:25:26.977553+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.977553+0000","last_undegraded":"2024-04-24T23:25:26.977553+0000","last_fullsized":"2024-04-24T23:25:26.977553+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972318+0000","last_change":"2024-04-24T23:25:26.972318+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972318+0000","last_undegraded":"2024-04-24T23:25:26.972318+0000","last_fullsized":"2024-04-24T23:25:26.972318+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973045+0000","last_change":"2024-04-24T23:25:26.973045+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973045+0000","last_undegraded":"2024-04-24T23:25:26.973045+0000","last_fullsized":"2024-04-24T23:25:26.973045+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973018+0000","last_change":"2024-04-24T23:25:26.973018+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973018+0000","last_undegraded":"2024-04-24T23:25:26.973018+0000","last_fullsized":"2024-04-24T23:25:26.973018+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975110+0000","last_change":"2024-04-24T23:25:26.975110+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975110+0000","last_undegraded":"2024-04-24T23:25:26.975110+0000","last_fullsized":"2024-04-24T23:25:26.975110+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973033+0000","last_change":"2024-04-24T23:25:26.973033+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973033+0000","last_undegraded":"2024-04-24T23:25:26.973033+0000","last_fullsized":"2024-04-24T23:25:26.973033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975740+0000","last_change":"2024-04-24T23:25:26.975740+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975740+0000","last_undegraded":"2024-04-24T23:25:26.975740+0000","last_fullsized":"2024-04-24T23:25:26.975740+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972531+0000","last_change":"2024-04-24T23:25:26.972531+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972531+0000","last_undegraded":"2024-04-24T23:25:26.972531+0000","last_fullsized":"2024-04-24T23:25:26.972531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972789+0000","last_change":"2024-04-24T23:25:26.972789+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972789+0000","last_undegraded":"2024-04-24T23:25:26.972789+0000","last_fullsized":"2024-04-24T23:25:26.972789+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975257+0000","last_change":"2024-04-24T23:25:26.975257+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975257+0000","last_undegraded":"2024-04-24T23:25:26.975257+0000","last_fullsized":"2024-04-24T23:25:26.975257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974439+0000","last_change":"2024-04-24T23:25:26.974439+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974439+0000","last_undegraded":"2024-04-24T23:25:26.974439+0000","last_fullsized":"2024-04-24T23:25:26.974439+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970819+0000","last_change":"2024-04-24T23:25:26.970819+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970819+0000","last_undegraded":"2024-04-24T23:25:26.970819+0000","last_fullsized":"2024-04-24T23:25:26.970819+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971038+0000","last_change":"2024-04-24T23:25:26.971038+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971038+0000","last_undegraded":"2024-04-24T23:25:26.971038+0000","last_fullsized":"2024-04-24T23:25:26.971038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973311+0000","last_change":"2024-04-24T23:25:26.973311+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973311+0000","last_undegraded":"2024-04-24T23:25:26.973311+0000","last_fullsized":"2024-04-24T23:25:26.973311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968913+0000","last_change":"2024-04-24T23:25:26.968913+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968913+0000","last_undegraded":"2024-04-24T23:25:26.968913+0000","last_fullsized":"2024-04-24T23:25:26.968913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971174+0000","last_change":"2024-04-24T23:25:26.971174+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971174+0000","last_undegraded":"2024-04-24T23:25:26.971174+0000","last_fullsized":"2024-04-24T23:25:26.971174+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972444+0000","last_change":"2024-04-24T23:25:26.972444+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972444+0000","last_undegraded":"2024-04-24T23:25:26.972444+0000","last_fullsized":"2024-04-24T23:25:26.972444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971143+0000","last_change":"2024-04-24T23:25:26.971143+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971143+0000","last_undegraded":"2024-04-24T23:25:26.971143+0000","last_fullsized":"2024-04-24T23:25:26.971143+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967626+0000","last_change":"2024-04-24T23:25:26.967626+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967626+0000","last_undegraded":"2024-04-24T23:25:26.967626+0000","last_fullsized":"2024-04-24T23:25:26.967626+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967495+0000","last_change":"2024-04-24T23:25:26.967495+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967495+0000","last_undegraded":"2024-04-24T23:25:26.967495+0000","last_fullsized":"2024-04-24T23:25:26.967495+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.976210+0000","last_change":"2024-04-24T23:25:26.976210+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.976210+0000","last_undegraded":"2024-04-24T23:25:26.976210+0000","last_fullsized":"2024-04-24T23:25:26.976210+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971298+0000","last_change":"2024-04-24T23:25:26.971298+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971298+0000","last_undegraded":"2024-04-24T23:25:26.971298+0000","last_fullsized":"2024-04-24T23:25:26.971298+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985622+0000","last_change":"2024-04-24T23:25:26.984830+0000","last_active":"2024-04-24T23:25:26.985622+0000","last_peered":"2024-04-24T23:25:26.985622+0000","last_clean":"2024-04-24T23:25:26.985622+0000","last_became_active":"2024-04-24T23:25:26.983743+0000","last_became_peered":"2024-04-24T23:25:26.983743+0000","last_unstale":"2024-04-24T23:25:26.985622+0000","last_undegraded":"2024-04-24T23:25:26.985622+0000","last_fullsized":"2024-04-24T23:25:26.985622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:01.501491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972060+0000","last_change":"2024-04-24T23:25:26.972060+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972060+0000","last_undegraded":"2024-04-24T23:25:26.972060+0000","last_fullsized":"2024-04-24T23:25:26.972060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.988350+0000","last_change":"2024-04-24T23:25:26.986600+0000","last_active":"2024-04-24T23:25:26.988350+0000","last_peered":"2024-04-24T23:25:26.988350+0000","last_clean":"2024-04-24T23:25:26.988350+0000","last_became_active":"2024-04-24T23:25:26.986208+0000","last_became_peered":"2024-04-24T23:25:26.986208+0000","last_unstale":"2024-04-24T23:25:26.988350+0000","last_undegraded":"2024-04-24T23:25:26.988350+0000","last_fullsized":"2024-04-24T23:25:26.988350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:06.044715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974992+0000","last_change":"2024-04-24T23:25:26.974992+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974992+0000","last_undegraded":"2024-04-24T23:25:26.974992+0000","last_fullsized":"2024-04-24T23:25:26.974992+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.989266+0000","last_change":"2024-04-24T23:25:26.988051+0000","last_active":"2024-04-24T23:25:26.989266+0000","last_peered":"2024-04-24T23:25:26.989266+0000","last_clean":"2024-04-24T23:25:26.989266+0000","last_became_active":"2024-04-24T23:25:26.987663+0000","last_became_peered":"2024-04-24T23:25:26.987663+0000","last_unstale":"2024-04-24T23:25:26.989266+0000","last_undegraded":"2024-04-24T23:25:26.989266+0000","last_fullsized":"2024-04-24T23:25:26.989266+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:11.588401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972006+0000","last_change":"2024-04-24T23:25:26.972006+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972006+0000","last_undegraded":"2024-04-24T23:25:26.972006+0000","last_fullsized":"2024-04-24T23:25:26.972006+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987433+0000","last_change":"2024-04-24T23:25:26.986624+0000","last_active":"2024-04-24T23:25:26.987433+0000","last_peered":"2024-04-24T23:25:26.987433+0000","last_clean":"2024-04-24T23:25:26.987433+0000","last_became_active":"2024-04-24T23:25:26.986323+0000","last_became_peered":"2024-04-24T23:25:26.986323+0000","last_unstale":"2024-04-24T23:25:26.987433+0000","last_undegraded":"2024-04-24T23:25:26.987433+0000","last_fullsized":"2024-04-24T23:25:26.987433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:11.677239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967001+0000","last_change":"2024-04-24T23:25:26.967001+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967001+0000","last_undegraded":"2024-04-24T23:25:26.967001+0000","last_fullsized":"2024-04-24T23:25:26.967001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.978242+0000","last_change":"2024-04-24T23:25:26.977218+0000","last_active":"2024-04-24T23:25:26.978242+0000","last_peered":"2024-04-24T23:25:26.978242+0000","last_clean":"2024-04-24T23:25:26.978242+0000","last_became_active":"2024-04-24T23:25:26.976913+0000","last_became_peered":"2024-04-24T23:25:26.976913+0000","last_unstale":"2024-04-24T23:25:26.978242+0000","last_undegraded":"2024-04-24T23:25:26.978242+0000","last_fullsized":"2024-04-24T23:25:26.978242+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:15.039445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972886+0000","last_change":"2024-04-24T23:25:26.972886+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972886+0000","last_undegraded":"2024-04-24T23:25:26.972886+0000","last_fullsized":"2024-04-24T23:25:26.972886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986090+0000","last_change":"2024-04-24T23:25:26.984947+0000","last_active":"2024-04-24T23:25:26.986090+0000","last_peered":"2024-04-24T23:25:26.986090+0000","last_clean":"2024-04-24T23:25:26.986090+0000","last_became_active":"2024-04-24T23:25:26.984604+0000","last_became_peered":"2024-04-24T23:25:26.984604+0000","last_unstale":"2024-04-24T23:25:26.986090+0000","last_undegraded":"2024-04-24T23:25:26.986090+0000","last_fullsized":"2024-04-24T23:25:26.986090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:27.118553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972021+0000","last_change":"2024-04-24T23:25:26.972021+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972021+0000","last_undegraded":"2024-04-24T23:25:26.972021+0000","last_fullsized":"2024-04-24T23:25:26.972021+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985711+0000","last_change":"2024-04-24T23:25:26.982771+0000","last_active":"2024-04-24T23:25:26.985711+0000","last_peered":"2024-04-24T23:25:26.985711+0000","last_clean":"2024-04-24T23:25:26.985711+0000","last_became_active":"2024-04-24T23:25:26.982504+0000","last_became_peered":"2024-04-24T23:25:26.982504+0000","last_unstale":"2024-04-24T23:25:26.985711+0000","last_undegraded":"2024-04-24T23:25:26.985711+0000","last_fullsized":"2024-04-24T23:25:26.985711+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:10.968125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972021+0000","last_change":"2024-04-24T23:25:26.972021+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972021+0000","last_undegraded":"2024-04-24T23:25:26.972021+0000","last_fullsized":"2024-04-24T23:25:26.972021+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987666+0000","last_change":"2024-04-24T23:25:26.985502+0000","last_active":"2024-04-24T23:25:26.987666+0000","last_peered":"2024-04-24T23:25:26.987666+0000","last_clean":"2024-04-24T23:25:26.987666+0000","last_became_active":"2024-04-24T23:25:26.985236+0000","last_became_peered":"2024-04-24T23:25:26.985236+0000","last_unstale":"2024-04-24T23:25:26.987666+0000","last_undegraded":"2024-04-24T23:25:26.987666+0000","last_fullsized":"2024-04-24T23:25:26.987666+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:42.270581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968067+0000","last_change":"2024-04-24T23:25:26.968067+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968067+0000","last_undegraded":"2024-04-24T23:25:26.968067+0000","last_fullsized":"2024-04-24T23:25:26.968067+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.978235+0000","last_change":"2024-04-24T23:25:26.976690+0000","last_active":"2024-04-24T23:25:26.978235+0000","last_peered":"2024-04-24T23:25:26.978235+0000","last_clean":"2024-04-24T23:25:26.978235+0000","last_became_active":"2024-04-24T23:25:26.976221+0000","last_became_peered":"2024-04-24T23:25:26.976221+0000","last_unstale":"2024-04-24T23:25:26.978235+0000","last_undegraded":"2024-04-24T23:25:26.978235+0000","last_fullsized":"2024-04-24T23:25:26.978235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:26.544193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971698+0000","last_change":"2024-04-24T23:25:26.971698+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971698+0000","last_undegraded":"2024-04-24T23:25:26.971698+0000","last_fullsized":"2024-04-24T23:25:26.971698+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.984328+0000","last_change":"2024-04-24T23:25:26.979704+0000","last_active":"2024-04-24T23:25:26.984328+0000","last_peered":"2024-04-24T23:25:26.984328+0000","last_clean":"2024-04-24T23:25:26.984328+0000","last_became_active":"2024-04-24T23:25:26.979294+0000","last_became_peered":"2024-04-24T23:25:26.979294+0000","last_unstale":"2024-04-24T23:25:26.984328+0000","last_undegraded":"2024-04-24T23:25:26.984328+0000","last_fullsized":"2024-04-24T23:25:26.984328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:59.557263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974519+0000","last_change":"2024-04-24T23:25:26.974519+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974519+0000","last_undegraded":"2024-04-24T23:25:26.974519+0000","last_fullsized":"2024-04-24T23:25:26.974519+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.974980+0000","last_change":"2024-04-24T23:25:26.973701+0000","last_active":"2024-04-24T23:25:26.974980+0000","last_peered":"2024-04-24T23:25:26.974980+0000","last_clean":"2024-04-24T23:25:26.974980+0000","last_became_active":"2024-04-24T23:25:26.973424+0000","last_became_peered":"2024-04-24T23:25:26.973424+0000","last_unstale":"2024-04-24T23:25:26.974980+0000","last_undegraded":"2024-04-24T23:25:26.974980+0000","last_fullsized":"2024-04-24T23:25:26.974980+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:54.920525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972192+0000","last_change":"2024-04-24T23:25:26.972192+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972192+0000","last_undegraded":"2024-04-24T23:25:26.972192+0000","last_fullsized":"2024-04-24T23:25:26.972192+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987465+0000","last_change":"2024-04-24T23:25:26.985049+0000","last_active":"2024-04-24T23:25:26.987465+0000","last_peered":"2024-04-24T23:25:26.987465+0000","last_clean":"2024-04-24T23:25:26.987465+0000","last_became_active":"2024-04-24T23:25:26.984293+0000","last_became_peered":"2024-04-24T23:25:26.984293+0000","last_unstale":"2024-04-24T23:25:26.987465+0000","last_undegraded":"2024-04-24T23:25:26.987465+0000","last_fullsized":"2024-04-24T23:25:26.987465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:12.258864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970929+0000","last_change":"2024-04-24T23:25:26.970929+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970929+0000","last_undegraded":"2024-04-24T23:25:26.970929+0000","last_fullsized":"2024-04-24T23:25:26.970929+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986717+0000","last_change":"2024-04-24T23:25:26.982431+0000","last_active":"2024-04-24T23:25:26.986717+0000","last_peered":"2024-04-24T23:25:26.986717+0000","last_clean":"2024-04-24T23:25:26.986717+0000","last_became_active":"2024-04-24T23:25:26.982147+0000","last_became_peered":"2024-04-24T23:25:26.982147+0000","last_unstale":"2024-04-24T23:25:26.986717+0000","last_undegraded":"2024-04-24T23:25:26.986717+0000","last_fullsized":"2024-04-24T23:25:26.986717+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:59.723643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967531+0000","last_change":"2024-04-24T23:25:26.967531+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967531+0000","last_undegraded":"2024-04-24T23:25:26.967531+0000","last_fullsized":"2024-04-24T23:25:26.967531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985013+0000","last_change":"2024-04-24T23:25:26.979601+0000","last_active":"2024-04-24T23:25:26.985013+0000","last_peered":"2024-04-24T23:25:26.985013+0000","last_clean":"2024-04-24T23:25:26.985013+0000","last_became_active":"2024-04-24T23:25:26.978063+0000","last_became_peered":"2024-04-24T23:25:26.978063+0000","last_unstale":"2024-04-24T23:25:26.985013+0000","last_undegraded":"2024-04-24T23:25:26.985013+0000","last_fullsized":"2024-04-24T23:25:26.985013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:32.655714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972297+0000","last_change":"2024-04-24T23:25:26.972297+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972297+0000","last_undegraded":"2024-04-24T23:25:26.972297+0000","last_fullsized":"2024-04-24T23:25:26.972297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.981043+0000","last_change":"2024-04-24T23:25:26.979978+0000","last_active":"2024-04-24T23:25:26.981043+0000","last_peered":"2024-04-24T23:25:26.981043+0000","last_clean":"2024-04-24T23:25:26.981043+0000","last_became_active":"2024-04-24T23:25:26.978846+0000","last_became_peered":"2024-04-24T23:25:26.978846+0000","last_unstale":"2024-04-24T23:25:26.981043+0000","last_undegraded":"2024-04-24T23:25:26.981043+0000","last_fullsized":"2024-04-24T23:25:26.981043+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:06.044715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971257+0000","last_change":"2024-04-24T23:25:26.971257+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971257+0000","last_undegraded":"2024-04-24T23:25:26.971257+0000","last_fullsized":"2024-04-24T23:25:26.971257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985464+0000","last_change":"2024-04-24T23:25:26.983021+0000","last_active":"2024-04-24T23:25:26.985464+0000","last_peered":"2024-04-24T23:25:26.985464+0000","last_clean":"2024-04-24T23:25:26.985464+0000","last_became_active":"2024-04-24T23:25:26.982757+0000","last_became_peered":"2024-04-24T23:25:26.982757+0000","last_unstale":"2024-04-24T23:25:26.985464+0000","last_undegraded":"2024-04-24T23:25:26.985464+0000","last_fullsized":"2024-04-24T23:25:26.985464+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:28.905967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972173+0000","last_change":"2024-04-24T23:25:26.972173+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972173+0000","last_undegraded":"2024-04-24T23:25:26.972173+0000","last_fullsized":"2024-04-24T23:25:26.972173+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985354+0000","last_change":"2024-04-24T23:25:26.984211+0000","last_active":"2024-04-24T23:25:26.985354+0000","last_peered":"2024-04-24T23:25:26.985354+0000","last_clean":"2024-04-24T23:25:26.985354+0000","last_became_active":"2024-04-24T23:25:26.983878+0000","last_became_peered":"2024-04-24T23:25:26.983878+0000","last_unstale":"2024-04-24T23:25:26.985354+0000","last_undegraded":"2024-04-24T23:25:26.985354+0000","last_fullsized":"2024-04-24T23:25:26.985354+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:39.951844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971011+0000","last_change":"2024-04-24T23:25:26.971011+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971011+0000","last_undegraded":"2024-04-24T23:25:26.971011+0000","last_fullsized":"2024-04-24T23:25:26.971011+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.989233+0000","last_change":"2024-04-24T23:25:26.988180+0000","last_active":"2024-04-24T23:25:26.989233+0000","last_peered":"2024-04-24T23:25:26.989233+0000","last_clean":"2024-04-24T23:25:26.989233+0000","last_became_active":"2024-04-24T23:25:26.987706+0000","last_became_peered":"2024-04-24T23:25:26.987706+0000","last_unstale":"2024-04-24T23:25:26.989233+0000","last_undegraded":"2024-04-24T23:25:26.989233+0000","last_fullsized":"2024-04-24T23:25:26.989233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:04.387670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.977087+0000","last_change":"2024-04-24T23:25:26.977087+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.977087+0000","last_undegraded":"2024-04-24T23:25:26.977087+0000","last_fullsized":"2024-04-24T23:25:26.977087+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.990338+0000","last_change":"2024-04-24T23:25:26.989891+0000","last_active":"2024-04-24T23:25:26.990338+0000","last_peered":"2024-04-24T23:25:26.990338+0000","last_clean":"2024-04-24T23:25:26.990338+0000","last_became_active":"2024-04-24T23:25:26.989524+0000","last_became_peered":"2024-04-24T23:25:26.989524+0000","last_unstale":"2024-04-24T23:25:26.990338+0000","last_undegraded":"2024-04-24T23:25:26.990338+0000","last_fullsized":"2024-04-24T23:25:26.990338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:44.608576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971175+0000","last_change":"2024-04-24T23:25:26.971175+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971175+0000","last_undegraded":"2024-04-24T23:25:26.971175+0000","last_fullsized":"2024-04-24T23:25:26.971175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.982172+0000","last_change":"2024-04-24T23:25:26.980697+0000","last_active":"2024-04-24T23:25:26.982172+0000","last_peered":"2024-04-24T23:25:26.982172+0000","last_clean":"2024-04-24T23:25:26.982172+0000","last_became_active":"2024-04-24T23:25:26.980315+0000","last_became_peered":"2024-04-24T23:25:26.980315+0000","last_unstale":"2024-04-24T23:25:26.982172+0000","last_undegraded":"2024-04-24T23:25:26.982172+0000","last_fullsized":"2024-04-24T23:25:26.982172+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:43.615264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970605+0000","last_change":"2024-04-24T23:25:26.970605+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970605+0000","last_undegraded":"2024-04-24T23:25:26.970605+0000","last_fullsized":"2024-04-24T23:25:26.970605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986068+0000","last_change":"2024-04-24T23:25:26.984839+0000","last_active":"2024-04-24T23:25:26.986068+0000","last_peered":"2024-04-24T23:25:26.986068+0000","last_clean":"2024-04-24T23:25:26.986068+0000","last_became_active":"2024-04-24T23:25:26.984489+0000","last_became_peered":"2024-04-24T23:25:26.984489+0000","last_unstale":"2024-04-24T23:25:26.986068+0000","last_undegraded":"2024-04-24T23:25:26.986068+0000","last_fullsized":"2024-04-24T23:25:26.986068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:50.577760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.969773+0000","last_change":"2024-04-24T23:25:26.969773+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.969773+0000","last_undegraded":"2024-04-24T23:25:26.969773+0000","last_fullsized":"2024-04-24T23:25:26.969773+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.976875+0000","last_change":"2024-04-24T23:25:26.975028+0000","last_active":"2024-04-24T23:25:26.976875+0000","last_peered":"2024-04-24T23:25:26.976875+0000","last_clean":"2024-04-24T23:25:26.976875+0000","last_became_active":"2024-04-24T23:25:26.974741+0000","last_became_peered":"2024-04-24T23:25:26.974741+0000","last_unstale":"2024-04-24T23:25:26.976875+0000","last_undegraded":"2024-04-24T23:25:26.976875+0000","last_fullsized":"2024-04-24T23:25:26.976875+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:51.782921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970234+0000","last_change":"2024-04-24T23:25:26.970234+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970234+0000","last_undegraded":"2024-04-24T23:25:26.970234+0000","last_fullsized":"2024-04-24T23:25:26.970234+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.988362+0000","last_change":"2024-04-24T23:25:26.986864+0000","last_active":"2024-04-24T23:25:26.988362+0000","last_peered":"2024-04-24T23:25:26.988362+0000","last_clean":"2024-04-24T23:25:26.988362+0000","last_became_active":"2024-04-24T23:25:26.986575+0000","last_became_peered":"2024-04-24T23:25:26.986575+0000","last_unstale":"2024-04-24T23:25:26.988362+0000","last_undegraded":"2024-04-24T23:25:26.988362+0000","last_fullsized":"2024-04-24T23:25:26.988362+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:57.895459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975340+0000","last_change":"2024-04-24T23:25:26.975340+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975340+0000","last_undegraded":"2024-04-24T23:25:26.975340+0000","last_fullsized":"2024-04-24T23:25:26.975340+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.984885+0000","last_change":"2024-04-24T23:25:26.983803+0000","last_active":"2024-04-24T23:25:26.984885+0000","last_peered":"2024-04-24T23:25:26.984885+0000","last_clean":"2024-04-24T23:25:26.984885+0000","last_became_active":"2024-04-24T23:25:26.983465+0000","last_became_peered":"2024-04-24T23:25:26.983465+0000","last_unstale":"2024-04-24T23:25:26.984885+0000","last_undegraded":"2024-04-24T23:25:26.984885+0000","last_fullsized":"2024-04-24T23:25:26.984885+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:48.673865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968080+0000","last_change":"2024-04-24T23:25:26.968080+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968080+0000","last_undegraded":"2024-04-24T23:25:26.968080+0000","last_fullsized":"2024-04-24T23:25:26.968080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.981884+0000","last_change":"2024-04-24T23:25:26.980977+0000","last_active":"2024-04-24T23:25:26.981884+0000","last_peered":"2024-04-24T23:25:26.981884+0000","last_clean":"2024-04-24T23:25:26.981884+0000","last_became_active":"2024-04-24T23:25:26.979025+0000","last_became_peered":"2024-04-24T23:25:26.979025+0000","last_unstale":"2024-04-24T23:25:26.981884+0000","last_undegraded":"2024-04-24T23:25:26.981884+0000","last_fullsized":"2024-04-24T23:25:26.981884+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:57.895459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971378+0000","last_change":"2024-04-24T23:25:26.971378+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971378+0000","last_undegraded":"2024-04-24T23:25:26.971378+0000","last_fullsized":"2024-04-24T23:25:26.971378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985813+0000","last_change":"2024-04-24T23:25:26.984615+0000","last_active":"2024-04-24T23:25:26.985813+0000","last_peered":"2024-04-24T23:25:26.985813+0000","last_clean":"2024-04-24T23:25:26.985813+0000","last_became_active":"2024-04-24T23:25:26.984358+0000","last_became_peered":"2024-04-24T23:25:26.984358+0000","last_unstale":"2024-04-24T23:25:26.985813+0000","last_undegraded":"2024-04-24T23:25:26.985813+0000","last_fullsized":"2024-04-24T23:25:26.985813+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:05.698635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971399+0000","last_change":"2024-04-24T23:25:26.971399+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971399+0000","last_undegraded":"2024-04-24T23:25:26.971399+0000","last_fullsized":"2024-04-24T23:25:26.971399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.988937+0000","last_change":"2024-04-24T23:25:26.988155+0000","last_active":"2024-04-24T23:25:26.988937+0000","last_peered":"2024-04-24T23:25:26.988937+0000","last_clean":"2024-04-24T23:25:26.988937+0000","last_became_active":"2024-04-24T23:25:26.987824+0000","last_became_peered":"2024-04-24T23:25:26.987824+0000","last_unstale":"2024-04-24T23:25:26.988937+0000","last_undegraded":"2024-04-24T23:25:26.988937+0000","last_fullsized":"2024-04-24T23:25:26.988937+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:42.850369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.969781+0000","last_change":"2024-04-24T23:25:26.969781+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.969781+0000","last_undegraded":"2024-04-24T23:25:26.969781+0000","last_fullsized":"2024-04-24T23:25:26.969781+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.979989+0000","last_change":"2024-04-24T23:25:26.976419+0000","last_active":"2024-04-24T23:25:26.979989+0000","last_peered":"2024-04-24T23:25:26.979989+0000","last_clean":"2024-04-24T23:25:26.979989+0000","last_became_active":"2024-04-24T23:25:26.976109+0000","last_became_peered":"2024-04-24T23:25:26.976109+0000","last_unstale":"2024-04-24T23:25:26.979989+0000","last_undegraded":"2024-04-24T23:25:26.979989+0000","last_fullsized":"2024-04-24T23:25:26.979989+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:19.806058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.978803+0000","last_change":"2024-04-24T23:25:26.978803+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.978803+0000","last_undegraded":"2024-04-24T23:25:26.978803+0000","last_fullsized":"2024-04-24T23:25:26.978803+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986234+0000","last_change":"2024-04-24T23:25:26.985538+0000","last_active":"2024-04-24T23:25:26.986234+0000","last_peered":"2024-04-24T23:25:26.986234+0000","last_clean":"2024-04-24T23:25:26.986234+0000","last_became_active":"2024-04-24T23:25:26.985216+0000","last_became_peered":"2024-04-24T23:25:26.985216+0000","last_unstale":"2024-04-24T23:25:26.986234+0000","last_undegraded":"2024-04-24T23:25:26.986234+0000","last_fullsized":"2024-04-24T23:25:26.986234+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:15.381258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975739+0000","last_change":"2024-04-24T23:25:26.975739+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975739+0000","last_undegraded":"2024-04-24T23:25:26.975739+0000","last_fullsized":"2024-04-24T23:25:26.975739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.979824+0000","last_change":"2024-04-24T23:25:26.978070+0000","last_active":"2024-04-24T23:25:26.979824+0000","last_peered":"2024-04-24T23:25:26.979824+0000","last_clean":"2024-04-24T23:25:26.979824+0000","last_became_active":"2024-04-24T23:25:26.977796+0000","last_became_peered":"2024-04-24T23:25:26.977796+0000","last_unstale":"2024-04-24T23:25:26.979824+0000","last_undegraded":"2024-04-24T23:25:26.979824+0000","last_fullsized":"2024-04-24T23:25:26.979824+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:27.254375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970188+0000","last_change":"2024-04-24T23:25:26.970188+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970188+0000","last_undegraded":"2024-04-24T23:25:26.970188+0000","last_fullsized":"2024-04-24T23:25:26.970188+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986754+0000","last_change":"2024-04-24T23:25:26.985857+0000","last_active":"2024-04-24T23:25:26.986754+0000","last_peered":"2024-04-24T23:25:26.986754+0000","last_clean":"2024-04-24T23:25:26.986754+0000","last_became_active":"2024-04-24T23:25:26.985337+0000","last_became_peered":"2024-04-24T23:25:26.985337+0000","last_unstale":"2024-04-24T23:25:26.986754+0000","last_undegraded":"2024-04-24T23:25:26.986754+0000","last_fullsized":"2024-04-24T23:25:26.986754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:17.024902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970226+0000","last_change":"2024-04-24T23:25:26.970226+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970226+0000","last_undegraded":"2024-04-24T23:25:26.970226+0000","last_fullsized":"2024-04-24T23:25:26.970226+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987375+0000","last_change":"2024-04-24T23:25:26.986149+0000","last_active":"2024-04-24T23:25:26.987375+0000","last_peered":"2024-04-24T23:25:26.987375+0000","last_clean":"2024-04-24T23:25:26.987375+0000","last_became_active":"2024-04-24T23:25:26.985763+0000","last_became_peered":"2024-04-24T23:25:26.985763+0000","last_unstale":"2024-04-24T23:25:26.987375+0000","last_undegraded":"2024-04-24T23:25:26.987375+0000","last_fullsized":"2024-04-24T23:25:26.987375+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:06.469168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":254,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.973212+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:25:26.973212+0000","last_peered":"2024-04-24T23:25:26.973212+0000","last_clean":"2024-04-24T23:25:26.973212+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:25:26.973212+0000","last_undegraded":"2024-04-24T23:25:26.973212+0000","last_fullsized":"2024-04-24T23:25:26.973212+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":928199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":337831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":344106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":344106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":337831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":337831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":344106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36720,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335120,"statfs":{"total":96636764160,"available":96599162880,"internally_reserved":0,"allocated":1409024,"data_stored":934474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:25:31.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:25:32.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.132+0000 7f9c34eae640 1 Processor -- start 2024-04-24T23:25:32.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.132+0000 7f9c34eae640 1 -- start start 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.132+0000 7f9c34eae640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c30143c40 0x7f9c300cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c34eae640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f9c300cc860 0x7f9c30072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c34eae640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c300cd0b0 con 0x7f9c3015eb00 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c34eae640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f9c30071b70 con 0x7f9c300cc860 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c34eae640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9c30071240 con 0x7f9c30143c40 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9c3015eb00 0x7f9c300b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32864/0 (socket says 172.21.15.26:32864) 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2ed9d640 1 -- 172.21.15.26:0/1514574681 learned_addr learned my addr 172.21.15.26:0/1514574681 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2e59c640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c30143c40 0x7f9c300cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2dd9b640 1 --2- 172.21.15.26:0/1514574681 >> v2:172.21.15.29:3300/0 conn(0x7f9c300cc860 0x7f9c30072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1507045362 0 0) 0x7f9c300cd0b0 con 0x7f9c3015eb00 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c18003520 con 0x7f9c3015eb00 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3627817164 0 0) 0x7f9c18003520 con 0x7f9c3015eb00 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c30143c40 msgr2=0x7f9c300cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c30143c40 0x7f9c300cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 >> v2:172.21.15.29:3300/0 conn(0x7f9c300cc860 msgr2=0x7f9c30072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 --2- 172.21.15.26:0/1514574681 >> v2:172.21.15.29:3300/0 conn(0x7f9c300cc860 0x7f9c30072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c30071520 con 0x7f9c3015eb00 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f9c280019f0 con 0x7f9c3015eb00 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 >> v1:172.21.15.26:6789/0 conn(0x7f9c3015eb00 legacy=0x7f9c300b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c30143c40 0x7f9c180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c18003f30 0x7f9c18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c30071b70 con 0x7f9c18004d40 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9c30071240 con 0x7f9c18003f30 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.133+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1800b9c0 con 0x7f9c30143c40 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.134+0000 7f9c2ed9d640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.134+0000 7f9c2e59c640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c30143c40 0x7f9c180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.134+0000 7f9c2dd9b640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c18003f30 0x7f9c18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.134+0000 7f9c2ed9d640 1 -- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c30143c40 msgr2=0x7f9c180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.134+0000 7f9c2ed9d640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c30143c40 0x7f9c180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.134+0000 7f9c2ed9d640 1 -- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c18003f30 msgr2=0x7f9c18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.134+0000 7f9c2ed9d640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c18003f30 0x7f9c18004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.134+0000 7f9c2ed9d640 1 -- 172.21.15.26:0/1514574681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c18003520 con 0x7f9c18004d40 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.134+0000 7f9c2ed9d640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c180051c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800a920 tx=0x7f9c18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c3432503d120cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:32.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2800a4c0 con 0x7f9c18004d40 2024-04-24T23:25:32.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2802f070 con 0x7f9c18004d40 2024-04-24T23:25:32.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c2d59a640 1 -- 172.21.15.26:0/1514574681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2802bd40 con 0x7f9c18004d40 2024-04-24T23:25:32.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c34eae640 1 -- 172.21.15.26:0/1514574681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 msgr2=0x7f9c180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c34eae640 1 --2- 172.21.15.26:0/1514574681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c180051c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800a920 tx=0x7f9c18005830 comp rx=0 tx=0).stop 2024-04-24T23:25:32.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c2ed9d640 1 -- 172.21.15.26:0/1514574681 reap_dead start 2024-04-24T23:25:32.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c34eae640 1 -- 172.21.15.26:0/1514574681 shutdown_connections 2024-04-24T23:25:32.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c34eae640 1 -- 172.21.15.26:0/1514574681 >> 172.21.15.26:0/1514574681 conn(0x7f9c3006b300 msgr2=0x7f9c3006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:32.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c34eae640 1 -- 172.21.15.26:0/1514574681 shutdown_connections 2024-04-24T23:25:32.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.135+0000 7f9c34eae640 1 -- 172.21.15.26:0/1514574681 wait complete. 2024-04-24T23:25:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c34eae640 1 Processor -- start 2024-04-24T23:25:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c34eae640 1 -- start start 2024-04-24T23:25:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c34eae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c300b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c34eae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3006de50 0x7f9c300ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c34eae640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c3006e2d0 0x7f9c300aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c34eae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c30072540 con 0x7f9c18004d40 2024-04-24T23:25:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c34eae640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9c30144020 con 0x7f9c3006de50 2024-04-24T23:25:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c34eae640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9c300b2c10 con 0x7f9c3006e2d0 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c2e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c300b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c2e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c300b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59054/0 (socket says 172.21.15.26:59054) 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.137+0000 7f9c2e59c640 1 -- 172.21.15.26:0/2822296093 learned_addr learned my addr 172.21.15.26:0/2822296093 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c2dd9b640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3006de50 0x7f9c300ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c2ed9d640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c3006e2d0 0x7f9c300aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c2e59c640 1 -- 172.21.15.26:0/2822296093 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c3006e2d0 msgr2=0x7f9c300aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c2e59c640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c3006e2d0 0x7f9c300aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c2e59c640 1 -- 172.21.15.26:0/2822296093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3006de50 msgr2=0x7f9c300ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c2e59c640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3006de50 0x7f9c300ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c2e59c640 1 -- 172.21.15.26:0/2822296093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c300b2ef0 con 0x7f9c18004d40 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c2e59c640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c300b39d0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9c200119a0 tx=0x7f9c20011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c1effd640 1 -- 172.21.15.26:0/2822296093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2000fa30 con 0x7f9c18004d40 2024-04-24T23:25:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c1effd640 1 -- 172.21.15.26:0/2822296093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2000fbd0 con 0x7f9c18004d40 2024-04-24T23:25:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c1effd640 1 -- 172.21.15.26:0/2822296093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20014440 con 0x7f9c18004d40 2024-04-24T23:25:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c300b31d0 con 0x7f9c18004d40 2024-04-24T23:25:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.138+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c300b3f10 con 0x7f9c18004d40 2024-04-24T23:25:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.139+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c300694c0 con 0x7f9c18004d40 2024-04-24T23:25:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.140+0000 7f9c1effd640 1 -- 172.21.15.26:0/2822296093 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c20012040 con 0x7f9c18004d40 2024-04-24T23:25:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.140+0000 7f9c1effd640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9c1006be30 0x7f9c1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.140+0000 7f9c1effd640 1 -- 172.21.15.26:0/2822296093 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f9c2008b530 con 0x7f9c18004d40 2024-04-24T23:25:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.140+0000 7f9c2dd9b640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9c1006be30 0x7f9c1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.141+0000 7f9c2dd9b640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9c1006be30 0x7f9c1006e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c3013d3d0 tx=0x7f9c24009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:32.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.144+0000 7f9c1effd640 1 -- 172.21.15.26:0/2822296093 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9c20019030 con 0x7f9c18004d40 2024-04-24T23:25:32.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.324+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c30072250 con 0x7f9c1006be30 2024-04-24T23:25:32.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.330+0000 7f9c1effd640 1 -- 172.21.15.26:0/2822296093 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246752 (secure 0 0 0) 0x7f9c30072250 con 0x7f9c1006be30 2024-04-24T23:25:32.333 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:25:32.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9c1006be30 msgr2=0x7f9c1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9c1006be30 0x7f9c1006e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c3013d3d0 tx=0x7f9c24009190 comp rx=0 tx=0).stop 2024-04-24T23:25:32.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 msgr2=0x7f9c300b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c300b39d0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9c200119a0 tx=0x7f9c20011da0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 shutdown_connections 2024-04-24T23:25:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9c1006be30 0x7f9c1006e2f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9c3006e2d0 0x7f9c300aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c3006de50 0x7f9c300ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 --2- 172.21.15.26:0/2822296093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c18004d40 0x7f9c300b39d0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 >> 172.21.15.26:0/2822296093 conn(0x7f9c3006b300 msgr2=0x7f9c3006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 shutdown_connections 2024-04-24T23:25:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.333+0000 7f9c34eae640 1 -- 172.21.15.26:0/2822296093 wait complete. 2024-04-24T23:25:32.345 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-04-24T23:25:31.376080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265256,"kb_used_data":7480,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754709464,"statfs":{"total":773094113280,"available":772822491136,"internally_reserved":0,"allocated":7659520,"data_stored":3908484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.413779"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972124+0000","last_change":"2024-04-24T23:25:26.972124+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972124+0000","last_undegraded":"2024-04-24T23:25:26.972124+0000","last_fullsized":"2024-04-24T23:25:26.972124+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973940+0000","last_change":"2024-04-24T23:25:26.973940+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973940+0000","last_undegraded":"2024-04-24T23:25:26.973940+0000","last_fullsized":"2024-04-24T23:25:26.973940+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971708+0000","last_change":"2024-04-24T23:25:26.971708+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971708+0000","last_undegraded":"2024-04-24T23:25:26.971708+0000","last_fullsized":"2024-04-24T23:25:26.971708+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.977999+0000","last_change":"2024-04-24T23:25:26.977999+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.977999+0000","last_undegraded":"2024-04-24T23:25:26.977999+0000","last_fullsized":"2024-04-24T23:25:26.977999+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974555+0000","last_change":"2024-04-24T23:25:26.974555+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974555+0000","last_undegraded":"2024-04-24T23:25:26.974555+0000","last_fullsized":"2024-04-24T23:25:26.974555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971249+0000","last_change":"2024-04-24T23:25:26.971249+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971249+0000","last_undegraded":"2024-04-24T23:25:26.971249+0000","last_fullsized":"2024-04-24T23:25:26.971249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968229+0000","last_change":"2024-04-24T23:25:26.968229+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968229+0000","last_undegraded":"2024-04-24T23:25:26.968229+0000","last_fullsized":"2024-04-24T23:25:26.968229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.976683+0000","last_change":"2024-04-24T23:25:26.976683+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.976683+0000","last_undegraded":"2024-04-24T23:25:26.976683+0000","last_fullsized":"2024-04-24T23:25:26.976683+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973701+0000","last_change":"2024-04-24T23:25:26.973701+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973701+0000","last_undegraded":"2024-04-24T23:25:26.973701+0000","last_fullsized":"2024-04-24T23:25:26.973701+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975677+0000","last_change":"2024-04-24T23:25:26.975677+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975677+0000","last_undegraded":"2024-04-24T23:25:26.975677+0000","last_fullsized":"2024-04-24T23:25:26.975677+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972646+0000","last_change":"2024-04-24T23:25:26.972646+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972646+0000","last_undegraded":"2024-04-24T23:25:26.972646+0000","last_fullsized":"2024-04-24T23:25:26.972646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.977553+0000","last_change":"2024-04-24T23:25:26.977553+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.977553+0000","last_undegraded":"2024-04-24T23:25:26.977553+0000","last_fullsized":"2024-04-24T23:25:26.977553+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972318+0000","last_change":"2024-04-24T23:25:26.972318+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972318+0000","last_undegraded":"2024-04-24T23:25:26.972318+0000","last_fullsized":"2024-04-24T23:25:26.972318+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973045+0000","last_change":"2024-04-24T23:25:26.973045+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973045+0000","last_undegraded":"2024-04-24T23:25:26.973045+0000","last_fullsized":"2024-04-24T23:25:26.973045+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973018+0000","last_change":"2024-04-24T23:25:26.973018+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973018+0000","last_undegraded":"2024-04-24T23:25:26.973018+0000","last_fullsized":"2024-04-24T23:25:26.973018+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975110+0000","last_change":"2024-04-24T23:25:26.975110+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975110+0000","last_undegraded":"2024-04-24T23:25:26.975110+0000","last_fullsized":"2024-04-24T23:25:26.975110+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973033+0000","last_change":"2024-04-24T23:25:26.973033+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973033+0000","last_undegraded":"2024-04-24T23:25:26.973033+0000","last_fullsized":"2024-04-24T23:25:26.973033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975740+0000","last_change":"2024-04-24T23:25:26.975740+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975740+0000","last_undegraded":"2024-04-24T23:25:26.975740+0000","last_fullsized":"2024-04-24T23:25:26.975740+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972531+0000","last_change":"2024-04-24T23:25:26.972531+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972531+0000","last_undegraded":"2024-04-24T23:25:26.972531+0000","last_fullsized":"2024-04-24T23:25:26.972531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972789+0000","last_change":"2024-04-24T23:25:26.972789+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972789+0000","last_undegraded":"2024-04-24T23:25:26.972789+0000","last_fullsized":"2024-04-24T23:25:26.972789+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975257+0000","last_change":"2024-04-24T23:25:26.975257+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975257+0000","last_undegraded":"2024-04-24T23:25:26.975257+0000","last_fullsized":"2024-04-24T23:25:26.975257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974439+0000","last_change":"2024-04-24T23:25:26.974439+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974439+0000","last_undegraded":"2024-04-24T23:25:26.974439+0000","last_fullsized":"2024-04-24T23:25:26.974439+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970819+0000","last_change":"2024-04-24T23:25:26.970819+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970819+0000","last_undegraded":"2024-04-24T23:25:26.970819+0000","last_fullsized":"2024-04-24T23:25:26.970819+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971038+0000","last_change":"2024-04-24T23:25:26.971038+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971038+0000","last_undegraded":"2024-04-24T23:25:26.971038+0000","last_fullsized":"2024-04-24T23:25:26.971038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973311+0000","last_change":"2024-04-24T23:25:26.973311+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973311+0000","last_undegraded":"2024-04-24T23:25:26.973311+0000","last_fullsized":"2024-04-24T23:25:26.973311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968913+0000","last_change":"2024-04-24T23:25:26.968913+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968913+0000","last_undegraded":"2024-04-24T23:25:26.968913+0000","last_fullsized":"2024-04-24T23:25:26.968913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971174+0000","last_change":"2024-04-24T23:25:26.971174+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971174+0000","last_undegraded":"2024-04-24T23:25:26.971174+0000","last_fullsized":"2024-04-24T23:25:26.971174+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972444+0000","last_change":"2024-04-24T23:25:26.972444+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972444+0000","last_undegraded":"2024-04-24T23:25:26.972444+0000","last_fullsized":"2024-04-24T23:25:26.972444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971143+0000","last_change":"2024-04-24T23:25:26.971143+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971143+0000","last_undegraded":"2024-04-24T23:25:26.971143+0000","last_fullsized":"2024-04-24T23:25:26.971143+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967626+0000","last_change":"2024-04-24T23:25:26.967626+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967626+0000","last_undegraded":"2024-04-24T23:25:26.967626+0000","last_fullsized":"2024-04-24T23:25:26.967626+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967495+0000","last_change":"2024-04-24T23:25:26.967495+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967495+0000","last_undegraded":"2024-04-24T23:25:26.967495+0000","last_fullsized":"2024-04-24T23:25:26.967495+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.976210+0000","last_change":"2024-04-24T23:25:26.976210+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.976210+0000","last_undegraded":"2024-04-24T23:25:26.976210+0000","last_fullsized":"2024-04-24T23:25:26.976210+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971298+0000","last_change":"2024-04-24T23:25:26.971298+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971298+0000","last_undegraded":"2024-04-24T23:25:26.971298+0000","last_fullsized":"2024-04-24T23:25:26.971298+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985622+0000","last_change":"2024-04-24T23:25:26.984830+0000","last_active":"2024-04-24T23:25:26.985622+0000","last_peered":"2024-04-24T23:25:26.985622+0000","last_clean":"2024-04-24T23:25:26.985622+0000","last_became_active":"2024-04-24T23:25:26.983743+0000","last_became_peered":"2024-04-24T23:25:26.983743+0000","last_unstale":"2024-04-24T23:25:26.985622+0000","last_undegraded":"2024-04-24T23:25:26.985622+0000","last_fullsized":"2024-04-24T23:25:26.985622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:01.501491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972060+0000","last_change":"2024-04-24T23:25:26.972060+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972060+0000","last_undegraded":"2024-04-24T23:25:26.972060+0000","last_fullsized":"2024-04-24T23:25:26.972060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.988350+0000","last_change":"2024-04-24T23:25:26.986600+0000","last_active":"2024-04-24T23:25:26.988350+0000","last_peered":"2024-04-24T23:25:26.988350+0000","last_clean":"2024-04-24T23:25:26.988350+0000","last_became_active":"2024-04-24T23:25:26.986208+0000","last_became_peered":"2024-04-24T23:25:26.986208+0000","last_unstale":"2024-04-24T23:25:26.988350+0000","last_undegraded":"2024-04-24T23:25:26.988350+0000","last_fullsized":"2024-04-24T23:25:26.988350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:06.044715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974992+0000","last_change":"2024-04-24T23:25:26.974992+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974992+0000","last_undegraded":"2024-04-24T23:25:26.974992+0000","last_fullsized":"2024-04-24T23:25:26.974992+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.989266+0000","last_change":"2024-04-24T23:25:26.988051+0000","last_active":"2024-04-24T23:25:26.989266+0000","last_peered":"2024-04-24T23:25:26.989266+0000","last_clean":"2024-04-24T23:25:26.989266+0000","last_became_active":"2024-04-24T23:25:26.987663+0000","last_became_peered":"2024-04-24T23:25:26.987663+0000","last_unstale":"2024-04-24T23:25:26.989266+0000","last_undegraded":"2024-04-24T23:25:26.989266+0000","last_fullsized":"2024-04-24T23:25:26.989266+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:11.588401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972006+0000","last_change":"2024-04-24T23:25:26.972006+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972006+0000","last_undegraded":"2024-04-24T23:25:26.972006+0000","last_fullsized":"2024-04-24T23:25:26.972006+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987433+0000","last_change":"2024-04-24T23:25:26.986624+0000","last_active":"2024-04-24T23:25:26.987433+0000","last_peered":"2024-04-24T23:25:26.987433+0000","last_clean":"2024-04-24T23:25:26.987433+0000","last_became_active":"2024-04-24T23:25:26.986323+0000","last_became_peered":"2024-04-24T23:25:26.986323+0000","last_unstale":"2024-04-24T23:25:26.987433+0000","last_undegraded":"2024-04-24T23:25:26.987433+0000","last_fullsized":"2024-04-24T23:25:26.987433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:11.677239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967001+0000","last_change":"2024-04-24T23:25:26.967001+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967001+0000","last_undegraded":"2024-04-24T23:25:26.967001+0000","last_fullsized":"2024-04-24T23:25:26.967001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.978242+0000","last_change":"2024-04-24T23:25:26.977218+0000","last_active":"2024-04-24T23:25:26.978242+0000","last_peered":"2024-04-24T23:25:26.978242+0000","last_clean":"2024-04-24T23:25:26.978242+0000","last_became_active":"2024-04-24T23:25:26.976913+0000","last_became_peered":"2024-04-24T23:25:26.976913+0000","last_unstale":"2024-04-24T23:25:26.978242+0000","last_undegraded":"2024-04-24T23:25:26.978242+0000","last_fullsized":"2024-04-24T23:25:26.978242+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:15.039445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972886+0000","last_change":"2024-04-24T23:25:26.972886+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972886+0000","last_undegraded":"2024-04-24T23:25:26.972886+0000","last_fullsized":"2024-04-24T23:25:26.972886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986090+0000","last_change":"2024-04-24T23:25:26.984947+0000","last_active":"2024-04-24T23:25:26.986090+0000","last_peered":"2024-04-24T23:25:26.986090+0000","last_clean":"2024-04-24T23:25:26.986090+0000","last_became_active":"2024-04-24T23:25:26.984604+0000","last_became_peered":"2024-04-24T23:25:26.984604+0000","last_unstale":"2024-04-24T23:25:26.986090+0000","last_undegraded":"2024-04-24T23:25:26.986090+0000","last_fullsized":"2024-04-24T23:25:26.986090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:27.118553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972021+0000","last_change":"2024-04-24T23:25:26.972021+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972021+0000","last_undegraded":"2024-04-24T23:25:26.972021+0000","last_fullsized":"2024-04-24T23:25:26.972021+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985711+0000","last_change":"2024-04-24T23:25:26.982771+0000","last_active":"2024-04-24T23:25:26.985711+0000","last_peered":"2024-04-24T23:25:26.985711+0000","last_clean":"2024-04-24T23:25:26.985711+0000","last_became_active":"2024-04-24T23:25:26.982504+0000","last_became_peered":"2024-04-24T23:25:26.982504+0000","last_unstale":"2024-04-24T23:25:26.985711+0000","last_undegraded":"2024-04-24T23:25:26.985711+0000","last_fullsized":"2024-04-24T23:25:26.985711+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:10.968125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972021+0000","last_change":"2024-04-24T23:25:26.972021+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972021+0000","last_undegraded":"2024-04-24T23:25:26.972021+0000","last_fullsized":"2024-04-24T23:25:26.972021+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987666+0000","last_change":"2024-04-24T23:25:26.985502+0000","last_active":"2024-04-24T23:25:26.987666+0000","last_peered":"2024-04-24T23:25:26.987666+0000","last_clean":"2024-04-24T23:25:26.987666+0000","last_became_active":"2024-04-24T23:25:26.985236+0000","last_became_peered":"2024-04-24T23:25:26.985236+0000","last_unstale":"2024-04-24T23:25:26.987666+0000","last_undegraded":"2024-04-24T23:25:26.987666+0000","last_fullsized":"2024-04-24T23:25:26.987666+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:42.270581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968067+0000","last_change":"2024-04-24T23:25:26.968067+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968067+0000","last_undegraded":"2024-04-24T23:25:26.968067+0000","last_fullsized":"2024-04-24T23:25:26.968067+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.978235+0000","last_change":"2024-04-24T23:25:26.976690+0000","last_active":"2024-04-24T23:25:26.978235+0000","last_peered":"2024-04-24T23:25:26.978235+0000","last_clean":"2024-04-24T23:25:26.978235+0000","last_became_active":"2024-04-24T23:25:26.976221+0000","last_became_peered":"2024-04-24T23:25:26.976221+0000","last_unstale":"2024-04-24T23:25:26.978235+0000","last_undegraded":"2024-04-24T23:25:26.978235+0000","last_fullsized":"2024-04-24T23:25:26.978235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:26.544193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971698+0000","last_change":"2024-04-24T23:25:26.971698+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971698+0000","last_undegraded":"2024-04-24T23:25:26.971698+0000","last_fullsized":"2024-04-24T23:25:26.971698+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.984328+0000","last_change":"2024-04-24T23:25:26.979704+0000","last_active":"2024-04-24T23:25:26.984328+0000","last_peered":"2024-04-24T23:25:26.984328+0000","last_clean":"2024-04-24T23:25:26.984328+0000","last_became_active":"2024-04-24T23:25:26.979294+0000","last_became_peered":"2024-04-24T23:25:26.979294+0000","last_unstale":"2024-04-24T23:25:26.984328+0000","last_undegraded":"2024-04-24T23:25:26.984328+0000","last_fullsized":"2024-04-24T23:25:26.984328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:59.557263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974519+0000","last_change":"2024-04-24T23:25:26.974519+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974519+0000","last_undegraded":"2024-04-24T23:25:26.974519+0000","last_fullsized":"2024-04-24T23:25:26.974519+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.974980+0000","last_change":"2024-04-24T23:25:26.973701+0000","last_active":"2024-04-24T23:25:26.974980+0000","last_peered":"2024-04-24T23:25:26.974980+0000","last_clean":"2024-04-24T23:25:26.974980+0000","last_became_active":"2024-04-24T23:25:26.973424+0000","last_became_peered":"2024-04-24T23:25:26.973424+0000","last_unstale":"2024-04-24T23:25:26.974980+0000","last_undegraded":"2024-04-24T23:25:26.974980+0000","last_fullsized":"2024-04-24T23:25:26.974980+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:54.920525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972192+0000","last_change":"2024-04-24T23:25:26.972192+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972192+0000","last_undegraded":"2024-04-24T23:25:26.972192+0000","last_fullsized":"2024-04-24T23:25:26.972192+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987465+0000","last_change":"2024-04-24T23:25:26.985049+0000","last_active":"2024-04-24T23:25:26.987465+0000","last_peered":"2024-04-24T23:25:26.987465+0000","last_clean":"2024-04-24T23:25:26.987465+0000","last_became_active":"2024-04-24T23:25:26.984293+0000","last_became_peered":"2024-04-24T23:25:26.984293+0000","last_unstale":"2024-04-24T23:25:26.987465+0000","last_undegraded":"2024-04-24T23:25:26.987465+0000","last_fullsized":"2024-04-24T23:25:26.987465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:12.258864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970929+0000","last_change":"2024-04-24T23:25:26.970929+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970929+0000","last_undegraded":"2024-04-24T23:25:26.970929+0000","last_fullsized":"2024-04-24T23:25:26.970929+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986717+0000","last_change":"2024-04-24T23:25:26.982431+0000","last_active":"2024-04-24T23:25:26.986717+0000","last_peered":"2024-04-24T23:25:26.986717+0000","last_clean":"2024-04-24T23:25:26.986717+0000","last_became_active":"2024-04-24T23:25:26.982147+0000","last_became_peered":"2024-04-24T23:25:26.982147+0000","last_unstale":"2024-04-24T23:25:26.986717+0000","last_undegraded":"2024-04-24T23:25:26.986717+0000","last_fullsized":"2024-04-24T23:25:26.986717+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:59.723643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967531+0000","last_change":"2024-04-24T23:25:26.967531+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967531+0000","last_undegraded":"2024-04-24T23:25:26.967531+0000","last_fullsized":"2024-04-24T23:25:26.967531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985013+0000","last_change":"2024-04-24T23:25:26.979601+0000","last_active":"2024-04-24T23:25:26.985013+0000","last_peered":"2024-04-24T23:25:26.985013+0000","last_clean":"2024-04-24T23:25:26.985013+0000","last_became_active":"2024-04-24T23:25:26.978063+0000","last_became_peered":"2024-04-24T23:25:26.978063+0000","last_unstale":"2024-04-24T23:25:26.985013+0000","last_undegraded":"2024-04-24T23:25:26.985013+0000","last_fullsized":"2024-04-24T23:25:26.985013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:32.655714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972297+0000","last_change":"2024-04-24T23:25:26.972297+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972297+0000","last_undegraded":"2024-04-24T23:25:26.972297+0000","last_fullsized":"2024-04-24T23:25:26.972297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.981043+0000","last_change":"2024-04-24T23:25:26.979978+0000","last_active":"2024-04-24T23:25:26.981043+0000","last_peered":"2024-04-24T23:25:26.981043+0000","last_clean":"2024-04-24T23:25:26.981043+0000","last_became_active":"2024-04-24T23:25:26.978846+0000","last_became_peered":"2024-04-24T23:25:26.978846+0000","last_unstale":"2024-04-24T23:25:26.981043+0000","last_undegraded":"2024-04-24T23:25:26.981043+0000","last_fullsized":"2024-04-24T23:25:26.981043+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:06.044715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971257+0000","last_change":"2024-04-24T23:25:26.971257+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971257+0000","last_undegraded":"2024-04-24T23:25:26.971257+0000","last_fullsized":"2024-04-24T23:25:26.971257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985464+0000","last_change":"2024-04-24T23:25:26.983021+0000","last_active":"2024-04-24T23:25:26.985464+0000","last_peered":"2024-04-24T23:25:26.985464+0000","last_clean":"2024-04-24T23:25:26.985464+0000","last_became_active":"2024-04-24T23:25:26.982757+0000","last_became_peered":"2024-04-24T23:25:26.982757+0000","last_unstale":"2024-04-24T23:25:26.985464+0000","last_undegraded":"2024-04-24T23:25:26.985464+0000","last_fullsized":"2024-04-24T23:25:26.985464+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:28.905967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972173+0000","last_change":"2024-04-24T23:25:26.972173+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972173+0000","last_undegraded":"2024-04-24T23:25:26.972173+0000","last_fullsized":"2024-04-24T23:25:26.972173+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985354+0000","last_change":"2024-04-24T23:25:26.984211+0000","last_active":"2024-04-24T23:25:26.985354+0000","last_peered":"2024-04-24T23:25:26.985354+0000","last_clean":"2024-04-24T23:25:26.985354+0000","last_became_active":"2024-04-24T23:25:26.983878+0000","last_became_peered":"2024-04-24T23:25:26.983878+0000","last_unstale":"2024-04-24T23:25:26.985354+0000","last_undegraded":"2024-04-24T23:25:26.985354+0000","last_fullsized":"2024-04-24T23:25:26.985354+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:39.951844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971011+0000","last_change":"2024-04-24T23:25:26.971011+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971011+0000","last_undegraded":"2024-04-24T23:25:26.971011+0000","last_fullsized":"2024-04-24T23:25:26.971011+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.989233+0000","last_change":"2024-04-24T23:25:26.988180+0000","last_active":"2024-04-24T23:25:26.989233+0000","last_peered":"2024-04-24T23:25:26.989233+0000","last_clean":"2024-04-24T23:25:26.989233+0000","last_became_active":"2024-04-24T23:25:26.987706+0000","last_became_peered":"2024-04-24T23:25:26.987706+0000","last_unstale":"2024-04-24T23:25:26.989233+0000","last_undegraded":"2024-04-24T23:25:26.989233+0000","last_fullsized":"2024-04-24T23:25:26.989233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:04.387670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.977087+0000","last_change":"2024-04-24T23:25:26.977087+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.977087+0000","last_undegraded":"2024-04-24T23:25:26.977087+0000","last_fullsized":"2024-04-24T23:25:26.977087+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.990338+0000","last_change":"2024-04-24T23:25:26.989891+0000","last_active":"2024-04-24T23:25:26.990338+0000","last_peered":"2024-04-24T23:25:26.990338+0000","last_clean":"2024-04-24T23:25:26.990338+0000","last_became_active":"2024-04-24T23:25:26.989524+0000","last_became_peered":"2024-04-24T23:25:26.989524+0000","last_unstale":"2024-04-24T23:25:26.990338+0000","last_undegraded":"2024-04-24T23:25:26.990338+0000","last_fullsized":"2024-04-24T23:25:26.990338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:44.608576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971175+0000","last_change":"2024-04-24T23:25:26.971175+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971175+0000","last_undegraded":"2024-04-24T23:25:26.971175+0000","last_fullsized":"2024-04-24T23:25:26.971175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.982172+0000","last_change":"2024-04-24T23:25:26.980697+0000","last_active":"2024-04-24T23:25:26.982172+0000","last_peered":"2024-04-24T23:25:26.982172+0000","last_clean":"2024-04-24T23:25:26.982172+0000","last_became_active":"2024-04-24T23:25:26.980315+0000","last_became_peered":"2024-04-24T23:25:26.980315+0000","last_unstale":"2024-04-24T23:25:26.982172+0000","last_undegraded":"2024-04-24T23:25:26.982172+0000","last_fullsized":"2024-04-24T23:25:26.982172+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:43.615264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970605+0000","last_change":"2024-04-24T23:25:26.970605+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970605+0000","last_undegraded":"2024-04-24T23:25:26.970605+0000","last_fullsized":"2024-04-24T23:25:26.970605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986068+0000","last_change":"2024-04-24T23:25:26.984839+0000","last_active":"2024-04-24T23:25:26.986068+0000","last_peered":"2024-04-24T23:25:26.986068+0000","last_clean":"2024-04-24T23:25:26.986068+0000","last_became_active":"2024-04-24T23:25:26.984489+0000","last_became_peered":"2024-04-24T23:25:26.984489+0000","last_unstale":"2024-04-24T23:25:26.986068+0000","last_undegraded":"2024-04-24T23:25:26.986068+0000","last_fullsized":"2024-04-24T23:25:26.986068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:50.577760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.969773+0000","last_change":"2024-04-24T23:25:26.969773+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.969773+0000","last_undegraded":"2024-04-24T23:25:26.969773+0000","last_fullsized":"2024-04-24T23:25:26.969773+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.976875+0000","last_change":"2024-04-24T23:25:26.975028+0000","last_active":"2024-04-24T23:25:26.976875+0000","last_peered":"2024-04-24T23:25:26.976875+0000","last_clean":"2024-04-24T23:25:26.976875+0000","last_became_active":"2024-04-24T23:25:26.974741+0000","last_became_peered":"2024-04-24T23:25:26.974741+0000","last_unstale":"2024-04-24T23:25:26.976875+0000","last_undegraded":"2024-04-24T23:25:26.976875+0000","last_fullsized":"2024-04-24T23:25:26.976875+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:51.782921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970234+0000","last_change":"2024-04-24T23:25:26.970234+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970234+0000","last_undegraded":"2024-04-24T23:25:26.970234+0000","last_fullsized":"2024-04-24T23:25:26.970234+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.988362+0000","last_change":"2024-04-24T23:25:26.986864+0000","last_active":"2024-04-24T23:25:26.988362+0000","last_peered":"2024-04-24T23:25:26.988362+0000","last_clean":"2024-04-24T23:25:26.988362+0000","last_became_active":"2024-04-24T23:25:26.986575+0000","last_became_peered":"2024-04-24T23:25:26.986575+0000","last_unstale":"2024-04-24T23:25:26.988362+0000","last_undegraded":"2024-04-24T23:25:26.988362+0000","last_fullsized":"2024-04-24T23:25:26.988362+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:57.895459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975340+0000","last_change":"2024-04-24T23:25:26.975340+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975340+0000","last_undegraded":"2024-04-24T23:25:26.975340+0000","last_fullsized":"2024-04-24T23:25:26.975340+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.984885+0000","last_change":"2024-04-24T23:25:26.983803+0000","last_active":"2024-04-24T23:25:26.984885+0000","last_peered":"2024-04-24T23:25:26.984885+0000","last_clean":"2024-04-24T23:25:26.984885+0000","last_became_active":"2024-04-24T23:25:26.983465+0000","last_became_peered":"2024-04-24T23:25:26.983465+0000","last_unstale":"2024-04-24T23:25:26.984885+0000","last_undegraded":"2024-04-24T23:25:26.984885+0000","last_fullsized":"2024-04-24T23:25:26.984885+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:48.673865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968080+0000","last_change":"2024-04-24T23:25:26.968080+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968080+0000","last_undegraded":"2024-04-24T23:25:26.968080+0000","last_fullsized":"2024-04-24T23:25:26.968080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.981884+0000","last_change":"2024-04-24T23:25:26.980977+0000","last_active":"2024-04-24T23:25:26.981884+0000","last_peered":"2024-04-24T23:25:26.981884+0000","last_clean":"2024-04-24T23:25:26.981884+0000","last_became_active":"2024-04-24T23:25:26.979025+0000","last_became_peered":"2024-04-24T23:25:26.979025+0000","last_unstale":"2024-04-24T23:25:26.981884+0000","last_undegraded":"2024-04-24T23:25:26.981884+0000","last_fullsized":"2024-04-24T23:25:26.981884+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:57.895459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971378+0000","last_change":"2024-04-24T23:25:26.971378+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971378+0000","last_undegraded":"2024-04-24T23:25:26.971378+0000","last_fullsized":"2024-04-24T23:25:26.971378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985813+0000","last_change":"2024-04-24T23:25:26.984615+0000","last_active":"2024-04-24T23:25:26.985813+0000","last_peered":"2024-04-24T23:25:26.985813+0000","last_clean":"2024-04-24T23:25:26.985813+0000","last_became_active":"2024-04-24T23:25:26.984358+0000","last_became_peered":"2024-04-24T23:25:26.984358+0000","last_unstale":"2024-04-24T23:25:26.985813+0000","last_undegraded":"2024-04-24T23:25:26.985813+0000","last_fullsized":"2024-04-24T23:25:26.985813+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:05.698635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971399+0000","last_change":"2024-04-24T23:25:26.971399+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971399+0000","last_undegraded":"2024-04-24T23:25:26.971399+0000","last_fullsized":"2024-04-24T23:25:26.971399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.988937+0000","last_change":"2024-04-24T23:25:26.988155+0000","last_active":"2024-04-24T23:25:26.988937+0000","last_peered":"2024-04-24T23:25:26.988937+0000","last_clean":"2024-04-24T23:25:26.988937+0000","last_became_active":"2024-04-24T23:25:26.987824+0000","last_became_peered":"2024-04-24T23:25:26.987824+0000","last_unstale":"2024-04-24T23:25:26.988937+0000","last_undegraded":"2024-04-24T23:25:26.988937+0000","last_fullsized":"2024-04-24T23:25:26.988937+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:42.850369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.969781+0000","last_change":"2024-04-24T23:25:26.969781+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.969781+0000","last_undegraded":"2024-04-24T23:25:26.969781+0000","last_fullsized":"2024-04-24T23:25:26.969781+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.979989+0000","last_change":"2024-04-24T23:25:26.976419+0000","last_active":"2024-04-24T23:25:26.979989+0000","last_peered":"2024-04-24T23:25:26.979989+0000","last_clean":"2024-04-24T23:25:26.979989+0000","last_became_active":"2024-04-24T23:25:26.976109+0000","last_became_peered":"2024-04-24T23:25:26.976109+0000","last_unstale":"2024-04-24T23:25:26.979989+0000","last_undegraded":"2024-04-24T23:25:26.979989+0000","last_fullsized":"2024-04-24T23:25:26.979989+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:19.806058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.978803+0000","last_change":"2024-04-24T23:25:26.978803+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.978803+0000","last_undegraded":"2024-04-24T23:25:26.978803+0000","last_fullsized":"2024-04-24T23:25:26.978803+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986234+0000","last_change":"2024-04-24T23:25:26.985538+0000","last_active":"2024-04-24T23:25:26.986234+0000","last_peered":"2024-04-24T23:25:26.986234+0000","last_clean":"2024-04-24T23:25:26.986234+0000","last_became_active":"2024-04-24T23:25:26.985216+0000","last_became_peered":"2024-04-24T23:25:26.985216+0000","last_unstale":"2024-04-24T23:25:26.986234+0000","last_undegraded":"2024-04-24T23:25:26.986234+0000","last_fullsized":"2024-04-24T23:25:26.986234+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:15.381258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975739+0000","last_change":"2024-04-24T23:25:26.975739+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975739+0000","last_undegraded":"2024-04-24T23:25:26.975739+0000","last_fullsized":"2024-04-24T23:25:26.975739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.979824+0000","last_change":"2024-04-24T23:25:26.978070+0000","last_active":"2024-04-24T23:25:26.979824+0000","last_peered":"2024-04-24T23:25:26.979824+0000","last_clean":"2024-04-24T23:25:26.979824+0000","last_became_active":"2024-04-24T23:25:26.977796+0000","last_became_peered":"2024-04-24T23:25:26.977796+0000","last_unstale":"2024-04-24T23:25:26.979824+0000","last_undegraded":"2024-04-24T23:25:26.979824+0000","last_fullsized":"2024-04-24T23:25:26.979824+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:27.254375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970188+0000","last_change":"2024-04-24T23:25:26.970188+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970188+0000","last_undegraded":"2024-04-24T23:25:26.970188+0000","last_fullsized":"2024-04-24T23:25:26.970188+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986754+0000","last_change":"2024-04-24T23:25:26.985857+0000","last_active":"2024-04-24T23:25:26.986754+0000","last_peered":"2024-04-24T23:25:26.986754+0000","last_clean":"2024-04-24T23:25:26.986754+0000","last_became_active":"2024-04-24T23:25:26.985337+0000","last_became_peered":"2024-04-24T23:25:26.985337+0000","last_unstale":"2024-04-24T23:25:26.986754+0000","last_undegraded":"2024-04-24T23:25:26.986754+0000","last_fullsized":"2024-04-24T23:25:26.986754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:17.024902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970226+0000","last_change":"2024-04-24T23:25:26.970226+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970226+0000","last_undegraded":"2024-04-24T23:25:26.970226+0000","last_fullsized":"2024-04-24T23:25:26.970226+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987375+0000","last_change":"2024-04-24T23:25:26.986149+0000","last_active":"2024-04-24T23:25:26.987375+0000","last_peered":"2024-04-24T23:25:26.987375+0000","last_clean":"2024-04-24T23:25:26.987375+0000","last_became_active":"2024-04-24T23:25:26.985763+0000","last_became_peered":"2024-04-24T23:25:26.985763+0000","last_unstale":"2024-04-24T23:25:26.987375+0000","last_undegraded":"2024-04-24T23:25:26.987375+0000","last_fullsized":"2024-04-24T23:25:26.987375+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:06.469168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":254,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.973212+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:25:26.973212+0000","last_peered":"2024-04-24T23:25:26.973212+0000","last_clean":"2024-04-24T23:25:26.973212+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:25:26.973212+0000","last_undegraded":"2024-04-24T23:25:26.973212+0000","last_fullsized":"2024-04-24T23:25:26.973212+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":928199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":337831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":344106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":344106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":337831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":337831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":344106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36720,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335120,"statfs":{"total":96636764160,"available":96599162880,"internally_reserved":0,"allocated":1409024,"data_stored":934474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:25:32.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:25:32.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.485+0000 7ff67d7ce640 1 Processor -- start 2024-04-24T23:25:32.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.485+0000 7ff67d7ce640 1 -- start start 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.486+0000 7ff67d7ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff67815e570 0x7ff6780d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.486+0000 7ff67d7ce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff6780d22e0 0x7ff6780d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.486+0000 7ff67d7ce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6780d32f0 con 0x7ff6780b3130 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.486+0000 7ff67d7ce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff6780d1ab0 con 0x7ff6780d22e0 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.486+0000 7ff67d7ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff6780d2ae0 con 0x7ff67815e570 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.486+0000 7ff6767fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff6780b3130 0x7ff6780d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32878/0 (socket says 172.21.15.26:32878) 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.486+0000 7ff6767fc640 1 -- 172.21.15.26:0/1296100743 learned_addr learned my addr 172.21.15.26:0/1296100743 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.486+0000 7ff676ffd640 1 --2- 172.21.15.26:0/1296100743 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff67815e570 0x7ff6780d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.486+0000 7ff6777fe640 1 --2- 172.21.15.26:0/1296100743 >> v2:172.21.15.29:3300/0 conn(0x7ff6780d22e0 0x7ff6780d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.487+0000 7ff675ffb640 1 -- 172.21.15.26:0/1296100743 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2413652502 0 0) 0x7ff66c001110 con 0x7ff6780b3130 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.487+0000 7ff675ffb640 1 -- 172.21.15.26:0/1296100743 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff660003520 con 0x7ff6780b3130 2024-04-24T23:25:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.487+0000 7ff6777fe640 1 -- 172.21.15.26:0/1296100743 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff67815e570 msgr2=0x7ff6780d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.487+0000 7ff6777fe640 1 --2- 172.21.15.26:0/1296100743 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff67815e570 0x7ff6780d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.487+0000 7ff675ffb640 1 -- 172.21.15.26:0/1296100743 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 801827151 0 0) 0x7ff660003520 con 0x7ff6780b3130 2024-04-24T23:25:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.487+0000 7ff6777fe640 1 -- 172.21.15.26:0/1296100743 >> v1:172.21.15.26:6789/0 conn(0x7ff6780b3130 legacy=0x7ff6780d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.487+0000 7ff6777fe640 1 -- 172.21.15.26:0/1296100743 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6780d0270 con 0x7ff6780d22e0 2024-04-24T23:25:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.487+0000 7ff676ffd640 1 --2- 172.21.15.26:0/1296100743 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff67815e570 0x7ff6780d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff6777fe640 1 --2- 172.21.15.26:0/1296100743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6780d22e0 0x7ff6780d46a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff668009d90 tx=0x7ff66802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=414adef72fbe21f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff675ffb640 1 -- 172.21.15.26:0/1296100743 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c001840 con 0x7ff6780d22e0 2024-04-24T23:25:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff675ffb640 1 -- 172.21.15.26:0/1296100743 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff668007540 con 0x7ff6780d22e0 2024-04-24T23:25:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff675ffb640 1 -- 172.21.15.26:0/1296100743 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668007910 con 0x7ff6780d22e0 2024-04-24T23:25:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1296100743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6780d22e0 msgr2=0x7ff6780d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff67d7ce640 1 --2- 172.21.15.26:0/1296100743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6780d22e0 0x7ff6780d46a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff668009d90 tx=0x7ff66802a5f0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1296100743 shutdown_connections 2024-04-24T23:25:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff67d7ce640 1 --2- 172.21.15.26:0/1296100743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6780d22e0 0x7ff6780d46a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff67d7ce640 1 --2- 172.21.15.26:0/1296100743 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff67815e570 0x7ff6780d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1296100743 >> 172.21.15.26:0/1296100743 conn(0x7ff67808dbe0 msgr2=0x7ff67809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1296100743 shutdown_connections 2024-04-24T23:25:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.488+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1296100743 wait complete. 2024-04-24T23:25:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.490+0000 7ff67d7ce640 1 Processor -- start 2024-04-24T23:25:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.490+0000 7ff67d7ce640 1 -- start start 2024-04-24T23:25:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.490+0000 7ff67d7ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6780b3130 0x7ff6781516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.490+0000 7ff67d7ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6780d22e0 0x7ff6780b9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.490+0000 7ff67d7ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff67815e570 0x7ff6780b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.490+0000 7ff67d7ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff6780cfa70 con 0x7ff6780d22e0 2024-04-24T23:25:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.490+0000 7ff67d7ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff6780d3010 con 0x7ff67815e570 2024-04-24T23:25:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.490+0000 7ff67d7ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff6780cfd50 con 0x7ff6780b3130 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6767fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6780d22e0 0x7ff6780b9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6767fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6780d22e0 0x7ff6780b9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59058/0 (socket says 172.21.15.26:59058) 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6767fc640 1 -- 172.21.15.26:0/1130863317 learned_addr learned my addr 172.21.15.26:0/1130863317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6777fe640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff67815e570 0x7ff6780b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff676ffd640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6780b3130 0x7ff6781516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6767fc640 1 -- 172.21.15.26:0/1130863317 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6780b3130 msgr2=0x7ff6781516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6767fc640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6780b3130 0x7ff6781516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6767fc640 1 -- 172.21.15.26:0/1130863317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff67815e570 msgr2=0x7ff6780b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6767fc640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff67815e570 0x7ff6780b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6767fc640 1 -- 172.21.15.26:0/1130863317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff678151c10 con 0x7ff6780d22e0 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6777fe640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff67815e570 0x7ff6780b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff6767fc640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6780d22e0 0x7ff6780b9830 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff66c001eb0 tx=0x7ff66c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:32.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff65f7fe640 1 -- 172.21.15.26:0/1130863317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c017070 con 0x7ff6780d22e0 2024-04-24T23:25:32.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff65f7fe640 1 -- 172.21.15.26:0/1130863317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66c005ad0 con 0x7ff6780d22e0 2024-04-24T23:25:32.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6781f7f10 con 0x7ff6780d22e0 2024-04-24T23:25:32.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff65f7fe640 1 -- 172.21.15.26:0/1130863317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c007070 con 0x7ff6780d22e0 2024-04-24T23:25:32.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.491+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6781f84d0 con 0x7ff6780d22e0 2024-04-24T23:25:32.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.492+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6780694c0 con 0x7ff6780d22e0 2024-04-24T23:25:32.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.493+0000 7ff65f7fe640 1 -- 172.21.15.26:0/1130863317 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff66c01a070 con 0x7ff6780d22e0 2024-04-24T23:25:32.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.493+0000 7ff65f7fe640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff66406be60 0x7ff66406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.493+0000 7ff65f7fe640 1 -- 172.21.15.26:0/1130863317 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7ff66c08a2d0 con 0x7ff6780d22e0 2024-04-24T23:25:32.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.493+0000 7ff676ffd640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff66406be60 0x7ff66406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.494+0000 7ff676ffd640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff66406be60 0x7ff66406e320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff670009a30 tx=0x7ff670008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:32.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.498+0000 7ff65f7fe640 1 -- 172.21.15.26:0/1130863317 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff6780d32f0 con 0x7ff6780d22e0 2024-04-24T23:25:32.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.676+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6781f7cc0 con 0x7ff66406be60 2024-04-24T23:25:32.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.682+0000 7ff65f7fe640 1 -- 172.21.15.26:0/1130863317 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246752 (secure 0 0 0) 0x7ff6781f7cc0 con 0x7ff66406be60 2024-04-24T23:25:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff66406be60 msgr2=0x7ff66406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff66406be60 0x7ff66406e320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff670009a30 tx=0x7ff670008040 comp rx=0 tx=0).stop 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6780d22e0 msgr2=0x7ff6780b9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6780d22e0 0x7ff6780b9830 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff66c001eb0 tx=0x7ff66c00cb90 comp rx=0 tx=0).stop 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 shutdown_connections 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff66406be60 0x7ff66406e320 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff67815e570 0x7ff6780b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6780d22e0 0x7ff6780b9830 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 --2- 172.21.15.26:0/1130863317 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6780b3130 0x7ff6781516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.685+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 >> 172.21.15.26:0/1130863317 conn(0x7ff67808dbe0 msgr2=0x7ff67815d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.686+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 shutdown_connections 2024-04-24T23:25:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.686+0000 7ff67d7ce640 1 -- 172.21.15.26:0/1130863317 wait complete. 2024-04-24T23:25:32.697 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-04-24T23:25:31.376080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265256,"kb_used_data":7480,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754709464,"statfs":{"total":773094113280,"available":772822491136,"internally_reserved":0,"allocated":7659520,"data_stored":3908484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.413779"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972124+0000","last_change":"2024-04-24T23:25:26.972124+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972124+0000","last_undegraded":"2024-04-24T23:25:26.972124+0000","last_fullsized":"2024-04-24T23:25:26.972124+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973940+0000","last_change":"2024-04-24T23:25:26.973940+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973940+0000","last_undegraded":"2024-04-24T23:25:26.973940+0000","last_fullsized":"2024-04-24T23:25:26.973940+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971708+0000","last_change":"2024-04-24T23:25:26.971708+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971708+0000","last_undegraded":"2024-04-24T23:25:26.971708+0000","last_fullsized":"2024-04-24T23:25:26.971708+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.977999+0000","last_change":"2024-04-24T23:25:26.977999+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.977999+0000","last_undegraded":"2024-04-24T23:25:26.977999+0000","last_fullsized":"2024-04-24T23:25:26.977999+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974555+0000","last_change":"2024-04-24T23:25:26.974555+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974555+0000","last_undegraded":"2024-04-24T23:25:26.974555+0000","last_fullsized":"2024-04-24T23:25:26.974555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971249+0000","last_change":"2024-04-24T23:25:26.971249+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971249+0000","last_undegraded":"2024-04-24T23:25:26.971249+0000","last_fullsized":"2024-04-24T23:25:26.971249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968229+0000","last_change":"2024-04-24T23:25:26.968229+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968229+0000","last_undegraded":"2024-04-24T23:25:26.968229+0000","last_fullsized":"2024-04-24T23:25:26.968229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.976683+0000","last_change":"2024-04-24T23:25:26.976683+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.976683+0000","last_undegraded":"2024-04-24T23:25:26.976683+0000","last_fullsized":"2024-04-24T23:25:26.976683+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973701+0000","last_change":"2024-04-24T23:25:26.973701+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973701+0000","last_undegraded":"2024-04-24T23:25:26.973701+0000","last_fullsized":"2024-04-24T23:25:26.973701+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975677+0000","last_change":"2024-04-24T23:25:26.975677+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975677+0000","last_undegraded":"2024-04-24T23:25:26.975677+0000","last_fullsized":"2024-04-24T23:25:26.975677+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972646+0000","last_change":"2024-04-24T23:25:26.972646+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972646+0000","last_undegraded":"2024-04-24T23:25:26.972646+0000","last_fullsized":"2024-04-24T23:25:26.972646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.977553+0000","last_change":"2024-04-24T23:25:26.977553+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.977553+0000","last_undegraded":"2024-04-24T23:25:26.977553+0000","last_fullsized":"2024-04-24T23:25:26.977553+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972318+0000","last_change":"2024-04-24T23:25:26.972318+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972318+0000","last_undegraded":"2024-04-24T23:25:26.972318+0000","last_fullsized":"2024-04-24T23:25:26.972318+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973045+0000","last_change":"2024-04-24T23:25:26.973045+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973045+0000","last_undegraded":"2024-04-24T23:25:26.973045+0000","last_fullsized":"2024-04-24T23:25:26.973045+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973018+0000","last_change":"2024-04-24T23:25:26.973018+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973018+0000","last_undegraded":"2024-04-24T23:25:26.973018+0000","last_fullsized":"2024-04-24T23:25:26.973018+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975110+0000","last_change":"2024-04-24T23:25:26.975110+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975110+0000","last_undegraded":"2024-04-24T23:25:26.975110+0000","last_fullsized":"2024-04-24T23:25:26.975110+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973033+0000","last_change":"2024-04-24T23:25:26.973033+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973033+0000","last_undegraded":"2024-04-24T23:25:26.973033+0000","last_fullsized":"2024-04-24T23:25:26.973033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975740+0000","last_change":"2024-04-24T23:25:26.975740+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975740+0000","last_undegraded":"2024-04-24T23:25:26.975740+0000","last_fullsized":"2024-04-24T23:25:26.975740+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972531+0000","last_change":"2024-04-24T23:25:26.972531+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972531+0000","last_undegraded":"2024-04-24T23:25:26.972531+0000","last_fullsized":"2024-04-24T23:25:26.972531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972789+0000","last_change":"2024-04-24T23:25:26.972789+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972789+0000","last_undegraded":"2024-04-24T23:25:26.972789+0000","last_fullsized":"2024-04-24T23:25:26.972789+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975257+0000","last_change":"2024-04-24T23:25:26.975257+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975257+0000","last_undegraded":"2024-04-24T23:25:26.975257+0000","last_fullsized":"2024-04-24T23:25:26.975257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974439+0000","last_change":"2024-04-24T23:25:26.974439+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974439+0000","last_undegraded":"2024-04-24T23:25:26.974439+0000","last_fullsized":"2024-04-24T23:25:26.974439+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970819+0000","last_change":"2024-04-24T23:25:26.970819+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970819+0000","last_undegraded":"2024-04-24T23:25:26.970819+0000","last_fullsized":"2024-04-24T23:25:26.970819+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971038+0000","last_change":"2024-04-24T23:25:26.971038+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971038+0000","last_undegraded":"2024-04-24T23:25:26.971038+0000","last_fullsized":"2024-04-24T23:25:26.971038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.973311+0000","last_change":"2024-04-24T23:25:26.973311+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.973311+0000","last_undegraded":"2024-04-24T23:25:26.973311+0000","last_fullsized":"2024-04-24T23:25:26.973311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968913+0000","last_change":"2024-04-24T23:25:26.968913+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968913+0000","last_undegraded":"2024-04-24T23:25:26.968913+0000","last_fullsized":"2024-04-24T23:25:26.968913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971174+0000","last_change":"2024-04-24T23:25:26.971174+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971174+0000","last_undegraded":"2024-04-24T23:25:26.971174+0000","last_fullsized":"2024-04-24T23:25:26.971174+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972444+0000","last_change":"2024-04-24T23:25:26.972444+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972444+0000","last_undegraded":"2024-04-24T23:25:26.972444+0000","last_fullsized":"2024-04-24T23:25:26.972444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971143+0000","last_change":"2024-04-24T23:25:26.971143+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971143+0000","last_undegraded":"2024-04-24T23:25:26.971143+0000","last_fullsized":"2024-04-24T23:25:26.971143+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967626+0000","last_change":"2024-04-24T23:25:26.967626+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967626+0000","last_undegraded":"2024-04-24T23:25:26.967626+0000","last_fullsized":"2024-04-24T23:25:26.967626+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967495+0000","last_change":"2024-04-24T23:25:26.967495+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967495+0000","last_undegraded":"2024-04-24T23:25:26.967495+0000","last_fullsized":"2024-04-24T23:25:26.967495+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.976210+0000","last_change":"2024-04-24T23:25:26.976210+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.976210+0000","last_undegraded":"2024-04-24T23:25:26.976210+0000","last_fullsized":"2024-04-24T23:25:26.976210+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971298+0000","last_change":"2024-04-24T23:25:26.971298+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971298+0000","last_undegraded":"2024-04-24T23:25:26.971298+0000","last_fullsized":"2024-04-24T23:25:26.971298+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985622+0000","last_change":"2024-04-24T23:25:26.984830+0000","last_active":"2024-04-24T23:25:26.985622+0000","last_peered":"2024-04-24T23:25:26.985622+0000","last_clean":"2024-04-24T23:25:26.985622+0000","last_became_active":"2024-04-24T23:25:26.983743+0000","last_became_peered":"2024-04-24T23:25:26.983743+0000","last_unstale":"2024-04-24T23:25:26.985622+0000","last_undegraded":"2024-04-24T23:25:26.985622+0000","last_fullsized":"2024-04-24T23:25:26.985622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:01.501491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972060+0000","last_change":"2024-04-24T23:25:26.972060+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972060+0000","last_undegraded":"2024-04-24T23:25:26.972060+0000","last_fullsized":"2024-04-24T23:25:26.972060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.988350+0000","last_change":"2024-04-24T23:25:26.986600+0000","last_active":"2024-04-24T23:25:26.988350+0000","last_peered":"2024-04-24T23:25:26.988350+0000","last_clean":"2024-04-24T23:25:26.988350+0000","last_became_active":"2024-04-24T23:25:26.986208+0000","last_became_peered":"2024-04-24T23:25:26.986208+0000","last_unstale":"2024-04-24T23:25:26.988350+0000","last_undegraded":"2024-04-24T23:25:26.988350+0000","last_fullsized":"2024-04-24T23:25:26.988350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:06.044715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974992+0000","last_change":"2024-04-24T23:25:26.974992+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974992+0000","last_undegraded":"2024-04-24T23:25:26.974992+0000","last_fullsized":"2024-04-24T23:25:26.974992+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.989266+0000","last_change":"2024-04-24T23:25:26.988051+0000","last_active":"2024-04-24T23:25:26.989266+0000","last_peered":"2024-04-24T23:25:26.989266+0000","last_clean":"2024-04-24T23:25:26.989266+0000","last_became_active":"2024-04-24T23:25:26.987663+0000","last_became_peered":"2024-04-24T23:25:26.987663+0000","last_unstale":"2024-04-24T23:25:26.989266+0000","last_undegraded":"2024-04-24T23:25:26.989266+0000","last_fullsized":"2024-04-24T23:25:26.989266+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:11.588401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972006+0000","last_change":"2024-04-24T23:25:26.972006+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972006+0000","last_undegraded":"2024-04-24T23:25:26.972006+0000","last_fullsized":"2024-04-24T23:25:26.972006+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987433+0000","last_change":"2024-04-24T23:25:26.986624+0000","last_active":"2024-04-24T23:25:26.987433+0000","last_peered":"2024-04-24T23:25:26.987433+0000","last_clean":"2024-04-24T23:25:26.987433+0000","last_became_active":"2024-04-24T23:25:26.986323+0000","last_became_peered":"2024-04-24T23:25:26.986323+0000","last_unstale":"2024-04-24T23:25:26.987433+0000","last_undegraded":"2024-04-24T23:25:26.987433+0000","last_fullsized":"2024-04-24T23:25:26.987433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:11.677239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967001+0000","last_change":"2024-04-24T23:25:26.967001+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967001+0000","last_undegraded":"2024-04-24T23:25:26.967001+0000","last_fullsized":"2024-04-24T23:25:26.967001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.978242+0000","last_change":"2024-04-24T23:25:26.977218+0000","last_active":"2024-04-24T23:25:26.978242+0000","last_peered":"2024-04-24T23:25:26.978242+0000","last_clean":"2024-04-24T23:25:26.978242+0000","last_became_active":"2024-04-24T23:25:26.976913+0000","last_became_peered":"2024-04-24T23:25:26.976913+0000","last_unstale":"2024-04-24T23:25:26.978242+0000","last_undegraded":"2024-04-24T23:25:26.978242+0000","last_fullsized":"2024-04-24T23:25:26.978242+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:15.039445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972886+0000","last_change":"2024-04-24T23:25:26.972886+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972886+0000","last_undegraded":"2024-04-24T23:25:26.972886+0000","last_fullsized":"2024-04-24T23:25:26.972886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986090+0000","last_change":"2024-04-24T23:25:26.984947+0000","last_active":"2024-04-24T23:25:26.986090+0000","last_peered":"2024-04-24T23:25:26.986090+0000","last_clean":"2024-04-24T23:25:26.986090+0000","last_became_active":"2024-04-24T23:25:26.984604+0000","last_became_peered":"2024-04-24T23:25:26.984604+0000","last_unstale":"2024-04-24T23:25:26.986090+0000","last_undegraded":"2024-04-24T23:25:26.986090+0000","last_fullsized":"2024-04-24T23:25:26.986090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:27.118553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972021+0000","last_change":"2024-04-24T23:25:26.972021+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972021+0000","last_undegraded":"2024-04-24T23:25:26.972021+0000","last_fullsized":"2024-04-24T23:25:26.972021+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985711+0000","last_change":"2024-04-24T23:25:26.982771+0000","last_active":"2024-04-24T23:25:26.985711+0000","last_peered":"2024-04-24T23:25:26.985711+0000","last_clean":"2024-04-24T23:25:26.985711+0000","last_became_active":"2024-04-24T23:25:26.982504+0000","last_became_peered":"2024-04-24T23:25:26.982504+0000","last_unstale":"2024-04-24T23:25:26.985711+0000","last_undegraded":"2024-04-24T23:25:26.985711+0000","last_fullsized":"2024-04-24T23:25:26.985711+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:10.968125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972021+0000","last_change":"2024-04-24T23:25:26.972021+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972021+0000","last_undegraded":"2024-04-24T23:25:26.972021+0000","last_fullsized":"2024-04-24T23:25:26.972021+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987666+0000","last_change":"2024-04-24T23:25:26.985502+0000","last_active":"2024-04-24T23:25:26.987666+0000","last_peered":"2024-04-24T23:25:26.987666+0000","last_clean":"2024-04-24T23:25:26.987666+0000","last_became_active":"2024-04-24T23:25:26.985236+0000","last_became_peered":"2024-04-24T23:25:26.985236+0000","last_unstale":"2024-04-24T23:25:26.987666+0000","last_undegraded":"2024-04-24T23:25:26.987666+0000","last_fullsized":"2024-04-24T23:25:26.987666+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:42.270581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968067+0000","last_change":"2024-04-24T23:25:26.968067+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968067+0000","last_undegraded":"2024-04-24T23:25:26.968067+0000","last_fullsized":"2024-04-24T23:25:26.968067+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.978235+0000","last_change":"2024-04-24T23:25:26.976690+0000","last_active":"2024-04-24T23:25:26.978235+0000","last_peered":"2024-04-24T23:25:26.978235+0000","last_clean":"2024-04-24T23:25:26.978235+0000","last_became_active":"2024-04-24T23:25:26.976221+0000","last_became_peered":"2024-04-24T23:25:26.976221+0000","last_unstale":"2024-04-24T23:25:26.978235+0000","last_undegraded":"2024-04-24T23:25:26.978235+0000","last_fullsized":"2024-04-24T23:25:26.978235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:26.544193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971698+0000","last_change":"2024-04-24T23:25:26.971698+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971698+0000","last_undegraded":"2024-04-24T23:25:26.971698+0000","last_fullsized":"2024-04-24T23:25:26.971698+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.984328+0000","last_change":"2024-04-24T23:25:26.979704+0000","last_active":"2024-04-24T23:25:26.984328+0000","last_peered":"2024-04-24T23:25:26.984328+0000","last_clean":"2024-04-24T23:25:26.984328+0000","last_became_active":"2024-04-24T23:25:26.979294+0000","last_became_peered":"2024-04-24T23:25:26.979294+0000","last_unstale":"2024-04-24T23:25:26.984328+0000","last_undegraded":"2024-04-24T23:25:26.984328+0000","last_fullsized":"2024-04-24T23:25:26.984328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:59.557263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.974519+0000","last_change":"2024-04-24T23:25:26.974519+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.974519+0000","last_undegraded":"2024-04-24T23:25:26.974519+0000","last_fullsized":"2024-04-24T23:25:26.974519+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.974980+0000","last_change":"2024-04-24T23:25:26.973701+0000","last_active":"2024-04-24T23:25:26.974980+0000","last_peered":"2024-04-24T23:25:26.974980+0000","last_clean":"2024-04-24T23:25:26.974980+0000","last_became_active":"2024-04-24T23:25:26.973424+0000","last_became_peered":"2024-04-24T23:25:26.973424+0000","last_unstale":"2024-04-24T23:25:26.974980+0000","last_undegraded":"2024-04-24T23:25:26.974980+0000","last_fullsized":"2024-04-24T23:25:26.974980+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:54.920525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972192+0000","last_change":"2024-04-24T23:25:26.972192+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972192+0000","last_undegraded":"2024-04-24T23:25:26.972192+0000","last_fullsized":"2024-04-24T23:25:26.972192+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987465+0000","last_change":"2024-04-24T23:25:26.985049+0000","last_active":"2024-04-24T23:25:26.987465+0000","last_peered":"2024-04-24T23:25:26.987465+0000","last_clean":"2024-04-24T23:25:26.987465+0000","last_became_active":"2024-04-24T23:25:26.984293+0000","last_became_peered":"2024-04-24T23:25:26.984293+0000","last_unstale":"2024-04-24T23:25:26.987465+0000","last_undegraded":"2024-04-24T23:25:26.987465+0000","last_fullsized":"2024-04-24T23:25:26.987465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:12.258864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970929+0000","last_change":"2024-04-24T23:25:26.970929+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970929+0000","last_undegraded":"2024-04-24T23:25:26.970929+0000","last_fullsized":"2024-04-24T23:25:26.970929+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986717+0000","last_change":"2024-04-24T23:25:26.982431+0000","last_active":"2024-04-24T23:25:26.986717+0000","last_peered":"2024-04-24T23:25:26.986717+0000","last_clean":"2024-04-24T23:25:26.986717+0000","last_became_active":"2024-04-24T23:25:26.982147+0000","last_became_peered":"2024-04-24T23:25:26.982147+0000","last_unstale":"2024-04-24T23:25:26.986717+0000","last_undegraded":"2024-04-24T23:25:26.986717+0000","last_fullsized":"2024-04-24T23:25:26.986717+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:59.723643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.967531+0000","last_change":"2024-04-24T23:25:26.967531+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.967531+0000","last_undegraded":"2024-04-24T23:25:26.967531+0000","last_fullsized":"2024-04-24T23:25:26.967531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985013+0000","last_change":"2024-04-24T23:25:26.979601+0000","last_active":"2024-04-24T23:25:26.985013+0000","last_peered":"2024-04-24T23:25:26.985013+0000","last_clean":"2024-04-24T23:25:26.985013+0000","last_became_active":"2024-04-24T23:25:26.978063+0000","last_became_peered":"2024-04-24T23:25:26.978063+0000","last_unstale":"2024-04-24T23:25:26.985013+0000","last_undegraded":"2024-04-24T23:25:26.985013+0000","last_fullsized":"2024-04-24T23:25:26.985013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:32.655714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972297+0000","last_change":"2024-04-24T23:25:26.972297+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972297+0000","last_undegraded":"2024-04-24T23:25:26.972297+0000","last_fullsized":"2024-04-24T23:25:26.972297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.981043+0000","last_change":"2024-04-24T23:25:26.979978+0000","last_active":"2024-04-24T23:25:26.981043+0000","last_peered":"2024-04-24T23:25:26.981043+0000","last_clean":"2024-04-24T23:25:26.981043+0000","last_became_active":"2024-04-24T23:25:26.978846+0000","last_became_peered":"2024-04-24T23:25:26.978846+0000","last_unstale":"2024-04-24T23:25:26.981043+0000","last_undegraded":"2024-04-24T23:25:26.981043+0000","last_fullsized":"2024-04-24T23:25:26.981043+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:06.044715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971257+0000","last_change":"2024-04-24T23:25:26.971257+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971257+0000","last_undegraded":"2024-04-24T23:25:26.971257+0000","last_fullsized":"2024-04-24T23:25:26.971257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985464+0000","last_change":"2024-04-24T23:25:26.983021+0000","last_active":"2024-04-24T23:25:26.985464+0000","last_peered":"2024-04-24T23:25:26.985464+0000","last_clean":"2024-04-24T23:25:26.985464+0000","last_became_active":"2024-04-24T23:25:26.982757+0000","last_became_peered":"2024-04-24T23:25:26.982757+0000","last_unstale":"2024-04-24T23:25:26.985464+0000","last_undegraded":"2024-04-24T23:25:26.985464+0000","last_fullsized":"2024-04-24T23:25:26.985464+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:28.905967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.972173+0000","last_change":"2024-04-24T23:25:26.972173+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.972173+0000","last_undegraded":"2024-04-24T23:25:26.972173+0000","last_fullsized":"2024-04-24T23:25:26.972173+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985354+0000","last_change":"2024-04-24T23:25:26.984211+0000","last_active":"2024-04-24T23:25:26.985354+0000","last_peered":"2024-04-24T23:25:26.985354+0000","last_clean":"2024-04-24T23:25:26.985354+0000","last_became_active":"2024-04-24T23:25:26.983878+0000","last_became_peered":"2024-04-24T23:25:26.983878+0000","last_unstale":"2024-04-24T23:25:26.985354+0000","last_undegraded":"2024-04-24T23:25:26.985354+0000","last_fullsized":"2024-04-24T23:25:26.985354+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:39.951844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971011+0000","last_change":"2024-04-24T23:25:26.971011+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971011+0000","last_undegraded":"2024-04-24T23:25:26.971011+0000","last_fullsized":"2024-04-24T23:25:26.971011+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.989233+0000","last_change":"2024-04-24T23:25:26.988180+0000","last_active":"2024-04-24T23:25:26.989233+0000","last_peered":"2024-04-24T23:25:26.989233+0000","last_clean":"2024-04-24T23:25:26.989233+0000","last_became_active":"2024-04-24T23:25:26.987706+0000","last_became_peered":"2024-04-24T23:25:26.987706+0000","last_unstale":"2024-04-24T23:25:26.989233+0000","last_undegraded":"2024-04-24T23:25:26.989233+0000","last_fullsized":"2024-04-24T23:25:26.989233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:04.387670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.977087+0000","last_change":"2024-04-24T23:25:26.977087+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.977087+0000","last_undegraded":"2024-04-24T23:25:26.977087+0000","last_fullsized":"2024-04-24T23:25:26.977087+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.990338+0000","last_change":"2024-04-24T23:25:26.989891+0000","last_active":"2024-04-24T23:25:26.990338+0000","last_peered":"2024-04-24T23:25:26.990338+0000","last_clean":"2024-04-24T23:25:26.990338+0000","last_became_active":"2024-04-24T23:25:26.989524+0000","last_became_peered":"2024-04-24T23:25:26.989524+0000","last_unstale":"2024-04-24T23:25:26.990338+0000","last_undegraded":"2024-04-24T23:25:26.990338+0000","last_fullsized":"2024-04-24T23:25:26.990338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:44.608576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971175+0000","last_change":"2024-04-24T23:25:26.971175+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971175+0000","last_undegraded":"2024-04-24T23:25:26.971175+0000","last_fullsized":"2024-04-24T23:25:26.971175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.982172+0000","last_change":"2024-04-24T23:25:26.980697+0000","last_active":"2024-04-24T23:25:26.982172+0000","last_peered":"2024-04-24T23:25:26.982172+0000","last_clean":"2024-04-24T23:25:26.982172+0000","last_became_active":"2024-04-24T23:25:26.980315+0000","last_became_peered":"2024-04-24T23:25:26.980315+0000","last_unstale":"2024-04-24T23:25:26.982172+0000","last_undegraded":"2024-04-24T23:25:26.982172+0000","last_fullsized":"2024-04-24T23:25:26.982172+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:43.615264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970605+0000","last_change":"2024-04-24T23:25:26.970605+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970605+0000","last_undegraded":"2024-04-24T23:25:26.970605+0000","last_fullsized":"2024-04-24T23:25:26.970605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986068+0000","last_change":"2024-04-24T23:25:26.984839+0000","last_active":"2024-04-24T23:25:26.986068+0000","last_peered":"2024-04-24T23:25:26.986068+0000","last_clean":"2024-04-24T23:25:26.986068+0000","last_became_active":"2024-04-24T23:25:26.984489+0000","last_became_peered":"2024-04-24T23:25:26.984489+0000","last_unstale":"2024-04-24T23:25:26.986068+0000","last_undegraded":"2024-04-24T23:25:26.986068+0000","last_fullsized":"2024-04-24T23:25:26.986068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:50.577760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.969773+0000","last_change":"2024-04-24T23:25:26.969773+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.969773+0000","last_undegraded":"2024-04-24T23:25:26.969773+0000","last_fullsized":"2024-04-24T23:25:26.969773+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.976875+0000","last_change":"2024-04-24T23:25:26.975028+0000","last_active":"2024-04-24T23:25:26.976875+0000","last_peered":"2024-04-24T23:25:26.976875+0000","last_clean":"2024-04-24T23:25:26.976875+0000","last_became_active":"2024-04-24T23:25:26.974741+0000","last_became_peered":"2024-04-24T23:25:26.974741+0000","last_unstale":"2024-04-24T23:25:26.976875+0000","last_undegraded":"2024-04-24T23:25:26.976875+0000","last_fullsized":"2024-04-24T23:25:26.976875+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:51.782921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970234+0000","last_change":"2024-04-24T23:25:26.970234+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970234+0000","last_undegraded":"2024-04-24T23:25:26.970234+0000","last_fullsized":"2024-04-24T23:25:26.970234+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.988362+0000","last_change":"2024-04-24T23:25:26.986864+0000","last_active":"2024-04-24T23:25:26.988362+0000","last_peered":"2024-04-24T23:25:26.988362+0000","last_clean":"2024-04-24T23:25:26.988362+0000","last_became_active":"2024-04-24T23:25:26.986575+0000","last_became_peered":"2024-04-24T23:25:26.986575+0000","last_unstale":"2024-04-24T23:25:26.988362+0000","last_undegraded":"2024-04-24T23:25:26.988362+0000","last_fullsized":"2024-04-24T23:25:26.988362+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:57.895459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975340+0000","last_change":"2024-04-24T23:25:26.975340+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975340+0000","last_undegraded":"2024-04-24T23:25:26.975340+0000","last_fullsized":"2024-04-24T23:25:26.975340+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.984885+0000","last_change":"2024-04-24T23:25:26.983803+0000","last_active":"2024-04-24T23:25:26.984885+0000","last_peered":"2024-04-24T23:25:26.984885+0000","last_clean":"2024-04-24T23:25:26.984885+0000","last_became_active":"2024-04-24T23:25:26.983465+0000","last_became_peered":"2024-04-24T23:25:26.983465+0000","last_unstale":"2024-04-24T23:25:26.984885+0000","last_undegraded":"2024-04-24T23:25:26.984885+0000","last_fullsized":"2024-04-24T23:25:26.984885+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:48.673865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.968080+0000","last_change":"2024-04-24T23:25:26.968080+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.968080+0000","last_undegraded":"2024-04-24T23:25:26.968080+0000","last_fullsized":"2024-04-24T23:25:26.968080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.981884+0000","last_change":"2024-04-24T23:25:26.980977+0000","last_active":"2024-04-24T23:25:26.981884+0000","last_peered":"2024-04-24T23:25:26.981884+0000","last_clean":"2024-04-24T23:25:26.981884+0000","last_became_active":"2024-04-24T23:25:26.979025+0000","last_became_peered":"2024-04-24T23:25:26.979025+0000","last_unstale":"2024-04-24T23:25:26.981884+0000","last_undegraded":"2024-04-24T23:25:26.981884+0000","last_fullsized":"2024-04-24T23:25:26.981884+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:57.895459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971378+0000","last_change":"2024-04-24T23:25:26.971378+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971378+0000","last_undegraded":"2024-04-24T23:25:26.971378+0000","last_fullsized":"2024-04-24T23:25:26.971378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.985813+0000","last_change":"2024-04-24T23:25:26.984615+0000","last_active":"2024-04-24T23:25:26.985813+0000","last_peered":"2024-04-24T23:25:26.985813+0000","last_clean":"2024-04-24T23:25:26.985813+0000","last_became_active":"2024-04-24T23:25:26.984358+0000","last_became_peered":"2024-04-24T23:25:26.984358+0000","last_unstale":"2024-04-24T23:25:26.985813+0000","last_undegraded":"2024-04-24T23:25:26.985813+0000","last_fullsized":"2024-04-24T23:25:26.985813+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:05.698635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.971399+0000","last_change":"2024-04-24T23:25:26.971399+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.971399+0000","last_undegraded":"2024-04-24T23:25:26.971399+0000","last_fullsized":"2024-04-24T23:25:26.971399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.988937+0000","last_change":"2024-04-24T23:25:26.988155+0000","last_active":"2024-04-24T23:25:26.988937+0000","last_peered":"2024-04-24T23:25:26.988937+0000","last_clean":"2024-04-24T23:25:26.988937+0000","last_became_active":"2024-04-24T23:25:26.987824+0000","last_became_peered":"2024-04-24T23:25:26.987824+0000","last_unstale":"2024-04-24T23:25:26.988937+0000","last_undegraded":"2024-04-24T23:25:26.988937+0000","last_fullsized":"2024-04-24T23:25:26.988937+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:42.850369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.969781+0000","last_change":"2024-04-24T23:25:26.969781+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.969781+0000","last_undegraded":"2024-04-24T23:25:26.969781+0000","last_fullsized":"2024-04-24T23:25:26.969781+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.979989+0000","last_change":"2024-04-24T23:25:26.976419+0000","last_active":"2024-04-24T23:25:26.979989+0000","last_peered":"2024-04-24T23:25:26.979989+0000","last_clean":"2024-04-24T23:25:26.979989+0000","last_became_active":"2024-04-24T23:25:26.976109+0000","last_became_peered":"2024-04-24T23:25:26.976109+0000","last_unstale":"2024-04-24T23:25:26.979989+0000","last_undegraded":"2024-04-24T23:25:26.979989+0000","last_fullsized":"2024-04-24T23:25:26.979989+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:19.806058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.978803+0000","last_change":"2024-04-24T23:25:26.978803+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.978803+0000","last_undegraded":"2024-04-24T23:25:26.978803+0000","last_fullsized":"2024-04-24T23:25:26.978803+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986234+0000","last_change":"2024-04-24T23:25:26.985538+0000","last_active":"2024-04-24T23:25:26.986234+0000","last_peered":"2024-04-24T23:25:26.986234+0000","last_clean":"2024-04-24T23:25:26.986234+0000","last_became_active":"2024-04-24T23:25:26.985216+0000","last_became_peered":"2024-04-24T23:25:26.985216+0000","last_unstale":"2024-04-24T23:25:26.986234+0000","last_undegraded":"2024-04-24T23:25:26.986234+0000","last_fullsized":"2024-04-24T23:25:26.986234+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:15.381258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.975739+0000","last_change":"2024-04-24T23:25:26.975739+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.975739+0000","last_undegraded":"2024-04-24T23:25:26.975739+0000","last_fullsized":"2024-04-24T23:25:26.975739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.979824+0000","last_change":"2024-04-24T23:25:26.978070+0000","last_active":"2024-04-24T23:25:26.979824+0000","last_peered":"2024-04-24T23:25:26.979824+0000","last_clean":"2024-04-24T23:25:26.979824+0000","last_became_active":"2024-04-24T23:25:26.977796+0000","last_became_peered":"2024-04-24T23:25:26.977796+0000","last_unstale":"2024-04-24T23:25:26.979824+0000","last_undegraded":"2024-04-24T23:25:26.979824+0000","last_fullsized":"2024-04-24T23:25:26.979824+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:27.254375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970188+0000","last_change":"2024-04-24T23:25:26.970188+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970188+0000","last_undegraded":"2024-04-24T23:25:26.970188+0000","last_fullsized":"2024-04-24T23:25:26.970188+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.986754+0000","last_change":"2024-04-24T23:25:26.985857+0000","last_active":"2024-04-24T23:25:26.986754+0000","last_peered":"2024-04-24T23:25:26.986754+0000","last_clean":"2024-04-24T23:25:26.986754+0000","last_became_active":"2024-04-24T23:25:26.985337+0000","last_became_peered":"2024-04-24T23:25:26.985337+0000","last_unstale":"2024-04-24T23:25:26.986754+0000","last_undegraded":"2024-04-24T23:25:26.986754+0000","last_fullsized":"2024-04-24T23:25:26.986754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:17.024902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-24T23:25:26.970226+0000","last_change":"2024-04-24T23:25:26.970226+0000","last_active":"2024-04-24T23:25:26.961783+0000","last_peered":"2024-04-24T23:25:26.961783+0000","last_clean":"2024-04-24T23:25:26.961783+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:25:26.970226+0000","last_undegraded":"2024-04-24T23:25:26.970226+0000","last_fullsized":"2024-04-24T23:25:26.970226+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.987375+0000","last_change":"2024-04-24T23:25:26.986149+0000","last_active":"2024-04-24T23:25:26.987375+0000","last_peered":"2024-04-24T23:25:26.987375+0000","last_clean":"2024-04-24T23:25:26.987375+0000","last_became_active":"2024-04-24T23:25:26.985763+0000","last_became_peered":"2024-04-24T23:25:26.985763+0000","last_unstale":"2024-04-24T23:25:26.987375+0000","last_undegraded":"2024-04-24T23:25:26.987375+0000","last_fullsized":"2024-04-24T23:25:26.987375+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:06.469168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":254,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-24T23:25:26.973212+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:25:26.973212+0000","last_peered":"2024-04-24T23:25:26.973212+0000","last_clean":"2024-04-24T23:25:26.973212+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:25:26.973212+0000","last_undegraded":"2024-04-24T23:25:26.973212+0000","last_fullsized":"2024-04-24T23:25:26.973212+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":928199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":337831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":344106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":344106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":337831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":337831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":344106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36720,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335120,"statfs":{"total":96636764160,"available":96599162880,"internally_reserved":0,"allocated":1409024,"data_stored":934474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:25:32.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-04-24T23:25:32.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972124+0000', 'last_change': '2024-04-24T23:25:26.972124+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972124+0000', 'last_undegraded': '2024-04-24T23:25:26.972124+0000', 'last_fullsized': '2024-04-24T23:25:26.972124+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-04-24T23:25:32.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.973940+0000', 'last_change': '2024-04-24T23:25:26.973940+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.973940+0000', 'last_undegraded': '2024-04-24T23:25:26.973940+0000', 'last_fullsized': '2024-04-24T23:25:26.973940+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:25:32.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971708+0000', 'last_change': '2024-04-24T23:25:26.971708+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971708+0000', 'last_undegraded': '2024-04-24T23:25:26.971708+0000', 'last_fullsized': '2024-04-24T23:25:26.971708+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.977999+0000', 'last_change': '2024-04-24T23:25:26.977999+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.977999+0000', 'last_undegraded': '2024-04-24T23:25:26.977999+0000', 'last_fullsized': '2024-04-24T23:25:26.977999+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.974555+0000', 'last_change': '2024-04-24T23:25:26.974555+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.974555+0000', 'last_undegraded': '2024-04-24T23:25:26.974555+0000', 'last_fullsized': '2024-04-24T23:25:26.974555+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971249+0000', 'last_change': '2024-04-24T23:25:26.971249+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971249+0000', 'last_undegraded': '2024-04-24T23:25:26.971249+0000', 'last_fullsized': '2024-04-24T23:25:26.971249+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.968229+0000', 'last_change': '2024-04-24T23:25:26.968229+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.968229+0000', 'last_undegraded': '2024-04-24T23:25:26.968229+0000', 'last_fullsized': '2024-04-24T23:25:26.968229+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.976683+0000', 'last_change': '2024-04-24T23:25:26.976683+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.976683+0000', 'last_undegraded': '2024-04-24T23:25:26.976683+0000', 'last_fullsized': '2024-04-24T23:25:26.976683+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:25:32.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.973701+0000', 'last_change': '2024-04-24T23:25:26.973701+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.973701+0000', 'last_undegraded': '2024-04-24T23:25:26.973701+0000', 'last_fullsized': '2024-04-24T23:25:26.973701+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.975677+0000', 'last_change': '2024-04-24T23:25:26.975677+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.975677+0000', 'last_undegraded': '2024-04-24T23:25:26.975677+0000', 'last_fullsized': '2024-04-24T23:25:26.975677+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972646+0000', 'last_change': '2024-04-24T23:25:26.972646+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972646+0000', 'last_undegraded': '2024-04-24T23:25:26.972646+0000', 'last_fullsized': '2024-04-24T23:25:26.972646+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.977553+0000', 'last_change': '2024-04-24T23:25:26.977553+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.977553+0000', 'last_undegraded': '2024-04-24T23:25:26.977553+0000', 'last_fullsized': '2024-04-24T23:25:26.977553+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972318+0000', 'last_change': '2024-04-24T23:25:26.972318+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972318+0000', 'last_undegraded': '2024-04-24T23:25:26.972318+0000', 'last_fullsized': '2024-04-24T23:25:26.972318+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.973045+0000', 'last_change': '2024-04-24T23:25:26.973045+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.973045+0000', 'last_undegraded': '2024-04-24T23:25:26.973045+0000', 'last_fullsized': '2024-04-24T23:25:26.973045+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.973018+0000', 'last_change': '2024-04-24T23:25:26.973018+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.973018+0000', 'last_undegraded': '2024-04-24T23:25:26.973018+0000', 'last_fullsized': '2024-04-24T23:25:26.973018+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.975110+0000', 'last_change': '2024-04-24T23:25:26.975110+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.975110+0000', 'last_undegraded': '2024-04-24T23:25:26.975110+0000', 'last_fullsized': '2024-04-24T23:25:26.975110+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.973033+0000', 'last_change': '2024-04-24T23:25:26.973033+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.973033+0000', 'last_undegraded': '2024-04-24T23:25:26.973033+0000', 'last_fullsized': '2024-04-24T23:25:26.973033+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.975740+0000', 'last_change': '2024-04-24T23:25:26.975740+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.975740+0000', 'last_undegraded': '2024-04-24T23:25:26.975740+0000', 'last_fullsized': '2024-04-24T23:25:26.975740+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972531+0000', 'last_change': '2024-04-24T23:25:26.972531+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972531+0000', 'last_undegraded': '2024-04-24T23:25:26.972531+0000', 'last_fullsized': '2024-04-24T23:25:26.972531+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972789+0000', 'last_change': '2024-04-24T23:25:26.972789+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972789+0000', 'last_undegraded': '2024-04-24T23:25:26.972789+0000', 'last_fullsized': '2024-04-24T23:25:26.972789+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-04-24T23:25:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.975257+0000', 'last_change': '2024-04-24T23:25:26.975257+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.975257+0000', 'last_undegraded': '2024-04-24T23:25:26.975257+0000', 'last_fullsized': '2024-04-24T23:25:26.975257+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.974439+0000', 'last_change': '2024-04-24T23:25:26.974439+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.974439+0000', 'last_undegraded': '2024-04-24T23:25:26.974439+0000', 'last_fullsized': '2024-04-24T23:25:26.974439+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.970819+0000', 'last_change': '2024-04-24T23:25:26.970819+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.970819+0000', 'last_undegraded': '2024-04-24T23:25:26.970819+0000', 'last_fullsized': '2024-04-24T23:25:26.970819+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971038+0000', 'last_change': '2024-04-24T23:25:26.971038+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971038+0000', 'last_undegraded': '2024-04-24T23:25:26.971038+0000', 'last_fullsized': '2024-04-24T23:25:26.971038+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.973311+0000', 'last_change': '2024-04-24T23:25:26.973311+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.973311+0000', 'last_undegraded': '2024-04-24T23:25:26.973311+0000', 'last_fullsized': '2024-04-24T23:25:26.973311+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.968913+0000', 'last_change': '2024-04-24T23:25:26.968913+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.968913+0000', 'last_undegraded': '2024-04-24T23:25:26.968913+0000', 'last_fullsized': '2024-04-24T23:25:26.968913+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-04-24T23:25:32.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971174+0000', 'last_change': '2024-04-24T23:25:26.971174+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971174+0000', 'last_undegraded': '2024-04-24T23:25:26.971174+0000', 'last_fullsized': '2024-04-24T23:25:26.971174+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972444+0000', 'last_change': '2024-04-24T23:25:26.972444+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972444+0000', 'last_undegraded': '2024-04-24T23:25:26.972444+0000', 'last_fullsized': '2024-04-24T23:25:26.972444+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971143+0000', 'last_change': '2024-04-24T23:25:26.971143+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971143+0000', 'last_undegraded': '2024-04-24T23:25:26.971143+0000', 'last_fullsized': '2024-04-24T23:25:26.971143+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.967626+0000', 'last_change': '2024-04-24T23:25:26.967626+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.967626+0000', 'last_undegraded': '2024-04-24T23:25:26.967626+0000', 'last_fullsized': '2024-04-24T23:25:26.967626+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.967495+0000', 'last_change': '2024-04-24T23:25:26.967495+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.967495+0000', 'last_undegraded': '2024-04-24T23:25:26.967495+0000', 'last_fullsized': '2024-04-24T23:25:26.967495+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.976210+0000', 'last_change': '2024-04-24T23:25:26.976210+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.976210+0000', 'last_undegraded': '2024-04-24T23:25:26.976210+0000', 'last_fullsized': '2024-04-24T23:25:26.976210+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-04-24T23:25:32.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971298+0000', 'last_change': '2024-04-24T23:25:26.971298+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971298+0000', 'last_undegraded': '2024-04-24T23:25:26.971298+0000', 'last_fullsized': '2024-04-24T23:25:26.971298+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972060+0000', 'last_change': '2024-04-24T23:25:26.972060+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972060+0000', 'last_undegraded': '2024-04-24T23:25:26.972060+0000', 'last_fullsized': '2024-04-24T23:25:26.972060+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.974992+0000', 'last_change': '2024-04-24T23:25:26.974992+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.974992+0000', 'last_undegraded': '2024-04-24T23:25:26.974992+0000', 'last_fullsized': '2024-04-24T23:25:26.974992+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972006+0000', 'last_change': '2024-04-24T23:25:26.972006+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972006+0000', 'last_undegraded': '2024-04-24T23:25:26.972006+0000', 'last_fullsized': '2024-04-24T23:25:26.972006+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.967001+0000', 'last_change': '2024-04-24T23:25:26.967001+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.967001+0000', 'last_undegraded': '2024-04-24T23:25:26.967001+0000', 'last_fullsized': '2024-04-24T23:25:26.967001+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972886+0000', 'last_change': '2024-04-24T23:25:26.972886+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972886+0000', 'last_undegraded': '2024-04-24T23:25:26.972886+0000', 'last_fullsized': '2024-04-24T23:25:26.972886+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972021+0000', 'last_change': '2024-04-24T23:25:26.972021+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972021+0000', 'last_undegraded': '2024-04-24T23:25:26.972021+0000', 'last_fullsized': '2024-04-24T23:25:26.972021+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972021+0000', 'last_change': '2024-04-24T23:25:26.972021+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972021+0000', 'last_undegraded': '2024-04-24T23:25:26.972021+0000', 'last_fullsized': '2024-04-24T23:25:26.972021+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.968067+0000', 'last_change': '2024-04-24T23:25:26.968067+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.968067+0000', 'last_undegraded': '2024-04-24T23:25:26.968067+0000', 'last_fullsized': '2024-04-24T23:25:26.968067+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971698+0000', 'last_change': '2024-04-24T23:25:26.971698+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971698+0000', 'last_undegraded': '2024-04-24T23:25:26.971698+0000', 'last_fullsized': '2024-04-24T23:25:26.971698+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.974519+0000', 'last_change': '2024-04-24T23:25:26.974519+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.974519+0000', 'last_undegraded': '2024-04-24T23:25:26.974519+0000', 'last_fullsized': '2024-04-24T23:25:26.974519+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972192+0000', 'last_change': '2024-04-24T23:25:26.972192+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972192+0000', 'last_undegraded': '2024-04-24T23:25:26.972192+0000', 'last_fullsized': '2024-04-24T23:25:26.972192+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.970929+0000', 'last_change': '2024-04-24T23:25:26.970929+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.970929+0000', 'last_undegraded': '2024-04-24T23:25:26.970929+0000', 'last_fullsized': '2024-04-24T23:25:26.970929+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-04-24T23:25:32.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.967531+0000', 'last_change': '2024-04-24T23:25:26.967531+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.967531+0000', 'last_undegraded': '2024-04-24T23:25:26.967531+0000', 'last_fullsized': '2024-04-24T23:25:26.967531+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972297+0000', 'last_change': '2024-04-24T23:25:26.972297+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972297+0000', 'last_undegraded': '2024-04-24T23:25:26.972297+0000', 'last_fullsized': '2024-04-24T23:25:26.972297+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971257+0000', 'last_change': '2024-04-24T23:25:26.971257+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971257+0000', 'last_undegraded': '2024-04-24T23:25:26.971257+0000', 'last_fullsized': '2024-04-24T23:25:26.971257+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.972173+0000', 'last_change': '2024-04-24T23:25:26.972173+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.972173+0000', 'last_undegraded': '2024-04-24T23:25:26.972173+0000', 'last_fullsized': '2024-04-24T23:25:26.972173+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971011+0000', 'last_change': '2024-04-24T23:25:26.971011+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971011+0000', 'last_undegraded': '2024-04-24T23:25:26.971011+0000', 'last_fullsized': '2024-04-24T23:25:26.971011+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.977087+0000', 'last_change': '2024-04-24T23:25:26.977087+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.977087+0000', 'last_undegraded': '2024-04-24T23:25:26.977087+0000', 'last_fullsized': '2024-04-24T23:25:26.977087+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971175+0000', 'last_change': '2024-04-24T23:25:26.971175+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971175+0000', 'last_undegraded': '2024-04-24T23:25:26.971175+0000', 'last_fullsized': '2024-04-24T23:25:26.971175+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.970605+0000', 'last_change': '2024-04-24T23:25:26.970605+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.970605+0000', 'last_undegraded': '2024-04-24T23:25:26.970605+0000', 'last_fullsized': '2024-04-24T23:25:26.970605+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.969773+0000', 'last_change': '2024-04-24T23:25:26.969773+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.969773+0000', 'last_undegraded': '2024-04-24T23:25:26.969773+0000', 'last_fullsized': '2024-04-24T23:25:26.969773+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.970234+0000', 'last_change': '2024-04-24T23:25:26.970234+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.970234+0000', 'last_undegraded': '2024-04-24T23:25:26.970234+0000', 'last_fullsized': '2024-04-24T23:25:26.970234+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.975340+0000', 'last_change': '2024-04-24T23:25:26.975340+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.975340+0000', 'last_undegraded': '2024-04-24T23:25:26.975340+0000', 'last_fullsized': '2024-04-24T23:25:26.975340+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.968080+0000', 'last_change': '2024-04-24T23:25:26.968080+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.968080+0000', 'last_undegraded': '2024-04-24T23:25:26.968080+0000', 'last_fullsized': '2024-04-24T23:25:26.968080+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971378+0000', 'last_change': '2024-04-24T23:25:26.971378+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971378+0000', 'last_undegraded': '2024-04-24T23:25:26.971378+0000', 'last_fullsized': '2024-04-24T23:25:26.971378+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-04-24T23:25:32.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.971399+0000', 'last_change': '2024-04-24T23:25:26.971399+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.971399+0000', 'last_undegraded': '2024-04-24T23:25:26.971399+0000', 'last_fullsized': '2024-04-24T23:25:26.971399+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.969781+0000', 'last_change': '2024-04-24T23:25:26.969781+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.969781+0000', 'last_undegraded': '2024-04-24T23:25:26.969781+0000', 'last_fullsized': '2024-04-24T23:25:26.969781+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.978803+0000', 'last_change': '2024-04-24T23:25:26.978803+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.978803+0000', 'last_undegraded': '2024-04-24T23:25:26.978803+0000', 'last_fullsized': '2024-04-24T23:25:26.978803+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.975739+0000', 'last_change': '2024-04-24T23:25:26.975739+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.975739+0000', 'last_undegraded': '2024-04-24T23:25:26.975739+0000', 'last_fullsized': '2024-04-24T23:25:26.975739+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.970188+0000', 'last_change': '2024-04-24T23:25:26.970188+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.970188+0000', 'last_undegraded': '2024-04-24T23:25:26.970188+0000', 'last_fullsized': '2024-04-24T23:25:26.970188+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-04-24T23:25:32.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:25:26.970226+0000', 'last_change': '2024-04-24T23:25:26.970226+0000', 'last_active': '2024-04-24T23:25:26.961783+0000', 'last_peered': '2024-04-24T23:25:26.961783+0000', 'last_clean': '2024-04-24T23:25:26.961783+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:25:26.970226+0000', 'last_undegraded': '2024-04-24T23:25:26.970226+0000', 'last_fullsized': '2024-04-24T23:25:26.970226+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'last_clean_scrub_stamp': '2024-04-24T23:25:26.961783+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:25:32.712 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:25:32.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.837+0000 7f38d1b09640 1 Processor -- start 2024-04-24T23:25:32.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.837+0000 7f38d1b09640 1 -- start start 2024-04-24T23:25:32.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38d1b09640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f38cc145a80 0x7f38cc0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38d1b09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc15ca40 0x7f38cc0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38d1b09640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc15cee0 con 0x7f38cc15c5b0 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38d1b09640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f38cc0b3300 con 0x7f38cc145a80 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38d1b09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f38cc15d1c0 con 0x7f38cc15ca40 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38caffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38cc15c5b0 0x7f38cc06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32882/0 (socket says 172.21.15.26:32882) 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38caffd640 1 -- 172.21.15.26:0/1344423548 learned_addr learned my addr 172.21.15.26:0/1344423548 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38cbfff640 1 --2- 172.21.15.26:0/1344423548 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc15ca40 0x7f38cc0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38cb7fe640 1 --2- 172.21.15.26:0/1344423548 >> v2:172.21.15.29:3300/0 conn(0x7f38cc145a80 0x7f38cc0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38ca7fc640 1 -- 172.21.15.26:0/1344423548 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 861481458 0 0) 0x7f38cc15cee0 con 0x7f38cc15c5b0 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.838+0000 7f38ca7fc640 1 -- 172.21.15.26:0/1344423548 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38b4003520 con 0x7f38cc15c5b0 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.839+0000 7f38cb7fe640 1 -- 172.21.15.26:0/1344423548 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc15ca40 msgr2=0x7f38cc0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.839+0000 7f38ca7fc640 1 -- 172.21.15.26:0/1344423548 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1151815587 0 0) 0x7f38b4003520 con 0x7f38cc15c5b0 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.839+0000 7f38cb7fe640 1 --2- 172.21.15.26:0/1344423548 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc15ca40 0x7f38cc0b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.839+0000 7f38cb7fe640 1 -- 172.21.15.26:0/1344423548 >> v1:172.21.15.26:6789/0 conn(0x7f38cc15c5b0 legacy=0x7f38cc06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.839+0000 7f38cb7fe640 1 -- 172.21.15.26:0/1344423548 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc15d4a0 con 0x7f38cc145a80 2024-04-24T23:25:32.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.839+0000 7f38cbfff640 1 --2- 172.21.15.26:0/1344423548 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc15ca40 0x7f38cc0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:32.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.839+0000 7f38cb7fe640 1 --2- 172.21.15.26:0/1344423548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38cc145a80 0x7f38cc0ce040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0097a0 tx=0x7f38bc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05c900018750637 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:32.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38ca7fc640 1 -- 172.21.15.26:0/1344423548 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc037070 con 0x7f38cc145a80 2024-04-24T23:25:32.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38ca7fc640 1 -- 172.21.15.26:0/1344423548 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38bc02aba0 con 0x7f38cc145a80 2024-04-24T23:25:32.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38ca7fc640 1 -- 172.21.15.26:0/1344423548 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0001670 con 0x7f38cc145a80 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38d1b09640 1 -- 172.21.15.26:0/1344423548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38cc145a80 msgr2=0x7f38cc0ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38d1b09640 1 --2- 172.21.15.26:0/1344423548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38cc145a80 0x7f38cc0ce040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0097a0 tx=0x7f38bc02a110 comp rx=0 tx=0).stop 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38d1b09640 1 -- 172.21.15.26:0/1344423548 shutdown_connections 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38d1b09640 1 --2- 172.21.15.26:0/1344423548 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc15ca40 0x7f38cc0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38d1b09640 1 --2- 172.21.15.26:0/1344423548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38cc145a80 0x7f38cc0ce040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38d1b09640 1 -- 172.21.15.26:0/1344423548 >> 172.21.15.26:0/1344423548 conn(0x7f38cc06b540 msgr2=0x7f38cc06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38d1b09640 1 -- 172.21.15.26:0/1344423548 shutdown_connections 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.840+0000 7f38d1b09640 1 -- 172.21.15.26:0/1344423548 wait complete. 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.841+0000 7f38d1b09640 1 Processor -- start 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.841+0000 7f38d1b09640 1 -- start start 2024-04-24T23:25:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38d1b09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc145a80 0x7f38cc0c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38d1b09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc15c5b0 0x7f38cc0adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38d1b09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38cc15ca40 0x7f38cc0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38d1b09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc06ddc0 con 0x7f38cc15c5b0 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38d1b09640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc06dc40 con 0x7f38cc15ca40 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38d1b09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f38cc0b3cd0 con 0x7f38cc145a80 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38caffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc15c5b0 0x7f38cc0adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38caffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc15c5b0 0x7f38cc0adf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59066/0 (socket says 172.21.15.26:59066) 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38caffd640 1 -- 172.21.15.26:0/3525994366 learned_addr learned my addr 172.21.15.26:0/3525994366 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38cb7fe640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc145a80 0x7f38cc0c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.842+0000 7f38cbfff640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38cc15ca40 0x7f38cc0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38caffd640 1 -- 172.21.15.26:0/3525994366 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc145a80 msgr2=0x7f38cc0c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38caffd640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc145a80 0x7f38cc0c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38caffd640 1 -- 172.21.15.26:0/3525994366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38cc15ca40 msgr2=0x7f38cc0c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38caffd640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38cc15ca40 0x7f38cc0c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38caffd640 1 -- 172.21.15.26:0/3525994366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc0ae680 con 0x7f38cc15c5b0 2024-04-24T23:25:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38caffd640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc15c5b0 0x7f38cc0adf80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c7b0 tx=0x7f38c000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38b3fff640 1 -- 172.21.15.26:0/3525994366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0017070 con 0x7f38cc15c5b0 2024-04-24T23:25:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38b3fff640 1 -- 172.21.15.26:0/3525994366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c0005ad0 con 0x7f38cc15c5b0 2024-04-24T23:25:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38b3fff640 1 -- 172.21.15.26:0/3525994366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0007070 con 0x7f38cc15c5b0 2024-04-24T23:25:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38cc0ae960 con 0x7f38cc15c5b0 2024-04-24T23:25:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.843+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38cc0aef20 con 0x7f38cc15c5b0 2024-04-24T23:25:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.844+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38cc0694c0 con 0x7f38cc15c5b0 2024-04-24T23:25:32.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.844+0000 7f38b3fff640 1 -- 172.21.15.26:0/3525994366 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38c001a070 con 0x7f38cc15c5b0 2024-04-24T23:25:32.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.845+0000 7f38b3fff640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f38b806be60 0x7f38b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:32.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.845+0000 7f38b3fff640 1 -- 172.21.15.26:0/3525994366 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f38c008a2b0 con 0x7f38cc15c5b0 2024-04-24T23:25:32.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.845+0000 7f38cb7fe640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f38b806be60 0x7f38b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:32.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.845+0000 7f38cb7fe640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f38b806be60 0x7f38b806e320 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f38bc034000 tx=0x7f38bc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:32.849+0000 7f38b3fff640 1 -- 172.21.15.26:0/3525994366 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f38c0058600 con 0x7f38cc15c5b0 2024-04-24T23:25:33.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.106+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f38cc0b10c0 con 0x7f38cc15c5b0 2024-04-24T23:25:33.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.106+0000 7f38b3fff640 1 -- 172.21.15.26:0/3525994366 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1454 (secure 0 0 0) 0x7f38c005c900 con 0x7f38cc15c5b0 2024-04-24T23:25:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:33.108 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":302,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":69,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":271622144,"bytes_avail":772822491136,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":67,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":7227}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-24T23:25:11.372341+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:25:33.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f38b806be60 msgr2=0x7f38b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:33.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f38b806be60 0x7f38b806e320 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f38bc034000 tx=0x7f38bc02e000 comp rx=0 tx=0).stop 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc15c5b0 msgr2=0x7f38cc0adf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc15c5b0 0x7f38cc0adf80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c7b0 tx=0x7f38c000cc70 comp rx=0 tx=0).stop 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 shutdown_connections 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f38b806be60 0x7f38b806e320 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38cc15ca40 0x7f38cc0c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc15c5b0 0x7f38cc0adf80 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 --2- 172.21.15.26:0/3525994366 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f38cc145a80 0x7f38cc0c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 >> 172.21.15.26:0/3525994366 conn(0x7f38cc06b540 msgr2=0x7f38cc0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 shutdown_connections 2024-04-24T23:25:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.108+0000 7f38d1b09640 1 -- 172.21.15.26:0/3525994366 wait complete. 2024-04-24T23:25:33.120 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-24T23:25:33.120 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:25:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.220+0000 7f595eca5640 1 Processor -- start 2024-04-24T23:25:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.220+0000 7f595eca5640 1 -- start start 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.221+0000 7f595eca5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595815e290 0x7f59580d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.221+0000 7f595eca5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f59580b3ac0 0x7f59580b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.221+0000 7f595eca5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59580b5840 con 0x7f595806c230 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.221+0000 7f595eca5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f59580b4460 con 0x7f59580b3ac0 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.221+0000 7f595eca5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59580b5e50 con 0x7f595815e290 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.221+0000 7f5957fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f59580b3ac0 0x7f59580b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.221+0000 7f595d242640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f595806c230 0x7f595806c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32890/0 (socket says 172.21.15.26:32890) 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.221+0000 7f595d242640 1 -- 172.21.15.26:0/2714028812 learned_addr learned my addr 172.21.15.26:0/2714028812 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.221+0000 7f595ca41640 1 --2- 172.21.15.26:0/2714028812 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595815e290 0x7f59580d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.222+0000 7f5957fff640 1 -- 172.21.15.26:0/2714028812 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595815e290 msgr2=0x7f59580d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.222+0000 7f5957fff640 1 --2- 172.21.15.26:0/2714028812 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595815e290 0x7f59580d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.222+0000 7f5957fff640 1 -- 172.21.15.26:0/2714028812 >> v1:172.21.15.26:6789/0 conn(0x7f595806c230 legacy=0x7f595806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.222+0000 7f59577fe640 1 -- 172.21.15.26:0/2714028812 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3641679020 0 0) 0x7f59580b5840 con 0x7f595806c230 2024-04-24T23:25:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.222+0000 7f5957fff640 1 -- 172.21.15.26:0/2714028812 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59580cfb30 con 0x7f59580b3ac0 2024-04-24T23:25:33.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.222+0000 7f5957fff640 1 --2- 172.21.15.26:0/2714028812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59580b3ac0 0x7f59580b3f20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f594c0097a0 tx=0x7f594c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49a1e7a7228f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:33.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.222+0000 7f59577fe640 1 -- 172.21.15.26:0/2714028812 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c037070 con 0x7f59580b3ac0 2024-04-24T23:25:33.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.222+0000 7f59577fe640 1 -- 172.21.15.26:0/2714028812 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594c032440 con 0x7f59580b3ac0 2024-04-24T23:25:33.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.222+0000 7f59577fe640 1 -- 172.21.15.26:0/2714028812 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c032810 con 0x7f59580b3ac0 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.223+0000 7f595eca5640 1 -- 172.21.15.26:0/2714028812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59580b3ac0 msgr2=0x7f59580b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.223+0000 7f595eca5640 1 --2- 172.21.15.26:0/2714028812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59580b3ac0 0x7f59580b3f20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f594c0097a0 tx=0x7f594c02a180 comp rx=0 tx=0).stop 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.223+0000 7f595eca5640 1 -- 172.21.15.26:0/2714028812 shutdown_connections 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.223+0000 7f595eca5640 1 --2- 172.21.15.26:0/2714028812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59580b3ac0 0x7f59580b3f20 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.223+0000 7f595eca5640 1 --2- 172.21.15.26:0/2714028812 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595815e290 0x7f59580d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.223+0000 7f595eca5640 1 -- 172.21.15.26:0/2714028812 >> 172.21.15.26:0/2714028812 conn(0x7f595806b120 msgr2=0x7f595806b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.223+0000 7f595eca5640 1 -- 172.21.15.26:0/2714028812 shutdown_connections 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.223+0000 7f595eca5640 1 -- 172.21.15.26:0/2714028812 wait complete. 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.224+0000 7f595eca5640 1 Processor -- start 2024-04-24T23:25:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.224+0000 7f595eca5640 1 -- start start 2024-04-24T23:25:33.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595eca5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595806c230 0x7f59580b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:33.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595eca5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59580b3ac0 0x7f59580cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:33.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595eca5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f595815e290 0x7f59580cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:33.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595eca5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59580b5ae0 con 0x7f595815e290 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595eca5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59580d00f0 con 0x7f59580b3ac0 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595eca5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f595815e670 con 0x7f595806c230 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595d242640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f595815e290 0x7f59580cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595d242640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f595815e290 0x7f59580cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59094/0 (socket says 172.21.15.26:59094) 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595d242640 1 -- 172.21.15.26:0/3063713170 learned_addr learned my addr 172.21.15.26:0/3063713170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f595ca41640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595806c230 0x7f59580b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.225+0000 7f5957fff640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59580b3ac0 0x7f59580cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f595d242640 1 -- 172.21.15.26:0/3063713170 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595806c230 msgr2=0x7f59580b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f595d242640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595806c230 0x7f59580b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f595d242640 1 -- 172.21.15.26:0/3063713170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59580b3ac0 msgr2=0x7f59580cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f595d242640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59580b3ac0 0x7f59580cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f595d242640 1 -- 172.21.15.26:0/3063713170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59581f3d50 con 0x7f595815e290 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f595d242640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f595815e290 0x7f59580cbd10 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f595000caf0 tx=0x7f59500021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:33.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f59557fa640 1 -- 172.21.15.26:0/3063713170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595000ac00 con 0x7f595815e290 2024-04-24T23:25:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f59557fa640 1 -- 172.21.15.26:0/3063713170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595000ada0 con 0x7f595815e290 2024-04-24T23:25:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59581f42f0 con 0x7f595815e290 2024-04-24T23:25:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f59557fa640 1 -- 172.21.15.26:0/3063713170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950012640 con 0x7f595815e290 2024-04-24T23:25:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.226+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581f48b0 con 0x7f595815e290 2024-04-24T23:25:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.227+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59580694c0 con 0x7f595815e290 2024-04-24T23:25:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.228+0000 7f59557fa640 1 -- 172.21.15.26:0/3063713170 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59500127e0 con 0x7f595815e290 2024-04-24T23:25:33.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.228+0000 7f59557fa640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f594406be30 0x7f594406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:33.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.228+0000 7f595ca41640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f594406be30 0x7f594406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:33.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.228+0000 7f59557fa640 1 -- 172.21.15.26:0/3063713170 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f595008b240 con 0x7f595815e290 2024-04-24T23:25:33.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.229+0000 7f595ca41640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f594406be30 0x7f594406e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5948009b40 tx=0x7f5948009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:33.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.232+0000 7f59557fa640 1 -- 172.21.15.26:0/3063713170 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5950059630 con 0x7f595815e290 2024-04-24T23:25:33.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.412+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f595806b880 con 0x7f594406be30 2024-04-24T23:25:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.418+0000 7f59557fa640 1 -- 172.21.15.26:0/3063713170 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f595806b880 con 0x7f594406be30 2024-04-24T23:25:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f594406be30 msgr2=0x7f594406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f594406be30 0x7f594406e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5948009b40 tx=0x7f5948009190 comp rx=0 tx=0).stop 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f595815e290 msgr2=0x7f59580cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f595815e290 0x7f59580cbd10 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f595000caf0 tx=0x7f59500021c0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 shutdown_connections 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f594406be30 0x7f594406e2f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f595815e290 0x7f59580cbd10 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59580b3ac0 0x7f59580cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 --2- 172.21.15.26:0/3063713170 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f595806c230 0x7f59580b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 >> 172.21.15.26:0/3063713170 conn(0x7f595806b120 msgr2=0x7f59580cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 shutdown_connections 2024-04-24T23:25:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:33.421+0000 7f595eca5640 1 -- 172.21.15.26:0/3063713170 wait complete. 2024-04-24T23:25:33.433 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-04-24T23:25:33.376374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265568,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754709152,"statfs":{"total":773094113280,"available":772822171648,"internally_reserved":0,"allocated":7970816,"data_stored":4043004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.411789"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164104+0000","last_change":"2024-04-24T23:25:28.001691+0000","last_active":"2024-04-24T23:25:29.164104+0000","last_peered":"2024-04-24T23:25:29.164104+0000","last_clean":"2024-04-24T23:25:29.164104+0000","last_became_active":"2024-04-24T23:25:28.001394+0000","last_became_peered":"2024-04-24T23:25:28.001394+0000","last_unstale":"2024-04-24T23:25:29.164104+0000","last_undegraded":"2024-04-24T23:25:29.164104+0000","last_fullsized":"2024-04-24T23:25:29.164104+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:36.386100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.408066+0000","last_change":"2024-04-24T23:25:28.003937+0000","last_active":"2024-04-24T23:25:30.408066+0000","last_peered":"2024-04-24T23:25:30.408066+0000","last_clean":"2024-04-24T23:25:30.408066+0000","last_became_active":"2024-04-24T23:25:28.001712+0000","last_became_peered":"2024-04-24T23:25:28.001712+0000","last_unstale":"2024-04-24T23:25:30.408066+0000","last_undegraded":"2024-04-24T23:25:30.408066+0000","last_fullsized":"2024-04-24T23:25:30.408066+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:47.560758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236442+0000","last_change":"2024-04-24T23:25:27.999495+0000","last_active":"2024-04-24T23:25:29.236442+0000","last_peered":"2024-04-24T23:25:29.236442+0000","last_clean":"2024-04-24T23:25:29.236442+0000","last_became_active":"2024-04-24T23:25:27.999220+0000","last_became_peered":"2024-04-24T23:25:27.999220+0000","last_unstale":"2024-04-24T23:25:29.236442+0000","last_undegraded":"2024-04-24T23:25:29.236442+0000","last_fullsized":"2024-04-24T23:25:29.236442+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:19.252039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406426+0000","last_change":"2024-04-24T23:25:27.997705+0000","last_active":"2024-04-24T23:25:30.406426+0000","last_peered":"2024-04-24T23:25:30.406426+0000","last_clean":"2024-04-24T23:25:30.406426+0000","last_became_active":"2024-04-24T23:25:27.997415+0000","last_became_peered":"2024-04-24T23:25:27.997415+0000","last_unstale":"2024-04-24T23:25:30.406426+0000","last_undegraded":"2024-04-24T23:25:30.406426+0000","last_fullsized":"2024-04-24T23:25:30.406426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:08.524873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993378+0000","last_change":"2024-04-24T23:25:27.996522+0000","last_active":"2024-04-24T23:25:29.993378+0000","last_peered":"2024-04-24T23:25:29.993378+0000","last_clean":"2024-04-24T23:25:29.993378+0000","last_became_active":"2024-04-24T23:25:27.996253+0000","last_became_peered":"2024-04-24T23:25:27.996253+0000","last_unstale":"2024-04-24T23:25:29.993378+0000","last_undegraded":"2024-04-24T23:25:29.993378+0000","last_fullsized":"2024-04-24T23:25:29.993378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:07.473497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407620+0000","last_change":"2024-04-24T23:25:27.999832+0000","last_active":"2024-04-24T23:25:30.407620+0000","last_peered":"2024-04-24T23:25:30.407620+0000","last_clean":"2024-04-24T23:25:30.407620+0000","last_became_active":"2024-04-24T23:25:27.999381+0000","last_became_peered":"2024-04-24T23:25:27.999381+0000","last_unstale":"2024-04-24T23:25:30.407620+0000","last_undegraded":"2024-04-24T23:25:30.407620+0000","last_fullsized":"2024-04-24T23:25:30.407620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:57.738709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992886+0000","last_change":"2024-04-24T23:25:27.993916+0000","last_active":"2024-04-24T23:25:28.992886+0000","last_peered":"2024-04-24T23:25:28.992886+0000","last_clean":"2024-04-24T23:25:28.992886+0000","last_became_active":"2024-04-24T23:25:27.993650+0000","last_became_peered":"2024-04-24T23:25:27.993650+0000","last_unstale":"2024-04-24T23:25:28.992886+0000","last_undegraded":"2024-04-24T23:25:28.992886+0000","last_fullsized":"2024-04-24T23:25:28.992886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:39.831794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407073+0000","last_change":"2024-04-24T23:25:28.002407+0000","last_active":"2024-04-24T23:25:30.407073+0000","last_peered":"2024-04-24T23:25:30.407073+0000","last_clean":"2024-04-24T23:25:30.407073+0000","last_became_active":"2024-04-24T23:25:28.000515+0000","last_became_peered":"2024-04-24T23:25:28.000515+0000","last_unstale":"2024-04-24T23:25:30.407073+0000","last_undegraded":"2024-04-24T23:25:30.407073+0000","last_fullsized":"2024-04-24T23:25:30.407073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:35.022298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993608+0000","last_change":"2024-04-24T23:25:27.994672+0000","last_active":"2024-04-24T23:25:29.993608+0000","last_peered":"2024-04-24T23:25:29.993608+0000","last_clean":"2024-04-24T23:25:29.993608+0000","last_became_active":"2024-04-24T23:25:27.994408+0000","last_became_peered":"2024-04-24T23:25:27.994408+0000","last_unstale":"2024-04-24T23:25:29.993608+0000","last_undegraded":"2024-04-24T23:25:29.993608+0000","last_fullsized":"2024-04-24T23:25:29.993608+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:13.263193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407106+0000","last_change":"2024-04-24T23:25:28.004663+0000","last_active":"2024-04-24T23:25:30.407106+0000","last_peered":"2024-04-24T23:25:30.407106+0000","last_clean":"2024-04-24T23:25:30.407106+0000","last_became_active":"2024-04-24T23:25:28.002731+0000","last_became_peered":"2024-04-24T23:25:28.002731+0000","last_unstale":"2024-04-24T23:25:30.407106+0000","last_undegraded":"2024-04-24T23:25:30.407106+0000","last_fullsized":"2024-04-24T23:25:30.407106+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:29.630244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163844+0000","last_change":"2024-04-24T23:25:28.003371+0000","last_active":"2024-04-24T23:25:29.163844+0000","last_peered":"2024-04-24T23:25:29.163844+0000","last_clean":"2024-04-24T23:25:29.163844+0000","last_became_active":"2024-04-24T23:25:28.003065+0000","last_became_peered":"2024-04-24T23:25:28.003065+0000","last_unstale":"2024-04-24T23:25:29.163844+0000","last_undegraded":"2024-04-24T23:25:29.163844+0000","last_fullsized":"2024-04-24T23:25:29.163844+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:55.924853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406422+0000","last_change":"2024-04-24T23:25:27.996620+0000","last_active":"2024-04-24T23:25:30.406422+0000","last_peered":"2024-04-24T23:25:30.406422+0000","last_clean":"2024-04-24T23:25:30.406422+0000","last_became_active":"2024-04-24T23:25:27.996355+0000","last_became_peered":"2024-04-24T23:25:27.996355+0000","last_unstale":"2024-04-24T23:25:30.406422+0000","last_undegraded":"2024-04-24T23:25:30.406422+0000","last_fullsized":"2024-04-24T23:25:30.406422+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:30.598436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992392+0000","last_change":"2024-04-24T23:25:28.005958+0000","last_active":"2024-04-24T23:25:28.992392+0000","last_peered":"2024-04-24T23:25:28.992392+0000","last_clean":"2024-04-24T23:25:28.992392+0000","last_became_active":"2024-04-24T23:25:28.005394+0000","last_became_peered":"2024-04-24T23:25:28.005394+0000","last_unstale":"2024-04-24T23:25:28.992392+0000","last_undegraded":"2024-04-24T23:25:28.992392+0000","last_fullsized":"2024-04-24T23:25:28.992392+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:41.763030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993440+0000","last_change":"2024-04-24T23:25:27.992504+0000","last_active":"2024-04-24T23:25:29.993440+0000","last_peered":"2024-04-24T23:25:29.993440+0000","last_clean":"2024-04-24T23:25:29.993440+0000","last_became_active":"2024-04-24T23:25:27.989888+0000","last_became_peered":"2024-04-24T23:25:27.989888+0000","last_unstale":"2024-04-24T23:25:29.993440+0000","last_undegraded":"2024-04-24T23:25:29.993440+0000","last_fullsized":"2024-04-24T23:25:29.993440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:12.681568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992149+0000","last_change":"2024-04-24T23:25:27.993212+0000","last_active":"2024-04-24T23:25:28.992149+0000","last_peered":"2024-04-24T23:25:28.992149+0000","last_clean":"2024-04-24T23:25:28.992149+0000","last_became_active":"2024-04-24T23:25:27.992908+0000","last_became_peered":"2024-04-24T23:25:27.992908+0000","last_unstale":"2024-04-24T23:25:28.992149+0000","last_undegraded":"2024-04-24T23:25:28.992149+0000","last_fullsized":"2024-04-24T23:25:28.992149+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:33.603412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406397+0000","last_change":"2024-04-24T23:25:28.010143+0000","last_active":"2024-04-24T23:25:30.406397+0000","last_peered":"2024-04-24T23:25:30.406397+0000","last_clean":"2024-04-24T23:25:30.406397+0000","last_became_active":"2024-04-24T23:25:28.009856+0000","last_became_peered":"2024-04-24T23:25:28.009856+0000","last_unstale":"2024-04-24T23:25:30.406397+0000","last_undegraded":"2024-04-24T23:25:30.406397+0000","last_fullsized":"2024-04-24T23:25:30.406397+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:43.854697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993750+0000","last_change":"2024-04-24T23:25:27.995041+0000","last_active":"2024-04-24T23:25:29.993750+0000","last_peered":"2024-04-24T23:25:29.993750+0000","last_clean":"2024-04-24T23:25:29.993750+0000","last_became_active":"2024-04-24T23:25:27.991668+0000","last_became_peered":"2024-04-24T23:25:27.991668+0000","last_unstale":"2024-04-24T23:25:29.993750+0000","last_undegraded":"2024-04-24T23:25:29.993750+0000","last_fullsized":"2024-04-24T23:25:29.993750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:16.385587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406481+0000","last_change":"2024-04-24T23:25:27.998171+0000","last_active":"2024-04-24T23:25:30.406481+0000","last_peered":"2024-04-24T23:25:30.406481+0000","last_clean":"2024-04-24T23:25:30.406481+0000","last_became_active":"2024-04-24T23:25:27.997720+0000","last_became_peered":"2024-04-24T23:25:27.997720+0000","last_unstale":"2024-04-24T23:25:30.406481+0000","last_undegraded":"2024-04-24T23:25:30.406481+0000","last_fullsized":"2024-04-24T23:25:30.406481+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:02.362039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407872+0000","last_change":"2024-04-24T23:25:27.994062+0000","last_active":"2024-04-24T23:25:30.407872+0000","last_peered":"2024-04-24T23:25:30.407872+0000","last_clean":"2024-04-24T23:25:30.407872+0000","last_became_active":"2024-04-24T23:25:27.993608+0000","last_became_peered":"2024-04-24T23:25:27.993608+0000","last_unstale":"2024-04-24T23:25:30.407872+0000","last_undegraded":"2024-04-24T23:25:30.407872+0000","last_fullsized":"2024-04-24T23:25:30.407872+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:07.049044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992400+0000","last_change":"2024-04-24T23:25:28.001925+0000","last_active":"2024-04-24T23:25:28.992400+0000","last_peered":"2024-04-24T23:25:28.992400+0000","last_clean":"2024-04-24T23:25:28.992400+0000","last_became_active":"2024-04-24T23:25:28.001658+0000","last_became_peered":"2024-04-24T23:25:28.001658+0000","last_unstale":"2024-04-24T23:25:28.992400+0000","last_undegraded":"2024-04-24T23:25:28.992400+0000","last_fullsized":"2024-04-24T23:25:28.992400+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:12.033609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406776+0000","last_change":"2024-04-24T23:25:27.996181+0000","last_active":"2024-04-24T23:25:30.406776+0000","last_peered":"2024-04-24T23:25:30.406776+0000","last_clean":"2024-04-24T23:25:30.406776+0000","last_became_active":"2024-04-24T23:25:27.995846+0000","last_became_peered":"2024-04-24T23:25:27.995846+0000","last_unstale":"2024-04-24T23:25:30.406776+0000","last_undegraded":"2024-04-24T23:25:30.406776+0000","last_fullsized":"2024-04-24T23:25:30.406776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:44.619593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406653+0000","last_change":"2024-04-24T23:25:28.002416+0000","last_active":"2024-04-24T23:25:30.406653+0000","last_peered":"2024-04-24T23:25:30.406653+0000","last_clean":"2024-04-24T23:25:30.406653+0000","last_became_active":"2024-04-24T23:25:28.002073+0000","last_became_peered":"2024-04-24T23:25:28.002073+0000","last_unstale":"2024-04-24T23:25:30.406653+0000","last_undegraded":"2024-04-24T23:25:30.406653+0000","last_fullsized":"2024-04-24T23:25:30.406653+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:47.560758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163642+0000","last_change":"2024-04-24T23:25:27.999950+0000","last_active":"2024-04-24T23:25:29.163642+0000","last_peered":"2024-04-24T23:25:29.163642+0000","last_clean":"2024-04-24T23:25:29.163642+0000","last_became_active":"2024-04-24T23:25:27.999540+0000","last_became_peered":"2024-04-24T23:25:27.999540+0000","last_unstale":"2024-04-24T23:25:29.163642+0000","last_undegraded":"2024-04-24T23:25:29.163642+0000","last_fullsized":"2024-04-24T23:25:29.163642+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:07.983258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163505+0000","last_change":"2024-04-24T23:25:28.008767+0000","last_active":"2024-04-24T23:25:29.163505+0000","last_peered":"2024-04-24T23:25:29.163505+0000","last_clean":"2024-04-24T23:25:29.163505+0000","last_became_active":"2024-04-24T23:25:28.008459+0000","last_became_peered":"2024-04-24T23:25:28.008459+0000","last_unstale":"2024-04-24T23:25:29.163505+0000","last_undegraded":"2024-04-24T23:25:29.163505+0000","last_fullsized":"2024-04-24T23:25:29.163505+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:53.596699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407139+0000","last_change":"2024-04-24T23:25:28.002914+0000","last_active":"2024-04-24T23:25:30.407139+0000","last_peered":"2024-04-24T23:25:30.407139+0000","last_clean":"2024-04-24T23:25:30.407139+0000","last_became_active":"2024-04-24T23:25:28.002018+0000","last_became_peered":"2024-04-24T23:25:28.002018+0000","last_unstale":"2024-04-24T23:25:30.407139+0000","last_undegraded":"2024-04-24T23:25:30.407139+0000","last_fullsized":"2024-04-24T23:25:30.407139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:48.807209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993321+0000","last_change":"2024-04-24T23:25:28.004061+0000","last_active":"2024-04-24T23:25:28.993321+0000","last_peered":"2024-04-24T23:25:28.993321+0000","last_clean":"2024-04-24T23:25:28.993321+0000","last_became_active":"2024-04-24T23:25:28.003756+0000","last_became_peered":"2024-04-24T23:25:28.003756+0000","last_unstale":"2024-04-24T23:25:28.993321+0000","last_undegraded":"2024-04-24T23:25:28.993321+0000","last_fullsized":"2024-04-24T23:25:28.993321+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:12.033609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236574+0000","last_change":"2024-04-24T23:25:27.990021+0000","last_active":"2024-04-24T23:25:29.236574+0000","last_peered":"2024-04-24T23:25:29.236574+0000","last_clean":"2024-04-24T23:25:29.236574+0000","last_became_active":"2024-04-24T23:25:27.989687+0000","last_became_peered":"2024-04-24T23:25:27.989687+0000","last_unstale":"2024-04-24T23:25:29.236574+0000","last_undegraded":"2024-04-24T23:25:29.236574+0000","last_fullsized":"2024-04-24T23:25:29.236574+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:55.576823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407040+0000","last_change":"2024-04-24T23:25:27.999162+0000","last_active":"2024-04-24T23:25:30.407040+0000","last_peered":"2024-04-24T23:25:30.407040+0000","last_clean":"2024-04-24T23:25:30.407040+0000","last_became_active":"2024-04-24T23:25:27.998838+0000","last_became_peered":"2024-04-24T23:25:27.998838+0000","last_unstale":"2024-04-24T23:25:30.407040+0000","last_undegraded":"2024-04-24T23:25:30.407040+0000","last_fullsized":"2024-04-24T23:25:30.407040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:21.326927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236422+0000","last_change":"2024-04-24T23:25:27.997325+0000","last_active":"2024-04-24T23:25:29.236422+0000","last_peered":"2024-04-24T23:25:29.236422+0000","last_clean":"2024-04-24T23:25:29.236422+0000","last_became_active":"2024-04-24T23:25:27.997036+0000","last_became_peered":"2024-04-24T23:25:27.997036+0000","last_unstale":"2024-04-24T23:25:29.236422+0000","last_undegraded":"2024-04-24T23:25:29.236422+0000","last_fullsized":"2024-04-24T23:25:29.236422+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:50.668477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992943+0000","last_change":"2024-04-24T23:25:28.007569+0000","last_active":"2024-04-24T23:25:28.992943+0000","last_peered":"2024-04-24T23:25:28.992943+0000","last_clean":"2024-04-24T23:25:28.992943+0000","last_became_active":"2024-04-24T23:25:28.007220+0000","last_became_peered":"2024-04-24T23:25:28.007220+0000","last_unstale":"2024-04-24T23:25:28.992943+0000","last_undegraded":"2024-04-24T23:25:28.992943+0000","last_fullsized":"2024-04-24T23:25:28.992943+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:56.132083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993329+0000","last_change":"2024-04-24T23:25:27.993307+0000","last_active":"2024-04-24T23:25:28.993329+0000","last_peered":"2024-04-24T23:25:28.993329+0000","last_clean":"2024-04-24T23:25:28.993329+0000","last_became_active":"2024-04-24T23:25:27.993030+0000","last_became_peered":"2024-04-24T23:25:27.993030+0000","last_unstale":"2024-04-24T23:25:28.993329+0000","last_undegraded":"2024-04-24T23:25:28.993329+0000","last_fullsized":"2024-04-24T23:25:28.993329+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:10.421054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407328+0000","last_change":"2024-04-24T23:25:28.002497+0000","last_active":"2024-04-24T23:25:30.407328+0000","last_peered":"2024-04-24T23:25:30.407328+0000","last_clean":"2024-04-24T23:25:30.407328+0000","last_became_active":"2024-04-24T23:25:28.002164+0000","last_became_peered":"2024-04-24T23:25:28.002164+0000","last_unstale":"2024-04-24T23:25:30.407328+0000","last_undegraded":"2024-04-24T23:25:30.407328+0000","last_fullsized":"2024-04-24T23:25:30.407328+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:38.792880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163995+0000","last_change":"2024-04-24T23:25:28.005542+0000","last_active":"2024-04-24T23:25:29.163995+0000","last_peered":"2024-04-24T23:25:29.163995+0000","last_clean":"2024-04-24T23:25:29.163995+0000","last_became_active":"2024-04-24T23:25:28.005236+0000","last_became_peered":"2024-04-24T23:25:28.005236+0000","last_unstale":"2024-04-24T23:25:29.163995+0000","last_undegraded":"2024-04-24T23:25:29.163995+0000","last_fullsized":"2024-04-24T23:25:29.163995+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:10.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164196+0000","last_change":"2024-04-24T23:25:26.984830+0000","last_active":"2024-04-24T23:25:29.164196+0000","last_peered":"2024-04-24T23:25:29.164196+0000","last_clean":"2024-04-24T23:25:29.164196+0000","last_became_active":"2024-04-24T23:25:26.983743+0000","last_became_peered":"2024-04-24T23:25:26.983743+0000","last_unstale":"2024-04-24T23:25:29.164196+0000","last_undegraded":"2024-04-24T23:25:29.164196+0000","last_fullsized":"2024-04-24T23:25:29.164196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:28.625915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993757+0000","last_change":"2024-04-24T23:25:27.991300+0000","last_active":"2024-04-24T23:25:29.993757+0000","last_peered":"2024-04-24T23:25:29.993757+0000","last_clean":"2024-04-24T23:25:29.993757+0000","last_became_active":"2024-04-24T23:25:27.991006+0000","last_became_peered":"2024-04-24T23:25:27.991006+0000","last_unstale":"2024-04-24T23:25:29.993757+0000","last_undegraded":"2024-04-24T23:25:29.993757+0000","last_fullsized":"2024-04-24T23:25:29.993757+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:50.527988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992493+0000","last_change":"2024-04-24T23:25:26.986600+0000","last_active":"2024-04-24T23:25:28.992493+0000","last_peered":"2024-04-24T23:25:28.992493+0000","last_clean":"2024-04-24T23:25:28.992493+0000","last_became_active":"2024-04-24T23:25:26.986208+0000","last_became_peered":"2024-04-24T23:25:26.986208+0000","last_unstale":"2024-04-24T23:25:28.992493+0000","last_undegraded":"2024-04-24T23:25:28.992493+0000","last_fullsized":"2024-04-24T23:25:28.992493+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:20.322598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407797+0000","last_change":"2024-04-24T23:25:27.995888+0000","last_active":"2024-04-24T23:25:30.407797+0000","last_peered":"2024-04-24T23:25:30.407797+0000","last_clean":"2024-04-24T23:25:30.407797+0000","last_became_active":"2024-04-24T23:25:27.995055+0000","last_became_peered":"2024-04-24T23:25:27.995055+0000","last_unstale":"2024-04-24T23:25:30.407797+0000","last_undegraded":"2024-04-24T23:25:30.407797+0000","last_fullsized":"2024-04-24T23:25:30.407797+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:52.787249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407024+0000","last_change":"2024-04-24T23:25:26.988051+0000","last_active":"2024-04-24T23:25:30.407024+0000","last_peered":"2024-04-24T23:25:30.407024+0000","last_clean":"2024-04-24T23:25:30.407024+0000","last_became_active":"2024-04-24T23:25:26.987663+0000","last_became_peered":"2024-04-24T23:25:26.987663+0000","last_unstale":"2024-04-24T23:25:30.407024+0000","last_undegraded":"2024-04-24T23:25:30.407024+0000","last_fullsized":"2024-04-24T23:25:30.407024+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:29.594107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163966+0000","last_change":"2024-04-24T23:25:28.002606+0000","last_active":"2024-04-24T23:25:29.163966+0000","last_peered":"2024-04-24T23:25:29.163966+0000","last_clean":"2024-04-24T23:25:29.163966+0000","last_became_active":"2024-04-24T23:25:28.002311+0000","last_became_peered":"2024-04-24T23:25:28.002311+0000","last_unstale":"2024-04-24T23:25:29.163966+0000","last_undegraded":"2024-04-24T23:25:29.163966+0000","last_fullsized":"2024-04-24T23:25:29.163966+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:16.043774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407997+0000","last_change":"2024-04-24T23:25:26.986624+0000","last_active":"2024-04-24T23:25:30.407997+0000","last_peered":"2024-04-24T23:25:30.407997+0000","last_clean":"2024-04-24T23:25:30.407997+0000","last_became_active":"2024-04-24T23:25:26.986323+0000","last_became_peered":"2024-04-24T23:25:26.986323+0000","last_unstale":"2024-04-24T23:25:30.407997+0000","last_undegraded":"2024-04-24T23:25:30.407997+0000","last_fullsized":"2024-04-24T23:25:30.407997+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:27.254375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993045+0000","last_change":"2024-04-24T23:25:28.001568+0000","last_active":"2024-04-24T23:25:28.993045+0000","last_peered":"2024-04-24T23:25:28.993045+0000","last_clean":"2024-04-24T23:25:28.993045+0000","last_became_active":"2024-04-24T23:25:28.001250+0000","last_became_peered":"2024-04-24T23:25:28.001250+0000","last_unstale":"2024-04-24T23:25:28.993045+0000","last_undegraded":"2024-04-24T23:25:28.993045+0000","last_fullsized":"2024-04-24T23:25:28.993045+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:33.603412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236474+0000","last_change":"2024-04-24T23:25:26.977218+0000","last_active":"2024-04-24T23:25:29.236474+0000","last_peered":"2024-04-24T23:25:29.236474+0000","last_clean":"2024-04-24T23:25:29.236474+0000","last_became_active":"2024-04-24T23:25:26.976913+0000","last_became_peered":"2024-04-24T23:25:26.976913+0000","last_unstale":"2024-04-24T23:25:29.236474+0000","last_undegraded":"2024-04-24T23:25:29.236474+0000","last_fullsized":"2024-04-24T23:25:29.236474+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:10.387500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406824+0000","last_change":"2024-04-24T23:25:28.001226+0000","last_active":"2024-04-24T23:25:30.406824+0000","last_peered":"2024-04-24T23:25:30.406824+0000","last_clean":"2024-04-24T23:25:30.406824+0000","last_became_active":"2024-04-24T23:25:28.000908+0000","last_became_peered":"2024-04-24T23:25:28.000908+0000","last_unstale":"2024-04-24T23:25:30.406824+0000","last_undegraded":"2024-04-24T23:25:30.406824+0000","last_fullsized":"2024-04-24T23:25:30.406824+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:02.505819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407528+0000","last_change":"2024-04-24T23:25:26.984947+0000","last_active":"2024-04-24T23:25:30.407528+0000","last_peered":"2024-04-24T23:25:30.407528+0000","last_clean":"2024-04-24T23:25:30.407528+0000","last_became_active":"2024-04-24T23:25:26.984604+0000","last_became_peered":"2024-04-24T23:25:26.984604+0000","last_unstale":"2024-04-24T23:25:30.407528+0000","last_undegraded":"2024-04-24T23:25:30.407528+0000","last_fullsized":"2024-04-24T23:25:30.407528+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:10.968125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993068+0000","last_change":"2024-04-24T23:25:27.997229+0000","last_active":"2024-04-24T23:25:29.993068+0000","last_peered":"2024-04-24T23:25:29.993068+0000","last_clean":"2024-04-24T23:25:29.993068+0000","last_became_active":"2024-04-24T23:25:27.996393+0000","last_became_peered":"2024-04-24T23:25:27.996393+0000","last_unstale":"2024-04-24T23:25:29.993068+0000","last_undegraded":"2024-04-24T23:25:29.993068+0000","last_fullsized":"2024-04-24T23:25:29.993068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:18.029231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406190+0000","last_change":"2024-04-24T23:25:26.982771+0000","last_active":"2024-04-24T23:25:30.406190+0000","last_peered":"2024-04-24T23:25:30.406190+0000","last_clean":"2024-04-24T23:25:30.406190+0000","last_became_active":"2024-04-24T23:25:26.982504+0000","last_became_peered":"2024-04-24T23:25:26.982504+0000","last_unstale":"2024-04-24T23:25:30.406190+0000","last_undegraded":"2024-04-24T23:25:30.406190+0000","last_fullsized":"2024-04-24T23:25:30.406190+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:26.544193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992272+0000","last_change":"2024-04-24T23:25:27.998554+0000","last_active":"2024-04-24T23:25:28.992272+0000","last_peered":"2024-04-24T23:25:28.992272+0000","last_clean":"2024-04-24T23:25:28.992272+0000","last_became_active":"2024-04-24T23:25:27.997924+0000","last_became_peered":"2024-04-24T23:25:27.997924+0000","last_unstale":"2024-04-24T23:25:28.992272+0000","last_undegraded":"2024-04-24T23:25:28.992272+0000","last_fullsized":"2024-04-24T23:25:28.992272+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:42.780286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993086+0000","last_change":"2024-04-24T23:25:26.985502+0000","last_active":"2024-04-24T23:25:29.993086+0000","last_peered":"2024-04-24T23:25:29.993086+0000","last_clean":"2024-04-24T23:25:29.993086+0000","last_became_active":"2024-04-24T23:25:26.985236+0000","last_became_peered":"2024-04-24T23:25:26.985236+0000","last_unstale":"2024-04-24T23:25:29.993086+0000","last_undegraded":"2024-04-24T23:25:29.993086+0000","last_fullsized":"2024-04-24T23:25:29.993086+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:11.001663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993393+0000","last_change":"2024-04-24T23:25:27.997744+0000","last_active":"2024-04-24T23:25:28.993393+0000","last_peered":"2024-04-24T23:25:28.993393+0000","last_clean":"2024-04-24T23:25:28.993393+0000","last_became_active":"2024-04-24T23:25:27.996890+0000","last_became_peered":"2024-04-24T23:25:27.996890+0000","last_unstale":"2024-04-24T23:25:28.993393+0000","last_undegraded":"2024-04-24T23:25:28.993393+0000","last_fullsized":"2024-04-24T23:25:28.993393+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:32.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407250+0000","last_change":"2024-04-24T23:25:26.976690+0000","last_active":"2024-04-24T23:25:30.407250+0000","last_peered":"2024-04-24T23:25:30.407250+0000","last_clean":"2024-04-24T23:25:30.407250+0000","last_became_active":"2024-04-24T23:25:26.976221+0000","last_became_peered":"2024-04-24T23:25:26.976221+0000","last_unstale":"2024-04-24T23:25:30.407250+0000","last_undegraded":"2024-04-24T23:25:30.407250+0000","last_fullsized":"2024-04-24T23:25:30.407250+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:07.520545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164100+0000","last_change":"2024-04-24T23:25:28.011082+0000","last_active":"2024-04-24T23:25:29.164100+0000","last_peered":"2024-04-24T23:25:29.164100+0000","last_clean":"2024-04-24T23:25:29.164100+0000","last_became_active":"2024-04-24T23:25:28.010786+0000","last_became_peered":"2024-04-24T23:25:28.010786+0000","last_unstale":"2024-04-24T23:25:29.164100+0000","last_undegraded":"2024-04-24T23:25:29.164100+0000","last_fullsized":"2024-04-24T23:25:29.164100+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:46.820679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407999+0000","last_change":"2024-04-24T23:25:26.979704+0000","last_active":"2024-04-24T23:25:30.407999+0000","last_peered":"2024-04-24T23:25:30.407999+0000","last_clean":"2024-04-24T23:25:30.407999+0000","last_became_active":"2024-04-24T23:25:26.979294+0000","last_became_peered":"2024-04-24T23:25:26.979294+0000","last_unstale":"2024-04-24T23:25:30.407999+0000","last_undegraded":"2024-04-24T23:25:30.407999+0000","last_fullsized":"2024-04-24T23:25:30.407999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:12.592820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407841+0000","last_change":"2024-04-24T23:25:28.001051+0000","last_active":"2024-04-24T23:25:30.407841+0000","last_peered":"2024-04-24T23:25:30.407841+0000","last_clean":"2024-04-24T23:25:30.407841+0000","last_became_active":"2024-04-24T23:25:28.000548+0000","last_became_peered":"2024-04-24T23:25:28.000548+0000","last_unstale":"2024-04-24T23:25:30.407841+0000","last_undegraded":"2024-04-24T23:25:30.407841+0000","last_fullsized":"2024-04-24T23:25:30.407841+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:43.854697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236638+0000","last_change":"2024-04-24T23:25:26.973701+0000","last_active":"2024-04-24T23:25:29.236638+0000","last_peered":"2024-04-24T23:25:29.236638+0000","last_clean":"2024-04-24T23:25:29.236638+0000","last_became_active":"2024-04-24T23:25:26.973424+0000","last_became_peered":"2024-04-24T23:25:26.973424+0000","last_unstale":"2024-04-24T23:25:29.236638+0000","last_undegraded":"2024-04-24T23:25:29.236638+0000","last_fullsized":"2024-04-24T23:25:29.236638+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:09.556305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993538+0000","last_change":"2024-04-24T23:25:27.989474+0000","last_active":"2024-04-24T23:25:29.993538+0000","last_peered":"2024-04-24T23:25:29.993538+0000","last_clean":"2024-04-24T23:25:29.993538+0000","last_became_active":"2024-04-24T23:25:27.989194+0000","last_became_peered":"2024-04-24T23:25:27.989194+0000","last_unstale":"2024-04-24T23:25:29.993538+0000","last_undegraded":"2024-04-24T23:25:29.993538+0000","last_fullsized":"2024-04-24T23:25:29.993538+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:11.972454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406545+0000","last_change":"2024-04-24T23:25:26.985049+0000","last_active":"2024-04-24T23:25:30.406545+0000","last_peered":"2024-04-24T23:25:30.406545+0000","last_clean":"2024-04-24T23:25:30.406545+0000","last_became_active":"2024-04-24T23:25:26.984293+0000","last_became_peered":"2024-04-24T23:25:26.984293+0000","last_unstale":"2024-04-24T23:25:30.406545+0000","last_undegraded":"2024-04-24T23:25:30.406545+0000","last_fullsized":"2024-04-24T23:25:30.406545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:12.592820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163585+0000","last_change":"2024-04-24T23:25:27.996333+0000","last_active":"2024-04-24T23:25:29.163585+0000","last_peered":"2024-04-24T23:25:29.163585+0000","last_clean":"2024-04-24T23:25:29.163585+0000","last_became_active":"2024-04-24T23:25:27.996005+0000","last_became_peered":"2024-04-24T23:25:27.996005+0000","last_unstale":"2024-04-24T23:25:29.163585+0000","last_undegraded":"2024-04-24T23:25:29.163585+0000","last_fullsized":"2024-04-24T23:25:29.163585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:14.057707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406350+0000","last_change":"2024-04-24T23:25:26.982431+0000","last_active":"2024-04-24T23:25:30.406350+0000","last_peered":"2024-04-24T23:25:30.406350+0000","last_clean":"2024-04-24T23:25:30.406350+0000","last_became_active":"2024-04-24T23:25:26.982147+0000","last_became_peered":"2024-04-24T23:25:26.982147+0000","last_unstale":"2024-04-24T23:25:30.406350+0000","last_undegraded":"2024-04-24T23:25:30.406350+0000","last_fullsized":"2024-04-24T23:25:30.406350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:47.513215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993164+0000","last_change":"2024-04-24T23:25:27.997375+0000","last_active":"2024-04-24T23:25:28.993164+0000","last_peered":"2024-04-24T23:25:28.993164+0000","last_clean":"2024-04-24T23:25:28.993164+0000","last_became_active":"2024-04-24T23:25:27.997031+0000","last_became_peered":"2024-04-24T23:25:27.997031+0000","last_unstale":"2024-04-24T23:25:28.993164+0000","last_undegraded":"2024-04-24T23:25:28.993164+0000","last_fullsized":"2024-04-24T23:25:28.993164+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:42.780286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407777+0000","last_change":"2024-04-24T23:25:26.979601+0000","last_active":"2024-04-24T23:25:30.407777+0000","last_peered":"2024-04-24T23:25:30.407777+0000","last_clean":"2024-04-24T23:25:30.407777+0000","last_became_active":"2024-04-24T23:25:26.978063+0000","last_became_peered":"2024-04-24T23:25:26.978063+0000","last_unstale":"2024-04-24T23:25:30.407777+0000","last_undegraded":"2024-04-24T23:25:30.407777+0000","last_fullsized":"2024-04-24T23:25:30.407777+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:59.723643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992089+0000","last_change":"2024-04-24T23:25:28.006596+0000","last_active":"2024-04-24T23:25:28.992089+0000","last_peered":"2024-04-24T23:25:28.992089+0000","last_clean":"2024-04-24T23:25:28.992089+0000","last_became_active":"2024-04-24T23:25:28.005475+0000","last_became_peered":"2024-04-24T23:25:28.005475+0000","last_unstale":"2024-04-24T23:25:28.992089+0000","last_undegraded":"2024-04-24T23:25:28.992089+0000","last_fullsized":"2024-04-24T23:25:28.992089+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:56.132083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992912+0000","last_change":"2024-04-24T23:25:26.979978+0000","last_active":"2024-04-24T23:25:28.992912+0000","last_peered":"2024-04-24T23:25:28.992912+0000","last_clean":"2024-04-24T23:25:28.992912+0000","last_became_active":"2024-04-24T23:25:26.978846+0000","last_became_peered":"2024-04-24T23:25:26.978846+0000","last_unstale":"2024-04-24T23:25:28.992912+0000","last_undegraded":"2024-04-24T23:25:28.992912+0000","last_fullsized":"2024-04-24T23:25:28.992912+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:42.850369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993066+0000","last_change":"2024-04-24T23:25:27.991199+0000","last_active":"2024-04-24T23:25:29.993066+0000","last_peered":"2024-04-24T23:25:29.993066+0000","last_clean":"2024-04-24T23:25:29.993066+0000","last_became_active":"2024-04-24T23:25:27.990922+0000","last_became_peered":"2024-04-24T23:25:27.990922+0000","last_unstale":"2024-04-24T23:25:29.993066+0000","last_undegraded":"2024-04-24T23:25:29.993066+0000","last_fullsized":"2024-04-24T23:25:29.993066+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:28.122882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406431+0000","last_change":"2024-04-24T23:25:26.983021+0000","last_active":"2024-04-24T23:25:30.406431+0000","last_peered":"2024-04-24T23:25:30.406431+0000","last_clean":"2024-04-24T23:25:30.406431+0000","last_became_active":"2024-04-24T23:25:26.982757+0000","last_became_peered":"2024-04-24T23:25:26.982757+0000","last_unstale":"2024-04-24T23:25:30.406431+0000","last_undegraded":"2024-04-24T23:25:30.406431+0000","last_fullsized":"2024-04-24T23:25:30.406431+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:01.357710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407142+0000","last_change":"2024-04-24T23:25:27.994927+0000","last_active":"2024-04-24T23:25:30.407142+0000","last_peered":"2024-04-24T23:25:30.407142+0000","last_clean":"2024-04-24T23:25:30.407142+0000","last_became_active":"2024-04-24T23:25:27.994616+0000","last_became_peered":"2024-04-24T23:25:27.994616+0000","last_unstale":"2024-04-24T23:25:30.407142+0000","last_undegraded":"2024-04-24T23:25:30.407142+0000","last_fullsized":"2024-04-24T23:25:30.407142+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:10.750730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.408036+0000","last_change":"2024-04-24T23:25:26.984211+0000","last_active":"2024-04-24T23:25:30.408036+0000","last_peered":"2024-04-24T23:25:30.408036+0000","last_clean":"2024-04-24T23:25:30.408036+0000","last_became_active":"2024-04-24T23:25:26.983878+0000","last_became_peered":"2024-04-24T23:25:26.983878+0000","last_unstale":"2024-04-24T23:25:30.408036+0000","last_undegraded":"2024-04-24T23:25:30.408036+0000","last_fullsized":"2024-04-24T23:25:30.408036+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:15.381258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164226+0000","last_change":"2024-04-24T23:25:27.998147+0000","last_active":"2024-04-24T23:25:29.164226+0000","last_peered":"2024-04-24T23:25:29.164226+0000","last_clean":"2024-04-24T23:25:29.164226+0000","last_became_active":"2024-04-24T23:25:27.997877+0000","last_became_peered":"2024-04-24T23:25:27.997877+0000","last_unstale":"2024-04-24T23:25:29.164226+0000","last_undegraded":"2024-04-24T23:25:29.164226+0000","last_fullsized":"2024-04-24T23:25:29.164226+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:51.479069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407227+0000","last_change":"2024-04-24T23:25:26.988180+0000","last_active":"2024-04-24T23:25:30.407227+0000","last_peered":"2024-04-24T23:25:30.407227+0000","last_clean":"2024-04-24T23:25:30.407227+0000","last_became_active":"2024-04-24T23:25:26.987706+0000","last_became_peered":"2024-04-24T23:25:26.987706+0000","last_unstale":"2024-04-24T23:25:30.407227+0000","last_undegraded":"2024-04-24T23:25:30.407227+0000","last_fullsized":"2024-04-24T23:25:30.407227+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:16.777467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406533+0000","last_change":"2024-04-24T23:25:27.993350+0000","last_active":"2024-04-24T23:25:30.406533+0000","last_peered":"2024-04-24T23:25:30.406533+0000","last_clean":"2024-04-24T23:25:30.406533+0000","last_became_active":"2024-04-24T23:25:27.993011+0000","last_became_peered":"2024-04-24T23:25:27.993011+0000","last_unstale":"2024-04-24T23:25:30.406533+0000","last_undegraded":"2024-04-24T23:25:30.406533+0000","last_fullsized":"2024-04-24T23:25:30.406533+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:32.094641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407990+0000","last_change":"2024-04-24T23:25:26.989891+0000","last_active":"2024-04-24T23:25:30.407990+0000","last_peered":"2024-04-24T23:25:30.407990+0000","last_clean":"2024-04-24T23:25:30.407990+0000","last_became_active":"2024-04-24T23:25:26.989524+0000","last_became_peered":"2024-04-24T23:25:26.989524+0000","last_unstale":"2024-04-24T23:25:30.407990+0000","last_undegraded":"2024-04-24T23:25:30.407990+0000","last_fullsized":"2024-04-24T23:25:30.407990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:12.258864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993464+0000","last_change":"2024-04-24T23:25:27.998329+0000","last_active":"2024-04-24T23:25:29.993464+0000","last_peered":"2024-04-24T23:25:29.993464+0000","last_clean":"2024-04-24T23:25:29.993464+0000","last_became_active":"2024-04-24T23:25:27.998051+0000","last_became_peered":"2024-04-24T23:25:27.998051+0000","last_unstale":"2024-04-24T23:25:29.993464+0000","last_undegraded":"2024-04-24T23:25:29.993464+0000","last_fullsized":"2024-04-24T23:25:29.993464+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:45.612905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163856+0000","last_change":"2024-04-24T23:25:26.980697+0000","last_active":"2024-04-24T23:25:29.163856+0000","last_peered":"2024-04-24T23:25:29.163856+0000","last_clean":"2024-04-24T23:25:29.163856+0000","last_became_active":"2024-04-24T23:25:26.980315+0000","last_became_peered":"2024-04-24T23:25:26.980315+0000","last_unstale":"2024-04-24T23:25:29.163856+0000","last_undegraded":"2024-04-24T23:25:29.163856+0000","last_fullsized":"2024-04-24T23:25:29.163856+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:20.322598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236354+0000","last_change":"2024-04-24T23:25:27.994129+0000","last_active":"2024-04-24T23:25:29.236354+0000","last_peered":"2024-04-24T23:25:29.236354+0000","last_clean":"2024-04-24T23:25:29.236354+0000","last_became_active":"2024-04-24T23:25:27.993811+0000","last_became_peered":"2024-04-24T23:25:27.993811+0000","last_unstale":"2024-04-24T23:25:29.236354+0000","last_undegraded":"2024-04-24T23:25:29.236354+0000","last_fullsized":"2024-04-24T23:25:29.236354+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:39.084955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407955+0000","last_change":"2024-04-24T23:25:26.984839+0000","last_active":"2024-04-24T23:25:30.407955+0000","last_peered":"2024-04-24T23:25:30.407955+0000","last_clean":"2024-04-24T23:25:30.407955+0000","last_became_active":"2024-04-24T23:25:26.984489+0000","last_became_peered":"2024-04-24T23:25:26.984489+0000","last_unstale":"2024-04-24T23:25:30.407955+0000","last_undegraded":"2024-04-24T23:25:30.407955+0000","last_fullsized":"2024-04-24T23:25:30.407955+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:49.523659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236325+0000","last_change":"2024-04-24T23:25:28.005544+0000","last_active":"2024-04-24T23:25:29.236325+0000","last_peered":"2024-04-24T23:25:29.236325+0000","last_clean":"2024-04-24T23:25:29.236325+0000","last_became_active":"2024-04-24T23:25:28.005222+0000","last_became_peered":"2024-04-24T23:25:28.005222+0000","last_unstale":"2024-04-24T23:25:29.236325+0000","last_undegraded":"2024-04-24T23:25:29.236325+0000","last_fullsized":"2024-04-24T23:25:29.236325+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:06.443167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993024+0000","last_change":"2024-04-24T23:25:26.975028+0000","last_active":"2024-04-24T23:25:28.993024+0000","last_peered":"2024-04-24T23:25:28.993024+0000","last_clean":"2024-04-24T23:25:28.993024+0000","last_became_active":"2024-04-24T23:25:26.974741+0000","last_became_peered":"2024-04-24T23:25:26.974741+0000","last_unstale":"2024-04-24T23:25:28.993024+0000","last_undegraded":"2024-04-24T23:25:28.993024+0000","last_fullsized":"2024-04-24T23:25:28.993024+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:46.556429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236384+0000","last_change":"2024-04-24T23:25:28.003440+0000","last_active":"2024-04-24T23:25:29.236384+0000","last_peered":"2024-04-24T23:25:29.236384+0000","last_clean":"2024-04-24T23:25:29.236384+0000","last_became_active":"2024-04-24T23:25:28.003156+0000","last_became_peered":"2024-04-24T23:25:28.003156+0000","last_unstale":"2024-04-24T23:25:29.236384+0000","last_undegraded":"2024-04-24T23:25:29.236384+0000","last_fullsized":"2024-04-24T23:25:29.236384+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:05.319581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992007+0000","last_change":"2024-04-24T23:25:26.986864+0000","last_active":"2024-04-24T23:25:28.992007+0000","last_peered":"2024-04-24T23:25:28.992007+0000","last_clean":"2024-04-24T23:25:28.992007+0000","last_became_active":"2024-04-24T23:25:26.986575+0000","last_became_peered":"2024-04-24T23:25:26.986575+0000","last_unstale":"2024-04-24T23:25:28.992007+0000","last_undegraded":"2024-04-24T23:25:28.992007+0000","last_fullsized":"2024-04-24T23:25:28.992007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:43.615264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992504+0000","last_change":"2024-04-24T23:25:28.004963+0000","last_active":"2024-04-24T23:25:28.992504+0000","last_peered":"2024-04-24T23:25:28.992504+0000","last_clean":"2024-04-24T23:25:28.992504+0000","last_became_active":"2024-04-24T23:25:28.004707+0000","last_became_peered":"2024-04-24T23:25:28.004707+0000","last_unstale":"2024-04-24T23:25:28.992504+0000","last_undegraded":"2024-04-24T23:25:28.992504+0000","last_fullsized":"2024-04-24T23:25:28.992504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:10.421054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.408136+0000","last_change":"2024-04-24T23:25:26.983803+0000","last_active":"2024-04-24T23:25:30.408136+0000","last_peered":"2024-04-24T23:25:30.408136+0000","last_clean":"2024-04-24T23:25:30.408136+0000","last_became_active":"2024-04-24T23:25:26.983465+0000","last_became_peered":"2024-04-24T23:25:26.983465+0000","last_unstale":"2024-04-24T23:25:30.408136+0000","last_undegraded":"2024-04-24T23:25:30.408136+0000","last_fullsized":"2024-04-24T23:25:30.408136+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:47.513215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993311+0000","last_change":"2024-04-24T23:25:27.993787+0000","last_active":"2024-04-24T23:25:28.993311+0000","last_peered":"2024-04-24T23:25:28.993311+0000","last_clean":"2024-04-24T23:25:28.993311+0000","last_became_active":"2024-04-24T23:25:27.993506+0000","last_became_peered":"2024-04-24T23:25:27.993506+0000","last_unstale":"2024-04-24T23:25:28.993311+0000","last_undegraded":"2024-04-24T23:25:28.993311+0000","last_fullsized":"2024-04-24T23:25:28.993311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:41.763030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993486+0000","last_change":"2024-04-24T23:25:26.980977+0000","last_active":"2024-04-24T23:25:28.993486+0000","last_peered":"2024-04-24T23:25:28.993486+0000","last_clean":"2024-04-24T23:25:28.993486+0000","last_became_active":"2024-04-24T23:25:26.979025+0000","last_became_peered":"2024-04-24T23:25:26.979025+0000","last_unstale":"2024-04-24T23:25:28.993486+0000","last_undegraded":"2024-04-24T23:25:28.993486+0000","last_fullsized":"2024-04-24T23:25:28.993486+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:43.615264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163839+0000","last_change":"2024-04-24T23:25:28.009828+0000","last_active":"2024-04-24T23:25:29.163839+0000","last_peered":"2024-04-24T23:25:29.163839+0000","last_clean":"2024-04-24T23:25:29.163839+0000","last_became_active":"2024-04-24T23:25:28.009151+0000","last_became_peered":"2024-04-24T23:25:28.009151+0000","last_unstale":"2024-04-24T23:25:29.163839+0000","last_undegraded":"2024-04-24T23:25:29.163839+0000","last_fullsized":"2024-04-24T23:25:29.163839+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:58.729537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993610+0000","last_change":"2024-04-24T23:25:26.984615+0000","last_active":"2024-04-24T23:25:29.993610+0000","last_peered":"2024-04-24T23:25:29.993610+0000","last_clean":"2024-04-24T23:25:29.993610+0000","last_became_active":"2024-04-24T23:25:26.984358+0000","last_became_peered":"2024-04-24T23:25:26.984358+0000","last_unstale":"2024-04-24T23:25:29.993610+0000","last_undegraded":"2024-04-24T23:25:29.993610+0000","last_fullsized":"2024-04-24T23:25:29.993610+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:50.649710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164064+0000","last_change":"2024-04-24T23:25:28.007442+0000","last_active":"2024-04-24T23:25:29.164064+0000","last_peered":"2024-04-24T23:25:29.164064+0000","last_clean":"2024-04-24T23:25:29.164064+0000","last_became_active":"2024-04-24T23:25:28.007170+0000","last_became_peered":"2024-04-24T23:25:28.007170+0000","last_unstale":"2024-04-24T23:25:29.164064+0000","last_undegraded":"2024-04-24T23:25:29.164064+0000","last_fullsized":"2024-04-24T23:25:29.164064+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:11.391828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992356+0000","last_change":"2024-04-24T23:25:26.988155+0000","last_active":"2024-04-24T23:25:28.992356+0000","last_peered":"2024-04-24T23:25:28.992356+0000","last_clean":"2024-04-24T23:25:28.992356+0000","last_became_active":"2024-04-24T23:25:26.987824+0000","last_became_peered":"2024-04-24T23:25:26.987824+0000","last_unstale":"2024-04-24T23:25:28.992356+0000","last_undegraded":"2024-04-24T23:25:28.992356+0000","last_fullsized":"2024-04-24T23:25:28.992356+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:01.501491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993514+0000","last_change":"2024-04-24T23:25:27.992164+0000","last_active":"2024-04-24T23:25:29.993514+0000","last_peered":"2024-04-24T23:25:29.993514+0000","last_clean":"2024-04-24T23:25:29.993514+0000","last_became_active":"2024-04-24T23:25:27.991877+0000","last_became_peered":"2024-04-24T23:25:27.991877+0000","last_unstale":"2024-04-24T23:25:29.993514+0000","last_undegraded":"2024-04-24T23:25:29.993514+0000","last_fullsized":"2024-04-24T23:25:29.993514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:00.727972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993402+0000","last_change":"2024-04-24T23:25:26.976419+0000","last_active":"2024-04-24T23:25:29.993402+0000","last_peered":"2024-04-24T23:25:29.993402+0000","last_clean":"2024-04-24T23:25:29.993402+0000","last_became_active":"2024-04-24T23:25:26.976109+0000","last_became_peered":"2024-04-24T23:25:26.976109+0000","last_unstale":"2024-04-24T23:25:29.993402+0000","last_undegraded":"2024-04-24T23:25:29.993402+0000","last_fullsized":"2024-04-24T23:25:29.993402+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:03.090729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407897+0000","last_change":"2024-04-24T23:25:27.998536+0000","last_active":"2024-04-24T23:25:30.407897+0000","last_peered":"2024-04-24T23:25:30.407897+0000","last_clean":"2024-04-24T23:25:30.407897+0000","last_became_active":"2024-04-24T23:25:27.997773+0000","last_became_peered":"2024-04-24T23:25:27.997773+0000","last_unstale":"2024-04-24T23:25:30.407897+0000","last_undegraded":"2024-04-24T23:25:30.407897+0000","last_fullsized":"2024-04-24T23:25:30.407897+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:58.899788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406177+0000","last_change":"2024-04-24T23:25:26.985538+0000","last_active":"2024-04-24T23:25:30.406177+0000","last_peered":"2024-04-24T23:25:30.406177+0000","last_clean":"2024-04-24T23:25:30.406177+0000","last_became_active":"2024-04-24T23:25:26.985216+0000","last_became_peered":"2024-04-24T23:25:26.985216+0000","last_unstale":"2024-04-24T23:25:30.406177+0000","last_undegraded":"2024-04-24T23:25:30.406177+0000","last_fullsized":"2024-04-24T23:25:30.406177+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:32.695590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406217+0000","last_change":"2024-04-24T23:25:28.001468+0000","last_active":"2024-04-24T23:25:30.406217+0000","last_peered":"2024-04-24T23:25:30.406217+0000","last_clean":"2024-04-24T23:25:30.406217+0000","last_became_active":"2024-04-24T23:25:28.001179+0000","last_became_peered":"2024-04-24T23:25:28.001179+0000","last_unstale":"2024-04-24T23:25:30.406217+0000","last_undegraded":"2024-04-24T23:25:30.406217+0000","last_fullsized":"2024-04-24T23:25:30.406217+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:17.781795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406372+0000","last_change":"2024-04-24T23:25:26.978070+0000","last_active":"2024-04-24T23:25:30.406372+0000","last_peered":"2024-04-24T23:25:30.406372+0000","last_clean":"2024-04-24T23:25:30.406372+0000","last_became_active":"2024-04-24T23:25:26.977796+0000","last_became_peered":"2024-04-24T23:25:26.977796+0000","last_unstale":"2024-04-24T23:25:30.406372+0000","last_undegraded":"2024-04-24T23:25:30.406372+0000","last_fullsized":"2024-04-24T23:25:30.406372+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:53.598202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407502+0000","last_change":"2024-04-24T23:25:27.994749+0000","last_active":"2024-04-24T23:25:30.407502+0000","last_peered":"2024-04-24T23:25:30.407502+0000","last_clean":"2024-04-24T23:25:30.407502+0000","last_became_active":"2024-04-24T23:25:27.994402+0000","last_became_peered":"2024-04-24T23:25:27.994402+0000","last_unstale":"2024-04-24T23:25:30.407502+0000","last_undegraded":"2024-04-24T23:25:30.407502+0000","last_fullsized":"2024-04-24T23:25:30.407502+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:42.428738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407300+0000","last_change":"2024-04-24T23:25:26.985857+0000","last_active":"2024-04-24T23:25:30.407300+0000","last_peered":"2024-04-24T23:25:30.407300+0000","last_clean":"2024-04-24T23:25:30.407300+0000","last_became_active":"2024-04-24T23:25:26.985337+0000","last_became_peered":"2024-04-24T23:25:26.985337+0000","last_unstale":"2024-04-24T23:25:30.407300+0000","last_undegraded":"2024-04-24T23:25:30.407300+0000","last_fullsized":"2024-04-24T23:25:30.407300+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:26.544193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236451+0000","last_change":"2024-04-24T23:25:28.004409+0000","last_active":"2024-04-24T23:25:29.236451+0000","last_peered":"2024-04-24T23:25:29.236451+0000","last_clean":"2024-04-24T23:25:29.236451+0000","last_became_active":"2024-04-24T23:25:28.004111+0000","last_became_peered":"2024-04-24T23:25:28.004111+0000","last_unstale":"2024-04-24T23:25:29.236451+0000","last_undegraded":"2024-04-24T23:25:29.236451+0000","last_fullsized":"2024-04-24T23:25:29.236451+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:50.232135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407598+0000","last_change":"2024-04-24T23:25:26.986149+0000","last_active":"2024-04-24T23:25:30.407598+0000","last_peered":"2024-04-24T23:25:30.407598+0000","last_clean":"2024-04-24T23:25:30.407598+0000","last_became_active":"2024-04-24T23:25:26.985763+0000","last_became_peered":"2024-04-24T23:25:26.985763+0000","last_unstale":"2024-04-24T23:25:30.407598+0000","last_undegraded":"2024-04-24T23:25:30.407598+0000","last_fullsized":"2024-04-24T23:25:30.407598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:32.695590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":258,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163495+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:25:29.163495+0000","last_peered":"2024-04-24T23:25:29.163495+0000","last_clean":"2024-04-24T23:25:29.163495+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:25:29.163495+0000","last_undegraded":"2024-04-24T23:25:29.163495+0000","last_fullsized":"2024-04-24T23:25:29.163495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":948165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":357273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":357271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":357251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":847872,"data_stored":357293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36172,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335668,"statfs":{"total":96636764160,"available":96599724032,"internally_reserved":0,"allocated":851968,"data_stored":358793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":359409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":1433600,"data_stored":947549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:25:36.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:25:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.551+0000 7efc06770640 1 Processor -- start 2024-04-24T23:25:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.551+0000 7efc06770640 1 -- start start 2024-04-24T23:25:36.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efc06770640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc0015e290 0x7efc000d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efc06770640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc000b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efc06770640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc000b5840 con 0x7efc000b3ac0 2024-04-24T23:25:36.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efc06770640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7efc000b5e50 con 0x7efc0015e290 2024-04-24T23:25:36.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efc06770640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc000cfb30 con 0x7efc000b72f0 2024-04-24T23:25:36.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efbfffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc0015e290 0x7efc000d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efbff7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efc000b3ac0 0x7efc0006c230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41184/0 (socket says 172.21.15.26:41184) 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efbff7fe640 1 -- 172.21.15.26:0/2164401759 learned_addr learned my addr 172.21.15.26:0/2164401759 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efc04d0d640 1 --2- 172.21.15.26:0/2164401759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc000b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efbfffff640 1 -- 172.21.15.26:0/2164401759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 msgr2=0x7efc000b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efbfffff640 1 --2- 172.21.15.26:0/2164401759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc000b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efbfffff640 1 -- 172.21.15.26:0/2164401759 >> v1:172.21.15.26:6789/0 conn(0x7efc000b3ac0 legacy=0x7efc0006c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efbfeffd640 1 -- 172.21.15.26:0/2164401759 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3615594828 0 0) 0x7efc000b5840 con 0x7efc000b3ac0 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efbfffff640 1 -- 172.21.15.26:0/2164401759 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc000cfe10 con 0x7efc0015e290 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.552+0000 7efc04d0d640 1 --2- 172.21.15.26:0/2164401759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc000b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.553+0000 7efbfffff640 1 --2- 172.21.15.26:0/2164401759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 0x7efc000d1730 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7efbe80097a0 tx=0x7efbe802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629a94239a7eaa26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.553+0000 7efbfeffd640 1 -- 172.21.15.26:0/2164401759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8037070 con 0x7efc0015e290 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.553+0000 7efbfeffd640 1 -- 172.21.15.26:0/2164401759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe8032440 con 0x7efc0015e290 2024-04-24T23:25:36.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.553+0000 7efbfeffd640 1 -- 172.21.15.26:0/2164401759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8032810 con 0x7efc0015e290 2024-04-24T23:25:36.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.554+0000 7efc06770640 1 -- 172.21.15.26:0/2164401759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 msgr2=0x7efc000d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:36.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.554+0000 7efc06770640 1 --2- 172.21.15.26:0/2164401759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 0x7efc000d1730 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7efbe80097a0 tx=0x7efbe802a180 comp rx=0 tx=0).stop 2024-04-24T23:25:36.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.554+0000 7efc06770640 1 -- 172.21.15.26:0/2164401759 shutdown_connections 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.554+0000 7efc06770640 1 --2- 172.21.15.26:0/2164401759 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc000b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.554+0000 7efc06770640 1 --2- 172.21.15.26:0/2164401759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 0x7efc000d1730 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.554+0000 7efc06770640 1 -- 172.21.15.26:0/2164401759 >> 172.21.15.26:0/2164401759 conn(0x7efc0006b120 msgr2=0x7efc0006b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.554+0000 7efc06770640 1 -- 172.21.15.26:0/2164401759 shutdown_connections 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.554+0000 7efc06770640 1 -- 172.21.15.26:0/2164401759 wait complete. 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.555+0000 7efc06770640 1 Processor -- start 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.555+0000 7efc06770640 1 -- start start 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc06770640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc000b3ac0 0x7efc000cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc06770640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc0006c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc06770640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 0x7efc0015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc06770640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc0015e670 con 0x7efc0015e290 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc06770640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc000b5ae0 con 0x7efc000b3ac0 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc06770640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc000d1c70 con 0x7efc000b72f0 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc04d0d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 0x7efc0015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc04d0d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 0x7efc0015c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55272/0 (socket says 172.21.15.26:55272) 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc04d0d640 1 -- 172.21.15.26:0/2484709705 learned_addr learned my addr 172.21.15.26:0/2484709705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efbff7fe640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc0006c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efbfffff640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc000b3ac0 0x7efc000cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc04d0d640 1 -- 172.21.15.26:0/2484709705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 msgr2=0x7efc0006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc04d0d640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc0006c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc04d0d640 1 -- 172.21.15.26:0/2484709705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc000b3ac0 msgr2=0x7efc000cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc04d0d640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc000b3ac0 0x7efc000cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efc04d0d640 1 -- 172.21.15.26:0/2484709705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc000c9c50 con 0x7efc0015e290 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.556+0000 7efbff7fe640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc0006c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:36.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.557+0000 7efc04d0d640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 0x7efc0015c090 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7efbf800cef0 tx=0x7efbf800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:36.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.557+0000 7efbfcff9640 1 -- 172.21.15.26:0/2484709705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf801a070 con 0x7efc0015e290 2024-04-24T23:25:36.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.557+0000 7efbfcff9640 1 -- 172.21.15.26:0/2484709705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf800d860 con 0x7efc0015e290 2024-04-24T23:25:36.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.557+0000 7efbfcff9640 1 -- 172.21.15.26:0/2484709705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf800dc50 con 0x7efc0015e290 2024-04-24T23:25:36.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.557+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc000c9f30 con 0x7efc0015e290 2024-04-24T23:25:36.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.557+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0015fa90 con 0x7efc0015e290 2024-04-24T23:25:36.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.558+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0015fc90 con 0x7efc0015e290 2024-04-24T23:25:36.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.558+0000 7efbfcff9640 1 -- 172.21.15.26:0/2484709705 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efbf801d070 con 0x7efc0015e290 2024-04-24T23:25:36.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.559+0000 7efbfcff9640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbe006be30 0x7efbe006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.559+0000 7efbfffff640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbe006be30 0x7efbe006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.559+0000 7efbfcff9640 1 -- 172.21.15.26:0/2484709705 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5767+0+0 (secure 0 0 0) 0x7efbf808b6d0 con 0x7efc0015e290 2024-04-24T23:25:36.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.559+0000 7efbfffff640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbe006be30 0x7efbe006e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efbe8034000 tx=0x7efbe802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:36.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.564+0000 7efbfcff9640 1 -- 172.21.15.26:0/2484709705 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc0015fc90 con 0x7efc0015e290 2024-04-24T23:25:36.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.742+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc000694c0 con 0x7efbe006be30 2024-04-24T23:25:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.748+0000 7efbfcff9640 1 -- 172.21.15.26:0/2484709705 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7efc000694c0 con 0x7efbe006be30 2024-04-24T23:25:36.752 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbe006be30 msgr2=0x7efbe006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbe006be30 0x7efbe006e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efbe8034000 tx=0x7efbe802e000 comp rx=0 tx=0).stop 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 msgr2=0x7efc0015c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 0x7efc0015c090 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7efbf800cef0 tx=0x7efbf800a710 comp rx=0 tx=0).stop 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 shutdown_connections 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbe006be30 0x7efbe006e2f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0015e290 0x7efc0015c090 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc000b72f0 0x7efc0006c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 --2- 172.21.15.26:0/2484709705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc000b3ac0 0x7efc000cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 >> 172.21.15.26:0/2484709705 conn(0x7efc0006b120 msgr2=0x7efc0006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 shutdown_connections 2024-04-24T23:25:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.751+0000 7efc06770640 1 -- 172.21.15.26:0/2484709705 wait complete. 2024-04-24T23:25:36.763 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-04-24T23:25:36.019232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265568,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754709152,"statfs":{"total":773094113280,"available":772822171648,"internally_reserved":0,"allocated":7970816,"data_stored":4043004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.038949"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164104+0000","last_change":"2024-04-24T23:25:28.001691+0000","last_active":"2024-04-24T23:25:29.164104+0000","last_peered":"2024-04-24T23:25:29.164104+0000","last_clean":"2024-04-24T23:25:29.164104+0000","last_became_active":"2024-04-24T23:25:28.001394+0000","last_became_peered":"2024-04-24T23:25:28.001394+0000","last_unstale":"2024-04-24T23:25:29.164104+0000","last_undegraded":"2024-04-24T23:25:29.164104+0000","last_fullsized":"2024-04-24T23:25:29.164104+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:36.386100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.408066+0000","last_change":"2024-04-24T23:25:28.003937+0000","last_active":"2024-04-24T23:25:30.408066+0000","last_peered":"2024-04-24T23:25:30.408066+0000","last_clean":"2024-04-24T23:25:30.408066+0000","last_became_active":"2024-04-24T23:25:28.001712+0000","last_became_peered":"2024-04-24T23:25:28.001712+0000","last_unstale":"2024-04-24T23:25:30.408066+0000","last_undegraded":"2024-04-24T23:25:30.408066+0000","last_fullsized":"2024-04-24T23:25:30.408066+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:47.560758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236442+0000","last_change":"2024-04-24T23:25:27.999495+0000","last_active":"2024-04-24T23:25:29.236442+0000","last_peered":"2024-04-24T23:25:29.236442+0000","last_clean":"2024-04-24T23:25:29.236442+0000","last_became_active":"2024-04-24T23:25:27.999220+0000","last_became_peered":"2024-04-24T23:25:27.999220+0000","last_unstale":"2024-04-24T23:25:29.236442+0000","last_undegraded":"2024-04-24T23:25:29.236442+0000","last_fullsized":"2024-04-24T23:25:29.236442+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:19.252039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406426+0000","last_change":"2024-04-24T23:25:27.997705+0000","last_active":"2024-04-24T23:25:30.406426+0000","last_peered":"2024-04-24T23:25:30.406426+0000","last_clean":"2024-04-24T23:25:30.406426+0000","last_became_active":"2024-04-24T23:25:27.997415+0000","last_became_peered":"2024-04-24T23:25:27.997415+0000","last_unstale":"2024-04-24T23:25:30.406426+0000","last_undegraded":"2024-04-24T23:25:30.406426+0000","last_fullsized":"2024-04-24T23:25:30.406426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:08.524873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993378+0000","last_change":"2024-04-24T23:25:27.996522+0000","last_active":"2024-04-24T23:25:29.993378+0000","last_peered":"2024-04-24T23:25:29.993378+0000","last_clean":"2024-04-24T23:25:29.993378+0000","last_became_active":"2024-04-24T23:25:27.996253+0000","last_became_peered":"2024-04-24T23:25:27.996253+0000","last_unstale":"2024-04-24T23:25:29.993378+0000","last_undegraded":"2024-04-24T23:25:29.993378+0000","last_fullsized":"2024-04-24T23:25:29.993378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:29:07.473497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407620+0000","last_change":"2024-04-24T23:25:27.999832+0000","last_active":"2024-04-24T23:25:30.407620+0000","last_peered":"2024-04-24T23:25:30.407620+0000","last_clean":"2024-04-24T23:25:30.407620+0000","last_became_active":"2024-04-24T23:25:27.999381+0000","last_became_peered":"2024-04-24T23:25:27.999381+0000","last_unstale":"2024-04-24T23:25:30.407620+0000","last_undegraded":"2024-04-24T23:25:30.407620+0000","last_fullsized":"2024-04-24T23:25:30.407620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:57.738709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992886+0000","last_change":"2024-04-24T23:25:27.993916+0000","last_active":"2024-04-24T23:25:28.992886+0000","last_peered":"2024-04-24T23:25:28.992886+0000","last_clean":"2024-04-24T23:25:28.992886+0000","last_became_active":"2024-04-24T23:25:27.993650+0000","last_became_peered":"2024-04-24T23:25:27.993650+0000","last_unstale":"2024-04-24T23:25:28.992886+0000","last_undegraded":"2024-04-24T23:25:28.992886+0000","last_fullsized":"2024-04-24T23:25:28.992886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:39.831794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407073+0000","last_change":"2024-04-24T23:25:28.002407+0000","last_active":"2024-04-24T23:25:30.407073+0000","last_peered":"2024-04-24T23:25:30.407073+0000","last_clean":"2024-04-24T23:25:30.407073+0000","last_became_active":"2024-04-24T23:25:28.000515+0000","last_became_peered":"2024-04-24T23:25:28.000515+0000","last_unstale":"2024-04-24T23:25:30.407073+0000","last_undegraded":"2024-04-24T23:25:30.407073+0000","last_fullsized":"2024-04-24T23:25:30.407073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:35.022298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993608+0000","last_change":"2024-04-24T23:25:27.994672+0000","last_active":"2024-04-24T23:25:29.993608+0000","last_peered":"2024-04-24T23:25:29.993608+0000","last_clean":"2024-04-24T23:25:29.993608+0000","last_became_active":"2024-04-24T23:25:27.994408+0000","last_became_peered":"2024-04-24T23:25:27.994408+0000","last_unstale":"2024-04-24T23:25:29.993608+0000","last_undegraded":"2024-04-24T23:25:29.993608+0000","last_fullsized":"2024-04-24T23:25:29.993608+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:13.263193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407106+0000","last_change":"2024-04-24T23:25:28.004663+0000","last_active":"2024-04-24T23:25:30.407106+0000","last_peered":"2024-04-24T23:25:30.407106+0000","last_clean":"2024-04-24T23:25:30.407106+0000","last_became_active":"2024-04-24T23:25:28.002731+0000","last_became_peered":"2024-04-24T23:25:28.002731+0000","last_unstale":"2024-04-24T23:25:30.407106+0000","last_undegraded":"2024-04-24T23:25:30.407106+0000","last_fullsized":"2024-04-24T23:25:30.407106+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:29.630244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163844+0000","last_change":"2024-04-24T23:25:28.003371+0000","last_active":"2024-04-24T23:25:29.163844+0000","last_peered":"2024-04-24T23:25:29.163844+0000","last_clean":"2024-04-24T23:25:29.163844+0000","last_became_active":"2024-04-24T23:25:28.003065+0000","last_became_peered":"2024-04-24T23:25:28.003065+0000","last_unstale":"2024-04-24T23:25:29.163844+0000","last_undegraded":"2024-04-24T23:25:29.163844+0000","last_fullsized":"2024-04-24T23:25:29.163844+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:55.924853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406422+0000","last_change":"2024-04-24T23:25:27.996620+0000","last_active":"2024-04-24T23:25:30.406422+0000","last_peered":"2024-04-24T23:25:30.406422+0000","last_clean":"2024-04-24T23:25:30.406422+0000","last_became_active":"2024-04-24T23:25:27.996355+0000","last_became_peered":"2024-04-24T23:25:27.996355+0000","last_unstale":"2024-04-24T23:25:30.406422+0000","last_undegraded":"2024-04-24T23:25:30.406422+0000","last_fullsized":"2024-04-24T23:25:30.406422+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:30.598436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992392+0000","last_change":"2024-04-24T23:25:28.005958+0000","last_active":"2024-04-24T23:25:28.992392+0000","last_peered":"2024-04-24T23:25:28.992392+0000","last_clean":"2024-04-24T23:25:28.992392+0000","last_became_active":"2024-04-24T23:25:28.005394+0000","last_became_peered":"2024-04-24T23:25:28.005394+0000","last_unstale":"2024-04-24T23:25:28.992392+0000","last_undegraded":"2024-04-24T23:25:28.992392+0000","last_fullsized":"2024-04-24T23:25:28.992392+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:41.763030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993440+0000","last_change":"2024-04-24T23:25:27.992504+0000","last_active":"2024-04-24T23:25:29.993440+0000","last_peered":"2024-04-24T23:25:29.993440+0000","last_clean":"2024-04-24T23:25:29.993440+0000","last_became_active":"2024-04-24T23:25:27.989888+0000","last_became_peered":"2024-04-24T23:25:27.989888+0000","last_unstale":"2024-04-24T23:25:29.993440+0000","last_undegraded":"2024-04-24T23:25:29.993440+0000","last_fullsized":"2024-04-24T23:25:29.993440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:12.681568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992149+0000","last_change":"2024-04-24T23:25:27.993212+0000","last_active":"2024-04-24T23:25:28.992149+0000","last_peered":"2024-04-24T23:25:28.992149+0000","last_clean":"2024-04-24T23:25:28.992149+0000","last_became_active":"2024-04-24T23:25:27.992908+0000","last_became_peered":"2024-04-24T23:25:27.992908+0000","last_unstale":"2024-04-24T23:25:28.992149+0000","last_undegraded":"2024-04-24T23:25:28.992149+0000","last_fullsized":"2024-04-24T23:25:28.992149+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:33.603412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406397+0000","last_change":"2024-04-24T23:25:28.010143+0000","last_active":"2024-04-24T23:25:30.406397+0000","last_peered":"2024-04-24T23:25:30.406397+0000","last_clean":"2024-04-24T23:25:30.406397+0000","last_became_active":"2024-04-24T23:25:28.009856+0000","last_became_peered":"2024-04-24T23:25:28.009856+0000","last_unstale":"2024-04-24T23:25:30.406397+0000","last_undegraded":"2024-04-24T23:25:30.406397+0000","last_fullsized":"2024-04-24T23:25:30.406397+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:43.854697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993750+0000","last_change":"2024-04-24T23:25:27.995041+0000","last_active":"2024-04-24T23:25:29.993750+0000","last_peered":"2024-04-24T23:25:29.993750+0000","last_clean":"2024-04-24T23:25:29.993750+0000","last_became_active":"2024-04-24T23:25:27.991668+0000","last_became_peered":"2024-04-24T23:25:27.991668+0000","last_unstale":"2024-04-24T23:25:29.993750+0000","last_undegraded":"2024-04-24T23:25:29.993750+0000","last_fullsized":"2024-04-24T23:25:29.993750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:16.385587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406481+0000","last_change":"2024-04-24T23:25:27.998171+0000","last_active":"2024-04-24T23:25:30.406481+0000","last_peered":"2024-04-24T23:25:30.406481+0000","last_clean":"2024-04-24T23:25:30.406481+0000","last_became_active":"2024-04-24T23:25:27.997720+0000","last_became_peered":"2024-04-24T23:25:27.997720+0000","last_unstale":"2024-04-24T23:25:30.406481+0000","last_undegraded":"2024-04-24T23:25:30.406481+0000","last_fullsized":"2024-04-24T23:25:30.406481+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:02.362039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407872+0000","last_change":"2024-04-24T23:25:27.994062+0000","last_active":"2024-04-24T23:25:30.407872+0000","last_peered":"2024-04-24T23:25:30.407872+0000","last_clean":"2024-04-24T23:25:30.407872+0000","last_became_active":"2024-04-24T23:25:27.993608+0000","last_became_peered":"2024-04-24T23:25:27.993608+0000","last_unstale":"2024-04-24T23:25:30.407872+0000","last_undegraded":"2024-04-24T23:25:30.407872+0000","last_fullsized":"2024-04-24T23:25:30.407872+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:07.049044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992400+0000","last_change":"2024-04-24T23:25:28.001925+0000","last_active":"2024-04-24T23:25:28.992400+0000","last_peered":"2024-04-24T23:25:28.992400+0000","last_clean":"2024-04-24T23:25:28.992400+0000","last_became_active":"2024-04-24T23:25:28.001658+0000","last_became_peered":"2024-04-24T23:25:28.001658+0000","last_unstale":"2024-04-24T23:25:28.992400+0000","last_undegraded":"2024-04-24T23:25:28.992400+0000","last_fullsized":"2024-04-24T23:25:28.992400+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:12.033609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406776+0000","last_change":"2024-04-24T23:25:27.996181+0000","last_active":"2024-04-24T23:25:30.406776+0000","last_peered":"2024-04-24T23:25:30.406776+0000","last_clean":"2024-04-24T23:25:30.406776+0000","last_became_active":"2024-04-24T23:25:27.995846+0000","last_became_peered":"2024-04-24T23:25:27.995846+0000","last_unstale":"2024-04-24T23:25:30.406776+0000","last_undegraded":"2024-04-24T23:25:30.406776+0000","last_fullsized":"2024-04-24T23:25:30.406776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:44.619593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406653+0000","last_change":"2024-04-24T23:25:28.002416+0000","last_active":"2024-04-24T23:25:30.406653+0000","last_peered":"2024-04-24T23:25:30.406653+0000","last_clean":"2024-04-24T23:25:30.406653+0000","last_became_active":"2024-04-24T23:25:28.002073+0000","last_became_peered":"2024-04-24T23:25:28.002073+0000","last_unstale":"2024-04-24T23:25:30.406653+0000","last_undegraded":"2024-04-24T23:25:30.406653+0000","last_fullsized":"2024-04-24T23:25:30.406653+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:47.560758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163642+0000","last_change":"2024-04-24T23:25:27.999950+0000","last_active":"2024-04-24T23:25:29.163642+0000","last_peered":"2024-04-24T23:25:29.163642+0000","last_clean":"2024-04-24T23:25:29.163642+0000","last_became_active":"2024-04-24T23:25:27.999540+0000","last_became_peered":"2024-04-24T23:25:27.999540+0000","last_unstale":"2024-04-24T23:25:29.163642+0000","last_undegraded":"2024-04-24T23:25:29.163642+0000","last_fullsized":"2024-04-24T23:25:29.163642+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:07.983258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163505+0000","last_change":"2024-04-24T23:25:28.008767+0000","last_active":"2024-04-24T23:25:29.163505+0000","last_peered":"2024-04-24T23:25:29.163505+0000","last_clean":"2024-04-24T23:25:29.163505+0000","last_became_active":"2024-04-24T23:25:28.008459+0000","last_became_peered":"2024-04-24T23:25:28.008459+0000","last_unstale":"2024-04-24T23:25:29.163505+0000","last_undegraded":"2024-04-24T23:25:29.163505+0000","last_fullsized":"2024-04-24T23:25:29.163505+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:53.596699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407139+0000","last_change":"2024-04-24T23:25:28.002914+0000","last_active":"2024-04-24T23:25:30.407139+0000","last_peered":"2024-04-24T23:25:30.407139+0000","last_clean":"2024-04-24T23:25:30.407139+0000","last_became_active":"2024-04-24T23:25:28.002018+0000","last_became_peered":"2024-04-24T23:25:28.002018+0000","last_unstale":"2024-04-24T23:25:30.407139+0000","last_undegraded":"2024-04-24T23:25:30.407139+0000","last_fullsized":"2024-04-24T23:25:30.407139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:48.807209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993321+0000","last_change":"2024-04-24T23:25:28.004061+0000","last_active":"2024-04-24T23:25:28.993321+0000","last_peered":"2024-04-24T23:25:28.993321+0000","last_clean":"2024-04-24T23:25:28.993321+0000","last_became_active":"2024-04-24T23:25:28.003756+0000","last_became_peered":"2024-04-24T23:25:28.003756+0000","last_unstale":"2024-04-24T23:25:28.993321+0000","last_undegraded":"2024-04-24T23:25:28.993321+0000","last_fullsized":"2024-04-24T23:25:28.993321+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:12.033609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236574+0000","last_change":"2024-04-24T23:25:27.990021+0000","last_active":"2024-04-24T23:25:29.236574+0000","last_peered":"2024-04-24T23:25:29.236574+0000","last_clean":"2024-04-24T23:25:29.236574+0000","last_became_active":"2024-04-24T23:25:27.989687+0000","last_became_peered":"2024-04-24T23:25:27.989687+0000","last_unstale":"2024-04-24T23:25:29.236574+0000","last_undegraded":"2024-04-24T23:25:29.236574+0000","last_fullsized":"2024-04-24T23:25:29.236574+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:55.576823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407040+0000","last_change":"2024-04-24T23:25:27.999162+0000","last_active":"2024-04-24T23:25:30.407040+0000","last_peered":"2024-04-24T23:25:30.407040+0000","last_clean":"2024-04-24T23:25:30.407040+0000","last_became_active":"2024-04-24T23:25:27.998838+0000","last_became_peered":"2024-04-24T23:25:27.998838+0000","last_unstale":"2024-04-24T23:25:30.407040+0000","last_undegraded":"2024-04-24T23:25:30.407040+0000","last_fullsized":"2024-04-24T23:25:30.407040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:21.326927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236422+0000","last_change":"2024-04-24T23:25:27.997325+0000","last_active":"2024-04-24T23:25:29.236422+0000","last_peered":"2024-04-24T23:25:29.236422+0000","last_clean":"2024-04-24T23:25:29.236422+0000","last_became_active":"2024-04-24T23:25:27.997036+0000","last_became_peered":"2024-04-24T23:25:27.997036+0000","last_unstale":"2024-04-24T23:25:29.236422+0000","last_undegraded":"2024-04-24T23:25:29.236422+0000","last_fullsized":"2024-04-24T23:25:29.236422+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:50.668477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992943+0000","last_change":"2024-04-24T23:25:28.007569+0000","last_active":"2024-04-24T23:25:28.992943+0000","last_peered":"2024-04-24T23:25:28.992943+0000","last_clean":"2024-04-24T23:25:28.992943+0000","last_became_active":"2024-04-24T23:25:28.007220+0000","last_became_peered":"2024-04-24T23:25:28.007220+0000","last_unstale":"2024-04-24T23:25:28.992943+0000","last_undegraded":"2024-04-24T23:25:28.992943+0000","last_fullsized":"2024-04-24T23:25:28.992943+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:56.132083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993329+0000","last_change":"2024-04-24T23:25:27.993307+0000","last_active":"2024-04-24T23:25:28.993329+0000","last_peered":"2024-04-24T23:25:28.993329+0000","last_clean":"2024-04-24T23:25:28.993329+0000","last_became_active":"2024-04-24T23:25:27.993030+0000","last_became_peered":"2024-04-24T23:25:27.993030+0000","last_unstale":"2024-04-24T23:25:28.993329+0000","last_undegraded":"2024-04-24T23:25:28.993329+0000","last_fullsized":"2024-04-24T23:25:28.993329+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:10.421054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407328+0000","last_change":"2024-04-24T23:25:28.002497+0000","last_active":"2024-04-24T23:25:30.407328+0000","last_peered":"2024-04-24T23:25:30.407328+0000","last_clean":"2024-04-24T23:25:30.407328+0000","last_became_active":"2024-04-24T23:25:28.002164+0000","last_became_peered":"2024-04-24T23:25:28.002164+0000","last_unstale":"2024-04-24T23:25:30.407328+0000","last_undegraded":"2024-04-24T23:25:30.407328+0000","last_fullsized":"2024-04-24T23:25:30.407328+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:38.792880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163995+0000","last_change":"2024-04-24T23:25:28.005542+0000","last_active":"2024-04-24T23:25:29.163995+0000","last_peered":"2024-04-24T23:25:29.163995+0000","last_clean":"2024-04-24T23:25:29.163995+0000","last_became_active":"2024-04-24T23:25:28.005236+0000","last_became_peered":"2024-04-24T23:25:28.005236+0000","last_unstale":"2024-04-24T23:25:29.163995+0000","last_undegraded":"2024-04-24T23:25:29.163995+0000","last_fullsized":"2024-04-24T23:25:29.163995+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:10.560634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164196+0000","last_change":"2024-04-24T23:25:26.984830+0000","last_active":"2024-04-24T23:25:29.164196+0000","last_peered":"2024-04-24T23:25:29.164196+0000","last_clean":"2024-04-24T23:25:29.164196+0000","last_became_active":"2024-04-24T23:25:26.983743+0000","last_became_peered":"2024-04-24T23:25:26.983743+0000","last_unstale":"2024-04-24T23:25:29.164196+0000","last_undegraded":"2024-04-24T23:25:29.164196+0000","last_fullsized":"2024-04-24T23:25:29.164196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:28.625915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993757+0000","last_change":"2024-04-24T23:25:27.991300+0000","last_active":"2024-04-24T23:25:29.993757+0000","last_peered":"2024-04-24T23:25:29.993757+0000","last_clean":"2024-04-24T23:25:29.993757+0000","last_became_active":"2024-04-24T23:25:27.991006+0000","last_became_peered":"2024-04-24T23:25:27.991006+0000","last_unstale":"2024-04-24T23:25:29.993757+0000","last_undegraded":"2024-04-24T23:25:29.993757+0000","last_fullsized":"2024-04-24T23:25:29.993757+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:50.527988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992493+0000","last_change":"2024-04-24T23:25:26.986600+0000","last_active":"2024-04-24T23:25:28.992493+0000","last_peered":"2024-04-24T23:25:28.992493+0000","last_clean":"2024-04-24T23:25:28.992493+0000","last_became_active":"2024-04-24T23:25:26.986208+0000","last_became_peered":"2024-04-24T23:25:26.986208+0000","last_unstale":"2024-04-24T23:25:28.992493+0000","last_undegraded":"2024-04-24T23:25:28.992493+0000","last_fullsized":"2024-04-24T23:25:28.992493+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:20.322598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407797+0000","last_change":"2024-04-24T23:25:27.995888+0000","last_active":"2024-04-24T23:25:30.407797+0000","last_peered":"2024-04-24T23:25:30.407797+0000","last_clean":"2024-04-24T23:25:30.407797+0000","last_became_active":"2024-04-24T23:25:27.995055+0000","last_became_peered":"2024-04-24T23:25:27.995055+0000","last_unstale":"2024-04-24T23:25:30.407797+0000","last_undegraded":"2024-04-24T23:25:30.407797+0000","last_fullsized":"2024-04-24T23:25:30.407797+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:52.787249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407024+0000","last_change":"2024-04-24T23:25:26.988051+0000","last_active":"2024-04-24T23:25:30.407024+0000","last_peered":"2024-04-24T23:25:30.407024+0000","last_clean":"2024-04-24T23:25:30.407024+0000","last_became_active":"2024-04-24T23:25:26.987663+0000","last_became_peered":"2024-04-24T23:25:26.987663+0000","last_unstale":"2024-04-24T23:25:30.407024+0000","last_undegraded":"2024-04-24T23:25:30.407024+0000","last_fullsized":"2024-04-24T23:25:30.407024+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:29.594107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163966+0000","last_change":"2024-04-24T23:25:28.002606+0000","last_active":"2024-04-24T23:25:29.163966+0000","last_peered":"2024-04-24T23:25:29.163966+0000","last_clean":"2024-04-24T23:25:29.163966+0000","last_became_active":"2024-04-24T23:25:28.002311+0000","last_became_peered":"2024-04-24T23:25:28.002311+0000","last_unstale":"2024-04-24T23:25:29.163966+0000","last_undegraded":"2024-04-24T23:25:29.163966+0000","last_fullsized":"2024-04-24T23:25:29.163966+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:16.043774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407997+0000","last_change":"2024-04-24T23:25:26.986624+0000","last_active":"2024-04-24T23:25:30.407997+0000","last_peered":"2024-04-24T23:25:30.407997+0000","last_clean":"2024-04-24T23:25:30.407997+0000","last_became_active":"2024-04-24T23:25:26.986323+0000","last_became_peered":"2024-04-24T23:25:26.986323+0000","last_unstale":"2024-04-24T23:25:30.407997+0000","last_undegraded":"2024-04-24T23:25:30.407997+0000","last_fullsized":"2024-04-24T23:25:30.407997+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:29:27.254375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993045+0000","last_change":"2024-04-24T23:25:28.001568+0000","last_active":"2024-04-24T23:25:28.993045+0000","last_peered":"2024-04-24T23:25:28.993045+0000","last_clean":"2024-04-24T23:25:28.993045+0000","last_became_active":"2024-04-24T23:25:28.001250+0000","last_became_peered":"2024-04-24T23:25:28.001250+0000","last_unstale":"2024-04-24T23:25:28.993045+0000","last_undegraded":"2024-04-24T23:25:28.993045+0000","last_fullsized":"2024-04-24T23:25:28.993045+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:33.603412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236474+0000","last_change":"2024-04-24T23:25:26.977218+0000","last_active":"2024-04-24T23:25:29.236474+0000","last_peered":"2024-04-24T23:25:29.236474+0000","last_clean":"2024-04-24T23:25:29.236474+0000","last_became_active":"2024-04-24T23:25:26.976913+0000","last_became_peered":"2024-04-24T23:25:26.976913+0000","last_unstale":"2024-04-24T23:25:29.236474+0000","last_undegraded":"2024-04-24T23:25:29.236474+0000","last_fullsized":"2024-04-24T23:25:29.236474+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:10.387500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406824+0000","last_change":"2024-04-24T23:25:28.001226+0000","last_active":"2024-04-24T23:25:30.406824+0000","last_peered":"2024-04-24T23:25:30.406824+0000","last_clean":"2024-04-24T23:25:30.406824+0000","last_became_active":"2024-04-24T23:25:28.000908+0000","last_became_peered":"2024-04-24T23:25:28.000908+0000","last_unstale":"2024-04-24T23:25:30.406824+0000","last_undegraded":"2024-04-24T23:25:30.406824+0000","last_fullsized":"2024-04-24T23:25:30.406824+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:02.505819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407528+0000","last_change":"2024-04-24T23:25:26.984947+0000","last_active":"2024-04-24T23:25:30.407528+0000","last_peered":"2024-04-24T23:25:30.407528+0000","last_clean":"2024-04-24T23:25:30.407528+0000","last_became_active":"2024-04-24T23:25:26.984604+0000","last_became_peered":"2024-04-24T23:25:26.984604+0000","last_unstale":"2024-04-24T23:25:30.407528+0000","last_undegraded":"2024-04-24T23:25:30.407528+0000","last_fullsized":"2024-04-24T23:25:30.407528+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:10.968125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993068+0000","last_change":"2024-04-24T23:25:27.997229+0000","last_active":"2024-04-24T23:25:29.993068+0000","last_peered":"2024-04-24T23:25:29.993068+0000","last_clean":"2024-04-24T23:25:29.993068+0000","last_became_active":"2024-04-24T23:25:27.996393+0000","last_became_peered":"2024-04-24T23:25:27.996393+0000","last_unstale":"2024-04-24T23:25:29.993068+0000","last_undegraded":"2024-04-24T23:25:29.993068+0000","last_fullsized":"2024-04-24T23:25:29.993068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:18.029231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406190+0000","last_change":"2024-04-24T23:25:26.982771+0000","last_active":"2024-04-24T23:25:30.406190+0000","last_peered":"2024-04-24T23:25:30.406190+0000","last_clean":"2024-04-24T23:25:30.406190+0000","last_became_active":"2024-04-24T23:25:26.982504+0000","last_became_peered":"2024-04-24T23:25:26.982504+0000","last_unstale":"2024-04-24T23:25:30.406190+0000","last_undegraded":"2024-04-24T23:25:30.406190+0000","last_fullsized":"2024-04-24T23:25:30.406190+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:26.544193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992272+0000","last_change":"2024-04-24T23:25:27.998554+0000","last_active":"2024-04-24T23:25:28.992272+0000","last_peered":"2024-04-24T23:25:28.992272+0000","last_clean":"2024-04-24T23:25:28.992272+0000","last_became_active":"2024-04-24T23:25:27.997924+0000","last_became_peered":"2024-04-24T23:25:27.997924+0000","last_unstale":"2024-04-24T23:25:28.992272+0000","last_undegraded":"2024-04-24T23:25:28.992272+0000","last_fullsized":"2024-04-24T23:25:28.992272+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:42.780286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993086+0000","last_change":"2024-04-24T23:25:26.985502+0000","last_active":"2024-04-24T23:25:29.993086+0000","last_peered":"2024-04-24T23:25:29.993086+0000","last_clean":"2024-04-24T23:25:29.993086+0000","last_became_active":"2024-04-24T23:25:26.985236+0000","last_became_peered":"2024-04-24T23:25:26.985236+0000","last_unstale":"2024-04-24T23:25:29.993086+0000","last_undegraded":"2024-04-24T23:25:29.993086+0000","last_fullsized":"2024-04-24T23:25:29.993086+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:11.001663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993393+0000","last_change":"2024-04-24T23:25:27.997744+0000","last_active":"2024-04-24T23:25:28.993393+0000","last_peered":"2024-04-24T23:25:28.993393+0000","last_clean":"2024-04-24T23:25:28.993393+0000","last_became_active":"2024-04-24T23:25:27.996890+0000","last_became_peered":"2024-04-24T23:25:27.996890+0000","last_unstale":"2024-04-24T23:25:28.993393+0000","last_undegraded":"2024-04-24T23:25:28.993393+0000","last_fullsized":"2024-04-24T23:25:28.993393+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:32.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407250+0000","last_change":"2024-04-24T23:25:26.976690+0000","last_active":"2024-04-24T23:25:30.407250+0000","last_peered":"2024-04-24T23:25:30.407250+0000","last_clean":"2024-04-24T23:25:30.407250+0000","last_became_active":"2024-04-24T23:25:26.976221+0000","last_became_peered":"2024-04-24T23:25:26.976221+0000","last_unstale":"2024-04-24T23:25:30.407250+0000","last_undegraded":"2024-04-24T23:25:30.407250+0000","last_fullsized":"2024-04-24T23:25:30.407250+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:07.520545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164100+0000","last_change":"2024-04-24T23:25:28.011082+0000","last_active":"2024-04-24T23:25:29.164100+0000","last_peered":"2024-04-24T23:25:29.164100+0000","last_clean":"2024-04-24T23:25:29.164100+0000","last_became_active":"2024-04-24T23:25:28.010786+0000","last_became_peered":"2024-04-24T23:25:28.010786+0000","last_unstale":"2024-04-24T23:25:29.164100+0000","last_undegraded":"2024-04-24T23:25:29.164100+0000","last_fullsized":"2024-04-24T23:25:29.164100+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:46.820679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407999+0000","last_change":"2024-04-24T23:25:26.979704+0000","last_active":"2024-04-24T23:25:30.407999+0000","last_peered":"2024-04-24T23:25:30.407999+0000","last_clean":"2024-04-24T23:25:30.407999+0000","last_became_active":"2024-04-24T23:25:26.979294+0000","last_became_peered":"2024-04-24T23:25:26.979294+0000","last_unstale":"2024-04-24T23:25:30.407999+0000","last_undegraded":"2024-04-24T23:25:30.407999+0000","last_fullsized":"2024-04-24T23:25:30.407999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:12.592820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407841+0000","last_change":"2024-04-24T23:25:28.001051+0000","last_active":"2024-04-24T23:25:30.407841+0000","last_peered":"2024-04-24T23:25:30.407841+0000","last_clean":"2024-04-24T23:25:30.407841+0000","last_became_active":"2024-04-24T23:25:28.000548+0000","last_became_peered":"2024-04-24T23:25:28.000548+0000","last_unstale":"2024-04-24T23:25:30.407841+0000","last_undegraded":"2024-04-24T23:25:30.407841+0000","last_fullsized":"2024-04-24T23:25:30.407841+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:43.854697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236638+0000","last_change":"2024-04-24T23:25:26.973701+0000","last_active":"2024-04-24T23:25:29.236638+0000","last_peered":"2024-04-24T23:25:29.236638+0000","last_clean":"2024-04-24T23:25:29.236638+0000","last_became_active":"2024-04-24T23:25:26.973424+0000","last_became_peered":"2024-04-24T23:25:26.973424+0000","last_unstale":"2024-04-24T23:25:29.236638+0000","last_undegraded":"2024-04-24T23:25:29.236638+0000","last_fullsized":"2024-04-24T23:25:29.236638+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:09.556305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993538+0000","last_change":"2024-04-24T23:25:27.989474+0000","last_active":"2024-04-24T23:25:29.993538+0000","last_peered":"2024-04-24T23:25:29.993538+0000","last_clean":"2024-04-24T23:25:29.993538+0000","last_became_active":"2024-04-24T23:25:27.989194+0000","last_became_peered":"2024-04-24T23:25:27.989194+0000","last_unstale":"2024-04-24T23:25:29.993538+0000","last_undegraded":"2024-04-24T23:25:29.993538+0000","last_fullsized":"2024-04-24T23:25:29.993538+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:11.972454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406545+0000","last_change":"2024-04-24T23:25:26.985049+0000","last_active":"2024-04-24T23:25:30.406545+0000","last_peered":"2024-04-24T23:25:30.406545+0000","last_clean":"2024-04-24T23:25:30.406545+0000","last_became_active":"2024-04-24T23:25:26.984293+0000","last_became_peered":"2024-04-24T23:25:26.984293+0000","last_unstale":"2024-04-24T23:25:30.406545+0000","last_undegraded":"2024-04-24T23:25:30.406545+0000","last_fullsized":"2024-04-24T23:25:30.406545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:12.592820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163585+0000","last_change":"2024-04-24T23:25:27.996333+0000","last_active":"2024-04-24T23:25:29.163585+0000","last_peered":"2024-04-24T23:25:29.163585+0000","last_clean":"2024-04-24T23:25:29.163585+0000","last_became_active":"2024-04-24T23:25:27.996005+0000","last_became_peered":"2024-04-24T23:25:27.996005+0000","last_unstale":"2024-04-24T23:25:29.163585+0000","last_undegraded":"2024-04-24T23:25:29.163585+0000","last_fullsized":"2024-04-24T23:25:29.163585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:14.057707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406350+0000","last_change":"2024-04-24T23:25:26.982431+0000","last_active":"2024-04-24T23:25:30.406350+0000","last_peered":"2024-04-24T23:25:30.406350+0000","last_clean":"2024-04-24T23:25:30.406350+0000","last_became_active":"2024-04-24T23:25:26.982147+0000","last_became_peered":"2024-04-24T23:25:26.982147+0000","last_unstale":"2024-04-24T23:25:30.406350+0000","last_undegraded":"2024-04-24T23:25:30.406350+0000","last_fullsized":"2024-04-24T23:25:30.406350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:47.513215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993164+0000","last_change":"2024-04-24T23:25:27.997375+0000","last_active":"2024-04-24T23:25:28.993164+0000","last_peered":"2024-04-24T23:25:28.993164+0000","last_clean":"2024-04-24T23:25:28.993164+0000","last_became_active":"2024-04-24T23:25:27.997031+0000","last_became_peered":"2024-04-24T23:25:27.997031+0000","last_unstale":"2024-04-24T23:25:28.993164+0000","last_undegraded":"2024-04-24T23:25:28.993164+0000","last_fullsized":"2024-04-24T23:25:28.993164+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:42.780286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407777+0000","last_change":"2024-04-24T23:25:26.979601+0000","last_active":"2024-04-24T23:25:30.407777+0000","last_peered":"2024-04-24T23:25:30.407777+0000","last_clean":"2024-04-24T23:25:30.407777+0000","last_became_active":"2024-04-24T23:25:26.978063+0000","last_became_peered":"2024-04-24T23:25:26.978063+0000","last_unstale":"2024-04-24T23:25:30.407777+0000","last_undegraded":"2024-04-24T23:25:30.407777+0000","last_fullsized":"2024-04-24T23:25:30.407777+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:59.723643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992089+0000","last_change":"2024-04-24T23:25:28.006596+0000","last_active":"2024-04-24T23:25:28.992089+0000","last_peered":"2024-04-24T23:25:28.992089+0000","last_clean":"2024-04-24T23:25:28.992089+0000","last_became_active":"2024-04-24T23:25:28.005475+0000","last_became_peered":"2024-04-24T23:25:28.005475+0000","last_unstale":"2024-04-24T23:25:28.992089+0000","last_undegraded":"2024-04-24T23:25:28.992089+0000","last_fullsized":"2024-04-24T23:25:28.992089+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:56.132083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992912+0000","last_change":"2024-04-24T23:25:26.979978+0000","last_active":"2024-04-24T23:25:28.992912+0000","last_peered":"2024-04-24T23:25:28.992912+0000","last_clean":"2024-04-24T23:25:28.992912+0000","last_became_active":"2024-04-24T23:25:26.978846+0000","last_became_peered":"2024-04-24T23:25:26.978846+0000","last_unstale":"2024-04-24T23:25:28.992912+0000","last_undegraded":"2024-04-24T23:25:28.992912+0000","last_fullsized":"2024-04-24T23:25:28.992912+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:42.850369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993066+0000","last_change":"2024-04-24T23:25:27.991199+0000","last_active":"2024-04-24T23:25:29.993066+0000","last_peered":"2024-04-24T23:25:29.993066+0000","last_clean":"2024-04-24T23:25:29.993066+0000","last_became_active":"2024-04-24T23:25:27.990922+0000","last_became_peered":"2024-04-24T23:25:27.990922+0000","last_unstale":"2024-04-24T23:25:29.993066+0000","last_undegraded":"2024-04-24T23:25:29.993066+0000","last_fullsized":"2024-04-24T23:25:29.993066+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:28.122882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406431+0000","last_change":"2024-04-24T23:25:26.983021+0000","last_active":"2024-04-24T23:25:30.406431+0000","last_peered":"2024-04-24T23:25:30.406431+0000","last_clean":"2024-04-24T23:25:30.406431+0000","last_became_active":"2024-04-24T23:25:26.982757+0000","last_became_peered":"2024-04-24T23:25:26.982757+0000","last_unstale":"2024-04-24T23:25:30.406431+0000","last_undegraded":"2024-04-24T23:25:30.406431+0000","last_fullsized":"2024-04-24T23:25:30.406431+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:01.357710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407142+0000","last_change":"2024-04-24T23:25:27.994927+0000","last_active":"2024-04-24T23:25:30.407142+0000","last_peered":"2024-04-24T23:25:30.407142+0000","last_clean":"2024-04-24T23:25:30.407142+0000","last_became_active":"2024-04-24T23:25:27.994616+0000","last_became_peered":"2024-04-24T23:25:27.994616+0000","last_unstale":"2024-04-24T23:25:30.407142+0000","last_undegraded":"2024-04-24T23:25:30.407142+0000","last_fullsized":"2024-04-24T23:25:30.407142+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:10.750730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.408036+0000","last_change":"2024-04-24T23:25:26.984211+0000","last_active":"2024-04-24T23:25:30.408036+0000","last_peered":"2024-04-24T23:25:30.408036+0000","last_clean":"2024-04-24T23:25:30.408036+0000","last_became_active":"2024-04-24T23:25:26.983878+0000","last_became_peered":"2024-04-24T23:25:26.983878+0000","last_unstale":"2024-04-24T23:25:30.408036+0000","last_undegraded":"2024-04-24T23:25:30.408036+0000","last_fullsized":"2024-04-24T23:25:30.408036+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:15.381258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164226+0000","last_change":"2024-04-24T23:25:27.998147+0000","last_active":"2024-04-24T23:25:29.164226+0000","last_peered":"2024-04-24T23:25:29.164226+0000","last_clean":"2024-04-24T23:25:29.164226+0000","last_became_active":"2024-04-24T23:25:27.997877+0000","last_became_peered":"2024-04-24T23:25:27.997877+0000","last_unstale":"2024-04-24T23:25:29.164226+0000","last_undegraded":"2024-04-24T23:25:29.164226+0000","last_fullsized":"2024-04-24T23:25:29.164226+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:51.479069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407227+0000","last_change":"2024-04-24T23:25:26.988180+0000","last_active":"2024-04-24T23:25:30.407227+0000","last_peered":"2024-04-24T23:25:30.407227+0000","last_clean":"2024-04-24T23:25:30.407227+0000","last_became_active":"2024-04-24T23:25:26.987706+0000","last_became_peered":"2024-04-24T23:25:26.987706+0000","last_unstale":"2024-04-24T23:25:30.407227+0000","last_undegraded":"2024-04-24T23:25:30.407227+0000","last_fullsized":"2024-04-24T23:25:30.407227+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:16.777467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406533+0000","last_change":"2024-04-24T23:25:27.993350+0000","last_active":"2024-04-24T23:25:30.406533+0000","last_peered":"2024-04-24T23:25:30.406533+0000","last_clean":"2024-04-24T23:25:30.406533+0000","last_became_active":"2024-04-24T23:25:27.993011+0000","last_became_peered":"2024-04-24T23:25:27.993011+0000","last_unstale":"2024-04-24T23:25:30.406533+0000","last_undegraded":"2024-04-24T23:25:30.406533+0000","last_fullsized":"2024-04-24T23:25:30.406533+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:32.094641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407990+0000","last_change":"2024-04-24T23:25:26.989891+0000","last_active":"2024-04-24T23:25:30.407990+0000","last_peered":"2024-04-24T23:25:30.407990+0000","last_clean":"2024-04-24T23:25:30.407990+0000","last_became_active":"2024-04-24T23:25:26.989524+0000","last_became_peered":"2024-04-24T23:25:26.989524+0000","last_unstale":"2024-04-24T23:25:30.407990+0000","last_undegraded":"2024-04-24T23:25:30.407990+0000","last_fullsized":"2024-04-24T23:25:30.407990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:12.258864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993464+0000","last_change":"2024-04-24T23:25:27.998329+0000","last_active":"2024-04-24T23:25:29.993464+0000","last_peered":"2024-04-24T23:25:29.993464+0000","last_clean":"2024-04-24T23:25:29.993464+0000","last_became_active":"2024-04-24T23:25:27.998051+0000","last_became_peered":"2024-04-24T23:25:27.998051+0000","last_unstale":"2024-04-24T23:25:29.993464+0000","last_undegraded":"2024-04-24T23:25:29.993464+0000","last_fullsized":"2024-04-24T23:25:29.993464+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:45.612905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163856+0000","last_change":"2024-04-24T23:25:26.980697+0000","last_active":"2024-04-24T23:25:29.163856+0000","last_peered":"2024-04-24T23:25:29.163856+0000","last_clean":"2024-04-24T23:25:29.163856+0000","last_became_active":"2024-04-24T23:25:26.980315+0000","last_became_peered":"2024-04-24T23:25:26.980315+0000","last_unstale":"2024-04-24T23:25:29.163856+0000","last_undegraded":"2024-04-24T23:25:29.163856+0000","last_fullsized":"2024-04-24T23:25:29.163856+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:20.322598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236354+0000","last_change":"2024-04-24T23:25:27.994129+0000","last_active":"2024-04-24T23:25:29.236354+0000","last_peered":"2024-04-24T23:25:29.236354+0000","last_clean":"2024-04-24T23:25:29.236354+0000","last_became_active":"2024-04-24T23:25:27.993811+0000","last_became_peered":"2024-04-24T23:25:27.993811+0000","last_unstale":"2024-04-24T23:25:29.236354+0000","last_undegraded":"2024-04-24T23:25:29.236354+0000","last_fullsized":"2024-04-24T23:25:29.236354+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:39.084955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407955+0000","last_change":"2024-04-24T23:25:26.984839+0000","last_active":"2024-04-24T23:25:30.407955+0000","last_peered":"2024-04-24T23:25:30.407955+0000","last_clean":"2024-04-24T23:25:30.407955+0000","last_became_active":"2024-04-24T23:25:26.984489+0000","last_became_peered":"2024-04-24T23:25:26.984489+0000","last_unstale":"2024-04-24T23:25:30.407955+0000","last_undegraded":"2024-04-24T23:25:30.407955+0000","last_fullsized":"2024-04-24T23:25:30.407955+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:49.523659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236325+0000","last_change":"2024-04-24T23:25:28.005544+0000","last_active":"2024-04-24T23:25:29.236325+0000","last_peered":"2024-04-24T23:25:29.236325+0000","last_clean":"2024-04-24T23:25:29.236325+0000","last_became_active":"2024-04-24T23:25:28.005222+0000","last_became_peered":"2024-04-24T23:25:28.005222+0000","last_unstale":"2024-04-24T23:25:29.236325+0000","last_undegraded":"2024-04-24T23:25:29.236325+0000","last_fullsized":"2024-04-24T23:25:29.236325+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:06.443167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993024+0000","last_change":"2024-04-24T23:25:26.975028+0000","last_active":"2024-04-24T23:25:28.993024+0000","last_peered":"2024-04-24T23:25:28.993024+0000","last_clean":"2024-04-24T23:25:28.993024+0000","last_became_active":"2024-04-24T23:25:26.974741+0000","last_became_peered":"2024-04-24T23:25:26.974741+0000","last_unstale":"2024-04-24T23:25:28.993024+0000","last_undegraded":"2024-04-24T23:25:28.993024+0000","last_fullsized":"2024-04-24T23:25:28.993024+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:46.556429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236384+0000","last_change":"2024-04-24T23:25:28.003440+0000","last_active":"2024-04-24T23:25:29.236384+0000","last_peered":"2024-04-24T23:25:29.236384+0000","last_clean":"2024-04-24T23:25:29.236384+0000","last_became_active":"2024-04-24T23:25:28.003156+0000","last_became_peered":"2024-04-24T23:25:28.003156+0000","last_unstale":"2024-04-24T23:25:29.236384+0000","last_undegraded":"2024-04-24T23:25:29.236384+0000","last_fullsized":"2024-04-24T23:25:29.236384+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:05.319581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992007+0000","last_change":"2024-04-24T23:25:26.986864+0000","last_active":"2024-04-24T23:25:28.992007+0000","last_peered":"2024-04-24T23:25:28.992007+0000","last_clean":"2024-04-24T23:25:28.992007+0000","last_became_active":"2024-04-24T23:25:26.986575+0000","last_became_peered":"2024-04-24T23:25:26.986575+0000","last_unstale":"2024-04-24T23:25:28.992007+0000","last_undegraded":"2024-04-24T23:25:28.992007+0000","last_fullsized":"2024-04-24T23:25:28.992007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:43.615264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992504+0000","last_change":"2024-04-24T23:25:28.004963+0000","last_active":"2024-04-24T23:25:28.992504+0000","last_peered":"2024-04-24T23:25:28.992504+0000","last_clean":"2024-04-24T23:25:28.992504+0000","last_became_active":"2024-04-24T23:25:28.004707+0000","last_became_peered":"2024-04-24T23:25:28.004707+0000","last_unstale":"2024-04-24T23:25:28.992504+0000","last_undegraded":"2024-04-24T23:25:28.992504+0000","last_fullsized":"2024-04-24T23:25:28.992504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:10.421054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.408136+0000","last_change":"2024-04-24T23:25:26.983803+0000","last_active":"2024-04-24T23:25:30.408136+0000","last_peered":"2024-04-24T23:25:30.408136+0000","last_clean":"2024-04-24T23:25:30.408136+0000","last_became_active":"2024-04-24T23:25:26.983465+0000","last_became_peered":"2024-04-24T23:25:26.983465+0000","last_unstale":"2024-04-24T23:25:30.408136+0000","last_undegraded":"2024-04-24T23:25:30.408136+0000","last_fullsized":"2024-04-24T23:25:30.408136+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:47.513215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993311+0000","last_change":"2024-04-24T23:25:27.993787+0000","last_active":"2024-04-24T23:25:28.993311+0000","last_peered":"2024-04-24T23:25:28.993311+0000","last_clean":"2024-04-24T23:25:28.993311+0000","last_became_active":"2024-04-24T23:25:27.993506+0000","last_became_peered":"2024-04-24T23:25:27.993506+0000","last_unstale":"2024-04-24T23:25:28.993311+0000","last_undegraded":"2024-04-24T23:25:28.993311+0000","last_fullsized":"2024-04-24T23:25:28.993311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:41.763030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.993486+0000","last_change":"2024-04-24T23:25:26.980977+0000","last_active":"2024-04-24T23:25:28.993486+0000","last_peered":"2024-04-24T23:25:28.993486+0000","last_clean":"2024-04-24T23:25:28.993486+0000","last_became_active":"2024-04-24T23:25:26.979025+0000","last_became_peered":"2024-04-24T23:25:26.979025+0000","last_unstale":"2024-04-24T23:25:28.993486+0000","last_undegraded":"2024-04-24T23:25:28.993486+0000","last_fullsized":"2024-04-24T23:25:28.993486+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:43.615264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163839+0000","last_change":"2024-04-24T23:25:28.009828+0000","last_active":"2024-04-24T23:25:29.163839+0000","last_peered":"2024-04-24T23:25:29.163839+0000","last_clean":"2024-04-24T23:25:29.163839+0000","last_became_active":"2024-04-24T23:25:28.009151+0000","last_became_peered":"2024-04-24T23:25:28.009151+0000","last_unstale":"2024-04-24T23:25:29.163839+0000","last_undegraded":"2024-04-24T23:25:29.163839+0000","last_fullsized":"2024-04-24T23:25:29.163839+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:58.729537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993610+0000","last_change":"2024-04-24T23:25:26.984615+0000","last_active":"2024-04-24T23:25:29.993610+0000","last_peered":"2024-04-24T23:25:29.993610+0000","last_clean":"2024-04-24T23:25:29.993610+0000","last_became_active":"2024-04-24T23:25:26.984358+0000","last_became_peered":"2024-04-24T23:25:26.984358+0000","last_unstale":"2024-04-24T23:25:29.993610+0000","last_undegraded":"2024-04-24T23:25:29.993610+0000","last_fullsized":"2024-04-24T23:25:29.993610+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:50.649710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.164064+0000","last_change":"2024-04-24T23:25:28.007442+0000","last_active":"2024-04-24T23:25:29.164064+0000","last_peered":"2024-04-24T23:25:29.164064+0000","last_clean":"2024-04-24T23:25:29.164064+0000","last_became_active":"2024-04-24T23:25:28.007170+0000","last_became_peered":"2024-04-24T23:25:28.007170+0000","last_unstale":"2024-04-24T23:25:29.164064+0000","last_undegraded":"2024-04-24T23:25:29.164064+0000","last_fullsized":"2024-04-24T23:25:29.164064+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:11.391828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:28.992356+0000","last_change":"2024-04-24T23:25:26.988155+0000","last_active":"2024-04-24T23:25:28.992356+0000","last_peered":"2024-04-24T23:25:28.992356+0000","last_clean":"2024-04-24T23:25:28.992356+0000","last_became_active":"2024-04-24T23:25:26.987824+0000","last_became_peered":"2024-04-24T23:25:26.987824+0000","last_unstale":"2024-04-24T23:25:28.992356+0000","last_undegraded":"2024-04-24T23:25:28.992356+0000","last_fullsized":"2024-04-24T23:25:28.992356+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:01.501491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993514+0000","last_change":"2024-04-24T23:25:27.992164+0000","last_active":"2024-04-24T23:25:29.993514+0000","last_peered":"2024-04-24T23:25:29.993514+0000","last_clean":"2024-04-24T23:25:29.993514+0000","last_became_active":"2024-04-24T23:25:27.991877+0000","last_became_peered":"2024-04-24T23:25:27.991877+0000","last_unstale":"2024-04-24T23:25:29.993514+0000","last_undegraded":"2024-04-24T23:25:29.993514+0000","last_fullsized":"2024-04-24T23:25:29.993514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:00.727972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.993402+0000","last_change":"2024-04-24T23:25:26.976419+0000","last_active":"2024-04-24T23:25:29.993402+0000","last_peered":"2024-04-24T23:25:29.993402+0000","last_clean":"2024-04-24T23:25:29.993402+0000","last_became_active":"2024-04-24T23:25:26.976109+0000","last_became_peered":"2024-04-24T23:25:26.976109+0000","last_unstale":"2024-04-24T23:25:29.993402+0000","last_undegraded":"2024-04-24T23:25:29.993402+0000","last_fullsized":"2024-04-24T23:25:29.993402+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:03.090729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407897+0000","last_change":"2024-04-24T23:25:27.998536+0000","last_active":"2024-04-24T23:25:30.407897+0000","last_peered":"2024-04-24T23:25:30.407897+0000","last_clean":"2024-04-24T23:25:30.407897+0000","last_became_active":"2024-04-24T23:25:27.997773+0000","last_became_peered":"2024-04-24T23:25:27.997773+0000","last_unstale":"2024-04-24T23:25:30.407897+0000","last_undegraded":"2024-04-24T23:25:30.407897+0000","last_fullsized":"2024-04-24T23:25:30.407897+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:58.899788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406177+0000","last_change":"2024-04-24T23:25:26.985538+0000","last_active":"2024-04-24T23:25:30.406177+0000","last_peered":"2024-04-24T23:25:30.406177+0000","last_clean":"2024-04-24T23:25:30.406177+0000","last_became_active":"2024-04-24T23:25:26.985216+0000","last_became_peered":"2024-04-24T23:25:26.985216+0000","last_unstale":"2024-04-24T23:25:30.406177+0000","last_undegraded":"2024-04-24T23:25:30.406177+0000","last_fullsized":"2024-04-24T23:25:30.406177+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:32.695590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406217+0000","last_change":"2024-04-24T23:25:28.001468+0000","last_active":"2024-04-24T23:25:30.406217+0000","last_peered":"2024-04-24T23:25:30.406217+0000","last_clean":"2024-04-24T23:25:30.406217+0000","last_became_active":"2024-04-24T23:25:28.001179+0000","last_became_peered":"2024-04-24T23:25:28.001179+0000","last_unstale":"2024-04-24T23:25:30.406217+0000","last_undegraded":"2024-04-24T23:25:30.406217+0000","last_fullsized":"2024-04-24T23:25:30.406217+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:17.781795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.406372+0000","last_change":"2024-04-24T23:25:26.978070+0000","last_active":"2024-04-24T23:25:30.406372+0000","last_peered":"2024-04-24T23:25:30.406372+0000","last_clean":"2024-04-24T23:25:30.406372+0000","last_became_active":"2024-04-24T23:25:26.977796+0000","last_became_peered":"2024-04-24T23:25:26.977796+0000","last_unstale":"2024-04-24T23:25:30.406372+0000","last_undegraded":"2024-04-24T23:25:30.406372+0000","last_fullsized":"2024-04-24T23:25:30.406372+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:53.598202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407502+0000","last_change":"2024-04-24T23:25:27.994749+0000","last_active":"2024-04-24T23:25:30.407502+0000","last_peered":"2024-04-24T23:25:30.407502+0000","last_clean":"2024-04-24T23:25:30.407502+0000","last_became_active":"2024-04-24T23:25:27.994402+0000","last_became_peered":"2024-04-24T23:25:27.994402+0000","last_unstale":"2024-04-24T23:25:30.407502+0000","last_undegraded":"2024-04-24T23:25:30.407502+0000","last_fullsized":"2024-04-24T23:25:30.407502+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:42.428738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407300+0000","last_change":"2024-04-24T23:25:26.985857+0000","last_active":"2024-04-24T23:25:30.407300+0000","last_peered":"2024-04-24T23:25:30.407300+0000","last_clean":"2024-04-24T23:25:30.407300+0000","last_became_active":"2024-04-24T23:25:26.985337+0000","last_became_peered":"2024-04-24T23:25:26.985337+0000","last_unstale":"2024-04-24T23:25:30.407300+0000","last_undegraded":"2024-04-24T23:25:30.407300+0000","last_fullsized":"2024-04-24T23:25:30.407300+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:31:26.544193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.236451+0000","last_change":"2024-04-24T23:25:28.004409+0000","last_active":"2024-04-24T23:25:29.236451+0000","last_peered":"2024-04-24T23:25:29.236451+0000","last_clean":"2024-04-24T23:25:29.236451+0000","last_became_active":"2024-04-24T23:25:28.004111+0000","last_became_peered":"2024-04-24T23:25:28.004111+0000","last_unstale":"2024-04-24T23:25:29.236451+0000","last_undegraded":"2024-04-24T23:25:29.236451+0000","last_fullsized":"2024-04-24T23:25:29.236451+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:26.961783+0000","last_clean_scrub_stamp":"2024-04-24T23:25:26.961783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:50.232135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:30.407598+0000","last_change":"2024-04-24T23:25:26.986149+0000","last_active":"2024-04-24T23:25:30.407598+0000","last_peered":"2024-04-24T23:25:30.407598+0000","last_clean":"2024-04-24T23:25:30.407598+0000","last_became_active":"2024-04-24T23:25:26.985763+0000","last_became_peered":"2024-04-24T23:25:26.985763+0000","last_unstale":"2024-04-24T23:25:30.407598+0000","last_undegraded":"2024-04-24T23:25:30.407598+0000","last_fullsized":"2024-04-24T23:25:30.407598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:25:25.957454+0000","last_clean_scrub_stamp":"2024-04-24T23:25:25.957454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:32.695590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":258,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T23:25:29.163495+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:25:29.163495+0000","last_peered":"2024-04-24T23:25:29.163495+0000","last_clean":"2024-04-24T23:25:29.163495+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:25:29.163495+0000","last_undegraded":"2024-04-24T23:25:29.163495+0000","last_fullsized":"2024-04-24T23:25:29.163495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":948165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":357273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":357271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":357251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":847872,"data_stored":357293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36172,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335668,"statfs":{"total":96636764160,"available":96599724032,"internally_reserved":0,"allocated":851968,"data_stored":358793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":359409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":1433600,"data_stored":947549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:25:36.767 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:25:36.767 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:25:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.911+0000 7f6fcbd9e640 1 Processor -- start 2024-04-24T23:25:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.911+0000 7f6fcbd9e640 1 -- start start 2024-04-24T23:25:36.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.912+0000 7f6fcbd9e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6fc415adc0 0x7f6fc40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.912+0000 7f6fcbd9e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6fc40ad7b0 0x7f6fc40d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.912+0000 7f6fcbd9e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fc415ef60 con 0x7f6fc415e9c0 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.912+0000 7f6fcbd9e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6fc415f240 con 0x7f6fc40ad7b0 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.912+0000 7f6fcbd9e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6fc415f520 con 0x7f6fc415adc0 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.912+0000 7f6fc9339640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6fc40ad7b0 0x7f6fc40d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.912+0000 7f6fc9339640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6fc40ad7b0 0x7f6fc40d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55288/0 (socket says 172.21.15.26:55288) 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.912+0000 7f6fc9339640 1 -- 172.21.15.26:0/3343971185 learned_addr learned my addr 172.21.15.26:0/3343971185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.913+0000 7f6fc9b3a640 1 --2- 172.21.15.26:0/3343971185 >> v2:172.21.15.29:3300/0 conn(0x7f6fc415adc0 0x7f6fc40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.913+0000 7f6fc9339640 1 -- 172.21.15.26:0/3343971185 >> v2:172.21.15.29:3300/0 conn(0x7f6fc415adc0 msgr2=0x7f6fc40b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.913+0000 7f6fc9339640 1 --2- 172.21.15.26:0/3343971185 >> v2:172.21.15.29:3300/0 conn(0x7f6fc415adc0 0x7f6fc40b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.913+0000 7f6fc9339640 1 -- 172.21.15.26:0/3343971185 >> v1:172.21.15.29:6789/0 conn(0x7f6fc415e9c0 legacy=0x7f6fc406dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.913+0000 7f6fc9339640 1 -- 172.21.15.26:0/3343971185 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc415f800 con 0x7f6fc40ad7b0 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.913+0000 7f6fc9b3a640 1 --2- 172.21.15.26:0/3343971185 >> v2:172.21.15.29:3300/0 conn(0x7f6fc415adc0 0x7f6fc40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.913+0000 7f6fc9339640 1 --2- 172.21.15.26:0/3343971185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc40ad7b0 0x7f6fc40d0040 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009870 tx=0x7f6fb402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cc1af648ccaa43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fc8b38640 1 -- 172.21.15.26:0/3343971185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4037070 con 0x7f6fc40ad7b0 2024-04-24T23:25:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fc8b38640 1 -- 172.21.15.26:0/3343971185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb402ae40 con 0x7f6fc40ad7b0 2024-04-24T23:25:36.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fc8b38640 1 -- 172.21.15.26:0/3343971185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4032660 con 0x7f6fc40ad7b0 2024-04-24T23:25:36.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/3343971185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc40ad7b0 msgr2=0x7f6fc40d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:36.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fcbd9e640 1 --2- 172.21.15.26:0/3343971185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc40ad7b0 0x7f6fc40d0040 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009870 tx=0x7f6fb402a130 comp rx=0 tx=0).stop 2024-04-24T23:25:36.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/3343971185 shutdown_connections 2024-04-24T23:25:36.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fcbd9e640 1 --2- 172.21.15.26:0/3343971185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc40ad7b0 0x7f6fc40d0040 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fcbd9e640 1 --2- 172.21.15.26:0/3343971185 >> v2:172.21.15.29:3300/0 conn(0x7f6fc415adc0 0x7f6fc40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/3343971185 >> 172.21.15.26:0/3343971185 conn(0x7f6fc406b540 msgr2=0x7f6fc406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/3343971185 shutdown_connections 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.914+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/3343971185 wait complete. 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.915+0000 7f6fcbd9e640 1 Processor -- start 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.915+0000 7f6fcbd9e640 1 -- start start 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fcbd9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fc40ad7b0 0x7f6fc409ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fcbd9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fc415adc0 0x7f6fc414c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fcbd9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc415e9c0 0x7f6fc414cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fcbd9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc415b1a0 con 0x7f6fc415e9c0 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fcbd9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc40b48b0 con 0x7f6fc415adc0 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fcbd9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc40d0580 con 0x7f6fc40ad7b0 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fca33b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc415e9c0 0x7f6fc414cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fca33b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc415e9c0 0x7f6fc414cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55300/0 (socket says 172.21.15.26:55300) 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fca33b640 1 -- 172.21.15.26:0/1544978051 learned_addr learned my addr 172.21.15.26:0/1544978051 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fc9339640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fc415adc0 0x7f6fc414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fc9b3a640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fc40ad7b0 0x7f6fc409ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fca33b640 1 -- 172.21.15.26:0/1544978051 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fc40ad7b0 msgr2=0x7f6fc409ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fca33b640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fc40ad7b0 0x7f6fc409ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fca33b640 1 -- 172.21.15.26:0/1544978051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fc415adc0 msgr2=0x7f6fc414c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.916+0000 7f6fca33b640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fc415adc0 0x7f6fc414c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.917+0000 7f6fca33b640 1 -- 172.21.15.26:0/1544978051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc415c480 con 0x7f6fc415e9c0 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.917+0000 7f6fc9339640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fc415adc0 0x7f6fc414c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.917+0000 7f6fca33b640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc415e9c0 0x7f6fc414cb50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000b970 tx=0x7f6fc000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.917+0000 7f6fba7fc640 1 -- 172.21.15.26:0/1544978051 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0019070 con 0x7f6fc415e9c0 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.917+0000 7f6fba7fc640 1 -- 172.21.15.26:0/1544978051 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc000cb40 con 0x7f6fc415e9c0 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.917+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc415c760 con 0x7f6fc415e9c0 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.917+0000 7f6fba7fc640 1 -- 172.21.15.26:0/1544978051 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0014440 con 0x7f6fc415e9c0 2024-04-24T23:25:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.917+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc415cd20 con 0x7f6fc415e9c0 2024-04-24T23:25:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.918+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc4075610 con 0x7f6fc415e9c0 2024-04-24T23:25:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.919+0000 7f6fba7fc640 1 -- 172.21.15.26:0/1544978051 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fc001c070 con 0x7f6fc415e9c0 2024-04-24T23:25:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.919+0000 7f6fba7fc640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6fa806be30 0x7f6fa806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.919+0000 7f6fba7fc640 1 -- 172.21.15.26:0/1544978051 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5767+0+0 (secure 0 0 0) 0x7f6fc008aa70 con 0x7f6fc415e9c0 2024-04-24T23:25:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.919+0000 7f6fc9b3a640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6fa806be30 0x7f6fa806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.919+0000 7f6fc9b3a640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6fa806be30 0x7f6fa806e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40b2e60 tx=0x7f6fbc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:36.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:36.923+0000 7f6fba7fc640 1 -- 172.21.15.26:0/1544978051 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6fc0059470 con 0x7f6fc415e9c0 2024-04-24T23:25:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.187+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6fc415e730 con 0x7f6fc415e9c0 2024-04-24T23:25:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.189+0000 7f6fba7fc640 1 -- 172.21.15.26:0/1544978051 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f6fc0058d70 con 0x7f6fc415e9c0 2024-04-24T23:25:37.190 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-24T23:25:37.190 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ== 2024-04-24T23:25:37.190 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-04-24T23:25:37.190 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-04-24T23:25:37.190 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:25:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6fa806be30 msgr2=0x7f6fa806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6fa806be30 0x7f6fa806e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40b2e60 tx=0x7f6fbc008040 comp rx=0 tx=0).stop 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc415e9c0 msgr2=0x7f6fc414cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc415e9c0 0x7f6fc414cb50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000b970 tx=0x7f6fc000be30 comp rx=0 tx=0).stop 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 shutdown_connections 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6fa806be30 0x7f6fa806e2f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc415e9c0 0x7f6fc414cb50 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fc415adc0 0x7f6fc414c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 --2- 172.21.15.26:0/1544978051 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6fc40ad7b0 0x7f6fc409ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 >> 172.21.15.26:0/1544978051 conn(0x7f6fc406b540 msgr2=0x7f6fc406e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 shutdown_connections 2024-04-24T23:25:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.191+0000 7f6fcbd9e640 1 -- 172.21.15.26:0/1544978051 wait complete. 2024-04-24T23:25:37.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.303+0000 7f17ae488640 1 Processor -- start 2024-04-24T23:25:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.303+0000 7f17ae488640 1 -- start start 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17ae488640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f17a8145910 0x7f17a806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17ae488640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a806d350 0x7f17a815d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17ae488640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a815d570 con 0x7f17a80cd660 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17ae488640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f17a8145cf0 con 0x7f17a8145910 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17ae488640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f17a80cfbc0 con 0x7f17a806d350 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17aca25640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f17a80cd660 0x7f17a815d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41196/0 (socket says 172.21.15.26:41196) 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17aca25640 1 -- 172.21.15.26:0/1608760676 learned_addr learned my addr 172.21.15.26:0/1608760676 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1215750055 0 0) 0x7f17a0001490 con 0x7f17a80cd660 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a7fff640 1 --2- 172.21.15.26:0/1608760676 >> v2:172.21.15.29:3300/0 conn(0x7f17a8145910 0x7f17a806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1794003520 con 0x7f17a80cd660 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a77fe640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a806d350 0x7f17a815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 432724560 0 0) 0x7f1794003520 con 0x7f17a80cd660 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a806d350 msgr2=0x7f17a815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a6ffd640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a806d350 0x7f17a815d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 >> v2:172.21.15.29:3300/0 conn(0x7f17a8145910 msgr2=0x7f17a806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a6ffd640 1 --2- 172.21.15.26:0/1608760676 >> v2:172.21.15.29:3300/0 conn(0x7f17a8145910 0x7f17a806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a806d7f0 con 0x7f17a80cd660 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.304+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f17a0001bc0 con 0x7f17a80cd660 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 >> v1:172.21.15.26:6789/0 conn(0x7f17a80cd660 legacy=0x7f17a815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17a6ffd640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a806d350 0x7f17940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17a6ffd640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1794003f30 0x7f1794004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17a6ffd640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794004d40 0x7f17940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8145cf0 con 0x7f1794004d40 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f17a80cfbc0 con 0x7f1794003f30 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f179400b9c0 con 0x7f17a806d350 2024-04-24T23:25:37.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17aca25640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794004d40 0x7f17940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17a7fff640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a806d350 0x7f17940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17a77fe640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1794003f30 0x7f1794004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17aca25640 1 -- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a806d350 msgr2=0x7f17940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17aca25640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a806d350 0x7f17940039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17aca25640 1 -- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1794003f30 msgr2=0x7f1794004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17aca25640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1794003f30 0x7f1794004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.305+0000 7f17aca25640 1 -- 172.21.15.26:0/1608760676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1794003520 con 0x7f1794004d40 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17aca25640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794004d40 0x7f17940051c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f17a000aa40 tx=0x7f1794005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1f6c3a497aee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a000a690 con 0x7f1794004d40 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a002f070 con 0x7f1794004d40 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17a6ffd640 1 -- 172.21.15.26:0/1608760676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0034470 con 0x7f1794004d40 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17ae488640 1 -- 172.21.15.26:0/1608760676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794004d40 msgr2=0x7f17940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17ae488640 1 --2- 172.21.15.26:0/1608760676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794004d40 0x7f17940051c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f17a000aa40 tx=0x7f1794005830 comp rx=0 tx=0).stop 2024-04-24T23:25:37.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17aca25640 1 -- 172.21.15.26:0/1608760676 reap_dead start 2024-04-24T23:25:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17ae488640 1 -- 172.21.15.26:0/1608760676 shutdown_connections 2024-04-24T23:25:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17ae488640 1 -- 172.21.15.26:0/1608760676 >> 172.21.15.26:0/1608760676 conn(0x7f17a806b300 msgr2=0x7f17a806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17ae488640 1 -- 172.21.15.26:0/1608760676 shutdown_connections 2024-04-24T23:25:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.306+0000 7f17ae488640 1 -- 172.21.15.26:0/1608760676 wait complete. 2024-04-24T23:25:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.308+0000 7f17ae488640 1 Processor -- start 2024-04-24T23:25:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.308+0000 7f17ae488640 1 -- start start 2024-04-24T23:25:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17ae488640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a81385b0 0x7f17a8139220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17ae488640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17a80b4ae0 0x7f17a81e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17ae488640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a81e9190 0x7f17a80b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17ae488640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17a806e090 con 0x7f17a80b4ae0 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17ae488640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f17a806ddb0 con 0x7f17a81e9190 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17ae488640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f17a806e370 con 0x7f17a81385b0 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17a80b4ae0 0x7f17a81e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17a80b4ae0 0x7f17a81e8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55328/0 (socket says 172.21.15.26:55328) 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a77fe640 1 -- 172.21.15.26:0/1314304910 learned_addr learned my addr 172.21.15.26:0/1314304910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17aca25640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a81e9190 0x7f17a80b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a7fff640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a81385b0 0x7f17a8139220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a77fe640 1 -- 172.21.15.26:0/1314304910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a81385b0 msgr2=0x7f17a8139220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a77fe640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a81385b0 0x7f17a8139220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a77fe640 1 -- 172.21.15.26:0/1314304910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a81e9190 msgr2=0x7f17a80b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a77fe640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a81e9190 0x7f17a80b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a77fe640 1 -- 172.21.15.26:0/1314304910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a81398c0 con 0x7f17a80b4ae0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17aca25640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a81e9190 0x7f17a80b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a77fe640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17a80b4ae0 0x7f17a81e8c50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1798011940 tx=0x7f1798011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a4ff9640 1 -- 172.21.15.26:0/1314304910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179800faf0 con 0x7f17a80b4ae0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a4ff9640 1 -- 172.21.15.26:0/1314304910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f179800fc90 con 0x7f17a80b4ae0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a8139ba0 con 0x7f17a80b4ae0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17a4ff9640 1 -- 172.21.15.26:0/1314304910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798014440 con 0x7f17a80b4ae0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.309+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a813a160 con 0x7f17a80b4ae0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.310+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a80694c0 con 0x7f17a80b4ae0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.311+0000 7f17a4ff9640 1 -- 172.21.15.26:0/1314304910 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1798012040 con 0x7f17a80b4ae0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.311+0000 7f17a4ff9640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f178806be30 0x7f178806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.311+0000 7f17a4ff9640 1 -- 172.21.15.26:0/1314304910 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f179808b590 con 0x7f17a80b4ae0 2024-04-24T23:25:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.312+0000 7f17a7fff640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f178806be30 0x7f178806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.312+0000 7f17a7fff640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f178806be30 0x7f178806e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1790009a60 tx=0x7f1790008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.316+0000 7f17a4ff9640 1 -- 172.21.15.26:0/1314304910 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1798019030 con 0x7f17a80b4ae0 2024-04-24T23:25:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.537+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17a80b18d0 con 0x7f17a80b4ae0 2024-04-24T23:25:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.538+0000 7f17a4ff9640 1 -- 172.21.15.26:0/1314304910 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7f1798059f00 con 0x7f17a80b4ae0 2024-04-24T23:25:37.539 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:37.539 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:25:30.362268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[7227]},"id":5}]} 2024-04-24T23:25:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 67 2024-04-24T23:25:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f178806be30 msgr2=0x7f178806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f178806be30 0x7f178806e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1790009a60 tx=0x7f1790008040 comp rx=0 tx=0).stop 2024-04-24T23:25:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17a80b4ae0 msgr2=0x7f17a81e8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17a80b4ae0 0x7f17a81e8c50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1798011940 tx=0x7f1798011e00 comp rx=0 tx=0).stop 2024-04-24T23:25:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 shutdown_connections 2024-04-24T23:25:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f178806be30 0x7f178806e2f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f17a81e9190 0x7f17a80b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17a80b4ae0 0x7f17a81e8c50 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 --2- 172.21.15.26:0/1314304910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f17a81385b0 0x7f17a8139220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 >> 172.21.15.26:0/1314304910 conn(0x7f17a806b300 msgr2=0x7f17a815efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 shutdown_connections 2024-04-24T23:25:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:37.540+0000 7f17ae488640 1 -- 172.21.15.26:0/1314304910 wait complete. 2024-04-24T23:25:37.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:25:27.340834+0000', 'modified': '2024-04-24T23:25:30.362268+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7227}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7227': {'gid': 7227, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6837/2537133693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6835', 'nonce': 2537133693}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 2537133693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7227, 'qdb_cluster': [7227]} 2024-04-24T23:25:37.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:25:37.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:25:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.060+0000 7fd4b10a5640 1 Processor -- start 2024-04-24T23:25:38.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.060+0000 7fd4b10a5640 1 -- start start 2024-04-24T23:25:38.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4b10a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4ac143c40 0x7fd4ac0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4b10a5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd4ac0cc860 0x7fd4ac072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4b10a5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ac0cd0b0 con 0x7fd4ac15eb00 2024-04-24T23:25:38.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4b10a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd4ac071b70 con 0x7fd4ac143c40 2024-04-24T23:25:38.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4b10a5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd4ac071240 con 0x7fd4ac0cc860 2024-04-24T23:25:38.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4aad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4ac143c40 0x7fd4ac0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4aad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4ac143c40 0x7fd4ac0cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55338/0 (socket says 172.21.15.26:55338) 2024-04-24T23:25:38.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4aad9d640 1 -- 172.21.15.26:0/3603823749 learned_addr learned my addr 172.21.15.26:0/3603823749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:38.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/3603823749 >> v2:172.21.15.29:3300/0 conn(0x7fd4ac0cc860 0x7fd4ac072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4aad9d640 1 -- 172.21.15.26:0/3603823749 >> v2:172.21.15.29:3300/0 conn(0x7fd4ac0cc860 msgr2=0x7fd4ac072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/3603823749 >> v2:172.21.15.29:3300/0 conn(0x7fd4ac0cc860 0x7fd4ac072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4aad9d640 1 -- 172.21.15.26:0/3603823749 >> v1:172.21.15.29:6789/0 conn(0x7fd4ac15eb00 legacy=0x7fd4ac15ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.061+0000 7fd4aad9d640 1 -- 172.21.15.26:0/3603823749 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac071520 con 0x7fd4ac143c40 2024-04-24T23:25:38.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.062+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/3603823749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac143c40 0x7fd4ac0cf870 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac06db50 tx=0x7fd4a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23280057a1bd8727 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.062+0000 7fd4a9d9b640 1 -- 172.21.15.26:0/3603823749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4037070 con 0x7fd4ac143c40 2024-04-24T23:25:38.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.062+0000 7fd4a9d9b640 1 -- 172.21.15.26:0/3603823749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a4032440 con 0x7fd4ac143c40 2024-04-24T23:25:38.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.062+0000 7fd4a9d9b640 1 -- 172.21.15.26:0/3603823749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4032810 con 0x7fd4ac143c40 2024-04-24T23:25:38.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.063+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3603823749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac143c40 msgr2=0x7fd4ac0cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.063+0000 7fd4b10a5640 1 --2- 172.21.15.26:0/3603823749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac143c40 0x7fd4ac0cf870 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac06db50 tx=0x7fd4a402a230 comp rx=0 tx=0).stop 2024-04-24T23:25:38.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.063+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3603823749 shutdown_connections 2024-04-24T23:25:38.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.063+0000 7fd4b10a5640 1 --2- 172.21.15.26:0/3603823749 >> v2:172.21.15.29:3300/0 conn(0x7fd4ac0cc860 0x7fd4ac072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.063+0000 7fd4b10a5640 1 --2- 172.21.15.26:0/3603823749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac143c40 0x7fd4ac0cf870 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.063+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3603823749 >> 172.21.15.26:0/3603823749 conn(0x7fd4ac06b300 msgr2=0x7fd4ac06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.063+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3603823749 shutdown_connections 2024-04-24T23:25:38.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.063+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3603823749 wait complete. 2024-04-24T23:25:38.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.064+0000 7fd4b10a5640 1 Processor -- start 2024-04-24T23:25:38.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.064+0000 7fd4b10a5640 1 -- start start 2024-04-24T23:25:38.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4b10a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd4ac0cc860 0x7fd4ac1e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4b10a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ac143c40 0x7fd4ac15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4b10a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac15eb00 0x7fd4ac15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4b10a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac072540 con 0x7fd4ac15eb00 2024-04-24T23:25:38.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4b10a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac144020 con 0x7fd4ac143c40 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4b10a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ac0b0570 con 0x7fd4ac0cc860 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4ab59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac15eb00 0x7fd4ac15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4ab59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac15eb00 0x7fd4ac15aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55350/0 (socket says 172.21.15.26:55350) 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4ab59e640 1 -- 172.21.15.26:0/3716095022 learned_addr learned my addr 172.21.15.26:0/3716095022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ac143c40 0x7fd4ac15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.065+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd4ac0cc860 0x7fd4ac1e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4ab59e640 1 -- 172.21.15.26:0/3716095022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd4ac0cc860 msgr2=0x7fd4ac1e83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4ab59e640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd4ac0cc860 0x7fd4ac1e83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4ab59e640 1 -- 172.21.15.26:0/3716095022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ac143c40 msgr2=0x7fd4ac15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4ab59e640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ac143c40 0x7fd4ac15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4ab59e640 1 -- 172.21.15.26:0/3716095022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac1e7040 con 0x7fd4ac15eb00 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4ab59e640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac15eb00 0x7fd4ac15aa00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00b970 tx=0x7fd49c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4937fe640 1 -- 172.21.15.26:0/3716095022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c019070 con 0x7fd4ac15eb00 2024-04-24T23:25:38.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4937fe640 1 -- 172.21.15.26:0/3716095022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd49c00cb40 con 0x7fd4ac15eb00 2024-04-24T23:25:38.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4937fe640 1 -- 172.21.15.26:0/3716095022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c014440 con 0x7fd4ac15eb00 2024-04-24T23:25:38.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3716095022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ac1e7320 con 0x7fd4ac15eb00 2024-04-24T23:25:38.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.066+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3716095022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ac1e78e0 con 0x7fd4ac15eb00 2024-04-24T23:25:38.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.068+0000 7fd4937fe640 1 -- 172.21.15.26:0/3716095022 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd49c01c070 con 0x7fd4ac15eb00 2024-04-24T23:25:38.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.068+0000 7fd4937fe640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd49806be30 0x7fd49806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.068+0000 7fd4937fe640 1 -- 172.21.15.26:0/3716095022 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fd49c08ab50 con 0x7fd4ac15eb00 2024-04-24T23:25:38.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.068+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd49806be30 0x7fd49806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.069+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd49806be30 0x7fd49806e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4034000 tx=0x7fd4a402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.069+0000 7fd4b3dba740 1 Processor -- start 2024-04-24T23:25:38.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.069+0000 7fd4b3dba740 1 -- start start 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911a4540 0x564d91179350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564d910dcf00 con 0x564d911c1b60 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564d91085750 con 0x564d911a4c80 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x564d910d8ec0 con 0x564d911a4540 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55352/0 (socket says 172.21.15.26:55352) 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aad9d640 1 -- 172.21.15.26:0/1834052603 learned_addr learned my addr 172.21.15.26:0/1834052603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4ab59e640 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911a4540 0x564d91179350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aad9d640 1 -- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911a4540 msgr2=0x564d91179350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911a4540 0x564d91179350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aad9d640 1 -- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 msgr2=0x564d911a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aad9d640 1 -- 172.21.15.26:0/1834052603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d90df12d0 con 0x564d911c1b60 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4ab59e640 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911a4540 0x564d91179350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.070+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.071+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911744f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd4a402a820 tx=0x7fd4a402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58b45701ece1ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.071+0000 7fd490ff9640 1 -- 172.21.15.26:0/1834052603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4037040 con 0x564d911c1b60 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.071+0000 7fd490ff9640 1 -- 172.21.15.26:0/1834052603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a404d070 con 0x564d911c1b60 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.071+0000 7fd490ff9640 1 -- 172.21.15.26:0/1834052603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4032de0 con 0x564d911c1b60 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.071+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1834052603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 msgr2=0x564d911744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.071+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911744f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd4a402a820 tx=0x7fd4a402c000 comp rx=0 tx=0).stop 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.071+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1834052603 shutdown_connections 2024-04-24T23:25:38.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.072+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911a4540 0x564d91179350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.072+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.072+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1834052603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911744f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.072+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1834052603 >> 172.21.15.26:0/1834052603 conn(0x564d91176760 msgr2=0x564d911bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.072+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1834052603 shutdown_connections 2024-04-24T23:25:38.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.072+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1834052603 wait complete. 2024-04-24T23:25:38.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.072+0000 7fd4b3dba740 1 -- start start 2024-04-24T23:25:38.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.072+0000 7fd4b3dba740 1 Processor -- start 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911a5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 0x564d911cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564d91178910 con 0x564d911c1b60 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564d91098cf0 con 0x564d911a4c80 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x564d91177d00 con 0x564d91191a30 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4b3dba740 10 client.0 _ll_register_callbacks cb 0x564d910986e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911a5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55354/0 (socket says 172.21.15.26:55354) 2024-04-24T23:25:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aa59c640 1 -- 172.21.15.26:0/1239955744 learned_addr learned my addr 172.21.15.26:0/1239955744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4ab59e640 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 0x564d911cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aa59c640 1 -- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 msgr2=0x564d911cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 0x564d911cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aa59c640 1 -- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 msgr2=0x564d911b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aa59c640 1 -- 172.21.15.26:0/1239955744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d90ebb6b0 con 0x564d911c1b60 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.073+0000 7fd4ab59e640 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 0x564d911cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911a5170 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0012660 tx=0x7fd4a0012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776b2c6fdcb9eb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd489ffb640 1 -- 172.21.15.26:0/1239955744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a001b020 con 0x564d911c1b60 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd489ffb640 1 -- 172.21.15.26:0/1239955744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fd4a0004030 con 0x564d911c1b60 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1239955744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a404d070 con 0x564d911c1b60 2024-04-24T23:25:38.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd489ffb640 1 -- 172.21.15.26:0/1239955744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd4a0025050 con 0x564d911c1b60 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4b3dba740 10 client.7398 fetch_fsmap learned FSMap version 67 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1239955744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x564d911b6580 con 0x564d911c1b60 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd489ffb640 1 -- 172.21.15.26:0/1239955744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a404d070 con 0x564d911c1b60 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd489ffb640 1 -- 172.21.15.26:0/1239955744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fd4a0028020 con 0x564d911c1b60 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4b3dba740 10 client.7398 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4b3dba740 10 client.7398 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4b3dba740 20 client.7398 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x564d911d2b50 0x564d911d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4b3dba740 4 client.7398 mds_command: new command op to 7227 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1239955744 --> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d910ea440 con 0x564d911d2b50 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.074+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x564d911d2b50 0x564d911d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.075+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x564d911d2b50 0x564d911d2f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca2929c20047b7ef server_cookie=eb5976c2cfa59f44 in_seq=0 out_seq=0 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.075+0000 7fd489ffb640 10 client.7398 ms_handle_connect on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.075+0000 7fd489ffb640 1 -- 172.21.15.26:0/1239955744 <== mds.0 v2:172.21.15.26:6835/2537133693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d910ea440 con 0x564d911d2b50 2024-04-24T23:25:38.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.075+0000 7fd489ffb640 10 client.7398 handle_command_reply: tid=0 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.075+0000 7fd4b3dba740 1 client.7398 shutdown 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.075+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1239955744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 msgr2=0x564d911a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.075+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911a5170 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0012660 tx=0x7fd4a0012b20 comp rx=0 tx=0).stop 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1239955744 shutdown_connections 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x564d911d2b50 0x564d911d2f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 0x564d911cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911c1b60 0x564d911a5170 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1239955744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d911b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1239955744 >> 172.21.15.26:0/1239955744 conn(0x564d911a4540 msgr2=0x564d911a1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd489ffb640 0 client.7398 ms_handle_reset on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1239955744 shutdown_connections 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1239955744 wait complete. 2024-04-24T23:25:38.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.076+0000 7fd4b3dba740 20 client.7398 trim_cache size 0 max 16384 2024-04-24T23:25:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.094+0000 7fd4b3dba740 1 Processor -- start 2024-04-24T23:25:38.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.094+0000 7fd4b3dba740 1 -- start start 2024-04-24T23:25:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d91191a30 0x564d91175230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d91175770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911c1b60 0x564d911a7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac200880 con 0x564d91191a30 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac0b1cc0 con 0x564d911a4c80 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ac140d30 con 0x564d911c1b60 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d91191a30 0x564d91175230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d91191a30 0x564d91175230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55362/0 (socket says 172.21.15.26:55362) 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aad9d640 1 -- 172.21.15.26:0/1877389536 learned_addr learned my addr 172.21.15.26:0/1877389536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4ab59e640 1 --2- 172.21.15.26:0/1877389536 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911c1b60 0x564d911a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/1877389536 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d91175770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aad9d640 1 -- 172.21.15.26:0/1877389536 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911c1b60 msgr2=0x564d911a7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1877389536 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911c1b60 0x564d911a7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aad9d640 1 -- 172.21.15.26:0/1877389536 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 msgr2=0x564d91175770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1877389536 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d91175770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aad9d640 1 -- 172.21.15.26:0/1877389536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d911778a0 con 0x564d91191a30 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.095+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1877389536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d91191a30 0x564d91175230 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd4a403e040 tx=0x7fd4a403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a9ee0eb5f2f5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd48bfff640 1 -- 172.21.15.26:0/1877389536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4032ba0 con 0x564d91191a30 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd48bfff640 1 -- 172.21.15.26:0/1877389536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a405c070 con 0x564d91191a30 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd48bfff640 1 -- 172.21.15.26:0/1877389536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a40561a0 con 0x564d91191a30 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1877389536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d91191a30 msgr2=0x564d91175230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1877389536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d91191a30 0x564d91175230 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd4a403e040 tx=0x7fd4a403c000 comp rx=0 tx=0).stop 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1877389536 shutdown_connections 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1877389536 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d911c1b60 0x564d911a7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1877389536 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911a4c80 0x564d91175770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1877389536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d91191a30 0x564d91175230 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1877389536 >> 172.21.15.26:0/1877389536 conn(0x564d911a4540 msgr2=0x564d911be340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1877389536 shutdown_connections 2024-04-24T23:25:38.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.096+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1877389536 wait complete. 2024-04-24T23:25:38.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.097+0000 7fd4b3dba740 1 -- start start 2024-04-24T23:25:38.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.097+0000 7fd4b3dba740 1 Processor -- start 2024-04-24T23:25:38.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.097+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 0x564d912381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.097+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911a4c80 0x564d91174280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4b3dba740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911c1b60 0x564d911cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564d911d0030 con 0x564d911a4c80 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac0b1670 con 0x564d911c1b60 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4b3dba740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x564d91173620 con 0x564d91191a30 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4b3dba740 10 client.0 _ll_register_callbacks cb 0x7fd4a0007df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911a4c80 0x564d91174280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911a4c80 0x564d91174280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55378/0 (socket says 172.21.15.26:55378) 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aa59c640 1 -- 172.21.15.26:0/1554033602 learned_addr learned my addr 172.21.15.26:0/1554033602 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 0x564d912381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4ab59e640 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911c1b60 0x564d911cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aa59c640 1 -- 172.21.15.26:0/1554033602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 msgr2=0x564d912381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 0x564d912381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aa59c640 1 -- 172.21.15.26:0/1554033602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911c1b60 msgr2=0x564d911cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911c1b60 0x564d911cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aa59c640 1 -- 172.21.15.26:0/1554033602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d911c3210 con 0x564d911a4c80 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4aa59c640 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911a4c80 0x564d91174280 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0010f40 tx=0x7fd4a0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27040ae258bd76a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd489ffb640 1 -- 172.21.15.26:0/1554033602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a001b030 con 0x564d911a4c80 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd489ffb640 1 -- 172.21.15.26:0/1554033602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fd4a0015860 con 0x564d911a4c80 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1554033602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a405c070 con 0x564d911a4c80 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd489ffb640 1 -- 172.21.15.26:0/1554033602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd4a0037050 con 0x564d911a4c80 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4b3dba740 10 client.7404 fetch_fsmap learned FSMap version 67 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.098+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1554033602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x564d91194090 con 0x564d911a4c80 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd489ffb640 1 -- 172.21.15.26:0/1554033602 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a405c070 con 0x564d911a4c80 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd489ffb640 1 -- 172.21.15.26:0/1554033602 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fd4a003a020 con 0x564d911a4c80 2024-04-24T23:25:38.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4b3dba740 10 client.7404 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4b3dba740 10 client.7404 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4b3dba740 20 client.7404 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x564d911a78d0 0x564d911b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4b3dba740 4 client.7404 mds_command: new command op to 7227 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1554033602 --> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] -- command(tid 0: {"prefix": "status"}) -- 0x564d910e4790 con 0x564d911a78d0 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x564d911a78d0 0x564d911b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4aad9d640 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x564d911a78d0 0x564d911b4990 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f20555f7b08c4a6e server_cookie=11023a530e7da817 in_seq=0 out_seq=0 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd489ffb640 10 client.7404 ms_handle_connect on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd489ffb640 1 -- 172.21.15.26:0/1554033602 <== mds.0 v2:172.21.15.26:6835/2537133693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x564d910e4790 con 0x564d911a78d0 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd489ffb640 10 client.7404 handle_command_reply: tid=0 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4b3dba740 1 client.7404 shutdown 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1554033602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911a4c80 msgr2=0x564d91174280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.099+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911a4c80 0x564d91174280 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0010f40 tx=0x7fd4a0015000 comp rx=0 tx=0).stop 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1554033602 shutdown_connections 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x564d911a78d0 0x564d911b4990 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564d911c1b60 0x564d911cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd489ffb640 0 client.7404 ms_handle_reset on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d911a4c80 0x564d91174280 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b3dba740 1 --2- 172.21.15.26:0/1554033602 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564d91191a30 0x564d912381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1554033602 >> 172.21.15.26:0/1554033602 conn(0x564d911a4540 msgr2=0x564d911a5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1554033602 shutdown_connections 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b3dba740 1 -- 172.21.15.26:0/1554033602 wait complete. 2024-04-24T23:25:38.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b3dba740 20 client.7404 trim_cache size 0 max 16384 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7227, 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 10.10956565, 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 67, 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 71, 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 69, 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.136914451999999 2024-04-24T23:25:38.111 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd49806be30 msgr2=0x7fd49806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd49806be30 0x7fd49806e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4034000 tx=0x7fd4a402e000 comp rx=0 tx=0).stop 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac15eb00 msgr2=0x7fd4ac15aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac15eb00 0x7fd4ac15aa00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00b970 tx=0x7fd49c00be30 comp rx=0 tx=0).stop 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3716095022 shutdown_connections 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd49806be30 0x7fd49806e2f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac15eb00 0x7fd4ac15aa00 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4ac143c40 0x7fd4ac15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 --2- 172.21.15.26:0/3716095022 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd4ac0cc860 0x7fd4ac1e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3716095022 >> 172.21.15.26:0/3716095022 conn(0x7fd4ac06b300 msgr2=0x7fd4ac0ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3716095022 shutdown_connections 2024-04-24T23:25:38.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.100+0000 7fd4b10a5640 1 -- 172.21.15.26:0/3716095022 wait complete. 2024-04-24T23:25:38.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:25:38.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.214+0000 7f9b293ee640 1 Processor -- start 2024-04-24T23:25:38.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.215+0000 7f9b293ee640 1 -- start start 2024-04-24T23:25:38.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.215+0000 7f9b293ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 0x7f9b240d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.215+0000 7f9b293ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b240d22e0 0x7f9b240d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.215+0000 7f9b293ee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b240d32f0 con 0x7f9b240b3130 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.215+0000 7f9b293ee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9b240d1ab0 con 0x7f9b240d22e0 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.215+0000 7f9b293ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9b240d2ae0 con 0x7f9b2415e570 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.215+0000 7f9b237fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b240d22e0 0x7f9b240d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.215+0000 7f9b237fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b240d22e0 0x7f9b240d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55382/0 (socket says 172.21.15.26:55382) 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.216+0000 7f9b237fe640 1 -- 172.21.15.26:0/1810020874 learned_addr learned my addr 172.21.15.26:0/1810020874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.216+0000 7f9b22ffd640 1 --2- 172.21.15.26:0/1810020874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 0x7f9b240d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.216+0000 7f9b237fe640 1 -- 172.21.15.26:0/1810020874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 msgr2=0x7f9b240d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.216+0000 7f9b237fe640 1 --2- 172.21.15.26:0/1810020874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 0x7f9b240d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.216+0000 7f9b237fe640 1 -- 172.21.15.26:0/1810020874 >> v1:172.21.15.29:6789/0 conn(0x7f9b240b3130 legacy=0x7f9b240d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:38.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.216+0000 7f9b237fe640 1 -- 172.21.15.26:0/1810020874 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b240d0270 con 0x7f9b240d22e0 2024-04-24T23:25:38.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.216+0000 7f9b237fe640 1 --2- 172.21.15.26:0/1810020874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240d22e0 0x7f9b240d46a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009e50 tx=0x7f9b1402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56eaa8be342fa252 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b21ffb640 1 -- 172.21.15.26:0/1810020874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14036070 con 0x7f9b240d22e0 2024-04-24T23:25:38.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b21ffb640 1 -- 172.21.15.26:0/1810020874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b14031600 con 0x7f9b240d22e0 2024-04-24T23:25:38.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b21ffb640 1 -- 172.21.15.26:0/1810020874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140319d0 con 0x7f9b240d22e0 2024-04-24T23:25:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b293ee640 1 -- 172.21.15.26:0/1810020874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240d22e0 msgr2=0x7f9b240d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b293ee640 1 --2- 172.21.15.26:0/1810020874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240d22e0 0x7f9b240d46a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009e50 tx=0x7f9b1402a710 comp rx=0 tx=0).stop 2024-04-24T23:25:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b293ee640 1 -- 172.21.15.26:0/1810020874 shutdown_connections 2024-04-24T23:25:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b293ee640 1 --2- 172.21.15.26:0/1810020874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240d22e0 0x7f9b240d46a0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b293ee640 1 --2- 172.21.15.26:0/1810020874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 0x7f9b240d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b293ee640 1 -- 172.21.15.26:0/1810020874 >> 172.21.15.26:0/1810020874 conn(0x7f9b2408dbe0 msgr2=0x7f9b2409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b293ee640 1 -- 172.21.15.26:0/1810020874 shutdown_connections 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.217+0000 7f9b293ee640 1 -- 172.21.15.26:0/1810020874 wait complete. 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b293ee640 1 Processor -- start 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b293ee640 1 -- start start 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b293ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240b3130 0x7f9b241f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b293ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b240d22e0 0x7f9b240d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b293ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 0x7f9b240b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b293ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b240cfa70 con 0x7f9b240b3130 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b293ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9b240d3010 con 0x7f9b240d22e0 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b293ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9b240cfd50 con 0x7f9b2415e570 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b22ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240b3130 0x7f9b241f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b22ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240b3130 0x7f9b241f5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55384/0 (socket says 172.21.15.26:55384) 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.219+0000 7f9b22ffd640 1 -- 172.21.15.26:0/884096853 learned_addr learned my addr 172.21.15.26:0/884096853 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:38.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b237fe640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 0x7f9b240b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b227fc640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b240d22e0 0x7f9b240d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b22ffd640 1 -- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 msgr2=0x7f9b240b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b22ffd640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 0x7f9b240b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b22ffd640 1 -- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b240d22e0 msgr2=0x7f9b240d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b22ffd640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b240d22e0 0x7f9b240d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b22ffd640 1 -- 172.21.15.26:0/884096853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b241f7c40 con 0x7f9b240b3130 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b227fc640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b240d22e0 0x7f9b240d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b237fe640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 0x7f9b240b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b22ffd640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240b3130 0x7f9b241f5710 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00d620 tx=0x7f9b1c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b1b7fe640 1 -- 172.21.15.26:0/884096853 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00b620 con 0x7f9b240b3130 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b1b7fe640 1 -- 172.21.15.26:0/884096853 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c019070 con 0x7f9b240b3130 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b241f7f20 con 0x7f9b240b3130 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b1b7fe640 1 -- 172.21.15.26:0/884096853 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00ba00 con 0x7f9b240b3130 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.220+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b241f84e0 con 0x7f9b240b3130 2024-04-24T23:25:38.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.221+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b240694c0 con 0x7f9b240b3130 2024-04-24T23:25:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.222+0000 7f9b1b7fe640 1 -- 172.21.15.26:0/884096853 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b1c00bba0 con 0x7f9b240b3130 2024-04-24T23:25:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.222+0000 7f9b1b7fe640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9b0406be30 0x7f9b0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.222+0000 7f9b1b7fe640 1 -- 172.21.15.26:0/884096853 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f9b1c08b210 con 0x7f9b240b3130 2024-04-24T23:25:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.222+0000 7f9b227fc640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9b0406be30 0x7f9b0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:38.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.223+0000 7f9b227fc640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9b0406be30 0x7f9b0406e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9b240a21b0 tx=0x7f9b0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:38.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.226+0000 7f9b1b7fe640 1 -- 172.21.15.26:0/884096853 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9b1c059c10 con 0x7f9b240b3130 2024-04-24T23:25:38.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.445+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9b240745c0 con 0x7f9b240b3130 2024-04-24T23:25:38.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.445+0000 7f9b1b7fe640 1 -- 172.21.15.26:0/884096853 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f9b1c059510 con 0x7f9b240b3130 2024-04-24T23:25:38.446 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:25:38.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9b0406be30 msgr2=0x7f9b0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9b0406be30 0x7f9b0406e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9b240a21b0 tx=0x7f9b0c009040 comp rx=0 tx=0).stop 2024-04-24T23:25:38.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240b3130 msgr2=0x7f9b241f5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240b3130 0x7f9b241f5710 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00d620 tx=0x7f9b1c00dae0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 shutdown_connections 2024-04-24T23:25:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9b0406be30 0x7f9b0406e2f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9b2415e570 0x7f9b240b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b240d22e0 0x7f9b240d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 --2- 172.21.15.26:0/884096853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b240b3130 0x7f9b241f5710 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 >> 172.21.15.26:0/884096853 conn(0x7f9b2408dbe0 msgr2=0x7f9b2406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 shutdown_connections 2024-04-24T23:25:38.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38.447+0000 7f9b293ee640 1 -- 172.21.15.26:0/884096853 wait complete. 2024-04-24T23:25:38.459 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:25:38.459 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:25:38.459 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-24T23:25:38.459 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:25:38.459 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:25:38.459 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:25:38.459 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:25:38.459 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:25:38.460 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:25:38.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:38.460 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-24T23:25:38.475 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:25:38.475 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:25:38.475 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:25:38.475 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:25:38.475 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42454sec preferred_lft 42454sec 2024-04-24T23:25:38.476 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ccf2:1eff:fe9b:4749/64 scope link 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c2:ea:e0:db:00:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c0ea:e0ff:fedb:e0/64 scope link 2024-04-24T23:25:38.477 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:25:38.478 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:25:38.478 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:25:38.478 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::1c82:d0ff:fefa:5b3f/64 scope link 2024-04-24T23:25:38.478 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:25:38.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:38.478 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:25:38.539 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-24T23:25:38.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:38.541 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-24T23:25:38.596 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-24T23:25:38.596 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:25:38.596 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:25:38.596 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:25:38.596 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:25:38.597 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:25:38.597 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:25:38.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:25:38.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:25:38.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:25:38.597 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:25:38.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:25:38.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:25:38.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:38.723 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:25:38.738 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:25:38.738 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:25:38.738 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:25:38.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:38.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-24T23:25:38.802 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:25:38.802 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:25:38.858 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:25:38.858 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:25:38.858 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:25:38.913 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:25:38.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:38.913 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:25:38.995 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:25:38.995 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:25:38.995 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:25:39.004 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:25:39.004 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:25:39.004 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:25:39.004 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:25:39.004 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:25:39.009 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:25:39.009 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:25:39.009 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:25:39.009 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:25:39.009 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:25:39.009 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:25:39.009 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:25:39.009 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:25:39.012 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:25:39.012 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:25:39.012 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:25:39.012 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:25:39.012 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:25:39.012 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:25:39.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:39.013 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:25:39.077 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:25:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:25:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:25:39.179 INFO:teuthology.orchestra.run.smithi026.stdout:client7413 2024-04-24T23:25:39.179 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:25:39.179 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:25:39.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T23:25:39.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.337+0000 7faef768c640 1 Processor -- start 2024-04-24T23:25:39.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.338+0000 7faef768c640 1 -- start start 2024-04-24T23:25:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.338+0000 7faef768c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faef00d2c30 0x7faef00cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.338+0000 7faef768c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef00d2120 con 0x7faef015e750 2024-04-24T23:25:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.338+0000 7faef768c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef00b3740 con 0x7faef015e370 2024-04-24T23:25:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.338+0000 7faef768c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7faef00d1510 con 0x7faef00d2c30 2024-04-24T23:25:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.338+0000 7faef5c29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faef00d2c30 0x7faef00cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.338+0000 7faef4c27640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faef015e750 0x7faef00cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41206/0 (socket says 172.21.15.26:41206) 2024-04-24T23:25:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.338+0000 7faef4c27640 1 -- 172.21.15.26:0/3299237038 learned_addr learned my addr 172.21.15.26:0/3299237038 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.339+0000 7faef5c29640 1 -- 172.21.15.26:0/3299237038 >> v1:172.21.15.29:6789/0 conn(0x7faef015e370 legacy=0x7faef00d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.339+0000 7faef5c29640 1 -- 172.21.15.26:0/3299237038 >> v1:172.21.15.26:6789/0 conn(0x7faef015e750 legacy=0x7faef00cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.339+0000 7faeeffff640 1 -- 172.21.15.26:0/3299237038 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375342703 0 0) 0x7faef00d2120 con 0x7faef015e750 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.339+0000 7faef5c29640 1 -- 172.21.15.26:0/3299237038 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef00b3a20 con 0x7faef00d2c30 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faef5c29640 1 --2- 172.21.15.26:0/3299237038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00d2c30 0x7faef00cf3a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7faee0009d90 tx=0x7faee002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b035c03370e6ab53 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faeeffff640 1 -- 172.21.15.26:0/3299237038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0007740 con 0x7faef00d2c30 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faeeffff640 1 -- 172.21.15.26:0/3299237038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee00078e0 con 0x7faef00d2c30 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faeeffff640 1 -- 172.21.15.26:0/3299237038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0007cb0 con 0x7faef00d2c30 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faef768c640 1 -- 172.21.15.26:0/3299237038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00d2c30 msgr2=0x7faef00cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faef768c640 1 --2- 172.21.15.26:0/3299237038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00d2c30 0x7faef00cf3a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7faee0009d90 tx=0x7faee002a6f0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faef768c640 1 -- 172.21.15.26:0/3299237038 shutdown_connections 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faef768c640 1 --2- 172.21.15.26:0/3299237038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00d2c30 0x7faef00cf3a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faef768c640 1 -- 172.21.15.26:0/3299237038 >> 172.21.15.26:0/3299237038 conn(0x7faef00d40f0 msgr2=0x7faef00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faef768c640 1 -- 172.21.15.26:0/3299237038 shutdown_connections 2024-04-24T23:25:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.340+0000 7faef768c640 1 -- 172.21.15.26:0/3299237038 wait complete. 2024-04-24T23:25:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.341+0000 7faef768c640 1 Processor -- start 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef768c640 1 -- start start 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef768c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faef00d2c30 0x7faef00b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef768c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef015e370 0x7faef01e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef768c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faef0158180 0x7faef01e8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef768c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faef00b5230 con 0x7faef015e370 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef768c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faef00d1ed0 con 0x7faef00d2c30 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef768c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faef00b53b0 con 0x7faef0158180 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef4c27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef015e370 0x7faef01e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef4c27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef015e370 0x7faef01e7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55410/0 (socket says 172.21.15.26:55410) 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef4c27640 1 -- 172.21.15.26:0/669283482 learned_addr learned my addr 172.21.15.26:0/669283482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.342+0000 7faef5428640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faef00d2c30 0x7faef00b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faef5c29640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faef0158180 0x7faef01e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faef4c27640 1 -- 172.21.15.26:0/669283482 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faef0158180 msgr2=0x7faef01e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faef4c27640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faef0158180 0x7faef01e8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faef4c27640 1 -- 172.21.15.26:0/669283482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faef00d2c30 msgr2=0x7faef00b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faef4c27640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faef00d2c30 0x7faef00b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faef4c27640 1 -- 172.21.15.26:0/669283482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef01e7000 con 0x7faef015e370 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faef4c27640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef015e370 0x7faef01e7d30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7faee400c4d0 tx=0x7faee400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faeedffb640 1 -- 172.21.15.26:0/669283482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4018070 con 0x7faef015e370 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faeedffb640 1 -- 172.21.15.26:0/669283482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee40074e0 con 0x7faef015e370 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef00cb1c0 con 0x7faef015e370 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faeedffb640 1 -- 172.21.15.26:0/669283482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4004a20 con 0x7faef015e370 2024-04-24T23:25:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.343+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef00cb780 con 0x7faef015e370 2024-04-24T23:25:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.344+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef0075610 con 0x7faef015e370 2024-04-24T23:25:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.345+0000 7faeedffb640 1 -- 172.21.15.26:0/669283482 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faee401b070 con 0x7faef015e370 2024-04-24T23:25:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.345+0000 7faeedffb640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faedc06be30 0x7faedc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.345+0000 7faeedffb640 1 -- 172.21.15.26:0/669283482 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7faee408a080 con 0x7faef015e370 2024-04-24T23:25:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.345+0000 7faef5428640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faedc06be30 0x7faedc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.346+0000 7faef5428640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faedc06be30 0x7faedc06e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faef006db50 tx=0x7faee8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:39.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.350+0000 7faeedffb640 1 -- 172.21.15.26:0/669283482 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faee4058430 con 0x7faef015e370 2024-04-24T23:25:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:39.573+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7faef006be40 con 0x7faef015e370 2024-04-24T23:25:40.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.030+0000 7faeedffb640 1 -- 172.21.15.26:0/669283482 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v68) ==== 105+0+0 (secure 0 0 0) 0x7faee405c730 con 0x7faef015e370 2024-04-24T23:25:40.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faedc06be30 msgr2=0x7faedc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faedc06be30 0x7faedc06e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faef006db50 tx=0x7faee8009040 comp rx=0 tx=0).stop 2024-04-24T23:25:40.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef015e370 msgr2=0x7faef01e7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef015e370 0x7faef01e7d30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7faee400c4d0 tx=0x7faee400c990 comp rx=0 tx=0).stop 2024-04-24T23:25:40.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 shutdown_connections 2024-04-24T23:25:40.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faedc06be30 0x7faedc06e2f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faef0158180 0x7faef01e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef015e370 0x7faef01e7d30 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 --2- 172.21.15.26:0/669283482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faef00d2c30 0x7faef00b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.032+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 >> 172.21.15.26:0/669283482 conn(0x7faef00d40f0 msgr2=0x7faef00cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.033+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 shutdown_connections 2024-04-24T23:25:40.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.033+0000 7faef768c640 1 -- 172.21.15.26:0/669283482 wait complete. 2024-04-24T23:25:40.051 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.155+0000 7febff909640 1 Processor -- start 2024-04-24T23:25:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.155+0000 7febff909640 1 -- start start 2024-04-24T23:25:40.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febff909640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7febf80cf870 0x7febf8072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febff909640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf80cd0b0 con 0x7febf815eb00 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febff909640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf8071240 con 0x7febf8143c40 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febff909640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7febf8071b70 con 0x7febf80cf870 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febfdea6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7febf815eb00 0x7febf815ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41218/0 (socket says 172.21.15.26:41218) 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febfdea6640 1 -- 172.21.15.26:0/2048790828 learned_addr learned my addr 172.21.15.26:0/2048790828 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febfcea4640 1 --2- 172.21.15.26:0/2048790828 >> v2:172.21.15.29:3300/0 conn(0x7febf80cf870 0x7febf8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1709233211 0 0) 0x7febf80cd0b0 con 0x7febf815eb00 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febe0003520 con 0x7febf815eb00 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 270210483 0 0) 0x7febe0003520 con 0x7febf815eb00 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 >> v2:172.21.15.29:3300/0 conn(0x7febf80cf870 msgr2=0x7febf8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 --2- 172.21.15.26:0/2048790828 >> v2:172.21.15.29:3300/0 conn(0x7febf80cf870 0x7febf8072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 >> v1:172.21.15.29:6789/0 conn(0x7febf8143c40 legacy=0x7febf80cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf8071520 con 0x7febf815eb00 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7febe8001670 con 0x7febf815eb00 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 >> v1:172.21.15.26:6789/0 conn(0x7febf815eb00 legacy=0x7febf815ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febe00039b0 0x7febe0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febe0004800 0x7febe0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febe00051c0 0x7febe000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7febf8071b70 con 0x7febe00051c0 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febe000c060 con 0x7febe00039b0 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.156+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7febe000c260 con 0x7febe0004800 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.157+0000 7febfdea6640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febe00051c0 0x7febe000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.157+0000 7febfd6a5640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febe00039b0 0x7febe0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.157+0000 7febfcea4640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febe0004800 0x7febe0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.157+0000 7febfdea6640 1 -- 172.21.15.26:0/2048790828 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febe0004800 msgr2=0x7febe0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.157+0000 7febfdea6640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febe0004800 0x7febe0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.157+0000 7febfdea6640 1 -- 172.21.15.26:0/2048790828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febe00039b0 msgr2=0x7febe0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.157+0000 7febfdea6640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febe00039b0 0x7febe0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.157+0000 7febfdea6640 1 -- 172.21.15.26:0/2048790828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe0003520 con 0x7febe00051c0 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.157+0000 7febfdea6640 1 -- 172.21.15.26:0/2048790828 reap_dead start 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febfdea6640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febe00051c0 0x7febe000b9c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7febe800c240 tx=0x7febe0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd49ec534b507e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8007cf0 con 0x7febe00051c0 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe80058d0 con 0x7febe00051c0 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febf7fff640 1 -- 172.21.15.26:0/2048790828 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8005ce0 con 0x7febe00051c0 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febff909640 1 -- 172.21.15.26:0/2048790828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febe00051c0 msgr2=0x7febe000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febff909640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febe00051c0 0x7febe000b9c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7febe800c240 tx=0x7febe0005830 comp rx=0 tx=0).stop 2024-04-24T23:25:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febff909640 1 -- 172.21.15.26:0/2048790828 shutdown_connections 2024-04-24T23:25:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febff909640 1 --2- 172.21.15.26:0/2048790828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febe00051c0 0x7febe000b9c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febff909640 1 -- 172.21.15.26:0/2048790828 >> 172.21.15.26:0/2048790828 conn(0x7febf806b300 msgr2=0x7febf806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febff909640 1 -- 172.21.15.26:0/2048790828 shutdown_connections 2024-04-24T23:25:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.158+0000 7febff909640 1 -- 172.21.15.26:0/2048790828 wait complete. 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febff909640 1 Processor -- start 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febff909640 1 -- start start 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febff909640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febe00051c0 0x7febf81597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febff909640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf81e5e10 0x7febf813fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febff909640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febf8141bb0 0x7febf8142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febff909640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7febf815ef10 con 0x7febf81e5e10 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febff909640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7febf8144020 con 0x7febf8141bb0 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febff909640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7febf815f090 con 0x7febe00051c0 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febfcea4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf81e5e10 0x7febf813fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febfcea4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf81e5e10 0x7febf813fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55436/0 (socket says 172.21.15.26:55436) 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febfcea4640 1 -- 172.21.15.26:0/884616479 learned_addr learned my addr 172.21.15.26:0/884616479 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febfd6a5640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febe00051c0 0x7febf81597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.160+0000 7febfdea6640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febf8141bb0 0x7febf8142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febfcea4640 1 -- 172.21.15.26:0/884616479 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febe00051c0 msgr2=0x7febf81597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febfcea4640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febe00051c0 0x7febf81597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febfcea4640 1 -- 172.21.15.26:0/884616479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febf8141bb0 msgr2=0x7febf8142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febfcea4640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febf8141bb0 0x7febf8142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febfcea4640 1 -- 172.21.15.26:0/884616479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf8127ea0 con 0x7febf81e5e10 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febfcea4640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf81e5e10 0x7febf813fed0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7febec011940 tx=0x7febec011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febf5ffb640 1 -- 172.21.15.26:0/884616479 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec00faf0 con 0x7febf81e5e10 2024-04-24T23:25:40.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febf5ffb640 1 -- 172.21.15.26:0/884616479 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec00fc90 con 0x7febf81e5e10 2024-04-24T23:25:40.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febff909640 1 -- 172.21.15.26:0/884616479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf8128180 con 0x7febf81e5e10 2024-04-24T23:25:40.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febf5ffb640 1 -- 172.21.15.26:0/884616479 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec014440 con 0x7febf81e5e10 2024-04-24T23:25:40.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.161+0000 7febff909640 1 -- 172.21.15.26:0/884616479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf8142570 con 0x7febf81e5e10 2024-04-24T23:25:40.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.162+0000 7febff909640 1 -- 172.21.15.26:0/884616479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf80694c0 con 0x7febf81e5e10 2024-04-24T23:25:40.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.163+0000 7febf5ffb640 1 -- 172.21.15.26:0/884616479 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febec012040 con 0x7febf81e5e10 2024-04-24T23:25:40.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.163+0000 7febf5ffb640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7febe406be30 0x7febe406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:40.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.164+0000 7febf5ffb640 1 -- 172.21.15.26:0/884616479 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7febec08b3e0 con 0x7febf81e5e10 2024-04-24T23:25:40.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.164+0000 7febfd6a5640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7febe406be30 0x7febe406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:40.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.164+0000 7febfd6a5640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7febe406be30 0x7febe406e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f80 tx=0x7febf0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:40.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.168+0000 7febf5ffb640 1 -- 172.21.15.26:0/884616479 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7febec019030 con 0x7febf81e5e10 2024-04-24T23:25:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.387+0000 7febff909640 1 -- 172.21.15.26:0/884616479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febf80b4d20 con 0x7febf81e5e10 2024-04-24T23:25:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.388+0000 7febf5ffb640 1 -- 172.21.15.26:0/884616479 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5199 (secure 0 0 0) 0x7febec059700 con 0x7febf81e5e10 2024-04-24T23:25:40.389 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:40.389 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":69,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:25:40.033082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7227,"mds_1":7224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7224":{"gid":7224,"name":"a","rank":1,"incarnation":69,"state":"up:creating","state_seq":1,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[7227]},"id":5}]} 2024-04-24T23:25:40.389 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 69 2024-04-24T23:25:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 -- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7febe406be30 msgr2=0x7febe406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7febe406be30 0x7febe406e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f80 tx=0x7febf0009040 comp rx=0 tx=0).stop 2024-04-24T23:25:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 -- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf81e5e10 msgr2=0x7febf813fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf81e5e10 0x7febf813fed0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7febec011940 tx=0x7febec011e00 comp rx=0 tx=0).stop 2024-04-24T23:25:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 -- 172.21.15.26:0/884616479 shutdown_connections 2024-04-24T23:25:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7febe406be30 0x7febe406e2f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7febf8141bb0 0x7febf8142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf81e5e10 0x7febf813fed0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 --2- 172.21.15.26:0/884616479 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7febe00051c0 0x7febf81597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 -- 172.21.15.26:0/884616479 >> 172.21.15.26:0/884616479 conn(0x7febf806b300 msgr2=0x7febf80afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 -- 172.21.15.26:0/884616479 shutdown_connections 2024-04-24T23:25:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:40.390+0000 7febff909640 1 -- 172.21.15.26:0/884616479 wait complete. 2024-04-24T23:25:40.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:25:27.340834+0000', 'modified': '2024-04-24T23:25:40.033082+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7227, 'mds_1': 7224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7224': {'gid': 7224, 'name': 'a', 'rank': 1, 'incarnation': 69, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.26:6836/1431580336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1431580336}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 1431580336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7227': {'gid': 7227, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6837/2537133693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6835', 'nonce': 2537133693}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 2537133693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7227, 'qdb_cluster': [7227]} 2024-04-24T23:25:40.805 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7224:up:creating 2024-04-24T23:25:41.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.906+0000 7f534b488640 1 Processor -- start 2024-04-24T23:25:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.907+0000 7f534b488640 1 -- start start 2024-04-24T23:25:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.907+0000 7f534b488640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f53440d4870 0x7f53440b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.907+0000 7f534b488640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53440d32f0 con 0x7f53440b3130 2024-04-24T23:25:41.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.907+0000 7f534b488640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53440d0270 con 0x7f534415e570 2024-04-24T23:25:41.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.907+0000 7f534b488640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f53440d1ab0 con 0x7f53440d4870 2024-04-24T23:25:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.908+0000 7f5348a23640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f53440d4870 0x7f53440b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.908+0000 7f5349a25640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f53440b3130 0x7f53440b4640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41234/0 (socket says 172.21.15.26:41234) 2024-04-24T23:25:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.908+0000 7f5349a25640 1 -- 172.21.15.26:0/1248563682 learned_addr learned my addr 172.21.15.26:0/1248563682 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.908+0000 7f5348a23640 1 -- 172.21.15.26:0/1248563682 >> v1:172.21.15.29:6789/0 conn(0x7f534415e570 legacy=0x7f53440d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.908+0000 7f5348a23640 1 -- 172.21.15.26:0/1248563682 >> v1:172.21.15.26:6789/0 conn(0x7f53440b3130 legacy=0x7f53440b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.908+0000 7f5348a23640 1 -- 172.21.15.26:0/1248563682 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53440d2d30 con 0x7f53440d4870 2024-04-24T23:25:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f5348a23640 1 --2- 172.21.15.26:0/1248563682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 0x7f53440b4100 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f53400097a0 tx=0x7f534002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bccb7daab434902 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f533bfff640 1 -- 172.21.15.26:0/1248563682 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340037070 con 0x7f53440d4870 2024-04-24T23:25:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f533bfff640 1 -- 172.21.15.26:0/1248563682 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5340032440 con 0x7f53440d4870 2024-04-24T23:25:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f533bfff640 1 -- 172.21.15.26:0/1248563682 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340032810 con 0x7f53440d4870 2024-04-24T23:25:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f534b488640 1 -- 172.21.15.26:0/1248563682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 msgr2=0x7f53440b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f534b488640 1 --2- 172.21.15.26:0/1248563682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 0x7f53440b4100 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f53400097a0 tx=0x7f534002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:25:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f534b488640 1 -- 172.21.15.26:0/1248563682 shutdown_connections 2024-04-24T23:25:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f534b488640 1 --2- 172.21.15.26:0/1248563682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 0x7f53440b4100 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f534b488640 1 -- 172.21.15.26:0/1248563682 >> 172.21.15.26:0/1248563682 conn(0x7f534408dbe0 msgr2=0x7f534409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f534b488640 1 -- 172.21.15.26:0/1248563682 shutdown_connections 2024-04-24T23:25:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.909+0000 7f534b488640 1 -- 172.21.15.26:0/1248563682 wait complete. 2024-04-24T23:25:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f534b488640 1 Processor -- start 2024-04-24T23:25:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f534b488640 1 -- start start 2024-04-24T23:25:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f534b488640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53440b3130 0x7f5344156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f534b488640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 0x7f5344157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f534b488640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f534415e570 0x7f53440b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f534b488640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f53440cfa70 con 0x7f53440d4870 2024-04-24T23:25:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f534b488640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f53440d2ae0 con 0x7f53440b3130 2024-04-24T23:25:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f534b488640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f53440cfd50 con 0x7f534415e570 2024-04-24T23:25:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f5348a23640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 0x7f5344157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f5348a23640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 0x7f5344157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55458/0 (socket says 172.21.15.26:55458) 2024-04-24T23:25:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.911+0000 7f5348a23640 1 -- 172.21.15.26:0/4141357720 learned_addr learned my addr 172.21.15.26:0/4141357720 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5349224640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53440b3130 0x7f5344156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5349a25640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f534415e570 0x7f53440b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5348a23640 1 -- 172.21.15.26:0/4141357720 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f534415e570 msgr2=0x7f53440b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5348a23640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f534415e570 0x7f53440b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5348a23640 1 -- 172.21.15.26:0/4141357720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53440b3130 msgr2=0x7f5344156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5348a23640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53440b3130 0x7f5344156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5348a23640 1 -- 172.21.15.26:0/4141357720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5344158b80 con 0x7f53440d4870 2024-04-24T23:25:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5348a23640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 0x7f5344157250 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5340035040 tx=0x7f53400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5339ffb640 1 -- 172.21.15.26:0/4141357720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340037040 con 0x7f53440d4870 2024-04-24T23:25:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5339ffb640 1 -- 172.21.15.26:0/4141357720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5340030770 con 0x7f53440d4870 2024-04-24T23:25:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5344157ae0 con 0x7f53440d4870 2024-04-24T23:25:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f5339ffb640 1 -- 172.21.15.26:0/4141357720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340030b40 con 0x7f53440d4870 2024-04-24T23:25:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.912+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53441580a0 con 0x7f53440d4870 2024-04-24T23:25:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.913+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53440694c0 con 0x7f53440d4870 2024-04-24T23:25:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.914+0000 7f5339ffb640 1 -- 172.21.15.26:0/4141357720 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5340044020 con 0x7f53440d4870 2024-04-24T23:25:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.914+0000 7f5339ffb640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f532806be30 0x7f532806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.914+0000 7f5339ffb640 1 -- 172.21.15.26:0/4141357720 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f53400ab430 con 0x7f53440d4870 2024-04-24T23:25:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.914+0000 7f5349224640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f532806be30 0x7f532806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.915+0000 7f5349224640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f532806be30 0x7f532806e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f53441f7d50 tx=0x7f5334009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:41.919+0000 7f5339ffb640 1 -- 172.21.15.26:0/4141357720 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5340032e20 con 0x7f53440d4870 2024-04-24T23:25:42.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.146+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53440b2d80 con 0x7f53440d4870 2024-04-24T23:25:42.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.147+0000 7f5339ffb640 1 -- 172.21.15.26:0/4141357720 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5221 (secure 0 0 0) 0x7f5340079da0 con 0x7f53440d4870 2024-04-24T23:25:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:42.148 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:25:42.042533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7227,"mds_1":7224,"mds_2":6364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6364":{"gid":6364,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7224":{"gid":7224,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[6364,7224,7227]},"id":5}]} 2024-04-24T23:25:42.148 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 72 2024-04-24T23:25:42.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f532806be30 msgr2=0x7f532806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f532806be30 0x7f532806e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f53441f7d50 tx=0x7f5334009040 comp rx=0 tx=0).stop 2024-04-24T23:25:42.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 msgr2=0x7f5344157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 0x7f5344157250 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5340035040 tx=0x7f53400046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 shutdown_connections 2024-04-24T23:25:42.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f532806be30 0x7f532806e2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f534415e570 0x7f53440b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53440d4870 0x7f5344157250 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 --2- 172.21.15.26:0/4141357720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f53440b3130 0x7f5344156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 >> 172.21.15.26:0/4141357720 conn(0x7f534408dbe0 msgr2=0x7f534406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 shutdown_connections 2024-04-24T23:25:42.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.149+0000 7f534b488640 1 -- 172.21.15.26:0/4141357720 wait complete. 2024-04-24T23:25:42.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:25:27.340834+0000', 'modified': '2024-04-24T23:25:42.042533+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7227, 'mds_1': 7224, 'mds_2': 6364}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6364': {'gid': 6364, 'name': 'd', 'rank': 2, 'incarnation': 71, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.29:6835/2057165440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 2057165440}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 2057165440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7224': {'gid': 7224, 'name': 'a', 'rank': 1, 'incarnation': 69, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.26:6836/1431580336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1431580336}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 1431580336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7227': {'gid': 7227, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6837/2537133693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6835', 'nonce': 2537133693}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 2537133693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7227, 'qdb_cluster': [6364, 7224, 7227]} 2024-04-24T23:25:42.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:25:42.564 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:25:42.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.664+0000 7f0a9517e640 1 Processor -- start 2024-04-24T23:25:42.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.664+0000 7f0a9517e640 1 -- start start 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.665+0000 7f0a9517e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f0a900cdd70 0x7f0a9006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.665+0000 7f0a9517e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a900cfbc0 con 0x7f0a90145910 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.665+0000 7f0a9517e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a9006d7f0 con 0x7f0a9006d350 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.665+0000 7f0a9517e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f0a90145cf0 con 0x7f0a900cdd70 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.665+0000 7f0a8ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0a90145910 0x7f0a9015d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41238/0 (socket says 172.21.15.26:41238) 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.665+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/2833041701 learned_addr learned my addr 172.21.15.26:0/2833041701 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.665+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3649237637 0 0) 0x7f0a900cfbc0 con 0x7f0a90145910 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.665+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a7c003520 con 0x7f0a90145910 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.665+0000 7f0a8f59e640 1 --2- 172.21.15.26:0/2833041701 >> v2:172.21.15.29:3300/0 conn(0x7f0a900cdd70 0x7f0a9006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3839695573 0 0) 0x7f0a7c003520 con 0x7f0a90145910 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 >> v2:172.21.15.29:3300/0 conn(0x7f0a900cdd70 msgr2=0x7f0a9006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 --2- 172.21.15.26:0/2833041701 >> v2:172.21.15.29:3300/0 conn(0x7f0a900cdd70 0x7f0a9006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 >> v1:172.21.15.29:6789/0 conn(0x7f0a9006d350 legacy=0x7f0a900cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9006dad0 con 0x7f0a90145910 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f0a80001670 con 0x7f0a90145910 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 >> v1:172.21.15.26:6789/0 conn(0x7f0a90145910 legacy=0x7f0a9015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a7c0039b0 0x7f0a7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a7c004800 0x7f0a7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a7c0051c0 0x7f0a7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90145cf0 con 0x7f0a7c0051c0 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c00c160 con 0x7f0a7c004800 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7c00c360 con 0x7f0a7c0039b0 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8f59e640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a7c0051c0 0x7f0a7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a87fff640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a7c004800 0x7f0a7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a7c0039b0 0x7f0a7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8f59e640 1 -- 172.21.15.26:0/2833041701 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a7c0039b0 msgr2=0x7f0a7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8f59e640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a7c0039b0 0x7f0a7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8f59e640 1 -- 172.21.15.26:0/2833041701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a7c004800 msgr2=0x7f0a7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8f59e640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a7c004800 0x7f0a7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.666+0000 7f0a8f59e640 1 -- 172.21.15.26:0/2833041701 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c003520 con 0x7f0a7c0051c0 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.667+0000 7f0a8f59e640 1 -- 172.21.15.26:0/2833041701 reap_dead start 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.667+0000 7f0a8f59e640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a7c0051c0 0x7f0a7c00b9c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009fc0 tx=0x7f0a7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209e68d9fd724020 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.667+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88035020 con 0x7f0a7c0051c0 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.667+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a88003bd0 con 0x7f0a7c0051c0 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.667+0000 7f0a8e59c640 1 -- 172.21.15.26:0/2833041701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88009c10 con 0x7f0a7c0051c0 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.668+0000 7f0a9517e640 1 -- 172.21.15.26:0/2833041701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a7c0051c0 msgr2=0x7f0a7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.668+0000 7f0a9517e640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a7c0051c0 0x7f0a7c00b9c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009fc0 tx=0x7f0a7c005830 comp rx=0 tx=0).stop 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.668+0000 7f0a9517e640 1 -- 172.21.15.26:0/2833041701 shutdown_connections 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.668+0000 7f0a9517e640 1 --2- 172.21.15.26:0/2833041701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a7c0051c0 0x7f0a7c00b9c0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.668+0000 7f0a9517e640 1 -- 172.21.15.26:0/2833041701 >> 172.21.15.26:0/2833041701 conn(0x7f0a9006b300 msgr2=0x7f0a9006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.668+0000 7f0a9517e640 1 -- 172.21.15.26:0/2833041701 shutdown_connections 2024-04-24T23:25:42.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.668+0000 7f0a9517e640 1 -- 172.21.15.26:0/2833041701 wait complete. 2024-04-24T23:25:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.669+0000 7f0a9517e640 1 Processor -- start 2024-04-24T23:25:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.669+0000 7f0a9517e640 1 -- start start 2024-04-24T23:25:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a9517e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a7c0051c0 0x7f0a901325c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a9517e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a9013c3f0 0x7f0a90158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a9517e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a90155100 0x7f0a9015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a9517e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9006e370 con 0x7f0a9013c3f0 2024-04-24T23:25:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a9517e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9006e090 con 0x7f0a90155100 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a9517e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9006e890 con 0x7f0a7c0051c0 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a9013c3f0 0x7f0a90158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a9013c3f0 0x7f0a90158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55484/0 (socket says 172.21.15.26:55484) 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a87fff640 1 -- 172.21.15.26:0/3790006581 learned_addr learned my addr 172.21.15.26:0/3790006581 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a7c0051c0 0x7f0a901325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a87fff640 1 -- 172.21.15.26:0/3790006581 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a7c0051c0 msgr2=0x7f0a901325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a8f59e640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a90155100 0x7f0a9015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a87fff640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a7c0051c0 0x7f0a901325c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a87fff640 1 -- 172.21.15.26:0/3790006581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a90155100 msgr2=0x7f0a9015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a87fff640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a90155100 0x7f0a9015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a87fff640 1 -- 172.21.15.26:0/3790006581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a90154060 con 0x7f0a9013c3f0 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.670+0000 7f0a87fff640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a9013c3f0 0x7f0a90158a50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800f5f0 tx=0x7f0a7800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.671+0000 7f0a877fe640 1 -- 172.21.15.26:0/3790006581 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a780052e0 con 0x7f0a9013c3f0 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.671+0000 7f0a877fe640 1 -- 172.21.15.26:0/3790006581 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a78018070 con 0x7f0a9013c3f0 2024-04-24T23:25:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.671+0000 7f0a9517e640 1 -- 172.21.15.26:0/3790006581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a90154340 con 0x7f0a9013c3f0 2024-04-24T23:25:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.671+0000 7f0a877fe640 1 -- 172.21.15.26:0/3790006581 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78003810 con 0x7f0a9013c3f0 2024-04-24T23:25:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.671+0000 7f0a9517e640 1 -- 172.21.15.26:0/3790006581 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a90154900 con 0x7f0a9013c3f0 2024-04-24T23:25:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.672+0000 7f0a877fe640 1 -- 172.21.15.26:0/3790006581 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a780039b0 con 0x7f0a9013c3f0 2024-04-24T23:25:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.672+0000 7f0a877fe640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a7006be60 0x7f0a7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.673+0000 7f0a877fe640 1 -- 172.21.15.26:0/3790006581 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f0a7808ac90 con 0x7f0a9013c3f0 2024-04-24T23:25:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.673+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a7006be60 0x7f0a7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.673+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a7006be60 0x7f0a7006e320 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0a80001050 tx=0x7f0a80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.674+0000 7f0a97f6c740 1 Processor -- start 2024-04-24T23:25:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.675+0000 7f0a97f6c740 1 -- start start 2024-04-24T23:25:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847fe50 0x55a1d8483c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d8457b60 0x55a1d8480230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d8483000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1d83bbf00 con 0x55a1d8457b60 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a1d8364750 con 0x55a1d8480770 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a1d83b7ec0 con 0x55a1d847fe50 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d8457b60 0x55a1d8480230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d8457b60 0x55a1d8480230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55494/0 (socket says 172.21.15.26:55494) 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a87fff640 1 -- 172.21.15.26:0/1743798879 learned_addr learned my addr 172.21.15.26:0/1743798879 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a87fff640 1 -- 172.21.15.26:0/1743798879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847fe50 msgr2=0x55a1d8483c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a87fff640 1 --2- 172.21.15.26:0/1743798879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847fe50 0x55a1d8483c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a87fff640 1 -- 172.21.15.26:0/1743798879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 msgr2=0x55a1d8483000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a87fff640 1 --2- 172.21.15.26:0/1743798879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d8483000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a87fff640 1 -- 172.21.15.26:0/1743798879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1d80d02d0 con 0x55a1d8457b60 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.676+0000 7f0a87fff640 1 --2- 172.21.15.26:0/1743798879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d8457b60 0x55a1d8480230 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0a78058e30 tx=0x7f0a78005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50330d344e85078a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a84ff9640 1 -- 172.21.15.26:0/1743798879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78058ef0 con 0x55a1d8457b60 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a84ff9640 1 -- 172.21.15.26:0/1743798879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a78087070 con 0x55a1d8457b60 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a84ff9640 1 -- 172.21.15.26:0/1743798879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7805d2a0 con 0x55a1d8457b60 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a97f6c740 1 -- 172.21.15.26:0/1743798879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d8457b60 msgr2=0x55a1d8480230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/1743798879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d8457b60 0x55a1d8480230 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0a78058e30 tx=0x7f0a78005480 comp rx=0 tx=0).stop 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a97f6c740 1 -- 172.21.15.26:0/1743798879 shutdown_connections 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/1743798879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d8483000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/1743798879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d8457b60 0x55a1d8480230 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/1743798879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847fe50 0x55a1d8483c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a97f6c740 1 -- 172.21.15.26:0/1743798879 >> 172.21.15.26:0/1743798879 conn(0x55a1d8455760 msgr2=0x55a1d849e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a97f6c740 1 -- 172.21.15.26:0/1743798879 shutdown_connections 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.677+0000 7f0a97f6c740 1 -- 172.21.15.26:0/1743798879 wait complete. 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.678+0000 7f0a97f6c740 1 -- start start 2024-04-24T23:25:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.678+0000 7f0a97f6c740 1 Processor -- start 2024-04-24T23:25:42.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847fe50 0x55a1d8473a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d84a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847f210 0x55a1d84a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1d8453ab0 con 0x55a1d847fe50 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a1d8377cf0 con 0x55a1d8480770 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a1d84534f0 con 0x55a1d847f210 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a97f6c740 10 client.0 _ll_register_callbacks cb 0x55a1d83776e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847fe50 0x55a1d8473a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847fe50 0x55a1d8473a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55508/0 (socket says 172.21.15.26:55508) 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/3169799541 learned_addr learned my addr 172.21.15.26:0/3169799541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8f59e640 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847f210 0x55a1d84a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a87fff640 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d84a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847f210 msgr2=0x55a1d84a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847f210 0x55a1d84a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 msgr2=0x55a1d84a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d84a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/3169799541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1d819a6b0 con 0x55a1d847fe50 2024-04-24T23:25:42.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8f59e640 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847f210 0x55a1d84a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a87fff640 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d84a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847fe50 0x55a1d8473a60 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f0a800077c0 tx=0x7f0a800078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb9b9dc8dccf3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.679+0000 7f0a6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a6dffb640 1 -- 172.21.15.26:0/3169799541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80022450 con 0x55a1d847fe50 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a6dffb640 1 -- 172.21.15.26:0/3169799541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f0a800225f0 con 0x55a1d847fe50 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a97f6c740 1 -- 172.21.15.26:0/3169799541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a78087070 con 0x55a1d847fe50 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a6dffb640 1 -- 172.21.15.26:0/3169799541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f0a800227f0 con 0x55a1d847fe50 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a97f6c740 10 client.7449 fetch_fsmap learned FSMap version 72 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a97f6c740 1 -- 172.21.15.26:0/3169799541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55a1d8477f00 con 0x55a1d847fe50 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a6dffb640 1 -- 172.21.15.26:0/3169799541 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a78087070 con 0x55a1d847fe50 2024-04-24T23:25:42.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a6dffb640 1 -- 172.21.15.26:0/3169799541 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 72) ==== 3534+0+0 (secure 0 0 0) 0x7f0a80022990 con 0x55a1d847fe50 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a97f6c740 10 client.7449 fetch_fsmap finished waiting for FSMap version 72 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a97f6c740 10 client.7449 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a97f6c740 20 client.7449 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x55a1d847d790 0x55a1d847db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a97f6c740 4 client.7449 mds_command: new command op to 6364 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a97f6c740 1 -- 172.21.15.26:0/3169799541 --> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1d83c9440 con 0x55a1d847d790 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.680+0000 7f0a87fff640 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x55a1d847d790 0x55a1d847db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.681+0000 7f0a87fff640 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x55a1d847d790 0x55a1d847db70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66b5b06229c8d3ed server_cookie=89df89563de70dd3 in_seq=0 out_seq=0 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.681+0000 7f0a6dffb640 10 client.7449 ms_handle_connect on v2:172.21.15.29:6833/2057165440 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.681+0000 7f0a6dffb640 1 -- 172.21.15.26:0/3169799541 <== mds.2 v2:172.21.15.29:6833/2057165440 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1d83c9440 con 0x55a1d847d790 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.681+0000 7f0a6dffb640 10 client.7449 handle_command_reply: tid=0 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.681+0000 7f0a97f6c740 1 client.7449 shutdown 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 -- 172.21.15.26:0/3169799541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847fe50 msgr2=0x55a1d8473a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847fe50 0x55a1d8473a60 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f0a800077c0 tx=0x7f0a800078f0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 -- 172.21.15.26:0/3169799541 shutdown_connections 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x55a1d847d790 0x55a1d847db70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847f210 0x55a1d84a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a6dffb640 0 client.7449 ms_handle_reset on v2:172.21.15.29:6833/2057165440 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d84a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/3169799541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847fe50 0x55a1d8473a60 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 -- 172.21.15.26:0/3169799541 >> 172.21.15.26:0/3169799541 conn(0x55a1d8457b60 msgr2=0x55a1d8458b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 -- 172.21.15.26:0/3169799541 shutdown_connections 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 1 -- 172.21.15.26:0/3169799541 wait complete. 2024-04-24T23:25:42.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.682+0000 7f0a97f6c740 20 client.7449 trim_cache size 0 max 16384 2024-04-24T23:25:42.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.700+0000 7f0a97f6c740 1 Processor -- start 2024-04-24T23:25:42.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.700+0000 7f0a97f6c740 1 -- start start 2024-04-24T23:25:42.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d8473990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d847fe50 0x55a1d849e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d8480770 0x55a1d84863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901fbe70 con 0x55a1d847f210 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901fc3b0 con 0x55a1d847fe50 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0a900b4190 con 0x55a1d8480770 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a8ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d8473990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a8ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d8473990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55512/0 (socket says 172.21.15.26:55512) 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/181797867 learned_addr learned my addr 172.21.15.26:0/181797867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/181797867 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d8480770 msgr2=0x55a1d84863a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/181797867 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d8480770 0x55a1d84863a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a87fff640 1 --2- 172.21.15.26:0/181797867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d847fe50 0x55a1d849e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/181797867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d847fe50 msgr2=0x55a1d849e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/181797867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d847fe50 0x55a1d849e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/181797867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1d84a0a90 con 0x55a1d847f210 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a87fff640 1 --2- 172.21.15.26:0/181797867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d847fe50 0x55a1d849e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/181797867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d8473990 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0a80023f70 tx=0x7f0a8001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ca793e00979d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a6ffff640 1 -- 172.21.15.26:0/181797867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032070 con 0x55a1d847f210 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a6ffff640 1 -- 172.21.15.26:0/181797867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a80022990 con 0x55a1d847f210 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.701+0000 7f0a6ffff640 1 -- 172.21.15.26:0/181797867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002e450 con 0x55a1d847f210 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.702+0000 7f0a97f6c740 1 -- 172.21.15.26:0/181797867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 msgr2=0x55a1d8473990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.702+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/181797867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d8473990 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0a80023f70 tx=0x7f0a8001e000 comp rx=0 tx=0).stop 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.702+0000 7f0a97f6c740 1 -- 172.21.15.26:0/181797867 shutdown_connections 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.702+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/181797867 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d8480770 0x55a1d84863a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.702+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/181797867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d847fe50 0x55a1d849e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.702+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/181797867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d8473990 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.702+0000 7f0a97f6c740 1 -- 172.21.15.26:0/181797867 >> 172.21.15.26:0/181797867 conn(0x55a1d8457b60 msgr2=0x55a1d8452840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.702+0000 7f0a97f6c740 1 -- 172.21.15.26:0/181797867 shutdown_connections 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.702+0000 7f0a97f6c740 1 -- 172.21.15.26:0/181797867 wait complete. 2024-04-24T23:25:42.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.703+0000 7f0a97f6c740 1 -- start start 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.703+0000 7f0a97f6c740 1 Processor -- start 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.703+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d84867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.703+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847fe50 0x55a1d8472640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.703+0000 7f0a97f6c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d8472b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.703+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1d8475b50 con 0x55a1d847f210 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.703+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a1d8481f30 con 0x55a1d8480770 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.703+0000 7f0a97f6c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a1d84760e0 con 0x55a1d847fe50 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.703+0000 7f0a97f6c740 10 client.0 _ll_register_callbacks cb 0x7f0a80007410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d84867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d84867a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55528/0 (socket says 172.21.15.26:55528) 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/147010080 learned_addr learned my addr 172.21.15.26:0/147010080 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a87fff640 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847fe50 0x55a1d8472640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8f59e640 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d8472b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847fe50 msgr2=0x55a1d8472640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847fe50 0x55a1d8472640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 msgr2=0x55a1d8472b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d8472b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8ed9d640 1 -- 172.21.15.26:0/147010080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1d8486ce0 con 0x55a1d847f210 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a8ed9d640 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d84867a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000fb0 tx=0x7f0a80006070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3013dfc0117375c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a6dffb640 1 -- 172.21.15.26:0/147010080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8003b060 con 0x55a1d847f210 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a6dffb640 1 -- 172.21.15.26:0/147010080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f0a8002e650 con 0x55a1d847f210 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a97f6c740 1 -- 172.21.15.26:0/147010080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a80022990 con 0x55a1d847f210 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a6dffb640 1 -- 172.21.15.26:0/147010080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f0a80041050 con 0x55a1d847f210 2024-04-24T23:25:42.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a97f6c740 10 client.7455 fetch_fsmap learned FSMap version 72 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a97f6c740 1 -- 172.21.15.26:0/147010080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55a1d84c6000 con 0x55a1d847f210 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.704+0000 7f0a6dffb640 1 -- 172.21.15.26:0/147010080 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a80022990 con 0x55a1d847f210 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a6dffb640 1 -- 172.21.15.26:0/147010080 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 72) ==== 3534+0+0 (secure 0 0 0) 0x7f0a80044020 con 0x55a1d847f210 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a97f6c740 10 client.7455 fetch_fsmap finished waiting for FSMap version 72 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a97f6c740 10 client.7455 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a97f6c740 20 client.7455 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x55a1d849f910 0x55a1d851c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a97f6c740 4 client.7455 mds_command: new command op to 6364 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a97f6c740 1 -- 172.21.15.26:0/147010080 --> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] -- command(tid 0: {"prefix": "status"}) -- 0x55a1d83c3790 con 0x55a1d849f910 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a87fff640 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x55a1d849f910 0x55a1d851c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a87fff640 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x55a1d849f910 0x55a1d851c0f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7f931eb3296ba7a server_cookie=f5eb8d9e9f92d9b4 in_seq=0 out_seq=0 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a6dffb640 10 client.7455 ms_handle_connect on v2:172.21.15.29:6833/2057165440 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a6dffb640 1 -- 172.21.15.26:0/147010080 <== mds.2 v2:172.21.15.29:6833/2057165440 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a1d83c3790 con 0x55a1d849f910 2024-04-24T23:25:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a6dffb640 10 client.7455 handle_command_reply: tid=0 2024-04-24T23:25:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.705+0000 7f0a97f6c740 1 client.7455 shutdown 2024-04-24T23:25:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 -- 172.21.15.26:0/147010080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 msgr2=0x55a1d84867a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d84867a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000fb0 tx=0x7f0a80006070 comp rx=0 tx=0).stop 2024-04-24T23:25:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 -- 172.21.15.26:0/147010080 shutdown_connections 2024-04-24T23:25:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1d8480770 0x55a1d8472b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x55a1d849f910 0x55a1d851c0f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1d847fe50 0x55a1d8472640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 --2- 172.21.15.26:0/147010080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1d847f210 0x55a1d84867a0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 -- 172.21.15.26:0/147010080 >> 172.21.15.26:0/147010080 conn(0x55a1d8457b60 msgr2=0x55a1d847ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a6dffb640 0 client.7455 ms_handle_reset on v2:172.21.15.29:6833/2057165440 2024-04-24T23:25:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 -- 172.21.15.26:0/147010080 shutdown_connections 2024-04-24T23:25:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 1 -- 172.21.15.26:0/147010080 wait complete. 2024-04-24T23:25:42.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a97f6c740 20 client.7455 trim_cache size 0 max 16384 2024-04-24T23:25:42.715 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:25:42.715 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:25:42.715 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-04-24T23:25:42.715 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6364, 2024-04-24T23:25:42.715 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:25:42.715 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:25:42.715 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:25:42.716 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 1.6670546509999999, 2024-04-24T23:25:42.716 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 72, 2024-04-24T23:25:42.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 71, 2024-04-24T23:25:42.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 71, 2024-04-24T23:25:42.716 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 19.727930841999999 2024-04-24T23:25:42.716 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:25:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 -- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a7006be60 msgr2=0x7f0a7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a7006be60 0x7f0a7006e320 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0a80001050 tx=0x7f0a80009040 comp rx=0 tx=0).stop 2024-04-24T23:25:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 -- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a9013c3f0 msgr2=0x7f0a90158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a9013c3f0 0x7f0a90158a50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800f5f0 tx=0x7f0a7800fab0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 -- 172.21.15.26:0/3790006581 shutdown_connections 2024-04-24T23:25:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a7006be60 0x7f0a7006e320 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a90155100 0x7f0a9015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a9013c3f0 0x7f0a90158a50 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 --2- 172.21.15.26:0/3790006581 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a7c0051c0 0x7f0a901325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 -- 172.21.15.26:0/3790006581 >> 172.21.15.26:0/3790006581 conn(0x7f0a9006b300 msgr2=0x7f0a90143f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 -- 172.21.15.26:0/3790006581 shutdown_connections 2024-04-24T23:25:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.706+0000 7f0a9517e640 1 -- 172.21.15.26:0/3790006581 wait complete. 2024-04-24T23:25:42.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:25:42.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.818+0000 7f59dc076640 1 Processor -- start 2024-04-24T23:25:42.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.819+0000 7f59dc076640 1 -- start start 2024-04-24T23:25:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.819+0000 7f59dc076640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59d415e570 0x7f59d40d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.819+0000 7f59dc076640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40b3130 0x7f59d40d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.819+0000 7f59dc076640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d40d32f0 con 0x7f59d40d22e0 2024-04-24T23:25:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.819+0000 7f59dc076640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f59d40d1ab0 con 0x7f59d415e570 2024-04-24T23:25:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.819+0000 7f59dc076640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59d40d2ae0 con 0x7f59d40b3130 2024-04-24T23:25:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59d9611640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40b3130 0x7f59d40d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59d9e12640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59d415e570 0x7f59d40d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59da613640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f59d40d22e0 0x7f59d40d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:56592/0 (socket says 172.21.15.26:56592) 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59da613640 1 -- 172.21.15.26:0/1690322815 learned_addr learned my addr 172.21.15.26:0/1690322815 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59d9611640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40b3130 0x7f59d40d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:57146/0 (socket says 172.21.15.26:57146) 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1690322815 >> v2:172.21.15.29:3300/0 conn(0x7f59d415e570 0x7f59d40d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:43074/0 (socket says 172.21.15.26:43074) 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59d8e10640 1 -- 172.21.15.26:0/1690322815 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2874078028 0 0) 0x7f59cc001490 con 0x7f59d40d22e0 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59d9611640 1 -- 172.21.15.26:0/1690322815 >> v2:172.21.15.29:3300/0 conn(0x7f59d415e570 msgr2=0x7f59d40d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59d9611640 1 --2- 172.21.15.26:0/1690322815 >> v2:172.21.15.29:3300/0 conn(0x7f59d415e570 0x7f59d40d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59d9611640 1 -- 172.21.15.26:0/1690322815 >> v1:172.21.15.29:6789/0 conn(0x7f59d40d22e0 legacy=0x7f59d40d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.820+0000 7f59d9611640 1 -- 172.21.15.26:0/1690322815 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d40d0270 con 0x7f59d40b3130 2024-04-24T23:25:42.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.821+0000 7f59d9611640 1 --2- 172.21.15.26:0/1690322815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40b3130 0x7f59d40d46a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f59d00097a0 tx=0x7f59d002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2cd6c462dcbd26b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.821+0000 7f59d8e10640 1 -- 172.21.15.26:0/1690322815 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0037070 con 0x7f59d40b3130 2024-04-24T23:25:42.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.821+0000 7f59d8e10640 1 -- 172.21.15.26:0/1690322815 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d0032440 con 0x7f59d40b3130 2024-04-24T23:25:42.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.821+0000 7f59d8e10640 1 -- 172.21.15.26:0/1690322815 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0032810 con 0x7f59d40b3130 2024-04-24T23:25:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.822+0000 7f59dc076640 1 -- 172.21.15.26:0/1690322815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40b3130 msgr2=0x7f59d40d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.822+0000 7f59dc076640 1 --2- 172.21.15.26:0/1690322815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40b3130 0x7f59d40d46a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f59d00097a0 tx=0x7f59d002a180 comp rx=0 tx=0).stop 2024-04-24T23:25:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.822+0000 7f59dc076640 1 -- 172.21.15.26:0/1690322815 shutdown_connections 2024-04-24T23:25:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.822+0000 7f59dc076640 1 --2- 172.21.15.26:0/1690322815 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40b3130 0x7f59d40d46a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.822+0000 7f59dc076640 1 --2- 172.21.15.26:0/1690322815 >> v2:172.21.15.29:3300/0 conn(0x7f59d415e570 0x7f59d40d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.822+0000 7f59dc076640 1 -- 172.21.15.26:0/1690322815 >> 172.21.15.26:0/1690322815 conn(0x7f59d408dbe0 msgr2=0x7f59d409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.822+0000 7f59dc076640 1 -- 172.21.15.26:0/1690322815 shutdown_connections 2024-04-24T23:25:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.822+0000 7f59dc076640 1 -- 172.21.15.26:0/1690322815 wait complete. 2024-04-24T23:25:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.823+0000 7f59dc076640 1 Processor -- start 2024-04-24T23:25:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.823+0000 7f59dc076640 1 -- start start 2024-04-24T23:25:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59dc076640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d40b3130 0x7f59d41527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59dc076640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40d22e0 0x7f59d40cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59dc076640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d415e570 0x7f59d406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59dc076640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59d40cfa70 con 0x7f59d415e570 2024-04-24T23:25:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59dc076640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59d40d3010 con 0x7f59d40b3130 2024-04-24T23:25:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59dc076640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59d40cfd50 con 0x7f59d40d22e0 2024-04-24T23:25:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59da613640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d415e570 0x7f59d406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59da613640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d415e570 0x7f59d406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55538/0 (socket says 172.21.15.26:55538) 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59da613640 1 -- 172.21.15.26:0/2093527218 learned_addr learned my addr 172.21.15.26:0/2093527218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59d9611640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40d22e0 0x7f59d40cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59d9e12640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d40b3130 0x7f59d41527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59da613640 1 -- 172.21.15.26:0/2093527218 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40d22e0 msgr2=0x7f59d40cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59da613640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40d22e0 0x7f59d40cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59da613640 1 -- 172.21.15.26:0/2093527218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d40b3130 msgr2=0x7f59d41527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59da613640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d40b3130 0x7f59d41527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59da613640 1 -- 172.21.15.26:0/2093527218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d409df70 con 0x7f59d415e570 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59da613640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d415e570 0x7f59d406c7c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00c820 tx=0x7f59cc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59ca7fc640 1 -- 172.21.15.26:0/2093527218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc018070 con 0x7f59d415e570 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.824+0000 7f59ca7fc640 1 -- 172.21.15.26:0/2093527218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59cc00a5e0 con 0x7f59d415e570 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.825+0000 7f59dc076640 1 -- 172.21.15.26:0/2093527218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d41f4500 con 0x7f59d415e570 2024-04-24T23:25:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.825+0000 7f59ca7fc640 1 -- 172.21.15.26:0/2093527218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00a9d0 con 0x7f59d415e570 2024-04-24T23:25:42.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.825+0000 7f59dc076640 1 -- 172.21.15.26:0/2093527218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d41f4ac0 con 0x7f59d415e570 2024-04-24T23:25:42.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.826+0000 7f59ca7fc640 1 -- 172.21.15.26:0/2093527218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59cc01b070 con 0x7f59d415e570 2024-04-24T23:25:42.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.826+0000 7f59ca7fc640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b806be30 0x7f59b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.827+0000 7f59ca7fc640 1 -- 172.21.15.26:0/2093527218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f59cc08a460 con 0x7f59d415e570 2024-04-24T23:25:42.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.827+0000 7f59d9e12640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b806be30 0x7f59b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.827+0000 7f59d9e12640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b806be30 0x7f59b806e2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f59d40cd390 tx=0x7f59c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59ded85740 1 Processor -- start 2024-04-24T23:25:42.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59ded85740 1 -- start start 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59ded85740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cb8e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59ded85740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59ded85740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbbe540 0x56546cb93350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59ded85740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56546caf6f00 con 0x56546cbbec80 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59ded85740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56546ca9f750 con 0x56546cbbe540 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59ded85740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56546caf2ec0 con 0x56546cbdbb60 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59d9611640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59d9611640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55554/0 (socket says 172.21.15.26:55554) 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59d9611640 1 -- 172.21.15.26:0/1724749131 learned_addr learned my addr 172.21.15.26:0/1724749131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.828+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1724749131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cb8e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59da613640 1 --2- 172.21.15.26:0/1724749131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbbe540 0x56546cb93350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59d9611640 1 -- 172.21.15.26:0/1724749131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 msgr2=0x56546cb8e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59d9611640 1 --2- 172.21.15.26:0/1724749131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cb8e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59d9611640 1 -- 172.21.15.26:0/1724749131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbbe540 msgr2=0x56546cb93350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59d9611640 1 --2- 172.21.15.26:0/1724749131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbbe540 0x56546cb93350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59d9611640 1 -- 172.21.15.26:0/1724749131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546c80b2d0 con 0x56546cbbec80 2024-04-24T23:25:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59d9611640 1 --2- 172.21.15.26:0/1724749131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbe000 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f59d00098c0 tx=0x7f59d002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9576060a996fee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59b77fe640 1 -- 172.21.15.26:0/1724749131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0037040 con 0x56546cbbec80 2024-04-24T23:25:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59b77fe640 1 -- 172.21.15.26:0/1724749131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d003e720 con 0x56546cbbec80 2024-04-24T23:25:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.829+0000 7f59b77fe640 1 -- 172.21.15.26:0/1724749131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d003eaf0 con 0x56546cbbec80 2024-04-24T23:25:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 -- 172.21.15.26:0/1724749131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 msgr2=0x56546cbbe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 --2- 172.21.15.26:0/1724749131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbe000 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f59d00098c0 tx=0x7f59d002c000 comp rx=0 tx=0).stop 2024-04-24T23:25:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 -- 172.21.15.26:0/1724749131 shutdown_connections 2024-04-24T23:25:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 --2- 172.21.15.26:0/1724749131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbbe540 0x56546cb93350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 --2- 172.21.15.26:0/1724749131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbe000 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 --2- 172.21.15.26:0/1724749131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cb8e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 -- 172.21.15.26:0/1724749131 >> 172.21.15.26:0/1724749131 conn(0x56546cb90760 msgr2=0x56546cbd9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 -- 172.21.15.26:0/1724749131 shutdown_connections 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 -- 172.21.15.26:0/1724749131 wait complete. 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 -- start start 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.830+0000 7f59ded85740 1 Processor -- start 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59ded85740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59ded85740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cbbabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59ded85740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbc0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59ded85740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56546cb92910 con 0x56546cbbec80 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59ded85740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56546cab2cf0 con 0x56546cbe9fe0 2024-04-24T23:25:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59ded85740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56546cb91d00 con 0x56546cbdbb60 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59ded85740 10 client.0 _ll_register_callbacks cb 0x56546cab26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59d9e12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59d9e12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbba680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55558/0 (socket says 172.21.15.26:55558) 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.831+0000 7f59d9e12640 1 -- 172.21.15.26:0/1219862211 learned_addr learned my addr 172.21.15.26:0/1219862211 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59d9611640 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cbbabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59da613640 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbc0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59d9e12640 1 -- 172.21.15.26:0/1219862211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 msgr2=0x56546cbbabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cbbabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59d9e12640 1 -- 172.21.15.26:0/1219862211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 msgr2=0x56546cbc0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbc0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59d9e12640 1 -- 172.21.15.26:0/1219862211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56546c8d56b0 con 0x56546cbbec80 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59da613640 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbc0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbba680 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f59c4015740 tx=0x7f59c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc857b4c6cb12be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1219862211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4014620 con 0x56546cbbec80 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1219862211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f59c40147c0 con 0x56546cbbec80 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59ded85740 1 -- 172.21.15.26:0/1219862211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d003e720 con 0x56546cbbec80 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1219862211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f59c40149c0 con 0x56546cbbec80 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59ded85740 10 client.7464 fetch_fsmap learned FSMap version 72 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59ded85740 1 -- 172.21.15.26:0/1219862211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x56546cbc0f40 con 0x56546cbbec80 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.832+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1219862211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d003e720 con 0x56546cbbec80 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1219862211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 72) ==== 3534+0+0 (secure 0 0 0) 0x7f59c4035080 con 0x56546cbbec80 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59ded85740 10 client.7464 fetch_fsmap finished waiting for FSMap version 72 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59ded85740 10 client.7464 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59ded85740 20 client.7464 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59ded85740 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x56546cbaaf90 0x56546cbbf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59ded85740 4 client.7464 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59ded85740 1 -- 172.21.15.26:0/1219862211 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56546cb04440 con 0x56546cbaaf90 2024-04-24T23:25:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59d9611640 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x56546cbaaf90 0x56546cbbf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59d9611640 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x56546cbaaf90 0x56546cbbf7d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=190f9cc7bbbe052c server_cookie=b2ef3cada5371762 in_seq=0 out_seq=0 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.833+0000 7f59b4ff9640 10 client.7464 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1219862211 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56546cb04440 con 0x56546cbaaf90 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59b4ff9640 10 client.7464 handle_command_reply: tid=0 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 client.7464 shutdown 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 -- 172.21.15.26:0/1219862211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 msgr2=0x56546cbba680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbba680 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f59c4015740 tx=0x7f59c40046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 -- 172.21.15.26:0/1219862211 shutdown_connections 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x56546cbaaf90 0x56546cbbf7d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbc0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cbbabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 --2- 172.21.15.26:0/1219862211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbba680 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 -- 172.21.15.26:0/1219862211 >> 172.21.15.26:0/1219862211 conn(0x56546cbbe540 msgr2=0x56546cbd9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59b4ff9640 0 client.7464 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.834+0000 7f59ded85740 1 -- 172.21.15.26:0/1219862211 shutdown_connections 2024-04-24T23:25:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.835+0000 7f59ded85740 1 -- 172.21.15.26:0/1219862211 wait complete. 2024-04-24T23:25:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.835+0000 7f59ded85740 20 client.7464 trim_cache size 0 max 16384 2024-04-24T23:25:42.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.853+0000 7f59ded85740 1 Processor -- start 2024-04-24T23:25:42.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.853+0000 7f59ded85740 1 -- start start 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59ded85740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbfcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59ded85740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cbba400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59ded85740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59ded85740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59d41ffc50 con 0x56546cbbec80 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59ded85740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59d4200320 con 0x56546cbe9fe0 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59ded85740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59d41fece0 con 0x56546cbdbb60 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9e12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbfcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9e12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbfcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55562/0 (socket says 172.21.15.26:55562) 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9e12640 1 -- 172.21.15.26:0/1475242563 learned_addr learned my addr 172.21.15.26:0/1475242563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59da613640 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9611640 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cbba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9e12640 1 -- 172.21.15.26:0/1475242563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 msgr2=0x56546cbba400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cbba400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9e12640 1 -- 172.21.15.26:0/1475242563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 msgr2=0x56546cbba940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbba940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9e12640 1 -- 172.21.15.26:0/1475242563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546cb95060 con 0x56546cbbec80 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59da613640 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbfcf40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f59c40116d0 tx=0x7f59c4023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf3880e7305807c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59b6ffd640 1 -- 172.21.15.26:0/1475242563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4014620 con 0x56546cbbec80 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.854+0000 7f59b6ffd640 1 -- 172.21.15.26:0/1475242563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c401a030 con 0x56546cbbec80 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59b6ffd640 1 -- 172.21.15.26:0/1475242563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4004020 con 0x56546cbbec80 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59ded85740 1 -- 172.21.15.26:0/1475242563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 msgr2=0x56546cbfcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59ded85740 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbfcf40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f59c40116d0 tx=0x7f59c4023000 comp rx=0 tx=0).stop 2024-04-24T23:25:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59ded85740 1 -- 172.21.15.26:0/1475242563 shutdown_connections 2024-04-24T23:25:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59ded85740 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbe9fe0 0x56546cbba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59ded85740 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbdbb60 0x56546cbba400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59ded85740 1 --2- 172.21.15.26:0/1475242563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbfcf40 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59ded85740 1 -- 172.21.15.26:0/1475242563 >> 172.21.15.26:0/1475242563 conn(0x56546cbbe540 msgr2=0x56546cbb9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59ded85740 1 -- 172.21.15.26:0/1475242563 shutdown_connections 2024-04-24T23:25:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.855+0000 7f59ded85740 1 -- 172.21.15.26:0/1475242563 wait complete. 2024-04-24T23:25:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.856+0000 7f59ded85740 1 -- start start 2024-04-24T23:25:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.856+0000 7f59ded85740 1 Processor -- start 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59ded85740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59ded85740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbdbb60 0x56546cc52fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59ded85740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbe9fe0 0x56546cbb84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59ded85740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56546cbaae10 con 0x56546cbbec80 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59ded85740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56546cbaa990 con 0x56546cbdbb60 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59ded85740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56546cbab460 con 0x56546cbe9fe0 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59ded85740 10 client.0 _ll_register_callbacks cb 0x7f59c4011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9e12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9e12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55572/0 (socket says 172.21.15.26:55572) 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9e12640 1 -- 172.21.15.26:0/1331935845 learned_addr learned my addr 172.21.15.26:0/1331935845 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9611640 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbdbb60 0x56546cc52fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59da613640 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbe9fe0 0x56546cbb84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9e12640 1 -- 172.21.15.26:0/1331935845 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbe9fe0 msgr2=0x56546cbb84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbe9fe0 0x56546cbb84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9e12640 1 -- 172.21.15.26:0/1331935845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbdbb60 msgr2=0x56546cc52fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbdbb60 0x56546cc52fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9e12640 1 -- 172.21.15.26:0/1331935845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56546cbcf970 con 0x56546cbbec80 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59d9e12640 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbaa10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f59c4014b80 tx=0x7f59c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679cfabf47b1eabc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1331935845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4004230 con 0x56546cbbec80 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1331935845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f59c40043d0 con 0x56546cbbec80 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59ded85740 1 -- 172.21.15.26:0/1331935845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59c401a030 con 0x56546cbbec80 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.857+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1331935845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f59c404c050 con 0x56546cbbec80 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59ded85740 10 client.7470 fetch_fsmap learned FSMap version 72 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59ded85740 1 -- 172.21.15.26:0/1331935845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x56546cbcf650 con 0x56546cbbec80 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1331935845 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c401a030 con 0x56546cbbec80 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1331935845 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 72) ==== 3534+0+0 (secure 0 0 0) 0x7f59c404f020 con 0x56546cbbec80 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59ded85740 10 client.7470 fetch_fsmap finished waiting for FSMap version 72 2024-04-24T23:25:42.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59ded85740 10 client.7470 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59ded85740 20 client.7470 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59ded85740 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x56546cbd1a50 0x56546cbd1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59ded85740 4 client.7470 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59ded85740 1 -- 172.21.15.26:0/1331935845 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "status"}) -- 0x56546cafe790 con 0x56546cbd1a50 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59d9611640 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x56546cbd1a50 0x56546cbd1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59b4ff9640 10 client.7470 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59d9611640 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x56546cbd1a50 0x56546cbd1e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=555e70a680f321cb server_cookie=63a865a1fab0f54c in_seq=0 out_seq=0 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59b4ff9640 1 -- 172.21.15.26:0/1331935845 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56546cafe790 con 0x56546cbd1a50 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.858+0000 7f59b4ff9640 10 client.7470 handle_command_reply: tid=0 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 client.7470 shutdown 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 -- 172.21.15.26:0/1331935845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 msgr2=0x56546cbbaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbaa10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f59c4014b80 tx=0x7f59c4016000 comp rx=0 tx=0).stop 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 -- 172.21.15.26:0/1331935845 shutdown_connections 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x56546cbd1a50 0x56546cbd1e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56546cbe9fe0 0x56546cbb84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59b4ff9640 0 client.7470 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56546cbdbb60 0x56546cc52fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 --2- 172.21.15.26:0/1331935845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56546cbbec80 0x56546cbbaa10 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 -- 172.21.15.26:0/1331935845 >> 172.21.15.26:0/1331935845 conn(0x56546cbbe540 msgr2=0x56546cbe7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 -- 172.21.15.26:0/1331935845 shutdown_connections 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 1 -- 172.21.15.26:0/1331935845 wait complete. 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59ded85740 20 client.7470 trim_cache size 0 max 16384 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7224, 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:25:42.866 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:25:42.867 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:25:42.867 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 2.8250776059999998, 2024-04-24T23:25:42.867 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 72, 2024-04-24T23:25:42.867 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 71, 2024-04-24T23:25:42.867 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 71, 2024-04-24T23:25:42.867 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 19.896208686000001 2024-04-24T23:25:42.867 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 -- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b806be30 msgr2=0x7f59b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b806be30 0x7f59b806e2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f59d40cd390 tx=0x7f59c4008040 comp rx=0 tx=0).stop 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 -- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d415e570 msgr2=0x7f59d406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d415e570 0x7f59d406c7c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00c820 tx=0x7f59cc00cce0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 -- 172.21.15.26:0/2093527218 shutdown_connections 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b806be30 0x7f59b806e2f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d415e570 0x7f59d406c7c0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d40d22e0 0x7f59d40cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 --2- 172.21.15.26:0/2093527218 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d40b3130 0x7f59d41527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.859+0000 7f59dc076640 1 -- 172.21.15.26:0/2093527218 >> 172.21.15.26:0/2093527218 conn(0x7f59d408dbe0 msgr2=0x7f59d415d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.860+0000 7f59dc076640 1 -- 172.21.15.26:0/2093527218 shutdown_connections 2024-04-24T23:25:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.860+0000 7f59dc076640 1 -- 172.21.15.26:0/2093527218 wait complete. 2024-04-24T23:25:42.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:25:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.972+0000 7f7ff773b640 1 Processor -- start 2024-04-24T23:25:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.972+0000 7f7ff773b640 1 -- start start 2024-04-24T23:25:42.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.973+0000 7f7ff773b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ff015e570 0x7f7ff00d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.973+0000 7f7ff773b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00b3130 0x7f7ff00d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.973+0000 7f7ff773b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff00d32f0 con 0x7f7ff00d22e0 2024-04-24T23:25:42.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.973+0000 7f7ff773b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff00d1ab0 con 0x7f7ff015e570 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.973+0000 7f7ff773b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00d2ae0 con 0x7f7ff00b3130 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.973+0000 7f7ff54d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ff015e570 0x7f7ff00d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.973+0000 7f7ff54d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ff015e570 0x7f7ff00d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55582/0 (socket says 172.21.15.26:55582) 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.973+0000 7f7ff54d7640 1 -- 172.21.15.26:0/2589443844 learned_addr learned my addr 172.21.15.26:0/2589443844 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.973+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/2589443844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00b3130 0x7f7ff00d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.974+0000 7f7ff54d7640 1 -- 172.21.15.26:0/2589443844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00b3130 msgr2=0x7f7ff00d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.974+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/2589443844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00b3130 0x7f7ff00d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.974+0000 7f7ff54d7640 1 -- 172.21.15.26:0/2589443844 >> v1:172.21.15.29:6789/0 conn(0x7f7ff00d22e0 legacy=0x7f7ff00d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.974+0000 7f7ff54d7640 1 -- 172.21.15.26:0/2589443844 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff00d0270 con 0x7f7ff015e570 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.974+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/2589443844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff015e570 0x7f7ff00d4160 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009870 tx=0x7f7fe002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1bae056a10e26a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7fe7fff640 1 -- 172.21.15.26:0/2589443844 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0037070 con 0x7f7ff015e570 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7fe7fff640 1 -- 172.21.15.26:0/2589443844 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe0032440 con 0x7f7ff015e570 2024-04-24T23:25:42.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7fe7fff640 1 -- 172.21.15.26:0/2589443844 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0032810 con 0x7f7ff015e570 2024-04-24T23:25:42.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7ff773b640 1 -- 172.21.15.26:0/2589443844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff015e570 msgr2=0x7f7ff00d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7ff773b640 1 --2- 172.21.15.26:0/2589443844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff015e570 0x7f7ff00d4160 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009870 tx=0x7f7fe002a270 comp rx=0 tx=0).stop 2024-04-24T23:25:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7ff773b640 1 -- 172.21.15.26:0/2589443844 shutdown_connections 2024-04-24T23:25:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7ff773b640 1 --2- 172.21.15.26:0/2589443844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00b3130 0x7f7ff00d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7ff773b640 1 --2- 172.21.15.26:0/2589443844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff015e570 0x7f7ff00d4160 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7ff773b640 1 -- 172.21.15.26:0/2589443844 >> 172.21.15.26:0/2589443844 conn(0x7f7ff008dbe0 msgr2=0x7f7ff009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7ff773b640 1 -- 172.21.15.26:0/2589443844 shutdown_connections 2024-04-24T23:25:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.975+0000 7f7ff773b640 1 -- 172.21.15.26:0/2589443844 wait complete. 2024-04-24T23:25:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff773b640 1 Processor -- start 2024-04-24T23:25:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff773b640 1 -- start start 2024-04-24T23:25:42.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff773b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff00b3130 0x7f7ff01505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff773b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00d22e0 0x7f7ff015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff773b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ff015e570 0x7f7ff0151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff773b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff00cfa70 con 0x7f7ff00b3130 2024-04-24T23:25:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff773b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff00d3010 con 0x7f7ff015e570 2024-04-24T23:25:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff773b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00cfd50 con 0x7f7ff00d22e0 2024-04-24T23:25:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff54d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff00b3130 0x7f7ff01505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff54d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff00b3130 0x7f7ff01505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55594/0 (socket says 172.21.15.26:55594) 2024-04-24T23:25:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.977+0000 7f7ff54d7640 1 -- 172.21.15.26:0/1637087139 learned_addr learned my addr 172.21.15.26:0/1637087139 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00d22e0 0x7f7ff015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ff015e570 0x7f7ff0151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff54d7640 1 -- 172.21.15.26:0/1637087139 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00d22e0 msgr2=0x7f7ff015fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00d22e0 0x7f7ff015fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff54d7640 1 -- 172.21.15.26:0/1637087139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ff015e570 msgr2=0x7f7ff0151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ff015e570 0x7f7ff0151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff54d7640 1 -- 172.21.15.26:0/1637087139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff015eaf0 con 0x7f7ff00b3130 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ff015e570 0x7f7ff0151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff00b3130 0x7f7ff01505a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0000c00 tx=0x7f7fe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7fe5ffb640 1 -- 172.21.15.26:0/1637087139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0041020 con 0x7f7ff00b3130 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7fe5ffb640 1 -- 172.21.15.26:0/1637087139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe00306c0 con 0x7f7ff00b3130 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff773b640 1 -- 172.21.15.26:0/1637087139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff015edd0 con 0x7f7ff00b3130 2024-04-24T23:25:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7fe5ffb640 1 -- 172.21.15.26:0/1637087139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0037040 con 0x7f7ff00b3130 2024-04-24T23:25:42.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.978+0000 7f7ff773b640 1 -- 172.21.15.26:0/1637087139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff015f390 con 0x7f7ff00b3130 2024-04-24T23:25:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.980+0000 7f7fe5ffb640 1 -- 172.21.15.26:0/1637087139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fe0044020 con 0x7f7ff00b3130 2024-04-24T23:25:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.981+0000 7f7fe5ffb640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fd406be30 0x7f7fd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.981+0000 7f7fe5ffb640 1 -- 172.21.15.26:0/1637087139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f7fe00ab650 con 0x7f7ff00b3130 2024-04-24T23:25:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.981+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fd406be30 0x7f7fd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.981+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fd406be30 0x7f7fd406e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7ff015cb90 tx=0x7f7fec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.982+0000 7f7ffa44a740 1 Processor -- start 2024-04-24T23:25:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.982+0000 7f7ffa44a740 1 -- start start 2024-04-24T23:25:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.982+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e91b60 0x55ec45e444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.982+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e74c80 0x55ec45e74000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.982+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74540 0x55ec45e49350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ec45dacf00 con 0x55ec45e74c80 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ec45d55750 con 0x55ec45e91b60 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ec45da8ec0 con 0x55ec45e74540 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff4cd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e74c80 0x55ec45e74000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff4cd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e74c80 0x55ec45e74000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55600/0 (socket says 172.21.15.26:55600) 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff54d7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e91b60 0x55ec45e444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff4cd6640 1 -- 172.21.15.26:0/3627097910 learned_addr learned my addr 172.21.15.26:0/3627097910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/3627097910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74540 0x55ec45e49350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff4cd6640 1 -- 172.21.15.26:0/3627097910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74540 msgr2=0x55ec45e49350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/3627097910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74540 0x55ec45e49350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff4cd6640 1 -- 172.21.15.26:0/3627097910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e91b60 msgr2=0x55ec45e444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/3627097910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e91b60 0x55ec45e444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff4cd6640 1 -- 172.21.15.26:0/3627097910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec45ac12d0 con 0x55ec45e74c80 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.983+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/3627097910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e74c80 0x55ec45e74000 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7fec010b30 tx=0x7f7fec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4196ef5935011a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7fdaffd640 1 -- 172.21.15.26:0/3627097910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec01c050 con 0x55ec45e74c80 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7fdaffd640 1 -- 172.21.15.26:0/3627097910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fec026070 con 0x55ec45e74c80 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7fdaffd640 1 -- 172.21.15.26:0/3627097910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec021440 con 0x55ec45e74c80 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3627097910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e74c80 msgr2=0x55ec45e74000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3627097910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e74c80 0x55ec45e74000 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7fec010b30 tx=0x7f7fec013000 comp rx=0 tx=0).stop 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3627097910 shutdown_connections 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3627097910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74540 0x55ec45e49350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3627097910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e74c80 0x55ec45e74000 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3627097910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e91b60 0x55ec45e444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3627097910 >> 172.21.15.26:0/3627097910 conn(0x55ec45e46760 msgr2=0x55ec45e8f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3627097910 shutdown_connections 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.984+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3627097910 wait complete. 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.985+0000 7f7ffa44a740 1 -- start start 2024-04-24T23:25:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.985+0000 7f7ffa44a740 1 Processor -- start 2024-04-24T23:25:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e74c80 0x55ec45e8bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e6f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e6f930 0x55ec45e43250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ec45e48910 con 0x55ec45e91b60 2024-04-24T23:25:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ec45d68cf0 con 0x55ec45e74c80 2024-04-24T23:25:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ec45e47d00 con 0x55ec45e6f930 2024-04-24T23:25:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ffa44a740 10 client.0 _ll_register_callbacks cb 0x55ec45d686e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ff4cd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e6f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ff4cd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e6f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55610/0 (socket says 172.21.15.26:55610) 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ff4cd6640 1 -- 172.21.15.26:0/3975459879 learned_addr learned my addr 172.21.15.26:0/3975459879 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e6f930 0x55ec45e43250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.986+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e74c80 0x55ec45e8bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ff4cd6640 1 -- 172.21.15.26:0/3975459879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e6f930 msgr2=0x55ec45e43250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e6f930 0x55ec45e43250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ff4cd6640 1 -- 172.21.15.26:0/3975459879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e74c80 msgr2=0x55ec45e8bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e74c80 0x55ec45e8bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ff4cd6640 1 -- 172.21.15.26:0/3975459879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec45b8b6b0 con 0x55ec45e91b60 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e6f350 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7fec007fd0 tx=0x7f7fec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80a20c4c1b5c673 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7fcffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7fcffff640 1 -- 172.21.15.26:0/3975459879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec006240 con 0x55ec45e91b60 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7fcffff640 1 -- 172.21.15.26:0/3975459879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f7fec034b10 con 0x55ec45e91b60 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3975459879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fec026070 con 0x55ec45e91b60 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7fcffff640 1 -- 172.21.15.26:0/3975459879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f7fec01c020 con 0x55ec45e91b60 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ffa44a740 10 client.7482 fetch_fsmap learned FSMap version 72 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3975459879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55ec45e90c40 con 0x55ec45e91b60 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7fcffff640 1 -- 172.21.15.26:0/3975459879 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fec026070 con 0x55ec45e91b60 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7fcffff640 1 -- 172.21.15.26:0/3975459879 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 72) ==== 3534+0+0 (secure 0 0 0) 0x7f7fec0063e0 con 0x55ec45e91b60 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ffa44a740 10 client.7482 fetch_fsmap finished waiting for FSMap version 72 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ffa44a740 10 client.7482 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.987+0000 7f7ffa44a740 20 client.7482 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.988+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x55ec45e73320 0x55ec45e75dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.988+0000 7f7ffa44a740 4 client.7482 mds_command: new command op to 7227 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.988+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3975459879 --> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec45dba440 con 0x55ec45e73320 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.988+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x55ec45e73320 0x55ec45e75dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.988+0000 7f7fcffff640 10 client.7482 ms_handle_connect on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.988+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x55ec45e73320 0x55ec45e75dd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf20b913fe2d8d05 server_cookie=389e1eabc7679f18 in_seq=0 out_seq=0 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7fcffff640 1 -- 172.21.15.26:0/3975459879 <== mds.0 v2:172.21.15.26:6835/2537133693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec45dba440 con 0x55ec45e73320 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7fcffff640 10 client.7482 handle_command_reply: tid=0 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 client.7482 shutdown 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3975459879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 msgr2=0x55ec45e6f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e6f350 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7fec007fd0 tx=0x7f7fec004730 comp rx=0 tx=0).stop 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3975459879 shutdown_connections 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x55ec45e73320 0x55ec45e75dd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e6f930 0x55ec45e43250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e6f350 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7fcffff640 0 client.7482 ms_handle_reset on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:42.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/3975459879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e74c80 0x55ec45e8bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:42.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3975459879 >> 172.21.15.26:0/3975459879 conn(0x55ec45e74540 msgr2=0x55ec45e8f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:42.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3975459879 shutdown_connections 2024-04-24T23:25:42.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 1 -- 172.21.15.26:0/3975459879 wait complete. 2024-04-24T23:25:42.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:42.989+0000 7f7ffa44a740 20 client.7482 trim_cache size 0 max 16384 2024-04-24T23:25:43.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.008+0000 7f7ffa44a740 1 Processor -- start 2024-04-24T23:25:43.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.008+0000 7f7ffa44a740 1 -- start start 2024-04-24T23:25:43.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.008+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 0x55ec45e9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.008+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 0x55ec45e90ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.008+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e91430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.008+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff01ffb30 con 0x55ec45e91b60 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.008+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff0200200 con 0x55ec45e6f930 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.008+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00b78f0 con 0x55ec45e74c80 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff5cd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e91430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff5cd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e91430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55616/0 (socket says 172.21.15.26:55616) 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff5cd8640 1 -- 172.21.15.26:0/1027370882 learned_addr learned my addr 172.21.15.26:0/1027370882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 0x55ec45e90ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 0x55ec45e9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff5cd8640 1 -- 172.21.15.26:0/1027370882 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 msgr2=0x55ec45e90ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 0x55ec45e90ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff5cd8640 1 -- 172.21.15.26:0/1027370882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 msgr2=0x55ec45e9eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 0x55ec45e9eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff5cd8640 1 -- 172.21.15.26:0/1027370882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec45e861d0 con 0x55ec45e91b60 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 0x55ec45e9eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e91430 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001c30 tx=0x7f7fe80020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae111ebb13ec5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7fda7fc640 1 -- 172.21.15.26:0/1027370882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8006e20 con 0x55ec45e91b60 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7fda7fc640 1 -- 172.21.15.26:0/1027370882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe801b020 con 0x55ec45e91b60 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.009+0000 7f7fda7fc640 1 -- 172.21.15.26:0/1027370882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80149c0 con 0x55ec45e91b60 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1027370882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 msgr2=0x55ec45e91430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e91430 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001c30 tx=0x7f7fe80020f0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1027370882 shutdown_connections 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e91430 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 0x55ec45e90ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1027370882 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 0x55ec45e9eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1027370882 >> 172.21.15.26:0/1027370882 conn(0x55ec45e74540 msgr2=0x55ec45e3f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1027370882 shutdown_connections 2024-04-24T23:25:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1027370882 wait complete. 2024-04-24T23:25:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.010+0000 7f7ffa44a740 1 -- start start 2024-04-24T23:25:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.011+0000 7f7ffa44a740 1 Processor -- start 2024-04-24T23:25:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.011+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 0x55ec45eeee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.011+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 0x55ec45e9e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.011+0000 7f7ffa44a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e9e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.011+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ec45e731a0 con 0x55ec45e91b60 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.011+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ec45e76b40 con 0x55ec45e6f930 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.011+0000 7f7ffa44a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ec45e737f0 con 0x55ec45e74c80 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.011+0000 7f7ffa44a740 10 client.0 _ll_register_callbacks cb 0x7f7fec0109f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff5cd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e9e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff5cd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e9e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55622/0 (socket says 172.21.15.26:55622) 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff5cd8640 1 -- 172.21.15.26:0/1565367302 learned_addr learned my addr 172.21.15.26:0/1565367302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff4cd6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 0x55ec45e9e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 0x55ec45eeee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff5cd8640 1 -- 172.21.15.26:0/1565367302 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 msgr2=0x55ec45e9e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 0x55ec45e9e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff5cd8640 1 -- 172.21.15.26:0/1565367302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 msgr2=0x55ec45eeee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 0x55ec45eeee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff5cd8640 1 -- 172.21.15.26:0/1565367302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec45e6c2f0 con 0x55ec45e91b60 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff4cd6640 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 0x55ec45e9e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ff5cd8640 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e9e9c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80012e0 tx=0x7f7fe8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b556efd1eab8f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7fcffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7fcffff640 1 -- 172.21.15.26:0/1565367302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe801a070 con 0x55ec45e91b60 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7fcffff640 1 -- 172.21.15.26:0/1565367302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f7fe8003d80 con 0x55ec45e91b60 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1565367302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe801b020 con 0x55ec45e91b60 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7fcffff640 1 -- 172.21.15.26:0/1565367302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f7fe8031050 con 0x55ec45e91b60 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ffa44a740 10 client.7488 fetch_fsmap learned FSMap version 72 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.012+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1565367302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55ec45eb1a70 con 0x55ec45e91b60 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7fcffff640 1 -- 172.21.15.26:0/1565367302 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe801b020 con 0x55ec45e91b60 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7fcffff640 1 -- 172.21.15.26:0/1565367302 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 72) ==== 3534+0+0 (secure 0 0 0) 0x7f7fe8034020 con 0x55ec45e91b60 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7ffa44a740 10 client.7488 fetch_fsmap finished waiting for FSMap version 72 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7ffa44a740 10 client.7488 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:25:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7ffa44a740 20 client.7488 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x55ec45eeff40 0x55ec45e84990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7ffa44a740 4 client.7488 mds_command: new command op to 7227 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1565367302 --> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] -- command(tid 0: {"prefix": "status"}) -- 0x55ec45db4790 con 0x55ec45eeff40 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x55ec45eeff40 0x55ec45e84990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7fcffff640 10 client.7488 ms_handle_connect on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7ff54d7640 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x55ec45eeff40 0x55ec45e84990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f986f700318e536 server_cookie=b317122612f775e0 in_seq=0 out_seq=0 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7fcffff640 1 -- 172.21.15.26:0/1565367302 <== mds.0 v2:172.21.15.26:6835/2537133693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ec45db4790 con 0x55ec45eeff40 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7fcffff640 10 client.7488 handle_command_reply: tid=0 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.013+0000 7f7ffa44a740 1 client.7488 shutdown 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1565367302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 msgr2=0x55ec45e9e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e9e9c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80012e0 tx=0x7f7fe8001250 comp rx=0 tx=0).stop 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1565367302 shutdown_connections 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x55ec45eeff40 0x55ec45e84990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec45e91b60 0x55ec45e9e9c0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7fcffff640 0 client.7488 ms_handle_reset on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ec45e74c80 0x55ec45e9e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 --2- 172.21.15.26:0/1565367302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ec45e6f930 0x55ec45eeee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1565367302 >> 172.21.15.26:0/1565367302 conn(0x55ec45e74540 msgr2=0x55ec45e3f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1565367302 shutdown_connections 2024-04-24T23:25:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 1 -- 172.21.15.26:0/1565367302 wait complete. 2024-04-24T23:25:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ffa44a740 20 client.7488 trim_cache size 0 max 16384 2024-04-24T23:25:43.023 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:25:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7227, 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 15.023618106000001, 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 72, 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 71, 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 71, 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 20.050970002 2024-04-24T23:25:43.024 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:25:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 -- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fd406be30 msgr2=0x7f7fd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fd406be30 0x7f7fd406e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7ff015cb90 tx=0x7f7fec008040 comp rx=0 tx=0).stop 2024-04-24T23:25:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 -- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff00b3130 msgr2=0x7f7ff01505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff00b3130 0x7f7ff01505a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0000c00 tx=0x7f7fe00046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 -- 172.21.15.26:0/1637087139 shutdown_connections 2024-04-24T23:25:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fd406be30 0x7f7fd406e2f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ff015e570 0x7f7ff0151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7ff00d22e0 0x7f7ff015fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 --2- 172.21.15.26:0/1637087139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ff00b3130 0x7f7ff01505a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.014+0000 7f7ff773b640 1 -- 172.21.15.26:0/1637087139 >> 172.21.15.26:0/1637087139 conn(0x7f7ff008dbe0 msgr2=0x7f7ff00b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.015+0000 7f7ff773b640 1 -- 172.21.15.26:0/1637087139 shutdown_connections 2024-04-24T23:25:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.015+0000 7f7ff773b640 1 -- 172.21.15.26:0/1637087139 wait complete. 2024-04-24T23:25:43.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:43.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.128+0000 7f0357453640 1 Processor -- start 2024-04-24T23:25:43.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.128+0000 7f0357453640 1 -- start start 2024-04-24T23:25:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.129+0000 7f0357453640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.129+0000 7f0357453640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f03500b3130 0x7f03500d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.129+0000 7f0357453640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03500d32f0 con 0x7f03500d22e0 2024-04-24T23:25:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.129+0000 7f0357453640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f03500d1ab0 con 0x7f03500b3130 2024-04-24T23:25:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.129+0000 7f0357453640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f03500d2ae0 con 0x7f035015e570 2024-04-24T23:25:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03551ef640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03549ee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f03500b3130 0x7f03500d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03551ef640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:57252/0 (socket says 172.21.15.26:57252) 2024-04-24T23:25:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03549ee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f03500b3130 0x7f03500d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:43160/0 (socket says 172.21.15.26:43160) 2024-04-24T23:25:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03551ef640 1 -- 172.21.15.26:0/3606524625 learned_addr learned my addr 172.21.15.26:0/3606524625 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03559f0640 1 --1- 172.21.15.26:0/3606524625 >> v1:172.21.15.29:6789/0 conn(0x7f03500d22e0 0x7f03500d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:56612/0 (socket says 172.21.15.26:56612) 2024-04-24T23:25:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03549ee640 1 -- 172.21.15.26:0/3606524625 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 msgr2=0x7f03500d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03549ee640 1 --2- 172.21.15.26:0/3606524625 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03549ee640 1 -- 172.21.15.26:0/3606524625 >> v1:172.21.15.29:6789/0 conn(0x7f03500d22e0 legacy=0x7f03500d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03549ee640 1 -- 172.21.15.26:0/3606524625 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03500d0270 con 0x7f03500b3130 2024-04-24T23:25:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.130+0000 7f03551ef640 1 --2- 172.21.15.26:0/3606524625 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:43.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f03549ee640 1 --2- 172.21.15.26:0/3606524625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03500b3130 0x7f03500d46a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f03400097a0 tx=0x7f034002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba809a6d383578a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:43.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f0347fff640 1 -- 172.21.15.26:0/3606524625 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340037070 con 0x7f03500b3130 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f0347fff640 1 -- 172.21.15.26:0/3606524625 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0340032440 con 0x7f03500b3130 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f0347fff640 1 -- 172.21.15.26:0/3606524625 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340032810 con 0x7f03500b3130 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f0357453640 1 -- 172.21.15.26:0/3606524625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03500b3130 msgr2=0x7f03500d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f0357453640 1 --2- 172.21.15.26:0/3606524625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03500b3130 0x7f03500d46a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f03400097a0 tx=0x7f034002a180 comp rx=0 tx=0).stop 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f0357453640 1 -- 172.21.15.26:0/3606524625 shutdown_connections 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f0357453640 1 --2- 172.21.15.26:0/3606524625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03500b3130 0x7f03500d46a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f0357453640 1 --2- 172.21.15.26:0/3606524625 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.131+0000 7f0357453640 1 -- 172.21.15.26:0/3606524625 >> 172.21.15.26:0/3606524625 conn(0x7f035008dbe0 msgr2=0x7f035009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.132+0000 7f0357453640 1 -- 172.21.15.26:0/3606524625 shutdown_connections 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.132+0000 7f0357453640 1 -- 172.21.15.26:0/3606524625 wait complete. 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.133+0000 7f0357453640 1 Processor -- start 2024-04-24T23:25:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.133+0000 7f0357453640 1 -- start start 2024-04-24T23:25:43.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.133+0000 7f0357453640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03500b3130 0x7f0350150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.133+0000 7f0357453640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03500d22e0 0x7f035015fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.133+0000 7f0357453640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.133+0000 7f0357453640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f03500cfa70 con 0x7f03500d22e0 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.133+0000 7f0357453640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f03500d3010 con 0x7f03500b3130 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.133+0000 7f0357453640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f03500cfd50 con 0x7f035015e570 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03549ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03500d22e0 0x7f035015fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03549ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03500d22e0 0x7f035015fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55628/0 (socket says 172.21.15.26:55628) 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03549ee640 1 -- 172.21.15.26:0/3744813037 learned_addr learned my addr 172.21.15.26:0/3744813037 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03559f0640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03551ef640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03500b3130 0x7f0350150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03549ee640 1 -- 172.21.15.26:0/3744813037 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 msgr2=0x7f03500b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03549ee640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03549ee640 1 -- 172.21.15.26:0/3744813037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03500b3130 msgr2=0x7f0350150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03549ee640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03500b3130 0x7f0350150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03549ee640 1 -- 172.21.15.26:0/3744813037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035015eba0 con 0x7f03500d22e0 2024-04-24T23:25:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f03549ee640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03500d22e0 0x7f035015fc40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f03400098c0 tx=0x7f03400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f0345ffb640 1 -- 172.21.15.26:0/3744813037 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340037040 con 0x7f03500d22e0 2024-04-24T23:25:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f0345ffb640 1 -- 172.21.15.26:0/3744813037 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0340030500 con 0x7f03500d22e0 2024-04-24T23:25:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035015ee80 con 0x7f03500d22e0 2024-04-24T23:25:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.134+0000 7f0345ffb640 1 -- 172.21.15.26:0/3744813037 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400308d0 con 0x7f03500d22e0 2024-04-24T23:25:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.135+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035015f440 con 0x7f03500d22e0 2024-04-24T23:25:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.135+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03500694c0 con 0x7f03500d22e0 2024-04-24T23:25:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.136+0000 7f0345ffb640 1 -- 172.21.15.26:0/3744813037 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0340044020 con 0x7f03500d22e0 2024-04-24T23:25:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.136+0000 7f0345ffb640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f033406be30 0x7f033406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.136+0000 7f0345ffb640 1 -- 172.21.15.26:0/3744813037 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f03400ab0f0 con 0x7f03500d22e0 2024-04-24T23:25:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.137+0000 7f03551ef640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f033406be30 0x7f033406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.137+0000 7f03551ef640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f033406be30 0x7f033406e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f034c009ce0 tx=0x7f034c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.141+0000 7f0345ffb640 1 -- 172.21.15.26:0/3744813037 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0340079a60 con 0x7f03500d22e0 2024-04-24T23:25:43.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.361+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03500ceea0 con 0x7f03500d22e0 2024-04-24T23:25:43.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.362+0000 7f0345ffb640 1 -- 172.21.15.26:0/3744813037 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5221 (secure 0 0 0) 0x7f0340079360 con 0x7f03500d22e0 2024-04-24T23:25:43.363 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:43.363 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:25:42.042533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7227,"mds_1":7224,"mds_2":6364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6364":{"gid":6364,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7224":{"gid":7224,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[6364,7224,7227]},"id":5}]} 2024-04-24T23:25:43.364 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 72 2024-04-24T23:25:43.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f033406be30 msgr2=0x7f033406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f033406be30 0x7f033406e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f034c009ce0 tx=0x7f034c00b040 comp rx=0 tx=0).stop 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03500d22e0 msgr2=0x7f035015fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03500d22e0 0x7f035015fc40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f03400098c0 tx=0x7f03400046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 shutdown_connections 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f033406be30 0x7f033406e2f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f035015e570 0x7f03500b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03500d22e0 0x7f035015fc40 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 --2- 172.21.15.26:0/3744813037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03500b3130 0x7f0350150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 >> 172.21.15.26:0/3744813037 conn(0x7f035008dbe0 msgr2=0x7f035006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 shutdown_connections 2024-04-24T23:25:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:43.364+0000 7f0357453640 1 -- 172.21.15.26:0/3744813037 wait complete. 2024-04-24T23:25:43.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:43.376 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:25:43.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:43.399 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:25:43.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:43.460 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-04-24T23:25:43.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:43.516 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-04-24T23:25:43.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:25:43.571 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-04-24T23:25:43.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:25:43.785 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:25:43.786 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:25:43.786 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:25:45.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:25:45.917 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:25:45.917 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:25:45.918 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:25:47.919 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:25:48.050 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:25:48.051 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:25:48.051 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:25:50.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:25:50.185 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:25:50.185 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:25:50.185 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:25:52.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:25:52.316 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:25:52.317 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:25:52.317 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:25:54.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:25:54.449 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-04-24T23:25:54.450 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:25:54.450 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:25:54.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:25:54.568 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:25:54.568 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:25:54.569 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:25:56.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:25:56.699 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:25:56.699 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:25:56.699 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:25:58.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:25:58.830 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-04-24T23:25:58.831 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:25:58.831 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:25:58.831 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:25:58.961 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:25:58.962 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:25:58.962 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:25:58.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-04-24T23:25:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.069+0000 7effa74da640 1 Processor -- start 2024-04-24T23:25:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.069+0000 7effa74da640 1 -- start start 2024-04-24T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.070+0000 7effa74da640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa015e750 0x7effa00cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.070+0000 7effa74da640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effa00d2c30 0x7effa00cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.070+0000 7effa74da640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa00d2120 con 0x7effa015e370 2024-04-24T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.070+0000 7effa74da640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7effa00d1510 con 0x7effa00d2c30 2024-04-24T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.070+0000 7effa74da640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7effa00d1ed0 con 0x7effa015e750 2024-04-24T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.070+0000 7effa5a77640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effa00d2c30 0x7effa00cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.070+0000 7effa5276640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7effa015e370 0x7effa00d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56774/0 (socket says 172.21.15.26:56774) 2024-04-24T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.070+0000 7effa5276640 1 -- 172.21.15.26:0/2809947343 learned_addr learned my addr 172.21.15.26:0/2809947343 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.071+0000 7effa4a75640 1 --2- 172.21.15.26:0/2809947343 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa015e750 0x7effa00cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.071+0000 7effa5a77640 1 -- 172.21.15.26:0/2809947343 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa015e750 msgr2=0x7effa00cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.071+0000 7effa5a77640 1 --2- 172.21.15.26:0/2809947343 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa015e750 0x7effa00cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.071+0000 7effa5a77640 1 -- 172.21.15.26:0/2809947343 >> v1:172.21.15.26:6789/0 conn(0x7effa015e370 legacy=0x7effa00d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.071+0000 7eff9ffff640 1 -- 172.21.15.26:0/2809947343 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203433970 0 0) 0x7effa00d2120 con 0x7effa015e370 2024-04-24T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.071+0000 7effa5a77640 1 -- 172.21.15.26:0/2809947343 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa00b3740 con 0x7effa00d2c30 2024-04-24T23:25:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.071+0000 7effa5a77640 1 --2- 172.21.15.26:0/2809947343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 0x7effa00cf1d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7effa00d2050 tx=0x7eff9002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d170de96cc8b03f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7eff9ffff640 1 -- 172.21.15.26:0/2809947343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90037070 con 0x7effa00d2c30 2024-04-24T23:25:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7eff9ffff640 1 -- 172.21.15.26:0/2809947343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff90032440 con 0x7effa00d2c30 2024-04-24T23:25:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7eff9ffff640 1 -- 172.21.15.26:0/2809947343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90032810 con 0x7effa00d2c30 2024-04-24T23:25:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7effa74da640 1 -- 172.21.15.26:0/2809947343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 msgr2=0x7effa00cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7effa74da640 1 --2- 172.21.15.26:0/2809947343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 0x7effa00cf1d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7effa00d2050 tx=0x7eff9002a180 comp rx=0 tx=0).stop 2024-04-24T23:25:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7effa74da640 1 -- 172.21.15.26:0/2809947343 shutdown_connections 2024-04-24T23:25:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7effa74da640 1 --2- 172.21.15.26:0/2809947343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 0x7effa00cf1d0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7effa74da640 1 --2- 172.21.15.26:0/2809947343 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa015e750 0x7effa00cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7effa74da640 1 -- 172.21.15.26:0/2809947343 >> 172.21.15.26:0/2809947343 conn(0x7effa00d40f0 msgr2=0x7effa00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7effa74da640 1 -- 172.21.15.26:0/2809947343 shutdown_connections 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.072+0000 7effa74da640 1 -- 172.21.15.26:0/2809947343 wait complete. 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.073+0000 7effa74da640 1 Processor -- start 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa74da640 1 -- start start 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa74da640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 0x7effa01f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa74da640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa015e370 0x7effa01e6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa74da640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa01e7e30 0x7effa00b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa74da640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7effa00b5230 con 0x7effa00d2c30 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa74da640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7effa00b3d00 con 0x7effa015e370 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa74da640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7effa00b53b0 con 0x7effa01e7e30 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa5276640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 0x7effa01f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa5276640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 0x7effa01f35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49994/0 (socket says 172.21.15.26:49994) 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.074+0000 7effa5276640 1 -- 172.21.15.26:0/2463022021 learned_addr learned my addr 172.21.15.26:0/2463022021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa4a75640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa015e370 0x7effa01e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa5a77640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa01e7e30 0x7effa00b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa5276640 1 -- 172.21.15.26:0/2463022021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa01e7e30 msgr2=0x7effa00b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa5276640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa01e7e30 0x7effa00b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa5276640 1 -- 172.21.15.26:0/2463022021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa015e370 msgr2=0x7effa01e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa5276640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa015e370 0x7effa01e6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa5276640 1 -- 172.21.15.26:0/2463022021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa01e6e40 con 0x7effa00d2c30 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa5276640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 0x7effa01f35b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c890 tx=0x7eff9800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7eff9dffb640 1 -- 172.21.15.26:0/2463022021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98018070 con 0x7effa00d2c30 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7eff9dffb640 1 -- 172.21.15.26:0/2463022021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff9800a640 con 0x7effa00d2c30 2024-04-24T23:25:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa74da640 1 -- 172.21.15.26:0/2463022021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01e7120 con 0x7effa00d2c30 2024-04-24T23:25:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7eff9dffb640 1 -- 172.21.15.26:0/2463022021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9800aa30 con 0x7effa00d2c30 2024-04-24T23:25:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.075+0000 7effa74da640 1 -- 172.21.15.26:0/2463022021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa00c8810 con 0x7effa00d2c30 2024-04-24T23:25:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.077+0000 7eff9dffb640 1 -- 172.21.15.26:0/2463022021 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff9801b070 con 0x7effa00d2c30 2024-04-24T23:25:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.077+0000 7eff9dffb640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7eff8c06be30 0x7eff8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.077+0000 7eff9dffb640 1 -- 172.21.15.26:0/2463022021 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7eff9808a520 con 0x7effa00d2c30 2024-04-24T23:25:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.077+0000 7effa4a75640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7eff8c06be30 0x7eff8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.078+0000 7effa4a75640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7eff8c06be30 0x7eff8c06e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff94006fd0 tx=0x7eff94008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.078+0000 7effaa1ec740 1 Processor -- start 2024-04-24T23:25:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.078+0000 7effaa1ec740 1 -- start start 2024-04-24T23:25:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e43d60 0x555b78e44140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e61600 0x555b78e61b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e19a40 0x555b78e5fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effaa1ec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b78d7d6b0 con 0x555b78e61600 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effaa1ec740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b78d25880 con 0x555b78e43d60 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effaa1ec740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555b78d79670 con 0x555b78e19a40 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa4a75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e61600 0x555b78e61b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa4a75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e61600 0x555b78e61b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49998/0 (socket says 172.21.15.26:49998) 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa4a75640 1 -- 172.21.15.26:0/3686833643 learned_addr learned my addr 172.21.15.26:0/3686833643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa5276640 1 --2- 172.21.15.26:0/3686833643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e43d60 0x555b78e44140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa5a77640 1 --2- 172.21.15.26:0/3686833643 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e19a40 0x555b78e5fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa4a75640 1 -- 172.21.15.26:0/3686833643 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e19a40 msgr2=0x555b78e5fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa4a75640 1 --2- 172.21.15.26:0/3686833643 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e19a40 0x555b78e5fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa4a75640 1 -- 172.21.15.26:0/3686833643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e43d60 msgr2=0x555b78e44140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa4a75640 1 --2- 172.21.15.26:0/3686833643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e43d60 0x555b78e44140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.079+0000 7effa4a75640 1 -- 172.21.15.26:0/3686833643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b78b5b7d0 con 0x555b78e61600 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7effa4a75640 1 --2- 172.21.15.26:0/3686833643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e61600 0x555b78e61b50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7effa00d0840 tx=0x7eff94012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8fbbb1029a9469 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7eff86ffd640 1 -- 172.21.15.26:0/3686833643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9401b050 con 0x555b78e61600 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7eff86ffd640 1 -- 172.21.15.26:0/3686833643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff94025070 con 0x555b78e61600 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7eff86ffd640 1 -- 172.21.15.26:0/3686833643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94014de0 con 0x555b78e61600 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7effaa1ec740 1 -- 172.21.15.26:0/3686833643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e61600 msgr2=0x555b78e61b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3686833643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e61600 0x555b78e61b50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7effa00d0840 tx=0x7eff94012000 comp rx=0 tx=0).stop 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7effaa1ec740 1 -- 172.21.15.26:0/3686833643 shutdown_connections 2024-04-24T23:25:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3686833643 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e19a40 0x555b78e5fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3686833643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e61600 0x555b78e61b50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3686833643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e43d60 0x555b78e44140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.080+0000 7effaa1ec740 1 -- 172.21.15.26:0/3686833643 >> 172.21.15.26:0/3686833643 conn(0x555b78e16600 msgr2=0x555b78e5eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.081+0000 7effaa1ec740 1 -- 172.21.15.26:0/3686833643 shutdown_connections 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.081+0000 7effaa1ec740 1 -- 172.21.15.26:0/3686833643 wait complete. 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.081+0000 7effaa1ec740 1 -- start start 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.081+0000 7effaa1ec740 1 Processor -- start 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 0x555b78e42220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 0x555b78e30830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78e31360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effaa1ec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b78e1a7e0 con 0x555b78e13d00 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effaa1ec740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b78d38f50 con 0x555b78e61600 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effaa1ec740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555b78e14530 con 0x555b78e43d60 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effaa1ec740 10 client.0 _ll_register_callbacks cb 0x555b78d74000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effa5a77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78e31360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effa5a77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78e31360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50002/0 (socket says 172.21.15.26:50002) 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effa5a77640 1 -- 172.21.15.26:0/3551208957 learned_addr learned my addr 172.21.15.26:0/3551208957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effa5276640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 0x555b78e42220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.082+0000 7effa4a75640 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 0x555b78e30830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7effa5a77640 1 -- 172.21.15.26:0/3551208957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 msgr2=0x555b78e42220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7effa5a77640 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 0x555b78e42220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7effa5a77640 1 -- 172.21.15.26:0/3551208957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 msgr2=0x555b78e30830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7effa5a77640 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 0x555b78e30830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7effa5a77640 1 -- 172.21.15.26:0/3551208957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b78a913f0 con 0x555b78e13d00 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7effa5a77640 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78e31360 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7eff90035040 tx=0x7eff90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3342a9d6ee2ed507 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7eff7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7eff7ffff640 1 -- 172.21.15.26:0/3551208957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90042030 con 0x555b78e13d00 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7eff7ffff640 1 -- 172.21.15.26:0/3551208957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7eff90003b50 con 0x555b78e13d00 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.083+0000 7effaa1ec740 1 -- 172.21.15.26:0/3551208957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff94025070 con 0x555b78e13d00 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7eff7ffff640 1 -- 172.21.15.26:0/3551208957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7eff9004e050 con 0x555b78e13d00 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effaa1ec740 10 client.7506 fetch_fsmap learned FSMap version 74 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effaa1ec740 1 -- 172.21.15.26:0/3551208957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x555b78a510f0 con 0x555b78e13d00 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7eff7ffff640 1 -- 172.21.15.26:0/3551208957 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff94025070 con 0x555b78e13d00 2024-04-24T23:25:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7eff7ffff640 1 -- 172.21.15.26:0/3551208957 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7eff90051020 con 0x555b78e13d00 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effaa1ec740 10 client.7506 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effaa1ec740 10 client.7506 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effaa1ec740 20 client.7506 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x555b78e2e3b0 0x555b78e2e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effaa1ec740 4 client.7506 mds_command: new command op to 7227 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effaa1ec740 1 -- 172.21.15.26:0/3551208957 --> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b78d8abf0 con 0x555b78e2e3b0 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effa5276640 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x555b78e2e3b0 0x555b78e2e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7eff7ffff640 10 client.7506 ms_handle_connect on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.084+0000 7effa5276640 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x555b78e2e3b0 0x555b78e2e7c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01b87d33e79a1f2 server_cookie=b127108bb2c4edba in_seq=0 out_seq=0 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7eff7ffff640 1 -- 172.21.15.26:0/3551208957 <== mds.0 v2:172.21.15.26:6835/2537133693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b78d8abf0 con 0x555b78e2e3b0 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7eff7ffff640 10 client.7506 handle_command_reply: tid=0 2024-04-24T23:25:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 client.7506 shutdown 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 -- 172.21.15.26:0/3551208957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 msgr2=0x555b78e31360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78e31360 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7eff90035040 tx=0x7eff90004730 comp rx=0 tx=0).stop 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 -- 172.21.15.26:0/3551208957 shutdown_connections 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x555b78e2e3b0 0x555b78e2e7c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78e31360 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 0x555b78e30830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7eff7ffff640 0 client.7506 ms_handle_reset on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3551208957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 0x555b78e42220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 -- 172.21.15.26:0/3551208957 >> 172.21.15.26:0/3551208957 conn(0x555b78e19a40 msgr2=0x555b78e620f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 -- 172.21.15.26:0/3551208957 shutdown_connections 2024-04-24T23:25:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 1 -- 172.21.15.26:0/3551208957 wait complete. 2024-04-24T23:25:59.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.085+0000 7effaa1ec740 20 client.7506 trim_cache size 0 max 16384 2024-04-24T23:25:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.100+0000 7effaa1ec740 1 Processor -- start 2024-04-24T23:25:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.100+0000 7effaa1ec740 1 -- start start 2024-04-24T23:25:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e13d00 0x555b78e2e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e43d60 0x555b78e2eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e61600 0x555b78ed2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effaa1ec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7effa0200790 con 0x555b78e43d60 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effaa1ec740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7effa01ff960 con 0x555b78e13d00 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effaa1ec740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7effa0200060 con 0x555b78e61600 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effa4a75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e43d60 0x555b78e2eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effa4a75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e43d60 0x555b78e2eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50014/0 (socket says 172.21.15.26:50014) 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effa4a75640 1 -- 172.21.15.26:0/3965619847 learned_addr learned my addr 172.21.15.26:0/3965619847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effa4a75640 1 -- 172.21.15.26:0/3965619847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e61600 msgr2=0x555b78ed2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effa4a75640 1 --2- 172.21.15.26:0/3965619847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e61600 0x555b78ed2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effa4a75640 1 -- 172.21.15.26:0/3965619847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e13d00 msgr2=0x555b78e2e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effa4a75640 1 --2- 172.21.15.26:0/3965619847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e13d00 0x555b78e2e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effa4a75640 1 -- 172.21.15.26:0/3965619847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b78d3a260 con 0x555b78e43d60 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7effa4a75640 1 --2- 172.21.15.26:0/3965619847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e43d60 0x555b78e2eec0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7eff94022040 tx=0x7eff94014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459c56261f47a6fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.101+0000 7eff867fc640 1 -- 172.21.15.26:0/3965619847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94020640 con 0x555b78e43d60 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7eff867fc640 1 -- 172.21.15.26:0/3965619847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff94030070 con 0x555b78e43d60 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7eff867fc640 1 -- 172.21.15.26:0/3965619847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94020a20 con 0x555b78e43d60 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7effaa1ec740 1 -- 172.21.15.26:0/3965619847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e43d60 msgr2=0x555b78e2eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3965619847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e43d60 0x555b78e2eec0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7eff94022040 tx=0x7eff94014000 comp rx=0 tx=0).stop 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7effaa1ec740 1 -- 172.21.15.26:0/3965619847 shutdown_connections 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3965619847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e61600 0x555b78ed2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3965619847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e43d60 0x555b78e2eec0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7effaa1ec740 1 --2- 172.21.15.26:0/3965619847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e13d00 0x555b78e2e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7effaa1ec740 1 -- 172.21.15.26:0/3965619847 >> 172.21.15.26:0/3965619847 conn(0x555b78e19a40 msgr2=0x555b78e19e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7effaa1ec740 1 -- 172.21.15.26:0/3965619847 shutdown_connections 2024-04-24T23:25:59.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.102+0000 7effaa1ec740 1 -- 172.21.15.26:0/3965619847 wait complete. 2024-04-24T23:25:59.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.103+0000 7effaa1ec740 1 -- start start 2024-04-24T23:25:59.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.103+0000 7effaa1ec740 1 Processor -- start 2024-04-24T23:25:59.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78ed1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 0x555b78e5ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 0x555b78e78810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b78e06090 con 0x555b78e13d00 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7effa00c9920 con 0x555b78e61600 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555b78e16e70 con 0x555b78e43d60 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 10 client.0 _ll_register_callbacks cb 0x7eff90035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5276640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78ed1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5276640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78ed1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50022/0 (socket says 172.21.15.26:50022) 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5276640 1 -- 172.21.15.26:0/371583579 learned_addr learned my addr 172.21.15.26:0/371583579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5a77640 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 0x555b78e78810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa4a75640 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 0x555b78e5ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5276640 1 -- 172.21.15.26:0/371583579 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 msgr2=0x555b78e5ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5276640 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 0x555b78e5ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5276640 1 -- 172.21.15.26:0/371583579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 msgr2=0x555b78e78810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5276640 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 0x555b78e78810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5276640 1 -- 172.21.15.26:0/371583579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b78e0ff90 con 0x555b78e13d00 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effa5276640 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78ed1590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7eff980582f0 tx=0x7eff9801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d13b6fabe89614 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7eff7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7eff7ffff640 1 -- 172.21.15.26:0/371583579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98087060 con 0x555b78e13d00 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7eff7ffff640 1 -- 172.21.15.26:0/371583579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7eff98056b20 con 0x555b78e13d00 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 1 -- 172.21.15.26:0/371583579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff94030070 con 0x555b78e13d00 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7eff7ffff640 1 -- 172.21.15.26:0/371583579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7eff98075050 con 0x555b78e13d00 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 10 client.7512 fetch_fsmap learned FSMap version 74 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.104+0000 7effaa1ec740 1 -- 172.21.15.26:0/371583579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x555b78ed1dc0 con 0x555b78e13d00 2024-04-24T23:25:59.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7eff7ffff640 1 -- 172.21.15.26:0/371583579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff94030070 con 0x555b78e13d00 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7eff7ffff640 1 -- 172.21.15.26:0/371583579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7eff9800abd0 con 0x555b78e13d00 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7effaa1ec740 10 client.7512 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7effaa1ec740 10 client.7512 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7effaa1ec740 20 client.7512 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7effaa1ec740 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x555b78e78d50 0x555b78e46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7effaa1ec740 4 client.7512 mds_command: new command op to 7227 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7effaa1ec740 1 -- 172.21.15.26:0/371583579 --> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b78d84f40 con 0x555b78e78d50 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7effa4a75640 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x555b78e78d50 0x555b78e46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7eff7ffff640 10 client.7512 ms_handle_connect on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:59.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.105+0000 7effa4a75640 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x555b78e78d50 0x555b78e46da0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7af4c624cc54594 server_cookie=15ad7522071499b7 in_seq=0 out_seq=0 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7eff7ffff640 1 -- 172.21.15.26:0/371583579 <== mds.0 v2:172.21.15.26:6835/2537133693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b78d84f40 con 0x555b78e78d50 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7eff7ffff640 10 client.7512 handle_command_reply: tid=0 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 client.7512 shutdown 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 -- 172.21.15.26:0/371583579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 msgr2=0x555b78ed1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78ed1590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7eff980582f0 tx=0x7eff9801f040 comp rx=0 tx=0).stop 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 -- 172.21.15.26:0/371583579 shutdown_connections 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.26:6835/2537133693,v1:172.21.15.26:6837/2537133693] conn(0x555b78e78d50 0x555b78e46da0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b78e61600 0x555b78e78810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7eff7ffff640 0 client.7512 ms_handle_reset on v2:172.21.15.26:6835/2537133693 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x555b78e43d60 0x555b78e5ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 --2- 172.21.15.26:0/371583579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b78e13d00 0x555b78ed1590 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 -- 172.21.15.26:0/371583579 >> 172.21.15.26:0/371583579 conn(0x555b78e19a40 msgr2=0x555b78e56f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 -- 172.21.15.26:0/371583579 shutdown_connections 2024-04-24T23:25:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 1 -- 172.21.15.26:0/371583579 wait complete. 2024-04-24T23:25:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.116+0000 7effaa1ec740 20 client.7512 trim_cache size 0 max 16384 2024-04-24T23:25:59.119 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:25:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:25:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:25:59.119 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:25:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 -- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7eff8c06be30 msgr2=0x7eff8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7eff8c06be30 0x7eff8c06e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff94006fd0 tx=0x7eff94008040 comp rx=0 tx=0).stop 2024-04-24T23:25:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 -- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 msgr2=0x7effa01f35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 0x7effa01f35b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c890 tx=0x7eff9800cd50 comp rx=0 tx=0).stop 2024-04-24T23:25:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 -- 172.21.15.26:0/2463022021 shutdown_connections 2024-04-24T23:25:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7eff8c06be30 0x7eff8c06e2f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7effa01e7e30 0x7effa00b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7effa015e370 0x7effa01e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 --2- 172.21.15.26:0/2463022021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa00d2c30 0x7effa01f35b0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 -- 172.21.15.26:0/2463022021 >> 172.21.15.26:0/2463022021 conn(0x7effa00d40f0 msgr2=0x7effa00b5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 -- 172.21.15.26:0/2463022021 shutdown_connections 2024-04-24T23:25:59.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.117+0000 7effa74da640 1 -- 172.21.15.26:0/2463022021 wait complete. 2024-04-24T23:25:59.128 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:1 flush journal 2024-04-24T23:25:59.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.230+0000 7f1cc3da5640 1 Processor -- start 2024-04-24T23:25:59.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.230+0000 7f1cc3da5640 1 -- start start 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc3da5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc0b4ce0 0x7f1cbc0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc3da5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cbc0aeb10 con 0x7f1cbc15be40 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc3da5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cbc0aedf0 con 0x7f1cbc0d0750 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc3da5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc09df20 con 0x7f1cbc0b4ce0 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc1b41640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1cbc15be40 0x7f1cbc0d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56784/0 (socket says 172.21.15.26:56784) 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc1b41640 1 -- 172.21.15.26:0/3370581786 learned_addr learned my addr 172.21.15.26:0/3370581786 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064923625 0 0) 0x7f1cbc0aeb10 con 0x7f1cbc15be40 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ca8003520 con 0x7f1cbc15be40 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc2342640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc0b4ce0 0x7f1cbc0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1885140163 0 0) 0x7f1ca8003520 con 0x7f1cbc15be40 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc0b4ce0 msgr2=0x7f1cbc0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc0b3f640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc0b4ce0 0x7f1cbc0acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 >> v1:172.21.15.29:6789/0 conn(0x7f1cbc0d0750 legacy=0x7f1cbc15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc0af0d0 con 0x7f1cbc15be40 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f1cb8001670 con 0x7f1cbc15be40 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.231+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 >> v1:172.21.15.26:6789/0 conn(0x7f1cbc15be40 legacy=0x7f1cbc0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc0b3f640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca80039e0 0x7f1ca8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc0b3f640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ca8004800 0x7f1ca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc0b3f640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ca80051c0 0x7f1ca800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca800c090 con 0x7f1ca80039e0 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca800c210 con 0x7f1ca80051c0 2024-04-24T23:25:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca800c410 con 0x7f1ca8004800 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc2342640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc0b4ce0 0x7f1cbc0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca80039e0 0x7f1ca8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc1340640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ca8004800 0x7f1ca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc2342640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ca80051c0 0x7f1ca800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc1b41640 1 -- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ca8004800 msgr2=0x7f1ca8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ca8004800 0x7f1ca8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc1b41640 1 -- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ca80051c0 msgr2=0x7f1ca800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ca80051c0 0x7f1ca800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.232+0000 7f1cc1b41640 1 -- 172.21.15.26:0/3370581786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca8003520 con 0x7f1ca80039e0 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca80039e0 0x7f1ca8003e00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c240 tx=0x7f1ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fc2460ad833eea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8007cf0 con 0x7f1ca80039e0 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb80058d0 con 0x7f1ca80039e0 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc0b3f640 1 -- 172.21.15.26:0/3370581786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8005ce0 con 0x7f1ca80039e0 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc3da5640 1 -- 172.21.15.26:0/3370581786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca80039e0 msgr2=0x7f1ca8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc3da5640 1 --2- 172.21.15.26:0/3370581786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ca80039e0 0x7f1ca8003e00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c240 tx=0x7f1ca8005830 comp rx=0 tx=0).stop 2024-04-24T23:25:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc2342640 1 -- 172.21.15.26:0/3370581786 reap_dead start 2024-04-24T23:25:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc3da5640 1 -- 172.21.15.26:0/3370581786 shutdown_connections 2024-04-24T23:25:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc3da5640 1 -- 172.21.15.26:0/3370581786 >> 172.21.15.26:0/3370581786 conn(0x7f1cbc06b540 msgr2=0x7f1cbc06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc3da5640 1 -- 172.21.15.26:0/3370581786 shutdown_connections 2024-04-24T23:25:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.233+0000 7f1cc3da5640 1 -- 172.21.15.26:0/3370581786 wait complete. 2024-04-24T23:25:59.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc3da5640 1 Processor -- start 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc3da5640 1 -- start start 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc3da5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ca80039e0 0x7f1cbc152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc3da5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc14e500 0x7f1cbc14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc3da5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cbc154690 0x7f1cbc154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc3da5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc06cd10 con 0x7f1cbc154690 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc3da5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc06cb90 con 0x7f1ca80039e0 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc3da5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc0acd30 con 0x7f1cbc14e500 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc2342640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cbc154690 0x7f1cbc154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc2342640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cbc154690 0x7f1cbc154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50042/0 (socket says 172.21.15.26:50042) 2024-04-24T23:25:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.235+0000 7f1cc2342640 1 -- 172.21.15.26:0/275789674 learned_addr learned my addr 172.21.15.26:0/275789674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ca80039e0 0x7f1cbc152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc1340640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc14e500 0x7f1cbc14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc2342640 1 -- 172.21.15.26:0/275789674 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc14e500 msgr2=0x7f1cbc14dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc2342640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc14e500 0x7f1cbc14dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc2342640 1 -- 172.21.15.26:0/275789674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ca80039e0 msgr2=0x7f1cbc152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc2342640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ca80039e0 0x7f1cbc152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc2342640 1 -- 172.21.15.26:0/275789674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc155510 con 0x7f1cbc154690 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc2342640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cbc154690 0x7f1cbc154b10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0af3d0 tx=0x7f1cb4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cb27fc640 1 -- 172.21.15.26:0/275789674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4015480 con 0x7f1cbc154690 2024-04-24T23:25:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cb27fc640 1 -- 172.21.15.26:0/275789674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb401a070 con 0x7f1cbc154690 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc3da5640 1 -- 172.21.15.26:0/275789674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc1557f0 con 0x7f1cbc154690 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cb27fc640 1 -- 172.21.15.26:0/275789674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4015860 con 0x7f1cbc154690 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.236+0000 7f1cc3da5640 1 -- 172.21.15.26:0/275789674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc155db0 con 0x7f1cbc154690 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.238+0000 7f1cb27fc640 1 -- 172.21.15.26:0/275789674 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cb4015a00 con 0x7f1cbc154690 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.238+0000 7f1cb27fc640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ca006be30 0x7f1ca006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.238+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ca006be30 0x7f1ca006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.238+0000 7f1cb27fc640 1 -- 172.21.15.26:0/275789674 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f1cb408b910 con 0x7f1cbc154690 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.239+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ca006be30 0x7f1ca006e2f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c240 tx=0x7f1cb8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.239+0000 7f1cc6b8f740 1 Processor -- start 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc6b8f740 1 -- start start 2024-04-24T23:25:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab310 0x563dad983970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 0x563dad9adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad981910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563dad8e76b0 con 0x563dad9ab310 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563dad88f880 con 0x563dad9ab6f0 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563dad8e3670 con 0x563dad9ae0b0 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc1b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab310 0x563dad983970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc1b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab310 0x563dad983970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50058/0 (socket says 172.21.15.26:50058) 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.240+0000 7f1cc1b41640 1 -- 172.21.15.26:0/2232311098 learned_addr learned my addr 172.21.15.26:0/2232311098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1cc2342640 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad981910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1cc1340640 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 0x563dad9adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1cc1b41640 1 -- 172.21.15.26:0/2232311098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 msgr2=0x563dad981910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad981910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1cc1b41640 1 -- 172.21.15.26:0/2232311098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 msgr2=0x563dad9adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 0x563dad9adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1cc1b41640 1 -- 172.21.15.26:0/2232311098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dad6c57d0 con 0x563dad9ab310 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1cc2342640 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad981910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab310 0x563dad983970 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c330 tx=0x7f1cb8007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb3ff28879c398c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1c9f7fe640 1 -- 172.21.15.26:0/2232311098 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb804e070 con 0x563dad9ab310 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1c9f7fe640 1 -- 172.21.15.26:0/2232311098 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb8005d30 con 0x563dad9ab310 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.241+0000 7f1c9f7fe640 1 -- 172.21.15.26:0/2232311098 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8042030 con 0x563dad9ab310 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2232311098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab310 msgr2=0x563dad983970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab310 0x563dad983970 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c330 tx=0x7f1cb8007000 comp rx=0 tx=0).stop 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2232311098 shutdown_connections 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad981910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 0x563dad9adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/2232311098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab310 0x563dad983970 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2232311098 >> 172.21.15.26:0/2232311098 conn(0x563dad980600 msgr2=0x563dad9c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2232311098 shutdown_connections 2024-04-24T23:25:59.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2232311098 wait complete. 2024-04-24T23:25:59.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 -- start start 2024-04-24T23:25:59.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.242+0000 7f1cc6b8f740 1 Processor -- start 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 0x563dad995570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ae0b0 0x563dad9c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9bfbe0 0x563dad9c0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563dad8a2f50 con 0x563dad9ae0b0 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563dad9c6870 con 0x563dad9ab6f0 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563dad9979e0 con 0x563dad9bfbe0 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc6b8f740 10 client.0 _ll_register_callbacks cb 0x563dad8de000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc1340640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ae0b0 0x563dad9c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc1340640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ae0b0 0x563dad9c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50060/0 (socket says 172.21.15.26:50060) 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc1b41640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 0x563dad995570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.243+0000 7f1cc1340640 1 -- 172.21.15.26:0/283854078 learned_addr learned my addr 172.21.15.26:0/283854078 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc2342640 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9bfbe0 0x563dad9c0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc1340640 1 -- 172.21.15.26:0/283854078 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9bfbe0 msgr2=0x563dad9c0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc1340640 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9bfbe0 0x563dad9c0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc1340640 1 -- 172.21.15.26:0/283854078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 msgr2=0x563dad995570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc1340640 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 0x563dad995570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc1340640 1 -- 172.21.15.26:0/283854078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dad5fb3f0 con 0x563dad9ae0b0 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc1340640 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ae0b0 0x563dad9c8760 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1cac003c20 tx=0x7f1cac0017d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d447c9ee192b8d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1c9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1c9cff9640 1 -- 172.21.15.26:0/283854078 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac015680 con 0x563dad9ae0b0 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1c9cff9640 1 -- 172.21.15.26:0/283854078 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f1cac015820 con 0x563dad9ae0b0 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/283854078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb8005d30 con 0x563dad9ae0b0 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1c9cff9640 1 -- 172.21.15.26:0/283854078 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f1cac015a20 con 0x563dad9ae0b0 2024-04-24T23:25:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc6b8f740 10 client.7524 fetch_fsmap learned FSMap version 74 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/283854078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x563dad5bb0f0 con 0x563dad9ae0b0 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.244+0000 7f1c9cff9640 1 -- 172.21.15.26:0/283854078 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb8005d30 con 0x563dad9ae0b0 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1c9cff9640 1 -- 172.21.15.26:0/283854078 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7f1cac015bc0 con 0x563dad9ae0b0 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1cc6b8f740 10 client.7524 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1cc6b8f740 10 client.7524 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.a 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1cc6b8f740 20 client.7524 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x563dad996b70 0x563dad996f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1cc6b8f740 4 client.7524 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/283854078 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dad8f4bf0 con 0x563dad996b70 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x563dad996b70 0x563dad996f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x563dad996b70 0x563dad996f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2c6df30f1b55bbc server_cookie=1ed7c00951a5d46 in_seq=0 out_seq=0 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.245+0000 7f1c9cff9640 10 client.7524 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1c9cff9640 1 -- 172.21.15.26:0/283854078 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563dad8f4bf0 con 0x563dad996b70 2024-04-24T23:25:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1c9cff9640 10 client.7524 handle_command_reply: tid=0 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 client.7524 shutdown 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/283854078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ae0b0 msgr2=0x563dad9c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ae0b0 0x563dad9c8760 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1cac003c20 tx=0x7f1cac0017d0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/283854078 shutdown_connections 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x563dad996b70 0x563dad996f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9bfbe0 0x563dad9c0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1c9cff9640 0 client.7524 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ae0b0 0x563dad9c8760 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/283854078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9ab6f0 0x563dad995570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/283854078 >> 172.21.15.26:0/283854078 conn(0x563dad9ab310 msgr2=0x563dad9cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/283854078 shutdown_connections 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/283854078 wait complete. 2024-04-24T23:25:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.246+0000 7f1cc6b8f740 20 client.7524 trim_cache size 0 max 16384 2024-04-24T23:25:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.261+0000 7f1cc6b8f740 1 Processor -- start 2024-04-24T23:25:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.261+0000 7f1cc6b8f740 1 -- start start 2024-04-24T23:25:59.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dad997140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad9c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 0x563dad9c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1fc9c0 con 0x563dad9ab6f0 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc072a50 con 0x563dad9bfbe0 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc1fc290 con 0x563dad9ae0b0 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc1b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dad997140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc1b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dad997140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50076/0 (socket says 172.21.15.26:50076) 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc1b41640 1 -- 172.21.15.26:0/2664372025 learned_addr learned my addr 172.21.15.26:0/2664372025 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc1b41640 1 -- 172.21.15.26:0/2664372025 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 msgr2=0x563dad9c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/2664372025 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad9c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc1b41640 1 -- 172.21.15.26:0/2664372025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 msgr2=0x563dad9c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/2664372025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 0x563dad9c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc1b41640 1 -- 172.21.15.26:0/2664372025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dad982b30 con 0x563dad9ab6f0 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/2664372025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dad997140 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c360 tx=0x7f1cb803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d4971ac4c94475 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1c9effd640 1 -- 172.21.15.26:0/2664372025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb805e070 con 0x563dad9ab6f0 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1c9effd640 1 -- 172.21.15.26:0/2664372025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb8059e50 con 0x563dad9ab6f0 2024-04-24T23:25:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.262+0000 7f1c9effd640 1 -- 172.21.15.26:0/2664372025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb804e040 con 0x563dad9ab6f0 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.263+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2664372025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 msgr2=0x563dad997140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.263+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/2664372025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dad997140 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c360 tx=0x7f1cb803e000 comp rx=0 tx=0).stop 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.263+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2664372025 shutdown_connections 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.263+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/2664372025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 0x563dad9c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.263+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/2664372025 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad9c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.263+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/2664372025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dad997140 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.263+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2664372025 >> 172.21.15.26:0/2664372025 conn(0x563dad9ab310 msgr2=0x563dad9cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.263+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2664372025 shutdown_connections 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.263+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/2664372025 wait complete. 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.264+0000 7f1cc6b8f740 1 -- start start 2024-04-24T23:25:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.264+0000 7f1cc6b8f740 1 Processor -- start 2024-04-24T23:25:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.264+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dada460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.264+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad9cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.264+0000 7f1cc6b8f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 0x563dad9cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.264+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563dad9b1580 con 0x563dad9ab6f0 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.264+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0b0c00 con 0x563dad9bfbe0 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.264+0000 7f1cc6b8f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563dad9c7820 con 0x563dad9ae0b0 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.264+0000 7f1cc6b8f740 10 client.0 _ll_register_callbacks cb 0x7f1cac005050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dada460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dada460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50084/0 (socket says 172.21.15.26:50084) 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1b41640 1 -- 172.21.15.26:0/3152820397 learned_addr learned my addr 172.21.15.26:0/3152820397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc2342640 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 0x563dad9cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1340640 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad9cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1b41640 1 -- 172.21.15.26:0/3152820397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 msgr2=0x563dad9cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad9cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1b41640 1 -- 172.21.15.26:0/3152820397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 msgr2=0x563dad9cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 0x563dad9cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1b41640 1 -- 172.21.15.26:0/3152820397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dad9aae50 con 0x563dad9ab6f0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc2342640 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 0x563dad9cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc1b41640 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dada460a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8001710 tx=0x7f1cbc0aece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce0989a47049207 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1c9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1c9cff9640 1 -- 172.21.15.26:0/3152820397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb805e070 con 0x563dad9ab6f0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1c9cff9640 1 -- 172.21.15.26:0/3152820397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f1cb8007630 con 0x563dad9ab6f0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/3152820397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb8059e50 con 0x563dad9ab6f0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1c9cff9640 1 -- 172.21.15.26:0/3152820397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f1cb806a050 con 0x563dad9ab6f0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc6b8f740 10 client.7530 fetch_fsmap learned FSMap version 74 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.265+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/3152820397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x563dad97e670 con 0x563dad9ab6f0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1c9cff9640 1 -- 172.21.15.26:0/3152820397 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb8059e50 con 0x563dad9ab6f0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1c9cff9640 1 -- 172.21.15.26:0/3152820397 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7f1cb806d020 con 0x563dad9ab6f0 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1cc6b8f740 10 client.7530 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1cc6b8f740 10 client.7530 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.a 2024-04-24T23:25:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1cc6b8f740 20 client.7530 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x563dad9a98c0 0x563dad9c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1cc6b8f740 4 client.7530 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:25:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/3152820397 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563dad8eef40 con 0x563dad9a98c0 2024-04-24T23:25:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1cc1340640 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x563dad9a98c0 0x563dad9c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1c9cff9640 10 client.7530 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.266+0000 7f1cc1340640 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x563dad9a98c0 0x563dad9c6830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34430e31eba6695c server_cookie=77caa9f7d590cccf in_seq=0 out_seq=0 2024-04-24T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.273+0000 7f1c9cff9640 1 -- 172.21.15.26:0/3152820397 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563dad8eef40 con 0x563dad9a98c0 2024-04-24T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.273+0000 7f1c9cff9640 10 client.7530 handle_command_reply: tid=0 2024-04-24T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.273+0000 7f1cc6b8f740 1 client.7530 shutdown 2024-04-24T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/3152820397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 msgr2=0x563dada460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dada460a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8001710 tx=0x7f1cbc0aece0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/3152820397 shutdown_connections 2024-04-24T23:25:59.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x563dad9a98c0 0x563dad9c6830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563dad9bfbe0 0x563dad9cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563dad9ae0b0 0x563dad9cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 --2- 172.21.15.26:0/3152820397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563dad9ab6f0 0x563dada460a0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/3152820397 >> 172.21.15.26:0/3152820397 conn(0x563dad9ab310 msgr2=0x563dad9a9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1c9cff9640 0 client.7530 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/3152820397 shutdown_connections 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 1 -- 172.21.15.26:0/3152820397 wait complete. 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc6b8f740 20 client.7530 trim_cache size 0 max 16384 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:25:59.276 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:25:59.277 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:25:59.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.274+0000 7f1cc3da5640 1 -- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ca006be30 msgr2=0x7f1ca006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ca006be30 0x7f1ca006e2f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c240 tx=0x7f1cb8009030 comp rx=0 tx=0).stop 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 -- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cbc154690 msgr2=0x7f1cbc154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cbc154690 0x7f1cbc154b10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0af3d0 tx=0x7f1cb4007700 comp rx=0 tx=0).stop 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 -- 172.21.15.26:0/275789674 shutdown_connections 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ca006be30 0x7f1ca006e2f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cbc154690 0x7f1cbc154b10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1cbc14e500 0x7f1cbc14dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 --2- 172.21.15.26:0/275789674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ca80039e0 0x7f1cbc152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 -- 172.21.15.26:0/275789674 >> 172.21.15.26:0/275789674 conn(0x7f1cbc06b540 msgr2=0x7f1cbc0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 -- 172.21.15.26:0/275789674 shutdown_connections 2024-04-24T23:25:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.275+0000 7f1cc3da5640 1 -- 172.21.15.26:0/275789674 wait complete. 2024-04-24T23:25:59.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:2 flush journal 2024-04-24T23:25:59.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.388+0000 7fe5fbaa5640 1 Processor -- start 2024-04-24T23:25:59.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.388+0000 7fe5fbaa5640 1 -- start start 2024-04-24T23:25:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5fbaa5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5f415adc0 0x7fe5f40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5fbaa5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f40ad7b0 0x7fe5f40d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5fbaa5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f415ef60 con 0x7fe5f415e9c0 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5fbaa5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe5f415f240 con 0x7fe5f415adc0 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5fbaa5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f415f520 con 0x7fe5f40ad7b0 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5f9841640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5f415adc0 0x7fe5f40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5fa042640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe5f415e9c0 0x7fe5f406dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56786/0 (socket says 172.21.15.26:56786) 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5fa042640 1 -- 172.21.15.26:0/4026970127 learned_addr learned my addr 172.21.15.26:0/4026970127 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5f9040640 1 --2- 172.21.15.26:0/4026970127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f40ad7b0 0x7fe5f40d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5f9841640 1 -- 172.21.15.26:0/4026970127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f40ad7b0 msgr2=0x7fe5f40d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5f9841640 1 --2- 172.21.15.26:0/4026970127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f40ad7b0 0x7fe5f40d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5f9841640 1 -- 172.21.15.26:0/4026970127 >> v1:172.21.15.26:6789/0 conn(0x7fe5f415e9c0 legacy=0x7fe5f406dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5f9841640 1 -- 172.21.15.26:0/4026970127 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f415f800 con 0x7fe5f415adc0 2024-04-24T23:25:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.389+0000 7fe5f883f640 1 -- 172.21.15.26:0/4026970127 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2026960066 0 0) 0x7fe5f415ef60 con 0x7fe5f415e9c0 2024-04-24T23:25:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.390+0000 7fe5f9841640 1 --2- 172.21.15.26:0/4026970127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415adc0 0x7fe5f40b4370 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00097a0 tx=0x7fe5f002c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a791e8c01a4c4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.390+0000 7fe5f883f640 1 -- 172.21.15.26:0/4026970127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0039070 con 0x7fe5f415adc0 2024-04-24T23:25:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.390+0000 7fe5f883f640 1 -- 172.21.15.26:0/4026970127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f0034640 con 0x7fe5f415adc0 2024-04-24T23:25:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.390+0000 7fe5f883f640 1 -- 172.21.15.26:0/4026970127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f002ce60 con 0x7fe5f415adc0 2024-04-24T23:25:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.391+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/4026970127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415adc0 msgr2=0x7fe5f40b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.391+0000 7fe5fbaa5640 1 --2- 172.21.15.26:0/4026970127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415adc0 0x7fe5f40b4370 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00097a0 tx=0x7fe5f002c310 comp rx=0 tx=0).stop 2024-04-24T23:25:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.391+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/4026970127 shutdown_connections 2024-04-24T23:25:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.391+0000 7fe5fbaa5640 1 --2- 172.21.15.26:0/4026970127 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f40ad7b0 0x7fe5f40d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.391+0000 7fe5fbaa5640 1 --2- 172.21.15.26:0/4026970127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415adc0 0x7fe5f40b4370 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.391+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/4026970127 >> 172.21.15.26:0/4026970127 conn(0x7fe5f406b540 msgr2=0x7fe5f406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.391+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/4026970127 shutdown_connections 2024-04-24T23:25:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.391+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/4026970127 wait complete. 2024-04-24T23:25:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.392+0000 7fe5fbaa5640 1 Processor -- start 2024-04-24T23:25:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.392+0000 7fe5fbaa5640 1 -- start start 2024-04-24T23:25:59.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fbaa5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f40ad7b0 0x7fe5f415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fbaa5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f415adc0 0x7fe5f4150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fbaa5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415e9c0 0x7fe5f409d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fbaa5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f415b1a0 con 0x7fe5f415e9c0 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fbaa5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f40b48b0 con 0x7fe5f40ad7b0 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fbaa5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f40d0580 con 0x7fe5f415adc0 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fa042640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415e9c0 0x7fe5f409d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fa042640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415e9c0 0x7fe5f409d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50088/0 (socket says 172.21.15.26:50088) 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fa042640 1 -- 172.21.15.26:0/2607786056 learned_addr learned my addr 172.21.15.26:0/2607786056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5f9841640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f40ad7b0 0x7fe5f415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5f9040640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f415adc0 0x7fe5f4150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fa042640 1 -- 172.21.15.26:0/2607786056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f415adc0 msgr2=0x7fe5f4150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fa042640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f415adc0 0x7fe5f4150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fa042640 1 -- 172.21.15.26:0/2607786056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f40ad7b0 msgr2=0x7fe5f415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fa042640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f40ad7b0 0x7fe5f415dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fa042640 1 -- 172.21.15.26:0/2607786056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f415ce20 con 0x7fe5f415e9c0 2024-04-24T23:25:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5f9040640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f415adc0 0x7fe5f4150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.393+0000 7fe5fa042640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415e9c0 0x7fe5f409d6c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00c700 tx=0x7fe5ec00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.394+0000 7fe5ea7fc640 1 -- 172.21.15.26:0/2607786056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec018070 con 0x7fe5f415e9c0 2024-04-24T23:25:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.394+0000 7fe5ea7fc640 1 -- 172.21.15.26:0/2607786056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ec00ce10 con 0x7fe5f415e9c0 2024-04-24T23:25:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.394+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/2607786056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f415d160 con 0x7fe5f415e9c0 2024-04-24T23:25:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.394+0000 7fe5ea7fc640 1 -- 172.21.15.26:0/2607786056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00a860 con 0x7fe5f415e9c0 2024-04-24T23:25:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.394+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/2607786056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f415d9a0 con 0x7fe5f415e9c0 2024-04-24T23:25:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.396+0000 7fe5ea7fc640 1 -- 172.21.15.26:0/2607786056 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5ec01b070 con 0x7fe5f415e9c0 2024-04-24T23:25:59.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.396+0000 7fe5ea7fc640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe5d806be30 0x7fe5d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.396+0000 7fe5ea7fc640 1 -- 172.21.15.26:0/2607786056 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fe5ec08a4c0 con 0x7fe5f415e9c0 2024-04-24T23:25:59.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.396+0000 7fe5f9841640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe5d806be30 0x7fe5d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.397+0000 7fe5f9841640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe5d806be30 0x7fe5d806e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0036000 tx=0x7fe5f0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.397+0000 7fe5fe7bb740 1 Processor -- start 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.397+0000 7fe5fe7bb740 1 -- start start 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 0x5568bab87140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568baba4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568bab5ca40 0x5568baba2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5568baac06b0 con 0x5568baba4600 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5568baa68880 con 0x5568bab5ca40 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5568baabc670 con 0x5568bab86d60 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9040640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568baba4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9040640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568baba4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50090/0 (socket says 172.21.15.26:50090) 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9040640 1 -- 172.21.15.26:0/1228146678 learned_addr learned my addr 172.21.15.26:0/1228146678 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9841640 1 --2- 172.21.15.26:0/1228146678 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 0x5568bab87140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5fa042640 1 --2- 172.21.15.26:0/1228146678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568bab5ca40 0x5568baba2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9040640 1 -- 172.21.15.26:0/1228146678 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 msgr2=0x5568bab87140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9040640 1 --2- 172.21.15.26:0/1228146678 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 0x5568bab87140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9040640 1 -- 172.21.15.26:0/1228146678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568bab5ca40 msgr2=0x5568baba2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9040640 1 --2- 172.21.15.26:0/1228146678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568bab5ca40 0x5568baba2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9040640 1 -- 172.21.15.26:0/1228146678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568ba89e7d0 con 0x5568baba4600 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5f9040640 1 --2- 172.21.15.26:0/1228146678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568baba4b50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009b70 tx=0x7fe5e4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c03d3940e9e93c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.398+0000 7fe5d77fe640 1 -- 172.21.15.26:0/1228146678 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e400f040 con 0x5568baba4600 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5d77fe640 1 -- 172.21.15.26:0/1228146678 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e4010e60 con 0x5568baba4600 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5d77fe640 1 -- 172.21.15.26:0/1228146678 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4004270 con 0x5568baba4600 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1228146678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 msgr2=0x5568baba4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1228146678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568baba4b50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009b70 tx=0x7fe5e4009190 comp rx=0 tx=0).stop 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1228146678 shutdown_connections 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1228146678 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568bab5ca40 0x5568baba2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1228146678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568baba4b50 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1228146678 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 0x5568bab87140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1228146678 >> 172.21.15.26:0/1228146678 conn(0x5568bab59600 msgr2=0x5568baba1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1228146678 shutdown_connections 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.399+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1228146678 wait complete. 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.400+0000 7fe5fe7bb740 1 -- start start 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.400+0000 7fe5fe7bb740 1 Processor -- start 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 0x5568bab729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568babb5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5568bab5d7e0 con 0x5568baba4600 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5568baa7bf50 con 0x5568babb6020 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5568bab57530 con 0x5568bab86d60 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5fe7bb740 10 client.0 _ll_register_callbacks cb 0x5568baab7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9040640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568babb5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9040640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568babb5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50102/0 (socket says 172.21.15.26:50102) 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9040640 1 -- 172.21.15.26:0/3912154890 learned_addr learned my addr 172.21.15.26:0/3912154890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5fa042640 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab9b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9841640 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 0x5568bab729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9040640 1 -- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 msgr2=0x5568bab729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9040640 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 0x5568bab729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9040640 1 -- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 msgr2=0x5568bab9b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9040640 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab9b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9040640 1 -- 172.21.15.26:0/3912154890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568ba7d43f0 con 0x5568baba4600 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5f9040640 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568babb5ae0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4010a50 tx=0x7fe5e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac07ed13c8c4757 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.401+0000 7fe5d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/3912154890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e402c060 con 0x5568baba4600 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/3912154890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fe5e4010c60 con 0x5568baba4600 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/3912154890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5e4010e60 con 0x5568baba4600 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/3912154890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fe5e4032050 con 0x5568baba4600 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5fe7bb740 10 client.7545 fetch_fsmap learned FSMap version 74 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/3912154890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5568ba7940f0 con 0x5568baba4600 2024-04-24T23:25:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/3912154890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e4010e60 con 0x5568baba4600 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/3912154890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7fe5e4035020 con 0x5568baba4600 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5fe7bb740 10 client.7545 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5fe7bb740 10 client.7545 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.d 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5fe7bb740 20 client.7545 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x5568bab89da0 0x5568babb23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5fe7bb740 4 client.7545 mds_command: new command op to 6364 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/3912154890 --> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568baacdbf0 con 0x5568bab89da0 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.402+0000 7fe5f9841640 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x5568bab89da0 0x5568babb23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5d4ff9640 10 client.7545 ms_handle_connect on v2:172.21.15.29:6833/2057165440 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5f9841640 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x5568bab89da0 0x5568babb23e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd2100a0aedfb2cb server_cookie=967889ff2d78083f in_seq=0 out_seq=0 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/3912154890 <== mds.2 v2:172.21.15.29:6833/2057165440 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568baacdbf0 con 0x5568bab89da0 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5d4ff9640 10 client.7545 handle_command_reply: tid=0 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5fe7bb740 1 client.7545 shutdown 2024-04-24T23:25:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/3912154890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 msgr2=0x5568babb5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568babb5ae0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4010a50 tx=0x7fe5e4004730 comp rx=0 tx=0).stop 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/3912154890 shutdown_connections 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab9b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568baba4600 0x5568babb5ae0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x5568bab89da0 0x5568babb23e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/3912154890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568bab86d60 0x5568bab729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5d4ff9640 0 client.7545 ms_handle_reset on v2:172.21.15.29:6833/2057165440 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.403+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/3912154890 >> 172.21.15.26:0/3912154890 conn(0x5568bab5ca40 msgr2=0x5568baba5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.404+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/3912154890 shutdown_connections 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.404+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/3912154890 wait complete. 2024-04-24T23:25:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.404+0000 7fe5fe7bb740 20 client.7545 trim_cache size 0 max 16384 2024-04-24T23:25:59.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.419+0000 7fe5fe7bb740 1 Processor -- start 2024-04-24T23:25:59.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.419+0000 7fe5fe7bb740 1 -- start start 2024-04-24T23:25:59.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.419+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568bab56e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.419+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 0x5568bac15480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.419+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab5ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.419+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41fc4e0 con 0x5568bab86d60 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.419+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41fb840 con 0x5568babb6020 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.419+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f41fbdb0 con 0x5568baba4600 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9841640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568bab56e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9841640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568bab56e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50104/0 (socket says 172.21.15.26:50104) 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9841640 1 -- 172.21.15.26:0/1865108463 learned_addr learned my addr 172.21.15.26:0/1865108463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9040640 1 --2- 172.21.15.26:0/1865108463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 0x5568bac15480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5fa042640 1 --2- 172.21.15.26:0/1865108463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab5ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9841640 1 -- 172.21.15.26:0/1865108463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 msgr2=0x5568bac15480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9841640 1 --2- 172.21.15.26:0/1865108463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 0x5568bac15480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9841640 1 -- 172.21.15.26:0/1865108463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 msgr2=0x5568bab5ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9841640 1 --2- 172.21.15.26:0/1865108463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab5ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9841640 1 -- 172.21.15.26:0/1865108463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568baba0b90 con 0x5568bab86d60 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5f9841640 1 --2- 172.21.15.26:0/1865108463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568bab56e30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00097a0 tx=0x7fe5f002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2720a49803aa075f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5d6ffd640 1 -- 172.21.15.26:0/1865108463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0039040 con 0x5568bab86d60 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5d6ffd640 1 -- 172.21.15.26:0/1865108463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f0057070 con 0x5568bab86d60 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.420+0000 7fe5d6ffd640 1 -- 172.21.15.26:0/1865108463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f002ce60 con 0x5568bab86d60 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1865108463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 msgr2=0x5568bab56e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1865108463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568bab56e30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00097a0 tx=0x7fe5f002e000 comp rx=0 tx=0).stop 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1865108463 shutdown_connections 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1865108463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab5ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1865108463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 0x5568bac15480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1865108463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568bab56e30 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1865108463 >> 172.21.15.26:0/1865108463 conn(0x5568bab5ca40 msgr2=0x5568bab727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1865108463 shutdown_connections 2024-04-24T23:25:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1865108463 wait complete. 2024-04-24T23:25:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 -- start start 2024-04-24T23:25:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.421+0000 7fe5fe7bb740 1 Processor -- start 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568babc5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 0x5568bab5a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5fe7bb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab5ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41f34d0 con 0x5568bab86d60 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5568bab5de10 con 0x5568babb6020 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5fe7bb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5568babb5160 con 0x5568baba4600 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5fe7bb740 10 client.0 _ll_register_callbacks cb 0x7fe5ec0549d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5f9841640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568babc5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5f9841640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568babc5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50106/0 (socket says 172.21.15.26:50106) 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.422+0000 7fe5f9841640 1 -- 172.21.15.26:0/1818880524 learned_addr learned my addr 172.21.15.26:0/1818880524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5f9040640 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 0x5568bab5a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5fa042640 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab5ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5f9841640 1 -- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 msgr2=0x5568bab5a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5f9841640 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 0x5568bab5a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5f9841640 1 -- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 msgr2=0x5568bab5ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5f9841640 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab5ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5f9841640 1 -- 172.21.15.26:0/1818880524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568babc5050 con 0x5568bab86d60 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5fa042640 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab5ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5f9841640 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568babc5b10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00098c0 tx=0x7fe5f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd21dea640f7bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/1818880524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0060060 con 0x5568bab86d60 2024-04-24T23:25:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/1818880524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fe5f00524f0 con 0x5568bab86d60 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1818880524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f0057070 con 0x5568bab86d60 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/1818880524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fe5f0066050 con 0x5568bab86d60 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5fe7bb740 10 client.7551 fetch_fsmap learned FSMap version 74 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1818880524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5568babb4ea0 con 0x5568bab86d60 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/1818880524 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f0057070 con 0x5568bab86d60 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/1818880524 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7fe5f0069020 con 0x5568bab86d60 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.423+0000 7fe5fe7bb740 10 client.7551 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.424+0000 7fe5fe7bb740 10 client.7551 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.d 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.424+0000 7fe5fe7bb740 20 client.7551 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.424+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x5568baba41d0 0x5568babb42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.424+0000 7fe5fe7bb740 4 client.7551 mds_command: new command op to 6364 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:25:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.424+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1818880524 --> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568baac7f40 con 0x5568baba41d0 2024-04-24T23:25:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.424+0000 7fe5f9040640 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x5568baba41d0 0x5568babb42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.424+0000 7fe5f9040640 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x5568baba41d0 0x5568babb42b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b0a9201f9c70108 server_cookie=42be53922fc92943 in_seq=0 out_seq=0 2024-04-24T23:25:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.424+0000 7fe5d4ff9640 10 client.7551 ms_handle_connect on v2:172.21.15.29:6833/2057165440 2024-04-24T23:25:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.430+0000 7fe5d4ff9640 1 -- 172.21.15.26:0/1818880524 <== mds.2 v2:172.21.15.29:6833/2057165440 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568baac7f40 con 0x5568baba41d0 2024-04-24T23:25:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.430+0000 7fe5d4ff9640 10 client.7551 handle_command_reply: tid=0 2024-04-24T23:25:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 client.7551 shutdown 2024-04-24T23:25:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1818880524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 msgr2=0x5568babc5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568babc5b10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00098c0 tx=0x7fe5f0004730 comp rx=0 tx=0).stop 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1818880524 shutdown_connections 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568babb6020 0x5568bab5ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:6833/2057165440,v1:172.21.15.29:6835/2057165440] conn(0x5568baba41d0 0x5568babb42b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5568baba4600 0x5568bab5a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5d4ff9640 0 client.7551 ms_handle_reset on v2:172.21.15.29:6833/2057165440 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 --2- 172.21.15.26:0/1818880524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568bab86d60 0x5568babc5b10 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1818880524 >> 172.21.15.26:0/1818880524 conn(0x5568bab5ca40 msgr2=0x5568bac149e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1818880524 shutdown_connections 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 1 -- 172.21.15.26:0/1818880524 wait complete. 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fe7bb740 20 client.7551 trim_cache size 0 max 16384 2024-04-24T23:25:59.433 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:25:59.434 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:25:59.434 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:25:59.434 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe5d806be30 msgr2=0x7fe5d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fbaa5640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe5d806be30 0x7fe5d806e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0036000 tx=0x7fe5f0030000 comp rx=0 tx=0).stop 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415e9c0 msgr2=0x7fe5f409d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fbaa5640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415e9c0 0x7fe5f409d6c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00c700 tx=0x7fe5ec00cbc0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/2607786056 shutdown_connections 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fbaa5640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe5d806be30 0x7fe5d806e2f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fbaa5640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5f415e9c0 0x7fe5f409d6c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fbaa5640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5f415adc0 0x7fe5f4150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.431+0000 7fe5fbaa5640 1 --2- 172.21.15.26:0/2607786056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe5f40ad7b0 0x7fe5f415dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.432+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/2607786056 >> 172.21.15.26:0/2607786056 conn(0x7fe5f406b540 msgr2=0x7fe5f406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.432+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/2607786056 shutdown_connections 2024-04-24T23:25:59.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.432+0000 7fe5fbaa5640 1 -- 172.21.15.26:0/2607786056 wait complete. 2024-04-24T23:25:59.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:25:59.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.543+0000 7fc75a91d640 1 Processor -- start 2024-04-24T23:25:59.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.543+0000 7fc75a91d640 1 -- start start 2024-04-24T23:25:59.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc75a91d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7540af9e0 0x7fc7540b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc75a91d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7540b3780 con 0x7fc75415b1e0 2024-04-24T23:25:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc75a91d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7540b33e0 con 0x7fc7540aed80 2024-04-24T23:25:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc75a91d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc7540b3190 con 0x7fc7540af9e0 2024-04-24T23:25:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc7537fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7540af9e0 0x7fc7540b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc753fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc75415b1e0 0x7fc7540b0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56792/0 (socket says 172.21.15.26:56792) 2024-04-24T23:25:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc753fff640 1 -- 172.21.15.26:0/4019573316 learned_addr learned my addr 172.21.15.26:0/4019573316 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc7537fe640 1 -- 172.21.15.26:0/4019573316 >> v1:172.21.15.29:6789/0 conn(0x7fc7540aed80 legacy=0x7fc75415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc7537fe640 1 -- 172.21.15.26:0/4019573316 >> v1:172.21.15.26:6789/0 conn(0x7fc75415b1e0 legacy=0x7fc7540b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc7537fe640 1 -- 172.21.15.26:0/4019573316 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7540688f0 con 0x7fc7540af9e0 2024-04-24T23:25:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.544+0000 7fc752ffd640 1 -- 172.21.15.26:0/4019573316 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 108111201 0 0) 0x7fc7540b3780 con 0x7fc75415b1e0 2024-04-24T23:25:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.545+0000 7fc7537fe640 1 --2- 172.21.15.26:0/4019573316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540af9e0 0x7fc7540b1620 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc7440097a0 tx=0x7fc74402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b86b9dae60e2ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.545+0000 7fc752ffd640 1 -- 172.21.15.26:0/4019573316 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744037070 con 0x7fc7540af9e0 2024-04-24T23:25:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.545+0000 7fc752ffd640 1 -- 172.21.15.26:0/4019573316 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc744032440 con 0x7fc7540af9e0 2024-04-24T23:25:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.545+0000 7fc752ffd640 1 -- 172.21.15.26:0/4019573316 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74402ae60 con 0x7fc7540af9e0 2024-04-24T23:25:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.546+0000 7fc75a91d640 1 -- 172.21.15.26:0/4019573316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540af9e0 msgr2=0x7fc7540b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.546+0000 7fc75a91d640 1 --2- 172.21.15.26:0/4019573316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540af9e0 0x7fc7540b1620 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc7440097a0 tx=0x7fc74402a110 comp rx=0 tx=0).stop 2024-04-24T23:25:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.546+0000 7fc75a91d640 1 -- 172.21.15.26:0/4019573316 shutdown_connections 2024-04-24T23:25:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.546+0000 7fc75a91d640 1 --2- 172.21.15.26:0/4019573316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7540af9e0 0x7fc7540b1620 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.546+0000 7fc75a91d640 1 -- 172.21.15.26:0/4019573316 >> 172.21.15.26:0/4019573316 conn(0x7fc754072790 msgr2=0x7fc754072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.546+0000 7fc75a91d640 1 -- 172.21.15.26:0/4019573316 shutdown_connections 2024-04-24T23:25:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.546+0000 7fc75a91d640 1 -- 172.21.15.26:0/4019573316 wait complete. 2024-04-24T23:25:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.547+0000 7fc75a91d640 1 Processor -- start 2024-04-24T23:25:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.547+0000 7fc75a91d640 1 -- start start 2024-04-24T23:25:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc75a91d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7540aed80 0x7fc7541f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc75a91d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7540af9e0 0x7fc754151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc75a91d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc75415b1e0 0x7fc754152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc75a91d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7540ae8f0 con 0x7fc75415b1e0 2024-04-24T23:25:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc75a91d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc7540b3a60 con 0x7fc7540af9e0 2024-04-24T23:25:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc75a91d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc75415bcd0 con 0x7fc7540aed80 2024-04-24T23:25:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc758eba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc75415b1e0 0x7fc754152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc758eba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc75415b1e0 0x7fc754152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50124/0 (socket says 172.21.15.26:50124) 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc758eba640 1 -- 172.21.15.26:0/2697072262 learned_addr learned my addr 172.21.15.26:0/2697072262 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc7537fe640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7540af9e0 0x7fc754151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc753fff640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7540aed80 0x7fc7541f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc758eba640 1 -- 172.21.15.26:0/2697072262 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7540aed80 msgr2=0x7fc7541f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc758eba640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7540aed80 0x7fc7541f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc758eba640 1 -- 172.21.15.26:0/2697072262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7540af9e0 msgr2=0x7fc754151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc758eba640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7540af9e0 0x7fc754151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc758eba640 1 -- 172.21.15.26:0/2697072262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75409b100 con 0x7fc75415b1e0 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.548+0000 7fc753fff640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7540aed80 0x7fc7541f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.549+0000 7fc758eba640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc75415b1e0 0x7fc754152500 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00c7a0 tx=0x7fc74c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.549+0000 7fc750ff9640 1 -- 172.21.15.26:0/2697072262 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c003b70 con 0x7fc75415b1e0 2024-04-24T23:25:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.549+0000 7fc750ff9640 1 -- 172.21.15.26:0/2697072262 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74c003d30 con 0x7fc75415b1e0 2024-04-24T23:25:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.549+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75409baf0 con 0x7fc75415b1e0 2024-04-24T23:25:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.549+0000 7fc750ff9640 1 -- 172.21.15.26:0/2697072262 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00aa90 con 0x7fc75415b1e0 2024-04-24T23:25:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.549+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75409c0b0 con 0x7fc75415b1e0 2024-04-24T23:25:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.550+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7540694c0 con 0x7fc75415b1e0 2024-04-24T23:25:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.550+0000 7fc750ff9640 1 -- 172.21.15.26:0/2697072262 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc74c00ac30 con 0x7fc75415b1e0 2024-04-24T23:25:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.551+0000 7fc750ff9640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc73406be30 0x7fc73406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.551+0000 7fc753fff640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc73406be30 0x7fc73406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.551+0000 7fc750ff9640 1 -- 172.21.15.26:0/2697072262 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fc74c08ae40 con 0x7fc75415b1e0 2024-04-24T23:25:59.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.551+0000 7fc753fff640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc73406be30 0x7fc73406e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc73c007fd0 tx=0x7fc73c007dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.556+0000 7fc750ff9640 1 -- 172.21.15.26:0/2697072262 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc74c059710 con 0x7fc75415b1e0 2024-04-24T23:25:59.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:25:59.633 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:25:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.783+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7540b4be0 con 0x7fc75415b1e0 2024-04-24T23:25:59.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.783+0000 7fc750ff9640 1 -- 172.21.15.26:0/2697072262 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5225 (secure 0 0 0) 0x7fc74c059010 con 0x7fc75415b1e0 2024-04-24T23:25:59.784 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:25:59.784 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:25:57.966897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7227,"mds_1":7224,"mds_2":6364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6364":{"gid":6364,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7224":{"gid":7224,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[6364,7224,7227]},"id":5}]} 2024-04-24T23:25:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 74 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc73406be30 msgr2=0x7fc73406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc73406be30 0x7fc73406e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc73c007fd0 tx=0x7fc73c007dc0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc75415b1e0 msgr2=0x7fc754152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc75415b1e0 0x7fc754152500 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00c7a0 tx=0x7fc74c00cc60 comp rx=0 tx=0).stop 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 shutdown_connections 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc73406be30 0x7fc73406e2f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc75415b1e0 0x7fc754152500 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7540af9e0 0x7fc754151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 --2- 172.21.15.26:0/2697072262 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc7540aed80 0x7fc7541f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.785+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 >> 172.21.15.26:0/2697072262 conn(0x7fc754072790 msgr2=0x7fc754074140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.786+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 shutdown_connections 2024-04-24T23:25:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.786+0000 7fc75a91d640 1 -- 172.21.15.26:0/2697072262 wait complete. 2024-04-24T23:25:59.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub start /d1/d2/d3 2024-04-24T23:25:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.900+0000 7f22932bd640 1 Processor -- start 2024-04-24T23:25:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.900+0000 7f22932bd640 1 -- start start 2024-04-24T23:25:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.901+0000 7f22932bd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f228c15e120 0x7f228c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.901+0000 7f22932bd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228c070f30 con 0x7f228c15e500 2024-04-24T23:25:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.901+0000 7f22932bd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228c06dcf0 con 0x7f228c150e60 2024-04-24T23:25:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.901+0000 7f22932bd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f228c06dfd0 con 0x7f228c15e120 2024-04-24T23:25:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.901+0000 7f2290858640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f228c15e500 0x7f228c0b37e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56804/0 (socket says 172.21.15.26:56804) 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.901+0000 7f2290858640 1 -- 172.21.15.26:0/63562838 learned_addr learned my addr 172.21.15.26:0/63562838 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.901+0000 7f2291059640 1 --2- 172.21.15.26:0/63562838 >> v2:172.21.15.29:3300/0 conn(0x7f228c15e120 0x7f228c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.901+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2259615758 0 0) 0x7f228c070f30 con 0x7f228c15e500 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2278003520 con 0x7f228c15e500 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3828728448 0 0) 0x7f2278003520 con 0x7f228c15e500 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 >> v2:172.21.15.29:3300/0 conn(0x7f228c15e120 msgr2=0x7f228c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 --2- 172.21.15.26:0/63562838 >> v2:172.21.15.29:3300/0 conn(0x7f228c15e120 0x7f228c06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 >> v1:172.21.15.29:6789/0 conn(0x7f228c150e60 legacy=0x7f228c1512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c06e2b0 con 0x7f228c15e500 2024-04-24T23:25:59.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f227c001670 con 0x7f228c15e500 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 >> v1:172.21.15.26:6789/0 conn(0x7f228c15e500 legacy=0x7f228c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22780039b0 0x7f2278003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2278004800 0x7f2278004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22780051c0 0x7f227800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f228c06dfd0 con 0x7f2278004800 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f227800c060 con 0x7f22780039b0 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f227800c260 con 0x7f22780051c0 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2290858640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2278004800 0x7f2278004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2291059640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22780039b0 0x7f2278003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f229185a640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22780051c0 0x7f227800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2290858640 1 -- 172.21.15.26:0/63562838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22780051c0 msgr2=0x7f227800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2290858640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22780051c0 0x7f227800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2290858640 1 -- 172.21.15.26:0/63562838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22780039b0 msgr2=0x7f2278003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2290858640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22780039b0 0x7f2278003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f2290858640 1 -- 172.21.15.26:0/63562838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2278003520 con 0x7f2278004800 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f229185a640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22780051c0 0x7f227800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.902+0000 7f229185a640 1 -- 172.21.15.26:0/63562838 reap_dead start 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.903+0000 7f2290858640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2278004800 0x7f2278004c80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c240 tx=0x7f2278005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2c884f62a272dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.903+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c007d10 con 0x7f2278004800 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.903+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227c005ad0 con 0x7f2278004800 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.903+0000 7f2283fff640 1 -- 172.21.15.26:0/63562838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c007070 con 0x7f2278004800 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.904+0000 7f22932bd640 1 -- 172.21.15.26:0/63562838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2278004800 msgr2=0x7f2278004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.904+0000 7f22932bd640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2278004800 0x7f2278004c80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c240 tx=0x7f2278005830 comp rx=0 tx=0).stop 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.904+0000 7f22932bd640 1 -- 172.21.15.26:0/63562838 shutdown_connections 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.904+0000 7f22932bd640 1 --2- 172.21.15.26:0/63562838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2278004800 0x7f2278004c80 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.904+0000 7f22932bd640 1 -- 172.21.15.26:0/63562838 >> 172.21.15.26:0/63562838 conn(0x7f228c06b540 msgr2=0x7f228c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.904+0000 7f22932bd640 1 -- 172.21.15.26:0/63562838 shutdown_connections 2024-04-24T23:25:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.904+0000 7f22932bd640 1 -- 172.21.15.26:0/63562838 wait complete. 2024-04-24T23:25:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.905+0000 7f22932bd640 1 Processor -- start 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.905+0000 7f22932bd640 1 -- start start 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f22932bd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2278004800 0x7f228c0cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f22932bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f228c14aef0 0x7f228c0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f22932bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f228c0ccd10 0x7f228c1492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f22932bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f228c06d5a0 con 0x7f228c14aef0 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f22932bd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f228c15e970 con 0x7f228c0ccd10 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f22932bd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f228c06d720 con 0x7f2278004800 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2290858640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f228c14aef0 0x7f228c0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2290858640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f228c14aef0 0x7f228c0b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50144/0 (socket says 172.21.15.26:50144) 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2290858640 1 -- 172.21.15.26:0/4146823961 learned_addr learned my addr 172.21.15.26:0/4146823961 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2291059640 1 --2- 172.21.15.26:0/4146823961 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2278004800 0x7f228c0cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f229185a640 1 --2- 172.21.15.26:0/4146823961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f228c0ccd10 0x7f228c1492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2290858640 1 -- 172.21.15.26:0/4146823961 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2278004800 msgr2=0x7f228c0cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2290858640 1 --2- 172.21.15.26:0/4146823961 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2278004800 0x7f228c0cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2290858640 1 -- 172.21.15.26:0/4146823961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f228c0ccd10 msgr2=0x7f228c1492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2290858640 1 --2- 172.21.15.26:0/4146823961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f228c0ccd10 0x7f228c1492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2290858640 1 -- 172.21.15.26:0/4146823961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c0ce7e0 con 0x7f228c14aef0 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2290858640 1 --2- 172.21.15.26:0/4146823961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f228c14aef0 0x7f228c0b2060 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f227c034040 tx=0x7f228c06dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2281ffb640 1 -- 172.21.15.26:0/4146823961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c041070 con 0x7f228c14aef0 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2281ffb640 1 -- 172.21.15.26:0/4146823961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227c004da0 con 0x7f228c14aef0 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f22932bd640 1 -- 172.21.15.26:0/4146823961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c0cc230 con 0x7f228c14aef0 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.906+0000 7f2281ffb640 1 -- 172.21.15.26:0/4146823961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c007d10 con 0x7f228c14aef0 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.907+0000 7f22932bd640 1 -- 172.21.15.26:0/4146823961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c0cc7f0 con 0x7f228c14aef0 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.908+0000 7f2281ffb640 1 -- 172.21.15.26:0/4146823961 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f227c044070 con 0x7f228c14aef0 2024-04-24T23:25:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.909+0000 7f2281ffb640 1 --2- 172.21.15.26:0/4146823961 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f227006be30 0x7f227006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.909+0000 7f2281ffb640 1 -- 172.21.15.26:0/4146823961 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f227c0ab550 con 0x7f228c14aef0 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.909+0000 7f2291059640 1 --2- 172.21.15.26:0/4146823961 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f227006be30 0x7f227006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.909+0000 7f2291059640 1 --2- 172.21.15.26:0/4146823961 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f227006be30 0x7f227006e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f228800a230 tx=0x7f228800c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.910+0000 7f2295fd5740 1 Processor -- start 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.910+0000 7f2295fd5740 1 -- start start 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.910+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69e030 0x556f2a69e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.910+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a698ec0 0x556f2a69cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.910+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a69d170 0x556f2a69d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.910+0000 7f2295fd5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556f2a601220 con 0x556f2a698ec0 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.910+0000 7f2295fd5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556f2a5a98b0 con 0x556f2a69e030 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.910+0000 7f2295fd5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556f2a5fd1e0 con 0x556f2a69d170 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2290858640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a698ec0 0x556f2a69cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2290858640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a698ec0 0x556f2a69cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50150/0 (socket says 172.21.15.26:50150) 2024-04-24T23:25:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2290858640 1 -- 172.21.15.26:0/2094998731 learned_addr learned my addr 172.21.15.26:0/2094998731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f229185a640 1 --2- 172.21.15.26:0/2094998731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a69d170 0x556f2a69d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2291059640 1 --2- 172.21.15.26:0/2094998731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69e030 0x556f2a69e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2290858640 1 -- 172.21.15.26:0/2094998731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a69d170 msgr2=0x556f2a69d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2290858640 1 --2- 172.21.15.26:0/2094998731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a69d170 0x556f2a69d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2290858640 1 -- 172.21.15.26:0/2094998731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69e030 msgr2=0x556f2a69e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2290858640 1 --2- 172.21.15.26:0/2094998731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69e030 0x556f2a69e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2290858640 1 -- 172.21.15.26:0/2094998731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2a3e1690 con 0x556f2a698ec0 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2290858640 1 --2- 172.21.15.26:0/2094998731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a698ec0 0x556f2a69cc30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f227c081fe0 tx=0x7f227c079350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0a5e980969dd99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2276ffd640 1 -- 172.21.15.26:0/2094998731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c07dce0 con 0x556f2a698ec0 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2276ffd640 1 -- 172.21.15.26:0/2094998731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227c082510 con 0x556f2a698ec0 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.911+0000 7f2276ffd640 1 -- 172.21.15.26:0/2094998731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c075fa0 con 0x556f2a698ec0 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 -- 172.21.15.26:0/2094998731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a698ec0 msgr2=0x556f2a69cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2094998731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a698ec0 0x556f2a69cc30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f227c081fe0 tx=0x7f227c079350 comp rx=0 tx=0).stop 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 -- 172.21.15.26:0/2094998731 shutdown_connections 2024-04-24T23:25:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2094998731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a69d170 0x556f2a69d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2094998731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a698ec0 0x556f2a69cc30 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2094998731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69e030 0x556f2a69e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 -- 172.21.15.26:0/2094998731 >> 172.21.15.26:0/2094998731 conn(0x556f2a69b130 msgr2=0x556f2a6e3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 -- 172.21.15.26:0/2094998731 shutdown_connections 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 -- 172.21.15.26:0/2094998731 wait complete. 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.912+0000 7f2295fd5740 1 -- start start 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.913+0000 7f2295fd5740 1 Processor -- start 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.913+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 0x556f2a6c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.913+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6c8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.913+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 0x556f2a6bb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.913+0000 7f2295fd5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556f2a5fd1e0 con 0x556f2a69e030 2024-04-24T23:25:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.913+0000 7f2295fd5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556f2a5bce40 con 0x556f2a69d170 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.913+0000 7f2295fd5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556f2a601220 con 0x556f2a6e7450 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.913+0000 7f2295fd5740 10 client.0 _ll_register_callbacks cb 0x556f2a5c1080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2290858640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2290858640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50158/0 (socket says 172.21.15.26:50158) 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2290858640 1 -- 172.21.15.26:0/3439858236 learned_addr learned my addr 172.21.15.26:0/3439858236 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2291059640 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 0x556f2a6c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f229185a640 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 0x556f2a6bb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2290858640 1 -- 172.21.15.26:0/3439858236 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 msgr2=0x556f2a6bb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2290858640 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 0x556f2a6bb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2290858640 1 -- 172.21.15.26:0/3439858236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 msgr2=0x556f2a6c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2290858640 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 0x556f2a6c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2290858640 1 -- 172.21.15.26:0/3439858236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2a441e20 con 0x556f2a69e030 2024-04-24T23:25:59.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2290858640 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6c8000 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f227c075d90 tx=0x7f227c0078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf42a4e7a3dc9f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f226bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f226bfff640 1 -- 172.21.15.26:0/3439858236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c082280 con 0x556f2a69e030 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f226bfff640 1 -- 172.21.15.26:0/3439858236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f227c091b90 con 0x556f2a69e030 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2295fd5740 1 -- 172.21.15.26:0/3439858236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f227c082510 con 0x556f2a69e030 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f226bfff640 1 -- 172.21.15.26:0/3439858236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f227c07dce0 con 0x556f2a69e030 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2295fd5740 10 client.7572 fetch_fsmap learned FSMap version 74 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f2295fd5740 1 -- 172.21.15.26:0/3439858236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x556f2a2d4f80 con 0x556f2a69e030 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.914+0000 7f226bfff640 1 -- 172.21.15.26:0/3439858236 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227c082510 con 0x556f2a69e030 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f226bfff640 1 -- 172.21.15.26:0/3439858236 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7f227c04e060 con 0x556f2a69e030 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f2295fd5740 10 client.7572 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f2295fd5740 10 client.7572 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f2295fd5740 20 client.7572 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f2295fd5740 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x556f2a6f4470 0x556f2a6f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f2295fd5740 4 client.7572 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f2295fd5740 1 -- 172.21.15.26:0/3439858236 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f2a5becc0 con 0x556f2a6f4470 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f2291059640 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x556f2a6f4470 0x556f2a6f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f226bfff640 10 client.7572 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.915+0000 7f2291059640 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x556f2a6f4470 0x556f2a6f4850 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6de570752ce019ca server_cookie=47311452b19766f8 in_seq=0 out_seq=0 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f226bfff640 1 -- 172.21.15.26:0/3439858236 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f2a5becc0 con 0x556f2a6f4470 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f226bfff640 10 client.7572 handle_command_reply: tid=0 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 client.7572 shutdown 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 -- 172.21.15.26:0/3439858236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 msgr2=0x556f2a6c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6c8000 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f227c075d90 tx=0x7f227c0078f0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 -- 172.21.15.26:0/3439858236 shutdown_connections 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x556f2a6f4470 0x556f2a6f4850 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 0x556f2a6bb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6c8000 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 --2- 172.21.15.26:0/3439858236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 0x556f2a6c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f226bfff640 0 client.7572 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:59.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 -- 172.21.15.26:0/3439858236 >> 172.21.15.26:0/3439858236 conn(0x556f2a698ec0 msgr2=0x556f2a6985e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 -- 172.21.15.26:0/3439858236 shutdown_connections 2024-04-24T23:25:59.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 1 -- 172.21.15.26:0/3439858236 wait complete. 2024-04-24T23:25:59.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.916+0000 7f2295fd5740 20 client.7572 trim_cache size 0 max 16384 2024-04-24T23:25:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.934+0000 7f2295fd5740 1 Processor -- start 2024-04-24T23:25:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.934+0000 7f2295fd5740 1 -- start start 2024-04-24T23:25:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a69d170 0x556f2a707ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69e030 0x556f2a6e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a6e7450 0x556f2a6f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f2295fd5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f228c200270 con 0x556f2a6e7450 2024-04-24T23:25:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f2295fd5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f228c200970 con 0x556f2a69e030 2024-04-24T23:25:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f2295fd5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f228c1f7fe0 con 0x556f2a69d170 2024-04-24T23:25:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f229185a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a6e7450 0x556f2a6f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f229185a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a6e7450 0x556f2a6f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50164/0 (socket says 172.21.15.26:50164) 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f229185a640 1 -- 172.21.15.26:0/2762982952 learned_addr learned my addr 172.21.15.26:0/2762982952 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f229185a640 1 -- 172.21.15.26:0/2762982952 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a69d170 msgr2=0x556f2a707ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f229185a640 1 --2- 172.21.15.26:0/2762982952 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a69d170 0x556f2a707ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f229185a640 1 -- 172.21.15.26:0/2762982952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69e030 msgr2=0x556f2a6e62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f229185a640 1 --2- 172.21.15.26:0/2762982952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69e030 0x556f2a6e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f229185a640 1 -- 172.21.15.26:0/2762982952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2a2d4f80 con 0x556f2a6e7450 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f229185a640 1 --2- 172.21.15.26:0/2762982952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a6e7450 0x556f2a6f67e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2284001510 tx=0x7f22840019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598ff2ae0ccc92c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f22767fc640 1 -- 172.21.15.26:0/2762982952 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22840050d0 con 0x556f2a6e7450 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f22767fc640 1 -- 172.21.15.26:0/2762982952 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2284005270 con 0x556f2a6e7450 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.935+0000 7f22767fc640 1 -- 172.21.15.26:0/2762982952 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22840158a0 con 0x556f2a6e7450 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 -- 172.21.15.26:0/2762982952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a6e7450 msgr2=0x556f2a6f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2762982952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a6e7450 0x556f2a6f67e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2284001510 tx=0x7f22840019d0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 -- 172.21.15.26:0/2762982952 shutdown_connections 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2762982952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a6e7450 0x556f2a6f67e0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2762982952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69e030 0x556f2a6e62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2762982952 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a69d170 0x556f2a707ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 -- 172.21.15.26:0/2762982952 >> 172.21.15.26:0/2762982952 conn(0x556f2a698ec0 msgr2=0x556f2a69c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 -- 172.21.15.26:0/2762982952 shutdown_connections 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 -- 172.21.15.26:0/2762982952 wait complete. 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 -- start start 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.936+0000 7f2295fd5740 1 Processor -- start 2024-04-24T23:25:59.943 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.937+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 0x556f2a6f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.937+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.937+0000 7f2295fd5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 0x556f2a6f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.937+0000 7f2295fd5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556f2a441e20 con 0x556f2a69e030 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.937+0000 7f2295fd5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556f2a5a98b0 con 0x556f2a69d170 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.937+0000 7f2295fd5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556f2a6ba900 con 0x556f2a6e7450 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.937+0000 7f2295fd5740 10 client.0 _ll_register_callbacks cb 0x7f228800ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.937+0000 7f2290858640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2290858640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50166/0 (socket says 172.21.15.26:50166) 2024-04-24T23:25:59.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2290858640 1 -- 172.21.15.26:0/2955166797 learned_addr learned my addr 172.21.15.26:0/2955166797 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2291059640 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 0x556f2a6f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f229185a640 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 0x556f2a6f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2290858640 1 -- 172.21.15.26:0/2955166797 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 msgr2=0x556f2a6f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2290858640 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 0x556f2a6f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2290858640 1 -- 172.21.15.26:0/2955166797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 msgr2=0x556f2a6f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2290858640 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 0x556f2a6f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2290858640 1 -- 172.21.15.26:0/2955166797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2a6e36f0 con 0x556f2a69e030 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2290858640 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6f4d90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f227c081fe0 tx=0x7f227c0a9a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd99cf3103961ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f226bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f226bfff640 1 -- 172.21.15.26:0/2955166797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c075bd0 con 0x556f2a69e030 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f226bfff640 1 -- 172.21.15.26:0/2955166797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f227c078d80 con 0x556f2a69e030 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2295fd5740 1 -- 172.21.15.26:0/2955166797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2284005270 con 0x556f2a69e030 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f226bfff640 1 -- 172.21.15.26:0/2955166797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f227c082510 con 0x556f2a69e030 2024-04-24T23:25:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2295fd5740 10 client.7578 fetch_fsmap learned FSMap version 74 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2295fd5740 1 -- 172.21.15.26:0/2955166797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x556f2a6e36f0 con 0x556f2a69e030 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f226bfff640 1 -- 172.21.15.26:0/2955166797 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2284005270 con 0x556f2a69e030 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f226bfff640 1 -- 172.21.15.26:0/2955166797 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7f227c091b90 con 0x556f2a69e030 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2295fd5740 10 client.7578 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.938+0000 7f2295fd5740 10 client.7578 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 20 client.7578 populate_metadata read hostname 'smithi026' 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x556f2a6f2b10 0x556f2a6b8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 4 client.7578 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3"}] 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 -- 172.21.15.26:0/2955166797 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3"}) -- 0x556f2a60e760 con 0x556f2a6f2b10 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2291059640 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x556f2a6f2b10 0x556f2a6b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f226bfff640 10 client.7578 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2291059640 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x556f2a6f2b10 0x556f2a6b8560 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=251a16b01327483b server_cookie=b88ef40b0036f6ae in_seq=0 out_seq=0 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f226bfff640 1 -- 172.21.15.26:0/2955166797 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x556f2a60e760 con 0x556f2a6f2b10 2024-04-24T23:25:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f226bfff640 10 client.7578 handle_command_reply: tid=0 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 client.7578 shutdown 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 -- 172.21.15.26:0/2955166797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 msgr2=0x556f2a6f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6f4d90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f227c081fe0 tx=0x7f227c0a9a80 comp rx=0 tx=0).stop 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 -- 172.21.15.26:0/2955166797 shutdown_connections 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x556f2a6f2b10 0x556f2a6b8560 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556f2a6e7450 0x556f2a6f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f2a69e030 0x556f2a6f4d90 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f226bfff640 0 client.7578 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 --2- 172.21.15.26:0/2955166797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f2a69d170 0x556f2a6f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 -- 172.21.15.26:0/2955166797 >> 172.21.15.26:0/2955166797 conn(0x556f2a698ec0 msgr2=0x556f2a6b9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.939+0000 7f2295fd5740 1 -- 172.21.15.26:0/2955166797 shutdown_connections 2024-04-24T23:25:59.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.940+0000 7f2295fd5740 1 -- 172.21.15.26:0/2955166797 wait complete. 2024-04-24T23:25:59.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:25:59.940+0000 7f2295fd5740 20 client.7578 trim_cache size 0 max 16384 2024-04-24T23:25:59.948 INFO:teuthology.orchestra.run.smithi026.stderr:Error EXDEV: Not rank 0 2024-04-24T23:25:59.948 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub abort 2024-04-24T23:26:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.052+0000 7ffa45fb7640 1 Processor -- start 2024-04-24T23:26:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.052+0000 7ffa45fb7640 1 -- start start 2024-04-24T23:26:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa45fb7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa400ce640 0x7ffa400d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa45fb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffa400d28e0 0x7ffa400cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa45fb7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa4006d750 con 0x7ffa4006c4d0 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa45fb7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ffa400d1110 con 0x7ffa400ce640 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa45fb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ffa400cc410 con 0x7ffa400d28e0 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa3f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa400ce640 0x7ffa400d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa3f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa400ce640 0x7ffa400d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50176/0 (socket says 172.21.15.26:50176) 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2153497292 learned_addr learned my addr 172.21.15.26:0/2153497292 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2153497292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffa400d28e0 msgr2=0x7ffa400cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2153497292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffa400d28e0 0x7ffa400cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2153497292 >> v1:172.21.15.29:6789/0 conn(0x7ffa4006c4d0 legacy=0x7ffa4006c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.053+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2153497292 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa400b1720 con 0x7ffa400ce640 2024-04-24T23:26:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.054+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2153497292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa400ce640 0x7ffa400d00f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009870 tx=0x7ffa3002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c871b5ba20fc493 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.054+0000 7ffa3e7fc640 1 -- 172.21.15.26:0/2153497292 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30037070 con 0x7ffa400ce640 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.054+0000 7ffa3e7fc640 1 -- 172.21.15.26:0/2153497292 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa30032440 con 0x7ffa400ce640 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.054+0000 7ffa3e7fc640 1 -- 172.21.15.26:0/2153497292 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30032810 con 0x7ffa400ce640 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.055+0000 7ffa45fb7640 1 -- 172.21.15.26:0/2153497292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa400ce640 msgr2=0x7ffa400d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.055+0000 7ffa45fb7640 1 --2- 172.21.15.26:0/2153497292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa400ce640 0x7ffa400d00f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009870 tx=0x7ffa3002a2a0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.055+0000 7ffa45fb7640 1 -- 172.21.15.26:0/2153497292 shutdown_connections 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.055+0000 7ffa45fb7640 1 --2- 172.21.15.26:0/2153497292 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffa400d28e0 0x7ffa400cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.055+0000 7ffa45fb7640 1 --2- 172.21.15.26:0/2153497292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa400ce640 0x7ffa400d00f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.055+0000 7ffa45fb7640 1 -- 172.21.15.26:0/2153497292 >> 172.21.15.26:0/2153497292 conn(0x7ffa400d1dc0 msgr2=0x7ffa400d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.055+0000 7ffa45fb7640 1 -- 172.21.15.26:0/2153497292 shutdown_connections 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.055+0000 7ffa45fb7640 1 -- 172.21.15.26:0/2153497292 wait complete. 2024-04-24T23:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.056+0000 7ffa45fb7640 1 Processor -- start 2024-04-24T23:26:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.056+0000 7ffa45fb7640 1 -- start start 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa45fb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa4006c4d0 0x7ffa401523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa45fb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffa400ce640 0x7ffa400b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa45fb7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa400d28e0 0x7ffa4015cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa45fb7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa400cc660 con 0x7ffa4006c4d0 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa45fb7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffa400d1360 con 0x7ffa400d28e0 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa45fb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ffa400cc940 con 0x7ffa400ce640 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa4006c4d0 0x7ffa401523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa4006c4d0 0x7ffa401523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50190/0 (socket says 172.21.15.26:50190) 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/3413032357 learned_addr learned my addr 172.21.15.26:0/3413032357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3effd640 1 --2- 172.21.15.26:0/3413032357 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffa400ce640 0x7ffa400b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3ffff640 1 --2- 172.21.15.26:0/3413032357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa400d28e0 0x7ffa4015cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/3413032357 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffa400ce640 msgr2=0x7ffa400b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/3413032357 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffa400ce640 0x7ffa400b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/3413032357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa400d28e0 msgr2=0x7ffa4015cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/3413032357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa400d28e0 0x7ffa4015cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/3413032357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4015ec40 con 0x7ffa4006c4d0 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3ffff640 1 --2- 172.21.15.26:0/3413032357 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffa400d28e0 0x7ffa4015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.057+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/3413032357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa4006c4d0 0x7ffa401523f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009870 tx=0x7ffa300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.058+0000 7ffa27fff640 1 -- 172.21.15.26:0/3413032357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30041070 con 0x7ffa4006c4d0 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.058+0000 7ffa27fff640 1 -- 172.21.15.26:0/3413032357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3003c870 con 0x7ffa4006c4d0 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.058+0000 7ffa45fb7640 1 -- 172.21.15.26:0/3413032357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4015ef20 con 0x7ffa4006c4d0 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.058+0000 7ffa27fff640 1 -- 172.21.15.26:0/3413032357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30037040 con 0x7ffa4006c4d0 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.058+0000 7ffa45fb7640 1 -- 172.21.15.26:0/3413032357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4015f4e0 con 0x7ffa4006c4d0 2024-04-24T23:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.059+0000 7ffa27fff640 1 -- 172.21.15.26:0/3413032357 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa30044070 con 0x7ffa4006c4d0 2024-04-24T23:26:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.059+0000 7ffa27fff640 1 --2- 172.21.15.26:0/3413032357 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffa2c06be30 0x7ffa2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.060+0000 7ffa27fff640 1 -- 172.21.15.26:0/3413032357 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7ffa300ab960 con 0x7ffa4006c4d0 2024-04-24T23:26:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.060+0000 7ffa3effd640 1 --2- 172.21.15.26:0/3413032357 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffa2c06be30 0x7ffa2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.060+0000 7ffa3effd640 1 --2- 172.21.15.26:0/3413032357 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffa2c06be30 0x7ffa2c06e2f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffa4015d900 tx=0x7ffa34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.060+0000 7ffa48da7740 1 Processor -- start 2024-04-24T23:26:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.061+0000 7ffa48da7740 1 -- start start 2024-04-24T23:26:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.061+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e2f8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.061+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f2be0 0x55f61e2f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.061+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e2f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.061+0000 7ffa48da7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f61e25b1b0 con 0x55f61e2f7d50 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.061+0000 7ffa48da7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f61e2038d0 con 0x55f61e2f2be0 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.061+0000 7ffa48da7740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f61e257170 con 0x55f61e2f6e90 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.061+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e2f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.061+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e2f8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50194/0 (socket says 172.21.15.26:50194) 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2086649992 learned_addr learned my addr 172.21.15.26:0/2086649992 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa3effd640 1 --2- 172.21.15.26:0/2086649992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f2be0 0x55f61e2f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa3ffff640 1 --2- 172.21.15.26:0/2086649992 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e2f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2086649992 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 msgr2=0x55f61e2f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2086649992 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e2f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2086649992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f2be0 msgr2=0x55f61e2f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2086649992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f2be0 0x55f61e2f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2086649992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61e039500 con 0x55f61e2f7d50 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2086649992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e2f8130 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ffa300a9e80 tx=0x7ffa300795c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ad6013284a43fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa257fa640 1 -- 172.21.15.26:0/2086649992 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3004b040 con 0x55f61e2f7d50 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa257fa640 1 -- 172.21.15.26:0/2086649992 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3007e010 con 0x55f61e2f7d50 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.062+0000 7ffa257fa640 1 -- 172.21.15.26:0/2086649992 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30082840 con 0x55f61e2f7d50 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 -- 172.21.15.26:0/2086649992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 msgr2=0x55f61e2f8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2086649992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e2f8130 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ffa300a9e80 tx=0x7ffa300795c0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 -- 172.21.15.26:0/2086649992 shutdown_connections 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2086649992 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e2f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2086649992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f2be0 0x55f61e2f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2086649992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e2f8130 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 -- 172.21.15.26:0/2086649992 >> 172.21.15.26:0/2086649992 conn(0x55f61e2f4e50 msgr2=0x55f61e33d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 -- 172.21.15.26:0/2086649992 shutdown_connections 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 -- 172.21.15.26:0/2086649992 wait complete. 2024-04-24T23:26:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 -- start start 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.063+0000 7ffa48da7740 1 Processor -- start 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f6e90 0x55f61e313450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 0x55f61e340780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e3246c0 0x55f61e324b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa48da7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f61e25b1b0 con 0x55f61e2f6e90 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa48da7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f61e216d90 con 0x55f61e2f7d50 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa48da7740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f61e3238e0 con 0x55f61e3246c0 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa48da7740 10 client.0 _ll_register_callbacks cb 0x55f61df6e910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f6e90 0x55f61e313450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f6e90 0x55f61e313450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50198/0 (socket says 172.21.15.26:50198) 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2912616248 learned_addr learned my addr 172.21.15.26:0/2912616248 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa3ffff640 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e3246c0 0x55f61e324b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.064+0000 7ffa3effd640 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 0x55f61e340780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2912616248 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e3246c0 msgr2=0x55f61e324b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e3246c0 0x55f61e324b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2912616248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 msgr2=0x55f61e340780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 0x55f61e340780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa3f7fe640 1 -- 172.21.15.26:0/2912616248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61e099c90 con 0x55f61e2f6e90 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f6e90 0x55f61e313450 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ffa30032fd0 tx=0x7ffa30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f283436b458b584 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2912616248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30096060 con 0x55f61e2f6e90 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2912616248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7ffa300823b0 con 0x55f61e2f6e90 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa48da7740 1 -- 172.21.15.26:0/2912616248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3007e010 con 0x55f61e2f6e90 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2912616248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7ffa3004b040 con 0x55f61e2f6e90 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa48da7740 10 client.7590 fetch_fsmap learned FSMap version 74 2024-04-24T23:26:00.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa48da7740 1 -- 172.21.15.26:0/2912616248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55f61df2ee20 con 0x55f61e2f6e90 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2912616248 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3007e010 con 0x55f61e2f6e90 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2912616248 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7ffa300790b0 con 0x55f61e2f6e90 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa48da7740 10 client.7590 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa48da7740 10 client.7590 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa48da7740 20 client.7590 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55f61e33f5e0 0x55f61e34da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa48da7740 4 client.7590 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.065+0000 7ffa48da7740 1 -- 172.21.15.26:0/2912616248 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61e2686f0 con 0x55f61e33f5e0 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.066+0000 7ffa3effd640 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55f61e33f5e0 0x55f61e34da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.066+0000 7ffa1e7fc640 10 client.7590 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.066+0000 7ffa3effd640 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55f61e33f5e0 0x55f61e34da80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d63cb670ff12f68 server_cookie=df5401560afb1c13 in_seq=0 out_seq=0 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.066+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2912616248 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f61e2686f0 con 0x55f61e33f5e0 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.066+0000 7ffa1e7fc640 10 client.7590 handle_command_reply: tid=0 2024-04-24T23:26:00.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.066+0000 7ffa48da7740 1 client.7590 shutdown 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 -- 172.21.15.26:0/2912616248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f6e90 msgr2=0x55f61e313450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f6e90 0x55f61e313450 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ffa30032fd0 tx=0x7ffa30000bc0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 -- 172.21.15.26:0/2912616248 shutdown_connections 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55f61e33f5e0 0x55f61e34da80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e3246c0 0x55f61e324b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 0x55f61e340780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2912616248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f6e90 0x55f61e313450 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa1e7fc640 0 client.7590 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 -- 172.21.15.26:0/2912616248 >> 172.21.15.26:0/2912616248 conn(0x55f61e2f2be0 msgr2=0x55f61e323180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 -- 172.21.15.26:0/2912616248 shutdown_connections 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 1 -- 172.21.15.26:0/2912616248 wait complete. 2024-04-24T23:26:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.067+0000 7ffa48da7740 20 client.7590 trim_cache size 0 max 16384 2024-04-24T23:26:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.084+0000 7ffa48da7740 1 Processor -- start 2024-04-24T23:26:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.084+0000 7ffa48da7740 1 -- start start 2024-04-24T23:26:00.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e323110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e3577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e3246c0 0x55f61e2f8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa48da7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401ffcf0 con 0x55f61e2f7d50 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa48da7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffa402003f0 con 0x55f61e3246c0 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa48da7740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ffa400b6930 con 0x55f61e2f6e90 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e3577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e3577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50204/0 (socket says 172.21.15.26:50204) 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3effd640 1 -- 172.21.15.26:0/2649378143 learned_addr learned my addr 172.21.15.26:0/2649378143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e323110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3ffff640 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e3246c0 0x55f61e2f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3effd640 1 -- 172.21.15.26:0/2649378143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 msgr2=0x55f61e323110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3effd640 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e323110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3effd640 1 -- 172.21.15.26:0/2649378143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e3246c0 msgr2=0x55f61e2f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3effd640 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e3246c0 0x55f61e2f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3effd640 1 -- 172.21.15.26:0/2649378143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61e2ee3c0 con 0x55f61e2f7d50 2024-04-24T23:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.085+0000 7ffa3ffff640 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e3246c0 0x55f61e2f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:00.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.086+0000 7ffa3effd640 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e3577c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ffa400c8dd0 tx=0x7ffa34013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc284bc2a26dcfa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa24ff9640 1 -- 172.21.15.26:0/2649378143 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34031070 con 0x55f61e2f7d50 2024-04-24T23:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa24ff9640 1 -- 172.21.15.26:0/2649378143 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa34014430 con 0x55f61e2f7d50 2024-04-24T23:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa24ff9640 1 -- 172.21.15.26:0/2649378143 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340145e0 con 0x55f61e2f7d50 2024-04-24T23:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa48da7740 1 -- 172.21.15.26:0/2649378143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 msgr2=0x55f61e3577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e3577c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ffa400c8dd0 tx=0x7ffa34013000 comp rx=0 tx=0).stop 2024-04-24T23:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa48da7740 1 -- 172.21.15.26:0/2649378143 shutdown_connections 2024-04-24T23:26:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e3246c0 0x55f61e2f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e2f7d50 0x55f61e3577c0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2649378143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e323110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.087+0000 7ffa48da7740 1 -- 172.21.15.26:0/2649378143 >> 172.21.15.26:0/2649378143 conn(0x55f61e2f2be0 msgr2=0x55f61e3412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.088+0000 7ffa48da7740 1 -- 172.21.15.26:0/2649378143 shutdown_connections 2024-04-24T23:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.088+0000 7ffa48da7740 1 -- 172.21.15.26:0/2649378143 wait complete. 2024-04-24T23:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.088+0000 7ffa48da7740 1 -- start start 2024-04-24T23:26:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.088+0000 7ffa48da7740 1 Processor -- start 2024-04-24T23:26:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e323370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 0x55f61e3191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa48da7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e3246c0 0x55f61e319710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa48da7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f61e25b1b0 con 0x55f61e3246c0 2024-04-24T23:26:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa48da7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f61e3238e0 con 0x55f61e2f7d50 2024-04-24T23:26:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa48da7740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f61e216d90 con 0x55f61e2f6e90 2024-04-24T23:26:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa48da7740 10 client.0 _ll_register_callbacks cb 0x7ffa30079ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e3246c0 0x55f61e319710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e3246c0 0x55f61e319710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50210/0 (socket says 172.21.15.26:50210) 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3ffff640 1 -- 172.21.15.26:0/2343286597 learned_addr learned my addr 172.21.15.26:0/2343286597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e323370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3effd640 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 0x55f61e3191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3ffff640 1 -- 172.21.15.26:0/2343286597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 msgr2=0x55f61e323370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3ffff640 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e323370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3ffff640 1 -- 172.21.15.26:0/2343286597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 msgr2=0x55f61e3191d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3ffff640 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 0x55f61e3191d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3ffff640 1 -- 172.21.15.26:0/2343286597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61e3226f0 con 0x55f61e3246c0 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa3ffff640 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e3246c0 0x55f61e319710 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ffa400c8e00 tx=0x7ffa38005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29686f3a69567db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.089+0000 7ffa1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2343286597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38006de0 con 0x55f61e3246c0 2024-04-24T23:26:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2343286597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7ffa38010040 con 0x55f61e3246c0 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa48da7740 1 -- 172.21.15.26:0/2343286597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa34014430 con 0x55f61e3246c0 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2343286597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7ffa380164e0 con 0x55f61e3246c0 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa48da7740 10 client.7596 fetch_fsmap learned FSMap version 74 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa48da7740 1 -- 172.21.15.26:0/2343286597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55f61e3b7c60 con 0x55f61e3246c0 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2343286597 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa34014430 con 0x55f61e3246c0 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2343286597 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7ffa38016680 con 0x55f61e3246c0 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa48da7740 10 client.7596 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa48da7740 10 client.7596 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa48da7740 20 client.7596 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55f61e3b6a60 0x55f61e3b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa48da7740 4 client.7596 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa48da7740 1 -- 172.21.15.26:0/2343286597 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x55f61e2f43f0 con 0x55f61e3b6a60 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.090+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55f61e3b6a60 0x55f61e3b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa1e7fc640 10 client.7596 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa3f7fe640 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55f61e3b6a60 0x55f61e3b6e40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcb9545a63bf806f server_cookie=fd1ff7fe8728c0d8 in_seq=0 out_seq=0 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa1e7fc640 1 -- 172.21.15.26:0/2343286597 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x55f61e2f43f0 con 0x55f61e3b6a60 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa1e7fc640 10 client.7596 handle_command_reply: tid=0 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 client.7596 shutdown 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 -- 172.21.15.26:0/2343286597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e3246c0 msgr2=0x55f61e319710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e3246c0 0x55f61e319710 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ffa400c8e00 tx=0x7ffa38005420 comp rx=0 tx=0).stop 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 -- 172.21.15.26:0/2343286597 shutdown_connections 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55f61e3b6a60 0x55f61e3b6e40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f61e3246c0 0x55f61e319710 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f61e2f7d50 0x55f61e3191d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa1e7fc640 0 client.7596 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 --2- 172.21.15.26:0/2343286597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f61e2f6e90 0x55f61e323370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 -- 172.21.15.26:0/2343286597 >> 172.21.15.26:0/2343286597 conn(0x55f61e2f2be0 msgr2=0x55f61e3412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 -- 172.21.15.26:0/2343286597 shutdown_connections 2024-04-24T23:26:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 1 -- 172.21.15.26:0/2343286597 wait complete. 2024-04-24T23:26:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.091+0000 7ffa48da7740 20 client.7596 trim_cache size 0 max 16384 2024-04-24T23:26:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:Error EXDEV: Not rank 0 2024-04-24T23:26:00.098 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-04-24T23:26:00.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub pause 2024-04-24T23:26:00.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.200+0000 7fc23d5d5640 1 Processor -- start 2024-04-24T23:26:00.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.200+0000 7fc23d5d5640 1 -- start start 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.201+0000 7fc23d5d5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2380b4ce0 0x7fc2380acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.201+0000 7fc23d5d5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2380aeb10 con 0x7fc23815be40 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.201+0000 7fc23d5d5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2380aedf0 con 0x7fc2380d0750 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.201+0000 7fc23d5d5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc23809df20 con 0x7fc2380b4ce0 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.201+0000 7fc236ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc23815be40 0x7fc2380d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56812/0 (socket says 172.21.15.26:56812) 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.201+0000 7fc236ffd640 1 -- 172.21.15.26:0/1709931475 learned_addr learned my addr 172.21.15.26:0/1709931475 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.201+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665915928 0 0) 0x7fc2380aeb10 con 0x7fc23815be40 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.201+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc220003520 con 0x7fc23815be40 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.201+0000 7fc2377fe640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2380b4ce0 0x7fc2380acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3383008682 0 0) 0x7fc220003520 con 0x7fc23815be40 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2380b4ce0 msgr2=0x7fc2380acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2380b4ce0 0x7fc2380acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 >> v1:172.21.15.29:6789/0 conn(0x7fc2380d0750 legacy=0x7fc23815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2380af0d0 con 0x7fc23815be40 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fc228001670 con 0x7fc23815be40 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 >> v1:172.21.15.26:6789/0 conn(0x7fc23815be40 legacy=0x7fc2380d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2380b4ce0 0x7fc2200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc220003f60 0x7fc220004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc220004d40 0x7fc2200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc22000ba70 con 0x7fc220004d40 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc22000bc20 con 0x7fc220003f60 2024-04-24T23:26:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc22000be20 con 0x7fc2380b4ce0 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc2377fe640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc220004d40 0x7fc2200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc236ffd640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2380b4ce0 0x7fc2200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc2367fc640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc220003f60 0x7fc220004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc2377fe640 1 -- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2380b4ce0 msgr2=0x7fc2200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc2377fe640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc2380b4ce0 0x7fc2200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc2377fe640 1 -- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc220003f60 msgr2=0x7fc220004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc2377fe640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc220003f60 0x7fc220004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.202+0000 7fc2377fe640 1 -- 172.21.15.26:0/1709931475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc220003520 con 0x7fc220004d40 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.203+0000 7fc2377fe640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc220004d40 0x7fc2200051c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc230009fc0 tx=0x7fc220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb4e8593536029 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.203+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230009c10 con 0x7fc220004d40 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.203+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc230036070 con 0x7fc220004d40 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.203+0000 7fc235ffb640 1 -- 172.21.15.26:0/1709931475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230031880 con 0x7fc220004d40 2024-04-24T23:26:00.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.204+0000 7fc23d5d5640 1 -- 172.21.15.26:0/1709931475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc220004d40 msgr2=0x7fc2200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.204+0000 7fc23d5d5640 1 --2- 172.21.15.26:0/1709931475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc220004d40 0x7fc2200051c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc230009fc0 tx=0x7fc220005830 comp rx=0 tx=0).stop 2024-04-24T23:26:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.204+0000 7fc2377fe640 1 -- 172.21.15.26:0/1709931475 reap_dead start 2024-04-24T23:26:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.204+0000 7fc23d5d5640 1 -- 172.21.15.26:0/1709931475 shutdown_connections 2024-04-24T23:26:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.204+0000 7fc23d5d5640 1 -- 172.21.15.26:0/1709931475 >> 172.21.15.26:0/1709931475 conn(0x7fc23806b540 msgr2=0x7fc23806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.204+0000 7fc23d5d5640 1 -- 172.21.15.26:0/1709931475 shutdown_connections 2024-04-24T23:26:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.204+0000 7fc23d5d5640 1 -- 172.21.15.26:0/1709931475 wait complete. 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.205+0000 7fc23d5d5640 1 Processor -- start 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.205+0000 7fc23d5d5640 1 -- start start 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc23d5d5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc23814f8c0 0x7fc2381f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc23d5d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc23814fca0 0x7fc2380b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc23d5d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2380b1490 0x7fc23815e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc23d5d5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc23806cd10 con 0x7fc23814fca0 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc23d5d5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc23806cb90 con 0x7fc2380b1490 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc23d5d5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc2380acd30 con 0x7fc23814f8c0 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc23814fca0 0x7fc2380b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc23814fca0 0x7fc2380b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50234/0 (socket says 172.21.15.26:50234) 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2367fc640 1 -- 172.21.15.26:0/1165547146 learned_addr learned my addr 172.21.15.26:0/1165547146 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2367fc640 1 -- 172.21.15.26:0/1165547146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc23814f8c0 msgr2=0x7fc2381f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2377fe640 1 --2- 172.21.15.26:0/1165547146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2380b1490 0x7fc23815e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc236ffd640 1 --2- 172.21.15.26:0/1165547146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc23814f8c0 0x7fc2381f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2367fc640 1 --2- 172.21.15.26:0/1165547146 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc23814f8c0 0x7fc2381f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2367fc640 1 -- 172.21.15.26:0/1165547146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2380b1490 msgr2=0x7fc23815e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2367fc640 1 --2- 172.21.15.26:0/1165547146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2380b1490 0x7fc23815e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2367fc640 1 -- 172.21.15.26:0/1165547146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2381f3c70 con 0x7fc23814fca0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc2367fc640 1 --2- 172.21.15.26:0/1165547146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc23814fca0 0x7fc2380b0f50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00f820 tx=0x7fc22c00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc21f7fe640 1 -- 172.21.15.26:0/1165547146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0050d0 con 0x7fc23814fca0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc21f7fe640 1 -- 172.21.15.26:0/1165547146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c018070 con 0x7fc23814fca0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc21f7fe640 1 -- 172.21.15.26:0/1165547146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c005490 con 0x7fc23814fca0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.206+0000 7fc23d5d5640 1 -- 172.21.15.26:0/1165547146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2381f4c30 con 0x7fc23814fca0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.207+0000 7fc23d5d5640 1 -- 172.21.15.26:0/1165547146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2380b1c80 con 0x7fc23814fca0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.208+0000 7fc21f7fe640 1 -- 172.21.15.26:0/1165547146 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc22c00d730 con 0x7fc23814fca0 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.208+0000 7fc21f7fe640 1 --2- 172.21.15.26:0/1165547146 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc22406be60 0x7fc22406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.208+0000 7fc21f7fe640 1 -- 172.21.15.26:0/1165547146 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fc22c08ae20 con 0x7fc23814fca0 2024-04-24T23:26:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.208+0000 7fc236ffd640 1 --2- 172.21.15.26:0/1165547146 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc22406be60 0x7fc22406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.209+0000 7fc236ffd640 1 --2- 172.21.15.26:0/1165547146 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc22406be60 0x7fc22406e320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc228007c70 tx=0x7fc228009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.209+0000 7fc2403c4740 1 Processor -- start 2024-04-24T23:26:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc2403c4740 1 -- start start 2024-04-24T23:26:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e65296be0 0x560e6529a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 0x560e6529b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc2403c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560e651ff1b0 con 0x560e6529bd50 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc2403c4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560e651a78d0 con 0x560e6529ae90 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc2403c4740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560e651fb170 con 0x560e65296be0 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc236ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc236ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50246/0 (socket says 172.21.15.26:50246) 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.210+0000 7fc236ffd640 1 -- 172.21.15.26:0/2616386503 learned_addr learned my addr 172.21.15.26:0/2616386503 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc2377fe640 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 0x560e6529b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc2367fc640 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e65296be0 0x560e6529a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc236ffd640 1 -- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e65296be0 msgr2=0x560e6529a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc236ffd640 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e65296be0 0x560e6529a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc236ffd640 1 -- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 msgr2=0x560e6529b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc236ffd640 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 0x560e6529b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc236ffd640 1 -- 172.21.15.26:0/2616386503 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e64fdd500 con 0x560e6529bd50 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc2367fc640 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e65296be0 0x560e6529a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc2377fe640 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 0x560e6529b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc236ffd640 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529c130 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc228007d20 tx=0x7fc228013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43bf686c87f1e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc21cff9640 1 -- 172.21.15.26:0/2616386503 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228026070 con 0x560e6529bd50 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc21cff9640 1 -- 172.21.15.26:0/2616386503 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22801c050 con 0x560e6529bd50 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.211+0000 7fc21cff9640 1 -- 172.21.15.26:0/2616386503 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228021410 con 0x560e6529bd50 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 -- 172.21.15.26:0/2616386503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 msgr2=0x560e6529c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529c130 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc228007d20 tx=0x7fc228013000 comp rx=0 tx=0).stop 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 -- 172.21.15.26:0/2616386503 shutdown_connections 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 0x560e6529b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e65296be0 0x560e6529a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 --2- 172.21.15.26:0/2616386503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529c130 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 -- 172.21.15.26:0/2616386503 >> 172.21.15.26:0/2616386503 conn(0x560e65298e50 msgr2=0x560e652e1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 -- 172.21.15.26:0/2616386503 shutdown_connections 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 -- 172.21.15.26:0/2616386503 wait complete. 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 -- start start 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.212+0000 7fc2403c4740 1 Processor -- start 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529ae90 0x560e652b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529bd50 0x560e652f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e652e9a70 0x560e652f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc2403c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560e652c78e0 con 0x560e6529ae90 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc2403c4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560e651bad90 con 0x560e6529bd50 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc2403c4740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560e652c6d60 con 0x560e652e9a70 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc2403c4740 10 client.0 _ll_register_callbacks cb 0x560e64f12910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc236ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529ae90 0x560e652b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc236ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529ae90 0x560e652b8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50260/0 (socket says 172.21.15.26:50260) 2024-04-24T23:26:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc236ffd640 1 -- 172.21.15.26:0/160719023 learned_addr learned my addr 172.21.15.26:0/160719023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc2377fe640 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e652e9a70 0x560e652f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.213+0000 7fc2367fc640 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529bd50 0x560e652f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc236ffd640 1 -- 172.21.15.26:0/160719023 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e652e9a70 msgr2=0x560e652f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc236ffd640 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e652e9a70 0x560e652f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc236ffd640 1 -- 172.21.15.26:0/160719023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529bd50 msgr2=0x560e652f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc236ffd640 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529bd50 0x560e652f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc236ffd640 1 -- 172.21.15.26:0/160719023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6503dc90 con 0x560e6529ae90 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2367fc640 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529bd50 0x560e652f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc236ffd640 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529ae90 0x560e652b8590 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc2280078c0 tx=0x7fc2380aefc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8183bf61db068713 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc215ffb640 1 -- 172.21.15.26:0/160719023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228030060 con 0x560e6529ae90 2024-04-24T23:26:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc215ffb640 1 -- 172.21.15.26:0/160719023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fc228022820 con 0x560e6529ae90 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2403c4740 1 -- 172.21.15.26:0/160719023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc22801c050 con 0x560e6529ae90 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc215ffb640 1 -- 172.21.15.26:0/160719023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fc228036050 con 0x560e6529ae90 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2403c4740 10 client.7608 fetch_fsmap learned FSMap version 74 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2403c4740 1 -- 172.21.15.26:0/160719023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x560e64ed2e20 con 0x560e6529ae90 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc215ffb640 1 -- 172.21.15.26:0/160719023 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22801c050 con 0x560e6529ae90 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc215ffb640 1 -- 172.21.15.26:0/160719023 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7fc228039020 con 0x560e6529ae90 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2403c4740 10 client.7608 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2403c4740 10 client.7608 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2403c4740 20 client.7608 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2403c4740 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x560e652b6e90 0x560e652b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2403c4740 4 client.7608 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.214+0000 7fc2403c4740 1 -- 172.21.15.26:0/160719023 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e6520c6f0 con 0x560e652b6e90 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.215+0000 7fc2367fc640 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x560e652b6e90 0x560e652b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.215+0000 7fc2367fc640 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x560e652b6e90 0x560e652b7270 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f19e2073c2694d1 server_cookie=ca87f8b41f956044 in_seq=0 out_seq=0 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.215+0000 7fc215ffb640 10 client.7608 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.215+0000 7fc215ffb640 1 -- 172.21.15.26:0/160719023 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e6520c6f0 con 0x560e652b6e90 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.215+0000 7fc215ffb640 10 client.7608 handle_command_reply: tid=0 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.215+0000 7fc2403c4740 1 client.7608 shutdown 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.215+0000 7fc2403c4740 1 -- 172.21.15.26:0/160719023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529ae90 msgr2=0x560e652b8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.215+0000 7fc2403c4740 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529ae90 0x560e652b8590 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc2280078c0 tx=0x7fc2380aefc0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc2403c4740 1 -- 172.21.15.26:0/160719023 shutdown_connections 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc2403c4740 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x560e652b6e90 0x560e652b7270 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc2403c4740 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e652e9a70 0x560e652f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc215ffb640 0 client.7608 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc2403c4740 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529bd50 0x560e652f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc2403c4740 1 --2- 172.21.15.26:0/160719023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529ae90 0x560e652b8590 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc2403c4740 1 -- 172.21.15.26:0/160719023 >> 172.21.15.26:0/160719023 conn(0x560e65296be0 msgr2=0x560e6529c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc2403c4740 1 -- 172.21.15.26:0/160719023 shutdown_connections 2024-04-24T23:26:00.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc2403c4740 1 -- 172.21.15.26:0/160719023 wait complete. 2024-04-24T23:26:00.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.216+0000 7fc2403c4740 20 client.7608 trim_cache size 0 max 16384 2024-04-24T23:26:00.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.233+0000 7fc2403c4740 1 Processor -- start 2024-04-24T23:26:00.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.233+0000 7fc2403c4740 1 -- start start 2024-04-24T23:26:00.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 0x560e652b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e652e9a70 0x560e6529c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2403c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc238200500 con 0x560e6529bd50 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2403c4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc238200c00 con 0x560e6529ae90 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2403c4740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc2381ff5a0 con 0x560e652e9a70 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50272/0 (socket says 172.21.15.26:50272) 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2367fc640 1 -- 172.21.15.26:0/2960863148 learned_addr learned my addr 172.21.15.26:0/2960863148 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2367fc640 1 -- 172.21.15.26:0/2960863148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e652e9a70 msgr2=0x560e6529c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2367fc640 1 --2- 172.21.15.26:0/2960863148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e652e9a70 0x560e6529c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2367fc640 1 -- 172.21.15.26:0/2960863148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 msgr2=0x560e652b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2367fc640 1 --2- 172.21.15.26:0/2960863148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 0x560e652b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2367fc640 1 -- 172.21.15.26:0/2960863148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e652b73b0 con 0x560e6529bd50 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc2367fc640 1 --2- 172.21.15.26:0/2960863148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529ca30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fc22c055450 tx=0x7fc22c001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cb0f58218eda94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.234+0000 7fc217fff640 1 -- 172.21.15.26:0/2960863148 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c061bd0 con 0x560e6529bd50 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc217fff640 1 -- 172.21.15.26:0/2960863148 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c012e30 con 0x560e6529bd50 2024-04-24T23:26:00.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc217fff640 1 -- 172.21.15.26:0/2960863148 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0572f0 con 0x560e6529bd50 2024-04-24T23:26:00.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc2403c4740 1 -- 172.21.15.26:0/2960863148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 msgr2=0x560e6529ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc2403c4740 1 --2- 172.21.15.26:0/2960863148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529ca30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fc22c055450 tx=0x7fc22c001190 comp rx=0 tx=0).stop 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc2403c4740 1 -- 172.21.15.26:0/2960863148 shutdown_connections 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc2403c4740 1 --2- 172.21.15.26:0/2960863148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e652e9a70 0x560e6529c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc2403c4740 1 --2- 172.21.15.26:0/2960863148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e6529ca30 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc2403c4740 1 --2- 172.21.15.26:0/2960863148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e6529ae90 0x560e652b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc2403c4740 1 -- 172.21.15.26:0/2960863148 >> 172.21.15.26:0/2960863148 conn(0x560e65296be0 msgr2=0x560e652bdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc2403c4740 1 -- 172.21.15.26:0/2960863148 shutdown_connections 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.235+0000 7fc2403c4740 1 -- 172.21.15.26:0/2960863148 wait complete. 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.236+0000 7fc2403c4740 1 -- start start 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.236+0000 7fc2403c4740 1 Processor -- start 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.236+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e6529ae90 0x560e6535bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.236+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e652c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.236+0000 7fc2403c4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e652e9a70 0x560e652c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.236+0000 7fc2403c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560e652e8f60 con 0x560e6529bd50 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.236+0000 7fc2403c4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560e652c78e0 con 0x560e652e9a70 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.236+0000 7fc2403c4740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560e652b2b90 con 0x560e6529ae90 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.236+0000 7fc2403c4740 10 client.0 _ll_register_callbacks cb 0x7fc228024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e652c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e652c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50276/0 (socket says 172.21.15.26:50276) 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2367fc640 1 -- 172.21.15.26:0/1395701716 learned_addr learned my addr 172.21.15.26:0/1395701716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2377fe640 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e652e9a70 0x560e652c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc236ffd640 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e6529ae90 0x560e6535bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2367fc640 1 -- 172.21.15.26:0/1395701716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e6529ae90 msgr2=0x560e6535bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2367fc640 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e6529ae90 0x560e6535bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2367fc640 1 -- 172.21.15.26:0/1395701716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e652e9a70 msgr2=0x560e652c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2367fc640 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e652e9a70 0x560e652c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2367fc640 1 -- 172.21.15.26:0/1395701716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e652e85b0 con 0x560e6529bd50 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2367fc640 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e652c6cc0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d520 tx=0x7fc22c070040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20393ff89c494baf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc215ffb640 1 -- 172.21.15.26:0/1395701716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c080060 con 0x560e6529bd50 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc215ffb640 1 -- 172.21.15.26:0/1395701716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fc22c055660 con 0x560e6529bd50 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2403c4740 1 -- 172.21.15.26:0/1395701716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc22c012e30 con 0x560e6529bd50 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc215ffb640 1 -- 172.21.15.26:0/1395701716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fc22c061bd0 con 0x560e6529bd50 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2403c4740 10 client.7614 fetch_fsmap learned FSMap version 74 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc2403c4740 1 -- 172.21.15.26:0/1395701716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x560e652e8330 con 0x560e6529bd50 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.237+0000 7fc215ffb640 1 -- 172.21.15.26:0/1395701716 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c012e30 con 0x560e6529bd50 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc215ffb640 1 -- 172.21.15.26:0/1395701716 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7fc22c061740 con 0x560e6529bd50 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 10 client.7614 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:26:00.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 10 client.7614 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 20 client.7614 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x560e6535ef80 0x560e652f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 4 client.7614 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 -- 172.21.15.26:0/1395701716 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x560e65206a40 con 0x560e6535ef80 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc236ffd640 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x560e6535ef80 0x560e652f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc215ffb640 10 client.7614 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc236ffd640 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x560e6535ef80 0x560e652f3ee0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4a34a4be63b7683 server_cookie=4da5bb3e40ba302a in_seq=0 out_seq=0 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc215ffb640 1 -- 172.21.15.26:0/1395701716 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x560e65206a40 con 0x560e6535ef80 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc215ffb640 10 client.7614 handle_command_reply: tid=0 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 client.7614 shutdown 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 -- 172.21.15.26:0/1395701716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 msgr2=0x560e652c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e652c6cc0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d520 tx=0x7fc22c070040 comp rx=0 tx=0).stop 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 -- 172.21.15.26:0/1395701716 shutdown_connections 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x560e6535ef80 0x560e652f3ee0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560e652e9a70 0x560e652c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc215ffb640 0 client.7614 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e6529bd50 0x560e652c6cc0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 --2- 172.21.15.26:0/1395701716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560e6529ae90 0x560e6535bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 -- 172.21.15.26:0/1395701716 >> 172.21.15.26:0/1395701716 conn(0x560e65296be0 msgr2=0x560e6529c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 -- 172.21.15.26:0/1395701716 shutdown_connections 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.238+0000 7fc2403c4740 1 -- 172.21.15.26:0/1395701716 wait complete. 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.239+0000 7fc2403c4740 20 client.7614 trim_cache size 0 max 16384 2024-04-24T23:26:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:Error EXDEV: Not rank 0 2024-04-24T23:26:00.247 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-04-24T23:26:00.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub resume 2024-04-24T23:26:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.349+0000 7f2f58c1c640 1 Processor -- start 2024-04-24T23:26:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.349+0000 7f2f58c1c640 1 -- start start 2024-04-24T23:26:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f58c1c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f540d0750 0x7f2f540acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f58c1c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2f540b4ce0 0x7f2f5415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f58c1c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f540aeb10 con 0x7f2f5415be40 2024-04-24T23:26:00.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f58c1c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2f5409df20 con 0x7f2f540b4ce0 2024-04-24T23:26:00.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f58c1c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2f540aedf0 con 0x7f2f540d0750 2024-04-24T23:26:00.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2f5415be40 0x7f2f540d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56814/0 (socket says 172.21.15.26:56814) 2024-04-24T23:26:00.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5259c640 1 -- 172.21.15.26:0/3635911124 learned_addr learned my addr 172.21.15.26:0/3635911124 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:00.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871623323 0 0) 0x7f2f540aeb10 con 0x7f2f5415be40 2024-04-24T23:26:00.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f540d0750 0x7f2f540acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/3635911124 >> v2:172.21.15.29:3300/0 conn(0x7f2f540b4ce0 0x7f2f5415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f3c003520 con 0x7f2f5415be40 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2621983047 0 0) 0x7f2f3c003520 con 0x7f2f5415be40 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f540d0750 msgr2=0x7f2f540acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f540d0750 0x7f2f540acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 >> v2:172.21.15.29:3300/0 conn(0x7f2f540b4ce0 msgr2=0x7f2f5415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 --2- 172.21.15.26:0/3635911124 >> v2:172.21.15.29:3300/0 conn(0x7f2f540b4ce0 0x7f2f5415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f540af0d0 con 0x7f2f5415be40 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f2f4c001670 con 0x7f2f5415be40 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 >> v1:172.21.15.26:6789/0 conn(0x7f2f5415be40 legacy=0x7f2f540d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f540d0750 0x7f2f3c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c003f30 0x7f2f3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3c004d40 0x7f2f3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2f540aedf0 con 0x7f2f3c004d40 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c00b9c0 con 0x7f2f3c003f30 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.350+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c00bb40 con 0x7f2f540d0750 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.351+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3c004d40 0x7f2f3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.351+0000 7f2f5259c640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f540d0750 0x7f2f3c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.351+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c003f30 0x7f2f3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.351+0000 7f2f52d9d640 1 -- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f540d0750 msgr2=0x7f2f3c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.351+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f540d0750 0x7f2f3c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.351+0000 7f2f52d9d640 1 -- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c003f30 msgr2=0x7f2f3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.351+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c003f30 0x7f2f3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.351+0000 7f2f52d9d640 1 -- 172.21.15.26:0/3635911124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c003520 con 0x7f2f3c004d40 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3c004d40 0x7f2f3c0051c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009fc0 tx=0x7f2f3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93b04fabe9fb357 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44009c10 con 0x7f2f3c004d40 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f44036070 con 0x7f2f3c004d40 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f5159a640 1 -- 172.21.15.26:0/3635911124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4402d070 con 0x7f2f3c004d40 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f58c1c640 1 -- 172.21.15.26:0/3635911124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3c004d40 msgr2=0x7f2f3c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f58c1c640 1 --2- 172.21.15.26:0/3635911124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3c004d40 0x7f2f3c0051c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009fc0 tx=0x7f2f3c005830 comp rx=0 tx=0).stop 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f52d9d640 1 -- 172.21.15.26:0/3635911124 reap_dead start 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f58c1c640 1 -- 172.21.15.26:0/3635911124 shutdown_connections 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f58c1c640 1 -- 172.21.15.26:0/3635911124 >> 172.21.15.26:0/3635911124 conn(0x7f2f5406b540 msgr2=0x7f2f5406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f58c1c640 1 -- 172.21.15.26:0/3635911124 shutdown_connections 2024-04-24T23:26:00.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.352+0000 7f2f58c1c640 1 -- 172.21.15.26:0/3635911124 wait complete. 2024-04-24T23:26:00.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.354+0000 7f2f58c1c640 1 Processor -- start 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.354+0000 7f2f58c1c640 1 -- start start 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.354+0000 7f2f58c1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c004d40 0x7f2f54152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.354+0000 7f2f58c1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5409cc70 0x7f2f541554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.354+0000 7f2f58c1c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f5409d0d0 0x7f2f541559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.354+0000 7f2f58c1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5406cd10 con 0x7f2f5409cc70 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.354+0000 7f2f58c1c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5406cb90 con 0x7f2f3c004d40 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.354+0000 7f2f58c1c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2f540acd30 con 0x7f2f5409d0d0 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f51d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5409cc70 0x7f2f541554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f51d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5409cc70 0x7f2f541554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50290/0 (socket says 172.21.15.26:50290) 2024-04-24T23:26:00.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f51d9b640 1 -- 172.21.15.26:0/548169990 learned_addr learned my addr 172.21.15.26:0/548169990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f51d9b640 1 -- 172.21.15.26:0/548169990 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f5409d0d0 msgr2=0x7f2f541559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/548169990 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2f5409d0d0 0x7f2f541559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f51d9b640 1 -- 172.21.15.26:0/548169990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c004d40 msgr2=0x7f2f54152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/548169990 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f3c004d40 0x7f2f54152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f51d9b640 1 -- 172.21.15.26:0/548169990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5409bbd0 con 0x7f2f5409cc70 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/548169990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5409cc70 0x7f2f541554b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2f48011910 tx=0x7f2f48011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f42ffd640 1 -- 172.21.15.26:0/548169990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4800fa70 con 0x7f2f5409cc70 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f42ffd640 1 -- 172.21.15.26:0/548169990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4800fc10 con 0x7f2f5409cc70 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f58c1c640 1 -- 172.21.15.26:0/548169990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5409beb0 con 0x7f2f5409cc70 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f42ffd640 1 -- 172.21.15.26:0/548169990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48014440 con 0x7f2f5409cc70 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.355+0000 7f2f58c1c640 1 -- 172.21.15.26:0/548169990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5409c470 con 0x7f2f5409cc70 2024-04-24T23:26:00.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.357+0000 7f2f42ffd640 1 -- 172.21.15.26:0/548169990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f48012040 con 0x7f2f5409cc70 2024-04-24T23:26:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.358+0000 7f2f42ffd640 1 --2- 172.21.15.26:0/548169990 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2f3406be60 0x7f2f3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.358+0000 7f2f42ffd640 1 -- 172.21.15.26:0/548169990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f2f4808b3e0 con 0x7f2f5409cc70 2024-04-24T23:26:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.358+0000 7f2f5259c640 1 --2- 172.21.15.26:0/548169990 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2f3406be60 0x7f2f3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.358+0000 7f2f5259c640 1 --2- 172.21.15.26:0/548169990 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2f3406be60 0x7f2f3406e320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c001ba0 tx=0x7f2f4c001b10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.358+0000 7f2f5ba08740 1 Processor -- start 2024-04-24T23:26:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.359+0000 7f2f5ba08740 1 -- start start 2024-04-24T23:26:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.359+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c0701c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.359+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06d5b80 0x55a4c06fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.359+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c0701020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.359+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a4c063a330 con 0x55a4c06d5b80 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.359+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a4c05e2ab0 con 0x55a4c06fe790 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.359+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a4c06362f0 con 0x55a4c06fde70 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f51d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06d5b80 0x55a4c06fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f51d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06d5b80 0x55a4c06fe250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50292/0 (socket says 172.21.15.26:50292) 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f51d9b640 1 -- 172.21.15.26:0/2531740909 learned_addr learned my addr 172.21.15.26:0/2531740909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/2531740909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c0701020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f5259c640 1 --2- 172.21.15.26:0/2531740909 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c0701c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f51d9b640 1 -- 172.21.15.26:0/2531740909 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 msgr2=0x55a4c0701c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/2531740909 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c0701c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f51d9b640 1 -- 172.21.15.26:0/2531740909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 msgr2=0x55a4c0701020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/2531740909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c0701020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f51d9b640 1 -- 172.21.15.26:0/2531740909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c04187d0 con 0x55a4c06d5b80 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/2531740909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06d5b80 0x55a4c06fe250 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2f480898d0 tx=0x7f2f480595e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8a518d25b8c679 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f33fff640 1 -- 172.21.15.26:0/2531740909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480622c0 con 0x55a4c06d5b80 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f33fff640 1 -- 172.21.15.26:0/2531740909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f480579e0 con 0x55a4c06d5b80 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.360+0000 7f2f33fff640 1 -- 172.21.15.26:0/2531740909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48055d50 con 0x55a4c06d5b80 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2531740909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06d5b80 msgr2=0x55a4c06fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/2531740909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06d5b80 0x55a4c06fe250 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2f480898d0 tx=0x7f2f480595e0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2531740909 shutdown_connections 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/2531740909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c0701020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/2531740909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06d5b80 0x55a4c06fe250 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/2531740909 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c0701c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2531740909 >> 172.21.15.26:0/2531740909 conn(0x55a4c06d3780 msgr2=0x55a4c071c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2531740909 shutdown_connections 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2531740909 wait complete. 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.361+0000 7f2f5ba08740 1 -- start start 2024-04-24T23:26:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.362+0000 7f2f5ba08740 1 Processor -- start 2024-04-24T23:26:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.362+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c071dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.362+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c071e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.362+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c06f6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.362+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a4c06d1ad0 con 0x55a4c06f3f60 2024-04-24T23:26:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.362+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a4c05f5f40 con 0x55a4c06fe790 2024-04-24T23:26:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.362+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a4c06d1510 con 0x55a4c06fde70 2024-04-24T23:26:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f5ba08740 10 client.0 _ll_register_callbacks cb 0x55a4c034dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f52d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c06f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f52d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c06f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50296/0 (socket says 172.21.15.26:50296) 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f52d9d640 1 -- 172.21.15.26:0/1462042922 learned_addr learned my addr 172.21.15.26:0/1462042922 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f5259c640 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c071dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c071e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f52d9d640 1 -- 172.21.15.26:0/1462042922 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 msgr2=0x55a4c071dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c071dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f52d9d640 1 -- 172.21.15.26:0/1462042922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 msgr2=0x55a4c071e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c071e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f52d9d640 1 -- 172.21.15.26:0/1462042922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c030e0f0 con 0x55a4c06f3f60 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c06f6800 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400e200 tx=0x7f2f44009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af39739f0cecf40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f317fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f317fa640 1 -- 172.21.15.26:0/1462042922 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44031cb0 con 0x55a4c06f3f60 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f317fa640 1 -- 172.21.15.26:0/1462042922 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f2f44003990 con 0x55a4c06f3f60 2024-04-24T23:26:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1462042922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f480579e0 con 0x55a4c06f3f60 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.363+0000 7f2f317fa640 1 -- 172.21.15.26:0/1462042922 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f2f4404b050 con 0x55a4c06f3f60 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5ba08740 10 client.7626 fetch_fsmap learned FSMap version 74 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1462042922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a4c0478f60 con 0x55a4c06f3f60 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f317fa640 1 -- 172.21.15.26:0/1462042922 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f480579e0 con 0x55a4c06f3f60 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f317fa640 1 -- 172.21.15.26:0/1462042922 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7f2f4404e020 con 0x55a4c06f3f60 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5ba08740 10 client.7626 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5ba08740 10 client.7626 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5ba08740 20 client.7626 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55a4c07130a0 0x55a4c0719ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5ba08740 4 client.7626 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1462042922 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4c0647870 con 0x55a4c07130a0 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5259c640 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55a4c07130a0 0x55a4c0719ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f317fa640 10 client.7626 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.364+0000 7f2f5259c640 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55a4c07130a0 0x55a4c0719ec0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37f9279fad4d10f7 server_cookie=27a20abce0563bf0 in_seq=0 out_seq=0 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f317fa640 1 -- 172.21.15.26:0/1462042922 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4c0647870 con 0x55a4c07130a0 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f317fa640 10 client.7626 handle_command_reply: tid=0 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 client.7626 shutdown 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1462042922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 msgr2=0x55a4c06f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c06f6800 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400e200 tx=0x7f2f44009750 comp rx=0 tx=0).stop 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1462042922 shutdown_connections 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55a4c07130a0 0x55a4c0719ec0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c06f6800 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c071e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f317fa640 0 client.7626 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1462042922 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c071dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1462042922 >> 172.21.15.26:0/1462042922 conn(0x55a4c06d5b80 msgr2=0x55a4c06d6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1462042922 shutdown_connections 2024-04-24T23:26:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1462042922 wait complete. 2024-04-24T23:26:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.365+0000 7f2f5ba08740 20 client.7626 trim_cache size 0 max 16384 2024-04-24T23:26:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.383+0000 7f2f5ba08740 1 Processor -- start 2024-04-24T23:26:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.383+0000 7f2f5ba08740 1 -- start start 2024-04-24T23:26:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.383+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c0718970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.383+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c06d6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.383+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c06f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.383+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541fca60 con 0x55a4c06f3f60 2024-04-24T23:26:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.383+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f540723b0 con 0x55a4c06fe790 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.383+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2f54072ab0 con 0x55a4c06fde70 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f5259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c0718970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f5259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c0718970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50300/0 (socket says 172.21.15.26:50300) 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f5259c640 1 -- 172.21.15.26:0/2926485976 learned_addr learned my addr 172.21.15.26:0/2926485976 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/2926485976 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c06d6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/2926485976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c06f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f5259c640 1 -- 172.21.15.26:0/2926485976 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 msgr2=0x55a4c06d6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f5259c640 1 --2- 172.21.15.26:0/2926485976 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c06d6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f5259c640 1 -- 172.21.15.26:0/2926485976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 msgr2=0x55a4c06f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f5259c640 1 --2- 172.21.15.26:0/2926485976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c06f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f5259c640 1 -- 172.21.15.26:0/2926485976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c02e71b0 con 0x55a4c06f3f60 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f5259c640 1 --2- 172.21.15.26:0/2926485976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c0718970 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c007430 tx=0x7f2f4c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb71abadf9dc6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f337fe640 1 -- 172.21.15.26:0/2926485976 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c019040 con 0x55a4c06f3f60 2024-04-24T23:26:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f337fe640 1 -- 172.21.15.26:0/2926485976 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c02b020 con 0x55a4c06f3f60 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.384+0000 7f2f337fe640 1 -- 172.21.15.26:0/2926485976 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c004cf0 con 0x55a4c06f3f60 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2926485976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 msgr2=0x55a4c0718970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/2926485976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c0718970 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c007430 tx=0x7f2f4c013000 comp rx=0 tx=0).stop 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2926485976 shutdown_connections 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/2926485976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c06f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/2926485976 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c06d6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/2926485976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c0718970 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2926485976 >> 172.21.15.26:0/2926485976 conn(0x55a4c06d5b80 msgr2=0x55a4c071c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2926485976 shutdown_connections 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 -- 172.21.15.26:0/2926485976 wait complete. 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 -- start start 2024-04-24T23:26:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.385+0000 7f2f5ba08740 1 Processor -- start 2024-04-24T23:26:00.392 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c079a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c071a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5ba08740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c071a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a4c06f6060 con 0x55a4c06f3f60 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541426a0 con 0x55a4c06fe790 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5ba08740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a4c0713560 con 0x55a4c06fde70 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5ba08740 10 client.0 _ll_register_callbacks cb 0x7f2f3c0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c079a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c079a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50310/0 (socket says 172.21.15.26:50310) 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.386+0000 7f2f5259c640 1 -- 172.21.15.26:0/1037380058 learned_addr learned my addr 172.21.15.26:0/1037380058 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c071a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f52d9d640 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c071a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5259c640 1 -- 172.21.15.26:0/1037380058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 msgr2=0x55a4c071a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5259c640 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c071a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5259c640 1 -- 172.21.15.26:0/1037380058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 msgr2=0x55a4c071a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5259c640 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c071a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5259c640 1 -- 172.21.15.26:0/1037380058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c06f4fb0 con 0x55a4c06f3f60 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5259c640 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c079a340 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c001ce0 tx=0x7f2f4c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b119b4c158f00749 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f317fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f317fa640 1 -- 172.21.15.26:0/1037380058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c019040 con 0x55a4c06f3f60 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f317fa640 1 -- 172.21.15.26:0/1037380058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f2f4c02c040 con 0x55a4c06f3f60 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1037380058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4c02b020 con 0x55a4c06f3f60 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f317fa640 1 -- 172.21.15.26:0/1037380058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f2f4c03a050 con 0x55a4c06f3f60 2024-04-24T23:26:00.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5ba08740 10 client.7632 fetch_fsmap learned FSMap version 74 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1037380058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a4c071dc40 con 0x55a4c06f3f60 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f317fa640 1 -- 172.21.15.26:0/1037380058 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c02b020 con 0x55a4c06f3f60 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f317fa640 1 -- 172.21.15.26:0/1037380058 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 74) ==== 3546+0+0 (secure 0 0 0) 0x7f2f4c03d020 con 0x55a4c06f3f60 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5ba08740 10 client.7632 fetch_fsmap finished waiting for FSMap version 74 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5ba08740 10 client.7632 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5ba08740 20 client.7632 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55a4c0720120 0x55a4c0743550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5ba08740 4 client.7632 mds_command: new command op to 7224 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.387+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1037380058 --> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x55a4c0641bc0 con 0x55a4c0720120 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55a4c0720120 0x55a4c0743550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f317fa640 10 client.7632 ms_handle_connect on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f51d9b640 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55a4c0720120 0x55a4c0743550 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32d16ed0dc3fb2d7 server_cookie=92fb53db9891f242 in_seq=0 out_seq=0 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f317fa640 1 -- 172.21.15.26:0/1037380058 <== mds.1 v2:172.21.15.26:6834/1431580336 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x55a4c0641bc0 con 0x55a4c0720120 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f317fa640 10 client.7632 handle_command_reply: tid=0 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 client.7632 shutdown 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1037380058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 msgr2=0x55a4c079a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c079a340 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c001ce0 tx=0x7f2f4c012000 comp rx=0 tx=0).stop 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1037380058 shutdown_connections 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.26:6834/1431580336,v1:172.21.15.26:6836/1431580336] conn(0x55a4c0720120 0x55a4c0743550 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a4c06fe790 0x55a4c071a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a4c06fde70 0x55a4c071a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f317fa640 0 client.7632 ms_handle_reset on v2:172.21.15.26:6834/1431580336 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 --2- 172.21.15.26:0/1037380058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a4c06f3f60 0x55a4c079a340 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1037380058 >> 172.21.15.26:0/1037380058 conn(0x55a4c06d5b80 msgr2=0x55a4c06d8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1037380058 shutdown_connections 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 1 -- 172.21.15.26:0/1037380058 wait complete. 2024-04-24T23:26:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.388+0000 7f2f5ba08740 20 client.7632 trim_cache size 0 max 16384 2024-04-24T23:26:00.397 INFO:teuthology.orchestra.run.smithi026.stderr:Error EXDEV: Not rank 0 2024-04-24T23:26:00.397 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:26:00.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:26:00.431 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:26:00.480 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:26:00.480 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:26:00.480 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:26:00.480 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:00.496 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:26:00.496 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:00.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:00.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.692+0000 7fca1951b640 1 Processor -- start 2024-04-24T23:26:00.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.692+0000 7fca1951b640 1 -- start start 2024-04-24T23:26:00.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.692+0000 7fca1951b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fca14145910 0x7fca1406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.692+0000 7fca1951b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fca140cd660 0x7fca140cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.692+0000 7fca1951b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca140cfbc0 con 0x7fca1406d350 2024-04-24T23:26:00.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.692+0000 7fca1951b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fca14145cf0 con 0x7fca14145910 2024-04-24T23:26:00.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.692+0000 7fca1951b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fca1406d7f0 con 0x7fca140cd660 2024-04-24T23:26:00.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca12ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fca14145910 0x7fca1406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca127fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fca1406d350 0x7fca1415d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56818/0 (socket says 172.21.15.26:56818) 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca12ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fca14145910 0x7fca1406e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50318/0 (socket says 172.21.15.26:50318) 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca127fc640 1 -- 172.21.15.26:0/3184010912 learned_addr learned my addr 172.21.15.26:0/3184010912 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca137fe640 1 --2- 172.21.15.26:0/3184010912 >> v2:172.21.15.29:3300/0 conn(0x7fca140cd660 0x7fca140cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca12ffd640 1 -- 172.21.15.26:0/3184010912 >> v2:172.21.15.29:3300/0 conn(0x7fca140cd660 msgr2=0x7fca140cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca12ffd640 1 --2- 172.21.15.26:0/3184010912 >> v2:172.21.15.29:3300/0 conn(0x7fca140cd660 0x7fca140cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca12ffd640 1 -- 172.21.15.26:0/3184010912 >> v1:172.21.15.26:6789/0 conn(0x7fca1406d350 legacy=0x7fca1415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca12ffd640 1 -- 172.21.15.26:0/3184010912 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1406dad0 con 0x7fca14145910 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca11ffb640 1 -- 172.21.15.26:0/3184010912 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1874401846 0 0) 0x7fca140cfbc0 con 0x7fca1406d350 2024-04-24T23:26:00.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.693+0000 7fca137fe640 1 --2- 172.21.15.26:0/3184010912 >> v2:172.21.15.29:3300/0 conn(0x7fca140cd660 0x7fca140cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca12ffd640 1 --2- 172.21.15.26:0/3184010912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 0x7fca1406e890 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fca040097a0 tx=0x7fca0402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab8070304cc52c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca11ffb640 1 -- 172.21.15.26:0/3184010912 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04037070 con 0x7fca14145910 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca11ffb640 1 -- 172.21.15.26:0/3184010912 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0402ae50 con 0x7fca14145910 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca11ffb640 1 -- 172.21.15.26:0/3184010912 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04032660 con 0x7fca14145910 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca1951b640 1 -- 172.21.15.26:0/3184010912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 msgr2=0x7fca1406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca1951b640 1 --2- 172.21.15.26:0/3184010912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 0x7fca1406e890 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fca040097a0 tx=0x7fca0402a140 comp rx=0 tx=0).stop 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca1951b640 1 -- 172.21.15.26:0/3184010912 shutdown_connections 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca1951b640 1 --2- 172.21.15.26:0/3184010912 >> v2:172.21.15.29:3300/0 conn(0x7fca140cd660 0x7fca140cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca1951b640 1 --2- 172.21.15.26:0/3184010912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 0x7fca1406e890 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca1951b640 1 -- 172.21.15.26:0/3184010912 >> 172.21.15.26:0/3184010912 conn(0x7fca1406b300 msgr2=0x7fca1406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca1951b640 1 -- 172.21.15.26:0/3184010912 shutdown_connections 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.694+0000 7fca1951b640 1 -- 172.21.15.26:0/3184010912 wait complete. 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.696+0000 7fca1951b640 1 Processor -- start 2024-04-24T23:26:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.696+0000 7fca1951b640 1 -- start start 2024-04-24T23:26:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.696+0000 7fca1951b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca1406d350 0x7fca141f1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.696+0000 7fca1951b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca140cd660 0x7fca1413d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.696+0000 7fca1951b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 0x7fca14139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.696+0000 7fca1951b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fca1406e370 con 0x7fca14145910 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.696+0000 7fca1951b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fca1406e090 con 0x7fca1406d350 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.696+0000 7fca1951b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fca1415d770 con 0x7fca140cd660 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 0x7fca14139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 0x7fca14139fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50330/0 (socket says 172.21.15.26:50330) 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca137fe640 1 -- 172.21.15.26:0/1075094005 learned_addr learned my addr 172.21.15.26:0/1075094005 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca12ffd640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca1406d350 0x7fca141f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca127fc640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca140cd660 0x7fca1413d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca137fe640 1 -- 172.21.15.26:0/1075094005 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca140cd660 msgr2=0x7fca1413d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca137fe640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca140cd660 0x7fca1413d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca137fe640 1 -- 172.21.15.26:0/1075094005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca1406d350 msgr2=0x7fca141f1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca137fe640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca1406d350 0x7fca141f1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca137fe640 1 -- 172.21.15.26:0/1075094005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1413d950 con 0x7fca14145910 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca12ffd640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca1406d350 0x7fca141f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca137fe640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 0x7fca14139fd0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00cef0 tx=0x7fca0c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fc9fb7fe640 1 -- 172.21.15.26:0/1075094005 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c01a070 con 0x7fca14145910 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fc9fb7fe640 1 -- 172.21.15.26:0/1075094005 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c00d860 con 0x7fca14145910 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1413a510 con 0x7fca14145910 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fc9fb7fe640 1 -- 172.21.15.26:0/1075094005 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00dc50 con 0x7fca14145910 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.697+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1413aa90 con 0x7fca14145910 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.698+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca140694c0 con 0x7fca14145910 2024-04-24T23:26:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.699+0000 7fc9fb7fe640 1 -- 172.21.15.26:0/1075094005 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca0c01d070 con 0x7fca14145910 2024-04-24T23:26:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.699+0000 7fc9fb7fe640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca0006be30 0x7fca0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.699+0000 7fca12ffd640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca0006be30 0x7fca0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.699+0000 7fc9fb7fe640 1 -- 172.21.15.26:0/1075094005 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fca0c08b680 con 0x7fca14145910 2024-04-24T23:26:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.700+0000 7fca12ffd640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca0006be30 0x7fca0006e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fca04034000 tx=0x7fca0402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.704+0000 7fc9fb7fe640 1 -- 172.21.15.26:0/1075094005 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fca0c05a080 con 0x7fca14145910 2024-04-24T23:26:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.887+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca1415e720 con 0x7fca14145910 2024-04-24T23:26:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.889+0000 7fc9fb7fe640 1 -- 172.21.15.26:0/1075094005 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16706 (secure 0 0 0) 0x7fca0c059980 con 0x7fca14145910 2024-04-24T23:26:00.890 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:00.890 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:25:37.018656+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:25:25.626812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:25:26.319330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca0006be30 msgr2=0x7fca0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca0006be30 0x7fca0006e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fca04034000 tx=0x7fca0402e000 comp rx=0 tx=0).stop 2024-04-24T23:26:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 msgr2=0x7fca14139fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 0x7fca14139fd0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00cef0 tx=0x7fca0c00a710 comp rx=0 tx=0).stop 2024-04-24T23:26:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 shutdown_connections 2024-04-24T23:26:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca0006be30 0x7fca0006e2f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca14145910 0x7fca14139fd0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fca140cd660 0x7fca1413d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 --2- 172.21.15.26:0/1075094005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca1406d350 0x7fca141f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 >> 172.21.15.26:0/1075094005 conn(0x7fca1406b300 msgr2=0x7fca1415ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 shutdown_connections 2024-04-24T23:26:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:00.891+0000 7fca1951b640 1 -- 172.21.15.26:0/1075094005 wait complete. 2024-04-24T23:26:00.903 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:25:25.626812+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:25:26.319330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:26:00.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:26:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.006+0000 7fda86b6a640 1 Processor -- start 2024-04-24T23:26:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.006+0000 7fda86b6a640 1 -- start start 2024-04-24T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.007+0000 7fda86b6a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fda800d0750 0x7fda800acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.007+0000 7fda86b6a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda800aeb10 con 0x7fda8015be40 2024-04-24T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.007+0000 7fda86b6a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda800aedf0 con 0x7fda800b4ce0 2024-04-24T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.007+0000 7fda86b6a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fda8009df20 con 0x7fda800d0750 2024-04-24T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.007+0000 7fda84906640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fda8015be40 0x7fda800d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56822/0 (socket says 172.21.15.26:56822) 2024-04-24T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.007+0000 7fda84906640 1 -- 172.21.15.26:0/2557033569 learned_addr learned my addr 172.21.15.26:0/2557033569 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7ffff640 1 --2- 172.21.15.26:0/2557033569 >> v2:172.21.15.29:3300/0 conn(0x7fda800d0750 0x7fda800acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 234697508 0 0) 0x7fda800aeb10 con 0x7fda8015be40 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda68003520 con 0x7fda8015be40 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 549267106 0 0) 0x7fda68003520 con 0x7fda8015be40 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 >> v2:172.21.15.29:3300/0 conn(0x7fda800d0750 msgr2=0x7fda800acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 --2- 172.21.15.26:0/2557033569 >> v2:172.21.15.29:3300/0 conn(0x7fda800d0750 0x7fda800acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 >> v1:172.21.15.29:6789/0 conn(0x7fda800b4ce0 legacy=0x7fda8015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda800af0d0 con 0x7fda8015be40 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fda780019f0 con 0x7fda8015be40 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7ffff640 1 --2- 172.21.15.26:0/2557033569 >> v2:172.21.15.29:3300/0 conn(0x7fda800d0750 0x7fda800acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 >> v1:172.21.15.26:6789/0 conn(0x7fda8015be40 legacy=0x7fda800d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda680039b0 0x7fda68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda68004800 0x7fda68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680051c0 0x7fda6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda6800c0e0 con 0x7fda680039b0 2024-04-24T23:26:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda6800c290 con 0x7fda680051c0 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fda6800c490 con 0x7fda68004800 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda84906640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda680039b0 0x7fda68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7ffff640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda68004800 0x7fda68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda85107640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680051c0 0x7fda6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda84906640 1 -- 172.21.15.26:0/2557033569 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda68004800 msgr2=0x7fda68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda84906640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda68004800 0x7fda68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda84906640 1 -- 172.21.15.26:0/2557033569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680051c0 msgr2=0x7fda6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda84906640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680051c0 0x7fda6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda85107640 1 -- 172.21.15.26:0/2557033569 reap_dead start 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda84906640 1 -- 172.21.15.26:0/2557033569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda68003520 con 0x7fda680039b0 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.008+0000 7fda7ffff640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda68004800 0x7fda68004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.009+0000 7fda84906640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda680039b0 0x7fda68003dd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fda7800a840 tx=0x7fda68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac31a0e12bae61db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.009+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7800a460 con 0x7fda680039b0 2024-04-24T23:26:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.009+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7802f070 con 0x7fda680039b0 2024-04-24T23:26:01.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.009+0000 7fda7f7fe640 1 -- 172.21.15.26:0/2557033569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7802bcf0 con 0x7fda680039b0 2024-04-24T23:26:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.010+0000 7fda86b6a640 1 -- 172.21.15.26:0/2557033569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda680039b0 msgr2=0x7fda68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.010+0000 7fda86b6a640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda680039b0 0x7fda68003dd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fda7800a840 tx=0x7fda68005830 comp rx=0 tx=0).stop 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.010+0000 7fda86b6a640 1 -- 172.21.15.26:0/2557033569 shutdown_connections 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.010+0000 7fda86b6a640 1 --2- 172.21.15.26:0/2557033569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda680039b0 0x7fda68003dd0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.010+0000 7fda86b6a640 1 -- 172.21.15.26:0/2557033569 >> 172.21.15.26:0/2557033569 conn(0x7fda8006b540 msgr2=0x7fda8006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.010+0000 7fda86b6a640 1 -- 172.21.15.26:0/2557033569 shutdown_connections 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.010+0000 7fda86b6a640 1 -- 172.21.15.26:0/2557033569 wait complete. 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.011+0000 7fda86b6a640 1 Processor -- start 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.011+0000 7fda86b6a640 1 -- start start 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda86b6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680039b0 0x7fda8009d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda86b6a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda80147b80 0x7fda8015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda86b6a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015dd60 0x7fda8015e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda86b6a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda8006cd10 con 0x7fda8015dd60 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda86b6a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda8006cb90 con 0x7fda680039b0 2024-04-24T23:26:01.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda86b6a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fda800acd30 con 0x7fda80147b80 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda85107640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015dd60 0x7fda8015e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda85107640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015dd60 0x7fda8015e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50348/0 (socket says 172.21.15.26:50348) 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda85107640 1 -- 172.21.15.26:0/1429944463 learned_addr learned my addr 172.21.15.26:0/1429944463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda84906640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680039b0 0x7fda8009d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda7ffff640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda80147b80 0x7fda8015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda85107640 1 -- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda80147b80 msgr2=0x7fda8015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda85107640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda80147b80 0x7fda8015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda85107640 1 -- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680039b0 msgr2=0x7fda8009d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda85107640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680039b0 0x7fda8009d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda85107640 1 -- 172.21.15.26:0/1429944463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8015c990 con 0x7fda8015dd60 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda84906640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680039b0 0x7fda8009d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.012+0000 7fda7ffff640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda80147b80 0x7fda8015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.013+0000 7fda85107640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015dd60 0x7fda8015e1e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fda800cfbc0 tx=0x7fda7000e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.013+0000 7fda7d7fa640 1 -- 172.21.15.26:0/1429944463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70019070 con 0x7fda8015dd60 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.013+0000 7fda7d7fa640 1 -- 172.21.15.26:0/1429944463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7000ee50 con 0x7fda8015dd60 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.013+0000 7fda7d7fa640 1 -- 172.21.15.26:0/1429944463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700035f0 con 0x7fda8015dd60 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.013+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8015cc70 con 0x7fda8015dd60 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.013+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8015d230 con 0x7fda8015dd60 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.014+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda800ad8b0 con 0x7fda8015dd60 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.014+0000 7fda7d7fa640 1 -- 172.21.15.26:0/1429944463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda7001c070 con 0x7fda8015dd60 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.015+0000 7fda7d7fa640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda6c06be60 0x7fda6c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.015+0000 7fda7d7fa640 1 -- 172.21.15.26:0/1429944463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fda7008a0a0 con 0x7fda8015dd60 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.015+0000 7fda84906640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda6c06be60 0x7fda6c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.015+0000 7fda84906640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda6c06be60 0x7fda6c06e320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fda78033000 tx=0x7fda78007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.019+0000 7fda7d7fa640 1 -- 172.21.15.26:0/1429944463 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fda70058a10 con 0x7fda8015dd60 2024-04-24T23:26:01.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.203+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda80075610 con 0x7fda8015dd60 2024-04-24T23:26:01.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.204+0000 7fda7d7fa640 1 -- 172.21.15.26:0/1429944463 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fda70058310 con 0x7fda8015dd60 2024-04-24T23:26:01.205 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:26:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda6c06be60 msgr2=0x7fda6c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda6c06be60 0x7fda6c06e320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fda78033000 tx=0x7fda78007000 comp rx=0 tx=0).stop 2024-04-24T23:26:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015dd60 msgr2=0x7fda8015e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015dd60 0x7fda8015e1e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fda800cfbc0 tx=0x7fda7000e5f0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 shutdown_connections 2024-04-24T23:26:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda6c06be60 0x7fda6c06e320 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015dd60 0x7fda8015e1e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda80147b80 0x7fda8015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 --2- 172.21.15.26:0/1429944463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda680039b0 0x7fda8009d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 >> 172.21.15.26:0/1429944463 conn(0x7fda8006b540 msgr2=0x7fda800b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 shutdown_connections 2024-04-24T23:26:01.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.206+0000 7fda86b6a640 1 -- 172.21.15.26:0/1429944463 wait complete. 2024-04-24T23:26:01.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:26:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.320+0000 7f09aad6a640 1 Processor -- start 2024-04-24T23:26:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.320+0000 7f09aad6a640 1 -- start start 2024-04-24T23:26:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.321+0000 7f09aad6a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09a415e750 0x7f09a40cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.321+0000 7f09aad6a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f09a40d2c30 0x7f09a40cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.321+0000 7f09aad6a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09a40d2120 con 0x7f09a415e370 2024-04-24T23:26:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.321+0000 7f09aad6a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f09a40d1510 con 0x7f09a415e750 2024-04-24T23:26:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.321+0000 7f09aad6a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f09a40d1ed0 con 0x7f09a40d2c30 2024-04-24T23:26:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.321+0000 7f09a3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09a415e750 0x7f09a40cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.321+0000 7f09a3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09a415e750 0x7f09a40cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50362/0 (socket says 172.21.15.26:50362) 2024-04-24T23:26:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.321+0000 7f09a3fff640 1 -- 172.21.15.26:0/1325681512 learned_addr learned my addr 172.21.15.26:0/1325681512 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.322+0000 7f09a9307640 1 --2- 172.21.15.26:0/1325681512 >> v2:172.21.15.29:3300/0 conn(0x7f09a40d2c30 0x7f09a40cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.322+0000 7f09a3fff640 1 -- 172.21.15.26:0/1325681512 >> v2:172.21.15.29:3300/0 conn(0x7f09a40d2c30 msgr2=0x7f09a40cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.322+0000 7f09a3fff640 1 --2- 172.21.15.26:0/1325681512 >> v2:172.21.15.29:3300/0 conn(0x7f09a40d2c30 0x7f09a40cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.322+0000 7f09a3fff640 1 -- 172.21.15.26:0/1325681512 >> v1:172.21.15.29:6789/0 conn(0x7f09a415e370 legacy=0x7f09a40d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.322+0000 7f09a3fff640 1 -- 172.21.15.26:0/1325681512 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a40b3740 con 0x7f09a415e750 2024-04-24T23:26:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.322+0000 7f09a9307640 1 --2- 172.21.15.26:0/1325681512 >> v2:172.21.15.29:3300/0 conn(0x7f09a40d2c30 0x7f09a40cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.322+0000 7f09a3fff640 1 --2- 172.21.15.26:0/1325681512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a415e750 0x7f09a40cec90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f09a408d9a0 tx=0x7f099802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97169b1a3e27575 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09a37fe640 1 -- 172.21.15.26:0/1325681512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998037070 con 0x7f09a415e750 2024-04-24T23:26:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09a37fe640 1 -- 172.21.15.26:0/1325681512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0998032440 con 0x7f09a415e750 2024-04-24T23:26:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09a37fe640 1 -- 172.21.15.26:0/1325681512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998032810 con 0x7f09a415e750 2024-04-24T23:26:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09aad6a640 1 -- 172.21.15.26:0/1325681512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a415e750 msgr2=0x7f09a40cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09aad6a640 1 --2- 172.21.15.26:0/1325681512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a415e750 0x7f09a40cec90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f09a408d9a0 tx=0x7f099802a230 comp rx=0 tx=0).stop 2024-04-24T23:26:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09aad6a640 1 -- 172.21.15.26:0/1325681512 shutdown_connections 2024-04-24T23:26:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09aad6a640 1 --2- 172.21.15.26:0/1325681512 >> v2:172.21.15.29:3300/0 conn(0x7f09a40d2c30 0x7f09a40cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09aad6a640 1 --2- 172.21.15.26:0/1325681512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a415e750 0x7f09a40cec90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09aad6a640 1 -- 172.21.15.26:0/1325681512 >> 172.21.15.26:0/1325681512 conn(0x7f09a40d40f0 msgr2=0x7f09a40d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09aad6a640 1 -- 172.21.15.26:0/1325681512 shutdown_connections 2024-04-24T23:26:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.323+0000 7f09aad6a640 1 -- 172.21.15.26:0/1325681512 wait complete. 2024-04-24T23:26:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.324+0000 7f09aad6a640 1 Processor -- start 2024-04-24T23:26:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.324+0000 7f09aad6a640 1 -- start start 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.325+0000 7f09aad6a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09a40d2c30 0x7f09a415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.325+0000 7f09aad6a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09a415e370 0x7f09a40b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.325+0000 7f09aad6a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a40b85c0 0x7f09a41f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.325+0000 7f09aad6a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f09a40b5230 con 0x7f09a40b85c0 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.325+0000 7f09aad6a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f09a40b3d00 con 0x7f09a40d2c30 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.325+0000 7f09aad6a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f09a40b53b0 con 0x7f09a415e370 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.325+0000 7f09a9307640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a40b85c0 0x7f09a41f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a9307640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a40b85c0 0x7f09a41f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50370/0 (socket says 172.21.15.26:50370) 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a8b06640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09a40d2c30 0x7f09a415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a9307640 1 -- 172.21.15.26:0/3583460817 learned_addr learned my addr 172.21.15.26:0/3583460817 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a3fff640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09a415e370 0x7f09a40b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a9307640 1 -- 172.21.15.26:0/3583460817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09a415e370 msgr2=0x7f09a40b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a9307640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09a415e370 0x7f09a40b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a9307640 1 -- 172.21.15.26:0/3583460817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09a40d2c30 msgr2=0x7f09a415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a9307640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09a40d2c30 0x7f09a415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a9307640 1 -- 172.21.15.26:0/3583460817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a40b7530 con 0x7f09a40b85c0 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a9307640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a40b85c0 0x7f09a41f38f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f099400c7c0 tx=0x7f099400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a17fa640 1 -- 172.21.15.26:0/3583460817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994015020 con 0x7f09a40b85c0 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a17fa640 1 -- 172.21.15.26:0/3583460817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0994004230 con 0x7f09a40b85c0 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09a17fa640 1 -- 172.21.15.26:0/3583460817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994007620 con 0x7f09a40b85c0 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a40b7810 con 0x7f09a40b85c0 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.326+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a40b8d40 con 0x7f09a40b85c0 2024-04-24T23:26:01.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.327+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a4075610 con 0x7f09a40b85c0 2024-04-24T23:26:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.328+0000 7f09a17fa640 1 -- 172.21.15.26:0/3583460817 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0994018020 con 0x7f09a40b85c0 2024-04-24T23:26:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.328+0000 7f09a17fa640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099006be30 0x7f099006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.328+0000 7f09a17fa640 1 -- 172.21.15.26:0/3583460817 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f099408add0 con 0x7f09a40b85c0 2024-04-24T23:26:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.328+0000 7f09a8b06640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099006be30 0x7f099006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.329+0000 7f09a8b06640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099006be30 0x7f099006e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f099c005660 tx=0x7f099c003d90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.332+0000 7f09a17fa640 1 -- 172.21.15.26:0/3583460817 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f099401f020 con 0x7f09a40b85c0 2024-04-24T23:26:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.517+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f09a40b8af0 con 0x7f09a40b85c0 2024-04-24T23:26:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.518+0000 7f09a17fa640 1 -- 172.21.15.26:0/3583460817 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f099408c020 con 0x7f09a40b85c0 2024-04-24T23:26:01.519 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:26:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099006be30 msgr2=0x7f099006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099006be30 0x7f099006e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f099c005660 tx=0x7f099c003d90 comp rx=0 tx=0).stop 2024-04-24T23:26:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a40b85c0 msgr2=0x7f09a41f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a40b85c0 0x7f09a41f38f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f099400c7c0 tx=0x7f099400cc80 comp rx=0 tx=0).stop 2024-04-24T23:26:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 shutdown_connections 2024-04-24T23:26:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f099006be30 0x7f099006e2f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09a40b85c0 0x7f09a41f38f0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f09a415e370 0x7f09a40b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 --2- 172.21.15.26:0/3583460817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f09a40d2c30 0x7f09a415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 >> 172.21.15.26:0/3583460817 conn(0x7f09a40d40f0 msgr2=0x7f09a40b5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 shutdown_connections 2024-04-24T23:26:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.520+0000 7f09aad6a640 1 -- 172.21.15.26:0/3583460817 wait complete. 2024-04-24T23:26:01.532 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:26:01.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.634+0000 7f87dd56a640 1 Processor -- start 2024-04-24T23:26:01.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.635+0000 7f87dd56a640 1 -- start start 2024-04-24T23:26:01.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.635+0000 7f87dd56a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87d8071e30 0x7f87d806e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.635+0000 7f87dd56a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d8161fe0 con 0x7f87d80d0ce0 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.635+0000 7f87dd56a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d806b830 con 0x7f87d8158ae0 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.635+0000 7f87dd56a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f87d80d02c0 con 0x7f87d8071e30 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.636+0000 7f87d77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87d8071e30 0x7f87d806e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.636+0000 7f87d67fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87d80d0ce0 0x7f87d8071720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56832/0 (socket says 172.21.15.26:56832) 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.636+0000 7f87d67fc640 1 -- 172.21.15.26:0/4266223872 learned_addr learned my addr 172.21.15.26:0/4266223872 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.636+0000 7f87d77fe640 1 -- 172.21.15.26:0/4266223872 >> v1:172.21.15.29:6789/0 conn(0x7f87d8158ae0 legacy=0x7f87d806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.636+0000 7f87d77fe640 1 -- 172.21.15.26:0/4266223872 >> v1:172.21.15.26:6789/0 conn(0x7f87d80d0ce0 legacy=0x7f87d8071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.636+0000 7f87d77fe640 1 -- 172.21.15.26:0/4266223872 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d806ec90 con 0x7f87d8071e30 2024-04-24T23:26:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.636+0000 7f87d5ffb640 1 -- 172.21.15.26:0/4266223872 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625914865 0 0) 0x7f87d8161fe0 con 0x7f87d80d0ce0 2024-04-24T23:26:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87d77fe640 1 --2- 172.21.15.26:0/4266223872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 0x7f87d806e620 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009d90 tx=0x7f87d002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2fbed94079438b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87d5ffb640 1 -- 172.21.15.26:0/4266223872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0007740 con 0x7f87d8071e30 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87d5ffb640 1 -- 172.21.15.26:0/4266223872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d00078e0 con 0x7f87d8071e30 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87d5ffb640 1 -- 172.21.15.26:0/4266223872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0007cb0 con 0x7f87d8071e30 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87dd56a640 1 -- 172.21.15.26:0/4266223872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 msgr2=0x7f87d806e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87dd56a640 1 --2- 172.21.15.26:0/4266223872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 0x7f87d806e620 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009d90 tx=0x7f87d002a660 comp rx=0 tx=0).stop 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87dd56a640 1 -- 172.21.15.26:0/4266223872 shutdown_connections 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87dd56a640 1 --2- 172.21.15.26:0/4266223872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 0x7f87d806e620 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87dd56a640 1 -- 172.21.15.26:0/4266223872 >> 172.21.15.26:0/4266223872 conn(0x7f87d80b2970 msgr2=0x7f87d80d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87dd56a640 1 -- 172.21.15.26:0/4266223872 shutdown_connections 2024-04-24T23:26:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.637+0000 7f87dd56a640 1 -- 172.21.15.26:0/4266223872 wait complete. 2024-04-24T23:26:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.639+0000 7f87dd56a640 1 Processor -- start 2024-04-24T23:26:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.639+0000 7f87dd56a640 1 -- start start 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.639+0000 7f87dd56a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 0x7f87d81e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.639+0000 7f87dd56a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d80d0ce0 0x7f87d815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.639+0000 7f87dd56a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87d8158ae0 0x7f87d8157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.639+0000 7f87dd56a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f87d806f530 con 0x7f87d8071e30 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.639+0000 7f87dd56a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f87d806f250 con 0x7f87d80d0ce0 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.639+0000 7f87dd56a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f87d80aff90 con 0x7f87d8158ae0 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 0x7f87d81e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 0x7f87d81e7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50392/0 (socket says 172.21.15.26:50392) 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d6ffd640 1 -- 172.21.15.26:0/2296658478 learned_addr learned my addr 172.21.15.26:0/2296658478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d6ffd640 1 -- 172.21.15.26:0/2296658478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87d8158ae0 msgr2=0x7f87d8157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d6ffd640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87d8158ae0 0x7f87d8157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d67fc640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d80d0ce0 0x7f87d815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d6ffd640 1 -- 172.21.15.26:0/2296658478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d80d0ce0 msgr2=0x7f87d815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d6ffd640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d80d0ce0 0x7f87d815d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d6ffd640 1 -- 172.21.15.26:0/2296658478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d81567e0 con 0x7f87d8071e30 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d67fc640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d80d0ce0 0x7f87d815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87d6ffd640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 0x7f87d81e7250 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c400 tx=0x7f87c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87bf7fe640 1 -- 172.21.15.26:0/2296658478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8017070 con 0x7f87d8071e30 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87bf7fe640 1 -- 172.21.15.26:0/2296658478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c80052d0 con 0x7f87d8071e30 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87bf7fe640 1 -- 172.21.15.26:0/2296658478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8003830 con 0x7f87d8071e30 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d8156ac0 con 0x7f87d8071e30 2024-04-24T23:26:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.640+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d8157080 con 0x7f87d8071e30 2024-04-24T23:26:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.642+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d80b5b90 con 0x7f87d8071e30 2024-04-24T23:26:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.642+0000 7f87bf7fe640 1 -- 172.21.15.26:0/2296658478 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87c801a070 con 0x7f87d8071e30 2024-04-24T23:26:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.642+0000 7f87bf7fe640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c406be30 0x7f87c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.642+0000 7f87bf7fe640 1 -- 172.21.15.26:0/2296658478 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f87c808a210 con 0x7f87d8071e30 2024-04-24T23:26:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.642+0000 7f87d67fc640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c406be30 0x7f87c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.643+0000 7f87d67fc640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c406be30 0x7f87c406e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f87d81f7360 tx=0x7f87cc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.647+0000 7f87bf7fe640 1 -- 172.21.15.26:0/2296658478 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f87c8058b90 con 0x7f87d8071e30 2024-04-24T23:26:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.828+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87d8075610 con 0x7f87d8071e30 2024-04-24T23:26:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.829+0000 7f87bf7fe640 1 -- 172.21.15.26:0/2296658478 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f87c8058490 con 0x7f87d8071e30 2024-04-24T23:26:01.830 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:26:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c406be30 msgr2=0x7f87c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c406be30 0x7f87c406e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f87d81f7360 tx=0x7f87cc009040 comp rx=0 tx=0).stop 2024-04-24T23:26:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 msgr2=0x7f87d81e7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 0x7f87d81e7250 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c400 tx=0x7f87c800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 shutdown_connections 2024-04-24T23:26:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f87c406be30 0x7f87c406e2f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f87d8158ae0 0x7f87d8157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87d80d0ce0 0x7f87d815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 --2- 172.21.15.26:0/2296658478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87d8071e30 0x7f87d81e7250 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 >> 172.21.15.26:0/2296658478 conn(0x7f87d80b2970 msgr2=0x7f87d806c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 shutdown_connections 2024-04-24T23:26:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.831+0000 7f87dd56a640 1 -- 172.21.15.26:0/2296658478 wait complete. 2024-04-24T23:26:01.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:26:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.943+0000 7f5023bdc640 1 Processor -- start 2024-04-24T23:26:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.944+0000 7f5023bdc640 1 -- start start 2024-04-24T23:26:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.944+0000 7f5023bdc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 0x7f501c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.944+0000 7f5023bdc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f501c0b3130 0x7f501c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.944+0000 7f5023bdc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f501c0d32f0 con 0x7f501c0d22e0 2024-04-24T23:26:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.944+0000 7f5023bdc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f501c0d1ab0 con 0x7f501c0b3130 2024-04-24T23:26:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.944+0000 7f5023bdc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f501c0d2ae0 con 0x7f501c15e570 2024-04-24T23:26:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.944+0000 7f5022179640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f501c0d22e0 0x7f501c0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56834/0 (socket says 172.21.15.26:56834) 2024-04-24T23:26:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5022179640 1 -- 172.21.15.26:0/39983672 learned_addr learned my addr 172.21.15.26:0/39983672 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5021978640 1 --2- 172.21.15.26:0/39983672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 0x7f501c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5021177640 1 --2- 172.21.15.26:0/39983672 >> v2:172.21.15.29:3300/0 conn(0x7f501c0b3130 0x7f501c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5020976640 1 -- 172.21.15.26:0/39983672 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1843767056 0 0) 0x7f5014001490 con 0x7f501c0d22e0 2024-04-24T23:26:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5020976640 1 -- 172.21.15.26:0/39983672 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5008003520 con 0x7f501c0d22e0 2024-04-24T23:26:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5020976640 1 -- 172.21.15.26:0/39983672 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 427491162 0 0) 0x7f5008003520 con 0x7f501c0d22e0 2024-04-24T23:26:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5021978640 1 -- 172.21.15.26:0/39983672 >> v2:172.21.15.29:3300/0 conn(0x7f501c0b3130 msgr2=0x7f501c0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5021978640 1 --2- 172.21.15.26:0/39983672 >> v2:172.21.15.29:3300/0 conn(0x7f501c0b3130 0x7f501c0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5021978640 1 -- 172.21.15.26:0/39983672 >> v1:172.21.15.26:6789/0 conn(0x7f501c0d22e0 legacy=0x7f501c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.945+0000 7f5021978640 1 -- 172.21.15.26:0/39983672 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c0d0270 con 0x7f501c15e570 2024-04-24T23:26:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.946+0000 7f5021978640 1 --2- 172.21.15.26:0/39983672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 0x7f501c0d4160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50180097a0 tx=0x7f501802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63d2d8653fafead5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.946+0000 7f5020976640 1 -- 172.21.15.26:0/39983672 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018037070 con 0x7f501c15e570 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.946+0000 7f5020976640 1 -- 172.21.15.26:0/39983672 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f501802aba0 con 0x7f501c15e570 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.946+0000 7f5020976640 1 -- 172.21.15.26:0/39983672 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014001bc0 con 0x7f501c15e570 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.946+0000 7f5023bdc640 1 -- 172.21.15.26:0/39983672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 msgr2=0x7f501c0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.946+0000 7f5023bdc640 1 --2- 172.21.15.26:0/39983672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 0x7f501c0d4160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50180097a0 tx=0x7f501802a110 comp rx=0 tx=0).stop 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.947+0000 7f5023bdc640 1 -- 172.21.15.26:0/39983672 shutdown_connections 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.947+0000 7f5023bdc640 1 --2- 172.21.15.26:0/39983672 >> v2:172.21.15.29:3300/0 conn(0x7f501c0b3130 0x7f501c0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.947+0000 7f5023bdc640 1 --2- 172.21.15.26:0/39983672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 0x7f501c0d4160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.947+0000 7f5023bdc640 1 -- 172.21.15.26:0/39983672 >> 172.21.15.26:0/39983672 conn(0x7f501c08dbe0 msgr2=0x7f501c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.947+0000 7f5023bdc640 1 -- 172.21.15.26:0/39983672 shutdown_connections 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.947+0000 7f5023bdc640 1 -- 172.21.15.26:0/39983672 wait complete. 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.948+0000 7f5023bdc640 1 Processor -- start 2024-04-24T23:26:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.948+0000 7f5023bdc640 1 -- start start 2024-04-24T23:26:01.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.948+0000 7f5023bdc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f501c0b3130 0x7f501c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.948+0000 7f5023bdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f501c0d22e0 0x7f501c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.948+0000 7f5023bdc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 0x7f501c153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.948+0000 7f5023bdc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f501c0cfa70 con 0x7f501c0d22e0 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.948+0000 7f5023bdc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f501c0d3010 con 0x7f501c0b3130 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.948+0000 7f5023bdc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f501c0cfd50 con 0x7f501c15e570 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021177640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f501c0d22e0 0x7f501c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021177640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f501c0d22e0 0x7f501c1529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50404/0 (socket says 172.21.15.26:50404) 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021177640 1 -- 172.21.15.26:0/1735268392 learned_addr learned my addr 172.21.15.26:0/1735268392 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5022179640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 0x7f501c153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021978640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f501c0b3130 0x7f501c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021177640 1 -- 172.21.15.26:0/1735268392 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 msgr2=0x7f501c153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021177640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 0x7f501c153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021177640 1 -- 172.21.15.26:0/1735268392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f501c0b3130 msgr2=0x7f501c14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021177640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f501c0b3130 0x7f501c14f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021177640 1 -- 172.21.15.26:0/1735268392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c14ff70 con 0x7f501c0d22e0 2024-04-24T23:26:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5021177640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f501c0d22e0 0x7f501c1529c0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f500c00e930 tx=0x7f500c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f50127fc640 1 -- 172.21.15.26:0/1735268392 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c017070 con 0x7f501c0d22e0 2024-04-24T23:26:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f50127fc640 1 -- 172.21.15.26:0/1735268392 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c00ca00 con 0x7f501c0d22e0 2024-04-24T23:26:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f50127fc640 1 -- 172.21.15.26:0/1735268392 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c00cdf0 con 0x7f501c0d22e0 2024-04-24T23:26:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c152f00 con 0x7f501c0d22e0 2024-04-24T23:26:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.949+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c1504f0 con 0x7f501c0d22e0 2024-04-24T23:26:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.950+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501c0694c0 con 0x7f501c0d22e0 2024-04-24T23:26:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.951+0000 7f50127fc640 1 -- 172.21.15.26:0/1735268392 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f500c01a070 con 0x7f501c0d22e0 2024-04-24T23:26:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.951+0000 7f50127fc640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f500006be30 0x7f500006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.951+0000 7f50127fc640 1 -- 172.21.15.26:0/1735268392 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f500c08af10 con 0x7f501c0d22e0 2024-04-24T23:26:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.951+0000 7f5021978640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f500006be30 0x7f500006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.952+0000 7f5021978640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f500006be30 0x7f500006e2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5018034000 tx=0x7f501802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:01.956+0000 7f50127fc640 1 -- 172.21.15.26:0/1735268392 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f500c059940 con 0x7f501c0d22e0 2024-04-24T23:26:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.153+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f501c0b4df0 con 0x7f501c0d22e0 2024-04-24T23:26:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.153+0000 7f50127fc640 1 -- 172.21.15.26:0/1735268392 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f500c059240 con 0x7f501c0d22e0 2024-04-24T23:26:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:26:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.155+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f500006be30 msgr2=0x7f500006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.155+0000 7f5023bdc640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f500006be30 0x7f500006e2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5018034000 tx=0x7f501802e000 comp rx=0 tx=0).stop 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.155+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f501c0d22e0 msgr2=0x7f501c1529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.155+0000 7f5023bdc640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f501c0d22e0 0x7f501c1529c0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f500c00e930 tx=0x7f500c00edf0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.155+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 shutdown_connections 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.155+0000 7f5023bdc640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f500006be30 0x7f500006e2f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.156+0000 7f5023bdc640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f501c15e570 0x7f501c153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.156+0000 7f5023bdc640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f501c0d22e0 0x7f501c1529c0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.156+0000 7f5023bdc640 1 --2- 172.21.15.26:0/1735268392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f501c0b3130 0x7f501c14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.156+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 >> 172.21.15.26:0/1735268392 conn(0x7f501c08dbe0 msgr2=0x7f501c06cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.156+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 shutdown_connections 2024-04-24T23:26:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.156+0000 7f5023bdc640 1 -- 172.21.15.26:0/1735268392 wait complete. 2024-04-24T23:26:02.167 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:26:02.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:02.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:26:02.195 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:26:02.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:02.195 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:26:02.251 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:26:02.251 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:26:02.251 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:02.271 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:26:02.271 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:26:02.271 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:26:02.271 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:02.307 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:26:02.307 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:26:02.307 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:26:02.307 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:02.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:26:02.329 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:02.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.448+0000 7f5482916640 1 Processor -- start 2024-04-24T23:26:02.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.448+0000 7f5482916640 1 -- start start 2024-04-24T23:26:02.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f5482916640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f547c15a480 0x7f547c0b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f5482916640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f547c15a860 0x7f547c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f5482916640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f547c0b2e80 con 0x7f547c0b4e30 2024-04-24T23:26:02.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f5482916640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f547c0b3120 con 0x7f547c15a480 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f5482916640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f547c0b33c0 con 0x7f547c15a860 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f5480eb3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f547c0b4e30 0x7f547c15e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56842/0 (socket says 172.21.15.26:56842) 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f5480eb3640 1 -- 172.21.15.26:0/2598829491 learned_addr learned my addr 172.21.15.26:0/2598829491 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547bfff640 1 --2- 172.21.15.26:0/2598829491 >> v2:172.21.15.29:3300/0 conn(0x7f547c15a480 0x7f547c0b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547b7fe640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f547c15a860 0x7f547c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504757798 0 0) 0x7f547c0b2e80 con 0x7f547c0b4e30 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5464003520 con 0x7f547c0b4e30 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 543789358 0 0) 0x7f5464003520 con 0x7f547c0b4e30 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f547c15a860 msgr2=0x7f547c06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f547c15a860 0x7f547c06dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 >> v2:172.21.15.29:3300/0 conn(0x7f547c15a480 msgr2=0x7f547c0b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 --2- 172.21.15.26:0/2598829491 >> v2:172.21.15.29:3300/0 conn(0x7f547c15a480 0x7f547c0b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c0b36a0 con 0x7f547c0b4e30 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f5474001a10 con 0x7f547c0b4e30 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 >> v1:172.21.15.26:6789/0 conn(0x7f547c0b4e30 legacy=0x7f547c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54640039b0 0x7f5464003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.449+0000 7f547affd640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f5464004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547affd640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54640051c0 0x7f546400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0b3120 con 0x7f5464004800 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0b33c0 con 0x7f54640051c0 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f546400c090 con 0x7f54640039b0 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547b7fe640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f547c15a860 0x7f547c06dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547bfff640 1 --2- 172.21.15.26:0/2598829491 >> v2:172.21.15.29:3300/0 conn(0x7f547c15a480 0x7f547c0b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547b7fe640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f5464004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f5480eb3640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54640051c0 0x7f546400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547bfff640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54640039b0 0x7f5464003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547b7fe640 1 -- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54640039b0 msgr2=0x7f5464003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547b7fe640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54640039b0 0x7f5464003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547b7fe640 1 -- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54640051c0 msgr2=0x7f546400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547b7fe640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f54640051c0 0x7f546400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547b7fe640 1 -- 172.21.15.26:0/2598829491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5464003520 con 0x7f5464004800 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.450+0000 7f547bfff640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f54640039b0 0x7f5464003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f547b7fe640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f5464004c80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5470006fd0 tx=0x7f5464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5eed42ecda130fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470039070 con 0x7f5464004800 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547002ccf0 con 0x7f5464004800 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f547affd640 1 -- 172.21.15.26:0/2598829491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547000f040 con 0x7f5464004800 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f5482916640 1 -- 172.21.15.26:0/2598829491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 msgr2=0x7f5464004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f5482916640 1 --2- 172.21.15.26:0/2598829491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f5464004c80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5470006fd0 tx=0x7f5464005830 comp rx=0 tx=0).stop 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f5480eb3640 1 -- 172.21.15.26:0/2598829491 reap_dead start 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f5482916640 1 -- 172.21.15.26:0/2598829491 shutdown_connections 2024-04-24T23:26:02.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f5482916640 1 -- 172.21.15.26:0/2598829491 >> 172.21.15.26:0/2598829491 conn(0x7f547c06b300 msgr2=0x7f547c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f5482916640 1 -- 172.21.15.26:0/2598829491 shutdown_connections 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.451+0000 7f5482916640 1 -- 172.21.15.26:0/2598829491 wait complete. 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.453+0000 7f5482916640 1 Processor -- start 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.453+0000 7f5482916640 1 -- start start 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.453+0000 7f5482916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f547c13b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.453+0000 7f5482916640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f547c13bbb0 0x7f547c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.453+0000 7f5482916640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f547c15d6b0 0x7f547c157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f5482916640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f547c072080 con 0x7f5464004800 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f5482916640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0b3c60 con 0x7f547c15d6b0 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f5482916640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f547c072200 con 0x7f547c13bbb0 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f547bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f547c13b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f547bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f547c13b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50408/0 (socket says 172.21.15.26:50408) 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f547bfff640 1 -- 172.21.15.26:0/737397082 learned_addr learned my addr 172.21.15.26:0/737397082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f547bfff640 1 -- 172.21.15.26:0/737397082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f547c13bbb0 msgr2=0x7f547c15d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f547bfff640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f547c13bbb0 0x7f547c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f547bfff640 1 -- 172.21.15.26:0/737397082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f547c15d6b0 msgr2=0x7f547c157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f547bfff640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f547c15d6b0 0x7f547c157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f547bfff640 1 -- 172.21.15.26:0/737397082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c15db40 con 0x7f5464004800 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f547bfff640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f547c13b670 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f546c010890 tx=0x7f546c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f5478ff9640 1 -- 172.21.15.26:0/737397082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c01a020 con 0x7f5464004800 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f5478ff9640 1 -- 172.21.15.26:0/737397082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f546c00eaa0 con 0x7f5464004800 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c1587a0 con 0x7f5464004800 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f5478ff9640 1 -- 172.21.15.26:0/737397082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c00f040 con 0x7f5464004800 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.454+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c158d60 con 0x7f5464004800 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.455+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547c0694c0 con 0x7f5464004800 2024-04-24T23:26:02.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.456+0000 7f5478ff9640 1 -- 172.21.15.26:0/737397082 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f546c01d020 con 0x7f5464004800 2024-04-24T23:26:02.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.456+0000 7f5478ff9640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f546806be30 0x7f546806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.456+0000 7f5478ff9640 1 -- 172.21.15.26:0/737397082 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f546c08b490 con 0x7f5464004800 2024-04-24T23:26:02.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.456+0000 7f547b7fe640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f546806be30 0x7f546806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.457+0000 7f547b7fe640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f546806be30 0x7f546806e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5470036000 tx=0x7f547002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.460+0000 7f5478ff9640 1 -- 172.21.15.26:0/737397082 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f546c024030 con 0x7f5464004800 2024-04-24T23:26:02.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.682+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f547c0b44a0 con 0x7f5464004800 2024-04-24T23:26:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.682+0000 7f5478ff9640 1 -- 172.21.15.26:0/737397082 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5225 (secure 0 0 0) 0x7f546c059e00 con 0x7f5464004800 2024-04-24T23:26:02.683 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:02.683 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:25:57.966897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7227,"mds_1":7224,"mds_2":6364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6364":{"gid":6364,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7224":{"gid":7224,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[6364,7224,7227]},"id":5}]} 2024-04-24T23:26:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 74 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.684+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f546806be30 msgr2=0x7f546806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.684+0000 7f5482916640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f546806be30 0x7f546806e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5470036000 tx=0x7f547002e000 comp rx=0 tx=0).stop 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.684+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 msgr2=0x7f547c13b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.684+0000 7f5482916640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f547c13b670 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f546c010890 tx=0x7f546c010d50 comp rx=0 tx=0).stop 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.685+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 shutdown_connections 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.685+0000 7f5482916640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f546806be30 0x7f546806e2f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.685+0000 7f5482916640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f547c15d6b0 0x7f547c157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.685+0000 7f5482916640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f547c13bbb0 0x7f547c15d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.685+0000 7f5482916640 1 --2- 172.21.15.26:0/737397082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464004800 0x7f547c13b670 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.685+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 >> 172.21.15.26:0/737397082 conn(0x7f547c06b300 msgr2=0x7f547c0ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.685+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 shutdown_connections 2024-04-24T23:26:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.685+0000 7f5482916640 1 -- 172.21.15.26:0/737397082 wait complete. 2024-04-24T23:26:02.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.796+0000 7fd7def61640 1 Processor -- start 2024-04-24T23:26:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.796+0000 7fd7def61640 1 -- start start 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.797+0000 7fd7def61640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d815e570 0x7fd7d80d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.797+0000 7fd7def61640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7d80b3130 0x7fd7d80d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.797+0000 7fd7def61640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd7d80d22e0 0x7fd7d80d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.797+0000 7fd7def61640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd7d80d1ab0 con 0x7fd7d80b3130 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.797+0000 7fd7def61640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd7d80d2ae0 con 0x7fd7d80d22e0 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.797+0000 7fd7def61640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d80d32f0 con 0x7fd7d815e570 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.797+0000 7fd7d7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7d80b3130 0x7fd7d80d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.797+0000 7fd7d7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7d80b3130 0x7fd7d80d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50410/0 (socket says 172.21.15.26:50410) 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.797+0000 7fd7d7fff640 1 -- 172.21.15.26:0/1043559033 learned_addr learned my addr 172.21.15.26:0/1043559033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.798+0000 7fd7dd4fe640 1 --2- 172.21.15.26:0/1043559033 >> v2:172.21.15.29:3300/0 conn(0x7fd7d80d22e0 0x7fd7d80d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.798+0000 7fd7dccfd640 1 --2- 172.21.15.26:0/1043559033 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d815e570 0x7fd7d80d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.798+0000 7fd7d7fff640 1 -- 172.21.15.26:0/1043559033 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d815e570 msgr2=0x7fd7d80d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.798+0000 7fd7d7fff640 1 --2- 172.21.15.26:0/1043559033 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d815e570 0x7fd7d80d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.798+0000 7fd7d7fff640 1 -- 172.21.15.26:0/1043559033 >> v2:172.21.15.29:3300/0 conn(0x7fd7d80d22e0 msgr2=0x7fd7d80d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.798+0000 7fd7d7fff640 1 --2- 172.21.15.26:0/1043559033 >> v2:172.21.15.29:3300/0 conn(0x7fd7d80d22e0 0x7fd7d80d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.798+0000 7fd7d7fff640 1 -- 172.21.15.26:0/1043559033 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d80d0270 con 0x7fd7d80b3130 2024-04-24T23:26:02.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7d7fff640 1 --2- 172.21.15.26:0/1043559033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80b3130 0x7fd7d80d46a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80ba280 tx=0x7fd7cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9044d8d56ede4dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7d77fe640 1 -- 172.21.15.26:0/1043559033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc037070 con 0x7fd7d80b3130 2024-04-24T23:26:02.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7d77fe640 1 -- 172.21.15.26:0/1043559033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7cc032440 con 0x7fd7d80b3130 2024-04-24T23:26:02.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7d77fe640 1 -- 172.21.15.26:0/1043559033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc032810 con 0x7fd7d80b3130 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7def61640 1 -- 172.21.15.26:0/1043559033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80b3130 msgr2=0x7fd7d80d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7def61640 1 --2- 172.21.15.26:0/1043559033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80b3130 0x7fd7d80d46a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80ba280 tx=0x7fd7cc02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7def61640 1 -- 172.21.15.26:0/1043559033 shutdown_connections 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7def61640 1 --2- 172.21.15.26:0/1043559033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80b3130 0x7fd7d80d46a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7def61640 1 --2- 172.21.15.26:0/1043559033 >> v2:172.21.15.29:3300/0 conn(0x7fd7d80d22e0 0x7fd7d80d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7def61640 1 --2- 172.21.15.26:0/1043559033 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d815e570 0x7fd7d80d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7def61640 1 -- 172.21.15.26:0/1043559033 >> 172.21.15.26:0/1043559033 conn(0x7fd7d808dbe0 msgr2=0x7fd7d809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7def61640 1 -- 172.21.15.26:0/1043559033 shutdown_connections 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.799+0000 7fd7def61640 1 -- 172.21.15.26:0/1043559033 wait complete. 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.800+0000 7fd7def61640 1 Processor -- start 2024-04-24T23:26:02.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.800+0000 7fd7def61640 1 -- start start 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.801+0000 7fd7def61640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d80b3130 0x7fd7d80b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.801+0000 7fd7def61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80d22e0 0x7fd7d80b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.801+0000 7fd7def61640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d815e570 0x7fd7d81590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.801+0000 7fd7def61640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d80cfa70 con 0x7fd7d80d22e0 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.801+0000 7fd7def61640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d80d3010 con 0x7fd7d815e570 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.801+0000 7fd7def61640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d80cfd50 con 0x7fd7d80b3130 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.801+0000 7fd7d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80d22e0 0x7fd7d80b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.801+0000 7fd7d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80d22e0 0x7fd7d80b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50414/0 (socket says 172.21.15.26:50414) 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.801+0000 7fd7d7fff640 1 -- 172.21.15.26:0/2250227359 learned_addr learned my addr 172.21.15.26:0/2250227359 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7dd4fe640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d815e570 0x7fd7d81590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7dccfd640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d80b3130 0x7fd7d80b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7d7fff640 1 -- 172.21.15.26:0/2250227359 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d80b3130 msgr2=0x7fd7d80b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7d7fff640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d80b3130 0x7fd7d80b91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7d7fff640 1 -- 172.21.15.26:0/2250227359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d815e570 msgr2=0x7fd7d81590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7d7fff640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d815e570 0x7fd7d81590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7d7fff640 1 -- 172.21.15.26:0/2250227359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d80b96e0 con 0x7fd7d80d22e0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7d7fff640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80d22e0 0x7fd7d80b1910 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc02a220 tx=0x7fd7cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7d57fa640 1 -- 172.21.15.26:0/2250227359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc041020 con 0x7fd7d80d22e0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7d57fa640 1 -- 172.21.15.26:0/2250227359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7cc0306c0 con 0x7fd7d80d22e0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d80b8700 con 0x7fd7d80d22e0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7d57fa640 1 -- 172.21.15.26:0/2250227359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc037040 con 0x7fd7d80d22e0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.802+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d80b8c80 con 0x7fd7d80d22e0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.803+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d80694c0 con 0x7fd7d80d22e0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.804+0000 7fd7d57fa640 1 -- 172.21.15.26:0/2250227359 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7cc044020 con 0x7fd7d80d22e0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.804+0000 7fd7d57fa640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7c406be30 0x7fd7c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.804+0000 7fd7d57fa640 1 -- 172.21.15.26:0/2250227359 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fd7cc0ab5b0 con 0x7fd7d80d22e0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.804+0000 7fd7dccfd640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7c406be30 0x7fd7c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.805+0000 7fd7dccfd640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7c406be30 0x7fd7c406e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0003aa0 tx=0x7fd7d0009830 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.808+0000 7fd7d57fa640 1 -- 172.21.15.26:0/2250227359 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7cc04b030 con 0x7fd7d80d22e0 2024-04-24T23:26:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.991+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7d80b8e50 con 0x7fd7d80d22e0 2024-04-24T23:26:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.992+0000 7fd7d57fa640 1 -- 172.21.15.26:0/2250227359 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16706 (secure 0 0 0) 0x7fd7cc0798d0 con 0x7fd7d80d22e0 2024-04-24T23:26:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:02.994 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:25:37.018656+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:25:25.626812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:25:26.319330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7c406be30 msgr2=0x7fd7c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7c406be30 0x7fd7c406e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0003aa0 tx=0x7fd7d0009830 comp rx=0 tx=0).stop 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80d22e0 msgr2=0x7fd7d80b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80d22e0 0x7fd7d80b1910 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc02a220 tx=0x7fd7cc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 shutdown_connections 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd7c406be30 0x7fd7c406e2f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7d815e570 0x7fd7d81590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7d80d22e0 0x7fd7d80b1910 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 --2- 172.21.15.26:0/2250227359 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7d80b3130 0x7fd7d80b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 >> 172.21.15.26:0/2250227359 conn(0x7fd7d808dbe0 msgr2=0x7fd7d806cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 shutdown_connections 2024-04-24T23:26:02.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:02.995+0000 7fd7def61640 1 -- 172.21.15.26:0/2250227359 wait complete. 2024-04-24T23:26:03.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:25:25.626812+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:25:26.319330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:26:03.008 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.112+0000 7fb327df9640 1 Processor -- start 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.112+0000 7fb327df9640 1 -- start start 2024-04-24T23:26:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb327df9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3200d0750 0x7fb3200acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb327df9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb3200b4ce0 0x7fb32015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb327df9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3200aeb10 con 0x7fb32015be40 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb327df9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb32009df20 con 0x7fb3200b4ce0 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb327df9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb3200aedf0 con 0x7fb3200d0750 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb325b95640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb32015be40 0x7fb3200d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56852/0 (socket says 172.21.15.26:56852) 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb325b95640 1 -- 172.21.15.26:0/2045616631 learned_addr learned my addr 172.21.15.26:0/2045616631 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb325394640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3200d0750 0x7fb3200acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb326396640 1 --2- 172.21.15.26:0/2045616631 >> v2:172.21.15.29:3300/0 conn(0x7fb3200b4ce0 0x7fb32015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3256099752 0 0) 0x7fb3200aeb10 con 0x7fb32015be40 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb30c003520 con 0x7fb32015be40 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2893924740 0 0) 0x7fb30c003520 con 0x7fb32015be40 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3200d0750 msgr2=0x7fb3200acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3200d0750 0x7fb3200acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 >> v2:172.21.15.29:3300/0 conn(0x7fb3200b4ce0 msgr2=0x7fb32015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 --2- 172.21.15.26:0/2045616631 >> v2:172.21.15.29:3300/0 conn(0x7fb3200b4ce0 0x7fb32015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3200af0d0 con 0x7fb32015be40 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fb31c001670 con 0x7fb32015be40 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 >> v1:172.21.15.26:6789/0 conn(0x7fb32015be40 legacy=0x7fb3200d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb30c0039b0 0x7fb30c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c004800 0x7fb30c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3200d0750 0x7fb30c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb3200aedf0 con 0x7fb30c0039b0 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c00b9c0 con 0x7fb30c004800 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.113+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb30c00bb40 con 0x7fb3200d0750 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.114+0000 7fb325b95640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb30c0039b0 0x7fb30c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.114+0000 7fb325394640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c004800 0x7fb30c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.114+0000 7fb326396640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3200d0750 0x7fb30c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.114+0000 7fb325b95640 1 -- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3200d0750 msgr2=0x7fb30c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.114+0000 7fb325b95640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3200d0750 0x7fb30c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.114+0000 7fb325b95640 1 -- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c004800 msgr2=0x7fb30c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.114+0000 7fb325b95640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c004800 0x7fb30c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.114+0000 7fb325b95640 1 -- 172.21.15.26:0/2045616631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c003520 con 0x7fb30c0039b0 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.114+0000 7fb326396640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb3200d0750 0x7fb30c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb325b95640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb30c0039b0 0x7fb30c003dd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00c240 tx=0x7fb30c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab41a8fc1a86f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c02e070 con 0x7fb30c0039b0 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb31c005af0 con 0x7fb30c0039b0 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb324b93640 1 -- 172.21.15.26:0/2045616631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c007d30 con 0x7fb30c0039b0 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb327df9640 1 -- 172.21.15.26:0/2045616631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb30c0039b0 msgr2=0x7fb30c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb327df9640 1 --2- 172.21.15.26:0/2045616631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb30c0039b0 0x7fb30c003dd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00c240 tx=0x7fb30c005830 comp rx=0 tx=0).stop 2024-04-24T23:26:03.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb326396640 1 -- 172.21.15.26:0/2045616631 reap_dead start 2024-04-24T23:26:03.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb327df9640 1 -- 172.21.15.26:0/2045616631 shutdown_connections 2024-04-24T23:26:03.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb327df9640 1 -- 172.21.15.26:0/2045616631 >> 172.21.15.26:0/2045616631 conn(0x7fb32006b540 msgr2=0x7fb32006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb327df9640 1 -- 172.21.15.26:0/2045616631 shutdown_connections 2024-04-24T23:26:03.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.115+0000 7fb327df9640 1 -- 172.21.15.26:0/2045616631 wait complete. 2024-04-24T23:26:03.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb327df9640 1 Processor -- start 2024-04-24T23:26:03.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb327df9640 1 -- start start 2024-04-24T23:26:03.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb327df9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c0039b0 0x7fb32009c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb327df9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb32014bd40 0x7fb32009c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb327df9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200722b0 0x7fb320072760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb327df9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb32006cd10 con 0x7fb3200722b0 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb327df9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb32006cb90 con 0x7fb30c0039b0 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb327df9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb3200acd30 con 0x7fb32014bd40 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb326396640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200722b0 0x7fb320072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb326396640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200722b0 0x7fb320072760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50438/0 (socket says 172.21.15.26:50438) 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.117+0000 7fb326396640 1 -- 172.21.15.26:0/92476563 learned_addr learned my addr 172.21.15.26:0/92476563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb325b95640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c0039b0 0x7fb32009c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb325394640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb32014bd40 0x7fb32009c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb326396640 1 -- 172.21.15.26:0/92476563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb32014bd40 msgr2=0x7fb32009c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb326396640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb32014bd40 0x7fb32009c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb326396640 1 -- 172.21.15.26:0/92476563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c0039b0 msgr2=0x7fb32009c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb326396640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c0039b0 0x7fb32009c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb326396640 1 -- 172.21.15.26:0/92476563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb320072ca0 con 0x7fb3200722b0 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb325b95640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c0039b0 0x7fb32009c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb326396640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200722b0 0x7fb320072760 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb32006ef20 tx=0x7fb318010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb3167fc640 1 -- 172.21.15.26:0/92476563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318010da0 con 0x7fb3200722b0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb3167fc640 1 -- 172.21.15.26:0/92476563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb318003990 con 0x7fb3200722b0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb320070fb0 con 0x7fb3200722b0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb3167fc640 1 -- 172.21.15.26:0/92476563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318003dc0 con 0x7fb3200722b0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.118+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb320071570 con 0x7fb3200722b0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.119+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb320075610 con 0x7fb3200722b0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.119+0000 7fb3167fc640 1 -- 172.21.15.26:0/92476563 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb31801e020 con 0x7fb3200722b0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.120+0000 7fb3167fc640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb30406be30 0x7fb30406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.120+0000 7fb3167fc640 1 -- 172.21.15.26:0/92476563 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fb31808b3f0 con 0x7fb3200722b0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.120+0000 7fb325b95640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb30406be30 0x7fb30406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.120+0000 7fb325b95640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb30406be30 0x7fb30406e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb31c033000 tx=0x7fb31c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.124+0000 7fb3167fc640 1 -- 172.21.15.26:0/92476563 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb318059d60 con 0x7fb3200722b0 2024-04-24T23:26:03.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.307+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb32006d120 con 0x7fb3200722b0 2024-04-24T23:26:03.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.307+0000 7fb3167fc640 1 -- 172.21.15.26:0/92476563 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fb318059660 con 0x7fb3200722b0 2024-04-24T23:26:03.308 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:26:03.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb30406be30 msgr2=0x7fb30406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb30406be30 0x7fb30406e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb31c033000 tx=0x7fb31c009030 comp rx=0 tx=0).stop 2024-04-24T23:26:03.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200722b0 msgr2=0x7fb320072760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200722b0 0x7fb320072760 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb32006ef20 tx=0x7fb318010590 comp rx=0 tx=0).stop 2024-04-24T23:26:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 shutdown_connections 2024-04-24T23:26:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb30406be30 0x7fb30406e2f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200722b0 0x7fb320072760 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb32014bd40 0x7fb32009c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 --2- 172.21.15.26:0/92476563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb30c0039b0 0x7fb32009c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 >> 172.21.15.26:0/92476563 conn(0x7fb32006b540 msgr2=0x7fb3201596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 shutdown_connections 2024-04-24T23:26:03.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.309+0000 7fb327df9640 1 -- 172.21.15.26:0/92476563 wait complete. 2024-04-24T23:26:03.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:26:03.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.424+0000 7f82b42f5640 1 Processor -- start 2024-04-24T23:26:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.424+0000 7f82b42f5640 1 -- start start 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b42f5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82ac15adc0 0x7f82ac0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b42f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac0ad7b0 0x7f82ac0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b42f5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82ac15ef60 con 0x7f82ac15e9c0 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b42f5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f82ac15f240 con 0x7f82ac15adc0 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b42f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac15f520 con 0x7f82ac0ad7b0 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b2091640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82ac15adc0 0x7f82ac0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b2091640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82ac15adc0 0x7f82ac0b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50440/0 (socket says 172.21.15.26:50440) 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b2091640 1 -- 172.21.15.26:0/2945145084 learned_addr learned my addr 172.21.15.26:0/2945145084 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b2091640 1 -- 172.21.15.26:0/2945145084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac0ad7b0 msgr2=0x7f82ac0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b1890640 1 --2- 172.21.15.26:0/2945145084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac0ad7b0 0x7f82ac0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b2091640 1 --2- 172.21.15.26:0/2945145084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac0ad7b0 0x7f82ac0d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b2091640 1 -- 172.21.15.26:0/2945145084 >> v1:172.21.15.29:6789/0 conn(0x7f82ac15e9c0 legacy=0x7f82ac06dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:03.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b2091640 1 -- 172.21.15.26:0/2945145084 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac15f800 con 0x7f82ac15adc0 2024-04-24T23:26:03.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.425+0000 7f82b1890640 1 --2- 172.21.15.26:0/2945145084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac0ad7b0 0x7f82ac0d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:03.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.426+0000 7f82b2091640 1 --2- 172.21.15.26:0/2945145084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 0x7f82ac0b4370 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009870 tx=0x7f82a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c60ef01ae5dace server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.426+0000 7f82b108f640 1 -- 172.21.15.26:0/2945145084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8037070 con 0x7f82ac15adc0 2024-04-24T23:26:03.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.426+0000 7f82b108f640 1 -- 172.21.15.26:0/2945145084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a8032440 con 0x7f82ac15adc0 2024-04-24T23:26:03.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.426+0000 7f82b108f640 1 -- 172.21.15.26:0/2945145084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8032810 con 0x7f82ac15adc0 2024-04-24T23:26:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.426+0000 7f82b42f5640 1 -- 172.21.15.26:0/2945145084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 msgr2=0x7f82ac0b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.426+0000 7f82b42f5640 1 --2- 172.21.15.26:0/2945145084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 0x7f82ac0b4370 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009870 tx=0x7f82a802a180 comp rx=0 tx=0).stop 2024-04-24T23:26:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.427+0000 7f82b42f5640 1 -- 172.21.15.26:0/2945145084 shutdown_connections 2024-04-24T23:26:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.427+0000 7f82b42f5640 1 --2- 172.21.15.26:0/2945145084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac0ad7b0 0x7f82ac0d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.427+0000 7f82b42f5640 1 --2- 172.21.15.26:0/2945145084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 0x7f82ac0b4370 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.427+0000 7f82b42f5640 1 -- 172.21.15.26:0/2945145084 >> 172.21.15.26:0/2945145084 conn(0x7f82ac06b540 msgr2=0x7f82ac06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.427+0000 7f82b42f5640 1 -- 172.21.15.26:0/2945145084 shutdown_connections 2024-04-24T23:26:03.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.427+0000 7f82b42f5640 1 -- 172.21.15.26:0/2945145084 wait complete. 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.428+0000 7f82b42f5640 1 Processor -- start 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.428+0000 7f82b42f5640 1 -- start start 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.428+0000 7f82b42f5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac0ad7b0 0x7f82ac152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.428+0000 7f82b42f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 0x7f82ac0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.428+0000 7f82b42f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac15e9c0 0x7f82ac0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.428+0000 7f82b42f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac15b1a0 con 0x7f82ac15adc0 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.428+0000 7f82b42f5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac0b48b0 con 0x7f82ac0ad7b0 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.428+0000 7f82b42f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac0d0580 con 0x7f82ac15e9c0 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b1890640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 0x7f82ac0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b1890640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 0x7f82ac0b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50442/0 (socket says 172.21.15.26:50442) 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b1890640 1 -- 172.21.15.26:0/133349331 learned_addr learned my addr 172.21.15.26:0/133349331 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b2091640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac0ad7b0 0x7f82ac152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b2892640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac15e9c0 0x7f82ac0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b1890640 1 -- 172.21.15.26:0/133349331 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac15e9c0 msgr2=0x7f82ac0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b1890640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac15e9c0 0x7f82ac0cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b1890640 1 -- 172.21.15.26:0/133349331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac0ad7b0 msgr2=0x7f82ac152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b1890640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac0ad7b0 0x7f82ac152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b1890640 1 -- 172.21.15.26:0/133349331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac0ce220 con 0x7f82ac15adc0 2024-04-24T23:26:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b1890640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 0x7f82ac0b1a00 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c7f0 tx=0x7f829c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82a2ffd640 1 -- 172.21.15.26:0/133349331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c015070 con 0x7f82ac15adc0 2024-04-24T23:26:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82a2ffd640 1 -- 172.21.15.26:0/133349331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c007a30 con 0x7f82ac15adc0 2024-04-24T23:26:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac0ce500 con 0x7f82ac15adc0 2024-04-24T23:26:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82a2ffd640 1 -- 172.21.15.26:0/133349331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c007e00 con 0x7f82ac15adc0 2024-04-24T23:26:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.429+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac15c8c0 con 0x7f82ac15adc0 2024-04-24T23:26:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.430+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac075610 con 0x7f82ac15adc0 2024-04-24T23:26:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.431+0000 7f82a2ffd640 1 -- 172.21.15.26:0/133349331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f829c018070 con 0x7f82ac15adc0 2024-04-24T23:26:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.431+0000 7f82a2ffd640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f829006be60 0x7f829006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.431+0000 7f82a2ffd640 1 -- 172.21.15.26:0/133349331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f829c08ad50 con 0x7f82ac15adc0 2024-04-24T23:26:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.431+0000 7f82b2091640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f829006be60 0x7f829006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.432+0000 7f82b2091640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f829006be60 0x7f829006e320 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f82a8034000 tx=0x7f82a802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.436+0000 7f82a2ffd640 1 -- 172.21.15.26:0/133349331 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f829c08e030 con 0x7f82ac15adc0 2024-04-24T23:26:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.622+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82ac15e730 con 0x7f82ac15adc0 2024-04-24T23:26:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.622+0000 7f82a2ffd640 1 -- 172.21.15.26:0/133349331 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f829c059100 con 0x7f82ac15adc0 2024-04-24T23:26:03.623 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f829006be60 msgr2=0x7f829006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f829006be60 0x7f829006e320 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f82a8034000 tx=0x7f82a802e000 comp rx=0 tx=0).stop 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 msgr2=0x7f82ac0b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 0x7f82ac0b1a00 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c7f0 tx=0x7f829c00cbf0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 shutdown_connections 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f829006be60 0x7f829006e320 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82ac15e9c0 0x7f82ac0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15adc0 0x7f82ac0b1a00 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 --2- 172.21.15.26:0/133349331 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82ac0ad7b0 0x7f82ac152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 >> 172.21.15.26:0/133349331 conn(0x7f82ac06b540 msgr2=0x7f82ac0b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 shutdown_connections 2024-04-24T23:26:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.624+0000 7f82b42f5640 1 -- 172.21.15.26:0/133349331 wait complete. 2024-04-24T23:26:03.636 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:26:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.738+0000 7f6f1266b640 1 Processor -- start 2024-04-24T23:26:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.738+0000 7f6f1266b640 1 -- start start 2024-04-24T23:26:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f1266b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0cdf30 0x7f6f0c0ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f1266b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f0c0b3150 0x7f6f0c0cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f1266b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f0c0b1760 0x7f6f0c0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f1266b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0c0d00d0 con 0x7f6f0c0b1760 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f1266b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0c06c5f0 con 0x7f6f0c0b3150 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f1266b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c06c840 con 0x7f6f0c0cdf30 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f10c08640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f0c0b1760 0x7f6f0c0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f10c08640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f0c0b1760 0x7f6f0c0b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53410/0 (socket says 172.21.15.26:53410) 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f10c08640 1 -- 172.21.15.26:0/1428539844 learned_addr learned my addr 172.21.15.26:0/1428539844 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f0bfff640 1 --2- 172.21.15.26:0/1428539844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0cdf30 0x7f6f0c0ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.739+0000 7f6f0b7fe640 1 --2- 172.21.15.26:0/1428539844 >> v2:172.21.15.29:3300/0 conn(0x7f6f0c0b3150 0x7f6f0c0cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.740+0000 7f6f10c08640 1 -- 172.21.15.26:0/1428539844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0cdf30 msgr2=0x7f6f0c0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.740+0000 7f6f10c08640 1 --2- 172.21.15.26:0/1428539844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0cdf30 0x7f6f0c0ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.740+0000 7f6f10c08640 1 -- 172.21.15.26:0/1428539844 >> v2:172.21.15.29:3300/0 conn(0x7f6f0c0b3150 msgr2=0x7f6f0c0cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.740+0000 7f6f10c08640 1 --2- 172.21.15.26:0/1428539844 >> v2:172.21.15.29:3300/0 conn(0x7f6f0c0b3150 0x7f6f0c0cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.740+0000 7f6f10c08640 1 -- 172.21.15.26:0/1428539844 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c06cb20 con 0x7f6f0c0b1760 2024-04-24T23:26:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.740+0000 7f6f0b7fe640 1 --2- 172.21.15.26:0/1428539844 >> v2:172.21.15.29:3300/0 conn(0x7f6f0c0b3150 0x7f6f0c0cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.740+0000 7f6f10c08640 1 --2- 172.21.15.26:0/1428539844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b1760 0x7f6f0c0b1bf0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009e50 tx=0x7f6f0402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c919765c2a6273 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.740+0000 7f6f0affd640 1 -- 172.21.15.26:0/1428539844 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04036070 con 0x7f6f0c0b1760 2024-04-24T23:26:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f0affd640 1 -- 172.21.15.26:0/1428539844 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f04031600 con 0x7f6f0c0b1760 2024-04-24T23:26:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f0affd640 1 -- 172.21.15.26:0/1428539844 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040319d0 con 0x7f6f0c0b1760 2024-04-24T23:26:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f1266b640 1 -- 172.21.15.26:0/1428539844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b1760 msgr2=0x7f6f0c0b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f1266b640 1 --2- 172.21.15.26:0/1428539844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b1760 0x7f6f0c0b1bf0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009e50 tx=0x7f6f0402a6c0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f1266b640 1 -- 172.21.15.26:0/1428539844 shutdown_connections 2024-04-24T23:26:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f1266b640 1 --2- 172.21.15.26:0/1428539844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b1760 0x7f6f0c0b1bf0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f1266b640 1 --2- 172.21.15.26:0/1428539844 >> v2:172.21.15.29:3300/0 conn(0x7f6f0c0b3150 0x7f6f0c0cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f1266b640 1 --2- 172.21.15.26:0/1428539844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0cdf30 0x7f6f0c0ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f1266b640 1 -- 172.21.15.26:0/1428539844 >> 172.21.15.26:0/1428539844 conn(0x7f6f0c0d1e00 msgr2=0x7f6f0c0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f1266b640 1 -- 172.21.15.26:0/1428539844 shutdown_connections 2024-04-24T23:26:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.741+0000 7f6f1266b640 1 -- 172.21.15.26:0/1428539844 wait complete. 2024-04-24T23:26:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.742+0000 7f6f1266b640 1 Processor -- start 2024-04-24T23:26:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.742+0000 7f6f1266b640 1 -- start start 2024-04-24T23:26:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f1266b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0b1760 0x7f6f0c153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f1266b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b3150 0x7f6f0c0c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f1266b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f0c0cdf30 0x7f6f0c0b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f1266b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c06d3c0 con 0x7f6f0c0b3150 2024-04-24T23:26:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f1266b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c06d0e0 con 0x7f6f0c0cdf30 2024-04-24T23:26:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f1266b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c06be30 con 0x7f6f0c0b1760 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f0b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b3150 0x7f6f0c0c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f0b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b3150 0x7f6f0c0c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53420/0 (socket says 172.21.15.26:53420) 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f0b7fe640 1 -- 172.21.15.26:0/3399148494 learned_addr learned my addr 172.21.15.26:0/3399148494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f10c08640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f0c0cdf30 0x7f6f0c0b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.743+0000 7f6f0bfff640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0b1760 0x7f6f0c153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f0b7fe640 1 -- 172.21.15.26:0/3399148494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0b1760 msgr2=0x7f6f0c153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f0b7fe640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0b1760 0x7f6f0c153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f0b7fe640 1 -- 172.21.15.26:0/3399148494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f0c0cdf30 msgr2=0x7f6f0c0b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f0b7fe640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f0c0cdf30 0x7f6f0c0b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f0b7fe640 1 -- 172.21.15.26:0/3399148494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c0b56d0 con 0x7f6f0c0b3150 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f10c08640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f0c0cdf30 0x7f6f0c0b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f0b7fe640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b3150 0x7f6f0c0c7f60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000e810 tx=0x7f6f0000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f08ff9640 1 -- 172.21.15.26:0/3399148494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000f040 con 0x7f6f0c0b3150 2024-04-24T23:26:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f08ff9640 1 -- 172.21.15.26:0/3399148494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f00004030 con 0x7f6f0c0b3150 2024-04-24T23:26:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c0b59b0 con 0x7f6f0c0b3150 2024-04-24T23:26:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f08ff9640 1 -- 172.21.15.26:0/3399148494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00004400 con 0x7f6f0c0b3150 2024-04-24T23:26:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.744+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c154540 con 0x7f6f0c0b3150 2024-04-24T23:26:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.745+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0c075610 con 0x7f6f0c0b3150 2024-04-24T23:26:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.745+0000 7f6f08ff9640 1 -- 172.21.15.26:0/3399148494 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f0000c6d0 con 0x7f6f0c0b3150 2024-04-24T23:26:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.746+0000 7f6f08ff9640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ef806be60 0x7f6ef806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.746+0000 7f6f08ff9640 1 -- 172.21.15.26:0/3399148494 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f6f0008aac0 con 0x7f6f0c0b3150 2024-04-24T23:26:03.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.746+0000 7f6f0bfff640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ef806be60 0x7f6ef806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.746+0000 7f6f0bfff640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ef806be60 0x7f6ef806e320 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009970 tx=0x7f6efc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.750+0000 7f6f08ff9640 1 -- 172.21.15.26:0/3399148494 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6f00017020 con 0x7f6f0c0b3150 2024-04-24T23:26:03.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.932+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6f0c0d1720 con 0x7f6f0c0b3150 2024-04-24T23:26:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.932+0000 7f6f08ff9640 1 -- 172.21.15.26:0/3399148494 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f6f000594c0 con 0x7f6f0c0b3150 2024-04-24T23:26:03.934 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:26:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ef806be60 msgr2=0x7f6ef806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ef806be60 0x7f6ef806e320 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009970 tx=0x7f6efc008040 comp rx=0 tx=0).stop 2024-04-24T23:26:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b3150 msgr2=0x7f6f0c0c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b3150 0x7f6f0c0c7f60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000e810 tx=0x7f6f0000ecd0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 shutdown_connections 2024-04-24T23:26:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ef806be60 0x7f6ef806e320 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f0c0cdf30 0x7f6f0c0b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0b3150 0x7f6f0c0c7f60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 --2- 172.21.15.26:0/3399148494 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f0c0b1760 0x7f6f0c153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 >> 172.21.15.26:0/3399148494 conn(0x7f6f0c0d1e00 msgr2=0x7f6f0c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 shutdown_connections 2024-04-24T23:26:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:03.934+0000 7f6f1266b640 1 -- 172.21.15.26:0/3399148494 wait complete. 2024-04-24T23:26:03.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:04.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.046+0000 7fd77f7ee640 1 Processor -- start 2024-04-24T23:26:04.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.046+0000 7fd77f7ee640 1 -- start start 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77f7ee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd778145980 0x7fd778073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77f7ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd77806b830 0x7fd7780cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77f7ee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd77815c7c0 con 0x7fd77815d7e0 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77f7ee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd778145d60 con 0x7fd77806b830 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77f7ee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd77815caa0 con 0x7fd778145980 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77cd89640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd77815d7e0 0x7fd77806b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55302/0 (socket says 172.21.15.26:55302) 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77dd8b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd77806b830 0x7fd7780cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77cd89640 1 -- 172.21.15.26:0/2794861393 learned_addr learned my addr 172.21.15.26:0/2794861393 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77d58a640 1 --2- 172.21.15.26:0/2794861393 >> v2:172.21.15.29:3300/0 conn(0x7fd778145980 0x7fd778073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd76ffff640 1 -- 172.21.15.26:0/2794861393 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 450755852 0 0) 0x7fd77815c7c0 con 0x7fd77815d7e0 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd76ffff640 1 -- 172.21.15.26:0/2794861393 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd764003520 con 0x7fd77815d7e0 2024-04-24T23:26:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77dd8b640 1 -- 172.21.15.26:0/2794861393 >> v2:172.21.15.29:3300/0 conn(0x7fd778145980 msgr2=0x7fd778073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77dd8b640 1 --2- 172.21.15.26:0/2794861393 >> v2:172.21.15.29:3300/0 conn(0x7fd778145980 0x7fd778073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77dd8b640 1 -- 172.21.15.26:0/2794861393 >> v1:172.21.15.26:6789/0 conn(0x7fd77815d7e0 legacy=0x7fd77806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.047+0000 7fd77dd8b640 1 -- 172.21.15.26:0/2794861393 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77815cd80 con 0x7fd77806b830 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.048+0000 7fd77dd8b640 1 --2- 172.21.15.26:0/2794861393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 0x7fd7780cd950 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd77806bf00 tx=0x7fd77002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd0fa8c9aa21623 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.048+0000 7fd76ffff640 1 -- 172.21.15.26:0/2794861393 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770007740 con 0x7fd77806b830 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.048+0000 7fd76ffff640 1 -- 172.21.15.26:0/2794861393 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7700078e0 con 0x7fd77806b830 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.048+0000 7fd76ffff640 1 -- 172.21.15.26:0/2794861393 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770007cb0 con 0x7fd77806b830 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.049+0000 7fd77f7ee640 1 -- 172.21.15.26:0/2794861393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 msgr2=0x7fd7780cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.049+0000 7fd77f7ee640 1 --2- 172.21.15.26:0/2794861393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 0x7fd7780cd950 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd77806bf00 tx=0x7fd77002a680 comp rx=0 tx=0).stop 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.049+0000 7fd77f7ee640 1 -- 172.21.15.26:0/2794861393 shutdown_connections 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.049+0000 7fd77f7ee640 1 --2- 172.21.15.26:0/2794861393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 0x7fd7780cd950 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.049+0000 7fd77f7ee640 1 --2- 172.21.15.26:0/2794861393 >> v2:172.21.15.29:3300/0 conn(0x7fd778145980 0x7fd778073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.049+0000 7fd77f7ee640 1 -- 172.21.15.26:0/2794861393 >> 172.21.15.26:0/2794861393 conn(0x7fd77806e2c0 msgr2=0x7fd77806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.049+0000 7fd77f7ee640 1 -- 172.21.15.26:0/2794861393 shutdown_connections 2024-04-24T23:26:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.049+0000 7fd77f7ee640 1 -- 172.21.15.26:0/2794861393 wait complete. 2024-04-24T23:26:04.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.050+0000 7fd77f7ee640 1 Processor -- start 2024-04-24T23:26:04.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.050+0000 7fd77f7ee640 1 -- start start 2024-04-24T23:26:04.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77f7ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 0x7fd7781e2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77f7ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd778145980 0x7fd778159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77f7ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77815d7e0 0x7fd778142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77f7ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd778072320 con 0x7fd77806b830 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77f7ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd7780721a0 con 0x7fd77815d7e0 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77f7ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd7780724a0 con 0x7fd778145980 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77d58a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 0x7fd7781e2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77d58a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 0x7fd7781e2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53426/0 (socket says 172.21.15.26:53426) 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77d58a640 1 -- 172.21.15.26:0/699656832 learned_addr learned my addr 172.21.15.26:0/699656832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77cd89640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd778145980 0x7fd778159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77dd8b640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77815d7e0 0x7fd778142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77d58a640 1 -- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd778145980 msgr2=0x7fd778159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77d58a640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd778145980 0x7fd778159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77d58a640 1 -- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77815d7e0 msgr2=0x7fd778142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77d58a640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77815d7e0 0x7fd778142ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77d58a640 1 -- 172.21.15.26:0/699656832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd778143020 con 0x7fd77806b830 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77cd89640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd778145980 0x7fd778159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77dd8b640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77815d7e0 0x7fd778142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.051+0000 7fd77d58a640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 0x7fd7781e2940 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd77400e9b0 tx=0x7fd77400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.052+0000 7fd76dffb640 1 -- 172.21.15.26:0/699656832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774017070 con 0x7fd77806b830 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.052+0000 7fd76dffb640 1 -- 172.21.15.26:0/699656832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77400c9a0 con 0x7fd77806b830 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.052+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7781e3040 con 0x7fd77806b830 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.052+0000 7fd76dffb640 1 -- 172.21.15.26:0/699656832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77400cd90 con 0x7fd77806b830 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.052+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7781e3600 con 0x7fd77806b830 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.053+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7780694c0 con 0x7fd77806b830 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.053+0000 7fd76dffb640 1 -- 172.21.15.26:0/699656832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd77401a070 con 0x7fd77806b830 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.054+0000 7fd76dffb640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd75c06be30 0x7fd75c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.054+0000 7fd76dffb640 1 -- 172.21.15.26:0/699656832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fd77408aee0 con 0x7fd77806b830 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.054+0000 7fd77cd89640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd75c06be30 0x7fd75c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.054+0000 7fd77cd89640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd75c06be30 0x7fd75c06e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd768007fd0 tx=0x7fd768009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.058+0000 7fd76dffb640 1 -- 172.21.15.26:0/699656832 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd778143020 con 0x7fd77806b830 2024-04-24T23:26:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.281+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd77806e8b0 con 0x7fd77806b830 2024-04-24T23:26:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.282+0000 7fd76dffb640 1 -- 172.21.15.26:0/699656832 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5227 (secure 0 0 0) 0x7fd774059840 con 0x7fd77806b830 2024-04-24T23:26:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:04.283 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:26:02.966810+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7227,"mds_1":7224,"mds_2":6364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6364":{"gid":6364,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7224":{"gid":7224,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[7227,7224,6364]},"id":5}]} 2024-04-24T23:26:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 75 2024-04-24T23:26:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd75c06be30 msgr2=0x7fd75c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd75c06be30 0x7fd75c06e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd768007fd0 tx=0x7fd768009040 comp rx=0 tx=0).stop 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 msgr2=0x7fd7781e2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 0x7fd7781e2940 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd77400e9b0 tx=0x7fd77400ee70 comp rx=0 tx=0).stop 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 shutdown_connections 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd75c06be30 0x7fd75c06e2f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd77815d7e0 0x7fd778142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd778145980 0x7fd778159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 --2- 172.21.15.26:0/699656832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77806b830 0x7fd7781e2940 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 >> 172.21.15.26:0/699656832 conn(0x7fd77806e2c0 msgr2=0x7fd77815bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 shutdown_connections 2024-04-24T23:26:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.284+0000 7fd77f7ee640 1 -- 172.21.15.26:0/699656832 wait complete. 2024-04-24T23:26:04.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.396+0000 7f507f5b0640 1 Processor -- start 2024-04-24T23:26:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.396+0000 7f507f5b0640 1 -- start start 2024-04-24T23:26:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507f5b0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5078143c40 0x7f50780cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507f5b0640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 0x7f5078072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507f5b0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50780cd0b0 con 0x7f507815eb00 2024-04-24T23:26:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507f5b0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5078071b70 con 0x7f5078143c40 2024-04-24T23:26:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507f5b0640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5078071240 con 0x7f50780cc860 2024-04-24T23:26:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507d34c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5078143c40 0x7f50780cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507db4d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f507815eb00 0x7f50780b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55310/0 (socket says 172.21.15.26:55310) 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507d34c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5078143c40 0x7f50780cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53436/0 (socket says 172.21.15.26:53436) 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507db4d640 1 -- 172.21.15.26:0/1735204608 learned_addr learned my addr 172.21.15.26:0/1735204608 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507cb4b640 1 --2- 172.21.15.26:0/1735204608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 0x7f5078072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507d34c640 1 -- 172.21.15.26:0/1735204608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 msgr2=0x7f5078072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507d34c640 1 --2- 172.21.15.26:0/1735204608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 0x7f5078072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.397+0000 7f507d34c640 1 -- 172.21.15.26:0/1735204608 >> v1:172.21.15.26:6789/0 conn(0x7f507815eb00 legacy=0x7f50780b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.398+0000 7f507d34c640 1 -- 172.21.15.26:0/1735204608 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5078071520 con 0x7f5078143c40 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.398+0000 7f5077fff640 1 -- 172.21.15.26:0/1735204608 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2820381685 0 0) 0x7f50780cd0b0 con 0x7f507815eb00 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.398+0000 7f507d34c640 1 --2- 172.21.15.26:0/1735204608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 0x7f50780cf870 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f50700097a0 tx=0x7f507002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540e6c94adbcbd3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.398+0000 7f5077fff640 1 -- 172.21.15.26:0/1735204608 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070037070 con 0x7f5078143c40 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.398+0000 7f5077fff640 1 -- 172.21.15.26:0/1735204608 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5070032440 con 0x7f5078143c40 2024-04-24T23:26:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.398+0000 7f5077fff640 1 -- 172.21.15.26:0/1735204608 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070032810 con 0x7f5078143c40 2024-04-24T23:26:04.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.399+0000 7f507f5b0640 1 -- 172.21.15.26:0/1735204608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 msgr2=0x7f50780cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.399+0000 7f507f5b0640 1 --2- 172.21.15.26:0/1735204608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 0x7f50780cf870 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f50700097a0 tx=0x7f507002a180 comp rx=0 tx=0).stop 2024-04-24T23:26:04.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.399+0000 7f507f5b0640 1 -- 172.21.15.26:0/1735204608 shutdown_connections 2024-04-24T23:26:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.399+0000 7f507f5b0640 1 --2- 172.21.15.26:0/1735204608 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 0x7f5078072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.399+0000 7f507f5b0640 1 --2- 172.21.15.26:0/1735204608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 0x7f50780cf870 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.399+0000 7f507f5b0640 1 -- 172.21.15.26:0/1735204608 >> 172.21.15.26:0/1735204608 conn(0x7f507806b300 msgr2=0x7f507806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.399+0000 7f507f5b0640 1 -- 172.21.15.26:0/1735204608 shutdown_connections 2024-04-24T23:26:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.399+0000 7f507f5b0640 1 -- 172.21.15.26:0/1735204608 wait complete. 2024-04-24T23:26:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.400+0000 7f507f5b0640 1 Processor -- start 2024-04-24T23:26:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.400+0000 7f507f5b0640 1 -- start start 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507f5b0640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 0x7f50780a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507f5b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 0x7f50780c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507f5b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f507815eb00 0x7f507815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507f5b0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5078072540 con 0x7f5078143c40 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507f5b0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5078144020 con 0x7f507815eb00 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507f5b0640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f50780aa790 con 0x7f50780cc860 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507cb4b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 0x7f50780c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507cb4b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 0x7f50780c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53448/0 (socket says 172.21.15.26:53448) 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507cb4b640 1 -- 172.21.15.26:0/1330699773 learned_addr learned my addr 172.21.15.26:0/1330699773 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507d34c640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 0x7f50780a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507db4d640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f507815eb00 0x7f507815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507cb4b640 1 -- 172.21.15.26:0/1330699773 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 msgr2=0x7f50780a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507cb4b640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 0x7f50780a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507cb4b640 1 -- 172.21.15.26:0/1330699773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f507815eb00 msgr2=0x7f507815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507cb4b640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f507815eb00 0x7f507815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507cb4b640 1 -- 172.21.15.26:0/1330699773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5078150c90 con 0x7f5078143c40 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.401+0000 7f507d34c640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 0x7f50780a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.402+0000 7f507cb4b640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 0x7f50780c9e10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f506c00d800 tx=0x7f506c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.402+0000 7f5075ffb640 1 -- 172.21.15.26:0/1330699773 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c00b6c0 con 0x7f5078143c40 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.402+0000 7f5075ffb640 1 -- 172.21.15.26:0/1330699773 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c019070 con 0x7f5078143c40 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.402+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50780c9330 con 0x7f5078143c40 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.402+0000 7f5075ffb640 1 -- 172.21.15.26:0/1330699773 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c00ba80 con 0x7f5078143c40 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.402+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50780ca350 con 0x7f5078143c40 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.403+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50780694c0 con 0x7f5078143c40 2024-04-24T23:26:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.403+0000 7f5075ffb640 1 -- 172.21.15.26:0/1330699773 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f506c00bc20 con 0x7f5078143c40 2024-04-24T23:26:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.404+0000 7f5075ffb640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f506406be30 0x7f506406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.404+0000 7f5075ffb640 1 -- 172.21.15.26:0/1330699773 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f506c08b2e0 con 0x7f5078143c40 2024-04-24T23:26:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.404+0000 7f507d34c640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f506406be30 0x7f506406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.404+0000 7f507d34c640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f506406be30 0x7f506406e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5070034000 tx=0x7f507002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.408+0000 7f5075ffb640 1 -- 172.21.15.26:0/1330699773 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f506c059ce0 con 0x7f5078143c40 2024-04-24T23:26:04.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.591+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50780b4d20 con 0x7f5078143c40 2024-04-24T23:26:04.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.593+0000 7f5075ffb640 1 -- 172.21.15.26:0/1330699773 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16706 (secure 0 0 0) 0x7f506c0595e0 con 0x7f5078143c40 2024-04-24T23:26:04.594 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:04.594 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:25:37.018656+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:25:25.626812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:25:26.319330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:04.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f506406be30 msgr2=0x7f506406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f506406be30 0x7f506406e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5070034000 tx=0x7f507002e000 comp rx=0 tx=0).stop 2024-04-24T23:26:04.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 msgr2=0x7f50780c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 0x7f50780c9e10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f506c00d800 tx=0x7f506c00dcc0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 shutdown_connections 2024-04-24T23:26:04.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f506406be30 0x7f506406e2f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f507815eb00 0x7f507815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5078143c40 0x7f50780c9e10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 --2- 172.21.15.26:0/1330699773 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f50780cc860 0x7f50780a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 >> 172.21.15.26:0/1330699773 conn(0x7f507806b300 msgr2=0x7f507813f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 shutdown_connections 2024-04-24T23:26:04.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.595+0000 7f507f5b0640 1 -- 172.21.15.26:0/1330699773 wait complete. 2024-04-24T23:26:04.607 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:26:04.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.706+0000 7f7156429640 1 Processor -- start 2024-04-24T23:26:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.706+0000 7f7156429640 1 -- start start 2024-04-24T23:26:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.707+0000 7f7156429640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 0x7f71500d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.707+0000 7f7156429640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71500b3130 0x7f71500d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.707+0000 7f7156429640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71500d32f0 con 0x7f71500d22e0 2024-04-24T23:26:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.707+0000 7f7156429640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f71500d1ab0 con 0x7f71500b3130 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.707+0000 7f7156429640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f71500d2ae0 con 0x7f715015e570 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.707+0000 7f714f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71500b3130 0x7f71500d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.707+0000 7f714f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71500b3130 0x7f71500d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53458/0 (socket says 172.21.15.26:53458) 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.707+0000 7f714f7fe640 1 -- 172.21.15.26:0/1741740204 learned_addr learned my addr 172.21.15.26:0/1741740204 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.708+0000 7f714ffff640 1 --2- 172.21.15.26:0/1741740204 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 0x7f71500d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.708+0000 7f714f7fe640 1 -- 172.21.15.26:0/1741740204 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 msgr2=0x7f71500d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.708+0000 7f714f7fe640 1 --2- 172.21.15.26:0/1741740204 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 0x7f71500d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.708+0000 7f714f7fe640 1 -- 172.21.15.26:0/1741740204 >> v1:172.21.15.29:6789/0 conn(0x7f71500d22e0 legacy=0x7f71500d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.708+0000 7f714f7fe640 1 -- 172.21.15.26:0/1741740204 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71500d0270 con 0x7f71500b3130 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.708+0000 7f714f7fe640 1 --2- 172.21.15.26:0/1741740204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500b3130 0x7f71500d46a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7144009870 tx=0x7f714402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6585024de8a20347 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f714effd640 1 -- 172.21.15.26:0/1741740204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144037070 con 0x7f71500b3130 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f714effd640 1 -- 172.21.15.26:0/1741740204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7144032440 con 0x7f71500b3130 2024-04-24T23:26:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f714effd640 1 -- 172.21.15.26:0/1741740204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144032810 con 0x7f71500b3130 2024-04-24T23:26:04.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f7156429640 1 -- 172.21.15.26:0/1741740204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500b3130 msgr2=0x7f71500d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f7156429640 1 --2- 172.21.15.26:0/1741740204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500b3130 0x7f71500d46a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7144009870 tx=0x7f714402a210 comp rx=0 tx=0).stop 2024-04-24T23:26:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f7156429640 1 -- 172.21.15.26:0/1741740204 shutdown_connections 2024-04-24T23:26:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f7156429640 1 --2- 172.21.15.26:0/1741740204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71500b3130 0x7f71500d46a0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f7156429640 1 --2- 172.21.15.26:0/1741740204 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f715015e570 0x7f71500d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f7156429640 1 -- 172.21.15.26:0/1741740204 >> 172.21.15.26:0/1741740204 conn(0x7f715008dbe0 msgr2=0x7f715009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f7156429640 1 -- 172.21.15.26:0/1741740204 shutdown_connections 2024-04-24T23:26:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.709+0000 7f7156429640 1 -- 172.21.15.26:0/1741740204 wait complete. 2024-04-24T23:26:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.710+0000 7f7156429640 1 Processor -- start 2024-04-24T23:26:04.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.710+0000 7f7156429640 1 -- start start 2024-04-24T23:26:04.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f7156429640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500b3130 0x7f715014f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f7156429640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500d22e0 0x7f715015e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f7156429640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f715015e570 0x7f71500cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f7156429640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71500cfa70 con 0x7f715015e570 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f7156429640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f71500d3010 con 0x7f71500d22e0 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f7156429640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f71500cfd50 con 0x7f71500b3130 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f71549c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f715015e570 0x7f71500cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f71549c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f715015e570 0x7f71500cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53468/0 (socket says 172.21.15.26:53468) 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f71549c6640 1 -- 172.21.15.26:0/1003754653 learned_addr learned my addr 172.21.15.26:0/1003754653 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f714ffff640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500b3130 0x7f715014f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.711+0000 7f714f7fe640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500d22e0 0x7f715015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f71549c6640 1 -- 172.21.15.26:0/1003754653 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500b3130 msgr2=0x7f715014f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f71549c6640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500b3130 0x7f715014f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f71549c6640 1 -- 172.21.15.26:0/1003754653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500d22e0 msgr2=0x7f715015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f71549c6640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500d22e0 0x7f715015e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f71549c6640 1 -- 172.21.15.26:0/1003754653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71500cc8d0 con 0x7f715015e570 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f71549c6640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f715015e570 0x7f71500cd970 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f714800c7b0 tx=0x7f714800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f714cff9640 1 -- 172.21.15.26:0/1003754653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148004b60 con 0x7f715015e570 2024-04-24T23:26:04.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f714cff9640 1 -- 172.21.15.26:0/1003754653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7148004d00 con 0x7f715015e570 2024-04-24T23:26:04.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71500ccbb0 con 0x7f715015e570 2024-04-24T23:26:04.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f714cff9640 1 -- 172.21.15.26:0/1003754653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714800aa90 con 0x7f715015e570 2024-04-24T23:26:04.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.712+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71500cd170 con 0x7f715015e570 2024-04-24T23:26:04.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.713+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71500d3d70 con 0x7f715015e570 2024-04-24T23:26:04.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.713+0000 7f714cff9640 1 -- 172.21.15.26:0/1003754653 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f714800ac30 con 0x7f715015e570 2024-04-24T23:26:04.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.714+0000 7f714cff9640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713c06be60 0x7f713c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.714+0000 7f714cff9640 1 -- 172.21.15.26:0/1003754653 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f714808b040 con 0x7f715015e570 2024-04-24T23:26:04.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.714+0000 7f714ffff640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713c06be60 0x7f713c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.714+0000 7f714ffff640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713c06be60 0x7f713c06e320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f71400079a0 tx=0x7f7140008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.718+0000 7f714cff9640 1 -- 172.21.15.26:0/1003754653 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f71500d3d70 con 0x7f715015e570 2024-04-24T23:26:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.941+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f715008d9c0 con 0x7f715015e570 2024-04-24T23:26:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.942+0000 7f714cff9640 1 -- 172.21.15.26:0/1003754653 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5227 (secure 0 0 0) 0x7f715008d9c0 con 0x7f715015e570 2024-04-24T23:26:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:04.943 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:26:02.966810+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7227,"mds_1":7224,"mds_2":6364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6364":{"gid":6364,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7224":{"gid":7224,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[7227,7224,6364]},"id":5}]} 2024-04-24T23:26:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 75 2024-04-24T23:26:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713c06be60 msgr2=0x7f713c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713c06be60 0x7f713c06e320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f71400079a0 tx=0x7f7140008040 comp rx=0 tx=0).stop 2024-04-24T23:26:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f715015e570 msgr2=0x7f71500cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f715015e570 0x7f71500cd970 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f714800c7b0 tx=0x7f714800cc70 comp rx=0 tx=0).stop 2024-04-24T23:26:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 shutdown_connections 2024-04-24T23:26:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713c06be60 0x7f713c06e320 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f715015e570 0x7f71500cd970 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71500d22e0 0x7f715015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 --2- 172.21.15.26:0/1003754653 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71500b3130 0x7f715014f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 >> 172.21.15.26:0/1003754653 conn(0x7f715008dbe0 msgr2=0x7f71500d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 shutdown_connections 2024-04-24T23:26:04.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:04.944+0000 7f7156429640 1 -- 172.21.15.26:0/1003754653 wait complete. 2024-04-24T23:26:04.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.056+0000 7ffb03838640 1 Processor -- start 2024-04-24T23:26:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.056+0000 7ffb03838640 1 -- start start 2024-04-24T23:26:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb03838640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc15e290 0x7ffafc0d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb03838640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffafc0b3ac0 0x7ffafc0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb03838640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffafc06c230 0x7ffafc06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb03838640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ffafc0b4460 con 0x7ffafc06c230 2024-04-24T23:26:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb03838640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ffafc0b5e50 con 0x7ffafc0b3ac0 2024-04-24T23:26:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb03838640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ffafc0b5840 con 0x7ffafc15e290 2024-04-24T23:26:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb01dd5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffafc06c230 0x7ffafc06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb01dd5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffafc06c230 0x7ffafc06c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53484/0 (socket says 172.21.15.26:53484) 2024-04-24T23:26:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb01dd5640 1 -- 172.21.15.26:0/3279010859 learned_addr learned my addr 172.21.15.26:0/3279010859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb015d4640 1 --2- 172.21.15.26:0/3279010859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc15e290 0x7ffafc0d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb01dd5640 1 -- 172.21.15.26:0/3279010859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc15e290 msgr2=0x7ffafc0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb00dd3640 1 --2- 172.21.15.26:0/3279010859 >> v2:172.21.15.29:3300/0 conn(0x7ffafc0b3ac0 0x7ffafc0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb01dd5640 1 --2- 172.21.15.26:0/3279010859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc15e290 0x7ffafc0d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb01dd5640 1 -- 172.21.15.26:0/3279010859 >> v2:172.21.15.29:3300/0 conn(0x7ffafc0b3ac0 msgr2=0x7ffafc0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb01dd5640 1 --2- 172.21.15.26:0/3279010859 >> v2:172.21.15.29:3300/0 conn(0x7ffafc0b3ac0 0x7ffafc0b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.057+0000 7ffb01dd5640 1 -- 172.21.15.26:0/3279010859 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc0cfb30 con 0x7ffafc06c230 2024-04-24T23:26:05.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.058+0000 7ffb01dd5640 1 --2- 172.21.15.26:0/3279010859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 0x7ffafc06c6a0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ffafc06bde0 tx=0x7ffaf402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7593567c620c363 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.058+0000 7ffaf3fff640 1 -- 172.21.15.26:0/3279010859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4036070 con 0x7ffafc06c230 2024-04-24T23:26:05.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.058+0000 7ffaf3fff640 1 -- 172.21.15.26:0/3279010859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf4031600 con 0x7ffafc06c230 2024-04-24T23:26:05.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.058+0000 7ffaf3fff640 1 -- 172.21.15.26:0/3279010859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40319d0 con 0x7ffafc06c230 2024-04-24T23:26:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.059+0000 7ffb03838640 1 -- 172.21.15.26:0/3279010859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 msgr2=0x7ffafc06c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.059+0000 7ffb03838640 1 --2- 172.21.15.26:0/3279010859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 0x7ffafc06c6a0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ffafc06bde0 tx=0x7ffaf402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.059+0000 7ffb03838640 1 -- 172.21.15.26:0/3279010859 shutdown_connections 2024-04-24T23:26:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.059+0000 7ffb03838640 1 --2- 172.21.15.26:0/3279010859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 0x7ffafc06c6a0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.059+0000 7ffb03838640 1 --2- 172.21.15.26:0/3279010859 >> v2:172.21.15.29:3300/0 conn(0x7ffafc0b3ac0 0x7ffafc0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.059+0000 7ffb03838640 1 --2- 172.21.15.26:0/3279010859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc15e290 0x7ffafc0d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.059+0000 7ffb03838640 1 -- 172.21.15.26:0/3279010859 >> 172.21.15.26:0/3279010859 conn(0x7ffafc06b120 msgr2=0x7ffafc06b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.059+0000 7ffb03838640 1 -- 172.21.15.26:0/3279010859 shutdown_connections 2024-04-24T23:26:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.059+0000 7ffb03838640 1 -- 172.21.15.26:0/3279010859 wait complete. 2024-04-24T23:26:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.060+0000 7ffb03838640 1 Processor -- start 2024-04-24T23:26:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.060+0000 7ffb03838640 1 -- start start 2024-04-24T23:26:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb03838640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 0x7ffafc14e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb03838640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 0x7ffafc15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb03838640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 0x7ffafc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb03838640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc0b5ae0 con 0x7ffafc06c230 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb03838640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc0d00f0 con 0x7ffafc15e290 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb03838640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ffafc15e670 con 0x7ffafc0b3ac0 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb015d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 0x7ffafc14e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb015d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 0x7ffafc14e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53492/0 (socket says 172.21.15.26:53492) 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb015d4640 1 -- 172.21.15.26:0/2805623857 learned_addr learned my addr 172.21.15.26:0/2805623857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb015d4640 1 -- 172.21.15.26:0/2805623857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 msgr2=0x7ffafc15e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb01dd5640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 0x7ffafc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb00dd3640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 0x7ffafc15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb015d4640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 0x7ffafc15e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb015d4640 1 -- 172.21.15.26:0/2805623857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 msgr2=0x7ffafc15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb015d4640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 0x7ffafc15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb015d4640 1 -- 172.21.15.26:0/2805623857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc15f2c0 con 0x7ffafc06c230 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb01dd5640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 0x7ffafc15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.061+0000 7ffb015d4640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 0x7ffafc14e630 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c820 tx=0x7ffaf800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.062+0000 7ffaf1ffb640 1 -- 172.21.15.26:0/2805623857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8015070 con 0x7ffafc06c230 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.062+0000 7ffaf1ffb640 1 -- 172.21.15.26:0/2805623857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf8004020 con 0x7ffafc06c230 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.062+0000 7ffaf1ffb640 1 -- 172.21.15.26:0/2805623857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80043f0 con 0x7ffafc06c230 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.062+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffafc15f5a0 con 0x7ffafc06c230 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.062+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffafc15fb60 con 0x7ffafc06c230 2024-04-24T23:26:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.063+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffafc0694c0 con 0x7ffafc06c230 2024-04-24T23:26:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.063+0000 7ffaf1ffb640 1 -- 172.21.15.26:0/2805623857 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffaf80077d0 con 0x7ffafc06c230 2024-04-24T23:26:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.064+0000 7ffaf1ffb640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffae006be30 0x7ffae006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.064+0000 7ffaf1ffb640 1 -- 172.21.15.26:0/2805623857 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7ffaf808abb0 con 0x7ffafc06c230 2024-04-24T23:26:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.064+0000 7ffb00dd3640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffae006be30 0x7ffae006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.064+0000 7ffb00dd3640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffae006be30 0x7ffae006e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0cda40 tx=0x7ffaec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.068+0000 7ffaf1ffb640 1 -- 172.21.15.26:0/2805623857 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffaf80595b0 con 0x7ffafc06c230 2024-04-24T23:26:05.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.289+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffafc0cf4a0 con 0x7ffafc06c230 2024-04-24T23:26:05.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.290+0000 7ffaf1ffb640 1 -- 172.21.15.26:0/2805623857 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5227 (secure 0 0 0) 0x7ffaf8058eb0 con 0x7ffafc06c230 2024-04-24T23:26:05.291 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:05.291 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:25:27.340834+0000","modified":"2024-04-24T23:26:02.966810+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7227,"mds_1":7224,"mds_2":6364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6364":{"gid":6364,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/2057165440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2057165440},{"type":"v1","addr":"172.21.15.29:6835","nonce":2057165440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7224":{"gid":7224,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/1431580336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1431580336},{"type":"v1","addr":"172.21.15.26:6836","nonce":1431580336}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6837/2537133693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2537133693},{"type":"v1","addr":"172.21.15.26:6837","nonce":2537133693}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7227,"qdb_cluster":[7227,7224,6364]},"id":5}]} 2024-04-24T23:26:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 75 2024-04-24T23:26:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffae006be30 msgr2=0x7ffae006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffae006be30 0x7ffae006e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0cda40 tx=0x7ffaec008040 comp rx=0 tx=0).stop 2024-04-24T23:26:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 msgr2=0x7ffafc14e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 0x7ffafc14e630 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c820 tx=0x7ffaf800cce0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 shutdown_connections 2024-04-24T23:26:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffae006be30 0x7ffae006e2f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffafc15e290 0x7ffafc15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffafc0b3ac0 0x7ffafc15e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 --2- 172.21.15.26:0/2805623857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc06c230 0x7ffafc14e630 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 >> 172.21.15.26:0/2805623857 conn(0x7ffafc06b120 msgr2=0x7ffafc06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 shutdown_connections 2024-04-24T23:26:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.292+0000 7ffb03838640 1 -- 172.21.15.26:0/2805623857 wait complete. 2024-04-24T23:26:05.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:05.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.402+0000 7fb41b4d4640 1 Processor -- start 2024-04-24T23:26:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.403+0000 7fb41b4d4640 1 -- start start 2024-04-24T23:26:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb41b4d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140cf870 0x7fb414072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb41b4d4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4140cd0b0 con 0x7fb414143c40 2024-04-24T23:26:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb41b4d4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb414071240 con 0x7fb41415eb00 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb41b4d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb414071b70 con 0x7fb4140cf870 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb419270640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb414143c40 0x7fb4140cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55316/0 (socket says 172.21.15.26:55316) 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb419270640 1 -- 172.21.15.26:0/3133164747 learned_addr learned my addr 172.21.15.26:0/3133164747 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb418a6f640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140cf870 0x7fb414072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567459305 0 0) 0x7fb4140cd0b0 con 0x7fb414143c40 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3fc003520 con 0x7fb414143c40 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1610367653 0 0) 0x7fb3fc003520 con 0x7fb414143c40 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140cf870 msgr2=0x7fb414072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140cf870 0x7fb414072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 >> v1:172.21.15.29:6789/0 conn(0x7fb41415eb00 legacy=0x7fb4140b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb414071520 con 0x7fb414143c40 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fb40c001670 con 0x7fb414143c40 2024-04-24T23:26:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 >> v1:172.21.15.26:6789/0 conn(0x7fb414143c40 legacy=0x7fb4140cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140cf870 0x7fb3fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3fc003f20 0x7fb3fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb3fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb414071b70 con 0x7fb3fc004d40 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc00ba20 con 0x7fb3fc003f20 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb3fc00bc20 con 0x7fb4140cf870 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.404+0000 7fb419a71640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb3fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.405+0000 7fb419270640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140cf870 0x7fb3fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.405+0000 7fb418a6f640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3fc003f20 0x7fb3fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.405+0000 7fb419a71640 1 -- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140cf870 msgr2=0x7fb3fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.405+0000 7fb419a71640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140cf870 0x7fb3fc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.405+0000 7fb419a71640 1 -- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3fc003f20 msgr2=0x7fb3fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.405+0000 7fb419a71640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb3fc003f20 0x7fb3fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.405+0000 7fb419a71640 1 -- 172.21.15.26:0/3133164747 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc003520 con 0x7fb3fc004d40 2024-04-24T23:26:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.405+0000 7fb419a71640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb3fc0051c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fb404001f60 tx=0x7fb3fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c77ee2239b077d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404001b30 con 0x7fb3fc004d40 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb404031620 con 0x7fb3fc004d40 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb413fff640 1 -- 172.21.15.26:0/3133164747 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404031a10 con 0x7fb3fc004d40 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb41b4d4640 1 -- 172.21.15.26:0/3133164747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 msgr2=0x7fb3fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb41b4d4640 1 --2- 172.21.15.26:0/3133164747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb3fc0051c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fb404001f60 tx=0x7fb3fc005830 comp rx=0 tx=0).stop 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb419a71640 1 -- 172.21.15.26:0/3133164747 reap_dead start 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb41b4d4640 1 -- 172.21.15.26:0/3133164747 shutdown_connections 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb41b4d4640 1 -- 172.21.15.26:0/3133164747 >> 172.21.15.26:0/3133164747 conn(0x7fb41406b300 msgr2=0x7fb41406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb41b4d4640 1 -- 172.21.15.26:0/3133164747 shutdown_connections 2024-04-24T23:26:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.406+0000 7fb41b4d4640 1 -- 172.21.15.26:0/3133164747 wait complete. 2024-04-24T23:26:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.407+0000 7fb41b4d4640 1 Processor -- start 2024-04-24T23:26:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.407+0000 7fb41b4d4640 1 -- start start 2024-04-24T23:26:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb41b4d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb4141f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb41b4d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140c65f0 0x7fb4140c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb41b4d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4140ae9f0 0x7fb4140aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb41b4d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4140b3590 con 0x7fb3fc004d40 2024-04-24T23:26:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb41b4d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb414144020 con 0x7fb4140ae9f0 2024-04-24T23:26:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb41b4d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb4140b3710 con 0x7fb4140c65f0 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb419270640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb4141f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb419270640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb4141f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53508/0 (socket says 172.21.15.26:53508) 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb419270640 1 -- 172.21.15.26:0/1317315509 learned_addr learned my addr 172.21.15.26:0/1317315509 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb419a71640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4140ae9f0 0x7fb4140aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb418a6f640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140c65f0 0x7fb4140c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb419270640 1 -- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140c65f0 msgr2=0x7fb4140c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb419270640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140c65f0 0x7fb4140c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb419270640 1 -- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4140ae9f0 msgr2=0x7fb4140aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb419270640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4140ae9f0 0x7fb4140aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.408+0000 7fb419270640 1 -- 172.21.15.26:0/1317315509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4140c9680 con 0x7fb3fc004d40 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.409+0000 7fb418a6f640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140c65f0 0x7fb4140c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.409+0000 7fb419a71640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4140ae9f0 0x7fb4140aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.409+0000 7fb419270640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb4141f48e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fb40c007340 tx=0x7fb40c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.409+0000 7fb411ffb640 1 -- 172.21.15.26:0/1317315509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00f8e0 con 0x7fb3fc004d40 2024-04-24T23:26:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.409+0000 7fb411ffb640 1 -- 172.21.15.26:0/1317315509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c018070 con 0x7fb3fc004d40 2024-04-24T23:26:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.409+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4141f3e00 con 0x7fb3fc004d40 2024-04-24T23:26:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.409+0000 7fb411ffb640 1 -- 172.21.15.26:0/1317315509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00fce0 con 0x7fb3fc004d40 2024-04-24T23:26:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.409+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4140c92c0 con 0x7fb3fc004d40 2024-04-24T23:26:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.410+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4140694c0 con 0x7fb3fc004d40 2024-04-24T23:26:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.410+0000 7fb411ffb640 1 -- 172.21.15.26:0/1317315509 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb40c01d460 con 0x7fb3fc004d40 2024-04-24T23:26:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.411+0000 7fb411ffb640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb40006be30 0x7fb40006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.411+0000 7fb411ffb640 1 -- 172.21.15.26:0/1317315509 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fb40c08af40 con 0x7fb3fc004d40 2024-04-24T23:26:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.411+0000 7fb418a6f640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb40006be30 0x7fb40006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.411+0000 7fb418a6f640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb40006be30 0x7fb40006e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb4140cdaa0 tx=0x7fb408009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.415+0000 7fb411ffb640 1 -- 172.21.15.26:0/1317315509 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb40c059940 con 0x7fb3fc004d40 2024-04-24T23:26:05.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.598+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4140cc540 con 0x7fb3fc004d40 2024-04-24T23:26:05.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.599+0000 7fb411ffb640 1 -- 172.21.15.26:0/1317315509 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16706 (secure 0 0 0) 0x7fb40c082e20 con 0x7fb3fc004d40 2024-04-24T23:26:05.600 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:05.600 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:25:37.018656+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:25:25.626812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T23:25:26.319330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb40006be30 msgr2=0x7fb40006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb40006be30 0x7fb40006e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb4140cdaa0 tx=0x7fb408009190 comp rx=0 tx=0).stop 2024-04-24T23:26:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 msgr2=0x7fb4141f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb4141f48e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fb40c007340 tx=0x7fb40c007800 comp rx=0 tx=0).stop 2024-04-24T23:26:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 shutdown_connections 2024-04-24T23:26:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb40006be30 0x7fb40006e2f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4140ae9f0 0x7fb4140aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4140c65f0 0x7fb4140c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 --2- 172.21.15.26:0/1317315509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3fc004d40 0x7fb4141f48e0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 >> 172.21.15.26:0/1317315509 conn(0x7fb41406b300 msgr2=0x7fb41415d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 shutdown_connections 2024-04-24T23:26:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.601+0000 7fb41b4d4640 1 -- 172.21.15.26:0/1317315509 wait complete. 2024-04-24T23:26:05.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:26:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.714+0000 7f55efad4640 1 Processor -- start 2024-04-24T23:26:05.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.714+0000 7f55efad4640 1 -- start start 2024-04-24T23:26:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55efad4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55e806d350 0x7f55e806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55efad4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f55e80cd660 0x7f55e80cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55efad4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e80cfbc0 con 0x7f55e8145910 2024-04-24T23:26:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55efad4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f55e8145cf0 con 0x7f55e806d350 2024-04-24T23:26:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55efad4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f55e806d7f0 con 0x7f55e80cd660 2024-04-24T23:26:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55ed06f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55e806d350 0x7f55e806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55ed06f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55e806d350 0x7f55e806e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53518/0 (socket says 172.21.15.26:53518) 2024-04-24T23:26:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55ed06f640 1 -- 172.21.15.26:0/1533467727 learned_addr learned my addr 172.21.15.26:0/1533467727 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55ee071640 1 --2- 172.21.15.26:0/1533467727 >> v2:172.21.15.29:3300/0 conn(0x7f55e80cd660 0x7f55e80cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55ed06f640 1 -- 172.21.15.26:0/1533467727 >> v2:172.21.15.29:3300/0 conn(0x7f55e80cd660 msgr2=0x7f55e80cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55ed06f640 1 --2- 172.21.15.26:0/1533467727 >> v2:172.21.15.29:3300/0 conn(0x7f55e80cd660 0x7f55e80cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55ed06f640 1 -- 172.21.15.26:0/1533467727 >> v1:172.21.15.29:6789/0 conn(0x7f55e8145910 legacy=0x7f55e815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55ed06f640 1 -- 172.21.15.26:0/1533467727 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e806dad0 con 0x7f55e806d350 2024-04-24T23:26:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.715+0000 7f55ee071640 1 --2- 172.21.15.26:0/1533467727 >> v2:172.21.15.29:3300/0 conn(0x7f55e80cd660 0x7f55e80cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:05.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.716+0000 7f55ed06f640 1 --2- 172.21.15.26:0/1533467727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e806d350 0x7f55e806e890 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009870 tx=0x7f55d802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690e013f88b18edc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.716+0000 7f55ec86e640 1 -- 172.21.15.26:0/1533467727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8037070 con 0x7f55e806d350 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.716+0000 7f55ec86e640 1 -- 172.21.15.26:0/1533467727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d8032440 con 0x7f55e806d350 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.716+0000 7f55ec86e640 1 -- 172.21.15.26:0/1533467727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8032810 con 0x7f55e806d350 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.716+0000 7f55efad4640 1 -- 172.21.15.26:0/1533467727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e806d350 msgr2=0x7f55e806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.716+0000 7f55efad4640 1 --2- 172.21.15.26:0/1533467727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e806d350 0x7f55e806e890 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009870 tx=0x7f55d802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.717+0000 7f55efad4640 1 -- 172.21.15.26:0/1533467727 shutdown_connections 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.717+0000 7f55efad4640 1 --2- 172.21.15.26:0/1533467727 >> v2:172.21.15.29:3300/0 conn(0x7f55e80cd660 0x7f55e80cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.717+0000 7f55efad4640 1 --2- 172.21.15.26:0/1533467727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e806d350 0x7f55e806e890 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.717+0000 7f55efad4640 1 -- 172.21.15.26:0/1533467727 >> 172.21.15.26:0/1533467727 conn(0x7f55e806b300 msgr2=0x7f55e806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.717+0000 7f55efad4640 1 -- 172.21.15.26:0/1533467727 shutdown_connections 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.717+0000 7f55efad4640 1 -- 172.21.15.26:0/1533467727 wait complete. 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.718+0000 7f55efad4640 1 Processor -- start 2024-04-24T23:26:05.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.718+0000 7f55efad4640 1 -- start start 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55efad4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55e806d350 0x7f55e81f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55efad4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55e80cd660 0x7f55e814c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55efad4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e8145910 0x7f55e8144940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55efad4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55e806e370 con 0x7f55e8145910 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55efad4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f55e806e090 con 0x7f55e80cd660 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55efad4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f55e815d770 con 0x7f55e806d350 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ee071640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e8145910 0x7f55e8144940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ee071640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e8145910 0x7f55e8144940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53534/0 (socket says 172.21.15.26:53534) 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ee071640 1 -- 172.21.15.26:0/814899250 learned_addr learned my addr 172.21.15.26:0/814899250 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ed06f640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55e80cd660 0x7f55e814c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ed870640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55e806d350 0x7f55e81f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ee071640 1 -- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55e806d350 msgr2=0x7f55e81f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ee071640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55e806d350 0x7f55e81f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ee071640 1 -- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55e80cd660 msgr2=0x7f55e814c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ee071640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55e80cd660 0x7f55e814c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ee071640 1 -- 172.21.15.26:0/814899250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e81f5c50 con 0x7f55e8145910 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ed06f640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55e80cd660 0x7f55e814c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ed870640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55e806d350 0x7f55e81f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55ee071640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e8145910 0x7f55e8144940 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f55e000cef0 tx=0x7f55e000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55de7fc640 1 -- 172.21.15.26:0/814899250 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e001a070 con 0x7f55e8145910 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55de7fc640 1 -- 172.21.15.26:0/814899250 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e000d860 con 0x7f55e8145910 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e813b540 con 0x7f55e8145910 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55de7fc640 1 -- 172.21.15.26:0/814899250 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e000dc50 con 0x7f55e8145910 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.719+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e813bd80 con 0x7f55e8145910 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.720+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e80694c0 con 0x7f55e8145910 2024-04-24T23:26:05.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.721+0000 7f55de7fc640 1 -- 172.21.15.26:0/814899250 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55e001d070 con 0x7f55e8145910 2024-04-24T23:26:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.721+0000 7f55de7fc640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55cc06be30 0x7f55cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.721+0000 7f55de7fc640 1 -- 172.21.15.26:0/814899250 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f55e008b820 con 0x7f55e8145910 2024-04-24T23:26:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.721+0000 7f55ed870640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55cc06be30 0x7f55cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.722+0000 7f55ed870640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55cc06be30 0x7f55cc06e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f55e400a830 tx=0x7f55e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.726+0000 7f55de7fc640 1 -- 172.21.15.26:0/814899250 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f55e813b540 con 0x7f55e8145910 2024-04-24T23:26:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.946+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f55e814f330 con 0x7f55e8145910 2024-04-24T23:26:05.947 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:26:05.946+0000 7f26a0f44640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:26:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.947+0000 7f55de7fc640 1 -- 172.21.15.26:0/814899250 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7f55e005a1b0 con 0x7f55e8145910 2024-04-24T23:26:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:26:05.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55cc06be30 msgr2=0x7f55cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55cc06be30 0x7f55cc06e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f55e400a830 tx=0x7f55e4008040 comp rx=0 tx=0).stop 2024-04-24T23:26:05.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e8145910 msgr2=0x7f55e8144940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e8145910 0x7f55e8144940 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f55e000cef0 tx=0x7f55e000a710 comp rx=0 tx=0).stop 2024-04-24T23:26:05.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 shutdown_connections 2024-04-24T23:26:05.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f55cc06be30 0x7f55cc06e2f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e8145910 0x7f55e8144940 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55e80cd660 0x7f55e814c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 --2- 172.21.15.26:0/814899250 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f55e806d350 0x7f55e81f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 >> 172.21.15.26:0/814899250 conn(0x7f55e806b300 msgr2=0x7f55e80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 shutdown_connections 2024-04-24T23:26:05.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:05.951+0000 7f55efad4640 1 -- 172.21.15.26:0/814899250 wait complete. 2024-04-24T23:26:05.954 INFO:tasks.ceph.mds.c.smithi026.stderr: -9425> 2024-04-24T23:25:29.960+0000 7f59be5ed640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:26:05.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:26:05.981 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:26:05.982 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:26:05.991 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:26:06.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.069+0000 7f196cbdc640 1 Processor -- start 2024-04-24T23:26:06.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.069+0000 7f196cbdc640 1 -- start start 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f196cbdc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 0x7f19680d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f196cbdc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f196815e9c0 0x7f19680d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f196cbdc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196815ef60 con 0x7f196815adc0 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f196cbdc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f196815f240 con 0x7f196815e9c0 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f196cbdc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f196815f520 con 0x7f19680ad7b0 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f196659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f196815adc0 0x7f19680b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55334/0 (socket says 172.21.15.26:55334) 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f196659c640 1 -- 172.21.15.26:0/1495541603 learned_addr learned my addr 172.21.15.26:0/1495541603 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f1966d9d640 1 --2- 172.21.15.26:0/1495541603 >> v2:172.21.15.29:3300/0 conn(0x7f196815e9c0 0x7f19680d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f195ffff640 1 --2- 172.21.15.26:0/1495541603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 0x7f19680d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f1965d9b640 1 -- 172.21.15.26:0/1495541603 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2694593470 0 0) 0x7f196815ef60 con 0x7f196815adc0 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f1965d9b640 1 -- 172.21.15.26:0/1495541603 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1954003520 con 0x7f196815adc0 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f1966d9d640 1 -- 172.21.15.26:0/1495541603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 msgr2=0x7f19680d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f1966d9d640 1 --2- 172.21.15.26:0/1495541603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 0x7f19680d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f1966d9d640 1 -- 172.21.15.26:0/1495541603 >> v1:172.21.15.26:6789/0 conn(0x7f196815adc0 legacy=0x7f19680b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.070+0000 7f1966d9d640 1 -- 172.21.15.26:0/1495541603 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196815f800 con 0x7f196815e9c0 2024-04-24T23:26:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.071+0000 7f1966d9d640 1 --2- 172.21.15.26:0/1495541603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196815e9c0 0x7f19680d0580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1960009d90 tx=0x7f196002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77dd319c56552fef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.071+0000 7f1965d9b640 1 -- 172.21.15.26:0/1495541603 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960031600 con 0x7f196815e9c0 2024-04-24T23:26:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.071+0000 7f1965d9b640 1 -- 172.21.15.26:0/1495541603 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19600317a0 con 0x7f196815e9c0 2024-04-24T23:26:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.071+0000 7f1965d9b640 1 -- 172.21.15.26:0/1495541603 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960031b70 con 0x7f196815e9c0 2024-04-24T23:26:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.072+0000 7f196cbdc640 1 -- 172.21.15.26:0/1495541603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196815e9c0 msgr2=0x7f19680d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.072+0000 7f196cbdc640 1 --2- 172.21.15.26:0/1495541603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196815e9c0 0x7f19680d0580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1960009d90 tx=0x7f196002a5d0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.072+0000 7f196cbdc640 1 -- 172.21.15.26:0/1495541603 shutdown_connections 2024-04-24T23:26:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.072+0000 7f196cbdc640 1 --2- 172.21.15.26:0/1495541603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196815e9c0 0x7f19680d0580 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.072+0000 7f196cbdc640 1 --2- 172.21.15.26:0/1495541603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 0x7f19680d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.072+0000 7f196cbdc640 1 -- 172.21.15.26:0/1495541603 >> 172.21.15.26:0/1495541603 conn(0x7f196806b540 msgr2=0x7f196806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.072+0000 7f196cbdc640 1 -- 172.21.15.26:0/1495541603 shutdown_connections 2024-04-24T23:26:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.072+0000 7f196cbdc640 1 -- 172.21.15.26:0/1495541603 wait complete. 2024-04-24T23:26:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.073+0000 7f196cbdc640 1 Processor -- start 2024-04-24T23:26:06.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.073+0000 7f196cbdc640 1 -- start start 2024-04-24T23:26:06.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f196cbdc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 0x7f19681460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f196cbdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196815adc0 0x7f1968142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f196cbdc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196815e9c0 0x7f1968153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f196cbdc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1968072130 con 0x7f196815adc0 2024-04-24T23:26:06.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f196cbdc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f196815b1a0 con 0x7f196815e9c0 2024-04-24T23:26:06.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f196cbdc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f19680722b0 con 0x7f19680ad7b0 2024-04-24T23:26:06.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f195ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196815adc0 0x7f1968142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f195ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196815adc0 0x7f1968142d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53558/0 (socket says 172.21.15.26:53558) 2024-04-24T23:26:06.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f195ffff640 1 -- 172.21.15.26:0/456161962 learned_addr learned my addr 172.21.15.26:0/456161962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f196659c640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 0x7f19681460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f1966d9d640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196815e9c0 0x7f1968153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f195ffff640 1 -- 172.21.15.26:0/456161962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 msgr2=0x7f19681460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f195ffff640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 0x7f19681460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f195ffff640 1 -- 172.21.15.26:0/456161962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196815e9c0 msgr2=0x7f1968153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f195ffff640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196815e9c0 0x7f1968153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.074+0000 7f195ffff640 1 -- 172.21.15.26:0/456161962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19681432c0 con 0x7f196815adc0 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.075+0000 7f195ffff640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196815adc0 0x7f1968142d80 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f195000e9b0 tx=0x7f195000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.075+0000 7f195effd640 1 -- 172.21.15.26:0/456161962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950017070 con 0x7f196815adc0 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.075+0000 7f195effd640 1 -- 172.21.15.26:0/456161962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195000c9a0 con 0x7f196815adc0 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.075+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1968153540 con 0x7f196815adc0 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.075+0000 7f195effd640 1 -- 172.21.15.26:0/456161962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195000cd90 con 0x7f196815adc0 2024-04-24T23:26:06.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.075+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1968153aa0 con 0x7f196815adc0 2024-04-24T23:26:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.076+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1968075610 con 0x7f196815adc0 2024-04-24T23:26:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.076+0000 7f195effd640 1 -- 172.21.15.26:0/456161962 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f195001a070 con 0x7f196815adc0 2024-04-24T23:26:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.077+0000 7f195effd640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f194806be30 0x7f194806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.077+0000 7f195effd640 1 -- 172.21.15.26:0/456161962 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6025+0+0 (secure 0 0 0) 0x7f195008af20 con 0x7f196815adc0 2024-04-24T23:26:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.077+0000 7f196659c640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f194806be30 0x7f194806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.077+0000 7f196659c640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f194806be30 0x7f194806e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1958005880 tx=0x7f1958009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.081+0000 7f195effd640 1 -- 172.21.15.26:0/456161962 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1968153540 con 0x7f196815adc0 2024-04-24T23:26:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.307+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f196815e730 con 0x7f196815adc0 2024-04-24T23:26:06.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.949+0000 7f195effd640 1 -- 172.21.15.26:0/456161962 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) ==== 104+0+0 (secure 0 0 0) 0x7f1950059140 con 0x7f196815adc0 2024-04-24T23:26:06.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f194806be30 msgr2=0x7f194806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f194806be30 0x7f194806e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1958005880 tx=0x7f1958009040 comp rx=0 tx=0).stop 2024-04-24T23:26:06.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196815adc0 msgr2=0x7f1968142d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196815adc0 0x7f1968142d80 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f195000e9b0 tx=0x7f195000ee70 comp rx=0 tx=0).stop 2024-04-24T23:26:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 shutdown_connections 2024-04-24T23:26:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f194806be30 0x7f194806e2f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196815e9c0 0x7f1968153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196815adc0 0x7f1968142d80 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 --2- 172.21.15.26:0/456161962 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19680ad7b0 0x7f19681460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 >> 172.21.15.26:0/456161962 conn(0x7f196806b540 msgr2=0x7f19680adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 shutdown_connections 2024-04-24T23:26:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:06.952+0000 7f196cbdc640 1 -- 172.21.15.26:0/456161962 wait complete. 2024-04-24T23:26:06.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:26:07.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.122+0000 7f6d057a2640 1 Processor -- start 2024-04-24T23:26:07.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.122+0000 7f6d057a2640 1 -- start start 2024-04-24T23:26:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.123+0000 7f6d057a2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6d0015e270 0x7f6d0016a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.123+0000 7f6d057a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d0006f650 con 0x7f6d0009aa00 2024-04-24T23:26:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.123+0000 7f6d057a2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d0006f930 con 0x7f6d000d1210 2024-04-24T23:26:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.123+0000 7f6d057a2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6d000706f0 con 0x7f6d0015e270 2024-04-24T23:26:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.123+0000 7f6cfeffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6d0009aa00 0x7f6d0009ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55340/0 (socket says 172.21.15.26:55340) 2024-04-24T23:26:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.123+0000 7f6cfeffd640 1 -- 172.21.15.26:0/3383763125 learned_addr learned my addr 172.21.15.26:0/3383763125 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.123+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807464649 0 0) 0x7f6d0006f650 con 0x7f6d0009aa00 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.123+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cec003520 con 0x7f6d0009aa00 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.123+0000 7f6cff7fe640 1 --2- 172.21.15.26:0/3383763125 >> v2:172.21.15.29:3300/0 conn(0x7f6d0015e270 0x7f6d0016a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2182005407 0 0) 0x7f6cec003520 con 0x7f6d0009aa00 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 >> v2:172.21.15.29:3300/0 conn(0x7f6d0015e270 msgr2=0x7f6d0016a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 --2- 172.21.15.26:0/3383763125 >> v2:172.21.15.29:3300/0 conn(0x7f6d0015e270 0x7f6d0016a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 >> v1:172.21.15.29:6789/0 conn(0x7f6d000d1210 legacy=0x7f6d0015db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0006fc10 con 0x7f6d0009aa00 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f6cf8001670 con 0x7f6d0009aa00 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 >> v1:172.21.15.26:6789/0 conn(0x7f6d0009aa00 legacy=0x7f6d0009ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6cec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cec004800 0x7f6cec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6cec0051c0 0x7f6cec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d000706f0 con 0x7f6cec0039b0 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec00c110 con 0x7f6cec004800 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6cec00c310 con 0x7f6cec0051c0 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfeffd640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6cec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfe7fc640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cec004800 0x7f6cec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cff7fe640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6cec0051c0 0x7f6cec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfeffd640 1 -- 172.21.15.26:0/3383763125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6cec0051c0 msgr2=0x7f6cec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfeffd640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6cec0051c0 0x7f6cec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfeffd640 1 -- 172.21.15.26:0/3383763125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cec004800 msgr2=0x7f6cec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfeffd640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cec004800 0x7f6cec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cfeffd640 1 -- 172.21.15.26:0/3383763125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec003520 con 0x7f6cec0039b0 2024-04-24T23:26:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.124+0000 7f6cff7fe640 1 -- 172.21.15.26:0/3383763125 reap_dead start 2024-04-24T23:26:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.125+0000 7f6cfeffd640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6cec003dd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c210 tx=0x7f6cec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba59f1206466213c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.125+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8007d30 con 0x7f6cec0039b0 2024-04-24T23:26:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.125+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf80058d0 con 0x7f6cec0039b0 2024-04-24T23:26:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.125+0000 7f6cfdffb640 1 -- 172.21.15.26:0/3383763125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8005d20 con 0x7f6cec0039b0 2024-04-24T23:26:07.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.126+0000 7f6d057a2640 1 -- 172.21.15.26:0/3383763125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 msgr2=0x7f6cec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.126+0000 7f6d057a2640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6cec003dd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c210 tx=0x7f6cec005830 comp rx=0 tx=0).stop 2024-04-24T23:26:07.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.126+0000 7f6d057a2640 1 -- 172.21.15.26:0/3383763125 shutdown_connections 2024-04-24T23:26:07.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.126+0000 7f6d057a2640 1 --2- 172.21.15.26:0/3383763125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6cec003dd0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.126+0000 7f6d057a2640 1 -- 172.21.15.26:0/3383763125 >> 172.21.15.26:0/3383763125 conn(0x7f6d000b39e0 msgr2=0x7f6d000b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.126+0000 7f6d057a2640 1 -- 172.21.15.26:0/3383763125 shutdown_connections 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.126+0000 7f6d057a2640 1 -- 172.21.15.26:0/3383763125 wait complete. 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.127+0000 7f6d057a2640 1 Processor -- start 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.127+0000 7f6d057a2640 1 -- start start 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6d057a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6d001f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6d057a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d000b5420 0x7f6d000c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6d057a2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d000b58a0 0x7f6d000ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6d057a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d00070260 con 0x7f6cec0039b0 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6d057a2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6d000d1650 con 0x7f6d000b5420 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6d057a2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0015e700 con 0x7f6d000b58a0 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cfeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6d001f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cfeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6d001f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53568/0 (socket says 172.21.15.26:53568) 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cfeffd640 1 -- 172.21.15.26:0/493993455 learned_addr learned my addr 172.21.15.26:0/493993455 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cff7fe640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d000b58a0 0x7f6d000ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cfe7fc640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d000b5420 0x7f6d000c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cfeffd640 1 -- 172.21.15.26:0/493993455 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d000b58a0 msgr2=0x7f6d000ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cfeffd640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d000b58a0 0x7f6d000ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cfeffd640 1 -- 172.21.15.26:0/493993455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d000b5420 msgr2=0x7f6d000c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cfeffd640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d000b5420 0x7f6d000c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cfeffd640 1 -- 172.21.15.26:0/493993455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d000b7010 con 0x7f6cec0039b0 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.128+0000 7f6cff7fe640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d000b58a0 0x7f6d000ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.129+0000 7f6cfeffd640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6d001f4380 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c210 tx=0x7f6cf8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.129+0000 7f6cf77fe640 1 -- 172.21.15.26:0/493993455 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8042020 con 0x7f6cec0039b0 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.129+0000 7f6cf77fe640 1 -- 172.21.15.26:0/493993455 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf8039040 con 0x7f6cec0039b0 2024-04-24T23:26:07.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.129+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d000b7290 con 0x7f6cec0039b0 2024-04-24T23:26:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.129+0000 7f6cf77fe640 1 -- 172.21.15.26:0/493993455 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8007680 con 0x7f6cec0039b0 2024-04-24T23:26:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.129+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d000b7850 con 0x7f6cec0039b0 2024-04-24T23:26:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.130+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d000694c0 con 0x7f6cec0039b0 2024-04-24T23:26:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.130+0000 7f6cf77fe640 1 -- 172.21.15.26:0/493993455 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6cf8045020 con 0x7f6cec0039b0 2024-04-24T23:26:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.131+0000 7f6cf77fe640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ce006be30 0x7f6ce006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.131+0000 7f6cf77fe640 1 -- 172.21.15.26:0/493993455 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6025+0+0 (secure 0 0 0) 0x7f6cf80ab390 con 0x7f6cec0039b0 2024-04-24T23:26:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.131+0000 7f6cfe7fc640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ce006be30 0x7f6ce006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.131+0000 7f6cfe7fc640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ce006be30 0x7f6ce006e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6d000c8970 tx=0x7f6ce8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.135+0000 7f6cf77fe640 1 -- 172.21.15.26:0/493993455 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6cf804c030 con 0x7f6cec0039b0 2024-04-24T23:26:07.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:07.321+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6d000b48f0 con 0x7f6cec0039b0 2024-04-24T23:26:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.111+0000 7f6cf77fe640 1 -- 172.21.15.26:0/493993455 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) ==== 181+0+0 (secure 0 0 0) 0x7f6d000b7010 con 0x7f6cec0039b0 2024-04-24T23:26:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:26:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ce006be30 msgr2=0x7f6ce006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ce006be30 0x7f6ce006e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6d000c8970 tx=0x7f6ce8009040 comp rx=0 tx=0).stop 2024-04-24T23:26:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 msgr2=0x7f6d001f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6d001f4380 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c210 tx=0x7f6cf8001a40 comp rx=0 tx=0).stop 2024-04-24T23:26:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 shutdown_connections 2024-04-24T23:26:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6ce006be30 0x7f6ce006e2f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d000b58a0 0x7f6d000ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d000b5420 0x7f6d000c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 --2- 172.21.15.26:0/493993455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec0039b0 0x7f6d001f4380 secure :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c210 tx=0x7f6cf8001a40 comp rx=0 tx=0).stop 2024-04-24T23:26:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 >> 172.21.15.26:0/493993455 conn(0x7f6d000b39e0 msgr2=0x7f6d000b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 shutdown_connections 2024-04-24T23:26:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.114+0000 7f6d057a2640 1 -- 172.21.15.26:0/493993455 wait complete. 2024-04-24T23:26:08.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:26:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.240+0000 7ffadf012640 1 Processor -- start 2024-04-24T23:26:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.240+0000 7ffadf012640 1 -- start start 2024-04-24T23:26:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffadf012640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffad8147bc0 0x7ffad80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffadf012640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad80722f0 0x7ffad8072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffadf012640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffad80b4260 con 0x7ffad815e950 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffadf012640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ffad80b44d0 con 0x7ffad8147bc0 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffadf012640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ffad80b47b0 con 0x7ffad80722f0 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffad7fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad80722f0 0x7ffad8072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffadcdae640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffad8147bc0 0x7ffad80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffad7fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad80722f0 0x7ffad8072770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:57936/0 (socket says 172.21.15.26:57936) 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffadcdae640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ffad8147bc0 0x7ffad80b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:48870/0 (socket says 172.21.15.26:48870) 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffadd5af640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ffad815e950 0x7ffad815ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:34884/0 (socket says 172.21.15.26:34884) 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.241+0000 7ffad7fff640 1 -- 172.21.15.26:0/912662187 learned_addr learned my addr 172.21.15.26:0/912662187 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.242+0000 7ffad7fff640 1 -- 172.21.15.26:0/912662187 >> v2:172.21.15.29:3300/0 conn(0x7ffad8147bc0 msgr2=0x7ffad80b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.242+0000 7ffad7fff640 1 --2- 172.21.15.26:0/912662187 >> v2:172.21.15.29:3300/0 conn(0x7ffad8147bc0 0x7ffad80b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.242+0000 7ffad7fff640 1 -- 172.21.15.26:0/912662187 >> v1:172.21.15.29:6789/0 conn(0x7ffad815e950 legacy=0x7ffad815ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.242+0000 7ffad7fff640 1 -- 172.21.15.26:0/912662187 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad80b4a90 con 0x7ffad80722f0 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.242+0000 7ffadcdae640 1 --2- 172.21.15.26:0/912662187 >> v2:172.21.15.29:3300/0 conn(0x7ffad8147bc0 0x7ffad80b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.242+0000 7ffad7fff640 1 --2- 172.21.15.26:0/912662187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad80722f0 0x7ffad8072770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009870 tx=0x7ffac402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3925fa59f41729ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffad77fe640 1 -- 172.21.15.26:0/912662187 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4037070 con 0x7ffad80722f0 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffad77fe640 1 -- 172.21.15.26:0/912662187 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac4032440 con 0x7ffad80722f0 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffad77fe640 1 -- 172.21.15.26:0/912662187 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4032810 con 0x7ffad80722f0 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffadf012640 1 -- 172.21.15.26:0/912662187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad80722f0 msgr2=0x7ffad8072770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffadf012640 1 --2- 172.21.15.26:0/912662187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad80722f0 0x7ffad8072770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009870 tx=0x7ffac402a270 comp rx=0 tx=0).stop 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffadf012640 1 -- 172.21.15.26:0/912662187 shutdown_connections 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffadf012640 1 --2- 172.21.15.26:0/912662187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad80722f0 0x7ffad8072770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffadf012640 1 --2- 172.21.15.26:0/912662187 >> v2:172.21.15.29:3300/0 conn(0x7ffad8147bc0 0x7ffad80b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffadf012640 1 -- 172.21.15.26:0/912662187 >> 172.21.15.26:0/912662187 conn(0x7ffad806b300 msgr2=0x7ffad806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffadf012640 1 -- 172.21.15.26:0/912662187 shutdown_connections 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.243+0000 7ffadf012640 1 -- 172.21.15.26:0/912662187 wait complete. 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.244+0000 7ffadf012640 1 Processor -- start 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.244+0000 7ffadf012640 1 -- start start 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadf012640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad80722f0 0x7ffad80c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadf012640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad8147bc0 0x7ffad81f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadf012640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad815e950 0x7ffad815da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadf012640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffad806d4d0 con 0x7ffad815e950 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadf012640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffad806d350 con 0x7ffad80722f0 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadf012640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ffad806d650 con 0x7ffad8147bc0 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadd5af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad815e950 0x7ffad815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffad7fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad8147bc0 0x7ffad81f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadd5af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad815e950 0x7ffad815da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53570/0 (socket says 172.21.15.26:53570) 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadd5af640 1 -- 172.21.15.26:0/708082769 learned_addr learned my addr 172.21.15.26:0/708082769 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.245+0000 7ffadcdae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad80722f0 0x7ffad80c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffadd5af640 1 -- 172.21.15.26:0/708082769 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad8147bc0 msgr2=0x7ffad81f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffadd5af640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad8147bc0 0x7ffad81f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffadd5af640 1 -- 172.21.15.26:0/708082769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad80722f0 msgr2=0x7ffad80c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffadd5af640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad80722f0 0x7ffad80c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffadd5af640 1 -- 172.21.15.26:0/708082769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad80c57c0 con 0x7ffad815e950 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffadd5af640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad815e950 0x7ffad815da90 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ffad000c730 tx=0x7ffad000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffad57fa640 1 -- 172.21.15.26:0/708082769 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0017070 con 0x7ffad815e950 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffad57fa640 1 -- 172.21.15.26:0/708082769 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad000ce40 con 0x7ffad815e950 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad81f4060 con 0x7ffad815e950 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffad57fa640 1 -- 172.21.15.26:0/708082769 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad000a840 con 0x7ffad815e950 2024-04-24T23:26:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.246+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad81f45c0 con 0x7ffad815e950 2024-04-24T23:26:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.247+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad80694c0 con 0x7ffad815e950 2024-04-24T23:26:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.248+0000 7ffad57fa640 1 -- 172.21.15.26:0/708082769 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffad001a070 con 0x7ffad815e950 2024-04-24T23:26:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.248+0000 7ffad57fa640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffabc06be30 0x7ffabc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.248+0000 7ffad57fa640 1 -- 172.21.15.26:0/708082769 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5587+0+0 (secure 0 0 0) 0x7ffad008a240 con 0x7ffad815e950 2024-04-24T23:26:08.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.248+0000 7ffadcdae640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffabc06be30 0x7ffabc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.249+0000 7ffadcdae640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffabc06be30 0x7ffabc06e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009560 tx=0x7ffacc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:08.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.252+0000 7ffad57fa640 1 -- 172.21.15.26:0/708082769 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffad81f4060 con 0x7ffad815e950 2024-04-24T23:26:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:08.438+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffad815dfd0 con 0x7ffad815e950 2024-04-24T23:26:09.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.115+0000 7ffad57fa640 1 -- 172.21.15.26:0/708082769 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) ==== 169+0+0 (secure 0 0 0) 0x7ffad0058610 con 0x7ffad815e950 2024-04-24T23:26:09.117 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-24T23:26:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffabc06be30 msgr2=0x7ffabc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffabc06be30 0x7ffabc06e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009560 tx=0x7ffacc009190 comp rx=0 tx=0).stop 2024-04-24T23:26:09.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad815e950 msgr2=0x7ffad815da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad815e950 0x7ffad815da90 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ffad000c730 tx=0x7ffad000cbf0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 shutdown_connections 2024-04-24T23:26:09.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ffabc06be30 0x7ffabc06e2f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad815e950 0x7ffad815da90 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ffad8147bc0 0x7ffad81f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 --2- 172.21.15.26:0/708082769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad80722f0 0x7ffad80c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 >> 172.21.15.26:0/708082769 conn(0x7ffad806b300 msgr2=0x7ffad8072920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:09.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 shutdown_connections 2024-04-24T23:26:09.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.118+0000 7ffadf012640 1 -- 172.21.15.26:0/708082769 wait complete. 2024-04-24T23:26:09.133 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:26:09.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.241+0000 7ff388b90640 1 Processor -- start 2024-04-24T23:26:09.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.241+0000 7ff388b90640 1 -- start start 2024-04-24T23:26:09.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff388b90640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff38415adc0 0x7ff3840b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff388b90640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff3840d0750 0x7ff384072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff388b90640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38415ef60 con 0x7ff3840ad7b0 2024-04-24T23:26:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff388b90640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff38415f240 con 0x7ff38415adc0 2024-04-24T23:26:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff388b90640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff38415f520 con 0x7ff3840d0750 2024-04-24T23:26:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff38259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff38415adc0 0x7ff3840b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff38259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff38415adc0 0x7ff3840b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53578/0 (socket says 172.21.15.26:53578) 2024-04-24T23:26:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff38259c640 1 -- 172.21.15.26:0/1800684118 learned_addr learned my addr 172.21.15.26:0/1800684118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff38259c640 1 -- 172.21.15.26:0/1800684118 >> v2:172.21.15.29:3300/0 conn(0x7ff3840d0750 msgr2=0x7ff384072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff38259c640 1 --2- 172.21.15.26:0/1800684118 >> v2:172.21.15.29:3300/0 conn(0x7ff3840d0750 0x7ff384072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff38259c640 1 -- 172.21.15.26:0/1800684118 >> v1:172.21.15.29:6789/0 conn(0x7ff3840ad7b0 legacy=0x7ff3840d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.242+0000 7ff38259c640 1 -- 172.21.15.26:0/1800684118 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38415f800 con 0x7ff38415adc0 2024-04-24T23:26:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.243+0000 7ff38259c640 1 --2- 172.21.15.26:0/1800684118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 0x7ff3840b4370 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff374009870 tx=0x7ff37402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958b958327dda5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.243+0000 7ff381d9b640 1 -- 172.21.15.26:0/1800684118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374037070 con 0x7ff38415adc0 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.243+0000 7ff381d9b640 1 -- 172.21.15.26:0/1800684118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff374032440 con 0x7ff38415adc0 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.243+0000 7ff381d9b640 1 -- 172.21.15.26:0/1800684118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374032810 con 0x7ff38415adc0 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.244+0000 7ff388b90640 1 -- 172.21.15.26:0/1800684118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 msgr2=0x7ff3840b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.244+0000 7ff388b90640 1 --2- 172.21.15.26:0/1800684118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 0x7ff3840b4370 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff374009870 tx=0x7ff37402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.244+0000 7ff388b90640 1 -- 172.21.15.26:0/1800684118 shutdown_connections 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.244+0000 7ff388b90640 1 --2- 172.21.15.26:0/1800684118 >> v2:172.21.15.29:3300/0 conn(0x7ff3840d0750 0x7ff384072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.244+0000 7ff388b90640 1 --2- 172.21.15.26:0/1800684118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 0x7ff3840b4370 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.244+0000 7ff388b90640 1 -- 172.21.15.26:0/1800684118 >> 172.21.15.26:0/1800684118 conn(0x7ff38406b540 msgr2=0x7ff38406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.244+0000 7ff388b90640 1 -- 172.21.15.26:0/1800684118 shutdown_connections 2024-04-24T23:26:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.244+0000 7ff388b90640 1 -- 172.21.15.26:0/1800684118 wait complete. 2024-04-24T23:26:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.245+0000 7ff388b90640 1 Processor -- start 2024-04-24T23:26:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.245+0000 7ff388b90640 1 -- start start 2024-04-24T23:26:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff388b90640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3840ad7b0 0x7ff38415b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff388b90640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3840d0750 0x7ff3840c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff388b90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 0x7ff3840cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff388b90640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff38415b1a0 con 0x7ff38415adc0 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff388b90640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff3840b48b0 con 0x7ff3840ad7b0 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff388b90640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff38415e9c0 con 0x7ff3840d0750 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff382d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 0x7ff3840cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff382d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 0x7ff3840cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53588/0 (socket says 172.21.15.26:53588) 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff382d9d640 1 -- 172.21.15.26:0/1902784906 learned_addr learned my addr 172.21.15.26:0/1902784906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff38259c640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3840ad7b0 0x7ff38415b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff37bfff640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3840d0750 0x7ff3840c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff382d9d640 1 -- 172.21.15.26:0/1902784906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3840d0750 msgr2=0x7ff3840c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff382d9d640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3840d0750 0x7ff3840c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff382d9d640 1 -- 172.21.15.26:0/1902784906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3840ad7b0 msgr2=0x7ff38415b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff382d9d640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3840ad7b0 0x7ff38415b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff382d9d640 1 -- 172.21.15.26:0/1902784906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38415b9b0 con 0x7ff38415adc0 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.246+0000 7ff37bfff640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3840d0750 0x7ff3840c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.247+0000 7ff382d9d640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 0x7ff3840cb5d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00b9e0 tx=0x7ff37c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.247+0000 7ff37affd640 1 -- 172.21.15.26:0/1902784906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c019070 con 0x7ff38415adc0 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.247+0000 7ff37affd640 1 -- 172.21.15.26:0/1902784906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff37c00cb40 con 0x7ff38415adc0 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.247+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38415bc90 con 0x7ff38415adc0 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.247+0000 7ff37affd640 1 -- 172.21.15.26:0/1902784906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c014440 con 0x7ff38415adc0 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.247+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38415c250 con 0x7ff38415adc0 2024-04-24T23:26:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.248+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff384075610 con 0x7ff38415adc0 2024-04-24T23:26:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.248+0000 7ff37affd640 1 -- 172.21.15.26:0/1902784906 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff37c01c070 con 0x7ff38415adc0 2024-04-24T23:26:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.249+0000 7ff37affd640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff36406be60 0x7ff36406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.249+0000 7ff37affd640 1 -- 172.21.15.26:0/1902784906 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7ff37c08a7c0 con 0x7ff38415adc0 2024-04-24T23:26:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.249+0000 7ff38259c640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff36406be60 0x7ff36406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.249+0000 7ff38259c640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff36406be60 0x7ff36406e320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff374034000 tx=0x7ff37402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.253+0000 7ff37affd640 1 -- 172.21.15.26:0/1902784906 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff37c058de0 con 0x7ff38415adc0 2024-04-24T23:26:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.436+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ff38415e730 con 0x7ff38415adc0 2024-04-24T23:26:09.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.439+0000 7ff37affd640 1 -- 172.21.15.26:0/1902784906 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7ff37c05d0e0 con 0x7ff38415adc0 2024-04-24T23:26:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff36406be60 msgr2=0x7ff36406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff36406be60 0x7ff36406e320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff374034000 tx=0x7ff37402e000 comp rx=0 tx=0).stop 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 msgr2=0x7ff3840cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 0x7ff3840cb5d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00b9e0 tx=0x7ff37c00bea0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 shutdown_connections 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff36406be60 0x7ff36406e320 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff38415adc0 0x7ff3840cb5d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff3840d0750 0x7ff3840c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 --2- 172.21.15.26:0/1902784906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3840ad7b0 0x7ff38415b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 >> 172.21.15.26:0/1902784906 conn(0x7ff38406b540 msgr2=0x7ff3840b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 shutdown_connections 2024-04-24T23:26:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.441+0000 7ff388b90640 1 -- 172.21.15.26:0/1902784906 wait complete. 2024-04-24T23:26:09.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-04-24T23:26:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.562+0000 7fce3dff4640 1 Processor -- start 2024-04-24T23:26:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.562+0000 7fce3dff4640 1 -- start start 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.563+0000 7fce3dff4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fce3815e570 0x7fce380d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.563+0000 7fce3dff4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce380d32f0 con 0x7fce380d22e0 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.563+0000 7fce3dff4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce380d1600 con 0x7fce380b3130 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.563+0000 7fce3dff4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fce380d1ab0 con 0x7fce3815e570 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.563+0000 7fce37fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fce380d22e0 0x7fce380d0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55356/0 (socket says 172.21.15.26:55356) 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.563+0000 7fce37fff640 1 -- 172.21.15.26:0/3410221721 learned_addr learned my addr 172.21.15.26:0/3410221721 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce377fe640 1 --2- 172.21.15.26:0/3410221721 >> v2:172.21.15.29:3300/0 conn(0x7fce3815e570 0x7fce380d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124099506 0 0) 0x7fce30001490 con 0x7fce380d22e0 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce20003520 con 0x7fce380d22e0 2024-04-24T23:26:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1813325166 0 0) 0x7fce380d1600 con 0x7fce380b3130 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce30001490 con 0x7fce380b3130 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3440128325 0 0) 0x7fce20003520 con 0x7fce380d22e0 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 >> v2:172.21.15.29:3300/0 conn(0x7fce3815e570 msgr2=0x7fce380d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 --2- 172.21.15.26:0/3410221721 >> v2:172.21.15.29:3300/0 conn(0x7fce3815e570 0x7fce380d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 >> v1:172.21.15.29:6789/0 conn(0x7fce380b3130 legacy=0x7fce380d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce380d0270 con 0x7fce380d22e0 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fce30001bc0 con 0x7fce380d22e0 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 >> v1:172.21.15.26:6789/0 conn(0x7fce380d22e0 legacy=0x7fce380d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce20003b90 0x7fce20003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce20004800 0x7fce20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce200051c0 0x7fce2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce380d1ab0 con 0x7fce20003b90 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce2000c090 con 0x7fce200051c0 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fce2000c290 con 0x7fce20004800 2024-04-24T23:26:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce377fe640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce20003b90 0x7fce20003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce36ffd640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce20004800 0x7fce20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce37fff640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce200051c0 0x7fce2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce377fe640 1 -- 172.21.15.26:0/3410221721 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce20004800 msgr2=0x7fce20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce377fe640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce20004800 0x7fce20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce377fe640 1 -- 172.21.15.26:0/3410221721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce200051c0 msgr2=0x7fce2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce377fe640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce200051c0 0x7fce2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce377fe640 1 -- 172.21.15.26:0/3410221721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce20003520 con 0x7fce20003b90 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce37fff640 1 -- 172.21.15.26:0/3410221721 reap_dead start 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.564+0000 7fce36ffd640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce20004800 0x7fce20004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.565+0000 7fce377fe640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce20003b90 0x7fce20003f90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fce28009ac0 tx=0x7fce20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757c19aef705ba57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.565+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28036070 con 0x7fce20003b90 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.565+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce28031400 con 0x7fce20003b90 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.565+0000 7fce367fc640 1 -- 172.21.15.26:0/3410221721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28009690 con 0x7fce20003b90 2024-04-24T23:26:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.566+0000 7fce3dff4640 1 -- 172.21.15.26:0/3410221721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce20003b90 msgr2=0x7fce20003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.566+0000 7fce3dff4640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce20003b90 0x7fce20003f90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fce28009ac0 tx=0x7fce20005830 comp rx=0 tx=0).stop 2024-04-24T23:26:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.566+0000 7fce3dff4640 1 -- 172.21.15.26:0/3410221721 shutdown_connections 2024-04-24T23:26:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.566+0000 7fce3dff4640 1 --2- 172.21.15.26:0/3410221721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce20003b90 0x7fce20003f90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.566+0000 7fce3dff4640 1 -- 172.21.15.26:0/3410221721 >> 172.21.15.26:0/3410221721 conn(0x7fce3808dbe0 msgr2=0x7fce3809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.566+0000 7fce3dff4640 1 -- 172.21.15.26:0/3410221721 shutdown_connections 2024-04-24T23:26:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.566+0000 7fce3dff4640 1 -- 172.21.15.26:0/3410221721 wait complete. 2024-04-24T23:26:09.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.567+0000 7fce3dff4640 1 Processor -- start 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.567+0000 7fce3dff4640 1 -- start start 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce3dff4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce20003b90 0x7fce3815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce3dff4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce380b80d0 0x7fce380b8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce3dff4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3814cfb0 0x7fce3814d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce3dff4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce380d3010 con 0x7fce3814cfb0 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce3dff4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce380d2ae0 con 0x7fce20003b90 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce3dff4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fce380cfa70 con 0x7fce380b80d0 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3814cfb0 0x7fce3814d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3814cfb0 0x7fce3814d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53602/0 (socket says 172.21.15.26:53602) 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce37fff640 1 -- 172.21.15.26:0/2593866339 learned_addr learned my addr 172.21.15.26:0/2593866339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce377fe640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce20003b90 0x7fce3815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce36ffd640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce380b80d0 0x7fce380b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce37fff640 1 -- 172.21.15.26:0/2593866339 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce380b80d0 msgr2=0x7fce380b8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce37fff640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce380b80d0 0x7fce380b8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce37fff640 1 -- 172.21.15.26:0/2593866339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce20003b90 msgr2=0x7fce3815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce37fff640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce20003b90 0x7fce3815c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.568+0000 7fce37fff640 1 -- 172.21.15.26:0/2593866339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce380b8d90 con 0x7fce3814cfb0 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.569+0000 7fce37fff640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3814cfb0 0x7fce3814d430 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fce3808c9f0 tx=0x7fce3000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.569+0000 7fce1ffff640 1 -- 172.21.15.26:0/2593866339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3000ec10 con 0x7fce3814cfb0 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.569+0000 7fce1ffff640 1 -- 172.21.15.26:0/2593866339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce30018070 con 0x7fce3814cfb0 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.569+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce380b9070 con 0x7fce3814cfb0 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.569+0000 7fce1ffff640 1 -- 172.21.15.26:0/2593866339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30013420 con 0x7fce3814cfb0 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.569+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce380b9630 con 0x7fce3814cfb0 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.570+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce380694c0 con 0x7fce3814cfb0 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.570+0000 7fce1ffff640 1 -- 172.21.15.26:0/2593866339 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce300135c0 con 0x7fce3814cfb0 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.571+0000 7fce1ffff640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce2406be30 0x7fce2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.571+0000 7fce1ffff640 1 -- 172.21.15.26:0/2593866339 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7fce3008af80 con 0x7fce3814cfb0 2024-04-24T23:26:09.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.571+0000 7fce377fe640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce2406be30 0x7fce2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.571+0000 7fce377fe640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce2406be30 0x7fce2406e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fce28033000 tx=0x7fce28008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.575+0000 7fce1ffff640 1 -- 172.21.15.26:0/2593866339 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce380d32f0 con 0x7fce3814cfb0 2024-04-24T23:26:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:09.829+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) -- 0x7fce380d13f0 con 0x7fce3814cfb0 2024-04-24T23:26:10.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.120+0000 7fce1ffff640 1 -- 172.21.15.26:0/2593866339 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v278) ==== 141+0+0 (secure 0 0 0) 0x7fce30059530 con 0x7fce3814cfb0 2024-04-24T23:26:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce2406be30 msgr2=0x7fce2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce2406be30 0x7fce2406e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fce28033000 tx=0x7fce28008030 comp rx=0 tx=0).stop 2024-04-24T23:26:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3814cfb0 msgr2=0x7fce3814d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3814cfb0 0x7fce3814d430 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fce3808c9f0 tx=0x7fce3000e530 comp rx=0 tx=0).stop 2024-04-24T23:26:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 shutdown_connections 2024-04-24T23:26:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce2406be30 0x7fce2406e2f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3814cfb0 0x7fce3814d430 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce380b80d0 0x7fce380b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 --2- 172.21.15.26:0/2593866339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce20003b90 0x7fce3815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 >> 172.21.15.26:0/2593866339 conn(0x7fce3808dbe0 msgr2=0x7fce380d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 shutdown_connections 2024-04-24T23:26:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.123+0000 7fce3dff4640 1 -- 172.21.15.26:0/2593866339 wait complete. 2024-04-24T23:26:10.136 INFO:tasks.cephfs_test_runner:test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-04-24T23:26:10.136 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-04-24T23:26:10.137 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.239+0000 7f45e68a7640 1 Processor -- start 2024-04-24T23:26:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.239+0000 7f45e68a7640 1 -- start start 2024-04-24T23:26:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e68a7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f45e0143c40 0x7f45e00cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e68a7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 0x7f45e0072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e68a7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45e015eb00 0x7f45e015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e68a7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f45e015f490 con 0x7f45e015eb00 2024-04-24T23:26:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e68a7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f45e0071b70 con 0x7f45e0143c40 2024-04-24T23:26:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e68a7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f45e00cd0b0 con 0x7f45e00cc860 2024-04-24T23:26:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e4e44640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45e015eb00 0x7f45e015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e4e44640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45e015eb00 0x7f45e015ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53610/0 (socket says 172.21.15.26:53610) 2024-04-24T23:26:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e4e44640 1 -- 172.21.15.26:0/595253554 learned_addr learned my addr 172.21.15.26:0/595253554 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45df7fe640 1 --2- 172.21.15.26:0/595253554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 0x7f45e0072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e4e44640 1 -- 172.21.15.26:0/595253554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 msgr2=0x7f45e0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e4e44640 1 --2- 172.21.15.26:0/595253554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 0x7f45e0072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45dffff640 1 --2- 172.21.15.26:0/595253554 >> v2:172.21.15.29:3300/0 conn(0x7f45e0143c40 0x7f45e00cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e4e44640 1 -- 172.21.15.26:0/595253554 >> v2:172.21.15.29:3300/0 conn(0x7f45e0143c40 msgr2=0x7f45e00cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e4e44640 1 --2- 172.21.15.26:0/595253554 >> v2:172.21.15.29:3300/0 conn(0x7f45e0143c40 0x7f45e00cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.240+0000 7f45e4e44640 1 -- 172.21.15.26:0/595253554 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e0071240 con 0x7f45e015eb00 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.241+0000 7f45e4e44640 1 --2- 172.21.15.26:0/595253554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 0x7f45e015ef50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f45e006ecf0 tx=0x7f45d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84938857d1a156f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.241+0000 7f45deffd640 1 -- 172.21.15.26:0/595253554 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8036070 con 0x7f45e015eb00 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.241+0000 7f45deffd640 1 -- 172.21.15.26:0/595253554 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d8031600 con 0x7f45e015eb00 2024-04-24T23:26:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.241+0000 7f45deffd640 1 -- 172.21.15.26:0/595253554 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d80319d0 con 0x7f45e015eb00 2024-04-24T23:26:10.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.242+0000 7f45e68a7640 1 -- 172.21.15.26:0/595253554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 msgr2=0x7f45e015ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.242+0000 7f45e68a7640 1 --2- 172.21.15.26:0/595253554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 0x7f45e015ef50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f45e006ecf0 tx=0x7f45d802a6a0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.242+0000 7f45e68a7640 1 -- 172.21.15.26:0/595253554 shutdown_connections 2024-04-24T23:26:10.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.242+0000 7f45e68a7640 1 --2- 172.21.15.26:0/595253554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 0x7f45e015ef50 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.242+0000 7f45e68a7640 1 --2- 172.21.15.26:0/595253554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 0x7f45e0072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.242+0000 7f45e68a7640 1 --2- 172.21.15.26:0/595253554 >> v2:172.21.15.29:3300/0 conn(0x7f45e0143c40 0x7f45e00cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.242+0000 7f45e68a7640 1 -- 172.21.15.26:0/595253554 >> 172.21.15.26:0/595253554 conn(0x7f45e006b300 msgr2=0x7f45e006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.242+0000 7f45e68a7640 1 -- 172.21.15.26:0/595253554 shutdown_connections 2024-04-24T23:26:10.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.242+0000 7f45e68a7640 1 -- 172.21.15.26:0/595253554 wait complete. 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.243+0000 7f45e68a7640 1 Processor -- start 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.243+0000 7f45e68a7640 1 -- start start 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e68a7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 0x7f45e0135270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e68a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45e0143c40 0x7f45e0136ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e68a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 0x7f45e0137420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e68a7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0144020 con 0x7f45e015eb00 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e68a7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0071800 con 0x7f45e0143c40 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e68a7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f45e0072540 con 0x7f45e00cc860 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e4e44640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 0x7f45e0137420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e4e44640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 0x7f45e0137420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53616/0 (socket says 172.21.15.26:53616) 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e4e44640 1 -- 172.21.15.26:0/4180502058 learned_addr learned my addr 172.21.15.26:0/4180502058 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45dffff640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 0x7f45e0135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45df7fe640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45e0143c40 0x7f45e0136ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e4e44640 1 -- 172.21.15.26:0/4180502058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 msgr2=0x7f45e0135270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e4e44640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 0x7f45e0135270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e4e44640 1 -- 172.21.15.26:0/4180502058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45e0143c40 msgr2=0x7f45e0136ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e4e44640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45e0143c40 0x7f45e0136ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e4e44640 1 -- 172.21.15.26:0/4180502058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e0141510 con 0x7f45e015eb00 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45df7fe640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45e0143c40 0x7f45e0136ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.244+0000 7f45e4e44640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 0x7f45e0137420 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f45d802ac90 tx=0x7f45d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.245+0000 7f45dcff9640 1 -- 172.21.15.26:0/4180502058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d803f020 con 0x7f45e015eb00 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.245+0000 7f45dcff9640 1 -- 172.21.15.26:0/4180502058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d8037040 con 0x7f45e015eb00 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.245+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e01417f0 con 0x7f45e015eb00 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.245+0000 7f45dcff9640 1 -- 172.21.15.26:0/4180502058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8036070 con 0x7f45e015eb00 2024-04-24T23:26:10.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.245+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e0141db0 con 0x7f45e015eb00 2024-04-24T23:26:10.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.246+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e00694c0 con 0x7f45e015eb00 2024-04-24T23:26:10.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.246+0000 7f45dcff9640 1 -- 172.21.15.26:0/4180502058 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45d8042020 con 0x7f45e015eb00 2024-04-24T23:26:10.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.247+0000 7f45dcff9640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f45c006be30 0x7f45c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.247+0000 7f45dcff9640 1 -- 172.21.15.26:0/4180502058 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f45d80ab0f0 con 0x7f45e015eb00 2024-04-24T23:26:10.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.247+0000 7f45dffff640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f45c006be30 0x7f45c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.247+0000 7f45dffff640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f45c006be30 0x7f45c006e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f45d000aac0 tx=0x7f45d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.251+0000 7f45dcff9640 1 -- 172.21.15.26:0/4180502058 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45d8049030 con 0x7f45e015eb00 2024-04-24T23:26:10.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.438+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45e015e210 con 0x7f45e015eb00 2024-04-24T23:26:10.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.438+0000 7f45dcff9640 1 -- 172.21.15.26:0/4180502058 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13221 (secure 0 0 0) 0x7f45d8079cf0 con 0x7f45e015eb00 2024-04-24T23:26:10.440 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:10.440 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":74,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:26:09.112163+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6834/1431580336":"2024-04-25T23:26:05.947305+0000","172.21.15.29:6835/2057165440":"2024-04-25T23:26:05.947294+0000","172.21.15.29:6833/2057165440":"2024-04-25T23:26:05.947294+0000","172.21.15.26:6836/1431580336":"2024-04-25T23:26:05.947305+0000","172.21.15.26:6837/2537133693":"2024-04-25T23:26:05.947310+0000","172.21.15.26:6835/2537133693":"2024-04-25T23:26:05.947310+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.440+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f45c006be30 msgr2=0x7f45c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.440+0000 7f45e68a7640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f45c006be30 0x7f45c006e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f45d000aac0 tx=0x7f45d0008040 comp rx=0 tx=0).stop 2024-04-24T23:26:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.440+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 msgr2=0x7f45e0137420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.440+0000 7f45e68a7640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 0x7f45e0137420 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f45d802ac90 tx=0x7f45d8004a40 comp rx=0 tx=0).stop 2024-04-24T23:26:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.441+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 shutdown_connections 2024-04-24T23:26:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.441+0000 7f45e68a7640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f45c006be30 0x7f45c006e2f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.441+0000 7f45e68a7640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45e015eb00 0x7f45e0137420 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.441+0000 7f45e68a7640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45e0143c40 0x7f45e0136ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.441+0000 7f45e68a7640 1 --2- 172.21.15.26:0/4180502058 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45e00cc860 0x7f45e0135270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.441+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 >> 172.21.15.26:0/4180502058 conn(0x7f45e006b300 msgr2=0x7f45e00cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.441+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 shutdown_connections 2024-04-24T23:26:10.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.441+0000 7f45e68a7640 1 -- 172.21.15.26:0/4180502058 wait complete. 2024-04-24T23:26:10.453 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:26:10.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:26:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.556+0000 7f905936a640 1 Processor -- start 2024-04-24T23:26:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.556+0000 7f905936a640 1 -- start start 2024-04-24T23:26:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.556+0000 7f905936a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 0x7f90540acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.556+0000 7f905936a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90540b4ce0 0x7f905415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.556+0000 7f905936a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90540aeb10 con 0x7f905415be40 2024-04-24T23:26:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f905936a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f905409df20 con 0x7f90540b4ce0 2024-04-24T23:26:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f905936a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f90540aedf0 con 0x7f90540d0750 2024-04-24T23:26:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f90537fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90540b4ce0 0x7f905415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f90537fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90540b4ce0 0x7f905415c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53624/0 (socket says 172.21.15.26:53624) 2024-04-24T23:26:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f90537fe640 1 -- 172.21.15.26:0/2014436480 learned_addr learned my addr 172.21.15.26:0/2014436480 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f9052ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f905415be40 0x7f90540d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55368/0 (socket says 172.21.15.26:55368) 2024-04-24T23:26:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f904bfff640 1 --2- 172.21.15.26:0/2014436480 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 0x7f90540acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f90537fe640 1 -- 172.21.15.26:0/2014436480 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 msgr2=0x7f90540acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f90537fe640 1 --2- 172.21.15.26:0/2014436480 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 0x7f90540acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f90537fe640 1 -- 172.21.15.26:0/2014436480 >> v1:172.21.15.26:6789/0 conn(0x7f905415be40 legacy=0x7f90540d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f90537fe640 1 -- 172.21.15.26:0/2014436480 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90540af0d0 con 0x7f90540b4ce0 2024-04-24T23:26:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.557+0000 7f904bfff640 1 --2- 172.21.15.26:0/2014436480 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 0x7f90540acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.558+0000 7f90537fe640 1 --2- 172.21.15.26:0/2014436480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 0x7f905415c220 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f904c009e50 tx=0x7f904c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634a8391d4df2aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.558+0000 7f90527fc640 1 -- 172.21.15.26:0/2014436480 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c036070 con 0x7f90540b4ce0 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.558+0000 7f90527fc640 1 -- 172.21.15.26:0/2014436480 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904c031600 con 0x7f90540b4ce0 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.558+0000 7f90527fc640 1 -- 172.21.15.26:0/2014436480 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c0319d0 con 0x7f90540b4ce0 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.558+0000 7f905936a640 1 -- 172.21.15.26:0/2014436480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 msgr2=0x7f905415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.558+0000 7f905936a640 1 --2- 172.21.15.26:0/2014436480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 0x7f905415c220 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f904c009e50 tx=0x7f904c02a710 comp rx=0 tx=0).stop 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.558+0000 7f905936a640 1 -- 172.21.15.26:0/2014436480 shutdown_connections 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.558+0000 7f905936a640 1 --2- 172.21.15.26:0/2014436480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 0x7f905415c220 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.558+0000 7f905936a640 1 --2- 172.21.15.26:0/2014436480 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 0x7f90540acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.559+0000 7f905936a640 1 -- 172.21.15.26:0/2014436480 >> 172.21.15.26:0/2014436480 conn(0x7f905406b540 msgr2=0x7f905406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.559+0000 7f905936a640 1 -- 172.21.15.26:0/2014436480 shutdown_connections 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.559+0000 7f905936a640 1 -- 172.21.15.26:0/2014436480 wait complete. 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.560+0000 7f905936a640 1 Processor -- start 2024-04-24T23:26:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.560+0000 7f905936a640 1 -- start start 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.560+0000 7f905936a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 0x7f9054143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f905936a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 0x7f905409cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f905936a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f905415be40 0x7f905409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f905936a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f905406cd10 con 0x7f90540b4ce0 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f905936a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f905406cb90 con 0x7f905415be40 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f905936a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f905415c760 con 0x7f90540d0750 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f9052ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 0x7f9054143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f9052ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 0x7f9054143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53636/0 (socket says 172.21.15.26:53636) 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f9052ffd640 1 -- 172.21.15.26:0/2148810727 learned_addr learned my addr 172.21.15.26:0/2148810727 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f904bfff640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 0x7f905409cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f90537fe640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f905415be40 0x7f905409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f9052ffd640 1 -- 172.21.15.26:0/2148810727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 msgr2=0x7f905409cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f9052ffd640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 0x7f905409cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f9052ffd640 1 -- 172.21.15.26:0/2148810727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f905415be40 msgr2=0x7f905409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f9052ffd640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f905415be40 0x7f905409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f9052ffd640 1 -- 172.21.15.26:0/2148810727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905415cc90 con 0x7f90540b4ce0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f9052ffd640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 0x7f9054143e00 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f904400c3c0 tx=0x7f904400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f904b7fe640 1 -- 172.21.15.26:0/2148810727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440074e0 con 0x7f90540b4ce0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f904b7fe640 1 -- 172.21.15.26:0/2148810727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9044007680 con 0x7f90540b4ce0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90541443e0 con 0x7f90540b4ce0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f904b7fe640 1 -- 172.21.15.26:0/2148810727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044004a00 con 0x7f90540b4ce0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.561+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90541449a0 con 0x7f90540b4ce0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.562+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9054067a40 con 0x7f90540b4ce0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.563+0000 7f904b7fe640 1 -- 172.21.15.26:0/2148810727 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f904401a020 con 0x7f90540b4ce0 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.563+0000 7f904b7fe640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f903406be30 0x7f903406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.563+0000 7f904b7fe640 1 -- 172.21.15.26:0/2148810727 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f904408a390 con 0x7f90540b4ce0 2024-04-24T23:26:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.563+0000 7f904bfff640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f903406be30 0x7f903406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.564+0000 7f904bfff640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f903406be30 0x7f903406e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9054151260 tx=0x7f903c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.568+0000 7f904b7fe640 1 -- 172.21.15.26:0/2148810727 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9054067a40 con 0x7f90540b4ce0 2024-04-24T23:26:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.749+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9054075610 con 0x7f90540b4ce0 2024-04-24T23:26:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.750+0000 7f904b7fe640 1 -- 172.21.15.26:0/2148810727 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f9044058f90 con 0x7f90540b4ce0 2024-04-24T23:26:10.751 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f903406be30 msgr2=0x7f903406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f903406be30 0x7f903406e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9054151260 tx=0x7f903c008040 comp rx=0 tx=0).stop 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 msgr2=0x7f9054143e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 0x7f9054143e00 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f904400c3c0 tx=0x7f904400c880 comp rx=0 tx=0).stop 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 shutdown_connections 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f903406be30 0x7f903406e2f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f905415be40 0x7f905409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f90540d0750 0x7f905409cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 --2- 172.21.15.26:0/2148810727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90540b4ce0 0x7f9054143e00 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 >> 172.21.15.26:0/2148810727 conn(0x7f905406b540 msgr2=0x7f90540b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 shutdown_connections 2024-04-24T23:26:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.752+0000 7f905936a640 1 -- 172.21.15.26:0/2148810727 wait complete. 2024-04-24T23:26:10.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-04-24T23:26:10.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.864+0000 7f51bd79d640 1 Processor -- start 2024-04-24T23:26:10.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.864+0000 7f51bd79d640 1 -- start start 2024-04-24T23:26:10.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51bd79d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51b8143c40 0x7f51b80cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51bd79d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f51b815eb00 0x7f51b815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51bd79d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b815f450 con 0x7f51b80cc860 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51bd79d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f51b8071b70 con 0x7f51b8143c40 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51bd79d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f51b80cd0b0 con 0x7f51b815eb00 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51b6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51b8143c40 0x7f51b80cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51b6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51b8143c40 0x7f51b80cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53640/0 (socket says 172.21.15.26:53640) 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51b6ffd640 1 -- 172.21.15.26:0/261160222 learned_addr learned my addr 172.21.15.26:0/261160222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51b6ffd640 1 -- 172.21.15.26:0/261160222 >> v2:172.21.15.29:3300/0 conn(0x7f51b815eb00 msgr2=0x7f51b815ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51b77fe640 1 --2- 172.21.15.26:0/261160222 >> v2:172.21.15.29:3300/0 conn(0x7f51b815eb00 0x7f51b815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51b6ffd640 1 --2- 172.21.15.26:0/261160222 >> v2:172.21.15.29:3300/0 conn(0x7f51b815eb00 0x7f51b815ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51b6ffd640 1 -- 172.21.15.26:0/261160222 >> v1:172.21.15.29:6789/0 conn(0x7f51b80cc860 legacy=0x7f51b8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.865+0000 7f51b6ffd640 1 -- 172.21.15.26:0/261160222 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b8071240 con 0x7f51b8143c40 2024-04-24T23:26:10.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.866+0000 7f51b6ffd640 1 --2- 172.21.15.26:0/261160222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b8143c40 0x7f51b80cf870 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f51b8144d10 tx=0x7f51b002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f82cea4ebabb83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.866+0000 7f51b5ffb640 1 -- 172.21.15.26:0/261160222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b00325a0 con 0x7f51b8143c40 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51b5ffb640 1 -- 172.21.15.26:0/261160222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b0038070 con 0x7f51b8143c40 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51b5ffb640 1 -- 172.21.15.26:0/261160222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0032960 con 0x7f51b8143c40 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51bd79d640 1 -- 172.21.15.26:0/261160222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b8143c40 msgr2=0x7f51b80cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51bd79d640 1 --2- 172.21.15.26:0/261160222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b8143c40 0x7f51b80cf870 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f51b8144d10 tx=0x7f51b002c8e0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51bd79d640 1 -- 172.21.15.26:0/261160222 shutdown_connections 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51bd79d640 1 --2- 172.21.15.26:0/261160222 >> v2:172.21.15.29:3300/0 conn(0x7f51b815eb00 0x7f51b815ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51bd79d640 1 --2- 172.21.15.26:0/261160222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b8143c40 0x7f51b80cf870 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51bd79d640 1 -- 172.21.15.26:0/261160222 >> 172.21.15.26:0/261160222 conn(0x7f51b806b300 msgr2=0x7f51b806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51bd79d640 1 -- 172.21.15.26:0/261160222 shutdown_connections 2024-04-24T23:26:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.867+0000 7f51bd79d640 1 -- 172.21.15.26:0/261160222 wait complete. 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.868+0000 7f51bd79d640 1 Processor -- start 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.868+0000 7f51bd79d640 1 -- start start 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51bd79d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51b80cc860 0x7f51b815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51bd79d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51b8143c40 0x7f51b80b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51bd79d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b815eb00 0x7f51b815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51bd79d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8144020 con 0x7f51b815eb00 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51bd79d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8071800 con 0x7f51b8143c40 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51bd79d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f51b80cccd0 con 0x7f51b80cc860 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b815eb00 0x7f51b815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b815eb00 0x7f51b815a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53652/0 (socket says 172.21.15.26:53652) 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51b77fe640 1 -- 172.21.15.26:0/2581240317 learned_addr learned my addr 172.21.15.26:0/2581240317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51b6ffd640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51b80cc860 0x7f51b815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.869+0000 7f51b67fc640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51b8143c40 0x7f51b80b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51b77fe640 1 -- 172.21.15.26:0/2581240317 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51b80cc860 msgr2=0x7f51b815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51b77fe640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51b80cc860 0x7f51b815df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51b77fe640 1 -- 172.21.15.26:0/2581240317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51b8143c40 msgr2=0x7f51b80b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51b77fe640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51b8143c40 0x7f51b80b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51b77fe640 1 -- 172.21.15.26:0/2581240317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b80b0f90 con 0x7f51b815eb00 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51b77fe640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b815eb00 0x7f51b815a210 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f51a800cc30 tx=0x7f51a800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51af7fe640 1 -- 172.21.15.26:0/2581240317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a800ce40 con 0x7f51b815eb00 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51af7fe640 1 -- 172.21.15.26:0/2581240317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a8003990 con 0x7f51b815eb00 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b815dc70 con 0x7f51b815eb00 2024-04-24T23:26:10.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51af7fe640 1 -- 172.21.15.26:0/2581240317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8003d40 con 0x7f51b815eb00 2024-04-24T23:26:10.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.870+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b815e490 con 0x7f51b815eb00 2024-04-24T23:26:10.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.871+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b80694c0 con 0x7f51b815eb00 2024-04-24T23:26:10.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.871+0000 7f51af7fe640 1 -- 172.21.15.26:0/2581240317 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51a8003ee0 con 0x7f51b815eb00 2024-04-24T23:26:10.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.872+0000 7f51af7fe640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f519806be30 0x7f519806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.872+0000 7f51af7fe640 1 -- 172.21.15.26:0/2581240317 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f51a808af10 con 0x7f51b815eb00 2024-04-24T23:26:10.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.872+0000 7f51b6ffd640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f519806be30 0x7f519806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.872+0000 7f51b6ffd640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f519806be30 0x7f519806e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f51b0037000 tx=0x7f51b0032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:10.876+0000 7f51af7fe640 1 -- 172.21.15.26:0/2581240317 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f51a8014030 con 0x7f51b815eb00 2024-04-24T23:26:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:11.133+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) -- 0x7f51b80b1160 con 0x7f51b815eb00 2024-04-24T23:26:12.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.126+0000 7f51af7fe640 1 -- 172.21.15.26:0/2581240317 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v280) ==== 157+0+0 (secure 0 0 0) 0x7f51b80b1160 con 0x7f51b815eb00 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f519806be30 msgr2=0x7f519806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f519806be30 0x7f519806e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f51b0037000 tx=0x7f51b0032000 comp rx=0 tx=0).stop 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b815eb00 msgr2=0x7f51b815a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b815eb00 0x7f51b815a210 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f51a800cc30 tx=0x7f51a800a5c0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 shutdown_connections 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f519806be30 0x7f519806e2f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51b815eb00 0x7f51b815a210 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f51b8143c40 0x7f51b80b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 --2- 172.21.15.26:0/2581240317 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51b80cc860 0x7f51b815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.129+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 >> 172.21.15.26:0/2581240317 conn(0x7f51b806b300 msgr2=0x7f51b813e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.130+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 shutdown_connections 2024-04-24T23:26:12.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.130+0000 7f51bd79d640 1 -- 172.21.15.26:0/2581240317 wait complete. 2024-04-24T23:26:12.143 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:26:12.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.249+0000 7f228e3ee640 1 Processor -- start 2024-04-24T23:26:12.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.249+0000 7f228e3ee640 1 -- start start 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228e3ee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f22880cd120 0x7f22880af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228e3ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22880c5350 0x7f228806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228e3ee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228806f0a0 con 0x7f22880c4ef0 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228e3ee640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f22880c69e0 con 0x7f22880c5350 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228e3ee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f22880c6ef0 con 0x7f22880cd120 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228c98b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22880c5350 0x7f228806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228c98b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22880c5350 0x7f228806d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53654/0 (socket says 172.21.15.26:53654) 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228c98b640 1 -- 172.21.15.26:0/1756668570 learned_addr learned my addr 172.21.15.26:0/1756668570 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228c98b640 1 -- 172.21.15.26:0/1756668570 >> v2:172.21.15.29:3300/0 conn(0x7f22880cd120 msgr2=0x7f22880af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228c98b640 1 --2- 172.21.15.26:0/1756668570 >> v2:172.21.15.29:3300/0 conn(0x7f22880cd120 0x7f22880af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228c98b640 1 -- 172.21.15.26:0/1756668570 >> v1:172.21.15.29:6789/0 conn(0x7f22880c4ef0 legacy=0x7f228806e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.250+0000 7f228c98b640 1 -- 172.21.15.26:0/1756668570 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22880711e0 con 0x7f22880c5350 2024-04-24T23:26:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.251+0000 7f228c98b640 1 --2- 172.21.15.26:0/1756668570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c5350 0x7f228806d240 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2288068950 tx=0x7f228002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eede5d07d5e5c378 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.251+0000 7f2286ffd640 1 -- 172.21.15.26:0/1756668570 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280036070 con 0x7f22880c5350 2024-04-24T23:26:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.251+0000 7f2286ffd640 1 -- 172.21.15.26:0/1756668570 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2280031600 con 0x7f22880c5350 2024-04-24T23:26:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.251+0000 7f2286ffd640 1 -- 172.21.15.26:0/1756668570 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22800319d0 con 0x7f22880c5350 2024-04-24T23:26:12.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.252+0000 7f228e3ee640 1 -- 172.21.15.26:0/1756668570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c5350 msgr2=0x7f228806d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.252+0000 7f228e3ee640 1 --2- 172.21.15.26:0/1756668570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c5350 0x7f228806d240 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2288068950 tx=0x7f228002a5d0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.252+0000 7f228e3ee640 1 -- 172.21.15.26:0/1756668570 shutdown_connections 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.252+0000 7f228e3ee640 1 --2- 172.21.15.26:0/1756668570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c5350 0x7f228806d240 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.252+0000 7f228e3ee640 1 --2- 172.21.15.26:0/1756668570 >> v2:172.21.15.29:3300/0 conn(0x7f22880cd120 0x7f22880af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.252+0000 7f228e3ee640 1 -- 172.21.15.26:0/1756668570 >> 172.21.15.26:0/1756668570 conn(0x7f22880b1c80 msgr2=0x7f22880b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.252+0000 7f228e3ee640 1 -- 172.21.15.26:0/1756668570 shutdown_connections 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.252+0000 7f228e3ee640 1 -- 172.21.15.26:0/1756668570 wait complete. 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.253+0000 7f228e3ee640 1 Processor -- start 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.253+0000 7f228e3ee640 1 -- start start 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f228e3ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c4ef0 0x7f228815f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f228e3ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22880c5350 0x7f228814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f228e3ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22880cd120 0x7f22880b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f228e3ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22880cfd10 con 0x7f22880c4ef0 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f228e3ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f22880cfa30 con 0x7f22880c5350 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f228e3ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f22880cee80 con 0x7f22880cd120 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2287fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c4ef0 0x7f228815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2287fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c4ef0 0x7f228815f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53660/0 (socket says 172.21.15.26:53660) 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2287fff640 1 -- 172.21.15.26:0/3430843344 learned_addr learned my addr 172.21.15.26:0/3430843344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f228c98b640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22880cd120 0x7f22880b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f22877fe640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22880c5350 0x7f228814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2287fff640 1 -- 172.21.15.26:0/3430843344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22880cd120 msgr2=0x7f22880b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2287fff640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22880cd120 0x7f22880b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2287fff640 1 -- 172.21.15.26:0/3430843344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22880c5350 msgr2=0x7f228814baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2287fff640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22880c5350 0x7f228814baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2287fff640 1 -- 172.21.15.26:0/3430843344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228815fef0 con 0x7f22880c4ef0 2024-04-24T23:26:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f228c98b640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22880cd120 0x7f22880b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2287fff640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c4ef0 0x7f228815f910 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f227000b440 tx=0x7f227000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.254+0000 7f2284ff9640 1 -- 172.21.15.26:0/3430843344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270015020 con 0x7f22880c4ef0 2024-04-24T23:26:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.255+0000 7f2284ff9640 1 -- 172.21.15.26:0/3430843344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2270011890 con 0x7f22880c4ef0 2024-04-24T23:26:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.255+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22881601d0 con 0x7f22880c4ef0 2024-04-24T23:26:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.255+0000 7f2284ff9640 1 -- 172.21.15.26:0/3430843344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270011c80 con 0x7f22880c4ef0 2024-04-24T23:26:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.255+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2288160790 con 0x7f22880c4ef0 2024-04-24T23:26:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.256+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2288075610 con 0x7f22880c4ef0 2024-04-24T23:26:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.256+0000 7f2284ff9640 1 -- 172.21.15.26:0/3430843344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2270018020 con 0x7f22880c4ef0 2024-04-24T23:26:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.257+0000 7f2284ff9640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f226806be30 0x7f226806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.257+0000 7f22877fe640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f226806be30 0x7f226806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.257+0000 7f2284ff9640 1 -- 172.21.15.26:0/3430843344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f227008ae50 con 0x7f22880c4ef0 2024-04-24T23:26:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.257+0000 7f22877fe640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f226806be30 0x7f226806e2f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f22880afc80 tx=0x7f2278009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.261+0000 7f2284ff9640 1 -- 172.21.15.26:0/3430843344 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f227001f020 con 0x7f22880c4ef0 2024-04-24T23:26:12.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.449+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f228809de00 con 0x7f22880c4ef0 2024-04-24T23:26:12.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.452+0000 7f2284ff9640 1 -- 172.21.15.26:0/3430843344 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f227000d040 con 0x7f22880c4ef0 2024-04-24T23:26:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f226806be30 msgr2=0x7f226806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f226806be30 0x7f226806e2f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f22880afc80 tx=0x7f2278009040 comp rx=0 tx=0).stop 2024-04-24T23:26:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c4ef0 msgr2=0x7f228815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c4ef0 0x7f228815f910 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f227000b440 tx=0x7f227000b900 comp rx=0 tx=0).stop 2024-04-24T23:26:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 shutdown_connections 2024-04-24T23:26:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f226806be30 0x7f226806e2f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f22880cd120 0x7f22880b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f22880c5350 0x7f228814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 --2- 172.21.15.26:0/3430843344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22880c4ef0 0x7f228815f910 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 >> 172.21.15.26:0/3430843344 conn(0x7f22880b1c80 msgr2=0x7f22880b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 shutdown_connections 2024-04-24T23:26:12.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.454+0000 7f228e3ee640 1 -- 172.21.15.26:0/3430843344 wait complete. 2024-04-24T23:26:12.466 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:26:12.466 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:26:12.507 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:26:12.557 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:26:12.557 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:26:12.557 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:12.573 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:26:12.573 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:26:12.573 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:12.591 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:26:12.591 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:26:12.591 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:12.628 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:26:12.628 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:26:12.628 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:12.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:26:12.648 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:26:12.648 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:12.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.767+0000 7f88a1f63640 1 Processor -- start 2024-04-24T23:26:12.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.767+0000 7f88a1f63640 1 -- start start 2024-04-24T23:26:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f88a1f63640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f889c15e570 0x7f889c0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f88a1f63640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f889c0b3130 0x7f889c0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f88a1f63640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0d22e0 0x7f889c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f88a1f63640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f889c0d1ab0 con 0x7f889c0b3130 2024-04-24T23:26:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f88a1f63640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f889c0d2ae0 con 0x7f889c15e570 2024-04-24T23:26:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f88a1f63640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f889c0d32f0 con 0x7f889c0d22e0 2024-04-24T23:26:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f889c0b3130 0x7f889c0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f889c0b3130 0x7f889c0d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53662/0 (socket says 172.21.15.26:53662) 2024-04-24T23:26:12.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889affd640 1 -- 172.21.15.26:0/3881566910 learned_addr learned my addr 172.21.15.26:0/3881566910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889affd640 1 -- 172.21.15.26:0/3881566910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0d22e0 msgr2=0x7f889c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889bfff640 1 --2- 172.21.15.26:0/3881566910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0d22e0 0x7f889c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889b7fe640 1 --2- 172.21.15.26:0/3881566910 >> v2:172.21.15.29:3300/0 conn(0x7f889c15e570 0x7f889c0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889affd640 1 --2- 172.21.15.26:0/3881566910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0d22e0 0x7f889c0d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889affd640 1 -- 172.21.15.26:0/3881566910 >> v2:172.21.15.29:3300/0 conn(0x7f889c15e570 msgr2=0x7f889c0d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889affd640 1 --2- 172.21.15.26:0/3881566910 >> v2:172.21.15.29:3300/0 conn(0x7f889c15e570 0x7f889c0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.768+0000 7f889affd640 1 -- 172.21.15.26:0/3881566910 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c0d0270 con 0x7f889c0b3130 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.769+0000 7f889affd640 1 --2- 172.21.15.26:0/3881566910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0b3130 0x7f889c0d46a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f889c0d52d0 tx=0x7f889002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50e63597302ef67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.769+0000 7f889a7fc640 1 -- 172.21.15.26:0/3881566910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890037070 con 0x7f889c0b3130 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.769+0000 7f889a7fc640 1 -- 172.21.15.26:0/3881566910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8890032440 con 0x7f889c0b3130 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.769+0000 7f889a7fc640 1 -- 172.21.15.26:0/3881566910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890032810 con 0x7f889c0b3130 2024-04-24T23:26:12.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.770+0000 7f88a1f63640 1 -- 172.21.15.26:0/3881566910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0b3130 msgr2=0x7f889c0d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.770+0000 7f88a1f63640 1 --2- 172.21.15.26:0/3881566910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0b3130 0x7f889c0d46a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f889c0d52d0 tx=0x7f889002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.770+0000 7f88a1f63640 1 -- 172.21.15.26:0/3881566910 shutdown_connections 2024-04-24T23:26:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.770+0000 7f88a1f63640 1 --2- 172.21.15.26:0/3881566910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0b3130 0x7f889c0d46a0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.770+0000 7f88a1f63640 1 --2- 172.21.15.26:0/3881566910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0d22e0 0x7f889c0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.770+0000 7f88a1f63640 1 --2- 172.21.15.26:0/3881566910 >> v2:172.21.15.29:3300/0 conn(0x7f889c15e570 0x7f889c0d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.770+0000 7f88a1f63640 1 -- 172.21.15.26:0/3881566910 >> 172.21.15.26:0/3881566910 conn(0x7f889c08dbe0 msgr2=0x7f889c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.770+0000 7f88a1f63640 1 -- 172.21.15.26:0/3881566910 shutdown_connections 2024-04-24T23:26:12.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.770+0000 7f88a1f63640 1 -- 172.21.15.26:0/3881566910 wait complete. 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.771+0000 7f88a1f63640 1 Processor -- start 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.771+0000 7f88a1f63640 1 -- start start 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f88a1f63640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0b3130 0x7f889c150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f88a1f63640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d22e0 0x7f889c0b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f88a1f63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c15e570 0x7f889c0b9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f88a1f63640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f889c0cfa70 con 0x7f889c0d22e0 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f88a1f63640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f889c0d3010 con 0x7f889c15e570 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f88a1f63640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f889c0cfd50 con 0x7f889c0b3130 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d22e0 0x7f889c0b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d22e0 0x7f889c0b72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53666/0 (socket says 172.21.15.26:53666) 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889affd640 1 -- 172.21.15.26:0/2878639555 learned_addr learned my addr 172.21.15.26:0/2878639555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889b7fe640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0b3130 0x7f889c150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889bfff640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c15e570 0x7f889c0b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889affd640 1 -- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0b3130 msgr2=0x7f889c150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889affd640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0b3130 0x7f889c150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889affd640 1 -- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c15e570 msgr2=0x7f889c0b9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889affd640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c15e570 0x7f889c0b9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889affd640 1 -- 172.21.15.26:0/2878639555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c0b7cd0 con 0x7f889c0d22e0 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889b7fe640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0b3130 0x7f889c150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889bfff640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c15e570 0x7f889c0b9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.772+0000 7f889affd640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d22e0 0x7f889c0b72f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f889002a220 tx=0x7f88900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.773+0000 7f8883fff640 1 -- 172.21.15.26:0/2878639555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890037040 con 0x7f889c0d22e0 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.773+0000 7f8883fff640 1 -- 172.21.15.26:0/2878639555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8890042070 con 0x7f889c0d22e0 2024-04-24T23:26:12.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.773+0000 7f8883fff640 1 -- 172.21.15.26:0/2878639555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890030bb0 con 0x7f889c0d22e0 2024-04-24T23:26:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.773+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c0b7fb0 con 0x7f889c0d22e0 2024-04-24T23:26:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.773+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c0b8850 con 0x7f889c0d22e0 2024-04-24T23:26:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.774+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889c0b83a0 con 0x7f889c0d22e0 2024-04-24T23:26:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.774+0000 7f8883fff640 1 -- 172.21.15.26:0/2878639555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8890047460 con 0x7f889c0d22e0 2024-04-24T23:26:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.774+0000 7f8883fff640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f888806be30 0x7f888806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.775+0000 7f8883fff640 1 -- 172.21.15.26:0/2878639555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f88900ab8e0 con 0x7f889c0d22e0 2024-04-24T23:26:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.775+0000 7f889b7fe640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f888806be30 0x7f888806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.775+0000 7f889b7fe640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f888806be30 0x7f888806e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b820 tx=0x7f888c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:12.779+0000 7f8883fff640 1 -- 172.21.15.26:0/2878639555 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8890045360 con 0x7f889c0d22e0 2024-04-24T23:26:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.002+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f889c0b8180 con 0x7f889c0d22e0 2024-04-24T23:26:13.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.003+0000 7f8883fff640 1 -- 172.21.15.26:0/2878639555 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3869 (secure 0 0 0) 0x7f889007a440 con 0x7f889c0d22e0 2024-04-24T23:26:13.004 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3667201771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3667201771},{"type":"v1","addr":"172.21.15.29:6835","nonce":3667201771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6361,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/1256833211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":1256833211},{"type":"v1","addr":"172.21.15.29:6834","nonce":1256833211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2462388508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2462388508},{"type":"v1","addr":"172.21.15.26:6835","nonce":2462388508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3699025480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3699025480},{"type":"v1","addr":"172.21.15.26:6837","nonce":3699025480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[]} 2024-04-24T23:26:13.005 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 77 2024-04-24T23:26:13.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f888806be30 msgr2=0x7f888806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:13.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f888806be30 0x7f888806e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b820 tx=0x7f888c008040 comp rx=0 tx=0).stop 2024-04-24T23:26:13.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d22e0 msgr2=0x7f889c0b72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:13.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d22e0 0x7f889c0b72f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f889002a220 tx=0x7f88900046c0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 shutdown_connections 2024-04-24T23:26:13.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f888806be30 0x7f888806e2f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f889c15e570 0x7f889c0b9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d22e0 0x7f889c0b72f0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 --2- 172.21.15.26:0/2878639555 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f889c0b3130 0x7f889c150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 >> 172.21.15.26:0/2878639555 conn(0x7f889c08dbe0 msgr2=0x7f889c0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:13.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 shutdown_connections 2024-04-24T23:26:13.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.005+0000 7f88a1f63640 1 -- 172.21.15.26:0/2878639555 wait complete. 2024-04-24T23:26:13.017 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:26:13.017 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:26:13.017 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:26:13.017 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:26:13.018 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:26:13.018 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:26:13.018 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:26:13.018 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:26:13.018 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:26:13.018 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:26:13.019 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:26:13.019 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:26:13.019 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:26:13.019 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:26:13.019 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:26:13.019 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:26:13.027 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:26:13.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:26:13.028 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:26:13.028 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:26:13.031 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:26:13.031 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:26:13.032 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:26:13.032 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:26:13.070 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:26:13.070 DEBUG:teuthology.parallel:result is None 2024-04-24T23:26:13.071 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:26:13.071 DEBUG:teuthology.parallel:result is None 2024-04-24T23:26:13.074 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:26:13.074 DEBUG:teuthology.parallel:result is None 2024-04-24T23:26:13.075 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:26:13.075 DEBUG:teuthology.parallel:result is None 2024-04-24T23:26:13.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:26:13.150 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:26:13.150 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:26:13.160 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:26:13.160 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:26:13.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.180+0000 7fb4dbd28640 1 Processor -- start 2024-04-24T23:26:13.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.180+0000 7fb4dbd28640 1 -- start start 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4dbd28640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb4d40d4870 0x7fb4d40b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4dbd28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4d40b3130 0x7fb4d40b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4dbd28640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d40d32f0 con 0x7fb4d415e570 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4dbd28640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d40d1ab0 con 0x7fb4d40b3130 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4dbd28640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d40d2ae0 con 0x7fb4d40d4870 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4da2c5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4d40b3130 0x7fb4d40b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4d9ac4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb4d415e570 0x7fb4d40d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55392/0 (socket says 172.21.15.26:55392) 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4da2c5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4d40b3130 0x7fb4d40b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53724/0 (socket says 172.21.15.26:53724) 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4d9ac4640 1 -- 172.21.15.26:0/1143173888 learned_addr learned my addr 172.21.15.26:0/1143173888 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4da2c5640 1 -- 172.21.15.26:0/1143173888 >> v2:172.21.15.29:3300/0 conn(0x7fb4d40d4870 msgr2=0x7fb4d40b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4da2c5640 1 --2- 172.21.15.26:0/1143173888 >> v2:172.21.15.29:3300/0 conn(0x7fb4d40d4870 0x7fb4d40b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4da2c5640 1 -- 172.21.15.26:0/1143173888 >> v1:172.21.15.26:6789/0 conn(0x7fb4d415e570 legacy=0x7fb4d40d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.181+0000 7fb4da2c5640 1 -- 172.21.15.26:0/1143173888 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d40d0270 con 0x7fb4d40b3130 2024-04-24T23:26:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.182+0000 7fb4da2c5640 1 --2- 172.21.15.26:0/1143173888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d40b3130 0x7fb4d40b4640 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb4d408cae0 tx=0x7fb4cc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f985f2caabf9e1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.182+0000 7fb4d8ac2640 1 -- 172.21.15.26:0/1143173888 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc007740 con 0x7fb4d40b3130 2024-04-24T23:26:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.182+0000 7fb4d8ac2640 1 -- 172.21.15.26:0/1143173888 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc0078e0 con 0x7fb4d40b3130 2024-04-24T23:26:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.182+0000 7fb4d8ac2640 1 -- 172.21.15.26:0/1143173888 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc007cb0 con 0x7fb4d40b3130 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.182+0000 7fb4dbd28640 1 -- 172.21.15.26:0/1143173888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d40b3130 msgr2=0x7fb4d40b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.182+0000 7fb4dbd28640 1 --2- 172.21.15.26:0/1143173888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d40b3130 0x7fb4d40b4640 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb4d408cae0 tx=0x7fb4cc02a5f0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.183+0000 7fb4dbd28640 1 -- 172.21.15.26:0/1143173888 shutdown_connections 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.183+0000 7fb4dbd28640 1 --2- 172.21.15.26:0/1143173888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d40b3130 0x7fb4d40b4640 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.183+0000 7fb4dbd28640 1 --2- 172.21.15.26:0/1143173888 >> v2:172.21.15.29:3300/0 conn(0x7fb4d40d4870 0x7fb4d40b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.183+0000 7fb4dbd28640 1 -- 172.21.15.26:0/1143173888 >> 172.21.15.26:0/1143173888 conn(0x7fb4d408dbe0 msgr2=0x7fb4d409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.183+0000 7fb4dbd28640 1 -- 172.21.15.26:0/1143173888 shutdown_connections 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.183+0000 7fb4dbd28640 1 -- 172.21.15.26:0/1143173888 wait complete. 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.184+0000 7fb4dbd28640 1 Processor -- start 2024-04-24T23:26:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.184+0000 7fb4dbd28640 1 -- start start 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4dbd28640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d40b3130 0x7fb4d40b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4dbd28640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4d40d4870 0x7fb4d40b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4dbd28640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d415e570 0x7fb4d414c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4dbd28640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d40cfa70 con 0x7fb4d415e570 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4dbd28640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d40d3010 con 0x7fb4d40b3130 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4dbd28640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d40cfd50 con 0x7fb4d40d4870 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4da2c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d415e570 0x7fb4d414c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4da2c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d415e570 0x7fb4d414c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53726/0 (socket says 172.21.15.26:53726) 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4da2c5640 1 -- 172.21.15.26:0/3651522925 learned_addr learned my addr 172.21.15.26:0/3651522925 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4d9ac4640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d40b3130 0x7fb4d40b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4d92c3640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4d40d4870 0x7fb4d40b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4da2c5640 1 -- 172.21.15.26:0/3651522925 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4d40d4870 msgr2=0x7fb4d40b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4da2c5640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4d40d4870 0x7fb4d40b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4da2c5640 1 -- 172.21.15.26:0/3651522925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d40b3130 msgr2=0x7fb4d40b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4da2c5640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d40b3130 0x7fb4d40b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4da2c5640 1 -- 172.21.15.26:0/3651522925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d40b9420 con 0x7fb4d415e570 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4d9ac4640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d40b3130 0x7fb4d40b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4da2c5640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d415e570 0x7fb4d414c4c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02abe0 tx=0x7fb4cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4ca7fc640 1 -- 172.21.15.26:0/3651522925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc040070 con 0x7fb4d415e570 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4ca7fc640 1 -- 172.21.15.26:0/3651522925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc031bc0 con 0x7fb4d415e570 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d41f7a40 con 0x7fb4d415e570 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.185+0000 7fb4ca7fc640 1 -- 172.21.15.26:0/3651522925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc045460 con 0x7fb4d415e570 2024-04-24T23:26:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.186+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41f8000 con 0x7fb4d415e570 2024-04-24T23:26:13.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.187+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d40d3d70 con 0x7fb4d415e570 2024-04-24T23:26:13.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.187+0000 7fb4ca7fc640 1 -- 172.21.15.26:0/3651522925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4cc039040 con 0x7fb4d415e570 2024-04-24T23:26:13.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.187+0000 7fb4ca7fc640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4b806be30 0x7fb4b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:13.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.187+0000 7fb4ca7fc640 1 -- 172.21.15.26:0/3651522925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7fb4cc0ac5b0 con 0x7fb4d415e570 2024-04-24T23:26:13.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.188+0000 7fb4d9ac4640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4b806be30 0x7fb4b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:13.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.188+0000 7fb4d9ac4640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4b806be30 0x7fb4b806e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0000f60 tx=0x7fb4d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:13.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.192+0000 7fb4ca7fc640 1 -- 172.21.15.26:0/3651522925 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb4d40d3d70 con 0x7fb4d415e570 2024-04-24T23:26:13.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:13.387+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb4d40b5430 con 0x7fb4d415e570 2024-04-24T23:26:14.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.136+0000 7fb4ca7fc640 1 -- 172.21.15.26:0/3651522925 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) ==== 97+0+0 (secure 0 0 0) 0x7fb4d40b5430 con 0x7fb4d415e570 2024-04-24T23:26:14.138 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-04-24T23:26:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4b806be30 msgr2=0x7fb4b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4b806be30 0x7fb4b806e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0000f60 tx=0x7fb4d0009040 comp rx=0 tx=0).stop 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d415e570 msgr2=0x7fb4d414c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d415e570 0x7fb4d414c4c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02abe0 tx=0x7fb4cc004a40 comp rx=0 tx=0).stop 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 shutdown_connections 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb4b806be30 0x7fb4b806e2f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d415e570 0x7fb4d414c4c0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb4d40d4870 0x7fb4d40b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 --2- 172.21.15.26:0/3651522925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb4d40b3130 0x7fb4d40b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 >> 172.21.15.26:0/3651522925 conn(0x7fb4d408dbe0 msgr2=0x7fb4d40d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.140+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 shutdown_connections 2024-04-24T23:26:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.141+0000 7fb4dbd28640 1 -- 172.21.15.26:0/3651522925 wait complete. 2024-04-24T23:26:14.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:26:14.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.263+0000 7f003d8a5640 1 Processor -- start 2024-04-24T23:26:14.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.264+0000 7f003d8a5640 1 -- start start 2024-04-24T23:26:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.264+0000 7f003d8a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f003815e3a0 0x7f00380d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.264+0000 7f003d8a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 0x7f00380d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.264+0000 7f003d8a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00380d3130 con 0x7f00380b4310 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.264+0000 7f003d8a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f00380b5060 con 0x7f003815e3a0 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.264+0000 7f003d8a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f00380d0bf0 con 0x7f00380d2770 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.265+0000 7f0036ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f003815e3a0 0x7f00380d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.265+0000 7f00377fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f00380b4310 0x7f00380b47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55790/0 (socket says 172.21.15.26:55790) 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.265+0000 7f00377fe640 1 -- 172.21.15.26:0/762089484 learned_addr learned my addr 172.21.15.26:0/762089484 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.265+0000 7f0036ffd640 1 -- 172.21.15.26:0/762089484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 msgr2=0x7f00380d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.265+0000 7f0036ffd640 1 --2- 172.21.15.26:0/762089484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 0x7f00380d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.265+0000 7f0036ffd640 1 -- 172.21.15.26:0/762089484 >> v1:172.21.15.26:6789/0 conn(0x7f00380b4310 legacy=0x7f00380b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.265+0000 7f0036ffd640 1 -- 172.21.15.26:0/762089484 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00380d0ed0 con 0x7f003815e3a0 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.265+0000 7f0035ffb640 1 -- 172.21.15.26:0/762089484 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 659694581 0 0) 0x7f0028001490 con 0x7f00380b4310 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f0036ffd640 1 --2- 172.21.15.26:0/762089484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 0x7f00380d16d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f00300097a0 tx=0x7f003002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c42b2d4cccd1c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f0035ffb640 1 -- 172.21.15.26:0/762089484 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030037070 con 0x7f003815e3a0 2024-04-24T23:26:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f0035ffb640 1 -- 172.21.15.26:0/762089484 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0030032440 con 0x7f003815e3a0 2024-04-24T23:26:14.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f0035ffb640 1 -- 172.21.15.26:0/762089484 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030032810 con 0x7f003815e3a0 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f003d8a5640 1 -- 172.21.15.26:0/762089484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 msgr2=0x7f00380d16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f003d8a5640 1 --2- 172.21.15.26:0/762089484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 0x7f00380d16d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f00300097a0 tx=0x7f003002a180 comp rx=0 tx=0).stop 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f003d8a5640 1 -- 172.21.15.26:0/762089484 shutdown_connections 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f003d8a5640 1 --2- 172.21.15.26:0/762089484 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 0x7f00380d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f003d8a5640 1 --2- 172.21.15.26:0/762089484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 0x7f00380d16d0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f003d8a5640 1 -- 172.21.15.26:0/762089484 >> 172.21.15.26:0/762089484 conn(0x7f003808dbd0 msgr2=0x7f003809d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f003d8a5640 1 -- 172.21.15.26:0/762089484 shutdown_connections 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.266+0000 7f003d8a5640 1 -- 172.21.15.26:0/762089484 wait complete. 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.268+0000 7f003d8a5640 1 Processor -- start 2024-04-24T23:26:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.268+0000 7f003d8a5640 1 -- start start 2024-04-24T23:26:14.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.268+0000 7f003d8a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00380b4310 0x7f00381f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.268+0000 7f003d8a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 0x7f00380d3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.268+0000 7f003d8a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 0x7f00380d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.268+0000 7f003d8a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f00380b2e70 con 0x7f003815e3a0 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.268+0000 7f003d8a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f003815e780 con 0x7f00380b4310 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.268+0000 7f003d8a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f00380b2ff0 con 0x7f00380d2770 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 0x7f00380d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 0x7f00380d39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56066/0 (socket says 172.21.15.26:56066) 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00377fe640 1 -- 172.21.15.26:0/1118392544 learned_addr learned my addr 172.21.15.26:0/1118392544 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00367fc640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 0x7f00380d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f0036ffd640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00380b4310 0x7f00381f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00377fe640 1 -- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 msgr2=0x7f00380d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00377fe640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 0x7f00380d3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00377fe640 1 -- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00380b4310 msgr2=0x7f00381f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00377fe640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00380b4310 0x7f00381f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00377fe640 1 -- 172.21.15.26:0/1118392544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00380cbc20 con 0x7f003815e3a0 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00367fc640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 0x7f00380d3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f0036ffd640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00380b4310 0x7f00381f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f00377fe640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 0x7f00380d39a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f002800c7f0 tx=0x7f002800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f001ffff640 1 -- 172.21.15.26:0/1118392544 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028018070 con 0x7f003815e3a0 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f001ffff640 1 -- 172.21.15.26:0/1118392544 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002800a5e0 con 0x7f003815e3a0 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f001ffff640 1 -- 172.21.15.26:0/1118392544 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002800aa10 con 0x7f003815e3a0 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00380cbf00 con 0x7f003815e3a0 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.269+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00380cc4c0 con 0x7f003815e3a0 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.271+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00380681e0 con 0x7f003815e3a0 2024-04-24T23:26:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.271+0000 7f001ffff640 1 -- 172.21.15.26:0/1118392544 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f002801b070 con 0x7f003815e3a0 2024-04-24T23:26:14.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.271+0000 7f001ffff640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.272+0000 7f001ffff640 1 -- 172.21.15.26:0/1118392544 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f002808a1b0 con 0x7f003815e3a0 2024-04-24T23:26:14.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.272+0000 7f0036ffd640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.272+0000 7f0036ffd640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f002406be30 0x7f002406e2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0030034000 tx=0x7f003002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:14.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.276+0000 7f001ffff640 1 -- 172.21.15.26:0/1118392544 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00380d3130 con 0x7f003815e3a0 2024-04-24T23:26:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.529+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f00380c7740 con 0x7f003815e3a0 2024-04-24T23:26:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.530+0000 7f001ffff640 1 -- 172.21.15.26:0/1118392544 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f0028058e10 con 0x7f003815e3a0 2024-04-24T23:26:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:14.531 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-04-24T23:26:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-04-24T23:26:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmg5ZUFBAAkapM9La3f+wSyLczFkm/0w==", 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPS3cHxAASacBQN+95Ds6N5668Je+DA==", 2024-04-24T23:26:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmStpLGxAAz7molt8venundBs+CcFSbg==", 2024-04-24T23:26:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylml0TBJhAAXQkjExbw1EXc90XlcAEBBw==", 2024-04-24T23:26:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjkylmQt74DBAAZLk9JTH3qTKRouUawqNURw==", 2024-04-24T23:26:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCmkylmi6IJCxAAWBnNl2y0XIUW/YoAIt6VIQ==", 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:26:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCpkylmRcgOCRAA79pdcH0OlB2VROLSRE4dcw==", 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-04-24T23:26:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCskylm8SopBxAArvUph95+zD5/0LmNM8WPIA==", 2024-04-24T23:26:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:26:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:26:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-04-24T23:26:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCwkylmbHipDxAAVUI9OpiMFiM/HFo03fEzSA==", 2024-04-24T23:26:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:26:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:26:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCzkylmDm4CDhAA1T/X+ywXEUCL96OGyi6S9Q==", 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.540 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.541 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-04-24T23:26:14.541 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC2kylmwWyVDBAAJsFFLltURqVqGLNjK9OCJQ==", 2024-04-24T23:26:14.541 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.541 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:26:14.541 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5kylm/nLXChAA6kk0UI2hchkN2lhPO1sXHw==", 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.542 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ==", 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-04-24T23:26:14.543 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylm/BVuMxAAAdEb6dd3Q5eMc+I3oszOyg==", 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmjnJGMBAAvwAmiHrkGkDFTeL/WZbdWw==", 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:26:14.544 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPh3qOBAAa4cbQeF96mdpgNVb/l3xPg==", 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCgkylm1ZfsMxAAExymnmhW44TcXzRblp95UA==", 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmq+8yERAAMKjK+5f2Qr35ral8L2qJMw==", 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.546 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmqYXWDBAAnuAn2h4Gb9qSFBoAZX73qQ==", 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-24T23:26:14.547 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f002406be30 msgr2=0x7f002406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f002406be30 0x7f002406e2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0030034000 tx=0x7f003002e000 comp rx=0 tx=0).stop 2024-04-24T23:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 msgr2=0x7f00380d39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 0x7f00380d39a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f002800c7f0 tx=0x7f002800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 shutdown_connections 2024-04-24T23:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f003815e3a0 0x7f00380d39a0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f00380d2770 0x7f00380d3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 --2- 172.21.15.26:0/1118392544 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f00380b4310 0x7f00381f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 >> 172.21.15.26:0/1118392544 conn(0x7f003808dbd0 msgr2=0x7f003815c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 shutdown_connections 2024-04-24T23:26:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.532+0000 7f003d8a5640 1 -- 172.21.15.26:0/1118392544 wait complete. 2024-04-24T23:26:14.550 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:14.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.650+0000 7fe97fe4d640 1 Processor -- start 2024-04-24T23:26:14.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.650+0000 7fe97fe4d640 1 -- start start 2024-04-24T23:26:14.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.651+0000 7fe97fe4d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 0x7fe9780ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.651+0000 7fe97fe4d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe9780addb0 0x7fe978154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.651+0000 7fe97fe4d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9780b0220 con 0x7fe97806f110 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.651+0000 7fe97fe4d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe9780ae220 con 0x7fe9780addb0 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.651+0000 7fe97fe4d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe9780af180 con 0x7fe97806f4f0 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.651+0000 7fe97e3ea640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe9780addb0 0x7fe978154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.651+0000 7fe97dbe9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe97806f110 0x7fe97815f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55804/0 (socket says 172.21.15.26:55804) 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.651+0000 7fe97dbe9640 1 -- 172.21.15.26:0/337743467 learned_addr learned my addr 172.21.15.26:0/337743467 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.652+0000 7fe97d3e8640 1 --2- 172.21.15.26:0/337743467 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 0x7fe9780ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.652+0000 7fe97e3ea640 1 -- 172.21.15.26:0/337743467 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 msgr2=0x7fe9780ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.652+0000 7fe97e3ea640 1 --2- 172.21.15.26:0/337743467 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 0x7fe9780ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.652+0000 7fe97cbe7640 1 -- 172.21.15.26:0/337743467 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 38878944 0 0) 0x7fe9780b0220 con 0x7fe97806f110 2024-04-24T23:26:14.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.652+0000 7fe97e3ea640 1 -- 172.21.15.26:0/337743467 >> v1:172.21.15.26:6789/0 conn(0x7fe97806f110 legacy=0x7fe97815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.652+0000 7fe97e3ea640 1 -- 172.21.15.26:0/337743467 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9780af460 con 0x7fe9780addb0 2024-04-24T23:26:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.652+0000 7fe97e3ea640 1 --2- 172.21.15.26:0/337743467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 0x7fe978154a50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe97402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ce31c571152adb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97cbe7640 1 -- 172.21.15.26:0/337743467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974007740 con 0x7fe9780addb0 2024-04-24T23:26:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97cbe7640 1 -- 172.21.15.26:0/337743467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9740078e0 con 0x7fe9780addb0 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97cbe7640 1 -- 172.21.15.26:0/337743467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974007cb0 con 0x7fe9780addb0 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97fe4d640 1 -- 172.21.15.26:0/337743467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 msgr2=0x7fe978154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97fe4d640 1 --2- 172.21.15.26:0/337743467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 0x7fe978154a50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe97402a660 comp rx=0 tx=0).stop 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97fe4d640 1 -- 172.21.15.26:0/337743467 shutdown_connections 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97fe4d640 1 --2- 172.21.15.26:0/337743467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 0x7fe978154a50 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97fe4d640 1 --2- 172.21.15.26:0/337743467 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 0x7fe9780ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97fe4d640 1 -- 172.21.15.26:0/337743467 >> 172.21.15.26:0/337743467 conn(0x7fe9780729a0 msgr2=0x7fe97806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97fe4d640 1 -- 172.21.15.26:0/337743467 shutdown_connections 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.653+0000 7fe97fe4d640 1 -- 172.21.15.26:0/337743467 wait complete. 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.654+0000 7fe97fe4d640 1 Processor -- start 2024-04-24T23:26:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.654+0000 7fe97fe4d640 1 -- start start 2024-04-24T23:26:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97fe4d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe97806f110 0x7fe978155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97fe4d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 0x7fe97814d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97fe4d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 0x7fe9781f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97fe4d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe9780afd00 con 0x7fe9780addb0 2024-04-24T23:26:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97fe4d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe9780afa20 con 0x7fe97806f110 2024-04-24T23:26:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97fe4d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe9780aee50 con 0x7fe97806f4f0 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97e3ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 0x7fe9781f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97e3ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 0x7fe9781f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56088/0 (socket says 172.21.15.26:56088) 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97e3ea640 1 -- 172.21.15.26:0/958201313 learned_addr learned my addr 172.21.15.26:0/958201313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97d3e8640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 0x7fe97814d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97e3ea640 1 -- 172.21.15.26:0/958201313 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 msgr2=0x7fe97814d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97e3ea640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 0x7fe97814d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97e3ea640 1 -- 172.21.15.26:0/958201313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe97806f110 msgr2=0x7fe978155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.655+0000 7fe97e3ea640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe97806f110 0x7fe978155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.656+0000 7fe97e3ea640 1 -- 172.21.15.26:0/958201313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe978156010 con 0x7fe9780addb0 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.656+0000 7fe97e3ea640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 0x7fe9781f3c60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.656+0000 7fe96e7fc640 1 -- 172.21.15.26:0/958201313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97403f070 con 0x7fe9780addb0 2024-04-24T23:26:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.656+0000 7fe96e7fc640 1 -- 172.21.15.26:0/958201313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe974031c90 con 0x7fe9780addb0 2024-04-24T23:26:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.656+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9781f47b0 con 0x7fe9780addb0 2024-04-24T23:26:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.656+0000 7fe96e7fc640 1 -- 172.21.15.26:0/958201313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974043630 con 0x7fe9780addb0 2024-04-24T23:26:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.656+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9781552d0 con 0x7fe9780addb0 2024-04-24T23:26:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.657+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9781550c0 con 0x7fe9780addb0 2024-04-24T23:26:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.657+0000 7fe96e7fc640 1 -- 172.21.15.26:0/958201313 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe97403b410 con 0x7fe9780addb0 2024-04-24T23:26:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.658+0000 7fe96e7fc640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe95c06be30 0x7fe95c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.658+0000 7fe96e7fc640 1 -- 172.21.15.26:0/958201313 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7fe9740ac100 con 0x7fe9780addb0 2024-04-24T23:26:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.658+0000 7fe97dbe9640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe95c06be30 0x7fe95c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.658+0000 7fe97dbe9640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe95c06be30 0x7fe95c06e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe970007fd0 tx=0x7fe970007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:14.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.662+0000 7fe96e7fc640 1 -- 172.21.15.26:0/958201313 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe97407aeb0 con 0x7fe9780addb0 2024-04-24T23:26:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.849+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9780694c0 con 0x7fe9780addb0 2024-04-24T23:26:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.850+0000 7fe96e7fc640 1 -- 172.21.15.26:0/958201313 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+12832 (secure 0 0 0) 0x7fe97407a7b0 con 0x7fe9780addb0 2024-04-24T23:26:14.851 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:14.851 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":78,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:26:14.144664+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe95c06be30 msgr2=0x7fe95c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe95c06be30 0x7fe95c06e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe970007fd0 tx=0x7fe970007920 comp rx=0 tx=0).stop 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 msgr2=0x7fe9781f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 0x7fe9781f3c60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe974004a40 comp rx=0 tx=0).stop 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 shutdown_connections 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe95c06be30 0x7fe95c06e2f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9780addb0 0x7fe9781f3c60 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe97806f4f0 0x7fe97814d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 --2- 172.21.15.26:0/958201313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe97806f110 0x7fe978155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 >> 172.21.15.26:0/958201313 conn(0x7fe9780729a0 msgr2=0x7fe9780b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 shutdown_connections 2024-04-24T23:26:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.852+0000 7fe97fe4d640 1 -- 172.21.15.26:0/958201313 wait complete. 2024-04-24T23:26:14.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:26:14.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:26:14.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.967+0000 7fbaaadf9640 1 Processor -- start 2024-04-24T23:26:14.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.968+0000 7fbaaadf9640 1 -- start start 2024-04-24T23:26:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.968+0000 7fbaaadf9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbaa415be40 0x7fbaa40acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.968+0000 7fbaaadf9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa40b4ce0 0x7fbaa40d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.968+0000 7fbaaadf9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa40aeb10 con 0x7fbaa40d0580 2024-04-24T23:26:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.968+0000 7fbaaadf9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa409df20 con 0x7fbaa415be40 2024-04-24T23:26:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.968+0000 7fbaaadf9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa40d09e0 con 0x7fbaa40b4ce0 2024-04-24T23:26:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa3fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa40b4ce0 0x7fbaa40d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa8b95640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbaa415be40 0x7fbaa40acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa3fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa40b4ce0 0x7fbaa40d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:40622/0 (socket says 172.21.15.26:40622) 2024-04-24T23:26:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa3fff640 1 -- 172.21.15.26:0/2621981433 learned_addr learned my addr 172.21.15.26:0/2621981433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa8b95640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbaa415be40 0x7fbaa40acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:42610/0 (socket says 172.21.15.26:42610) 2024-04-24T23:26:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa3fff640 1 -- 172.21.15.26:0/2621981433 >> v2:172.21.15.29:3300/0 conn(0x7fbaa415be40 msgr2=0x7fbaa40acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa3fff640 1 --2- 172.21.15.26:0/2621981433 >> v2:172.21.15.29:3300/0 conn(0x7fbaa415be40 0x7fbaa40acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa3fff640 1 -- 172.21.15.26:0/2621981433 >> v1:172.21.15.29:6789/0 conn(0x7fbaa40d0580 legacy=0x7fbaa40b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa3fff640 1 -- 172.21.15.26:0/2621981433 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa40aedf0 con 0x7fbaa40b4ce0 2024-04-24T23:26:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.969+0000 7fbaa8b95640 1 --2- 172.21.15.26:0/2621981433 >> v2:172.21.15.29:3300/0 conn(0x7fbaa415be40 0x7fbaa40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:14.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.970+0000 7fbaa3fff640 1 --2- 172.21.15.26:0/2621981433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa40b4ce0 0x7fbaa40d0040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba90009870 tx=0x7fba9002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c55f41364bead68d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:14.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.970+0000 7fbaa37fe640 1 -- 172.21.15.26:0/2621981433 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90037070 con 0x7fbaa40b4ce0 2024-04-24T23:26:14.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.970+0000 7fbaa37fe640 1 -- 172.21.15.26:0/2621981433 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba90032440 con 0x7fbaa40b4ce0 2024-04-24T23:26:14.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.970+0000 7fbaa37fe640 1 -- 172.21.15.26:0/2621981433 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90032810 con 0x7fbaa40b4ce0 2024-04-24T23:26:14.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.971+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2621981433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa40b4ce0 msgr2=0x7fbaa40d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.971+0000 7fbaaadf9640 1 --2- 172.21.15.26:0/2621981433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa40b4ce0 0x7fbaa40d0040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba90009870 tx=0x7fba9002a270 comp rx=0 tx=0).stop 2024-04-24T23:26:14.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.971+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2621981433 shutdown_connections 2024-04-24T23:26:14.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.971+0000 7fbaaadf9640 1 --2- 172.21.15.26:0/2621981433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa40b4ce0 0x7fbaa40d0040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.971+0000 7fbaaadf9640 1 --2- 172.21.15.26:0/2621981433 >> v2:172.21.15.29:3300/0 conn(0x7fbaa415be40 0x7fbaa40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.971+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2621981433 >> 172.21.15.26:0/2621981433 conn(0x7fbaa406b540 msgr2=0x7fbaa406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:14.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.971+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2621981433 shutdown_connections 2024-04-24T23:26:14.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.971+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2621981433 wait complete. 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.972+0000 7fbaaadf9640 1 Processor -- start 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.972+0000 7fbaaadf9640 1 -- start start 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaaadf9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbaa40b4ce0 0x7fbaa4152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaaadf9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaa40d0580 0x7fbaa4147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaaadf9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa415be40 0x7fbaa4147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaaadf9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa406cb90 con 0x7fbaa40d0580 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaaadf9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40ae800 con 0x7fbaa40b4ce0 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaaadf9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa406cd10 con 0x7fbaa415be40 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaa40d0580 0x7fbaa4147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaa40d0580 0x7fbaa4147310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56100/0 (socket says 172.21.15.26:56100) 2024-04-24T23:26:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa3fff640 1 -- 172.21.15.26:0/2061904889 learned_addr learned my addr 172.21.15.26:0/2061904889 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa8b95640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbaa40b4ce0 0x7fbaa4152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa9396640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa415be40 0x7fbaa4147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa3fff640 1 -- 172.21.15.26:0/2061904889 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa415be40 msgr2=0x7fbaa4147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa3fff640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa415be40 0x7fbaa4147850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa3fff640 1 -- 172.21.15.26:0/2061904889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbaa40b4ce0 msgr2=0x7fbaa4152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa3fff640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbaa40b4ce0 0x7fbaa4152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa3fff640 1 -- 172.21.15.26:0/2061904889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa409b730 con 0x7fbaa40d0580 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.973+0000 7fbaa3fff640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaa40d0580 0x7fbaa4147310 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fba90000c00 tx=0x7fba900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.974+0000 7fbaa17fa640 1 -- 172.21.15.26:0/2061904889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90037040 con 0x7fbaa40d0580 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.974+0000 7fbaa17fa640 1 -- 172.21.15.26:0/2061904889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba90042070 con 0x7fbaa40d0580 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.974+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa409ba10 con 0x7fbaa40d0580 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.974+0000 7fbaa17fa640 1 -- 172.21.15.26:0/2061904889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90030860 con 0x7fbaa40d0580 2024-04-24T23:26:14.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.974+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa409bfd0 con 0x7fbaa40d0580 2024-04-24T23:26:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.975+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa4075610 con 0x7fbaa40d0580 2024-04-24T23:26:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.975+0000 7fbaa17fa640 1 -- 172.21.15.26:0/2061904889 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba90030a00 con 0x7fbaa40d0580 2024-04-24T23:26:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.976+0000 7fbaa17fa640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba8806be60 0x7fba8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.976+0000 7fbaa8b95640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba8806be60 0x7fba8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.976+0000 7fbaa17fa640 1 -- 172.21.15.26:0/2061904889 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7fba900ab1e0 con 0x7fbaa40d0580 2024-04-24T23:26:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.976+0000 7fbaa8b95640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba8806be60 0x7fba8806e320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fba98009560 tx=0x7fba98009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:14.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:14.980+0000 7fbaa17fa640 1 -- 172.21.15.26:0/2061904889 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba90079890 con 0x7fbaa40d0580 2024-04-24T23:26:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.166+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbaa406dac0 con 0x7fbaa40d0580 2024-04-24T23:26:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.166+0000 7fbaa17fa640 1 -- 172.21.15.26:0/2061904889 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fba9007db90 con 0x7fbaa40d0580 2024-04-24T23:26:15.168 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.168+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba8806be60 msgr2=0x7fba8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.168+0000 7fbaaadf9640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba8806be60 0x7fba8806e320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fba98009560 tx=0x7fba98009190 comp rx=0 tx=0).stop 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.168+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaa40d0580 msgr2=0x7fbaa4147310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.168+0000 7fbaaadf9640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaa40d0580 0x7fbaa4147310 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fba90000c00 tx=0x7fba900046c0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.169+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 shutdown_connections 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.169+0000 7fbaaadf9640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba8806be60 0x7fba8806e320 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.169+0000 7fbaaadf9640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbaa415be40 0x7fbaa4147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.169+0000 7fbaaadf9640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaa40d0580 0x7fbaa4147310 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.169+0000 7fbaaadf9640 1 --2- 172.21.15.26:0/2061904889 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbaa40b4ce0 0x7fbaa4152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.169+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 >> 172.21.15.26:0/2061904889 conn(0x7fbaa406b540 msgr2=0x7fbaa415ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.169+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 shutdown_connections 2024-04-24T23:26:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.169+0000 7fbaaadf9640 1 -- 172.21.15.26:0/2061904889 wait complete. 2024-04-24T23:26:15.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:26:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.280+0000 7f0fd2a9a640 1 Processor -- start 2024-04-24T23:26:15.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.280+0000 7f0fd2a9a640 1 -- start start 2024-04-24T23:26:15.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.281+0000 7f0fd2a9a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc0d4870 0x7f0fcc0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.281+0000 7f0fd2a9a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fcc0d32f0 con 0x7f0fcc15e570 2024-04-24T23:26:15.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.281+0000 7f0fd2a9a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fcc0d0270 con 0x7f0fcc0b3130 2024-04-24T23:26:15.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.281+0000 7f0fd2a9a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0fcc0d1ab0 con 0x7f0fcc0d4870 2024-04-24T23:26:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.281+0000 7f0fd0836640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0fcc15e570 0x7f0fcc0d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55818/0 (socket says 172.21.15.26:55818) 2024-04-24T23:26:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.281+0000 7f0fd0836640 1 -- 172.21.15.26:0/417899716 learned_addr learned my addr 172.21.15.26:0/417899716 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 187586112 0 0) 0x7f0fbc001110 con 0x7f0fcc15e570 2024-04-24T23:26:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fb4003520 con 0x7f0fcc15e570 2024-04-24T23:26:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcbfff640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc0d4870 0x7f0fcc0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 52938763 0 0) 0x7f0fb4003520 con 0x7f0fcc15e570 2024-04-24T23:26:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc0d4870 msgr2=0x7f0fcc0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc0d4870 0x7f0fcc0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 >> v1:172.21.15.29:6789/0 conn(0x7f0fcc0b3130 legacy=0x7f0fcc0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc0d2d30 con 0x7f0fcc15e570 2024-04-24T23:26:15.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f0fbc001840 con 0x7f0fcc15e570 2024-04-24T23:26:15.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 >> v1:172.21.15.26:6789/0 conn(0x7f0fcc15e570 legacy=0x7f0fcc0d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb40039e0 0x7f0fb4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4004800 0x7f0fb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc0d4870 0x7f0fb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc0d1ab0 con 0x7f0fb4004800 2024-04-24T23:26:15.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb400ba70 con 0x7f0fb40039e0 2024-04-24T23:26:15.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb400bc70 con 0x7f0fcc0d4870 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcbfff640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4004800 0x7f0fb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fd0836640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb40039e0 0x7f0fb4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fd1037640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc0d4870 0x7f0fb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcbfff640 1 -- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc0d4870 msgr2=0x7f0fb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcbfff640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc0d4870 0x7f0fb40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcbfff640 1 -- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb40039e0 msgr2=0x7f0fb4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcbfff640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb40039e0 0x7f0fb4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.282+0000 7f0fcbfff640 1 -- 172.21.15.26:0/417899716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb4003520 con 0x7f0fb4004800 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.283+0000 7f0fcbfff640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4004800 0x7f0fb4004c80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009a90 tx=0x7f0fb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16275e160c86ccbc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.283+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0036070 con 0x7f0fb4004800 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.283+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc0031400 con 0x7f0fb4004800 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.283+0000 7f0fcb7fe640 1 -- 172.21.15.26:0/417899716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0009690 con 0x7f0fb4004800 2024-04-24T23:26:15.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.284+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/417899716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4004800 msgr2=0x7f0fb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.284+0000 7f0fd2a9a640 1 --2- 172.21.15.26:0/417899716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4004800 0x7f0fb4004c80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009a90 tx=0x7f0fb4005830 comp rx=0 tx=0).stop 2024-04-24T23:26:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.284+0000 7f0fd1037640 1 -- 172.21.15.26:0/417899716 reap_dead start 2024-04-24T23:26:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.284+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/417899716 shutdown_connections 2024-04-24T23:26:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.284+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/417899716 >> 172.21.15.26:0/417899716 conn(0x7f0fcc08dbe0 msgr2=0x7f0fcc09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.284+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/417899716 shutdown_connections 2024-04-24T23:26:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.284+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/417899716 wait complete. 2024-04-24T23:26:15.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.285+0000 7f0fd2a9a640 1 Processor -- start 2024-04-24T23:26:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.285+0000 7f0fd2a9a640 1 -- start start 2024-04-24T23:26:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd2a9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb4004800 0x7f0fcc14c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd2a9a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc15bd20 0x7f0fcc15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd2a9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fcc09df70 0x7f0fcc09e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd2a9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc0cfa70 con 0x7f0fcc09df70 2024-04-24T23:26:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd2a9a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc0d2ae0 con 0x7f0fb4004800 2024-04-24T23:26:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd2a9a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0fcc0cfd50 con 0x7f0fcc15bd20 2024-04-24T23:26:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd1037640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fcc09df70 0x7f0fcc09e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd1037640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fcc09df70 0x7f0fcc09e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56118/0 (socket says 172.21.15.26:56118) 2024-04-24T23:26:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd1037640 1 -- 172.21.15.26:0/42436329 learned_addr learned my addr 172.21.15.26:0/42436329 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd0836640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb4004800 0x7f0fcc14c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fcbfff640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc15bd20 0x7f0fcc15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd1037640 1 -- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc15bd20 msgr2=0x7f0fcc15c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd1037640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc15bd20 0x7f0fcc15c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd1037640 1 -- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb4004800 msgr2=0x7f0fcc14c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd1037640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb4004800 0x7f0fcc14c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd1037640 1 -- 172.21.15.26:0/42436329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc15ac80 con 0x7f0fcc09df70 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fd0836640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb4004800 0x7f0fcc14c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.286+0000 7f0fcbfff640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc15bd20 0x7f0fcc15c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.287+0000 7f0fd1037640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fcc09df70 0x7f0fcc09e3f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc08e4a0 tx=0x7f0fc400fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.287+0000 7f0fc97fa640 1 -- 172.21.15.26:0/42436329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc401a070 con 0x7f0fcc09df70 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.287+0000 7f0fc97fa640 1 -- 172.21.15.26:0/42436329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc4006d60 con 0x7f0fcc09df70 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.287+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fcc15af60 con 0x7f0fcc09df70 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.287+0000 7f0fc97fa640 1 -- 172.21.15.26:0/42436329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4003b90 con 0x7f0fcc09df70 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.287+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fcc15b520 con 0x7f0fcc09df70 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.288+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fcc0694c0 con 0x7f0fcc09df70 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.288+0000 7f0fc97fa640 1 -- 172.21.15.26:0/42436329 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fc4005070 con 0x7f0fcc09df70 2024-04-24T23:26:15.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.289+0000 7f0fc97fa640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0fb806be60 0x7f0fb806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.289+0000 7f0fc97fa640 1 -- 172.21.15.26:0/42436329 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f0fc4089ff0 con 0x7f0fcc09df70 2024-04-24T23:26:15.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.289+0000 7f0fd0836640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0fb806be60 0x7f0fb806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.289+0000 7f0fd0836640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0fb806be60 0x7f0fb806e320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0016c0 tx=0x7f0fbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:15.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.293+0000 7f0fc97fa640 1 -- 172.21.15.26:0/42436329 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0fc4058c90 con 0x7f0fcc09df70 2024-04-24T23:26:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.480+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0fcc0b5320 con 0x7f0fcc09df70 2024-04-24T23:26:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.480+0000 7f0fc97fa640 1 -- 172.21.15.26:0/42436329 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7f0fc4058590 con 0x7f0fcc09df70 2024-04-24T23:26:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:15.482 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:26:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:26:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-24T23:26:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-24T23:26:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:15.482 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:26:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0fb806be60 msgr2=0x7f0fb806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0fb806be60 0x7f0fb806e320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0016c0 tx=0x7f0fbc009040 comp rx=0 tx=0).stop 2024-04-24T23:26:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fcc09df70 msgr2=0x7f0fcc09e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fcc09df70 0x7f0fcc09e3f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc08e4a0 tx=0x7f0fc400fd40 comp rx=0 tx=0).stop 2024-04-24T23:26:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 shutdown_connections 2024-04-24T23:26:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0fb806be60 0x7f0fb806e320 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fcc09df70 0x7f0fcc09e3f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0fcc15bd20 0x7f0fcc15c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 --2- 172.21.15.26:0/42436329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0fb4004800 0x7f0fcc14c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.482+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 >> 172.21.15.26:0/42436329 conn(0x7f0fcc08dbe0 msgr2=0x7f0fcc0b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.483+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 shutdown_connections 2024-04-24T23:26:15.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.483+0000 7f0fd2a9a640 1 -- 172.21.15.26:0/42436329 wait complete. 2024-04-24T23:26:15.494 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:26:15.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:26:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.598+0000 7ff45ad09640 1 Processor -- start 2024-04-24T23:26:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.598+0000 7ff45ad09640 1 -- start start 2024-04-24T23:26:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff45ad09640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff454145650 0x7ff45406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff45ad09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff4540cde20 0x7ff45415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff45ad09640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45406ecc0 con 0x7ff4540b39e0 2024-04-24T23:26:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff45ad09640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff4540b3e70 con 0x7ff4540cde20 2024-04-24T23:26:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff45ad09640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff45406e420 con 0x7ff454145650 2024-04-24T23:26:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff4592a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff4540cde20 0x7ff45415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff44bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff4540b39e0 0x7ff4540cd710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55830/0 (socket says 172.21.15.26:55830) 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff44bfff640 1 -- 172.21.15.26:0/2544785264 learned_addr learned my addr 172.21.15.26:0/2544785264 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff458aa5640 1 --2- 172.21.15.26:0/2544785264 >> v2:172.21.15.29:3300/0 conn(0x7ff454145650 0x7ff45406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff4592a6640 1 -- 172.21.15.26:0/2544785264 >> v2:172.21.15.29:3300/0 conn(0x7ff454145650 msgr2=0x7ff45406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff4592a6640 1 --2- 172.21.15.26:0/2544785264 >> v2:172.21.15.29:3300/0 conn(0x7ff454145650 0x7ff45406dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff4592a6640 1 -- 172.21.15.26:0/2544785264 >> v1:172.21.15.26:6789/0 conn(0x7ff4540b39e0 legacy=0x7ff4540cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff4592a6640 1 -- 172.21.15.26:0/2544785264 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45406e700 con 0x7ff4540cde20 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff44b7fe640 1 -- 172.21.15.26:0/2544785264 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462836859 0 0) 0x7ff45406ecc0 con 0x7ff4540b39e0 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.599+0000 7ff458aa5640 1 --2- 172.21.15.26:0/2544785264 >> v2:172.21.15.29:3300/0 conn(0x7ff454145650 0x7ff45406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.600+0000 7ff4592a6640 1 --2- 172.21.15.26:0/2544785264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 0x7ff45415cc80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff45406f090 tx=0x7ff44c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17df445f2686fc3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.600+0000 7ff44b7fe640 1 -- 172.21.15.26:0/2544785264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c007740 con 0x7ff4540cde20 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.600+0000 7ff44b7fe640 1 -- 172.21.15.26:0/2544785264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff44c0078e0 con 0x7ff4540cde20 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.600+0000 7ff44b7fe640 1 -- 172.21.15.26:0/2544785264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c007cb0 con 0x7ff4540cde20 2024-04-24T23:26:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.601+0000 7ff45ad09640 1 -- 172.21.15.26:0/2544785264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 msgr2=0x7ff45415cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.601+0000 7ff45ad09640 1 --2- 172.21.15.26:0/2544785264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 0x7ff45415cc80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff45406f090 tx=0x7ff44c02a680 comp rx=0 tx=0).stop 2024-04-24T23:26:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.601+0000 7ff45ad09640 1 -- 172.21.15.26:0/2544785264 shutdown_connections 2024-04-24T23:26:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.601+0000 7ff45ad09640 1 --2- 172.21.15.26:0/2544785264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 0x7ff45415cc80 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.601+0000 7ff45ad09640 1 --2- 172.21.15.26:0/2544785264 >> v2:172.21.15.29:3300/0 conn(0x7ff454145650 0x7ff45406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.601+0000 7ff45ad09640 1 -- 172.21.15.26:0/2544785264 >> 172.21.15.26:0/2544785264 conn(0x7ff45406b300 msgr2=0x7ff45406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.601+0000 7ff45ad09640 1 -- 172.21.15.26:0/2544785264 shutdown_connections 2024-04-24T23:26:15.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.601+0000 7ff45ad09640 1 -- 172.21.15.26:0/2544785264 wait complete. 2024-04-24T23:26:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.602+0000 7ff45ad09640 1 Processor -- start 2024-04-24T23:26:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.602+0000 7ff45ad09640 1 -- start start 2024-04-24T23:26:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff45ad09640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4540b39e0 0x7ff4541f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff45ad09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 0x7ff4540a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff45ad09640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff454145650 0x7ff4540b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff45ad09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff454145a30 con 0x7ff4540cde20 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff45ad09640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff45415c760 con 0x7ff4540b39e0 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff45ad09640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff454145bb0 con 0x7ff454145650 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff44bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 0x7ff4540a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff44bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 0x7ff4540a9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56140/0 (socket says 172.21.15.26:56140) 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff44bfff640 1 -- 172.21.15.26:0/21977667 learned_addr learned my addr 172.21.15.26:0/21977667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff458aa5640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4540b39e0 0x7ff4541f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff44bfff640 1 -- 172.21.15.26:0/21977667 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff454145650 msgr2=0x7ff4540b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff44bfff640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff454145650 0x7ff4540b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff44bfff640 1 -- 172.21.15.26:0/21977667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4540b39e0 msgr2=0x7ff4541f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff44bfff640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4540b39e0 0x7ff4541f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:15.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff44bfff640 1 -- 172.21.15.26:0/21977667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4541f1930 con 0x7ff4540cde20 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.603+0000 7ff44bfff640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 0x7ff4540a9700 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff44400c4d0 tx=0x7ff44400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.604+0000 7ff4497fa640 1 -- 172.21.15.26:0/21977667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444018070 con 0x7ff4540cde20 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.604+0000 7ff4497fa640 1 -- 172.21.15.26:0/21977667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff444007500 con 0x7ff4540cde20 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.604+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4541f1c50 con 0x7ff4540cde20 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.604+0000 7ff4497fa640 1 -- 172.21.15.26:0/21977667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444004a20 con 0x7ff4540cde20 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.604+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4541f2490 con 0x7ff4540cde20 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.605+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4540694c0 con 0x7ff4540cde20 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.605+0000 7ff4497fa640 1 -- 172.21.15.26:0/21977667 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff44401b070 con 0x7ff4540cde20 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.606+0000 7ff4497fa640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff44006be30 0x7ff44006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.606+0000 7ff458aa5640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff44006be30 0x7ff44006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.606+0000 7ff4497fa640 1 -- 172.21.15.26:0/21977667 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7ff444089e40 con 0x7ff4540cde20 2024-04-24T23:26:15.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.606+0000 7ff458aa5640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff44006be30 0x7ff44006e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff45000b840 tx=0x7ff450008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:15.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.610+0000 7ff4497fa640 1 -- 172.21.15.26:0/21977667 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff4541f1c50 con 0x7ff4540cde20 2024-04-24T23:26:15.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:15.792+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff4540b1720 con 0x7ff4540cde20 2024-04-24T23:26:16.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.142+0000 7ff4497fa640 1 -- 172.21.15.26:0/21977667 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7ff444058b40 con 0x7ff4540cde20 2024-04-24T23:26:16.143 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-24T23:26:16.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff44006be30 msgr2=0x7ff44006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:16.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff44006be30 0x7ff44006e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff45000b840 tx=0x7ff450008040 comp rx=0 tx=0).stop 2024-04-24T23:26:16.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 msgr2=0x7ff4540a9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 0x7ff4540a9700 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff44400c4d0 tx=0x7ff44400c990 comp rx=0 tx=0).stop 2024-04-24T23:26:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 shutdown_connections 2024-04-24T23:26:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff44006be30 0x7ff44006e2f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff454145650 0x7ff4540b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4540cde20 0x7ff4540a9700 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 --2- 172.21.15.26:0/21977667 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4540b39e0 0x7ff4541f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 >> 172.21.15.26:0/21977667 conn(0x7ff45406b300 msgr2=0x7ff4540ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.144+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 shutdown_connections 2024-04-24T23:26:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.145+0000 7ff45ad09640 1 -- 172.21.15.26:0/21977667 wait complete. 2024-04-24T23:26:16.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:26:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.271+0000 7fdf95eae640 1 Processor -- start 2024-04-24T23:26:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.271+0000 7fdf95eae640 1 -- start start 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf95eae640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 0x7fdf9006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf95eae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf9015e500 0x7fdf900b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf95eae640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdf900b3d20 0x7fdf90150e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf95eae640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fdf90070f30 con 0x7fdf9015e500 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf95eae640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9006dcf0 con 0x7fdf900b3d20 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf95eae640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9006dfd0 con 0x7fdf9015e120 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf9015e500 0x7fdf900b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf9015e500 0x7fdf900b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56142/0 (socket says 172.21.15.26:56142) 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8effd640 1 -- 172.21.15.26:0/4247496072 learned_addr learned my addr 172.21.15.26:0/4247496072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8f7fe640 1 --2- 172.21.15.26:0/4247496072 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 0x7fdf9006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8ffff640 1 --2- 172.21.15.26:0/4247496072 >> v2:172.21.15.29:3300/0 conn(0x7fdf900b3d20 0x7fdf90150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8effd640 1 -- 172.21.15.26:0/4247496072 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 msgr2=0x7fdf9006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8effd640 1 --2- 172.21.15.26:0/4247496072 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 0x7fdf9006d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8effd640 1 -- 172.21.15.26:0/4247496072 >> v2:172.21.15.29:3300/0 conn(0x7fdf900b3d20 msgr2=0x7fdf90150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8effd640 1 --2- 172.21.15.26:0/4247496072 >> v2:172.21.15.29:3300/0 conn(0x7fdf900b3d20 0x7fdf90150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.272+0000 7fdf8effd640 1 -- 172.21.15.26:0/4247496072 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9006e2b0 con 0x7fdf9015e500 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.273+0000 7fdf8effd640 1 --2- 172.21.15.26:0/4247496072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9015e500 0x7fdf900b37e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdf900cc060 tx=0x7fdf8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886d6b116d6d3f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.273+0000 7fdf8e7fc640 1 -- 172.21.15.26:0/4247496072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84037070 con 0x7fdf9015e500 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.273+0000 7fdf8e7fc640 1 -- 172.21.15.26:0/4247496072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf84032440 con 0x7fdf9015e500 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.273+0000 7fdf8e7fc640 1 -- 172.21.15.26:0/4247496072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84032810 con 0x7fdf9015e500 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.274+0000 7fdf95eae640 1 -- 172.21.15.26:0/4247496072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9015e500 msgr2=0x7fdf900b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.274+0000 7fdf95eae640 1 --2- 172.21.15.26:0/4247496072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9015e500 0x7fdf900b37e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdf900cc060 tx=0x7fdf8402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.274+0000 7fdf95eae640 1 -- 172.21.15.26:0/4247496072 shutdown_connections 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.274+0000 7fdf95eae640 1 --2- 172.21.15.26:0/4247496072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9015e500 0x7fdf900b37e0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.274+0000 7fdf95eae640 1 --2- 172.21.15.26:0/4247496072 >> v2:172.21.15.29:3300/0 conn(0x7fdf900b3d20 0x7fdf90150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.274+0000 7fdf95eae640 1 --2- 172.21.15.26:0/4247496072 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 0x7fdf9006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.274+0000 7fdf95eae640 1 -- 172.21.15.26:0/4247496072 >> 172.21.15.26:0/4247496072 conn(0x7fdf9006b540 msgr2=0x7fdf9006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.274+0000 7fdf95eae640 1 -- 172.21.15.26:0/4247496072 shutdown_connections 2024-04-24T23:26:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.274+0000 7fdf95eae640 1 -- 172.21.15.26:0/4247496072 wait complete. 2024-04-24T23:26:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.275+0000 7fdf95eae640 1 Processor -- start 2024-04-24T23:26:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.275+0000 7fdf95eae640 1 -- start start 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf95eae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900b3d20 0x7fdf9009c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf95eae640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 0x7fdf9015d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf95eae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf9015e500 0x7fdf9014b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf95eae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9015e940 con 0x7fdf900b3d20 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf95eae640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdf900d0040 con 0x7fdf9015e500 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf95eae640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9009ca70 con 0x7fdf9015e120 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900b3d20 0x7fdf9009c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900b3d20 0x7fdf9009c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56152/0 (socket says 172.21.15.26:56152) 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8f7fe640 1 -- 172.21.15.26:0/2063421421 learned_addr learned my addr 172.21.15.26:0/2063421421 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8ffff640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf9015e500 0x7fdf9014b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8effd640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 0x7fdf9015d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8f7fe640 1 -- 172.21.15.26:0/2063421421 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 msgr2=0x7fdf9015d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8f7fe640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 0x7fdf9015d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8f7fe640 1 -- 172.21.15.26:0/2063421421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf9015e500 msgr2=0x7fdf9014b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8f7fe640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf9015e500 0x7fdf9014b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8f7fe640 1 -- 172.21.15.26:0/2063421421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf90159600 con 0x7fdf900b3d20 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.276+0000 7fdf8ffff640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf9015e500 0x7fdf9014b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.277+0000 7fdf8f7fe640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900b3d20 0x7fdf9009c530 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000d800 tx=0x7fdf8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.277+0000 7fdf77fff640 1 -- 172.21.15.26:0/2063421421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8000b660 con 0x7fdf900b3d20 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.277+0000 7fdf77fff640 1 -- 172.21.15.26:0/2063421421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf80019070 con 0x7fdf900b3d20 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.277+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf901598e0 con 0x7fdf900b3d20 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.277+0000 7fdf77fff640 1 -- 172.21.15.26:0/2063421421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8000ba20 con 0x7fdf900b3d20 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.277+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf90159ea0 con 0x7fdf900b3d20 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.278+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf90075610 con 0x7fdf900b3d20 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.278+0000 7fdf77fff640 1 -- 172.21.15.26:0/2063421421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf8000bbc0 con 0x7fdf900b3d20 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.279+0000 7fdf77fff640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf7c06be60 0x7fdf7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.279+0000 7fdf77fff640 1 -- 172.21.15.26:0/2063421421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5227+0+0 (secure 0 0 0) 0x7fdf8008aea0 con 0x7fdf900b3d20 2024-04-24T23:26:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.279+0000 7fdf8effd640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf7c06be60 0x7fdf7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.279+0000 7fdf8effd640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf7c06be60 0x7fdf7c06e320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdf84034000 tx=0x7fdf8402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:16.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.284+0000 7fdf77fff640 1 -- 172.21.15.26:0/2063421421 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdf80059ac0 con 0x7fdf900b3d20 2024-04-24T23:26:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:16.466+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdf90074eb0 con 0x7fdf900b3d20 2024-04-24T23:26:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.144+0000 7fdf77fff640 1 -- 172.21.15.26:0/2063421421 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7fdf800593c0 con 0x7fdf900b3d20 2024-04-24T23:26:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-24T23:26:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf7c06be60 msgr2=0x7fdf7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf7c06be60 0x7fdf7c06e320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdf84034000 tx=0x7fdf8402e000 comp rx=0 tx=0).stop 2024-04-24T23:26:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900b3d20 msgr2=0x7fdf9009c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900b3d20 0x7fdf9009c530 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000d800 tx=0x7fdf8000dcc0 comp rx=0 tx=0).stop 2024-04-24T23:26:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 shutdown_connections 2024-04-24T23:26:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdf7c06be60 0x7fdf7c06e320 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdf9015e500 0x7fdf9014b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdf9015e120 0x7fdf9015d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 --2- 172.21.15.26:0/2063421421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900b3d20 0x7fdf9009c530 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 >> 172.21.15.26:0/2063421421 conn(0x7fdf9006b540 msgr2=0x7fdf90152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 shutdown_connections 2024-04-24T23:26:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.147+0000 7fdf95eae640 1 -- 172.21.15.26:0/2063421421 wait complete. 2024-04-24T23:26:17.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:26:17.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.283+0000 7f3fb0a27640 1 Processor -- start 2024-04-24T23:26:17.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.284+0000 7f3fb0a27640 1 -- start start 2024-04-24T23:26:17.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.284+0000 7f3fb0a27640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fac0ad7b0 0x7f3fac0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:17.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.284+0000 7f3fb0a27640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fac15ef60 con 0x7f3fac15e9c0 2024-04-24T23:26:17.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.284+0000 7f3fb0a27640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fac15f240 con 0x7f3fac15adc0 2024-04-24T23:26:17.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.284+0000 7f3fb0a27640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3fac15f520 con 0x7f3fac0ad7b0 2024-04-24T23:26:17.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.285+0000 7f3fa3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fac0ad7b0 0x7f3fac0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:17.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.285+0000 7f3faad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3fac15e9c0 0x7f3fac072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55834/0 (socket says 172.21.15.26:55834) 2024-04-24T23:26:17.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.285+0000 7f3faad9d640 1 -- 172.21.15.26:0/2624858349 learned_addr learned my addr 172.21.15.26:0/2624858349 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:17.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.285+0000 7f3fa3fff640 1 -- 172.21.15.26:0/2624858349 >> v1:172.21.15.29:6789/0 conn(0x7f3fac15adc0 legacy=0x7f3fac0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:17.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.285+0000 7f3fa3fff640 1 -- 172.21.15.26:0/2624858349 >> v1:172.21.15.26:6789/0 conn(0x7f3fac15e9c0 legacy=0x7f3fac072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:17.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.285+0000 7f3fa3fff640 1 -- 172.21.15.26:0/2624858349 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac15f800 con 0x7f3fac0ad7b0 2024-04-24T23:26:17.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.285+0000 7f3fa9d9b640 1 -- 172.21.15.26:0/2624858349 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237001534 0 0) 0x7f3fac15ef60 con 0x7f3fac15e9c0 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fa3fff640 1 --2- 172.21.15.26:0/2624858349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac0ad7b0 0x7f3fac0d0040 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3f940097d0 tx=0x7f3f9402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1adfe6e05a5971 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fa9d9b640 1 -- 172.21.15.26:0/2624858349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94037070 con 0x7f3fac0ad7b0 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fa9d9b640 1 -- 172.21.15.26:0/2624858349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9402ae10 con 0x7f3fac0ad7b0 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fa9d9b640 1 -- 172.21.15.26:0/2624858349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94032660 con 0x7f3fac0ad7b0 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fb0a27640 1 -- 172.21.15.26:0/2624858349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac0ad7b0 msgr2=0x7f3fac0d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fb0a27640 1 --2- 172.21.15.26:0/2624858349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac0ad7b0 0x7f3fac0d0040 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3f940097d0 tx=0x7f3f9402a100 comp rx=0 tx=0).stop 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fb0a27640 1 -- 172.21.15.26:0/2624858349 shutdown_connections 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fb0a27640 1 --2- 172.21.15.26:0/2624858349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac0ad7b0 0x7f3fac0d0040 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fb0a27640 1 -- 172.21.15.26:0/2624858349 >> 172.21.15.26:0/2624858349 conn(0x7f3fac06b540 msgr2=0x7f3fac06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fb0a27640 1 -- 172.21.15.26:0/2624858349 shutdown_connections 2024-04-24T23:26:17.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.286+0000 7f3fb0a27640 1 -- 172.21.15.26:0/2624858349 wait complete. 2024-04-24T23:26:17.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.288+0000 7f3fb0a27640 1 Processor -- start 2024-04-24T23:26:17.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.288+0000 7f3fb0a27640 1 -- start start 2024-04-24T23:26:17.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.288+0000 7f3fb0a27640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3fac0ad7b0 0x7f3fac09d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:17.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.288+0000 7f3fb0a27640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fac15adc0 0x7f3fac158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:17.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.288+0000 7f3fb0a27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac15e9c0 0x7f3fac159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:17.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.288+0000 7f3fb0a27640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac0d0580 con 0x7f3fac15e9c0 2024-04-24T23:26:17.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.288+0000 7f3fb0a27640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac15b1a0 con 0x7f3fac15adc0 2024-04-24T23:26:17.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.288+0000 7f3fb0a27640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3fac0d0700 con 0x7f3fac0ad7b0 2024-04-24T23:26:17.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac15e9c0 0x7f3fac159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac15e9c0 0x7f3fac159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56164/0 (socket says 172.21.15.26:56164) 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faad9d640 1 -- 172.21.15.26:0/4201368687 learned_addr learned my addr 172.21.15.26:0/4201368687 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3fa3fff640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fac15adc0 0x7f3fac158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faa59c640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3fac0ad7b0 0x7f3fac09d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faad9d640 1 -- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3fac0ad7b0 msgr2=0x7f3fac09d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faad9d640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3fac0ad7b0 0x7f3fac09d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faad9d640 1 -- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fac15adc0 msgr2=0x7f3fac158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faad9d640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fac15adc0 0x7f3fac158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faad9d640 1 -- 172.21.15.26:0/4201368687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac09b980 con 0x7f3fac15e9c0 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faa59c640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3fac0ad7b0 0x7f3fac09d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3fa3fff640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fac15adc0 0x7f3fac158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3faad9d640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac15e9c0 0x7f3fac159470 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400c700 tx=0x7f3fa400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3fa2ffd640 1 -- 172.21.15.26:0/4201368687 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4018070 con 0x7f3fac15e9c0 2024-04-24T23:26:17.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3fa2ffd640 1 -- 172.21.15.26:0/4201368687 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa400ce10 con 0x7f3fac15e9c0 2024-04-24T23:26:17.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fac09bc60 con 0x7f3fac15e9c0 2024-04-24T23:26:17.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3fa2ffd640 1 -- 172.21.15.26:0/4201368687 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400a860 con 0x7f3fac15e9c0 2024-04-24T23:26:17.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.289+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fac158730 con 0x7f3fac15e9c0 2024-04-24T23:26:17.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.290+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fac075610 con 0x7f3fac15e9c0 2024-04-24T23:26:17.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.291+0000 7f3fa2ffd640 1 -- 172.21.15.26:0/4201368687 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3fa401b070 con 0x7f3fac15e9c0 2024-04-24T23:26:17.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.291+0000 7f3fa2ffd640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f8c06be30 0x7f3f8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:17.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.291+0000 7f3fa2ffd640 1 -- 172.21.15.26:0/4201368687 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5609+0+0 (secure 0 0 0) 0x7f3fa408a390 con 0x7f3fac15e9c0 2024-04-24T23:26:17.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.291+0000 7f3faa59c640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f8c06be30 0x7f3f8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:17.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.292+0000 7f3faa59c640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f8c06be30 0x7f3f8c06e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3fac152490 tx=0x7f3f9c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:17.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.296+0000 7f3fa2ffd640 1 -- 172.21.15.26:0/4201368687 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3fa4058e30 con 0x7f3fac15e9c0 2024-04-24T23:26:17.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:17.521+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3fac15e730 con 0x7f3fac15e9c0 2024-04-24T23:26:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.161+0000 7f3fa2ffd640 1 -- 172.21.15.26:0/4201368687 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v80) ==== 174+0+0 (secure 0 0 0) 0x7f3fa4058730 con 0x7f3fac15e9c0 2024-04-24T23:26:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 12 and data pool 13 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.166+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f8c06be30 msgr2=0x7f3f8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.166+0000 7f3fb0a27640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f8c06be30 0x7f3f8c06e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3fac152490 tx=0x7f3f9c009040 comp rx=0 tx=0).stop 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.166+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac15e9c0 msgr2=0x7f3fac159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.167+0000 7f3fb0a27640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac15e9c0 0x7f3fac159470 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400c700 tx=0x7f3fa400cbc0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.167+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 shutdown_connections 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.167+0000 7f3fb0a27640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f8c06be30 0x7f3f8c06e2f0 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3fac152490 tx=0x7f3f9c009040 comp rx=0 tx=0).stop 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.167+0000 7f3fb0a27640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac15e9c0 0x7f3fac159470 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.167+0000 7f3fb0a27640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fac15adc0 0x7f3fac158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.167+0000 7f3fb0a27640 1 --2- 172.21.15.26:0/4201368687 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3fac0ad7b0 0x7f3fac09d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.167+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 >> 172.21.15.26:0/4201368687 conn(0x7f3fac06b540 msgr2=0x7f3fac0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.167+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 shutdown_connections 2024-04-24T23:26:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.168+0000 7f3fb0a27640 1 -- 172.21.15.26:0/4201368687 wait complete. 2024-04-24T23:26:18.199 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:18.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.302+0000 7f5da7577640 1 Processor -- start 2024-04-24T23:26:18.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.303+0000 7f5da7577640 1 -- start start 2024-04-24T23:26:18.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.303+0000 7f5da7577640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5da00d4870 0x7f5da00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.303+0000 7f5da7577640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5da00b3130 0x7f5da00b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.303+0000 7f5da7577640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da00d32f0 con 0x7f5da015e570 2024-04-24T23:26:18.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.303+0000 7f5da7577640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5da00d1ab0 con 0x7f5da00b3130 2024-04-24T23:26:18.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.303+0000 7f5da7577640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5da00d2ae0 con 0x7f5da00d4870 2024-04-24T23:26:18.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.303+0000 7f5da5313640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5da015e570 0x7f5da00d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55850/0 (socket says 172.21.15.26:55850) 2024-04-24T23:26:18.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.303+0000 7f5da5313640 1 -- 172.21.15.26:0/1505237818 learned_addr learned my addr 172.21.15.26:0/1505237818 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:18.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.304+0000 7f5da5b14640 1 --2- 172.21.15.26:0/1505237818 >> v2:172.21.15.26:3300/0 conn(0x7f5da00b3130 0x7f5da00b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.304+0000 7f5d9ffff640 1 -- 172.21.15.26:0/1505237818 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977017123 0 0) 0x7f5d90001490 con 0x7f5da015e570 2024-04-24T23:26:18.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.304+0000 7f5d9ffff640 1 -- 172.21.15.26:0/1505237818 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d88003520 con 0x7f5da015e570 2024-04-24T23:26:18.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.304+0000 7f5da4b12640 1 --2- 172.21.15.26:0/1505237818 >> v2:172.21.15.29:3300/0 conn(0x7f5da00d4870 0x7f5da00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.304+0000 7f5da5b14640 1 -- 172.21.15.26:0/1505237818 >> v2:172.21.15.29:3300/0 conn(0x7f5da00d4870 msgr2=0x7f5da00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.304+0000 7f5da5b14640 1 --2- 172.21.15.26:0/1505237818 >> v2:172.21.15.29:3300/0 conn(0x7f5da00d4870 0x7f5da00b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.304+0000 7f5da5b14640 1 -- 172.21.15.26:0/1505237818 >> v1:172.21.15.26:6789/0 conn(0x7f5da015e570 legacy=0x7f5da00d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.304+0000 7f5da5b14640 1 -- 172.21.15.26:0/1505237818 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da00d0270 con 0x7f5da00b3130 2024-04-24T23:26:18.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5da5b14640 1 --2- 172.21.15.26:0/1505237818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 0x7f5da00b4640 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f5da008cae0 tx=0x7f5d9802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a80b3234512fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5d9ffff640 1 -- 172.21.15.26:0/1505237818 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98037070 con 0x7f5da00b3130 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5d9ffff640 1 -- 172.21.15.26:0/1505237818 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d98032440 con 0x7f5da00b3130 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5d9ffff640 1 -- 172.21.15.26:0/1505237818 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9802ae60 con 0x7f5da00b3130 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5da7577640 1 -- 172.21.15.26:0/1505237818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 msgr2=0x7f5da00b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5da7577640 1 --2- 172.21.15.26:0/1505237818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 0x7f5da00b4640 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f5da008cae0 tx=0x7f5d9802a110 comp rx=0 tx=0).stop 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5da7577640 1 -- 172.21.15.26:0/1505237818 shutdown_connections 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5da7577640 1 --2- 172.21.15.26:0/1505237818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 0x7f5da00b4640 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5da7577640 1 --2- 172.21.15.26:0/1505237818 >> v2:172.21.15.29:3300/0 conn(0x7f5da00d4870 0x7f5da00b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5da7577640 1 -- 172.21.15.26:0/1505237818 >> 172.21.15.26:0/1505237818 conn(0x7f5da008dbe0 msgr2=0x7f5da009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5da7577640 1 -- 172.21.15.26:0/1505237818 shutdown_connections 2024-04-24T23:26:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.305+0000 7f5da7577640 1 -- 172.21.15.26:0/1505237818 wait complete. 2024-04-24T23:26:18.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da7577640 1 Processor -- start 2024-04-24T23:26:18.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da7577640 1 -- start start 2024-04-24T23:26:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da7577640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 0x7f5da015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da7577640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5da00d4870 0x7f5da0155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da7577640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5da015e570 0x7f5da00b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da7577640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00cfa70 con 0x7f5da00b3130 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da7577640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00d3010 con 0x7f5da00d4870 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da7577640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5da00cfd50 con 0x7f5da015e570 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da5313640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 0x7f5da015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da5313640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 0x7f5da015bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56176/0 (socket says 172.21.15.26:56176) 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.307+0000 7f5da5313640 1 -- 172.21.15.26:0/1967968744 learned_addr learned my addr 172.21.15.26:0/1967968744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da4b12640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5da00d4870 0x7f5da0155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da5b14640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5da015e570 0x7f5da00b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da5313640 1 -- 172.21.15.26:0/1967968744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5da015e570 msgr2=0x7f5da00b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da5313640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5da015e570 0x7f5da00b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da5313640 1 -- 172.21.15.26:0/1967968744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5da00d4870 msgr2=0x7f5da0155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da5313640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5da00d4870 0x7f5da0155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da5313640 1 -- 172.21.15.26:0/1967968744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da015eaf0 con 0x7f5da00b3130 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da5b14640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5da015e570 0x7f5da00b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da5313640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 0x7f5da015bd30 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000cb10 tx=0x7f5d9000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5d9dffb640 1 -- 172.21.15.26:0/1967968744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90017070 con 0x7f5da00b3130 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5d9dffb640 1 -- 172.21.15.26:0/1967968744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9000abb0 con 0x7f5da00b3130 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da00b1930 con 0x7f5da00b3130 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5d9dffb640 1 -- 172.21.15.26:0/1967968744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90012400 con 0x7f5da00b3130 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.308+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da0155540 con 0x7f5da00b3130 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.309+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da00694c0 con 0x7f5da00b3130 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.310+0000 7f5d9dffb640 1 -- 172.21.15.26:0/1967968744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d9001a070 con 0x7f5da00b3130 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.310+0000 7f5d9dffb640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5d8c06be30 0x7f5d8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.310+0000 7f5d9dffb640 1 -- 172.21.15.26:0/1967968744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f5d9008aac0 con 0x7f5da00b3130 2024-04-24T23:26:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.310+0000 7f5da4b12640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5d8c06be30 0x7f5d8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.311+0000 7f5da4b12640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5d8c06be30 0x7f5d8c06e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5da015c270 tx=0x7f5d94008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.314+0000 7f5d9dffb640 1 -- 172.21.15.26:0/1967968744 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5da00d32f0 con 0x7f5da00b3130 2024-04-24T23:26:18.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.497+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5da00b5570 con 0x7f5da00b3130 2024-04-24T23:26:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.498+0000 7f5d9dffb640 1 -- 172.21.15.26:0/1967968744 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16613 (secure 0 0 0) 0x7f5d90058eb0 con 0x7f5da00b3130 2024-04-24T23:26:18.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:18.500 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:26:17.523097+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:26:15.824299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:26:16.498456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.500+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5d8c06be30 msgr2=0x7f5d8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.500+0000 7f5da7577640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5d8c06be30 0x7f5d8c06e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5da015c270 tx=0x7f5d94008040 comp rx=0 tx=0).stop 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.500+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 msgr2=0x7f5da015bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.500+0000 7f5da7577640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 0x7f5da015bd30 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000cb10 tx=0x7f5d9000a060 comp rx=0 tx=0).stop 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.501+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 shutdown_connections 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.501+0000 7f5da7577640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5d8c06be30 0x7f5d8c06e2f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.501+0000 7f5da7577640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5da015e570 0x7f5da00b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.501+0000 7f5da7577640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5da00d4870 0x7f5da0155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.501+0000 7f5da7577640 1 --2- 172.21.15.26:0/1967968744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5da00b3130 0x7f5da015bd30 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.501+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 >> 172.21.15.26:0/1967968744 conn(0x7f5da008dbe0 msgr2=0x7f5da00caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.501+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 shutdown_connections 2024-04-24T23:26:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.501+0000 7f5da7577640 1 -- 172.21.15.26:0/1967968744 wait complete. 2024-04-24T23:26:18.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:18.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.612+0000 7f714f65d640 1 Processor -- start 2024-04-24T23:26:18.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.613+0000 7f714f65d640 1 -- start start 2024-04-24T23:26:18.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.613+0000 7f714f65d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480d4870 0x7f71480b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.613+0000 7f714f65d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71480d32f0 con 0x7f714815e570 2024-04-24T23:26:18.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.613+0000 7f714f65d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71480d0270 con 0x7f71480b3130 2024-04-24T23:26:18.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.613+0000 7f714f65d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f71480d1ab0 con 0x7f71480d4870 2024-04-24T23:26:18.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.613+0000 7f714d3f9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f714815e570 0x7f71480d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55860/0 (socket says 172.21.15.26:55860) 2024-04-24T23:26:18.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.613+0000 7f714d3f9640 1 -- 172.21.15.26:0/3984807147 learned_addr learned my addr 172.21.15.26:0/3984807147 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1923998910 0 0) 0x7f7140001110 con 0x7f714815e570 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7130003520 con 0x7f714815e570 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f714cbf8640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480d4870 0x7f71480b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 438336392 0 0) 0x7f7130003520 con 0x7f714815e570 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480d4870 msgr2=0x7f71480b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480d4870 0x7f71480b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 >> v1:172.21.15.29:6789/0 conn(0x7f71480b3130 legacy=0x7f71480b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71480d2d30 con 0x7f714815e570 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f7140001840 con 0x7f714815e570 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 >> v1:172.21.15.26:6789/0 conn(0x7f714815e570 legacy=0x7f71480d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480d4870 0x7f71300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f7130004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7130004d40 0x7f71300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71480d1ab0 con 0x7f7130003f60 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f713000baa0 con 0x7f7130004d40 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f713000bca0 con 0x7f71480d4870 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f714cbf8640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f7130004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f714d3f9640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480d4870 0x7f71300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f714dbfa640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7130004d40 0x7f71300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f714cbf8640 1 -- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480d4870 msgr2=0x7f71300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.614+0000 7f714cbf8640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480d4870 0x7f71300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.615+0000 7f714cbf8640 1 -- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7130004d40 msgr2=0x7f71300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.615+0000 7f714cbf8640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7130004d40 0x7f71300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.615+0000 7f714cbf8640 1 -- 172.21.15.26:0/3984807147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7130003520 con 0x7f7130003f60 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.615+0000 7f714d3f9640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480d4870 0x7f71300039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:18.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.615+0000 7f714dbfa640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7130004d40 0x7f71300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.615+0000 7f714cbf8640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f7130004800 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f713c009ac0 tx=0x7f7130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ae963239560cba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.615+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c036070 con 0x7f7130003f60 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.615+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713c031400 con 0x7f7130003f60 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.615+0000 7f7147fff640 1 -- 172.21.15.26:0/3984807147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c009690 con 0x7f7130003f60 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.616+0000 7f714f65d640 1 -- 172.21.15.26:0/3984807147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 msgr2=0x7f7130004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.616+0000 7f714f65d640 1 --2- 172.21.15.26:0/3984807147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f7130004800 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f713c009ac0 tx=0x7f7130005830 comp rx=0 tx=0).stop 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.616+0000 7f714dbfa640 1 -- 172.21.15.26:0/3984807147 reap_dead start 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.616+0000 7f714f65d640 1 -- 172.21.15.26:0/3984807147 shutdown_connections 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.616+0000 7f714f65d640 1 -- 172.21.15.26:0/3984807147 >> 172.21.15.26:0/3984807147 conn(0x7f714808dbe0 msgr2=0x7f714809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.616+0000 7f714f65d640 1 -- 172.21.15.26:0/3984807147 shutdown_connections 2024-04-24T23:26:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.616+0000 7f714f65d640 1 -- 172.21.15.26:0/3984807147 wait complete. 2024-04-24T23:26:18.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.617+0000 7f714f65d640 1 Processor -- start 2024-04-24T23:26:18.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.617+0000 7f714f65d640 1 -- start start 2024-04-24T23:26:18.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714f65d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f71481514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714f65d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480cbf50 0x7f71480cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714f65d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71480b4190 0x7f71480b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714f65d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71480cfa70 con 0x7f7130003f60 2024-04-24T23:26:18.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714f65d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f71480d2ae0 con 0x7f71480b4190 2024-04-24T23:26:18.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714f65d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f71480cfd50 con 0x7f71480cbf50 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714d3f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f71481514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714d3f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f71481514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56194/0 (socket says 172.21.15.26:56194) 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714d3f9640 1 -- 172.21.15.26:0/1369993450 learned_addr learned my addr 172.21.15.26:0/1369993450 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714dbfa640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71480b4190 0x7f71480b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.618+0000 7f714cbf8640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480cbf50 0x7f71480cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f714d3f9640 1 -- 172.21.15.26:0/1369993450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480cbf50 msgr2=0x7f71480cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f714d3f9640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480cbf50 0x7f71480cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f714d3f9640 1 -- 172.21.15.26:0/1369993450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71480b4190 msgr2=0x7f71480b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f714d3f9640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71480b4190 0x7f71480b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f714d3f9640 1 -- 172.21.15.26:0/1369993450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71480b8480 con 0x7f7130003f60 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f714dbfa640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71480b4190 0x7f71480b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f714d3f9640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f71481514a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f7140001eb0 tx=0x7f7140007790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f7145ffb640 1 -- 172.21.15.26:0/1369993450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714000e8c0 con 0x7f7130003f60 2024-04-24T23:26:18.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f7145ffb640 1 -- 172.21.15.26:0/1369993450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7140018070 con 0x7f7130003f60 2024-04-24T23:26:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71480b8760 con 0x7f7130003f60 2024-04-24T23:26:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f7145ffb640 1 -- 172.21.15.26:0/1369993450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714000ec60 con 0x7f7130003f60 2024-04-24T23:26:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.619+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71480b8d20 con 0x7f7130003f60 2024-04-24T23:26:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.620+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71480694c0 con 0x7f7130003f60 2024-04-24T23:26:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.620+0000 7f7145ffb640 1 -- 172.21.15.26:0/1369993450 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f714001d460 con 0x7f7130003f60 2024-04-24T23:26:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.621+0000 7f7145ffb640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be60 0x7f713406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.621+0000 7f7145ffb640 1 -- 172.21.15.26:0/1369993450 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f714008ae30 con 0x7f7130003f60 2024-04-24T23:26:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.621+0000 7f714cbf8640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be60 0x7f713406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.621+0000 7f714cbf8640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be60 0x7f713406e320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f713c033000 tx=0x7f713c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.625+0000 7f7145ffb640 1 -- 172.21.15.26:0/1369993450 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f71480d32f0 con 0x7f7130003f60 2024-04-24T23:26:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.811+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71480b2ef0 con 0x7f7130003f60 2024-04-24T23:26:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.812+0000 7f7145ffb640 1 -- 172.21.15.26:0/1369993450 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16613 (secure 0 0 0) 0x7f71400597e0 con 0x7f7130003f60 2024-04-24T23:26:18.814 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:18.814 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:26:17.523097+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:26:15.824299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:26:16.498456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.814+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be60 msgr2=0x7f713406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.814+0000 7f714f65d640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be60 0x7f713406e320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f713c033000 tx=0x7f713c008030 comp rx=0 tx=0).stop 2024-04-24T23:26:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.814+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 msgr2=0x7f71481514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.814+0000 7f714f65d640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f71481514a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f7140001eb0 tx=0x7f7140007790 comp rx=0 tx=0).stop 2024-04-24T23:26:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.814+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 shutdown_connections 2024-04-24T23:26:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.814+0000 7f714f65d640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f713406be60 0x7f713406e320 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.814+0000 7f714f65d640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71480b4190 0x7f71480b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.814+0000 7f714f65d640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f71480cbf50 0x7f71480cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.815+0000 7f714f65d640 1 --2- 172.21.15.26:0/1369993450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7130003f60 0x7f71481514a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.815+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 >> 172.21.15.26:0/1369993450 conn(0x7f714808dbe0 msgr2=0x7f714806d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.815+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 shutdown_connections 2024-04-24T23:26:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.815+0000 7f714f65d640 1 -- 172.21.15.26:0/1369993450 wait complete. 2024-04-24T23:26:18.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:26:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.931+0000 7ff32b110640 1 Processor -- start 2024-04-24T23:26:18.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.932+0000 7ff32b110640 1 -- start start 2024-04-24T23:26:18.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.932+0000 7ff32b110640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff32406d5a0 0x7ff3240d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.932+0000 7ff32b110640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff3240b37e0 0x7ff3240b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.932+0000 7ff32b110640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff324070f30 con 0x7ff32415e120 2024-04-24T23:26:18.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.932+0000 7ff32b110640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff32406dcf0 con 0x7ff3240b37e0 2024-04-24T23:26:18.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.932+0000 7ff32b110640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff32406dfd0 con 0x7ff32406d5a0 2024-04-24T23:26:18.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.932+0000 7ff3296ad640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff32415e120 0x7ff324150e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55868/0 (socket says 172.21.15.26:55868) 2024-04-24T23:26:18.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.932+0000 7ff3296ad640 1 -- 172.21.15.26:0/351722907 learned_addr learned my addr 172.21.15.26:0/351722907 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:18.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff328eac640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff32406d5a0 0x7ff3240d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff323fff640 1 --2- 172.21.15.26:0/351722907 >> v2:172.21.15.29:3300/0 conn(0x7ff3240b37e0 0x7ff3240b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4123448042 0 0) 0x7ff324070f30 con 0x7ff32415e120 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff30c003520 con 0x7ff32415e120 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 336964527 0 0) 0x7ff30c003520 con 0x7ff32415e120 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff32406d5a0 msgr2=0x7ff3240d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff32406d5a0 0x7ff3240d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 >> v2:172.21.15.29:3300/0 conn(0x7ff3240b37e0 msgr2=0x7ff3240b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 --2- 172.21.15.26:0/351722907 >> v2:172.21.15.29:3300/0 conn(0x7ff3240b37e0 0x7ff3240b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32406e2b0 con 0x7ff32415e120 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7ff31c0019f0 con 0x7ff32415e120 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 >> v1:172.21.15.26:6789/0 conn(0x7ff32415e120 legacy=0x7ff324150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff30c0039b0 0x7ff30c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff32406d5a0 0x7ff30c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff30c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff32406dcf0 con 0x7ff30c004d40 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff32406dfd0 con 0x7ff30c0039b0 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c00b9c0 con 0x7ff32406d5a0 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff3296ad640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff30c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff323fff640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff32406d5a0 0x7ff30c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.933+0000 7ff328eac640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff30c0039b0 0x7ff30c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.934+0000 7ff3296ad640 1 -- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff32406d5a0 msgr2=0x7ff30c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.934+0000 7ff3296ad640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff32406d5a0 0x7ff30c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.934+0000 7ff3296ad640 1 -- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff30c0039b0 msgr2=0x7ff30c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.934+0000 7ff3296ad640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff30c0039b0 0x7ff30c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.934+0000 7ff3296ad640 1 -- 172.21.15.26:0/351722907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c003520 con 0x7ff30c004d40 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.934+0000 7ff3296ad640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff30c0051c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00a480 tx=0x7ff30c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996c48c0c28a33b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.934+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c00a4c0 con 0x7ff30c004d40 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.934+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31c031070 con 0x7ff30c004d40 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.934+0000 7ff3237fe640 1 -- 172.21.15.26:0/351722907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c036470 con 0x7ff30c004d40 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.935+0000 7ff32b110640 1 -- 172.21.15.26:0/351722907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 msgr2=0x7ff30c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.935+0000 7ff32b110640 1 --2- 172.21.15.26:0/351722907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff30c0051c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00a480 tx=0x7ff30c005830 comp rx=0 tx=0).stop 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.935+0000 7ff3296ad640 1 -- 172.21.15.26:0/351722907 reap_dead start 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.935+0000 7ff32b110640 1 -- 172.21.15.26:0/351722907 shutdown_connections 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.935+0000 7ff32b110640 1 -- 172.21.15.26:0/351722907 >> 172.21.15.26:0/351722907 conn(0x7ff32406b540 msgr2=0x7ff32406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.935+0000 7ff32b110640 1 -- 172.21.15.26:0/351722907 shutdown_connections 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.935+0000 7ff32b110640 1 -- 172.21.15.26:0/351722907 wait complete. 2024-04-24T23:26:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.936+0000 7ff32b110640 1 Processor -- start 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.936+0000 7ff32b110640 1 -- start start 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff32b110640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff3240af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff32b110640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff324157a00 0x7ff324150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff32b110640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff32409c200 0x7ff32409c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff32b110640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff32415e590 con 0x7ff30c004d40 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff32b110640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff32406d980 con 0x7ff32409c200 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff32b110640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff32415e710 con 0x7ff324157a00 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff328eac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff3240af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff328eac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff3240af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56206/0 (socket says 172.21.15.26:56206) 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff328eac640 1 -- 172.21.15.26:0/4103960407 learned_addr learned my addr 172.21.15.26:0/4103960407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff328eac640 1 -- 172.21.15.26:0/4103960407 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff324157a00 msgr2=0x7ff324150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff323fff640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff324157a00 0x7ff324150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff328eac640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff324157a00 0x7ff324150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff328eac640 1 -- 172.21.15.26:0/4103960407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff32409c200 msgr2=0x7ff32409c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff328eac640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff32409c200 0x7ff32409c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.937+0000 7ff328eac640 1 -- 172.21.15.26:0/4103960407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3240b01c0 con 0x7ff30c004d40 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.938+0000 7ff328eac640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff3240af8c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff314011910 tx=0x7ff314011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.938+0000 7ff3217fa640 1 -- 172.21.15.26:0/4103960407 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31400fa70 con 0x7ff30c004d40 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.938+0000 7ff3217fa640 1 -- 172.21.15.26:0/4103960407 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31400fc10 con 0x7ff30c004d40 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.938+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3240b04a0 con 0x7ff30c004d40 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.938+0000 7ff3217fa640 1 -- 172.21.15.26:0/4103960407 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314014440 con 0x7ff30c004d40 2024-04-24T23:26:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.938+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3240cbfe0 con 0x7ff30c004d40 2024-04-24T23:26:18.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.939+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3240cbde0 con 0x7ff30c004d40 2024-04-24T23:26:18.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.940+0000 7ff3217fa640 1 -- 172.21.15.26:0/4103960407 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff314012040 con 0x7ff30c004d40 2024-04-24T23:26:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.940+0000 7ff3217fa640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff31006be30 0x7ff31006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.940+0000 7ff3217fa640 1 -- 172.21.15.26:0/4103960407 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7ff31408b490 con 0x7ff30c004d40 2024-04-24T23:26:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.940+0000 7ff323fff640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff31006be30 0x7ff31006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.941+0000 7ff323fff640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff31006be30 0x7ff31006e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff324157e40 tx=0x7ff318008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:18.944+0000 7ff3217fa640 1 -- 172.21.15.26:0/4103960407 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff314019030 con 0x7ff30c004d40 2024-04-24T23:26:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:19.167+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff324075610 con 0x7ff30c004d40 2024-04-24T23:26:20.144 INFO:tasks.ceph.mds.b.smithi029.stderr:2024-04-24T23:26:20.143+0000 7f440015b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:26:20.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.168+0000 7ff3217fa640 1 -- 172.21.15.26:0/4103960407 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) ==== 118+0+0 (secure 0 0 0) 0x7ff314059e40 con 0x7ff30c004d40 2024-04-24T23:26:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff31006be30 msgr2=0x7ff31006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff31006be30 0x7ff31006e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff324157e40 tx=0x7ff318008040 comp rx=0 tx=0).stop 2024-04-24T23:26:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 msgr2=0x7ff3240af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff3240af8c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff314011910 tx=0x7ff314011dd0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 shutdown_connections 2024-04-24T23:26:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff31006be30 0x7ff31006e2f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff32409c200 0x7ff32409c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff324157a00 0x7ff324150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 --2- 172.21.15.26:0/4103960407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff30c004d40 0x7ff3240af8c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 >> 172.21.15.26:0/4103960407 conn(0x7ff32406b540 msgr2=0x7ff3240ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 shutdown_connections 2024-04-24T23:26:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.171+0000 7ff32b110640 1 -- 172.21.15.26:0/4103960407 wait complete. 2024-04-24T23:26:20.185 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:26:20.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:26:20.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.295+0000 7f47baa9a640 1 Processor -- start 2024-04-24T23:26:20.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.295+0000 7f47baa9a640 1 -- start start 2024-04-24T23:26:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.296+0000 7f47baa9a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f47b40ce640 0x7f47b40d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.296+0000 7f47baa9a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b415da30 0x7f47b415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.296+0000 7f47baa9a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47b406d750 con 0x7f47b40d28e0 2024-04-24T23:26:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.296+0000 7f47baa9a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f47b40d1110 con 0x7f47b40ce640 2024-04-24T23:26:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.296+0000 7f47baa9a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f47b40cc410 con 0x7f47b415da30 2024-04-24T23:26:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.296+0000 7f47b8836640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f47b40ce640 0x7f47b40d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.296+0000 7f47b9037640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b415da30 0x7f47b415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.296+0000 7f47b3fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f47b40d28e0 0x7f47b40cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:53656/0 (socket says 172.21.15.26:53656) 2024-04-24T23:26:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.296+0000 7f47b3fff640 1 -- 172.21.15.26:0/3282559026 learned_addr learned my addr 172.21.15.26:0/3282559026 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.297+0000 7f47b8836640 1 -- 172.21.15.26:0/3282559026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b415da30 msgr2=0x7f47b415deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.297+0000 7f47b8836640 1 --2- 172.21.15.26:0/3282559026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b415da30 0x7f47b415deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.297+0000 7f47b8836640 1 -- 172.21.15.26:0/3282559026 >> v1:172.21.15.29:6789/0 conn(0x7f47b40d28e0 legacy=0x7f47b40cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.297+0000 7f47b8836640 1 -- 172.21.15.26:0/3282559026 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b40b1720 con 0x7f47b40ce640 2024-04-24T23:26:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.297+0000 7f47b9037640 1 --2- 172.21.15.26:0/3282559026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b415da30 0x7f47b415deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47b8836640 1 --2- 172.21.15.26:0/3282559026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47b40ce640 0x7f47b40d00f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f47a40097a0 tx=0x7f47a402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cbddb8e763cc86b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47b37fe640 1 -- 172.21.15.26:0/3282559026 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4037070 con 0x7f47b40ce640 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47b37fe640 1 -- 172.21.15.26:0/3282559026 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a4032440 con 0x7f47b40ce640 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47b37fe640 1 -- 172.21.15.26:0/3282559026 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4032810 con 0x7f47b40ce640 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47baa9a640 1 -- 172.21.15.26:0/3282559026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47b40ce640 msgr2=0x7f47b40d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47baa9a640 1 --2- 172.21.15.26:0/3282559026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47b40ce640 0x7f47b40d00f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f47a40097a0 tx=0x7f47a402a210 comp rx=0 tx=0).stop 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47baa9a640 1 -- 172.21.15.26:0/3282559026 shutdown_connections 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47baa9a640 1 --2- 172.21.15.26:0/3282559026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b415da30 0x7f47b415deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47baa9a640 1 --2- 172.21.15.26:0/3282559026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47b40ce640 0x7f47b40d00f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47baa9a640 1 -- 172.21.15.26:0/3282559026 >> 172.21.15.26:0/3282559026 conn(0x7f47b40d1dc0 msgr2=0x7f47b40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:20.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47baa9a640 1 -- 172.21.15.26:0/3282559026 shutdown_connections 2024-04-24T23:26:20.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.298+0000 7f47baa9a640 1 -- 172.21.15.26:0/3282559026 wait complete. 2024-04-24T23:26:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.300+0000 7f47baa9a640 1 Processor -- start 2024-04-24T23:26:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.300+0000 7f47baa9a640 1 -- start start 2024-04-24T23:26:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.300+0000 7f47baa9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b40ce640 0x7f47b4156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.300+0000 7f47baa9a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b40d28e0 0x7f47b40cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.300+0000 7f47baa9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47b415da30 0x7f47b40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.300+0000 7f47baa9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47b40cc940 con 0x7f47b40ce640 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.300+0000 7f47baa9a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f47b40cc660 con 0x7f47b415da30 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.300+0000 7f47baa9a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f47b40ccc20 con 0x7f47b40d28e0 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b8836640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b40ce640 0x7f47b4156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b8836640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b40ce640 0x7f47b4156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56220/0 (socket says 172.21.15.26:56220) 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b8836640 1 -- 172.21.15.26:0/1140730662 learned_addr learned my addr 172.21.15.26:0/1140730662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b3fff640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b40d28e0 0x7f47b40cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b9037640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47b415da30 0x7f47b40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b8836640 1 -- 172.21.15.26:0/1140730662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b40d28e0 msgr2=0x7f47b40cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b8836640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b40d28e0 0x7f47b40cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b8836640 1 -- 172.21.15.26:0/1140730662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47b415da30 msgr2=0x7f47b40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b8836640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47b415da30 0x7f47b40b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b8836640 1 -- 172.21.15.26:0/1140730662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b40cc0b0 con 0x7f47b40ce640 2024-04-24T23:26:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b8836640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b40ce640 0x7f47b4156310 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f47a4035040 tx=0x7f47a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b17fa640 1 -- 172.21.15.26:0/1140730662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4037040 con 0x7f47b40ce640 2024-04-24T23:26:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b17fa640 1 -- 172.21.15.26:0/1140730662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a4030500 con 0x7f47b40ce640 2024-04-24T23:26:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b40cb190 con 0x7f47b40ce640 2024-04-24T23:26:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47b17fa640 1 -- 172.21.15.26:0/1140730662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4030910 con 0x7f47b40ce640 2024-04-24T23:26:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.301+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b40ca3d0 con 0x7f47b40ce640 2024-04-24T23:26:20.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.303+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b4075610 con 0x7f47b40ce640 2024-04-24T23:26:20.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.303+0000 7f47b17fa640 1 -- 172.21.15.26:0/1140730662 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47a4044020 con 0x7f47b40ce640 2024-04-24T23:26:20.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.303+0000 7f47b17fa640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f47a006be30 0x7f47a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:20.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.303+0000 7f47b17fa640 1 -- 172.21.15.26:0/1140730662 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f47a40ab150 con 0x7f47b40ce640 2024-04-24T23:26:20.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.304+0000 7f47b3fff640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f47a006be30 0x7f47a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:20.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.304+0000 7f47b3fff640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f47a006be30 0x7f47a006e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f47b40cd740 tx=0x7f47a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:20.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.308+0000 7f47b17fa640 1 -- 172.21.15.26:0/1140730662 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f47b40cb190 con 0x7f47b40ce640 2024-04-24T23:26:20.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:20.528+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f47b40b1500 con 0x7f47b40ce640 2024-04-24T23:26:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.169+0000 7f47b17fa640 1 -- 172.21.15.26:0/1140730662 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7f47b40b1720 con 0x7f47b40ce640 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f47a006be30 msgr2=0x7f47a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f47a006be30 0x7f47a006e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f47b40cd740 tx=0x7f47a8009040 comp rx=0 tx=0).stop 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b40ce640 msgr2=0x7f47b4156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b40ce640 0x7f47b4156310 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f47a4035040 tx=0x7f47a40046c0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 shutdown_connections 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f47a006be30 0x7f47a006e2f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47b415da30 0x7f47b40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f47b40d28e0 0x7f47b40cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 --2- 172.21.15.26:0/1140730662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b40ce640 0x7f47b4156310 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 >> 172.21.15.26:0/1140730662 conn(0x7f47b40d1dc0 msgr2=0x7f47b40741d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 shutdown_connections 2024-04-24T23:26:21.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.172+0000 7f47baa9a640 1 -- 172.21.15.26:0/1140730662 wait complete. 2024-04-24T23:26:21.186 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:26:21.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:21.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.293+0000 7f1abc922640 1 Processor -- start 2024-04-24T23:26:21.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.293+0000 7f1abc922640 1 -- start start 2024-04-24T23:26:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.294+0000 7f1abc922640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ab8143c40 0x7f1ab80cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.294+0000 7f1abc922640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab80cc860 0x7f1ab8072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.294+0000 7f1abc922640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab80cd0b0 con 0x7f1ab815eb00 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.294+0000 7f1abc922640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab8071b70 con 0x7f1ab8143c40 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.294+0000 7f1abc922640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab8071240 con 0x7f1ab80cc860 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.294+0000 7f1ab659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ab8143c40 0x7f1ab80cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.294+0000 7f1ab6d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ab815eb00 0x7f1ab80b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55884/0 (socket says 172.21.15.26:55884) 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.294+0000 7f1ab6d9d640 1 -- 172.21.15.26:0/408050735 learned_addr learned my addr 172.21.15.26:0/408050735 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.294+0000 7f1ab5d9b640 1 --2- 172.21.15.26:0/408050735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab80cc860 0x7f1ab8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.295+0000 7f1ab659c640 1 -- 172.21.15.26:0/408050735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab80cc860 msgr2=0x7f1ab8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.295+0000 7f1ab659c640 1 --2- 172.21.15.26:0/408050735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab80cc860 0x7f1ab8072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.295+0000 7f1ab659c640 1 -- 172.21.15.26:0/408050735 >> v1:172.21.15.26:6789/0 conn(0x7f1ab815eb00 legacy=0x7f1ab80b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.295+0000 7f1ab659c640 1 -- 172.21.15.26:0/408050735 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab8071520 con 0x7f1ab8143c40 2024-04-24T23:26:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.295+0000 7f1ab659c640 1 --2- 172.21.15.26:0/408050735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8143c40 0x7f1ab80cf870 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00097a0 tx=0x7f1ab002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d5e4924fcd7d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.295+0000 7f1ab559a640 1 -- 172.21.15.26:0/408050735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0037070 con 0x7f1ab8143c40 2024-04-24T23:26:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1ab559a640 1 -- 172.21.15.26:0/408050735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0032440 con 0x7f1ab8143c40 2024-04-24T23:26:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1ab559a640 1 -- 172.21.15.26:0/408050735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0032810 con 0x7f1ab8143c40 2024-04-24T23:26:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1abc922640 1 -- 172.21.15.26:0/408050735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8143c40 msgr2=0x7f1ab80cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1abc922640 1 --2- 172.21.15.26:0/408050735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8143c40 0x7f1ab80cf870 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00097a0 tx=0x7f1ab002a180 comp rx=0 tx=0).stop 2024-04-24T23:26:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1abc922640 1 -- 172.21.15.26:0/408050735 shutdown_connections 2024-04-24T23:26:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1abc922640 1 --2- 172.21.15.26:0/408050735 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab80cc860 0x7f1ab8072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1abc922640 1 --2- 172.21.15.26:0/408050735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8143c40 0x7f1ab80cf870 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1abc922640 1 -- 172.21.15.26:0/408050735 >> 172.21.15.26:0/408050735 conn(0x7f1ab806b300 msgr2=0x7f1ab806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1abc922640 1 -- 172.21.15.26:0/408050735 shutdown_connections 2024-04-24T23:26:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.296+0000 7f1abc922640 1 -- 172.21.15.26:0/408050735 wait complete. 2024-04-24T23:26:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.297+0000 7f1abc922640 1 Processor -- start 2024-04-24T23:26:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1abc922640 1 -- start start 2024-04-24T23:26:21.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1abc922640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab80cc860 0x7f1ab81f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1abc922640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ab8143c40 0x7f1ab815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1abc922640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab815eb00 0x7f1ab813d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1abc922640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8072540 con 0x7f1ab80cc860 2024-04-24T23:26:21.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1abc922640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8144020 con 0x7f1ab8143c40 2024-04-24T23:26:21.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1abc922640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab81f7690 con 0x7f1ab815eb00 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1ab659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab80cc860 0x7f1ab81f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1ab659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab80cc860 0x7f1ab81f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56238/0 (socket says 172.21.15.26:56238) 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.298+0000 7f1ab659c640 1 -- 172.21.15.26:0/1198216833 learned_addr learned my addr 172.21.15.26:0/1198216833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1ab6d9d640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab815eb00 0x7f1ab813d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1ab5d9b640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ab8143c40 0x7f1ab815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1ab659c640 1 -- 172.21.15.26:0/1198216833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab815eb00 msgr2=0x7f1ab813d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1ab659c640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab815eb00 0x7f1ab813d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1ab659c640 1 -- 172.21.15.26:0/1198216833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ab8143c40 msgr2=0x7f1ab815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1ab659c640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ab8143c40 0x7f1ab815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1ab659c640 1 -- 172.21.15.26:0/1198216833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab80c9630 con 0x7f1ab80cc860 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1ab659c640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab80cc860 0x7f1ab81f7150 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f1ab002a1c0 tx=0x7f1ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1aaeffd640 1 -- 172.21.15.26:0/1198216833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0037040 con 0x7f1ab80cc860 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1aaeffd640 1 -- 172.21.15.26:0/1198216833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0042070 con 0x7f1ab80cc860 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab80c9910 con 0x7f1ab80cc860 2024-04-24T23:26:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1aaeffd640 1 -- 172.21.15.26:0/1198216833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0030bb0 con 0x7f1ab80cc860 2024-04-24T23:26:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.299+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab80c9ed0 con 0x7f1ab80cc860 2024-04-24T23:26:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.300+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab80694c0 con 0x7f1ab80cc860 2024-04-24T23:26:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.301+0000 7f1aaeffd640 1 -- 172.21.15.26:0/1198216833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ab0047460 con 0x7f1ab80cc860 2024-04-24T23:26:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.301+0000 7f1aaeffd640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a9806be30 0x7f1a9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.301+0000 7f1aaeffd640 1 -- 172.21.15.26:0/1198216833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f1ab00ab9c0 con 0x7f1ab80cc860 2024-04-24T23:26:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.301+0000 7f1ab5d9b640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a9806be30 0x7f1a9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.302+0000 7f1ab5d9b640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a9806be30 0x7f1a9806e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8140bc0 tx=0x7f1aa0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:21.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.306+0000 7f1aaeffd640 1 -- 172.21.15.26:0/1198216833 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ab00ade20 con 0x7f1ab80cc860 2024-04-24T23:26:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.528+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ab806e7f0 con 0x7f1ab80cc860 2024-04-24T23:26:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.529+0000 7f1aaeffd640 1 -- 172.21.15.26:0/1198216833 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5180 (secure 0 0 0) 0x7f1ab0045360 con 0x7f1ab80cc860 2024-04-24T23:26:21.530 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:21.530 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":84,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:20.530093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6790},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790]},"id":6}]} 2024-04-24T23:26:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 84 2024-04-24T23:26:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a9806be30 msgr2=0x7f1a9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a9806be30 0x7f1a9806e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8140bc0 tx=0x7f1aa0008040 comp rx=0 tx=0).stop 2024-04-24T23:26:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab80cc860 msgr2=0x7f1ab81f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab80cc860 0x7f1ab81f7150 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f1ab002a1c0 tx=0x7f1ab00046c0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 shutdown_connections 2024-04-24T23:26:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a9806be30 0x7f1a9806e2f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ab815eb00 0x7f1ab813d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ab8143c40 0x7f1ab815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 --2- 172.21.15.26:0/1198216833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab80cc860 0x7f1ab81f7150 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 >> 172.21.15.26:0/1198216833 conn(0x7f1ab806b300 msgr2=0x7f1ab806d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 shutdown_connections 2024-04-24T23:26:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.531+0000 7f1abc922640 1 -- 172.21.15.26:0/1198216833 wait complete. 2024-04-24T23:26:21.543 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:21.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.643+0000 7ff9f1e72640 1 Processor -- start 2024-04-24T23:26:21.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.643+0000 7ff9f1e72640 1 -- start start 2024-04-24T23:26:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9f1e72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec145910 0x7ff9ec06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9f1e72640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff9ec06d350 0x7ff9ec15d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9f1e72640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9ec15d570 con 0x7ff9ec0cd660 2024-04-24T23:26:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9f1e72640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff9ec145cf0 con 0x7ff9ec06d350 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9f1e72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ec0cfbc0 con 0x7ff9ec145910 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ebfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff9ec0cd660 0x7ff9ec15d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55900/0 (socket says 172.21.15.26:55900) 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ebfff640 1 -- 172.21.15.26:0/1195145200 learned_addr learned my addr 172.21.15.26:0/1195145200 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9eb7fe640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec145910 0x7ff9ec06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2788829470 0 0) 0x7ff9e4001490 con 0x7ff9ec0cd660 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9d4003520 con 0x7ff9ec0cd660 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9eaffd640 1 --2- 172.21.15.26:0/1195145200 >> v2:172.21.15.29:3300/0 conn(0x7ff9ec06d350 0x7ff9ec15d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1302810018 0 0) 0x7ff9d4003520 con 0x7ff9ec0cd660 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec145910 msgr2=0x7ff9ec06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec145910 0x7ff9ec06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 >> v2:172.21.15.29:3300/0 conn(0x7ff9ec06d350 msgr2=0x7ff9ec15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 --2- 172.21.15.26:0/1195145200 >> v2:172.21.15.29:3300/0 conn(0x7ff9ec06d350 0x7ff9ec15d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec06d7f0 con 0x7ff9ec0cd660 2024-04-24T23:26:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7ff9e4001bc0 con 0x7ff9ec0cd660 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9eaffd640 1 --2- 172.21.15.26:0/1195145200 >> v2:172.21.15.29:3300/0 conn(0x7ff9ec06d350 0x7ff9ec15d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 >> v1:172.21.15.26:6789/0 conn(0x7ff9ec0cd660 legacy=0x7ff9ec15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9d40039b0 0x7ff9d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.644+0000 7ff9ea7fc640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec145910 0x7ff9d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec145cf0 con 0x7ff9d4004800 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec0cfbc0 con 0x7ff9d40039b0 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d400b9c0 con 0x7ff9ec145910 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9eaffd640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9ebfff640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec145910 0x7ff9d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9eb7fe640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9d40039b0 0x7ff9d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9eaffd640 1 -- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec145910 msgr2=0x7ff9d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9eaffd640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec145910 0x7ff9d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9eaffd640 1 -- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9d40039b0 msgr2=0x7ff9d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:21.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9eaffd640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9d40039b0 0x7ff9d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.645+0000 7ff9eaffd640 1 -- 172.21.15.26:0/1195145200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d4003520 con 0x7ff9d4004800 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9eaffd640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9d4004c80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0098a0 tx=0x7ff9d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f4683ed5ffbbed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00f040 con 0x7ff9d4004800 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9dc02ad20 con 0x7ff9d4004800 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9ea7fc640 1 -- 172.21.15.26:0/1195145200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc004250 con 0x7ff9d4004800 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9f1e72640 1 -- 172.21.15.26:0/1195145200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 msgr2=0x7ff9d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9f1e72640 1 --2- 172.21.15.26:0/1195145200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9d4004c80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0098a0 tx=0x7ff9d4005830 comp rx=0 tx=0).stop 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9ebfff640 1 -- 172.21.15.26:0/1195145200 reap_dead start 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9f1e72640 1 -- 172.21.15.26:0/1195145200 shutdown_connections 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9f1e72640 1 -- 172.21.15.26:0/1195145200 >> 172.21.15.26:0/1195145200 conn(0x7ff9ec06b300 msgr2=0x7ff9ec06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9f1e72640 1 -- 172.21.15.26:0/1195145200 shutdown_connections 2024-04-24T23:26:21.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.646+0000 7ff9f1e72640 1 -- 172.21.15.26:0/1195145200 wait complete. 2024-04-24T23:26:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.647+0000 7ff9f1e72640 1 Processor -- start 2024-04-24T23:26:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9f1e72640 1 -- start start 2024-04-24T23:26:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9f1e72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9ec15b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9f1e72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec13c8c0 0x7ff9ec13ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9f1e72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ec15e0f0 0x7ff9ec15e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9f1e72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec06e090 con 0x7ff9d4004800 2024-04-24T23:26:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9f1e72640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec06ddb0 con 0x7ff9ec15e0f0 2024-04-24T23:26:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9f1e72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ec06e370 con 0x7ff9ec13c8c0 2024-04-24T23:26:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9ec15b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9ec15b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56270/0 (socket says 172.21.15.26:56270) 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.648+0000 7ff9eb7fe640 1 -- 172.21.15.26:0/2871986475 learned_addr learned my addr 172.21.15.26:0/2871986475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9ebfff640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ec15e0f0 0x7ff9ec15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9eaffd640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec13c8c0 0x7ff9ec13ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9eb7fe640 1 -- 172.21.15.26:0/2871986475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec13c8c0 msgr2=0x7ff9ec13ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9eb7fe640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec13c8c0 0x7ff9ec13ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9eb7fe640 1 -- 172.21.15.26:0/2871986475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ec15e0f0 msgr2=0x7ff9ec15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9eb7fe640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ec15e0f0 0x7ff9ec15e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9eb7fe640 1 -- 172.21.15.26:0/2871986475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec13b290 con 0x7ff9d4004800 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9eb7fe640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9ec15b920 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0011910 tx=0x7ff9e0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9d3fff640 1 -- 172.21.15.26:0/2871986475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000faf0 con 0x7ff9d4004800 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9d3fff640 1 -- 172.21.15.26:0/2871986475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e000fc90 con 0x7ff9d4004800 2024-04-24T23:26:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ec13ce60 con 0x7ff9d4004800 2024-04-24T23:26:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9d3fff640 1 -- 172.21.15.26:0/2871986475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0014440 con 0x7ff9d4004800 2024-04-24T23:26:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.649+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ec13d420 con 0x7ff9d4004800 2024-04-24T23:26:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.650+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ec0694c0 con 0x7ff9d4004800 2024-04-24T23:26:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.651+0000 7ff9d3fff640 1 -- 172.21.15.26:0/2871986475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9e0012040 con 0x7ff9d4004800 2024-04-24T23:26:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.651+0000 7ff9d3fff640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff9d806be30 0x7ff9d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.651+0000 7ff9eaffd640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff9d806be30 0x7ff9d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.651+0000 7ff9d3fff640 1 -- 172.21.15.26:0/2871986475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7ff9e008b480 con 0x7ff9d4004800 2024-04-24T23:26:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.652+0000 7ff9eaffd640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff9d806be30 0x7ff9d806e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc033000 tx=0x7ff9dc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.656+0000 7ff9d3fff640 1 -- 172.21.15.26:0/2871986475 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9e0019030 con 0x7ff9d4004800 2024-04-24T23:26:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.840+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9ec1f1e50 con 0x7ff9d4004800 2024-04-24T23:26:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.841+0000 7ff9d3fff640 1 -- 172.21.15.26:0/2871986475 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16600 (secure 0 0 0) 0x7ff9e0059da0 con 0x7ff9d4004800 2024-04-24T23:26:21.842 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:21.842 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:26:19.159795+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:26:15.824299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:26:16.498456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.843+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff9d806be30 msgr2=0x7ff9d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.843+0000 7ff9f1e72640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff9d806be30 0x7ff9d806e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc033000 tx=0x7ff9dc02e000 comp rx=0 tx=0).stop 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.843+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 msgr2=0x7ff9ec15b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.843+0000 7ff9f1e72640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9ec15b920 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0011910 tx=0x7ff9e0011dd0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.843+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 shutdown_connections 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.843+0000 7ff9f1e72640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff9d806be30 0x7ff9d806e2f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.843+0000 7ff9f1e72640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9ec15e0f0 0x7ff9ec15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.843+0000 7ff9f1e72640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9ec13c8c0 0x7ff9ec13ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.843+0000 7ff9f1e72640 1 --2- 172.21.15.26:0/2871986475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d4004800 0x7ff9ec15b920 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.844+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 >> 172.21.15.26:0/2871986475 conn(0x7ff9ec06b300 msgr2=0x7ff9ec0721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.844+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 shutdown_connections 2024-04-24T23:26:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.844+0000 7ff9f1e72640 1 -- 172.21.15.26:0/2871986475 wait complete. 2024-04-24T23:26:21.856 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:26:21.856 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:26:21.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.956+0000 7f8b8a7f9640 1 Processor -- start 2024-04-24T23:26:21.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.956+0000 7f8b8a7f9640 1 -- start start 2024-04-24T23:26:21.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b8a7f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b8406f110 0x7f8b8415f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b8a7f9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b840adf80 0x7f8b84154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b8a7f9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b8415f750 con 0x7f8b8406f4f0 2024-04-24T23:26:21.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b8a7f9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8b840b0220 con 0x7f8b840adf80 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b8a7f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8b840af180 con 0x7f8b8406f110 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b88d96640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8b840adf80 0x7f8b84154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b83fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b8406f110 0x7f8b8415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b837fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f8b8406f4f0 0x7f8b840ad870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:53660/0 (socket says 172.21.15.26:53660) 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b837fe640 1 -- 172.21.15.26:0/1421151727 learned_addr learned my addr 172.21.15.26:0/1421151727 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b82ffd640 1 -- 172.21.15.26:0/1421151727 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 686994122 0 0) 0x7f8b78001110 con 0x7f8b8406f4f0 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b82ffd640 1 -- 172.21.15.26:0/1421151727 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b6c003520 con 0x7f8b8406f4f0 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b88d96640 1 -- 172.21.15.26:0/1421151727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b8406f110 msgr2=0x7f8b8415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b88d96640 1 --2- 172.21.15.26:0/1421151727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b8406f110 0x7f8b8415f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b88d96640 1 -- 172.21.15.26:0/1421151727 >> v1:172.21.15.29:6789/0 conn(0x7f8b8406f4f0 legacy=0x7f8b840ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.957+0000 7f8b88d96640 1 -- 172.21.15.26:0/1421151727 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b840af460 con 0x7f8b840adf80 2024-04-24T23:26:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.958+0000 7f8b88d96640 1 --2- 172.21.15.26:0/1421151727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b840adf80 0x7f8b84154a50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009d90 tx=0x7f8b7c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b8b46ba1b75fa72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.958+0000 7f8b82ffd640 1 -- 172.21.15.26:0/1421151727 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c007740 con 0x7f8b840adf80 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.958+0000 7f8b82ffd640 1 -- 172.21.15.26:0/1421151727 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7c0078e0 con 0x7f8b840adf80 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.958+0000 7f8b82ffd640 1 -- 172.21.15.26:0/1421151727 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c007cb0 con 0x7f8b840adf80 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.959+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/1421151727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b840adf80 msgr2=0x7f8b84154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.959+0000 7f8b8a7f9640 1 --2- 172.21.15.26:0/1421151727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b840adf80 0x7f8b84154a50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009d90 tx=0x7f8b7c02a660 comp rx=0 tx=0).stop 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.959+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/1421151727 shutdown_connections 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.959+0000 7f8b8a7f9640 1 --2- 172.21.15.26:0/1421151727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b840adf80 0x7f8b84154a50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.959+0000 7f8b8a7f9640 1 --2- 172.21.15.26:0/1421151727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b8406f110 0x7f8b8415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.959+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/1421151727 >> 172.21.15.26:0/1421151727 conn(0x7f8b840729a0 msgr2=0x7f8b8406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.959+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/1421151727 shutdown_connections 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.959+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/1421151727 wait complete. 2024-04-24T23:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.960+0000 7f8b8a7f9640 1 Processor -- start 2024-04-24T23:26:21.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.960+0000 7f8b8a7f9640 1 -- start start 2024-04-24T23:26:21.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b8a7f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8406f110 0x7f8b8414b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b8a7f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b8406f4f0 0x7f8b8415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b8a7f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b840adf80 0x7f8b8415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b8a7f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8b840afd00 con 0x7f8b8406f110 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b8a7f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8b840afa20 con 0x7f8b8406f4f0 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b8a7f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8b840aee50 con 0x7f8b840adf80 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b83fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8406f110 0x7f8b8414b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b83fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8406f110 0x7f8b8414b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56276/0 (socket says 172.21.15.26:56276) 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b83fff640 1 -- 172.21.15.26:0/3891793523 learned_addr learned my addr 172.21.15.26:0/3891793523 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b837fe640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b8406f4f0 0x7f8b8415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b88d96640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b840adf80 0x7f8b8415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b83fff640 1 -- 172.21.15.26:0/3891793523 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b840adf80 msgr2=0x7f8b8415abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b83fff640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b840adf80 0x7f8b8415abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b83fff640 1 -- 172.21.15.26:0/3891793523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b8406f4f0 msgr2=0x7f8b8415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b83fff640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b8406f4f0 0x7f8b8415a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.961+0000 7f8b83fff640 1 -- 172.21.15.26:0/3891793523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b841e81b0 con 0x7f8b8406f110 2024-04-24T23:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.962+0000 7f8b83fff640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8406f110 0x7f8b8414b310 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400e9e0 tx=0x7f8b7400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.962+0000 7f8b80ff9640 1 -- 172.21.15.26:0/3891793523 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74017070 con 0x7f8b8406f110 2024-04-24T23:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.962+0000 7f8b80ff9640 1 -- 172.21.15.26:0/3891793523 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7400c9d0 con 0x7f8b8406f110 2024-04-24T23:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.962+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b841e8490 con 0x7f8b8406f110 2024-04-24T23:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.962+0000 7f8b80ff9640 1 -- 172.21.15.26:0/3891793523 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7400cdc0 con 0x7f8b8406f110 2024-04-24T23:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.962+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b841e8a50 con 0x7f8b8406f110 2024-04-24T23:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.963+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b840694c0 con 0x7f8b8406f110 2024-04-24T23:26:21.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.963+0000 7f8b80ff9640 1 -- 172.21.15.26:0/3891793523 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b7401a070 con 0x7f8b8406f110 2024-04-24T23:26:21.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.964+0000 7f8b80ff9640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b7006be30 0x7f8b7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:21.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.964+0000 7f8b80ff9640 1 -- 172.21.15.26:0/3891793523 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8b7408b110 con 0x7f8b8406f110 2024-04-24T23:26:21.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.964+0000 7f8b837fe640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b7006be30 0x7f8b7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:21.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.964+0000 7f8b837fe640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b7006be30 0x7f8b7006e2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8b841e71f0 tx=0x7f8b78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:21.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:21.968+0000 7f8b80ff9640 1 -- 172.21.15.26:0/3891793523 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8b74059500 con 0x7f8b8406f110 2024-04-24T23:26:22.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.147+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b8406bf90 con 0x7f8b7006be30 2024-04-24T23:26:22.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.153+0000 7f8b80ff9640 1 -- 172.21.15.26:0/3891793523 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246741 (secure 0 0 0) 0x7f8b8406bf90 con 0x7f8b7006be30 2024-04-24T23:26:22.157 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:22.158 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b7006be30 msgr2=0x7f8b7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b7006be30 0x7f8b7006e2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8b841e71f0 tx=0x7f8b78009040 comp rx=0 tx=0).stop 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8406f110 msgr2=0x7f8b8414b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8406f110 0x7f8b8414b310 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400e9e0 tx=0x7f8b7400eea0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 shutdown_connections 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8b7006be30 0x7f8b7006e2f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8b840adf80 0x7f8b8415abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8b8406f4f0 0x7f8b8415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 --2- 172.21.15.26:0/3891793523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8406f110 0x7f8b8414b310 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 >> 172.21.15.26:0/3891793523 conn(0x7f8b840729a0 msgr2=0x7f8b840b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 shutdown_connections 2024-04-24T23:26:22.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.156+0000 7f8b8a7f9640 1 -- 172.21.15.26:0/3891793523 wait complete. 2024-04-24T23:26:22.168 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-04-24T23:26:21.386191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270732,"kb_used_data":8780,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754703988,"statfs":{"total":773094113280,"available":772816883712,"internally_reserved":0,"allocated":8990720,"data_stored":4513009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.242490"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.161035+0000","last_change":"2024-04-24T23:26:17.161035+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.161035+0000","last_undegraded":"2024-04-24T23:26:17.161035+0000","last_fullsized":"2024-04-24T23:26:17.161035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152937+0000","last_change":"2024-04-24T23:26:17.152937+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152937+0000","last_undegraded":"2024-04-24T23:26:17.152937+0000","last_fullsized":"2024-04-24T23:26:17.152937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152655+0000","last_change":"2024-04-24T23:26:17.152655+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152655+0000","last_undegraded":"2024-04-24T23:26:17.152655+0000","last_fullsized":"2024-04-24T23:26:17.152655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.156363+0000","last_change":"2024-04-24T23:26:17.156363+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.156363+0000","last_undegraded":"2024-04-24T23:26:17.156363+0000","last_fullsized":"2024-04-24T23:26:17.156363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159303+0000","last_change":"2024-04-24T23:26:17.159303+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159303+0000","last_undegraded":"2024-04-24T23:26:17.159303+0000","last_fullsized":"2024-04-24T23:26:17.159303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.149287+0000","last_change":"2024-04-24T23:26:17.149287+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.149287+0000","last_undegraded":"2024-04-24T23:26:17.149287+0000","last_fullsized":"2024-04-24T23:26:17.149287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155893+0000","last_change":"2024-04-24T23:26:17.155893+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155893+0000","last_undegraded":"2024-04-24T23:26:17.155893+0000","last_fullsized":"2024-04-24T23:26:17.155893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148447+0000","last_change":"2024-04-24T23:26:17.148447+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148447+0000","last_undegraded":"2024-04-24T23:26:17.148447+0000","last_fullsized":"2024-04-24T23:26:17.148447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.149122+0000","last_change":"2024-04-24T23:26:17.149122+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.149122+0000","last_undegraded":"2024-04-24T23:26:17.149122+0000","last_fullsized":"2024-04-24T23:26:17.149122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159010+0000","last_change":"2024-04-24T23:26:17.159010+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159010+0000","last_undegraded":"2024-04-24T23:26:17.159010+0000","last_fullsized":"2024-04-24T23:26:17.159010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152125+0000","last_change":"2024-04-24T23:26:17.152125+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152125+0000","last_undegraded":"2024-04-24T23:26:17.152125+0000","last_fullsized":"2024-04-24T23:26:17.152125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.160127+0000","last_change":"2024-04-24T23:26:17.160127+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.160127+0000","last_undegraded":"2024-04-24T23:26:17.160127+0000","last_fullsized":"2024-04-24T23:26:17.160127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154328+0000","last_change":"2024-04-24T23:26:17.154328+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154328+0000","last_undegraded":"2024-04-24T23:26:17.154328+0000","last_fullsized":"2024-04-24T23:26:17.154328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154638+0000","last_change":"2024-04-24T23:26:17.154638+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154638+0000","last_undegraded":"2024-04-24T23:26:17.154638+0000","last_fullsized":"2024-04-24T23:26:17.154638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.156466+0000","last_change":"2024-04-24T23:26:17.156466+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.156466+0000","last_undegraded":"2024-04-24T23:26:17.156466+0000","last_fullsized":"2024-04-24T23:26:17.156466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158764+0000","last_change":"2024-04-24T23:26:17.158764+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158764+0000","last_undegraded":"2024-04-24T23:26:17.158764+0000","last_fullsized":"2024-04-24T23:26:17.158764+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158461+0000","last_change":"2024-04-24T23:26:17.158461+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158461+0000","last_undegraded":"2024-04-24T23:26:17.158461+0000","last_fullsized":"2024-04-24T23:26:17.158461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153010+0000","last_change":"2024-04-24T23:26:17.153010+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153010+0000","last_undegraded":"2024-04-24T23:26:17.153010+0000","last_fullsized":"2024-04-24T23:26:17.153010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159129+0000","last_change":"2024-04-24T23:26:17.159129+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159129+0000","last_undegraded":"2024-04-24T23:26:17.159129+0000","last_fullsized":"2024-04-24T23:26:17.159129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155234+0000","last_change":"2024-04-24T23:26:17.155234+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155234+0000","last_undegraded":"2024-04-24T23:26:17.155234+0000","last_fullsized":"2024-04-24T23:26:17.155234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155233+0000","last_change":"2024-04-24T23:26:17.155233+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155233+0000","last_undegraded":"2024-04-24T23:26:17.155233+0000","last_fullsized":"2024-04-24T23:26:17.155233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154445+0000","last_change":"2024-04-24T23:26:17.154445+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154445+0000","last_undegraded":"2024-04-24T23:26:17.154445+0000","last_fullsized":"2024-04-24T23:26:17.154445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159956+0000","last_change":"2024-04-24T23:26:17.159956+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159956+0000","last_undegraded":"2024-04-24T23:26:17.159956+0000","last_fullsized":"2024-04-24T23:26:17.159956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153261+0000","last_change":"2024-04-24T23:26:17.153261+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153261+0000","last_undegraded":"2024-04-24T23:26:17.153261+0000","last_fullsized":"2024-04-24T23:26:17.153261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153219+0000","last_change":"2024-04-24T23:26:17.153219+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153219+0000","last_undegraded":"2024-04-24T23:26:17.153219+0000","last_fullsized":"2024-04-24T23:26:17.153219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153550+0000","last_change":"2024-04-24T23:26:17.153550+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153550+0000","last_undegraded":"2024-04-24T23:26:17.153550+0000","last_fullsized":"2024-04-24T23:26:17.153550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159847+0000","last_change":"2024-04-24T23:26:17.159847+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159847+0000","last_undegraded":"2024-04-24T23:26:17.159847+0000","last_fullsized":"2024-04-24T23:26:17.159847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159058+0000","last_change":"2024-04-24T23:26:17.159058+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159058+0000","last_undegraded":"2024-04-24T23:26:17.159058+0000","last_fullsized":"2024-04-24T23:26:17.159058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152080+0000","last_change":"2024-04-24T23:26:17.152080+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152080+0000","last_undegraded":"2024-04-24T23:26:17.152080+0000","last_fullsized":"2024-04-24T23:26:17.152080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154029+0000","last_change":"2024-04-24T23:26:17.154029+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154029+0000","last_undegraded":"2024-04-24T23:26:17.154029+0000","last_fullsized":"2024-04-24T23:26:17.154029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155927+0000","last_change":"2024-04-24T23:26:17.155927+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155927+0000","last_undegraded":"2024-04-24T23:26:17.155927+0000","last_fullsized":"2024-04-24T23:26:17.155927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154894+0000","last_change":"2024-04-24T23:26:17.154894+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154894+0000","last_undegraded":"2024-04-24T23:26:17.154894+0000","last_fullsized":"2024-04-24T23:26:17.154894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155905+0000","last_change":"2024-04-24T23:26:17.155905+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155905+0000","last_undegraded":"2024-04-24T23:26:17.155905+0000","last_fullsized":"2024-04-24T23:26:17.155905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167739+0000","last_change":"2024-04-24T23:26:17.166716+0000","last_active":"2024-04-24T23:26:17.167739+0000","last_peered":"2024-04-24T23:26:17.167739+0000","last_clean":"2024-04-24T23:26:17.167739+0000","last_became_active":"2024-04-24T23:26:17.166397+0000","last_became_peered":"2024-04-24T23:26:17.166397+0000","last_unstale":"2024-04-24T23:26:17.167739+0000","last_undegraded":"2024-04-24T23:26:17.167739+0000","last_fullsized":"2024-04-24T23:26:17.167739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:31.957370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.160325+0000","last_change":"2024-04-24T23:26:17.160325+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.160325+0000","last_undegraded":"2024-04-24T23:26:17.160325+0000","last_fullsized":"2024-04-24T23:26:17.160325+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164745+0000","last_change":"2024-04-24T23:26:17.162927+0000","last_active":"2024-04-24T23:26:17.164745+0000","last_peered":"2024-04-24T23:26:17.164745+0000","last_clean":"2024-04-24T23:26:17.164745+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:17.164745+0000","last_undegraded":"2024-04-24T23:26:17.164745+0000","last_fullsized":"2024-04-24T23:26:17.164745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:25.563184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158589+0000","last_change":"2024-04-24T23:26:17.158589+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158589+0000","last_undegraded":"2024-04-24T23:26:17.158589+0000","last_fullsized":"2024-04-24T23:26:17.158589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170289+0000","last_change":"2024-04-24T23:26:17.166726+0000","last_active":"2024-04-24T23:26:17.170289+0000","last_peered":"2024-04-24T23:26:17.170289+0000","last_clean":"2024-04-24T23:26:17.170289+0000","last_became_active":"2024-04-24T23:26:17.166370+0000","last_became_peered":"2024-04-24T23:26:17.166370+0000","last_unstale":"2024-04-24T23:26:17.170289+0000","last_undegraded":"2024-04-24T23:26:17.170289+0000","last_fullsized":"2024-04-24T23:26:17.170289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:28.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152137+0000","last_change":"2024-04-24T23:26:17.152137+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152137+0000","last_undegraded":"2024-04-24T23:26:17.152137+0000","last_fullsized":"2024-04-24T23:26:17.152137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170551+0000","last_change":"2024-04-24T23:26:17.167115+0000","last_active":"2024-04-24T23:26:17.170551+0000","last_peered":"2024-04-24T23:26:17.170551+0000","last_clean":"2024-04-24T23:26:17.170551+0000","last_became_active":"2024-04-24T23:26:17.166838+0000","last_became_peered":"2024-04-24T23:26:17.166838+0000","last_unstale":"2024-04-24T23:26:17.170551+0000","last_undegraded":"2024-04-24T23:26:17.170551+0000","last_fullsized":"2024-04-24T23:26:17.170551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:39.845561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153602+0000","last_change":"2024-04-24T23:26:17.153602+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153602+0000","last_undegraded":"2024-04-24T23:26:17.153602+0000","last_fullsized":"2024-04-24T23:26:17.153602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167567+0000","last_change":"2024-04-24T23:26:17.166049+0000","last_active":"2024-04-24T23:26:17.167567+0000","last_peered":"2024-04-24T23:26:17.167567+0000","last_clean":"2024-04-24T23:26:17.167567+0000","last_became_active":"2024-04-24T23:26:17.165763+0000","last_became_peered":"2024-04-24T23:26:17.165763+0000","last_unstale":"2024-04-24T23:26:17.167567+0000","last_undegraded":"2024-04-24T23:26:17.167567+0000","last_fullsized":"2024-04-24T23:26:17.167567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:40.656153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155159+0000","last_change":"2024-04-24T23:26:17.155159+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155159+0000","last_undegraded":"2024-04-24T23:26:17.155159+0000","last_fullsized":"2024-04-24T23:26:17.155159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.163167+0000","last_change":"2024-04-24T23:26:17.161576+0000","last_active":"2024-04-24T23:26:17.163167+0000","last_peered":"2024-04-24T23:26:17.163167+0000","last_clean":"2024-04-24T23:26:17.163167+0000","last_became_active":"2024-04-24T23:26:17.161260+0000","last_became_peered":"2024-04-24T23:26:17.161260+0000","last_unstale":"2024-04-24T23:26:17.163167+0000","last_undegraded":"2024-04-24T23:26:17.163167+0000","last_fullsized":"2024-04-24T23:26:17.163167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:00.568912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152635+0000","last_change":"2024-04-24T23:26:17.152635+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152635+0000","last_undegraded":"2024-04-24T23:26:17.152635+0000","last_fullsized":"2024-04-24T23:26:17.152635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170205+0000","last_change":"2024-04-24T23:26:17.167646+0000","last_active":"2024-04-24T23:26:17.170205+0000","last_peered":"2024-04-24T23:26:17.170205+0000","last_clean":"2024-04-24T23:26:17.170205+0000","last_became_active":"2024-04-24T23:26:17.167351+0000","last_became_peered":"2024-04-24T23:26:17.167351+0000","last_unstale":"2024-04-24T23:26:17.170205+0000","last_undegraded":"2024-04-24T23:26:17.170205+0000","last_fullsized":"2024-04-24T23:26:17.170205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:39.409219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.151934+0000","last_change":"2024-04-24T23:26:17.151934+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.151934+0000","last_undegraded":"2024-04-24T23:26:17.151934+0000","last_fullsized":"2024-04-24T23:26:17.151934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.165107+0000","last_change":"2024-04-24T23:26:17.162901+0000","last_active":"2024-04-24T23:26:17.165107+0000","last_peered":"2024-04-24T23:26:17.165107+0000","last_clean":"2024-04-24T23:26:17.165107+0000","last_became_active":"2024-04-24T23:26:17.162604+0000","last_became_peered":"2024-04-24T23:26:17.162604+0000","last_unstale":"2024-04-24T23:26:17.165107+0000","last_undegraded":"2024-04-24T23:26:17.165107+0000","last_fullsized":"2024-04-24T23:26:17.165107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:37.984293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.151319+0000","last_change":"2024-04-24T23:26:17.151319+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.151319+0000","last_undegraded":"2024-04-24T23:26:17.151319+0000","last_fullsized":"2024-04-24T23:26:17.151319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170103+0000","last_change":"2024-04-24T23:26:17.167550+0000","last_active":"2024-04-24T23:26:17.170103+0000","last_peered":"2024-04-24T23:26:17.170103+0000","last_clean":"2024-04-24T23:26:17.170103+0000","last_became_active":"2024-04-24T23:26:17.167229+0000","last_became_peered":"2024-04-24T23:26:17.167229+0000","last_unstale":"2024-04-24T23:26:17.170103+0000","last_undegraded":"2024-04-24T23:26:17.170103+0000","last_fullsized":"2024-04-24T23:26:17.170103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:30.940114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148656+0000","last_change":"2024-04-24T23:26:17.148656+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148656+0000","last_undegraded":"2024-04-24T23:26:17.148656+0000","last_fullsized":"2024-04-24T23:26:17.148656+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170242+0000","last_change":"2024-04-24T23:26:17.168677+0000","last_active":"2024-04-24T23:26:17.170242+0000","last_peered":"2024-04-24T23:26:17.170242+0000","last_clean":"2024-04-24T23:26:17.170242+0000","last_became_active":"2024-04-24T23:26:17.168354+0000","last_became_peered":"2024-04-24T23:26:17.168354+0000","last_unstale":"2024-04-24T23:26:17.170242+0000","last_undegraded":"2024-04-24T23:26:17.170242+0000","last_fullsized":"2024-04-24T23:26:17.170242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:59.598138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.160360+0000","last_change":"2024-04-24T23:26:17.160360+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.160360+0000","last_undegraded":"2024-04-24T23:26:17.160360+0000","last_fullsized":"2024-04-24T23:26:17.160360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170673+0000","last_change":"2024-04-24T23:26:17.170110+0000","last_active":"2024-04-24T23:26:17.170673+0000","last_peered":"2024-04-24T23:26:17.170673+0000","last_clean":"2024-04-24T23:26:17.170673+0000","last_became_active":"2024-04-24T23:26:17.169755+0000","last_became_peered":"2024-04-24T23:26:17.169755+0000","last_unstale":"2024-04-24T23:26:17.170673+0000","last_undegraded":"2024-04-24T23:26:17.170673+0000","last_fullsized":"2024-04-24T23:26:17.170673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:03.234791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154473+0000","last_change":"2024-04-24T23:26:17.154473+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154473+0000","last_undegraded":"2024-04-24T23:26:17.154473+0000","last_fullsized":"2024-04-24T23:26:17.154473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.152470+0000","last_change":"2024-04-24T23:26:17.151318+0000","last_active":"2024-04-24T23:26:17.152470+0000","last_peered":"2024-04-24T23:26:17.152470+0000","last_clean":"2024-04-24T23:26:17.152470+0000","last_became_active":"2024-04-24T23:26:17.151022+0000","last_became_peered":"2024-04-24T23:26:17.151022+0000","last_unstale":"2024-04-24T23:26:17.152470+0000","last_undegraded":"2024-04-24T23:26:17.152470+0000","last_fullsized":"2024-04-24T23:26:17.152470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:25.593578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.156618+0000","last_change":"2024-04-24T23:26:17.156618+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.156618+0000","last_undegraded":"2024-04-24T23:26:17.156618+0000","last_fullsized":"2024-04-24T23:26:17.156618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.165078+0000","last_change":"2024-04-24T23:26:17.162943+0000","last_active":"2024-04-24T23:26:17.165078+0000","last_peered":"2024-04-24T23:26:17.165078+0000","last_clean":"2024-04-24T23:26:17.165078+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:17.165078+0000","last_undegraded":"2024-04-24T23:26:17.165078+0000","last_fullsized":"2024-04-24T23:26:17.165078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:59.927814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153902+0000","last_change":"2024-04-24T23:26:17.153902+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153902+0000","last_undegraded":"2024-04-24T23:26:17.153902+0000","last_fullsized":"2024-04-24T23:26:17.153902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164541+0000","last_change":"2024-04-24T23:26:17.163630+0000","last_active":"2024-04-24T23:26:17.164541+0000","last_peered":"2024-04-24T23:26:17.164541+0000","last_clean":"2024-04-24T23:26:17.164541+0000","last_became_active":"2024-04-24T23:26:17.163331+0000","last_became_peered":"2024-04-24T23:26:17.163331+0000","last_unstale":"2024-04-24T23:26:17.164541+0000","last_undegraded":"2024-04-24T23:26:17.164541+0000","last_fullsized":"2024-04-24T23:26:17.164541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:47.984272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152815+0000","last_change":"2024-04-24T23:26:17.152815+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152815+0000","last_undegraded":"2024-04-24T23:26:17.152815+0000","last_fullsized":"2024-04-24T23:26:17.152815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.161936+0000","last_change":"2024-04-24T23:26:17.160593+0000","last_active":"2024-04-24T23:26:17.161936+0000","last_peered":"2024-04-24T23:26:17.161936+0000","last_clean":"2024-04-24T23:26:17.161936+0000","last_became_active":"2024-04-24T23:26:17.160303+0000","last_became_peered":"2024-04-24T23:26:17.160303+0000","last_unstale":"2024-04-24T23:26:17.161936+0000","last_undegraded":"2024-04-24T23:26:17.161936+0000","last_fullsized":"2024-04-24T23:26:17.161936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:18.000135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.151917+0000","last_change":"2024-04-24T23:26:17.151917+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.151917+0000","last_undegraded":"2024-04-24T23:26:17.151917+0000","last_fullsized":"2024-04-24T23:26:17.151917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170145+0000","last_change":"2024-04-24T23:26:17.165945+0000","last_active":"2024-04-24T23:26:17.170145+0000","last_peered":"2024-04-24T23:26:17.170145+0000","last_clean":"2024-04-24T23:26:17.170145+0000","last_became_active":"2024-04-24T23:26:17.164750+0000","last_became_peered":"2024-04-24T23:26:17.164750+0000","last_unstale":"2024-04-24T23:26:17.170145+0000","last_undegraded":"2024-04-24T23:26:17.170145+0000","last_fullsized":"2024-04-24T23:26:17.170145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:55.620251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153107+0000","last_change":"2024-04-24T23:26:17.153107+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153107+0000","last_undegraded":"2024-04-24T23:26:17.153107+0000","last_fullsized":"2024-04-24T23:26:17.153107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167256+0000","last_change":"2024-04-24T23:26:17.165977+0000","last_active":"2024-04-24T23:26:17.167256+0000","last_peered":"2024-04-24T23:26:17.167256+0000","last_clean":"2024-04-24T23:26:17.167256+0000","last_became_active":"2024-04-24T23:26:17.165641+0000","last_became_peered":"2024-04-24T23:26:17.165641+0000","last_unstale":"2024-04-24T23:26:17.167256+0000","last_undegraded":"2024-04-24T23:26:17.167256+0000","last_fullsized":"2024-04-24T23:26:17.167256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:22.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159384+0000","last_change":"2024-04-24T23:26:17.159384+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159384+0000","last_undegraded":"2024-04-24T23:26:17.159384+0000","last_fullsized":"2024-04-24T23:26:17.159384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170579+0000","last_change":"2024-04-24T23:26:17.163913+0000","last_active":"2024-04-24T23:26:17.170579+0000","last_peered":"2024-04-24T23:26:17.170579+0000","last_clean":"2024-04-24T23:26:17.170579+0000","last_became_active":"2024-04-24T23:26:17.163640+0000","last_became_peered":"2024-04-24T23:26:17.163640+0000","last_unstale":"2024-04-24T23:26:17.170579+0000","last_undegraded":"2024-04-24T23:26:17.170579+0000","last_fullsized":"2024-04-24T23:26:17.170579+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:18.000135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153797+0000","last_change":"2024-04-24T23:26:17.153797+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153797+0000","last_undegraded":"2024-04-24T23:26:17.153797+0000","last_fullsized":"2024-04-24T23:26:17.153797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170573+0000","last_change":"2024-04-24T23:26:17.167926+0000","last_active":"2024-04-24T23:26:17.170573+0000","last_peered":"2024-04-24T23:26:17.170573+0000","last_clean":"2024-04-24T23:26:17.170573+0000","last_became_active":"2024-04-24T23:26:17.167660+0000","last_became_peered":"2024-04-24T23:26:17.167660+0000","last_unstale":"2024-04-24T23:26:17.170573+0000","last_undegraded":"2024-04-24T23:26:17.170573+0000","last_fullsized":"2024-04-24T23:26:17.170573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:54.496665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154193+0000","last_change":"2024-04-24T23:26:17.154193+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154193+0000","last_undegraded":"2024-04-24T23:26:17.154193+0000","last_fullsized":"2024-04-24T23:26:17.154193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.161464+0000","last_change":"2024-04-24T23:26:17.158322+0000","last_active":"2024-04-24T23:26:17.161464+0000","last_peered":"2024-04-24T23:26:17.161464+0000","last_clean":"2024-04-24T23:26:17.161464+0000","last_became_active":"2024-04-24T23:26:17.158052+0000","last_became_peered":"2024-04-24T23:26:17.158052+0000","last_unstale":"2024-04-24T23:26:17.161464+0000","last_undegraded":"2024-04-24T23:26:17.161464+0000","last_fullsized":"2024-04-24T23:26:17.161464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:07.133490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148457+0000","last_change":"2024-04-24T23:26:17.148457+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148457+0000","last_undegraded":"2024-04-24T23:26:17.148457+0000","last_fullsized":"2024-04-24T23:26:17.148457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.169586+0000","last_change":"2024-04-24T23:26:17.167861+0000","last_active":"2024-04-24T23:26:17.169586+0000","last_peered":"2024-04-24T23:26:17.169586+0000","last_clean":"2024-04-24T23:26:17.169586+0000","last_became_active":"2024-04-24T23:26:17.167598+0000","last_became_peered":"2024-04-24T23:26:17.167598+0000","last_unstale":"2024-04-24T23:26:17.169586+0000","last_undegraded":"2024-04-24T23:26:17.169586+0000","last_fullsized":"2024-04-24T23:26:17.169586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:27.969964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152735+0000","last_change":"2024-04-24T23:26:17.152735+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152735+0000","last_undegraded":"2024-04-24T23:26:17.152735+0000","last_fullsized":"2024-04-24T23:26:17.152735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.168211+0000","last_change":"2024-04-24T23:26:17.165956+0000","last_active":"2024-04-24T23:26:17.168211+0000","last_peered":"2024-04-24T23:26:17.168211+0000","last_clean":"2024-04-24T23:26:17.168211+0000","last_became_active":"2024-04-24T23:26:17.165633+0000","last_became_peered":"2024-04-24T23:26:17.165633+0000","last_unstale":"2024-04-24T23:26:17.168211+0000","last_undegraded":"2024-04-24T23:26:17.168211+0000","last_fullsized":"2024-04-24T23:26:17.168211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:24.199382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.149169+0000","last_change":"2024-04-24T23:26:17.149169+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.149169+0000","last_undegraded":"2024-04-24T23:26:17.149169+0000","last_fullsized":"2024-04-24T23:26:17.149169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.161402+0000","last_change":"2024-04-24T23:26:17.157597+0000","last_active":"2024-04-24T23:26:17.161402+0000","last_peered":"2024-04-24T23:26:17.161402+0000","last_clean":"2024-04-24T23:26:17.161402+0000","last_became_active":"2024-04-24T23:26:17.157311+0000","last_became_peered":"2024-04-24T23:26:17.157311+0000","last_unstale":"2024-04-24T23:26:17.161402+0000","last_undegraded":"2024-04-24T23:26:17.161402+0000","last_fullsized":"2024-04-24T23:26:17.161402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:18.468882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152830+0000","last_change":"2024-04-24T23:26:17.152830+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152830+0000","last_undegraded":"2024-04-24T23:26:17.152830+0000","last_fullsized":"2024-04-24T23:26:17.152830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.165174+0000","last_change":"2024-04-24T23:26:17.163231+0000","last_active":"2024-04-24T23:26:17.165174+0000","last_peered":"2024-04-24T23:26:17.165174+0000","last_clean":"2024-04-24T23:26:17.165174+0000","last_became_active":"2024-04-24T23:26:17.162790+0000","last_became_peered":"2024-04-24T23:26:17.162790+0000","last_unstale":"2024-04-24T23:26:17.165174+0000","last_undegraded":"2024-04-24T23:26:17.165174+0000","last_fullsized":"2024-04-24T23:26:17.165174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:42.373513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154400+0000","last_change":"2024-04-24T23:26:17.154400+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154400+0000","last_undegraded":"2024-04-24T23:26:17.154400+0000","last_fullsized":"2024-04-24T23:26:17.154400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164991+0000","last_change":"2024-04-24T23:26:17.161814+0000","last_active":"2024-04-24T23:26:17.164991+0000","last_peered":"2024-04-24T23:26:17.164991+0000","last_clean":"2024-04-24T23:26:17.164991+0000","last_became_active":"2024-04-24T23:26:17.161485+0000","last_became_peered":"2024-04-24T23:26:17.161485+0000","last_unstale":"2024-04-24T23:26:17.164991+0000","last_undegraded":"2024-04-24T23:26:17.164991+0000","last_fullsized":"2024-04-24T23:26:17.164991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:18.807328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158100+0000","last_change":"2024-04-24T23:26:17.158100+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158100+0000","last_undegraded":"2024-04-24T23:26:17.158100+0000","last_fullsized":"2024-04-24T23:26:17.158100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170350+0000","last_change":"2024-04-24T23:26:17.169621+0000","last_active":"2024-04-24T23:26:17.170350+0000","last_peered":"2024-04-24T23:26:17.170350+0000","last_clean":"2024-04-24T23:26:17.170350+0000","last_became_active":"2024-04-24T23:26:17.169195+0000","last_became_peered":"2024-04-24T23:26:17.169195+0000","last_unstale":"2024-04-24T23:26:17.170350+0000","last_undegraded":"2024-04-24T23:26:17.170350+0000","last_fullsized":"2024-04-24T23:26:17.170350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:42.773783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153079+0000","last_change":"2024-04-24T23:26:17.153079+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153079+0000","last_undegraded":"2024-04-24T23:26:17.153079+0000","last_fullsized":"2024-04-24T23:26:17.153079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170341+0000","last_change":"2024-04-24T23:26:17.169611+0000","last_active":"2024-04-24T23:26:17.170341+0000","last_peered":"2024-04-24T23:26:17.170341+0000","last_clean":"2024-04-24T23:26:17.170341+0000","last_became_active":"2024-04-24T23:26:17.169304+0000","last_became_peered":"2024-04-24T23:26:17.169304+0000","last_unstale":"2024-04-24T23:26:17.170341+0000","last_undegraded":"2024-04-24T23:26:17.170341+0000","last_fullsized":"2024-04-24T23:26:17.170341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:21.760106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148445+0000","last_change":"2024-04-24T23:26:17.148445+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148445+0000","last_undegraded":"2024-04-24T23:26:17.148445+0000","last_fullsized":"2024-04-24T23:26:17.148445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170099+0000","last_change":"2024-04-24T23:26:17.168469+0000","last_active":"2024-04-24T23:26:17.170099+0000","last_peered":"2024-04-24T23:26:17.170099+0000","last_clean":"2024-04-24T23:26:17.170099+0000","last_became_active":"2024-04-24T23:26:17.167305+0000","last_became_peered":"2024-04-24T23:26:17.167305+0000","last_unstale":"2024-04-24T23:26:17.170099+0000","last_undegraded":"2024-04-24T23:26:17.170099+0000","last_fullsized":"2024-04-24T23:26:17.170099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:08.429123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159242+0000","last_change":"2024-04-24T23:26:17.159242+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159242+0000","last_undegraded":"2024-04-24T23:26:17.159242+0000","last_fullsized":"2024-04-24T23:26:17.159242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164655+0000","last_change":"2024-04-24T23:26:17.162968+0000","last_active":"2024-04-24T23:26:17.164655+0000","last_peered":"2024-04-24T23:26:17.164655+0000","last_clean":"2024-04-24T23:26:17.164655+0000","last_became_active":"2024-04-24T23:26:17.162691+0000","last_became_peered":"2024-04-24T23:26:17.162691+0000","last_unstale":"2024-04-24T23:26:17.164655+0000","last_undegraded":"2024-04-24T23:26:17.164655+0000","last_fullsized":"2024-04-24T23:26:17.164655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.997763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152026+0000","last_change":"2024-04-24T23:26:17.152026+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152026+0000","last_undegraded":"2024-04-24T23:26:17.152026+0000","last_fullsized":"2024-04-24T23:26:17.152026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170205+0000","last_change":"2024-04-24T23:26:17.168302+0000","last_active":"2024-04-24T23:26:17.170205+0000","last_peered":"2024-04-24T23:26:17.170205+0000","last_clean":"2024-04-24T23:26:17.170205+0000","last_became_active":"2024-04-24T23:26:17.167711+0000","last_became_peered":"2024-04-24T23:26:17.167711+0000","last_unstale":"2024-04-24T23:26:17.170205+0000","last_undegraded":"2024-04-24T23:26:17.170205+0000","last_fullsized":"2024-04-24T23:26:17.170205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:01.210692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159455+0000","last_change":"2024-04-24T23:26:17.159455+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159455+0000","last_undegraded":"2024-04-24T23:26:17.159455+0000","last_fullsized":"2024-04-24T23:26:17.159455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.159915+0000","last_change":"2024-04-24T23:26:17.156009+0000","last_active":"2024-04-24T23:26:17.159915+0000","last_peered":"2024-04-24T23:26:17.159915+0000","last_clean":"2024-04-24T23:26:17.159915+0000","last_became_active":"2024-04-24T23:26:17.155739+0000","last_became_peered":"2024-04-24T23:26:17.155739+0000","last_unstale":"2024-04-24T23:26:17.159915+0000","last_undegraded":"2024-04-24T23:26:17.159915+0000","last_fullsized":"2024-04-24T23:26:17.159915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:59.737718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153256+0000","last_change":"2024-04-24T23:26:17.153256+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153256+0000","last_undegraded":"2024-04-24T23:26:17.153256+0000","last_fullsized":"2024-04-24T23:26:17.153256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167714+0000","last_change":"2024-04-24T23:26:17.166166+0000","last_active":"2024-04-24T23:26:17.167714+0000","last_peered":"2024-04-24T23:26:17.167714+0000","last_clean":"2024-04-24T23:26:17.167714+0000","last_became_active":"2024-04-24T23:26:17.165855+0000","last_became_peered":"2024-04-24T23:26:17.165855+0000","last_unstale":"2024-04-24T23:26:17.167714+0000","last_undegraded":"2024-04-24T23:26:17.167714+0000","last_fullsized":"2024-04-24T23:26:17.167714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:23.185199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":278,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.155269+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:26:17.155269+0000","last_peered":"2024-04-24T23:26:17.155269+0000","last_clean":"2024-04-24T23:26:17.155269+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:26:17.155269+0000","last_undegraded":"2024-04-24T23:26:17.155269+0000","last_fullsized":"2024-04-24T23:26:17.155269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1564672,"data_stored":1004549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":420456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335544,"statfs":{"total":96636764160,"available":96599597056,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":983040,"data_stored":420456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1576960,"data_stored":1010824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:26:22.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:26:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.309+0000 7f8270ea7640 1 Processor -- start 2024-04-24T23:26:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.309+0000 7f8270ea7640 1 -- start start 2024-04-24T23:26:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.310+0000 7f8270ea7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c0d4870 0x7f826c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.310+0000 7f8270ea7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f826c0b3130 0x7f826c0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.310+0000 7f8270ea7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826c0d32f0 con 0x7f826c15e570 2024-04-24T23:26:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.310+0000 7f8270ea7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f826c0d1ab0 con 0x7f826c0b3130 2024-04-24T23:26:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.310+0000 7f8270ea7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f826c0d2ae0 con 0x7f826c0d4870 2024-04-24T23:26:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.311+0000 7f8269d9b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c0d4870 0x7f826c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.311+0000 7f826ad9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f826c0b3130 0x7f826c0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.311+0000 7f826a59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f826c15e570 0x7f826c0d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:53674/0 (socket says 172.21.15.26:53674) 2024-04-24T23:26:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.311+0000 7f826a59c640 1 -- 172.21.15.26:0/1552613201 learned_addr learned my addr 172.21.15.26:0/1552613201 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.311+0000 7f826ad9d640 1 -- 172.21.15.26:0/1552613201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c0d4870 msgr2=0x7f826c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.311+0000 7f826ad9d640 1 --2- 172.21.15.26:0/1552613201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c0d4870 0x7f826c0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.311+0000 7f826ad9d640 1 -- 172.21.15.26:0/1552613201 >> v1:172.21.15.29:6789/0 conn(0x7f826c15e570 legacy=0x7f826c0d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.311+0000 7f826ad9d640 1 -- 172.21.15.26:0/1552613201 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c0d0270 con 0x7f826c0b3130 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f826ad9d640 1 --2- 172.21.15.26:0/1552613201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c0b3130 0x7f826c0b4640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f825c0097a0 tx=0x7f825c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c020685a1ec78d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f826959a640 1 -- 172.21.15.26:0/1552613201 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c037070 con 0x7f826c0b3130 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f826959a640 1 -- 172.21.15.26:0/1552613201 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825c032440 con 0x7f826c0b3130 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f826959a640 1 -- 172.21.15.26:0/1552613201 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c032810 con 0x7f826c0b3130 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f8270ea7640 1 -- 172.21.15.26:0/1552613201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c0b3130 msgr2=0x7f826c0b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f8270ea7640 1 --2- 172.21.15.26:0/1552613201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c0b3130 0x7f826c0b4640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f825c0097a0 tx=0x7f825c02a180 comp rx=0 tx=0).stop 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f8270ea7640 1 -- 172.21.15.26:0/1552613201 shutdown_connections 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f8270ea7640 1 --2- 172.21.15.26:0/1552613201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c0b3130 0x7f826c0b4640 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f8270ea7640 1 --2- 172.21.15.26:0/1552613201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c0d4870 0x7f826c0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.312+0000 7f8270ea7640 1 -- 172.21.15.26:0/1552613201 >> 172.21.15.26:0/1552613201 conn(0x7f826c08dbe0 msgr2=0x7f826c09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.313+0000 7f8270ea7640 1 -- 172.21.15.26:0/1552613201 shutdown_connections 2024-04-24T23:26:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.313+0000 7f8270ea7640 1 -- 172.21.15.26:0/1552613201 wait complete. 2024-04-24T23:26:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.314+0000 7f8270ea7640 1 Processor -- start 2024-04-24T23:26:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.314+0000 7f8270ea7640 1 -- start start 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8270ea7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c0b3130 0x7f826c0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8270ea7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f826c0d4870 0x7f826c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8270ea7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c15e570 0x7f826c161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8270ea7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0cfa70 con 0x7f826c0d4870 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8270ea7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0d3010 con 0x7f826c0b3130 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8270ea7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f826c0cfd50 con 0x7f826c15e570 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8269d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f826c0d4870 0x7f826c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8269d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f826c0d4870 0x7f826c1525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56282/0 (socket says 172.21.15.26:56282) 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8269d9b640 1 -- 172.21.15.26:0/971167179 learned_addr learned my addr 172.21.15.26:0/971167179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f826a59c640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c0b3130 0x7f826c0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f826ad9d640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c15e570 0x7f826c161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8269d9b640 1 -- 172.21.15.26:0/971167179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c15e570 msgr2=0x7f826c161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8269d9b640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c15e570 0x7f826c161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8269d9b640 1 -- 172.21.15.26:0/971167179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c0b3130 msgr2=0x7f826c0cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8269d9b640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c0b3130 0x7f826c0cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8269d9b640 1 -- 172.21.15.26:0/971167179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c1f7860 con 0x7f826c0d4870 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f8269d9b640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f826c0d4870 0x7f826c1525c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f826000e9b0 tx=0x7f826000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.315+0000 7f825affd640 1 -- 172.21.15.26:0/971167179 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260017070 con 0x7f826c0d4870 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.316+0000 7f825affd640 1 -- 172.21.15.26:0/971167179 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826000c9a0 con 0x7f826c0d4870 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.316+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c1f7b40 con 0x7f826c0d4870 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.316+0000 7f825affd640 1 -- 172.21.15.26:0/971167179 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826000cd90 con 0x7f826c0d4870 2024-04-24T23:26:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.316+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c1f8100 con 0x7f826c0d4870 2024-04-24T23:26:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.317+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826c0694c0 con 0x7f826c0d4870 2024-04-24T23:26:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.317+0000 7f825affd640 1 -- 172.21.15.26:0/971167179 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f826001a070 con 0x7f826c0d4870 2024-04-24T23:26:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.317+0000 7f825affd640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f824c06be30 0x7f824c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.318+0000 7f825affd640 1 -- 172.21.15.26:0/971167179 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f826008ae00 con 0x7f826c0d4870 2024-04-24T23:26:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.318+0000 7f826a59c640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f824c06be30 0x7f824c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.318+0000 7f826a59c640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f824c06be30 0x7f824c06e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f82640012e0 tx=0x7f8264009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.322+0000 7f825affd640 1 -- 172.21.15.26:0/971167179 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8260059840 con 0x7f826c0d4870 2024-04-24T23:26:22.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.501+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f826c0caca0 con 0x7f824c06be30 2024-04-24T23:26:22.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.507+0000 7f825affd640 1 -- 172.21.15.26:0/971167179 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246741 (secure 0 0 0) 0x7f826c0caca0 con 0x7f824c06be30 2024-04-24T23:26:22.511 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:26:22.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f824c06be30 msgr2=0x7f824c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f824c06be30 0x7f824c06e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f82640012e0 tx=0x7f8264009040 comp rx=0 tx=0).stop 2024-04-24T23:26:22.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f826c0d4870 msgr2=0x7f826c1525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f826c0d4870 0x7f826c1525c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f826000e9b0 tx=0x7f826000ee70 comp rx=0 tx=0).stop 2024-04-24T23:26:22.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 shutdown_connections 2024-04-24T23:26:22.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f824c06be30 0x7f824c06e2f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f826c15e570 0x7f826c161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f826c0d4870 0x7f826c1525c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 --2- 172.21.15.26:0/971167179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f826c0b3130 0x7f826c0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 >> 172.21.15.26:0/971167179 conn(0x7f826c08dbe0 msgr2=0x7f826c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:22.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 shutdown_connections 2024-04-24T23:26:22.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.510+0000 7f8270ea7640 1 -- 172.21.15.26:0/971167179 wait complete. 2024-04-24T23:26:22.522 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-04-24T23:26:21.386191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270732,"kb_used_data":8780,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754703988,"statfs":{"total":773094113280,"available":772816883712,"internally_reserved":0,"allocated":8990720,"data_stored":4513009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.242490"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.161035+0000","last_change":"2024-04-24T23:26:17.161035+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.161035+0000","last_undegraded":"2024-04-24T23:26:17.161035+0000","last_fullsized":"2024-04-24T23:26:17.161035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152937+0000","last_change":"2024-04-24T23:26:17.152937+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152937+0000","last_undegraded":"2024-04-24T23:26:17.152937+0000","last_fullsized":"2024-04-24T23:26:17.152937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152655+0000","last_change":"2024-04-24T23:26:17.152655+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152655+0000","last_undegraded":"2024-04-24T23:26:17.152655+0000","last_fullsized":"2024-04-24T23:26:17.152655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.156363+0000","last_change":"2024-04-24T23:26:17.156363+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.156363+0000","last_undegraded":"2024-04-24T23:26:17.156363+0000","last_fullsized":"2024-04-24T23:26:17.156363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159303+0000","last_change":"2024-04-24T23:26:17.159303+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159303+0000","last_undegraded":"2024-04-24T23:26:17.159303+0000","last_fullsized":"2024-04-24T23:26:17.159303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.149287+0000","last_change":"2024-04-24T23:26:17.149287+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.149287+0000","last_undegraded":"2024-04-24T23:26:17.149287+0000","last_fullsized":"2024-04-24T23:26:17.149287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155893+0000","last_change":"2024-04-24T23:26:17.155893+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155893+0000","last_undegraded":"2024-04-24T23:26:17.155893+0000","last_fullsized":"2024-04-24T23:26:17.155893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148447+0000","last_change":"2024-04-24T23:26:17.148447+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148447+0000","last_undegraded":"2024-04-24T23:26:17.148447+0000","last_fullsized":"2024-04-24T23:26:17.148447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.149122+0000","last_change":"2024-04-24T23:26:17.149122+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.149122+0000","last_undegraded":"2024-04-24T23:26:17.149122+0000","last_fullsized":"2024-04-24T23:26:17.149122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159010+0000","last_change":"2024-04-24T23:26:17.159010+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159010+0000","last_undegraded":"2024-04-24T23:26:17.159010+0000","last_fullsized":"2024-04-24T23:26:17.159010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152125+0000","last_change":"2024-04-24T23:26:17.152125+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152125+0000","last_undegraded":"2024-04-24T23:26:17.152125+0000","last_fullsized":"2024-04-24T23:26:17.152125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.160127+0000","last_change":"2024-04-24T23:26:17.160127+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.160127+0000","last_undegraded":"2024-04-24T23:26:17.160127+0000","last_fullsized":"2024-04-24T23:26:17.160127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154328+0000","last_change":"2024-04-24T23:26:17.154328+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154328+0000","last_undegraded":"2024-04-24T23:26:17.154328+0000","last_fullsized":"2024-04-24T23:26:17.154328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154638+0000","last_change":"2024-04-24T23:26:17.154638+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154638+0000","last_undegraded":"2024-04-24T23:26:17.154638+0000","last_fullsized":"2024-04-24T23:26:17.154638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.156466+0000","last_change":"2024-04-24T23:26:17.156466+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.156466+0000","last_undegraded":"2024-04-24T23:26:17.156466+0000","last_fullsized":"2024-04-24T23:26:17.156466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158764+0000","last_change":"2024-04-24T23:26:17.158764+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158764+0000","last_undegraded":"2024-04-24T23:26:17.158764+0000","last_fullsized":"2024-04-24T23:26:17.158764+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158461+0000","last_change":"2024-04-24T23:26:17.158461+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158461+0000","last_undegraded":"2024-04-24T23:26:17.158461+0000","last_fullsized":"2024-04-24T23:26:17.158461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153010+0000","last_change":"2024-04-24T23:26:17.153010+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153010+0000","last_undegraded":"2024-04-24T23:26:17.153010+0000","last_fullsized":"2024-04-24T23:26:17.153010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159129+0000","last_change":"2024-04-24T23:26:17.159129+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159129+0000","last_undegraded":"2024-04-24T23:26:17.159129+0000","last_fullsized":"2024-04-24T23:26:17.159129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155234+0000","last_change":"2024-04-24T23:26:17.155234+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155234+0000","last_undegraded":"2024-04-24T23:26:17.155234+0000","last_fullsized":"2024-04-24T23:26:17.155234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155233+0000","last_change":"2024-04-24T23:26:17.155233+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155233+0000","last_undegraded":"2024-04-24T23:26:17.155233+0000","last_fullsized":"2024-04-24T23:26:17.155233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154445+0000","last_change":"2024-04-24T23:26:17.154445+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154445+0000","last_undegraded":"2024-04-24T23:26:17.154445+0000","last_fullsized":"2024-04-24T23:26:17.154445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159956+0000","last_change":"2024-04-24T23:26:17.159956+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159956+0000","last_undegraded":"2024-04-24T23:26:17.159956+0000","last_fullsized":"2024-04-24T23:26:17.159956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153261+0000","last_change":"2024-04-24T23:26:17.153261+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153261+0000","last_undegraded":"2024-04-24T23:26:17.153261+0000","last_fullsized":"2024-04-24T23:26:17.153261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153219+0000","last_change":"2024-04-24T23:26:17.153219+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153219+0000","last_undegraded":"2024-04-24T23:26:17.153219+0000","last_fullsized":"2024-04-24T23:26:17.153219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153550+0000","last_change":"2024-04-24T23:26:17.153550+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153550+0000","last_undegraded":"2024-04-24T23:26:17.153550+0000","last_fullsized":"2024-04-24T23:26:17.153550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159847+0000","last_change":"2024-04-24T23:26:17.159847+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159847+0000","last_undegraded":"2024-04-24T23:26:17.159847+0000","last_fullsized":"2024-04-24T23:26:17.159847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159058+0000","last_change":"2024-04-24T23:26:17.159058+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159058+0000","last_undegraded":"2024-04-24T23:26:17.159058+0000","last_fullsized":"2024-04-24T23:26:17.159058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152080+0000","last_change":"2024-04-24T23:26:17.152080+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152080+0000","last_undegraded":"2024-04-24T23:26:17.152080+0000","last_fullsized":"2024-04-24T23:26:17.152080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154029+0000","last_change":"2024-04-24T23:26:17.154029+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154029+0000","last_undegraded":"2024-04-24T23:26:17.154029+0000","last_fullsized":"2024-04-24T23:26:17.154029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155927+0000","last_change":"2024-04-24T23:26:17.155927+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155927+0000","last_undegraded":"2024-04-24T23:26:17.155927+0000","last_fullsized":"2024-04-24T23:26:17.155927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154894+0000","last_change":"2024-04-24T23:26:17.154894+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154894+0000","last_undegraded":"2024-04-24T23:26:17.154894+0000","last_fullsized":"2024-04-24T23:26:17.154894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155905+0000","last_change":"2024-04-24T23:26:17.155905+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155905+0000","last_undegraded":"2024-04-24T23:26:17.155905+0000","last_fullsized":"2024-04-24T23:26:17.155905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167739+0000","last_change":"2024-04-24T23:26:17.166716+0000","last_active":"2024-04-24T23:26:17.167739+0000","last_peered":"2024-04-24T23:26:17.167739+0000","last_clean":"2024-04-24T23:26:17.167739+0000","last_became_active":"2024-04-24T23:26:17.166397+0000","last_became_peered":"2024-04-24T23:26:17.166397+0000","last_unstale":"2024-04-24T23:26:17.167739+0000","last_undegraded":"2024-04-24T23:26:17.167739+0000","last_fullsized":"2024-04-24T23:26:17.167739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:31.957370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.160325+0000","last_change":"2024-04-24T23:26:17.160325+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.160325+0000","last_undegraded":"2024-04-24T23:26:17.160325+0000","last_fullsized":"2024-04-24T23:26:17.160325+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164745+0000","last_change":"2024-04-24T23:26:17.162927+0000","last_active":"2024-04-24T23:26:17.164745+0000","last_peered":"2024-04-24T23:26:17.164745+0000","last_clean":"2024-04-24T23:26:17.164745+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:17.164745+0000","last_undegraded":"2024-04-24T23:26:17.164745+0000","last_fullsized":"2024-04-24T23:26:17.164745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:25.563184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158589+0000","last_change":"2024-04-24T23:26:17.158589+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158589+0000","last_undegraded":"2024-04-24T23:26:17.158589+0000","last_fullsized":"2024-04-24T23:26:17.158589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170289+0000","last_change":"2024-04-24T23:26:17.166726+0000","last_active":"2024-04-24T23:26:17.170289+0000","last_peered":"2024-04-24T23:26:17.170289+0000","last_clean":"2024-04-24T23:26:17.170289+0000","last_became_active":"2024-04-24T23:26:17.166370+0000","last_became_peered":"2024-04-24T23:26:17.166370+0000","last_unstale":"2024-04-24T23:26:17.170289+0000","last_undegraded":"2024-04-24T23:26:17.170289+0000","last_fullsized":"2024-04-24T23:26:17.170289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:28.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152137+0000","last_change":"2024-04-24T23:26:17.152137+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152137+0000","last_undegraded":"2024-04-24T23:26:17.152137+0000","last_fullsized":"2024-04-24T23:26:17.152137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170551+0000","last_change":"2024-04-24T23:26:17.167115+0000","last_active":"2024-04-24T23:26:17.170551+0000","last_peered":"2024-04-24T23:26:17.170551+0000","last_clean":"2024-04-24T23:26:17.170551+0000","last_became_active":"2024-04-24T23:26:17.166838+0000","last_became_peered":"2024-04-24T23:26:17.166838+0000","last_unstale":"2024-04-24T23:26:17.170551+0000","last_undegraded":"2024-04-24T23:26:17.170551+0000","last_fullsized":"2024-04-24T23:26:17.170551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:39.845561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153602+0000","last_change":"2024-04-24T23:26:17.153602+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153602+0000","last_undegraded":"2024-04-24T23:26:17.153602+0000","last_fullsized":"2024-04-24T23:26:17.153602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167567+0000","last_change":"2024-04-24T23:26:17.166049+0000","last_active":"2024-04-24T23:26:17.167567+0000","last_peered":"2024-04-24T23:26:17.167567+0000","last_clean":"2024-04-24T23:26:17.167567+0000","last_became_active":"2024-04-24T23:26:17.165763+0000","last_became_peered":"2024-04-24T23:26:17.165763+0000","last_unstale":"2024-04-24T23:26:17.167567+0000","last_undegraded":"2024-04-24T23:26:17.167567+0000","last_fullsized":"2024-04-24T23:26:17.167567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:40.656153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155159+0000","last_change":"2024-04-24T23:26:17.155159+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155159+0000","last_undegraded":"2024-04-24T23:26:17.155159+0000","last_fullsized":"2024-04-24T23:26:17.155159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.163167+0000","last_change":"2024-04-24T23:26:17.161576+0000","last_active":"2024-04-24T23:26:17.163167+0000","last_peered":"2024-04-24T23:26:17.163167+0000","last_clean":"2024-04-24T23:26:17.163167+0000","last_became_active":"2024-04-24T23:26:17.161260+0000","last_became_peered":"2024-04-24T23:26:17.161260+0000","last_unstale":"2024-04-24T23:26:17.163167+0000","last_undegraded":"2024-04-24T23:26:17.163167+0000","last_fullsized":"2024-04-24T23:26:17.163167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:00.568912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152635+0000","last_change":"2024-04-24T23:26:17.152635+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152635+0000","last_undegraded":"2024-04-24T23:26:17.152635+0000","last_fullsized":"2024-04-24T23:26:17.152635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170205+0000","last_change":"2024-04-24T23:26:17.167646+0000","last_active":"2024-04-24T23:26:17.170205+0000","last_peered":"2024-04-24T23:26:17.170205+0000","last_clean":"2024-04-24T23:26:17.170205+0000","last_became_active":"2024-04-24T23:26:17.167351+0000","last_became_peered":"2024-04-24T23:26:17.167351+0000","last_unstale":"2024-04-24T23:26:17.170205+0000","last_undegraded":"2024-04-24T23:26:17.170205+0000","last_fullsized":"2024-04-24T23:26:17.170205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:39.409219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.151934+0000","last_change":"2024-04-24T23:26:17.151934+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.151934+0000","last_undegraded":"2024-04-24T23:26:17.151934+0000","last_fullsized":"2024-04-24T23:26:17.151934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.165107+0000","last_change":"2024-04-24T23:26:17.162901+0000","last_active":"2024-04-24T23:26:17.165107+0000","last_peered":"2024-04-24T23:26:17.165107+0000","last_clean":"2024-04-24T23:26:17.165107+0000","last_became_active":"2024-04-24T23:26:17.162604+0000","last_became_peered":"2024-04-24T23:26:17.162604+0000","last_unstale":"2024-04-24T23:26:17.165107+0000","last_undegraded":"2024-04-24T23:26:17.165107+0000","last_fullsized":"2024-04-24T23:26:17.165107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:37.984293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.151319+0000","last_change":"2024-04-24T23:26:17.151319+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.151319+0000","last_undegraded":"2024-04-24T23:26:17.151319+0000","last_fullsized":"2024-04-24T23:26:17.151319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170103+0000","last_change":"2024-04-24T23:26:17.167550+0000","last_active":"2024-04-24T23:26:17.170103+0000","last_peered":"2024-04-24T23:26:17.170103+0000","last_clean":"2024-04-24T23:26:17.170103+0000","last_became_active":"2024-04-24T23:26:17.167229+0000","last_became_peered":"2024-04-24T23:26:17.167229+0000","last_unstale":"2024-04-24T23:26:17.170103+0000","last_undegraded":"2024-04-24T23:26:17.170103+0000","last_fullsized":"2024-04-24T23:26:17.170103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:30.940114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148656+0000","last_change":"2024-04-24T23:26:17.148656+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148656+0000","last_undegraded":"2024-04-24T23:26:17.148656+0000","last_fullsized":"2024-04-24T23:26:17.148656+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170242+0000","last_change":"2024-04-24T23:26:17.168677+0000","last_active":"2024-04-24T23:26:17.170242+0000","last_peered":"2024-04-24T23:26:17.170242+0000","last_clean":"2024-04-24T23:26:17.170242+0000","last_became_active":"2024-04-24T23:26:17.168354+0000","last_became_peered":"2024-04-24T23:26:17.168354+0000","last_unstale":"2024-04-24T23:26:17.170242+0000","last_undegraded":"2024-04-24T23:26:17.170242+0000","last_fullsized":"2024-04-24T23:26:17.170242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:59.598138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.160360+0000","last_change":"2024-04-24T23:26:17.160360+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.160360+0000","last_undegraded":"2024-04-24T23:26:17.160360+0000","last_fullsized":"2024-04-24T23:26:17.160360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170673+0000","last_change":"2024-04-24T23:26:17.170110+0000","last_active":"2024-04-24T23:26:17.170673+0000","last_peered":"2024-04-24T23:26:17.170673+0000","last_clean":"2024-04-24T23:26:17.170673+0000","last_became_active":"2024-04-24T23:26:17.169755+0000","last_became_peered":"2024-04-24T23:26:17.169755+0000","last_unstale":"2024-04-24T23:26:17.170673+0000","last_undegraded":"2024-04-24T23:26:17.170673+0000","last_fullsized":"2024-04-24T23:26:17.170673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:03.234791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154473+0000","last_change":"2024-04-24T23:26:17.154473+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154473+0000","last_undegraded":"2024-04-24T23:26:17.154473+0000","last_fullsized":"2024-04-24T23:26:17.154473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.152470+0000","last_change":"2024-04-24T23:26:17.151318+0000","last_active":"2024-04-24T23:26:17.152470+0000","last_peered":"2024-04-24T23:26:17.152470+0000","last_clean":"2024-04-24T23:26:17.152470+0000","last_became_active":"2024-04-24T23:26:17.151022+0000","last_became_peered":"2024-04-24T23:26:17.151022+0000","last_unstale":"2024-04-24T23:26:17.152470+0000","last_undegraded":"2024-04-24T23:26:17.152470+0000","last_fullsized":"2024-04-24T23:26:17.152470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:25.593578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.156618+0000","last_change":"2024-04-24T23:26:17.156618+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.156618+0000","last_undegraded":"2024-04-24T23:26:17.156618+0000","last_fullsized":"2024-04-24T23:26:17.156618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.165078+0000","last_change":"2024-04-24T23:26:17.162943+0000","last_active":"2024-04-24T23:26:17.165078+0000","last_peered":"2024-04-24T23:26:17.165078+0000","last_clean":"2024-04-24T23:26:17.165078+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:17.165078+0000","last_undegraded":"2024-04-24T23:26:17.165078+0000","last_fullsized":"2024-04-24T23:26:17.165078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:59.927814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153902+0000","last_change":"2024-04-24T23:26:17.153902+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153902+0000","last_undegraded":"2024-04-24T23:26:17.153902+0000","last_fullsized":"2024-04-24T23:26:17.153902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164541+0000","last_change":"2024-04-24T23:26:17.163630+0000","last_active":"2024-04-24T23:26:17.164541+0000","last_peered":"2024-04-24T23:26:17.164541+0000","last_clean":"2024-04-24T23:26:17.164541+0000","last_became_active":"2024-04-24T23:26:17.163331+0000","last_became_peered":"2024-04-24T23:26:17.163331+0000","last_unstale":"2024-04-24T23:26:17.164541+0000","last_undegraded":"2024-04-24T23:26:17.164541+0000","last_fullsized":"2024-04-24T23:26:17.164541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:47.984272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152815+0000","last_change":"2024-04-24T23:26:17.152815+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152815+0000","last_undegraded":"2024-04-24T23:26:17.152815+0000","last_fullsized":"2024-04-24T23:26:17.152815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.161936+0000","last_change":"2024-04-24T23:26:17.160593+0000","last_active":"2024-04-24T23:26:17.161936+0000","last_peered":"2024-04-24T23:26:17.161936+0000","last_clean":"2024-04-24T23:26:17.161936+0000","last_became_active":"2024-04-24T23:26:17.160303+0000","last_became_peered":"2024-04-24T23:26:17.160303+0000","last_unstale":"2024-04-24T23:26:17.161936+0000","last_undegraded":"2024-04-24T23:26:17.161936+0000","last_fullsized":"2024-04-24T23:26:17.161936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:18.000135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.151917+0000","last_change":"2024-04-24T23:26:17.151917+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.151917+0000","last_undegraded":"2024-04-24T23:26:17.151917+0000","last_fullsized":"2024-04-24T23:26:17.151917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170145+0000","last_change":"2024-04-24T23:26:17.165945+0000","last_active":"2024-04-24T23:26:17.170145+0000","last_peered":"2024-04-24T23:26:17.170145+0000","last_clean":"2024-04-24T23:26:17.170145+0000","last_became_active":"2024-04-24T23:26:17.164750+0000","last_became_peered":"2024-04-24T23:26:17.164750+0000","last_unstale":"2024-04-24T23:26:17.170145+0000","last_undegraded":"2024-04-24T23:26:17.170145+0000","last_fullsized":"2024-04-24T23:26:17.170145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:55.620251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153107+0000","last_change":"2024-04-24T23:26:17.153107+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153107+0000","last_undegraded":"2024-04-24T23:26:17.153107+0000","last_fullsized":"2024-04-24T23:26:17.153107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167256+0000","last_change":"2024-04-24T23:26:17.165977+0000","last_active":"2024-04-24T23:26:17.167256+0000","last_peered":"2024-04-24T23:26:17.167256+0000","last_clean":"2024-04-24T23:26:17.167256+0000","last_became_active":"2024-04-24T23:26:17.165641+0000","last_became_peered":"2024-04-24T23:26:17.165641+0000","last_unstale":"2024-04-24T23:26:17.167256+0000","last_undegraded":"2024-04-24T23:26:17.167256+0000","last_fullsized":"2024-04-24T23:26:17.167256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:22.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159384+0000","last_change":"2024-04-24T23:26:17.159384+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159384+0000","last_undegraded":"2024-04-24T23:26:17.159384+0000","last_fullsized":"2024-04-24T23:26:17.159384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170579+0000","last_change":"2024-04-24T23:26:17.163913+0000","last_active":"2024-04-24T23:26:17.170579+0000","last_peered":"2024-04-24T23:26:17.170579+0000","last_clean":"2024-04-24T23:26:17.170579+0000","last_became_active":"2024-04-24T23:26:17.163640+0000","last_became_peered":"2024-04-24T23:26:17.163640+0000","last_unstale":"2024-04-24T23:26:17.170579+0000","last_undegraded":"2024-04-24T23:26:17.170579+0000","last_fullsized":"2024-04-24T23:26:17.170579+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:18.000135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153797+0000","last_change":"2024-04-24T23:26:17.153797+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153797+0000","last_undegraded":"2024-04-24T23:26:17.153797+0000","last_fullsized":"2024-04-24T23:26:17.153797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170573+0000","last_change":"2024-04-24T23:26:17.167926+0000","last_active":"2024-04-24T23:26:17.170573+0000","last_peered":"2024-04-24T23:26:17.170573+0000","last_clean":"2024-04-24T23:26:17.170573+0000","last_became_active":"2024-04-24T23:26:17.167660+0000","last_became_peered":"2024-04-24T23:26:17.167660+0000","last_unstale":"2024-04-24T23:26:17.170573+0000","last_undegraded":"2024-04-24T23:26:17.170573+0000","last_fullsized":"2024-04-24T23:26:17.170573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:54.496665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154193+0000","last_change":"2024-04-24T23:26:17.154193+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154193+0000","last_undegraded":"2024-04-24T23:26:17.154193+0000","last_fullsized":"2024-04-24T23:26:17.154193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.161464+0000","last_change":"2024-04-24T23:26:17.158322+0000","last_active":"2024-04-24T23:26:17.161464+0000","last_peered":"2024-04-24T23:26:17.161464+0000","last_clean":"2024-04-24T23:26:17.161464+0000","last_became_active":"2024-04-24T23:26:17.158052+0000","last_became_peered":"2024-04-24T23:26:17.158052+0000","last_unstale":"2024-04-24T23:26:17.161464+0000","last_undegraded":"2024-04-24T23:26:17.161464+0000","last_fullsized":"2024-04-24T23:26:17.161464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:07.133490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148457+0000","last_change":"2024-04-24T23:26:17.148457+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148457+0000","last_undegraded":"2024-04-24T23:26:17.148457+0000","last_fullsized":"2024-04-24T23:26:17.148457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.169586+0000","last_change":"2024-04-24T23:26:17.167861+0000","last_active":"2024-04-24T23:26:17.169586+0000","last_peered":"2024-04-24T23:26:17.169586+0000","last_clean":"2024-04-24T23:26:17.169586+0000","last_became_active":"2024-04-24T23:26:17.167598+0000","last_became_peered":"2024-04-24T23:26:17.167598+0000","last_unstale":"2024-04-24T23:26:17.169586+0000","last_undegraded":"2024-04-24T23:26:17.169586+0000","last_fullsized":"2024-04-24T23:26:17.169586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:27.969964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152735+0000","last_change":"2024-04-24T23:26:17.152735+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152735+0000","last_undegraded":"2024-04-24T23:26:17.152735+0000","last_fullsized":"2024-04-24T23:26:17.152735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.168211+0000","last_change":"2024-04-24T23:26:17.165956+0000","last_active":"2024-04-24T23:26:17.168211+0000","last_peered":"2024-04-24T23:26:17.168211+0000","last_clean":"2024-04-24T23:26:17.168211+0000","last_became_active":"2024-04-24T23:26:17.165633+0000","last_became_peered":"2024-04-24T23:26:17.165633+0000","last_unstale":"2024-04-24T23:26:17.168211+0000","last_undegraded":"2024-04-24T23:26:17.168211+0000","last_fullsized":"2024-04-24T23:26:17.168211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:24.199382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.149169+0000","last_change":"2024-04-24T23:26:17.149169+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.149169+0000","last_undegraded":"2024-04-24T23:26:17.149169+0000","last_fullsized":"2024-04-24T23:26:17.149169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.161402+0000","last_change":"2024-04-24T23:26:17.157597+0000","last_active":"2024-04-24T23:26:17.161402+0000","last_peered":"2024-04-24T23:26:17.161402+0000","last_clean":"2024-04-24T23:26:17.161402+0000","last_became_active":"2024-04-24T23:26:17.157311+0000","last_became_peered":"2024-04-24T23:26:17.157311+0000","last_unstale":"2024-04-24T23:26:17.161402+0000","last_undegraded":"2024-04-24T23:26:17.161402+0000","last_fullsized":"2024-04-24T23:26:17.161402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:18.468882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152830+0000","last_change":"2024-04-24T23:26:17.152830+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152830+0000","last_undegraded":"2024-04-24T23:26:17.152830+0000","last_fullsized":"2024-04-24T23:26:17.152830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.165174+0000","last_change":"2024-04-24T23:26:17.163231+0000","last_active":"2024-04-24T23:26:17.165174+0000","last_peered":"2024-04-24T23:26:17.165174+0000","last_clean":"2024-04-24T23:26:17.165174+0000","last_became_active":"2024-04-24T23:26:17.162790+0000","last_became_peered":"2024-04-24T23:26:17.162790+0000","last_unstale":"2024-04-24T23:26:17.165174+0000","last_undegraded":"2024-04-24T23:26:17.165174+0000","last_fullsized":"2024-04-24T23:26:17.165174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:42.373513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154400+0000","last_change":"2024-04-24T23:26:17.154400+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154400+0000","last_undegraded":"2024-04-24T23:26:17.154400+0000","last_fullsized":"2024-04-24T23:26:17.154400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164991+0000","last_change":"2024-04-24T23:26:17.161814+0000","last_active":"2024-04-24T23:26:17.164991+0000","last_peered":"2024-04-24T23:26:17.164991+0000","last_clean":"2024-04-24T23:26:17.164991+0000","last_became_active":"2024-04-24T23:26:17.161485+0000","last_became_peered":"2024-04-24T23:26:17.161485+0000","last_unstale":"2024-04-24T23:26:17.164991+0000","last_undegraded":"2024-04-24T23:26:17.164991+0000","last_fullsized":"2024-04-24T23:26:17.164991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:18.807328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158100+0000","last_change":"2024-04-24T23:26:17.158100+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158100+0000","last_undegraded":"2024-04-24T23:26:17.158100+0000","last_fullsized":"2024-04-24T23:26:17.158100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170350+0000","last_change":"2024-04-24T23:26:17.169621+0000","last_active":"2024-04-24T23:26:17.170350+0000","last_peered":"2024-04-24T23:26:17.170350+0000","last_clean":"2024-04-24T23:26:17.170350+0000","last_became_active":"2024-04-24T23:26:17.169195+0000","last_became_peered":"2024-04-24T23:26:17.169195+0000","last_unstale":"2024-04-24T23:26:17.170350+0000","last_undegraded":"2024-04-24T23:26:17.170350+0000","last_fullsized":"2024-04-24T23:26:17.170350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:42.773783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153079+0000","last_change":"2024-04-24T23:26:17.153079+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153079+0000","last_undegraded":"2024-04-24T23:26:17.153079+0000","last_fullsized":"2024-04-24T23:26:17.153079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170341+0000","last_change":"2024-04-24T23:26:17.169611+0000","last_active":"2024-04-24T23:26:17.170341+0000","last_peered":"2024-04-24T23:26:17.170341+0000","last_clean":"2024-04-24T23:26:17.170341+0000","last_became_active":"2024-04-24T23:26:17.169304+0000","last_became_peered":"2024-04-24T23:26:17.169304+0000","last_unstale":"2024-04-24T23:26:17.170341+0000","last_undegraded":"2024-04-24T23:26:17.170341+0000","last_fullsized":"2024-04-24T23:26:17.170341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:21.760106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148445+0000","last_change":"2024-04-24T23:26:17.148445+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148445+0000","last_undegraded":"2024-04-24T23:26:17.148445+0000","last_fullsized":"2024-04-24T23:26:17.148445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170099+0000","last_change":"2024-04-24T23:26:17.168469+0000","last_active":"2024-04-24T23:26:17.170099+0000","last_peered":"2024-04-24T23:26:17.170099+0000","last_clean":"2024-04-24T23:26:17.170099+0000","last_became_active":"2024-04-24T23:26:17.167305+0000","last_became_peered":"2024-04-24T23:26:17.167305+0000","last_unstale":"2024-04-24T23:26:17.170099+0000","last_undegraded":"2024-04-24T23:26:17.170099+0000","last_fullsized":"2024-04-24T23:26:17.170099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:08.429123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159242+0000","last_change":"2024-04-24T23:26:17.159242+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159242+0000","last_undegraded":"2024-04-24T23:26:17.159242+0000","last_fullsized":"2024-04-24T23:26:17.159242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164655+0000","last_change":"2024-04-24T23:26:17.162968+0000","last_active":"2024-04-24T23:26:17.164655+0000","last_peered":"2024-04-24T23:26:17.164655+0000","last_clean":"2024-04-24T23:26:17.164655+0000","last_became_active":"2024-04-24T23:26:17.162691+0000","last_became_peered":"2024-04-24T23:26:17.162691+0000","last_unstale":"2024-04-24T23:26:17.164655+0000","last_undegraded":"2024-04-24T23:26:17.164655+0000","last_fullsized":"2024-04-24T23:26:17.164655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.997763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152026+0000","last_change":"2024-04-24T23:26:17.152026+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152026+0000","last_undegraded":"2024-04-24T23:26:17.152026+0000","last_fullsized":"2024-04-24T23:26:17.152026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170205+0000","last_change":"2024-04-24T23:26:17.168302+0000","last_active":"2024-04-24T23:26:17.170205+0000","last_peered":"2024-04-24T23:26:17.170205+0000","last_clean":"2024-04-24T23:26:17.170205+0000","last_became_active":"2024-04-24T23:26:17.167711+0000","last_became_peered":"2024-04-24T23:26:17.167711+0000","last_unstale":"2024-04-24T23:26:17.170205+0000","last_undegraded":"2024-04-24T23:26:17.170205+0000","last_fullsized":"2024-04-24T23:26:17.170205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:01.210692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159455+0000","last_change":"2024-04-24T23:26:17.159455+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159455+0000","last_undegraded":"2024-04-24T23:26:17.159455+0000","last_fullsized":"2024-04-24T23:26:17.159455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.159915+0000","last_change":"2024-04-24T23:26:17.156009+0000","last_active":"2024-04-24T23:26:17.159915+0000","last_peered":"2024-04-24T23:26:17.159915+0000","last_clean":"2024-04-24T23:26:17.159915+0000","last_became_active":"2024-04-24T23:26:17.155739+0000","last_became_peered":"2024-04-24T23:26:17.155739+0000","last_unstale":"2024-04-24T23:26:17.159915+0000","last_undegraded":"2024-04-24T23:26:17.159915+0000","last_fullsized":"2024-04-24T23:26:17.159915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:59.737718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153256+0000","last_change":"2024-04-24T23:26:17.153256+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153256+0000","last_undegraded":"2024-04-24T23:26:17.153256+0000","last_fullsized":"2024-04-24T23:26:17.153256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167714+0000","last_change":"2024-04-24T23:26:17.166166+0000","last_active":"2024-04-24T23:26:17.167714+0000","last_peered":"2024-04-24T23:26:17.167714+0000","last_clean":"2024-04-24T23:26:17.167714+0000","last_became_active":"2024-04-24T23:26:17.165855+0000","last_became_peered":"2024-04-24T23:26:17.165855+0000","last_unstale":"2024-04-24T23:26:17.167714+0000","last_undegraded":"2024-04-24T23:26:17.167714+0000","last_fullsized":"2024-04-24T23:26:17.167714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:23.185199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":278,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.155269+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:26:17.155269+0000","last_peered":"2024-04-24T23:26:17.155269+0000","last_clean":"2024-04-24T23:26:17.155269+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:26:17.155269+0000","last_undegraded":"2024-04-24T23:26:17.155269+0000","last_fullsized":"2024-04-24T23:26:17.155269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1564672,"data_stored":1004549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":420456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335544,"statfs":{"total":96636764160,"available":96599597056,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":983040,"data_stored":420456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1576960,"data_stored":1010824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:26:22.526 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:26:22.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.663+0000 7fcc5559e640 1 Processor -- start 2024-04-24T23:26:22.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.663+0000 7fcc5559e640 1 -- start start 2024-04-24T23:26:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc5559e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc5015e290 0x7fcc500d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc5559e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcc500b72f0 0x7fcc500b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc5559e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc500b5840 con 0x7fcc500b3ac0 2024-04-24T23:26:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc5559e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fcc500b5e50 con 0x7fcc5015e290 2024-04-24T23:26:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc5559e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fcc500cfb30 con 0x7fcc500b72f0 2024-04-24T23:26:22.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc4effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc5015e290 0x7fcc500d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc4e7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcc500b3ac0 0x7fcc5006c230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55902/0 (socket says 172.21.15.26:55902) 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc4e7fc640 1 -- 172.21.15.26:0/3642522378 learned_addr learned my addr 172.21.15.26:0/3642522378 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc4f7fe640 1 --2- 172.21.15.26:0/3642522378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcc500b72f0 0x7fcc500b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc4effd640 1 -- 172.21.15.26:0/3642522378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcc500b72f0 msgr2=0x7fcc500b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc4effd640 1 --2- 172.21.15.26:0/3642522378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcc500b72f0 0x7fcc500b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc4effd640 1 -- 172.21.15.26:0/3642522378 >> v1:172.21.15.26:6789/0 conn(0x7fcc500b3ac0 legacy=0x7fcc5006c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc4effd640 1 -- 172.21.15.26:0/3642522378 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc500cfe10 con 0x7fcc5015e290 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.664+0000 7fcc4dffb640 1 -- 172.21.15.26:0/3642522378 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335691750 0 0) 0x7fcc500b5840 con 0x7fcc500b3ac0 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.665+0000 7fcc4effd640 1 --2- 172.21.15.26:0/3642522378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5015e290 0x7fcc500d1730 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009d90 tx=0x7fcc4002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c926829459f6d999 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.665+0000 7fcc4dffb640 1 -- 172.21.15.26:0/3642522378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40007740 con 0x7fcc5015e290 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.665+0000 7fcc4dffb640 1 -- 172.21.15.26:0/3642522378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc400078e0 con 0x7fcc5015e290 2024-04-24T23:26:22.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.665+0000 7fcc4dffb640 1 -- 172.21.15.26:0/3642522378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40007cb0 con 0x7fcc5015e290 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.666+0000 7fcc5559e640 1 -- 172.21.15.26:0/3642522378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5015e290 msgr2=0x7fcc500d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.666+0000 7fcc5559e640 1 --2- 172.21.15.26:0/3642522378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5015e290 0x7fcc500d1730 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009d90 tx=0x7fcc4002a660 comp rx=0 tx=0).stop 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.666+0000 7fcc5559e640 1 -- 172.21.15.26:0/3642522378 shutdown_connections 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.666+0000 7fcc5559e640 1 --2- 172.21.15.26:0/3642522378 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcc500b72f0 0x7fcc500b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.666+0000 7fcc5559e640 1 --2- 172.21.15.26:0/3642522378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc5015e290 0x7fcc500d1730 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.666+0000 7fcc5559e640 1 -- 172.21.15.26:0/3642522378 >> 172.21.15.26:0/3642522378 conn(0x7fcc5006b120 msgr2=0x7fcc5006b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.666+0000 7fcc5559e640 1 -- 172.21.15.26:0/3642522378 shutdown_connections 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.666+0000 7fcc5559e640 1 -- 172.21.15.26:0/3642522378 wait complete. 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.667+0000 7fcc5559e640 1 Processor -- start 2024-04-24T23:26:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.667+0000 7fcc5559e640 1 -- start start 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc5559e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc500b3ac0 0x7fcc501475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc5559e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc500b72f0 0x7fcc500b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc5559e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcc5015e290 0x7fcc500b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc5559e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5015e670 con 0x7fcc500b3ac0 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc5559e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcc500b5ae0 con 0x7fcc500b72f0 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc5559e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fcc500d1c70 con 0x7fcc5015e290 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc4effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc500b3ac0 0x7fcc501475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc4effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc500b3ac0 0x7fcc501475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56304/0 (socket says 172.21.15.26:56304) 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc4effd640 1 -- 172.21.15.26:0/3789219101 learned_addr learned my addr 172.21.15.26:0/3789219101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc4effd640 1 -- 172.21.15.26:0/3789219101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcc5015e290 msgr2=0x7fcc500b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc4effd640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcc5015e290 0x7fcc500b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc4effd640 1 -- 172.21.15.26:0/3789219101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc500b72f0 msgr2=0x7fcc500b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc4effd640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc500b72f0 0x7fcc500b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc4effd640 1 -- 172.21.15.26:0/3789219101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5014ae30 con 0x7fcc500b3ac0 2024-04-24T23:26:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.668+0000 7fcc4effd640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc500b3ac0 0x7fcc501475c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009eb0 tx=0x7fcc40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.669+0000 7fcc377fe640 1 -- 172.21.15.26:0/3789219101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4003f070 con 0x7fcc500b3ac0 2024-04-24T23:26:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.669+0000 7fcc377fe640 1 -- 172.21.15.26:0/3789219101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc40031c40 con 0x7fcc500b3ac0 2024-04-24T23:26:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.669+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc500cd5b0 con 0x7fcc500b3ac0 2024-04-24T23:26:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.669+0000 7fcc377fe640 1 -- 172.21.15.26:0/3789219101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40043410 con 0x7fcc500b3ac0 2024-04-24T23:26:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.669+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc500cdb70 con 0x7fcc500b3ac0 2024-04-24T23:26:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.670+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc500694c0 con 0x7fcc500b3ac0 2024-04-24T23:26:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.670+0000 7fcc377fe640 1 -- 172.21.15.26:0/3789219101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc400435b0 con 0x7fcc500b3ac0 2024-04-24T23:26:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.670+0000 7fcc377fe640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcc3c06be30 0x7fcc3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.671+0000 7fcc377fe640 1 -- 172.21.15.26:0/3789219101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fcc400ac3c0 con 0x7fcc500b3ac0 2024-04-24T23:26:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.671+0000 7fcc4e7fc640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcc3c06be30 0x7fcc3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.671+0000 7fcc4e7fc640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcc3c06be30 0x7fcc3c06e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcc44007fd0 tx=0x7fcc44007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:22.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.675+0000 7fcc377fe640 1 -- 172.21.15.26:0/3789219101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcc4007ae00 con 0x7fcc500b3ac0 2024-04-24T23:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.854+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc5006f4f0 con 0x7fcc3c06be30 2024-04-24T23:26:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.860+0000 7fcc377fe640 1 -- 172.21.15.26:0/3789219101 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246741 (secure 0 0 0) 0x7fcc5006f4f0 con 0x7fcc3c06be30 2024-04-24T23:26:22.864 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:22.865 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:26:22.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcc3c06be30 msgr2=0x7fcc3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcc3c06be30 0x7fcc3c06e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcc44007fd0 tx=0x7fcc44007900 comp rx=0 tx=0).stop 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc500b3ac0 msgr2=0x7fcc501475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc500b3ac0 0x7fcc501475c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009eb0 tx=0x7fcc40004a40 comp rx=0 tx=0).stop 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 shutdown_connections 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcc3c06be30 0x7fcc3c06e2f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcc5015e290 0x7fcc500b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc500b72f0 0x7fcc500b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 --2- 172.21.15.26:0/3789219101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc500b3ac0 0x7fcc501475c0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 >> 172.21.15.26:0/3789219101 conn(0x7fcc5006b120 msgr2=0x7fcc500b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 shutdown_connections 2024-04-24T23:26:22.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:22.863+0000 7fcc5559e640 1 -- 172.21.15.26:0/3789219101 wait complete. 2024-04-24T23:26:22.875 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-04-24T23:26:21.386191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270732,"kb_used_data":8780,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754703988,"statfs":{"total":773094113280,"available":772816883712,"internally_reserved":0,"allocated":8990720,"data_stored":4513009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.242490"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.161035+0000","last_change":"2024-04-24T23:26:17.161035+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.161035+0000","last_undegraded":"2024-04-24T23:26:17.161035+0000","last_fullsized":"2024-04-24T23:26:17.161035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152937+0000","last_change":"2024-04-24T23:26:17.152937+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152937+0000","last_undegraded":"2024-04-24T23:26:17.152937+0000","last_fullsized":"2024-04-24T23:26:17.152937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152655+0000","last_change":"2024-04-24T23:26:17.152655+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152655+0000","last_undegraded":"2024-04-24T23:26:17.152655+0000","last_fullsized":"2024-04-24T23:26:17.152655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.156363+0000","last_change":"2024-04-24T23:26:17.156363+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.156363+0000","last_undegraded":"2024-04-24T23:26:17.156363+0000","last_fullsized":"2024-04-24T23:26:17.156363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159303+0000","last_change":"2024-04-24T23:26:17.159303+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159303+0000","last_undegraded":"2024-04-24T23:26:17.159303+0000","last_fullsized":"2024-04-24T23:26:17.159303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.149287+0000","last_change":"2024-04-24T23:26:17.149287+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.149287+0000","last_undegraded":"2024-04-24T23:26:17.149287+0000","last_fullsized":"2024-04-24T23:26:17.149287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155893+0000","last_change":"2024-04-24T23:26:17.155893+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155893+0000","last_undegraded":"2024-04-24T23:26:17.155893+0000","last_fullsized":"2024-04-24T23:26:17.155893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148447+0000","last_change":"2024-04-24T23:26:17.148447+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148447+0000","last_undegraded":"2024-04-24T23:26:17.148447+0000","last_fullsized":"2024-04-24T23:26:17.148447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.149122+0000","last_change":"2024-04-24T23:26:17.149122+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.149122+0000","last_undegraded":"2024-04-24T23:26:17.149122+0000","last_fullsized":"2024-04-24T23:26:17.149122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159010+0000","last_change":"2024-04-24T23:26:17.159010+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159010+0000","last_undegraded":"2024-04-24T23:26:17.159010+0000","last_fullsized":"2024-04-24T23:26:17.159010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152125+0000","last_change":"2024-04-24T23:26:17.152125+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152125+0000","last_undegraded":"2024-04-24T23:26:17.152125+0000","last_fullsized":"2024-04-24T23:26:17.152125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.160127+0000","last_change":"2024-04-24T23:26:17.160127+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.160127+0000","last_undegraded":"2024-04-24T23:26:17.160127+0000","last_fullsized":"2024-04-24T23:26:17.160127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154328+0000","last_change":"2024-04-24T23:26:17.154328+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154328+0000","last_undegraded":"2024-04-24T23:26:17.154328+0000","last_fullsized":"2024-04-24T23:26:17.154328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154638+0000","last_change":"2024-04-24T23:26:17.154638+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154638+0000","last_undegraded":"2024-04-24T23:26:17.154638+0000","last_fullsized":"2024-04-24T23:26:17.154638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.156466+0000","last_change":"2024-04-24T23:26:17.156466+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.156466+0000","last_undegraded":"2024-04-24T23:26:17.156466+0000","last_fullsized":"2024-04-24T23:26:17.156466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158764+0000","last_change":"2024-04-24T23:26:17.158764+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158764+0000","last_undegraded":"2024-04-24T23:26:17.158764+0000","last_fullsized":"2024-04-24T23:26:17.158764+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158461+0000","last_change":"2024-04-24T23:26:17.158461+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158461+0000","last_undegraded":"2024-04-24T23:26:17.158461+0000","last_fullsized":"2024-04-24T23:26:17.158461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153010+0000","last_change":"2024-04-24T23:26:17.153010+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153010+0000","last_undegraded":"2024-04-24T23:26:17.153010+0000","last_fullsized":"2024-04-24T23:26:17.153010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159129+0000","last_change":"2024-04-24T23:26:17.159129+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159129+0000","last_undegraded":"2024-04-24T23:26:17.159129+0000","last_fullsized":"2024-04-24T23:26:17.159129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155234+0000","last_change":"2024-04-24T23:26:17.155234+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155234+0000","last_undegraded":"2024-04-24T23:26:17.155234+0000","last_fullsized":"2024-04-24T23:26:17.155234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155233+0000","last_change":"2024-04-24T23:26:17.155233+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155233+0000","last_undegraded":"2024-04-24T23:26:17.155233+0000","last_fullsized":"2024-04-24T23:26:17.155233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154445+0000","last_change":"2024-04-24T23:26:17.154445+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154445+0000","last_undegraded":"2024-04-24T23:26:17.154445+0000","last_fullsized":"2024-04-24T23:26:17.154445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159956+0000","last_change":"2024-04-24T23:26:17.159956+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159956+0000","last_undegraded":"2024-04-24T23:26:17.159956+0000","last_fullsized":"2024-04-24T23:26:17.159956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153261+0000","last_change":"2024-04-24T23:26:17.153261+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153261+0000","last_undegraded":"2024-04-24T23:26:17.153261+0000","last_fullsized":"2024-04-24T23:26:17.153261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153219+0000","last_change":"2024-04-24T23:26:17.153219+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153219+0000","last_undegraded":"2024-04-24T23:26:17.153219+0000","last_fullsized":"2024-04-24T23:26:17.153219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153550+0000","last_change":"2024-04-24T23:26:17.153550+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153550+0000","last_undegraded":"2024-04-24T23:26:17.153550+0000","last_fullsized":"2024-04-24T23:26:17.153550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159847+0000","last_change":"2024-04-24T23:26:17.159847+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159847+0000","last_undegraded":"2024-04-24T23:26:17.159847+0000","last_fullsized":"2024-04-24T23:26:17.159847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159058+0000","last_change":"2024-04-24T23:26:17.159058+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159058+0000","last_undegraded":"2024-04-24T23:26:17.159058+0000","last_fullsized":"2024-04-24T23:26:17.159058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152080+0000","last_change":"2024-04-24T23:26:17.152080+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152080+0000","last_undegraded":"2024-04-24T23:26:17.152080+0000","last_fullsized":"2024-04-24T23:26:17.152080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154029+0000","last_change":"2024-04-24T23:26:17.154029+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154029+0000","last_undegraded":"2024-04-24T23:26:17.154029+0000","last_fullsized":"2024-04-24T23:26:17.154029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155927+0000","last_change":"2024-04-24T23:26:17.155927+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155927+0000","last_undegraded":"2024-04-24T23:26:17.155927+0000","last_fullsized":"2024-04-24T23:26:17.155927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154894+0000","last_change":"2024-04-24T23:26:17.154894+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154894+0000","last_undegraded":"2024-04-24T23:26:17.154894+0000","last_fullsized":"2024-04-24T23:26:17.154894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155905+0000","last_change":"2024-04-24T23:26:17.155905+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155905+0000","last_undegraded":"2024-04-24T23:26:17.155905+0000","last_fullsized":"2024-04-24T23:26:17.155905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167739+0000","last_change":"2024-04-24T23:26:17.166716+0000","last_active":"2024-04-24T23:26:17.167739+0000","last_peered":"2024-04-24T23:26:17.167739+0000","last_clean":"2024-04-24T23:26:17.167739+0000","last_became_active":"2024-04-24T23:26:17.166397+0000","last_became_peered":"2024-04-24T23:26:17.166397+0000","last_unstale":"2024-04-24T23:26:17.167739+0000","last_undegraded":"2024-04-24T23:26:17.167739+0000","last_fullsized":"2024-04-24T23:26:17.167739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:31.957370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.160325+0000","last_change":"2024-04-24T23:26:17.160325+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.160325+0000","last_undegraded":"2024-04-24T23:26:17.160325+0000","last_fullsized":"2024-04-24T23:26:17.160325+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164745+0000","last_change":"2024-04-24T23:26:17.162927+0000","last_active":"2024-04-24T23:26:17.164745+0000","last_peered":"2024-04-24T23:26:17.164745+0000","last_clean":"2024-04-24T23:26:17.164745+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:17.164745+0000","last_undegraded":"2024-04-24T23:26:17.164745+0000","last_fullsized":"2024-04-24T23:26:17.164745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:25.563184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158589+0000","last_change":"2024-04-24T23:26:17.158589+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158589+0000","last_undegraded":"2024-04-24T23:26:17.158589+0000","last_fullsized":"2024-04-24T23:26:17.158589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170289+0000","last_change":"2024-04-24T23:26:17.166726+0000","last_active":"2024-04-24T23:26:17.170289+0000","last_peered":"2024-04-24T23:26:17.170289+0000","last_clean":"2024-04-24T23:26:17.170289+0000","last_became_active":"2024-04-24T23:26:17.166370+0000","last_became_peered":"2024-04-24T23:26:17.166370+0000","last_unstale":"2024-04-24T23:26:17.170289+0000","last_undegraded":"2024-04-24T23:26:17.170289+0000","last_fullsized":"2024-04-24T23:26:17.170289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:28.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152137+0000","last_change":"2024-04-24T23:26:17.152137+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152137+0000","last_undegraded":"2024-04-24T23:26:17.152137+0000","last_fullsized":"2024-04-24T23:26:17.152137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170551+0000","last_change":"2024-04-24T23:26:17.167115+0000","last_active":"2024-04-24T23:26:17.170551+0000","last_peered":"2024-04-24T23:26:17.170551+0000","last_clean":"2024-04-24T23:26:17.170551+0000","last_became_active":"2024-04-24T23:26:17.166838+0000","last_became_peered":"2024-04-24T23:26:17.166838+0000","last_unstale":"2024-04-24T23:26:17.170551+0000","last_undegraded":"2024-04-24T23:26:17.170551+0000","last_fullsized":"2024-04-24T23:26:17.170551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:39.845561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153602+0000","last_change":"2024-04-24T23:26:17.153602+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153602+0000","last_undegraded":"2024-04-24T23:26:17.153602+0000","last_fullsized":"2024-04-24T23:26:17.153602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167567+0000","last_change":"2024-04-24T23:26:17.166049+0000","last_active":"2024-04-24T23:26:17.167567+0000","last_peered":"2024-04-24T23:26:17.167567+0000","last_clean":"2024-04-24T23:26:17.167567+0000","last_became_active":"2024-04-24T23:26:17.165763+0000","last_became_peered":"2024-04-24T23:26:17.165763+0000","last_unstale":"2024-04-24T23:26:17.167567+0000","last_undegraded":"2024-04-24T23:26:17.167567+0000","last_fullsized":"2024-04-24T23:26:17.167567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:40.656153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.155159+0000","last_change":"2024-04-24T23:26:17.155159+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.155159+0000","last_undegraded":"2024-04-24T23:26:17.155159+0000","last_fullsized":"2024-04-24T23:26:17.155159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.163167+0000","last_change":"2024-04-24T23:26:17.161576+0000","last_active":"2024-04-24T23:26:17.163167+0000","last_peered":"2024-04-24T23:26:17.163167+0000","last_clean":"2024-04-24T23:26:17.163167+0000","last_became_active":"2024-04-24T23:26:17.161260+0000","last_became_peered":"2024-04-24T23:26:17.161260+0000","last_unstale":"2024-04-24T23:26:17.163167+0000","last_undegraded":"2024-04-24T23:26:17.163167+0000","last_fullsized":"2024-04-24T23:26:17.163167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:00.568912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152635+0000","last_change":"2024-04-24T23:26:17.152635+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152635+0000","last_undegraded":"2024-04-24T23:26:17.152635+0000","last_fullsized":"2024-04-24T23:26:17.152635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170205+0000","last_change":"2024-04-24T23:26:17.167646+0000","last_active":"2024-04-24T23:26:17.170205+0000","last_peered":"2024-04-24T23:26:17.170205+0000","last_clean":"2024-04-24T23:26:17.170205+0000","last_became_active":"2024-04-24T23:26:17.167351+0000","last_became_peered":"2024-04-24T23:26:17.167351+0000","last_unstale":"2024-04-24T23:26:17.170205+0000","last_undegraded":"2024-04-24T23:26:17.170205+0000","last_fullsized":"2024-04-24T23:26:17.170205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:39.409219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.151934+0000","last_change":"2024-04-24T23:26:17.151934+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.151934+0000","last_undegraded":"2024-04-24T23:26:17.151934+0000","last_fullsized":"2024-04-24T23:26:17.151934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.165107+0000","last_change":"2024-04-24T23:26:17.162901+0000","last_active":"2024-04-24T23:26:17.165107+0000","last_peered":"2024-04-24T23:26:17.165107+0000","last_clean":"2024-04-24T23:26:17.165107+0000","last_became_active":"2024-04-24T23:26:17.162604+0000","last_became_peered":"2024-04-24T23:26:17.162604+0000","last_unstale":"2024-04-24T23:26:17.165107+0000","last_undegraded":"2024-04-24T23:26:17.165107+0000","last_fullsized":"2024-04-24T23:26:17.165107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:37.984293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.151319+0000","last_change":"2024-04-24T23:26:17.151319+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.151319+0000","last_undegraded":"2024-04-24T23:26:17.151319+0000","last_fullsized":"2024-04-24T23:26:17.151319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170103+0000","last_change":"2024-04-24T23:26:17.167550+0000","last_active":"2024-04-24T23:26:17.170103+0000","last_peered":"2024-04-24T23:26:17.170103+0000","last_clean":"2024-04-24T23:26:17.170103+0000","last_became_active":"2024-04-24T23:26:17.167229+0000","last_became_peered":"2024-04-24T23:26:17.167229+0000","last_unstale":"2024-04-24T23:26:17.170103+0000","last_undegraded":"2024-04-24T23:26:17.170103+0000","last_fullsized":"2024-04-24T23:26:17.170103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:30.940114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148656+0000","last_change":"2024-04-24T23:26:17.148656+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148656+0000","last_undegraded":"2024-04-24T23:26:17.148656+0000","last_fullsized":"2024-04-24T23:26:17.148656+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170242+0000","last_change":"2024-04-24T23:26:17.168677+0000","last_active":"2024-04-24T23:26:17.170242+0000","last_peered":"2024-04-24T23:26:17.170242+0000","last_clean":"2024-04-24T23:26:17.170242+0000","last_became_active":"2024-04-24T23:26:17.168354+0000","last_became_peered":"2024-04-24T23:26:17.168354+0000","last_unstale":"2024-04-24T23:26:17.170242+0000","last_undegraded":"2024-04-24T23:26:17.170242+0000","last_fullsized":"2024-04-24T23:26:17.170242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:59.598138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.160360+0000","last_change":"2024-04-24T23:26:17.160360+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.160360+0000","last_undegraded":"2024-04-24T23:26:17.160360+0000","last_fullsized":"2024-04-24T23:26:17.160360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170673+0000","last_change":"2024-04-24T23:26:17.170110+0000","last_active":"2024-04-24T23:26:17.170673+0000","last_peered":"2024-04-24T23:26:17.170673+0000","last_clean":"2024-04-24T23:26:17.170673+0000","last_became_active":"2024-04-24T23:26:17.169755+0000","last_became_peered":"2024-04-24T23:26:17.169755+0000","last_unstale":"2024-04-24T23:26:17.170673+0000","last_undegraded":"2024-04-24T23:26:17.170673+0000","last_fullsized":"2024-04-24T23:26:17.170673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:03.234791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154473+0000","last_change":"2024-04-24T23:26:17.154473+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154473+0000","last_undegraded":"2024-04-24T23:26:17.154473+0000","last_fullsized":"2024-04-24T23:26:17.154473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.152470+0000","last_change":"2024-04-24T23:26:17.151318+0000","last_active":"2024-04-24T23:26:17.152470+0000","last_peered":"2024-04-24T23:26:17.152470+0000","last_clean":"2024-04-24T23:26:17.152470+0000","last_became_active":"2024-04-24T23:26:17.151022+0000","last_became_peered":"2024-04-24T23:26:17.151022+0000","last_unstale":"2024-04-24T23:26:17.152470+0000","last_undegraded":"2024-04-24T23:26:17.152470+0000","last_fullsized":"2024-04-24T23:26:17.152470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:25.593578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.156618+0000","last_change":"2024-04-24T23:26:17.156618+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.156618+0000","last_undegraded":"2024-04-24T23:26:17.156618+0000","last_fullsized":"2024-04-24T23:26:17.156618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.165078+0000","last_change":"2024-04-24T23:26:17.162943+0000","last_active":"2024-04-24T23:26:17.165078+0000","last_peered":"2024-04-24T23:26:17.165078+0000","last_clean":"2024-04-24T23:26:17.165078+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:17.165078+0000","last_undegraded":"2024-04-24T23:26:17.165078+0000","last_fullsized":"2024-04-24T23:26:17.165078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:59.927814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153902+0000","last_change":"2024-04-24T23:26:17.153902+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153902+0000","last_undegraded":"2024-04-24T23:26:17.153902+0000","last_fullsized":"2024-04-24T23:26:17.153902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164541+0000","last_change":"2024-04-24T23:26:17.163630+0000","last_active":"2024-04-24T23:26:17.164541+0000","last_peered":"2024-04-24T23:26:17.164541+0000","last_clean":"2024-04-24T23:26:17.164541+0000","last_became_active":"2024-04-24T23:26:17.163331+0000","last_became_peered":"2024-04-24T23:26:17.163331+0000","last_unstale":"2024-04-24T23:26:17.164541+0000","last_undegraded":"2024-04-24T23:26:17.164541+0000","last_fullsized":"2024-04-24T23:26:17.164541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:47.984272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152815+0000","last_change":"2024-04-24T23:26:17.152815+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152815+0000","last_undegraded":"2024-04-24T23:26:17.152815+0000","last_fullsized":"2024-04-24T23:26:17.152815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.161936+0000","last_change":"2024-04-24T23:26:17.160593+0000","last_active":"2024-04-24T23:26:17.161936+0000","last_peered":"2024-04-24T23:26:17.161936+0000","last_clean":"2024-04-24T23:26:17.161936+0000","last_became_active":"2024-04-24T23:26:17.160303+0000","last_became_peered":"2024-04-24T23:26:17.160303+0000","last_unstale":"2024-04-24T23:26:17.161936+0000","last_undegraded":"2024-04-24T23:26:17.161936+0000","last_fullsized":"2024-04-24T23:26:17.161936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:18.000135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.151917+0000","last_change":"2024-04-24T23:26:17.151917+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.151917+0000","last_undegraded":"2024-04-24T23:26:17.151917+0000","last_fullsized":"2024-04-24T23:26:17.151917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170145+0000","last_change":"2024-04-24T23:26:17.165945+0000","last_active":"2024-04-24T23:26:17.170145+0000","last_peered":"2024-04-24T23:26:17.170145+0000","last_clean":"2024-04-24T23:26:17.170145+0000","last_became_active":"2024-04-24T23:26:17.164750+0000","last_became_peered":"2024-04-24T23:26:17.164750+0000","last_unstale":"2024-04-24T23:26:17.170145+0000","last_undegraded":"2024-04-24T23:26:17.170145+0000","last_fullsized":"2024-04-24T23:26:17.170145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:55.620251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153107+0000","last_change":"2024-04-24T23:26:17.153107+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153107+0000","last_undegraded":"2024-04-24T23:26:17.153107+0000","last_fullsized":"2024-04-24T23:26:17.153107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167256+0000","last_change":"2024-04-24T23:26:17.165977+0000","last_active":"2024-04-24T23:26:17.167256+0000","last_peered":"2024-04-24T23:26:17.167256+0000","last_clean":"2024-04-24T23:26:17.167256+0000","last_became_active":"2024-04-24T23:26:17.165641+0000","last_became_peered":"2024-04-24T23:26:17.165641+0000","last_unstale":"2024-04-24T23:26:17.167256+0000","last_undegraded":"2024-04-24T23:26:17.167256+0000","last_fullsized":"2024-04-24T23:26:17.167256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:22.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159384+0000","last_change":"2024-04-24T23:26:17.159384+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159384+0000","last_undegraded":"2024-04-24T23:26:17.159384+0000","last_fullsized":"2024-04-24T23:26:17.159384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170579+0000","last_change":"2024-04-24T23:26:17.163913+0000","last_active":"2024-04-24T23:26:17.170579+0000","last_peered":"2024-04-24T23:26:17.170579+0000","last_clean":"2024-04-24T23:26:17.170579+0000","last_became_active":"2024-04-24T23:26:17.163640+0000","last_became_peered":"2024-04-24T23:26:17.163640+0000","last_unstale":"2024-04-24T23:26:17.170579+0000","last_undegraded":"2024-04-24T23:26:17.170579+0000","last_fullsized":"2024-04-24T23:26:17.170579+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:18.000135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153797+0000","last_change":"2024-04-24T23:26:17.153797+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153797+0000","last_undegraded":"2024-04-24T23:26:17.153797+0000","last_fullsized":"2024-04-24T23:26:17.153797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170573+0000","last_change":"2024-04-24T23:26:17.167926+0000","last_active":"2024-04-24T23:26:17.170573+0000","last_peered":"2024-04-24T23:26:17.170573+0000","last_clean":"2024-04-24T23:26:17.170573+0000","last_became_active":"2024-04-24T23:26:17.167660+0000","last_became_peered":"2024-04-24T23:26:17.167660+0000","last_unstale":"2024-04-24T23:26:17.170573+0000","last_undegraded":"2024-04-24T23:26:17.170573+0000","last_fullsized":"2024-04-24T23:26:17.170573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:54.496665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154193+0000","last_change":"2024-04-24T23:26:17.154193+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154193+0000","last_undegraded":"2024-04-24T23:26:17.154193+0000","last_fullsized":"2024-04-24T23:26:17.154193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.161464+0000","last_change":"2024-04-24T23:26:17.158322+0000","last_active":"2024-04-24T23:26:17.161464+0000","last_peered":"2024-04-24T23:26:17.161464+0000","last_clean":"2024-04-24T23:26:17.161464+0000","last_became_active":"2024-04-24T23:26:17.158052+0000","last_became_peered":"2024-04-24T23:26:17.158052+0000","last_unstale":"2024-04-24T23:26:17.161464+0000","last_undegraded":"2024-04-24T23:26:17.161464+0000","last_fullsized":"2024-04-24T23:26:17.161464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:07.133490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148457+0000","last_change":"2024-04-24T23:26:17.148457+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148457+0000","last_undegraded":"2024-04-24T23:26:17.148457+0000","last_fullsized":"2024-04-24T23:26:17.148457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.169586+0000","last_change":"2024-04-24T23:26:17.167861+0000","last_active":"2024-04-24T23:26:17.169586+0000","last_peered":"2024-04-24T23:26:17.169586+0000","last_clean":"2024-04-24T23:26:17.169586+0000","last_became_active":"2024-04-24T23:26:17.167598+0000","last_became_peered":"2024-04-24T23:26:17.167598+0000","last_unstale":"2024-04-24T23:26:17.169586+0000","last_undegraded":"2024-04-24T23:26:17.169586+0000","last_fullsized":"2024-04-24T23:26:17.169586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:27.969964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152735+0000","last_change":"2024-04-24T23:26:17.152735+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152735+0000","last_undegraded":"2024-04-24T23:26:17.152735+0000","last_fullsized":"2024-04-24T23:26:17.152735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.168211+0000","last_change":"2024-04-24T23:26:17.165956+0000","last_active":"2024-04-24T23:26:17.168211+0000","last_peered":"2024-04-24T23:26:17.168211+0000","last_clean":"2024-04-24T23:26:17.168211+0000","last_became_active":"2024-04-24T23:26:17.165633+0000","last_became_peered":"2024-04-24T23:26:17.165633+0000","last_unstale":"2024-04-24T23:26:17.168211+0000","last_undegraded":"2024-04-24T23:26:17.168211+0000","last_fullsized":"2024-04-24T23:26:17.168211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:24.199382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.149169+0000","last_change":"2024-04-24T23:26:17.149169+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.149169+0000","last_undegraded":"2024-04-24T23:26:17.149169+0000","last_fullsized":"2024-04-24T23:26:17.149169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.161402+0000","last_change":"2024-04-24T23:26:17.157597+0000","last_active":"2024-04-24T23:26:17.161402+0000","last_peered":"2024-04-24T23:26:17.161402+0000","last_clean":"2024-04-24T23:26:17.161402+0000","last_became_active":"2024-04-24T23:26:17.157311+0000","last_became_peered":"2024-04-24T23:26:17.157311+0000","last_unstale":"2024-04-24T23:26:17.161402+0000","last_undegraded":"2024-04-24T23:26:17.161402+0000","last_fullsized":"2024-04-24T23:26:17.161402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:18.468882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152830+0000","last_change":"2024-04-24T23:26:17.152830+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152830+0000","last_undegraded":"2024-04-24T23:26:17.152830+0000","last_fullsized":"2024-04-24T23:26:17.152830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.165174+0000","last_change":"2024-04-24T23:26:17.163231+0000","last_active":"2024-04-24T23:26:17.165174+0000","last_peered":"2024-04-24T23:26:17.165174+0000","last_clean":"2024-04-24T23:26:17.165174+0000","last_became_active":"2024-04-24T23:26:17.162790+0000","last_became_peered":"2024-04-24T23:26:17.162790+0000","last_unstale":"2024-04-24T23:26:17.165174+0000","last_undegraded":"2024-04-24T23:26:17.165174+0000","last_fullsized":"2024-04-24T23:26:17.165174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:42.373513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.154400+0000","last_change":"2024-04-24T23:26:17.154400+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.154400+0000","last_undegraded":"2024-04-24T23:26:17.154400+0000","last_fullsized":"2024-04-24T23:26:17.154400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164991+0000","last_change":"2024-04-24T23:26:17.161814+0000","last_active":"2024-04-24T23:26:17.164991+0000","last_peered":"2024-04-24T23:26:17.164991+0000","last_clean":"2024-04-24T23:26:17.164991+0000","last_became_active":"2024-04-24T23:26:17.161485+0000","last_became_peered":"2024-04-24T23:26:17.161485+0000","last_unstale":"2024-04-24T23:26:17.164991+0000","last_undegraded":"2024-04-24T23:26:17.164991+0000","last_fullsized":"2024-04-24T23:26:17.164991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:18.807328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.158100+0000","last_change":"2024-04-24T23:26:17.158100+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.158100+0000","last_undegraded":"2024-04-24T23:26:17.158100+0000","last_fullsized":"2024-04-24T23:26:17.158100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170350+0000","last_change":"2024-04-24T23:26:17.169621+0000","last_active":"2024-04-24T23:26:17.170350+0000","last_peered":"2024-04-24T23:26:17.170350+0000","last_clean":"2024-04-24T23:26:17.170350+0000","last_became_active":"2024-04-24T23:26:17.169195+0000","last_became_peered":"2024-04-24T23:26:17.169195+0000","last_unstale":"2024-04-24T23:26:17.170350+0000","last_undegraded":"2024-04-24T23:26:17.170350+0000","last_fullsized":"2024-04-24T23:26:17.170350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:42.773783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153079+0000","last_change":"2024-04-24T23:26:17.153079+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153079+0000","last_undegraded":"2024-04-24T23:26:17.153079+0000","last_fullsized":"2024-04-24T23:26:17.153079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170341+0000","last_change":"2024-04-24T23:26:17.169611+0000","last_active":"2024-04-24T23:26:17.170341+0000","last_peered":"2024-04-24T23:26:17.170341+0000","last_clean":"2024-04-24T23:26:17.170341+0000","last_became_active":"2024-04-24T23:26:17.169304+0000","last_became_peered":"2024-04-24T23:26:17.169304+0000","last_unstale":"2024-04-24T23:26:17.170341+0000","last_undegraded":"2024-04-24T23:26:17.170341+0000","last_fullsized":"2024-04-24T23:26:17.170341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:21.760106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.148445+0000","last_change":"2024-04-24T23:26:17.148445+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.148445+0000","last_undegraded":"2024-04-24T23:26:17.148445+0000","last_fullsized":"2024-04-24T23:26:17.148445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170099+0000","last_change":"2024-04-24T23:26:17.168469+0000","last_active":"2024-04-24T23:26:17.170099+0000","last_peered":"2024-04-24T23:26:17.170099+0000","last_clean":"2024-04-24T23:26:17.170099+0000","last_became_active":"2024-04-24T23:26:17.167305+0000","last_became_peered":"2024-04-24T23:26:17.167305+0000","last_unstale":"2024-04-24T23:26:17.170099+0000","last_undegraded":"2024-04-24T23:26:17.170099+0000","last_fullsized":"2024-04-24T23:26:17.170099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:08.429123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159242+0000","last_change":"2024-04-24T23:26:17.159242+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159242+0000","last_undegraded":"2024-04-24T23:26:17.159242+0000","last_fullsized":"2024-04-24T23:26:17.159242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.164655+0000","last_change":"2024-04-24T23:26:17.162968+0000","last_active":"2024-04-24T23:26:17.164655+0000","last_peered":"2024-04-24T23:26:17.164655+0000","last_clean":"2024-04-24T23:26:17.164655+0000","last_became_active":"2024-04-24T23:26:17.162691+0000","last_became_peered":"2024-04-24T23:26:17.162691+0000","last_unstale":"2024-04-24T23:26:17.164655+0000","last_undegraded":"2024-04-24T23:26:17.164655+0000","last_fullsized":"2024-04-24T23:26:17.164655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.997763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.152026+0000","last_change":"2024-04-24T23:26:17.152026+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.152026+0000","last_undegraded":"2024-04-24T23:26:17.152026+0000","last_fullsized":"2024-04-24T23:26:17.152026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.170205+0000","last_change":"2024-04-24T23:26:17.168302+0000","last_active":"2024-04-24T23:26:17.170205+0000","last_peered":"2024-04-24T23:26:17.170205+0000","last_clean":"2024-04-24T23:26:17.170205+0000","last_became_active":"2024-04-24T23:26:17.167711+0000","last_became_peered":"2024-04-24T23:26:17.167711+0000","last_unstale":"2024-04-24T23:26:17.170205+0000","last_undegraded":"2024-04-24T23:26:17.170205+0000","last_fullsized":"2024-04-24T23:26:17.170205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:01.210692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.159455+0000","last_change":"2024-04-24T23:26:17.159455+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.159455+0000","last_undegraded":"2024-04-24T23:26:17.159455+0000","last_fullsized":"2024-04-24T23:26:17.159455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.159915+0000","last_change":"2024-04-24T23:26:17.156009+0000","last_active":"2024-04-24T23:26:17.159915+0000","last_peered":"2024-04-24T23:26:17.159915+0000","last_clean":"2024-04-24T23:26:17.159915+0000","last_became_active":"2024-04-24T23:26:17.155739+0000","last_became_peered":"2024-04-24T23:26:17.155739+0000","last_unstale":"2024-04-24T23:26:17.159915+0000","last_undegraded":"2024-04-24T23:26:17.159915+0000","last_fullsized":"2024-04-24T23:26:17.159915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:59.737718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T23:26:17.153256+0000","last_change":"2024-04-24T23:26:17.153256+0000","last_active":"2024-04-24T23:26:17.143261+0000","last_peered":"2024-04-24T23:26:17.143261+0000","last_clean":"2024-04-24T23:26:17.143261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:26:17.153256+0000","last_undegraded":"2024-04-24T23:26:17.153256+0000","last_fullsized":"2024-04-24T23:26:17.153256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.167714+0000","last_change":"2024-04-24T23:26:17.166166+0000","last_active":"2024-04-24T23:26:17.167714+0000","last_peered":"2024-04-24T23:26:17.167714+0000","last_clean":"2024-04-24T23:26:17.167714+0000","last_became_active":"2024-04-24T23:26:17.165855+0000","last_became_peered":"2024-04-24T23:26:17.165855+0000","last_unstale":"2024-04-24T23:26:17.167714+0000","last_undegraded":"2024-04-24T23:26:17.167714+0000","last_fullsized":"2024-04-24T23:26:17.167714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:23.185199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":278,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T23:26:17.155269+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:26:17.155269+0000","last_peered":"2024-04-24T23:26:17.155269+0000","last_clean":"2024-04-24T23:26:17.155269+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:26:17.155269+0000","last_undegraded":"2024-04-24T23:26:17.155269+0000","last_fullsized":"2024-04-24T23:26:17.155269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1564672,"data_stored":1004549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":420456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335544,"statfs":{"total":96636764160,"available":96599597056,"internally_reserved":0,"allocated":970752,"data_stored":414181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":983040,"data_stored":420456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1576960,"data_stored":1010824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.161035+0000', 'last_change': '2024-04-24T23:26:17.161035+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.161035+0000', 'last_undegraded': '2024-04-24T23:26:17.161035+0000', 'last_fullsized': '2024-04-24T23:26:17.161035+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152937+0000', 'last_change': '2024-04-24T23:26:17.152937+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152937+0000', 'last_undegraded': '2024-04-24T23:26:17.152937+0000', 'last_fullsized': '2024-04-24T23:26:17.152937+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152655+0000', 'last_change': '2024-04-24T23:26:17.152655+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152655+0000', 'last_undegraded': '2024-04-24T23:26:17.152655+0000', 'last_fullsized': '2024-04-24T23:26:17.152655+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.156363+0000', 'last_change': '2024-04-24T23:26:17.156363+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.156363+0000', 'last_undegraded': '2024-04-24T23:26:17.156363+0000', 'last_fullsized': '2024-04-24T23:26:17.156363+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2024-04-24T23:26:22.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.159303+0000', 'last_change': '2024-04-24T23:26:17.159303+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.159303+0000', 'last_undegraded': '2024-04-24T23:26:17.159303+0000', 'last_fullsized': '2024-04-24T23:26:17.159303+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.149287+0000', 'last_change': '2024-04-24T23:26:17.149287+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.149287+0000', 'last_undegraded': '2024-04-24T23:26:17.149287+0000', 'last_fullsized': '2024-04-24T23:26:17.149287+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.155893+0000', 'last_change': '2024-04-24T23:26:17.155893+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.155893+0000', 'last_undegraded': '2024-04-24T23:26:17.155893+0000', 'last_fullsized': '2024-04-24T23:26:17.155893+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.148447+0000', 'last_change': '2024-04-24T23:26:17.148447+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.148447+0000', 'last_undegraded': '2024-04-24T23:26:17.148447+0000', 'last_fullsized': '2024-04-24T23:26:17.148447+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.149122+0000', 'last_change': '2024-04-24T23:26:17.149122+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.149122+0000', 'last_undegraded': '2024-04-24T23:26:17.149122+0000', 'last_fullsized': '2024-04-24T23:26:17.149122+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.159010+0000', 'last_change': '2024-04-24T23:26:17.159010+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.159010+0000', 'last_undegraded': '2024-04-24T23:26:17.159010+0000', 'last_fullsized': '2024-04-24T23:26:17.159010+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152125+0000', 'last_change': '2024-04-24T23:26:17.152125+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152125+0000', 'last_undegraded': '2024-04-24T23:26:17.152125+0000', 'last_fullsized': '2024-04-24T23:26:17.152125+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.160127+0000', 'last_change': '2024-04-24T23:26:17.160127+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.160127+0000', 'last_undegraded': '2024-04-24T23:26:17.160127+0000', 'last_fullsized': '2024-04-24T23:26:17.160127+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.154328+0000', 'last_change': '2024-04-24T23:26:17.154328+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.154328+0000', 'last_undegraded': '2024-04-24T23:26:17.154328+0000', 'last_fullsized': '2024-04-24T23:26:17.154328+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.154638+0000', 'last_change': '2024-04-24T23:26:17.154638+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.154638+0000', 'last_undegraded': '2024-04-24T23:26:17.154638+0000', 'last_fullsized': '2024-04-24T23:26:17.154638+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.156466+0000', 'last_change': '2024-04-24T23:26:17.156466+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.156466+0000', 'last_undegraded': '2024-04-24T23:26:17.156466+0000', 'last_fullsized': '2024-04-24T23:26:17.156466+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2024-04-24T23:26:22.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.158764+0000', 'last_change': '2024-04-24T23:26:17.158764+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.158764+0000', 'last_undegraded': '2024-04-24T23:26:17.158764+0000', 'last_fullsized': '2024-04-24T23:26:17.158764+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.158461+0000', 'last_change': '2024-04-24T23:26:17.158461+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.158461+0000', 'last_undegraded': '2024-04-24T23:26:17.158461+0000', 'last_fullsized': '2024-04-24T23:26:17.158461+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153010+0000', 'last_change': '2024-04-24T23:26:17.153010+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153010+0000', 'last_undegraded': '2024-04-24T23:26:17.153010+0000', 'last_fullsized': '2024-04-24T23:26:17.153010+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.159129+0000', 'last_change': '2024-04-24T23:26:17.159129+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.159129+0000', 'last_undegraded': '2024-04-24T23:26:17.159129+0000', 'last_fullsized': '2024-04-24T23:26:17.159129+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.155234+0000', 'last_change': '2024-04-24T23:26:17.155234+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.155234+0000', 'last_undegraded': '2024-04-24T23:26:17.155234+0000', 'last_fullsized': '2024-04-24T23:26:17.155234+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2024-04-24T23:26:22.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.155233+0000', 'last_change': '2024-04-24T23:26:17.155233+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.155233+0000', 'last_undegraded': '2024-04-24T23:26:17.155233+0000', 'last_fullsized': '2024-04-24T23:26:17.155233+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.154445+0000', 'last_change': '2024-04-24T23:26:17.154445+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.154445+0000', 'last_undegraded': '2024-04-24T23:26:17.154445+0000', 'last_fullsized': '2024-04-24T23:26:17.154445+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.159956+0000', 'last_change': '2024-04-24T23:26:17.159956+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.159956+0000', 'last_undegraded': '2024-04-24T23:26:17.159956+0000', 'last_fullsized': '2024-04-24T23:26:17.159956+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153261+0000', 'last_change': '2024-04-24T23:26:17.153261+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153261+0000', 'last_undegraded': '2024-04-24T23:26:17.153261+0000', 'last_fullsized': '2024-04-24T23:26:17.153261+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153219+0000', 'last_change': '2024-04-24T23:26:17.153219+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153219+0000', 'last_undegraded': '2024-04-24T23:26:17.153219+0000', 'last_fullsized': '2024-04-24T23:26:17.153219+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153550+0000', 'last_change': '2024-04-24T23:26:17.153550+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153550+0000', 'last_undegraded': '2024-04-24T23:26:17.153550+0000', 'last_fullsized': '2024-04-24T23:26:17.153550+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:26:22.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.159847+0000', 'last_change': '2024-04-24T23:26:17.159847+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.159847+0000', 'last_undegraded': '2024-04-24T23:26:17.159847+0000', 'last_fullsized': '2024-04-24T23:26:17.159847+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.159058+0000', 'last_change': '2024-04-24T23:26:17.159058+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.159058+0000', 'last_undegraded': '2024-04-24T23:26:17.159058+0000', 'last_fullsized': '2024-04-24T23:26:17.159058+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152080+0000', 'last_change': '2024-04-24T23:26:17.152080+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152080+0000', 'last_undegraded': '2024-04-24T23:26:17.152080+0000', 'last_fullsized': '2024-04-24T23:26:17.152080+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.154029+0000', 'last_change': '2024-04-24T23:26:17.154029+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.154029+0000', 'last_undegraded': '2024-04-24T23:26:17.154029+0000', 'last_fullsized': '2024-04-24T23:26:17.154029+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.155927+0000', 'last_change': '2024-04-24T23:26:17.155927+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.155927+0000', 'last_undegraded': '2024-04-24T23:26:17.155927+0000', 'last_fullsized': '2024-04-24T23:26:17.155927+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2024-04-24T23:26:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.154894+0000', 'last_change': '2024-04-24T23:26:17.154894+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.154894+0000', 'last_undegraded': '2024-04-24T23:26:17.154894+0000', 'last_fullsized': '2024-04-24T23:26:17.154894+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.155905+0000', 'last_change': '2024-04-24T23:26:17.155905+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.155905+0000', 'last_undegraded': '2024-04-24T23:26:17.155905+0000', 'last_fullsized': '2024-04-24T23:26:17.155905+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.160325+0000', 'last_change': '2024-04-24T23:26:17.160325+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.160325+0000', 'last_undegraded': '2024-04-24T23:26:17.160325+0000', 'last_fullsized': '2024-04-24T23:26:17.160325+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.158589+0000', 'last_change': '2024-04-24T23:26:17.158589+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.158589+0000', 'last_undegraded': '2024-04-24T23:26:17.158589+0000', 'last_fullsized': '2024-04-24T23:26:17.158589+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152137+0000', 'last_change': '2024-04-24T23:26:17.152137+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152137+0000', 'last_undegraded': '2024-04-24T23:26:17.152137+0000', 'last_fullsized': '2024-04-24T23:26:17.152137+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153602+0000', 'last_change': '2024-04-24T23:26:17.153602+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153602+0000', 'last_undegraded': '2024-04-24T23:26:17.153602+0000', 'last_fullsized': '2024-04-24T23:26:17.153602+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.155159+0000', 'last_change': '2024-04-24T23:26:17.155159+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.155159+0000', 'last_undegraded': '2024-04-24T23:26:17.155159+0000', 'last_fullsized': '2024-04-24T23:26:17.155159+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152635+0000', 'last_change': '2024-04-24T23:26:17.152635+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152635+0000', 'last_undegraded': '2024-04-24T23:26:17.152635+0000', 'last_fullsized': '2024-04-24T23:26:17.152635+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.151934+0000', 'last_change': '2024-04-24T23:26:17.151934+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.151934+0000', 'last_undegraded': '2024-04-24T23:26:17.151934+0000', 'last_fullsized': '2024-04-24T23:26:17.151934+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.151319+0000', 'last_change': '2024-04-24T23:26:17.151319+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.151319+0000', 'last_undegraded': '2024-04-24T23:26:17.151319+0000', 'last_fullsized': '2024-04-24T23:26:17.151319+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.148656+0000', 'last_change': '2024-04-24T23:26:17.148656+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.148656+0000', 'last_undegraded': '2024-04-24T23:26:17.148656+0000', 'last_fullsized': '2024-04-24T23:26:17.148656+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2024-04-24T23:26:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.160360+0000', 'last_change': '2024-04-24T23:26:17.160360+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.160360+0000', 'last_undegraded': '2024-04-24T23:26:17.160360+0000', 'last_fullsized': '2024-04-24T23:26:17.160360+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.154473+0000', 'last_change': '2024-04-24T23:26:17.154473+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.154473+0000', 'last_undegraded': '2024-04-24T23:26:17.154473+0000', 'last_fullsized': '2024-04-24T23:26:17.154473+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.156618+0000', 'last_change': '2024-04-24T23:26:17.156618+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.156618+0000', 'last_undegraded': '2024-04-24T23:26:17.156618+0000', 'last_fullsized': '2024-04-24T23:26:17.156618+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153902+0000', 'last_change': '2024-04-24T23:26:17.153902+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153902+0000', 'last_undegraded': '2024-04-24T23:26:17.153902+0000', 'last_fullsized': '2024-04-24T23:26:17.153902+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152815+0000', 'last_change': '2024-04-24T23:26:17.152815+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152815+0000', 'last_undegraded': '2024-04-24T23:26:17.152815+0000', 'last_fullsized': '2024-04-24T23:26:17.152815+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.151917+0000', 'last_change': '2024-04-24T23:26:17.151917+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.151917+0000', 'last_undegraded': '2024-04-24T23:26:17.151917+0000', 'last_fullsized': '2024-04-24T23:26:17.151917+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153107+0000', 'last_change': '2024-04-24T23:26:17.153107+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153107+0000', 'last_undegraded': '2024-04-24T23:26:17.153107+0000', 'last_fullsized': '2024-04-24T23:26:17.153107+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.159384+0000', 'last_change': '2024-04-24T23:26:17.159384+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.159384+0000', 'last_undegraded': '2024-04-24T23:26:17.159384+0000', 'last_fullsized': '2024-04-24T23:26:17.159384+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153797+0000', 'last_change': '2024-04-24T23:26:17.153797+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153797+0000', 'last_undegraded': '2024-04-24T23:26:17.153797+0000', 'last_fullsized': '2024-04-24T23:26:17.153797+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.154193+0000', 'last_change': '2024-04-24T23:26:17.154193+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.154193+0000', 'last_undegraded': '2024-04-24T23:26:17.154193+0000', 'last_fullsized': '2024-04-24T23:26:17.154193+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.148457+0000', 'last_change': '2024-04-24T23:26:17.148457+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.148457+0000', 'last_undegraded': '2024-04-24T23:26:17.148457+0000', 'last_fullsized': '2024-04-24T23:26:17.148457+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:26:22.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152735+0000', 'last_change': '2024-04-24T23:26:17.152735+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152735+0000', 'last_undegraded': '2024-04-24T23:26:17.152735+0000', 'last_fullsized': '2024-04-24T23:26:17.152735+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.149169+0000', 'last_change': '2024-04-24T23:26:17.149169+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.149169+0000', 'last_undegraded': '2024-04-24T23:26:17.149169+0000', 'last_fullsized': '2024-04-24T23:26:17.149169+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152830+0000', 'last_change': '2024-04-24T23:26:17.152830+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152830+0000', 'last_undegraded': '2024-04-24T23:26:17.152830+0000', 'last_fullsized': '2024-04-24T23:26:17.152830+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.154400+0000', 'last_change': '2024-04-24T23:26:17.154400+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.154400+0000', 'last_undegraded': '2024-04-24T23:26:17.154400+0000', 'last_fullsized': '2024-04-24T23:26:17.154400+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.158100+0000', 'last_change': '2024-04-24T23:26:17.158100+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.158100+0000', 'last_undegraded': '2024-04-24T23:26:17.158100+0000', 'last_fullsized': '2024-04-24T23:26:17.158100+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2024-04-24T23:26:22.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153079+0000', 'last_change': '2024-04-24T23:26:17.153079+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153079+0000', 'last_undegraded': '2024-04-24T23:26:17.153079+0000', 'last_fullsized': '2024-04-24T23:26:17.153079+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.148445+0000', 'last_change': '2024-04-24T23:26:17.148445+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.148445+0000', 'last_undegraded': '2024-04-24T23:26:17.148445+0000', 'last_fullsized': '2024-04-24T23:26:17.148445+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.159242+0000', 'last_change': '2024-04-24T23:26:17.159242+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.159242+0000', 'last_undegraded': '2024-04-24T23:26:17.159242+0000', 'last_fullsized': '2024-04-24T23:26:17.159242+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.152026+0000', 'last_change': '2024-04-24T23:26:17.152026+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.152026+0000', 'last_undegraded': '2024-04-24T23:26:17.152026+0000', 'last_fullsized': '2024-04-24T23:26:17.152026+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.159455+0000', 'last_change': '2024-04-24T23:26:17.159455+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.159455+0000', 'last_undegraded': '2024-04-24T23:26:17.159455+0000', 'last_fullsized': '2024-04-24T23:26:17.159455+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2024-04-24T23:26:22.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-24T23:26:17.153256+0000', 'last_change': '2024-04-24T23:26:17.153256+0000', 'last_active': '2024-04-24T23:26:17.143261+0000', 'last_peered': '2024-04-24T23:26:17.143261+0000', 'last_clean': '2024-04-24T23:26:17.143261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T23:26:17.153256+0000', 'last_undegraded': '2024-04-24T23:26:17.153256+0000', 'last_fullsized': '2024-04-24T23:26:17.153256+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'last_clean_scrub_stamp': '2024-04-24T23:26:17.143261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T23:26:22.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:26:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.015+0000 7f973a6d2640 1 Processor -- start 2024-04-24T23:26:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.015+0000 7f973a6d2640 1 -- start start 2024-04-24T23:26:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.016+0000 7f973a6d2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97340b2830 0x7f973406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.016+0000 7f973a6d2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97340afc50 con 0x7f97340af5e0 2024-04-24T23:26:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.016+0000 7f973a6d2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97340aff30 con 0x7f9734145c40 2024-04-24T23:26:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.016+0000 7f973a6d2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f97340b0860 con 0x7f97340b2830 2024-04-24T23:26:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.016+0000 7f97337fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97340b2830 0x7f973406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.016+0000 7f9738c6f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f97340af5e0 0x7f973415ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55908/0 (socket says 172.21.15.26:55908) 2024-04-24T23:26:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.016+0000 7f9738c6f640 1 -- 172.21.15.26:0/1429116741 learned_addr learned my addr 172.21.15.26:0/1429116741 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.016+0000 7f97337fe640 1 -- 172.21.15.26:0/1429116741 >> v1:172.21.15.29:6789/0 conn(0x7f9734145c40 legacy=0x7f97340b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.017+0000 7f97337fe640 1 -- 172.21.15.26:0/1429116741 >> v1:172.21.15.26:6789/0 conn(0x7f97340af5e0 legacy=0x7f973415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.017+0000 7f97337fe640 1 -- 172.21.15.26:0/1429116741 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97340b0210 con 0x7f97340b2830 2024-04-24T23:26:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.017+0000 7f9732ffd640 1 -- 172.21.15.26:0/1429116741 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609996961 0 0) 0x7f97340afc50 con 0x7f97340af5e0 2024-04-24T23:26:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.017+0000 7f97337fe640 1 --2- 172.21.15.26:0/1429116741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340b2830 0x7f973406bb30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f971c0097a0 tx=0x7f971c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabd86f176da69a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.017+0000 7f9732ffd640 1 -- 172.21.15.26:0/1429116741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c037070 con 0x7f97340b2830 2024-04-24T23:26:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.017+0000 7f9732ffd640 1 -- 172.21.15.26:0/1429116741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971c032440 con 0x7f97340b2830 2024-04-24T23:26:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.017+0000 7f9732ffd640 1 -- 172.21.15.26:0/1429116741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c032810 con 0x7f97340b2830 2024-04-24T23:26:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.018+0000 7f973a6d2640 1 -- 172.21.15.26:0/1429116741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340b2830 msgr2=0x7f973406bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.018+0000 7f973a6d2640 1 --2- 172.21.15.26:0/1429116741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340b2830 0x7f973406bb30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f971c0097a0 tx=0x7f971c02a230 comp rx=0 tx=0).stop 2024-04-24T23:26:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.018+0000 7f973a6d2640 1 -- 172.21.15.26:0/1429116741 shutdown_connections 2024-04-24T23:26:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.018+0000 7f973a6d2640 1 --2- 172.21.15.26:0/1429116741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340b2830 0x7f973406bb30 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.018+0000 7f973a6d2640 1 -- 172.21.15.26:0/1429116741 >> 172.21.15.26:0/1429116741 conn(0x7f973409aa00 msgr2=0x7f97340b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.018+0000 7f973a6d2640 1 -- 172.21.15.26:0/1429116741 shutdown_connections 2024-04-24T23:26:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.018+0000 7f973a6d2640 1 -- 172.21.15.26:0/1429116741 wait complete. 2024-04-24T23:26:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.019+0000 7f973a6d2640 1 Processor -- start 2024-04-24T23:26:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f973a6d2640 1 -- start start 2024-04-24T23:26:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f973a6d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340af5e0 0x7f9734152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f973a6d2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97340b2830 0x7f973413e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f973a6d2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9734145c40 0x7f973409b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f973a6d2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9734146020 con 0x7f97340af5e0 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f973a6d2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f97340af9f0 con 0x7f9734145c40 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f973a6d2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f97340b6f50 con 0x7f97340b2830 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f9733fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340af5e0 0x7f9734152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f9733fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340af5e0 0x7f9734152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56324/0 (socket says 172.21.15.26:56324) 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.020+0000 7f9733fff640 1 -- 172.21.15.26:0/3963071822 learned_addr learned my addr 172.21.15.26:0/3963071822 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9738c6f640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9734145c40 0x7f973409b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f97337fe640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97340b2830 0x7f973413e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9733fff640 1 -- 172.21.15.26:0/3963071822 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97340b2830 msgr2=0x7f973413e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9733fff640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97340b2830 0x7f973413e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9733fff640 1 -- 172.21.15.26:0/3963071822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9734145c40 msgr2=0x7f973409b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9733fff640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9734145c40 0x7f973409b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9733fff640 1 -- 172.21.15.26:0/3963071822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973409b5c0 con 0x7f97340af5e0 2024-04-24T23:26:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f97337fe640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97340b2830 0x7f973413e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9733fff640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340af5e0 0x7f9734152ce0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f972400c450 tx=0x7f972400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9730ff9640 1 -- 172.21.15.26:0/3963071822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97240052d0 con 0x7f97340af5e0 2024-04-24T23:26:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9730ff9640 1 -- 172.21.15.26:0/3963071822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9724005470 con 0x7f97340af5e0 2024-04-24T23:26:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973413be40 con 0x7f97340af5e0 2024-04-24T23:26:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f9730ff9640 1 -- 172.21.15.26:0/3963071822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724012870 con 0x7f97340af5e0 2024-04-24T23:26:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.021+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9734140f30 con 0x7f97340af5e0 2024-04-24T23:26:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.022+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973413c360 con 0x7f97340af5e0 2024-04-24T23:26:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.023+0000 7f9730ff9640 1 -- 172.21.15.26:0/3963071822 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9724012a10 con 0x7f97340af5e0 2024-04-24T23:26:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.023+0000 7f9730ff9640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971406be30 0x7f971406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.023+0000 7f9730ff9640 1 -- 172.21.15.26:0/3963071822 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f972408a960 con 0x7f97340af5e0 2024-04-24T23:26:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.023+0000 7f97337fe640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971406be30 0x7f971406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.024+0000 7f97337fe640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971406be30 0x7f971406e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f971c034000 tx=0x7f971c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.027+0000 7f9730ff9640 1 -- 172.21.15.26:0/3963071822 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f973413c360 con 0x7f97340af5e0 2024-04-24T23:26:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.279+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f97340694c0 con 0x7f97340af5e0 2024-04-24T23:26:23.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.279+0000 7f9730ff9640 1 -- 172.21.15.26:0/3963071822 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1454 (secure 0 0 0) 0x7f97240593a0 con 0x7f97340af5e0 2024-04-24T23:26:23.280 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:23.280 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":352,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":82,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":277229568,"bytes_avail":772816883712,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":84,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-24T23:25:11.372341+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:26:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971406be30 msgr2=0x7f971406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971406be30 0x7f971406e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f971c034000 tx=0x7f971c02e000 comp rx=0 tx=0).stop 2024-04-24T23:26:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340af5e0 msgr2=0x7f9734152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340af5e0 0x7f9734152ce0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f972400c450 tx=0x7f972400c910 comp rx=0 tx=0).stop 2024-04-24T23:26:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 shutdown_connections 2024-04-24T23:26:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971406be30 0x7f971406e2f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9734145c40 0x7f973409b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f97340b2830 0x7f973413e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 --2- 172.21.15.26:0/3963071822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97340af5e0 0x7f9734152ce0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 >> 172.21.15.26:0/3963071822 conn(0x7f973409aa00 msgr2=0x7f97340b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 shutdown_connections 2024-04-24T23:26:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.281+0000 7f973a6d2640 1 -- 172.21.15.26:0/3963071822 wait complete. 2024-04-24T23:26:23.293 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-24T23:26:23.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:26:23.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.393+0000 7efe26e2a640 1 Processor -- start 2024-04-24T23:26:23.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.394+0000 7efe26e2a640 1 -- start start 2024-04-24T23:26:23.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.394+0000 7efe26e2a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 0x7efe200d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.394+0000 7efe26e2a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efe200d22e0 0x7efe200d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.394+0000 7efe26e2a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe200d32f0 con 0x7efe200b3130 2024-04-24T23:26:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.394+0000 7efe26e2a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efe200d1ab0 con 0x7efe200d22e0 2024-04-24T23:26:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.394+0000 7efe26e2a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efe200d2ae0 con 0x7efe2015e570 2024-04-24T23:26:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe253c7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efe200d22e0 0x7efe200d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe24bc6640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 0x7efe200d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe1ffff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efe200b3130 0x7efe200d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:53688/0 (socket says 172.21.15.26:53688) 2024-04-24T23:26:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe1ffff640 1 -- 172.21.15.26:0/1416374049 learned_addr learned my addr 172.21.15.26:0/1416374049 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe1f7fe640 1 -- 172.21.15.26:0/1416374049 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967008530 0 0) 0x7efe0c001110 con 0x7efe200b3130 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe1f7fe640 1 -- 172.21.15.26:0/1416374049 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe10003520 con 0x7efe200b3130 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe24bc6640 1 -- 172.21.15.26:0/1416374049 >> v2:172.21.15.29:3300/0 conn(0x7efe200d22e0 msgr2=0x7efe200d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe24bc6640 1 --2- 172.21.15.26:0/1416374049 >> v2:172.21.15.29:3300/0 conn(0x7efe200d22e0 0x7efe200d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe24bc6640 1 -- 172.21.15.26:0/1416374049 >> v1:172.21.15.29:6789/0 conn(0x7efe200b3130 legacy=0x7efe200d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe24bc6640 1 -- 172.21.15.26:0/1416374049 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe200d0270 con 0x7efe2015e570 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.395+0000 7efe253c7640 1 --2- 172.21.15.26:0/1416374049 >> v2:172.21.15.29:3300/0 conn(0x7efe200d22e0 0x7efe200d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.396+0000 7efe24bc6640 1 --2- 172.21.15.26:0/1416374049 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 0x7efe200d4160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe140097a0 tx=0x7efe1402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bbeaa0bd6220c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.396+0000 7efe1f7fe640 1 -- 172.21.15.26:0/1416374049 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14037070 con 0x7efe2015e570 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.396+0000 7efe1f7fe640 1 -- 172.21.15.26:0/1416374049 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe14032440 con 0x7efe2015e570 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.396+0000 7efe1f7fe640 1 -- 172.21.15.26:0/1416374049 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe140327f0 con 0x7efe2015e570 2024-04-24T23:26:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.397+0000 7efe26e2a640 1 -- 172.21.15.26:0/1416374049 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 msgr2=0x7efe200d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.397+0000 7efe26e2a640 1 --2- 172.21.15.26:0/1416374049 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 0x7efe200d4160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe140097a0 tx=0x7efe1402a180 comp rx=0 tx=0).stop 2024-04-24T23:26:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.397+0000 7efe26e2a640 1 -- 172.21.15.26:0/1416374049 shutdown_connections 2024-04-24T23:26:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.397+0000 7efe26e2a640 1 --2- 172.21.15.26:0/1416374049 >> v2:172.21.15.29:3300/0 conn(0x7efe200d22e0 0x7efe200d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.397+0000 7efe26e2a640 1 --2- 172.21.15.26:0/1416374049 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 0x7efe200d4160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.397+0000 7efe26e2a640 1 -- 172.21.15.26:0/1416374049 >> 172.21.15.26:0/1416374049 conn(0x7efe2008dbe0 msgr2=0x7efe2009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.397+0000 7efe26e2a640 1 -- 172.21.15.26:0/1416374049 shutdown_connections 2024-04-24T23:26:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.397+0000 7efe26e2a640 1 -- 172.21.15.26:0/1416374049 wait complete. 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.398+0000 7efe26e2a640 1 Processor -- start 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.398+0000 7efe26e2a640 1 -- start start 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe26e2a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe200b3130 0x7efe2014f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe26e2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe200d22e0 0x7efe200d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe26e2a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 0x7efe200d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe26e2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe200cfa70 con 0x7efe200d22e0 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe26e2a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efe200d3010 con 0x7efe200b3130 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe26e2a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efe200cfd50 con 0x7efe2015e570 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe200d22e0 0x7efe200d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe200d22e0 0x7efe200d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56332/0 (socket says 172.21.15.26:56332) 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe1ffff640 1 -- 172.21.15.26:0/1761923564 learned_addr learned my addr 172.21.15.26:0/1761923564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe24bc6640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe200b3130 0x7efe2014f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe253c7640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 0x7efe200d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe1ffff640 1 -- 172.21.15.26:0/1761923564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 msgr2=0x7efe200d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe1ffff640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 0x7efe200d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe1ffff640 1 -- 172.21.15.26:0/1761923564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe200b3130 msgr2=0x7efe2014f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe1ffff640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe200b3130 0x7efe2014f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe1ffff640 1 -- 172.21.15.26:0/1761923564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2009e6e0 con 0x7efe200d22e0 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.399+0000 7efe1ffff640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe200d22e0 0x7efe200d0c00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00c730 tx=0x7efe0c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.400+0000 7efe1d7fa640 1 -- 172.21.15.26:0/1761923564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c017070 con 0x7efe200d22e0 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.400+0000 7efe1d7fa640 1 -- 172.21.15.26:0/1761923564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0c005ad0 con 0x7efe200d22e0 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.400+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe201f4dc0 con 0x7efe200d22e0 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.400+0000 7efe1d7fa640 1 -- 172.21.15.26:0/1761923564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c007e20 con 0x7efe200d22e0 2024-04-24T23:26:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.400+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2009ec80 con 0x7efe200d22e0 2024-04-24T23:26:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.401+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe200694c0 con 0x7efe200d22e0 2024-04-24T23:26:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.401+0000 7efe1d7fa640 1 -- 172.21.15.26:0/1761923564 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe0c01a070 con 0x7efe200d22e0 2024-04-24T23:26:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.401+0000 7efe1d7fa640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe0406be60 0x7efe0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:23.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.402+0000 7efe1d7fa640 1 -- 172.21.15.26:0/1761923564 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7efe0c08a220 con 0x7efe200d22e0 2024-04-24T23:26:23.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.402+0000 7efe24bc6640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe0406be60 0x7efe0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:23.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.402+0000 7efe24bc6640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe0406be60 0x7efe0406e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efe14034000 tx=0x7efe1402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:23.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.406+0000 7efe1d7fa640 1 -- 172.21.15.26:0/1761923564 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe200d32f0 con 0x7efe200d22e0 2024-04-24T23:26:23.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.585+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe2006d180 con 0x7efe0406be60 2024-04-24T23:26:23.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.591+0000 7efe1d7fa640 1 -- 172.21.15.26:0/1761923564 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252081 (secure 0 0 0) 0x7efe2006d180 con 0x7efe0406be60 2024-04-24T23:26:23.595 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:23.596 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:26:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe0406be60 msgr2=0x7efe0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe0406be60 0x7efe0406e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efe14034000 tx=0x7efe1402e000 comp rx=0 tx=0).stop 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe200d22e0 msgr2=0x7efe200d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe200d22e0 0x7efe200d0c00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00c730 tx=0x7efe0c00cbf0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 shutdown_connections 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe0406be60 0x7efe0406e320 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe2015e570 0x7efe200d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe200d22e0 0x7efe200d0c00 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 --2- 172.21.15.26:0/1761923564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe200b3130 0x7efe2014f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 >> 172.21.15.26:0/1761923564 conn(0x7efe2008dbe0 msgr2=0x7efe200b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 shutdown_connections 2024-04-24T23:26:23.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:23.594+0000 7efe26e2a640 1 -- 172.21.15.26:0/1761923564 wait complete. 2024-04-24T23:26:23.606 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-04-24T23:26:23.386491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271036,"kb_used_data":9096,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754703684,"statfs":{"total":773094113280,"available":772816572416,"internally_reserved":0,"allocated":9314304,"data_stored":4653804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.240394"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370066+0000","last_change":"2024-04-24T23:26:18.182579+0000","last_active":"2024-04-24T23:26:19.370066+0000","last_peered":"2024-04-24T23:26:19.370066+0000","last_clean":"2024-04-24T23:26:19.370066+0000","last_became_active":"2024-04-24T23:26:18.182259+0000","last_became_peered":"2024-04-24T23:26:18.182259+0000","last_unstale":"2024-04-24T23:26:19.370066+0000","last_undegraded":"2024-04-24T23:26:19.370066+0000","last_fullsized":"2024-04-24T23:26:19.370066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:37.471755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172178+0000","last_change":"2024-04-24T23:26:18.182127+0000","last_active":"2024-04-24T23:26:19.172178+0000","last_peered":"2024-04-24T23:26:19.172178+0000","last_clean":"2024-04-24T23:26:19.172178+0000","last_became_active":"2024-04-24T23:26:18.181815+0000","last_became_peered":"2024-04-24T23:26:18.181815+0000","last_unstale":"2024-04-24T23:26:19.172178+0000","last_undegraded":"2024-04-24T23:26:19.172178+0000","last_fullsized":"2024-04-24T23:26:19.172178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:26.597972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463793+0000","last_change":"2024-04-24T23:26:18.180748+0000","last_active":"2024-04-24T23:26:19.463793+0000","last_peered":"2024-04-24T23:26:19.463793+0000","last_clean":"2024-04-24T23:26:19.463793+0000","last_became_active":"2024-04-24T23:26:18.180423+0000","last_became_peered":"2024-04-24T23:26:18.180423+0000","last_unstale":"2024-04-24T23:26:19.463793+0000","last_undegraded":"2024-04-24T23:26:19.463793+0000","last_fullsized":"2024-04-24T23:26:19.463793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:28.897593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463767+0000","last_change":"2024-04-24T23:26:18.179588+0000","last_active":"2024-04-24T23:26:19.463767+0000","last_peered":"2024-04-24T23:26:19.463767+0000","last_clean":"2024-04-24T23:26:19.463767+0000","last_became_active":"2024-04-24T23:26:18.179338+0000","last_became_peered":"2024-04-24T23:26:18.179338+0000","last_unstale":"2024-04-24T23:26:19.463767+0000","last_undegraded":"2024-04-24T23:26:19.463767+0000","last_fullsized":"2024-04-24T23:26:19.463767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:46.313561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166713+0000","last_change":"2024-04-24T23:26:18.174615+0000","last_active":"2024-04-24T23:26:19.166713+0000","last_peered":"2024-04-24T23:26:19.166713+0000","last_clean":"2024-04-24T23:26:19.166713+0000","last_became_active":"2024-04-24T23:26:18.174345+0000","last_became_peered":"2024-04-24T23:26:18.174345+0000","last_unstale":"2024-04-24T23:26:19.166713+0000","last_undegraded":"2024-04-24T23:26:19.166713+0000","last_fullsized":"2024-04-24T23:26:19.166713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:07.648864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167619+0000","last_change":"2024-04-24T23:26:18.173531+0000","last_active":"2024-04-24T23:26:19.167619+0000","last_peered":"2024-04-24T23:26:19.167619+0000","last_clean":"2024-04-24T23:26:19.167619+0000","last_became_active":"2024-04-24T23:26:18.172003+0000","last_became_peered":"2024-04-24T23:26:18.172003+0000","last_unstale":"2024-04-24T23:26:19.167619+0000","last_undegraded":"2024-04-24T23:26:19.167619+0000","last_fullsized":"2024-04-24T23:26:19.167619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:37.471755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463409+0000","last_change":"2024-04-24T23:26:18.167012+0000","last_active":"2024-04-24T23:26:19.463409+0000","last_peered":"2024-04-24T23:26:19.463409+0000","last_clean":"2024-04-24T23:26:19.463409+0000","last_became_active":"2024-04-24T23:26:18.165907+0000","last_became_peered":"2024-04-24T23:26:18.165907+0000","last_unstale":"2024-04-24T23:26:19.463409+0000","last_undegraded":"2024-04-24T23:26:19.463409+0000","last_fullsized":"2024-04-24T23:26:19.463409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:00.932208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167076+0000","last_change":"2024-04-24T23:26:18.175501+0000","last_active":"2024-04-24T23:26:19.167076+0000","last_peered":"2024-04-24T23:26:19.167076+0000","last_clean":"2024-04-24T23:26:19.167076+0000","last_became_active":"2024-04-24T23:26:18.175151+0000","last_became_peered":"2024-04-24T23:26:18.175151+0000","last_unstale":"2024-04-24T23:26:19.167076+0000","last_undegraded":"2024-04-24T23:26:19.167076+0000","last_fullsized":"2024-04-24T23:26:19.167076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:53.405043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167839+0000","last_change":"2024-04-24T23:26:18.173971+0000","last_active":"2024-04-24T23:26:19.167839+0000","last_peered":"2024-04-24T23:26:19.167839+0000","last_clean":"2024-04-24T23:26:19.167839+0000","last_became_active":"2024-04-24T23:26:18.173649+0000","last_became_peered":"2024-04-24T23:26:18.173649+0000","last_unstale":"2024-04-24T23:26:19.167839+0000","last_undegraded":"2024-04-24T23:26:19.167839+0000","last_fullsized":"2024-04-24T23:26:19.167839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:04.106671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370199+0000","last_change":"2024-04-24T23:26:18.180126+0000","last_active":"2024-04-24T23:26:19.370199+0000","last_peered":"2024-04-24T23:26:19.370199+0000","last_clean":"2024-04-24T23:26:19.370199+0000","last_became_active":"2024-04-24T23:26:18.179798+0000","last_became_peered":"2024-04-24T23:26:18.179798+0000","last_unstale":"2024-04-24T23:26:19.370199+0000","last_undegraded":"2024-04-24T23:26:19.370199+0000","last_fullsized":"2024-04-24T23:26:19.370199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:59.026283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172112+0000","last_change":"2024-04-24T23:26:18.172770+0000","last_active":"2024-04-24T23:26:19.172112+0000","last_peered":"2024-04-24T23:26:19.172112+0000","last_clean":"2024-04-24T23:26:19.172112+0000","last_became_active":"2024-04-24T23:26:18.172478+0000","last_became_peered":"2024-04-24T23:26:18.172478+0000","last_unstale":"2024-04-24T23:26:19.172112+0000","last_undegraded":"2024-04-24T23:26:19.172112+0000","last_fullsized":"2024-04-24T23:26:19.172112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:41.660546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369673+0000","last_change":"2024-04-24T23:26:18.179272+0000","last_active":"2024-04-24T23:26:19.369673+0000","last_peered":"2024-04-24T23:26:19.369673+0000","last_clean":"2024-04-24T23:26:19.369673+0000","last_became_active":"2024-04-24T23:26:18.178927+0000","last_became_peered":"2024-04-24T23:26:18.178927+0000","last_unstale":"2024-04-24T23:26:19.369673+0000","last_undegraded":"2024-04-24T23:26:19.369673+0000","last_fullsized":"2024-04-24T23:26:19.369673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:08.371069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463828+0000","last_change":"2024-04-24T23:26:18.179069+0000","last_active":"2024-04-24T23:26:19.463828+0000","last_peered":"2024-04-24T23:26:19.463828+0000","last_clean":"2024-04-24T23:26:19.463828+0000","last_became_active":"2024-04-24T23:26:18.178734+0000","last_became_peered":"2024-04-24T23:26:18.178734+0000","last_unstale":"2024-04-24T23:26:19.463828+0000","last_undegraded":"2024-04-24T23:26:19.463828+0000","last_fullsized":"2024-04-24T23:26:19.463828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:32.655169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463644+0000","last_change":"2024-04-24T23:26:18.173011+0000","last_active":"2024-04-24T23:26:19.463644+0000","last_peered":"2024-04-24T23:26:19.463644+0000","last_clean":"2024-04-24T23:26:19.463644+0000","last_became_active":"2024-04-24T23:26:18.172727+0000","last_became_peered":"2024-04-24T23:26:18.172727+0000","last_unstale":"2024-04-24T23:26:19.463644+0000","last_undegraded":"2024-04-24T23:26:19.463644+0000","last_fullsized":"2024-04-24T23:26:19.463644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:22.487949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463030+0000","last_change":"2024-04-24T23:26:18.172692+0000","last_active":"2024-04-24T23:26:19.463030+0000","last_peered":"2024-04-24T23:26:19.463030+0000","last_clean":"2024-04-24T23:26:19.463030+0000","last_became_active":"2024-04-24T23:26:18.172391+0000","last_became_peered":"2024-04-24T23:26:18.172391+0000","last_unstale":"2024-04-24T23:26:19.463030+0000","last_undegraded":"2024-04-24T23:26:19.463030+0000","last_fullsized":"2024-04-24T23:26:19.463030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:02.363430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166356+0000","last_change":"2024-04-24T23:26:18.180886+0000","last_active":"2024-04-24T23:26:19.166356+0000","last_peered":"2024-04-24T23:26:19.166356+0000","last_clean":"2024-04-24T23:26:19.166356+0000","last_became_active":"2024-04-24T23:26:18.180601+0000","last_became_peered":"2024-04-24T23:26:18.180601+0000","last_unstale":"2024-04-24T23:26:19.166356+0000","last_undegraded":"2024-04-24T23:26:19.166356+0000","last_fullsized":"2024-04-24T23:26:19.166356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:37.471755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167034+0000","last_change":"2024-04-24T23:26:18.170493+0000","last_active":"2024-04-24T23:26:19.167034+0000","last_peered":"2024-04-24T23:26:19.167034+0000","last_clean":"2024-04-24T23:26:19.167034+0000","last_became_active":"2024-04-24T23:26:18.170190+0000","last_became_peered":"2024-04-24T23:26:18.170190+0000","last_unstale":"2024-04-24T23:26:19.167034+0000","last_undegraded":"2024-04-24T23:26:19.167034+0000","last_fullsized":"2024-04-24T23:26:19.167034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:54.744768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463875+0000","last_change":"2024-04-24T23:26:18.181808+0000","last_active":"2024-04-24T23:26:19.463875+0000","last_peered":"2024-04-24T23:26:19.463875+0000","last_clean":"2024-04-24T23:26:19.463875+0000","last_became_active":"2024-04-24T23:26:18.181513+0000","last_became_peered":"2024-04-24T23:26:18.181513+0000","last_unstale":"2024-04-24T23:26:19.463875+0000","last_undegraded":"2024-04-24T23:26:19.463875+0000","last_fullsized":"2024-04-24T23:26:19.463875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:23.784890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174236+0000","last_change":"2024-04-24T23:26:18.167771+0000","last_active":"2024-04-24T23:26:19.174236+0000","last_peered":"2024-04-24T23:26:19.174236+0000","last_clean":"2024-04-24T23:26:19.174236+0000","last_became_active":"2024-04-24T23:26:18.167488+0000","last_became_peered":"2024-04-24T23:26:18.167488+0000","last_unstale":"2024-04-24T23:26:19.174236+0000","last_undegraded":"2024-04-24T23:26:19.174236+0000","last_fullsized":"2024-04-24T23:26:19.174236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:08.137883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463947+0000","last_change":"2024-04-24T23:26:18.169427+0000","last_active":"2024-04-24T23:26:19.463947+0000","last_peered":"2024-04-24T23:26:19.463947+0000","last_clean":"2024-04-24T23:26:19.463947+0000","last_became_active":"2024-04-24T23:26:18.169139+0000","last_became_peered":"2024-04-24T23:26:18.169139+0000","last_unstale":"2024-04-24T23:26:19.463947+0000","last_undegraded":"2024-04-24T23:26:19.463947+0000","last_fullsized":"2024-04-24T23:26:19.463947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:32.961764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463579+0000","last_change":"2024-04-24T23:26:18.169656+0000","last_active":"2024-04-24T23:26:19.463579+0000","last_peered":"2024-04-24T23:26:19.463579+0000","last_clean":"2024-04-24T23:26:19.463579+0000","last_became_active":"2024-04-24T23:26:18.165677+0000","last_became_peered":"2024-04-24T23:26:18.165677+0000","last_unstale":"2024-04-24T23:26:19.463579+0000","last_undegraded":"2024-04-24T23:26:19.463579+0000","last_fullsized":"2024-04-24T23:26:19.463579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:45.726822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171871+0000","last_change":"2024-04-24T23:26:18.171619+0000","last_active":"2024-04-24T23:26:19.171871+0000","last_peered":"2024-04-24T23:26:19.171871+0000","last_clean":"2024-04-24T23:26:19.171871+0000","last_became_active":"2024-04-24T23:26:18.171306+0000","last_became_peered":"2024-04-24T23:26:18.171306+0000","last_unstale":"2024-04-24T23:26:19.171871+0000","last_undegraded":"2024-04-24T23:26:19.171871+0000","last_fullsized":"2024-04-24T23:26:19.171871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:01.573306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369707+0000","last_change":"2024-04-24T23:26:18.178885+0000","last_active":"2024-04-24T23:26:19.369707+0000","last_peered":"2024-04-24T23:26:19.369707+0000","last_clean":"2024-04-24T23:26:19.369707+0000","last_became_active":"2024-04-24T23:26:18.178432+0000","last_became_peered":"2024-04-24T23:26:18.178432+0000","last_unstale":"2024-04-24T23:26:19.369707+0000","last_undegraded":"2024-04-24T23:26:19.369707+0000","last_fullsized":"2024-04-24T23:26:19.369707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:38.603796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463237+0000","last_change":"2024-04-24T23:26:18.169319+0000","last_active":"2024-04-24T23:26:19.463237+0000","last_peered":"2024-04-24T23:26:19.463237+0000","last_clean":"2024-04-24T23:26:19.463237+0000","last_became_active":"2024-04-24T23:26:18.169050+0000","last_became_peered":"2024-04-24T23:26:18.169050+0000","last_unstale":"2024-04-24T23:26:19.463237+0000","last_undegraded":"2024-04-24T23:26:19.463237+0000","last_fullsized":"2024-04-24T23:26:19.463237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:38.988687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172127+0000","last_change":"2024-04-24T23:26:18.181916+0000","last_active":"2024-04-24T23:26:19.172127+0000","last_peered":"2024-04-24T23:26:19.172127+0000","last_clean":"2024-04-24T23:26:19.172127+0000","last_became_active":"2024-04-24T23:26:18.179385+0000","last_became_peered":"2024-04-24T23:26:18.179385+0000","last_unstale":"2024-04-24T23:26:19.172127+0000","last_undegraded":"2024-04-24T23:26:19.172127+0000","last_fullsized":"2024-04-24T23:26:19.172127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:08.940182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463949+0000","last_change":"2024-04-24T23:26:18.176128+0000","last_active":"2024-04-24T23:26:19.463949+0000","last_peered":"2024-04-24T23:26:19.463949+0000","last_clean":"2024-04-24T23:26:19.463949+0000","last_became_active":"2024-04-24T23:26:18.175801+0000","last_became_peered":"2024-04-24T23:26:18.175801+0000","last_unstale":"2024-04-24T23:26:19.463949+0000","last_undegraded":"2024-04-24T23:26:19.463949+0000","last_fullsized":"2024-04-24T23:26:19.463949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:17.771410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167252+0000","last_change":"2024-04-24T23:26:18.173268+0000","last_active":"2024-04-24T23:26:19.167252+0000","last_peered":"2024-04-24T23:26:19.167252+0000","last_clean":"2024-04-24T23:26:19.167252+0000","last_became_active":"2024-04-24T23:26:18.172981+0000","last_became_peered":"2024-04-24T23:26:18.172981+0000","last_unstale":"2024-04-24T23:26:19.167252+0000","last_undegraded":"2024-04-24T23:26:19.167252+0000","last_fullsized":"2024-04-24T23:26:19.167252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:35.874424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369835+0000","last_change":"2024-04-24T23:26:18.179248+0000","last_active":"2024-04-24T23:26:19.369835+0000","last_peered":"2024-04-24T23:26:19.369835+0000","last_clean":"2024-04-24T23:26:19.369835+0000","last_became_active":"2024-04-24T23:26:18.178895+0000","last_became_peered":"2024-04-24T23:26:18.178895+0000","last_unstale":"2024-04-24T23:26:19.369835+0000","last_undegraded":"2024-04-24T23:26:19.369835+0000","last_fullsized":"2024-04-24T23:26:19.369835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:24.189593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172026+0000","last_change":"2024-04-24T23:26:18.170742+0000","last_active":"2024-04-24T23:26:19.172026+0000","last_peered":"2024-04-24T23:26:19.172026+0000","last_clean":"2024-04-24T23:26:19.172026+0000","last_became_active":"2024-04-24T23:26:18.170475+0000","last_became_peered":"2024-04-24T23:26:18.170475+0000","last_unstale":"2024-04-24T23:26:19.172026+0000","last_undegraded":"2024-04-24T23:26:19.172026+0000","last_fullsized":"2024-04-24T23:26:19.172026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:26.567578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171882+0000","last_change":"2024-04-24T23:26:18.174295+0000","last_active":"2024-04-24T23:26:19.171882+0000","last_peered":"2024-04-24T23:26:19.171882+0000","last_clean":"2024-04-24T23:26:19.171882+0000","last_became_active":"2024-04-24T23:26:18.173905+0000","last_became_peered":"2024-04-24T23:26:18.173905+0000","last_unstale":"2024-04-24T23:26:19.171882+0000","last_undegraded":"2024-04-24T23:26:19.171882+0000","last_fullsized":"2024-04-24T23:26:19.171882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:37.002157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166346+0000","last_change":"2024-04-24T23:26:18.168808+0000","last_active":"2024-04-24T23:26:19.166346+0000","last_peered":"2024-04-24T23:26:19.166346+0000","last_clean":"2024-04-24T23:26:19.166346+0000","last_became_active":"2024-04-24T23:26:18.168532+0000","last_became_peered":"2024-04-24T23:26:18.168532+0000","last_unstale":"2024-04-24T23:26:19.166346+0000","last_undegraded":"2024-04-24T23:26:19.166346+0000","last_fullsized":"2024-04-24T23:26:19.166346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:53.405043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369573+0000","last_change":"2024-04-24T23:26:18.185209+0000","last_active":"2024-04-24T23:26:19.369573+0000","last_peered":"2024-04-24T23:26:19.369573+0000","last_clean":"2024-04-24T23:26:19.369573+0000","last_became_active":"2024-04-24T23:26:18.184905+0000","last_became_peered":"2024-04-24T23:26:18.184905+0000","last_unstale":"2024-04-24T23:26:19.369573+0000","last_undegraded":"2024-04-24T23:26:19.369573+0000","last_fullsized":"2024-04-24T23:26:19.369573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:45.758300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167020+0000","last_change":"2024-04-24T23:26:18.181402+0000","last_active":"2024-04-24T23:26:19.167020+0000","last_peered":"2024-04-24T23:26:19.167020+0000","last_clean":"2024-04-24T23:26:19.167020+0000","last_became_active":"2024-04-24T23:26:18.181116+0000","last_became_peered":"2024-04-24T23:26:18.181116+0000","last_unstale":"2024-04-24T23:26:19.167020+0000","last_undegraded":"2024-04-24T23:26:19.167020+0000","last_fullsized":"2024-04-24T23:26:19.167020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:04.106671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370302+0000","last_change":"2024-04-24T23:26:17.166716+0000","last_active":"2024-04-24T23:26:19.370302+0000","last_peered":"2024-04-24T23:26:19.370302+0000","last_clean":"2024-04-24T23:26:19.370302+0000","last_became_active":"2024-04-24T23:26:17.166397+0000","last_became_peered":"2024-04-24T23:26:17.166397+0000","last_unstale":"2024-04-24T23:26:19.370302+0000","last_undegraded":"2024-04-24T23:26:19.370302+0000","last_fullsized":"2024-04-24T23:26:19.370302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:59.737718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370213+0000","last_change":"2024-04-24T23:26:18.184041+0000","last_active":"2024-04-24T23:26:19.370213+0000","last_peered":"2024-04-24T23:26:19.370213+0000","last_clean":"2024-04-24T23:26:19.370213+0000","last_became_active":"2024-04-24T23:26:18.181518+0000","last_became_peered":"2024-04-24T23:26:18.181518+0000","last_unstale":"2024-04-24T23:26:19.370213+0000","last_undegraded":"2024-04-24T23:26:19.370213+0000","last_fullsized":"2024-04-24T23:26:19.370213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:04.106671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167264+0000","last_change":"2024-04-24T23:26:17.162927+0000","last_active":"2024-04-24T23:26:19.167264+0000","last_peered":"2024-04-24T23:26:19.167264+0000","last_clean":"2024-04-24T23:26:19.167264+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:19.167264+0000","last_undegraded":"2024-04-24T23:26:19.167264+0000","last_fullsized":"2024-04-24T23:26:19.167264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:18.000135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369912+0000","last_change":"2024-04-24T23:26:18.182994+0000","last_active":"2024-04-24T23:26:19.369912+0000","last_peered":"2024-04-24T23:26:19.369912+0000","last_clean":"2024-04-24T23:26:19.369912+0000","last_became_active":"2024-04-24T23:26:18.182675+0000","last_became_peered":"2024-04-24T23:26:18.182675+0000","last_unstale":"2024-04-24T23:26:19.369912+0000","last_undegraded":"2024-04-24T23:26:19.369912+0000","last_fullsized":"2024-04-24T23:26:19.369912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:53.405043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174279+0000","last_change":"2024-04-24T23:26:17.166726+0000","last_active":"2024-04-24T23:26:19.174279+0000","last_peered":"2024-04-24T23:26:19.174279+0000","last_clean":"2024-04-24T23:26:19.174279+0000","last_became_active":"2024-04-24T23:26:17.166370+0000","last_became_peered":"2024-04-24T23:26:17.166370+0000","last_unstale":"2024-04-24T23:26:19.174279+0000","last_undegraded":"2024-04-24T23:26:19.174279+0000","last_fullsized":"2024-04-24T23:26:19.174279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:23.185199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463949+0000","last_change":"2024-04-24T23:26:18.175108+0000","last_active":"2024-04-24T23:26:19.463949+0000","last_peered":"2024-04-24T23:26:19.463949+0000","last_clean":"2024-04-24T23:26:19.463949+0000","last_became_active":"2024-04-24T23:26:18.174822+0000","last_became_peered":"2024-04-24T23:26:18.174822+0000","last_unstale":"2024-04-24T23:26:19.463949+0000","last_undegraded":"2024-04-24T23:26:19.463949+0000","last_fullsized":"2024-04-24T23:26:19.463949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:55.214805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174102+0000","last_change":"2024-04-24T23:26:17.167115+0000","last_active":"2024-04-24T23:26:19.174102+0000","last_peered":"2024-04-24T23:26:19.174102+0000","last_clean":"2024-04-24T23:26:19.174102+0000","last_became_active":"2024-04-24T23:26:17.166838+0000","last_became_peered":"2024-04-24T23:26:17.166838+0000","last_unstale":"2024-04-24T23:26:19.174102+0000","last_undegraded":"2024-04-24T23:26:19.174102+0000","last_fullsized":"2024-04-24T23:26:19.174102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:52.400650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172127+0000","last_change":"2024-04-24T23:26:18.173810+0000","last_active":"2024-04-24T23:26:19.172127+0000","last_peered":"2024-04-24T23:26:19.172127+0000","last_clean":"2024-04-24T23:26:19.172127+0000","last_became_active":"2024-04-24T23:26:18.173146+0000","last_became_peered":"2024-04-24T23:26:18.173146+0000","last_unstale":"2024-04-24T23:26:19.172127+0000","last_undegraded":"2024-04-24T23:26:19.172127+0000","last_fullsized":"2024-04-24T23:26:19.172127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:43.460492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167102+0000","last_change":"2024-04-24T23:26:17.166049+0000","last_active":"2024-04-24T23:26:19.167102+0000","last_peered":"2024-04-24T23:26:19.167102+0000","last_clean":"2024-04-24T23:26:19.167102+0000","last_became_active":"2024-04-24T23:26:17.165763+0000","last_became_peered":"2024-04-24T23:26:17.165763+0000","last_unstale":"2024-04-24T23:26:19.167102+0000","last_undegraded":"2024-04-24T23:26:19.167102+0000","last_fullsized":"2024-04-24T23:26:19.167102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:01.255107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166897+0000","last_change":"2024-04-24T23:26:18.172391+0000","last_active":"2024-04-24T23:26:19.166897+0000","last_peered":"2024-04-24T23:26:19.166897+0000","last_clean":"2024-04-24T23:26:19.166897+0000","last_became_active":"2024-04-24T23:26:18.172096+0000","last_became_peered":"2024-04-24T23:26:18.172096+0000","last_unstale":"2024-04-24T23:26:19.166897+0000","last_undegraded":"2024-04-24T23:26:19.166897+0000","last_fullsized":"2024-04-24T23:26:19.166897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:59.026283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166463+0000","last_change":"2024-04-24T23:26:17.161576+0000","last_active":"2024-04-24T23:26:19.166463+0000","last_peered":"2024-04-24T23:26:19.166463+0000","last_clean":"2024-04-24T23:26:19.166463+0000","last_became_active":"2024-04-24T23:26:17.161260+0000","last_became_peered":"2024-04-24T23:26:17.161260+0000","last_unstale":"2024-04-24T23:26:19.166463+0000","last_undegraded":"2024-04-24T23:26:19.166463+0000","last_fullsized":"2024-04-24T23:26:19.166463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:42.456099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463602+0000","last_change":"2024-04-24T23:26:18.173192+0000","last_active":"2024-04-24T23:26:19.463602+0000","last_peered":"2024-04-24T23:26:19.463602+0000","last_clean":"2024-04-24T23:26:19.463602+0000","last_became_active":"2024-04-24T23:26:18.172915+0000","last_became_peered":"2024-04-24T23:26:18.172915+0000","last_unstale":"2024-04-24T23:26:19.463602+0000","last_undegraded":"2024-04-24T23:26:19.463602+0000","last_fullsized":"2024-04-24T23:26:19.463602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:08.137883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.173896+0000","last_change":"2024-04-24T23:26:17.167646+0000","last_active":"2024-04-24T23:26:19.173896+0000","last_peered":"2024-04-24T23:26:19.173896+0000","last_clean":"2024-04-24T23:26:19.173896+0000","last_became_active":"2024-04-24T23:26:17.167351+0000","last_became_peered":"2024-04-24T23:26:17.167351+0000","last_unstale":"2024-04-24T23:26:19.173896+0000","last_undegraded":"2024-04-24T23:26:19.173896+0000","last_fullsized":"2024-04-24T23:26:19.173896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:07.366676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463063+0000","last_change":"2024-04-24T23:26:18.164600+0000","last_active":"2024-04-24T23:26:19.463063+0000","last_peered":"2024-04-24T23:26:19.463063+0000","last_clean":"2024-04-24T23:26:19.463063+0000","last_became_active":"2024-04-24T23:26:18.164337+0000","last_became_peered":"2024-04-24T23:26:18.164337+0000","last_unstale":"2024-04-24T23:26:19.463063+0000","last_undegraded":"2024-04-24T23:26:19.463063+0000","last_fullsized":"2024-04-24T23:26:19.463063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:20.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171839+0000","last_change":"2024-04-24T23:26:17.162901+0000","last_active":"2024-04-24T23:26:19.171839+0000","last_peered":"2024-04-24T23:26:19.171839+0000","last_clean":"2024-04-24T23:26:19.171839+0000","last_became_active":"2024-04-24T23:26:17.162604+0000","last_became_peered":"2024-04-24T23:26:17.162604+0000","last_unstale":"2024-04-24T23:26:19.171839+0000","last_undegraded":"2024-04-24T23:26:19.171839+0000","last_fullsized":"2024-04-24T23:26:19.171839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:29.008878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463335+0000","last_change":"2024-04-24T23:26:18.172560+0000","last_active":"2024-04-24T23:26:19.463335+0000","last_peered":"2024-04-24T23:26:19.463335+0000","last_clean":"2024-04-24T23:26:19.463335+0000","last_became_active":"2024-04-24T23:26:18.172167+0000","last_became_peered":"2024-04-24T23:26:18.172167+0000","last_unstale":"2024-04-24T23:26:19.463335+0000","last_undegraded":"2024-04-24T23:26:19.463335+0000","last_fullsized":"2024-04-24T23:26:19.463335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:20.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370361+0000","last_change":"2024-04-24T23:26:17.167550+0000","last_active":"2024-04-24T23:26:19.370361+0000","last_peered":"2024-04-24T23:26:19.370361+0000","last_clean":"2024-04-24T23:26:19.370361+0000","last_became_active":"2024-04-24T23:26:17.167229+0000","last_became_peered":"2024-04-24T23:26:17.167229+0000","last_unstale":"2024-04-24T23:26:19.370361+0000","last_undegraded":"2024-04-24T23:26:19.370361+0000","last_fullsized":"2024-04-24T23:26:19.370361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:00.568912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167936+0000","last_change":"2024-04-24T23:26:18.176333+0000","last_active":"2024-04-24T23:26:19.167936+0000","last_peered":"2024-04-24T23:26:19.167936+0000","last_clean":"2024-04-24T23:26:19.167936+0000","last_became_active":"2024-04-24T23:26:18.176039+0000","last_became_peered":"2024-04-24T23:26:18.176039+0000","last_unstale":"2024-04-24T23:26:19.167936+0000","last_undegraded":"2024-04-24T23:26:19.167936+0000","last_fullsized":"2024-04-24T23:26:19.167936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:54.744768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370102+0000","last_change":"2024-04-24T23:26:17.168677+0000","last_active":"2024-04-24T23:26:19.370102+0000","last_peered":"2024-04-24T23:26:19.370102+0000","last_clean":"2024-04-24T23:26:19.370102+0000","last_became_active":"2024-04-24T23:26:17.168354+0000","last_became_peered":"2024-04-24T23:26:17.168354+0000","last_unstale":"2024-04-24T23:26:19.370102+0000","last_undegraded":"2024-04-24T23:26:19.370102+0000","last_fullsized":"2024-04-24T23:26:19.370102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:25.563184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174078+0000","last_change":"2024-04-24T23:26:18.180771+0000","last_active":"2024-04-24T23:26:19.174078+0000","last_peered":"2024-04-24T23:26:19.174078+0000","last_clean":"2024-04-24T23:26:19.174078+0000","last_became_active":"2024-04-24T23:26:18.180497+0000","last_became_peered":"2024-04-24T23:26:18.180497+0000","last_unstale":"2024-04-24T23:26:19.174078+0000","last_undegraded":"2024-04-24T23:26:19.174078+0000","last_fullsized":"2024-04-24T23:26:19.174078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:19.473276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369658+0000","last_change":"2024-04-24T23:26:17.170110+0000","last_active":"2024-04-24T23:26:19.369658+0000","last_peered":"2024-04-24T23:26:19.369658+0000","last_clean":"2024-04-24T23:26:19.369658+0000","last_became_active":"2024-04-24T23:26:17.169755+0000","last_became_peered":"2024-04-24T23:26:17.169755+0000","last_unstale":"2024-04-24T23:26:19.369658+0000","last_undegraded":"2024-04-24T23:26:19.369658+0000","last_fullsized":"2024-04-24T23:26:19.369658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:47.906621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167370+0000","last_change":"2024-04-24T23:26:18.172691+0000","last_active":"2024-04-24T23:26:19.167370+0000","last_peered":"2024-04-24T23:26:19.167370+0000","last_clean":"2024-04-24T23:26:19.167370+0000","last_became_active":"2024-04-24T23:26:18.172294+0000","last_became_peered":"2024-04-24T23:26:18.172294+0000","last_unstale":"2024-04-24T23:26:19.167370+0000","last_undegraded":"2024-04-24T23:26:19.167370+0000","last_fullsized":"2024-04-24T23:26:19.167370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:23.707858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167799+0000","last_change":"2024-04-24T23:26:17.151318+0000","last_active":"2024-04-24T23:26:19.167799+0000","last_peered":"2024-04-24T23:26:19.167799+0000","last_clean":"2024-04-24T23:26:19.167799+0000","last_became_active":"2024-04-24T23:26:17.151022+0000","last_became_peered":"2024-04-24T23:26:17.151022+0000","last_unstale":"2024-04-24T23:26:19.167799+0000","last_undegraded":"2024-04-24T23:26:19.167799+0000","last_fullsized":"2024-04-24T23:26:19.167799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:55.620251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.173969+0000","last_change":"2024-04-24T23:26:18.176225+0000","last_active":"2024-04-24T23:26:19.173969+0000","last_peered":"2024-04-24T23:26:19.173969+0000","last_clean":"2024-04-24T23:26:19.173969+0000","last_became_active":"2024-04-24T23:26:18.175893+0000","last_became_peered":"2024-04-24T23:26:18.175893+0000","last_unstale":"2024-04-24T23:26:19.173969+0000","last_undegraded":"2024-04-24T23:26:19.173969+0000","last_fullsized":"2024-04-24T23:26:19.173969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:23.707858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172012+0000","last_change":"2024-04-24T23:26:17.162943+0000","last_active":"2024-04-24T23:26:19.172012+0000","last_peered":"2024-04-24T23:26:19.172012+0000","last_clean":"2024-04-24T23:26:19.172012+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:19.172012+0000","last_undegraded":"2024-04-24T23:26:19.172012+0000","last_fullsized":"2024-04-24T23:26:19.172012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:21.483555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463963+0000","last_change":"2024-04-24T23:26:18.173884+0000","last_active":"2024-04-24T23:26:19.463963+0000","last_peered":"2024-04-24T23:26:19.463963+0000","last_clean":"2024-04-24T23:26:19.463963+0000","last_became_active":"2024-04-24T23:26:18.173532+0000","last_became_peered":"2024-04-24T23:26:18.173532+0000","last_unstale":"2024-04-24T23:26:19.463963+0000","last_undegraded":"2024-04-24T23:26:19.463963+0000","last_fullsized":"2024-04-24T23:26:19.463963+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:40.785566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463179+0000","last_change":"2024-04-24T23:26:17.163630+0000","last_active":"2024-04-24T23:26:19.463179+0000","last_peered":"2024-04-24T23:26:19.463179+0000","last_clean":"2024-04-24T23:26:19.463179+0000","last_became_active":"2024-04-24T23:26:17.163331+0000","last_became_peered":"2024-04-24T23:26:17.163331+0000","last_unstale":"2024-04-24T23:26:19.463179+0000","last_undegraded":"2024-04-24T23:26:19.463179+0000","last_fullsized":"2024-04-24T23:26:19.463179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:54.210411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171817+0000","last_change":"2024-04-24T23:26:18.174543+0000","last_active":"2024-04-24T23:26:19.171817+0000","last_peered":"2024-04-24T23:26:19.171817+0000","last_clean":"2024-04-24T23:26:19.171817+0000","last_became_active":"2024-04-24T23:26:18.173665+0000","last_became_peered":"2024-04-24T23:26:18.173665+0000","last_unstale":"2024-04-24T23:26:19.171817+0000","last_undegraded":"2024-04-24T23:26:19.171817+0000","last_fullsized":"2024-04-24T23:26:19.171817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:00.742112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167436+0000","last_change":"2024-04-24T23:26:17.160593+0000","last_active":"2024-04-24T23:26:19.167436+0000","last_peered":"2024-04-24T23:26:19.167436+0000","last_clean":"2024-04-24T23:26:19.167436+0000","last_became_active":"2024-04-24T23:26:17.160303+0000","last_became_peered":"2024-04-24T23:26:17.160303+0000","last_unstale":"2024-04-24T23:26:19.167436+0000","last_undegraded":"2024-04-24T23:26:19.167436+0000","last_fullsized":"2024-04-24T23:26:19.167436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:28.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171783+0000","last_change":"2024-04-24T23:26:18.179756+0000","last_active":"2024-04-24T23:26:19.171783+0000","last_peered":"2024-04-24T23:26:19.171783+0000","last_clean":"2024-04-24T23:26:19.171783+0000","last_became_active":"2024-04-24T23:26:18.179454+0000","last_became_peered":"2024-04-24T23:26:18.179454+0000","last_unstale":"2024-04-24T23:26:19.171783+0000","last_undegraded":"2024-04-24T23:26:19.171783+0000","last_fullsized":"2024-04-24T23:26:19.171783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:06.225251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.173835+0000","last_change":"2024-04-24T23:26:17.165945+0000","last_active":"2024-04-24T23:26:19.173835+0000","last_peered":"2024-04-24T23:26:19.173835+0000","last_clean":"2024-04-24T23:26:19.173835+0000","last_became_active":"2024-04-24T23:26:17.164750+0000","last_became_peered":"2024-04-24T23:26:17.164750+0000","last_unstale":"2024-04-24T23:26:19.173835+0000","last_undegraded":"2024-04-24T23:26:19.173835+0000","last_fullsized":"2024-04-24T23:26:19.173835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:51.395536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.464027+0000","last_change":"2024-04-24T23:26:18.168783+0000","last_active":"2024-04-24T23:26:19.464027+0000","last_peered":"2024-04-24T23:26:19.464027+0000","last_clean":"2024-04-24T23:26:19.464027+0000","last_became_active":"2024-04-24T23:26:18.168449+0000","last_became_peered":"2024-04-24T23:26:18.168449+0000","last_unstale":"2024-04-24T23:26:19.464027+0000","last_undegraded":"2024-04-24T23:26:19.464027+0000","last_fullsized":"2024-04-24T23:26:19.464027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:50.240258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370433+0000","last_change":"2024-04-24T23:26:17.165977+0000","last_active":"2024-04-24T23:26:19.370433+0000","last_peered":"2024-04-24T23:26:19.370433+0000","last_clean":"2024-04-24T23:26:19.370433+0000","last_became_active":"2024-04-24T23:26:17.165641+0000","last_became_peered":"2024-04-24T23:26:17.165641+0000","last_unstale":"2024-04-24T23:26:19.370433+0000","last_undegraded":"2024-04-24T23:26:19.370433+0000","last_fullsized":"2024-04-24T23:26:19.370433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.997763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370287+0000","last_change":"2024-04-24T23:26:18.180504+0000","last_active":"2024-04-24T23:26:19.370287+0000","last_peered":"2024-04-24T23:26:19.370287+0000","last_clean":"2024-04-24T23:26:19.370287+0000","last_became_active":"2024-04-24T23:26:18.179646+0000","last_became_peered":"2024-04-24T23:26:18.179646+0000","last_unstale":"2024-04-24T23:26:19.370287+0000","last_undegraded":"2024-04-24T23:26:19.370287+0000","last_fullsized":"2024-04-24T23:26:19.370287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:54.744768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174158+0000","last_change":"2024-04-24T23:26:17.163913+0000","last_active":"2024-04-24T23:26:19.174158+0000","last_peered":"2024-04-24T23:26:19.174158+0000","last_clean":"2024-04-24T23:26:19.174158+0000","last_became_active":"2024-04-24T23:26:17.163640+0000","last_became_peered":"2024-04-24T23:26:17.163640+0000","last_unstale":"2024-04-24T23:26:19.174158+0000","last_undegraded":"2024-04-24T23:26:19.174158+0000","last_fullsized":"2024-04-24T23:26:19.174158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:44.753907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167446+0000","last_change":"2024-04-24T23:26:18.175458+0000","last_active":"2024-04-24T23:26:19.167446+0000","last_peered":"2024-04-24T23:26:19.167446+0000","last_clean":"2024-04-24T23:26:19.167446+0000","last_became_active":"2024-04-24T23:26:18.175117+0000","last_became_peered":"2024-04-24T23:26:18.175117+0000","last_unstale":"2024-04-24T23:26:19.167446+0000","last_undegraded":"2024-04-24T23:26:19.167446+0000","last_fullsized":"2024-04-24T23:26:19.167446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:29.129749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174166+0000","last_change":"2024-04-24T23:26:17.167926+0000","last_active":"2024-04-24T23:26:19.174166+0000","last_peered":"2024-04-24T23:26:19.174166+0000","last_clean":"2024-04-24T23:26:19.174166+0000","last_became_active":"2024-04-24T23:26:17.167660+0000","last_became_peered":"2024-04-24T23:26:17.167660+0000","last_unstale":"2024-04-24T23:26:19.174166+0000","last_undegraded":"2024-04-24T23:26:19.174166+0000","last_fullsized":"2024-04-24T23:26:19.174166+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.467361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463642+0000","last_change":"2024-04-24T23:26:18.172592+0000","last_active":"2024-04-24T23:26:19.463642+0000","last_peered":"2024-04-24T23:26:19.463642+0000","last_clean":"2024-04-24T23:26:19.463642+0000","last_became_active":"2024-04-24T23:26:18.172290+0000","last_became_peered":"2024-04-24T23:26:18.172290+0000","last_unstale":"2024-04-24T23:26:19.463642+0000","last_undegraded":"2024-04-24T23:26:19.463642+0000","last_fullsized":"2024-04-24T23:26:19.463642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:30.013272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463374+0000","last_change":"2024-04-24T23:26:17.158322+0000","last_active":"2024-04-24T23:26:19.463374+0000","last_peered":"2024-04-24T23:26:19.463374+0000","last_clean":"2024-04-24T23:26:19.463374+0000","last_became_active":"2024-04-24T23:26:17.158052+0000","last_became_peered":"2024-04-24T23:26:17.158052+0000","last_unstale":"2024-04-24T23:26:19.463374+0000","last_undegraded":"2024-04-24T23:26:19.463374+0000","last_fullsized":"2024-04-24T23:26:19.463374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:31.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167801+0000","last_change":"2024-04-24T23:26:18.165683+0000","last_active":"2024-04-24T23:26:19.167801+0000","last_peered":"2024-04-24T23:26:19.167801+0000","last_clean":"2024-04-24T23:26:19.167801+0000","last_became_active":"2024-04-24T23:26:18.165354+0000","last_became_peered":"2024-04-24T23:26:18.165354+0000","last_unstale":"2024-04-24T23:26:19.167801+0000","last_undegraded":"2024-04-24T23:26:19.167801+0000","last_fullsized":"2024-04-24T23:26:19.167801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:59.026283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463731+0000","last_change":"2024-04-24T23:26:17.167861+0000","last_active":"2024-04-24T23:26:19.463731+0000","last_peered":"2024-04-24T23:26:19.463731+0000","last_clean":"2024-04-24T23:26:19.463731+0000","last_became_active":"2024-04-24T23:26:17.167598+0000","last_became_peered":"2024-04-24T23:26:17.167598+0000","last_unstale":"2024-04-24T23:26:19.463731+0000","last_undegraded":"2024-04-24T23:26:19.463731+0000","last_fullsized":"2024-04-24T23:26:19.463731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:46.915793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463466+0000","last_change":"2024-04-24T23:26:18.163644+0000","last_active":"2024-04-24T23:26:19.463466+0000","last_peered":"2024-04-24T23:26:19.463466+0000","last_clean":"2024-04-24T23:26:19.463466+0000","last_became_active":"2024-04-24T23:26:18.163326+0000","last_became_peered":"2024-04-24T23:26:18.163326+0000","last_unstale":"2024-04-24T23:26:19.463466+0000","last_undegraded":"2024-04-24T23:26:19.463466+0000","last_fullsized":"2024-04-24T23:26:19.463466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:43.377906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463456+0000","last_change":"2024-04-24T23:26:17.165956+0000","last_active":"2024-04-24T23:26:19.463456+0000","last_peered":"2024-04-24T23:26:19.463456+0000","last_clean":"2024-04-24T23:26:19.463456+0000","last_became_active":"2024-04-24T23:26:17.165633+0000","last_became_peered":"2024-04-24T23:26:17.165633+0000","last_unstale":"2024-04-24T23:26:19.463456+0000","last_undegraded":"2024-04-24T23:26:19.463456+0000","last_fullsized":"2024-04-24T23:26:19.463456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:31.605822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166935+0000","last_change":"2024-04-24T23:26:18.173214+0000","last_active":"2024-04-24T23:26:19.166935+0000","last_peered":"2024-04-24T23:26:19.166935+0000","last_clean":"2024-04-24T23:26:19.166935+0000","last_became_active":"2024-04-24T23:26:18.172837+0000","last_became_peered":"2024-04-24T23:26:18.172837+0000","last_unstale":"2024-04-24T23:26:19.166935+0000","last_undegraded":"2024-04-24T23:26:19.166935+0000","last_fullsized":"2024-04-24T23:26:19.166935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:24.189593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.462917+0000","last_change":"2024-04-24T23:26:17.157597+0000","last_active":"2024-04-24T23:26:19.462917+0000","last_peered":"2024-04-24T23:26:19.462917+0000","last_clean":"2024-04-24T23:26:19.462917+0000","last_became_active":"2024-04-24T23:26:17.157311+0000","last_became_peered":"2024-04-24T23:26:17.157311+0000","last_unstale":"2024-04-24T23:26:19.462917+0000","last_undegraded":"2024-04-24T23:26:19.462917+0000","last_fullsized":"2024-04-24T23:26:19.462917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.781173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167189+0000","last_change":"2024-04-24T23:26:18.164360+0000","last_active":"2024-04-24T23:26:19.167189+0000","last_peered":"2024-04-24T23:26:19.167189+0000","last_clean":"2024-04-24T23:26:19.167189+0000","last_became_active":"2024-04-24T23:26:18.164082+0000","last_became_peered":"2024-04-24T23:26:18.164082+0000","last_unstale":"2024-04-24T23:26:19.167189+0000","last_undegraded":"2024-04-24T23:26:19.167189+0000","last_fullsized":"2024-04-24T23:26:19.167189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:40.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172194+0000","last_change":"2024-04-24T23:26:17.163231+0000","last_active":"2024-04-24T23:26:19.172194+0000","last_peered":"2024-04-24T23:26:19.172194+0000","last_clean":"2024-04-24T23:26:19.172194+0000","last_became_active":"2024-04-24T23:26:17.162790+0000","last_became_peered":"2024-04-24T23:26:17.162790+0000","last_unstale":"2024-04-24T23:26:19.172194+0000","last_undegraded":"2024-04-24T23:26:19.172194+0000","last_fullsized":"2024-04-24T23:26:19.172194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:44.722428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.464151+0000","last_change":"2024-04-24T23:26:18.184007+0000","last_active":"2024-04-24T23:26:19.464151+0000","last_peered":"2024-04-24T23:26:19.464151+0000","last_clean":"2024-04-24T23:26:19.464151+0000","last_became_active":"2024-04-24T23:26:18.183723+0000","last_became_peered":"2024-04-24T23:26:18.183723+0000","last_unstale":"2024-04-24T23:26:19.464151+0000","last_undegraded":"2024-04-24T23:26:19.464151+0000","last_fullsized":"2024-04-24T23:26:19.464151+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:57.076073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172239+0000","last_change":"2024-04-24T23:26:17.161814+0000","last_active":"2024-04-24T23:26:19.172239+0000","last_peered":"2024-04-24T23:26:19.172239+0000","last_clean":"2024-04-24T23:26:19.172239+0000","last_became_active":"2024-04-24T23:26:17.161485+0000","last_became_peered":"2024-04-24T23:26:17.161485+0000","last_unstale":"2024-04-24T23:26:19.172239+0000","last_undegraded":"2024-04-24T23:26:19.172239+0000","last_fullsized":"2024-04-24T23:26:19.172239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:45.309167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370143+0000","last_change":"2024-04-24T23:26:18.188305+0000","last_active":"2024-04-24T23:26:19.370143+0000","last_peered":"2024-04-24T23:26:19.370143+0000","last_clean":"2024-04-24T23:26:19.370143+0000","last_became_active":"2024-04-24T23:26:18.187873+0000","last_became_peered":"2024-04-24T23:26:18.187873+0000","last_unstale":"2024-04-24T23:26:19.370143+0000","last_undegraded":"2024-04-24T23:26:19.370143+0000","last_fullsized":"2024-04-24T23:26:19.370143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:07.648864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369824+0000","last_change":"2024-04-24T23:26:17.169621+0000","last_active":"2024-04-24T23:26:19.369824+0000","last_peered":"2024-04-24T23:26:19.369824+0000","last_clean":"2024-04-24T23:26:19.369824+0000","last_became_active":"2024-04-24T23:26:17.169195+0000","last_became_peered":"2024-04-24T23:26:17.169195+0000","last_unstale":"2024-04-24T23:26:19.369824+0000","last_undegraded":"2024-04-24T23:26:19.369824+0000","last_fullsized":"2024-04-24T23:26:19.369824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:45.101937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167560+0000","last_change":"2024-04-24T23:26:18.170255+0000","last_active":"2024-04-24T23:26:19.167560+0000","last_peered":"2024-04-24T23:26:19.167560+0000","last_clean":"2024-04-24T23:26:19.167560+0000","last_became_active":"2024-04-24T23:26:18.169965+0000","last_became_peered":"2024-04-24T23:26:18.169965+0000","last_unstale":"2024-04-24T23:26:19.167560+0000","last_undegraded":"2024-04-24T23:26:19.167560+0000","last_fullsized":"2024-04-24T23:26:19.167560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:19.473276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370056+0000","last_change":"2024-04-24T23:26:17.169611+0000","last_active":"2024-04-24T23:26:19.370056+0000","last_peered":"2024-04-24T23:26:19.370056+0000","last_clean":"2024-04-24T23:26:19.370056+0000","last_became_active":"2024-04-24T23:26:17.169304+0000","last_became_peered":"2024-04-24T23:26:17.169304+0000","last_unstale":"2024-04-24T23:26:19.370056+0000","last_undegraded":"2024-04-24T23:26:19.370056+0000","last_fullsized":"2024-04-24T23:26:19.370056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:05.220857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167666+0000","last_change":"2024-04-24T23:26:18.165478+0000","last_active":"2024-04-24T23:26:19.167666+0000","last_peered":"2024-04-24T23:26:19.167666+0000","last_clean":"2024-04-24T23:26:19.167666+0000","last_became_active":"2024-04-24T23:26:18.165191+0000","last_became_peered":"2024-04-24T23:26:18.165191+0000","last_unstale":"2024-04-24T23:26:19.167666+0000","last_undegraded":"2024-04-24T23:26:19.167666+0000","last_fullsized":"2024-04-24T23:26:19.167666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:07.648864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174110+0000","last_change":"2024-04-24T23:26:17.168469+0000","last_active":"2024-04-24T23:26:19.174110+0000","last_peered":"2024-04-24T23:26:19.174110+0000","last_clean":"2024-04-24T23:26:19.174110+0000","last_became_active":"2024-04-24T23:26:17.167305+0000","last_became_peered":"2024-04-24T23:26:17.167305+0000","last_unstale":"2024-04-24T23:26:19.174110+0000","last_undegraded":"2024-04-24T23:26:19.174110+0000","last_fullsized":"2024-04-24T23:26:19.174110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:37.599403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174009+0000","last_change":"2024-04-24T23:26:18.176710+0000","last_active":"2024-04-24T23:26:19.174009+0000","last_peered":"2024-04-24T23:26:19.174009+0000","last_clean":"2024-04-24T23:26:19.174009+0000","last_became_active":"2024-04-24T23:26:18.176458+0000","last_became_peered":"2024-04-24T23:26:18.176458+0000","last_unstale":"2024-04-24T23:26:19.174009+0000","last_undegraded":"2024-04-24T23:26:19.174009+0000","last_fullsized":"2024-04-24T23:26:19.174009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:29.129749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166598+0000","last_change":"2024-04-24T23:26:17.162968+0000","last_active":"2024-04-24T23:26:19.166598+0000","last_peered":"2024-04-24T23:26:19.166598+0000","last_clean":"2024-04-24T23:26:19.166598+0000","last_became_active":"2024-04-24T23:26:17.162691+0000","last_became_peered":"2024-04-24T23:26:17.162691+0000","last_unstale":"2024-04-24T23:26:19.166598+0000","last_undegraded":"2024-04-24T23:26:19.166598+0000","last_fullsized":"2024-04-24T23:26:19.166598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:25.593578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463672+0000","last_change":"2024-04-24T23:26:18.176811+0000","last_active":"2024-04-24T23:26:19.463672+0000","last_peered":"2024-04-24T23:26:19.463672+0000","last_clean":"2024-04-24T23:26:19.463672+0000","last_became_active":"2024-04-24T23:26:18.176290+0000","last_became_peered":"2024-04-24T23:26:18.176290+0000","last_unstale":"2024-04-24T23:26:19.463672+0000","last_undegraded":"2024-04-24T23:26:19.463672+0000","last_fullsized":"2024-04-24T23:26:19.463672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:48.988665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369839+0000","last_change":"2024-04-24T23:26:17.168302+0000","last_active":"2024-04-24T23:26:19.369839+0000","last_peered":"2024-04-24T23:26:19.369839+0000","last_clean":"2024-04-24T23:26:19.369839+0000","last_became_active":"2024-04-24T23:26:17.167711+0000","last_became_peered":"2024-04-24T23:26:17.167711+0000","last_unstale":"2024-04-24T23:26:19.369839+0000","last_undegraded":"2024-04-24T23:26:19.369839+0000","last_fullsized":"2024-04-24T23:26:19.369839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:57.160342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369737+0000","last_change":"2024-04-24T23:26:18.187349+0000","last_active":"2024-04-24T23:26:19.369737+0000","last_peered":"2024-04-24T23:26:19.369737+0000","last_clean":"2024-04-24T23:26:19.369737+0000","last_became_active":"2024-04-24T23:26:18.187041+0000","last_became_peered":"2024-04-24T23:26:18.187041+0000","last_unstale":"2024-04-24T23:26:19.369737+0000","last_undegraded":"2024-04-24T23:26:19.369737+0000","last_fullsized":"2024-04-24T23:26:19.369737+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:52.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166354+0000","last_change":"2024-04-24T23:26:17.156009+0000","last_active":"2024-04-24T23:26:19.166354+0000","last_peered":"2024-04-24T23:26:19.166354+0000","last_clean":"2024-04-24T23:26:19.166354+0000","last_became_active":"2024-04-24T23:26:17.155739+0000","last_became_peered":"2024-04-24T23:26:17.155739+0000","last_unstale":"2024-04-24T23:26:19.166354+0000","last_undegraded":"2024-04-24T23:26:19.166354+0000","last_fullsized":"2024-04-24T23:26:19.166354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:07.935788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172160+0000","last_change":"2024-04-24T23:26:18.183679+0000","last_active":"2024-04-24T23:26:19.172160+0000","last_peered":"2024-04-24T23:26:19.172160+0000","last_clean":"2024-04-24T23:26:19.172160+0000","last_became_active":"2024-04-24T23:26:18.183138+0000","last_became_peered":"2024-04-24T23:26:18.183138+0000","last_unstale":"2024-04-24T23:26:19.172160+0000","last_undegraded":"2024-04-24T23:26:19.172160+0000","last_fullsized":"2024-04-24T23:26:19.172160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:02.259501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463966+0000","last_change":"2024-04-24T23:26:17.166166+0000","last_active":"2024-04-24T23:26:19.463966+0000","last_peered":"2024-04-24T23:26:19.463966+0000","last_clean":"2024-04-24T23:26:19.463966+0000","last_became_active":"2024-04-24T23:26:17.165855+0000","last_became_peered":"2024-04-24T23:26:17.165855+0000","last_unstale":"2024-04-24T23:26:19.463966+0000","last_undegraded":"2024-04-24T23:26:19.463966+0000","last_fullsized":"2024-04-24T23:26:19.463966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.467361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":282,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463814+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:26:19.463814+0000","last_peered":"2024-04-24T23:26:19.463814+0000","last_clean":"2024-04-24T23:26:19.463814+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:26:19.463814+0000","last_undegraded":"2024-04-24T23:26:19.463814+0000","last_fullsized":"2024-04-24T23:26:19.463814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1605632,"data_stored":1023969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":434225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":435165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":433531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":435143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1011712,"data_stored":434113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":433725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1605632,"data_stored":1023933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:26:26.611 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:26:26.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.723+0000 7f3be962c640 1 Processor -- start 2024-04-24T23:26:26.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.724+0000 7f3be962c640 1 -- start start 2024-04-24T23:26:26.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.724+0000 7f3be962c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3be40d4870 0x7f3be40b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:26.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.724+0000 7f3be962c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be40d32f0 con 0x7f3be415e570 2024-04-24T23:26:26.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.724+0000 7f3be962c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be40d0270 con 0x7f3be40b3130 2024-04-24T23:26:26.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.724+0000 7f3be962c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3be40d1ab0 con 0x7f3be40d4870 2024-04-24T23:26:26.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.724+0000 7f3be2ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3be415e570 0x7f3be40d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58342/0 (socket says 172.21.15.26:58342) 2024-04-24T23:26:26.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.724+0000 7f3be2ffd640 1 -- 172.21.15.26:0/1894978367 learned_addr learned my addr 172.21.15.26:0/1894978367 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:26.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.724+0000 7f3be27fc640 1 --2- 172.21.15.26:0/1894978367 >> v2:172.21.15.26:3300/0 conn(0x7f3be40d4870 0x7f3be40b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:26.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.725+0000 7f3be27fc640 1 -- 172.21.15.26:0/1894978367 >> v1:172.21.15.29:6789/0 conn(0x7f3be40b3130 legacy=0x7f3be40b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:26.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.725+0000 7f3be27fc640 1 -- 172.21.15.26:0/1894978367 >> v1:172.21.15.26:6789/0 conn(0x7f3be415e570 legacy=0x7f3be40d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:26.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.725+0000 7f3be27fc640 1 -- 172.21.15.26:0/1894978367 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be40d2d30 con 0x7f3be40d4870 2024-04-24T23:26:26.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be27fc640 1 --2- 172.21.15.26:0/1894978367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40d4870 0x7f3be40b4100 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80097a0 tx=0x7f3bd802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d259a70eae84047 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:26.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be1ffb640 1 -- 172.21.15.26:0/1894978367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8037070 con 0x7f3be40d4870 2024-04-24T23:26:26.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be1ffb640 1 -- 172.21.15.26:0/1894978367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd8032440 con 0x7f3be40d4870 2024-04-24T23:26:26.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be1ffb640 1 -- 172.21.15.26:0/1894978367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8032810 con 0x7f3be40d4870 2024-04-24T23:26:26.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be962c640 1 -- 172.21.15.26:0/1894978367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40d4870 msgr2=0x7f3be40b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:26.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be962c640 1 --2- 172.21.15.26:0/1894978367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40d4870 0x7f3be40b4100 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80097a0 tx=0x7f3bd802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:26:26.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be962c640 1 -- 172.21.15.26:0/1894978367 shutdown_connections 2024-04-24T23:26:26.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be962c640 1 --2- 172.21.15.26:0/1894978367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40d4870 0x7f3be40b4100 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:26.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be962c640 1 -- 172.21.15.26:0/1894978367 >> 172.21.15.26:0/1894978367 conn(0x7f3be408dbe0 msgr2=0x7f3be409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:26.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be962c640 1 -- 172.21.15.26:0/1894978367 shutdown_connections 2024-04-24T23:26:26.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.726+0000 7f3be962c640 1 -- 172.21.15.26:0/1894978367 wait complete. 2024-04-24T23:26:26.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.728+0000 7f3be962c640 1 Processor -- start 2024-04-24T23:26:26.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.728+0000 7f3be962c640 1 -- start start 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.728+0000 7f3be962c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40b3130 0x7f3be4152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.728+0000 7f3be962c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3be40d4870 0x7f3be40cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.728+0000 7f3be962c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be415e570 0x7f3be40b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.728+0000 7f3be962c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3be40cfa70 con 0x7f3be40b3130 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.728+0000 7f3be962c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3be40d2ae0 con 0x7f3be415e570 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.728+0000 7f3be962c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3be40cfd50 con 0x7f3be40d4870 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40b3130 0x7f3be4152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40b3130 0x7f3be4152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60424/0 (socket says 172.21.15.26:60424) 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be2ffd640 1 -- 172.21.15.26:0/2085589768 learned_addr learned my addr 172.21.15.26:0/2085589768 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be37fe640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be415e570 0x7f3be40b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:26.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be27fc640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3be40d4870 0x7f3be40cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be2ffd640 1 -- 172.21.15.26:0/2085589768 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3be40d4870 msgr2=0x7f3be40cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be2ffd640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3be40d4870 0x7f3be40cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be2ffd640 1 -- 172.21.15.26:0/2085589768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be415e570 msgr2=0x7f3be40b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be2ffd640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be415e570 0x7f3be40b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be2ffd640 1 -- 172.21.15.26:0/2085589768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be41f7560 con 0x7f3be40b3130 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be37fe640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be415e570 0x7f3be40b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be2ffd640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40b3130 0x7f3be4152750 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c4b0 tx=0x7f3bdc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3bcb7fe640 1 -- 172.21.15.26:0/2085589768 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc017070 con 0x7f3be40b3130 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3bcb7fe640 1 -- 172.21.15.26:0/2085589768 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bdc005ce0 con 0x7f3be40b3130 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be41f7820 con 0x7f3be40b3130 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3bcb7fe640 1 -- 172.21.15.26:0/2085589768 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc007270 con 0x7f3be40b3130 2024-04-24T23:26:26.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.729+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be41f7de0 con 0x7f3be40b3130 2024-04-24T23:26:26.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.730+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be40694c0 con 0x7f3be40b3130 2024-04-24T23:26:26.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.731+0000 7f3bcb7fe640 1 -- 172.21.15.26:0/2085589768 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3bdc01a070 con 0x7f3be40b3130 2024-04-24T23:26:26.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.731+0000 7f3bcb7fe640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3bd006be30 0x7f3bd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:26.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.732+0000 7f3bcb7fe640 1 -- 172.21.15.26:0/2085589768 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f3bdc08a040 con 0x7f3be40b3130 2024-04-24T23:26:26.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.732+0000 7f3be27fc640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3bd006be30 0x7f3bd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:26.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.732+0000 7f3be27fc640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3bd006be30 0x7f3bd006e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8034000 tx=0x7f3bd802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:26.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.736+0000 7f3bcb7fe640 1 -- 172.21.15.26:0/2085589768 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3be40d32f0 con 0x7f3be40b3130 2024-04-24T23:26:26.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.917+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3be40b55d0 con 0x7f3bd006be30 2024-04-24T23:26:26.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.923+0000 7f3bcb7fe640 1 -- 172.21.15.26:0/2085589768 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252081 (secure 0 0 0) 0x7f3be40b55d0 con 0x7f3bd006be30 2024-04-24T23:26:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:26.927 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:26:26.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3bd006be30 msgr2=0x7f3bd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:26.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3bd006be30 0x7f3bd006e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8034000 tx=0x7f3bd802e000 comp rx=0 tx=0).stop 2024-04-24T23:26:26.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40b3130 msgr2=0x7f3be4152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:26.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40b3130 0x7f3be4152750 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c4b0 tx=0x7f3bdc00c970 comp rx=0 tx=0).stop 2024-04-24T23:26:26.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 shutdown_connections 2024-04-24T23:26:26.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3bd006be30 0x7f3bd006e2f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:26.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3be415e570 0x7f3be40b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:26.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3be40d4870 0x7f3be40cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:26.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 --2- 172.21.15.26:0/2085589768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be40b3130 0x7f3be4152750 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:26.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 >> 172.21.15.26:0/2085589768 conn(0x7f3be408dbe0 msgr2=0x7f3be40d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:26.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 shutdown_connections 2024-04-24T23:26:26.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:26.926+0000 7f3be962c640 1 -- 172.21.15.26:0/2085589768 wait complete. 2024-04-24T23:26:26.938 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-04-24T23:26:25.387090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271036,"kb_used_data":9096,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754703684,"statfs":{"total":773094113280,"available":772816572416,"internally_reserved":0,"allocated":9314304,"data_stored":4653804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001852"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370066+0000","last_change":"2024-04-24T23:26:18.182579+0000","last_active":"2024-04-24T23:26:19.370066+0000","last_peered":"2024-04-24T23:26:19.370066+0000","last_clean":"2024-04-24T23:26:19.370066+0000","last_became_active":"2024-04-24T23:26:18.182259+0000","last_became_peered":"2024-04-24T23:26:18.182259+0000","last_unstale":"2024-04-24T23:26:19.370066+0000","last_undegraded":"2024-04-24T23:26:19.370066+0000","last_fullsized":"2024-04-24T23:26:19.370066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:37.471755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172178+0000","last_change":"2024-04-24T23:26:18.182127+0000","last_active":"2024-04-24T23:26:19.172178+0000","last_peered":"2024-04-24T23:26:19.172178+0000","last_clean":"2024-04-24T23:26:19.172178+0000","last_became_active":"2024-04-24T23:26:18.181815+0000","last_became_peered":"2024-04-24T23:26:18.181815+0000","last_unstale":"2024-04-24T23:26:19.172178+0000","last_undegraded":"2024-04-24T23:26:19.172178+0000","last_fullsized":"2024-04-24T23:26:19.172178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:26.597972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463793+0000","last_change":"2024-04-24T23:26:18.180748+0000","last_active":"2024-04-24T23:26:19.463793+0000","last_peered":"2024-04-24T23:26:19.463793+0000","last_clean":"2024-04-24T23:26:19.463793+0000","last_became_active":"2024-04-24T23:26:18.180423+0000","last_became_peered":"2024-04-24T23:26:18.180423+0000","last_unstale":"2024-04-24T23:26:19.463793+0000","last_undegraded":"2024-04-24T23:26:19.463793+0000","last_fullsized":"2024-04-24T23:26:19.463793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:28.897593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463767+0000","last_change":"2024-04-24T23:26:18.179588+0000","last_active":"2024-04-24T23:26:19.463767+0000","last_peered":"2024-04-24T23:26:19.463767+0000","last_clean":"2024-04-24T23:26:19.463767+0000","last_became_active":"2024-04-24T23:26:18.179338+0000","last_became_peered":"2024-04-24T23:26:18.179338+0000","last_unstale":"2024-04-24T23:26:19.463767+0000","last_undegraded":"2024-04-24T23:26:19.463767+0000","last_fullsized":"2024-04-24T23:26:19.463767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:46.313561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166713+0000","last_change":"2024-04-24T23:26:18.174615+0000","last_active":"2024-04-24T23:26:19.166713+0000","last_peered":"2024-04-24T23:26:19.166713+0000","last_clean":"2024-04-24T23:26:19.166713+0000","last_became_active":"2024-04-24T23:26:18.174345+0000","last_became_peered":"2024-04-24T23:26:18.174345+0000","last_unstale":"2024-04-24T23:26:19.166713+0000","last_undegraded":"2024-04-24T23:26:19.166713+0000","last_fullsized":"2024-04-24T23:26:19.166713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:07.648864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167619+0000","last_change":"2024-04-24T23:26:18.173531+0000","last_active":"2024-04-24T23:26:19.167619+0000","last_peered":"2024-04-24T23:26:19.167619+0000","last_clean":"2024-04-24T23:26:19.167619+0000","last_became_active":"2024-04-24T23:26:18.172003+0000","last_became_peered":"2024-04-24T23:26:18.172003+0000","last_unstale":"2024-04-24T23:26:19.167619+0000","last_undegraded":"2024-04-24T23:26:19.167619+0000","last_fullsized":"2024-04-24T23:26:19.167619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:37.471755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463409+0000","last_change":"2024-04-24T23:26:18.167012+0000","last_active":"2024-04-24T23:26:19.463409+0000","last_peered":"2024-04-24T23:26:19.463409+0000","last_clean":"2024-04-24T23:26:19.463409+0000","last_became_active":"2024-04-24T23:26:18.165907+0000","last_became_peered":"2024-04-24T23:26:18.165907+0000","last_unstale":"2024-04-24T23:26:19.463409+0000","last_undegraded":"2024-04-24T23:26:19.463409+0000","last_fullsized":"2024-04-24T23:26:19.463409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:00.932208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167076+0000","last_change":"2024-04-24T23:26:18.175501+0000","last_active":"2024-04-24T23:26:19.167076+0000","last_peered":"2024-04-24T23:26:19.167076+0000","last_clean":"2024-04-24T23:26:19.167076+0000","last_became_active":"2024-04-24T23:26:18.175151+0000","last_became_peered":"2024-04-24T23:26:18.175151+0000","last_unstale":"2024-04-24T23:26:19.167076+0000","last_undegraded":"2024-04-24T23:26:19.167076+0000","last_fullsized":"2024-04-24T23:26:19.167076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:53.405043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167839+0000","last_change":"2024-04-24T23:26:18.173971+0000","last_active":"2024-04-24T23:26:19.167839+0000","last_peered":"2024-04-24T23:26:19.167839+0000","last_clean":"2024-04-24T23:26:19.167839+0000","last_became_active":"2024-04-24T23:26:18.173649+0000","last_became_peered":"2024-04-24T23:26:18.173649+0000","last_unstale":"2024-04-24T23:26:19.167839+0000","last_undegraded":"2024-04-24T23:26:19.167839+0000","last_fullsized":"2024-04-24T23:26:19.167839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:04.106671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370199+0000","last_change":"2024-04-24T23:26:18.180126+0000","last_active":"2024-04-24T23:26:19.370199+0000","last_peered":"2024-04-24T23:26:19.370199+0000","last_clean":"2024-04-24T23:26:19.370199+0000","last_became_active":"2024-04-24T23:26:18.179798+0000","last_became_peered":"2024-04-24T23:26:18.179798+0000","last_unstale":"2024-04-24T23:26:19.370199+0000","last_undegraded":"2024-04-24T23:26:19.370199+0000","last_fullsized":"2024-04-24T23:26:19.370199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:59.026283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172112+0000","last_change":"2024-04-24T23:26:18.172770+0000","last_active":"2024-04-24T23:26:19.172112+0000","last_peered":"2024-04-24T23:26:19.172112+0000","last_clean":"2024-04-24T23:26:19.172112+0000","last_became_active":"2024-04-24T23:26:18.172478+0000","last_became_peered":"2024-04-24T23:26:18.172478+0000","last_unstale":"2024-04-24T23:26:19.172112+0000","last_undegraded":"2024-04-24T23:26:19.172112+0000","last_fullsized":"2024-04-24T23:26:19.172112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:41.660546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369673+0000","last_change":"2024-04-24T23:26:18.179272+0000","last_active":"2024-04-24T23:26:19.369673+0000","last_peered":"2024-04-24T23:26:19.369673+0000","last_clean":"2024-04-24T23:26:19.369673+0000","last_became_active":"2024-04-24T23:26:18.178927+0000","last_became_peered":"2024-04-24T23:26:18.178927+0000","last_unstale":"2024-04-24T23:26:19.369673+0000","last_undegraded":"2024-04-24T23:26:19.369673+0000","last_fullsized":"2024-04-24T23:26:19.369673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:08.371069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463828+0000","last_change":"2024-04-24T23:26:18.179069+0000","last_active":"2024-04-24T23:26:19.463828+0000","last_peered":"2024-04-24T23:26:19.463828+0000","last_clean":"2024-04-24T23:26:19.463828+0000","last_became_active":"2024-04-24T23:26:18.178734+0000","last_became_peered":"2024-04-24T23:26:18.178734+0000","last_unstale":"2024-04-24T23:26:19.463828+0000","last_undegraded":"2024-04-24T23:26:19.463828+0000","last_fullsized":"2024-04-24T23:26:19.463828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:32.655169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463644+0000","last_change":"2024-04-24T23:26:18.173011+0000","last_active":"2024-04-24T23:26:19.463644+0000","last_peered":"2024-04-24T23:26:19.463644+0000","last_clean":"2024-04-24T23:26:19.463644+0000","last_became_active":"2024-04-24T23:26:18.172727+0000","last_became_peered":"2024-04-24T23:26:18.172727+0000","last_unstale":"2024-04-24T23:26:19.463644+0000","last_undegraded":"2024-04-24T23:26:19.463644+0000","last_fullsized":"2024-04-24T23:26:19.463644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:22.487949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463030+0000","last_change":"2024-04-24T23:26:18.172692+0000","last_active":"2024-04-24T23:26:19.463030+0000","last_peered":"2024-04-24T23:26:19.463030+0000","last_clean":"2024-04-24T23:26:19.463030+0000","last_became_active":"2024-04-24T23:26:18.172391+0000","last_became_peered":"2024-04-24T23:26:18.172391+0000","last_unstale":"2024-04-24T23:26:19.463030+0000","last_undegraded":"2024-04-24T23:26:19.463030+0000","last_fullsized":"2024-04-24T23:26:19.463030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:02.363430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166356+0000","last_change":"2024-04-24T23:26:18.180886+0000","last_active":"2024-04-24T23:26:19.166356+0000","last_peered":"2024-04-24T23:26:19.166356+0000","last_clean":"2024-04-24T23:26:19.166356+0000","last_became_active":"2024-04-24T23:26:18.180601+0000","last_became_peered":"2024-04-24T23:26:18.180601+0000","last_unstale":"2024-04-24T23:26:19.166356+0000","last_undegraded":"2024-04-24T23:26:19.166356+0000","last_fullsized":"2024-04-24T23:26:19.166356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:37.471755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167034+0000","last_change":"2024-04-24T23:26:18.170493+0000","last_active":"2024-04-24T23:26:19.167034+0000","last_peered":"2024-04-24T23:26:19.167034+0000","last_clean":"2024-04-24T23:26:19.167034+0000","last_became_active":"2024-04-24T23:26:18.170190+0000","last_became_peered":"2024-04-24T23:26:18.170190+0000","last_unstale":"2024-04-24T23:26:19.167034+0000","last_undegraded":"2024-04-24T23:26:19.167034+0000","last_fullsized":"2024-04-24T23:26:19.167034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:54.744768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463875+0000","last_change":"2024-04-24T23:26:18.181808+0000","last_active":"2024-04-24T23:26:19.463875+0000","last_peered":"2024-04-24T23:26:19.463875+0000","last_clean":"2024-04-24T23:26:19.463875+0000","last_became_active":"2024-04-24T23:26:18.181513+0000","last_became_peered":"2024-04-24T23:26:18.181513+0000","last_unstale":"2024-04-24T23:26:19.463875+0000","last_undegraded":"2024-04-24T23:26:19.463875+0000","last_fullsized":"2024-04-24T23:26:19.463875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:23.784890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174236+0000","last_change":"2024-04-24T23:26:18.167771+0000","last_active":"2024-04-24T23:26:19.174236+0000","last_peered":"2024-04-24T23:26:19.174236+0000","last_clean":"2024-04-24T23:26:19.174236+0000","last_became_active":"2024-04-24T23:26:18.167488+0000","last_became_peered":"2024-04-24T23:26:18.167488+0000","last_unstale":"2024-04-24T23:26:19.174236+0000","last_undegraded":"2024-04-24T23:26:19.174236+0000","last_fullsized":"2024-04-24T23:26:19.174236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:08.137883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463947+0000","last_change":"2024-04-24T23:26:18.169427+0000","last_active":"2024-04-24T23:26:19.463947+0000","last_peered":"2024-04-24T23:26:19.463947+0000","last_clean":"2024-04-24T23:26:19.463947+0000","last_became_active":"2024-04-24T23:26:18.169139+0000","last_became_peered":"2024-04-24T23:26:18.169139+0000","last_unstale":"2024-04-24T23:26:19.463947+0000","last_undegraded":"2024-04-24T23:26:19.463947+0000","last_fullsized":"2024-04-24T23:26:19.463947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:37:32.961764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463579+0000","last_change":"2024-04-24T23:26:18.169656+0000","last_active":"2024-04-24T23:26:19.463579+0000","last_peered":"2024-04-24T23:26:19.463579+0000","last_clean":"2024-04-24T23:26:19.463579+0000","last_became_active":"2024-04-24T23:26:18.165677+0000","last_became_peered":"2024-04-24T23:26:18.165677+0000","last_unstale":"2024-04-24T23:26:19.463579+0000","last_undegraded":"2024-04-24T23:26:19.463579+0000","last_fullsized":"2024-04-24T23:26:19.463579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:45.726822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171871+0000","last_change":"2024-04-24T23:26:18.171619+0000","last_active":"2024-04-24T23:26:19.171871+0000","last_peered":"2024-04-24T23:26:19.171871+0000","last_clean":"2024-04-24T23:26:19.171871+0000","last_became_active":"2024-04-24T23:26:18.171306+0000","last_became_peered":"2024-04-24T23:26:18.171306+0000","last_unstale":"2024-04-24T23:26:19.171871+0000","last_undegraded":"2024-04-24T23:26:19.171871+0000","last_fullsized":"2024-04-24T23:26:19.171871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:01.573306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369707+0000","last_change":"2024-04-24T23:26:18.178885+0000","last_active":"2024-04-24T23:26:19.369707+0000","last_peered":"2024-04-24T23:26:19.369707+0000","last_clean":"2024-04-24T23:26:19.369707+0000","last_became_active":"2024-04-24T23:26:18.178432+0000","last_became_peered":"2024-04-24T23:26:18.178432+0000","last_unstale":"2024-04-24T23:26:19.369707+0000","last_undegraded":"2024-04-24T23:26:19.369707+0000","last_fullsized":"2024-04-24T23:26:19.369707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:38.603796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463237+0000","last_change":"2024-04-24T23:26:18.169319+0000","last_active":"2024-04-24T23:26:19.463237+0000","last_peered":"2024-04-24T23:26:19.463237+0000","last_clean":"2024-04-24T23:26:19.463237+0000","last_became_active":"2024-04-24T23:26:18.169050+0000","last_became_peered":"2024-04-24T23:26:18.169050+0000","last_unstale":"2024-04-24T23:26:19.463237+0000","last_undegraded":"2024-04-24T23:26:19.463237+0000","last_fullsized":"2024-04-24T23:26:19.463237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:38.988687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172127+0000","last_change":"2024-04-24T23:26:18.181916+0000","last_active":"2024-04-24T23:26:19.172127+0000","last_peered":"2024-04-24T23:26:19.172127+0000","last_clean":"2024-04-24T23:26:19.172127+0000","last_became_active":"2024-04-24T23:26:18.179385+0000","last_became_peered":"2024-04-24T23:26:18.179385+0000","last_unstale":"2024-04-24T23:26:19.172127+0000","last_undegraded":"2024-04-24T23:26:19.172127+0000","last_fullsized":"2024-04-24T23:26:19.172127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:08.940182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463949+0000","last_change":"2024-04-24T23:26:18.176128+0000","last_active":"2024-04-24T23:26:19.463949+0000","last_peered":"2024-04-24T23:26:19.463949+0000","last_clean":"2024-04-24T23:26:19.463949+0000","last_became_active":"2024-04-24T23:26:18.175801+0000","last_became_peered":"2024-04-24T23:26:18.175801+0000","last_unstale":"2024-04-24T23:26:19.463949+0000","last_undegraded":"2024-04-24T23:26:19.463949+0000","last_fullsized":"2024-04-24T23:26:19.463949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:17.771410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167252+0000","last_change":"2024-04-24T23:26:18.173268+0000","last_active":"2024-04-24T23:26:19.167252+0000","last_peered":"2024-04-24T23:26:19.167252+0000","last_clean":"2024-04-24T23:26:19.167252+0000","last_became_active":"2024-04-24T23:26:18.172981+0000","last_became_peered":"2024-04-24T23:26:18.172981+0000","last_unstale":"2024-04-24T23:26:19.167252+0000","last_undegraded":"2024-04-24T23:26:19.167252+0000","last_fullsized":"2024-04-24T23:26:19.167252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:35.874424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369835+0000","last_change":"2024-04-24T23:26:18.179248+0000","last_active":"2024-04-24T23:26:19.369835+0000","last_peered":"2024-04-24T23:26:19.369835+0000","last_clean":"2024-04-24T23:26:19.369835+0000","last_became_active":"2024-04-24T23:26:18.178895+0000","last_became_peered":"2024-04-24T23:26:18.178895+0000","last_unstale":"2024-04-24T23:26:19.369835+0000","last_undegraded":"2024-04-24T23:26:19.369835+0000","last_fullsized":"2024-04-24T23:26:19.369835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:24.189593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172026+0000","last_change":"2024-04-24T23:26:18.170742+0000","last_active":"2024-04-24T23:26:19.172026+0000","last_peered":"2024-04-24T23:26:19.172026+0000","last_clean":"2024-04-24T23:26:19.172026+0000","last_became_active":"2024-04-24T23:26:18.170475+0000","last_became_peered":"2024-04-24T23:26:18.170475+0000","last_unstale":"2024-04-24T23:26:19.172026+0000","last_undegraded":"2024-04-24T23:26:19.172026+0000","last_fullsized":"2024-04-24T23:26:19.172026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:26.567578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171882+0000","last_change":"2024-04-24T23:26:18.174295+0000","last_active":"2024-04-24T23:26:19.171882+0000","last_peered":"2024-04-24T23:26:19.171882+0000","last_clean":"2024-04-24T23:26:19.171882+0000","last_became_active":"2024-04-24T23:26:18.173905+0000","last_became_peered":"2024-04-24T23:26:18.173905+0000","last_unstale":"2024-04-24T23:26:19.171882+0000","last_undegraded":"2024-04-24T23:26:19.171882+0000","last_fullsized":"2024-04-24T23:26:19.171882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:37.002157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166346+0000","last_change":"2024-04-24T23:26:18.168808+0000","last_active":"2024-04-24T23:26:19.166346+0000","last_peered":"2024-04-24T23:26:19.166346+0000","last_clean":"2024-04-24T23:26:19.166346+0000","last_became_active":"2024-04-24T23:26:18.168532+0000","last_became_peered":"2024-04-24T23:26:18.168532+0000","last_unstale":"2024-04-24T23:26:19.166346+0000","last_undegraded":"2024-04-24T23:26:19.166346+0000","last_fullsized":"2024-04-24T23:26:19.166346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:53.405043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369573+0000","last_change":"2024-04-24T23:26:18.185209+0000","last_active":"2024-04-24T23:26:19.369573+0000","last_peered":"2024-04-24T23:26:19.369573+0000","last_clean":"2024-04-24T23:26:19.369573+0000","last_became_active":"2024-04-24T23:26:18.184905+0000","last_became_peered":"2024-04-24T23:26:18.184905+0000","last_unstale":"2024-04-24T23:26:19.369573+0000","last_undegraded":"2024-04-24T23:26:19.369573+0000","last_fullsized":"2024-04-24T23:26:19.369573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:45.758300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167020+0000","last_change":"2024-04-24T23:26:18.181402+0000","last_active":"2024-04-24T23:26:19.167020+0000","last_peered":"2024-04-24T23:26:19.167020+0000","last_clean":"2024-04-24T23:26:19.167020+0000","last_became_active":"2024-04-24T23:26:18.181116+0000","last_became_peered":"2024-04-24T23:26:18.181116+0000","last_unstale":"2024-04-24T23:26:19.167020+0000","last_undegraded":"2024-04-24T23:26:19.167020+0000","last_fullsized":"2024-04-24T23:26:19.167020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:04.106671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370302+0000","last_change":"2024-04-24T23:26:17.166716+0000","last_active":"2024-04-24T23:26:19.370302+0000","last_peered":"2024-04-24T23:26:19.370302+0000","last_clean":"2024-04-24T23:26:19.370302+0000","last_became_active":"2024-04-24T23:26:17.166397+0000","last_became_peered":"2024-04-24T23:26:17.166397+0000","last_unstale":"2024-04-24T23:26:19.370302+0000","last_undegraded":"2024-04-24T23:26:19.370302+0000","last_fullsized":"2024-04-24T23:26:19.370302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:59.737718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370213+0000","last_change":"2024-04-24T23:26:18.184041+0000","last_active":"2024-04-24T23:26:19.370213+0000","last_peered":"2024-04-24T23:26:19.370213+0000","last_clean":"2024-04-24T23:26:19.370213+0000","last_became_active":"2024-04-24T23:26:18.181518+0000","last_became_peered":"2024-04-24T23:26:18.181518+0000","last_unstale":"2024-04-24T23:26:19.370213+0000","last_undegraded":"2024-04-24T23:26:19.370213+0000","last_fullsized":"2024-04-24T23:26:19.370213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:04.106671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167264+0000","last_change":"2024-04-24T23:26:17.162927+0000","last_active":"2024-04-24T23:26:19.167264+0000","last_peered":"2024-04-24T23:26:19.167264+0000","last_clean":"2024-04-24T23:26:19.167264+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:19.167264+0000","last_undegraded":"2024-04-24T23:26:19.167264+0000","last_fullsized":"2024-04-24T23:26:19.167264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:18.000135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369912+0000","last_change":"2024-04-24T23:26:18.182994+0000","last_active":"2024-04-24T23:26:19.369912+0000","last_peered":"2024-04-24T23:26:19.369912+0000","last_clean":"2024-04-24T23:26:19.369912+0000","last_became_active":"2024-04-24T23:26:18.182675+0000","last_became_peered":"2024-04-24T23:26:18.182675+0000","last_unstale":"2024-04-24T23:26:19.369912+0000","last_undegraded":"2024-04-24T23:26:19.369912+0000","last_fullsized":"2024-04-24T23:26:19.369912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:53.405043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174279+0000","last_change":"2024-04-24T23:26:17.166726+0000","last_active":"2024-04-24T23:26:19.174279+0000","last_peered":"2024-04-24T23:26:19.174279+0000","last_clean":"2024-04-24T23:26:19.174279+0000","last_became_active":"2024-04-24T23:26:17.166370+0000","last_became_peered":"2024-04-24T23:26:17.166370+0000","last_unstale":"2024-04-24T23:26:19.174279+0000","last_undegraded":"2024-04-24T23:26:19.174279+0000","last_fullsized":"2024-04-24T23:26:19.174279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:23.185199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463949+0000","last_change":"2024-04-24T23:26:18.175108+0000","last_active":"2024-04-24T23:26:19.463949+0000","last_peered":"2024-04-24T23:26:19.463949+0000","last_clean":"2024-04-24T23:26:19.463949+0000","last_became_active":"2024-04-24T23:26:18.174822+0000","last_became_peered":"2024-04-24T23:26:18.174822+0000","last_unstale":"2024-04-24T23:26:19.463949+0000","last_undegraded":"2024-04-24T23:26:19.463949+0000","last_fullsized":"2024-04-24T23:26:19.463949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:55.214805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174102+0000","last_change":"2024-04-24T23:26:17.167115+0000","last_active":"2024-04-24T23:26:19.174102+0000","last_peered":"2024-04-24T23:26:19.174102+0000","last_clean":"2024-04-24T23:26:19.174102+0000","last_became_active":"2024-04-24T23:26:17.166838+0000","last_became_peered":"2024-04-24T23:26:17.166838+0000","last_unstale":"2024-04-24T23:26:19.174102+0000","last_undegraded":"2024-04-24T23:26:19.174102+0000","last_fullsized":"2024-04-24T23:26:19.174102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:52.400650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172127+0000","last_change":"2024-04-24T23:26:18.173810+0000","last_active":"2024-04-24T23:26:19.172127+0000","last_peered":"2024-04-24T23:26:19.172127+0000","last_clean":"2024-04-24T23:26:19.172127+0000","last_became_active":"2024-04-24T23:26:18.173146+0000","last_became_peered":"2024-04-24T23:26:18.173146+0000","last_unstale":"2024-04-24T23:26:19.172127+0000","last_undegraded":"2024-04-24T23:26:19.172127+0000","last_fullsized":"2024-04-24T23:26:19.172127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:43.460492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167102+0000","last_change":"2024-04-24T23:26:17.166049+0000","last_active":"2024-04-24T23:26:19.167102+0000","last_peered":"2024-04-24T23:26:19.167102+0000","last_clean":"2024-04-24T23:26:19.167102+0000","last_became_active":"2024-04-24T23:26:17.165763+0000","last_became_peered":"2024-04-24T23:26:17.165763+0000","last_unstale":"2024-04-24T23:26:19.167102+0000","last_undegraded":"2024-04-24T23:26:19.167102+0000","last_fullsized":"2024-04-24T23:26:19.167102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:01.255107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166897+0000","last_change":"2024-04-24T23:26:18.172391+0000","last_active":"2024-04-24T23:26:19.166897+0000","last_peered":"2024-04-24T23:26:19.166897+0000","last_clean":"2024-04-24T23:26:19.166897+0000","last_became_active":"2024-04-24T23:26:18.172096+0000","last_became_peered":"2024-04-24T23:26:18.172096+0000","last_unstale":"2024-04-24T23:26:19.166897+0000","last_undegraded":"2024-04-24T23:26:19.166897+0000","last_fullsized":"2024-04-24T23:26:19.166897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:59.026283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166463+0000","last_change":"2024-04-24T23:26:17.161576+0000","last_active":"2024-04-24T23:26:19.166463+0000","last_peered":"2024-04-24T23:26:19.166463+0000","last_clean":"2024-04-24T23:26:19.166463+0000","last_became_active":"2024-04-24T23:26:17.161260+0000","last_became_peered":"2024-04-24T23:26:17.161260+0000","last_unstale":"2024-04-24T23:26:19.166463+0000","last_undegraded":"2024-04-24T23:26:19.166463+0000","last_fullsized":"2024-04-24T23:26:19.166463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:42.456099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463602+0000","last_change":"2024-04-24T23:26:18.173192+0000","last_active":"2024-04-24T23:26:19.463602+0000","last_peered":"2024-04-24T23:26:19.463602+0000","last_clean":"2024-04-24T23:26:19.463602+0000","last_became_active":"2024-04-24T23:26:18.172915+0000","last_became_peered":"2024-04-24T23:26:18.172915+0000","last_unstale":"2024-04-24T23:26:19.463602+0000","last_undegraded":"2024-04-24T23:26:19.463602+0000","last_fullsized":"2024-04-24T23:26:19.463602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:08.137883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.173896+0000","last_change":"2024-04-24T23:26:17.167646+0000","last_active":"2024-04-24T23:26:19.173896+0000","last_peered":"2024-04-24T23:26:19.173896+0000","last_clean":"2024-04-24T23:26:19.173896+0000","last_became_active":"2024-04-24T23:26:17.167351+0000","last_became_peered":"2024-04-24T23:26:17.167351+0000","last_unstale":"2024-04-24T23:26:19.173896+0000","last_undegraded":"2024-04-24T23:26:19.173896+0000","last_fullsized":"2024-04-24T23:26:19.173896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:07.366676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463063+0000","last_change":"2024-04-24T23:26:18.164600+0000","last_active":"2024-04-24T23:26:19.463063+0000","last_peered":"2024-04-24T23:26:19.463063+0000","last_clean":"2024-04-24T23:26:19.463063+0000","last_became_active":"2024-04-24T23:26:18.164337+0000","last_became_peered":"2024-04-24T23:26:18.164337+0000","last_unstale":"2024-04-24T23:26:19.463063+0000","last_undegraded":"2024-04-24T23:26:19.463063+0000","last_fullsized":"2024-04-24T23:26:19.463063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:20.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171839+0000","last_change":"2024-04-24T23:26:17.162901+0000","last_active":"2024-04-24T23:26:19.171839+0000","last_peered":"2024-04-24T23:26:19.171839+0000","last_clean":"2024-04-24T23:26:19.171839+0000","last_became_active":"2024-04-24T23:26:17.162604+0000","last_became_peered":"2024-04-24T23:26:17.162604+0000","last_unstale":"2024-04-24T23:26:19.171839+0000","last_undegraded":"2024-04-24T23:26:19.171839+0000","last_fullsized":"2024-04-24T23:26:19.171839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:29.008878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463335+0000","last_change":"2024-04-24T23:26:18.172560+0000","last_active":"2024-04-24T23:26:19.463335+0000","last_peered":"2024-04-24T23:26:19.463335+0000","last_clean":"2024-04-24T23:26:19.463335+0000","last_became_active":"2024-04-24T23:26:18.172167+0000","last_became_peered":"2024-04-24T23:26:18.172167+0000","last_unstale":"2024-04-24T23:26:19.463335+0000","last_undegraded":"2024-04-24T23:26:19.463335+0000","last_fullsized":"2024-04-24T23:26:19.463335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:20.523125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370361+0000","last_change":"2024-04-24T23:26:17.167550+0000","last_active":"2024-04-24T23:26:19.370361+0000","last_peered":"2024-04-24T23:26:19.370361+0000","last_clean":"2024-04-24T23:26:19.370361+0000","last_became_active":"2024-04-24T23:26:17.167229+0000","last_became_peered":"2024-04-24T23:26:17.167229+0000","last_unstale":"2024-04-24T23:26:19.370361+0000","last_undegraded":"2024-04-24T23:26:19.370361+0000","last_fullsized":"2024-04-24T23:26:19.370361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:00.568912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167936+0000","last_change":"2024-04-24T23:26:18.176333+0000","last_active":"2024-04-24T23:26:19.167936+0000","last_peered":"2024-04-24T23:26:19.167936+0000","last_clean":"2024-04-24T23:26:19.167936+0000","last_became_active":"2024-04-24T23:26:18.176039+0000","last_became_peered":"2024-04-24T23:26:18.176039+0000","last_unstale":"2024-04-24T23:26:19.167936+0000","last_undegraded":"2024-04-24T23:26:19.167936+0000","last_fullsized":"2024-04-24T23:26:19.167936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:54.744768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370102+0000","last_change":"2024-04-24T23:26:17.168677+0000","last_active":"2024-04-24T23:26:19.370102+0000","last_peered":"2024-04-24T23:26:19.370102+0000","last_clean":"2024-04-24T23:26:19.370102+0000","last_became_active":"2024-04-24T23:26:17.168354+0000","last_became_peered":"2024-04-24T23:26:17.168354+0000","last_unstale":"2024-04-24T23:26:19.370102+0000","last_undegraded":"2024-04-24T23:26:19.370102+0000","last_fullsized":"2024-04-24T23:26:19.370102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:25.563184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174078+0000","last_change":"2024-04-24T23:26:18.180771+0000","last_active":"2024-04-24T23:26:19.174078+0000","last_peered":"2024-04-24T23:26:19.174078+0000","last_clean":"2024-04-24T23:26:19.174078+0000","last_became_active":"2024-04-24T23:26:18.180497+0000","last_became_peered":"2024-04-24T23:26:18.180497+0000","last_unstale":"2024-04-24T23:26:19.174078+0000","last_undegraded":"2024-04-24T23:26:19.174078+0000","last_fullsized":"2024-04-24T23:26:19.174078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:19.473276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369658+0000","last_change":"2024-04-24T23:26:17.170110+0000","last_active":"2024-04-24T23:26:19.369658+0000","last_peered":"2024-04-24T23:26:19.369658+0000","last_clean":"2024-04-24T23:26:19.369658+0000","last_became_active":"2024-04-24T23:26:17.169755+0000","last_became_peered":"2024-04-24T23:26:17.169755+0000","last_unstale":"2024-04-24T23:26:19.369658+0000","last_undegraded":"2024-04-24T23:26:19.369658+0000","last_fullsized":"2024-04-24T23:26:19.369658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:47.906621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167370+0000","last_change":"2024-04-24T23:26:18.172691+0000","last_active":"2024-04-24T23:26:19.167370+0000","last_peered":"2024-04-24T23:26:19.167370+0000","last_clean":"2024-04-24T23:26:19.167370+0000","last_became_active":"2024-04-24T23:26:18.172294+0000","last_became_peered":"2024-04-24T23:26:18.172294+0000","last_unstale":"2024-04-24T23:26:19.167370+0000","last_undegraded":"2024-04-24T23:26:19.167370+0000","last_fullsized":"2024-04-24T23:26:19.167370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:23.707858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167799+0000","last_change":"2024-04-24T23:26:17.151318+0000","last_active":"2024-04-24T23:26:19.167799+0000","last_peered":"2024-04-24T23:26:19.167799+0000","last_clean":"2024-04-24T23:26:19.167799+0000","last_became_active":"2024-04-24T23:26:17.151022+0000","last_became_peered":"2024-04-24T23:26:17.151022+0000","last_unstale":"2024-04-24T23:26:19.167799+0000","last_undegraded":"2024-04-24T23:26:19.167799+0000","last_fullsized":"2024-04-24T23:26:19.167799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:55.620251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.173969+0000","last_change":"2024-04-24T23:26:18.176225+0000","last_active":"2024-04-24T23:26:19.173969+0000","last_peered":"2024-04-24T23:26:19.173969+0000","last_clean":"2024-04-24T23:26:19.173969+0000","last_became_active":"2024-04-24T23:26:18.175893+0000","last_became_peered":"2024-04-24T23:26:18.175893+0000","last_unstale":"2024-04-24T23:26:19.173969+0000","last_undegraded":"2024-04-24T23:26:19.173969+0000","last_fullsized":"2024-04-24T23:26:19.173969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:23.707858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172012+0000","last_change":"2024-04-24T23:26:17.162943+0000","last_active":"2024-04-24T23:26:19.172012+0000","last_peered":"2024-04-24T23:26:19.172012+0000","last_clean":"2024-04-24T23:26:19.172012+0000","last_became_active":"2024-04-24T23:26:17.162650+0000","last_became_peered":"2024-04-24T23:26:17.162650+0000","last_unstale":"2024-04-24T23:26:19.172012+0000","last_undegraded":"2024-04-24T23:26:19.172012+0000","last_fullsized":"2024-04-24T23:26:19.172012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:21.483555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463963+0000","last_change":"2024-04-24T23:26:18.173884+0000","last_active":"2024-04-24T23:26:19.463963+0000","last_peered":"2024-04-24T23:26:19.463963+0000","last_clean":"2024-04-24T23:26:19.463963+0000","last_became_active":"2024-04-24T23:26:18.173532+0000","last_became_peered":"2024-04-24T23:26:18.173532+0000","last_unstale":"2024-04-24T23:26:19.463963+0000","last_undegraded":"2024-04-24T23:26:19.463963+0000","last_fullsized":"2024-04-24T23:26:19.463963+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:40.785566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463179+0000","last_change":"2024-04-24T23:26:17.163630+0000","last_active":"2024-04-24T23:26:19.463179+0000","last_peered":"2024-04-24T23:26:19.463179+0000","last_clean":"2024-04-24T23:26:19.463179+0000","last_became_active":"2024-04-24T23:26:17.163331+0000","last_became_peered":"2024-04-24T23:26:17.163331+0000","last_unstale":"2024-04-24T23:26:19.463179+0000","last_undegraded":"2024-04-24T23:26:19.463179+0000","last_fullsized":"2024-04-24T23:26:19.463179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:54.210411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171817+0000","last_change":"2024-04-24T23:26:18.174543+0000","last_active":"2024-04-24T23:26:19.171817+0000","last_peered":"2024-04-24T23:26:19.171817+0000","last_clean":"2024-04-24T23:26:19.171817+0000","last_became_active":"2024-04-24T23:26:18.173665+0000","last_became_peered":"2024-04-24T23:26:18.173665+0000","last_unstale":"2024-04-24T23:26:19.171817+0000","last_undegraded":"2024-04-24T23:26:19.171817+0000","last_fullsized":"2024-04-24T23:26:19.171817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:00.742112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167436+0000","last_change":"2024-04-24T23:26:17.160593+0000","last_active":"2024-04-24T23:26:19.167436+0000","last_peered":"2024-04-24T23:26:19.167436+0000","last_clean":"2024-04-24T23:26:19.167436+0000","last_became_active":"2024-04-24T23:26:17.160303+0000","last_became_peered":"2024-04-24T23:26:17.160303+0000","last_unstale":"2024-04-24T23:26:19.167436+0000","last_undegraded":"2024-04-24T23:26:19.167436+0000","last_fullsized":"2024-04-24T23:26:19.167436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:28.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.171783+0000","last_change":"2024-04-24T23:26:18.179756+0000","last_active":"2024-04-24T23:26:19.171783+0000","last_peered":"2024-04-24T23:26:19.171783+0000","last_clean":"2024-04-24T23:26:19.171783+0000","last_became_active":"2024-04-24T23:26:18.179454+0000","last_became_peered":"2024-04-24T23:26:18.179454+0000","last_unstale":"2024-04-24T23:26:19.171783+0000","last_undegraded":"2024-04-24T23:26:19.171783+0000","last_fullsized":"2024-04-24T23:26:19.171783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:06.225251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.173835+0000","last_change":"2024-04-24T23:26:17.165945+0000","last_active":"2024-04-24T23:26:19.173835+0000","last_peered":"2024-04-24T23:26:19.173835+0000","last_clean":"2024-04-24T23:26:19.173835+0000","last_became_active":"2024-04-24T23:26:17.164750+0000","last_became_peered":"2024-04-24T23:26:17.164750+0000","last_unstale":"2024-04-24T23:26:19.173835+0000","last_undegraded":"2024-04-24T23:26:19.173835+0000","last_fullsized":"2024-04-24T23:26:19.173835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:51.395536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.464027+0000","last_change":"2024-04-24T23:26:18.168783+0000","last_active":"2024-04-24T23:26:19.464027+0000","last_peered":"2024-04-24T23:26:19.464027+0000","last_clean":"2024-04-24T23:26:19.464027+0000","last_became_active":"2024-04-24T23:26:18.168449+0000","last_became_peered":"2024-04-24T23:26:18.168449+0000","last_unstale":"2024-04-24T23:26:19.464027+0000","last_undegraded":"2024-04-24T23:26:19.464027+0000","last_fullsized":"2024-04-24T23:26:19.464027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:50.240258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370433+0000","last_change":"2024-04-24T23:26:17.165977+0000","last_active":"2024-04-24T23:26:19.370433+0000","last_peered":"2024-04-24T23:26:19.370433+0000","last_clean":"2024-04-24T23:26:19.370433+0000","last_became_active":"2024-04-24T23:26:17.165641+0000","last_became_peered":"2024-04-24T23:26:17.165641+0000","last_unstale":"2024-04-24T23:26:19.370433+0000","last_undegraded":"2024-04-24T23:26:19.370433+0000","last_fullsized":"2024-04-24T23:26:19.370433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.997763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370287+0000","last_change":"2024-04-24T23:26:18.180504+0000","last_active":"2024-04-24T23:26:19.370287+0000","last_peered":"2024-04-24T23:26:19.370287+0000","last_clean":"2024-04-24T23:26:19.370287+0000","last_became_active":"2024-04-24T23:26:18.179646+0000","last_became_peered":"2024-04-24T23:26:18.179646+0000","last_unstale":"2024-04-24T23:26:19.370287+0000","last_undegraded":"2024-04-24T23:26:19.370287+0000","last_fullsized":"2024-04-24T23:26:19.370287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:54.744768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174158+0000","last_change":"2024-04-24T23:26:17.163913+0000","last_active":"2024-04-24T23:26:19.174158+0000","last_peered":"2024-04-24T23:26:19.174158+0000","last_clean":"2024-04-24T23:26:19.174158+0000","last_became_active":"2024-04-24T23:26:17.163640+0000","last_became_peered":"2024-04-24T23:26:17.163640+0000","last_unstale":"2024-04-24T23:26:19.174158+0000","last_undegraded":"2024-04-24T23:26:19.174158+0000","last_fullsized":"2024-04-24T23:26:19.174158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:44.753907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167446+0000","last_change":"2024-04-24T23:26:18.175458+0000","last_active":"2024-04-24T23:26:19.167446+0000","last_peered":"2024-04-24T23:26:19.167446+0000","last_clean":"2024-04-24T23:26:19.167446+0000","last_became_active":"2024-04-24T23:26:18.175117+0000","last_became_peered":"2024-04-24T23:26:18.175117+0000","last_unstale":"2024-04-24T23:26:19.167446+0000","last_undegraded":"2024-04-24T23:26:19.167446+0000","last_fullsized":"2024-04-24T23:26:19.167446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:29.129749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174166+0000","last_change":"2024-04-24T23:26:17.167926+0000","last_active":"2024-04-24T23:26:19.174166+0000","last_peered":"2024-04-24T23:26:19.174166+0000","last_clean":"2024-04-24T23:26:19.174166+0000","last_became_active":"2024-04-24T23:26:17.167660+0000","last_became_peered":"2024-04-24T23:26:17.167660+0000","last_unstale":"2024-04-24T23:26:19.174166+0000","last_undegraded":"2024-04-24T23:26:19.174166+0000","last_fullsized":"2024-04-24T23:26:19.174166+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.467361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463642+0000","last_change":"2024-04-24T23:26:18.172592+0000","last_active":"2024-04-24T23:26:19.463642+0000","last_peered":"2024-04-24T23:26:19.463642+0000","last_clean":"2024-04-24T23:26:19.463642+0000","last_became_active":"2024-04-24T23:26:18.172290+0000","last_became_peered":"2024-04-24T23:26:18.172290+0000","last_unstale":"2024-04-24T23:26:19.463642+0000","last_undegraded":"2024-04-24T23:26:19.463642+0000","last_fullsized":"2024-04-24T23:26:19.463642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:30.013272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463374+0000","last_change":"2024-04-24T23:26:17.158322+0000","last_active":"2024-04-24T23:26:19.463374+0000","last_peered":"2024-04-24T23:26:19.463374+0000","last_clean":"2024-04-24T23:26:19.463374+0000","last_became_active":"2024-04-24T23:26:17.158052+0000","last_became_peered":"2024-04-24T23:26:17.158052+0000","last_unstale":"2024-04-24T23:26:19.463374+0000","last_undegraded":"2024-04-24T23:26:19.463374+0000","last_fullsized":"2024-04-24T23:26:19.463374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:31.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167801+0000","last_change":"2024-04-24T23:26:18.165683+0000","last_active":"2024-04-24T23:26:19.167801+0000","last_peered":"2024-04-24T23:26:19.167801+0000","last_clean":"2024-04-24T23:26:19.167801+0000","last_became_active":"2024-04-24T23:26:18.165354+0000","last_became_peered":"2024-04-24T23:26:18.165354+0000","last_unstale":"2024-04-24T23:26:19.167801+0000","last_undegraded":"2024-04-24T23:26:19.167801+0000","last_fullsized":"2024-04-24T23:26:19.167801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:59.026283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463731+0000","last_change":"2024-04-24T23:26:17.167861+0000","last_active":"2024-04-24T23:26:19.463731+0000","last_peered":"2024-04-24T23:26:19.463731+0000","last_clean":"2024-04-24T23:26:19.463731+0000","last_became_active":"2024-04-24T23:26:17.167598+0000","last_became_peered":"2024-04-24T23:26:17.167598+0000","last_unstale":"2024-04-24T23:26:19.463731+0000","last_undegraded":"2024-04-24T23:26:19.463731+0000","last_fullsized":"2024-04-24T23:26:19.463731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:46.915793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463466+0000","last_change":"2024-04-24T23:26:18.163644+0000","last_active":"2024-04-24T23:26:19.463466+0000","last_peered":"2024-04-24T23:26:19.463466+0000","last_clean":"2024-04-24T23:26:19.463466+0000","last_became_active":"2024-04-24T23:26:18.163326+0000","last_became_peered":"2024-04-24T23:26:18.163326+0000","last_unstale":"2024-04-24T23:26:19.463466+0000","last_undegraded":"2024-04-24T23:26:19.463466+0000","last_fullsized":"2024-04-24T23:26:19.463466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:43.377906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463456+0000","last_change":"2024-04-24T23:26:17.165956+0000","last_active":"2024-04-24T23:26:19.463456+0000","last_peered":"2024-04-24T23:26:19.463456+0000","last_clean":"2024-04-24T23:26:19.463456+0000","last_became_active":"2024-04-24T23:26:17.165633+0000","last_became_peered":"2024-04-24T23:26:17.165633+0000","last_unstale":"2024-04-24T23:26:19.463456+0000","last_undegraded":"2024-04-24T23:26:19.463456+0000","last_fullsized":"2024-04-24T23:26:19.463456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:31.605822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166935+0000","last_change":"2024-04-24T23:26:18.173214+0000","last_active":"2024-04-24T23:26:19.166935+0000","last_peered":"2024-04-24T23:26:19.166935+0000","last_clean":"2024-04-24T23:26:19.166935+0000","last_became_active":"2024-04-24T23:26:18.172837+0000","last_became_peered":"2024-04-24T23:26:18.172837+0000","last_unstale":"2024-04-24T23:26:19.166935+0000","last_undegraded":"2024-04-24T23:26:19.166935+0000","last_fullsized":"2024-04-24T23:26:19.166935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:24.189593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.462917+0000","last_change":"2024-04-24T23:26:17.157597+0000","last_active":"2024-04-24T23:26:19.462917+0000","last_peered":"2024-04-24T23:26:19.462917+0000","last_clean":"2024-04-24T23:26:19.462917+0000","last_became_active":"2024-04-24T23:26:17.157311+0000","last_became_peered":"2024-04-24T23:26:17.157311+0000","last_unstale":"2024-04-24T23:26:19.462917+0000","last_undegraded":"2024-04-24T23:26:19.462917+0000","last_fullsized":"2024-04-24T23:26:19.462917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.781173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167189+0000","last_change":"2024-04-24T23:26:18.164360+0000","last_active":"2024-04-24T23:26:19.167189+0000","last_peered":"2024-04-24T23:26:19.167189+0000","last_clean":"2024-04-24T23:26:19.167189+0000","last_became_active":"2024-04-24T23:26:18.164082+0000","last_became_peered":"2024-04-24T23:26:18.164082+0000","last_unstale":"2024-04-24T23:26:19.167189+0000","last_undegraded":"2024-04-24T23:26:19.167189+0000","last_fullsized":"2024-04-24T23:26:19.167189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:40.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172194+0000","last_change":"2024-04-24T23:26:17.163231+0000","last_active":"2024-04-24T23:26:19.172194+0000","last_peered":"2024-04-24T23:26:19.172194+0000","last_clean":"2024-04-24T23:26:19.172194+0000","last_became_active":"2024-04-24T23:26:17.162790+0000","last_became_peered":"2024-04-24T23:26:17.162790+0000","last_unstale":"2024-04-24T23:26:19.172194+0000","last_undegraded":"2024-04-24T23:26:19.172194+0000","last_fullsized":"2024-04-24T23:26:19.172194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:44.722428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.464151+0000","last_change":"2024-04-24T23:26:18.184007+0000","last_active":"2024-04-24T23:26:19.464151+0000","last_peered":"2024-04-24T23:26:19.464151+0000","last_clean":"2024-04-24T23:26:19.464151+0000","last_became_active":"2024-04-24T23:26:18.183723+0000","last_became_peered":"2024-04-24T23:26:18.183723+0000","last_unstale":"2024-04-24T23:26:19.464151+0000","last_undegraded":"2024-04-24T23:26:19.464151+0000","last_fullsized":"2024-04-24T23:26:19.464151+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:57.076073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172239+0000","last_change":"2024-04-24T23:26:17.161814+0000","last_active":"2024-04-24T23:26:19.172239+0000","last_peered":"2024-04-24T23:26:19.172239+0000","last_clean":"2024-04-24T23:26:19.172239+0000","last_became_active":"2024-04-24T23:26:17.161485+0000","last_became_peered":"2024-04-24T23:26:17.161485+0000","last_unstale":"2024-04-24T23:26:19.172239+0000","last_undegraded":"2024-04-24T23:26:19.172239+0000","last_fullsized":"2024-04-24T23:26:19.172239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:45.309167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370143+0000","last_change":"2024-04-24T23:26:18.188305+0000","last_active":"2024-04-24T23:26:19.370143+0000","last_peered":"2024-04-24T23:26:19.370143+0000","last_clean":"2024-04-24T23:26:19.370143+0000","last_became_active":"2024-04-24T23:26:18.187873+0000","last_became_peered":"2024-04-24T23:26:18.187873+0000","last_unstale":"2024-04-24T23:26:19.370143+0000","last_undegraded":"2024-04-24T23:26:19.370143+0000","last_fullsized":"2024-04-24T23:26:19.370143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:07.648864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369824+0000","last_change":"2024-04-24T23:26:17.169621+0000","last_active":"2024-04-24T23:26:19.369824+0000","last_peered":"2024-04-24T23:26:19.369824+0000","last_clean":"2024-04-24T23:26:19.369824+0000","last_became_active":"2024-04-24T23:26:17.169195+0000","last_became_peered":"2024-04-24T23:26:17.169195+0000","last_unstale":"2024-04-24T23:26:19.369824+0000","last_undegraded":"2024-04-24T23:26:19.369824+0000","last_fullsized":"2024-04-24T23:26:19.369824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:45.101937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167560+0000","last_change":"2024-04-24T23:26:18.170255+0000","last_active":"2024-04-24T23:26:19.167560+0000","last_peered":"2024-04-24T23:26:19.167560+0000","last_clean":"2024-04-24T23:26:19.167560+0000","last_became_active":"2024-04-24T23:26:18.169965+0000","last_became_peered":"2024-04-24T23:26:18.169965+0000","last_unstale":"2024-04-24T23:26:19.167560+0000","last_undegraded":"2024-04-24T23:26:19.167560+0000","last_fullsized":"2024-04-24T23:26:19.167560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:19.473276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.370056+0000","last_change":"2024-04-24T23:26:17.169611+0000","last_active":"2024-04-24T23:26:19.370056+0000","last_peered":"2024-04-24T23:26:19.370056+0000","last_clean":"2024-04-24T23:26:19.370056+0000","last_became_active":"2024-04-24T23:26:17.169304+0000","last_became_peered":"2024-04-24T23:26:17.169304+0000","last_unstale":"2024-04-24T23:26:19.370056+0000","last_undegraded":"2024-04-24T23:26:19.370056+0000","last_fullsized":"2024-04-24T23:26:19.370056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:05.220857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.167666+0000","last_change":"2024-04-24T23:26:18.165478+0000","last_active":"2024-04-24T23:26:19.167666+0000","last_peered":"2024-04-24T23:26:19.167666+0000","last_clean":"2024-04-24T23:26:19.167666+0000","last_became_active":"2024-04-24T23:26:18.165191+0000","last_became_peered":"2024-04-24T23:26:18.165191+0000","last_unstale":"2024-04-24T23:26:19.167666+0000","last_undegraded":"2024-04-24T23:26:19.167666+0000","last_fullsized":"2024-04-24T23:26:19.167666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:07.648864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174110+0000","last_change":"2024-04-24T23:26:17.168469+0000","last_active":"2024-04-24T23:26:19.174110+0000","last_peered":"2024-04-24T23:26:19.174110+0000","last_clean":"2024-04-24T23:26:19.174110+0000","last_became_active":"2024-04-24T23:26:17.167305+0000","last_became_peered":"2024-04-24T23:26:17.167305+0000","last_unstale":"2024-04-24T23:26:19.174110+0000","last_undegraded":"2024-04-24T23:26:19.174110+0000","last_fullsized":"2024-04-24T23:26:19.174110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:37.599403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.174009+0000","last_change":"2024-04-24T23:26:18.176710+0000","last_active":"2024-04-24T23:26:19.174009+0000","last_peered":"2024-04-24T23:26:19.174009+0000","last_clean":"2024-04-24T23:26:19.174009+0000","last_became_active":"2024-04-24T23:26:18.176458+0000","last_became_peered":"2024-04-24T23:26:18.176458+0000","last_unstale":"2024-04-24T23:26:19.174009+0000","last_undegraded":"2024-04-24T23:26:19.174009+0000","last_fullsized":"2024-04-24T23:26:19.174009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:29.129749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166598+0000","last_change":"2024-04-24T23:26:17.162968+0000","last_active":"2024-04-24T23:26:19.166598+0000","last_peered":"2024-04-24T23:26:19.166598+0000","last_clean":"2024-04-24T23:26:19.166598+0000","last_became_active":"2024-04-24T23:26:17.162691+0000","last_became_peered":"2024-04-24T23:26:17.162691+0000","last_unstale":"2024-04-24T23:26:19.166598+0000","last_undegraded":"2024-04-24T23:26:19.166598+0000","last_fullsized":"2024-04-24T23:26:19.166598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:25.593578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463672+0000","last_change":"2024-04-24T23:26:18.176811+0000","last_active":"2024-04-24T23:26:19.463672+0000","last_peered":"2024-04-24T23:26:19.463672+0000","last_clean":"2024-04-24T23:26:19.463672+0000","last_became_active":"2024-04-24T23:26:18.176290+0000","last_became_peered":"2024-04-24T23:26:18.176290+0000","last_unstale":"2024-04-24T23:26:19.463672+0000","last_undegraded":"2024-04-24T23:26:19.463672+0000","last_fullsized":"2024-04-24T23:26:19.463672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:48.988665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369839+0000","last_change":"2024-04-24T23:26:17.168302+0000","last_active":"2024-04-24T23:26:19.369839+0000","last_peered":"2024-04-24T23:26:19.369839+0000","last_clean":"2024-04-24T23:26:19.369839+0000","last_became_active":"2024-04-24T23:26:17.167711+0000","last_became_peered":"2024-04-24T23:26:17.167711+0000","last_unstale":"2024-04-24T23:26:19.369839+0000","last_undegraded":"2024-04-24T23:26:19.369839+0000","last_fullsized":"2024-04-24T23:26:19.369839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:57.160342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.369737+0000","last_change":"2024-04-24T23:26:18.187349+0000","last_active":"2024-04-24T23:26:19.369737+0000","last_peered":"2024-04-24T23:26:19.369737+0000","last_clean":"2024-04-24T23:26:19.369737+0000","last_became_active":"2024-04-24T23:26:18.187041+0000","last_became_peered":"2024-04-24T23:26:18.187041+0000","last_unstale":"2024-04-24T23:26:19.369737+0000","last_undegraded":"2024-04-24T23:26:19.369737+0000","last_fullsized":"2024-04-24T23:26:19.369737+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:52.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.166354+0000","last_change":"2024-04-24T23:26:17.156009+0000","last_active":"2024-04-24T23:26:19.166354+0000","last_peered":"2024-04-24T23:26:19.166354+0000","last_clean":"2024-04-24T23:26:19.166354+0000","last_became_active":"2024-04-24T23:26:17.155739+0000","last_became_peered":"2024-04-24T23:26:17.155739+0000","last_unstale":"2024-04-24T23:26:19.166354+0000","last_undegraded":"2024-04-24T23:26:19.166354+0000","last_fullsized":"2024-04-24T23:26:19.166354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:07.935788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.172160+0000","last_change":"2024-04-24T23:26:18.183679+0000","last_active":"2024-04-24T23:26:19.172160+0000","last_peered":"2024-04-24T23:26:19.172160+0000","last_clean":"2024-04-24T23:26:19.172160+0000","last_became_active":"2024-04-24T23:26:18.183138+0000","last_became_peered":"2024-04-24T23:26:18.183138+0000","last_unstale":"2024-04-24T23:26:19.172160+0000","last_undegraded":"2024-04-24T23:26:19.172160+0000","last_fullsized":"2024-04-24T23:26:19.172160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:17.143261+0000","last_clean_scrub_stamp":"2024-04-24T23:26:17.143261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:02.259501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463966+0000","last_change":"2024-04-24T23:26:17.166166+0000","last_active":"2024-04-24T23:26:19.463966+0000","last_peered":"2024-04-24T23:26:19.463966+0000","last_clean":"2024-04-24T23:26:19.463966+0000","last_became_active":"2024-04-24T23:26:17.165855+0000","last_became_peered":"2024-04-24T23:26:17.165855+0000","last_unstale":"2024-04-24T23:26:19.463966+0000","last_undegraded":"2024-04-24T23:26:19.463966+0000","last_fullsized":"2024-04-24T23:26:19.463966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:26:16.138867+0000","last_clean_scrub_stamp":"2024-04-24T23:26:16.138867+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.467361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":282,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T23:26:19.463814+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:26:19.463814+0000","last_peered":"2024-04-24T23:26:19.463814+0000","last_clean":"2024-04-24T23:26:19.463814+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:26:19.463814+0000","last_undegraded":"2024-04-24T23:26:19.463814+0000","last_fullsized":"2024-04-24T23:26:19.463814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1605632,"data_stored":1023969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":434225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":435165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":433531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":435143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1011712,"data_stored":434113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":433725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1605632,"data_stored":1023933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:26:26.941 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:26:26.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:26:27.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.086+0000 7f2fd61b7640 1 Processor -- start 2024-04-24T23:26:27.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.086+0000 7f2fd61b7640 1 -- start start 2024-04-24T23:26:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fd61b7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fd0071e30 0x7f2fd006e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fd61b7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd0161fe0 con 0x7f2fd0158ae0 2024-04-24T23:26:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fd61b7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd006b830 con 0x7f2fd00d0ce0 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fd61b7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd00d02c0 con 0x7f2fd0071e30 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fcf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2fd0158ae0 0x7f2fd006b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58358/0 (socket says 172.21.15.26:58358) 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fcf7fe640 1 -- 172.21.15.26:0/2910379487 learned_addr learned my addr 172.21.15.26:0/2910379487 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153599227 0 0) 0x7f2fd0161fe0 con 0x7f2fd0158ae0 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fb8003520 con 0x7f2fd0158ae0 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fcffff640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fd0071e30 0x7f2fd006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3395677835 0 0) 0x7f2fb8003520 con 0x7f2fd0158ae0 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fd0071e30 msgr2=0x7f2fd006e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fce7fc640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fd0071e30 0x7f2fd006e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 >> v1:172.21.15.29:6789/0 conn(0x7f2fd00d0ce0 legacy=0x7f2fd0071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd006ec90 con 0x7f2fd0158ae0 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.087+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f2fc0001670 con 0x7f2fd0158ae0 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 >> v1:172.21.15.26:6789/0 conn(0x7f2fd0158ae0 legacy=0x7f2fd006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fce7fc640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb80039e0 0x7f2fb8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fce7fc640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fd0071e30 0x7f2fb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fce7fc640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fb8004d40 0x7f2fb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb800ba70 con 0x7f2fb80039e0 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb800bc20 con 0x7f2fb8004d40 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb800be20 con 0x7f2fd0071e30 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fcf7fe640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb80039e0 0x7f2fb8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fceffd640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fd0071e30 0x7f2fb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fcffff640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fb8004d40 0x7f2fb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fcf7fe640 1 -- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fd0071e30 msgr2=0x7f2fb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fcf7fe640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fd0071e30 0x7f2fb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fcf7fe640 1 -- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fb8004d40 msgr2=0x7f2fb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fcf7fe640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fb8004d40 0x7f2fb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fcf7fe640 1 -- 172.21.15.26:0/2910379487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb8003520 con 0x7f2fb80039e0 2024-04-24T23:26:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.088+0000 7f2fcffff640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fb8004d40 0x7f2fb80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fcf7fe640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb80039e0 0x7f2fb8003dc0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000c290 tx=0x7f2fb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74faaec61818926 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0007d50 con 0x7f2fb80039e0 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc0005af0 con 0x7f2fb80039e0 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fce7fc640 1 -- 172.21.15.26:0/2910379487 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0007070 con 0x7f2fb80039e0 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fd61b7640 1 -- 172.21.15.26:0/2910379487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb80039e0 msgr2=0x7f2fb8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fd61b7640 1 --2- 172.21.15.26:0/2910379487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb80039e0 0x7f2fb8003dc0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000c290 tx=0x7f2fb8005830 comp rx=0 tx=0).stop 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fcffff640 1 -- 172.21.15.26:0/2910379487 reap_dead start 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fd61b7640 1 -- 172.21.15.26:0/2910379487 shutdown_connections 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fd61b7640 1 -- 172.21.15.26:0/2910379487 >> 172.21.15.26:0/2910379487 conn(0x7f2fd00b2970 msgr2=0x7f2fd00d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fd61b7640 1 -- 172.21.15.26:0/2910379487 shutdown_connections 2024-04-24T23:26:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.089+0000 7f2fd61b7640 1 -- 172.21.15.26:0/2910379487 wait complete. 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.091+0000 7f2fd61b7640 1 Processor -- start 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.091+0000 7f2fd61b7640 1 -- start start 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.091+0000 7f2fd61b7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fb80039e0 0x7f2fd014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.091+0000 7f2fd61b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fd0152440 0x7f2fd01528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.091+0000 7f2fd61b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd006b120 0x7f2fd006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.091+0000 7f2fd61b7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd006f530 con 0x7f2fd0152440 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.091+0000 7f2fd61b7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd006f250 con 0x7f2fd006b120 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.091+0000 7f2fd61b7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd00aff90 con 0x7f2fb80039e0 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.091+0000 7f2fceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fd0152440 0x7f2fd01528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fd0152440 0x7f2fd01528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60446/0 (socket says 172.21.15.26:60446) 2024-04-24T23:26:27.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fceffd640 1 -- 172.21.15.26:0/3688592112 learned_addr learned my addr 172.21.15.26:0/3688592112 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fcf7fe640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fb80039e0 0x7f2fd014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fcffff640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd006b120 0x7f2fd006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fceffd640 1 -- 172.21.15.26:0/3688592112 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fb80039e0 msgr2=0x7f2fd014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fceffd640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fb80039e0 0x7f2fd014cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fceffd640 1 -- 172.21.15.26:0/3688592112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd006b120 msgr2=0x7f2fd006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fceffd640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd006b120 0x7f2fd006b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fceffd640 1 -- 172.21.15.26:0/3688592112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd0153000 con 0x7f2fd0152440 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fceffd640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fd0152440 0x7f2fd01528a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400e800 tx=0x7f2fc400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fb7fff640 1 -- 172.21.15.26:0/3688592112 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40050d0 con 0x7f2fd0152440 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fb7fff640 1 -- 172.21.15.26:0/3688592112 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc4018070 con 0x7f2fd0152440 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd01532e0 con 0x7f2fd0152440 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fb7fff640 1 -- 172.21.15.26:0/3688592112 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400c6c0 con 0x7f2fd0152440 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.092+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd0154580 con 0x7f2fd0152440 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.093+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd006bae0 con 0x7f2fd0152440 2024-04-24T23:26:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.094+0000 7f2fb7fff640 1 -- 172.21.15.26:0/3688592112 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fc400c860 con 0x7f2fd0152440 2024-04-24T23:26:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.094+0000 7f2fb7fff640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2fbc06be60 0x7f2fbc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.094+0000 7f2fb7fff640 1 -- 172.21.15.26:0/3688592112 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f2fc408ad30 con 0x7f2fd0152440 2024-04-24T23:26:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.094+0000 7f2fcf7fe640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2fbc06be60 0x7f2fbc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.095+0000 7f2fcf7fe640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2fbc06be60 0x7f2fbc06e320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0033000 tx=0x7f2fc002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.098+0000 7f2fb7fff640 1 -- 172.21.15.26:0/3688592112 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2fc4059650 con 0x7f2fd0152440 2024-04-24T23:26:27.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.350+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2fd0153800 con 0x7f2fd0152440 2024-04-24T23:26:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.353+0000 7f2fb7fff640 1 -- 172.21.15.26:0/3688592112 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f2fd0153800 con 0x7f2fd0152440 2024-04-24T23:26:27.354 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-24T23:26:27.354 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ== 2024-04-24T23:26:27.354 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-04-24T23:26:27.354 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-04-24T23:26:27.354 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:26:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-04-24T23:26:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2fbc06be60 msgr2=0x7f2fbc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2fbc06be60 0x7f2fbc06e320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0033000 tx=0x7f2fc002e000 comp rx=0 tx=0).stop 2024-04-24T23:26:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fd0152440 msgr2=0x7f2fd01528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fd0152440 0x7f2fd01528a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400e800 tx=0x7f2fc400ecc0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 shutdown_connections 2024-04-24T23:26:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2fbc06be60 0x7f2fbc06e320 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2fd006b120 0x7f2fd006b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fd0152440 0x7f2fd01528a0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 --2- 172.21.15.26:0/3688592112 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2fb80039e0 0x7f2fd014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 >> 172.21.15.26:0/3688592112 conn(0x7f2fd00b2970 msgr2=0x7f2fd00b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 shutdown_connections 2024-04-24T23:26:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.355+0000 7f2fd61b7640 1 -- 172.21.15.26:0/3688592112 wait complete. 2024-04-24T23:26:27.367 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.467+0000 7f8ee8ad2640 1 Processor -- start 2024-04-24T23:26:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.467+0000 7f8ee8ad2640 1 -- start start 2024-04-24T23:26:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee8ad2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ee4145910 0x7f8ee406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee8ad2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8ee406d350 0x7f8ee415d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee8ad2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee415d570 con 0x7f8ee40cd660 2024-04-24T23:26:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee8ad2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8ee4145cf0 con 0x7f8ee4145910 2024-04-24T23:26:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee8ad2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8ee40cfbc0 con 0x7f8ee406d350 2024-04-24T23:26:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ee4145910 0x7f8ee406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8ee40cd660 0x7f8ee415d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58372/0 (socket says 172.21.15.26:58372) 2024-04-24T23:26:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee2d9d640 1 -- 172.21.15.26:0/2713800560 learned_addr learned my addr 172.21.15.26:0/2713800560 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8edbfff640 1 --2- 172.21.15.26:0/2713800560 >> v2:172.21.15.29:3300/0 conn(0x7f8ee406d350 0x7f8ee415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee259c640 1 -- 172.21.15.26:0/2713800560 >> v2:172.21.15.29:3300/0 conn(0x7f8ee406d350 msgr2=0x7f8ee415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee259c640 1 --2- 172.21.15.26:0/2713800560 >> v2:172.21.15.29:3300/0 conn(0x7f8ee406d350 0x7f8ee415d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee259c640 1 -- 172.21.15.26:0/2713800560 >> v1:172.21.15.26:6789/0 conn(0x7f8ee40cd660 legacy=0x7f8ee415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee259c640 1 -- 172.21.15.26:0/2713800560 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee406d7f0 con 0x7f8ee4145910 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8ee1d9b640 1 -- 172.21.15.26:0/2713800560 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4097351055 0 0) 0x7f8edc001490 con 0x7f8ee40cd660 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.468+0000 7f8edbfff640 1 --2- 172.21.15.26:0/2713800560 >> v2:172.21.15.29:3300/0 conn(0x7f8ee406d350 0x7f8ee415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.469+0000 7f8ee259c640 1 --2- 172.21.15.26:0/2713800560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee4145910 0x7f8ee406e890 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40097a0 tx=0x7f8ed402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed7080db34b5596 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.469+0000 7f8ee1d9b640 1 -- 172.21.15.26:0/2713800560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4037070 con 0x7f8ee4145910 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.469+0000 7f8ee1d9b640 1 -- 172.21.15.26:0/2713800560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed402ae50 con 0x7f8ee4145910 2024-04-24T23:26:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.469+0000 7f8ee1d9b640 1 -- 172.21.15.26:0/2713800560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4032660 con 0x7f8ee4145910 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.470+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/2713800560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee4145910 msgr2=0x7f8ee406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.470+0000 7f8ee8ad2640 1 --2- 172.21.15.26:0/2713800560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee4145910 0x7f8ee406e890 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40097a0 tx=0x7f8ed402a140 comp rx=0 tx=0).stop 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.470+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/2713800560 shutdown_connections 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.470+0000 7f8ee8ad2640 1 --2- 172.21.15.26:0/2713800560 >> v2:172.21.15.29:3300/0 conn(0x7f8ee406d350 0x7f8ee415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.470+0000 7f8ee8ad2640 1 --2- 172.21.15.26:0/2713800560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee4145910 0x7f8ee406e890 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.470+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/2713800560 >> 172.21.15.26:0/2713800560 conn(0x7f8ee406b300 msgr2=0x7f8ee406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.470+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/2713800560 shutdown_connections 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.470+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/2713800560 wait complete. 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.471+0000 7f8ee8ad2640 1 Processor -- start 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.471+0000 7f8ee8ad2640 1 -- start start 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee8ad2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee406d350 0x7f8ee4140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee8ad2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee40cd660 0x7f8ee4144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee8ad2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ee4145910 0x7f8ee4144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee8ad2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee406e090 con 0x7f8ee406d350 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee8ad2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee406ddb0 con 0x7f8ee40cd660 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee8ad2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee406e370 con 0x7f8ee4145910 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee406d350 0x7f8ee4140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee406d350 0x7f8ee4140670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60464/0 (socket says 172.21.15.26:60464) 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee259c640 1 -- 172.21.15.26:0/381783449 learned_addr learned my addr 172.21.15.26:0/381783449 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee259c640 1 -- 172.21.15.26:0/381783449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ee4145910 msgr2=0x7f8ee4144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee259c640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ee4145910 0x7f8ee4144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee259c640 1 -- 172.21.15.26:0/381783449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee40cd660 msgr2=0x7f8ee4144740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee259c640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee40cd660 0x7f8ee4144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee259c640 1 -- 172.21.15.26:0/381783449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee40ae5f0 con 0x7f8ee406d350 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8ee259c640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee406d350 0x7f8ee4140670 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40097a0 tx=0x7f8ed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8edaffd640 1 -- 172.21.15.26:0/381783449 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4037040 con 0x7f8ee406d350 2024-04-24T23:26:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.472+0000 7f8edaffd640 1 -- 172.21.15.26:0/381783449 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed4004030 con 0x7f8ee406d350 2024-04-24T23:26:27.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.473+0000 7f8edaffd640 1 -- 172.21.15.26:0/381783449 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4004400 con 0x7f8ee406d350 2024-04-24T23:26:27.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.473+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee40ae8d0 con 0x7f8ee406d350 2024-04-24T23:26:27.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.473+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee414bb40 con 0x7f8ee406d350 2024-04-24T23:26:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.474+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee40694c0 con 0x7f8ee406d350 2024-04-24T23:26:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.474+0000 7f8edaffd640 1 -- 172.21.15.26:0/381783449 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ed4003610 con 0x7f8ee406d350 2024-04-24T23:26:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.474+0000 7f8edaffd640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8ec406be30 0x7f8ec406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.475+0000 7f8edaffd640 1 -- 172.21.15.26:0/381783449 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8ed40abb20 con 0x7f8ee406d350 2024-04-24T23:26:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.475+0000 7f8edbfff640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8ec406be30 0x7f8ec406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.475+0000 7f8edbfff640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8ec406be30 0x7f8ec406e2f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc006f00 tx=0x7f8ecc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.479+0000 7f8edaffd640 1 -- 172.21.15.26:0/381783449 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ed407a4d0 con 0x7f8ee406d350 2024-04-24T23:26:27.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.703+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ee415d200 con 0x7f8ee406d350 2024-04-24T23:26:27.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.704+0000 7f8edaffd640 1 -- 172.21.15.26:0/381783449 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5180 (secure 0 0 0) 0x7f8ed4079dd0 con 0x7f8ee406d350 2024-04-24T23:26:27.705 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:27.705 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":84,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:20.530093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6790},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790]},"id":6}]} 2024-04-24T23:26:27.705 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 84 2024-04-24T23:26:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8ec406be30 msgr2=0x7f8ec406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8ec406be30 0x7f8ec406e2f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc006f00 tx=0x7f8ecc008040 comp rx=0 tx=0).stop 2024-04-24T23:26:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee406d350 msgr2=0x7f8ee4140670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee406d350 0x7f8ee4140670 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40097a0 tx=0x7f8ed40046c0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 shutdown_connections 2024-04-24T23:26:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8ec406be30 0x7f8ec406e2f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8ee4145910 0x7f8ee4144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee40cd660 0x7f8ee4144740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 --2- 172.21.15.26:0/381783449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee406d350 0x7f8ee4140670 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 >> 172.21.15.26:0/381783449 conn(0x7f8ee406b300 msgr2=0x7f8ee40cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 shutdown_connections 2024-04-24T23:26:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:27.706+0000 7f8ee8ad2640 1 -- 172.21.15.26:0/381783449 wait complete. 2024-04-24T23:26:28.120 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:26:17.523402+0000', 'modified': '2024-04-24T23:26:20.530093+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6790}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6790': {'gid': 6790, 'name': 'b', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6834/2253149800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 2253149800}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 2253149800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6790, 'qdb_cluster': [6790]} 2024-04-24T23:26:28.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:26:28.121 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:26:28.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.226+0000 7f07fa42c640 1 Processor -- start 2024-04-24T23:26:28.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.226+0000 7f07fa42c640 1 -- start start 2024-04-24T23:26:28.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07fa42c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415eb00 0x7f07f415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07fa42c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f40cd0b0 con 0x7f07f40cf870 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07fa42c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f4071240 con 0x7f07f4143c40 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07fa42c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f07f4071b70 con 0x7f07f415eb00 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f07f40cf870 0x7f07f4072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58380/0 (socket says 172.21.15.26:58380) 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f37fe640 1 -- 172.21.15.26:0/226787178 learned_addr learned my addr 172.21.15.26:0/226787178 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f89c9640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415eb00 0x7f07f415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1726407323 0 0) 0x7f07f40cd0b0 con 0x7f07f40cf870 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07e0003520 con 0x7f07f40cf870 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3374995477 0 0) 0x7f07f4071240 con 0x7f07f4143c40 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07f40cd0b0 con 0x7f07f4143c40 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2921532642 0 0) 0x7f07e0003520 con 0x7f07f40cf870 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415eb00 msgr2=0x7f07f415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f2ffd640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415eb00 0x7f07f415ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 >> v1:172.21.15.29:6789/0 conn(0x7f07f4143c40 legacy=0x7f07f40cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.227+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f4071520 con 0x7f07f40cf870 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f07dc001670 con 0x7f07f40cf870 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 >> v1:172.21.15.26:6789/0 conn(0x7f07f40cf870 legacy=0x7f07f4072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f2ffd640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07e0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f2ffd640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07e0004800 0x7f07e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f2ffd640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415eb00 0x7f07e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07f4071b70 con 0x7f07e0003b70 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f07e000dc80 con 0x7f07e0004800 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f07e000de80 con 0x7f07f415eb00 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f3fff640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07e0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f37fe640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07e0004800 0x7f07e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f89c9640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415eb00 0x7f07e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f3fff640 1 -- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415eb00 msgr2=0x7f07e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f3fff640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415eb00 0x7f07e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f3fff640 1 -- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07e0004800 msgr2=0x7f07e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f3fff640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07e0004800 0x7f07e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.228+0000 7f07f3fff640 1 -- 172.21.15.26:0/226787178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e0003520 con 0x7f07e0003b70 2024-04-24T23:26:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.229+0000 7f07f3fff640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07e0003f70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f07e40018d0 tx=0x7f07e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d50813a8b6f937 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.229+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e40014f0 con 0x7f07e0003b70 2024-04-24T23:26:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.229+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e40052d0 con 0x7f07e0003b70 2024-04-24T23:26:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.229+0000 7f07f2ffd640 1 -- 172.21.15.26:0/226787178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e402aca0 con 0x7f07e0003b70 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.229+0000 7f07fa42c640 1 -- 172.21.15.26:0/226787178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 msgr2=0x7f07e0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.229+0000 7f07fa42c640 1 --2- 172.21.15.26:0/226787178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07e0003f70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f07e40018d0 tx=0x7f07e0005830 comp rx=0 tx=0).stop 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.229+0000 7f07f89c9640 1 -- 172.21.15.26:0/226787178 reap_dead start 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.230+0000 7f07fa42c640 1 -- 172.21.15.26:0/226787178 shutdown_connections 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.230+0000 7f07fa42c640 1 -- 172.21.15.26:0/226787178 >> 172.21.15.26:0/226787178 conn(0x7f07f406b300 msgr2=0x7f07f406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.230+0000 7f07fa42c640 1 -- 172.21.15.26:0/226787178 shutdown_connections 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.230+0000 7f07fa42c640 1 -- 172.21.15.26:0/226787178 wait complete. 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.231+0000 7f07fa42c640 1 Processor -- start 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.231+0000 7f07fa42c640 1 -- start start 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07fa42c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07f40a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07fa42c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415d6b0 0x7f07f41544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07fa42c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e540 0x7f07f41f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07fa42c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07f415ef30 con 0x7f07e0003b70 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07fa42c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f07f4144020 con 0x7f07f415e540 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07fa42c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f07f415f0b0 con 0x7f07f415d6b0 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07f40a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07f40a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60486/0 (socket says 172.21.15.26:60486) 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f3fff640 1 -- 172.21.15.26:0/2509893899 learned_addr learned my addr 172.21.15.26:0/2509893899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f37fe640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415d6b0 0x7f07f41544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f89c9640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e540 0x7f07f41f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f3fff640 1 -- 172.21.15.26:0/2509893899 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415d6b0 msgr2=0x7f07f41544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f3fff640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415d6b0 0x7f07f41544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f3fff640 1 -- 172.21.15.26:0/2509893899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e540 msgr2=0x7f07f41f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f3fff640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e540 0x7f07f41f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f3fff640 1 -- 172.21.15.26:0/2509893899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f415c610 con 0x7f07e0003b70 2024-04-24T23:26:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f3fff640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07f40a9460 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f07e4000c00 tx=0x7f07e40012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f0ff9640 1 -- 172.21.15.26:0/2509893899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4037040 con 0x7f07e0003b70 2024-04-24T23:26:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f0ff9640 1 -- 172.21.15.26:0/2509893899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e4006e50 con 0x7f07e0003b70 2024-04-24T23:26:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.232+0000 7f07f0ff9640 1 -- 172.21.15.26:0/2509893899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e40014f0 con 0x7f07e0003b70 2024-04-24T23:26:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.233+0000 7f07fa42c640 1 -- 172.21.15.26:0/2509893899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f415c8f0 con 0x7f07e0003b70 2024-04-24T23:26:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.233+0000 7f07fa42c640 1 -- 172.21.15.26:0/2509893899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f415ceb0 con 0x7f07e0003b70 2024-04-24T23:26:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.234+0000 7f07f0ff9640 1 -- 172.21.15.26:0/2509893899 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07e4044020 con 0x7f07e0003b70 2024-04-24T23:26:28.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.234+0000 7f07f0ff9640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d406be30 0x7f07d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.235+0000 7f07f0ff9640 1 -- 172.21.15.26:0/2509893899 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f07e40aaa20 con 0x7f07e0003b70 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.235+0000 7f07f37fe640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d406be30 0x7f07d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.235+0000 7f07f37fe640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d406be30 0x7f07d406e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f07dc001050 tx=0x7f07dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.235+0000 7f07fd210740 1 Processor -- start 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.235+0000 7f07fd210740 1 -- start start 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07fd210740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bd3b60 0x563d87b864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07fd210740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 0x563d87bb6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07fd210740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bb6540 0x563d87b8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07fd210740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563d87aeef00 con 0x563d87bb6540 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07fd210740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563d87a97750 con 0x563d87bb6c80 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07fd210740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563d87aeaec0 con 0x563d87bd3b60 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f89c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bb6540 0x563d87b8b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f89c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bb6540 0x563d87b8b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60502/0 (socket says 172.21.15.26:60502) 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f89c9640 1 -- 172.21.15.26:0/3624217162 learned_addr learned my addr 172.21.15.26:0/3624217162 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f3fff640 1 --2- 172.21.15.26:0/3624217162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bd3b60 0x563d87b864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f37fe640 1 --2- 172.21.15.26:0/3624217162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 0x563d87bb6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f89c9640 1 -- 172.21.15.26:0/3624217162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bd3b60 msgr2=0x563d87b864f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f89c9640 1 --2- 172.21.15.26:0/3624217162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bd3b60 0x563d87b864f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f89c9640 1 -- 172.21.15.26:0/3624217162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 msgr2=0x563d87bb6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f89c9640 1 --2- 172.21.15.26:0/3624217162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 0x563d87bb6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.236+0000 7f07f89c9640 1 -- 172.21.15.26:0/3624217162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d878032d0 con 0x563d87bb6540 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07f89c9640 1 --2- 172.21.15.26:0/3624217162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bb6540 0x563d87b8b350 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009f70 tx=0x7f07ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a612197346abe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07e9ffb640 1 -- 172.21.15.26:0/3624217162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec007ea0 con 0x563d87bb6540 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07e9ffb640 1 -- 172.21.15.26:0/3624217162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec01c070 con 0x563d87bb6540 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07e9ffb640 1 -- 172.21.15.26:0/3624217162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec004030 con 0x563d87bb6540 2024-04-24T23:26:28.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07fd210740 1 -- 172.21.15.26:0/3624217162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bb6540 msgr2=0x563d87b8b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07fd210740 1 --2- 172.21.15.26:0/3624217162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bb6540 0x563d87b8b350 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009f70 tx=0x7f07ec009750 comp rx=0 tx=0).stop 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07fd210740 1 -- 172.21.15.26:0/3624217162 shutdown_connections 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07fd210740 1 --2- 172.21.15.26:0/3624217162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bb6540 0x563d87b8b350 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07fd210740 1 --2- 172.21.15.26:0/3624217162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 0x563d87bb6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07fd210740 1 --2- 172.21.15.26:0/3624217162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bd3b60 0x563d87b864f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07fd210740 1 -- 172.21.15.26:0/3624217162 >> 172.21.15.26:0/3624217162 conn(0x563d87b88760 msgr2=0x563d87bd1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07fd210740 1 -- 172.21.15.26:0/3624217162 shutdown_connections 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.237+0000 7f07fd210740 1 -- 172.21.15.26:0/3624217162 wait complete. 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.238+0000 7f07fd210740 1 -- start start 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.238+0000 7f07fd210740 1 Processor -- start 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07fd210740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bb6c80 0x563d87bd4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07fd210740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bd3b60 0x563d87bc7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07fd210740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87ba3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07fd210740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563d87b8a910 con 0x563d87bc9d00 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07fd210740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563d87aaacf0 con 0x563d87bd3b60 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07fd210740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563d87b89d00 con 0x563d87bb6c80 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07fd210740 10 client.0 _ll_register_callbacks cb 0x563d87aaa6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f89c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87ba3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f89c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87ba3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60504/0 (socket says 172.21.15.26:60504) 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f89c9640 1 -- 172.21.15.26:0/3955075859 learned_addr learned my addr 172.21.15.26:0/3955075859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f3fff640 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bb6c80 0x563d87bd4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f37fe640 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bd3b60 0x563d87bc7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f89c9640 1 -- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bb6c80 msgr2=0x563d87bd4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f89c9640 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bb6c80 0x563d87bd4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f89c9640 1 -- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bd3b60 msgr2=0x563d87bc7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f89c9640 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bd3b60 0x563d87bc7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.239+0000 7f07f89c9640 1 -- 172.21.15.26:0/3955075859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d878cd6b0 con 0x563d87bc9d00 2024-04-24T23:26:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07f89c9640 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87ba3400 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f07ec019040 tx=0x7f07ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c665227419e0ad86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07d2ffd640 1 -- 172.21.15.26:0/3955075859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec007ea0 con 0x563d87bc9d00 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07fd210740 1 -- 172.21.15.26:0/3955075859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07ec01c070 con 0x563d87bc9d00 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07d2ffd640 1 -- 172.21.15.26:0/3955075859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f07ec02c680 con 0x563d87bc9d00 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07d2ffd640 1 -- 172.21.15.26:0/3955075859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f07ec034050 con 0x563d87bc9d00 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07fd210740 10 client.8013 fetch_fsmap learned FSMap version 84 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07fd210740 1 -- 172.21.15.26:0/3955075859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x563d87ba4ec0 con 0x563d87bc9d00 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07d2ffd640 1 -- 172.21.15.26:0/3955075859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec01c070 con 0x563d87bc9d00 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07d2ffd640 1 -- 172.21.15.26:0/3955075859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f07ec037020 con 0x563d87bc9d00 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07fd210740 10 client.8013 fetch_fsmap finished waiting for FSMap version 84 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07fd210740 10 client.8013 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07fd210740 20 client.8013 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07fd210740 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x563d87ba3ac0 0x563d87bb77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07fd210740 4 client.8013 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.240+0000 7f07fd210740 1 -- 172.21.15.26:0/3955075859 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d87afc440 con 0x563d87ba3ac0 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.241+0000 7f07f3fff640 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x563d87ba3ac0 0x563d87bb77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.241+0000 7f07d2ffd640 10 client.8013 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.241+0000 7f07f3fff640 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x563d87ba3ac0 0x563d87bb77f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1f8efe534f11b5 server_cookie=663ce05a34689921 in_seq=0 out_seq=0 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07d2ffd640 1 -- 172.21.15.26:0/3955075859 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d87afc440 con 0x563d87ba3ac0 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07d2ffd640 10 client.8013 handle_command_reply: tid=0 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 client.8013 shutdown 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 -- 172.21.15.26:0/3955075859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 msgr2=0x563d87ba3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87ba3400 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f07ec019040 tx=0x7f07ec004a40 comp rx=0 tx=0).stop 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 -- 172.21.15.26:0/3955075859 shutdown_connections 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x563d87ba3ac0 0x563d87bb77f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87ba3400 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07d2ffd640 0 client.8013 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bd3b60 0x563d87bc7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 --2- 172.21.15.26:0/3955075859 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bb6c80 0x563d87bd4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 -- 172.21.15.26:0/3955075859 >> 172.21.15.26:0/3955075859 conn(0x563d87bb6540 msgr2=0x563d87b87230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 -- 172.21.15.26:0/3955075859 shutdown_connections 2024-04-24T23:26:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 1 -- 172.21.15.26:0/3955075859 wait complete. 2024-04-24T23:26:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.242+0000 7f07fd210740 20 client.8013 trim_cache size 0 max 16384 2024-04-24T23:26:28.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.260+0000 7f07fd210740 1 Processor -- start 2024-04-24T23:26:28.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.260+0000 7f07fd210740 1 -- start start 2024-04-24T23:26:28.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07fd210740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87ba3ac0 0x563d87bb59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07fd210740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bb6c80 0x563d87bb15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07fd210740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bc9d00 0x563d87bb1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07fd210740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07f4200000 con 0x563d87ba3ac0 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07fd210740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f07f413a590 con 0x563d87bc9d00 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07fd210740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f07f413abe0 con 0x563d87bb6c80 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87ba3ac0 0x563d87bb59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87ba3ac0 0x563d87bb59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60520/0 (socket says 172.21.15.26:60520) 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07f3fff640 1 -- 172.21.15.26:0/3603969217 learned_addr learned my addr 172.21.15.26:0/3603969217 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07f3fff640 1 -- 172.21.15.26:0/3603969217 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bb6c80 msgr2=0x563d87bb15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07f3fff640 1 --2- 172.21.15.26:0/3603969217 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bb6c80 0x563d87bb15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07f3fff640 1 -- 172.21.15.26:0/3603969217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bc9d00 msgr2=0x563d87bb1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07f3fff640 1 --2- 172.21.15.26:0/3603969217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bc9d00 0x563d87bb1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.261+0000 7f07f3fff640 1 -- 172.21.15.26:0/3603969217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d87a75920 con 0x563d87ba3ac0 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07f3fff640 1 --2- 172.21.15.26:0/3603969217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87ba3ac0 0x563d87bb59c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f07e4078d80 tx=0x7f07e4048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834d694d1145297c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07e97fa640 1 -- 172.21.15.26:0/3603969217 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e40819d0 con 0x563d87ba3ac0 2024-04-24T23:26:28.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07e97fa640 1 -- 172.21.15.26:0/3603969217 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e403ce80 con 0x563d87ba3ac0 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07e97fa640 1 -- 172.21.15.26:0/3603969217 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e40770f0 con 0x563d87ba3ac0 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07fd210740 1 -- 172.21.15.26:0/3603969217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87ba3ac0 msgr2=0x563d87bb59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07fd210740 1 --2- 172.21.15.26:0/3603969217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87ba3ac0 0x563d87bb59c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f07e4078d80 tx=0x7f07e4048040 comp rx=0 tx=0).stop 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07fd210740 1 -- 172.21.15.26:0/3603969217 shutdown_connections 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07fd210740 1 --2- 172.21.15.26:0/3603969217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bc9d00 0x563d87bb1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07fd210740 1 --2- 172.21.15.26:0/3603969217 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87bb6c80 0x563d87bb15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07fd210740 1 --2- 172.21.15.26:0/3603969217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87ba3ac0 0x563d87bb59c0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07fd210740 1 -- 172.21.15.26:0/3603969217 >> 172.21.15.26:0/3603969217 conn(0x563d87bb6540 msgr2=0x563d87b87330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07fd210740 1 -- 172.21.15.26:0/3603969217 shutdown_connections 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.262+0000 7f07fd210740 1 -- 172.21.15.26:0/3603969217 wait complete. 2024-04-24T23:26:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.263+0000 7f07fd210740 1 -- start start 2024-04-24T23:26:28.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.263+0000 7f07fd210740 1 Processor -- start 2024-04-24T23:26:28.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07fd210740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87ba3ac0 0x563d87ba1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07fd210740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 0x563d87c4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07fd210740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87bf3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07fd210740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563d87ba4260 con 0x563d87bc9d00 2024-04-24T23:26:28.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07fd210740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563d87ba3940 con 0x563d87bb6c80 2024-04-24T23:26:28.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07fd210740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563d87ba48b0 con 0x563d87ba3ac0 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07fd210740 10 client.0 _ll_register_callbacks cb 0x7f07ec016840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f89c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87bf3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f89c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87bf3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60534/0 (socket says 172.21.15.26:60534) 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f89c9640 1 -- 172.21.15.26:0/2845421762 learned_addr learned my addr 172.21.15.26:0/2845421762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f37fe640 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 0x563d87c4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f3fff640 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87ba3ac0 0x563d87ba1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f89c9640 1 -- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87ba3ac0 msgr2=0x563d87ba1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f89c9640 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87ba3ac0 0x563d87ba1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f89c9640 1 -- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 msgr2=0x563d87c4ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f89c9640 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 0x563d87c4ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f89c9640 1 -- 172.21.15.26:0/2845421762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d87b866a0 con 0x563d87bc9d00 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f37fe640 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 0x563d87c4ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.264+0000 7f07f89c9640 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87bf3a70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f07ec016810 tx=0x7f07ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a4b7e3c1ba3e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07d2ffd640 1 -- 172.21.15.26:0/2845421762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec03d070 con 0x563d87bc9d00 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07d2ffd640 1 -- 172.21.15.26:0/2845421762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f07ec021810 con 0x563d87bc9d00 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07fd210740 1 -- 172.21.15.26:0/2845421762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07e403ce80 con 0x563d87bc9d00 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07d2ffd640 1 -- 172.21.15.26:0/2845421762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f07ec048050 con 0x563d87bc9d00 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07fd210740 10 client.8019 fetch_fsmap learned FSMap version 84 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07fd210740 1 -- 172.21.15.26:0/2845421762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x563d87ba25f0 con 0x563d87bc9d00 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07d2ffd640 1 -- 172.21.15.26:0/2845421762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e403ce80 con 0x563d87bc9d00 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07d2ffd640 1 -- 172.21.15.26:0/2845421762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f07ec04b020 con 0x563d87bc9d00 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07fd210740 10 client.8019 fetch_fsmap finished waiting for FSMap version 84 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07fd210740 10 client.8019 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07fd210740 20 client.8019 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07fd210740 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x563d87bd3b60 0x563d87bb45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07fd210740 4 client.8019 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07fd210740 1 -- 172.21.15.26:0/2845421762 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "status"}) -- 0x563d87af6790 con 0x563d87bd3b60 2024-04-24T23:26:28.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.265+0000 7f07f3fff640 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x563d87bd3b60 0x563d87bb45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07f3fff640 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x563d87bd3b60 0x563d87bb45f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b225433890360ee server_cookie=efb831bf0cb6e63d in_seq=0 out_seq=0 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07d2ffd640 10 client.8019 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07d2ffd640 1 -- 172.21.15.26:0/2845421762 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x563d87af6790 con 0x563d87bd3b60 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07d2ffd640 10 client.8019 handle_command_reply: tid=0 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 client.8019 shutdown 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 -- 172.21.15.26:0/2845421762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 msgr2=0x563d87bf3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87bf3a70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f07ec016810 tx=0x7f07ec004ab0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 -- 172.21.15.26:0/2845421762 shutdown_connections 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x563d87bd3b60 0x563d87bb45f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d87bc9d00 0x563d87bf3a70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563d87bb6c80 0x563d87c4ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07d2ffd640 0 client.8019 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 --2- 172.21.15.26:0/2845421762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563d87ba3ac0 0x563d87ba1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 -- 172.21.15.26:0/2845421762 >> 172.21.15.26:0/2845421762 conn(0x563d87bb6540 msgr2=0x563d87b85d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 -- 172.21.15.26:0/2845421762 shutdown_connections 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 1 -- 172.21.15.26:0/2845421762 wait complete. 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.266+0000 7f07fd210740 20 client.8019 trim_cache size 0 max 16384 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6790, 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 10.102823052, 2024-04-24T23:26:28.273 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 84, 2024-04-24T23:26:28.274 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-04-24T23:26:28.274 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 82, 2024-04-24T23:26:28.274 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.104955835 2024-04-24T23:26:28.274 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 -- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d406be30 msgr2=0x7f07d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d406be30 0x7f07d406e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f07dc001050 tx=0x7f07dc009040 comp rx=0 tx=0).stop 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 -- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 msgr2=0x7f07f40a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07f40a9460 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f07e4000c00 tx=0x7f07e40012d0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 -- 172.21.15.26:0/2509893899 shutdown_connections 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d406be30 0x7f07d406e2f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e540 0x7f07f41f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f415d6b0 0x7f07f41544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 --2- 172.21.15.26:0/2509893899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e0003b70 0x7f07f40a9460 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 -- 172.21.15.26:0/2509893899 >> 172.21.15.26:0/2509893899 conn(0x7f07f406b300 msgr2=0x7f07f406e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 -- 172.21.15.26:0/2509893899 shutdown_connections 2024-04-24T23:26:28.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.267+0000 7f07fa42c640 1 -- 172.21.15.26:0/2509893899 wait complete. 2024-04-24T23:26:28.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:26:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.380+0000 7f9945b89640 1 Processor -- start 2024-04-24T23:26:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.380+0000 7f9945b89640 1 -- start start 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f9945b89640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9940145910 0x7f994006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f9945b89640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f99400cd660 0x7f99400cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f9945b89640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99400cfbc0 con 0x7f994006d350 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f9945b89640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9940145cf0 con 0x7f9940145910 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f9945b89640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f994006d7f0 con 0x7f99400cd660 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9940145910 0x7f994006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f994006d350 0x7f994015d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58394/0 (socket says 172.21.15.26:58394) 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993effd640 1 -- 172.21.15.26:0/1599880630 learned_addr learned my addr 172.21.15.26:0/1599880630 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993ffff640 1 --2- 172.21.15.26:0/1599880630 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f99400cd660 0x7f99400cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993f7fe640 1 -- 172.21.15.26:0/1599880630 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f99400cd660 msgr2=0x7f99400cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993f7fe640 1 --2- 172.21.15.26:0/1599880630 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f99400cd660 0x7f99400cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993f7fe640 1 -- 172.21.15.26:0/1599880630 >> v1:172.21.15.26:6789/0 conn(0x7f994006d350 legacy=0x7f994015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993f7fe640 1 -- 172.21.15.26:0/1599880630 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994006dad0 con 0x7f9940145910 2024-04-24T23:26:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993e7fc640 1 -- 172.21.15.26:0/1599880630 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654225544 0 0) 0x7f99400cfbc0 con 0x7f994006d350 2024-04-24T23:26:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.381+0000 7f993ffff640 1 --2- 172.21.15.26:0/1599880630 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f99400cd660 0x7f99400cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.382+0000 7f993f7fe640 1 --2- 172.21.15.26:0/1599880630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9940145910 0x7f994006e890 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f99340097d0 tx=0x7f993402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7934be4906a020ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.382+0000 7f993e7fc640 1 -- 172.21.15.26:0/1599880630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934037070 con 0x7f9940145910 2024-04-24T23:26:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.382+0000 7f993e7fc640 1 -- 172.21.15.26:0/1599880630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934032440 con 0x7f9940145910 2024-04-24T23:26:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.382+0000 7f993e7fc640 1 -- 172.21.15.26:0/1599880630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934032810 con 0x7f9940145910 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.383+0000 7f9945b89640 1 -- 172.21.15.26:0/1599880630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9940145910 msgr2=0x7f994006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.383+0000 7f9945b89640 1 --2- 172.21.15.26:0/1599880630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9940145910 0x7f994006e890 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f99340097d0 tx=0x7f993402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.383+0000 7f9945b89640 1 -- 172.21.15.26:0/1599880630 shutdown_connections 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.383+0000 7f9945b89640 1 --2- 172.21.15.26:0/1599880630 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f99400cd660 0x7f99400cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.383+0000 7f9945b89640 1 --2- 172.21.15.26:0/1599880630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9940145910 0x7f994006e890 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.383+0000 7f9945b89640 1 -- 172.21.15.26:0/1599880630 >> 172.21.15.26:0/1599880630 conn(0x7f994006b300 msgr2=0x7f994006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.383+0000 7f9945b89640 1 -- 172.21.15.26:0/1599880630 shutdown_connections 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.383+0000 7f9945b89640 1 -- 172.21.15.26:0/1599880630 wait complete. 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.384+0000 7f9945b89640 1 Processor -- start 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.384+0000 7f9945b89640 1 -- start start 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9945b89640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994006d350 0x7f994013f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9945b89640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cd660 0x7f99400aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9945b89640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9940145910 0x7f994013d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9945b89640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f994006e370 con 0x7f99400cd660 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9945b89640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f994006e090 con 0x7f994006d350 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9945b89640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f994015d770 con 0x7f9940145910 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cd660 0x7f99400aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cd660 0x7f99400aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60562/0 (socket says 172.21.15.26:60562) 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993effd640 1 -- 172.21.15.26:0/2944438052 learned_addr learned my addr 172.21.15.26:0/2944438052 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993f7fe640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994006d350 0x7f994013f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993ffff640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9940145910 0x7f994013d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993effd640 1 -- 172.21.15.26:0/2944438052 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9940145910 msgr2=0x7f994013d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993effd640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9940145910 0x7f994013d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993effd640 1 -- 172.21.15.26:0/2944438052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994006d350 msgr2=0x7f994013f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993effd640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994006d350 0x7f994013f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993effd640 1 -- 172.21.15.26:0/2944438052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994013e1f0 con 0x7f99400cd660 2024-04-24T23:26:28.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f993effd640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cd660 0x7f99400aeb30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f993000c4d0 tx=0x7f993000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9927fff640 1 -- 172.21.15.26:0/2944438052 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930018070 con 0x7f99400cd660 2024-04-24T23:26:28.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994013e4d0 con 0x7f99400cd660 2024-04-24T23:26:28.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9927fff640 1 -- 172.21.15.26:0/2944438052 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9930007500 con 0x7f99400cd660 2024-04-24T23:26:28.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.385+0000 7f9927fff640 1 -- 172.21.15.26:0/2944438052 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930004a20 con 0x7f99400cd660 2024-04-24T23:26:28.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.386+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994013ed70 con 0x7f99400cd660 2024-04-24T23:26:28.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.387+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99400694c0 con 0x7f99400cd660 2024-04-24T23:26:28.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.387+0000 7f9927fff640 1 -- 172.21.15.26:0/2944438052 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9930004bc0 con 0x7f99400cd660 2024-04-24T23:26:28.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.387+0000 7f9927fff640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f992c06be30 0x7f992c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:28.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.387+0000 7f9927fff640 1 -- 172.21.15.26:0/2944438052 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f993008a100 con 0x7f99400cd660 2024-04-24T23:26:28.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.387+0000 7f993f7fe640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f992c06be30 0x7f992c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:28.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.388+0000 7f993f7fe640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f992c06be30 0x7f992c06e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9934034000 tx=0x7f993402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:28.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.392+0000 7f9927fff640 1 -- 172.21.15.26:0/2944438052 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f99300583e0 con 0x7f99400cd660 2024-04-24T23:26:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.613+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f994015bf70 con 0x7f99400cd660 2024-04-24T23:26:28.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.614+0000 7f9927fff640 1 -- 172.21.15.26:0/2944438052 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7f993005c6e0 con 0x7f99400cd660 2024-04-24T23:26:28.615 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f992c06be30 msgr2=0x7f992c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f992c06be30 0x7f992c06e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9934034000 tx=0x7f993402e000 comp rx=0 tx=0).stop 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cd660 msgr2=0x7f99400aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cd660 0x7f99400aeb30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f993000c4d0 tx=0x7f993000c990 comp rx=0 tx=0).stop 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 shutdown_connections 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f992c06be30 0x7f992c06e2f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9940145910 0x7f994013d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cd660 0x7f99400aeb30 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 --2- 172.21.15.26:0/2944438052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f994006d350 0x7f994013f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 >> 172.21.15.26:0/2944438052 conn(0x7f994006b300 msgr2=0x7f994015c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 shutdown_connections 2024-04-24T23:26:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28.616+0000 7f9945b89640 1 -- 172.21.15.26:0/2944438052 wait complete. 2024-04-24T23:26:28.630 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:26:28.630 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:26:28.631 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-24T23:26:28.631 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:26:28.631 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:26:28.631 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:26:28.631 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:26:28.631 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:26:28.631 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:26:28.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:28.631 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-24T23:26:28.647 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:26:28.647 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:26:28.647 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:26:28.647 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:26:28.647 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:26:28.647 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:26:28.647 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42404sec preferred_lft 42404sec 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:26:28.648 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ccf2:1eff:fe9b:4749/64 scope link 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c2:ea:e0:db:00:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c0ea:e0ff:fedb:e0/64 scope link 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:26:28.649 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::1c82:d0ff:fefa:5b3f/64 scope link 2024-04-24T23:26:28.650 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:26:28.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:28.650 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:26:28.710 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-24T23:26:28.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:28.712 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-24T23:26:28.768 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-24T23:26:28.769 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:26:28.769 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:26:28.769 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:26:28.769 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:26:28.769 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:26:28.770 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:26:28.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:26:28.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:26:28.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:26:28.770 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:26:28.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:26:28.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:26:28.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:28.895 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:26:28.910 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:26:28.910 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:26:28.910 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:26:28.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:28.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-24T23:26:28.975 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:26:28.975 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:26:29.031 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:26:29.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:26:29.031 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:26:29.085 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:26:29.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:29.085 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:26:29.168 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:26:29.168 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:26:29.168 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:26:29.177 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:26:29.177 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:26:29.177 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:26:29.177 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:26:29.177 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:26:29.185 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:26:29.186 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:26:29.186 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:26:29.186 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:26:29.186 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:26:29.186 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:26:29.186 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:26:29.186 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:26:29.188 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:26:29.188 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:26:29.188 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:26:29.188 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:26:29.188 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:26:29.189 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:26:29.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:29.189 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:26:29.253 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:26:29.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:26:29.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:26:29.357 INFO:teuthology.orchestra.run.smithi026.stdout:client6901 2024-04-24T23:26:29.357 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:26:29.357 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:29.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T23:26:29.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.517+0000 7f75e2fa2640 1 Processor -- start 2024-04-24T23:26:29.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.517+0000 7f75e2fa2640 1 -- start start 2024-04-24T23:26:29.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75e2fa2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f75dc15cc30 0x7f75dc15d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75e2fa2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc0afce0 0x7f75dc071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75e2fa2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75dc0b4780 con 0x7f75dc071c60 2024-04-24T23:26:29.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75e2fa2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f75dc0b5aa0 con 0x7f75dc15cc30 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75e2fa2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f75dc0b4a00 con 0x7f75dc0afce0 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75e153f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f75dc071c60 0x7f75dc06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58410/0 (socket says 172.21.15.26:58410) 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75e153f640 1 -- 172.21.15.26:0/3698656798 learned_addr learned my addr 172.21.15.26:0/3698656798 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75e0d3e640 1 --2- 172.21.15.26:0/3698656798 >> v2:172.21.15.29:3300/0 conn(0x7f75dc15cc30 0x7f75dc15d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75dbfff640 1 --2- 172.21.15.26:0/3698656798 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc0afce0 0x7f75dc071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75db7fe640 1 -- 172.21.15.26:0/3698656798 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2466053345 0 0) 0x7f75c4001110 con 0x7f75dc071c60 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.518+0000 7f75db7fe640 1 -- 172.21.15.26:0/3698656798 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75c8003520 con 0x7f75dc071c60 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.519+0000 7f75dbfff640 1 -- 172.21.15.26:0/3698656798 >> v2:172.21.15.29:3300/0 conn(0x7f75dc15cc30 msgr2=0x7f75dc15d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.519+0000 7f75dbfff640 1 --2- 172.21.15.26:0/3698656798 >> v2:172.21.15.29:3300/0 conn(0x7f75dc15cc30 0x7f75dc15d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.519+0000 7f75dbfff640 1 -- 172.21.15.26:0/3698656798 >> v1:172.21.15.26:6789/0 conn(0x7f75dc071c60 legacy=0x7f75dc06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.519+0000 7f75dbfff640 1 -- 172.21.15.26:0/3698656798 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc0b4ce0 con 0x7f75dc0afce0 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.519+0000 7f75dbfff640 1 --2- 172.21.15.26:0/3698656798 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc0afce0 0x7f75dc071720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009d90 tx=0x7f75cc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23435c0da9c62d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75db7fe640 1 -- 172.21.15.26:0/3698656798 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc007740 con 0x7f75dc0afce0 2024-04-24T23:26:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75db7fe640 1 -- 172.21.15.26:0/3698656798 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75cc0078e0 con 0x7f75dc0afce0 2024-04-24T23:26:29.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75db7fe640 1 -- 172.21.15.26:0/3698656798 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc007c90 con 0x7f75dc0afce0 2024-04-24T23:26:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75e2fa2640 1 -- 172.21.15.26:0/3698656798 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc0afce0 msgr2=0x7f75dc071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75e2fa2640 1 --2- 172.21.15.26:0/3698656798 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc0afce0 0x7f75dc071720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009d90 tx=0x7f75cc02a660 comp rx=0 tx=0).stop 2024-04-24T23:26:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75e2fa2640 1 -- 172.21.15.26:0/3698656798 shutdown_connections 2024-04-24T23:26:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75e2fa2640 1 --2- 172.21.15.26:0/3698656798 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc0afce0 0x7f75dc071720 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75e2fa2640 1 --2- 172.21.15.26:0/3698656798 >> v2:172.21.15.29:3300/0 conn(0x7f75dc15cc30 0x7f75dc15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75e2fa2640 1 -- 172.21.15.26:0/3698656798 >> 172.21.15.26:0/3698656798 conn(0x7f75dc0b2970 msgr2=0x7f75dc0d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75e2fa2640 1 -- 172.21.15.26:0/3698656798 shutdown_connections 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.520+0000 7f75e2fa2640 1 -- 172.21.15.26:0/3698656798 wait complete. 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.521+0000 7f75e2fa2640 1 Processor -- start 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.521+0000 7f75e2fa2640 1 -- start start 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.522+0000 7f75e2fa2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc071c60 0x7f75dc15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.522+0000 7f75e2fa2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc0afce0 0x7f75dc0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.522+0000 7f75e2fa2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75dc15cc30 0x7f75dc15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.522+0000 7f75e2fa2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc0b5580 con 0x7f75dc15cc30 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.522+0000 7f75e2fa2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc0b52a0 con 0x7f75dc0afce0 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.522+0000 7f75e2fa2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f75dc15d550 con 0x7f75dc071c60 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e153f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75dc15cc30 0x7f75dc15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e0d3e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc071c60 0x7f75dc15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e153f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75dc15cc30 0x7f75dc15d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60572/0 (socket says 172.21.15.26:60572) 2024-04-24T23:26:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e153f640 1 -- 172.21.15.26:0/493881266 learned_addr learned my addr 172.21.15.26:0/493881266 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75dbfff640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc0afce0 0x7f75dc0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e153f640 1 -- 172.21.15.26:0/493881266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc071c60 msgr2=0x7f75dc15f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e153f640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc071c60 0x7f75dc15f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e153f640 1 -- 172.21.15.26:0/493881266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc0afce0 msgr2=0x7f75dc0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e153f640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc0afce0 0x7f75dc0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e153f640 1 -- 172.21.15.26:0/493881266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc15f7f0 con 0x7f75dc15cc30 2024-04-24T23:26:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e0d3e640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc071c60 0x7f75dc15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e153f640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75dc15cc30 0x7f75dc15d890 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c3a0 tx=0x7f75c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75d97fa640 1 -- 172.21.15.26:0/493881266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4017070 con 0x7f75dc15cc30 2024-04-24T23:26:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75d97fa640 1 -- 172.21.15.26:0/493881266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c4005ce0 con 0x7f75dc15cc30 2024-04-24T23:26:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc151bc0 con 0x7f75dc15cc30 2024-04-24T23:26:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75d97fa640 1 -- 172.21.15.26:0/493881266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c40072f0 con 0x7f75dc15cc30 2024-04-24T23:26:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.523+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc152180 con 0x7f75dc15cc30 2024-04-24T23:26:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.524+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc075610 con 0x7f75dc15cc30 2024-04-24T23:26:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.525+0000 7f75d97fa640 1 -- 172.21.15.26:0/493881266 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75c401a070 con 0x7f75dc15cc30 2024-04-24T23:26:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.525+0000 7f75d97fa640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75bc06be60 0x7f75bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.525+0000 7f75d97fa640 1 -- 172.21.15.26:0/493881266 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f75c408a170 con 0x7f75dc15cc30 2024-04-24T23:26:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.525+0000 7f75e0d3e640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75bc06be60 0x7f75bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.526+0000 7f75e0d3e640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75bc06be60 0x7f75bc06e320 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009f30 tx=0x7f75d4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.530+0000 7f75d97fa640 1 -- 172.21.15.26:0/493881266 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75dc0b4780 con 0x7f75dc15cc30 2024-04-24T23:26:29.687 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:26:29.690 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:26:29.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:29.752+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f75dc159b30 con 0x7f75dc15cc30 2024-04-24T23:26:30.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.451+0000 7f75d97fa640 1 -- 172.21.15.26:0/493881266 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v85) ==== 105+0+0 (secure 0 0 0) 0x7f75c40584d0 con 0x7f75dc15cc30 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75bc06be60 msgr2=0x7f75bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75bc06be60 0x7f75bc06e320 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009f30 tx=0x7f75d4009190 comp rx=0 tx=0).stop 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75dc15cc30 msgr2=0x7f75dc15d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75dc15cc30 0x7f75dc15d890 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c3a0 tx=0x7f75c400c860 comp rx=0 tx=0).stop 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 shutdown_connections 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f75bc06be60 0x7f75bc06e320 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75dc15cc30 0x7f75dc15d890 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75dc0afce0 0x7f75dc0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 --2- 172.21.15.26:0/493881266 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f75dc071c60 0x7f75dc15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 >> 172.21.15.26:0/493881266 conn(0x7f75dc0b2970 msgr2=0x7f75dc15a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:30.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 shutdown_connections 2024-04-24T23:26:30.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.455+0000 7f75e2fa2640 1 -- 172.21.15.26:0/493881266 wait complete. 2024-04-24T23:26:30.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:30.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.624+0000 7fbbd45ce640 1 Processor -- start 2024-04-24T23:26:30.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.624+0000 7fbbd45ce640 1 -- start start 2024-04-24T23:26:30.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd45ce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbbcc06f110 0x7fbbcc15f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd45ce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbcc15f750 con 0x7fbbcc06f4f0 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd45ce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbcc0b0220 con 0x7fbbcc0adf80 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd45ce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbbcc0af180 con 0x7fbbcc06f110 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1b69640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbbcc06f4f0 0x7fbbcc0ad870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58418/0 (socket says 172.21.15.26:58418) 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1b69640 1 -- 172.21.15.26:0/2720719698 learned_addr learned my addr 172.21.15.26:0/2720719698 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186079421 0 0) 0x7fbbbc001110 con 0x7fbbcc06f4f0 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbb8003520 con 0x7fbbcc06f4f0 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd236a640 1 --2- 172.21.15.26:0/2720719698 >> v2:172.21.15.29:3300/0 conn(0x7fbbcc06f110 0x7fbbcc15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1322440561 0 0) 0x7fbbb8003520 con 0x7fbbcc06f4f0 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 >> v2:172.21.15.29:3300/0 conn(0x7fbbcc06f110 msgr2=0x7fbbcc15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 --2- 172.21.15.26:0/2720719698 >> v2:172.21.15.29:3300/0 conn(0x7fbbcc06f110 0x7fbbcc15f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 >> v1:172.21.15.29:6789/0 conn(0x7fbbcc0adf80 legacy=0x7fbbcc154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc0af460 con 0x7fbbcc06f4f0 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fbbbc001840 con 0x7fbbcc06f4f0 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 >> v1:172.21.15.26:6789/0 conn(0x7fbbcc06f4f0 legacy=0x7fbbcc0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.625+0000 7fbbd1368640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbb80039b0 0x7fbbb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1368640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb8004800 0x7fbbb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1368640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb80052e0 0x7fbbb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0af180 con 0x7fbbb8004800 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb800c1b0 con 0x7fbbb80052e0 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb800c3b0 con 0x7fbbb80039b0 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1b69640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb8004800 0x7fbbb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd236a640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbb80039b0 0x7fbbb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd2b6b640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb80052e0 0x7fbbb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1b69640 1 -- 172.21.15.26:0/2720719698 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbb80039b0 msgr2=0x7fbbb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:30.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1b69640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbb80039b0 0x7fbbb8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1b69640 1 -- 172.21.15.26:0/2720719698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb80052e0 msgr2=0x7fbbb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1b69640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb80052e0 0x7fbbb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd1b69640 1 -- 172.21.15.26:0/2720719698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb8003520 con 0x7fbbb8004800 2024-04-24T23:26:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.626+0000 7fbbd2b6b640 1 -- 172.21.15.26:0/2720719698 reap_dead start 2024-04-24T23:26:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.627+0000 7fbbd1b69640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb8004800 0x7fbbb8004c80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc06f900 tx=0x7fbbb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1955b85cb01f3d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.627+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc00c0e0 con 0x7fbbb8004800 2024-04-24T23:26:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.627+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbbc034070 con 0x7fbbb8004800 2024-04-24T23:26:30.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.627+0000 7fbbd1368640 1 -- 172.21.15.26:0/2720719698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc005d70 con 0x7fbbb8004800 2024-04-24T23:26:30.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.627+0000 7fbbd45ce640 1 -- 172.21.15.26:0/2720719698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb8004800 msgr2=0x7fbbb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.627+0000 7fbbd45ce640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb8004800 0x7fbbb8004c80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc06f900 tx=0x7fbbb8005830 comp rx=0 tx=0).stop 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.628+0000 7fbbd45ce640 1 -- 172.21.15.26:0/2720719698 shutdown_connections 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.628+0000 7fbbd45ce640 1 --2- 172.21.15.26:0/2720719698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb8004800 0x7fbbb8004c80 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.628+0000 7fbbd45ce640 1 -- 172.21.15.26:0/2720719698 >> 172.21.15.26:0/2720719698 conn(0x7fbbcc0729a0 msgr2=0x7fbbcc06b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.628+0000 7fbbd45ce640 1 -- 172.21.15.26:0/2720719698 shutdown_connections 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.628+0000 7fbbd45ce640 1 -- 172.21.15.26:0/2720719698 wait complete. 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.629+0000 7fbbd45ce640 1 Processor -- start 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.629+0000 7fbbd45ce640 1 -- start start 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd45ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb8004800 0x7fbbcc147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd45ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbcc159ea0 0x7fbbcc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd45ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbcc143e70 0x7fbbcc1e9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd45ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0afd00 con 0x7fbbcc143e70 2024-04-24T23:26:30.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd45ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0afa20 con 0x7fbbb8004800 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd45ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc0aee50 con 0x7fbbcc159ea0 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd2b6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbcc143e70 0x7fbbcc1e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd2b6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbcc143e70 0x7fbbcc1e9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60580/0 (socket says 172.21.15.26:60580) 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd2b6b640 1 -- 172.21.15.26:0/827206579 learned_addr learned my addr 172.21.15.26:0/827206579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd1b69640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbcc159ea0 0x7fbbcc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd236a640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb8004800 0x7fbbcc147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd2b6b640 1 -- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbcc159ea0 msgr2=0x7fbbcc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd2b6b640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbcc159ea0 0x7fbbcc15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd2b6b640 1 -- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb8004800 msgr2=0x7fbbcc147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd2b6b640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb8004800 0x7fbbcc147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd2b6b640 1 -- 172.21.15.26:0/827206579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc15bbe0 con 0x7fbbcc143e70 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd1b69640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbcc159ea0 0x7fbbcc15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd236a640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb8004800 0x7fbbcc147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:30.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd2b6b640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbcc143e70 0x7fbbcc1e9120 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0b3c70 tx=0x7fbbc400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbc2ffd640 1 -- 172.21.15.26:0/827206579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc401a070 con 0x7fbbcc143e70 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbc2ffd640 1 -- 172.21.15.26:0/827206579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc4006d60 con 0x7fbbcc143e70 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbc2ffd640 1 -- 172.21.15.26:0/827206579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4003bb0 con 0x7fbbcc143e70 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc15a620 con 0x7fbbcc143e70 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.630+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc15abe0 con 0x7fbbcc143e70 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.631+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbcc0694c0 con 0x7fbbcc143e70 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.632+0000 7fbbc2ffd640 1 -- 172.21.15.26:0/827206579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbc401d070 con 0x7fbbcc143e70 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.632+0000 7fbbc2ffd640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbbb006be30 0x7fbbb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.632+0000 7fbbc2ffd640 1 -- 172.21.15.26:0/827206579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fbbc408a490 con 0x7fbbcc143e70 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.632+0000 7fbbd236a640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbbb006be30 0x7fbbb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.633+0000 7fbbd236a640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbbb006be30 0x7fbbb006e2f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc068f60 tx=0x7fbbc8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.637+0000 7fbbc2ffd640 1 -- 172.21.15.26:0/827206579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbbc4058ed0 con 0x7fbbcc143e70 2024-04-24T23:26:30.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.857+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbcc161940 con 0x7fbbcc143e70 2024-04-24T23:26:30.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.858+0000 7fbbc2ffd640 1 -- 172.21.15.26:0/827206579 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5196 (secure 0 0 0) 0x7fbbc40587d0 con 0x7fbbcc143e70 2024-04-24T23:26:30.859 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:30.859 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":86,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:30.455354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6790,"mds_1":6778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:creating","state_seq":1,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790]},"id":6}]} 2024-04-24T23:26:30.860 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 86 2024-04-24T23:26:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbbb006be30 msgr2=0x7fbbb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbbb006be30 0x7fbbb006e2f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc068f60 tx=0x7fbbc8008040 comp rx=0 tx=0).stop 2024-04-24T23:26:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbcc143e70 msgr2=0x7fbbcc1e9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbcc143e70 0x7fbbcc1e9120 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0b3c70 tx=0x7fbbc400fc90 comp rx=0 tx=0).stop 2024-04-24T23:26:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 shutdown_connections 2024-04-24T23:26:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbbb006be30 0x7fbbb006e2f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbcc143e70 0x7fbbcc1e9120 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbbcc159ea0 0x7fbbcc15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 --2- 172.21.15.26:0/827206579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbbb8004800 0x7fbbcc147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:30.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 >> 172.21.15.26:0/827206579 conn(0x7fbbcc0729a0 msgr2=0x7fbbcc0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:30.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 shutdown_connections 2024-04-24T23:26:30.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:30.860+0000 7fbbd45ce640 1 -- 172.21.15.26:0/827206579 wait complete. 2024-04-24T23:26:31.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:26:17.523402+0000', 'modified': '2024-04-24T23:26:30.455354+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6790, 'mds_1': 6778}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6778': {'gid': 6778, 'name': 'a', 'rank': 1, 'incarnation': 86, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.26:6836/306957409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 306957409}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 306957409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6790': {'gid': 6790, 'name': 'b', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6834/2253149800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 2253149800}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 2253149800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6790, 'qdb_cluster': [6790]} 2024-04-24T23:26:31.275 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6778:up:creating 2024-04-24T23:26:32.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:32.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.388+0000 7f59d4fb7640 1 Processor -- start 2024-04-24T23:26:32.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.389+0000 7f59d4fb7640 1 -- start start 2024-04-24T23:26:32.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.389+0000 7f59d4fb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d00d4870 0x7f59d00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.389+0000 7f59d4fb7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59d00b3130 0x7f59d00b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59d4fb7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d00d32f0 con 0x7f59d015e570 2024-04-24T23:26:32.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59d4fb7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f59d00d1ab0 con 0x7f59d00b3130 2024-04-24T23:26:32.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59d4fb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59d00d2ae0 con 0x7f59d00d4870 2024-04-24T23:26:32.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59cdd9b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d00d4870 0x7f59d00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59ced9d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f59d00b3130 0x7f59d00b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59ce59c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f59d015e570 0x7f59d00d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:44960/0 (socket says 172.21.15.26:44960) 2024-04-24T23:26:32.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59ce59c640 1 -- 172.21.15.26:0/994670858 learned_addr learned my addr 172.21.15.26:0/994670858 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59ced9d640 1 -- 172.21.15.26:0/994670858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d00d4870 msgr2=0x7f59d00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59ced9d640 1 --2- 172.21.15.26:0/994670858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d00d4870 0x7f59d00b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59ced9d640 1 -- 172.21.15.26:0/994670858 >> v1:172.21.15.29:6789/0 conn(0x7f59d015e570 legacy=0x7f59d00d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59ced9d640 1 -- 172.21.15.26:0/994670858 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d00d0270 con 0x7f59d00b3130 2024-04-24T23:26:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59cd59a640 1 -- 172.21.15.26:0/994670858 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2771336799 0 0) 0x7f59c8001110 con 0x7f59d015e570 2024-04-24T23:26:32.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.390+0000 7f59cdd9b640 1 --2- 172.21.15.26:0/994670858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d00d4870 0x7f59d00b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.391+0000 7f59ced9d640 1 --2- 172.21.15.26:0/994670858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 0x7f59d00b4640 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009d90 tx=0x7f59c002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6118322772e4d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.391+0000 7f59cd59a640 1 -- 172.21.15.26:0/994670858 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0007740 con 0x7f59d00b3130 2024-04-24T23:26:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.391+0000 7f59cd59a640 1 -- 172.21.15.26:0/994670858 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c00078e0 con 0x7f59d00b3130 2024-04-24T23:26:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.391+0000 7f59cd59a640 1 -- 172.21.15.26:0/994670858 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0007cb0 con 0x7f59d00b3130 2024-04-24T23:26:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.392+0000 7f59d4fb7640 1 -- 172.21.15.26:0/994670858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 msgr2=0x7f59d00b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.392+0000 7f59d4fb7640 1 --2- 172.21.15.26:0/994670858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 0x7f59d00b4640 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009d90 tx=0x7f59c002a660 comp rx=0 tx=0).stop 2024-04-24T23:26:32.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.392+0000 7f59d4fb7640 1 -- 172.21.15.26:0/994670858 shutdown_connections 2024-04-24T23:26:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.392+0000 7f59d4fb7640 1 --2- 172.21.15.26:0/994670858 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 0x7f59d00b4640 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.392+0000 7f59d4fb7640 1 --2- 172.21.15.26:0/994670858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d00d4870 0x7f59d00b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.392+0000 7f59d4fb7640 1 -- 172.21.15.26:0/994670858 >> 172.21.15.26:0/994670858 conn(0x7f59d008dbe0 msgr2=0x7f59d009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.392+0000 7f59d4fb7640 1 -- 172.21.15.26:0/994670858 shutdown_connections 2024-04-24T23:26:32.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.392+0000 7f59d4fb7640 1 -- 172.21.15.26:0/994670858 wait complete. 2024-04-24T23:26:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.393+0000 7f59d4fb7640 1 Processor -- start 2024-04-24T23:26:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.393+0000 7f59d4fb7640 1 -- start start 2024-04-24T23:26:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59d4fb7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 0x7f59d00b9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59d4fb7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d00d4870 0x7f59d00b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59d4fb7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d015e570 0x7f59d00b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59d4fb7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00cfa70 con 0x7f59d00d4870 2024-04-24T23:26:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59d4fb7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00d3010 con 0x7f59d00b3130 2024-04-24T23:26:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59d4fb7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f59d00cfd50 con 0x7f59d015e570 2024-04-24T23:26:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59cdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d00d4870 0x7f59d00b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59ce59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 0x7f59d00b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59cdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d00d4870 0x7f59d00b98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60590/0 (socket says 172.21.15.26:60590) 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59ced9d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d015e570 0x7f59d00b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59cdd9b640 1 -- 172.21.15.26:0/3443421762 learned_addr learned my addr 172.21.15.26:0/3443421762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59ce59c640 1 -- 172.21.15.26:0/3443421762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d015e570 msgr2=0x7f59d00b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59ce59c640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d015e570 0x7f59d00b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59ce59c640 1 -- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d00d4870 msgr2=0x7f59d00b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59ce59c640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d00d4870 0x7f59d00b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59ce59c640 1 -- 172.21.15.26:0/3443421762 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d0161e70 con 0x7f59d00b3130 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.394+0000 7f59cdd9b640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d00d4870 0x7f59d00b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.395+0000 7f59ce59c640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 0x7f59d00b9370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59c800c410 tx=0x7f59c800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.395+0000 7f59c6ffd640 1 -- 172.21.15.26:0/3443421762 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80074e0 con 0x7f59d00b3130 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.395+0000 7f59c6ffd640 1 -- 172.21.15.26:0/3443421762 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c80047e0 con 0x7f59d00b3130 2024-04-24T23:26:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.395+0000 7f59c6ffd640 1 -- 172.21.15.26:0/3443421762 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8004bd0 con 0x7f59d00b3130 2024-04-24T23:26:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.395+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d00b7990 con 0x7f59d00b3130 2024-04-24T23:26:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.395+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d015eaf0 con 0x7f59d00b3130 2024-04-24T23:26:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.396+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d00d3a90 con 0x7f59d00b3130 2024-04-24T23:26:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.396+0000 7f59c6ffd640 1 -- 172.21.15.26:0/3443421762 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59c8011070 con 0x7f59d00b3130 2024-04-24T23:26:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.397+0000 7f59c6ffd640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b006be30 0x7f59b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.397+0000 7f59c6ffd640 1 -- 172.21.15.26:0/3443421762 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f59c808a8f0 con 0x7f59d00b3130 2024-04-24T23:26:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.397+0000 7f59cdd9b640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b006be30 0x7f59b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.397+0000 7f59cdd9b640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b006be30 0x7f59b006e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f59d01f7aa0 tx=0x7f59b8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:32.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.402+0000 7f59c6ffd640 1 -- 172.21.15.26:0/3443421762 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f59d00d3a90 con 0x7f59d00b3130 2024-04-24T23:26:32.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.627+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59d00694c0 con 0x7f59d00b3130 2024-04-24T23:26:32.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.628+0000 7f59c6ffd640 1 -- 172.21.15.26:0/3443421762 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5218 (secure 0 0 0) 0x7f59d00d32f0 con 0x7f59d00b3130 2024-04-24T23:26:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:32.630 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:32.463770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6299,6778,6790]},"id":6}]} 2024-04-24T23:26:32.630 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 89 2024-04-24T23:26:32.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b006be30 msgr2=0x7f59b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b006be30 0x7f59b006e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f59d01f7aa0 tx=0x7f59b8009190 comp rx=0 tx=0).stop 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 msgr2=0x7f59d00b9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 0x7f59d00b9370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59c800c410 tx=0x7f59c800c8d0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 shutdown_connections 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59b006be30 0x7f59b006e2f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f59d015e570 0x7f59d00b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d00d4870 0x7f59d00b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 --2- 172.21.15.26:0/3443421762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59d00b3130 0x7f59d00b9370 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.630+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 >> 172.21.15.26:0/3443421762 conn(0x7f59d008dbe0 msgr2=0x7f59d00d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.631+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 shutdown_connections 2024-04-24T23:26:32.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:32.631+0000 7f59d4fb7640 1 -- 172.21.15.26:0/3443421762 wait complete. 2024-04-24T23:26:33.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:26:17.523402+0000', 'modified': '2024-04-24T23:26:32.463770+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6790, 'mds_1': 6778, 'mds_2': 6299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6299': {'gid': 6299, 'name': 'd', 'rank': 2, 'incarnation': 88, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.29:6835/3727846607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 3727846607}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 3727846607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6778': {'gid': 6778, 'name': 'a', 'rank': 1, 'incarnation': 86, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.26:6836/306957409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 306957409}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 306957409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6790': {'gid': 6790, 'name': 'b', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6834/2253149800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 2253149800}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 2253149800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6790, 'qdb_cluster': [6299, 6778, 6790]} 2024-04-24T23:26:33.047 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:26:33.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:26:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.153+0000 7fa41214c640 1 Processor -- start 2024-04-24T23:26:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.153+0000 7fa41214c640 1 -- start start 2024-04-24T23:26:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa41214c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa40c142310 0x7fa40c142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa41214c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 0x7fa40c0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa41214c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa40c0b5cd0 0x7fa40c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa41214c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa40c06dc60 con 0x7fa40c0b5cd0 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa41214c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa40c06eac0 con 0x7fa40c142310 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa41214c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c06deb0 con 0x7fa40c145890 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa40c0b5cd0 0x7fa40c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa40c0b5cd0 0x7fa40c0b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60598/0 (socket says 172.21.15.26:60598) 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40bfff640 1 -- 172.21.15.26:0/2618000629 learned_addr learned my addr 172.21.15.26:0/2618000629 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40b7fe640 1 --2- 172.21.15.26:0/2618000629 >> v2:172.21.15.29:3300/0 conn(0x7fa40c142310 0x7fa40c142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40affd640 1 --2- 172.21.15.26:0/2618000629 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 0x7fa40c0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40bfff640 1 -- 172.21.15.26:0/2618000629 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 msgr2=0x7fa40c0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40bfff640 1 --2- 172.21.15.26:0/2618000629 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 0x7fa40c0d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40bfff640 1 -- 172.21.15.26:0/2618000629 >> v2:172.21.15.29:3300/0 conn(0x7fa40c142310 msgr2=0x7fa40c142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40bfff640 1 --2- 172.21.15.26:0/2618000629 >> v2:172.21.15.29:3300/0 conn(0x7fa40c142310 0x7fa40c142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.154+0000 7fa40bfff640 1 -- 172.21.15.26:0/2618000629 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c06e190 con 0x7fa40c0b5cd0 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.155+0000 7fa40bfff640 1 --2- 172.21.15.26:0/2618000629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c0b5cd0 0x7fa40c0b6160 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa40c072f00 tx=0x7fa40402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e053ae662a7253b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.155+0000 7fa40a7fc640 1 -- 172.21.15.26:0/2618000629 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404036070 con 0x7fa40c0b5cd0 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.155+0000 7fa40a7fc640 1 -- 172.21.15.26:0/2618000629 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa404031600 con 0x7fa40c0b5cd0 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.155+0000 7fa40a7fc640 1 -- 172.21.15.26:0/2618000629 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040319d0 con 0x7fa40c0b5cd0 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.156+0000 7fa41214c640 1 -- 172.21.15.26:0/2618000629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c0b5cd0 msgr2=0x7fa40c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.156+0000 7fa41214c640 1 --2- 172.21.15.26:0/2618000629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c0b5cd0 0x7fa40c0b6160 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa40c072f00 tx=0x7fa40402a610 comp rx=0 tx=0).stop 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.156+0000 7fa41214c640 1 -- 172.21.15.26:0/2618000629 shutdown_connections 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.156+0000 7fa41214c640 1 --2- 172.21.15.26:0/2618000629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c0b5cd0 0x7fa40c0b6160 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.156+0000 7fa41214c640 1 --2- 172.21.15.26:0/2618000629 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 0x7fa40c0d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.156+0000 7fa41214c640 1 --2- 172.21.15.26:0/2618000629 >> v2:172.21.15.29:3300/0 conn(0x7fa40c142310 0x7fa40c142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.156+0000 7fa41214c640 1 -- 172.21.15.26:0/2618000629 >> 172.21.15.26:0/2618000629 conn(0x7fa40c0729a0 msgr2=0x7fa40c06b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.156+0000 7fa41214c640 1 -- 172.21.15.26:0/2618000629 shutdown_connections 2024-04-24T23:26:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.156+0000 7fa41214c640 1 -- 172.21.15.26:0/2618000629 wait complete. 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.157+0000 7fa41214c640 1 Processor -- start 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.157+0000 7fa41214c640 1 -- start start 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa41214c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa40c0b5cd0 0x7fa40c1f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa41214c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c142310 0x7fa40c13cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa41214c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 0x7fa40c0af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa41214c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1426f0 con 0x7fa40c142310 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa41214c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c06e750 con 0x7fa40c0b5cd0 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa41214c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c142870 con 0x7fa40c145890 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c142310 0x7fa40c13cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c142310 0x7fa40c13cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60600/0 (socket says 172.21.15.26:60600) 2024-04-24T23:26:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40affd640 1 -- 172.21.15.26:0/3206423715 learned_addr learned my addr 172.21.15.26:0/3206423715 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40b7fe640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa40c0b5cd0 0x7fa40c1f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40bfff640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 0x7fa40c0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40affd640 1 -- 172.21.15.26:0/3206423715 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 msgr2=0x7fa40c0af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40affd640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 0x7fa40c0af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40affd640 1 -- 172.21.15.26:0/3206423715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa40c0b5cd0 msgr2=0x7fa40c1f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40affd640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa40c0b5cd0 0x7fa40c1f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40affd640 1 -- 172.21.15.26:0/3206423715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c13d080 con 0x7fa40c142310 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.158+0000 7fa40affd640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c142310 0x7fa40c13cb40 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e870 tx=0x7fa40000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.159+0000 7fa3f3fff640 1 -- 172.21.15.26:0/3206423715 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400017070 con 0x7fa40c142310 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.159+0000 7fa41214c640 1 -- 172.21.15.26:0/3206423715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c0afc70 con 0x7fa40c142310 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.159+0000 7fa3f3fff640 1 -- 172.21.15.26:0/3206423715 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40000c940 con 0x7fa40c142310 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.159+0000 7fa3f3fff640 1 -- 172.21.15.26:0/3206423715 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40000cd30 con 0x7fa40c142310 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.159+0000 7fa41214c640 1 -- 172.21.15.26:0/3206423715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c0b3b60 con 0x7fa40c142310 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.160+0000 7fa3f3fff640 1 -- 172.21.15.26:0/3206423715 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa40001a070 con 0x7fa40c142310 2024-04-24T23:26:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.161+0000 7fa3f3fff640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa3f806be30 0x7fa3f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.161+0000 7fa3f3fff640 1 -- 172.21.15.26:0/3206423715 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fa40008aca0 con 0x7fa40c142310 2024-04-24T23:26:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.161+0000 7fa40b7fe640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa3f806be30 0x7fa3f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.161+0000 7fa40b7fe640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa3f806be30 0x7fa3f806e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009ac0 tx=0x7fa3fc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.162+0000 7fa414e58740 1 Processor -- start 2024-04-24T23:26:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.162+0000 7fa414e58740 1 -- start start 2024-04-24T23:26:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa414e58740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179edaab60 0x56179ed5d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa414e58740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed8dc80 0x56179ed8d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa414e58740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179ed8d540 0x56179ed62350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa414e58740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56179ecc5f00 con 0x56179ed8d540 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa414e58740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56179ec6e750 con 0x56179ed8dc80 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa414e58740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56179ecc1ec0 con 0x56179edaab60 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179ed8d540 0x56179ed62350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179edaab60 0x56179ed5d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed8dc80 0x56179ed8d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179edaab60 0x56179ed5d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:47916/0 (socket says 172.21.15.26:47916) 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed8dc80 0x56179ed8d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:36702/0 (socket says 172.21.15.26:36702) 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40b7fe640 1 -- 172.21.15.26:0/969301212 learned_addr learned my addr 172.21.15.26:0/969301212 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179ed8d540 0x56179ed62350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60602/0 (socket says 172.21.15.26:60602) 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40bfff640 1 -- 172.21.15.26:0/969301212 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179edaab60 msgr2=0x56179ed5d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40bfff640 1 --2- 172.21.15.26:0/969301212 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179edaab60 0x56179ed5d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40bfff640 1 -- 172.21.15.26:0/969301212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed8dc80 msgr2=0x56179ed8d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40bfff640 1 --2- 172.21.15.26:0/969301212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed8dc80 0x56179ed8d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40bfff640 1 -- 172.21.15.26:0/969301212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179e9da2d0 con 0x56179ed8d540 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.163+0000 7fa40affd640 1 --2- 172.21.15.26:0/969301212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed8dc80 0x56179ed8d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.164+0000 7fa40bfff640 1 --2- 172.21.15.26:0/969301212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179ed8d540 0x56179ed62350 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fa40c06c9d0 tx=0x7fa40402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48040666e071df3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.164+0000 7fa3f17fa640 1 -- 172.21.15.26:0/969301212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404036040 con 0x56179ed8d540 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.164+0000 7fa3f17fa640 1 -- 172.21.15.26:0/969301212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40403d720 con 0x56179ed8d540 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.164+0000 7fa3f17fa640 1 -- 172.21.15.26:0/969301212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40403daf0 con 0x56179ed8d540 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 -- 172.21.15.26:0/969301212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179ed8d540 msgr2=0x56179ed62350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 --2- 172.21.15.26:0/969301212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179ed8d540 0x56179ed62350 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fa40c06c9d0 tx=0x7fa40402e000 comp rx=0 tx=0).stop 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 -- 172.21.15.26:0/969301212 shutdown_connections 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 --2- 172.21.15.26:0/969301212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179ed8d540 0x56179ed62350 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 --2- 172.21.15.26:0/969301212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed8dc80 0x56179ed8d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 --2- 172.21.15.26:0/969301212 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179edaab60 0x56179ed5d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 -- 172.21.15.26:0/969301212 >> 172.21.15.26:0/969301212 conn(0x56179ed5f760 msgr2=0x56179eda8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 -- 172.21.15.26:0/969301212 shutdown_connections 2024-04-24T23:26:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 -- 172.21.15.26:0/969301212 wait complete. 2024-04-24T23:26:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.165+0000 7fa414e58740 1 -- start start 2024-04-24T23:26:33.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.166+0000 7fa414e58740 1 Processor -- start 2024-04-24T23:26:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.166+0000 7fa414e58740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179ed8f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.166+0000 7fa414e58740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed62510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.166+0000 7fa414e58740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179ed62a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.166+0000 7fa414e58740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56179ed61910 con 0x56179edaab60 2024-04-24T23:26:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.166+0000 7fa414e58740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56179ec81cf0 con 0x56179ed88c10 2024-04-24T23:26:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.166+0000 7fa414e58740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56179ed60d00 con 0x56179ed8dc80 2024-04-24T23:26:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.166+0000 7fa414e58740 10 client.0 _ll_register_callbacks cb 0x56179ec816e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed62510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179ed8f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179ed62a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179ed8f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:47926/0 (socket says 172.21.15.26:47926) 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed62510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60606/0 (socket says 172.21.15.26:60606) 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40b7fe640 1 -- 172.21.15.26:0/3906851720 learned_addr learned my addr 172.21.15.26:0/3906851720 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40affd640 1 -- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 msgr2=0x56179ed8f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40affd640 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179ed8f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40affd640 1 -- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 msgr2=0x56179ed62a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40affd640 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179ed62a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40affd640 1 -- 172.21.15.26:0/3906851720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179eaa46b0 con 0x56179edaab60 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40bfff640 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179ed62a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa40affd640 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed62510 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fa4000891a0 tx=0x7fa400058f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142dc0ef3ef4aa15 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa3ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/3906851720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400021040 con 0x56179edaab60 2024-04-24T23:26:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa414e58740 1 -- 172.21.15.26:0/3906851720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40403d720 con 0x56179edaab60 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/3906851720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fa400076d70 con 0x56179edaab60 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/3906851720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa400061ab0 con 0x56179edaab60 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa414e58740 10 client.8055 fetch_fsmap learned FSMap version 89 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.167+0000 7fa414e58740 1 -- 172.21.15.26:0/3906851720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x56179ed90a60 con 0x56179edaab60 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/3906851720 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40403d720 con 0x56179edaab60 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/3906851720 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) ==== 3534+0+0 (secure 0 0 0) 0x7fa400079020 con 0x56179edaab60 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa414e58740 10 client.8055 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa414e58740 10 client.8055 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa414e58740 20 client.8055 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa414e58740 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x56179edb8a10 0x56179edb8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa414e58740 4 client.8055 mds_command: new command op to 6299 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa414e58740 1 -- 172.21.15.26:0/3906851720 --> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56179ecd3440 con 0x56179edb8a10 2024-04-24T23:26:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa40b7fe640 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x56179edb8a10 0x56179edb8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa3ea7fc640 10 client.8055 ms_handle_connect on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.168+0000 7fa40b7fe640 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x56179edb8a10 0x56179edb8df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a747adea08bac31 server_cookie=95a4a62f02a47ba0 in_seq=0 out_seq=0 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/3906851720 <== mds.2 v2:172.21.15.29:6833/3727846607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56179ecd3440 con 0x56179edb8a10 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa3ea7fc640 10 client.8055 handle_command_reply: tid=0 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 client.8055 shutdown 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 -- 172.21.15.26:0/3906851720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 msgr2=0x56179ed62510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed62510 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fa4000891a0 tx=0x7fa400058f80 comp rx=0 tx=0).stop 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 -- 172.21.15.26:0/3906851720 shutdown_connections 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x56179edb8a10 0x56179edb8df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa3ea7fc640 0 client.8055 ms_handle_reset on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179ed62a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed62510 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 --2- 172.21.15.26:0/3906851720 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179ed8f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 -- 172.21.15.26:0/3906851720 >> 172.21.15.26:0/3906851720 conn(0x56179ed8d540 msgr2=0x56179ed61d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 -- 172.21.15.26:0/3906851720 shutdown_connections 2024-04-24T23:26:33.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 1 -- 172.21.15.26:0/3906851720 wait complete. 2024-04-24T23:26:33.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.169+0000 7fa414e58740 20 client.8055 trim_cache size 0 max 16384 2024-04-24T23:26:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.188+0000 7fa414e58740 1 Processor -- start 2024-04-24T23:26:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.188+0000 7fa414e58740 1 -- start start 2024-04-24T23:26:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa414e58740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179edc2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa414e58740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179eda7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa414e58740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ee1bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa414e58740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1f7780 con 0x56179edaab60 2024-04-24T23:26:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa414e58740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1f7e50 con 0x56179ed88c10 2024-04-24T23:26:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa414e58740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c1f0e10 con 0x56179ed8dc80 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ee1bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ee1bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60612/0 (socket says 172.21.15.26:60612) 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40bfff640 1 -- 172.21.15.26:0/3417840225 learned_addr learned my addr 172.21.15.26:0/3417840225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40b7fe640 1 --2- 172.21.15.26:0/3417840225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179edc2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40affd640 1 --2- 172.21.15.26:0/3417840225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179eda7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40bfff640 1 -- 172.21.15.26:0/3417840225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 msgr2=0x56179eda7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40bfff640 1 --2- 172.21.15.26:0/3417840225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179eda7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40bfff640 1 -- 172.21.15.26:0/3417840225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 msgr2=0x56179edc2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40bfff640 1 --2- 172.21.15.26:0/3417840225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179edc2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.189+0000 7fa40bfff640 1 -- 172.21.15.26:0/3417840225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179ed8f860 con 0x56179edaab60 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.190+0000 7fa40bfff640 1 --2- 172.21.15.26:0/3417840225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ee1bed0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa40402a650 tx=0x7fa40402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d66750badbceb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.190+0000 7fa3f0ff9640 1 -- 172.21.15.26:0/3417840225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404036070 con 0x56179edaab60 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.190+0000 7fa3f0ff9640 1 -- 172.21.15.26:0/3417840225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40404c070 con 0x56179edaab60 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.190+0000 7fa3f0ff9640 1 -- 172.21.15.26:0/3417840225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404003d90 con 0x56179edaab60 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.190+0000 7fa414e58740 1 -- 172.21.15.26:0/3417840225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 msgr2=0x56179ee1bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.190+0000 7fa414e58740 1 --2- 172.21.15.26:0/3417840225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ee1bed0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa40402a650 tx=0x7fa40402f000 comp rx=0 tx=0).stop 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.191+0000 7fa414e58740 1 -- 172.21.15.26:0/3417840225 shutdown_connections 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.191+0000 7fa414e58740 1 --2- 172.21.15.26:0/3417840225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ee1bed0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.191+0000 7fa414e58740 1 --2- 172.21.15.26:0/3417840225 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179eda7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.191+0000 7fa414e58740 1 --2- 172.21.15.26:0/3417840225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179edc2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.191+0000 7fa414e58740 1 -- 172.21.15.26:0/3417840225 >> 172.21.15.26:0/3417840225 conn(0x56179ed8d540 msgr2=0x56179ed8eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.191+0000 7fa414e58740 1 -- 172.21.15.26:0/3417840225 shutdown_connections 2024-04-24T23:26:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.191+0000 7fa414e58740 1 -- 172.21.15.26:0/3417840225 wait complete. 2024-04-24T23:26:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.191+0000 7fa414e58740 1 -- start start 2024-04-24T23:26:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.191+0000 7fa414e58740 1 Processor -- start 2024-04-24T23:26:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa414e58740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179eda6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa414e58740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179ed893b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa414e58740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed9d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa414e58740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56179ed9f870 con 0x56179edaab60 2024-04-24T23:26:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa414e58740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56179ed8fca0 con 0x56179ed88c10 2024-04-24T23:26:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa414e58740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56179ed8e9a0 con 0x56179ed8dc80 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa414e58740 10 client.0 _ll_register_callbacks cb 0x7fa400058ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179eda6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed9d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179eda6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:36738/0 (socket says 172.21.15.26:36738) 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40b7fe640 1 -- 172.21.15.26:0/2336870296 learned_addr learned my addr 172.21.15.26:0/2336870296 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40affd640 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179ed893b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40bfff640 1 -- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 msgr2=0x56179ed893b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40bfff640 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179ed893b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40bfff640 1 -- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 msgr2=0x56179eda6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40bfff640 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179eda6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.192+0000 7fa40bfff640 1 -- 172.21.15.26:0/2336870296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179ed9ed00 con 0x56179edaab60 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa40bfff640 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed9d990 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa40404d000 tx=0x7fa404004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ad726cbdb2e96c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa3ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/2336870296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404036040 con 0x56179edaab60 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa414e58740 1 -- 172.21.15.26:0/2336870296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40404c070 con 0x56179edaab60 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/2336870296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fa404004450 con 0x56179edaab60 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/2336870296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa404061050 con 0x56179edaab60 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa414e58740 10 client.8061 fetch_fsmap learned FSMap version 89 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa414e58740 1 -- 172.21.15.26:0/2336870296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x56179eda62d0 con 0x56179edaab60 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/2336870296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40404c070 con 0x56179edaab60 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/2336870296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) ==== 3534+0+0 (secure 0 0 0) 0x7fa404008040 con 0x56179edaab60 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa414e58740 10 client.8061 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa414e58740 10 client.8061 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa414e58740 20 client.8061 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa414e58740 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x56179edb8a10 0x56179ed9ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa414e58740 4 client.8061 mds_command: new command op to 6299 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa414e58740 1 -- 172.21.15.26:0/2336870296 --> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] -- command(tid 0: {"prefix": "status"}) -- 0x56179eccd790 con 0x56179edb8a10 2024-04-24T23:26:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.193+0000 7fa40b7fe640 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x56179edb8a10 0x56179ed9ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa3ea7fc640 10 client.8061 ms_handle_connect on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa40b7fe640 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x56179edb8a10 0x56179ed9ded0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71d238e4f9e35e31 server_cookie=676686eadaaeb197 in_seq=0 out_seq=0 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa3ea7fc640 1 -- 172.21.15.26:0/2336870296 <== mds.2 v2:172.21.15.29:6833/3727846607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x56179eccd790 con 0x56179edb8a10 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa3ea7fc640 10 client.8061 handle_command_reply: tid=0 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 client.8061 shutdown 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 -- 172.21.15.26:0/2336870296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 msgr2=0x56179ed9d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed9d990 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa40404d000 tx=0x7fa404004ab0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 -- 172.21.15.26:0/2336870296 shutdown_connections 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x56179edb8a10 0x56179ed9ded0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56179edaab60 0x56179ed9d990 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa3ea7fc640 0 client.8061 ms_handle_reset on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56179ed8dc80 0x56179ed893b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 --2- 172.21.15.26:0/2336870296 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56179ed88c10 0x56179eda6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 -- 172.21.15.26:0/2336870296 >> 172.21.15.26:0/2336870296 conn(0x56179ed8d540 msgr2=0x56179ed8ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 -- 172.21.15.26:0/2336870296 shutdown_connections 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 1 -- 172.21.15.26:0/2336870296 wait complete. 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.194+0000 7fa414e58740 20 client.8061 trim_cache size 0 max 16384 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6299, 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:26:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 1.73227267, 2024-04-24T23:26:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 89, 2024-04-24T23:26:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-04-24T23:26:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 82, 2024-04-24T23:26:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 20.033689905999999 2024-04-24T23:26:33.205 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 -- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa3f806be30 msgr2=0x7fa3f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa3f806be30 0x7fa3f806e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009ac0 tx=0x7fa3fc009190 comp rx=0 tx=0).stop 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 -- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c142310 msgr2=0x7fa40c13cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c142310 0x7fa40c13cb40 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e870 tx=0x7fa40000ed30 comp rx=0 tx=0).stop 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 -- 172.21.15.26:0/3206423715 shutdown_connections 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa3f806be30 0x7fa3f806e2f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa40c145890 0x7fa40c0af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c142310 0x7fa40c13cb40 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 --2- 172.21.15.26:0/3206423715 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa40c0b5cd0 0x7fa40c1f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 -- 172.21.15.26:0/3206423715 >> 172.21.15.26:0/3206423715 conn(0x7fa40c0729a0 msgr2=0x7fa40c0cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 -- 172.21.15.26:0/3206423715 shutdown_connections 2024-04-24T23:26:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.195+0000 7fa41214c640 1 -- 172.21.15.26:0/3206423715 wait complete. 2024-04-24T23:26:33.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T23:26:33.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.307+0000 7fbf93f4f640 1 Processor -- start 2024-04-24T23:26:33.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.308+0000 7fbf93f4f640 1 -- start start 2024-04-24T23:26:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.308+0000 7fbf93f4f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf8c15e280 0x7fbf8c0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.308+0000 7fbf93f4f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.308+0000 7fbf93f4f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf8c0d1a20 con 0x7fbf8c0b4050 2024-04-24T23:26:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.308+0000 7fbf93f4f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbf8c0d1740 con 0x7fbf8c15e280 2024-04-24T23:26:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.308+0000 7fbf93f4f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c0cee60 con 0x7fbf8c0cfe80 2024-04-24T23:26:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.308+0000 7fbf91ceb640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf8c15e280 0x7fbf8c0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf924ec640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fbf8c0b4050 0x7fbf8c0d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:44964/0 (socket says 172.21.15.26:44964) 2024-04-24T23:26:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf924ec640 1 -- 172.21.15.26:0/4290307326 learned_addr learned my addr 172.21.15.26:0/4290307326 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf914ea640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf90ce9640 1 -- 172.21.15.26:0/4290307326 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468286115 0 0) 0x7fbf8c0d1a20 con 0x7fbf8c0b4050 2024-04-24T23:26:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf90ce9640 1 -- 172.21.15.26:0/4290307326 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf74003520 con 0x7fbf8c0b4050 2024-04-24T23:26:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf914ea640 1 -- 172.21.15.26:0/4290307326 >> v2:172.21.15.29:3300/0 conn(0x7fbf8c15e280 msgr2=0x7fbf8c0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf914ea640 1 --2- 172.21.15.26:0/4290307326 >> v2:172.21.15.29:3300/0 conn(0x7fbf8c15e280 0x7fbf8c0cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf914ea640 1 -- 172.21.15.26:0/4290307326 >> v1:172.21.15.29:6789/0 conn(0x7fbf8c0b4050 legacy=0x7fbf8c0d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf914ea640 1 -- 172.21.15.26:0/4290307326 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c0cf140 con 0x7fbf8c0cfe80 2024-04-24T23:26:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.309+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/4290307326 >> v2:172.21.15.29:3300/0 conn(0x7fbf8c15e280 0x7fbf8c0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf914ea640 1 --2- 172.21.15.26:0/4290307326 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b3b10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf800097a0 tx=0x7fbf8002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f838af55421f3ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf90ce9640 1 -- 172.21.15.26:0/4290307326 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80037070 con 0x7fbf8c0cfe80 2024-04-24T23:26:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf90ce9640 1 -- 172.21.15.26:0/4290307326 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8002ae50 con 0x7fbf8c0cfe80 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf90ce9640 1 -- 172.21.15.26:0/4290307326 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80032660 con 0x7fbf8c0cfe80 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf93f4f640 1 -- 172.21.15.26:0/4290307326 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0cfe80 msgr2=0x7fbf8c0b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf93f4f640 1 --2- 172.21.15.26:0/4290307326 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b3b10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf800097a0 tx=0x7fbf8002a100 comp rx=0 tx=0).stop 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf93f4f640 1 -- 172.21.15.26:0/4290307326 shutdown_connections 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf93f4f640 1 --2- 172.21.15.26:0/4290307326 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b3b10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf93f4f640 1 --2- 172.21.15.26:0/4290307326 >> v2:172.21.15.29:3300/0 conn(0x7fbf8c15e280 0x7fbf8c0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf93f4f640 1 -- 172.21.15.26:0/4290307326 >> 172.21.15.26:0/4290307326 conn(0x7fbf8c08dbd0 msgr2=0x7fbf8c09d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf93f4f640 1 -- 172.21.15.26:0/4290307326 shutdown_connections 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.310+0000 7fbf93f4f640 1 -- 172.21.15.26:0/4290307326 wait complete. 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf93f4f640 1 Processor -- start 2024-04-24T23:26:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf93f4f640 1 -- start start 2024-04-24T23:26:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf93f4f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0b4050 0x7fbf8c0b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf93f4f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf93f4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf8c15e280 0x7fbf8c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf93f4f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c15e660 con 0x7fbf8c15e280 2024-04-24T23:26:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf93f4f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c0b4450 con 0x7fbf8c0cfe80 2024-04-24T23:26:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf93f4f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c15e7e0 con 0x7fbf8c0b4050 2024-04-24T23:26:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf924ec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf8c15e280 0x7fbf8c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf91ceb640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0b4050 0x7fbf8c0b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf914ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf924ec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf8c15e280 0x7fbf8c0b8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60640/0 (socket says 172.21.15.26:60640) 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.312+0000 7fbf91ceb640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0b4050 0x7fbf8c0b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:47974/0 (socket says 172.21.15.26:47974) 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf924ec640 1 -- 172.21.15.26:0/94192131 learned_addr learned my addr 172.21.15.26:0/94192131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf924ec640 1 -- 172.21.15.26:0/94192131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0b4050 msgr2=0x7fbf8c0b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf924ec640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0b4050 0x7fbf8c0b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf924ec640 1 -- 172.21.15.26:0/94192131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf8c0cfe80 msgr2=0x7fbf8c0b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf924ec640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf924ec640 1 -- 172.21.15.26:0/94192131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c0cb910 con 0x7fbf8c15e280 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf914ea640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf924ec640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf8c15e280 0x7fbf8c0b8ce0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00caa0 tx=0x7fbf7c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf8a7fc640 1 -- 172.21.15.26:0/94192131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c017070 con 0x7fbf8c15e280 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf8a7fc640 1 -- 172.21.15.26:0/94192131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7c00ac00 con 0x7fbf8c15e280 2024-04-24T23:26:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf93f4f640 1 -- 172.21.15.26:0/94192131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8c0cbbf0 con 0x7fbf8c15e280 2024-04-24T23:26:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf8a7fc640 1 -- 172.21.15.26:0/94192131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c012400 con 0x7fbf8c15e280 2024-04-24T23:26:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.313+0000 7fbf93f4f640 1 -- 172.21.15.26:0/94192131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8c0cc1b0 con 0x7fbf8c15e280 2024-04-24T23:26:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.315+0000 7fbf8a7fc640 1 -- 172.21.15.26:0/94192131 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf7c01a070 con 0x7fbf8c15e280 2024-04-24T23:26:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.315+0000 7fbf8a7fc640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbf7806be30 0x7fbf7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.315+0000 7fbf8a7fc640 1 -- 172.21.15.26:0/94192131 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fbf7c08ab10 con 0x7fbf8c15e280 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.315+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbf7806be30 0x7fbf7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.316+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbf7806be30 0x7fbf7806e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009560 tx=0x7fbf84009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.316+0000 7fbf96c56740 1 Processor -- start 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.316+0000 7fbf96c56740 1 -- start start 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56bdb60 0x55dec56704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a0c80 0x55dec56a0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0540 0x55dec5675350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf96c56740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dec55d8f00 con 0x55dec56a0c80 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf96c56740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dec5581750 con 0x55dec56bdb60 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf96c56740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55dec55d4ec0 con 0x55dec56a0540 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf91ceb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56bdb60 0x55dec56704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf914ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a0c80 0x55dec56a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf91ceb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56bdb60 0x55dec56704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:36756/0 (socket says 172.21.15.26:36756) 2024-04-24T23:26:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf91ceb640 1 -- 172.21.15.26:0/2213789295 learned_addr learned my addr 172.21.15.26:0/2213789295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf914ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a0c80 0x55dec56a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60656/0 (socket says 172.21.15.26:60656) 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf924ec640 1 --2- 172.21.15.26:0/2213789295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0540 0x55dec5675350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf91ceb640 1 -- 172.21.15.26:0/2213789295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0540 msgr2=0x55dec5675350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/2213789295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0540 0x55dec5675350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf91ceb640 1 -- 172.21.15.26:0/2213789295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a0c80 msgr2=0x55dec56a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/2213789295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a0c80 0x55dec56a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf91ceb640 1 -- 172.21.15.26:0/2213789295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec52ed2d0 con 0x55dec56bdb60 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.317+0000 7fbf914ea640 1 --2- 172.21.15.26:0/2213789295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a0c80 0x55dec56a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/2213789295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56bdb60 0x55dec56704f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbf84011f20 tx=0x7fbf84014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7bcb1048b23320c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf737fe640 1 -- 172.21.15.26:0/2213789295 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8401b070 con 0x55dec56bdb60 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf737fe640 1 -- 172.21.15.26:0/2213789295 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf84027070 con 0x55dec56bdb60 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf737fe640 1 -- 172.21.15.26:0/2213789295 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84003a20 con 0x55dec56bdb60 2024-04-24T23:26:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf96c56740 1 -- 172.21.15.26:0/2213789295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56bdb60 msgr2=0x55dec56704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf96c56740 1 --2- 172.21.15.26:0/2213789295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56bdb60 0x55dec56704f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbf84011f20 tx=0x7fbf84014000 comp rx=0 tx=0).stop 2024-04-24T23:26:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf96c56740 1 -- 172.21.15.26:0/2213789295 shutdown_connections 2024-04-24T23:26:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf96c56740 1 --2- 172.21.15.26:0/2213789295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0540 0x55dec5675350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf96c56740 1 --2- 172.21.15.26:0/2213789295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a0c80 0x55dec56a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf96c56740 1 --2- 172.21.15.26:0/2213789295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56bdb60 0x55dec56704f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf96c56740 1 -- 172.21.15.26:0/2213789295 >> 172.21.15.26:0/2213789295 conn(0x55dec5672760 msgr2=0x55dec56bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.318+0000 7fbf96c56740 1 -- 172.21.15.26:0/2213789295 shutdown_connections 2024-04-24T23:26:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.319+0000 7fbf96c56740 1 -- 172.21.15.26:0/2213789295 wait complete. 2024-04-24T23:26:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.319+0000 7fbf96c56740 1 -- start start 2024-04-24T23:26:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.319+0000 7fbf96c56740 1 Processor -- start 2024-04-24T23:26:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0c80 0x55dec56cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a3380 0x55dec56a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf96c56740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dec5674910 con 0x55dec56bdb60 2024-04-24T23:26:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf96c56740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dec5594cf0 con 0x55dec56a3380 2024-04-24T23:26:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf96c56740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55dec5673d00 con 0x55dec56a0c80 2024-04-24T23:26:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf96c56740 10 client.0 _ll_register_callbacks cb 0x55dec55946e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf914ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf924ec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a3380 0x55dec56a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf91ceb640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0c80 0x55dec56cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf914ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60672/0 (socket says 172.21.15.26:60672) 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf914ea640 1 -- 172.21.15.26:0/496397730 learned_addr learned my addr 172.21.15.26:0/496397730 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf914ea640 1 -- 172.21.15.26:0/496397730 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0c80 msgr2=0x55dec56cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf914ea640 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0c80 0x55dec56cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf914ea640 1 -- 172.21.15.26:0/496397730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a3380 msgr2=0x55dec56a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf914ea640 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a3380 0x55dec56a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf914ea640 1 -- 172.21.15.26:0/496397730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec53b76b0 con 0x55dec56bdb60 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.320+0000 7fbf924ec640 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a3380 0x55dec56a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf914ea640 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56a2e40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fbf800098c0 tx=0x7fbf80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aedb3876b96a621 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf70ff9640 1 -- 172.21.15.26:0/496397730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80037040 con 0x55dec56bdb60 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf70ff9640 1 -- 172.21.15.26:0/496397730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fbf8003c820 con 0x55dec56bdb60 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf96c56740 1 -- 172.21.15.26:0/496397730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf84027070 con 0x55dec56bdb60 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf70ff9640 1 -- 172.21.15.26:0/496397730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fbf8004f050 con 0x55dec56bdb60 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf96c56740 10 client.8070 fetch_fsmap learned FSMap version 89 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf96c56740 1 -- 172.21.15.26:0/496397730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55dec56cd350 con 0x55dec56bdb60 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf70ff9640 1 -- 172.21.15.26:0/496397730 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf84027070 con 0x55dec56bdb60 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf70ff9640 1 -- 172.21.15.26:0/496397730 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) ==== 3534+0+0 (secure 0 0 0) 0x7fbf80052020 con 0x55dec56bdb60 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf96c56740 10 client.8070 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf96c56740 10 client.8070 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:26:33.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf96c56740 20 client.8070 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:33.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf96c56740 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55dec56d4ae0 0x55dec56d8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf96c56740 4 client.8070 mds_command: new command op to 6778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:33.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.321+0000 7fbf96c56740 1 -- 172.21.15.26:0/496397730 --> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dec55e6440 con 0x55dec56d4ae0 2024-04-24T23:26:33.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.322+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55dec56d4ae0 0x55dec56d8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.322+0000 7fbf70ff9640 10 client.8070 ms_handle_connect on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:33.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.322+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55dec56d4ae0 0x55dec56d8df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7318cde6768a366f server_cookie=5bbdf05836e627c9 in_seq=0 out_seq=0 2024-04-24T23:26:33.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.322+0000 7fbf70ff9640 1 -- 172.21.15.26:0/496397730 <== mds.1 v2:172.21.15.26:6834/306957409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dec55e6440 con 0x55dec56d4ae0 2024-04-24T23:26:33.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.322+0000 7fbf70ff9640 10 client.8070 handle_command_reply: tid=0 2024-04-24T23:26:33.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.322+0000 7fbf96c56740 1 client.8070 shutdown 2024-04-24T23:26:33.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 -- 172.21.15.26:0/496397730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 msgr2=0x55dec56a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56a2e40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fbf800098c0 tx=0x7fbf80004730 comp rx=0 tx=0).stop 2024-04-24T23:26:33.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 -- 172.21.15.26:0/496397730 shutdown_connections 2024-04-24T23:26:33.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55dec56d4ae0 0x55dec56d8df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a3380 0x55dec56a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56a2e40 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf70ff9640 0 client.8070 ms_handle_reset on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:33.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 --2- 172.21.15.26:0/496397730 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a0c80 0x55dec56cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 -- 172.21.15.26:0/496397730 >> 172.21.15.26:0/496397730 conn(0x55dec56a0540 msgr2=0x55dec56be420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 -- 172.21.15.26:0/496397730 shutdown_connections 2024-04-24T23:26:33.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 1 -- 172.21.15.26:0/496397730 wait complete. 2024-04-24T23:26:33.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.323+0000 7fbf96c56740 20 client.8070 trim_cache size 0 max 16384 2024-04-24T23:26:33.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.341+0000 7fbf96c56740 1 Processor -- start 2024-04-24T23:26:33.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.341+0000 7fbf96c56740 1 -- start start 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec569b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a3380 0x55dec56cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56bdb60 0x55dec56cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf96c56740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c1ff700 con 0x55dec56a3380 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf96c56740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c1ffdd0 con 0x55dec56a0c80 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf96c56740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c0b5f10 con 0x55dec56bdb60 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf914ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a3380 0x55dec56cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf914ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a3380 0x55dec56cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60678/0 (socket says 172.21.15.26:60678) 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf914ea640 1 -- 172.21.15.26:0/2581967846 learned_addr learned my addr 172.21.15.26:0/2581967846 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf91ceb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec569b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf924ec640 1 --2- 172.21.15.26:0/2581967846 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56bdb60 0x55dec56cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf914ea640 1 -- 172.21.15.26:0/2581967846 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56bdb60 msgr2=0x55dec56cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf914ea640 1 --2- 172.21.15.26:0/2581967846 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56bdb60 0x55dec56cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf914ea640 1 -- 172.21.15.26:0/2581967846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 msgr2=0x55dec569b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf914ea640 1 --2- 172.21.15.26:0/2581967846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec569b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf914ea640 1 -- 172.21.15.26:0/2581967846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec56738a0 con 0x55dec56a3380 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.342+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/2581967846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec569b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf914ea640 1 --2- 172.21.15.26:0/2581967846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a3380 0x55dec56cd6a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fbf80006dc0 tx=0x7fbf8003e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89abf4e34a074b60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf72ffd640 1 -- 172.21.15.26:0/2581967846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80058070 con 0x55dec56a3380 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf72ffd640 1 -- 172.21.15.26:0/2581967846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8003c820 con 0x55dec56a3380 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf72ffd640 1 -- 172.21.15.26:0/2581967846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80037030 con 0x55dec56a3380 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf96c56740 1 -- 172.21.15.26:0/2581967846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a3380 msgr2=0x55dec56cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf96c56740 1 --2- 172.21.15.26:0/2581967846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a3380 0x55dec56cd6a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fbf80006dc0 tx=0x7fbf8003e070 comp rx=0 tx=0).stop 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf96c56740 1 -- 172.21.15.26:0/2581967846 shutdown_connections 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf96c56740 1 --2- 172.21.15.26:0/2581967846 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56bdb60 0x55dec56cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf96c56740 1 --2- 172.21.15.26:0/2581967846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56a3380 0x55dec56cd6a0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf96c56740 1 --2- 172.21.15.26:0/2581967846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec569b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf96c56740 1 -- 172.21.15.26:0/2581967846 >> 172.21.15.26:0/2581967846 conn(0x55dec56a0540 msgr2=0x55dec56b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf96c56740 1 -- 172.21.15.26:0/2581967846 shutdown_connections 2024-04-24T23:26:33.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.343+0000 7fbf96c56740 1 -- 172.21.15.26:0/2581967846 wait complete. 2024-04-24T23:26:33.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.344+0000 7fbf96c56740 1 -- start start 2024-04-24T23:26:33.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.344+0000 7fbf96c56740 1 Processor -- start 2024-04-24T23:26:33.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec5734c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a3380 0x55dec56b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf96c56740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56b9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf96c56740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dec56d4960 con 0x55dec56bdb60 2024-04-24T23:26:33.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf96c56740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dec56d4530 con 0x55dec56a0c80 2024-04-24T23:26:33.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf96c56740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55dec56d4fb0 con 0x55dec56a3380 2024-04-24T23:26:33.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf96c56740 10 client.0 _ll_register_callbacks cb 0x7fbf80032bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:33.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf924ec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf914ea640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a3380 0x55dec56b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf91ceb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec5734c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf924ec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56b9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60694/0 (socket says 172.21.15.26:60694) 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf924ec640 1 -- 172.21.15.26:0/1327129244 learned_addr learned my addr 172.21.15.26:0/1327129244 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf924ec640 1 -- 172.21.15.26:0/1327129244 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a3380 msgr2=0x55dec56b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf924ec640 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a3380 0x55dec56b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf924ec640 1 -- 172.21.15.26:0/1327129244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 msgr2=0x55dec5734c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf924ec640 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec5734c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf924ec640 1 -- 172.21.15.26:0/1327129244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec56b1a20 con 0x55dec56bdb60 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec5734c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf924ec640 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56b9520 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c055880 tx=0x7fbf7c061af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbde6af99f49757 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.345+0000 7fbf70ff9640 1 -- 172.21.15.26:0/1327129244 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c004d60 con 0x55dec56bdb60 2024-04-24T23:26:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf70ff9640 1 -- 172.21.15.26:0/1327129244 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fbf7c058350 con 0x55dec56bdb60 2024-04-24T23:26:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf96c56740 1 -- 172.21.15.26:0/1327129244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8003c820 con 0x55dec56bdb60 2024-04-24T23:26:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf70ff9640 1 -- 172.21.15.26:0/1327129244 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fbf7c058570 con 0x55dec56bdb60 2024-04-24T23:26:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf96c56740 10 client.8076 fetch_fsmap learned FSMap version 89 2024-04-24T23:26:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf96c56740 1 -- 172.21.15.26:0/1327129244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55dec56b9e70 con 0x55dec56bdb60 2024-04-24T23:26:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf70ff9640 1 -- 172.21.15.26:0/1327129244 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8003c820 con 0x55dec56bdb60 2024-04-24T23:26:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf70ff9640 1 -- 172.21.15.26:0/1327129244 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) ==== 3534+0+0 (secure 0 0 0) 0x7fbf7c05f400 con 0x55dec56bdb60 2024-04-24T23:26:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf96c56740 10 client.8076 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T23:26:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf96c56740 10 client.8076 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T23:26:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf96c56740 20 client.8076 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf96c56740 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55dec569c760 0x55dec569cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf96c56740 4 client.8076 mds_command: new command op to 6778 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf96c56740 1 -- 172.21.15.26:0/1327129244 --> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] -- command(tid 0: {"prefix": "status"}) -- 0x55dec5671bb0 con 0x55dec569c760 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55dec569c760 0x55dec569cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf70ff9640 10 client.8076 ms_handle_connect on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.346+0000 7fbf91ceb640 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55dec569c760 0x55dec569cb40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88266aa083ad4b18 server_cookie=2781246a00f77076 in_seq=0 out_seq=0 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf70ff9640 1 -- 172.21.15.26:0/1327129244 <== mds.1 v2:172.21.15.26:6834/306957409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55dec5671bb0 con 0x55dec569c760 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf70ff9640 10 client.8076 handle_command_reply: tid=0 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 client.8076 shutdown 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 -- 172.21.15.26:0/1327129244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 msgr2=0x55dec56b9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56b9520 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c055880 tx=0x7fbf7c061af0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 -- 172.21.15.26:0/1327129244 shutdown_connections 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55dec569c760 0x55dec569cb40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec56bdb60 0x55dec56b9520 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55dec56a3380 0x55dec56b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 --2- 172.21.15.26:0/1327129244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dec56a0c80 0x55dec5734c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 -- 172.21.15.26:0/1327129244 >> 172.21.15.26:0/1327129244 conn(0x55dec56a0540 msgr2=0x55dec566f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf70ff9640 0 client.8076 ms_handle_reset on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 -- 172.21.15.26:0/1327129244 shutdown_connections 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 1 -- 172.21.15.26:0/1327129244 wait complete. 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf96c56740 20 client.8076 trim_cache size 0 max 16384 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:26:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6778, 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 2.8915004130000002, 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 89, 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 82, 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 20.196395978000002 2024-04-24T23:26:33.357 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf93f4f640 1 -- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbf7806be30 msgr2=0x7fbf7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf93f4f640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbf7806be30 0x7fbf7806e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009560 tx=0x7fbf84009190 comp rx=0 tx=0).stop 2024-04-24T23:26:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf93f4f640 1 -- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf8c15e280 msgr2=0x7fbf8c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf93f4f640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf8c15e280 0x7fbf8c0b8ce0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00caa0 tx=0x7fbf7c00cf60 comp rx=0 tx=0).stop 2024-04-24T23:26:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf93f4f640 1 -- 172.21.15.26:0/94192131 shutdown_connections 2024-04-24T23:26:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf93f4f640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbf7806be30 0x7fbf7806e2f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf93f4f640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf8c15e280 0x7fbf8c0b8ce0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.347+0000 7fbf93f4f640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf8c0cfe80 0x7fbf8c0b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.348+0000 7fbf93f4f640 1 --2- 172.21.15.26:0/94192131 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf8c0b4050 0x7fbf8c0b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.348+0000 7fbf93f4f640 1 -- 172.21.15.26:0/94192131 >> 172.21.15.26:0/94192131 conn(0x7fbf8c08dbd0 msgr2=0x7fbf8c0d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.348+0000 7fbf93f4f640 1 -- 172.21.15.26:0/94192131 shutdown_connections 2024-04-24T23:26:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.348+0000 7fbf93f4f640 1 -- 172.21.15.26:0/94192131 wait complete. 2024-04-24T23:26:33.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:26:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.460+0000 7f1e9bc31640 1 Processor -- start 2024-04-24T23:26:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.460+0000 7f1e9bc31640 1 -- start start 2024-04-24T23:26:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.461+0000 7f1e9bc31640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e940b3130 0x7f1e940b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.461+0000 7f1e9bc31640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e940d32f0 con 0x7f1e9415e570 2024-04-24T23:26:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.461+0000 7f1e9bc31640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e940d0270 con 0x7f1e940d4870 2024-04-24T23:26:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.461+0000 7f1e9bc31640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1e940d1ab0 con 0x7f1e940b3130 2024-04-24T23:26:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.461+0000 7f1e9a1ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e940b3130 0x7f1e940b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.461+0000 7f1e999cd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1e9415e570 0x7f1e940d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58422/0 (socket says 172.21.15.26:58422) 2024-04-24T23:26:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.461+0000 7f1e999cd640 1 -- 172.21.15.26:0/2240577033 learned_addr learned my addr 172.21.15.26:0/2240577033 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.462+0000 7f1e989cb640 1 -- 172.21.15.26:0/2240577033 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234932364 0 0) 0x7f1e940d0270 con 0x7f1e940d4870 2024-04-24T23:26:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.462+0000 7f1e9a1ce640 1 -- 172.21.15.26:0/2240577033 >> v1:172.21.15.29:6789/0 conn(0x7f1e940d4870 legacy=0x7f1e940b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.462+0000 7f1e9a1ce640 1 -- 172.21.15.26:0/2240577033 >> v1:172.21.15.26:6789/0 conn(0x7f1e9415e570 legacy=0x7f1e940d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.462+0000 7f1e9a1ce640 1 -- 172.21.15.26:0/2240577033 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e940d2d30 con 0x7f1e940b3130 2024-04-24T23:26:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.462+0000 7f1e9a1ce640 1 --2- 172.21.15.26:0/2240577033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940b3130 0x7f1e940b4810 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009d90 tx=0x7f1e8402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9d17fb74b0faed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e989cb640 1 -- 172.21.15.26:0/2240577033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84007740 con 0x7f1e940b3130 2024-04-24T23:26:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e989cb640 1 -- 172.21.15.26:0/2240577033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e840078e0 con 0x7f1e940b3130 2024-04-24T23:26:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e989cb640 1 -- 172.21.15.26:0/2240577033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84007cb0 con 0x7f1e940b3130 2024-04-24T23:26:33.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e9bc31640 1 -- 172.21.15.26:0/2240577033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940b3130 msgr2=0x7f1e940b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e9bc31640 1 --2- 172.21.15.26:0/2240577033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940b3130 0x7f1e940b4810 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009d90 tx=0x7f1e8402a560 comp rx=0 tx=0).stop 2024-04-24T23:26:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e9bc31640 1 -- 172.21.15.26:0/2240577033 shutdown_connections 2024-04-24T23:26:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e9bc31640 1 --2- 172.21.15.26:0/2240577033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940b3130 0x7f1e940b4810 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e9bc31640 1 -- 172.21.15.26:0/2240577033 >> 172.21.15.26:0/2240577033 conn(0x7f1e9408dbe0 msgr2=0x7f1e9409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e9bc31640 1 -- 172.21.15.26:0/2240577033 shutdown_connections 2024-04-24T23:26:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.463+0000 7f1e9bc31640 1 -- 172.21.15.26:0/2240577033 wait complete. 2024-04-24T23:26:33.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.464+0000 7f1e9bc31640 1 Processor -- start 2024-04-24T23:26:33.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e9bc31640 1 -- start start 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e9bc31640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940b3130 0x7f1e9415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e9bc31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940d4870 0x7f1e941527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e9bc31640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e9415e570 0x7f1e941f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e9bc31640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e940cfa70 con 0x7f1e940d4870 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e9bc31640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e940d2ae0 con 0x7f1e9415e570 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e9bc31640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1e940cfd50 con 0x7f1e940b3130 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e991cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940d4870 0x7f1e941527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e991cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940d4870 0x7f1e941527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60714/0 (socket says 172.21.15.26:60714) 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e991cc640 1 -- 172.21.15.26:0/1264173728 learned_addr learned my addr 172.21.15.26:0/1264173728 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e999cd640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940b3130 0x7f1e9415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.465+0000 7f1e9a1ce640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e9415e570 0x7f1e941f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.466+0000 7f1e991cc640 1 -- 172.21.15.26:0/1264173728 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940b3130 msgr2=0x7f1e9415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.466+0000 7f1e991cc640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940b3130 0x7f1e9415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.466+0000 7f1e991cc640 1 -- 172.21.15.26:0/1264173728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e9415e570 msgr2=0x7f1e941f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.466+0000 7f1e991cc640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e9415e570 0x7f1e941f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.466+0000 7f1e991cc640 1 -- 172.21.15.26:0/1264173728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e940b9ac0 con 0x7f1e940d4870 2024-04-24T23:26:33.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.466+0000 7f1e991cc640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940d4870 0x7f1e941527e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c4b0 tx=0x7f1e8800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.467+0000 7f1e927fc640 1 -- 172.21.15.26:0/1264173728 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88017070 con 0x7f1e940d4870 2024-04-24T23:26:33.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.467+0000 7f1e927fc640 1 -- 172.21.15.26:0/1264173728 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e88005480 con 0x7f1e940d4870 2024-04-24T23:26:33.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.467+0000 7f1e9bc31640 1 -- 172.21.15.26:0/1264173728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e940b9da0 con 0x7f1e940d4870 2024-04-24T23:26:33.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.468+0000 7f1e927fc640 1 -- 172.21.15.26:0/1264173728 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88003830 con 0x7f1e940d4870 2024-04-24T23:26:33.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.468+0000 7f1e9bc31640 1 -- 172.21.15.26:0/1264173728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e9415fdf0 con 0x7f1e940d4870 2024-04-24T23:26:33.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.469+0000 7f1e927fc640 1 -- 172.21.15.26:0/1264173728 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e8801a070 con 0x7f1e940d4870 2024-04-24T23:26:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.469+0000 7f1e927fc640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be60 0x7f1e8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.469+0000 7f1e927fc640 1 -- 172.21.15.26:0/1264173728 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f1e8808a0c0 con 0x7f1e940d4870 2024-04-24T23:26:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.469+0000 7f1e999cd640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be60 0x7f1e8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.470+0000 7f1e999cd640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be60 0x7f1e8006e320 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c000f60 tx=0x7f1e8c007e20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.471+0000 7f1e9ea1c740 1 Processor -- start 2024-04-24T23:26:33.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.471+0000 7f1e9ea1c740 1 -- start start 2024-04-24T23:26:33.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.471+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x5618268c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.471+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.471+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7540 0x5618268cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.471+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56182682ff00 con 0x561826914b60 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.471+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5618267d8750 con 0x5618268f7540 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.471+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56182682bec0 con 0x5618268f7c80 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e999cd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x5618268c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e999cd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x5618268c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60720/0 (socket says 172.21.15.26:60720) 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e999cd640 1 -- 172.21.15.26:0/785259052 learned_addr learned my addr 172.21.15.26:0/785259052 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e991cc640 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e9a1ce640 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7540 0x5618268cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e999cd640 1 -- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 msgr2=0x5618268f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e999cd640 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e999cd640 1 -- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7540 msgr2=0x5618268cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e999cd640 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7540 0x5618268cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e999cd640 1 -- 172.21.15.26:0/785259052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618265442d0 con 0x561826914b60 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e991cc640 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e9a1ce640 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7540 0x5618268cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e999cd640 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x5618268c74f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c001b80 tx=0x7f1e8c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec67ea1682d6f2a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/785259052 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c026070 con 0x561826914b60 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/785259052 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c013e00 con 0x561826914b60 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.472+0000 7f1e7b7fe640 1 -- 172.21.15.26:0/785259052 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c012550 con 0x561826914b60 2024-04-24T23:26:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.473+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/785259052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 msgr2=0x5618268c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.473+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x5618268c74f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c001b80 tx=0x7f1e8c013000 comp rx=0 tx=0).stop 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.473+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/785259052 shutdown_connections 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.473+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7540 0x5618268cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.473+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.473+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/785259052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x5618268c74f0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.473+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/785259052 >> 172.21.15.26:0/785259052 conn(0x5618268c9760 msgr2=0x561826912000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.473+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/785259052 shutdown_connections 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.473+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/785259052 wait complete. 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.474+0000 7f1e9ea1c740 1 -- start start 2024-04-24T23:26:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.474+0000 7f1e9ea1c740 1 Processor -- start 2024-04-24T23:26:33.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x561826909cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182690a200 0x561826913130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5618268cb910 con 0x561826914b60 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5618267ebcf0 con 0x56182690a200 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5618268cad00 con 0x5618268f7c80 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 10 client.0 _ll_register_callbacks cb 0x5618267eb6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e991cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x561826909cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e991cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x561826909cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60732/0 (socket says 172.21.15.26:60732) 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e991cc640 1 -- 172.21.15.26:0/3134025662 learned_addr learned my addr 172.21.15.26:0/3134025662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9a1ce640 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182690a200 0x561826913130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e999cd640 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e991cc640 1 -- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 msgr2=0x5618268f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e991cc640 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e991cc640 1 -- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182690a200 msgr2=0x561826913130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e991cc640 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182690a200 0x561826913130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e991cc640 1 -- 172.21.15.26:0/3134025662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182660e6b0 con 0x561826914b60 2024-04-24T23:26:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9a1ce640 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182690a200 0x561826913130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e999cd640 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e991cc640 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x561826909cc0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800ce70 tx=0x7f1e8805c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b347c9780edb69c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3134025662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88070180 con 0x561826914b60 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3134025662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f1e88070320 con 0x561826914b60 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3134025662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e8c013e00 con 0x561826914b60 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3134025662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f1e88088120 con 0x561826914b60 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 10 client.8091 fetch_fsmap learned FSMap version 89 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.475+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3134025662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x5618268f88d0 con 0x561826914b60 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3134025662 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c013e00 con 0x561826914b60 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3134025662 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) ==== 3534+0+0 (secure 0 0 0) 0x7f1e88057900 con 0x561826914b60 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e9ea1c740 10 client.8091 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T23:26:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e9ea1c740 10 client.8091 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e9ea1c740 20 client.8091 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x561826925a00 0x5618268f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e9ea1c740 4 client.8091 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3134025662 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56182683d440 con 0x561826925a00 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e999cd640 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x561826925a00 0x5618268f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e78ff9640 10 client.8091 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.476+0000 7f1e999cd640 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x561826925a00 0x5618268f9c30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2056d195c68abbb server_cookie=ed2bd7eafe2d7058 in_seq=0 out_seq=0 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3134025662 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56182683d440 con 0x561826925a00 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e78ff9640 10 client.8091 handle_command_reply: tid=0 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 client.8091 shutdown 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3134025662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 msgr2=0x561826909cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x561826909cc0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800ce70 tx=0x7f1e8805c870 comp rx=0 tx=0).stop 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3134025662 shutdown_connections 2024-04-24T23:26:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x561826925a00 0x5618268f9c30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56182690a200 0x561826913130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561826914b60 0x561826909cc0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e78ff9640 0 client.8091 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3134025662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x5618268f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3134025662 >> 172.21.15.26:0/3134025662 conn(0x5618268f7540 msgr2=0x5618269140a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3134025662 shutdown_connections 2024-04-24T23:26:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3134025662 wait complete. 2024-04-24T23:26:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.477+0000 7f1e9ea1c740 20 client.8091 trim_cache size 0 max 16384 2024-04-24T23:26:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.495+0000 7f1e9ea1c740 1 Processor -- start 2024-04-24T23:26:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e9ea1c740 1 -- start start 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x561826912610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x561826913d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561826914b60 0x5618268c6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e941ff9d0 con 0x56182690a200 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e941fff40 con 0x561826914b60 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1e94160af0 con 0x5618268f7c80 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e991cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x561826913d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e991cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x561826913d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60746/0 (socket says 172.21.15.26:60746) 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e991cc640 1 -- 172.21.15.26:0/1651269910 learned_addr learned my addr 172.21.15.26:0/1651269910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.496+0000 7f1e991cc640 1 -- 172.21.15.26:0/1651269910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 msgr2=0x561826912610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e991cc640 1 --2- 172.21.15.26:0/1651269910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x561826912610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e991cc640 1 -- 172.21.15.26:0/1651269910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561826914b60 msgr2=0x5618268c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e991cc640 1 --2- 172.21.15.26:0/1651269910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561826914b60 0x5618268c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e991cc640 1 -- 172.21.15.26:0/1651269910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182690aeb0 con 0x56182690a200 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e991cc640 1 --2- 172.21.15.26:0/1651269910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x561826913d30 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1e88000f60 tx=0x7f1e88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873dfc184fed7d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e7affd640 1 -- 172.21.15.26:0/1651269910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880882c0 con 0x56182690a200 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e7affd640 1 -- 172.21.15.26:0/1651269910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e88057900 con 0x56182690a200 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e7affd640 1 -- 172.21.15.26:0/1651269910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88070180 con 0x56182690a200 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/1651269910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 msgr2=0x561826913d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/1651269910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x561826913d30 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1e88000f60 tx=0x7f1e88014000 comp rx=0 tx=0).stop 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/1651269910 shutdown_connections 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/1651269910 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561826914b60 0x5618268c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/1651269910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x561826913d30 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/1651269910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5618268f7c80 0x561826912610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/1651269910 >> 172.21.15.26:0/1651269910 conn(0x5618268f7540 msgr2=0x5618269099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/1651269910 shutdown_connections 2024-04-24T23:26:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.497+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/1651269910 wait complete. 2024-04-24T23:26:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.498+0000 7f1e9ea1c740 1 -- start start 2024-04-24T23:26:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.498+0000 7f1e9ea1c740 1 Processor -- start 2024-04-24T23:26:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7c80 0x5618268eec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x5618268f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e9ea1c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561826914b60 0x561826912090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561826925880 con 0x56182690a200 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5618268f8c70 con 0x5618268f7c80 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e9ea1c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561826925ed0 con 0x561826914b60 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e9ea1c740 10 client.0 _ll_register_callbacks cb 0x7f1e880584b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e991cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x5618268f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e991cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x5618268f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60756/0 (socket says 172.21.15.26:60756) 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e991cc640 1 -- 172.21.15.26:0/3432724498 learned_addr learned my addr 172.21.15.26:0/3432724498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e9a1ce640 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561826914b60 0x561826912090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e991cc640 1 -- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561826914b60 msgr2=0x561826912090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e999cd640 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7c80 0x5618268eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e991cc640 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561826914b60 0x561826912090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e991cc640 1 -- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7c80 msgr2=0x5618268eec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e991cc640 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7c80 0x5618268eec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e991cc640 1 -- 172.21.15.26:0/3432724498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618269142e0 con 0x56182690a200 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e9a1ce640 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561826914b60 0x561826912090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.499+0000 7f1e999cd640 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7c80 0x5618268eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e991cc640 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x5618268f41d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1e88058440 tx=0x7f1e88088550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795e2daf469bc7f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3432724498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88058280 con 0x56182690a200 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3432724498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f1e88070380 con 0x56182690a200 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3432724498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e88057900 con 0x56182690a200 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3432724498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f1e88060fe0 con 0x56182690a200 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e9ea1c740 10 client.8097 fetch_fsmap learned FSMap version 89 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3432724498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x5618268f39d0 con 0x56182690a200 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3432724498 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e88057900 con 0x56182690a200 2024-04-24T23:26:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3432724498 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 89) ==== 3534+0+0 (secure 0 0 0) 0x7f1e88054200 con 0x56182690a200 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e9ea1c740 10 client.8097 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e9ea1c740 10 client.8097 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e9ea1c740 20 client.8097 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x561826913240 0x5618269125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e9ea1c740 4 client.8097 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3432724498 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "status"}) -- 0x5618268ce930 con 0x561826913240 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.500+0000 7f1e999cd640 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x561826913240 0x5618269125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e78ff9640 10 client.8097 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e999cd640 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x561826913240 0x5618269125d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef62204868fdcdd server_cookie=bf6057c01ec3b0a4 in_seq=0 out_seq=0 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e78ff9640 1 -- 172.21.15.26:0/3432724498 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5618268ce930 con 0x561826913240 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e78ff9640 10 client.8097 handle_command_reply: tid=0 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 client.8097 shutdown 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3432724498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 msgr2=0x5618268f41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x5618268f41d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1e88058440 tx=0x7f1e88088550 comp rx=0 tx=0).stop 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3432724498 shutdown_connections 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x561826913240 0x5618269125d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561826914b60 0x561826912090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56182690a200 0x5618268f41d0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e78ff9640 0 client.8097 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 --2- 172.21.15.26:0/3432724498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5618268f7c80 0x5618268eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3432724498 >> 172.21.15.26:0/3432724498 conn(0x5618268f7540 msgr2=0x561826935da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3432724498 shutdown_connections 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 1 -- 172.21.15.26:0/3432724498 wait complete. 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.501+0000 7f1e9ea1c740 20 client.8097 trim_cache size 0 max 16384 2024-04-24T23:26:33.509 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6790, 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 15.338668584000001, 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 89, 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 82, 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 82, 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 20.340800208000001 2024-04-24T23:26:33.510 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:33.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 -- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be60 msgr2=0x7f1e8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be60 0x7f1e8006e320 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c000f60 tx=0x7f1e8c007e20 comp rx=0 tx=0).stop 2024-04-24T23:26:33.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 -- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940d4870 msgr2=0x7f1e941527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940d4870 0x7f1e941527e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c4b0 tx=0x7f1e8800c970 comp rx=0 tx=0).stop 2024-04-24T23:26:33.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 -- 172.21.15.26:0/1264173728 shutdown_connections 2024-04-24T23:26:33.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e8006be60 0x7f1e8006e320 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e9415e570 0x7f1e941f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e940d4870 0x7f1e941527e0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 --2- 172.21.15.26:0/1264173728 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e940b3130 0x7f1e9415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 -- 172.21.15.26:0/1264173728 >> 172.21.15.26:0/1264173728 conn(0x7f1e9408dbe0 msgr2=0x7f1e940ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 -- 172.21.15.26:0/1264173728 shutdown_connections 2024-04-24T23:26:33.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.502+0000 7f1e9bc31640 1 -- 172.21.15.26:0/1264173728 wait complete. 2024-04-24T23:26:33.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:33.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.613+0000 7f617bd10640 1 Processor -- start 2024-04-24T23:26:33.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.614+0000 7f617bd10640 1 -- start start 2024-04-24T23:26:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.614+0000 7f617bd10640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f617406e2f0 0x7f6174165680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.614+0000 7f617bd10640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61740b2e80 con 0x7f617415a480 2024-04-24T23:26:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.614+0000 7f617bd10640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61740b3120 con 0x7f617415a860 2024-04-24T23:26:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.614+0000 7f617bd10640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f61740b33c0 con 0x7f617406e2f0 2024-04-24T23:26:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.614+0000 7f6179aac640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f617415a480 0x7f61740b48f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58424/0 (socket says 172.21.15.26:58424) 2024-04-24T23:26:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6179aac640 1 -- 172.21.15.26:0/901836312 learned_addr learned my addr 172.21.15.26:0/901836312 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f617a2ad640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f617406e2f0 0x7f6174165680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 178519582 0 0) 0x7f61740b2e80 con 0x7f617415a480 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6160003520 con 0x7f617415a480 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1391557984 0 0) 0x7f6160003520 con 0x7f617415a480 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f617406e2f0 msgr2=0x7f6174165680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f617406e2f0 0x7f6174165680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 >> v1:172.21.15.29:6789/0 conn(0x7f617415a860 legacy=0x7f617406dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61740b36a0 con 0x7f617415a480 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f6170001670 con 0x7f617415a480 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 >> v1:172.21.15.26:6789/0 conn(0x7f617415a480 legacy=0x7f61740b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f617406e2f0 0x7f61600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f6160004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6160004d40 0x7f61600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f61740b33c0 con 0x7f6160003f60 2024-04-24T23:26:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f616000bb50 con 0x7f6160004d40 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f616000bd50 con 0x7f617406e2f0 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f61792ab640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f6160004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f6179aac640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f617406e2f0 0x7f61600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.615+0000 7f617a2ad640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6160004d40 0x7f61600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.616+0000 7f61792ab640 1 -- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f617406e2f0 msgr2=0x7f61600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.616+0000 7f61792ab640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f617406e2f0 0x7f61600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.616+0000 7f61792ab640 1 -- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6160004d40 msgr2=0x7f61600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.616+0000 7f61792ab640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6160004d40 0x7f61600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.616+0000 7f61792ab640 1 -- 172.21.15.26:0/901836312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6160003520 con 0x7f6160003f60 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.616+0000 7f61792ab640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f6160004800 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f6164001720 tx=0x7f6160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5f7346c9970622 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.616+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164001370 con 0x7f6160003f60 2024-04-24T23:26:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.616+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61640052d0 con 0x7f6160003f60 2024-04-24T23:26:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.616+0000 7f6178aaa640 1 -- 172.21.15.26:0/901836312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616402abc0 con 0x7f6160003f60 2024-04-24T23:26:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.617+0000 7f617bd10640 1 -- 172.21.15.26:0/901836312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 msgr2=0x7f6160004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.617+0000 7f617bd10640 1 --2- 172.21.15.26:0/901836312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f6160004800 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f6164001720 tx=0x7f6160005830 comp rx=0 tx=0).stop 2024-04-24T23:26:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.617+0000 7f617a2ad640 1 -- 172.21.15.26:0/901836312 reap_dead start 2024-04-24T23:26:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.617+0000 7f617bd10640 1 -- 172.21.15.26:0/901836312 shutdown_connections 2024-04-24T23:26:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.617+0000 7f617bd10640 1 -- 172.21.15.26:0/901836312 >> 172.21.15.26:0/901836312 conn(0x7f617406b300 msgr2=0x7f617406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.617+0000 7f617bd10640 1 -- 172.21.15.26:0/901836312 shutdown_connections 2024-04-24T23:26:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.617+0000 7f617bd10640 1 -- 172.21.15.26:0/901836312 wait complete. 2024-04-24T23:26:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.618+0000 7f617bd10640 1 Processor -- start 2024-04-24T23:26:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.618+0000 7f617bd10640 1 -- start start 2024-04-24T23:26:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f617bd10640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f61741407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f617bd10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f617414c0d0 0x7f61740cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f617bd10640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f61740cf200 0x7f617414a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f617bd10640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f61740b5000 con 0x7f6160003f60 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f617bd10640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f61740b3c60 con 0x7f617414c0d0 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f617bd10640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f61740b5180 con 0x7f61740cf200 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f6179aac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f61741407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f6179aac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f61741407c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60776/0 (socket says 172.21.15.26:60776) 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f6179aac640 1 -- 172.21.15.26:0/682714902 learned_addr learned my addr 172.21.15.26:0/682714902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f61792ab640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f617414c0d0 0x7f61740cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f617a2ad640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f61740cf200 0x7f617414a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f6179aac640 1 -- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f61740cf200 msgr2=0x7f617414a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f6179aac640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f61740cf200 0x7f617414a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f6179aac640 1 -- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f617414c0d0 msgr2=0x7f61740cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f6179aac640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f617414c0d0 0x7f61740cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f6179aac640 1 -- 172.21.15.26:0/682714902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6174140d00 con 0x7f6160003f60 2024-04-24T23:26:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f61792ab640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f617414c0d0 0x7f61740cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.619+0000 7f617a2ad640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f61740cf200 0x7f617414a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.620+0000 7f6179aac640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f61741407c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f6170007290 tx=0x7f6170007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.620+0000 7f616a7fc640 1 -- 172.21.15.26:0/682714902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617000f960 con 0x7f6160003f60 2024-04-24T23:26:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.620+0000 7f616a7fc640 1 -- 172.21.15.26:0/682714902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6170018070 con 0x7f6160003f60 2024-04-24T23:26:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.620+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617414b030 con 0x7f6160003f60 2024-04-24T23:26:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.620+0000 7f616a7fc640 1 -- 172.21.15.26:0/682714902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617000fd60 con 0x7f6160003f60 2024-04-24T23:26:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.620+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617414b5f0 con 0x7f6160003f60 2024-04-24T23:26:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.621+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61740694c0 con 0x7f6160003f60 2024-04-24T23:26:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.621+0000 7f616a7fc640 1 -- 172.21.15.26:0/682714902 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f617001d460 con 0x7f6160003f60 2024-04-24T23:26:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.622+0000 7f616a7fc640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f615806be30 0x7f615806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.622+0000 7f616a7fc640 1 -- 172.21.15.26:0/682714902 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f617008af30 con 0x7f6160003f60 2024-04-24T23:26:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.622+0000 7f61792ab640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f615806be30 0x7f615806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.622+0000 7f61792ab640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f615806be30 0x7f615806e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6164001720 tx=0x7f616402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:33.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.626+0000 7f616a7fc640 1 -- 172.21.15.26:0/682714902 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6170059970 con 0x7f6160003f60 2024-04-24T23:26:33.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.847+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61740b44a0 con 0x7f6160003f60 2024-04-24T23:26:33.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.848+0000 7f616a7fc640 1 -- 172.21.15.26:0/682714902 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5218 (secure 0 0 0) 0x7f6170059270 con 0x7f6160003f60 2024-04-24T23:26:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:33.850 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:32.463770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6299,6778,6790]},"id":6}]} 2024-04-24T23:26:33.850 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 89 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f615806be30 msgr2=0x7f615806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f615806be30 0x7f615806e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6164001720 tx=0x7f616402e000 comp rx=0 tx=0).stop 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 msgr2=0x7f61741407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f61741407c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f6170007290 tx=0x7f6170007750 comp rx=0 tx=0).stop 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 shutdown_connections 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f615806be30 0x7f615806e2f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f61740cf200 0x7f617414a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f617414c0d0 0x7f61740cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 --2- 172.21.15.26:0/682714902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6160003f60 0x7f61741407c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 >> 172.21.15.26:0/682714902 conn(0x7f617406b300 msgr2=0x7f61740b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 shutdown_connections 2024-04-24T23:26:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:33.850+0000 7f617bd10640 1 -- 172.21.15.26:0/682714902 wait complete. 2024-04-24T23:26:33.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:33.863 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:26:33.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:33.887 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:26:33.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:33.951 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-04-24T23:26:34.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:34.006 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-04-24T23:26:34.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:34.063 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-04-24T23:26:34.118 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-04-24T23:26:34.242 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:26:34.243 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:26:34.243 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:26:36.244 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-04-24T23:26:36.373 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:26:36.374 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:26:36.374 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:26:38.376 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-04-24T23:26:38.510 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:26:38.511 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:26:38.511 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:26:40.512 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-04-24T23:26:40.642 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:26:40.643 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:26:40.643 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:26:42.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-04-24T23:26:42.774 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:26:42.775 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:26:42.775 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:26:44.776 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-04-24T23:26:44.905 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-04-24T23:26:44.906 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:26:44.906 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:26:44.907 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:26:45.036 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:26:45.037 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:26:45.037 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:26:47.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:26:47.169 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:26:47.169 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:26:47.169 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:26:49.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-04-24T23:26:49.301 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:26:49.302 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:26:49.302 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:26:49.302 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:26:49.432 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:26:49.432 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:26:49.433 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:26:49.433 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-04-24T23:26:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.541+0000 7f3301c61640 1 Processor -- start 2024-04-24T23:26:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.541+0000 7f3301c61640 1 -- start start 2024-04-24T23:26:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f3301c61640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f32fc0afec0 0x7f32fc06c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f3301c61640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f32fc06d4a0 0x7f32fc06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f3301c61640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32fc0d00a0 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f3301c61640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f32fc06b120 con 0x7f32fc06d4a0 2024-04-24T23:26:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f3301c61640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f32fc06b370 con 0x7f32fc0afec0 2024-04-24T23:26:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f32fbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f32fc06d4a0 0x7f32fc06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f32fbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f32fc06d4a0 0x7f32fc06d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40102/0 (socket says 172.21.15.26:40102) 2024-04-24T23:26:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f32fbfff640 1 -- 172.21.15.26:0/4258432906 learned_addr learned my addr 172.21.15.26:0/4258432906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f32faffd640 1 --2- 172.21.15.26:0/4258432906 >> v2:172.21.15.29:3300/0 conn(0x7f32fc0afec0 0x7f32fc06c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f32fbfff640 1 -- 172.21.15.26:0/4258432906 >> v2:172.21.15.29:3300/0 conn(0x7f32fc0afec0 msgr2=0x7f32fc06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f32fbfff640 1 --2- 172.21.15.26:0/4258432906 >> v2:172.21.15.29:3300/0 conn(0x7f32fc0afec0 0x7f32fc06c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f32fbfff640 1 -- 172.21.15.26:0/4258432906 >> v1:172.21.15.29:6789/0 conn(0x7f32fc0cfcc0 legacy=0x7f32fc15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.542+0000 7f32fbfff640 1 -- 172.21.15.26:0/4258432906 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc06de00 con 0x7f32fc06d4a0 2024-04-24T23:26:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.543+0000 7f32fbfff640 1 --2- 172.21.15.26:0/4258432906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc06d4a0 0x7f32fc06d8c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0b30d0 tx=0x7f32f402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310b76578adfb166 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.543+0000 7f32fa7fc640 1 -- 172.21.15.26:0/4258432906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4036070 con 0x7f32fc06d4a0 2024-04-24T23:26:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.543+0000 7f32fa7fc640 1 -- 172.21.15.26:0/4258432906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f4031600 con 0x7f32fc06d4a0 2024-04-24T23:26:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.543+0000 7f32fa7fc640 1 -- 172.21.15.26:0/4258432906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40319d0 con 0x7f32fc06d4a0 2024-04-24T23:26:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.544+0000 7f3301c61640 1 -- 172.21.15.26:0/4258432906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc06d4a0 msgr2=0x7f32fc06d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.544+0000 7f3301c61640 1 --2- 172.21.15.26:0/4258432906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc06d4a0 0x7f32fc06d8c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0b30d0 tx=0x7f32f402a6f0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.544+0000 7f3301c61640 1 -- 172.21.15.26:0/4258432906 shutdown_connections 2024-04-24T23:26:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.544+0000 7f3301c61640 1 --2- 172.21.15.26:0/4258432906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc06d4a0 0x7f32fc06d8c0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.544+0000 7f3301c61640 1 --2- 172.21.15.26:0/4258432906 >> v2:172.21.15.29:3300/0 conn(0x7f32fc0afec0 0x7f32fc06c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.544+0000 7f3301c61640 1 -- 172.21.15.26:0/4258432906 >> 172.21.15.26:0/4258432906 conn(0x7f32fc0b2970 msgr2=0x7f32fc0d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.544+0000 7f3301c61640 1 -- 172.21.15.26:0/4258432906 shutdown_connections 2024-04-24T23:26:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.544+0000 7f3301c61640 1 -- 172.21.15.26:0/4258432906 wait complete. 2024-04-24T23:26:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.545+0000 7f3301c61640 1 Processor -- start 2024-04-24T23:26:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.545+0000 7f3301c61640 1 -- start start 2024-04-24T23:26:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f3301c61640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32fc06d4a0 0x7f32fc153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f3301c61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc0cfcc0 0x7f32fc1e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f3301c61640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f32fc0b5490 0x7f32fc1e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f3301c61640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc06bc10 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f3301c61640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc06b930 con 0x7f32fc06d4a0 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f3301c61640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f32fc06bef0 con 0x7f32fc0b5490 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc0cfcc0 0x7f32fc1e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc0cfcc0 0x7f32fc1e7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40104/0 (socket says 172.21.15.26:40104) 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32faffd640 1 -- 172.21.15.26:0/2904930472 learned_addr learned my addr 172.21.15.26:0/2904930472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32fbfff640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f32fc0b5490 0x7f32fc1e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32fc06d4a0 0x7f32fc153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32faffd640 1 -- 172.21.15.26:0/2904930472 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f32fc0b5490 msgr2=0x7f32fc1e80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32faffd640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f32fc0b5490 0x7f32fc1e80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32faffd640 1 -- 172.21.15.26:0/2904930472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32fc06d4a0 msgr2=0x7f32fc153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32faffd640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32fc06d4a0 0x7f32fc153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.546+0000 7f32faffd640 1 -- 172.21.15.26:0/2904930472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc0b43f0 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.547+0000 7f32faffd640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc0cfcc0 0x7f32fc1e7b80 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c7f0 tx=0x7f32ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.547+0000 7f32e3fff640 1 -- 172.21.15.26:0/2904930472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec015070 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.547+0000 7f32e3fff640 1 -- 172.21.15.26:0/2904930472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ec004220 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.547+0000 7f3301c61640 1 -- 172.21.15.26:0/2904930472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc0b46d0 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.547+0000 7f32e3fff640 1 -- 172.21.15.26:0/2904930472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec007620 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.547+0000 7f3301c61640 1 -- 172.21.15.26:0/2904930472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc0b4c90 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.549+0000 7f32e3fff640 1 -- 172.21.15.26:0/2904930472 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f32ec0077c0 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.549+0000 7f32e3fff640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f32e806be30 0x7f32e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.549+0000 7f32e3fff640 1 -- 172.21.15.26:0/2904930472 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f32ec08b0b0 con 0x7f32fc0cfcc0 2024-04-24T23:26:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.549+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f32e806be30 0x7f32e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.549+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f32e806be30 0x7f32e806e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f32f0006fd0 tx=0x7f32f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.550+0000 7f3304968740 1 Processor -- start 2024-04-24T23:26:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.550+0000 7f3304968740 1 -- start start 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f3304968740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f3304968740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc8c600 0x55a1ebc8cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f3304968740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc44a40 0x55a1ebc8acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f3304968740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1ebba86b0 con 0x55a1ebc6ed60 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f3304968740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a1ebb50880 con 0x55a1ebc44a40 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f3304968740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a1ebba4670 con 0x55a1ebc8c600 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc6f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40120/0 (socket says 172.21.15.26:40120) 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fb7fe640 1 -- 172.21.15.26:0/1479650141 learned_addr learned my addr 172.21.15.26:0/1479650141 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fbfff640 1 --2- 172.21.15.26:0/1479650141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc44a40 0x55a1ebc8acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fb7fe640 1 -- 172.21.15.26:0/1479650141 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc8c600 msgr2=0x55a1ebc8cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/1479650141 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc8c600 0x55a1ebc8cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fb7fe640 1 -- 172.21.15.26:0/1479650141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc44a40 msgr2=0x55a1ebc8acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/1479650141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc44a40 0x55a1ebc8acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fb7fe640 1 -- 172.21.15.26:0/1479650141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1eb9867d0 con 0x55a1ebc6ed60 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/1479650141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc6f140 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d830 tx=0x7f32f0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fe7c6f733d6205 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32e17fa640 1 -- 172.21.15.26:0/1479650141 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0004020 con 0x55a1ebc6ed60 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32e17fa640 1 -- 172.21.15.26:0/1479650141 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f00041c0 con 0x55a1ebc6ed60 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.551+0000 7f32e17fa640 1 -- 172.21.15.26:0/1479650141 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f00149b0 con 0x55a1ebc6ed60 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.552+0000 7f3304968740 1 -- 172.21.15.26:0/1479650141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc6ed60 msgr2=0x55a1ebc6f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.552+0000 7f3304968740 1 --2- 172.21.15.26:0/1479650141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc6f140 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d830 tx=0x7f32f0010000 comp rx=0 tx=0).stop 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.552+0000 7f3304968740 1 -- 172.21.15.26:0/1479650141 shutdown_connections 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.552+0000 7f3304968740 1 --2- 172.21.15.26:0/1479650141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc44a40 0x55a1ebc8acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.552+0000 7f3304968740 1 --2- 172.21.15.26:0/1479650141 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc8c600 0x55a1ebc8cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.552+0000 7f3304968740 1 --2- 172.21.15.26:0/1479650141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc6f140 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.552+0000 7f3304968740 1 -- 172.21.15.26:0/1479650141 >> 172.21.15.26:0/1479650141 conn(0x55a1ebc41600 msgr2=0x55a1ebc89b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.552+0000 7f3304968740 1 -- 172.21.15.26:0/1479650141 shutdown_connections 2024-04-24T23:26:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.552+0000 7f3304968740 1 -- 172.21.15.26:0/1479650141 wait complete. 2024-04-24T23:26:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.553+0000 7f3304968740 1 -- start start 2024-04-24T23:26:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.553+0000 7f3304968740 1 Processor -- start 2024-04-24T23:26:49.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.553+0000 7f3304968740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 0x55a1ebc86650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.553+0000 7f3304968740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc80d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.553+0000 7f3304968740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 0x55a1ebc5edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f3304968740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1ebc457e0 con 0x55a1ebc8c600 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f3304968740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a1ebb63f50 con 0x55a1ebc81240 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f3304968740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a1ebc3f530 con 0x55a1ebc6ed60 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f3304968740 10 client.0 _ll_register_callbacks cb 0x55a1ebb9f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc80d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 0x55a1ebc5edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc80d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40122/0 (socket says 172.21.15.26:40122) 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32faffd640 1 -- 172.21.15.26:0/3499162028 learned_addr learned my addr 172.21.15.26:0/3499162028 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 0x55a1ebc86650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32faffd640 1 -- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 msgr2=0x55a1ebc86650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32faffd640 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 0x55a1ebc86650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32faffd640 1 -- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 msgr2=0x55a1ebc5edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32faffd640 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 0x55a1ebc5edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32faffd640 1 -- 172.21.15.26:0/3499162028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1eb8bc3f0 con 0x55a1ebc8c600 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32faffd640 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc80d00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c7c0 tx=0x7f32ec059af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8378b514c5c1646 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32da7fc640 1 -- 172.21.15.26:0/3499162028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec01e040 con 0x55a1ebc8c600 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32da7fc640 1 -- 172.21.15.26:0/3499162028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f32ec058780 con 0x55a1ebc8c600 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f3304968740 1 -- 172.21.15.26:0/3499162028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f00041c0 con 0x55a1ebc8c600 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f32da7fc640 1 -- 172.21.15.26:0/3499162028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f32ec061f10 con 0x55a1ebc8c600 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f3304968740 10 client.8115 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.554+0000 7f3304968740 1 -- 172.21.15.26:0/3499162028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55a1eb87c0f0 con 0x55a1ebc8c600 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f32da7fc640 1 -- 172.21.15.26:0/3499162028 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f00041c0 con 0x55a1ebc8c600 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f32da7fc640 1 -- 172.21.15.26:0/3499162028 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f32ec071020 con 0x55a1ebc8c600 2024-04-24T23:26:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f3304968740 10 client.8115 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f3304968740 10 client.8115 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f3304968740 20 client.8115 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f3304968740 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55a1ebc9df20 0x55a1ebc9e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f3304968740 4 client.8115 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f3304968740 1 -- 172.21.15.26:0/3499162028 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1ebbb5bf0 con 0x55a1ebc9df20 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55a1ebc9df20 0x55a1ebc9e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55a1ebc9df20 0x55a1ebc9e300 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40071bf7255ec2ac server_cookie=723c0d2640183e0d in_seq=0 out_seq=0 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.555+0000 7f32da7fc640 10 client.8115 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f32da7fc640 1 -- 172.21.15.26:0/3499162028 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1ebbb5bf0 con 0x55a1ebc9df20 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f32da7fc640 10 client.8115 handle_command_reply: tid=0 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 client.8115 shutdown 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 -- 172.21.15.26:0/3499162028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 msgr2=0x55a1ebc80d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc80d00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c7c0 tx=0x7f32ec059af0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 -- 172.21.15.26:0/3499162028 shutdown_connections 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55a1ebc9df20 0x55a1ebc9e300 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 0x55a1ebc5edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc80d00 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f32da7fc640 0 client.8115 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 --2- 172.21.15.26:0/3499162028 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 0x55a1ebc86650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 -- 172.21.15.26:0/3499162028 >> 172.21.15.26:0/3499162028 conn(0x55a1ebc44a40 msgr2=0x55a1ebc8d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 -- 172.21.15.26:0/3499162028 shutdown_connections 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 1 -- 172.21.15.26:0/3499162028 wait complete. 2024-04-24T23:26:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.556+0000 7f3304968740 20 client.8115 trim_cache size 0 max 16384 2024-04-24T23:26:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.571+0000 7f3304968740 1 Processor -- start 2024-04-24T23:26:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.571+0000 7f3304968740 1 -- start start 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f3304968740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc44e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f3304968740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc81240 0x55a1ebc45e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f3304968740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc8c600 0x55a1ebc8b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f3304968740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc2009f0 con 0x55a1ebc81240 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f3304968740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1f54a0 con 0x55a1ebc6ed60 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f3304968740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f32fc2002c0 con 0x55a1ebc8c600 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f32faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc81240 0x55a1ebc45e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f32faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc81240 0x55a1ebc45e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40134/0 (socket says 172.21.15.26:40134) 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f32fb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc44e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f32faffd640 1 -- 172.21.15.26:0/2674660920 learned_addr learned my addr 172.21.15.26:0/2674660920 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.572+0000 7f32fbfff640 1 --2- 172.21.15.26:0/2674660920 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc8c600 0x55a1ebc8b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32faffd640 1 -- 172.21.15.26:0/2674660920 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc8c600 msgr2=0x55a1ebc8b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32faffd640 1 --2- 172.21.15.26:0/2674660920 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc8c600 0x55a1ebc8b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32faffd640 1 -- 172.21.15.26:0/2674660920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc6ed60 msgr2=0x55a1ebc44e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32faffd640 1 --2- 172.21.15.26:0/2674660920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc44e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32faffd640 1 -- 172.21.15.26:0/2674660920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ebc834a0 con 0x55a1ebc81240 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/2674660920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc44e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32faffd640 1 --2- 172.21.15.26:0/2674660920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc81240 0x55a1ebc45e10 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f32ec007410 tx=0x7f32ec078000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe6f67a48e54778 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32e0ff9640 1 -- 172.21.15.26:0/2674660920 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec01e020 con 0x55a1ebc81240 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32e0ff9640 1 -- 172.21.15.26:0/2674660920 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ec055080 con 0x55a1ebc81240 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f32e0ff9640 1 -- 172.21.15.26:0/2674660920 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec05d160 con 0x55a1ebc81240 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f3304968740 1 -- 172.21.15.26:0/2674660920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc81240 msgr2=0x55a1ebc45e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.573+0000 7f3304968740 1 --2- 172.21.15.26:0/2674660920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc81240 0x55a1ebc45e10 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f32ec007410 tx=0x7f32ec078000 comp rx=0 tx=0).stop 2024-04-24T23:26:49.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.574+0000 7f3304968740 1 -- 172.21.15.26:0/2674660920 shutdown_connections 2024-04-24T23:26:49.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.574+0000 7f3304968740 1 --2- 172.21.15.26:0/2674660920 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc8c600 0x55a1ebc8b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.574+0000 7f3304968740 1 --2- 172.21.15.26:0/2674660920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc81240 0x55a1ebc45e10 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.574+0000 7f3304968740 1 --2- 172.21.15.26:0/2674660920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc6ed60 0x55a1ebc44e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.574+0000 7f3304968740 1 -- 172.21.15.26:0/2674660920 >> 172.21.15.26:0/2674660920 conn(0x55a1ebc44a40 msgr2=0x55a1ebc86460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.574+0000 7f3304968740 1 -- 172.21.15.26:0/2674660920 shutdown_connections 2024-04-24T23:26:49.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.574+0000 7f3304968740 1 -- 172.21.15.26:0/2674660920 wait complete. 2024-04-24T23:26:49.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.574+0000 7f3304968740 1 -- start start 2024-04-24T23:26:49.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.574+0000 7f3304968740 1 Processor -- start 2024-04-24T23:26:49.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f3304968740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 0x55a1ebcfa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f3304968740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 0x55a1ebc9cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f3304968740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc9d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f3304968740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1ebc9e390 con 0x55a1ebc8c600 2024-04-24T23:26:49.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f3304968740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc157e20 con 0x55a1ebc81240 2024-04-24T23:26:49.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f3304968740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a1ebc82f50 con 0x55a1ebc6ed60 2024-04-24T23:26:49.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f3304968740 10 client.0 _ll_register_callbacks cb 0x7f32ec00a440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:49.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc9d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc9d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40150/0 (socket says 172.21.15.26:40150) 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fbfff640 1 -- 172.21.15.26:0/1861719048 learned_addr learned my addr 172.21.15.26:0/1861719048 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 0x55a1ebcfa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32faffd640 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 0x55a1ebc9cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fbfff640 1 -- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 msgr2=0x55a1ebcfa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fbfff640 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 0x55a1ebcfa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fbfff640 1 -- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 msgr2=0x55a1ebc9cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fbfff640 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 0x55a1ebc9cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fbfff640 1 -- 172.21.15.26:0/1861719048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ebc9c240 con 0x55a1ebc8c600 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.575+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 0x55a1ebcfa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f32fbfff640 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc9d2c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009f40 tx=0x7f32f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e28c9b8172ff35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f32da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f32da7fc640 1 -- 172.21.15.26:0/1861719048 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f404b060 con 0x55a1ebc8c600 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f32da7fc640 1 -- 172.21.15.26:0/1861719048 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f32f403dc20 con 0x55a1ebc8c600 2024-04-24T23:26:49.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f3304968740 1 -- 172.21.15.26:0/1861719048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ec055080 con 0x55a1ebc8c600 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f32da7fc640 1 -- 172.21.15.26:0/1861719048 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f32f4036040 con 0x55a1ebc8c600 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f3304968740 10 client.8121 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f3304968740 1 -- 172.21.15.26:0/1861719048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55a1ebc978c0 con 0x55a1ebc8c600 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f32da7fc640 1 -- 172.21.15.26:0/1861719048 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ec055080 con 0x55a1ebc8c600 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f32da7fc640 1 -- 172.21.15.26:0/1861719048 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f32f4045070 con 0x55a1ebc8c600 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f3304968740 10 client.8121 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f3304968740 10 client.8121 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f3304968740 20 client.8121 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f3304968740 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55a1ebc971e0 0x55a1ebc7f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f3304968740 4 client.8121 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f3304968740 1 -- 172.21.15.26:0/1861719048 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1ebc87cf0 con 0x55a1ebc971e0 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.576+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55a1ebc971e0 0x55a1ebc7f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.577+0000 7f32fb7fe640 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55a1ebc971e0 0x55a1ebc7f500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3dda8c64b97ff9 server_cookie=c5ea9bd043915f9 in_seq=0 out_seq=0 2024-04-24T23:26:49.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.577+0000 7f32da7fc640 10 client.8121 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:49.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.587+0000 7f32da7fc640 1 -- 172.21.15.26:0/1861719048 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1ebc87cf0 con 0x55a1ebc971e0 2024-04-24T23:26:49.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.587+0000 7f32da7fc640 10 client.8121 handle_command_reply: tid=0 2024-04-24T23:26:49.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.587+0000 7f3304968740 1 client.8121 shutdown 2024-04-24T23:26:49.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 -- 172.21.15.26:0/1861719048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 msgr2=0x55a1ebc9d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc9d2c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009f40 tx=0x7f32f4004ab0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 -- 172.21.15.26:0/1861719048 shutdown_connections 2024-04-24T23:26:49.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55a1ebc971e0 0x55a1ebc7f500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ebc8c600 0x55a1ebc9d2c0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a1ebc81240 0x55a1ebc9cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f32da7fc640 0 client.8121 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 --2- 172.21.15.26:0/1861719048 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a1ebc6ed60 0x55a1ebcfa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 -- 172.21.15.26:0/1861719048 >> 172.21.15.26:0/1861719048 conn(0x55a1ebc44a40 msgr2=0x55a1ebcfe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 -- 172.21.15.26:0/1861719048 shutdown_connections 2024-04-24T23:26:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 1 -- 172.21.15.26:0/1861719048 wait complete. 2024-04-24T23:26:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3304968740 20 client.8121 trim_cache size 0 max 16384 2024-04-24T23:26:49.591 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:26:49.591 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:26:49.591 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 -- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f32e806be30 msgr2=0x7f32e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f32e806be30 0x7f32e806e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f32f0006fd0 tx=0x7f32f0008040 comp rx=0 tx=0).stop 2024-04-24T23:26:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 -- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc0cfcc0 msgr2=0x7f32fc1e7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc0cfcc0 0x7f32fc1e7b80 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c7f0 tx=0x7f32ec00ccb0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 -- 172.21.15.26:0/2904930472 shutdown_connections 2024-04-24T23:26:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f32e806be30 0x7f32e806e2f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f32fc0b5490 0x7f32fc1e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32fc0cfcc0 0x7f32fc1e7b80 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 --2- 172.21.15.26:0/2904930472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32fc06d4a0 0x7f32fc153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.588+0000 7f3301c61640 1 -- 172.21.15.26:0/2904930472 >> 172.21.15.26:0/2904930472 conn(0x7f32fc0b2970 msgr2=0x7f32fc15dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.589+0000 7f3301c61640 1 -- 172.21.15.26:0/2904930472 shutdown_connections 2024-04-24T23:26:49.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.589+0000 7f3301c61640 1 -- 172.21.15.26:0/2904930472 wait complete. 2024-04-24T23:26:49.600 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 flush journal 2024-04-24T23:26:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.703+0000 7f33c0ad2640 1 Processor -- start 2024-04-24T23:26:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.703+0000 7f33c0ad2640 1 -- start start 2024-04-24T23:26:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33c0ad2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f33bc15adc0 0x7f33bc0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33c0ad2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33bc0ad7b0 0x7f33bc0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33c0ad2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33bc15ef60 con 0x7f33bc15e9c0 2024-04-24T23:26:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33c0ad2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f33bc15f240 con 0x7f33bc0ad7b0 2024-04-24T23:26:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33c0ad2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f33bc15f520 con 0x7f33bc15adc0 2024-04-24T23:26:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33b3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33bc0ad7b0 0x7f33bc0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33b3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33bc0ad7b0 0x7f33bc0d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40160/0 (socket says 172.21.15.26:40160) 2024-04-24T23:26:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33b3fff640 1 -- 172.21.15.26:0/385711277 learned_addr learned my addr 172.21.15.26:0/385711277 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33b3fff640 1 -- 172.21.15.26:0/385711277 >> v2:172.21.15.29:3300/0 conn(0x7f33bc15adc0 msgr2=0x7f33bc0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33b3fff640 1 --2- 172.21.15.26:0/385711277 >> v2:172.21.15.29:3300/0 conn(0x7f33bc15adc0 0x7f33bc0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33b3fff640 1 -- 172.21.15.26:0/385711277 >> v1:172.21.15.29:6789/0 conn(0x7f33bc15e9c0 legacy=0x7f33bc06dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.704+0000 7f33b3fff640 1 -- 172.21.15.26:0/385711277 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc15f800 con 0x7f33bc0ad7b0 2024-04-24T23:26:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.705+0000 7f33b3fff640 1 --2- 172.21.15.26:0/385711277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc0ad7b0 0x7f33bc0d0040 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009870 tx=0x7f33a402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd00c23ec5a1d665 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.705+0000 7f33b9d9b640 1 -- 172.21.15.26:0/385711277 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4037070 con 0x7f33bc0ad7b0 2024-04-24T23:26:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.705+0000 7f33b9d9b640 1 -- 172.21.15.26:0/385711277 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a402ae40 con 0x7f33bc0ad7b0 2024-04-24T23:26:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.705+0000 7f33b9d9b640 1 -- 172.21.15.26:0/385711277 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4032660 con 0x7f33bc0ad7b0 2024-04-24T23:26:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.705+0000 7f33c0ad2640 1 -- 172.21.15.26:0/385711277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc0ad7b0 msgr2=0x7f33bc0d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.705+0000 7f33c0ad2640 1 --2- 172.21.15.26:0/385711277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc0ad7b0 0x7f33bc0d0040 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009870 tx=0x7f33a402a130 comp rx=0 tx=0).stop 2024-04-24T23:26:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.706+0000 7f33c0ad2640 1 -- 172.21.15.26:0/385711277 shutdown_connections 2024-04-24T23:26:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.706+0000 7f33c0ad2640 1 --2- 172.21.15.26:0/385711277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc0ad7b0 0x7f33bc0d0040 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.706+0000 7f33c0ad2640 1 --2- 172.21.15.26:0/385711277 >> v2:172.21.15.29:3300/0 conn(0x7f33bc15adc0 0x7f33bc0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.706+0000 7f33c0ad2640 1 -- 172.21.15.26:0/385711277 >> 172.21.15.26:0/385711277 conn(0x7f33bc06b540 msgr2=0x7f33bc06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.706+0000 7f33c0ad2640 1 -- 172.21.15.26:0/385711277 shutdown_connections 2024-04-24T23:26:49.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.706+0000 7f33c0ad2640 1 -- 172.21.15.26:0/385711277 wait complete. 2024-04-24T23:26:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.707+0000 7f33c0ad2640 1 Processor -- start 2024-04-24T23:26:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.707+0000 7f33c0ad2640 1 -- start start 2024-04-24T23:26:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33c0ad2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33bc0ad7b0 0x7f33bc1f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33c0ad2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc15adc0 0x7f33bc09c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33c0ad2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f33bc15e9c0 0x7f33bc14f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33c0ad2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc15b1a0 con 0x7f33bc15adc0 2024-04-24T23:26:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33c0ad2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc0b48b0 con 0x7f33bc0ad7b0 2024-04-24T23:26:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33c0ad2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc0d0580 con 0x7f33bc15e9c0 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc15adc0 0x7f33bc09c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc15adc0 0x7f33bc09c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40164/0 (socket says 172.21.15.26:40164) 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b3fff640 1 -- 172.21.15.26:0/1952005776 learned_addr learned my addr 172.21.15.26:0/1952005776 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33ba59c640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33bc0ad7b0 0x7f33bc1f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b3fff640 1 -- 172.21.15.26:0/1952005776 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f33bc15e9c0 msgr2=0x7f33bc14f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b3fff640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f33bc15e9c0 0x7f33bc14f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b3fff640 1 -- 172.21.15.26:0/1952005776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33bc0ad7b0 msgr2=0x7f33bc1f74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b3fff640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33bc0ad7b0 0x7f33bc1f74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b3fff640 1 -- 172.21.15.26:0/1952005776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc1f3fe0 con 0x7f33bc15adc0 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b3fff640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc15adc0 0x7f33bc09c410 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009870 tx=0x7f33a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b2ffd640 1 -- 172.21.15.26:0/1952005776 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4037040 con 0x7f33bc15adc0 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b2ffd640 1 -- 172.21.15.26:0/1952005776 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a4004030 con 0x7f33bc15adc0 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33c0ad2640 1 -- 172.21.15.26:0/1952005776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc1f7af0 con 0x7f33bc15adc0 2024-04-24T23:26:49.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33b2ffd640 1 -- 172.21.15.26:0/1952005776 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4004420 con 0x7f33bc15adc0 2024-04-24T23:26:49.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.708+0000 7f33c0ad2640 1 -- 172.21.15.26:0/1952005776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc1f80b0 con 0x7f33bc15adc0 2024-04-24T23:26:49.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.710+0000 7f33b2ffd640 1 -- 172.21.15.26:0/1952005776 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33a403a040 con 0x7f33bc15adc0 2024-04-24T23:26:49.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.710+0000 7f33b2ffd640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f339c06be30 0x7f339c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.710+0000 7f33b2ffd640 1 -- 172.21.15.26:0/1952005776 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f33a40ab550 con 0x7f33bc15adc0 2024-04-24T23:26:49.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.710+0000 7f33ba59c640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f339c06be30 0x7f339c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.711+0000 7f33ba59c640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f339c06be30 0x7f339c06e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f33bc150020 tx=0x7f33ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.711+0000 7f33c37dd740 1 Processor -- start 2024-04-24T23:26:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33c37dd740 1 -- start start 2024-04-24T23:26:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 0x55d5ed863140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed880600 0x55d5ed880b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed838a40 0x55d5ed87ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33c37dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed79c6b0 con 0x55d5ed838a40 2024-04-24T23:26:49.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33c37dd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed744880 con 0x55d5ed862d60 2024-04-24T23:26:49.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33c37dd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d5ed798670 con 0x55d5ed880600 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33bad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed838a40 0x55d5ed87ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33ba59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 0x55d5ed863140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33ba59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 0x55d5ed863140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:57656/0 (socket says 172.21.15.26:57656) 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33ba59c640 1 -- 172.21.15.26:0/1227777974 learned_addr learned my addr 172.21.15.26:0/1227777974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33bad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed838a40 0x55d5ed87ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40170/0 (socket says 172.21.15.26:40170) 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.712+0000 7f33b3fff640 1 --2- 172.21.15.26:0/1227777974 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed880600 0x55d5ed880b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f33ba59c640 1 -- 172.21.15.26:0/1227777974 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed880600 msgr2=0x55d5ed880b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f33ba59c640 1 --2- 172.21.15.26:0/1227777974 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed880600 0x55d5ed880b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f33ba59c640 1 -- 172.21.15.26:0/1227777974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed838a40 msgr2=0x55d5ed87ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f33ba59c640 1 --2- 172.21.15.26:0/1227777974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed838a40 0x55d5ed87ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f33ba59c640 1 -- 172.21.15.26:0/1227777974 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ed57a7d0 con 0x55d5ed862d60 2024-04-24T23:26:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f33bad9d640 1 --2- 172.21.15.26:0/1227777974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed838a40 0x55d5ed87ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f33ba59c640 1 --2- 172.21.15.26:0/1227777974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 0x55d5ed863140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00d840 tx=0x7f33ac010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221f3efeda45175d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f339bfff640 1 -- 172.21.15.26:0/1227777974 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac024070 con 0x55d5ed862d60 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f339bfff640 1 -- 172.21.15.26:0/1227777974 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac0149b0 con 0x55d5ed862d60 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.713+0000 7f339bfff640 1 -- 172.21.15.26:0/1227777974 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac014d60 con 0x55d5ed862d60 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 -- 172.21.15.26:0/1227777974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 msgr2=0x55d5ed863140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 --2- 172.21.15.26:0/1227777974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 0x55d5ed863140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00d840 tx=0x7f33ac010000 comp rx=0 tx=0).stop 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 -- 172.21.15.26:0/1227777974 shutdown_connections 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 --2- 172.21.15.26:0/1227777974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed838a40 0x55d5ed87ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 --2- 172.21.15.26:0/1227777974 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed880600 0x55d5ed880b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 --2- 172.21.15.26:0/1227777974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 0x55d5ed863140 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 -- 172.21.15.26:0/1227777974 >> 172.21.15.26:0/1227777974 conn(0x55d5ed835600 msgr2=0x55d5ed87db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 -- 172.21.15.26:0/1227777974 shutdown_connections 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 -- 172.21.15.26:0/1227777974 wait complete. 2024-04-24T23:26:49.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.714+0000 7f33c37dd740 1 -- start start 2024-04-24T23:26:49.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.715+0000 7f33c37dd740 1 Processor -- start 2024-04-24T23:26:49.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.715+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed862d60 0x55d5ed87cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.715+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed880600 0x55d5ed879ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.715+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed857320 0x55d5ed87a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.715+0000 7f33c37dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed8397e0 con 0x55d5ed880600 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.715+0000 7f33c37dd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed757f50 con 0x55d5ed857320 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.715+0000 7f33c37dd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d5ed833530 con 0x55d5ed862d60 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.715+0000 7f33c37dd740 10 client.0 _ll_register_callbacks cb 0x55d5ed793000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed880600 0x55d5ed879ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed880600 0x55d5ed879ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40178/0 (socket says 172.21.15.26:40178) 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33b3fff640 1 -- 172.21.15.26:0/203158824 learned_addr learned my addr 172.21.15.26:0/203158824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33ba59c640 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed862d60 0x55d5ed87cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33bad9d640 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed857320 0x55d5ed87a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33b3fff640 1 -- 172.21.15.26:0/203158824 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed862d60 msgr2=0x55d5ed87cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33b3fff640 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed862d60 0x55d5ed87cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33b3fff640 1 -- 172.21.15.26:0/203158824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed857320 msgr2=0x55d5ed87a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33b3fff640 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed857320 0x55d5ed87a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33b3fff640 1 -- 172.21.15.26:0/203158824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ed4b03f0 con 0x55d5ed880600 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33b3fff640 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed880600 0x55d5ed879ef0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f33a4035040 tx=0x7f33a4079120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2fa65f31eb10d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33997fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33997fa640 1 -- 172.21.15.26:0/203158824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a404b040 con 0x55d5ed880600 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33997fa640 1 -- 172.21.15.26:0/203158824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f33a4081f10 con 0x55d5ed880600 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33c37dd740 1 -- 172.21.15.26:0/203158824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ac0149b0 con 0x55d5ed880600 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33997fa640 1 -- 172.21.15.26:0/203158824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f33a40a9050 con 0x55d5ed880600 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33c37dd740 10 client.8133 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33c37dd740 1 -- 172.21.15.26:0/203158824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55d5ed4700f0 con 0x55d5ed880600 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.716+0000 7f33997fa640 1 -- 172.21.15.26:0/203158824 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac0149b0 con 0x55d5ed880600 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33997fa640 1 -- 172.21.15.26:0/203158824 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f33a40ab0b0 con 0x55d5ed880600 2024-04-24T23:26:49.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33c37dd740 10 client.8133 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:49.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33c37dd740 10 client.8133 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-04-24T23:26:49.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33c37dd740 20 client.8133 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:49.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33c37dd740 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55d5ed891b40 0x55d5ed891f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33c37dd740 4 client.8133 mds_command: new command op to 6778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:49.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33c37dd740 1 -- 172.21.15.26:0/203158824 --> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5ed7a9bf0 con 0x55d5ed891b40 2024-04-24T23:26:49.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33ba59c640 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55d5ed891b40 0x55d5ed891f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33997fa640 10 client.8133 ms_handle_connect on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:49.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.717+0000 7f33ba59c640 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55d5ed891b40 0x55d5ed891f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7847425d3bcd503 server_cookie=94e7b7009da28bed in_seq=0 out_seq=0 2024-04-24T23:26:49.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33997fa640 1 -- 172.21.15.26:0/203158824 <== mds.1 v2:172.21.15.26:6834/306957409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5ed7a9bf0 con 0x55d5ed891b40 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33997fa640 10 client.8133 handle_command_reply: tid=0 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 client.8133 shutdown 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 -- 172.21.15.26:0/203158824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed880600 msgr2=0x55d5ed879ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed880600 0x55d5ed879ef0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f33a4035040 tx=0x7f33a4079120 comp rx=0 tx=0).stop 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 -- 172.21.15.26:0/203158824 shutdown_connections 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55d5ed891b40 0x55d5ed891f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed857320 0x55d5ed87a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed880600 0x55d5ed879ef0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 --2- 172.21.15.26:0/203158824 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed862d60 0x55d5ed87cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33997fa640 0 client.8133 ms_handle_reset on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 -- 172.21.15.26:0/203158824 >> 172.21.15.26:0/203158824 conn(0x55d5ed838a40 msgr2=0x55d5ed881980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 -- 172.21.15.26:0/203158824 shutdown_connections 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 1 -- 172.21.15.26:0/203158824 wait complete. 2024-04-24T23:26:49.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.718+0000 7f33c37dd740 20 client.8133 trim_cache size 0 max 16384 2024-04-24T23:26:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.733+0000 7f33c37dd740 1 Processor -- start 2024-04-24T23:26:49.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.733+0000 7f33c37dd740 1 -- start start 2024-04-24T23:26:49.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed857320 0x55d5ed879ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 0x55d5ed87a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed880600 0x55d5ed836910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33c37dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc200a70 con 0x55d5ed857320 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33c37dd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1ffc40 con 0x55d5ed862d60 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33c37dd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc200340 con 0x55d5ed880600 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33ba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed857320 0x55d5ed879ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33ba59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed857320 0x55d5ed879ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40186/0 (socket says 172.21.15.26:40186) 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33ba59c640 1 -- 172.21.15.26:0/4004705143 learned_addr learned my addr 172.21.15.26:0/4004705143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33ba59c640 1 -- 172.21.15.26:0/4004705143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed880600 msgr2=0x55d5ed836910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33ba59c640 1 --2- 172.21.15.26:0/4004705143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed880600 0x55d5ed836910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33ba59c640 1 -- 172.21.15.26:0/4004705143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 msgr2=0x55d5ed87a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33ba59c640 1 --2- 172.21.15.26:0/4004705143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 0x55d5ed87a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33ba59c640 1 -- 172.21.15.26:0/4004705143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ed87b430 con 0x55d5ed857320 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f33ba59c640 1 --2- 172.21.15.26:0/4004705143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed857320 0x55d5ed879ea0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f33ac022040 tx=0x7f33ac01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad722efc910f8997 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f339b7fe640 1 -- 172.21.15.26:0/4004705143 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac024040 con 0x55d5ed857320 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f339b7fe640 1 -- 172.21.15.26:0/4004705143 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac014d60 con 0x55d5ed857320 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.734+0000 7f339b7fe640 1 -- 172.21.15.26:0/4004705143 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac02c2c0 con 0x55d5ed857320 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 -- 172.21.15.26:0/4004705143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed857320 msgr2=0x55d5ed879ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 --2- 172.21.15.26:0/4004705143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed857320 0x55d5ed879ea0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f33ac022040 tx=0x7f33ac01c000 comp rx=0 tx=0).stop 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 -- 172.21.15.26:0/4004705143 shutdown_connections 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 --2- 172.21.15.26:0/4004705143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed880600 0x55d5ed836910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 --2- 172.21.15.26:0/4004705143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed862d60 0x55d5ed87a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 --2- 172.21.15.26:0/4004705143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed857320 0x55d5ed879ea0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 -- 172.21.15.26:0/4004705143 >> 172.21.15.26:0/4004705143 conn(0x55d5ed838a40 msgr2=0x55d5ed87fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 -- 172.21.15.26:0/4004705143 shutdown_connections 2024-04-24T23:26:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 -- 172.21.15.26:0/4004705143 wait complete. 2024-04-24T23:26:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.735+0000 7f33c37dd740 1 -- start start 2024-04-24T23:26:49.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.736+0000 7f33c37dd740 1 Processor -- start 2024-04-24T23:26:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.736+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed857320 0x55d5ed8f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.736+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed862d60 0x55d5ed878190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.736+0000 7f33c37dd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed880600 0x55d5ed8786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.736+0000 7f33c37dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed833e30 con 0x55d5ed862d60 2024-04-24T23:26:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.736+0000 7f33c37dd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc0cd180 con 0x55d5ed880600 2024-04-24T23:26:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.736+0000 7f33c37dd740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d5ed882030 con 0x55d5ed857320 2024-04-24T23:26:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.736+0000 7f33c37dd740 10 client.0 _ll_register_callbacks cb 0x7f33a40094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:49.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed862d60 0x55d5ed878190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed862d60 0x55d5ed878190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40188/0 (socket says 172.21.15.26:40188) 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33b3fff640 1 -- 172.21.15.26:0/3925125598 learned_addr learned my addr 172.21.15.26:0/3925125598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33ba59c640 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed857320 0x55d5ed8f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33bad9d640 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed880600 0x55d5ed8786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33b3fff640 1 -- 172.21.15.26:0/3925125598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed857320 msgr2=0x55d5ed8f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33b3fff640 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed857320 0x55d5ed8f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33b3fff640 1 -- 172.21.15.26:0/3925125598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed880600 msgr2=0x55d5ed8786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33b3fff640 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed880600 0x55d5ed8786d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33b3fff640 1 -- 172.21.15.26:0/3925125598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ed87c6c0 con 0x55d5ed862d60 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33b3fff640 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed862d60 0x55d5ed878190 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f33a40ab870 tx=0x7f33a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c22d9e417945f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33997fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33997fa640 1 -- 172.21.15.26:0/3925125598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a404b070 con 0x55d5ed862d60 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33997fa640 1 -- 172.21.15.26:0/3925125598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f33a4082120 con 0x55d5ed862d60 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33c37dd740 1 -- 172.21.15.26:0/3925125598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ac014d60 con 0x55d5ed862d60 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33997fa640 1 -- 172.21.15.26:0/3925125598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f33a409e050 con 0x55d5ed862d60 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33c37dd740 10 client.8139 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33c37dd740 1 -- 172.21.15.26:0/3925125598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55d5ed87c6c0 con 0x55d5ed862d60 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33997fa640 1 -- 172.21.15.26:0/3925125598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac014d60 con 0x55d5ed862d60 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.737+0000 7f33997fa640 1 -- 172.21.15.26:0/3925125598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f33a40a1020 con 0x55d5ed862d60 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.738+0000 7f33c37dd740 10 client.8139 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.738+0000 7f33c37dd740 10 client.8139 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.738+0000 7f33c37dd740 20 client.8139 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.738+0000 7f33c37dd740 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55d5ed881060 0x55d5ed881470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.738+0000 7f33c37dd740 4 client.8139 mds_command: new command op to 6778 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:26:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.738+0000 7f33c37dd740 1 -- 172.21.15.26:0/3925125598 --> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5ed7a3f40 con 0x55d5ed881060 2024-04-24T23:26:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.738+0000 7f33ba59c640 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55d5ed881060 0x55d5ed881470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.738+0000 7f33997fa640 10 client.8139 ms_handle_connect on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.738+0000 7f33ba59c640 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55d5ed881060 0x55d5ed881470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb3869759c48fd68 server_cookie=a897bb6be0c72b58 in_seq=0 out_seq=0 2024-04-24T23:26:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33997fa640 1 -- 172.21.15.26:0/3925125598 <== mds.1 v2:172.21.15.26:6834/306957409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5ed7a3f40 con 0x55d5ed881060 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33997fa640 10 client.8139 handle_command_reply: tid=0 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 client.8139 shutdown 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 -- 172.21.15.26:0/3925125598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed862d60 msgr2=0x55d5ed878190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed862d60 0x55d5ed878190 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f33a40ab870 tx=0x7f33a4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 -- 172.21.15.26:0/3925125598 shutdown_connections 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x55d5ed881060 0x55d5ed881470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d5ed880600 0x55d5ed8786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5ed862d60 0x55d5ed878190 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 --2- 172.21.15.26:0/3925125598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d5ed857320 0x55d5ed8f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33997fa640 0 client.8139 ms_handle_reset on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 -- 172.21.15.26:0/3925125598 >> 172.21.15.26:0/3925125598 conn(0x55d5ed838a40 msgr2=0x55d5ed87bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 -- 172.21.15.26:0/3925125598 shutdown_connections 2024-04-24T23:26:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 1 -- 172.21.15.26:0/3925125598 wait complete. 2024-04-24T23:26:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.745+0000 7f33c37dd740 20 client.8139 trim_cache size 0 max 16384 2024-04-24T23:26:49.748 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:26:49.748 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:26:49.748 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 -- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f339c06be30 msgr2=0x7f339c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f339c06be30 0x7f339c06e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f33bc150020 tx=0x7f33ac008040 comp rx=0 tx=0).stop 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 -- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc15adc0 msgr2=0x7f33bc09c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc15adc0 0x7f33bc09c410 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009870 tx=0x7f33a40046c0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 -- 172.21.15.26:0/1952005776 shutdown_connections 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f339c06be30 0x7f339c06e2f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f33bc15e9c0 0x7f33bc14f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc15adc0 0x7f33bc09c410 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 --2- 172.21.15.26:0/1952005776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33bc0ad7b0 0x7f33bc1f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 -- 172.21.15.26:0/1952005776 >> 172.21.15.26:0/1952005776 conn(0x7f33bc06b540 msgr2=0x7f33bc1584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 -- 172.21.15.26:0/1952005776 shutdown_connections 2024-04-24T23:26:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.746+0000 7f33c0ad2640 1 -- 172.21.15.26:0/1952005776 wait complete. 2024-04-24T23:26:49.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 flush journal 2024-04-24T23:26:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.860+0000 7fe8e2a4c640 1 Processor -- start 2024-04-24T23:26:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.860+0000 7fe8e2a4c640 1 -- start start 2024-04-24T23:26:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.861+0000 7fe8e2a4c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8dc15e750 0x7fe8dc0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.861+0000 7fe8e2a4c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8dc0d2120 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.861+0000 7fe8e2a4c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8dc0b3740 con 0x7fe8dc15e370 2024-04-24T23:26:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.861+0000 7fe8e2a4c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe8dc0d1510 con 0x7fe8dc15e750 2024-04-24T23:26:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.861+0000 7fe8db7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8dc15e750 0x7fe8dc0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.861+0000 7fe8e0fe9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe8dc0d2c30 0x7fe8dc0cf1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51546/0 (socket says 172.21.15.26:51546) 2024-04-24T23:26:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.861+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/2457926156 learned_addr learned my addr 172.21.15.26:0/2457926156 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.862+0000 7fe8db7fe640 1 -- 172.21.15.26:0/2457926156 >> v1:172.21.15.29:6789/0 conn(0x7fe8dc15e370 legacy=0x7fe8dc0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.862+0000 7fe8db7fe640 1 -- 172.21.15.26:0/2457926156 >> v1:172.21.15.26:6789/0 conn(0x7fe8dc0d2c30 legacy=0x7fe8dc0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.862+0000 7fe8db7fe640 1 -- 172.21.15.26:0/2457926156 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc0b3a20 con 0x7fe8dc15e750 2024-04-24T23:26:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.862+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/2457926156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc15e750 0x7fe8dc0cec90 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0097a0 tx=0x7fe8cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad47f15ee7522165 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8daffd640 1 -- 172.21.15.26:0/2457926156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc037070 con 0x7fe8dc15e750 2024-04-24T23:26:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8daffd640 1 -- 172.21.15.26:0/2457926156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc032440 con 0x7fe8dc15e750 2024-04-24T23:26:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8daffd640 1 -- 172.21.15.26:0/2457926156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc032810 con 0x7fe8dc15e750 2024-04-24T23:26:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/2457926156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc15e750 msgr2=0x7fe8dc0cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8e2a4c640 1 --2- 172.21.15.26:0/2457926156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc15e750 0x7fe8dc0cec90 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0097a0 tx=0x7fe8cc02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/2457926156 shutdown_connections 2024-04-24T23:26:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8e2a4c640 1 --2- 172.21.15.26:0/2457926156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc15e750 0x7fe8dc0cec90 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/2457926156 >> 172.21.15.26:0/2457926156 conn(0x7fe8dc0d40f0 msgr2=0x7fe8dc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/2457926156 shutdown_connections 2024-04-24T23:26:49.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.863+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/2457926156 wait complete. 2024-04-24T23:26:49.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.864+0000 7fe8e2a4c640 1 Processor -- start 2024-04-24T23:26:49.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8e2a4c640 1 -- start start 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8e2a4c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc0d2c30 0x7fe8dc154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8e2a4c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8dc15e370 0x7fe8dc09ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8e2a4c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8dc09e410 0x7fe8dc0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8e2a4c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc0b5230 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8e2a4c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc0d1ed0 con 0x7fe8dc09e410 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8e2a4c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe8dc0b53b0 con 0x7fe8dc15e370 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc0d2c30 0x7fe8dc154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc0d2c30 0x7fe8dc154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40220/0 (socket says 172.21.15.26:40220) 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8dbfff640 1 -- 172.21.15.26:0/1126932551 learned_addr learned my addr 172.21.15.26:0/1126932551 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.865+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8dc09e410 0x7fe8dc0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8dc15e370 0x7fe8dc09ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8dbfff640 1 -- 172.21.15.26:0/1126932551 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8dc15e370 msgr2=0x7fe8dc09ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8dc15e370 0x7fe8dc09ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8dbfff640 1 -- 172.21.15.26:0/1126932551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8dc09e410 msgr2=0x7fe8dc0b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8dc09e410 0x7fe8dc0b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8dbfff640 1 -- 172.21.15.26:0/1126932551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc0a0a80 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc0d2c30 0x7fe8dc154720 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c430 tx=0x7fe8c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8d8ff9640 1 -- 172.21.15.26:0/1126932551 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4011790 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8d8ff9640 1 -- 172.21.15.26:0/1126932551 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c4017070 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/1126932551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8dc0a0d60 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8d8ff9640 1 -- 172.21.15.26:0/1126932551 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4011b70 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.866+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/1126932551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8dc0a1320 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.868+0000 7fe8d8ff9640 1 -- 172.21.15.26:0/1126932551 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8c4011d10 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.868+0000 7fe8d8ff9640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8bc06be30 0x7fe8bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.868+0000 7fe8d8ff9640 1 -- 172.21.15.26:0/1126932551 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fe8c408ac00 con 0x7fe8dc0d2c30 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.868+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8bc06be30 0x7fe8bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.868+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8bc06be30 0x7fe8bc06e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc034000 tx=0x7fe8cc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.869+0000 7fe8e5831740 1 Processor -- start 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.869+0000 7fe8e5831740 1 -- start start 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 0x5611d9e0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e29b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9de1a40 0x5611d9e27cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e5831740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5611d9d456b0 con 0x5611d9de1a40 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e5831740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5611d9ced880 con 0x5611d9e0bd60 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e5831740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5611d9d41670 con 0x5611d9e29600 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e0fe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9de1a40 0x5611d9e27cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e0fe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9de1a40 0x5611d9e27cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40228/0 (socket says 172.21.15.26:40228) 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/357973222 learned_addr learned my addr 172.21.15.26:0/357973222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/357973222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 0x5611d9e0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/357973222 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e29b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/357973222 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 msgr2=0x5611d9e29b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/357973222 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e29b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/357973222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 msgr2=0x5611d9e0c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/357973222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 0x5611d9e0c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/357973222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611d9b237d0 con 0x5611d9de1a40 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/357973222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9de1a40 0x5611d9e27cc0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400f1d0 tx=0x7fe8d4001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dc15d90ecd1959 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8d1ffb640 1 -- 172.21.15.26:0/357973222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400fe50 con 0x5611d9de1a40 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8d1ffb640 1 -- 172.21.15.26:0/357973222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d4018070 con 0x5611d9de1a40 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.870+0000 7fe8d1ffb640 1 -- 172.21.15.26:0/357973222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40074e0 con 0x5611d9de1a40 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.871+0000 7fe8e5831740 1 -- 172.21.15.26:0/357973222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9de1a40 msgr2=0x5611d9e27cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.871+0000 7fe8e5831740 1 --2- 172.21.15.26:0/357973222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9de1a40 0x5611d9e27cc0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400f1d0 tx=0x7fe8d4001950 comp rx=0 tx=0).stop 2024-04-24T23:26:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.871+0000 7fe8e5831740 1 -- 172.21.15.26:0/357973222 shutdown_connections 2024-04-24T23:26:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.871+0000 7fe8e5831740 1 --2- 172.21.15.26:0/357973222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9de1a40 0x5611d9e27cc0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.871+0000 7fe8e5831740 1 --2- 172.21.15.26:0/357973222 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e29b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.871+0000 7fe8e5831740 1 --2- 172.21.15.26:0/357973222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 0x5611d9e0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.871+0000 7fe8e5831740 1 -- 172.21.15.26:0/357973222 >> 172.21.15.26:0/357973222 conn(0x5611d9dde600 msgr2=0x5611d9e26b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.871+0000 7fe8e5831740 1 -- 172.21.15.26:0/357973222 shutdown_connections 2024-04-24T23:26:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.871+0000 7fe8e5831740 1 -- 172.21.15.26:0/357973222 wait complete. 2024-04-24T23:26:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.872+0000 7fe8e5831740 1 -- start start 2024-04-24T23:26:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.872+0000 7fe8e5831740 1 Processor -- start 2024-04-24T23:26:49.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e0bd60 0x5611d9e08590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e0f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e370c0 0x5611d9e27af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8e5831740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5611d9de27e0 con 0x5611d9e0bd60 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8e5831740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5611d9d00f50 con 0x5611d9e370c0 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8e5831740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5611d9ddc530 con 0x5611d9e29600 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8e5831740 10 client.0 _ll_register_callbacks cb 0x5611d9d3c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e0bd60 0x5611d9e08590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e0bd60 0x5611d9e08590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40236/0 (socket says 172.21.15.26:40236) 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8dbfff640 1 -- 172.21.15.26:0/2630239668 learned_addr learned my addr 172.21.15.26:0/2630239668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e0f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e370c0 0x5611d9e27af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8dbfff640 1 -- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 msgr2=0x5611d9e0f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e0f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8dbfff640 1 -- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e370c0 msgr2=0x5611d9e27af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e370c0 0x5611d9e27af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8dbfff640 1 -- 172.21.15.26:0/2630239668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611d9a593f0 con 0x5611d9e0bd60 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e0bd60 0x5611d9e08590 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40890f0 tx=0x7fe8c4058da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0ebde991721383 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8baffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8baffd640 1 -- 172.21.15.26:0/2630239668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4073020 con 0x5611d9e0bd60 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8baffd640 1 -- 172.21.15.26:0/2630239668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fe8c4058570 con 0x5611d9e0bd60 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.873+0000 7fe8e5831740 1 -- 172.21.15.26:0/2630239668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d4018070 con 0x5611d9e0bd60 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8baffd640 1 -- 172.21.15.26:0/2630239668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe8c4086070 con 0x5611d9e0bd60 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8e5831740 10 client.8154 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8e5831740 1 -- 172.21.15.26:0/2630239668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5611d9a190f0 con 0x5611d9e0bd60 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8baffd640 1 -- 172.21.15.26:0/2630239668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d4018070 con 0x5611d9e0bd60 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8baffd640 1 -- 172.21.15.26:0/2630239668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7fe8c405f420 con 0x5611d9e0bd60 2024-04-24T23:26:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8e5831740 10 client.8154 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8e5831740 10 client.8154 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8e5831740 20 client.8154 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5611d9e0e1c0 0x5611d9e0e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8e5831740 4 client.8154 mds_command: new command op to 6299 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8e5831740 1 -- 172.21.15.26:0/2630239668 --> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611d9d52bf0 con 0x5611d9e0e1c0 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.874+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5611d9e0e1c0 0x5611d9e0e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8baffd640 10 client.8154 ms_handle_connect on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5611d9e0e1c0 0x5611d9e0e5c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54e0afda82f10252 server_cookie=48ab1c4b10f1dfd1 in_seq=0 out_seq=0 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8baffd640 1 -- 172.21.15.26:0/2630239668 <== mds.2 v2:172.21.15.29:6833/3727846607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611d9d52bf0 con 0x5611d9e0e1c0 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8baffd640 10 client.8154 handle_command_reply: tid=0 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8e5831740 1 client.8154 shutdown 2024-04-24T23:26:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8e5831740 1 -- 172.21.15.26:0/2630239668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e0bd60 msgr2=0x5611d9e08590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e0bd60 0x5611d9e08590 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40890f0 tx=0x7fe8c4058da0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8e5831740 1 -- 172.21.15.26:0/2630239668 shutdown_connections 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5611d9e0e1c0 0x5611d9e0e5c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e370c0 0x5611d9e27af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e0f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8baffd640 0 client.8154 ms_handle_reset on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2630239668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e0bd60 0x5611d9e08590 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.875+0000 7fe8e5831740 1 -- 172.21.15.26:0/2630239668 >> 172.21.15.26:0/2630239668 conn(0x5611d9de1a40 msgr2=0x5611d9e2a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.876+0000 7fe8e5831740 1 -- 172.21.15.26:0/2630239668 shutdown_connections 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.876+0000 7fe8e5831740 1 -- 172.21.15.26:0/2630239668 wait complete. 2024-04-24T23:26:49.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.876+0000 7fe8e5831740 20 client.8154 trim_cache size 0 max 16384 2024-04-24T23:26:49.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.890+0000 7fe8e5831740 1 Processor -- start 2024-04-24T23:26:49.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e5831740 1 -- start start 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 0x5611d9e24fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e26b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e5831740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc2008b0 con 0x5611d9e370c0 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e5831740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc1ffa80 con 0x5611d9e0bd60 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e5831740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe8dc200180 con 0x5611d9e29600 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e0fe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e0fe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e270b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40244/0 (socket says 172.21.15.26:40244) 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.891+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/452738341 learned_addr learned my addr 172.21.15.26:0/452738341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/452738341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 0x5611d9e24fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/452738341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 msgr2=0x5611d9e26b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/452738341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e26b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/452738341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 msgr2=0x5611d9e24fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/452738341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 0x5611d9e24fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/452738341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611d9dddac0 con 0x5611d9e370c0 2024-04-24T23:26:49.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/452738341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e270b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000f80 tx=0x7fe8d400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2aa38982045764 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8d17fa640 1 -- 172.21.15.26:0/452738341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40072e0 con 0x5611d9e370c0 2024-04-24T23:26:49.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8d17fa640 1 -- 172.21.15.26:0/452738341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d4024070 con 0x5611d9e370c0 2024-04-24T23:26:49.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.892+0000 7fe8d17fa640 1 -- 172.21.15.26:0/452738341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40076e0 con 0x5611d9e370c0 2024-04-24T23:26:49.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 -- 172.21.15.26:0/452738341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 msgr2=0x5611d9e270b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 --2- 172.21.15.26:0/452738341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e270b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000f80 tx=0x7fe8d400a000 comp rx=0 tx=0).stop 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 -- 172.21.15.26:0/452738341 shutdown_connections 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 --2- 172.21.15.26:0/452738341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e270b0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 --2- 172.21.15.26:0/452738341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e29600 0x5611d9e26b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 --2- 172.21.15.26:0/452738341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e0bd60 0x5611d9e24fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 -- 172.21.15.26:0/452738341 >> 172.21.15.26:0/452738341 conn(0x5611d9de1a40 msgr2=0x5611d9e083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 -- 172.21.15.26:0/452738341 shutdown_connections 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 -- 172.21.15.26:0/452738341 wait complete. 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 -- start start 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.893+0000 7fe8e5831740 1 Processor -- start 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e0bd60 0x5611d9e1ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e29600 0x5611d9e26b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e5831740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e27080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e5831740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc0b1bc0 con 0x5611d9e370c0 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e5831740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5611d9ea1710 con 0x5611d9e29600 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e5831740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5611d9e0eab0 con 0x5611d9e0bd60 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e5831740 10 client.0 _ll_register_callbacks cb 0x7fe8c4058950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e0fe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e27080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e0fe9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e27080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40248/0 (socket says 172.21.15.26:40248) 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/2610610946 learned_addr learned my addr 172.21.15.26:0/2610610946 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e29600 0x5611d9e26b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.894+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e0bd60 0x5611d9e1ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e0bd60 msgr2=0x5611d9e1ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e0bd60 0x5611d9e1ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e29600 msgr2=0x5611d9e26b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e29600 0x5611d9e26b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8e0fe9640 1 -- 172.21.15.26:0/2610610946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611d9ddb4b0 con 0x5611d9e370c0 2024-04-24T23:26:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8db7fe640 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e29600 0x5611d9e26b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8e0fe9640 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e27080 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400f1d0 tx=0x7fe8d4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1c00977e688176 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8baffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8baffd640 1 -- 172.21.15.26:0/2610610946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4029460 con 0x5611d9e370c0 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8baffd640 1 -- 172.21.15.26:0/2610610946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fe8d4029600 con 0x5611d9e370c0 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8e5831740 1 -- 172.21.15.26:0/2610610946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d4024070 con 0x5611d9e370c0 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8baffd640 1 -- 172.21.15.26:0/2610610946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe8d403e050 con 0x5611d9e370c0 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8e5831740 10 client.8160 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8e5831740 1 -- 172.21.15.26:0/2610610946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5611d9e40c40 con 0x5611d9e370c0 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8baffd640 1 -- 172.21.15.26:0/2610610946 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d4024070 con 0x5611d9e370c0 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.895+0000 7fe8baffd640 1 -- 172.21.15.26:0/2610610946 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7fe8d4041020 con 0x5611d9e370c0 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.896+0000 7fe8e5831740 10 client.8160 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.896+0000 7fe8e5831740 10 client.8160 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.896+0000 7fe8e5831740 20 client.8160 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.896+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5611d9e25380 0x5611d9ea08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.896+0000 7fe8e5831740 4 client.8160 mds_command: new command op to 6299 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:26:49.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.896+0000 7fe8e5831740 1 -- 172.21.15.26:0/2610610946 --> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611d9d4cf40 con 0x5611d9e25380 2024-04-24T23:26:49.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.896+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5611d9e25380 0x5611d9ea08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.896+0000 7fe8dbfff640 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5611d9e25380 0x5611d9ea08a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e49ab36cbe8c1f7d server_cookie=e202553b864dcfaa in_seq=0 out_seq=0 2024-04-24T23:26:49.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.896+0000 7fe8baffd640 10 client.8160 ms_handle_connect on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:49.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8baffd640 1 -- 172.21.15.26:0/2610610946 <== mds.2 v2:172.21.15.29:6833/3727846607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611d9d4cf40 con 0x5611d9e25380 2024-04-24T23:26:49.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8baffd640 10 client.8160 handle_command_reply: tid=0 2024-04-24T23:26:49.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 client.8160 shutdown 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 -- 172.21.15.26:0/2610610946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 msgr2=0x5611d9e27080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e27080 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400f1d0 tx=0x7fe8d4012000 comp rx=0 tx=0).stop 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 -- 172.21.15.26:0/2610610946 shutdown_connections 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5611d9e25380 0x5611d9ea08a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611d9e370c0 0x5611d9e27080 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5611d9e29600 0x5611d9e26b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 --2- 172.21.15.26:0/2610610946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5611d9e0bd60 0x5611d9e1ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8baffd640 0 client.8160 ms_handle_reset on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 -- 172.21.15.26:0/2610610946 >> 172.21.15.26:0/2610610946 conn(0x5611d9de1a40 msgr2=0x5611d9e08430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 -- 172.21.15.26:0/2610610946 shutdown_connections 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 1 -- 172.21.15.26:0/2610610946 wait complete. 2024-04-24T23:26:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.903+0000 7fe8e5831740 20 client.8160 trim_cache size 0 max 16384 2024-04-24T23:26:49.906 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:26:49.906 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:26:49.906 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8bc06be30 msgr2=0x7fe8bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8bc06be30 0x7fe8bc06e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc034000 tx=0x7fe8cc02e000 comp rx=0 tx=0).stop 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc0d2c30 msgr2=0x7fe8dc154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc0d2c30 0x7fe8dc154720 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c430 tx=0x7fe8c400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/1126932551 shutdown_connections 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8bc06be30 0x7fe8bc06e2f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8dc09e410 0x7fe8dc0b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8dc15e370 0x7fe8dc09ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 --2- 172.21.15.26:0/1126932551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8dc0d2c30 0x7fe8dc154720 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/1126932551 >> 172.21.15.26:0/1126932551 conn(0x7fe8dc0d40f0 msgr2=0x7fe8dc0b5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/1126932551 shutdown_connections 2024-04-24T23:26:49.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:49.904+0000 7fe8e2a4c640 1 -- 172.21.15.26:0/1126932551 wait complete. 2024-04-24T23:26:49.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:49.916 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-04-24T23:26:49.939 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/ 2024-04-24T23:26:49.939 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4 2024-04-24T23:26:49.939 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5 2024-04-24T23:26:49.939 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6 2024-04-24T23:26:49.939 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7 2024-04-24T23:26:49.939 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-04-24T23:26:49.940 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:26:49.940 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:26:49.940 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:26:49.940 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:26:49.940 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-04-24T23:26:49.940 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:26:50.047 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627778 2024-04-24T23:26:50.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:26:50.055 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:26:50.055 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:26:50.056 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:26:50.056 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:26:50.056 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-04-24T23:26:50.056 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:26:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627779 2024-04-24T23:26:50.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:26:50.172 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:26:50.172 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:26:50.172 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:26:50.172 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:26:50.172 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-04-24T23:26:50.172 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:26:50.278 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627780 2024-04-24T23:26:50.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:26:50.288 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:26:50.288 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:26:50.288 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:26:50.288 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:26:50.288 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-04-24T23:26:50.288 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:26:50.394 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627781 2024-04-24T23:26:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:26:50.403 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:26:50.403 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:26:50.403 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:26:50.403 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:26:50.403 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-04-24T23:26:50.403 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:26:50.511 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627782 2024-04-24T23:26:50.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:26:50.520 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:26:50.520 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:26:50.521 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:26:50.521 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:26:50.521 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-04-24T23:26:50.521 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:26:50.627 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627783 2024-04-24T23:26:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:26:50.636 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-04-24T23:26:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.719+0000 7fca83a0cf40 1 Processor -- start 2024-04-24T23:26:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.719+0000 7fca83a0cf40 1 -- start start 2024-04-24T23:26:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.719+0000 7fca83a0cf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55be0da40190 0x55be0d9f9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.719+0000 7fca83a0cf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be0da430a0 0x55be0dac5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca83a0cf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0dace4f0 0x55be0dac5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca83a0cf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55be0dad0400 con 0x55be0dace4f0 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca83a0cf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55be0d9a0640 con 0x55be0da430a0 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca83a0cf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55be0dacf830 con 0x55be0da40190 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca81049640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0dace4f0 0x55be0dac5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca81049640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0dace4f0 0x55be0dac5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40270/0 (socket says 172.21.15.26:40270) 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca81049640 1 -- 172.21.15.26:0/330763426 learned_addr learned my addr 172.21.15.26:0/330763426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca80047640 1 --2- 172.21.15.26:0/330763426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be0da430a0 0x55be0dac5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca80848640 1 --2- 172.21.15.26:0/330763426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55be0da40190 0x55be0d9f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca81049640 1 -- 172.21.15.26:0/330763426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55be0da40190 msgr2=0x55be0d9f9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca81049640 1 --2- 172.21.15.26:0/330763426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55be0da40190 0x55be0d9f9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca81049640 1 -- 172.21.15.26:0/330763426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be0da430a0 msgr2=0x55be0dac5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca81049640 1 --2- 172.21.15.26:0/330763426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be0da430a0 0x55be0dac5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca81049640 1 -- 172.21.15.26:0/330763426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be0da3f270 con 0x55be0dace4f0 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca81049640 1 --2- 172.21.15.26:0/330763426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0dace4f0 0x55be0dac5c70 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fca7800a7a0 tx=0x7fca7800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7838d00652e138f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca7f045640 1 -- 172.21.15.26:0/330763426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78019070 con 0x55be0dace4f0 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca7f045640 1 -- 172.21.15.26:0/330763426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca78007600 con 0x55be0dace4f0 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.720+0000 7fca7f045640 1 -- 172.21.15.26:0/330763426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca780079d0 con 0x55be0dace4f0 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 -- 172.21.15.26:0/330763426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0dace4f0 msgr2=0x55be0dac5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 --2- 172.21.15.26:0/330763426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0dace4f0 0x55be0dac5c70 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fca7800a7a0 tx=0x7fca7800a890 comp rx=0 tx=0).stop 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 -- 172.21.15.26:0/330763426 shutdown_connections 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 --2- 172.21.15.26:0/330763426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0dace4f0 0x55be0dac5c70 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 --2- 172.21.15.26:0/330763426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be0da430a0 0x55be0dac5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 --2- 172.21.15.26:0/330763426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55be0da40190 0x55be0d9f9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 -- 172.21.15.26:0/330763426 >> 172.21.15.26:0/330763426 conn(0x55be0da23b90 msgr2=0x55be0da42990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 -- 172.21.15.26:0/330763426 shutdown_connections 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 -- 172.21.15.26:0/330763426 wait complete. 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 Processor -- start 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.721+0000 7fca83a0cf40 1 -- start start 2024-04-24T23:26:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca83a0cf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55be0da40190 0x55be0da41b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca83a0cf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0da430a0 0x55be0da40d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca83a0cf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be0dace4f0 0x55be0da412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca83a0cf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55be0da40780 con 0x55be0da430a0 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca83a0cf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55be0dacf150 con 0x55be0dace4f0 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca83a0cf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55be0da3ef90 con 0x55be0da40190 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca80047640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0da430a0 0x55be0da40d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca80047640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0da430a0 0x55be0da40d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40282/0 (socket says 172.21.15.26:40282) 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca80047640 1 -- 172.21.15.26:0/814609478 learned_addr learned my addr 172.21.15.26:0/814609478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca80848640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55be0da40190 0x55be0da41b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.722+0000 7fca81049640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be0dace4f0 0x55be0da412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca80047640 1 -- 172.21.15.26:0/814609478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55be0da40190 msgr2=0x55be0da41b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca80047640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55be0da40190 0x55be0da41b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca80047640 1 -- 172.21.15.26:0/814609478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be0dace4f0 msgr2=0x55be0da412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca80047640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55be0dace4f0 0x55be0da412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca80047640 1 -- 172.21.15.26:0/814609478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be0da42040 con 0x55be0da430a0 2024-04-24T23:26:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca80047640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0da430a0 0x55be0da40d70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fca68003d60 tx=0x7fca6800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca7e043640 1 -- 172.21.15.26:0/814609478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68015070 con 0x55be0da430a0 2024-04-24T23:26:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca7e043640 1 -- 172.21.15.26:0/814609478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca68011450 con 0x55be0da430a0 2024-04-24T23:26:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55be0d9f8530 con 0x55be0da430a0 2024-04-24T23:26:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca7e043640 1 -- 172.21.15.26:0/814609478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68011840 con 0x55be0da430a0 2024-04-24T23:26:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.723+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be0dac3440 con 0x55be0da430a0 2024-04-24T23:26:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.724+0000 7fca7e043640 1 -- 172.21.15.26:0/814609478 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca68018070 con 0x55be0da430a0 2024-04-24T23:26:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.725+0000 7fca7e043640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca6006a840 0x7fca6006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.725+0000 7fca7e043640 1 -- 172.21.15.26:0/814609478 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fca68089ba0 con 0x55be0da430a0 2024-04-24T23:26:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.725+0000 7fca80848640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca6006a840 0x7fca6006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.725+0000 7fca83a0cf40 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55be0dacd140 0x55be0dac7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.725+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 --> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] -- osd_op(unknown.0.0:1 12.12 12:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55be0dacccf0 con 0x55be0dacd140 2024-04-24T23:26:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.725+0000 7fca81049640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55be0dacd140 0x55be0dac7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.725+0000 7fca80848640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca6006a840 0x7fca6006cd00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fca70056000 tx=0x7fca7005d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.726+0000 7fca81049640 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55be0dacd140 0x55be0dac7840 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca81049640 1 -- 172.21.15.26:0/814609478 <== osd.3 v2:172.21.15.26:6808/2035350946 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v85'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fca7801c040 con 0x55be0dacd140 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55be0dacd140 msgr2=0x55be0dac7840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55be0dacd140 0x55be0dac7840 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca6006a840 msgr2=0x7fca6006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fca6006a840 0x7fca6006cd00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fca70056000 tx=0x7fca7005d040 comp rx=0 tx=0).stop 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0da430a0 msgr2=0x55be0da40d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 --2- 172.21.15.26:0/814609478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55be0da430a0 0x55be0da40d70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fca68003d60 tx=0x7fca6800c620 comp rx=0 tx=0).stop 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca81049640 1 -- 172.21.15.26:0/814609478 reap_dead start 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 shutdown_connections 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 >> 172.21.15.26:0/814609478 conn(0x55be0da23b90 msgr2=0x55be0da42990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 shutdown_connections 2024-04-24T23:26:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.727+0000 7fca83a0cf40 1 -- 172.21.15.26:0/814609478 wait complete. 2024-04-24T23:26:50.733 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-04-24T23:26:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.813+0000 7f8df67edf40 1 Processor -- start 2024-04-24T23:26:50.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.813+0000 7f8df67edf40 1 -- start start 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df67edf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0603cd010 0x55e0603cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df67edf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e060496440 0x55e06048d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df67edf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55e060498e30 0x55e06048d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df67edf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e060372640 con 0x55e0603cd010 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df67edf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e060376680 con 0x55e060496440 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df67edf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55e0603f57f0 con 0x55e060498e30 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df42ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0603cd010 0x55e0603cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df42ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0603cd010 0x55e0603cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40294/0 (socket says 172.21.15.26:40294) 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df42ee640 1 -- 172.21.15.26:0/3607635474 learned_addr learned my addr 172.21.15.26:0/3607635474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df42ee640 1 -- 172.21.15.26:0/3607635474 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55e060498e30 msgr2=0x55e06048d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df42ee640 1 --2- 172.21.15.26:0/3607635474 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55e060498e30 0x55e06048d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df42ee640 1 -- 172.21.15.26:0/3607635474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e060496440 msgr2=0x55e06048d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df3aed640 1 --2- 172.21.15.26:0/3607635474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e060496440 0x55e06048d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df42ee640 1 --2- 172.21.15.26:0/3607635474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e060496440 0x55e06048d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.814+0000 7f8df42ee640 1 -- 172.21.15.26:0/3607635474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e06048e2a0 con 0x55e0603cd010 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df42ee640 1 --2- 172.21.15.26:0/3607635474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0603cd010 0x55e0603cdc80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00c6a0 tx=0x7f8dec00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699d067461b32fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df2aeb640 1 -- 172.21.15.26:0/3607635474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec015020 con 0x55e0603cd010 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df2aeb640 1 -- 172.21.15.26:0/3607635474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dec00cdb0 con 0x55e0603cd010 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df2aeb640 1 -- 172.21.15.26:0/3607635474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec00eab0 con 0x55e0603cd010 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df67edf40 1 -- 172.21.15.26:0/3607635474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0603cd010 msgr2=0x55e0603cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df67edf40 1 --2- 172.21.15.26:0/3607635474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0603cd010 0x55e0603cdc80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00c6a0 tx=0x7f8dec00cb60 comp rx=0 tx=0).stop 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df67edf40 1 -- 172.21.15.26:0/3607635474 shutdown_connections 2024-04-24T23:26:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df67edf40 1 --2- 172.21.15.26:0/3607635474 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55e060498e30 0x55e06048d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df67edf40 1 --2- 172.21.15.26:0/3607635474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e060496440 0x55e06048d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df67edf40 1 --2- 172.21.15.26:0/3607635474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0603cd010 0x55e0603cdc80 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df67edf40 1 -- 172.21.15.26:0/3607635474 >> 172.21.15.26:0/3607635474 conn(0x55e0603f5b90 msgr2=0x55e0603c9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df67edf40 1 -- 172.21.15.26:0/3607635474 shutdown_connections 2024-04-24T23:26:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.815+0000 7f8df67edf40 1 -- 172.21.15.26:0/3607635474 wait complete. 2024-04-24T23:26:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.816+0000 7f8df67edf40 1 Processor -- start 2024-04-24T23:26:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.816+0000 7f8df67edf40 1 -- start start 2024-04-24T23:26:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df67edf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0603cd010 0x55e0604892b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df67edf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55e060496440 0x55e06048f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df67edf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e060498e30 0x55e06048f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df67edf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e06048dfc0 con 0x55e060498e30 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df67edf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e06048dcc0 con 0x55e0603cd010 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df67edf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55e06048e560 con 0x55e060496440 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df4aef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e060498e30 0x55e06048f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df4aef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e060498e30 0x55e06048f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40300/0 (socket says 172.21.15.26:40300) 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df4aef640 1 -- 172.21.15.26:0/3445242941 learned_addr learned my addr 172.21.15.26:0/3445242941 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df42ee640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0603cd010 0x55e0604892b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df3aed640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55e060496440 0x55e06048f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df4aef640 1 -- 172.21.15.26:0/3445242941 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55e060496440 msgr2=0x55e06048f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df4aef640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55e060496440 0x55e06048f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df4aef640 1 -- 172.21.15.26:0/3445242941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0603cd010 msgr2=0x55e0604892b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df4aef640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0603cd010 0x55e0604892b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df4aef640 1 -- 172.21.15.26:0/3445242941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0603cdc90 con 0x55e060498e30 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df42ee640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0603cd010 0x55e0604892b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df4aef640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e060498e30 0x55e06048f790 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8de4026070 tx=0x55e06049fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df1ae9640 1 -- 172.21.15.26:0/3445242941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4006950 con 0x55e060498e30 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df1ae9640 1 -- 172.21.15.26:0/3445242941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de4006270 con 0x55e060498e30 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e0603cdf70 con 0x55e060498e30 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df1ae9640 1 -- 172.21.15.26:0/3445242941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de404dad0 con 0x55e060498e30 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.817+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0603f9f90 con 0x55e060498e30 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.819+0000 7f8df1ae9640 1 -- 172.21.15.26:0/3445242941 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8de404dc70 con 0x55e060498e30 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.819+0000 7f8df1ae9640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8dd406a840 0x7f8dd406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.819+0000 7f8df1ae9640 1 -- 172.21.15.26:0/3445242941 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f8de40cc720 con 0x55e060498e30 2024-04-24T23:26:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.819+0000 7f8df42ee640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8dd406a840 0x7f8dd406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.819+0000 7f8df67edf40 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55e060489a50 0x55e06053a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.819+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 --> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] -- osd_op(unknown.0.0:1 12.12 12:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55e06053a6c0 con 0x55e060489a50 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.820+0000 7f8df3aed640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55e060489a50 0x55e06053a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.820+0000 7f8df42ee640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8dd406a840 0x7f8dd406cd00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00c6a0 tx=0x7f8dec009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.820+0000 7f8df3aed640 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55e060489a50 0x55e06053a180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df3aed640 1 -- 172.21.15.26:0/3445242941 <== osd.3 v2:172.21.15.26:6808/2035350946 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v85'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8ddc0099a0 con 0x55e060489a50 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55e060489a50 msgr2=0x55e06053a180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x55e060489a50 0x55e06053a180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8dd406a840 msgr2=0x7f8dd406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8dd406a840 0x7f8dd406cd00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00c6a0 tx=0x7f8dec009000 comp rx=0 tx=0).stop 2024-04-24T23:26:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e060498e30 msgr2=0x55e06048f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 --2- 172.21.15.26:0/3445242941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e060498e30 0x55e06048f790 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8de4026070 tx=0x55e06049fc60 comp rx=0 tx=0).stop 2024-04-24T23:26:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df4aef640 1 -- 172.21.15.26:0/3445242941 reap_dead start 2024-04-24T23:26:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 shutdown_connections 2024-04-24T23:26:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 >> 172.21.15.26:0/3445242941 conn(0x55e0603f5b90 msgr2=0x55e0603c9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 shutdown_connections 2024-04-24T23:26:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.821+0000 7f8df67edf40 1 -- 172.21.15.26:0/3445242941 wait complete. 2024-04-24T23:26:50.828 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-04-24T23:26:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.910+0000 7fba771edf40 1 Processor -- start 2024-04-24T23:26:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.910+0000 7fba771edf40 1 -- start start 2024-04-24T23:26:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba771edf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584953469f0 0x5584952a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba771edf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558495346e80 0x55849533bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba771edf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55849534d990 0x558495277ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba771edf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5584952a27f0 con 0x5584953469f0 2024-04-24T23:26:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba771edf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55849521f640 con 0x55849534d990 2024-04-24T23:26:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba771edf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5584952c1b50 con 0x558495346e80 2024-04-24T23:26:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba74cf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584953469f0 0x5584952a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba74cf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584953469f0 0x5584952a6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40312/0 (socket says 172.21.15.26:40312) 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba74cf0640 1 -- 172.21.15.26:0/662298530 learned_addr learned my addr 172.21.15.26:0/662298530 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba74cf0640 1 -- 172.21.15.26:0/662298530 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558495346e80 msgr2=0x55849533bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba74cf0640 1 --2- 172.21.15.26:0/662298530 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558495346e80 0x55849533bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba74cf0640 1 -- 172.21.15.26:0/662298530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55849534d990 msgr2=0x558495277ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba74cf0640 1 --2- 172.21.15.26:0/662298530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55849534d990 0x558495277ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba74cf0640 1 -- 172.21.15.26:0/662298530 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584952a8bf0 con 0x5584953469f0 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.911+0000 7fba74cf0640 1 --2- 172.21.15.26:0/662298530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584953469f0 0x5584952a6480 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fba6800b240 tx=0x7fba6800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbfa15174f3b0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba6effd640 1 -- 172.21.15.26:0/662298530 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68008070 con 0x5584953469f0 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba6effd640 1 -- 172.21.15.26:0/662298530 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6800d500 con 0x5584953469f0 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba6effd640 1 -- 172.21.15.26:0/662298530 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6800d8f0 con 0x5584953469f0 2024-04-24T23:26:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba771edf40 1 -- 172.21.15.26:0/662298530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584953469f0 msgr2=0x5584952a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba771edf40 1 --2- 172.21.15.26:0/662298530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584953469f0 0x5584952a6480 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fba6800b240 tx=0x7fba6800b700 comp rx=0 tx=0).stop 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba771edf40 1 -- 172.21.15.26:0/662298530 shutdown_connections 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba771edf40 1 --2- 172.21.15.26:0/662298530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55849534d990 0x558495277ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba771edf40 1 --2- 172.21.15.26:0/662298530 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558495346e80 0x55849533bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba771edf40 1 --2- 172.21.15.26:0/662298530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584953469f0 0x5584952a6480 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba771edf40 1 -- 172.21.15.26:0/662298530 >> 172.21.15.26:0/662298530 conn(0x5584952a2b90 msgr2=0x55849534f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba771edf40 1 -- 172.21.15.26:0/662298530 shutdown_connections 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.912+0000 7fba771edf40 1 -- 172.21.15.26:0/662298530 wait complete. 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.913+0000 7fba771edf40 1 Processor -- start 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.913+0000 7fba771edf40 1 -- start start 2024-04-24T23:26:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.913+0000 7fba771edf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5584953469f0 0x558495338940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.913+0000 7fba771edf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558495346e80 0x558495277ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.913+0000 7fba771edf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55849534d990 0x558495278010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.913+0000 7fba771edf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55849533c6b0 con 0x558495346e80 2024-04-24T23:26:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.913+0000 7fba771edf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5584952a89a0 con 0x5584953469f0 2024-04-24T23:26:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.913+0000 7fba771edf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5584952a5f60 con 0x55849534d990 2024-04-24T23:26:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558495346e80 0x558495277ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558495346e80 0x558495277ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40316/0 (socket says 172.21.15.26:40316) 2024-04-24T23:26:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6ffff640 1 -- 172.21.15.26:0/3519159283 learned_addr learned my addr 172.21.15.26:0/3519159283 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba74cf0640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5584953469f0 0x558495338940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6ffff640 1 -- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55849534d990 msgr2=0x558495278010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba754f1640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55849534d990 0x558495278010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6ffff640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55849534d990 0x558495278010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6ffff640 1 -- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5584953469f0 msgr2=0x558495338940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6ffff640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5584953469f0 0x558495338940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6ffff640 1 -- 172.21.15.26:0/3519159283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849533a620 con 0x558495346e80 2024-04-24T23:26:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba74cf0640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5584953469f0 0x558495338940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6ffff640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558495346e80 0x558495277ad0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fba60057040 tx=0x7fba60005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6dffb640 1 -- 172.21.15.26:0/3519159283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60064030 con 0x558495346e80 2024-04-24T23:26:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6dffb640 1 -- 172.21.15.26:0/3519159283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6000a020 con 0x558495346e80 2024-04-24T23:26:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55849533a900 con 0x558495346e80 2024-04-24T23:26:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba6dffb640 1 -- 172.21.15.26:0/3519159283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60059040 con 0x558495346e80 2024-04-24T23:26:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.914+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849533aec0 con 0x558495346e80 2024-04-24T23:26:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.916+0000 7fba6dffb640 1 -- 172.21.15.26:0/3519159283 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba60067070 con 0x558495346e80 2024-04-24T23:26:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.916+0000 7fba6dffb640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba5806a810 0x7fba5806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.916+0000 7fba6dffb640 1 -- 172.21.15.26:0/3519159283 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fba600cd720 con 0x558495346e80 2024-04-24T23:26:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.916+0000 7fba74cf0640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba5806a810 0x7fba5806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.916+0000 7fba771edf40 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55849534cd60 0x558495344a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.916+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- osd_op(unknown.0.0:1 12.5 12:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x558495344fd0 con 0x55849534cd60 2024-04-24T23:26:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.917+0000 7fba74cf0640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba5806a810 0x7fba5806ccd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fba6800b240 tx=0x7fba68008000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.917+0000 7fba754f1640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55849534cd60 0x558495344a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.917+0000 7fba754f1640 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55849534cd60 0x558495344a90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba754f1640 1 -- 172.21.15.26:0/3519159283 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v85'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fba7000c6e0 con 0x55849534cd60 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55849534cd60 msgr2=0x558495344a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x55849534cd60 0x558495344a90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba5806a810 msgr2=0x7fba5806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fba5806a810 0x7fba5806ccd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fba6800b240 tx=0x7fba68008000 comp rx=0 tx=0).stop 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558495346e80 msgr2=0x558495277ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 --2- 172.21.15.26:0/3519159283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558495346e80 0x558495277ad0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fba60057040 tx=0x7fba60005700 comp rx=0 tx=0).stop 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba754f1640 1 -- 172.21.15.26:0/3519159283 reap_dead start 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 shutdown_connections 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 >> 172.21.15.26:0/3519159283 conn(0x5584952a2b90 msgr2=0x558495276b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 shutdown_connections 2024-04-24T23:26:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:50.919+0000 7fba771edf40 1 -- 172.21.15.26:0/3519159283 wait complete. 2024-04-24T23:26:50.930 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-04-24T23:26:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.010+0000 7fbfd0063f40 1 Processor -- start 2024-04-24T23:26:51.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.010+0000 7fbfd0063f40 1 -- start start 2024-04-24T23:26:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfd0063f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac759190 0x55ceac876140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfd0063f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 0x55ceac7e4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfd0063f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac7e74f0 0x55ceac7e5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfd0063f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ceac6b9640 con 0x55ceac759190 2024-04-24T23:26:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfd0063f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ceac6bd680 con 0x55ceac75c0a0 2024-04-24T23:26:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfd0063f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ceac7e9400 con 0x55ceac7e74f0 2024-04-24T23:26:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfcdb64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac759190 0x55ceac876140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfcdb64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac759190 0x55ceac876140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40348/0 (socket says 172.21.15.26:40348) 2024-04-24T23:26:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfcdb64640 1 -- 172.21.15.26:0/812987719 learned_addr learned my addr 172.21.15.26:0/812987719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfcd363640 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 0x55ceac7e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfce365640 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac7e74f0 0x55ceac7e5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfcdb64640 1 -- 172.21.15.26:0/812987719 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac7e74f0 msgr2=0x55ceac7e5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfcdb64640 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac7e74f0 0x55ceac7e5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfcdb64640 1 -- 172.21.15.26:0/812987719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 msgr2=0x55ceac7e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfcdb64640 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 0x55ceac7e4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfcdb64640 1 -- 172.21.15.26:0/812987719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceac73c560 con 0x55ceac759190 2024-04-24T23:26:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.011+0000 7fbfce365640 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac7e74f0 0x55ceac7e5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfcdb64640 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac759190 0x55ceac876140 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800b240 tx=0x7fbfc800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bef7c57a0062b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfc7fff640 1 -- 172.21.15.26:0/812987719 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8008070 con 0x55ceac759190 2024-04-24T23:26:51.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfc7fff640 1 -- 172.21.15.26:0/812987719 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc800d500 con 0x55ceac759190 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfc7fff640 1 -- 172.21.15.26:0/812987719 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800d910 con 0x55ceac759190 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfd0063f40 1 -- 172.21.15.26:0/812987719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac759190 msgr2=0x55ceac876140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfd0063f40 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac759190 0x55ceac876140 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800b240 tx=0x7fbfc800b700 comp rx=0 tx=0).stop 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfd0063f40 1 -- 172.21.15.26:0/812987719 shutdown_connections 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfd0063f40 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac7e74f0 0x55ceac7e5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfd0063f40 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 0x55ceac7e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfd0063f40 1 --2- 172.21.15.26:0/812987719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac759190 0x55ceac876140 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfd0063f40 1 -- 172.21.15.26:0/812987719 >> 172.21.15.26:0/812987719 conn(0x55ceac73cb90 msgr2=0x55ceac75b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfd0063f40 1 -- 172.21.15.26:0/812987719 shutdown_connections 2024-04-24T23:26:51.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.012+0000 7fbfd0063f40 1 -- 172.21.15.26:0/812987719 wait complete. 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.013+0000 7fbfd0063f40 1 Processor -- start 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.013+0000 7fbfd0063f40 1 -- start start 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfd0063f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac759190 0x55ceac7e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfd0063f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 0x55ceac7dea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfd0063f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac7e74f0 0x55ceac7e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfd0063f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ceac7135a0 con 0x55ceac7e74f0 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfd0063f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ceac713350 con 0x55ceac75c0a0 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfd0063f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ceac7e4740 con 0x55ceac759190 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfce365640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac7e74f0 0x55ceac7e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfce365640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac7e74f0 0x55ceac7e5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40350/0 (socket says 172.21.15.26:40350) 2024-04-24T23:26:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfce365640 1 -- 172.21.15.26:0/2289996852 learned_addr learned my addr 172.21.15.26:0/2289996852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfcdb64640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac759190 0x55ceac7e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfcd363640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 0x55ceac7dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfce365640 1 -- 172.21.15.26:0/2289996852 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac759190 msgr2=0x55ceac7e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfce365640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac759190 0x55ceac7e4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfce365640 1 -- 172.21.15.26:0/2289996852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 msgr2=0x55ceac7dea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfce365640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 0x55ceac7dea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfce365640 1 -- 172.21.15.26:0/2289996852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceac742180 con 0x55ceac7e74f0 2024-04-24T23:26:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfcdb64640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ceac759190 0x55ceac7e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfcd363640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceac75c0a0 0x55ceac7dea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfce365640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac7e74f0 0x55ceac7e5b10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0025a80 tx=0x7fbfc00033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfc6ffd640 1 -- 172.21.15.26:0/2289996852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0063460 con 0x55ceac7e74f0 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfc6ffd640 1 -- 172.21.15.26:0/2289996852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc0068070 con 0x55ceac7e74f0 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ceac7e16d0 con 0x55ceac7e74f0 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.014+0000 7fbfc6ffd640 1 -- 172.21.15.26:0/2289996852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0063820 con 0x55ceac7e74f0 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.015+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceac7e1c90 con 0x55ceac7e74f0 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.016+0000 7fbfc6ffd640 1 -- 172.21.15.26:0/2289996852 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbfc0061030 con 0x55ceac7e74f0 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.016+0000 7fbfc6ffd640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfb006a810 0x7fbfb006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.016+0000 7fbfc6ffd640 1 -- 172.21.15.26:0/2289996852 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fbfc00cd800 con 0x55ceac7e74f0 2024-04-24T23:26:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.016+0000 7fbfcdb64640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfb006a810 0x7fbfb006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.017+0000 7fbfd0063f40 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55ceac711710 0x55ceac87e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.017+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 --> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] -- osd_op(unknown.0.0:1 12.9 12:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55ceac87eae0 con 0x55ceac711710 2024-04-24T23:26:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.017+0000 7fbfcd363640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55ceac711710 0x55ceac87e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.017+0000 7fbfcdb64640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfb006a810 0x7fbfb006ccd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x55ceac714630 tx=0x7fbfc8009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.017+0000 7fbfcd363640 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55ceac711710 0x55ceac87e5a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.018+0000 7fbfcd363640 1 -- 172.21.15.26:0/2289996852 <== osd.2 v2:172.21.15.26:6800/3780187135 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v85'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbfb800e3d0 con 0x55ceac711710 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.018+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55ceac711710 msgr2=0x55ceac87e5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.018+0000 7fbfd0063f40 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55ceac711710 0x55ceac87e5a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.018+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfb006a810 msgr2=0x7fbfb006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.018+0000 7fbfd0063f40 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfb006a810 0x7fbfb006ccd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x55ceac714630 tx=0x7fbfc8009000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.018+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac7e74f0 msgr2=0x55ceac7e5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.018+0000 7fbfd0063f40 1 --2- 172.21.15.26:0/2289996852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceac7e74f0 0x55ceac7e5b10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0025a80 tx=0x7fbfc00033a0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.018+0000 7fbfce365640 1 -- 172.21.15.26:0/2289996852 reap_dead start 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.019+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 shutdown_connections 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.019+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 >> 172.21.15.26:0/2289996852 conn(0x55ceac73cb90 msgr2=0x55ceac75b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.019+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 shutdown_connections 2024-04-24T23:26:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.019+0000 7fbfd0063f40 1 -- 172.21.15.26:0/2289996852 wait complete. 2024-04-24T23:26:51.027 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-04-24T23:26:51.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.108+0000 7f1385492f40 1 Processor -- start 2024-04-24T23:26:51.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.108+0000 7f1385492f40 1 -- start start 2024-04-24T23:26:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1385492f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581a90 0x563593646d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1385492f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563593581e70 0x5635936561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1385492f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635935c6250 0x5635936566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1385492f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5635935c69b0 con 0x563593581e70 2024-04-24T23:26:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1385492f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563593526640 con 0x563593581a90 2024-04-24T23:26:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1385492f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5635935c6c90 con 0x5635935c6250 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382794640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563593581e70 0x5635936561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382794640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563593581e70 0x5635936561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40356/0 (socket says 172.21.15.26:40356) 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382794640 1 -- 172.21.15.26:0/3799884631 learned_addr learned my addr 172.21.15.26:0/3799884631 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382794640 1 -- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635935c6250 msgr2=0x5635936566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382f95640 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581a90 0x563593646d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1383796640 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635935c6250 0x5635936566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382794640 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635935c6250 0x5635936566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382794640 1 -- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581a90 msgr2=0x563593646d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382794640 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581a90 0x563593646d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382794640 1 -- 172.21.15.26:0/3799884631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635935a9410 con 0x563593581e70 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1383796640 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635935c6250 0x5635936566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382f95640 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581a90 0x563593646d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1382794640 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563593581e70 0x5635936561b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f136c00a4f0 tx=0x7f136c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e6b50d9958a2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1381792640 1 -- 172.21.15.26:0/3799884631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c053800 con 0x563593581e70 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1381792640 1 -- 172.21.15.26:0/3799884631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c063070 con 0x563593581e70 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.109+0000 7f1381792640 1 -- 172.21.15.26:0/3799884631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c053ba0 con 0x563593581e70 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.110+0000 7f1385492f40 1 -- 172.21.15.26:0/3799884631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563593581e70 msgr2=0x5635936561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.110+0000 7f1385492f40 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563593581e70 0x5635936561b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f136c00a4f0 tx=0x7f136c051000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.110+0000 7f1385492f40 1 -- 172.21.15.26:0/3799884631 shutdown_connections 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.110+0000 7f1385492f40 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5635935c6250 0x5635936566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.110+0000 7f1385492f40 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563593581e70 0x5635936561b0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.110+0000 7f1385492f40 1 --2- 172.21.15.26:0/3799884631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581a90 0x563593646d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.110+0000 7f1385492f40 1 -- 172.21.15.26:0/3799884631 >> 172.21.15.26:0/3799884631 conn(0x5635935a9b90 msgr2=0x563593651140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.110+0000 7f1385492f40 1 -- 172.21.15.26:0/3799884631 shutdown_connections 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.110+0000 7f1385492f40 1 -- 172.21.15.26:0/3799884631 wait complete. 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1385492f40 1 Processor -- start 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1385492f40 1 -- start start 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1385492f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563593581a90 0x563593645340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1385492f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581e70 0x563593645880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1385492f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635935c6250 0x56359364a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1385492f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563593650320 con 0x5635935c6250 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1385492f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5635936475c0 con 0x563593581e70 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1385492f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563593647810 con 0x563593581a90 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1383796640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635935c6250 0x56359364a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1383796640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635935c6250 0x56359364a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40370/0 (socket says 172.21.15.26:40370) 2024-04-24T23:26:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1383796640 1 -- 172.21.15.26:0/3385787398 learned_addr learned my addr 172.21.15.26:0/3385787398 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1382f95640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563593581a90 0x563593645340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.111+0000 7f1382794640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581e70 0x563593645880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f1383796640 1 -- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563593581a90 msgr2=0x563593645340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f1383796640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563593581a90 0x563593645340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f1383796640 1 -- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581e70 msgr2=0x563593645880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f1383796640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563593581e70 0x563593645880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f1383796640 1 -- 172.21.15.26:0/3385787398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635936466b0 con 0x5635935c6250 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f1383796640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635935c6250 0x56359364a3c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1374004250 tx=0x7f137400c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f137bfff640 1 -- 172.21.15.26:0/3385787398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374017070 con 0x5635935c6250 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f137bfff640 1 -- 172.21.15.26:0/3385787398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137400ce10 con 0x5635935c6250 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563593647ca0 con 0x5635935c6250 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f137bfff640 1 -- 172.21.15.26:0/3385787398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374001760 con 0x5635935c6250 2024-04-24T23:26:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.112+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563593648260 con 0x5635935c6250 2024-04-24T23:26:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.113+0000 7f137bfff640 1 -- 172.21.15.26:0/3385787398 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f137401a070 con 0x5635935c6250 2024-04-24T23:26:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.114+0000 7f137bfff640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f136406a810 0x7f136406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.114+0000 7f137bfff640 1 -- 172.21.15.26:0/3385787398 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f1374089290 con 0x5635935c6250 2024-04-24T23:26:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.114+0000 7f1382f95640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f136406a810 0x7f136406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.114+0000 7f1385492f40 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56359364a900 0x563593648780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.114+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- osd_op(unknown.0.0:1 12.5 12:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x563593648cc0 con 0x56359364a900 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.114+0000 7f1382794640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56359364a900 0x563593648780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.114+0000 7f1382f95640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f136406a810 0x7f136406ccd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f137c009820 tx=0x7f137c005a00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.115+0000 7f1382794640 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56359364a900 0x563593648780 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1382794640 1 -- 172.21.15.26:0/3385787398 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v85'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f136c051070 con 0x56359364a900 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56359364a900 msgr2=0x563593648780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x56359364a900 0x563593648780 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f136406a810 msgr2=0x7f136406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f136406a810 0x7f136406ccd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f137c009820 tx=0x7f137c005a00 comp rx=0 tx=0).stop 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635935c6250 msgr2=0x56359364a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 --2- 172.21.15.26:0/3385787398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5635935c6250 0x56359364a3c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1374004250 tx=0x7f137400c5e0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1383796640 1 -- 172.21.15.26:0/3385787398 reap_dead start 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 shutdown_connections 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 >> 172.21.15.26:0/3385787398 conn(0x5635935a9b90 msgr2=0x563593656190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 shutdown_connections 2024-04-24T23:26:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.116+0000 7f1385492f40 1 -- 172.21.15.26:0/3385787398 wait complete. 2024-04-24T23:26:51.122 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-04-24T23:26:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.203+0000 7efe93e7af40 1 Processor -- start 2024-04-24T23:26:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.203+0000 7efe93e7af40 1 -- start start 2024-04-24T23:26:51.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.203+0000 7efe93e7af40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617a3d7acd0 0x5617a3dbb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.203+0000 7efe93e7af40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d92730 0x5617a3efa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.203+0000 7efe93e7af40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a3e4f090 0x5617a3efa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.203+0000 7efe93e7af40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5617a3d37640 con 0x5617a3d92730 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.203+0000 7efe93e7af40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617a3d3b680 con 0x5617a3e4f090 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.203+0000 7efe93e7af40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5617a3e66500 con 0x5617a3d7acd0 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9117a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d92730 0x5617a3efa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9117a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d92730 0x5617a3efa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40388/0 (socket says 172.21.15.26:40388) 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9117a640 1 -- 172.21.15.26:0/1120997438 learned_addr learned my addr 172.21.15.26:0/1120997438 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9217c640 1 --2- 172.21.15.26:0/1120997438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a3e4f090 0x5617a3efa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9117a640 1 -- 172.21.15.26:0/1120997438 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617a3d7acd0 msgr2=0x5617a3dbb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9197b640 1 --2- 172.21.15.26:0/1120997438 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617a3d7acd0 0x5617a3dbb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9117a640 1 --2- 172.21.15.26:0/1120997438 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617a3d7acd0 0x5617a3dbb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9117a640 1 -- 172.21.15.26:0/1120997438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a3e4f090 msgr2=0x5617a3efa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9117a640 1 --2- 172.21.15.26:0/1120997438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a3e4f090 0x5617a3efa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9117a640 1 -- 172.21.15.26:0/1120997438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a3e66070 con 0x5617a3d92730 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe9117a640 1 --2- 172.21.15.26:0/1120997438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d92730 0x5617a3efa090 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7efe7c001e00 tx=0x7efe7c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313e12397f4566ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe8bfff640 1 -- 172.21.15.26:0/1120997438 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00be40 con 0x5617a3d92730 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe8bfff640 1 -- 172.21.15.26:0/1120997438 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c00f400 con 0x5617a3d92730 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.204+0000 7efe8bfff640 1 -- 172.21.15.26:0/1120997438 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00f7d0 con 0x5617a3d92730 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 -- 172.21.15.26:0/1120997438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d92730 msgr2=0x5617a3efa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 --2- 172.21.15.26:0/1120997438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d92730 0x5617a3efa090 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7efe7c001e00 tx=0x7efe7c00b670 comp rx=0 tx=0).stop 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 -- 172.21.15.26:0/1120997438 shutdown_connections 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 --2- 172.21.15.26:0/1120997438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a3e4f090 0x5617a3efa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 --2- 172.21.15.26:0/1120997438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d92730 0x5617a3efa090 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 --2- 172.21.15.26:0/1120997438 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617a3d7acd0 0x5617a3dbb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 -- 172.21.15.26:0/1120997438 >> 172.21.15.26:0/1120997438 conn(0x5617a3d90fd0 msgr2=0x5617a3d8e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 -- 172.21.15.26:0/1120997438 shutdown_connections 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 -- 172.21.15.26:0/1120997438 wait complete. 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 Processor -- start 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.205+0000 7efe93e7af40 1 -- start start 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe93e7af40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d7acd0 0x5617a3dd07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe93e7af40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a3d92730 0x5617a3dd9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe93e7af40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617a3e4f090 0x5617a3e47c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe93e7af40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5617a3dbed60 con 0x5617a3d7acd0 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe93e7af40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617a3dda1a0 con 0x5617a3d92730 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe93e7af40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5617a3dbbb30 con 0x5617a3e4f090 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe9197b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d7acd0 0x5617a3dd07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe9197b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d7acd0 0x5617a3dd07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40392/0 (socket says 172.21.15.26:40392) 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe9197b640 1 -- 172.21.15.26:0/265955061 learned_addr learned my addr 172.21.15.26:0/265955061 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe9117a640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a3d92730 0x5617a3dd9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.206+0000 7efe9217c640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617a3e4f090 0x5617a3e47c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe9197b640 1 -- 172.21.15.26:0/265955061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617a3e4f090 msgr2=0x5617a3e47c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe9197b640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617a3e4f090 0x5617a3e47c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe9197b640 1 -- 172.21.15.26:0/265955061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a3d92730 msgr2=0x5617a3dd9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe9197b640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617a3d92730 0x5617a3dd9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe9197b640 1 -- 172.21.15.26:0/265955061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a3dd1100 con 0x5617a3d7acd0 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe9197b640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d7acd0 0x5617a3dd07d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7efe8400c480 tx=0x7efe8400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe8affd640 1 -- 172.21.15.26:0/265955061 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe840062e0 con 0x5617a3d7acd0 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe8affd640 1 -- 172.21.15.26:0/265955061 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe84003650 con 0x5617a3d7acd0 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5617a3dd13e0 con 0x5617a3d7acd0 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe8affd640 1 -- 172.21.15.26:0/265955061 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84003a40 con 0x5617a3d7acd0 2024-04-24T23:26:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.207+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a3dd19a0 con 0x5617a3d7acd0 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.208+0000 7efe8affd640 1 -- 172.21.15.26:0/265955061 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe8401a020 con 0x5617a3d7acd0 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.209+0000 7efe8affd640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe7406a810 0x7efe7406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.209+0000 7efe8affd640 1 -- 172.21.15.26:0/265955061 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7efe84089760 con 0x5617a3d7acd0 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.209+0000 7efe9117a640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe7406a810 0x7efe7406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.209+0000 7efe93e7af40 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5617a3dd9750 0x5617a3dd4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.209+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- osd_op(unknown.0.0:1 12.1d 12:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x5617a3dd4fe0 con 0x5617a3dd9750 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.209+0000 7efe9217c640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5617a3dd9750 0x5617a3dd4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.209+0000 7efe9117a640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe7406a810 0x7efe7406ccd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x5617a3d8f2d0 tx=0x7efe7c009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.210+0000 7efe9217c640 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5617a3dd9750 0x5617a3dd4aa0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe9217c640 1 -- 172.21.15.26:0/265955061 <== osd.0 v2:172.21.15.26:6819/256768257 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v85'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efe8c054650 con 0x5617a3dd9750 2024-04-24T23:26:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5617a3dd9750 msgr2=0x5617a3dd4aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5617a3dd9750 0x5617a3dd4aa0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe7406a810 msgr2=0x7efe7406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe7406a810 0x7efe7406ccd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x5617a3d8f2d0 tx=0x7efe7c009000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d7acd0 msgr2=0x5617a3dd07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 --2- 172.21.15.26:0/265955061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a3d7acd0 0x5617a3dd07d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7efe8400c480 tx=0x7efe8400c940 comp rx=0 tx=0).stop 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe9217c640 1 -- 172.21.15.26:0/265955061 reap_dead start 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 shutdown_connections 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 >> 172.21.15.26:0/265955061 conn(0x5617a3d90fd0 msgr2=0x5617a3dbf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 shutdown_connections 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.211+0000 7efe93e7af40 1 -- 172.21.15.26:0/265955061 wait complete. 2024-04-24T23:26:51.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:51.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /d1/d2/d3 recursive,force 2024-04-24T23:26:51.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.360+0000 7f8c1d590640 1 Processor -- start 2024-04-24T23:26:51.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.360+0000 7f8c1d590640 1 -- start start 2024-04-24T23:26:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.361+0000 7f8c1d590640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8c1815e330 0x7f8c180d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.361+0000 7f8c1d590640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c1815e710 0x7f8c180d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.361+0000 7f8c1d590640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c180d0170 con 0x7f8c180cfcc0 2024-04-24T23:26:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.361+0000 7f8c1d590640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8c180d27b0 con 0x7f8c1815e710 2024-04-24T23:26:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.361+0000 7f8c1d590640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8c180cef00 con 0x7f8c1815e330 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.361+0000 7f8c167fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c1815e710 0x7f8c180d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.361+0000 7f8c167fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c1815e710 0x7f8c180d15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40396/0 (socket says 172.21.15.26:40396) 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.361+0000 7f8c167fc640 1 -- 172.21.15.26:0/3905027899 learned_addr learned my addr 172.21.15.26:0/3905027899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.362+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/3905027899 >> v2:172.21.15.29:3300/0 conn(0x7f8c1815e330 0x7f8c180d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.362+0000 7f8c167fc640 1 -- 172.21.15.26:0/3905027899 >> v2:172.21.15.29:3300/0 conn(0x7f8c1815e330 msgr2=0x7f8c180d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.362+0000 7f8c167fc640 1 --2- 172.21.15.26:0/3905027899 >> v2:172.21.15.29:3300/0 conn(0x7f8c1815e330 0x7f8c180d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.362+0000 7f8c167fc640 1 -- 172.21.15.26:0/3905027899 >> v1:172.21.15.29:6789/0 conn(0x7f8c180cfcc0 legacy=0x7f8c180d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.362+0000 7f8c167fc640 1 -- 172.21.15.26:0/3905027899 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c180cf1e0 con 0x7f8c1815e710 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.362+0000 7f8c167fc640 1 --2- 172.21.15.26:0/3905027899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e710 0x7f8c180d15e0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f8c18068af0 tx=0x7f8c0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36e7466c380cdf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c15ffb640 1 -- 172.21.15.26:0/3905027899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00037070 con 0x7f8c1815e710 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c15ffb640 1 -- 172.21.15.26:0/3905027899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c00032440 con 0x7f8c1815e710 2024-04-24T23:26:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c15ffb640 1 -- 172.21.15.26:0/3905027899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00032810 con 0x7f8c1815e710 2024-04-24T23:26:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c1d590640 1 -- 172.21.15.26:0/3905027899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e710 msgr2=0x7f8c180d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c1d590640 1 --2- 172.21.15.26:0/3905027899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e710 0x7f8c180d15e0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f8c18068af0 tx=0x7f8c0002a270 comp rx=0 tx=0).stop 2024-04-24T23:26:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c1d590640 1 -- 172.21.15.26:0/3905027899 shutdown_connections 2024-04-24T23:26:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c1d590640 1 --2- 172.21.15.26:0/3905027899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e710 0x7f8c180d15e0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c1d590640 1 --2- 172.21.15.26:0/3905027899 >> v2:172.21.15.29:3300/0 conn(0x7f8c1815e330 0x7f8c180d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c1d590640 1 -- 172.21.15.26:0/3905027899 >> 172.21.15.26:0/3905027899 conn(0x7f8c1808dbc0 msgr2=0x7f8c1809d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c1d590640 1 -- 172.21.15.26:0/3905027899 shutdown_connections 2024-04-24T23:26:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.363+0000 7f8c1d590640 1 -- 172.21.15.26:0/3905027899 wait complete. 2024-04-24T23:26:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.364+0000 7f8c1d590640 1 Processor -- start 2024-04-24T23:26:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.364+0000 7f8c1d590640 1 -- start start 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c1d590640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c180cfcc0 0x7f8c181f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c1d590640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e330 0x7f8c180cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c1d590640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c1815e710 0x7f8c180cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c1d590640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c180d4630 con 0x7f8c1815e330 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c1d590640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c180cf7a0 con 0x7f8c180cfcc0 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c1d590640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8c180d47b0 con 0x7f8c1815e710 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c167fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e330 0x7f8c180cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c167fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e330 0x7f8c180cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40410/0 (socket says 172.21.15.26:40410) 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c167fc640 1 -- 172.21.15.26:0/533511067 learned_addr learned my addr 172.21.15.26:0/533511067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c180cfcc0 0x7f8c181f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.365+0000 7f8c177fe640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c1815e710 0x7f8c180cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c167fc640 1 -- 172.21.15.26:0/533511067 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c1815e710 msgr2=0x7f8c180cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c167fc640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c1815e710 0x7f8c180cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c167fc640 1 -- 172.21.15.26:0/533511067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c180cfcc0 msgr2=0x7f8c181f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c167fc640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c180cfcc0 0x7f8c181f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c167fc640 1 -- 172.21.15.26:0/533511067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c180ce960 con 0x7f8c1815e330 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c167fc640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e330 0x7f8c180cb280 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f8c00000c00 tx=0x7f8c000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c0f7fe640 1 -- 172.21.15.26:0/533511067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00041020 con 0x7f8c1815e330 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c0f7fe640 1 -- 172.21.15.26:0/533511067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c000306c0 con 0x7f8c1815e330 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c1d590640 1 -- 172.21.15.26:0/533511067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c180cd8c0 con 0x7f8c1815e330 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c0f7fe640 1 -- 172.21.15.26:0/533511067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00037040 con 0x7f8c1815e330 2024-04-24T23:26:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.366+0000 7f8c1d590640 1 -- 172.21.15.26:0/533511067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c180cde80 con 0x7f8c1815e330 2024-04-24T23:26:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.368+0000 7f8c0f7fe640 1 -- 172.21.15.26:0/533511067 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c00044020 con 0x7f8c1815e330 2024-04-24T23:26:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.368+0000 7f8c0f7fe640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bf806be30 0x7f8bf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.368+0000 7f8c0f7fe640 1 -- 172.21.15.26:0/533511067 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f8c000ab500 con 0x7f8c1815e330 2024-04-24T23:26:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.368+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bf806be30 0x7f8bf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.369+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bf806be30 0x7f8bf806e2f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8c10006fd0 tx=0x7f8c10008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.369+0000 7f8c20379740 1 Processor -- start 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.369+0000 7f8c20379740 1 -- start start 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c20379740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 0x55f2e340e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c20379740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e33f3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c20379740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e33ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c20379740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2e332b490 con 0x55f2e33f3580 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c20379740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f2e32d3ae0 con 0x55f2e33f2c60 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c20379740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f2e3327450 con 0x55f2e3404680 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e33ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e33ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40414/0 (socket says 172.21.15.26:40414) 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c177fe640 1 -- 172.21.15.26:0/1441705103 learned_addr learned my addr 172.21.15.26:0/1441705103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 0x55f2e340e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c167fc640 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e33f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c177fe640 1 -- 172.21.15.26:0/1441705103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 msgr2=0x55f2e33f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c177fe640 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e33f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c177fe640 1 -- 172.21.15.26:0/1441705103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 msgr2=0x55f2e340e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c177fe640 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 0x55f2e340e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c177fe640 1 -- 172.21.15.26:0/1441705103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2e3107a50 con 0x55f2e33f3580 2024-04-24T23:26:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.370+0000 7f8c167fc640 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e33f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c177fe640 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e33ef840 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8c08001f00 tx=0x7f8c08001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86879c0e4813f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c0cff9640 1 -- 172.21.15.26:0/1441705103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800fdf0 con 0x55f2e33f3580 2024-04-24T23:26:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c0cff9640 1 -- 172.21.15.26:0/1441705103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c08018070 con 0x55f2e33f3580 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c0cff9640 1 -- 172.21.15.26:0/1441705103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08003b70 con 0x55f2e33f3580 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c20379740 1 -- 172.21.15.26:0/1441705103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 msgr2=0x55f2e33ef840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c20379740 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e33ef840 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8c08001f00 tx=0x7f8c08001670 comp rx=0 tx=0).stop 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c20379740 1 -- 172.21.15.26:0/1441705103 shutdown_connections 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c20379740 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e33ef840 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c20379740 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e33f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c20379740 1 --2- 172.21.15.26:0/1441705103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 0x55f2e340e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c20379740 1 -- 172.21.15.26:0/1441705103 >> 172.21.15.26:0/1441705103 conn(0x55f2e338f5e0 msgr2=0x55f2e340da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c20379740 1 -- 172.21.15.26:0/1441705103 shutdown_connections 2024-04-24T23:26:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.371+0000 7f8c20379740 1 -- 172.21.15.26:0/1441705103 wait complete. 2024-04-24T23:26:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.372+0000 7f8c20379740 1 -- start start 2024-04-24T23:26:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.372+0000 7f8c20379740 1 Processor -- start 2024-04-24T23:26:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c20379740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f2c60 0x55f2e340bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c20379740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 0x55f2e340dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c20379740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e340e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c20379740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2e33c9fa0 con 0x55f2e33f2c60 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c20379740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f2e32e7060 con 0x55f2e33f3580 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c20379740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f2e33c4820 con 0x55f2e3404680 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c20379740 10 client.0 _ll_register_callbacks cb 0x55f2e3322970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c16ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f2c60 0x55f2e340bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c16ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f2c60 0x55f2e340bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40422/0 (socket says 172.21.15.26:40422) 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c16ffd640 1 -- 172.21.15.26:0/3632969937 learned_addr learned my addr 172.21.15.26:0/3632969937 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c167fc640 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 0x55f2e340dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c177fe640 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e340e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c16ffd640 1 -- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 msgr2=0x55f2e340e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e340e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c16ffd640 1 -- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 msgr2=0x55f2e340dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 0x55f2e340dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c16ffd640 1 -- 172.21.15.26:0/3632969937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2e2fff3e0 con 0x55f2e33f2c60 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c177fe640 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e340e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f2c60 0x55f2e340bc60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8c100167d0 tx=0x7f8c10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e49f701badc61a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.373+0000 7f8bf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/3632969937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10026020 con 0x55f2e33f2c60 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/3632969937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f8c100149b0 con 0x55f2e33f2c60 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c20379740 1 -- 172.21.15.26:0/3632969937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c08018070 con 0x55f2e33f2c60 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/3632969937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f8c10030050 con 0x55f2e33f2c60 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c20379740 10 client.8232 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c20379740 1 -- 172.21.15.26:0/3632969937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55f2e31681e0 con 0x55f2e33f2c60 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/3632969937 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c08018070 con 0x55f2e33f2c60 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/3632969937 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f8c10033020 con 0x55f2e33f2c60 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c20379740 10 client.8232 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c20379740 10 client.8232 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c20379740 20 client.8232 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c20379740 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f2e33ea0c0 0x55f2e340ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c20379740 4 client.8232 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c20379740 1 -- 172.21.15.26:0/3632969937 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2e33389d0 con 0x55f2e33ea0c0 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.374+0000 7f8c167fc640 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f2e33ea0c0 0x55f2e340ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.375+0000 7f8bf5ffb640 10 client.8232 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.375+0000 7f8c167fc640 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f2e33ea0c0 0x55f2e340ca50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84643b9c8f7ca415 server_cookie=461872f845e7722f in_seq=0 out_seq=0 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.375+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/3632969937 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2e33389d0 con 0x55f2e33ea0c0 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.375+0000 7f8bf5ffb640 10 client.8232 handle_command_reply: tid=0 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 client.8232 shutdown 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 -- 172.21.15.26:0/3632969937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f2c60 msgr2=0x55f2e340bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f2c60 0x55f2e340bc60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8c100167d0 tx=0x7f8c10004730 comp rx=0 tx=0).stop 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 -- 172.21.15.26:0/3632969937 shutdown_connections 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f2e33ea0c0 0x55f2e340ca50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e340e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 0x55f2e340dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 --2- 172.21.15.26:0/3632969937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f2c60 0x55f2e340bc60 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8bf5ffb640 0 client.8232 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 -- 172.21.15.26:0/3632969937 >> 172.21.15.26:0/3632969937 conn(0x55f2e338f5e0 msgr2=0x55f2e33eae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 -- 172.21.15.26:0/3632969937 shutdown_connections 2024-04-24T23:26:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 1 -- 172.21.15.26:0/3632969937 wait complete. 2024-04-24T23:26:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.376+0000 7f8c20379740 20 client.8232 trim_cache size 0 max 16384 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.394+0000 7f8c20379740 1 Processor -- start 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.394+0000 7f8c20379740 1 -- start start 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c20379740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e33f2c60 0x55f2e340a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c20379740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 0x55f2e340b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c20379740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e3404680 0x55f2e33f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c20379740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18200880 con 0x55f2e3404680 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c20379740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18151640 con 0x55f2e33f3580 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c20379740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8c18200150 con 0x55f2e33f2c60 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e3404680 0x55f2e33f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e3404680 0x55f2e33f56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40434/0 (socket says 172.21.15.26:40434) 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c177fe640 1 -- 172.21.15.26:0/3183681686 learned_addr learned my addr 172.21.15.26:0/3183681686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c167fc640 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 0x55f2e340b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e33f2c60 0x55f2e340a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c177fe640 1 -- 172.21.15.26:0/3183681686 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e33f2c60 msgr2=0x55f2e340a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c177fe640 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e33f2c60 0x55f2e340a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c177fe640 1 -- 172.21.15.26:0/3183681686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 msgr2=0x55f2e340b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c177fe640 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 0x55f2e340b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c177fe640 1 -- 172.21.15.26:0/3183681686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2e2fd84a0 con 0x55f2e3404680 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.395+0000 7f8c167fc640 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 0x55f2e340b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c177fe640 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e3404680 0x55f2e33f56a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8c08014000 tx=0x7f8c0800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2851cf3284ad68c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8bf7fff640 1 -- 172.21.15.26:0/3183681686 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08026070 con 0x55f2e3404680 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8bf7fff640 1 -- 172.21.15.26:0/3183681686 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c08018030 con 0x55f2e3404680 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8bf7fff640 1 -- 172.21.15.26:0/3183681686 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08003bb0 con 0x55f2e3404680 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c20379740 1 -- 172.21.15.26:0/3183681686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e3404680 msgr2=0x55f2e33f56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c20379740 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e3404680 0x55f2e33f56a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8c08014000 tx=0x7f8c0800a000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c20379740 1 -- 172.21.15.26:0/3183681686 shutdown_connections 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c20379740 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e3404680 0x55f2e33f56a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c20379740 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f3580 0x55f2e340b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c20379740 1 --2- 172.21.15.26:0/3183681686 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e33f2c60 0x55f2e340a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c20379740 1 -- 172.21.15.26:0/3183681686 >> 172.21.15.26:0/3183681686 conn(0x55f2e338f5e0 msgr2=0x55f2e341d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c20379740 1 -- 172.21.15.26:0/3183681686 shutdown_connections 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.396+0000 7f8c20379740 1 -- 172.21.15.26:0/3183681686 wait complete. 2024-04-24T23:26:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.397+0000 7f8c20379740 1 -- start start 2024-04-24T23:26:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.397+0000 7f8c20379740 1 Processor -- start 2024-04-24T23:26:51.401 INFO:tasks.ceph.mds.b.smithi029.stderr:2024-04-24T23:26:51.401+0000 7f43fc153640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1) n(v3 rc2024-04-24T23:26:34.115974+0000 6=0+6) (inest mix dirty) (ifile excl) caps={6901=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6901 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x556ffeee3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:34.115974+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-04-24T23:26:34.115974+0000 6=0+6)","error_str":""},"return_code":-61} 2024-04-24T23:26:51.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c20379740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 0x55f2e341c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c20379740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e3403ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c20379740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e3404000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c20379740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2e33389d0 con 0x55f2e33f3580 2024-04-24T23:26:51.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c20379740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1814e5b0 con 0x55f2e33f2c60 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c20379740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f2e33f5260 con 0x55f2e3404680 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c20379740 10 client.0 _ll_register_callbacks cb 0x7f8c10007790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c167fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e3403ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c167fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e3403ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40438/0 (socket says 172.21.15.26:40438) 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c167fc640 1 -- 172.21.15.26:0/1976901552 learned_addr learned my addr 172.21.15.26:0/1976901552 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 0x55f2e341c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c177fe640 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e3404000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c167fc640 1 -- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 msgr2=0x55f2e3404000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c167fc640 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e3404000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c167fc640 1 -- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 msgr2=0x55f2e341c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c167fc640 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 0x55f2e341c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c167fc640 1 -- 172.21.15.26:0/1976901552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2e341c890 con 0x55f2e33f3580 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c167fc640 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e3403ac0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8c00075d10 tx=0x7f8c000ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd1d9054cb79ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8bf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/1976901552 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c000a9060 con 0x55f2e33f3580 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/1976901552 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f8c00075f20 con 0x55f2e33f3580 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.398+0000 7f8c20379740 1 -- 172.21.15.26:0/1976901552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c08018030 con 0x55f2e33f3580 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/1976901552 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f8c00098050 con 0x55f2e33f3580 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c20379740 10 client.8238 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c20379740 1 -- 172.21.15.26:0/1976901552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55f2e33c6880 con 0x55f2e33f3580 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/1976901552 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c08018030 con 0x55f2e33f3580 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/1976901552 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f8c0003a030 con 0x55f2e33f3580 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c20379740 10 client.8238 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c20379740 10 client.8238 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c20379740 20 client.8238 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c20379740 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f2e3411640 0x55f2e3486970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c20379740 4 client.8238 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c20379740 1 -- 172.21.15.26:0/1976901552 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x55f2e3332d20 con 0x55f2e3411640 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f2e3411640 0x55f2e3486970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8c16ffd640 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f2e3411640 0x55f2e3486970 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a164f2cf4f7466b server_cookie=7da659be720435b2 in_seq=0 out_seq=0 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.399+0000 7f8bf5ffb640 10 client.8238 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8bf5ffb640 1 -- 172.21.15.26:0/1976901552 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f2e3332d20 con 0x55f2e3411640 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8bf5ffb640 10 client.8238 handle_command_reply: tid=0 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 client.8238 shutdown 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 -- 172.21.15.26:0/1976901552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 msgr2=0x55f2e3403ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e3403ac0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8c00075d10 tx=0x7f8c000ab040 comp rx=0 tx=0).stop 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 -- 172.21.15.26:0/1976901552 shutdown_connections 2024-04-24T23:26:51.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f2e3411640 0x55f2e3486970 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2e3404680 0x55f2e3404000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2e33f3580 0x55f2e3403ac0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8bf5ffb640 0 client.8238 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 --2- 172.21.15.26:0/1976901552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2e33f2c60 0x55f2e341c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 -- 172.21.15.26:0/1976901552 >> 172.21.15.26:0/1976901552 conn(0x55f2e338f5e0 msgr2=0x55f2e33f56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.400+0000 7f8c20379740 1 -- 172.21.15.26:0/1976901552 shutdown_connections 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c20379740 1 -- 172.21.15.26:0/1976901552 wait complete. 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c20379740 20 client.8238 trim_cache size 0 max 16384 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-04-24T23:26:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "cca3b3ba-5fb5-4b95-8bce-070d29d5babd", 2024-04-24T23:26:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-04-24T23:26:51.410 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:51.410 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f6800b1f640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1) n(v2 rc2024-04-24T23:26:34.115974+0000 4=0+4) (inest lock) (ifile excl) caps={6901=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6901 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x563badca3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:34.115974+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-04-24T23:26:34.115974+0000 4=0+4)","error_str":""},"return_code":-61} 2024-04-24T23:26:51.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 -- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bf806be30 msgr2=0x7f8bf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bf806be30 0x7f8bf806e2f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8c10006fd0 tx=0x7f8c10008040 comp rx=0 tx=0).stop 2024-04-24T23:26:51.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 -- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e330 msgr2=0x7f8c180cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e330 0x7f8c180cb280 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f8c00000c00 tx=0x7f8c000046c0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 -- 172.21.15.26:0/533511067 shutdown_connections 2024-04-24T23:26:51.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bf806be30 0x7f8bf806e2f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c1815e710 0x7f8c180cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c1815e330 0x7f8c180cb280 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 --2- 172.21.15.26:0/533511067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c180cfcc0 0x7f8c181f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 -- 172.21.15.26:0/533511067 >> 172.21.15.26:0/533511067 conn(0x7f8c1808dbc0 msgr2=0x7f8c180b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 -- 172.21.15.26:0/533511067 shutdown_connections 2024-04-24T23:26:51.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.401+0000 7f8c1d590640 1 -- 172.21.15.26:0/533511067 wait complete. 2024-04-24T23:26:51.412 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-04-24T23:26:51.403+0000 7f680732c640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1) n(v1 rc2024-04-24T23:26:34.115974+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={6901=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x563bacd47080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:33.884977+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-04-24T23:26:34.115974+0000 3=0+3)","error_str":""},"return_code":-61} 2024-04-24T23:26:51.413 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:26:51.402+0000 7f8211070640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-04-24T23:26:33.884977+0000 1=0+1) n(v0 rc2024-04-24T23:26:33.884977+0000 2=0+2) (inest lock) (ifile excl) caps={6901=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6901 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5632ac726580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.884977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:33.884977+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.884977+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-04-24T23:26:33.884977+0000 2=0+2)","error_str":""},"return_code":-61} 2024-04-24T23:26:51.413 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:26:51.402+0000 7f8211070640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={6901=pAsLsXsFsx/AsLsXsFsx@2},l=6901 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5632ac726b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-04-24T23:26:51.413 INFO:tasks.ceph.mds.b.smithi029.stderr:2024-04-24T23:26:51.402+0000 7f4402960640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1) n(v4 rc2024-04-24T23:26:34.115974+0000 5=0+5) (inest sync->mix g=1,2) caps={6901=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x556ffeee3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:34.115974+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-04-24T23:26:34.115974+0000 5=0+5)","error_str":""},"return_code":-61} 2024-04-24T23:26:51.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:51.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub pause 2024-04-24T23:26:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.518+0000 7fa882b5c640 1 Processor -- start 2024-04-24T23:26:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.518+0000 7fa882b5c640 1 -- start start 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa882b5c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa87c0d28e0 0x7fa87c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa882b5c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87c06d750 con 0x7fa87c0ce640 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa882b5c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87c0b1720 con 0x7fa87c15da30 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa882b5c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa87c0d1110 con 0x7fa87c0d28e0 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa87bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa87c0d28e0 0x7fa87c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa8808f8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa87c0ce640 0x7fa87c0cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51582/0 (socket says 172.21.15.26:51582) 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa8808f8640 1 -- 172.21.15.26:0/829734780 learned_addr learned my addr 172.21.15.26:0/829734780 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa87bfff640 1 -- 172.21.15.26:0/829734780 >> v1:172.21.15.29:6789/0 conn(0x7fa87c15da30 legacy=0x7fa87c15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa87bfff640 1 -- 172.21.15.26:0/829734780 >> v1:172.21.15.26:6789/0 conn(0x7fa87c0ce640 legacy=0x7fa87c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.519+0000 7fa87bfff640 1 -- 172.21.15.26:0/829734780 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c0d1360 con 0x7fa87c0d28e0 2024-04-24T23:26:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.520+0000 7fa87bfff640 1 --2- 172.21.15.26:0/829734780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c0d28e0 0x7fa87c0d00f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa8680097a0 tx=0x7fa86802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fd4f8438ebe037 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.520+0000 7fa87b7fe640 1 -- 172.21.15.26:0/829734780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868037070 con 0x7fa87c0d28e0 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.520+0000 7fa87b7fe640 1 -- 172.21.15.26:0/829734780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa868032440 con 0x7fa87c0d28e0 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.520+0000 7fa87b7fe640 1 -- 172.21.15.26:0/829734780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868032810 con 0x7fa87c0d28e0 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.520+0000 7fa882b5c640 1 -- 172.21.15.26:0/829734780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c0d28e0 msgr2=0x7fa87c0d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.520+0000 7fa882b5c640 1 --2- 172.21.15.26:0/829734780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c0d28e0 0x7fa87c0d00f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa8680097a0 tx=0x7fa86802a270 comp rx=0 tx=0).stop 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.521+0000 7fa882b5c640 1 -- 172.21.15.26:0/829734780 shutdown_connections 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.521+0000 7fa882b5c640 1 --2- 172.21.15.26:0/829734780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c0d28e0 0x7fa87c0d00f0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.521+0000 7fa882b5c640 1 -- 172.21.15.26:0/829734780 >> 172.21.15.26:0/829734780 conn(0x7fa87c0d1dc0 msgr2=0x7fa87c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.521+0000 7fa882b5c640 1 -- 172.21.15.26:0/829734780 shutdown_connections 2024-04-24T23:26:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.521+0000 7fa882b5c640 1 -- 172.21.15.26:0/829734780 wait complete. 2024-04-24T23:26:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.522+0000 7fa882b5c640 1 Processor -- start 2024-04-24T23:26:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.522+0000 7fa882b5c640 1 -- start start 2024-04-24T23:26:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa882b5c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa87c0ce640 0x7fa87c09c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa882b5c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa87c0d28e0 0x7fa87c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa882b5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c15da30 0x7fa87c156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa882b5c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c0cc940 con 0x7fa87c15da30 2024-04-24T23:26:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa882b5c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c0cc410 con 0x7fa87c0d28e0 2024-04-24T23:26:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa882b5c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c0ccc20 con 0x7fa87c0ce640 2024-04-24T23:26:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8810f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c15da30 0x7fa87c156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8810f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c15da30 0x7fa87c156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40456/0 (socket says 172.21.15.26:40456) 2024-04-24T23:26:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8810f9640 1 -- 172.21.15.26:0/2672958046 learned_addr learned my addr 172.21.15.26:0/2672958046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa87bfff640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa87c0d28e0 0x7fa87c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8808f8640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa87c0ce640 0x7fa87c09c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8810f9640 1 -- 172.21.15.26:0/2672958046 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa87c0ce640 msgr2=0x7fa87c09c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8810f9640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa87c0ce640 0x7fa87c09c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8810f9640 1 -- 172.21.15.26:0/2672958046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa87c0d28e0 msgr2=0x7fa87c070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8810f9640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa87c0d28e0 0x7fa87c070460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8810f9640 1 -- 172.21.15.26:0/2672958046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c0b4c10 con 0x7fa87c15da30 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa87bfff640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa87c0d28e0 0x7fa87c070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8810f9640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c15da30 0x7fa87c156e10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b970 tx=0x7fa87400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8797fa640 1 -- 172.21.15.26:0/2672958046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874019070 con 0x7fa87c15da30 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8797fa640 1 -- 172.21.15.26:0/2672958046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa87400cb40 con 0x7fa87c15da30 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa882b5c640 1 -- 172.21.15.26:0/2672958046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c0b4ef0 con 0x7fa87c15da30 2024-04-24T23:26:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa8797fa640 1 -- 172.21.15.26:0/2672958046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874014440 con 0x7fa87c15da30 2024-04-24T23:26:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.523+0000 7fa882b5c640 1 -- 172.21.15.26:0/2672958046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c0b5790 con 0x7fa87c15da30 2024-04-24T23:26:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.525+0000 7fa8797fa640 1 -- 172.21.15.26:0/2672958046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa87401c070 con 0x7fa87c15da30 2024-04-24T23:26:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.525+0000 7fa8797fa640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa86006be30 0x7fa86006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.526+0000 7fa8797fa640 1 -- 172.21.15.26:0/2672958046 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fa87408aa70 con 0x7fa87c15da30 2024-04-24T23:26:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.526+0000 7fa8808f8640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa86006be30 0x7fa86006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.526+0000 7fa8808f8640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa86006be30 0x7fa86006e2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa870000f60 tx=0x7fa870009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.526+0000 7fa885860740 1 Processor -- start 2024-04-24T23:26:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.526+0000 7fa885860740 1 -- start start 2024-04-24T23:26:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa885860740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 0x560053f98510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa885860740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fc8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa885860740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8560 0x560053f9d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa885860740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560053f01330 con 0x560053fc8ca0 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa885860740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560053ea9ab0 con 0x560053fe5b80 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa885860740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560053efd2f0 con 0x560053fc8560 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa87bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fc8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa87bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fc8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40462/0 (socket says 172.21.15.26:40462) 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa87bfff640 1 -- 172.21.15.26:0/2700696168 learned_addr learned my addr 172.21.15.26:0/2700696168 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa8810f9640 1 --2- 172.21.15.26:0/2700696168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8560 0x560053f9d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.527+0000 7fa8808f8640 1 --2- 172.21.15.26:0/2700696168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 0x560053f98510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.528+0000 7fa87bfff640 1 -- 172.21.15.26:0/2700696168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8560 msgr2=0x560053f9d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.528+0000 7fa87bfff640 1 --2- 172.21.15.26:0/2700696168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8560 0x560053f9d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.528+0000 7fa87bfff640 1 -- 172.21.15.26:0/2700696168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 msgr2=0x560053f98510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.528+0000 7fa87bfff640 1 --2- 172.21.15.26:0/2700696168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 0x560053f98510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.528+0000 7fa87bfff640 1 -- 172.21.15.26:0/2700696168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560053cdf7d0 con 0x560053fc8ca0 2024-04-24T23:26:51.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.528+0000 7fa87bfff640 1 --2- 172.21.15.26:0/2700696168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fc8020 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fa86802a860 tx=0x7fa86802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cb44428eb21a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.528+0000 7fa8667fc640 1 -- 172.21.15.26:0/2700696168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868037040 con 0x560053fc8ca0 2024-04-24T23:26:51.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.528+0000 7fa8667fc640 1 -- 172.21.15.26:0/2700696168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa868042070 con 0x560053fc8ca0 2024-04-24T23:26:51.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.528+0000 7fa8667fc640 1 -- 172.21.15.26:0/2700696168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868032940 con 0x560053fc8ca0 2024-04-24T23:26:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 -- 172.21.15.26:0/2700696168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 msgr2=0x560053fc8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 --2- 172.21.15.26:0/2700696168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fc8020 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fa86802a860 tx=0x7fa86802c000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 -- 172.21.15.26:0/2700696168 shutdown_connections 2024-04-24T23:26:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 --2- 172.21.15.26:0/2700696168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8560 0x560053f9d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 --2- 172.21.15.26:0/2700696168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fc8020 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 --2- 172.21.15.26:0/2700696168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 0x560053f98510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 -- 172.21.15.26:0/2700696168 >> 172.21.15.26:0/2700696168 conn(0x560053f9a780 msgr2=0x560053fe3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 -- 172.21.15.26:0/2700696168 shutdown_connections 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 -- 172.21.15.26:0/2700696168 wait complete. 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 -- start start 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.529+0000 7fa885860740 1 Processor -- start 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa885860740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fb8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa885860740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 0x560053fe4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa885860740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc9190 0x560053fe4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa885860740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560053f9c930 con 0x560053fc8ca0 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa885860740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560053ebcf40 con 0x560053fe5b80 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa885860740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560053f9bd20 con 0x560053fc9190 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa885860740 10 client.0 _ll_register_callbacks cb 0x560053c14be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa8808f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fb8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa8808f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fb8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40472/0 (socket says 172.21.15.26:40472) 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa8808f8640 1 -- 172.21.15.26:0/3996194433 learned_addr learned my addr 172.21.15.26:0/3996194433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa87bfff640 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 0x560053fe4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.530+0000 7fa8810f9640 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc9190 0x560053fe4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa8808f8640 1 -- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc9190 msgr2=0x560053fe4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa8808f8640 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc9190 0x560053fe4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa8808f8640 1 -- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 msgr2=0x560053fe4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa8808f8640 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 0x560053fe4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa8808f8640 1 -- 172.21.15.26:0/3996194433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560053bd50f0 con 0x560053fc8ca0 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa8810f9640 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc9190 0x560053fe4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa8808f8640 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fb8df0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa870007680 tx=0x7fa870000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a06f627b61fbeb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa85b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3996194433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870026060 con 0x560053fc8ca0 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3996194433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fa870021ae0 con 0x560053fc8ca0 2024-04-24T23:26:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa885860740 1 -- 172.21.15.26:0/3996194433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa868042070 con 0x560053fc8ca0 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3996194433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa870021ce0 con 0x560053fc8ca0 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa885860740 10 client.8253 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa885860740 1 -- 172.21.15.26:0/3996194433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x560053d3ff60 con 0x560053fc8ca0 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3996194433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa868042070 con 0x560053fc8ca0 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3996194433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7fa870012410 con 0x560053fc8ca0 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa885860740 10 client.8253 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa885860740 10 client.8253 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.531+0000 7fa885860740 20 client.8253 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.532+0000 7fa885860740 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x560053fb6750 0x560053fb6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.532+0000 7fa885860740 4 client.8253 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.532+0000 7fa885860740 1 -- 172.21.15.26:0/3996194433 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560053f0e870 con 0x560053fb6750 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.532+0000 7fa87bfff640 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x560053fb6750 0x560053fb6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.532+0000 7fa87bfff640 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x560053fb6750 0x560053fb6b60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28f8ba2d1f62e97 server_cookie=ae37d165ec4c7961 in_seq=0 out_seq=0 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.532+0000 7fa85b7fe640 10 client.8253 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3996194433 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560053f0e870 con 0x560053fb6750 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa85b7fe640 10 client.8253 handle_command_reply: tid=0 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 client.8253 shutdown 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 -- 172.21.15.26:0/3996194433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 msgr2=0x560053fb8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fb8df0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa870007680 tx=0x7fa870000ed0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 -- 172.21.15.26:0/3996194433 shutdown_connections 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x560053fb6750 0x560053fb6b60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc9190 0x560053fe4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fe5b80 0x560053fe4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 --2- 172.21.15.26:0/3996194433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053fb8df0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa85b7fe640 0 client.8253 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 -- 172.21.15.26:0/3996194433 >> 172.21.15.26:0/3996194433 conn(0x560053fc8560 msgr2=0x560053fe38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 -- 172.21.15.26:0/3996194433 shutdown_connections 2024-04-24T23:26:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 1 -- 172.21.15.26:0/3996194433 wait complete. 2024-04-24T23:26:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.533+0000 7fa885860740 20 client.8253 trim_cache size 0 max 16384 2024-04-24T23:26:51.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa885860740 1 Processor -- start 2024-04-24T23:26:51.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa885860740 1 -- start start 2024-04-24T23:26:51.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa885860740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053f99440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa885860740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 0x560053fb73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa885860740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fe5b80 0x560053fb78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa885860740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c200bb0 con 0x560053fc8ca0 2024-04-24T23:26:51.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa885860740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1ffd80 con 0x560053fc9190 2024-04-24T23:26:51.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa885860740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c200480 con 0x560053fe5b80 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa8808f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053f99440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa8808f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053f99440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40488/0 (socket says 172.21.15.26:40488) 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.551+0000 7fa8808f8640 1 -- 172.21.15.26:0/120517955 learned_addr learned my addr 172.21.15.26:0/120517955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa87bfff640 1 --2- 172.21.15.26:0/120517955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 0x560053fb73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa8810f9640 1 --2- 172.21.15.26:0/120517955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fe5b80 0x560053fb78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa8808f8640 1 -- 172.21.15.26:0/120517955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fe5b80 msgr2=0x560053fb78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa8808f8640 1 --2- 172.21.15.26:0/120517955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fe5b80 0x560053fb78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa8808f8640 1 -- 172.21.15.26:0/120517955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 msgr2=0x560053fb73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa8808f8640 1 --2- 172.21.15.26:0/120517955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 0x560053fb73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa8808f8640 1 -- 172.21.15.26:0/120517955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560053bae1b0 con 0x560053fc8ca0 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa8808f8640 1 --2- 172.21.15.26:0/120517955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053f99440 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa87000e010 tx=0x7fa870019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f1eed2ef27685e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa865ffb640 1 -- 172.21.15.26:0/120517955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870012960 con 0x560053fc8ca0 2024-04-24T23:26:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa865ffb640 1 -- 172.21.15.26:0/120517955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870012b00 con 0x560053fc8ca0 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa865ffb640 1 -- 172.21.15.26:0/120517955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870012070 con 0x560053fc8ca0 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa885860740 1 -- 172.21.15.26:0/120517955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 msgr2=0x560053f99440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.552+0000 7fa885860740 1 --2- 172.21.15.26:0/120517955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053f99440 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa87000e010 tx=0x7fa870019000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.553+0000 7fa885860740 1 -- 172.21.15.26:0/120517955 shutdown_connections 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.553+0000 7fa885860740 1 --2- 172.21.15.26:0/120517955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fe5b80 0x560053fb78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.553+0000 7fa885860740 1 --2- 172.21.15.26:0/120517955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 0x560053fb73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.553+0000 7fa885860740 1 --2- 172.21.15.26:0/120517955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fc8ca0 0x560053f99440 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.553+0000 7fa885860740 1 -- 172.21.15.26:0/120517955 >> 172.21.15.26:0/120517955 conn(0x560053fc8560 msgr2=0x560053ff5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.553+0000 7fa885860740 1 -- 172.21.15.26:0/120517955 shutdown_connections 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.553+0000 7fa885860740 1 -- 172.21.15.26:0/120517955 wait complete. 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.553+0000 7fa885860740 1 -- start start 2024-04-24T23:26:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.553+0000 7fa885860740 1 Processor -- start 2024-04-24T23:26:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa885860740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8ca0 0x56005405c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa885860740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 0x560053fc4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa885860740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fe5b80 0x560053fc5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa885860740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560053fc3b80 con 0x560053fe5b80 2024-04-24T23:26:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa885860740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c09efa0 con 0x560053fc9190 2024-04-24T23:26:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa885860740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x560053ea8e50 con 0x560053fc8ca0 2024-04-24T23:26:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa885860740 10 client.0 _ll_register_callbacks cb 0x7fa870005950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa8810f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fe5b80 0x560053fc5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa8810f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fe5b80 0x560053fc5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40490/0 (socket says 172.21.15.26:40490) 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa8810f9640 1 -- 172.21.15.26:0/3665434349 learned_addr learned my addr 172.21.15.26:0/3665434349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa8808f8640 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8ca0 0x56005405c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa87bfff640 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 0x560053fc4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa8810f9640 1 -- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8ca0 msgr2=0x56005405c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.554+0000 7fa8810f9640 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8ca0 0x56005405c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa8810f9640 1 -- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 msgr2=0x560053fc4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa8810f9640 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 0x560053fc4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa8810f9640 1 -- 172.21.15.26:0/3665434349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560053f98fc0 con 0x560053fe5b80 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa8810f9640 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fe5b80 0x560053fc5070 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa874061420 tx=0x7fa874058c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60855e67a5ddebab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa85b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3665434349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8740581c0 con 0x560053fe5b80 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3665434349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fa874058360 con 0x560053fe5b80 2024-04-24T23:26:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa885860740 1 -- 172.21.15.26:0/3665434349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa870012b00 con 0x560053fe5b80 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3665434349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa87405f320 con 0x560053fe5b80 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa885860740 10 client.8259 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa885860740 1 -- 172.21.15.26:0/3665434349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x560053fc5610 con 0x560053fe5b80 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3665434349 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870012b00 con 0x560053fe5b80 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3665434349 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7fa87405f4c0 con 0x560053fe5b80 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa885860740 10 client.8259 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa885860740 10 client.8259 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa885860740 20 client.8259 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa885860740 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x560053fb8ca0 0x56005405fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa885860740 4 client.8259 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa885860740 1 -- 172.21.15.26:0/3665434349 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x560053f08bc0 con 0x560053fb8ca0 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.555+0000 7fa8808f8640 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x560053fb8ca0 0x56005405fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa8808f8640 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x560053fb8ca0 0x56005405fc20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a279422503e8c18b server_cookie=e46d96f31befb7b0 in_seq=0 out_seq=0 2024-04-24T23:26:51.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa85b7fe640 10 client.8259 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa85b7fe640 1 -- 172.21.15.26:0/3665434349 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560053f08bc0 con 0x560053fb8ca0 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa85b7fe640 10 client.8259 handle_command_reply: tid=0 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 client.8259 shutdown 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 -- 172.21.15.26:0/3665434349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fe5b80 msgr2=0x560053fc5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fe5b80 0x560053fc5070 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa874061420 tx=0x7fa874058c70 comp rx=0 tx=0).stop 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 -- 172.21.15.26:0/3665434349 shutdown_connections 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x560053fb8ca0 0x56005405fc20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560053fe5b80 0x560053fc5070 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560053fc9190 0x560053fc4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa85b7fe640 0 client.8259 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 --2- 172.21.15.26:0/3665434349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x560053fc8ca0 0x56005405c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 -- 172.21.15.26:0/3665434349 >> 172.21.15.26:0/3665434349 conn(0x560053fc8560 msgr2=0x560054006870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 -- 172.21.15.26:0/3665434349 shutdown_connections 2024-04-24T23:26:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.556+0000 7fa885860740 1 -- 172.21.15.26:0/3665434349 wait complete. 2024-04-24T23:26:51.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa885860740 20 client.8259 trim_cache size 0 max 16384 2024-04-24T23:26:51.564 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:26:51.564 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 -- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa86006be30 msgr2=0x7fa86006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa86006be30 0x7fa86006e2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa870000f60 tx=0x7fa870009040 comp rx=0 tx=0).stop 2024-04-24T23:26:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 -- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c15da30 msgr2=0x7fa87c156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c15da30 0x7fa87c156e10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b970 tx=0x7fa87400be30 comp rx=0 tx=0).stop 2024-04-24T23:26:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 -- 172.21.15.26:0/2672958046 shutdown_connections 2024-04-24T23:26:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa86006be30 0x7fa86006e2f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c15da30 0x7fa87c156e10 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa87c0d28e0 0x7fa87c070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 --2- 172.21.15.26:0/2672958046 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa87c0ce640 0x7fa87c09c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 -- 172.21.15.26:0/2672958046 >> 172.21.15.26:0/2672958046 conn(0x7fa87c0d1dc0 msgr2=0x7fa87c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 -- 172.21.15.26:0/2672958046 shutdown_connections 2024-04-24T23:26:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.557+0000 7fa882b5c640 1 -- 172.21.15.26:0/2672958046 wait complete. 2024-04-24T23:26:51.569 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:26:51.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:1 scrub status 2024-04-24T23:26:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.670+0000 7f8f693e9640 1 Processor -- start 2024-04-24T23:26:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.670+0000 7f8f693e9640 1 -- start start 2024-04-24T23:26:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f693e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f64158ae0 0x7f8f6406e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f693e9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f640d0ce0 0x7f8f6415e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f693e9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8f6415ee90 0x7f8f64071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f693e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6406b1c0 con 0x7f8f64158ae0 2024-04-24T23:26:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f693e9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6406ec90 con 0x7f8f6415ee90 2024-04-24T23:26:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f693e9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6406ef70 con 0x7f8f640d0ce0 2024-04-24T23:26:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f62ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f64158ae0 0x7f8f6406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f62ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f64158ae0 0x7f8f6406e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40492/0 (socket says 172.21.15.26:40492) 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f62ffd640 1 -- 172.21.15.26:0/3500014210 learned_addr learned my addr 172.21.15.26:0/3500014210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3500014210 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f640d0ce0 0x7f8f6415e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f637fe640 1 --2- 172.21.15.26:0/3500014210 >> v2:172.21.15.29:3300/0 conn(0x7f8f6415ee90 0x7f8f64071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f62ffd640 1 -- 172.21.15.26:0/3500014210 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f640d0ce0 msgr2=0x7f8f6415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3500014210 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f640d0ce0 0x7f8f6415e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f62ffd640 1 -- 172.21.15.26:0/3500014210 >> v2:172.21.15.29:3300/0 conn(0x7f8f6415ee90 msgr2=0x7f8f64071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3500014210 >> v2:172.21.15.29:3300/0 conn(0x7f8f6415ee90 0x7f8f64071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.671+0000 7f8f62ffd640 1 -- 172.21.15.26:0/3500014210 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6406f250 con 0x7f8f64158ae0 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.672+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3500014210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 0x7f8f6406e620 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8f640716b0 tx=0x7f8f5c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4257ea8faa7286a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.672+0000 7f8f61ffb640 1 -- 172.21.15.26:0/3500014210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c037070 con 0x7f8f64158ae0 2024-04-24T23:26:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.672+0000 7f8f61ffb640 1 -- 172.21.15.26:0/3500014210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c032440 con 0x7f8f64158ae0 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.672+0000 7f8f61ffb640 1 -- 172.21.15.26:0/3500014210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c032810 con 0x7f8f64158ae0 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.673+0000 7f8f693e9640 1 -- 172.21.15.26:0/3500014210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 msgr2=0x7f8f6406e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.673+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3500014210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 0x7f8f6406e620 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8f640716b0 tx=0x7f8f5c02a140 comp rx=0 tx=0).stop 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.673+0000 7f8f693e9640 1 -- 172.21.15.26:0/3500014210 shutdown_connections 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.673+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3500014210 >> v2:172.21.15.29:3300/0 conn(0x7f8f6415ee90 0x7f8f64071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.673+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3500014210 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f640d0ce0 0x7f8f6415e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.673+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3500014210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 0x7f8f6406e620 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.673+0000 7f8f693e9640 1 -- 172.21.15.26:0/3500014210 >> 172.21.15.26:0/3500014210 conn(0x7f8f640b2970 msgr2=0x7f8f640d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.673+0000 7f8f693e9640 1 -- 172.21.15.26:0/3500014210 shutdown_connections 2024-04-24T23:26:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.673+0000 7f8f693e9640 1 -- 172.21.15.26:0/3500014210 wait complete. 2024-04-24T23:26:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.674+0000 7f8f693e9640 1 Processor -- start 2024-04-24T23:26:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.674+0000 7f8f693e9640 1 -- start start 2024-04-24T23:26:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f693e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f640d0ce0 0x7f8f64159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f693e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 0x7f8f641f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f693e9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f6415ee90 0x7f8f64159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f693e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f640b00e0 con 0x7f8f64158ae0 2024-04-24T23:26:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f693e9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f640aff60 con 0x7f8f640d0ce0 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f693e9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8f64071c60 con 0x7f8f6415ee90 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 0x7f8f641f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 0x7f8f641f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40504/0 (socket says 172.21.15.26:40504) 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f627fc640 1 -- 172.21.15.26:0/3890861125 learned_addr learned my addr 172.21.15.26:0/3890861125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f640d0ce0 0x7f8f64159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f637fe640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f6415ee90 0x7f8f64159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f627fc640 1 -- 172.21.15.26:0/3890861125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f6415ee90 msgr2=0x7f8f64159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f6415ee90 0x7f8f64159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f627fc640 1 -- 172.21.15.26:0/3890861125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f640d0ce0 msgr2=0x7f8f64159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.675+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f640d0ce0 0x7f8f64159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.676+0000 7f8f627fc640 1 -- 172.21.15.26:0/3890861125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f641f5740 con 0x7f8f64158ae0 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.676+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 0x7f8f641f4590 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00d800 tx=0x7f8f4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.676+0000 7f8f5b7fe640 1 -- 172.21.15.26:0/3890861125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c018020 con 0x7f8f64158ae0 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.676+0000 7f8f5b7fe640 1 -- 172.21.15.26:0/3890861125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c0096a0 con 0x7f8f64158ae0 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.676+0000 7f8f693e9640 1 -- 172.21.15.26:0/3890861125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f641f4ea0 con 0x7f8f64158ae0 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.676+0000 7f8f5b7fe640 1 -- 172.21.15.26:0/3890861125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c009a70 con 0x7f8f64158ae0 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.676+0000 7f8f693e9640 1 -- 172.21.15.26:0/3890861125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f641f5460 con 0x7f8f64158ae0 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.678+0000 7f8f5b7fe640 1 -- 172.21.15.26:0/3890861125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f4c01b020 con 0x7f8f64158ae0 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.678+0000 7f8f5b7fe640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4406be30 0x7f8f4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.678+0000 7f8f5b7fe640 1 -- 172.21.15.26:0/3890861125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f8f4c08b370 con 0x7f8f64158ae0 2024-04-24T23:26:51.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.678+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4406be30 0x7f8f4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.679+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4406be30 0x7f8f4406e2f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c034000 tx=0x7f8f5c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.679+0000 7f8f6c1cb740 1 Processor -- start 2024-04-24T23:26:51.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.679+0000 7f8f6c1cb740 1 -- start start 2024-04-24T23:26:51.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d58140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d75600 0x561753d75b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d2da40 0x561753d73cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561753c916b0 con 0x561753d75600 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561753c39880 con 0x561753d57d60 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561753c8d670 con 0x561753d2da40 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d75600 0x561753d75b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d75600 0x561753d75b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40510/0 (socket says 172.21.15.26:40510) 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f627fc640 1 -- 172.21.15.26:0/3067977355 learned_addr learned my addr 172.21.15.26:0/3067977355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d58140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f637fe640 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d2da40 0x561753d73cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f627fc640 1 -- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d2da40 msgr2=0x561753d73cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d2da40 0x561753d73cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f627fc640 1 -- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 msgr2=0x561753d58140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d58140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f627fc640 1 -- 172.21.15.26:0/3067977355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561753a6f7d0 con 0x561753d75600 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d58140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f637fe640 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d2da40 0x561753d73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.680+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d75600 0x561753d75b50 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c055b80 tx=0x7f8f4c0591b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b0919d75906626 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f58ff9640 1 -- 172.21.15.26:0/3067977355 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c057a20 con 0x561753d75600 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f58ff9640 1 -- 172.21.15.26:0/3067977355 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c08b0b0 con 0x561753d75600 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f58ff9640 1 -- 172.21.15.26:0/3067977355 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c009c10 con 0x561753d75600 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3067977355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d75600 msgr2=0x561753d75b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d75600 0x561753d75b50 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c055b80 tx=0x7f8f4c0591b0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3067977355 shutdown_connections 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d2da40 0x561753d73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d75600 0x561753d75b50 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3067977355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d58140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3067977355 >> 172.21.15.26:0/3067977355 conn(0x561753d2a600 msgr2=0x561753d72b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3067977355 shutdown_connections 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.681+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3067977355 wait complete. 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.682+0000 7f8f6c1cb740 1 -- start start 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.682+0000 7f8f6c1cb740 1 Processor -- start 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d6b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d75600 0x561753d5b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d5b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561753d2e7e0 con 0x561753d6bb90 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561753c4cf50 con 0x561753d57d60 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561753d28530 con 0x561753d75600 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f6c1cb740 10 client.0 _ll_register_callbacks cb 0x561753c88000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d5b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d5b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40522/0 (socket says 172.21.15.26:40522) 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f637fe640 1 -- 172.21.15.26:0/3269766298 learned_addr learned my addr 172.21.15.26:0/3269766298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d6b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d75600 0x561753d5b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f637fe640 1 -- 172.21.15.26:0/3269766298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d75600 msgr2=0x561753d5b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f637fe640 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d75600 0x561753d5b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f637fe640 1 -- 172.21.15.26:0/3269766298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 msgr2=0x561753d6b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f637fe640 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d6b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f637fe640 1 -- 172.21.15.26:0/3269766298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617539a53f0 con 0x561753d6bb90 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f637fe640 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d5b860 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8f540079d0 tx=0x7f8f54007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28649fdbaae434ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3269766298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54019020 con 0x561753d6bb90 2024-04-24T23:26:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3269766298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f8f54010c10 con 0x561753d6bb90 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.683+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3269766298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4c08b0b0 con 0x561753d6bb90 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3269766298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f8f54023050 con 0x561753d6bb90 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f6c1cb740 10 client.8271 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3269766298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5617539650f0 con 0x561753d6bb90 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3269766298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c08b0b0 con 0x561753d6bb90 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3269766298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f8f54026020 con 0x561753d6bb90 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f6c1cb740 10 client.8271 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f6c1cb740 10 client.8271 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f6c1cb740 20 client.8271 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x561753d86dd0 0x561753d871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f6c1cb740 4 client.8271 mds_command: new command op to 6778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3269766298 --> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561753c9ebf0 con 0x561753d86dd0 2024-04-24T23:26:51.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x561753d86dd0 0x561753d871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x561753d86dd0 0x561753d871b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e61ab5ac2eb345e2 server_cookie=9700db18165f035f in_seq=0 out_seq=0 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.684+0000 7f8f41ffb640 10 client.8271 ms_handle_connect on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3269766298 <== mds.1 v2:172.21.15.26:6834/306957409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561753c9ebf0 con 0x561753d86dd0 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f41ffb640 10 client.8271 handle_command_reply: tid=0 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 client.8271 shutdown 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3269766298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 msgr2=0x561753d5b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d5b860 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8f540079d0 tx=0x7f8f54007e90 comp rx=0 tx=0).stop 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3269766298 shutdown_connections 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x561753d86dd0 0x561753d871b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d5b860 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d75600 0x561753d5b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3269766298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d6b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f41ffb640 0 client.8271 ms_handle_reset on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3269766298 >> 172.21.15.26:0/3269766298 conn(0x561753d2da40 msgr2=0x561753d56de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3269766298 shutdown_connections 2024-04-24T23:26:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3269766298 wait complete. 2024-04-24T23:26:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.685+0000 7f8f6c1cb740 20 client.8271 trim_cache size 0 max 16384 2024-04-24T23:26:51.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.703+0000 7f8f6c1cb740 1 Processor -- start 2024-04-24T23:26:51.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.703+0000 7f8f6c1cb740 1 -- start start 2024-04-24T23:26:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d68c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d75ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d75600 0x561753d75ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64200a90 con 0x561753d6bb90 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641ffda0 con 0x561753d57d60 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8f64200360 con 0x561753d75600 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d75ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d75ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40530/0 (socket says 172.21.15.26:40530) 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f627fc640 1 -- 172.21.15.26:0/3846341833 learned_addr learned my addr 172.21.15.26:0/3846341833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f627fc640 1 -- 172.21.15.26:0/3846341833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d75600 msgr2=0x561753d75ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3846341833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d75600 0x561753d75ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f627fc640 1 -- 172.21.15.26:0/3846341833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 msgr2=0x561753d68c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3846341833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d68c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f627fc640 1 -- 172.21.15.26:0/3846341833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561753d6b780 con 0x561753d6bb90 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3846341833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d75ab0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009490 tx=0x7f8f4c059390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd80651f7ae1781 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f43fff640 1 -- 172.21.15.26:0/3846341833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c08b0b0 con 0x561753d6bb90 2024-04-24T23:26:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f43fff640 1 -- 172.21.15.26:0/3846341833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c055d90 con 0x561753d6bb90 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.704+0000 7f8f43fff640 1 -- 172.21.15.26:0/3846341833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c05ebb0 con 0x561753d6bb90 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3846341833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 msgr2=0x561753d75ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3846341833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d75ab0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009490 tx=0x7f8f4c059390 comp rx=0 tx=0).stop 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3846341833 shutdown_connections 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3846341833 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d75600 0x561753d75ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3846341833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d6bb90 0x561753d75ab0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3846341833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d57d60 0x561753d68c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3846341833 >> 172.21.15.26:0/3846341833 conn(0x561753d2da40 msgr2=0x561753d2de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3846341833 shutdown_connections 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3846341833 wait complete. 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 -- start start 2024-04-24T23:26:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.705+0000 7f8f6c1cb740 1 Processor -- start 2024-04-24T23:26:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d57d60 0x561753d7f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d6bb90 0x561753d5acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f6c1cb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d75600 0x561753d96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641472e0 con 0x561753d57d60 2024-04-24T23:26:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561753d97670 con 0x561753d75600 2024-04-24T23:26:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f6c1cb740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561753d7ff90 con 0x561753d6bb90 2024-04-24T23:26:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f6c1cb740 10 client.0 _ll_register_callbacks cb 0x7f8f54009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d57d60 0x561753d7f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d57d60 0x561753d7f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40546/0 (socket says 172.21.15.26:40546) 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f62ffd640 1 -- 172.21.15.26:0/3617524568 learned_addr learned my addr 172.21.15.26:0/3617524568 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f637fe640 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d75600 0x561753d96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.706+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d6bb90 0x561753d5acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f62ffd640 1 -- 172.21.15.26:0/3617524568 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d6bb90 msgr2=0x561753d5acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d6bb90 0x561753d5acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f62ffd640 1 -- 172.21.15.26:0/3617524568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d75600 msgr2=0x561753d96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d75600 0x561753d96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f62ffd640 1 -- 172.21.15.26:0/3617524568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561753d7fac0 con 0x561753d57d60 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d6bb90 0x561753d5acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f62ffd640 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d57d60 0x561753d7f280 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622243f5e6aff25e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3617524568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c037040 con 0x561753d57d60 2024-04-24T23:26:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3617524568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f8f5c04f3c0 con 0x561753d57d60 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3617524568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4c055d90 con 0x561753d57d60 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3617524568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f8f5c05e050 con 0x561753d57d60 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f6c1cb740 10 client.8277 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3617524568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x561753d71040 con 0x561753d57d60 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3617524568 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c055d90 con 0x561753d57d60 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3617524568 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f8f5c061020 con 0x561753d57d60 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f6c1cb740 10 client.8277 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f6c1cb740 10 client.8277 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f6c1cb740 20 client.8277 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x561753defeb0 0x561753df02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f6c1cb740 4 client.8277 mds_command: new command op to 6778 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3617524568 --> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561753d6d040 con 0x561753defeb0 2024-04-24T23:26:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.707+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x561753defeb0 0x561753df02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f41ffb640 10 client.8277 ms_handle_connect on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f627fc640 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x561753defeb0 0x561753df02c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c494d824fdadff3 server_cookie=75b207c41f714ba0 in_seq=0 out_seq=0 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f41ffb640 1 -- 172.21.15.26:0/3617524568 <== mds.1 v2:172.21.15.26:6834/306957409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x561753d6d040 con 0x561753defeb0 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f41ffb640 10 client.8277 handle_command_reply: tid=0 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 client.8277 shutdown 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3617524568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d57d60 msgr2=0x561753d7f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d57d60 0x561753d7f280 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3617524568 shutdown_connections 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x561753defeb0 0x561753df02c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561753d75600 0x561753d96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561753d6bb90 0x561753d5acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f41ffb640 0 client.8277 ms_handle_reset on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 --2- 172.21.15.26:0/3617524568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561753d57d60 0x561753d7f280 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3617524568 >> 172.21.15.26:0/3617524568 conn(0x561753d2da40 msgr2=0x561753d6aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3617524568 shutdown_connections 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 1 -- 172.21.15.26:0/3617524568 wait complete. 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.708+0000 7f8f6c1cb740 20 client.8277 trim_cache size 0 max 16384 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "cca3b3ba-5fb5-4b95-8bce-070d29d5babd": { 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:26:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "cca3b3ba-5fb5-4b95-8bce-070d29d5babd", 2024-04-24T23:26:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:26:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:51.717 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 -- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4406be30 msgr2=0x7f8f4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4406be30 0x7f8f4406e2f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c034000 tx=0x7f8f5c02e000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 -- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 msgr2=0x7f8f641f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 0x7f8f641f4590 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00d800 tx=0x7f8f4c00dcc0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 -- 172.21.15.26:0/3890861125 shutdown_connections 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8f4406be30 0x7f8f4406e2f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8f6415ee90 0x7f8f64159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f64158ae0 0x7f8f641f4590 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 --2- 172.21.15.26:0/3890861125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f640d0ce0 0x7f8f64159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 -- 172.21.15.26:0/3890861125 >> 172.21.15.26:0/3890861125 conn(0x7f8f640b2970 msgr2=0x7f8f6406b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 -- 172.21.15.26:0/3890861125 shutdown_connections 2024-04-24T23:26:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.709+0000 7f8f693e9640 1 -- 172.21.15.26:0/3890861125 wait complete. 2024-04-24T23:26:51.720 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:26:51.721 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:26:51.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:2 scrub status 2024-04-24T23:26:51.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.822+0000 7fe58bc1c640 1 Processor -- start 2024-04-24T23:26:51.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.823+0000 7fe58bc1c640 1 -- start start 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.823+0000 7fe58bc1c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe58415adc0 0x7fe5840b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.823+0000 7fe58bc1c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe5840ad7b0 0x7fe5840d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.823+0000 7fe58bc1c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58415ef60 con 0x7fe58415e9c0 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.823+0000 7fe58bc1c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe58415f240 con 0x7fe58415adc0 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.823+0000 7fe58bc1c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe58415f520 con 0x7fe5840ad7b0 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.824+0000 7fe5899b8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe58415adc0 0x7fe5840b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.824+0000 7fe58a1b9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe58415e9c0 0x7fe58406dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51594/0 (socket says 172.21.15.26:51594) 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.824+0000 7fe58a1b9640 1 -- 172.21.15.26:0/3035181282 learned_addr learned my addr 172.21.15.26:0/3035181282 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.824+0000 7fe5891b7640 1 --2- 172.21.15.26:0/3035181282 >> v2:172.21.15.29:3300/0 conn(0x7fe5840ad7b0 0x7fe5840d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.824+0000 7fe5899b8640 1 -- 172.21.15.26:0/3035181282 >> v2:172.21.15.29:3300/0 conn(0x7fe5840ad7b0 msgr2=0x7fe5840d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.824+0000 7fe5899b8640 1 --2- 172.21.15.26:0/3035181282 >> v2:172.21.15.29:3300/0 conn(0x7fe5840ad7b0 0x7fe5840d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.824+0000 7fe5899b8640 1 -- 172.21.15.26:0/3035181282 >> v1:172.21.15.26:6789/0 conn(0x7fe58415e9c0 legacy=0x7fe58406dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.824+0000 7fe5899b8640 1 -- 172.21.15.26:0/3035181282 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58415f800 con 0x7fe58415adc0 2024-04-24T23:26:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.824+0000 7fe5889b6640 1 -- 172.21.15.26:0/3035181282 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3537168726 0 0) 0x7fe58415ef60 con 0x7fe58415e9c0 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe5899b8640 1 --2- 172.21.15.26:0/3035181282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415adc0 0x7fe5840b4370 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009d90 tx=0x7fe57c02a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5e1096dfb0e015 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe5889b6640 1 -- 172.21.15.26:0/3035181282 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c007740 con 0x7fe58415adc0 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe5889b6640 1 -- 172.21.15.26:0/3035181282 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57c0078e0 con 0x7fe58415adc0 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe5889b6640 1 -- 172.21.15.26:0/3035181282 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c007cb0 con 0x7fe58415adc0 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe58bc1c640 1 -- 172.21.15.26:0/3035181282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415adc0 msgr2=0x7fe5840b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe58bc1c640 1 --2- 172.21.15.26:0/3035181282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415adc0 0x7fe5840b4370 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009d90 tx=0x7fe57c02a580 comp rx=0 tx=0).stop 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe58bc1c640 1 -- 172.21.15.26:0/3035181282 shutdown_connections 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe58bc1c640 1 --2- 172.21.15.26:0/3035181282 >> v2:172.21.15.29:3300/0 conn(0x7fe5840ad7b0 0x7fe5840d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe58bc1c640 1 --2- 172.21.15.26:0/3035181282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415adc0 0x7fe5840b4370 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.825+0000 7fe58bc1c640 1 -- 172.21.15.26:0/3035181282 >> 172.21.15.26:0/3035181282 conn(0x7fe58406b540 msgr2=0x7fe58406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.826+0000 7fe58bc1c640 1 -- 172.21.15.26:0/3035181282 shutdown_connections 2024-04-24T23:26:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.826+0000 7fe58bc1c640 1 -- 172.21.15.26:0/3035181282 wait complete. 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.827+0000 7fe58bc1c640 1 Processor -- start 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.827+0000 7fe58bc1c640 1 -- start start 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.827+0000 7fe58bc1c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5840ad7b0 0x7fe584148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.827+0000 7fe58bc1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58415adc0 0x7fe584147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.827+0000 7fe58bc1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415e9c0 0x7fe58415b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.827+0000 7fe58bc1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe58415b1a0 con 0x7fe58415e9c0 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.827+0000 7fe58bc1c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe5840b48b0 con 0x7fe58415adc0 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.827+0000 7fe58bc1c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe5840d0580 con 0x7fe5840ad7b0 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58a1b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415e9c0 0x7fe58415b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58a1b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415e9c0 0x7fe58415b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40562/0 (socket says 172.21.15.26:40562) 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58a1b9640 1 -- 172.21.15.26:0/1038357134 learned_addr learned my addr 172.21.15.26:0/1038357134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe5899b8640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5840ad7b0 0x7fe584148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe5891b7640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58415adc0 0x7fe584147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58a1b9640 1 -- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5840ad7b0 msgr2=0x7fe584148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58a1b9640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5840ad7b0 0x7fe584148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58a1b9640 1 -- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58415adc0 msgr2=0x7fe584147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58a1b9640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58415adc0 0x7fe584147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58a1b9640 1 -- 172.21.15.26:0/1038357134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe584146dc0 con 0x7fe58415e9c0 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe5891b7640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58415adc0 0x7fe584147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe5899b8640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5840ad7b0 0x7fe584148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58a1b9640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415e9c0 0x7fe58415b440 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe574001eb0 tx=0x7fe57400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe5827fc640 1 -- 172.21.15.26:0/1038357134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740074e0 con 0x7fe58415e9c0 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe5827fc640 1 -- 172.21.15.26:0/1038357134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5740047e0 con 0x7fe58415e9c0 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58bc1c640 1 -- 172.21.15.26:0/1038357134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5841470a0 con 0x7fe58415e9c0 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe5827fc640 1 -- 172.21.15.26:0/1038357134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574004bb0 con 0x7fe58415e9c0 2024-04-24T23:26:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.828+0000 7fe58bc1c640 1 -- 172.21.15.26:0/1038357134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe584147660 con 0x7fe58415e9c0 2024-04-24T23:26:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.830+0000 7fe5827fc640 1 -- 172.21.15.26:0/1038357134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe574011070 con 0x7fe58415e9c0 2024-04-24T23:26:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.830+0000 7fe5827fc640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.830+0000 7fe5827fc640 1 -- 172.21.15.26:0/1038357134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fe57408aab0 con 0x7fe58415e9c0 2024-04-24T23:26:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.830+0000 7fe5899b8640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.831+0000 7fe5899b8640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe57c034000 tx=0x7fe57c02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.831+0000 7fe58ea09740 1 Processor -- start 2024-04-24T23:26:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe58ea09740 1 -- start start 2024-04-24T23:26:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558973f0d60 0x5558973f1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589740eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973c6a40 0x55589740ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe58ea09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55589732a6b0 con 0x55589740e600 2024-04-24T23:26:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe58ea09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5558972d2880 con 0x5558973f0d60 2024-04-24T23:26:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe58ea09740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x555897326670 con 0x5558973c6a40 2024-04-24T23:26:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe5891b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589740eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe5891b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589740eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40566/0 (socket says 172.21.15.26:40566) 2024-04-24T23:26:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe5891b7640 1 -- 172.21.15.26:0/92845630 learned_addr learned my addr 172.21.15.26:0/92845630 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe5899b8640 1 --2- 172.21.15.26:0/92845630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558973f0d60 0x5558973f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.832+0000 7fe58a1b9640 1 --2- 172.21.15.26:0/92845630 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973c6a40 0x55589740ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.833+0000 7fe5891b7640 1 -- 172.21.15.26:0/92845630 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973c6a40 msgr2=0x55589740ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.833+0000 7fe5891b7640 1 --2- 172.21.15.26:0/92845630 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973c6a40 0x55589740ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.833+0000 7fe5891b7640 1 -- 172.21.15.26:0/92845630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558973f0d60 msgr2=0x5558973f1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.833+0000 7fe5891b7640 1 --2- 172.21.15.26:0/92845630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558973f0d60 0x5558973f1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.833+0000 7fe5891b7640 1 -- 172.21.15.26:0/92845630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558971087d0 con 0x55589740e600 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.833+0000 7fe5891b7640 1 --2- 172.21.15.26:0/92845630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589740eb50 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe578007950 tx=0x7fe578007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bf33e7ccd88539 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.833+0000 7fe56b7fe640 1 -- 172.21.15.26:0/92845630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578018020 con 0x55589740e600 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.833+0000 7fe56b7fe640 1 -- 172.21.15.26:0/92845630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57800fa80 con 0x55589740e600 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.833+0000 7fe56b7fe640 1 -- 172.21.15.26:0/92845630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57800fe30 con 0x55589740e600 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 -- 172.21.15.26:0/92845630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 msgr2=0x55589740eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 --2- 172.21.15.26:0/92845630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589740eb50 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe578007950 tx=0x7fe578007e10 comp rx=0 tx=0).stop 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 -- 172.21.15.26:0/92845630 shutdown_connections 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 --2- 172.21.15.26:0/92845630 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973c6a40 0x55589740ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 --2- 172.21.15.26:0/92845630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589740eb50 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 --2- 172.21.15.26:0/92845630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558973f0d60 0x5558973f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 -- 172.21.15.26:0/92845630 >> 172.21.15.26:0/92845630 conn(0x5558973c3600 msgr2=0x55589740bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 -- 172.21.15.26:0/92845630 shutdown_connections 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 -- 172.21.15.26:0/92845630 wait complete. 2024-04-24T23:26:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 -- start start 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.834+0000 7fe58ea09740 1 Processor -- start 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 0x555897402c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 0x5558973ed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589741f7e0 0x55589741c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58ea09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5558973c77e0 con 0x55589741f7e0 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58ea09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5558972e5f50 con 0x55589740e600 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58ea09740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5558973c1530 con 0x5558973f0d60 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58ea09740 10 client.0 _ll_register_callbacks cb 0x555897321000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58a1b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589741f7e0 0x55589741c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58a1b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589741f7e0 0x55589741c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40582/0 (socket says 172.21.15.26:40582) 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe58a1b9640 1 -- 172.21.15.26:0/3563405955 learned_addr learned my addr 172.21.15.26:0/3563405955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe5899b8640 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 0x555897402c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.835+0000 7fe5891b7640 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 0x5558973ed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58a1b9640 1 -- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 msgr2=0x555897402c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58a1b9640 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 0x555897402c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58a1b9640 1 -- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 msgr2=0x5558973ed6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58a1b9640 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 0x5558973ed6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58a1b9640 1 -- 172.21.15.26:0/3563405955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589703e3f0 con 0x55589741f7e0 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58a1b9640 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589741f7e0 0x55589741c920 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe574007fd0 tx=0x7fe574059560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a30908bac3b197 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe568ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe568ff9640 1 -- 172.21.15.26:0/3563405955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574004d50 con 0x55589741f7e0 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe568ff9640 1 -- 172.21.15.26:0/3563405955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fe57401ad90 con 0x55589741f7e0 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58ea09740 1 -- 172.21.15.26:0/3563405955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57800fa80 con 0x55589741f7e0 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe568ff9640 1 -- 172.21.15.26:0/3563405955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe574058170 con 0x55589741f7e0 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58ea09740 10 client.8292 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58ea09740 1 -- 172.21.15.26:0/3563405955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x555896ffe0f0 con 0x55589741f7e0 2024-04-24T23:26:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe568ff9640 1 -- 172.21.15.26:0/3563405955 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57800fa80 con 0x55589741f7e0 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe568ff9640 1 -- 172.21.15.26:0/3563405955 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7fe574058310 con 0x55589741f7e0 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58ea09740 10 client.8292 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58ea09740 10 client.8292 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58ea09740 20 client.8292 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58ea09740 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5558973c0260 0x5558973c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58ea09740 4 client.8292 mds_command: new command op to 6299 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.836+0000 7fe58ea09740 1 -- 172.21.15.26:0/3563405955 --> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555897337bf0 con 0x5558973c0260 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.837+0000 7fe5899b8640 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5558973c0260 0x5558973c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.837+0000 7fe568ff9640 10 client.8292 ms_handle_connect on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.837+0000 7fe5899b8640 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5558973c0260 0x5558973c0670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6aa7544bf482f79d server_cookie=7dfac8333ef202db in_seq=0 out_seq=0 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe568ff9640 1 -- 172.21.15.26:0/3563405955 <== mds.2 v2:172.21.15.29:6833/3727846607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555897337bf0 con 0x5558973c0260 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe568ff9640 10 client.8292 handle_command_reply: tid=0 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 client.8292 shutdown 2024-04-24T23:26:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 -- 172.21.15.26:0/3563405955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589741f7e0 msgr2=0x55589741c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589741f7e0 0x55589741c920 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe574007fd0 tx=0x7fe574059560 comp rx=0 tx=0).stop 2024-04-24T23:26:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 -- 172.21.15.26:0/3563405955 shutdown_connections 2024-04-24T23:26:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x5558973c0260 0x5558973c0670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589741f7e0 0x55589741c920 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe568ff9640 0 client.8292 ms_handle_reset on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 0x5558973ed6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 --2- 172.21.15.26:0/3563405955 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 0x555897402c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 -- 172.21.15.26:0/3563405955 >> 172.21.15.26:0/3563405955 conn(0x5558973c6a40 msgr2=0x55589740f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 -- 172.21.15.26:0/3563405955 shutdown_connections 2024-04-24T23:26:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 1 -- 172.21.15.26:0/3563405955 wait complete. 2024-04-24T23:26:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.838+0000 7fe58ea09740 20 client.8292 trim_cache size 0 max 16384 2024-04-24T23:26:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.856+0000 7fe58ea09740 1 Processor -- start 2024-04-24T23:26:51.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.856+0000 7fe58ea09740 1 -- start start 2024-04-24T23:26:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.856+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558973f0d60 0x555897403c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.856+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 0x5558973c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.856+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55589741f7e0 0x5558973f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.856+0000 7fe58ea09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe584200720 con 0x5558973f0d60 2024-04-24T23:26:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.856+0000 7fe58ea09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841ff8f0 con 0x55589740e600 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.856+0000 7fe58ea09740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe5841ffff0 con 0x55589741f7e0 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5899b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558973f0d60 0x555897403c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5899b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558973f0d60 0x555897403c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40592/0 (socket says 172.21.15.26:40592) 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5899b8640 1 -- 172.21.15.26:0/345746077 learned_addr learned my addr 172.21.15.26:0/345746077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5891b7640 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 0x5558973c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe58a1b9640 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55589741f7e0 0x5558973f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5899b8640 1 -- 172.21.15.26:0/345746077 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55589741f7e0 msgr2=0x5558973f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5899b8640 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55589741f7e0 0x5558973f0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5899b8640 1 -- 172.21.15.26:0/345746077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 msgr2=0x5558973c06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5899b8640 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 0x5558973c06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5899b8640 1 -- 172.21.15.26:0/345746077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558973c2ac0 con 0x5558973f0d60 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5891b7640 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 0x5558973c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe5899b8640 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558973f0d60 0x555897403c30 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe57c02ab70 tx=0x7fe57c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459c2cb60a075c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe56affd640 1 -- 172.21.15.26:0/345746077 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c053670 con 0x5558973f0d60 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe56affd640 1 -- 172.21.15.26:0/345746077 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57c058070 con 0x5558973f0d60 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.857+0000 7fe56affd640 1 -- 172.21.15.26:0/345746077 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c03d570 con 0x5558973f0d60 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 -- 172.21.15.26:0/345746077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558973f0d60 msgr2=0x555897403c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558973f0d60 0x555897403c30 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe57c02ab70 tx=0x7fe57c030000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 -- 172.21.15.26:0/345746077 shutdown_connections 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55589741f7e0 0x5558973f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589740e600 0x5558973c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 --2- 172.21.15.26:0/345746077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558973f0d60 0x555897403c30 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 -- 172.21.15.26:0/345746077 >> 172.21.15.26:0/345746077 conn(0x5558973c6a40 msgr2=0x5558973ecbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 -- 172.21.15.26:0/345746077 shutdown_connections 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 -- 172.21.15.26:0/345746077 wait complete. 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 -- start start 2024-04-24T23:26:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.858+0000 7fe58ea09740 1 Processor -- start 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 0x5558973f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589742ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe58ea09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589741f7e0 0x55589742f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe58ea09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5558974059a0 con 0x55589740e600 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe58ea09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841585c0 con 0x55589741f7e0 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe58ea09740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55589742fa90 con 0x5558973f0d60 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe58ea09740 10 client.0 _ll_register_callbacks cb 0x7fe574001170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe5891b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589742ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe5891b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589742ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40602/0 (socket says 172.21.15.26:40602) 2024-04-24T23:26:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe5891b7640 1 -- 172.21.15.26:0/854791108 learned_addr learned my addr 172.21.15.26:0/854791108 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.859+0000 7fe58a1b9640 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589741f7e0 0x55589742f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe5899b8640 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 0x5558973f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe5891b7640 1 -- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 msgr2=0x5558973f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe5891b7640 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 0x5558973f07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe5891b7640 1 -- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589741f7e0 msgr2=0x55589742f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe5891b7640 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589741f7e0 0x55589742f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe5891b7640 1 -- 172.21.15.26:0/854791108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558974099b0 con 0x55589740e600 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe5891b7640 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589742ec90 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe578008fd0 tx=0x7fe578012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bf1252979ddc2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe568ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe568ff9640 1 -- 172.21.15.26:0/854791108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57802d060 con 0x55589740e600 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe568ff9640 1 -- 172.21.15.26:0/854791108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fe578014670 con 0x55589740e600 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe58ea09740 1 -- 172.21.15.26:0/854791108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57c058070 con 0x55589740e600 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe568ff9640 1 -- 172.21.15.26:0/854791108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe578033050 con 0x55589740e600 2024-04-24T23:26:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe58ea09740 10 client.8298 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe58ea09740 1 -- 172.21.15.26:0/854791108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55589741baa0 con 0x55589740e600 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe568ff9640 1 -- 172.21.15.26:0/854791108 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57c058070 con 0x55589740e600 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.860+0000 7fe568ff9640 1 -- 172.21.15.26:0/854791108 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7fe578036020 con 0x55589740e600 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 10 client.8298 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 10 client.8298 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 20 client.8298 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x555897488810 0x55589740cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 4 client.8298 mds_command: new command op to 6299 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 1 -- 172.21.15.26:0/854791108 --> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555897406a80 con 0x555897488810 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe5899b8640 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x555897488810 0x55589740cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe5899b8640 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x555897488810 0x55589740cc20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ba583d2304e36f4 server_cookie=ebec6e5f2ed64898 in_seq=0 out_seq=0 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe568ff9640 10 client.8298 ms_handle_connect on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe568ff9640 1 -- 172.21.15.26:0/854791108 <== mds.2 v2:172.21.15.29:6833/3727846607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x555897406a80 con 0x555897488810 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe568ff9640 10 client.8298 handle_command_reply: tid=0 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 1 client.8298 shutdown 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 1 -- 172.21.15.26:0/854791108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 msgr2=0x55589742ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589742ec90 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe578008fd0 tx=0x7fe578012000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.861+0000 7fe58ea09740 1 -- 172.21.15.26:0/854791108 shutdown_connections 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58ea09740 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x555897488810 0x55589740cc20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58ea09740 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55589741f7e0 0x55589742f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58ea09740 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55589740e600 0x55589742ec90 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58ea09740 1 --2- 172.21.15.26:0/854791108 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5558973f0d60 0x5558973f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe568ff9640 0 client.8298 ms_handle_reset on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58ea09740 1 -- 172.21.15.26:0/854791108 >> 172.21.15.26:0/854791108 conn(0x5558973c6a40 msgr2=0x5558974037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58ea09740 1 -- 172.21.15.26:0/854791108 shutdown_connections 2024-04-24T23:26:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58ea09740 1 -- 172.21.15.26:0/854791108 wait complete. 2024-04-24T23:26:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58ea09740 20 client.8298 trim_cache size 0 max 16384 2024-04-24T23:26:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:26:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:26:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "cca3b3ba-5fb5-4b95-8bce-070d29d5babd": { 2024-04-24T23:26:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:26:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "cca3b3ba-5fb5-4b95-8bce-070d29d5babd", 2024-04-24T23:26:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:26:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 -- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 msgr2=0x7fe57006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe57c034000 tx=0x7fe57c02f000 comp rx=0 tx=0).stop 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 -- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415e9c0 msgr2=0x7fe58415b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415e9c0 0x7fe58415b440 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe574001eb0 tx=0x7fe57400c870 comp rx=0 tx=0).stop 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 -- 172.21.15.26:0/1038357134 shutdown_connections 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe57006be30 0x7fe57006e2f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe58415e9c0 0x7fe58415b440 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe58415adc0 0x7fe584147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 --2- 172.21.15.26:0/1038357134 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe5840ad7b0 0x7fe584148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 -- 172.21.15.26:0/1038357134 >> 172.21.15.26:0/1038357134 conn(0x7fe58406b540 msgr2=0x7fe58409cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 -- 172.21.15.26:0/1038357134 shutdown_connections 2024-04-24T23:26:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.862+0000 7fe58bc1c640 1 -- 172.21.15.26:0/1038357134 wait complete. 2024-04-24T23:26:51.874 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:26:51.874 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:26:51.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-24T23:26:51.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.976+0000 7f512a1c1640 1 Processor -- start 2024-04-24T23:26:51.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.976+0000 7f512a1c1640 1 -- start start 2024-04-24T23:26:51.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f512a1c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f512415adc0 0x7f51240b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f512a1c1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512415ef60 con 0x7f51240ad7b0 2024-04-24T23:26:51.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f512a1c1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512415f240 con 0x7f51240d0750 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f512a1c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f512415f520 con 0x7f512415adc0 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f5122ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f51240ad7b0 0x7f51240d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51602/0 (socket says 172.21.15.26:51602) 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f5122ffd640 1 -- 172.21.15.26:0/211456493 learned_addr learned my addr 172.21.15.26:0/211456493 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f51237fe640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f512415adc0 0x7f51240b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956672847 0 0) 0x7f512415ef60 con 0x7f51240ad7b0 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f510c003520 con 0x7f51240ad7b0 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1601365092 0 0) 0x7f510c003520 con 0x7f51240ad7b0 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f512415adc0 msgr2=0x7f51240b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f51227fc640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f512415adc0 0x7f51240b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 >> v1:172.21.15.29:6789/0 conn(0x7f51240d0750 legacy=0x7f5124072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512415f800 con 0x7f51240ad7b0 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.977+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f5118001670 con 0x7f51240ad7b0 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 >> v1:172.21.15.26:6789/0 conn(0x7f51240ad7b0 legacy=0x7f51240d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51227fc640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c0039e0 0x7f510c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51227fc640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c004390 0x7f510c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51227fc640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f512415adc0 0x7f510c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f512415f520 con 0x7f510c0039e0 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f510c00ba70 con 0x7f510c004390 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f510c00bc70 con 0x7f512415adc0 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51237fe640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c0039e0 0x7f510c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f5122ffd640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c004390 0x7f510c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f5123fff640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f512415adc0 0x7f510c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51237fe640 1 -- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f512415adc0 msgr2=0x7f510c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51237fe640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f512415adc0 0x7f510c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51237fe640 1 -- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c004390 msgr2=0x7f510c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51237fe640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c004390 0x7f510c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f51237fe640 1 -- 172.21.15.26:0/211456493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c003520 con 0x7f510c0039e0 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.978+0000 7f5122ffd640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c004390 0x7f510c004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f51237fe640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c0039e0 0x7f510c003dc0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f5114009ac0 tx=0x7f510c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264b60e3245ed532 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114036070 con 0x7f510c0039e0 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5114031400 con 0x7f510c0039e0 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f51227fc640 1 -- 172.21.15.26:0/211456493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114009690 con 0x7f510c0039e0 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f512a1c1640 1 -- 172.21.15.26:0/211456493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c0039e0 msgr2=0x7f510c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f512a1c1640 1 --2- 172.21.15.26:0/211456493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c0039e0 0x7f510c003dc0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f5114009ac0 tx=0x7f510c005830 comp rx=0 tx=0).stop 2024-04-24T23:26:51.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f5123fff640 1 -- 172.21.15.26:0/211456493 reap_dead start 2024-04-24T23:26:51.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f512a1c1640 1 -- 172.21.15.26:0/211456493 shutdown_connections 2024-04-24T23:26:51.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f512a1c1640 1 -- 172.21.15.26:0/211456493 >> 172.21.15.26:0/211456493 conn(0x7f512406b540 msgr2=0x7f512406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f512a1c1640 1 -- 172.21.15.26:0/211456493 shutdown_connections 2024-04-24T23:26:51.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.979+0000 7f512a1c1640 1 -- 172.21.15.26:0/211456493 wait complete. 2024-04-24T23:26:51.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.981+0000 7f512a1c1640 1 Processor -- start 2024-04-24T23:26:51.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.981+0000 7f512a1c1640 1 -- start start 2024-04-24T23:26:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f512a1c1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c0039e0 0x7f512415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f512a1c1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51240ce450 0x7f512415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f512a1c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5124156b00 0x7f512415c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f512a1c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f512415e9c0 con 0x7f5124156b00 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f512a1c1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f512415b1a0 con 0x7f510c0039e0 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f512a1c1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f512415eb40 con 0x7f51240ce450 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5123fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5124156b00 0x7f512415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5123fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5124156b00 0x7f512415c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40624/0 (socket says 172.21.15.26:40624) 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5123fff640 1 -- 172.21.15.26:0/1600885026 learned_addr learned my addr 172.21.15.26:0/1600885026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5122ffd640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51240ce450 0x7f512415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f51237fe640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c0039e0 0x7f512415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5123fff640 1 -- 172.21.15.26:0/1600885026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51240ce450 msgr2=0x7f512415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5123fff640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51240ce450 0x7f512415bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5123fff640 1 -- 172.21.15.26:0/1600885026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c0039e0 msgr2=0x7f512415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5123fff640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c0039e0 0x7f512415dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5123fff640 1 -- 172.21.15.26:0/1600885026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512409b990 con 0x7f5124156b00 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f51237fe640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c0039e0 0x7f512415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f5123fff640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5124156b00 0x7f512415c220 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f512406f210 tx=0x7f511c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f510bfff640 1 -- 172.21.15.26:0/1600885026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c00ee50 con 0x7f5124156b00 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f510bfff640 1 -- 172.21.15.26:0/1600885026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c018070 con 0x7f5124156b00 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f512a1c1640 1 -- 172.21.15.26:0/1600885026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512409bc70 con 0x7f5124156b00 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f510bfff640 1 -- 172.21.15.26:0/1600885026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c003dc0 con 0x7f5124156b00 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.982+0000 7f512a1c1640 1 -- 172.21.15.26:0/1600885026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512409c230 con 0x7f5124156b00 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.984+0000 7f510bfff640 1 -- 172.21.15.26:0/1600885026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f511c0050d0 con 0x7f5124156b00 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.984+0000 7f510bfff640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f511006bd70 0x7f511006e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.985+0000 7f51237fe640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f511006bd70 0x7f511006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.985+0000 7f510bfff640 1 -- 172.21.15.26:0/1600885026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f511c08abc0 con 0x7f5124156b00 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.985+0000 7f51237fe640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f511006bd70 0x7f511006e230 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5114033000 tx=0x7f5114008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.985+0000 7f512cfae740 1 Processor -- start 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.985+0000 7f512cfae740 1 -- start start 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.986+0000 7f512cfae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60305d60 0x55de60306140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.986+0000 7f512cfae740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60323600 0x55de60323b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.986+0000 7f512cfae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de602dba40 0x55de60321cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.986+0000 7f512cfae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55de6023f6b0 con 0x55de602dba40 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.986+0000 7f512cfae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55de601e7880 con 0x55de60305d60 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.986+0000 7f512cfae740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55de6023b670 con 0x55de60323600 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5123fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de602dba40 0x55de60321cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5123fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de602dba40 0x55de60321cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40640/0 (socket says 172.21.15.26:40640) 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5123fff640 1 -- 172.21.15.26:0/911731649 learned_addr learned my addr 172.21.15.26:0/911731649 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f51237fe640 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60305d60 0x55de60306140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5122ffd640 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60323600 0x55de60323b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5123fff640 1 -- 172.21.15.26:0/911731649 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60323600 msgr2=0x55de60323b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5123fff640 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60323600 0x55de60323b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5123fff640 1 -- 172.21.15.26:0/911731649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60305d60 msgr2=0x55de60306140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5123fff640 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60305d60 0x55de60306140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5123fff640 1 -- 172.21.15.26:0/911731649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de6001d7d0 con 0x55de602dba40 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5122ffd640 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60323600 0x55de60323b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f5123fff640 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de602dba40 0x55de60321cc0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f511c0614d0 tx=0x7f511c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87978715bc26442 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f51097fa640 1 -- 172.21.15.26:0/911731649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c087070 con 0x55de602dba40 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f51097fa640 1 -- 172.21.15.26:0/911731649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c05d1d0 con 0x55de602dba40 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.987+0000 7f51097fa640 1 -- 172.21.15.26:0/911731649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c061a00 con 0x55de602dba40 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 -- 172.21.15.26:0/911731649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de602dba40 msgr2=0x55de60321cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de602dba40 0x55de60321cc0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f511c0614d0 tx=0x7f511c001930 comp rx=0 tx=0).stop 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 -- 172.21.15.26:0/911731649 shutdown_connections 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de602dba40 0x55de60321cc0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60323600 0x55de60323b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 --2- 172.21.15.26:0/911731649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60305d60 0x55de60306140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 -- 172.21.15.26:0/911731649 >> 172.21.15.26:0/911731649 conn(0x55de602d8600 msgr2=0x55de60320b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 -- 172.21.15.26:0/911731649 shutdown_connections 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 -- 172.21.15.26:0/911731649 wait complete. 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.988+0000 7f512cfae740 1 -- start start 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f512cfae740 1 Processor -- start 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f512cfae740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 0x55de60330bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f512cfae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60323600 0x55de60327ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f512cfae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60308a90 0x55de60308f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f512cfae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55de602dc7e0 con 0x55de60308a90 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f512cfae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55de601faf50 con 0x55de60323600 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f512cfae740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55de602d6530 con 0x55de60305d60 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f512cfae740 10 client.0 _ll_register_callbacks cb 0x55de60236000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f5123fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60308a90 0x55de60308f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f5123fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60308a90 0x55de60308f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40656/0 (socket says 172.21.15.26:40656) 2024-04-24T23:26:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f5123fff640 1 -- 172.21.15.26:0/1904481493 learned_addr learned my addr 172.21.15.26:0/1904481493 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.989+0000 7f5122ffd640 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60323600 0x55de60327ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f51237fe640 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 0x55de60330bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f5123fff640 1 -- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 msgr2=0x55de60330bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f5123fff640 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 0x55de60330bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f5123fff640 1 -- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60323600 msgr2=0x55de60327ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f5123fff640 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60323600 0x55de60327ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f5123fff640 1 -- 172.21.15.26:0/1904481493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5ff533f0 con 0x55de60308a90 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f5123fff640 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60308a90 0x55de60308f10 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f511c058b20 tx=0x7f511c0015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb290af3ba7b25c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f51027fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f51027fc640 1 -- 172.21.15.26:0/1904481493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c087070 con 0x55de60308a90 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f51027fc640 1 -- 172.21.15.26:0/1904481493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f511c058270 con 0x55de60308a90 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f512cfae740 1 -- 172.21.15.26:0/1904481493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511c05d1d0 con 0x55de60308a90 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f51027fc640 1 -- 172.21.15.26:0/1904481493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f511c081050 con 0x55de60308a90 2024-04-24T23:26:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f512cfae740 10 client.8310 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f512cfae740 1 -- 172.21.15.26:0/1904481493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55de5ff130f0 con 0x55de60308a90 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f51027fc640 1 -- 172.21.15.26:0/1904481493 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c05d1d0 con 0x55de60308a90 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f51027fc640 1 -- 172.21.15.26:0/1904481493 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f511c05f3e0 con 0x55de60308a90 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f512cfae740 10 client.8310 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.990+0000 7f512cfae740 10 client.8310 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.991+0000 7f512cfae740 20 client.8310 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.991+0000 7f512cfae740 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55de603073d0 0x55de603077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.991+0000 7f512cfae740 4 client.8310 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.991+0000 7f512cfae740 1 -- 172.21.15.26:0/1904481493 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de6024cbf0 con 0x55de603073d0 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.991+0000 7f51237fe640 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55de603073d0 0x55de603077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.991+0000 7f51237fe640 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55de603073d0 0x55de603077b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f39e02ac473d01 server_cookie=77d5bf5304b28a28 in_seq=0 out_seq=0 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.991+0000 7f51027fc640 10 client.8310 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f51027fc640 1 -- 172.21.15.26:0/1904481493 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de6024cbf0 con 0x55de603073d0 2024-04-24T23:26:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f51027fc640 10 client.8310 handle_command_reply: tid=0 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 client.8310 shutdown 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 -- 172.21.15.26:0/1904481493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60308a90 msgr2=0x55de60308f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60308a90 0x55de60308f10 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f511c058b20 tx=0x7f511c0015b0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 -- 172.21.15.26:0/1904481493 shutdown_connections 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55de603073d0 0x55de603077b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60308a90 0x55de60308f10 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60323600 0x55de60327ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f51027fc640 0 client.8310 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 --2- 172.21.15.26:0/1904481493 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 0x55de60330bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 -- 172.21.15.26:0/1904481493 >> 172.21.15.26:0/1904481493 conn(0x55de602dba40 msgr2=0x55de60324980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 -- 172.21.15.26:0/1904481493 shutdown_connections 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 1 -- 172.21.15.26:0/1904481493 wait complete. 2024-04-24T23:26:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:51.992+0000 7f512cfae740 20 client.8310 trim_cache size 0 max 16384 2024-04-24T23:26:52.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.010+0000 7f512cfae740 1 Processor -- start 2024-04-24T23:26:52.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.010+0000 7f512cfae740 1 -- start start 2024-04-24T23:26:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.010+0000 7f512cfae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60305d60 0x55de602dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.010+0000 7f512cfae740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60308a90 0x55de602dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.010+0000 7f512cfae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60323600 0x55de60325240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.010+0000 7f512cfae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5124200ee0 con 0x55de60305d60 2024-04-24T23:26:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.010+0000 7f512cfae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f51242000b0 con 0x55de60323600 2024-04-24T23:26:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.010+0000 7f512cfae740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f51242007b0 con 0x55de60308a90 2024-04-24T23:26:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f51237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60305d60 0x55de602dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f51237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60305d60 0x55de602dce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40664/0 (socket says 172.21.15.26:40664) 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f51237fe640 1 -- 172.21.15.26:0/3385915349 learned_addr learned my addr 172.21.15.26:0/3385915349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f51237fe640 1 -- 172.21.15.26:0/3385915349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60308a90 msgr2=0x55de602dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f51237fe640 1 --2- 172.21.15.26:0/3385915349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60308a90 0x55de602dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f51237fe640 1 -- 172.21.15.26:0/3385915349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60323600 msgr2=0x55de60325240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f51237fe640 1 --2- 172.21.15.26:0/3385915349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60323600 0x55de60325240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f51237fe640 1 -- 172.21.15.26:0/3385915349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de602d6990 con 0x55de60305d60 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f51237fe640 1 --2- 172.21.15.26:0/3385915349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60305d60 0x55de602dce10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f51140042b0 tx=0x7f511402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ac0080c7fd93f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f5108ff9640 1 -- 172.21.15.26:0/3385915349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114056070 con 0x55de60305d60 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f5108ff9640 1 -- 172.21.15.26:0/3385915349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5114051dd0 con 0x55de60305d60 2024-04-24T23:26:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.011+0000 7f5108ff9640 1 -- 172.21.15.26:0/3385915349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114040040 con 0x55de60305d60 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 -- 172.21.15.26:0/3385915349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60305d60 msgr2=0x55de602dce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 --2- 172.21.15.26:0/3385915349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60305d60 0x55de602dce10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f51140042b0 tx=0x7f511402e000 comp rx=0 tx=0).stop 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 -- 172.21.15.26:0/3385915349 shutdown_connections 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 --2- 172.21.15.26:0/3385915349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60323600 0x55de60325240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 --2- 172.21.15.26:0/3385915349 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60308a90 0x55de602dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 --2- 172.21.15.26:0/3385915349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60305d60 0x55de602dce10 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 -- 172.21.15.26:0/3385915349 >> 172.21.15.26:0/3385915349 conn(0x55de602dba40 msgr2=0x55de603082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 -- 172.21.15.26:0/3385915349 shutdown_connections 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 -- 172.21.15.26:0/3385915349 wait complete. 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 -- start start 2024-04-24T23:26:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.012+0000 7f512cfae740 1 Processor -- start 2024-04-24T23:26:52.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f512cfae740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 0x55de60343d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f512cfae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60308a90 0x55de603249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f512cfae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60323600 0x55de60325220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f512cfae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55de6030a090 con 0x55de60323600 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f512cfae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5124153980 con 0x55de60308a90 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f512cfae740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55de6020cec0 con 0x55de60305d60 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f512cfae740 10 client.0 _ll_register_callbacks cb 0x7f511c085070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f5123fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60323600 0x55de60325220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f5123fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60323600 0x55de60325220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40672/0 (socket says 172.21.15.26:40672) 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f5123fff640 1 -- 172.21.15.26:0/4068611422 learned_addr learned my addr 172.21.15.26:0/4068611422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f5122ffd640 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60308a90 0x55de603249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.013+0000 7f51237fe640 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 0x55de60343d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f5123fff640 1 -- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 msgr2=0x55de60343d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f5123fff640 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 0x55de60343d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f5123fff640 1 -- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60308a90 msgr2=0x55de603249f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f5123fff640 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60308a90 0x55de603249f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f5123fff640 1 -- 172.21.15.26:0/4068611422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de60309100 con 0x55de60323600 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f5123fff640 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60323600 0x55de60325220 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f511c002230 tx=0x7f511c058ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b70ab5e564842d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f51027fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f51027fc640 1 -- 172.21.15.26:0/4068611422 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c065070 con 0x55de60323600 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f512cfae740 1 -- 172.21.15.26:0/4068611422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5114051dd0 con 0x55de60323600 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f51027fc640 1 -- 172.21.15.26:0/4068611422 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f511c054b70 con 0x55de60323600 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f51027fc640 1 -- 172.21.15.26:0/4068611422 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f511c048050 con 0x55de60323600 2024-04-24T23:26:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f512cfae740 10 client.8316 fetch_fsmap learned FSMap version 91 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f512cfae740 1 -- 172.21.15.26:0/4068611422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55de60328210 con 0x55de60323600 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f51027fc640 1 -- 172.21.15.26:0/4068611422 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5114051dd0 con 0x55de60323600 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f51027fc640 1 -- 172.21.15.26:0/4068611422 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 91) ==== 3546+0+0 (secure 0 0 0) 0x7f511c057ce0 con 0x55de60323600 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f512cfae740 10 client.8316 fetch_fsmap finished waiting for FSMap version 91 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f512cfae740 10 client.8316 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f512cfae740 20 client.8316 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f512cfae740 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55de602d6b60 0x55de60325760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f512cfae740 4 client.8316 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.014+0000 7f512cfae740 1 -- 172.21.15.26:0/4068611422 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55de60246f40 con 0x55de602d6b60 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f51237fe640 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55de602d6b60 0x55de60325760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f51027fc640 10 client.8316 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f51237fe640 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55de602d6b60 0x55de60325760 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40bf9612412de037 server_cookie=616766611148e732 in_seq=0 out_seq=0 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f51027fc640 1 -- 172.21.15.26:0/4068611422 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55de60246f40 con 0x55de602d6b60 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f51027fc640 10 client.8316 handle_command_reply: tid=0 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f512cfae740 1 client.8316 shutdown 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f512cfae740 1 -- 172.21.15.26:0/4068611422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60323600 msgr2=0x55de60325220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f512cfae740 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60323600 0x55de60325220 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f511c002230 tx=0x7f511c058ca0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f512cfae740 1 -- 172.21.15.26:0/4068611422 shutdown_connections 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f512cfae740 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55de602d6b60 0x55de60325760 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f512cfae740 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55de60323600 0x55de60325220 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f51027fc640 0 client.8316 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f512cfae740 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55de60308a90 0x55de603249f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f512cfae740 1 --2- 172.21.15.26:0/4068611422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55de60305d60 0x55de60343d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.015+0000 7f512cfae740 1 -- 172.21.15.26:0/4068611422 >> 172.21.15.26:0/4068611422 conn(0x55de602dba40 msgr2=0x55de602d5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512cfae740 1 -- 172.21.15.26:0/4068611422 shutdown_connections 2024-04-24T23:26:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512cfae740 1 -- 172.21.15.26:0/4068611422 wait complete. 2024-04-24T23:26:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512cfae740 20 client.8316 trim_cache size 0 max 16384 2024-04-24T23:26:52.024 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:52.024 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:26:52.024 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:26:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "cca3b3ba-5fb5-4b95-8bce-070d29d5babd": { 2024-04-24T23:26:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:26:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "cca3b3ba-5fb5-4b95-8bce-070d29d5babd", 2024-04-24T23:26:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:26:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:52.025 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 -- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f511006bd70 msgr2=0x7f511006e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f511006bd70 0x7f511006e230 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5114033000 tx=0x7f5114008030 comp rx=0 tx=0).stop 2024-04-24T23:26:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 -- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5124156b00 msgr2=0x7f512415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5124156b00 0x7f512415c220 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f512406f210 tx=0x7f511c00ec00 comp rx=0 tx=0).stop 2024-04-24T23:26:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 -- 172.21.15.26:0/1600885026 shutdown_connections 2024-04-24T23:26:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f511006bd70 0x7f511006e230 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5124156b00 0x7f512415c220 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f51240ce450 0x7f512415bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 --2- 172.21.15.26:0/1600885026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f510c0039e0 0x7f512415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 -- 172.21.15.26:0/1600885026 >> 172.21.15.26:0/1600885026 conn(0x7f512406b540 msgr2=0x7f5124159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 -- 172.21.15.26:0/1600885026 shutdown_connections 2024-04-24T23:26:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.016+0000 7f512a1c1640 1 -- 172.21.15.26:0/1600885026 wait complete. 2024-04-24T23:26:52.028 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:26:52.028 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.128+0000 7fc963963640 1 Processor -- start 2024-04-24T23:26:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.128+0000 7fc963963640 1 -- start start 2024-04-24T23:26:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.129+0000 7fc963963640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc95c0cf870 0x7fc95c072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.129+0000 7fc963963640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc95c15eb00 0x7fc95c15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.129+0000 7fc963963640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc95c15f450 con 0x7fc95c143c40 2024-04-24T23:26:52.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.129+0000 7fc963963640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc95c071b70 con 0x7fc95c0cf870 2024-04-24T23:26:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.129+0000 7fc963963640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc95c0cd0b0 con 0x7fc95c15eb00 2024-04-24T23:26:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.129+0000 7fc9616ff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc95c143c40 0x7fc95c0cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51618/0 (socket says 172.21.15.26:51618) 2024-04-24T23:26:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.129+0000 7fc9616ff640 1 -- 172.21.15.26:0/2652750781 learned_addr learned my addr 172.21.15.26:0/2652750781 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.129+0000 7fc960efe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc95c0cf870 0x7fc95c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.129+0000 7fc961f00640 1 --2- 172.21.15.26:0/2652750781 >> v2:172.21.15.29:3300/0 conn(0x7fc95c15eb00 0x7fc95c15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.130+0000 7fc960efe640 1 -- 172.21.15.26:0/2652750781 >> v2:172.21.15.29:3300/0 conn(0x7fc95c15eb00 msgr2=0x7fc95c15ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.130+0000 7fc960efe640 1 --2- 172.21.15.26:0/2652750781 >> v2:172.21.15.29:3300/0 conn(0x7fc95c15eb00 0x7fc95c15ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.130+0000 7fc960efe640 1 -- 172.21.15.26:0/2652750781 >> v1:172.21.15.26:6789/0 conn(0x7fc95c143c40 legacy=0x7fc95c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.130+0000 7fc960efe640 1 -- 172.21.15.26:0/2652750781 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95c071240 con 0x7fc95c0cf870 2024-04-24T23:26:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.130+0000 7fc960efe640 1 --2- 172.21.15.26:0/2652750781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 0x7fc95c072000 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc95c144d10 tx=0x7fc94c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c49f0c3c3a1b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.130+0000 7fc953fff640 1 -- 172.21.15.26:0/2652750781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c037070 con 0x7fc95c0cf870 2024-04-24T23:26:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.130+0000 7fc953fff640 1 -- 172.21.15.26:0/2652750781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94c032440 con 0x7fc95c0cf870 2024-04-24T23:26:52.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.130+0000 7fc953fff640 1 -- 172.21.15.26:0/2652750781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c032810 con 0x7fc95c0cf870 2024-04-24T23:26:52.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.131+0000 7fc963963640 1 -- 172.21.15.26:0/2652750781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 msgr2=0x7fc95c072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.131+0000 7fc963963640 1 --2- 172.21.15.26:0/2652750781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 0x7fc95c072000 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc95c144d10 tx=0x7fc94c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.131+0000 7fc963963640 1 -- 172.21.15.26:0/2652750781 shutdown_connections 2024-04-24T23:26:52.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.131+0000 7fc963963640 1 --2- 172.21.15.26:0/2652750781 >> v2:172.21.15.29:3300/0 conn(0x7fc95c15eb00 0x7fc95c15ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.131+0000 7fc963963640 1 --2- 172.21.15.26:0/2652750781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 0x7fc95c072000 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.131+0000 7fc963963640 1 -- 172.21.15.26:0/2652750781 >> 172.21.15.26:0/2652750781 conn(0x7fc95c06b300 msgr2=0x7fc95c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.131+0000 7fc963963640 1 -- 172.21.15.26:0/2652750781 shutdown_connections 2024-04-24T23:26:52.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.131+0000 7fc963963640 1 -- 172.21.15.26:0/2652750781 wait complete. 2024-04-24T23:26:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.132+0000 7fc963963640 1 Processor -- start 2024-04-24T23:26:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.132+0000 7fc963963640 1 -- start start 2024-04-24T23:26:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc963963640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 0x7fc95c1318a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc963963640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc95c143c40 0x7fc95c15e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc963963640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc95c15eb00 0x7fc95c13d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc963963640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c144020 con 0x7fc95c0cf870 2024-04-24T23:26:52.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc963963640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c071800 con 0x7fc95c143c40 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc963963640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc95c072540 con 0x7fc95c15eb00 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc9616ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 0x7fc95c1318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc9616ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 0x7fc95c1318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40684/0 (socket says 172.21.15.26:40684) 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc9616ff640 1 -- 172.21.15.26:0/2152114739 learned_addr learned my addr 172.21.15.26:0/2152114739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc960efe640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc95c143c40 0x7fc95c15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc9616ff640 1 -- 172.21.15.26:0/2152114739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc95c15eb00 msgr2=0x7fc95c13d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc961f00640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc95c15eb00 0x7fc95c13d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc9616ff640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc95c15eb00 0x7fc95c13d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc9616ff640 1 -- 172.21.15.26:0/2152114739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc95c143c40 msgr2=0x7fc95c15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc9616ff640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc95c143c40 0x7fc95c15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc9616ff640 1 -- 172.21.15.26:0/2152114739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95c13dc40 con 0x7fc95c0cf870 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.133+0000 7fc960efe640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc95c143c40 0x7fc95c15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.134+0000 7fc9616ff640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 0x7fc95c1318a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc95800c430 tx=0x7fc95800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.134+0000 7fc951ffb640 1 -- 172.21.15.26:0/2152114739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958017070 con 0x7fc95c0cf870 2024-04-24T23:26:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.134+0000 7fc951ffb640 1 -- 172.21.15.26:0/2152114739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc958005ce0 con 0x7fc95c0cf870 2024-04-24T23:26:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.134+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95c1327d0 con 0x7fc95c0cf870 2024-04-24T23:26:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.134+0000 7fc951ffb640 1 -- 172.21.15.26:0/2152114739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9580072f0 con 0x7fc95c0cf870 2024-04-24T23:26:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.134+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95c132210 con 0x7fc95c0cf870 2024-04-24T23:26:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.135+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc95c0694c0 con 0x7fc95c0cf870 2024-04-24T23:26:52.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.135+0000 7fc951ffb640 1 -- 172.21.15.26:0/2152114739 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc95801a070 con 0x7fc95c0cf870 2024-04-24T23:26:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.136+0000 7fc951ffb640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc94006be30 0x7fc94006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.136+0000 7fc951ffb640 1 -- 172.21.15.26:0/2152114739 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fc95808a210 con 0x7fc95c0cf870 2024-04-24T23:26:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.136+0000 7fc960efe640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc94006be30 0x7fc94006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.136+0000 7fc960efe640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc94006be30 0x7fc94006e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc94c034000 tx=0x7fc94c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.140+0000 7fc951ffb640 1 -- 172.21.15.26:0/2152114739 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc95c15f450 con 0x7fc95c0cf870 2024-04-24T23:26:52.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.362+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc95c0b4d20 con 0x7fc95c0cf870 2024-04-24T23:26:52.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.363+0000 7fc951ffb640 1 -- 172.21.15.26:0/2152114739 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5222 (secure 0 0 0) 0x7fc958058c10 con 0x7fc95c0cf870 2024-04-24T23:26:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:52.364 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:48.156525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6299,6778,6790]},"id":6}]} 2024-04-24T23:26:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 91 2024-04-24T23:26:52.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc94006be30 msgr2=0x7fc94006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc94006be30 0x7fc94006e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc94c034000 tx=0x7fc94c02e000 comp rx=0 tx=0).stop 2024-04-24T23:26:52.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 msgr2=0x7fc95c1318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 0x7fc95c1318a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc95800c430 tx=0x7fc95800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 shutdown_connections 2024-04-24T23:26:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc94006be30 0x7fc94006e2f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc95c15eb00 0x7fc95c13d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc95c143c40 0x7fc95c15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 --2- 172.21.15.26:0/2152114739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc95c0cf870 0x7fc95c1318a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 >> 172.21.15.26:0/2152114739 conn(0x7fc95c06b300 msgr2=0x7fc95c13f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 shutdown_connections 2024-04-24T23:26:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.365+0000 7fc963963640 1 -- 172.21.15.26:0/2152114739 wait complete. 2024-04-24T23:26:52.377 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'a', 'd'] 2024-04-24T23:26:52.377 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:26:52.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.476+0000 7faaf2f77640 1 Processor -- start 2024-04-24T23:26:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.477+0000 7faaf2f77640 1 -- start start 2024-04-24T23:26:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.477+0000 7faaf2f77640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec162cb0 0x7faaec0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.477+0000 7faaf2f77640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faaec0d4860 0x7faaec0d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.477+0000 7faaf2f77640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaec0d1fb0 con 0x7faaec0d10a0 2024-04-24T23:26:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.477+0000 7faaf2f77640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faaec0d1510 con 0x7faaec0d4860 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.477+0000 7faaf2f77640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faaec0cff00 con 0x7faaec162cb0 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaf1514640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faaec0d10a0 0x7faaec0d21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51628/0 (socket says 172.21.15.26:51628) 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaf1514640 1 -- 172.21.15.26:0/274350554 learned_addr learned my addr 172.21.15.26:0/274350554 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaebfff640 1 --2- 172.21.15.26:0/274350554 >> v2:172.21.15.29:3300/0 conn(0x7faaec0d4860 0x7faaec0d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaf0d13640 1 --2- 172.21.15.26:0/274350554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec162cb0 0x7faaec0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaeb7fe640 1 -- 172.21.15.26:0/274350554 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1754534681 0 0) 0x7faaec0d1fb0 con 0x7faaec0d10a0 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaeb7fe640 1 -- 172.21.15.26:0/274350554 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faad4003520 con 0x7faaec0d10a0 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaebfff640 1 -- 172.21.15.26:0/274350554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec162cb0 msgr2=0x7faaec0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaebfff640 1 --2- 172.21.15.26:0/274350554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec162cb0 0x7faaec0cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaebfff640 1 -- 172.21.15.26:0/274350554 >> v1:172.21.15.26:6789/0 conn(0x7faaec0d10a0 legacy=0x7faaec0d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaebfff640 1 -- 172.21.15.26:0/274350554 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec0b7140 con 0x7faaec0d4860 2024-04-24T23:26:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.478+0000 7faaf0d13640 1 --2- 172.21.15.26:0/274350554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec162cb0 0x7faaec0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.479+0000 7faaebfff640 1 --2- 172.21.15.26:0/274350554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec0d4860 0x7faaec0d0b60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faae00097a0 tx=0x7faae002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9378b72e66a421c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.479+0000 7faaeb7fe640 1 -- 172.21.15.26:0/274350554 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0037070 con 0x7faaec0d4860 2024-04-24T23:26:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.479+0000 7faaeb7fe640 1 -- 172.21.15.26:0/274350554 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae0032440 con 0x7faaec0d4860 2024-04-24T23:26:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.479+0000 7faaeb7fe640 1 -- 172.21.15.26:0/274350554 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae00327f0 con 0x7faaec0d4860 2024-04-24T23:26:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.479+0000 7faaf2f77640 1 -- 172.21.15.26:0/274350554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec0d4860 msgr2=0x7faaec0d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.479+0000 7faaf2f77640 1 --2- 172.21.15.26:0/274350554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec0d4860 0x7faaec0d0b60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faae00097a0 tx=0x7faae002a180 comp rx=0 tx=0).stop 2024-04-24T23:26:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.480+0000 7faaf2f77640 1 -- 172.21.15.26:0/274350554 shutdown_connections 2024-04-24T23:26:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.480+0000 7faaf2f77640 1 --2- 172.21.15.26:0/274350554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec0d4860 0x7faaec0d0b60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.480+0000 7faaf2f77640 1 --2- 172.21.15.26:0/274350554 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec162cb0 0x7faaec0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.480+0000 7faaf2f77640 1 -- 172.21.15.26:0/274350554 >> 172.21.15.26:0/274350554 conn(0x7faaec09d500 msgr2=0x7faaec0d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.480+0000 7faaf2f77640 1 -- 172.21.15.26:0/274350554 shutdown_connections 2024-04-24T23:26:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.480+0000 7faaf2f77640 1 -- 172.21.15.26:0/274350554 wait complete. 2024-04-24T23:26:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.481+0000 7faaf2f77640 1 Processor -- start 2024-04-24T23:26:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.481+0000 7faaf2f77640 1 -- start start 2024-04-24T23:26:52.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf2f77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec0d10a0 0x7faaec0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf2f77640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec0d4860 0x7faaec0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf2f77640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec162cb0 0x7faaec15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf2f77640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0b8db0 con 0x7faaec0d10a0 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf2f77640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0b8c30 con 0x7faaec162cb0 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf2f77640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faaec0b8f30 con 0x7faaec0d4860 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf0d13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec0d10a0 0x7faaec0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf0d13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec0d10a0 0x7faaec0ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40686/0 (socket says 172.21.15.26:40686) 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf0d13640 1 -- 172.21.15.26:0/4233826838 learned_addr learned my addr 172.21.15.26:0/4233826838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaebfff640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec0d4860 0x7faaec0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf1514640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec162cb0 0x7faaec15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf0d13640 1 -- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec0d4860 msgr2=0x7faaec0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf0d13640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec0d4860 0x7faaec0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf0d13640 1 -- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec162cb0 msgr2=0x7faaec15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf0d13640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec162cb0 0x7faaec15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf0d13640 1 -- 172.21.15.26:0/4233826838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec0c8cd0 con 0x7faaec0d10a0 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaebfff640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec0d4860 0x7faaec0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf1514640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec162cb0 0x7faaec15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf0d13640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec0d10a0 0x7faaec0ca050 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7faadc00e990 tx=0x7faadc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faae97fa640 1 -- 172.21.15.26:0/4233826838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc017070 con 0x7faaec0d10a0 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faae97fa640 1 -- 172.21.15.26:0/4233826838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc00c9e0 con 0x7faaec0d10a0 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec0c8fb0 con 0x7faaec0d10a0 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.482+0000 7faae97fa640 1 -- 172.21.15.26:0/4233826838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc00cdb0 con 0x7faaec0d10a0 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.483+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec0c9850 con 0x7faaec0d10a0 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.483+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec0694c0 con 0x7faaec0d10a0 2024-04-24T23:26:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.484+0000 7faae97fa640 1 -- 172.21.15.26:0/4233826838 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faadc01a070 con 0x7faaec0d10a0 2024-04-24T23:26:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.484+0000 7faae97fa640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad806be30 0x7faad806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.484+0000 7faae97fa640 1 -- 172.21.15.26:0/4233826838 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7faadc08b0b0 con 0x7faaec0d10a0 2024-04-24T23:26:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.485+0000 7faaebfff640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad806be30 0x7faad806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.485+0000 7faaebfff640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad806be30 0x7faad806e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faae0034000 tx=0x7faae002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.489+0000 7faae97fa640 1 -- 172.21.15.26:0/4233826838 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faadc059a20 con 0x7faaec0d10a0 2024-04-24T23:26:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.744+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7faaec0cc3b0 con 0x7faaec0d10a0 2024-04-24T23:26:52.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.744+0000 7faae97fa640 1 -- 172.21.15.26:0/4233826838 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1577 (secure 0 0 0) 0x7faadc059320 con 0x7faaec0d10a0 2024-04-24T23:26:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:52.745 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":381,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":62,"data_bytes":654430,"bytes_used":278011904,"bytes_avail":772816101376,"bytes_total":773094113280,"write_bytes_sec":3924,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":91,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6299},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":6778},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-24T23:25:11.372341+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:26:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.746+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad806be30 msgr2=0x7faad806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.746+0000 7faaf2f77640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad806be30 0x7faad806e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faae0034000 tx=0x7faae002e000 comp rx=0 tx=0).stop 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.746+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec0d10a0 msgr2=0x7faaec0ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.746+0000 7faaf2f77640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec0d10a0 0x7faaec0ca050 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7faadc00e990 tx=0x7faadc00ee50 comp rx=0 tx=0).stop 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.746+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 shutdown_connections 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.746+0000 7faaf2f77640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad806be30 0x7faad806e2f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.746+0000 7faaf2f77640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaec162cb0 0x7faaec15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.746+0000 7faaf2f77640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faaec0d4860 0x7faaec0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.746+0000 7faaf2f77640 1 --2- 172.21.15.26:0/4233826838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec0d10a0 0x7faaec0ca050 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.747+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 >> 172.21.15.26:0/4233826838 conn(0x7faaec09d500 msgr2=0x7faaec161ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.747+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 shutdown_connections 2024-04-24T23:26:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:52.747+0000 7faaf2f77640 1 -- 172.21.15.26:0/4233826838 wait complete. 2024-04-24T23:26:52.758 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-24T23:26:53.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.872+0000 7fda60e93640 1 Processor -- start 2024-04-24T23:26:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.872+0000 7fda60e93640 1 -- start start 2024-04-24T23:26:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.873+0000 7fda60e93640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fda5c06f4f0 0x7fda5c0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.873+0000 7fda60e93640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5c0b0220 con 0x7fda5c0addb0 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.873+0000 7fda60e93640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5c0af180 con 0x7fda5c06f110 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.873+0000 7fda60e93640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fda5c0af460 con 0x7fda5c06f4f0 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.873+0000 7fda5ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fda5c0addb0 0x7fda5c154a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40088/0 (socket says 172.21.15.26:40088) 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.873+0000 7fda5ad9d640 1 -- 172.21.15.26:0/4028616557 learned_addr learned my addr 172.21.15.26:0/4028616557 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.873+0000 7fda53fff640 1 --2- 172.21.15.26:0/4028616557 >> v2:172.21.15.29:3300/0 conn(0x7fda5c06f4f0 0x7fda5c0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.873+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4017335517 0 0) 0x7fda5c0b0220 con 0x7fda5c0addb0 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.873+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda48003520 con 0x7fda5c0addb0 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1723761094 0 0) 0x7fda48003520 con 0x7fda5c0addb0 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 >> v2:172.21.15.29:3300/0 conn(0x7fda5c06f4f0 msgr2=0x7fda5c0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 --2- 172.21.15.26:0/4028616557 >> v2:172.21.15.29:3300/0 conn(0x7fda5c06f4f0 0x7fda5c0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 >> v1:172.21.15.29:6789/0 conn(0x7fda5c06f110 legacy=0x7fda5c15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c0af740 con 0x7fda5c0addb0 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fda540019f0 con 0x7fda5c0addb0 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda53fff640 1 --2- 172.21.15.26:0/4028616557 >> v2:172.21.15.29:3300/0 conn(0x7fda5c06f4f0 0x7fda5c0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 >> v1:172.21.15.26:6789/0 conn(0x7fda5c0addb0 legacy=0x7fda5c154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda480039b0 0x7fda48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda480051c0 0x7fda4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c0af460 con 0x7fda48004800 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda4800c060 con 0x7fda480039b0 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fda4800c260 con 0x7fda480051c0 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda53fff640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda5a59c640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda480039b0 0x7fda48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda5ad9d640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda480051c0 0x7fda4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda53fff640 1 -- 172.21.15.26:0/4028616557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda480051c0 msgr2=0x7fda4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda53fff640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda480051c0 0x7fda4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda53fff640 1 -- 172.21.15.26:0/4028616557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda480039b0 msgr2=0x7fda48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda53fff640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda480039b0 0x7fda48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda53fff640 1 -- 172.21.15.26:0/4028616557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda48003520 con 0x7fda48004800 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.874+0000 7fda5ad9d640 1 -- 172.21.15.26:0/4028616557 reap_dead start 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.875+0000 7fda53fff640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda48004c80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fda44009c60 tx=0x7fda48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252ea61b1e7c7e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.875+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44036070 con 0x7fda48004800 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.875+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda44031400 con 0x7fda48004800 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.875+0000 7fda59d9b640 1 -- 172.21.15.26:0/4028616557 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44009860 con 0x7fda48004800 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.876+0000 7fda60e93640 1 -- 172.21.15.26:0/4028616557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 msgr2=0x7fda48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.876+0000 7fda60e93640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda48004c80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fda44009c60 tx=0x7fda48005830 comp rx=0 tx=0).stop 2024-04-24T23:26:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.876+0000 7fda60e93640 1 -- 172.21.15.26:0/4028616557 shutdown_connections 2024-04-24T23:26:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.876+0000 7fda60e93640 1 --2- 172.21.15.26:0/4028616557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda48004c80 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.876+0000 7fda60e93640 1 -- 172.21.15.26:0/4028616557 >> 172.21.15.26:0/4028616557 conn(0x7fda5c0729a0 msgr2=0x7fda5c06b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.876+0000 7fda60e93640 1 -- 172.21.15.26:0/4028616557 shutdown_connections 2024-04-24T23:26:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.876+0000 7fda60e93640 1 -- 172.21.15.26:0/4028616557 wait complete. 2024-04-24T23:26:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.877+0000 7fda60e93640 1 Processor -- start 2024-04-24T23:26:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.877+0000 7fda60e93640 1 -- start start 2024-04-24T23:26:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda60e93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda5c158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda60e93640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda5c15b0c0 0x7fda5c1473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda60e93640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda5c15dfa0 0x7fda5c15e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda60e93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c0aee50 con 0x7fda48004800 2024-04-24T23:26:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda60e93640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c0afd00 con 0x7fda5c15b0c0 2024-04-24T23:26:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda60e93640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fda5c06fb00 con 0x7fda5c15dfa0 2024-04-24T23:26:53.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda5c158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda5c158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52516/0 (socket says 172.21.15.26:52516) 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5a59c640 1 -- 172.21.15.26:0/516624227 learned_addr learned my addr 172.21.15.26:0/516624227 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5ad9d640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda5c15dfa0 0x7fda5c15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda53fff640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda5c15b0c0 0x7fda5c1473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5a59c640 1 -- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda5c15dfa0 msgr2=0x7fda5c15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5a59c640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda5c15dfa0 0x7fda5c15e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5a59c640 1 -- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda5c15b0c0 msgr2=0x7fda5c1473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5a59c640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda5c15b0c0 0x7fda5c1473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5a59c640 1 -- 172.21.15.26:0/516624227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c1594e0 con 0x7fda48004800 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda5ad9d640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda5c15dfa0 0x7fda5c15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.878+0000 7fda53fff640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda5c15b0c0 0x7fda5c1473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.879+0000 7fda5a59c640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda5c158e50 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00f880 tx=0x7fda4c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.879+0000 7fda52ffd640 1 -- 172.21.15.26:0/516624227 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c001d20 con 0x7fda48004800 2024-04-24T23:26:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.879+0000 7fda52ffd640 1 -- 172.21.15.26:0/516624227 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c018070 con 0x7fda48004800 2024-04-24T23:26:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.879+0000 7fda52ffd640 1 -- 172.21.15.26:0/516624227 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00d730 con 0x7fda48004800 2024-04-24T23:26:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.879+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c1597c0 con 0x7fda48004800 2024-04-24T23:26:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.879+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c159d80 con 0x7fda48004800 2024-04-24T23:26:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.880+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5c0ae840 con 0x7fda48004800 2024-04-24T23:26:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.880+0000 7fda52ffd640 1 -- 172.21.15.26:0/516624227 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda4c00d8d0 con 0x7fda48004800 2024-04-24T23:26:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.881+0000 7fda52ffd640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda3c06be30 0x7fda3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.881+0000 7fda52ffd640 1 -- 172.21.15.26:0/516624227 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fda4c08ac90 con 0x7fda48004800 2024-04-24T23:26:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.881+0000 7fda53fff640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda3c06be30 0x7fda3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.882+0000 7fda53fff640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda3c06be30 0x7fda3c06e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fda44033000 tx=0x7fda44008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:53.885+0000 7fda52ffd640 1 -- 172.21.15.26:0/516624227 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fda5c0ae840 con 0x7fda48004800 2024-04-24T23:26:54.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.104+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda5c0694c0 con 0x7fda48004800 2024-04-24T23:26:54.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.105+0000 7fda52ffd640 1 -- 172.21.15.26:0/516624227 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5224 (secure 0 0 0) 0x7fda4c059600 con 0x7fda48004800 2024-04-24T23:26:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:54.106 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:53.156327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:26:54.106 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 92 2024-04-24T23:26:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda3c06be30 msgr2=0x7fda3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda3c06be30 0x7fda3c06e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fda44033000 tx=0x7fda44008030 comp rx=0 tx=0).stop 2024-04-24T23:26:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 msgr2=0x7fda5c158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda5c158e50 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00f880 tx=0x7fda4c00fd40 comp rx=0 tx=0).stop 2024-04-24T23:26:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 shutdown_connections 2024-04-24T23:26:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fda3c06be30 0x7fda3c06e2f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fda5c15dfa0 0x7fda5c15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda5c15b0c0 0x7fda5c1473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 --2- 172.21.15.26:0/516624227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda48004800 0x7fda5c158e50 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 >> 172.21.15.26:0/516624227 conn(0x7fda5c0729a0 msgr2=0x7fda5c0b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 shutdown_connections 2024-04-24T23:26:54.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.107+0000 7fda60e93640 1 -- 172.21.15.26:0/516624227 wait complete. 2024-04-24T23:26:54.119 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'a', 'd'] 2024-04-24T23:26:54.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:26:54.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.218+0000 7f601e69a640 1 Processor -- start 2024-04-24T23:26:54.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.218+0000 7f601e69a640 1 -- start start 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.219+0000 7f601e69a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f60180b4100 0x7f60180b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.219+0000 7f601e69a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6018162860 con 0x7f60180b5e40 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.219+0000 7f601e69a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6018162b40 con 0x7f6018147e30 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.219+0000 7f601e69a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6018162e20 con 0x7f60180b4100 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.219+0000 7f601cc37640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f60180b5e40 0x7f60180b14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40104/0 (socket says 172.21.15.26:40104) 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.219+0000 7f601cc37640 1 -- 172.21.15.26:0/1423174817 learned_addr learned my addr 172.21.15.26:0/1423174817 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.219+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710543893 0 0) 0x7f6018162860 con 0x7f60180b5e40 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6004003520 con 0x7f60180b5e40 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f60177fe640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f60180b4100 0x7f60180b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 857503898 0 0) 0x7f6004003520 con 0x7f60180b5e40 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f60180b4100 msgr2=0x7f60180b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:54.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f60180b4100 0x7f60180b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 >> v1:172.21.15.29:6789/0 conn(0x7f6018147e30 legacy=0x7f6018161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60180b0cd0 con 0x7f60180b5e40 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f60100019f0 con 0x7f60180b5e40 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 >> v1:172.21.15.26:6789/0 conn(0x7f60180b5e40 legacy=0x7f60180b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60040039e0 0x7f6004003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f60180b4100 0x7f6004004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6004004d90 0x7f6004005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6018162e20 con 0x7f6004004d90 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f600400bc40 con 0x7f60040039e0 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f600400be40 con 0x7f60180b4100 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f601cc37640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6004004d90 0x7f6004005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f6017fff640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60040039e0 0x7f6004003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f60177fe640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f60180b4100 0x7f6004004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f601cc37640 1 -- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f60180b4100 msgr2=0x7f6004004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:54.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f601cc37640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f60180b4100 0x7f6004004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f601cc37640 1 -- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60040039e0 msgr2=0x7f6004003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f601cc37640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60040039e0 0x7f6004003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.220+0000 7f601cc37640 1 -- 172.21.15.26:0/1423174817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6004003520 con 0x7f6004004d90 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.221+0000 7f601cc37640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6004004d90 0x7f6004005210 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6010002230 tx=0x7f6004005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3babf2623cf0c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.221+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601000a4a0 con 0x7f6004004d90 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.221+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601002f070 con 0x7f6004004d90 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.221+0000 7f6016ffd640 1 -- 172.21.15.26:0/1423174817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601002bc80 con 0x7f6004004d90 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.222+0000 7f601e69a640 1 -- 172.21.15.26:0/1423174817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6004004d90 msgr2=0x7f6004005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.222+0000 7f601e69a640 1 --2- 172.21.15.26:0/1423174817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6004004d90 0x7f6004005210 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6010002230 tx=0x7f6004005830 comp rx=0 tx=0).stop 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.222+0000 7f601cc37640 1 -- 172.21.15.26:0/1423174817 reap_dead start 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.222+0000 7f601e69a640 1 -- 172.21.15.26:0/1423174817 shutdown_connections 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.222+0000 7f601e69a640 1 -- 172.21.15.26:0/1423174817 >> 172.21.15.26:0/1423174817 conn(0x7f601809aa00 msgr2=0x7f60180b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.222+0000 7f601e69a640 1 -- 172.21.15.26:0/1423174817 shutdown_connections 2024-04-24T23:26:54.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.222+0000 7f601e69a640 1 -- 172.21.15.26:0/1423174817 wait complete. 2024-04-24T23:26:54.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.223+0000 7f601e69a640 1 Processor -- start 2024-04-24T23:26:54.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.223+0000 7f601e69a640 1 -- start start 2024-04-24T23:26:54.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f601e69a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f601813fa70 0x7f60180ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f601e69a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f601813fe50 0x7f601809ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f601e69a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60181f8630 0x7f60180b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f601e69a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60180b5900 con 0x7f601813fa70 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f601e69a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f60180b62c0 con 0x7f60181f8630 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f601e69a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f60180b5a80 con 0x7f601813fe50 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f6017fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f601813fa70 0x7f60180ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f6017fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f601813fa70 0x7f60180ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52534/0 (socket says 172.21.15.26:52534) 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f6017fff640 1 -- 172.21.15.26:0/1463908254 learned_addr learned my addr 172.21.15.26:0/1463908254 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f601cc37640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60181f8630 0x7f60180b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f6017fff640 1 -- 172.21.15.26:0/1463908254 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f601813fe50 msgr2=0x7f601809ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f6017fff640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f601813fe50 0x7f601809ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f6017fff640 1 -- 172.21.15.26:0/1463908254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60181f8630 msgr2=0x7f60180b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f6017fff640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60181f8630 0x7f60180b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.224+0000 7f6017fff640 1 -- 172.21.15.26:0/1463908254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60180ae690 con 0x7f601813fa70 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.225+0000 7f6017fff640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f601813fa70 0x7f60180ae000 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f600000f850 tx=0x7f600000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.225+0000 7f6014ff9640 1 -- 172.21.15.26:0/1463908254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600000fe40 con 0x7f601813fa70 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.225+0000 7f6014ff9640 1 -- 172.21.15.26:0/1463908254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6000018070 con 0x7f601813fa70 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.225+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60180ae970 con 0x7f601813fa70 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.225+0000 7f6014ff9640 1 -- 172.21.15.26:0/1463908254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600000d730 con 0x7f601813fa70 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.225+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60181566c0 con 0x7f601813fa70 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.226+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60180694c0 con 0x7f601813fa70 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.226+0000 7f6014ff9640 1 -- 172.21.15.26:0/1463908254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f600000d8d0 con 0x7f601813fa70 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.227+0000 7f6014ff9640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5ff806be30 0x7f5ff806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.227+0000 7f6014ff9640 1 -- 172.21.15.26:0/1463908254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f600008ada0 con 0x7f601813fa70 2024-04-24T23:26:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.227+0000 7f60177fe640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5ff806be30 0x7f5ff806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.227+0000 7f60177fe640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5ff806be30 0x7f5ff806e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6008006a90 tx=0x7f6008006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:54.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.231+0000 7f6014ff9640 1 -- 172.21.15.26:0/1463908254 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6000059690 con 0x7f601813fa70 2024-04-24T23:26:54.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.485+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f60180aeb40 con 0x7f601813fa70 2024-04-24T23:26:54.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.486+0000 7f6014ff9640 1 -- 172.21.15.26:0/1463908254 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1597 (secure 0 0 0) 0x7f6000058f90 con 0x7f601813fa70 2024-04-24T23:26:54.487 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:54.487 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":383,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700179,"bytes_used":278085632,"bytes_avail":772816027648,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":10663,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6299},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":6778},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-24T23:25:11.372341+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:26:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5ff806be30 msgr2=0x7f5ff806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5ff806be30 0x7f5ff806e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6008006a90 tx=0x7f6008006a20 comp rx=0 tx=0).stop 2024-04-24T23:26:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f601813fa70 msgr2=0x7f60180ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f601813fa70 0x7f60180ae000 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f600000f850 tx=0x7f600000fd10 comp rx=0 tx=0).stop 2024-04-24T23:26:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 shutdown_connections 2024-04-24T23:26:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5ff806be30 0x7f5ff806e2f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f60181f8630 0x7f60180b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f601813fe50 0x7f601809ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 --2- 172.21.15.26:0/1463908254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f601813fa70 0x7f60180ae000 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 >> 172.21.15.26:0/1463908254 conn(0x7f601809aa00 msgr2=0x7f6018161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 shutdown_connections 2024-04-24T23:26:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:54.488+0000 7f601e69a640 1 -- 172.21.15.26:0/1463908254 wait complete. 2024-04-24T23:26:54.500 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-24T23:26:54.521 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:26:54.520+0000 7f26a3749640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T23:26:55.501 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.616+0000 7fbf15229640 1 Processor -- start 2024-04-24T23:26:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.616+0000 7fbf15229640 1 -- start start 2024-04-24T23:26:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.617+0000 7fbf15229640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf10145910 0x7fbf1006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.617+0000 7fbf15229640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbf1006d350 0x7fbf1015d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.617+0000 7fbf15229640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf1015d570 con 0x7fbf100cd660 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.617+0000 7fbf15229640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbf10145cf0 con 0x7fbf10145910 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.617+0000 7fbf15229640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbf100cfbc0 con 0x7fbf1006d350 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.617+0000 7fbf0ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf10145910 0x7fbf1006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.617+0000 7fbf0ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf10145910 0x7fbf1006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52540/0 (socket says 172.21.15.26:52540) 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.617+0000 7fbf0ed9d640 1 -- 172.21.15.26:0/2526568971 learned_addr learned my addr 172.21.15.26:0/2526568971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.617+0000 7fbf0e59c640 1 --2- 172.21.15.26:0/2526568971 >> v2:172.21.15.29:3300/0 conn(0x7fbf1006d350 0x7fbf1015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.618+0000 7fbf0ed9d640 1 -- 172.21.15.26:0/2526568971 >> v2:172.21.15.29:3300/0 conn(0x7fbf1006d350 msgr2=0x7fbf1015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.618+0000 7fbf0ed9d640 1 --2- 172.21.15.26:0/2526568971 >> v2:172.21.15.29:3300/0 conn(0x7fbf1006d350 0x7fbf1015d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.618+0000 7fbf0ed9d640 1 -- 172.21.15.26:0/2526568971 >> v1:172.21.15.29:6789/0 conn(0x7fbf100cd660 legacy=0x7fbf1015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.618+0000 7fbf0ed9d640 1 -- 172.21.15.26:0/2526568971 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1006d7f0 con 0x7fbf10145910 2024-04-24T23:26:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.618+0000 7fbf0ed9d640 1 --2- 172.21.15.26:0/2526568971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf10145910 0x7fbf1006e890 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009e50 tx=0x7fbf0002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5198914ea876867c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.618+0000 7fbf0dd9b640 1 -- 172.21.15.26:0/2526568971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00036070 con 0x7fbf10145910 2024-04-24T23:26:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf0dd9b640 1 -- 172.21.15.26:0/2526568971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf00031600 con 0x7fbf10145910 2024-04-24T23:26:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf0dd9b640 1 -- 172.21.15.26:0/2526568971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf000319d0 con 0x7fbf10145910 2024-04-24T23:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf15229640 1 -- 172.21.15.26:0/2526568971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf10145910 msgr2=0x7fbf1006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf15229640 1 --2- 172.21.15.26:0/2526568971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf10145910 0x7fbf1006e890 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009e50 tx=0x7fbf0002a6d0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf15229640 1 -- 172.21.15.26:0/2526568971 shutdown_connections 2024-04-24T23:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf15229640 1 --2- 172.21.15.26:0/2526568971 >> v2:172.21.15.29:3300/0 conn(0x7fbf1006d350 0x7fbf1015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf15229640 1 --2- 172.21.15.26:0/2526568971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf10145910 0x7fbf1006e890 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf15229640 1 -- 172.21.15.26:0/2526568971 >> 172.21.15.26:0/2526568971 conn(0x7fbf1006b300 msgr2=0x7fbf1006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf15229640 1 -- 172.21.15.26:0/2526568971 shutdown_connections 2024-04-24T23:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.619+0000 7fbf15229640 1 -- 172.21.15.26:0/2526568971 wait complete. 2024-04-24T23:26:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.621+0000 7fbf15229640 1 Processor -- start 2024-04-24T23:26:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.621+0000 7fbf15229640 1 -- start start 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.621+0000 7fbf15229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1006d350 0x7fbf101f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.621+0000 7fbf15229640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf100cd660 0x7fbf101558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.621+0000 7fbf15229640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf10145910 0x7fbf101e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.621+0000 7fbf15229640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1006e090 con 0x7fbf1006d350 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.621+0000 7fbf15229640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1006ddb0 con 0x7fbf10145910 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.621+0000 7fbf15229640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1006e370 con 0x7fbf100cd660 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1006d350 0x7fbf101f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1006d350 0x7fbf101f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52552/0 (socket says 172.21.15.26:52552) 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0ed9d640 1 -- 172.21.15.26:0/1866631278 learned_addr learned my addr 172.21.15.26:0/1866631278 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0f59e640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf10145910 0x7fbf101e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0e59c640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf100cd660 0x7fbf101558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0ed9d640 1 -- 172.21.15.26:0/1866631278 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf100cd660 msgr2=0x7fbf101558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0ed9d640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf100cd660 0x7fbf101558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0ed9d640 1 -- 172.21.15.26:0/1866631278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf10145910 msgr2=0x7fbf101e4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0ed9d640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf10145910 0x7fbf101e4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0ed9d640 1 -- 172.21.15.26:0/1866631278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf10155e00 con 0x7fbf1006d350 2024-04-24T23:26:55.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0e59c640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf100cd660 0x7fbf101558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf0ed9d640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1006d350 0x7fbf101f1cd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009f70 tx=0x7fbf00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf077fe640 1 -- 172.21.15.26:0/1866631278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00036050 con 0x7fbf1006d350 2024-04-24T23:26:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf077fe640 1 -- 172.21.15.26:0/1866631278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf00042070 con 0x7fbf1006d350 2024-04-24T23:26:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf101560a0 con 0x7fbf1006d350 2024-04-24T23:26:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf077fe640 1 -- 172.21.15.26:0/1866631278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00009510 con 0x7fbf1006d350 2024-04-24T23:26:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.622+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf10156660 con 0x7fbf1006d350 2024-04-24T23:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.623+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf100694c0 con 0x7fbf1006d350 2024-04-24T23:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.624+0000 7fbf077fe640 1 -- 172.21.15.26:0/1866631278 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf00007c50 con 0x7fbf1006d350 2024-04-24T23:26:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.624+0000 7fbf077fe640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.624+0000 7fbf077fe640 1 -- 172.21.15.26:0/1866631278 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fbf000abde0 con 0x7fbf1006d350 2024-04-24T23:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.625+0000 7fbf0e59c640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.625+0000 7fbf0e59c640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbef006be30 0x7fbef006e2f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbf1015b160 tx=0x7fbef8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.629+0000 7fbf077fe640 1 -- 172.21.15.26:0/1866631278 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf10155e00 con 0x7fbf1006d350 2024-04-24T23:26:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.850+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf100cdb70 con 0x7fbf1006d350 2024-04-24T23:26:55.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.850+0000 7fbf077fe640 1 -- 172.21.15.26:0/1866631278 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7fbf0007a6d0 con 0x7fbf1006d350 2024-04-24T23:26:55.851 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:55.851 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:26:55.852 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:26:55.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbef006be30 msgr2=0x7fbef006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbef006be30 0x7fbef006e2f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbf1015b160 tx=0x7fbef8008040 comp rx=0 tx=0).stop 2024-04-24T23:26:55.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1006d350 msgr2=0x7fbf101f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1006d350 0x7fbf101f1cd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009f70 tx=0x7fbf00004a40 comp rx=0 tx=0).stop 2024-04-24T23:26:55.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 shutdown_connections 2024-04-24T23:26:55.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbf10145910 0x7fbf101e4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbf100cd660 0x7fbf101558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 --2- 172.21.15.26:0/1866631278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf1006d350 0x7fbf101f1cd0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 >> 172.21.15.26:0/1866631278 conn(0x7fbf1006b300 msgr2=0x7fbf10144b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:55.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 shutdown_connections 2024-04-24T23:26:55.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.852+0000 7fbf15229640 1 -- 172.21.15.26:0/1866631278 wait complete. 2024-04-24T23:26:55.864 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'a', 'd'] 2024-04-24T23:26:55.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:26:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.964+0000 7f5e8705d640 1 Processor -- start 2024-04-24T23:26:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.964+0000 7f5e8705d640 1 -- start start 2024-04-24T23:26:55.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e8705d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e8006dc40 0x7f5e80166d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e8705d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e8015cee0 con 0x7f5e800ce040 2024-04-24T23:26:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e8705d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e8015d1c0 con 0x7f5e80145a80 2024-04-24T23:26:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e8705d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5e8015ccc0 con 0x7f5e8006dc40 2024-04-24T23:26:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e855fa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e8006dc40 0x7f5e80166d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e7ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5e800ce040 0x7f5e8006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40112/0 (socket says 172.21.15.26:40112) 2024-04-24T23:26:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e7ffff640 1 -- 172.21.15.26:0/3979048645 learned_addr learned my addr 172.21.15.26:0/3979048645 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e855fa640 1 -- 172.21.15.26:0/3979048645 >> v1:172.21.15.29:6789/0 conn(0x7f5e80145a80 legacy=0x7f5e8015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e855fa640 1 -- 172.21.15.26:0/3979048645 >> v1:172.21.15.26:6789/0 conn(0x7f5e800ce040 legacy=0x7f5e8006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.965+0000 7f5e855fa640 1 -- 172.21.15.26:0/3979048645 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8015d4a0 con 0x7f5e8006dc40 2024-04-24T23:26:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.966+0000 7f5e855fa640 1 --2- 172.21.15.26:0/3979048645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 0x7f5e80166d50 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009d90 tx=0x7f5e7802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6202e379cf79124 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.966+0000 7f5e7f7fe640 1 -- 172.21.15.26:0/3979048645 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78007740 con 0x7f5e8006dc40 2024-04-24T23:26:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.966+0000 7f5e7f7fe640 1 -- 172.21.15.26:0/3979048645 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e780078e0 con 0x7f5e8006dc40 2024-04-24T23:26:55.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.966+0000 7f5e7f7fe640 1 -- 172.21.15.26:0/3979048645 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78007cb0 con 0x7f5e8006dc40 2024-04-24T23:26:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.967+0000 7f5e8705d640 1 -- 172.21.15.26:0/3979048645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 msgr2=0x7f5e80166d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.967+0000 7f5e8705d640 1 --2- 172.21.15.26:0/3979048645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 0x7f5e80166d50 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009d90 tx=0x7f5e7802a660 comp rx=0 tx=0).stop 2024-04-24T23:26:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.967+0000 7f5e8705d640 1 -- 172.21.15.26:0/3979048645 shutdown_connections 2024-04-24T23:26:55.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.967+0000 7f5e8705d640 1 --2- 172.21.15.26:0/3979048645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 0x7f5e80166d50 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.967+0000 7f5e8705d640 1 -- 172.21.15.26:0/3979048645 >> 172.21.15.26:0/3979048645 conn(0x7f5e8006b540 msgr2=0x7f5e8006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.967+0000 7f5e8705d640 1 -- 172.21.15.26:0/3979048645 shutdown_connections 2024-04-24T23:26:55.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.967+0000 7f5e8705d640 1 -- 172.21.15.26:0/3979048645 wait complete. 2024-04-24T23:26:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.968+0000 7f5e8705d640 1 Processor -- start 2024-04-24T23:26:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.968+0000 7f5e8705d640 1 -- start start 2024-04-24T23:26:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e8705d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 0x7f5e8015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e8705d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e800ce040 0x7f5e80141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e8705d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e80145a80 0x7f5e80071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e8705d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e800b47a0 con 0x7f5e8006dc40 2024-04-24T23:26:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e8705d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5e800b3300 con 0x7f5e80145a80 2024-04-24T23:26:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e8705d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5e800b4920 con 0x7f5e800ce040 2024-04-24T23:26:55.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e84df9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 0x7f5e8015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e84df9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 0x7f5e8015bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52568/0 (socket says 172.21.15.26:52568) 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e84df9640 1 -- 172.21.15.26:0/789568595 learned_addr learned my addr 172.21.15.26:0/789568595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e855fa640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e80145a80 0x7f5e80071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e7ffff640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e800ce040 0x7f5e80141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e84df9640 1 -- 172.21.15.26:0/789568595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e800ce040 msgr2=0x7f5e80141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e84df9640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e800ce040 0x7f5e80141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e84df9640 1 -- 172.21.15.26:0/789568595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e80145a80 msgr2=0x7f5e80071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e84df9640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e80145a80 0x7f5e80071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e84df9640 1 -- 172.21.15.26:0/789568595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8015c000 con 0x7f5e8006dc40 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.969+0000 7f5e855fa640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e80145a80 0x7f5e80071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.970+0000 7f5e84df9640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 0x7f5e8015bac0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400c400 tx=0x7f5e7400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.970+0000 7f5e7d7fa640 1 -- 172.21.15.26:0/789568595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e740052d0 con 0x7f5e8006dc40 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.970+0000 7f5e7d7fa640 1 -- 172.21.15.26:0/789568595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e74005490 con 0x7f5e8006dc40 2024-04-24T23:26:55.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.970+0000 7f5e7d7fa640 1 -- 172.21.15.26:0/789568595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74007260 con 0x7f5e8006dc40 2024-04-24T23:26:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.970+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e80142830 con 0x7f5e8006dc40 2024-04-24T23:26:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.970+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e801e5b40 con 0x7f5e8006dc40 2024-04-24T23:26:55.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.971+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e800694c0 con 0x7f5e8006dc40 2024-04-24T23:26:55.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.971+0000 7f5e7d7fa640 1 -- 172.21.15.26:0/789568595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e74007400 con 0x7f5e8006dc40 2024-04-24T23:26:55.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.972+0000 7f5e7d7fa640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e6406be30 0x7f5e6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:55.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.972+0000 7f5e7d7fa640 1 -- 172.21.15.26:0/789568595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f5e7408a440 con 0x7f5e8006dc40 2024-04-24T23:26:55.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.972+0000 7f5e7ffff640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e6406be30 0x7f5e6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:55.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.972+0000 7f5e7ffff640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e6406be30 0x7f5e6406e2f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5e800ce010 tx=0x7f5e6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:55.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:55.976+0000 7f5e7d7fa640 1 -- 172.21.15.26:0/789568595 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5e74058da0 con 0x7f5e8006dc40 2024-04-24T23:26:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.232+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5e80142d50 con 0x7f5e8006dc40 2024-04-24T23:26:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.232+0000 7f5e7d7fa640 1 -- 172.21.15.26:0/789568595 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1967 (secure 0 0 0) 0x7f5e740586a0 con 0x7f5e8006dc40 2024-04-24T23:26:56.233 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:56.234 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":385,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700179,"bytes_used":278085632,"bytes_avail":772816027648,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":10066,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":94,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6299},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":6778},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-24T23:26:55.394721+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e6406be30 msgr2=0x7f5e6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e6406be30 0x7f5e6406e2f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5e800ce010 tx=0x7f5e6c009040 comp rx=0 tx=0).stop 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 msgr2=0x7f5e8015bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 0x7f5e8015bac0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400c400 tx=0x7f5e7400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 shutdown_connections 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e6406be30 0x7f5e6406e2f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e80145a80 0x7f5e80071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e800ce040 0x7f5e80141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 --2- 172.21.15.26:0/789568595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e8006dc40 0x7f5e8015bac0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 >> 172.21.15.26:0/789568595 conn(0x7f5e8006b540 msgr2=0x7f5e8015c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 shutdown_connections 2024-04-24T23:26:56.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.235+0000 7f5e8705d640 1 -- 172.21.15.26:0/789568595 wait complete. 2024-04-24T23:26:56.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:26:56.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:56.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub resume 2024-04-24T23:26:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.348+0000 7fc413d61640 1 Processor -- start 2024-04-24T23:26:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.349+0000 7fc413d61640 1 -- start start 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.349+0000 7fc413d61640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc40c15e750 0x7fc40c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.349+0000 7fc413d61640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc40c0d2120 con 0x7fc40c15e370 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.349+0000 7fc413d61640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc40c0b3740 con 0x7fc40c0d2c30 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.349+0000 7fc413d61640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c0d1510 con 0x7fc40c15e750 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.349+0000 7fc411afd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc40c15e370 0x7fc40c0d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40116/0 (socket says 172.21.15.26:40116) 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.349+0000 7fc411afd640 1 -- 172.21.15.26:0/116537893 learned_addr learned my addr 172.21.15.26:0/116537893 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc4112fc640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc40c15e750 0x7fc40c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507354148 0 0) 0x7fc40c0d2120 con 0x7fc40c15e370 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3f8003520 con 0x7fc40c15e370 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4241078104 0 0) 0x7fc40c0b3740 con 0x7fc40c0d2c30 2024-04-24T23:26:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc40c0d2120 con 0x7fc40c0d2c30 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1536366298 0 0) 0x7fc3f8003520 con 0x7fc40c15e370 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc40c15e750 msgr2=0x7fc40c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc40c15e750 0x7fc40c0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 >> v1:172.21.15.29:6789/0 conn(0x7fc40c0d2c30 legacy=0x7fc40c0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c0b3a20 con 0x7fc40c15e370 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fc408001670 con 0x7fc40c15e370 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 >> v1:172.21.15.26:6789/0 conn(0x7fc40c15e370 legacy=0x7fc40c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8003b70 0x7fc3f8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc3f8004800 0x7fc3f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f8005180 0x7fc3f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f800bf00 con 0x7fc3f8005180 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f800c080 con 0x7fc3f8003b70 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f800c200 con 0x7fc3f8004800 2024-04-24T23:26:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc4122fe640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f8005180 0x7fc3f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc411afd640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8003b70 0x7fc3f8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.350+0000 7fc4112fc640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc3f8004800 0x7fc3f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.351+0000 7fc4122fe640 1 -- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc3f8004800 msgr2=0x7fc3f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.351+0000 7fc4122fe640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc3f8004800 0x7fc3f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.351+0000 7fc4122fe640 1 -- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8003b70 msgr2=0x7fc3f8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.351+0000 7fc4122fe640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc3f8003b70 0x7fc3f8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.351+0000 7fc4122fe640 1 -- 172.21.15.26:0/116537893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f8003520 con 0x7fc3f8005180 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.351+0000 7fc4122fe640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f8005180 0x7fc3f800b9c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc404001fd0 tx=0x7fc3f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28954de8a2072ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.351+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404001bf0 con 0x7fc3f8005180 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.351+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc404003b70 con 0x7fc3f8005180 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.351+0000 7fc410afb640 1 -- 172.21.15.26:0/116537893 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4040050d0 con 0x7fc3f8005180 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.352+0000 7fc413d61640 1 -- 172.21.15.26:0/116537893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f8005180 msgr2=0x7fc3f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.352+0000 7fc413d61640 1 --2- 172.21.15.26:0/116537893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f8005180 0x7fc3f800b9c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc404001fd0 tx=0x7fc3f8005830 comp rx=0 tx=0).stop 2024-04-24T23:26:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.352+0000 7fc4122fe640 1 -- 172.21.15.26:0/116537893 reap_dead start 2024-04-24T23:26:56.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.352+0000 7fc413d61640 1 -- 172.21.15.26:0/116537893 shutdown_connections 2024-04-24T23:26:56.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.352+0000 7fc413d61640 1 -- 172.21.15.26:0/116537893 >> 172.21.15.26:0/116537893 conn(0x7fc40c0d40f0 msgr2=0x7fc40c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.352+0000 7fc413d61640 1 -- 172.21.15.26:0/116537893 shutdown_connections 2024-04-24T23:26:56.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.352+0000 7fc413d61640 1 -- 172.21.15.26:0/116537893 wait complete. 2024-04-24T23:26:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.353+0000 7fc413d61640 1 Processor -- start 2024-04-24T23:26:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.353+0000 7fc413d61640 1 -- start start 2024-04-24T23:26:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc413d61640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc40c14b3e0 0x7fc40c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc413d61640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc40c156390 0x7fc40c159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc413d61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c15a150 0x7fc40c1e4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc413d61640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c0b5230 con 0x7fc40c15a150 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc413d61640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c0d1ed0 con 0x7fc40c156390 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc413d61640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c0b53b0 con 0x7fc40c14b3e0 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc4122fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c15a150 0x7fc40c1e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc4122fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c15a150 0x7fc40c1e4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52590/0 (socket says 172.21.15.26:52590) 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc4122fe640 1 -- 172.21.15.26:0/3504998453 learned_addr learned my addr 172.21.15.26:0/3504998453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc40c156390 0x7fc40c159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc411afd640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc40c14b3e0 0x7fc40c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc4122fe640 1 -- 172.21.15.26:0/3504998453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc40c14b3e0 msgr2=0x7fc40c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc4122fe640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc40c14b3e0 0x7fc40c15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc4122fe640 1 -- 172.21.15.26:0/3504998453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc40c156390 msgr2=0x7fc40c159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc4122fe640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc40c156390 0x7fc40c159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.354+0000 7fc4122fe640 1 -- 172.21.15.26:0/3504998453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c15d860 con 0x7fc40c15a150 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.355+0000 7fc4122fe640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c15a150 0x7fc40c1e4070 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc4040020f0 tx=0x7fc404001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.355+0000 7fc3fe7fc640 1 -- 172.21.15.26:0/3504998453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404001bf0 con 0x7fc40c15a150 2024-04-24T23:26:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.355+0000 7fc3fe7fc640 1 -- 172.21.15.26:0/3504998453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc404003970 con 0x7fc40c15a150 2024-04-24T23:26:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.355+0000 7fc3fe7fc640 1 -- 172.21.15.26:0/3504998453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404005610 con 0x7fc40c15a150 2024-04-24T23:26:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.355+0000 7fc413d61640 1 -- 172.21.15.26:0/3504998453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c158b70 con 0x7fc40c15a150 2024-04-24T23:26:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.355+0000 7fc413d61640 1 -- 172.21.15.26:0/3504998453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c159130 con 0x7fc40c15a150 2024-04-24T23:26:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.356+0000 7fc3fe7fc640 1 -- 172.21.15.26:0/3504998453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc404045020 con 0x7fc40c15a150 2024-04-24T23:26:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.357+0000 7fc3fe7fc640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc3f006be30 0x7fc3f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.357+0000 7fc3fe7fc640 1 -- 172.21.15.26:0/3504998453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fc4040aadf0 con 0x7fc40c15a150 2024-04-24T23:26:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.357+0000 7fc411afd640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc3f006be30 0x7fc3f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.357+0000 7fc411afd640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc3f006be30 0x7fc3f006e2f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc408001eb0 tx=0x7fc408001e20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.358+0000 7fc416a6d740 1 Processor -- start 2024-04-24T23:26:56.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.358+0000 7fc416a6d740 1 -- start start 2024-04-24T23:26:56.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827579d60 0x55782757a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827597b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55782754fa40 0x557827595cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc416a6d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5578274b36b0 con 0x557827597600 2024-04-24T23:26:56.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc416a6d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55782745b880 con 0x55782754fa40 2024-04-24T23:26:56.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc416a6d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5578274af670 con 0x557827579d60 2024-04-24T23:26:56.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc4112fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827597b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc4112fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827597b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52600/0 (socket says 172.21.15.26:52600) 2024-04-24T23:26:56.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc4112fc640 1 -- 172.21.15.26:0/3508878451 learned_addr learned my addr 172.21.15.26:0/3508878451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc4112fc640 1 -- 172.21.15.26:0/3508878451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827579d60 msgr2=0x55782757a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3508878451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827579d60 0x55782757a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc4112fc640 1 -- 172.21.15.26:0/3508878451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55782754fa40 msgr2=0x557827595cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3508878451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55782754fa40 0x557827595cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc4112fc640 1 -- 172.21.15.26:0/3508878451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578272917d0 con 0x557827597600 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3508878451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827597b50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc400009c90 tx=0x7fc400014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87de0e2be7afba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc3ef7fe640 1 -- 172.21.15.26:0/3508878451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400022070 con 0x557827597600 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc3ef7fe640 1 -- 172.21.15.26:0/3508878451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40001d440 con 0x557827597600 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.359+0000 7fc3ef7fe640 1 -- 172.21.15.26:0/3508878451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40001d830 con 0x557827597600 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.360+0000 7fc416a6d740 1 -- 172.21.15.26:0/3508878451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 msgr2=0x557827597b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.360+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3508878451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827597b50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc400009c90 tx=0x7fc400014780 comp rx=0 tx=0).stop 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.360+0000 7fc416a6d740 1 -- 172.21.15.26:0/3508878451 shutdown_connections 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.360+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3508878451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55782754fa40 0x557827595cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.360+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3508878451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827597b50 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.360+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3508878451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827579d60 0x55782757a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.360+0000 7fc416a6d740 1 -- 172.21.15.26:0/3508878451 >> 172.21.15.26:0/3508878451 conn(0x55782754c600 msgr2=0x557827594b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.360+0000 7fc416a6d740 1 -- 172.21.15.26:0/3508878451 shutdown_connections 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.360+0000 7fc416a6d740 1 -- 172.21.15.26:0/3508878451 wait complete. 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.361+0000 7fc416a6d740 1 -- start start 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.361+0000 7fc416a6d740 1 Processor -- start 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.361+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827579d60 0x55782758d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.361+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 0x55782757c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.361+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827578750 0x557827575b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.361+0000 7fc416a6d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5578275507e0 con 0x557827579d60 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc416a6d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55782746ef50 con 0x557827578750 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc416a6d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55782754a530 con 0x557827597600 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc416a6d740 10 client.0 _ll_register_callbacks cb 0x5578274aa000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:56.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc411afd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827579d60 0x55782758d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc411afd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827579d60 0x55782758d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52616/0 (socket says 172.21.15.26:52616) 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc411afd640 1 -- 172.21.15.26:0/3204525973 learned_addr learned my addr 172.21.15.26:0/3204525973 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 0x55782757c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc4122fe640 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827578750 0x557827575b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc411afd640 1 -- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 msgr2=0x55782757c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc411afd640 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 0x55782757c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc411afd640 1 -- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827578750 msgr2=0x557827575b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc411afd640 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827578750 0x557827575b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc411afd640 1 -- 172.21.15.26:0/3204525973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578271c73f0 con 0x557827579d60 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc411afd640 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827579d60 0x55782758d2c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc4080159f0 tx=0x7fc40c0b3bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f123938ce8cd973e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc3ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3204525973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408028060 con 0x557827579d60 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3204525973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fc408023b10 con 0x557827579d60 2024-04-24T23:26:56.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc416a6d740 1 -- 172.21.15.26:0/3204525973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc40001d440 con 0x557827579d60 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3204525973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fc408007e40 con 0x557827579d60 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc416a6d740 10 client.8370 fetch_fsmap learned FSMap version 94 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.362+0000 7fc416a6d740 1 -- 172.21.15.26:0/3204525973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x5578271870f0 con 0x557827579d60 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3204525973 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40001d440 con 0x557827579d60 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3204525973 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) ==== 3550+0+0 (secure 0 0 0) 0x7fc4080145b0 con 0x557827579d60 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc416a6d740 10 client.8370 fetch_fsmap finished waiting for FSMap version 94 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc416a6d740 10 client.8370 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc416a6d740 20 client.8370 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x5578275a5b10 0x5578275a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc416a6d740 4 client.8370 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:56.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc416a6d740 1 -- 172.21.15.26:0/3204525973 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578274c0bf0 con 0x5578275a5b10 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x5578275a5b10 0x5578275a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x5578275a5b10 0x5578275a5f20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad790ae1959148d0 server_cookie=dba3629f18b59264 in_seq=0 out_seq=0 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.363+0000 7fc3ecff9640 10 client.8370 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3204525973 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578274c0bf0 con 0x5578275a5b10 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc3ecff9640 10 client.8370 handle_command_reply: tid=0 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 client.8370 shutdown 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 -- 172.21.15.26:0/3204525973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827579d60 msgr2=0x55782758d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827579d60 0x55782758d2c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc4080159f0 tx=0x7fc40c0b3bf0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 -- 172.21.15.26:0/3204525973 shutdown_connections 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x5578275a5b10 0x5578275a5f20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827578750 0x557827575b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 0x55782757c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc3ecff9640 0 client.8370 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:56.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3204525973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827579d60 0x55782758d2c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 -- 172.21.15.26:0/3204525973 >> 172.21.15.26:0/3204525973 conn(0x55782754fa40 msgr2=0x5578275980f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.364+0000 7fc416a6d740 1 -- 172.21.15.26:0/3204525973 shutdown_connections 2024-04-24T23:26:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.365+0000 7fc416a6d740 1 -- 172.21.15.26:0/3204525973 wait complete. 2024-04-24T23:26:56.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.365+0000 7fc416a6d740 20 client.8370 trim_cache size 0 max 16384 2024-04-24T23:26:56.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.382+0000 7fc416a6d740 1 Processor -- start 2024-04-24T23:26:56.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.382+0000 7fc416a6d740 1 -- start start 2024-04-24T23:26:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827578750 0x5578275af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x557827577df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827576450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc416a6d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c2010b0 con 0x557827597600 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc416a6d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c200250 con 0x557827579d60 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc416a6d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c200980 con 0x557827578750 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4122fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827576450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4122fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827576450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52630/0 (socket says 172.21.15.26:52630) 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4122fe640 1 -- 172.21.15.26:0/3720607426 learned_addr learned my addr 172.21.15.26:0/3720607426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc411afd640 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827578750 0x5578275af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x557827577df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4122fe640 1 -- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827578750 msgr2=0x5578275af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4122fe640 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827578750 0x5578275af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4122fe640 1 -- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 msgr2=0x557827577df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4122fe640 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x557827577df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4122fe640 1 -- 172.21.15.26:0/3720607426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55782754ab20 con 0x557827597600 2024-04-24T23:26:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc411afd640 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827578750 0x5578275af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x557827577df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc4122fe640 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827576450 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fc4040756a0 tx=0x7fc404078e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd03f870ab4b7955 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc3eeffd640 1 -- 172.21.15.26:0/3720607426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404079240 con 0x557827597600 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc3eeffd640 1 -- 172.21.15.26:0/3720607426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4040a8070 con 0x557827597600 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.383+0000 7fc3eeffd640 1 -- 172.21.15.26:0/3720607426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40407d5f0 con 0x557827597600 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 -- 172.21.15.26:0/3720607426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 msgr2=0x557827576450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827576450 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fc4040756a0 tx=0x7fc404078e40 comp rx=0 tx=0).stop 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 -- 172.21.15.26:0/3720607426 shutdown_connections 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827597600 0x557827576450 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x557827577df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3720607426 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827578750 0x5578275af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 -- 172.21.15.26:0/3720607426 >> 172.21.15.26:0/3720607426 conn(0x55782754fa40 msgr2=0x557827573810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 -- 172.21.15.26:0/3720607426 shutdown_connections 2024-04-24T23:26:56.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 -- 172.21.15.26:0/3720607426 wait complete. 2024-04-24T23:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.384+0000 7fc416a6d740 1 -- start start 2024-04-24T23:26:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.385+0000 7fc416a6d740 1 Processor -- start 2024-04-24T23:26:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.385+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827578750 0x55782757ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.385+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x55782757b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.385+0000 7fc416a6d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 0x5578275adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.385+0000 7fc416a6d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55782758d9a0 con 0x557827578750 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.385+0000 7fc416a6d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c157a50 con 0x557827579d60 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.385+0000 7fc416a6d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55782758afa0 con 0x557827597600 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.385+0000 7fc416a6d740 10 client.0 _ll_register_callbacks cb 0x7fc408007800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc411afd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827578750 0x55782757ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc411afd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827578750 0x55782757ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52634/0 (socket says 172.21.15.26:52634) 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc411afd640 1 -- 172.21.15.26:0/3328349294 learned_addr learned my addr 172.21.15.26:0/3328349294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x55782757b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc4122fe640 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 0x5578275adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc411afd640 1 -- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 msgr2=0x5578275adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc411afd640 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 0x5578275adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc411afd640 1 -- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 msgr2=0x55782757b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc411afd640 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x55782757b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc411afd640 1 -- 172.21.15.26:0/3328349294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578275af840 con 0x557827578750 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x55782757b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc411afd640 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827578750 0x55782757ab60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc4080018f0 tx=0x7fc408006030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3c7578ad98473d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc3ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3328349294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40802e640 con 0x557827578750 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3328349294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fc408033040 con 0x557827578750 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc416a6d740 1 -- 172.21.15.26:0/3328349294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4040a8070 con 0x557827578750 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3328349294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fc408042050 con 0x557827578750 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc416a6d740 10 client.8376 fetch_fsmap learned FSMap version 94 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc416a6d740 1 -- 172.21.15.26:0/3328349294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x5578275986a0 con 0x557827578750 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.386+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3328349294 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4040a8070 con 0x557827578750 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3328349294 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) ==== 3550+0+0 (secure 0 0 0) 0x7fc408045020 con 0x557827578750 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc416a6d740 10 client.8376 fetch_fsmap finished waiting for FSMap version 94 2024-04-24T23:26:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc416a6d740 10 client.8376 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc416a6d740 20 client.8376 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55782757d8f0 0x5578275ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc416a6d740 4 client.8376 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc416a6d740 1 -- 172.21.15.26:0/3328349294 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x5578274baf40 con 0x55782757d8f0 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55782757d8f0 0x5578275ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc4112fc640 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55782757d8f0 0x5578275ae520 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0ea4035a29f46a server_cookie=8d5259977da8c8cc in_seq=0 out_seq=0 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.387+0000 7fc3ecff9640 10 client.8376 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc3ecff9640 1 -- 172.21.15.26:0/3328349294 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5578274baf40 con 0x55782757d8f0 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc3ecff9640 10 client.8376 handle_command_reply: tid=0 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 client.8376 shutdown 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 -- 172.21.15.26:0/3328349294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827578750 msgr2=0x55782757ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827578750 0x55782757ab60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc4080018f0 tx=0x7fc408006030 comp rx=0 tx=0).stop 2024-04-24T23:26:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 -- 172.21.15.26:0/3328349294 shutdown_connections 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55782757d8f0 0x5578275ae520 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557827597600 0x5578275adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557827579d60 0x55782757b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc3ecff9640 0 client.8376 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 --2- 172.21.15.26:0/3328349294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557827578750 0x55782757ab60 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 -- 172.21.15.26:0/3328349294 >> 172.21.15.26:0/3328349294 conn(0x55782754fa40 msgr2=0x557827593e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 -- 172.21.15.26:0/3328349294 shutdown_connections 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 1 -- 172.21.15.26:0/3328349294 wait complete. 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc416a6d740 20 client.8376 trim_cache size 0 max 16384 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:56.395 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:26:56.396 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 -- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc3f006be30 msgr2=0x7fc3f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc3f006be30 0x7fc3f006e2f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc408001eb0 tx=0x7fc408001e20 comp rx=0 tx=0).stop 2024-04-24T23:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 -- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c15a150 msgr2=0x7fc40c1e4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c15a150 0x7fc40c1e4070 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc4040020f0 tx=0x7fc404001650 comp rx=0 tx=0).stop 2024-04-24T23:26:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 -- 172.21.15.26:0/3504998453 shutdown_connections 2024-04-24T23:26:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc3f006be30 0x7fc3f006e2f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc40c15a150 0x7fc40c1e4070 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc40c156390 0x7fc40c159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 --2- 172.21.15.26:0/3504998453 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc40c14b3e0 0x7fc40c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.388+0000 7fc413d61640 1 -- 172.21.15.26:0/3504998453 >> 172.21.15.26:0/3504998453 conn(0x7fc40c0d40f0 msgr2=0x7fc40c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.389+0000 7fc413d61640 1 -- 172.21.15.26:0/3504998453 shutdown_connections 2024-04-24T23:26:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.389+0000 7fc413d61640 1 -- 172.21.15.26:0/3504998453 wait complete. 2024-04-24T23:26:56.400 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:26:56.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:1 scrub status 2024-04-24T23:26:56.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.502+0000 7f7fcf522640 1 Processor -- start 2024-04-24T23:26:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.502+0000 7f7fcf522640 1 -- start start 2024-04-24T23:26:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fcf522640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 0x7f7fc80acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fcf522640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7fc80b4ce0 0x7f7fc815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fcf522640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fc80aeb10 con 0x7f7fc815be40 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fcf522640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7fc809df20 con 0x7f7fc80b4ce0 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fcf522640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc80aedf0 con 0x7f7fc80d0750 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fccabd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 0x7f7fc80acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fcdabf640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7fc80b4ce0 0x7f7fc815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fcd2be640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f7fc815be40 0x7f7fc80d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:42086/0 (socket says 172.21.15.26:42086) 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fcd2be640 1 -- 172.21.15.26:0/93549187 learned_addr learned my addr 172.21.15.26:0/93549187 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fccabd640 1 -- 172.21.15.26:0/93549187 >> v2:172.21.15.29:3300/0 conn(0x7f7fc80b4ce0 msgr2=0x7f7fc815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fccabd640 1 --2- 172.21.15.26:0/93549187 >> v2:172.21.15.29:3300/0 conn(0x7f7fc80b4ce0 0x7f7fc815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fccabd640 1 -- 172.21.15.26:0/93549187 >> v1:172.21.15.29:6789/0 conn(0x7f7fc815be40 legacy=0x7f7fc80d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.503+0000 7f7fccabd640 1 -- 172.21.15.26:0/93549187 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc80af0d0 con 0x7f7fc80d0750 2024-04-24T23:26:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.504+0000 7f7fccabd640 1 --2- 172.21.15.26:0/93549187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 0x7f7fc80acd30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80097a0 tx=0x7f7fb802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ef724394b2fceb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.504+0000 7f7fbffff640 1 -- 172.21.15.26:0/93549187 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8037070 con 0x7f7fc80d0750 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.504+0000 7f7fbffff640 1 -- 172.21.15.26:0/93549187 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb8032440 con 0x7f7fc80d0750 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.504+0000 7f7fbffff640 1 -- 172.21.15.26:0/93549187 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8032810 con 0x7f7fc80d0750 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.505+0000 7f7fcf522640 1 -- 172.21.15.26:0/93549187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 msgr2=0x7f7fc80acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.505+0000 7f7fcf522640 1 --2- 172.21.15.26:0/93549187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 0x7f7fc80acd30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80097a0 tx=0x7f7fb802a180 comp rx=0 tx=0).stop 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.505+0000 7f7fcf522640 1 -- 172.21.15.26:0/93549187 shutdown_connections 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.505+0000 7f7fcf522640 1 --2- 172.21.15.26:0/93549187 >> v2:172.21.15.29:3300/0 conn(0x7f7fc80b4ce0 0x7f7fc815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.505+0000 7f7fcf522640 1 --2- 172.21.15.26:0/93549187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 0x7f7fc80acd30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.505+0000 7f7fcf522640 1 -- 172.21.15.26:0/93549187 >> 172.21.15.26:0/93549187 conn(0x7f7fc806b540 msgr2=0x7f7fc806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.505+0000 7f7fcf522640 1 -- 172.21.15.26:0/93549187 shutdown_connections 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.505+0000 7f7fcf522640 1 -- 172.21.15.26:0/93549187 wait complete. 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.506+0000 7f7fcf522640 1 Processor -- start 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.506+0000 7f7fcf522640 1 -- start start 2024-04-24T23:26:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcf522640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fc80b4ce0 0x7f7fc8158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcf522640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 0x7f7fc815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcf522640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc815be40 0x7f7fc815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcf522640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc806cd10 con 0x7f7fc815be40 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcf522640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc806cb90 con 0x7f7fc80b4ce0 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcf522640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc815c760 con 0x7f7fc80d0750 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcdabf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc815be40 0x7f7fc815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcdabf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc815be40 0x7f7fc815acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52636/0 (socket says 172.21.15.26:52636) 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcdabf640 1 -- 172.21.15.26:0/3701276099 learned_addr learned my addr 172.21.15.26:0/3701276099 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fc80b4ce0 0x7f7fc8158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fccabd640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 0x7f7fc815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcdabf640 1 -- 172.21.15.26:0/3701276099 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 msgr2=0x7f7fc815a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 0x7f7fc815a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcdabf640 1 -- 172.21.15.26:0/3701276099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fc80b4ce0 msgr2=0x7f7fc8158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fc80b4ce0 0x7f7fc8158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcdabf640 1 -- 172.21.15.26:0/3701276099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc8071c40 con 0x7f7fc815be40 2024-04-24T23:26:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.507+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fc80b4ce0 0x7f7fc8158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.508+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc815be40 0x7f7fc815acb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009dd0 tx=0x7f7fc000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.508+0000 7f7fbdffb640 1 -- 172.21.15.26:0/3701276099 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0018020 con 0x7f7fc815be40 2024-04-24T23:26:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.508+0000 7f7fbdffb640 1 -- 172.21.15.26:0/3701276099 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc0014480 con 0x7f7fc815be40 2024-04-24T23:26:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.508+0000 7f7fbdffb640 1 -- 172.21.15.26:0/3701276099 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0014870 con 0x7f7fc815be40 2024-04-24T23:26:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.508+0000 7f7fcf522640 1 -- 172.21.15.26:0/3701276099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc8071f20 con 0x7f7fc815be40 2024-04-24T23:26:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.508+0000 7f7fcf522640 1 -- 172.21.15.26:0/3701276099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc80724e0 con 0x7f7fc815be40 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.509+0000 7f7fbdffb640 1 -- 172.21.15.26:0/3701276099 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fc001b020 con 0x7f7fc815be40 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.510+0000 7f7fbdffb640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fac06be30 0x7f7fac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.510+0000 7f7fbdffb640 1 -- 172.21.15.26:0/3701276099 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f7fc008bcc0 con 0x7f7fc815be40 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.510+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fac06be30 0x7f7fac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.510+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fac06be30 0x7f7fac06e2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000f60 tx=0x7f7fc4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.511+0000 7f7fd2232740 1 Processor -- start 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.511+0000 7f7fd2232740 1 -- start start 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.511+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 0x556642c38140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.511+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c55600 0x556642c55b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.511+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c0da40 0x556642c53cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.511+0000 7f7fd2232740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556642b716b0 con 0x556642c55600 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.511+0000 7f7fd2232740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556642b19880 con 0x556642c37d60 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.511+0000 7f7fd2232740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556642b6d670 con 0x556642c0da40 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fccabd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c55600 0x556642c55b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fccabd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c55600 0x556642c55b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52638/0 (socket says 172.21.15.26:52638) 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fccabd640 1 -- 172.21.15.26:0/3625139834 learned_addr learned my addr 172.21.15.26:0/3625139834 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/3625139834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 0x556642c38140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/3625139834 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c0da40 0x556642c53cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fccabd640 1 -- 172.21.15.26:0/3625139834 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c0da40 msgr2=0x556642c53cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fccabd640 1 --2- 172.21.15.26:0/3625139834 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c0da40 0x556642c53cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fccabd640 1 -- 172.21.15.26:0/3625139834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 msgr2=0x556642c38140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fccabd640 1 --2- 172.21.15.26:0/3625139834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 0x556642c38140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fccabd640 1 -- 172.21.15.26:0/3625139834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664294f7d0 con 0x556642c55600 2024-04-24T23:26:56.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7fccabd640 1 --2- 172.21.15.26:0/3625139834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c55600 0x556642c55b50 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80097a0 tx=0x7f7fb802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571d2673191b5632 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7faaffd640 1 -- 172.21.15.26:0/3625139834 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8043070 con 0x556642c55600 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7faaffd640 1 -- 172.21.15.26:0/3625139834 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb80327a0 con 0x556642c55600 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.512+0000 7f7faaffd640 1 -- 172.21.15.26:0/3625139834 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8037040 con 0x556642c55600 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.513+0000 7f7fd2232740 1 -- 172.21.15.26:0/3625139834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c55600 msgr2=0x556642c55b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.513+0000 7f7fd2232740 1 --2- 172.21.15.26:0/3625139834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c55600 0x556642c55b50 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80097a0 tx=0x7f7fb802c000 comp rx=0 tx=0).stop 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.513+0000 7f7fd2232740 1 -- 172.21.15.26:0/3625139834 shutdown_connections 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.513+0000 7f7fd2232740 1 --2- 172.21.15.26:0/3625139834 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c0da40 0x556642c53cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.513+0000 7f7fd2232740 1 --2- 172.21.15.26:0/3625139834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c55600 0x556642c55b50 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.513+0000 7f7fd2232740 1 --2- 172.21.15.26:0/3625139834 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 0x556642c38140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.513+0000 7f7fd2232740 1 -- 172.21.15.26:0/3625139834 >> 172.21.15.26:0/3625139834 conn(0x556642c0a600 msgr2=0x556642c52b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.513+0000 7f7fd2232740 1 -- 172.21.15.26:0/3625139834 shutdown_connections 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.513+0000 7f7fd2232740 1 -- 172.21.15.26:0/3625139834 wait complete. 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.514+0000 7f7fd2232740 1 -- start start 2024-04-24T23:26:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.514+0000 7f7fd2232740 1 Processor -- start 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.514+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 0x556642c34250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.514+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 0x556642c07260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.514+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.514+0000 7f7fd2232740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556642c0e7e0 con 0x556642c66c60 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.514+0000 7f7fd2232740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556642b2cf50 con 0x556642c37d60 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fd2232740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556642c08530 con 0x556642c55600 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fd2232740 10 client.0 _ll_register_callbacks cb 0x556642b68000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcdabf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcdabf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52640/0 (socket says 172.21.15.26:52640) 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcdabf640 1 -- 172.21.15.26:0/2109649319 learned_addr learned my addr 172.21.15.26:0/2109649319 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 0x556642c34250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fccabd640 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 0x556642c07260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcdabf640 1 -- 172.21.15.26:0/2109649319 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 msgr2=0x556642c07260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 0x556642c07260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcdabf640 1 -- 172.21.15.26:0/2109649319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 msgr2=0x556642c34250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 0x556642c34250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcdabf640 1 -- 172.21.15.26:0/2109649319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566428853f0 con 0x556642c66c60 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 0x556642c34250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c077a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000c990 tx=0x7f7fc0059ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f156292859fb68e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2109649319 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0062880 con 0x556642c66c60 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2109649319 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f7fc0059380 con 0x556642c66c60 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fd2232740 1 -- 172.21.15.26:0/2109649319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb80327a0 con 0x556642c66c60 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2109649319 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f7fc0062b10 con 0x556642c66c60 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fd2232740 10 client.8385 fetch_fsmap learned FSMap version 94 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.515+0000 7f7fd2232740 1 -- 172.21.15.26:0/2109649319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x5566428450f0 con 0x556642c66c60 2024-04-24T23:26:56.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2109649319 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb80327a0 con 0x556642c66c60 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2109649319 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) ==== 3550+0+0 (secure 0 0 0) 0x7f7fc0083b10 con 0x556642c66c60 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fd2232740 10 client.8385 fetch_fsmap finished waiting for FSMap version 94 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fd2232740 10 client.8385 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fd2232740 20 client.8385 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x556642c3a4b0 0x556642c3a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fd2232740 4 client.8385 mds_command: new command op to 6778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fd2232740 1 -- 172.21.15.26:0/2109649319 --> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556642b7ebf0 con 0x556642c3a4b0 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x556642c3a4b0 0x556642c3a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x556642c3a4b0 0x556642c3a8c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48d5e78b0352607d server_cookie=b8f5de8847f4f7df in_seq=0 out_seq=0 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.516+0000 7f7fa3fff640 10 client.8385 ms_handle_connect on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2109649319 <== mds.1 v2:172.21.15.26:6834/306957409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556642b7ebf0 con 0x556642c3a4b0 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fa3fff640 10 client.8385 handle_command_reply: tid=0 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 client.8385 shutdown 2024-04-24T23:26:56.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 -- 172.21.15.26:0/2109649319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 msgr2=0x556642c077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c077a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000c990 tx=0x7f7fc0059ff0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 -- 172.21.15.26:0/2109649319 shutdown_connections 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x556642c3a4b0 0x556642c3a8c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c077a0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 0x556642c07260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fa3fff640 0 client.8385 ms_handle_reset on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2109649319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c37d60 0x556642c34250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 -- 172.21.15.26:0/2109649319 >> 172.21.15.26:0/2109649319 conn(0x556642c0da40 msgr2=0x556642c52410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 -- 172.21.15.26:0/2109649319 shutdown_connections 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 1 -- 172.21.15.26:0/2109649319 wait complete. 2024-04-24T23:26:56.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.517+0000 7f7fd2232740 20 client.8385 trim_cache size 0 max 16384 2024-04-24T23:26:56.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.536+0000 7f7fd2232740 1 Processor -- start 2024-04-24T23:26:56.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.536+0000 7f7fd2232740 1 -- start start 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c37d60 0x556642c0cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c55600 0x556642c4fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fd2232740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8200a60 con 0x556642c66c60 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fd2232740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81ffc00 con 0x556642c55600 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fd2232740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc8200330 con 0x556642c37d60 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fcdabf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fcdabf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52646/0 (socket says 172.21.15.26:52646) 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fcdabf640 1 -- 172.21.15.26:0/1815642056 learned_addr learned my addr 172.21.15.26:0/1815642056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fccabd640 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c55600 0x556642c4fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.537+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c37d60 0x556642c0cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fcdabf640 1 -- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c37d60 msgr2=0x556642c0cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c37d60 0x556642c0cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fcdabf640 1 -- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c55600 msgr2=0x556642c4fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c55600 0x556642c4fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fcdabf640 1 -- 172.21.15.26:0/1815642056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556642c042a0 con 0x556642c66c60 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fccabd640 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c55600 0x556642c4fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c37d60 0x556642c0cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c503c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0059c30 tx=0x7f7fc008b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68eb3449afc06d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7faa7fc640 1 -- 172.21.15.26:0/1815642056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0080070 con 0x556642c66c60 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7faa7fc640 1 -- 172.21.15.26:0/1815642056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc0062680 con 0x556642c66c60 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7faa7fc640 1 -- 172.21.15.26:0/1815642056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0055c80 con 0x556642c66c60 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fd2232740 1 -- 172.21.15.26:0/1815642056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 msgr2=0x556642c503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fd2232740 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c503c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0059c30 tx=0x7f7fc008b000 comp rx=0 tx=0).stop 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fd2232740 1 -- 172.21.15.26:0/1815642056 shutdown_connections 2024-04-24T23:26:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.538+0000 7f7fd2232740 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c66c60 0x556642c503c0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.539+0000 7f7fd2232740 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c55600 0x556642c4fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.539+0000 7f7fd2232740 1 --2- 172.21.15.26:0/1815642056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c37d60 0x556642c0cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.539+0000 7f7fd2232740 1 -- 172.21.15.26:0/1815642056 >> 172.21.15.26:0/1815642056 conn(0x556642c0da40 msgr2=0x556642c07260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.539+0000 7f7fd2232740 1 -- 172.21.15.26:0/1815642056 shutdown_connections 2024-04-24T23:26:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.539+0000 7f7fd2232740 1 -- 172.21.15.26:0/1815642056 wait complete. 2024-04-24T23:26:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.539+0000 7f7fd2232740 1 -- start start 2024-04-24T23:26:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.539+0000 7f7fd2232740 1 Processor -- start 2024-04-24T23:26:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c37d60 0x556642c66040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 0x556642c66580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fd2232740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c66c60 0x556642c52110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fd2232740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81f5920 con 0x556642c37d60 2024-04-24T23:26:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fd2232740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556642c02f00 con 0x556642c66c60 2024-04-24T23:26:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fd2232740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556642c390a0 con 0x556642c55600 2024-04-24T23:26:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fd2232740 10 client.0 _ll_register_callbacks cb 0x7f7fc000cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fcd2be640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c37d60 0x556642c66040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fcd2be640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c37d60 0x556642c66040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52656/0 (socket says 172.21.15.26:52656) 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fcd2be640 1 -- 172.21.15.26:0/2440095598 learned_addr learned my addr 172.21.15.26:0/2440095598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fcdabf640 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c66c60 0x556642c52110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.540+0000 7f7fccabd640 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 0x556642c66580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fcd2be640 1 -- 172.21.15.26:0/2440095598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 msgr2=0x556642c66580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 0x556642c66580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fcd2be640 1 -- 172.21.15.26:0/2440095598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c66c60 msgr2=0x556642c52110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c66c60 0x556642c52110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fcd2be640 1 -- 172.21.15.26:0/2440095598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556642c54280 con 0x556642c37d60 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fcd2be640 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c37d60 0x556642c66040 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4007ab0 tx=0x7f7fc4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778ba9508604e5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2440095598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4007720 con 0x556642c37d60 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2440095598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f7fc4010040 con 0x556642c37d60 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fd2232740 1 -- 172.21.15.26:0/2440095598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc0062680 con 0x556642c37d60 2024-04-24T23:26:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2440095598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f7fc4026280 con 0x556642c37d60 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fd2232740 10 client.8391 fetch_fsmap learned FSMap version 94 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fd2232740 1 -- 172.21.15.26:0/2440095598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x556642c54800 con 0x556642c37d60 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2440095598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc0062680 con 0x556642c37d60 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2440095598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) ==== 3550+0+0 (secure 0 0 0) 0x7f7fc4026420 con 0x556642c37d60 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fd2232740 10 client.8391 fetch_fsmap finished waiting for FSMap version 94 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fd2232740 10 client.8391 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fd2232740 20 client.8391 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x556642c34770 0x556642c52650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fd2232740 4 client.8391 mds_command: new command op to 6778 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.541+0000 7f7fd2232740 1 -- 172.21.15.26:0/2440095598 --> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556642b78f40 con 0x556642c34770 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fccabd640 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x556642c34770 0x556642c52650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fa3fff640 10 client.8391 ms_handle_connect on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fccabd640 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x556642c34770 0x556642c52650 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e756b80e3f441d7a server_cookie=bc6068594946c4ca in_seq=0 out_seq=0 2024-04-24T23:26:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fa3fff640 1 -- 172.21.15.26:0/2440095598 <== mds.1 v2:172.21.15.26:6834/306957409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x556642b78f40 con 0x556642c34770 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fa3fff640 10 client.8391 handle_command_reply: tid=0 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 client.8391 shutdown 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 -- 172.21.15.26:0/2440095598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c37d60 msgr2=0x556642c66040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c37d60 0x556642c66040 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4007ab0 tx=0x7f7fc4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 -- 172.21.15.26:0/2440095598 shutdown_connections 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.26:6834/306957409,v1:172.21.15.26:6836/306957409] conn(0x556642c34770 0x556642c52650 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556642c66c60 0x556642c52110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556642c55600 0x556642c66580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fa3fff640 0 client.8391 ms_handle_reset on v2:172.21.15.26:6834/306957409 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 --2- 172.21.15.26:0/2440095598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556642c37d60 0x556642c66040 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 -- 172.21.15.26:0/2440095598 >> 172.21.15.26:0/2440095598 conn(0x556642c0da40 msgr2=0x556642c34060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 -- 172.21.15.26:0/2440095598 shutdown_connections 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.542+0000 7f7fd2232740 1 -- 172.21.15.26:0/2440095598 wait complete. 2024-04-24T23:26:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fd2232740 20 client.8391 trim_cache size 0 max 16384 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "cca3b3ba-5fb5-4b95-8bce-070d29d5babd": { 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "cca3b3ba-5fb5-4b95-8bce-070d29d5babd", 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:56.550 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 -- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fac06be30 msgr2=0x7f7fac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fac06be30 0x7f7fac06e2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000f60 tx=0x7f7fc4009040 comp rx=0 tx=0).stop 2024-04-24T23:26:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 -- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc815be40 msgr2=0x7f7fc815acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc815be40 0x7f7fc815acb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009dd0 tx=0x7f7fc000c490 comp rx=0 tx=0).stop 2024-04-24T23:26:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 -- 172.21.15.26:0/3701276099 shutdown_connections 2024-04-24T23:26:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fac06be30 0x7f7fac06e2f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fc815be40 0x7f7fc815acb0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fc80d0750 0x7f7fc815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 --2- 172.21.15.26:0/3701276099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fc80b4ce0 0x7f7fc8158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 -- 172.21.15.26:0/3701276099 >> 172.21.15.26:0/3701276099 conn(0x7f7fc806b540 msgr2=0x7f7fc806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 -- 172.21.15.26:0/3701276099 shutdown_connections 2024-04-24T23:26:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.543+0000 7f7fcf522640 1 -- 172.21.15.26:0/3701276099 wait complete. 2024-04-24T23:26:56.555 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:26:56.555 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:26:56.555 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:2 scrub status 2024-04-24T23:26:56.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.657+0000 7fa7ffa95640 1 Processor -- start 2024-04-24T23:26:56.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.657+0000 7fa7ffa95640 1 -- start start 2024-04-24T23:26:56.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.658+0000 7fa7ffa95640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7f80d0750 0x7fa7f8165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.658+0000 7fa7ffa95640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f815ef60 con 0x7fa7f815adc0 2024-04-24T23:26:56.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.658+0000 7fa7ffa95640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f815f240 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.658+0000 7fa7ffa95640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa7f815f520 con 0x7fa7f80d0750 2024-04-24T23:26:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.658+0000 7fa7fe032640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7f80d0750 0x7fa7f8165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.658+0000 7fa7fd831640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa7f815adc0 0x7fa7f80b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40126/0 (socket says 172.21.15.26:40126) 2024-04-24T23:26:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.658+0000 7fa7fd831640 1 -- 172.21.15.26:0/2537174675 learned_addr learned my addr 172.21.15.26:0/2537174675 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.659+0000 7fa7fe032640 1 -- 172.21.15.26:0/2537174675 >> v1:172.21.15.29:6789/0 conn(0x7fa7f80ad7b0 legacy=0x7fa7f80d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.659+0000 7fa7fe032640 1 -- 172.21.15.26:0/2537174675 >> v1:172.21.15.26:6789/0 conn(0x7fa7f815adc0 legacy=0x7fa7f80b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.659+0000 7fa7fe032640 1 -- 172.21.15.26:0/2537174675 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f815f800 con 0x7fa7f80d0750 2024-04-24T23:26:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.659+0000 7fa7fc82f640 1 -- 172.21.15.26:0/2537174675 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024228509 0 0) 0x7fa7f815ef60 con 0x7fa7f815adc0 2024-04-24T23:26:56.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.659+0000 7fa7fe032640 1 --2- 172.21.15.26:0/2537174675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80d0750 0x7fa7f8165d50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00097d0 tx=0x7fa7f002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fddd12b382a289a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.659+0000 7fa7fc82f640 1 -- 172.21.15.26:0/2537174675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0037070 con 0x7fa7f80d0750 2024-04-24T23:26:56.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.660+0000 7fa7fc82f640 1 -- 172.21.15.26:0/2537174675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f0032440 con 0x7fa7f80d0750 2024-04-24T23:26:56.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.660+0000 7fa7fc82f640 1 -- 172.21.15.26:0/2537174675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0032810 con 0x7fa7f80d0750 2024-04-24T23:26:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.660+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2537174675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80d0750 msgr2=0x7fa7f8165d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.660+0000 7fa7ffa95640 1 --2- 172.21.15.26:0/2537174675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80d0750 0x7fa7f8165d50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00097d0 tx=0x7fa7f002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.660+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2537174675 shutdown_connections 2024-04-24T23:26:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.660+0000 7fa7ffa95640 1 --2- 172.21.15.26:0/2537174675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80d0750 0x7fa7f8165d50 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.660+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2537174675 >> 172.21.15.26:0/2537174675 conn(0x7fa7f806b540 msgr2=0x7fa7f806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.660+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2537174675 shutdown_connections 2024-04-24T23:26:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.660+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2537174675 wait complete. 2024-04-24T23:26:56.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.661+0000 7fa7ffa95640 1 Processor -- start 2024-04-24T23:26:56.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.661+0000 7fa7ffa95640 1 -- start start 2024-04-24T23:26:56.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7ffa95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80ad7b0 0x7fa7f809c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7ffa95640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7f80d0750 0x7fa7f806cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7ffa95640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa7f815adc0 0x7fa7f806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7ffa95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f815e9c0 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7ffa95640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f815b1a0 con 0x7fa7f80d0750 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7ffa95640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f815eb40 con 0x7fa7f815adc0 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7fd831640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80ad7b0 0x7fa7f809c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7fd831640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80ad7b0 0x7fa7f809c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52674/0 (socket says 172.21.15.26:52674) 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7fd831640 1 -- 172.21.15.26:0/2699908940 learned_addr learned my addr 172.21.15.26:0/2699908940 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7fd030640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7f80d0750 0x7fa7f806cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.662+0000 7fa7fe032640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa7f815adc0 0x7fa7f806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7fd831640 1 -- 172.21.15.26:0/2699908940 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa7f815adc0 msgr2=0x7fa7f806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7fd831640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa7f815adc0 0x7fa7f806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7fd831640 1 -- 172.21.15.26:0/2699908940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7f80d0750 msgr2=0x7fa7f806cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7fd831640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7f80d0750 0x7fa7f806cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7fd831640 1 -- 172.21.15.26:0/2699908940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f81545d0 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7fd831640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80ad7b0 0x7fa7f809c8a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c780 tx=0x7fa7f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7ee7fc640 1 -- 172.21.15.26:0/2699908940 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4018070 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7ee7fc640 1 -- 172.21.15.26:0/2699908940 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f400a640 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2699908940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f8154870 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7ee7fc640 1 -- 172.21.15.26:0/2699908940 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f400aab0 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.663+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2699908940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f8154e30 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.664+0000 7fa7ee7fc640 1 -- 172.21.15.26:0/2699908940 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7f401b070 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.665+0000 7fa7ee7fc640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7dc06be30 0x7fa7dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.665+0000 7fa7ee7fc640 1 -- 172.21.15.26:0/2699908940 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fa7f408a4e0 con 0x7fa7f80ad7b0 2024-04-24T23:26:56.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.665+0000 7fa7fd030640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7dc06be30 0x7fa7dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.665+0000 7fa7fd030640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7dc06be30 0x7fa7dc06e2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa7f809c160 tx=0x7fa7e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.666+0000 7fa80279c740 1 Processor -- start 2024-04-24T23:26:56.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.666+0000 7fa80279c740 1 -- start start 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa80279c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 0x564de6141140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa80279c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de615eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa80279c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de6116a40 0x564de615ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa80279c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564de607a6b0 con 0x564de615e600 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa80279c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564de6022880 con 0x564de6140d60 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa80279c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x564de6076670 con 0x564de6116a40 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd030640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de615eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd030640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de615eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52682/0 (socket says 172.21.15.26:52682) 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd030640 1 -- 172.21.15.26:0/555346508 learned_addr learned my addr 172.21.15.26:0/555346508 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd831640 1 --2- 172.21.15.26:0/555346508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 0x564de6141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fe032640 1 --2- 172.21.15.26:0/555346508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de6116a40 0x564de615ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd030640 1 -- 172.21.15.26:0/555346508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de6116a40 msgr2=0x564de615ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd030640 1 --2- 172.21.15.26:0/555346508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de6116a40 0x564de615ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd030640 1 -- 172.21.15.26:0/555346508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 msgr2=0x564de6141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd030640 1 --2- 172.21.15.26:0/555346508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 0x564de6141140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd030640 1 -- 172.21.15.26:0/555346508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de5e587d0 con 0x564de615e600 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7fd030640 1 --2- 172.21.15.26:0/555346508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de615eb50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80adc20 tx=0x7fa7e8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3339a42b4dac6eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7db7fe640 1 -- 172.21.15.26:0/555346508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8025070 con 0x564de615e600 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7db7fe640 1 -- 172.21.15.26:0/555346508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e801b050 con 0x564de615e600 2024-04-24T23:26:56.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.667+0000 7fa7db7fe640 1 -- 172.21.15.26:0/555346508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8014de0 con 0x564de615e600 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.668+0000 7fa80279c740 1 -- 172.21.15.26:0/555346508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 msgr2=0x564de615eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.668+0000 7fa80279c740 1 --2- 172.21.15.26:0/555346508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de615eb50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80adc20 tx=0x7fa7e8012000 comp rx=0 tx=0).stop 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.668+0000 7fa80279c740 1 -- 172.21.15.26:0/555346508 shutdown_connections 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.668+0000 7fa80279c740 1 --2- 172.21.15.26:0/555346508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de6116a40 0x564de615ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.668+0000 7fa80279c740 1 --2- 172.21.15.26:0/555346508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de615eb50 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.668+0000 7fa80279c740 1 --2- 172.21.15.26:0/555346508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 0x564de6141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.668+0000 7fa80279c740 1 -- 172.21.15.26:0/555346508 >> 172.21.15.26:0/555346508 conn(0x564de6113600 msgr2=0x564de615bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.668+0000 7fa80279c740 1 -- 172.21.15.26:0/555346508 shutdown_connections 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.668+0000 7fa80279c740 1 -- 172.21.15.26:0/555346508 wait complete. 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.669+0000 7fa80279c740 1 -- start start 2024-04-24T23:26:56.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.669+0000 7fa80279c740 1 Processor -- start 2024-04-24T23:26:56.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa80279c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 0x564de6153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa80279c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de613ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa80279c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de613d600 0x564de6110260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa80279c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564de61177e0 con 0x564de615e600 2024-04-24T23:26:56.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa80279c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564de6035f50 con 0x564de6140d60 2024-04-24T23:26:56.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa80279c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x564de6111530 con 0x564de613d600 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa80279c740 10 client.0 _ll_register_callbacks cb 0x564de6071000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd030640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de613ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd030640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de613ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52698/0 (socket says 172.21.15.26:52698) 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd030640 1 -- 172.21.15.26:0/1279934527 learned_addr learned my addr 172.21.15.26:0/1279934527 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd831640 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 0x564de6153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd030640 1 -- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de613d600 msgr2=0x564de6110260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd030640 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de613d600 0x564de6110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd030640 1 -- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 msgr2=0x564de6153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd030640 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 0x564de6153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd030640 1 -- 172.21.15.26:0/1279934527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de5d8e3f0 con 0x564de615e600 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd831640 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 0x564de6153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7fd030640 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de613ce00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8023040 tx=0x7fa7e8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1881ecf3933b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1279934527 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8025040 con 0x564de615e600 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1279934527 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fa7e8021a50 con 0x564de615e600 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.670+0000 7fa80279c740 1 -- 172.21.15.26:0/1279934527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e801b050 con 0x564de615e600 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1279934527 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fa7e8035050 con 0x564de615e600 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa80279c740 10 client.8406 fetch_fsmap learned FSMap version 94 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa80279c740 1 -- 172.21.15.26:0/1279934527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x564de5d4e0f0 con 0x564de615e600 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1279934527 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e801b050 con 0x564de615e600 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1279934527 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) ==== 3550+0+0 (secure 0 0 0) 0x7fa7e8038020 con 0x564de615e600 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa80279c740 10 client.8406 fetch_fsmap finished waiting for FSMap version 94 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa80279c740 10 client.8406 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-04-24T23:26:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa80279c740 20 client.8406 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa80279c740 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x564de616c370 0x564de616c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa80279c740 4 client.8406 mds_command: new command op to 6299 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa80279c740 1 -- 172.21.15.26:0/1279934527 --> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564de6087bf0 con 0x564de616c370 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.671+0000 7fa7fd831640 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x564de616c370 0x564de616c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.672+0000 7fa7d8ff9640 10 client.8406 ms_handle_connect on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.672+0000 7fa7fd831640 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x564de616c370 0x564de616c7b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=442591afec6a37ba server_cookie=a1b0769389b2742e in_seq=0 out_seq=0 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.672+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1279934527 <== mds.2 v2:172.21.15.29:6833/3727846607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564de6087bf0 con 0x564de616c370 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.672+0000 7fa7d8ff9640 10 client.8406 handle_command_reply: tid=0 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 client.8406 shutdown 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 -- 172.21.15.26:0/1279934527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 msgr2=0x564de613ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de613ce00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8023040 tx=0x7fa7e8001120 comp rx=0 tx=0).stop 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 -- 172.21.15.26:0/1279934527 shutdown_connections 2024-04-24T23:26:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x564de616c370 0x564de616c7b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de613d600 0x564de6110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de615e600 0x564de613ce00 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa7d8ff9640 0 client.8406 ms_handle_reset on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:56.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 --2- 172.21.15.26:0/1279934527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de6140d60 0x564de6153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 -- 172.21.15.26:0/1279934527 >> 172.21.15.26:0/1279934527 conn(0x564de6116a40 msgr2=0x564de615f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 -- 172.21.15.26:0/1279934527 shutdown_connections 2024-04-24T23:26:56.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 1 -- 172.21.15.26:0/1279934527 wait complete. 2024-04-24T23:26:56.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.673+0000 7fa80279c740 20 client.8406 trim_cache size 0 max 16384 2024-04-24T23:26:56.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.691+0000 7fa80279c740 1 Processor -- start 2024-04-24T23:26:56.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.691+0000 7fa80279c740 1 -- start start 2024-04-24T23:26:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.691+0000 7fa80279c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de613d600 0x564de6117070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.691+0000 7fa80279c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de610cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.691+0000 7fa80279c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de615e600 0x564de613fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.691+0000 7fa80279c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8200820 con 0x564de6140d60 2024-04-24T23:26:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.691+0000 7fa80279c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81ff9c0 con 0x564de613d600 2024-04-24T23:26:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.691+0000 7fa80279c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f82000f0 con 0x564de615e600 2024-04-24T23:26:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.691+0000 7fa7fd030640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de610cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fd030640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de610cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52706/0 (socket says 172.21.15.26:52706) 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fd030640 1 -- 172.21.15.26:0/3349001169 learned_addr learned my addr 172.21.15.26:0/3349001169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fd831640 1 --2- 172.21.15.26:0/3349001169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de613d600 0x564de6117070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fe032640 1 --2- 172.21.15.26:0/3349001169 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de615e600 0x564de613fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fd030640 1 -- 172.21.15.26:0/3349001169 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de615e600 msgr2=0x564de613fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fd030640 1 --2- 172.21.15.26:0/3349001169 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de615e600 0x564de613fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fd030640 1 -- 172.21.15.26:0/3349001169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de613d600 msgr2=0x564de6117070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fd030640 1 --2- 172.21.15.26:0/3349001169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de613d600 0x564de6117070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fd030640 1 -- 172.21.15.26:0/3349001169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de6130c00 con 0x564de6140d60 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7fd030640 1 --2- 172.21.15.26:0/3349001169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de610cc50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8008fd0 tx=0x7fa7e8026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc76166a27b4391 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7daffd640 1 -- 172.21.15.26:0/3349001169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8025040 con 0x564de6140d60 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7daffd640 1 -- 172.21.15.26:0/3349001169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e801b070 con 0x564de6140d60 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.692+0000 7fa7daffd640 1 -- 172.21.15.26:0/3349001169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80072b0 con 0x564de6140d60 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 -- 172.21.15.26:0/3349001169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 msgr2=0x564de610cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 --2- 172.21.15.26:0/3349001169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de610cc50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8008fd0 tx=0x7fa7e8026000 comp rx=0 tx=0).stop 2024-04-24T23:26:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 -- 172.21.15.26:0/3349001169 shutdown_connections 2024-04-24T23:26:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 --2- 172.21.15.26:0/3349001169 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de615e600 0x564de613fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 --2- 172.21.15.26:0/3349001169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de610cc50 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 --2- 172.21.15.26:0/3349001169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de613d600 0x564de6117070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 -- 172.21.15.26:0/3349001169 >> 172.21.15.26:0/3349001169 conn(0x564de6116a40 msgr2=0x564de613cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 -- 172.21.15.26:0/3349001169 shutdown_connections 2024-04-24T23:26:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 -- 172.21.15.26:0/3349001169 wait complete. 2024-04-24T23:26:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 -- start start 2024-04-24T23:26:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.693+0000 7fa80279c740 1 Processor -- start 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa80279c740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de613d600 0x564de61533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa80279c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de617ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa80279c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de615e600 0x564de617f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa80279c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564de6116e20 con 0x564de6140d60 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa80279c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f80c9e00 con 0x564de615e600 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa80279c740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x564de6111b20 con 0x564de613d600 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa80279c740 10 client.0 _ll_register_callbacks cb 0x7fa7e8023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa7fd030640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de617ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa7fd030640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de617ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52718/0 (socket says 172.21.15.26:52718) 2024-04-24T23:26:56.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa7fd030640 1 -- 172.21.15.26:0/1670172560 learned_addr learned my addr 172.21.15.26:0/1670172560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa7fe032640 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de615e600 0x564de617f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.694+0000 7fa7fd831640 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de613d600 0x564de61533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7fd030640 1 -- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de613d600 msgr2=0x564de61533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7fd030640 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de613d600 0x564de61533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7fd030640 1 -- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de615e600 msgr2=0x564de617f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7fd030640 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de615e600 0x564de617f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7fd030640 1 -- 172.21.15.26:0/1670172560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de615c980 con 0x564de6140d60 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7fd030640 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de617ec50 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8007f30 tx=0x7fa7e800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d61aa1e9876190 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1670172560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8025040 con 0x564de6140d60 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1670172560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fa7e8007480 con 0x564de6140d60 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa80279c740 1 -- 172.21.15.26:0/1670172560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e801b070 con 0x564de6140d60 2024-04-24T23:26:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1670172560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fa7e804b050 con 0x564de6140d60 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa80279c740 10 client.8412 fetch_fsmap learned FSMap version 94 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa80279c740 1 -- 172.21.15.26:0/1670172560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x564de617fc70 con 0x564de6140d60 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1670172560 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e801b070 con 0x564de6140d60 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1670172560 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) ==== 3550+0+0 (secure 0 0 0) 0x7fa7e804e020 con 0x564de6140d60 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa80279c740 10 client.8412 fetch_fsmap finished waiting for FSMap version 94 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa80279c740 10 client.8412 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa80279c740 20 client.8412 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.695+0000 7fa80279c740 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x564de61538e0 0x564de61d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa80279c740 4 client.8412 mds_command: new command op to 6299 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa80279c740 1 -- 172.21.15.26:0/1670172560 --> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564de6081f40 con 0x564de61538e0 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa7fd831640 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x564de61538e0 0x564de61d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa7d8ff9640 10 client.8412 ms_handle_connect on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa7fd831640 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x564de61538e0 0x564de61d59c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e7fd7c33c5fd723 server_cookie=aebb2df94aa4b70b in_seq=0 out_seq=0 2024-04-24T23:26:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa7d8ff9640 1 -- 172.21.15.26:0/1670172560 <== mds.2 v2:172.21.15.29:6833/3727846607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x564de6081f40 con 0x564de61538e0 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa7d8ff9640 10 client.8412 handle_command_reply: tid=0 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa80279c740 1 client.8412 shutdown 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa80279c740 1 -- 172.21.15.26:0/1670172560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 msgr2=0x564de617ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa80279c740 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de617ec50 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8007f30 tx=0x7fa7e800a000 comp rx=0 tx=0).stop 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa80279c740 1 -- 172.21.15.26:0/1670172560 shutdown_connections 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.696+0000 7fa80279c740 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:6833/3727846607,v1:172.21.15.29:6835/3727846607] conn(0x564de61538e0 0x564de61d59c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa80279c740 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564de615e600 0x564de617f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa80279c740 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564de6140d60 0x564de617ec50 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7d8ff9640 0 client.8412 ms_handle_reset on v2:172.21.15.29:6833/3727846607 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa80279c740 1 --2- 172.21.15.26:0/1670172560 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x564de613d600 0x564de61533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa80279c740 1 -- 172.21.15.26:0/1670172560 >> 172.21.15.26:0/1670172560 conn(0x564de6116a40 msgr2=0x564de613fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa80279c740 1 -- 172.21.15.26:0/1670172560 shutdown_connections 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa80279c740 1 -- 172.21.15.26:0/1670172560 wait complete. 2024-04-24T23:26:56.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa80279c740 20 client.8412 trim_cache size 0 max 16384 2024-04-24T23:26:56.704 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:26:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:26:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "cca3b3ba-5fb5-4b95-8bce-070d29d5babd": { 2024-04-24T23:26:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:26:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "cca3b3ba-5fb5-4b95-8bce-070d29d5babd", 2024-04-24T23:26:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:26:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:56.704 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:56.705 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:56.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7dc06be30 msgr2=0x7fa7dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7dc06be30 0x7fa7dc06e2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa7f809c160 tx=0x7fa7e8009040 comp rx=0 tx=0).stop 2024-04-24T23:26:56.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80ad7b0 msgr2=0x7fa7f809c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80ad7b0 0x7fa7f809c8a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c780 tx=0x7fa7f400cc40 comp rx=0 tx=0).stop 2024-04-24T23:26:56.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2699908940 shutdown_connections 2024-04-24T23:26:56.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa7dc06be30 0x7fa7dc06e2f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa7f815adc0 0x7fa7f806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7f80d0750 0x7fa7f806cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 --2- 172.21.15.26:0/2699908940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7f80ad7b0 0x7fa7f809c8a0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2699908940 >> 172.21.15.26:0/2699908940 conn(0x7fa7f806b540 msgr2=0x7fa7f80d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2699908940 shutdown_connections 2024-04-24T23:26:56.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.697+0000 7fa7ffa95640 1 -- 172.21.15.26:0/2699908940 wait complete. 2024-04-24T23:26:56.711 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:26:56.711 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:26:56.712 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-24T23:26:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.814+0000 7f977bb63640 1 Processor -- start 2024-04-24T23:26:56.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.814+0000 7f977bb63640 1 -- start start 2024-04-24T23:26:56.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.815+0000 7f977bb63640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97740d0750 0x7f9774165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.815+0000 7f977bb63640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f977415ef60 con 0x7f977415adc0 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.815+0000 7f977bb63640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f977415f240 con 0x7f97740ad7b0 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.815+0000 7f977bb63640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f977415f520 con 0x7f97740d0750 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.815+0000 7f97798ff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f977415adc0 0x7f97740b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40138/0 (socket says 172.21.15.26:40138) 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.815+0000 7f97798ff640 1 -- 172.21.15.26:0/1085781582 learned_addr learned my addr 172.21.15.26:0/1085781582 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.815+0000 7f977a100640 1 --2- 172.21.15.26:0/1085781582 >> v2:172.21.15.26:3300/0 conn(0x7f97740d0750 0x7f9774165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.816+0000 7f97788fd640 1 -- 172.21.15.26:0/1085781582 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1302239097 0 0) 0x7f977415ef60 con 0x7f977415adc0 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.816+0000 7f97788fd640 1 -- 172.21.15.26:0/1085781582 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9760003520 con 0x7f977415adc0 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.816+0000 7f977a100640 1 -- 172.21.15.26:0/1085781582 >> v1:172.21.15.29:6789/0 conn(0x7f97740ad7b0 legacy=0x7f97740d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.816+0000 7f977a100640 1 -- 172.21.15.26:0/1085781582 >> v1:172.21.15.26:6789/0 conn(0x7f977415adc0 legacy=0x7f97740b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.816+0000 7f977a100640 1 -- 172.21.15.26:0/1085781582 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977415f800 con 0x7f97740d0750 2024-04-24T23:26:56.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.816+0000 7f977a100640 1 --2- 172.21.15.26:0/1085781582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 0x7f9774165d50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f976c009dc0 tx=0x7f976c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50f5e0eb396aad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f97788fd640 1 -- 172.21.15.26:0/1085781582 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c007740 con 0x7f97740d0750 2024-04-24T23:26:56.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f97788fd640 1 -- 172.21.15.26:0/1085781582 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c0078e0 con 0x7f97740d0750 2024-04-24T23:26:56.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f97788fd640 1 -- 172.21.15.26:0/1085781582 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c007cb0 con 0x7f97740d0750 2024-04-24T23:26:56.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f977bb63640 1 -- 172.21.15.26:0/1085781582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 msgr2=0x7f9774165d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f977bb63640 1 --2- 172.21.15.26:0/1085781582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 0x7f9774165d50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f976c009dc0 tx=0x7f976c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f977bb63640 1 -- 172.21.15.26:0/1085781582 shutdown_connections 2024-04-24T23:26:56.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f977bb63640 1 --2- 172.21.15.26:0/1085781582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 0x7f9774165d50 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f977bb63640 1 -- 172.21.15.26:0/1085781582 >> 172.21.15.26:0/1085781582 conn(0x7f977406b540 msgr2=0x7f977406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f977bb63640 1 -- 172.21.15.26:0/1085781582 shutdown_connections 2024-04-24T23:26:56.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.817+0000 7f977bb63640 1 -- 172.21.15.26:0/1085781582 wait complete. 2024-04-24T23:26:56.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.818+0000 7f977bb63640 1 Processor -- start 2024-04-24T23:26:56.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f977bb63640 1 -- start start 2024-04-24T23:26:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f977bb63640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97740ad7b0 0x7f97740cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f977bb63640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 0x7f9774140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f977bb63640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f977415adc0 0x7f97741e4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f977bb63640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f977415e9c0 con 0x7f97740d0750 2024-04-24T23:26:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f977bb63640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f977415b1a0 con 0x7f97740ad7b0 2024-04-24T23:26:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f977bb63640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f977415eb40 con 0x7f977415adc0 2024-04-24T23:26:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f97790fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 0x7f9774140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f97790fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 0x7f9774140f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52738/0 (socket says 172.21.15.26:52738) 2024-04-24T23:26:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f97790fe640 1 -- 172.21.15.26:0/3316766316 learned_addr learned my addr 172.21.15.26:0/3316766316 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f97798ff640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97740ad7b0 0x7f97740cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.819+0000 7f977a100640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f977415adc0 0x7f97741e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f97790fe640 1 -- 172.21.15.26:0/3316766316 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f977415adc0 msgr2=0x7f97741e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f97790fe640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f977415adc0 0x7f97741e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f97790fe640 1 -- 172.21.15.26:0/3316766316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97740ad7b0 msgr2=0x7f97740cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f97790fe640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97740ad7b0 0x7f97740cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f97790fe640 1 -- 172.21.15.26:0/3316766316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9774153a90 con 0x7f97740d0750 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f977a100640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f977415adc0 0x7f97741e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f97790fe640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 0x7f9774140f20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f977000c400 tx=0x7f977000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f976a7fc640 1 -- 172.21.15.26:0/3316766316 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770017070 con 0x7f97740d0750 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f976a7fc640 1 -- 172.21.15.26:0/3316766316 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97700052d0 con 0x7f97740d0750 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f977bb63640 1 -- 172.21.15.26:0/3316766316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97741418c0 con 0x7f97740d0750 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f976a7fc640 1 -- 172.21.15.26:0/3316766316 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770003830 con 0x7f97740d0750 2024-04-24T23:26:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.820+0000 7f977bb63640 1 -- 172.21.15.26:0/3316766316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9774141e80 con 0x7f97740d0750 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.822+0000 7f976a7fc640 1 -- 172.21.15.26:0/3316766316 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f977001a070 con 0x7f97740d0750 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.822+0000 7f976a7fc640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f975806be30 0x7f975806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.822+0000 7f976a7fc640 1 -- 172.21.15.26:0/3316766316 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f977008a190 con 0x7f97740d0750 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.822+0000 7f97798ff640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f975806be30 0x7f975806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.823+0000 7f97798ff640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f975806be30 0x7f975806e2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f97640011e0 tx=0x7f9764007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.823+0000 7f977e87b740 1 Processor -- start 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.823+0000 7f977e87b740 1 -- start start 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f977e87b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e650310 0x55f76e628970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f977e87b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 0x55f76e652b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f977e87b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e626910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f977e87b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f76e58c6b0 con 0x55f76e650310 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f977e87b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f76e534880 con 0x55f76e6506f0 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f977e87b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f76e588670 con 0x55f76e6530b0 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f97798ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e650310 0x55f76e628970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f97798ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e650310 0x55f76e628970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52748/0 (socket says 172.21.15.26:52748) 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f97798ff640 1 -- 172.21.15.26:0/313641799 learned_addr learned my addr 172.21.15.26:0/313641799 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f97790fe640 1 --2- 172.21.15.26:0/313641799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 0x55f76e652b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f977a100640 1 --2- 172.21.15.26:0/313641799 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e626910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f97798ff640 1 -- 172.21.15.26:0/313641799 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 msgr2=0x55f76e626910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f97798ff640 1 --2- 172.21.15.26:0/313641799 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e626910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f97798ff640 1 -- 172.21.15.26:0/313641799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 msgr2=0x55f76e652b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f97798ff640 1 --2- 172.21.15.26:0/313641799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 0x55f76e652b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.824+0000 7f97798ff640 1 -- 172.21.15.26:0/313641799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f76e36a7d0 con 0x55f76e650310 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f97798ff640 1 --2- 172.21.15.26:0/313641799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e650310 0x55f76e628970 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f9764005c10 tx=0x7f9764012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d91bea000af5702 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f97577fe640 1 -- 172.21.15.26:0/313641799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764026070 con 0x55f76e650310 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f97577fe640 1 -- 172.21.15.26:0/313641799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9764005e00 con 0x55f76e650310 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f97577fe640 1 -- 172.21.15.26:0/313641799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764001820 con 0x55f76e650310 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f977e87b740 1 -- 172.21.15.26:0/313641799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e650310 msgr2=0x55f76e628970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f977e87b740 1 --2- 172.21.15.26:0/313641799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e650310 0x55f76e628970 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f9764005c10 tx=0x7f9764012000 comp rx=0 tx=0).stop 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f977e87b740 1 -- 172.21.15.26:0/313641799 shutdown_connections 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f977e87b740 1 --2- 172.21.15.26:0/313641799 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e626910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f977e87b740 1 --2- 172.21.15.26:0/313641799 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 0x55f76e652b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f977e87b740 1 --2- 172.21.15.26:0/313641799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e650310 0x55f76e628970 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.825+0000 7f977e87b740 1 -- 172.21.15.26:0/313641799 >> 172.21.15.26:0/313641799 conn(0x55f76e625600 msgr2=0x55f76e66db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.826+0000 7f977e87b740 1 -- 172.21.15.26:0/313641799 shutdown_connections 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.826+0000 7f977e87b740 1 -- 172.21.15.26:0/313641799 wait complete. 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.826+0000 7f977e87b740 1 -- start start 2024-04-24T23:26:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.826+0000 7f977e87b740 1 Processor -- start 2024-04-24T23:26:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f977e87b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e656a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f977e87b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6530b0 0x55f76e64eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f977e87b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6655d0 0x55f76e682980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f977e87b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f76e547f50 con 0x55f76e6506f0 2024-04-24T23:26:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f977e87b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f76e66b870 con 0x55f76e6530b0 2024-04-24T23:26:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f977e87b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f76e656fa0 con 0x55f76e6655d0 2024-04-24T23:26:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f977e87b740 10 client.0 _ll_register_callbacks cb 0x55f76e583000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f97798ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e656a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f97798ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e656a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52750/0 (socket says 172.21.15.26:52750) 2024-04-24T23:26:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f97798ff640 1 -- 172.21.15.26:0/2310722921 learned_addr learned my addr 172.21.15.26:0/2310722921 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f977a100640 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6655d0 0x55f76e682980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f97790fe640 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6530b0 0x55f76e64eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f97798ff640 1 -- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6655d0 msgr2=0x55f76e682980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f97798ff640 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6655d0 0x55f76e682980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f97798ff640 1 -- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6530b0 msgr2=0x55f76e64eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f97798ff640 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6530b0 0x55f76e64eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.827+0000 7f97798ff640 1 -- 172.21.15.26:0/2310722921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f76e2a03f0 con 0x55f76e6506f0 2024-04-24T23:26:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f97798ff640 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e656a60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9764005fd0 tx=0x7f9764000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f08a13d0301862d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f9754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:56.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f9754ff9640 1 -- 172.21.15.26:0/2310722921 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764032060 con 0x55f76e6506f0 2024-04-24T23:26:56.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f9754ff9640 1 -- 172.21.15.26:0/2310722921 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f9764020420 con 0x55f76e6506f0 2024-04-24T23:26:56.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f977e87b740 1 -- 172.21.15.26:0/2310722921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9764005e00 con 0x55f76e6506f0 2024-04-24T23:26:56.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f9754ff9640 1 -- 172.21.15.26:0/2310722921 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f9764038050 con 0x55f76e6506f0 2024-04-24T23:26:56.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f977e87b740 10 client.8427 fetch_fsmap learned FSMap version 94 2024-04-24T23:26:56.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f977e87b740 1 -- 172.21.15.26:0/2310722921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55f76e2600f0 con 0x55f76e6506f0 2024-04-24T23:26:56.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f9754ff9640 1 -- 172.21.15.26:0/2310722921 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9764005e00 con 0x55f76e6506f0 2024-04-24T23:26:56.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f9754ff9640 1 -- 172.21.15.26:0/2310722921 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) ==== 3550+0+0 (secure 0 0 0) 0x7f976402c070 con 0x55f76e6506f0 2024-04-24T23:26:56.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f977e87b740 10 client.8427 fetch_fsmap finished waiting for FSMap version 94 2024-04-24T23:26:56.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f977e87b740 10 client.8427 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:56.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f977e87b740 20 client.8427 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:56.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f977e87b740 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f76e652380 0x55f76e6900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f977e87b740 4 client.8427 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:56.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.828+0000 7f977e87b740 1 -- 172.21.15.26:0/2310722921 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f76e599bf0 con 0x55f76e652380 2024-04-24T23:26:56.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.829+0000 7f97790fe640 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f76e652380 0x55f76e6900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.829+0000 7f97790fe640 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f76e652380 0x55f76e6900c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f96ccb06c57373a server_cookie=b797a624abddbf35 in_seq=0 out_seq=0 2024-04-24T23:26:56.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.829+0000 7f9754ff9640 10 client.8427 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:56.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f9754ff9640 1 -- 172.21.15.26:0/2310722921 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f76e599bf0 con 0x55f76e652380 2024-04-24T23:26:56.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f9754ff9640 10 client.8427 handle_command_reply: tid=0 2024-04-24T23:26:56.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 client.8427 shutdown 2024-04-24T23:26:56.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 -- 172.21.15.26:0/2310722921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 msgr2=0x55f76e656a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e656a60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9764005fd0 tx=0x7f9764000ed0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 -- 172.21.15.26:0/2310722921 shutdown_connections 2024-04-24T23:26:56.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f76e652380 0x55f76e6900c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6655d0 0x55f76e682980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6530b0 0x55f76e64eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f9754ff9640 0 client.8427 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:56.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 --2- 172.21.15.26:0/2310722921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e656a60 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 -- 172.21.15.26:0/2310722921 >> 172.21.15.26:0/2310722921 conn(0x55f76e650310 msgr2=0x55f76e626910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 -- 172.21.15.26:0/2310722921 shutdown_connections 2024-04-24T23:26:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 1 -- 172.21.15.26:0/2310722921 wait complete. 2024-04-24T23:26:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.830+0000 7f977e87b740 20 client.8427 trim_cache size 0 max 16384 2024-04-24T23:26:56.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.848+0000 7f977e87b740 1 Processor -- start 2024-04-24T23:26:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.848+0000 7f977e87b740 1 -- start start 2024-04-24T23:26:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.848+0000 7f977e87b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 0x55f76e629e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.848+0000 7f977e87b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e66db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.848+0000 7f977e87b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6655d0 0x55f76e66e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.848+0000 7f977e87b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97741fc3d0 con 0x55f76e6655d0 2024-04-24T23:26:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.848+0000 7f977e87b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f977409bc40 con 0x55f76e6506f0 2024-04-24T23:26:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.848+0000 7f977e87b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f977409c370 con 0x55f76e6530b0 2024-04-24T23:26:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f977a100640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6655d0 0x55f76e66e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f977a100640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6655d0 0x55f76e66e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52764/0 (socket says 172.21.15.26:52764) 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f977a100640 1 -- 172.21.15.26:0/3690398625 learned_addr learned my addr 172.21.15.26:0/3690398625 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f97798ff640 1 --2- 172.21.15.26:0/3690398625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 0x55f76e629e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f977a100640 1 -- 172.21.15.26:0/3690398625 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 msgr2=0x55f76e66db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f977a100640 1 --2- 172.21.15.26:0/3690398625 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e66db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f977a100640 1 -- 172.21.15.26:0/3690398625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 msgr2=0x55f76e629e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f977a100640 1 --2- 172.21.15.26:0/3690398625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 0x55f76e629e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f977a100640 1 -- 172.21.15.26:0/3690398625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f76e653f20 con 0x55f76e6655d0 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f977a100640 1 --2- 172.21.15.26:0/3690398625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6655d0 0x55f76e66e0a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f976c02ac90 tx=0x7f976c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14164a6936c04d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f9756ffd640 1 -- 172.21.15.26:0/3690398625 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c003b50 con 0x55f76e6655d0 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f9756ffd640 1 -- 172.21.15.26:0/3690398625 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c003cf0 con 0x55f76e6655d0 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.849+0000 7f9756ffd640 1 -- 172.21.15.26:0/3690398625 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c004240 con 0x55f76e6655d0 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 -- 172.21.15.26:0/3690398625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6655d0 msgr2=0x55f76e66e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 --2- 172.21.15.26:0/3690398625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6655d0 0x55f76e66e0a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f976c02ac90 tx=0x7f976c02e000 comp rx=0 tx=0).stop 2024-04-24T23:26:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 -- 172.21.15.26:0/3690398625 shutdown_connections 2024-04-24T23:26:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 --2- 172.21.15.26:0/3690398625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6655d0 0x55f76e66e0a0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 --2- 172.21.15.26:0/3690398625 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e66db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 --2- 172.21.15.26:0/3690398625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6506f0 0x55f76e629e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 -- 172.21.15.26:0/3690398625 >> 172.21.15.26:0/3690398625 conn(0x55f76e650310 msgr2=0x55f76e650ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 -- 172.21.15.26:0/3690398625 shutdown_connections 2024-04-24T23:26:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 -- 172.21.15.26:0/3690398625 wait complete. 2024-04-24T23:26:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 -- start start 2024-04-24T23:26:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.850+0000 7f977e87b740 1 Processor -- start 2024-04-24T23:26:56.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f977e87b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e654300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f977e87b740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e6e6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f977e87b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6655d0 0x55f76e663b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f977e87b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9774159850 con 0x55f76e6506f0 2024-04-24T23:26:56.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f977e87b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f76e656f40 con 0x55f76e6655d0 2024-04-24T23:26:56.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f977e87b740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f76e6ed5e0 con 0x55f76e6530b0 2024-04-24T23:26:56.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f977e87b740 10 client.0 _ll_register_callbacks cb 0x7f9764024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:56.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f97798ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e654300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f97798ff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e654300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52780/0 (socket says 172.21.15.26:52780) 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f97798ff640 1 -- 172.21.15.26:0/1583023535 learned_addr learned my addr 172.21.15.26:0/1583023535 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f977a100640 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6655d0 0x55f76e663b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.851+0000 7f97790fe640 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e6e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f97798ff640 1 -- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 msgr2=0x55f76e6e6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f97798ff640 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e6e6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f97798ff640 1 -- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6655d0 msgr2=0x55f76e663b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f97798ff640 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6655d0 0x55f76e663b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f97798ff640 1 -- 172.21.15.26:0/1583023535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f76e653880 con 0x55f76e6506f0 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f97798ff640 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e654300 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f9764007b20 tx=0x7f976400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689f1ef32ad6fecd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f9754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f9754ff9640 1 -- 172.21.15.26:0/1583023535 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764026070 con 0x55f76e6506f0 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f9754ff9640 1 -- 172.21.15.26:0/1583023535 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f976403eb60 con 0x55f76e6506f0 2024-04-24T23:26:56.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f977e87b740 1 -- 172.21.15.26:0/1583023535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f976c003cf0 con 0x55f76e6506f0 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f9754ff9640 1 -- 172.21.15.26:0/1583023535 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f9764005e00 con 0x55f76e6506f0 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f977e87b740 10 client.8433 fetch_fsmap learned FSMap version 94 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f977e87b740 1 -- 172.21.15.26:0/1583023535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55f76e664ce0 con 0x55f76e6506f0 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f9754ff9640 1 -- 172.21.15.26:0/1583023535 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c003cf0 con 0x55f76e6506f0 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f9754ff9640 1 -- 172.21.15.26:0/1583023535 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 94) ==== 3550+0+0 (secure 0 0 0) 0x7f976403b040 con 0x55f76e6506f0 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f977e87b740 10 client.8433 fetch_fsmap finished waiting for FSMap version 94 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f977e87b740 10 client.8433 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f977e87b740 20 client.8433 populate_metadata read hostname 'smithi026' 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f977e87b740 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f76e6ec850 0x55f76e6ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f977e87b740 4 client.8433 mds_command: new command op to 6790 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.852+0000 7f977e87b740 1 -- 172.21.15.26:0/1583023535 --> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f76e666ed0 con 0x55f76e6ec850 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f97790fe640 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f76e6ec850 0x55f76e6ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f97790fe640 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f76e6ec850 0x55f76e6ecc30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c1506544089476 server_cookie=76728f57dc79392e in_seq=0 out_seq=0 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f9754ff9640 10 client.8433 ms_handle_connect on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f9754ff9640 1 -- 172.21.15.26:0/1583023535 <== mds.0 v2:172.21.15.29:6832/2253149800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55f76e666ed0 con 0x55f76e6ec850 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f9754ff9640 10 client.8433 handle_command_reply: tid=0 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f977e87b740 1 client.8433 shutdown 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f977e87b740 1 -- 172.21.15.26:0/1583023535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 msgr2=0x55f76e654300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f977e87b740 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e654300 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f9764007b20 tx=0x7f976400a000 comp rx=0 tx=0).stop 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f977e87b740 1 -- 172.21.15.26:0/1583023535 shutdown_connections 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f977e87b740 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:6832/2253149800,v1:172.21.15.29:6834/2253149800] conn(0x55f76e6ec850 0x55f76e6ecc30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f9754ff9640 0 client.8433 ms_handle_reset on v2:172.21.15.29:6832/2253149800 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f977e87b740 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f76e6655d0 0x55f76e663b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f977e87b740 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f76e6530b0 0x55f76e6e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f977e87b740 1 --2- 172.21.15.26:0/1583023535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76e6506f0 0x55f76e654300 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.853+0000 7f977e87b740 1 -- 172.21.15.26:0/1583023535 >> 172.21.15.26:0/1583023535 conn(0x55f76e650310 msgr2=0x55f76e6528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977e87b740 1 -- 172.21.15.26:0/1583023535 shutdown_connections 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977e87b740 1 -- 172.21.15.26:0/1583023535 wait complete. 2024-04-24T23:26:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977e87b740 20 client.8433 trim_cache size 0 max 16384 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "cca3b3ba-5fb5-4b95-8bce-070d29d5babd": { 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "cca3b3ba-5fb5-4b95-8bce-070d29d5babd", 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:26:56.861 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:26:56.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 -- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f975806be30 msgr2=0x7f975806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f975806be30 0x7f975806e2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f97640011e0 tx=0x7f9764007d30 comp rx=0 tx=0).stop 2024-04-24T23:26:56.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 -- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 msgr2=0x7f9774140f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 0x7f9774140f20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f977000c400 tx=0x7f977000c8c0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 -- 172.21.15.26:0/3316766316 shutdown_connections 2024-04-24T23:26:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f975806be30 0x7f975806e2f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f977415adc0 0x7f97741e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97740d0750 0x7f9774140f20 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 --2- 172.21.15.26:0/3316766316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f97740ad7b0 0x7f97740cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 -- 172.21.15.26:0/3316766316 >> 172.21.15.26:0/3316766316 conn(0x7f977406b540 msgr2=0x7f977415b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 -- 172.21.15.26:0/3316766316 shutdown_connections 2024-04-24T23:26:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.854+0000 7f977bb63640 1 -- 172.21.15.26:0/3316766316 wait complete. 2024-04-24T23:26:56.865 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:26:56.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:56.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.965+0000 7fe8ebd3b640 1 Processor -- start 2024-04-24T23:26:56.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.965+0000 7fe8ebd3b640 1 -- start start 2024-04-24T23:26:56.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.966+0000 7fe8ebd3b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4145910 0x7fe8e406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.966+0000 7fe8ebd3b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e40cfbc0 con 0x7fe8e40cd660 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.966+0000 7fe8ebd3b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e406d7f0 con 0x7fe8e406d350 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.966+0000 7fe8ebd3b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e4145cf0 con 0x7fe8e4145910 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.966+0000 7fe8ea2d8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe8e40cd660 0x7fe8e415d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40142/0 (socket says 172.21.15.26:40142) 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.966+0000 7fe8ea2d8640 1 -- 172.21.15.26:0/1633545932 learned_addr learned my addr 172.21.15.26:0/1633545932 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e9ad7640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4145910 0x7fe8e406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3509005979 0 0) 0x7fe8e40cfbc0 con 0x7fe8e40cd660 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8cc003520 con 0x7fe8e40cd660 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 237969928 0 0) 0x7fe8cc003520 con 0x7fe8e40cd660 2024-04-24T23:26:56.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4145910 msgr2=0x7fe8e406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4145910 0x7fe8e406e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 >> v1:172.21.15.29:6789/0 conn(0x7fe8e406d350 legacy=0x7fe8e415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e406dad0 con 0x7fe8e40cd660 2024-04-24T23:26:56.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fe8d4001670 con 0x7fe8e40cd660 2024-04-24T23:26:56.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 >> v1:172.21.15.26:6789/0 conn(0x7fe8e40cd660 legacy=0x7fe8e415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8cc0039e0 0x7fe8cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4145910 0x7fe8cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8cc004d40 0x7fe8cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4145cf0 con 0x7fe8cc0039e0 2024-04-24T23:26:56.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc00ba70 con 0x7fe8cc004d40 2024-04-24T23:26:56.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe8cc00bc70 con 0x7fe8e4145910 2024-04-24T23:26:56.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e9ad7640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8cc0039e0 0x7fe8cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e92d6640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4145910 0x7fe8cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8ea2d8640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8cc004d40 0x7fe8cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e9ad7640 1 -- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4145910 msgr2=0x7fe8cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e9ad7640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4145910 0x7fe8cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e9ad7640 1 -- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8cc004d40 msgr2=0x7fe8cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e9ad7640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8cc004d40 0x7fe8cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e9ad7640 1 -- 172.21.15.26:0/1633545932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc003520 con 0x7fe8cc0039e0 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.967+0000 7fe8e92d6640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4145910 0x7fe8cc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.968+0000 7fe8e9ad7640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8cc0039e0 0x7fe8cc003e00 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009f90 tx=0x7fe8cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d1c809fe02a08d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.968+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc009c10 con 0x7fe8cc0039e0 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.968+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc036070 con 0x7fe8cc0039e0 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.968+0000 7fe8e8ad5640 1 -- 172.21.15.26:0/1633545932 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc031850 con 0x7fe8cc0039e0 2024-04-24T23:26:56.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.969+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/1633545932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8cc0039e0 msgr2=0x7fe8cc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.969+0000 7fe8ebd3b640 1 --2- 172.21.15.26:0/1633545932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8cc0039e0 0x7fe8cc003e00 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009f90 tx=0x7fe8cc005830 comp rx=0 tx=0).stop 2024-04-24T23:26:56.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.969+0000 7fe8ea2d8640 1 -- 172.21.15.26:0/1633545932 reap_dead start 2024-04-24T23:26:56.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.969+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/1633545932 shutdown_connections 2024-04-24T23:26:56.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.969+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/1633545932 >> 172.21.15.26:0/1633545932 conn(0x7fe8e406b300 msgr2=0x7fe8e406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:56.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.969+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/1633545932 shutdown_connections 2024-04-24T23:26:56.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.969+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/1633545932 wait complete. 2024-04-24T23:26:56.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.970+0000 7fe8ebd3b640 1 Processor -- start 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.970+0000 7fe8ebd3b640 1 -- start start 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8ebd3b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e4138420 0x7fe8e413f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8ebd3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8e4138800 0x7fe8e41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8ebd3b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4157f20 0x7fe8e41f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8ebd3b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e406e370 con 0x7fe8e4138800 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8ebd3b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e406e090 con 0x7fe8e4138420 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8ebd3b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e406e890 con 0x7fe8e4157f20 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e92d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8e4138800 0x7fe8e41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e92d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8e4138800 0x7fe8e41579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52810/0 (socket says 172.21.15.26:52810) 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e92d6640 1 -- 172.21.15.26:0/3824776662 learned_addr learned my addr 172.21.15.26:0/3824776662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e9ad7640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e4138420 0x7fe8e413f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8ea2d8640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4157f20 0x7fe8e41f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e92d6640 1 -- 172.21.15.26:0/3824776662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4157f20 msgr2=0x7fe8e41f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e92d6640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4157f20 0x7fe8e41f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e92d6640 1 -- 172.21.15.26:0/3824776662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e4138420 msgr2=0x7fe8e413f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e92d6640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e4138420 0x7fe8e413f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e92d6640 1 -- 172.21.15.26:0/3824776662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e41f3100 con 0x7fe8e4138800 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e92d6640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8e4138800 0x7fe8e41579e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800e600 tx=0x7fe8d800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.971+0000 7fe8e27fc640 1 -- 172.21.15.26:0/3824776662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80052e0 con 0x7fe8e4138800 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.972+0000 7fe8e27fc640 1 -- 172.21.15.26:0/3824776662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d8018070 con 0x7fe8e4138800 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.972+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e413fb90 con 0x7fe8e4138800 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.972+0000 7fe8e27fc640 1 -- 172.21.15.26:0/3824776662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8003810 con 0x7fe8e4138800 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.972+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e41f2860 con 0x7fe8e4138800 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.972+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e4158330 con 0x7fe8e4138800 2024-04-24T23:26:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.973+0000 7fe8e27fc640 1 -- 172.21.15.26:0/3824776662 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8d80039b0 con 0x7fe8e4138800 2024-04-24T23:26:56.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.973+0000 7fe8e27fc640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8d006be30 0x7fe8d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:56.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.974+0000 7fe8e27fc640 1 -- 172.21.15.26:0/3824776662 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fe8d808ac50 con 0x7fe8e4138800 2024-04-24T23:26:56.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.974+0000 7fe8e9ad7640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8d006be30 0x7fe8d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:56.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.974+0000 7fe8e9ad7640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8d006be30 0x7fe8d006e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc035000 tx=0x7fe8dc02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:56.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:56.978+0000 7fe8e27fc640 1 -- 172.21.15.26:0/3824776662 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe8e4158330 con 0x7fe8e4138800 2024-04-24T23:26:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.199+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8e40b11f0 con 0x7fe8e4138800 2024-04-24T23:26:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.200+0000 7fe8e27fc640 1 -- 172.21.15.26:0/3824776662 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7fe8d80595b0 con 0x7fe8e4138800 2024-04-24T23:26:57.201 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:57.202 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:26:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:26:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8d006be30 msgr2=0x7fe8d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8d006be30 0x7fe8d006e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc035000 tx=0x7fe8dc02f000 comp rx=0 tx=0).stop 2024-04-24T23:26:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8e4138800 msgr2=0x7fe8e41579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8e4138800 0x7fe8e41579e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800e600 tx=0x7fe8d800eac0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 shutdown_connections 2024-04-24T23:26:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8d006be30 0x7fe8d006e2f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8e4157f20 0x7fe8e41f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8e4138800 0x7fe8e41579e0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 --2- 172.21.15.26:0/3824776662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8e4138420 0x7fe8e413f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 >> 172.21.15.26:0/3824776662 conn(0x7fe8e406b300 msgr2=0x7fe8e4143d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 shutdown_connections 2024-04-24T23:26:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.202+0000 7fe8ebd3b640 1 -- 172.21.15.26:0/3824776662 wait complete. 2024-04-24T23:26:57.214 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'a', 'd'] 2024-04-24T23:26:57.214 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:26:57.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.313+0000 7f183d612640 1 Processor -- start 2024-04-24T23:26:57.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.313+0000 7f183d612640 1 -- start start 2024-04-24T23:26:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f183d612640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f183806dc30 0x7f18380d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f183d612640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183806f740 con 0x7f183809b110 2024-04-24T23:26:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f183d612640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183806fa20 con 0x7f183806e010 2024-04-24T23:26:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f183d612640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f183806fd00 con 0x7f183806dc30 2024-04-24T23:26:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18377fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f183809b110 0x7f183815dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40156/0 (socket says 172.21.15.26:40156) 2024-04-24T23:26:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18377fe640 1 -- 172.21.15.26:0/41299247 learned_addr learned my addr 172.21.15.26:0/41299247 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3514348025 0 0) 0x7f183806f740 con 0x7f183809b110 2024-04-24T23:26:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1824003520 con 0x7f183809b110 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f1836ffd640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f183806dc30 0x7f18380d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3364018136 0 0) 0x7f1824003520 con 0x7f183809b110 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f183806dc30 msgr2=0x7f18380d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f183806dc30 0x7f18380d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 >> v1:172.21.15.29:6789/0 conn(0x7f183806e010 legacy=0x7f183809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183806ffe0 con 0x7f183809b110 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f18300019f0 con 0x7f183809b110 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 >> v1:172.21.15.26:6789/0 conn(0x7f183809b110 legacy=0x7f183815dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f183806dc30 0x7f18240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1824004060 0x7f1824004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1824004d40 0x7f18240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f183806fd00 con 0x7f1824004060 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f182400bc00 con 0x7f1824004d40 2024-04-24T23:26:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.314+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f182400be00 con 0x7f183806dc30 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.315+0000 7f182ffff640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1824004060 0x7f1824004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.315+0000 7f1836ffd640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f183806dc30 0x7f18240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.315+0000 7f18377fe640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1824004d40 0x7f18240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.315+0000 7f182ffff640 1 -- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f183806dc30 msgr2=0x7f18240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.315+0000 7f182ffff640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f183806dc30 0x7f18240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.315+0000 7f182ffff640 1 -- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1824004d40 msgr2=0x7f18240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.315+0000 7f182ffff640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1824004d40 0x7f18240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.315+0000 7f182ffff640 1 -- 172.21.15.26:0/41299247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1824003520 con 0x7f1824004060 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.315+0000 7f18377fe640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1824004d40 0x7f18240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f182ffff640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1824004060 0x7f1824004800 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1820001720 tx=0x7f1824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415e40e7a0768311 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820001370 con 0x7f1824004060 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18200052d0 con 0x7f1824004060 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f18367fc640 1 -- 172.21.15.26:0/41299247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182002abc0 con 0x7f1824004060 2024-04-24T23:26:57.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f183d612640 1 -- 172.21.15.26:0/41299247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1824004060 msgr2=0x7f1824004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f183d612640 1 --2- 172.21.15.26:0/41299247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1824004060 0x7f1824004800 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1820001720 tx=0x7f1824005830 comp rx=0 tx=0).stop 2024-04-24T23:26:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f18377fe640 1 -- 172.21.15.26:0/41299247 reap_dead start 2024-04-24T23:26:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f183d612640 1 -- 172.21.15.26:0/41299247 shutdown_connections 2024-04-24T23:26:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f183d612640 1 -- 172.21.15.26:0/41299247 >> 172.21.15.26:0/41299247 conn(0x7f18380b3a60 msgr2=0x7f18380b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f183d612640 1 -- 172.21.15.26:0/41299247 shutdown_connections 2024-04-24T23:26:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.316+0000 7f183d612640 1 -- 172.21.15.26:0/41299247 wait complete. 2024-04-24T23:26:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f183d612640 1 Processor -- start 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f183d612640 1 -- start start 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f183d612640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1824004060 0x7f18380c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f183d612640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18380c7fe0 0x7f1838156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f183d612640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f183815ee30 0x7f183815f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f183d612640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f183806d240 con 0x7f183815ee30 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f183d612640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18380d0570 con 0x7f18380c7fe0 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f183d612640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f183806d3c0 con 0x7f1824004060 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f18377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f183815ee30 0x7f183815f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f18377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f183815ee30 0x7f183815f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52824/0 (socket says 172.21.15.26:52824) 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.318+0000 7f18377fe640 1 -- 172.21.15.26:0/585514626 learned_addr learned my addr 172.21.15.26:0/585514626 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f1836ffd640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1824004060 0x7f18380c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f182ffff640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18380c7fe0 0x7f1838156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f18377fe640 1 -- 172.21.15.26:0/585514626 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1824004060 msgr2=0x7f18380c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f18377fe640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1824004060 0x7f18380c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f18377fe640 1 -- 172.21.15.26:0/585514626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18380c7fe0 msgr2=0x7f1838156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f18377fe640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18380c7fe0 0x7f1838156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f18377fe640 1 -- 172.21.15.26:0/585514626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18380c6110 con 0x7f183815ee30 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f18377fe640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f183815ee30 0x7f183815f2b0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f18380cc550 tx=0x7f183000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f182f7fe640 1 -- 172.21.15.26:0/585514626 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183000ed40 con 0x7f183815ee30 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f182f7fe640 1 -- 172.21.15.26:0/585514626 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1830018070 con 0x7f183815ee30 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f182f7fe640 1 -- 172.21.15.26:0/585514626 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830013640 con 0x7f183815ee30 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18380c63f0 con 0x7f183815ee30 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.319+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18380c69b0 con 0x7f183815ee30 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.320+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183815f7f0 con 0x7f183815ee30 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.321+0000 7f182f7fe640 1 -- 172.21.15.26:0/585514626 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18300137e0 con 0x7f183815ee30 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.321+0000 7f182f7fe640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f181806be60 0x7f181806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.321+0000 7f182f7fe640 1 -- 172.21.15.26:0/585514626 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f183008b040 con 0x7f183815ee30 2024-04-24T23:26:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.321+0000 7f1836ffd640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f181806be60 0x7f181806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.322+0000 7f1836ffd640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f181806be60 0x7f181806e320 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1828006fd0 tx=0x7f1828008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.325+0000 7f182f7fe640 1 -- 172.21.15.26:0/585514626 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1830059a40 con 0x7f183815ee30 2024-04-24T23:26:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.578+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f18380694c0 con 0x7f183815ee30 2024-04-24T23:26:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.578+0000 7f182f7fe640 1 -- 172.21.15.26:0/585514626 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1967 (secure 0 0 0) 0x7f1830059340 con 0x7f183815ee30 2024-04-24T23:26:57.579 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:57.579 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":386,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700753,"bytes_used":278085632,"bytes_avail":772816027648,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":10152,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":94,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6299},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":6778},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-24T23:26:55.394721+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:26:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f181806be60 msgr2=0x7f181806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f181806be60 0x7f181806e320 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1828006fd0 tx=0x7f1828008040 comp rx=0 tx=0).stop 2024-04-24T23:26:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f183815ee30 msgr2=0x7f183815f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f183815ee30 0x7f183815f2b0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f18380cc550 tx=0x7f183000e530 comp rx=0 tx=0).stop 2024-04-24T23:26:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 shutdown_connections 2024-04-24T23:26:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f181806be60 0x7f181806e320 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f183815ee30 0x7f183815f2b0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18380c7fe0 0x7f1838156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 --2- 172.21.15.26:0/585514626 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1824004060 0x7f18380c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 >> 172.21.15.26:0/585514626 conn(0x7f18380b3a60 msgr2=0x7f18380cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 shutdown_connections 2024-04-24T23:26:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:57.580+0000 7f183d612640 1 -- 172.21.15.26:0/585514626 wait complete. 2024-04-24T23:26:57.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:26:58.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:58.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.705+0000 7fc556b1d640 1 Processor -- start 2024-04-24T23:26:58.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.705+0000 7fc556b1d640 1 -- start start 2024-04-24T23:26:58.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.706+0000 7fc556b1d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc55015e380 0x7fc5500b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.706+0000 7fc556b1d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5500b5440 con 0x7fc5500b4fd0 2024-04-24T23:26:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.706+0000 7fc556b1d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5500b5700 con 0x7fc5500b0030 2024-04-24T23:26:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.706+0000 7fc556b1d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc55015e760 con 0x7fc55015e380 2024-04-24T23:26:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.706+0000 7fc54ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc5500b4fd0 0x7fc550072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40160/0 (socket says 172.21.15.26:40160) 2024-04-24T23:26:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.706+0000 7fc54ffff640 1 -- 172.21.15.26:0/2922039363 learned_addr learned my addr 172.21.15.26:0/2922039363 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc5548b9640 1 --2- 172.21.15.26:0/2922039363 >> v2:172.21.15.29:3300/0 conn(0x7fc55015e380 0x7fc5500b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2038467215 0 0) 0x7fc5500b5440 con 0x7fc5500b4fd0 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc53c003520 con 0x7fc5500b4fd0 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3864322332 0 0) 0x7fc53c003520 con 0x7fc5500b4fd0 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 >> v2:172.21.15.29:3300/0 conn(0x7fc55015e380 msgr2=0x7fc5500b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 --2- 172.21.15.26:0/2922039363 >> v2:172.21.15.29:3300/0 conn(0x7fc55015e380 0x7fc5500b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 >> v1:172.21.15.29:6789/0 conn(0x7fc5500b0030 legacy=0x7fc5500b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5500b59e0 con 0x7fc5500b4fd0 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fc5400019f0 con 0x7fc5500b4fd0 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 >> v1:172.21.15.26:6789/0 conn(0x7fc5500b4fd0 legacy=0x7fc550072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc53c0039b0 0x7fc53c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc53c0043b0 0x7fc53c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc53c005080 0x7fc53c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc55015e760 con 0x7fc53c005080 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c00bf30 con 0x7fc53c0043b0 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c00c130 con 0x7fc53c0039b0 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc5550ba640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc53c005080 0x7fc53c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc54ffff640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc53c0043b0 0x7fc53c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.707+0000 7fc5548b9640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc53c0039b0 0x7fc53c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.708+0000 7fc5550ba640 1 -- 172.21.15.26:0/2922039363 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc53c0039b0 msgr2=0x7fc53c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.708+0000 7fc5550ba640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc53c0039b0 0x7fc53c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.708+0000 7fc5550ba640 1 -- 172.21.15.26:0/2922039363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc53c0043b0 msgr2=0x7fc53c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.708+0000 7fc5550ba640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc53c0043b0 0x7fc53c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.708+0000 7fc5550ba640 1 -- 172.21.15.26:0/2922039363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c003520 con 0x7fc53c005080 2024-04-24T23:26:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.708+0000 7fc5550ba640 1 -- 172.21.15.26:0/2922039363 reap_dead start 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.708+0000 7fc5550ba640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc53c005080 0x7fc53c00b9c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc538001860 tx=0x7fc53c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446dafb1689da8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538001430 con 0x7fc53c005080 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc538005330 con 0x7fc53c005080 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc54f7fe640 1 -- 172.21.15.26:0/2922039363 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53802ac10 con 0x7fc53c005080 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc556b1d640 1 -- 172.21.15.26:0/2922039363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc53c005080 msgr2=0x7fc53c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc556b1d640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc53c005080 0x7fc53c00b9c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc538001860 tx=0x7fc53c005830 comp rx=0 tx=0).stop 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc556b1d640 1 -- 172.21.15.26:0/2922039363 shutdown_connections 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc556b1d640 1 --2- 172.21.15.26:0/2922039363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc53c005080 0x7fc53c00b9c0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc556b1d640 1 -- 172.21.15.26:0/2922039363 >> 172.21.15.26:0/2922039363 conn(0x7fc55006e2c0 msgr2=0x7fc55006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc556b1d640 1 -- 172.21.15.26:0/2922039363 shutdown_connections 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.709+0000 7fc556b1d640 1 -- 172.21.15.26:0/2922039363 wait complete. 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.710+0000 7fc556b1d640 1 Processor -- start 2024-04-24T23:26:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.710+0000 7fc556b1d640 1 -- start start 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.711+0000 7fc556b1d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc53c005080 0x7fc550155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.711+0000 7fc556b1d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc55014c730 0x7fc5500c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.711+0000 7fc556b1d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc55014cb90 0x7fc5500c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.711+0000 7fc556b1d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc5500b4530 con 0x7fc55014cb90 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.711+0000 7fc556b1d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5500725b0 con 0x7fc55014c730 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.711+0000 7fc556b1d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc5500b46b0 con 0x7fc53c005080 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.711+0000 7fc5550ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc55014cb90 0x7fc5500c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.711+0000 7fc5550ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc55014cb90 0x7fc5500c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52852/0 (socket says 172.21.15.26:52852) 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.711+0000 7fc5550ba640 1 -- 172.21.15.26:0/25994905 learned_addr learned my addr 172.21.15.26:0/25994905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc54ffff640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc55014c730 0x7fc5500c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc5548b9640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc53c005080 0x7fc550155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc5550ba640 1 -- 172.21.15.26:0/25994905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc53c005080 msgr2=0x7fc550155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc5550ba640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc53c005080 0x7fc550155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc5550ba640 1 -- 172.21.15.26:0/25994905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc55014c730 msgr2=0x7fc5500c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc5550ba640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc55014c730 0x7fc5500c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc5550ba640 1 -- 172.21.15.26:0/25994905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5501548c0 con 0x7fc55014cb90 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc5550ba640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc55014cb90 0x7fc5500c6420 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc538001980 tx=0x7fc538001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc54d7fa640 1 -- 172.21.15.26:0/25994905 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538001430 con 0x7fc55014cb90 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc54d7fa640 1 -- 172.21.15.26:0/25994905 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc538003b30 con 0x7fc55014cb90 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc54d7fa640 1 -- 172.21.15.26:0/25994905 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538007030 con 0x7fc55014cb90 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc550154ba0 con 0x7fc55014cb90 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.712+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc550155160 con 0x7fc55014cb90 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.713+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5500694c0 con 0x7fc55014cb90 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.714+0000 7fc54d7fa640 1 -- 172.21.15.26:0/25994905 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc538005290 con 0x7fc55014cb90 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.714+0000 7fc54d7fa640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53006be30 0x7fc53006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.714+0000 7fc54d7fa640 1 -- 172.21.15.26:0/25994905 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fc5380aa180 con 0x7fc55014cb90 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.714+0000 7fc5548b9640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53006be30 0x7fc53006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.715+0000 7fc5548b9640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53006be30 0x7fc53006e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc548009b40 tx=0x7fc548009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.718+0000 7fc54d7fa640 1 -- 172.21.15.26:0/25994905 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc538044060 con 0x7fc55014cb90 2024-04-24T23:26:58.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.943+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc55015c8d0 con 0x7fc55014cb90 2024-04-24T23:26:58.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.943+0000 7fc54d7fa640 1 -- 172.21.15.26:0/25994905 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7fc538031350 con 0x7fc55014cb90 2024-04-24T23:26:58.944 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:58.945 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:26:58.945 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:26:58.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53006be30 msgr2=0x7fc53006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53006be30 0x7fc53006e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc548009b40 tx=0x7fc548009190 comp rx=0 tx=0).stop 2024-04-24T23:26:58.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc55014cb90 msgr2=0x7fc5500c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc55014cb90 0x7fc5500c6420 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc538001980 tx=0x7fc538001230 comp rx=0 tx=0).stop 2024-04-24T23:26:58.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 shutdown_connections 2024-04-24T23:26:58.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc53006be30 0x7fc53006e2f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc55014cb90 0x7fc5500c6420 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc55014c730 0x7fc5500c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 --2- 172.21.15.26:0/25994905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc53c005080 0x7fc550155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 >> 172.21.15.26:0/25994905 conn(0x7fc55006e2c0 msgr2=0x7fc55015d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:58.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 shutdown_connections 2024-04-24T23:26:58.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:58.945+0000 7fc556b1d640 1 -- 172.21.15.26:0/25994905 wait complete. 2024-04-24T23:26:58.958 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'a', 'd'] 2024-04-24T23:26:58.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:26:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.058+0000 7ff764329640 1 Processor -- start 2024-04-24T23:26:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.058+0000 7ff764329640 1 -- start start 2024-04-24T23:26:59.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff764329640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff75c15be90 0x7ff75c15f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff764329640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff75c15f850 con 0x7ff75c06dbc0 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff764329640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff75c162000 con 0x7ff75c06d760 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff764329640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff75c15c270 con 0x7ff75c15be90 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7628c6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff75c06dbc0 0x7ff75c0b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40162/0 (socket says 172.21.15.26:40162) 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7628c6640 1 -- 172.21.15.26:0/1453145849 learned_addr learned my addr 172.21.15.26:0/1453145849 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61935528 0 0) 0x7ff754001490 con 0x7ff75c06dbc0 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff748003520 con 0x7ff75c06dbc0 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7620c5640 1 --2- 172.21.15.26:0/1453145849 >> v2:172.21.15.29:3300/0 conn(0x7ff75c15be90 0x7ff75c15f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2574724092 0 0) 0x7ff748003520 con 0x7ff75c06dbc0 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 >> v2:172.21.15.29:3300/0 conn(0x7ff75c15be90 msgr2=0x7ff75c15f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7610c3640 1 --2- 172.21.15.26:0/1453145849 >> v2:172.21.15.29:3300/0 conn(0x7ff75c15be90 0x7ff75c15f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 >> v1:172.21.15.29:6789/0 conn(0x7ff75c06d760 legacy=0x7ff75c162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.059+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c071a70 con 0x7ff75c06dbc0 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7ff754001bc0 con 0x7ff75c06dbc0 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 >> v1:172.21.15.26:6789/0 conn(0x7ff75c06dbc0 legacy=0x7ff75c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7610c3640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7480039b0 0x7ff748003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7610c3640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748004800 0x7ff748004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7610c3640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff7480052e0 0x7ff74800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c15c270 con 0x7ff748004800 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff74800c1b0 con 0x7ff7480039b0 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff74800c3b0 con 0x7ff7480052e0 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7618c4640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748004800 0x7ff748004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7620c5640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7480039b0 0x7ff748003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7628c6640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff7480052e0 0x7ff74800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7618c4640 1 -- 172.21.15.26:0/1453145849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff7480052e0 msgr2=0x7ff74800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7618c4640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff7480052e0 0x7ff74800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7618c4640 1 -- 172.21.15.26:0/1453145849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7480039b0 msgr2=0x7ff748003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7618c4640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff7480039b0 0x7ff748003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7618c4640 1 -- 172.21.15.26:0/1453145849 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff748003520 con 0x7ff748004800 2024-04-24T23:26:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.060+0000 7ff7628c6640 1 -- 172.21.15.26:0/1453145849 reap_dead start 2024-04-24T23:26:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.061+0000 7ff7618c4640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748004800 0x7ff748004c80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff74c001720 tx=0x7ff748005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cda2dc55c7fa79d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.061+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c001370 con 0x7ff748004800 2024-04-24T23:26:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.061+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74c0052d0 con 0x7ff748004800 2024-04-24T23:26:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.061+0000 7ff7610c3640 1 -- 172.21.15.26:0/1453145849 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c02ab70 con 0x7ff748004800 2024-04-24T23:26:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.062+0000 7ff764329640 1 -- 172.21.15.26:0/1453145849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748004800 msgr2=0x7ff748004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.062+0000 7ff764329640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748004800 0x7ff748004c80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff74c001720 tx=0x7ff748005830 comp rx=0 tx=0).stop 2024-04-24T23:26:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.062+0000 7ff764329640 1 -- 172.21.15.26:0/1453145849 shutdown_connections 2024-04-24T23:26:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.062+0000 7ff764329640 1 --2- 172.21.15.26:0/1453145849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff748004800 0x7ff748004c80 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.062+0000 7ff764329640 1 -- 172.21.15.26:0/1453145849 >> 172.21.15.26:0/1453145849 conn(0x7ff75c0ccff0 msgr2=0x7ff75c0cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.062+0000 7ff764329640 1 -- 172.21.15.26:0/1453145849 shutdown_connections 2024-04-24T23:26:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.062+0000 7ff764329640 1 -- 172.21.15.26:0/1453145849 wait complete. 2024-04-24T23:26:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.063+0000 7ff764329640 1 Processor -- start 2024-04-24T23:26:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.063+0000 7ff764329640 1 -- start start 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff764329640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff748004800 0x7ff75c147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff764329640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff75c1481e0 0x7ff75c147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff764329640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c155b40 0x7ff75c1f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff764329640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c071740 con 0x7ff75c155b40 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff764329640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c072030 con 0x7ff748004800 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff764329640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff75c0b3650 con 0x7ff75c1481e0 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7628c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c155b40 0x7ff75c1f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7628c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c155b40 0x7ff75c1f7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52864/0 (socket says 172.21.15.26:52864) 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7628c6640 1 -- 172.21.15.26:0/2353513821 learned_addr learned my addr 172.21.15.26:0/2353513821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7628c6640 1 -- 172.21.15.26:0/2353513821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff75c1481e0 msgr2=0x7ff75c147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7628c6640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff75c1481e0 0x7ff75c147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7620c5640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff748004800 0x7ff75c147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7628c6640 1 -- 172.21.15.26:0/2353513821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff748004800 msgr2=0x7ff75c147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7628c6640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff748004800 0x7ff75c147000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7628c6640 1 -- 172.21.15.26:0/2353513821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c154d30 con 0x7ff75c155b40 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff7628c6640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c155b40 0x7ff75c1f7b40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff754000f80 tx=0x7ff75400aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff752ffd640 1 -- 172.21.15.26:0/2353513821 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540078c0 con 0x7ff75c155b40 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff752ffd640 1 -- 172.21.15.26:0/2353513821 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff754018070 con 0x7ff75c155b40 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c155010 con 0x7ff75c155b40 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff752ffd640 1 -- 172.21.15.26:0/2353513821 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754013630 con 0x7ff75c155b40 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.064+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c1f8080 con 0x7ff75c155b40 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.065+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c0694c0 con 0x7ff75c155b40 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.066+0000 7ff752ffd640 1 -- 172.21.15.26:0/2353513821 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7540137d0 con 0x7ff75c155b40 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.066+0000 7ff752ffd640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff74006be30 0x7ff74006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.066+0000 7ff752ffd640 1 -- 172.21.15.26:0/2353513821 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7ff75408b4f0 con 0x7ff75c155b40 2024-04-24T23:26:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.066+0000 7ff7620c5640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff74006be30 0x7ff74006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.067+0000 7ff7620c5640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff74006be30 0x7ff74006e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff75c068f60 tx=0x7ff758008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.070+0000 7ff752ffd640 1 -- 172.21.15.26:0/2353513821 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff75c15f850 con 0x7ff75c155b40 2024-04-24T23:26:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.326+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff75c0728a0 con 0x7ff75c155b40 2024-04-24T23:26:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.326+0000 7ff752ffd640 1 -- 172.21.15.26:0/2353513821 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1967 (secure 0 0 0) 0x7ff754059ef0 con 0x7ff75c155b40 2024-04-24T23:26:59.327 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:26:59.327 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":388,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700753,"bytes_used":278085632,"bytes_avail":772816027648,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":10152,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":94,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6299},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":6778},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-24T23:26:55.394721+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:26:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.328+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff74006be30 msgr2=0x7ff74006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.328+0000 7ff764329640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff74006be30 0x7ff74006e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff75c068f60 tx=0x7ff758008040 comp rx=0 tx=0).stop 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.328+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c155b40 msgr2=0x7ff75c1f7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.328+0000 7ff764329640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c155b40 0x7ff75c1f7b40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff754000f80 tx=0x7ff75400aef0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.329+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 shutdown_connections 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.329+0000 7ff764329640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff74006be30 0x7ff74006e2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.329+0000 7ff764329640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c155b40 0x7ff75c1f7b40 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.329+0000 7ff764329640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff75c1481e0 0x7ff75c147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.329+0000 7ff764329640 1 --2- 172.21.15.26:0/2353513821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff748004800 0x7ff75c147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.329+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 >> 172.21.15.26:0/2353513821 conn(0x7ff75c0ccff0 msgr2=0x7ff75c15ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.329+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 shutdown_connections 2024-04-24T23:26:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:26:59.329+0000 7ff764329640 1 -- 172.21.15.26:0/2353513821 wait complete. 2024-04-24T23:26:59.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:26:59.746 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:26:59.749 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:27:00.341 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:00.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.453+0000 7f5643b70640 1 Processor -- start 2024-04-24T23:27:00.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.453+0000 7f5643b70640 1 -- start start 2024-04-24T23:27:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.454+0000 7f5643b70640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f563c06b830 0x7f563c0720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.454+0000 7f5643b70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c073260 0x7f563c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.454+0000 7f5643b70640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f563c070a60 con 0x7f563c145960 2024-04-24T23:27:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.454+0000 7f5643b70640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f563c073040 con 0x7f563c06b830 2024-04-24T23:27:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.454+0000 7f5643b70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f563c15c7e0 con 0x7f563c073260 2024-04-24T23:27:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.454+0000 7f564210d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c073260 0x7f563c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564110b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f563c06b830 0x7f563c0720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564210d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c073260 0x7f563c15cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:49610/0 (socket says 172.21.15.26:49610) 2024-04-24T23:27:00.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564210d640 1 -- 172.21.15.26:0/1094842238 learned_addr learned my addr 172.21.15.26:0/1094842238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:00.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564110b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f563c06b830 0x7f563c0720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:42124/0 (socket says 172.21.15.26:42124) 2024-04-24T23:27:00.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564190c640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f563c145960 0x7f563c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:42128/0 (socket says 172.21.15.26:42128) 2024-04-24T23:27:00.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564210d640 1 -- 172.21.15.26:0/1094842238 >> v2:172.21.15.29:3300/0 conn(0x7f563c06b830 msgr2=0x7f563c0720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:00.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564210d640 1 --2- 172.21.15.26:0/1094842238 >> v2:172.21.15.29:3300/0 conn(0x7f563c06b830 0x7f563c0720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564210d640 1 -- 172.21.15.26:0/1094842238 >> v1:172.21.15.29:6789/0 conn(0x7f563c145960 legacy=0x7f563c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564210d640 1 -- 172.21.15.26:0/1094842238 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c0727a0 con 0x7f563c073260 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.455+0000 7f564110b640 1 --2- 172.21.15.26:0/1094842238 >> v2:172.21.15.29:3300/0 conn(0x7f563c06b830 0x7f563c0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f564210d640 1 --2- 172.21.15.26:0/1094842238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c073260 0x7f563c15cc70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5634009e50 tx=0x7f563402a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2e84cf9f6ba04b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f564090a640 1 -- 172.21.15.26:0/1094842238 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634036070 con 0x7f563c073260 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f564090a640 1 -- 172.21.15.26:0/1094842238 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5634031600 con 0x7f563c073260 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f564090a640 1 -- 172.21.15.26:0/1094842238 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340319d0 con 0x7f563c073260 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f5643b70640 1 -- 172.21.15.26:0/1094842238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c073260 msgr2=0x7f563c15cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f5643b70640 1 --2- 172.21.15.26:0/1094842238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c073260 0x7f563c15cc70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5634009e50 tx=0x7f563402a730 comp rx=0 tx=0).stop 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f5643b70640 1 -- 172.21.15.26:0/1094842238 shutdown_connections 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f5643b70640 1 --2- 172.21.15.26:0/1094842238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c073260 0x7f563c15cc70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f5643b70640 1 --2- 172.21.15.26:0/1094842238 >> v2:172.21.15.29:3300/0 conn(0x7f563c06b830 0x7f563c0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.456+0000 7f5643b70640 1 -- 172.21.15.26:0/1094842238 >> 172.21.15.26:0/1094842238 conn(0x7f563c06e2c0 msgr2=0x7f563c06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.457+0000 7f5643b70640 1 -- 172.21.15.26:0/1094842238 shutdown_connections 2024-04-24T23:27:00.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.457+0000 7f5643b70640 1 -- 172.21.15.26:0/1094842238 wait complete. 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.458+0000 7f5643b70640 1 Processor -- start 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.458+0000 7f5643b70640 1 -- start start 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f5643b70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c06b830 0x7f563c0ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f5643b70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f563c073260 0x7f563c0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f5643b70640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f563c145960 0x7f563c0a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f5643b70640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f563c15d1b0 con 0x7f563c073260 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f5643b70640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f563c072d60 con 0x7f563c145960 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f5643b70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f563c15d330 con 0x7f563c06b830 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564110b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f563c073260 0x7f563c0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564110b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f563c073260 0x7f563c0a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52866/0 (socket says 172.21.15.26:52866) 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564110b640 1 -- 172.21.15.26:0/3666793969 learned_addr learned my addr 172.21.15.26:0/3666793969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564190c640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c06b830 0x7f563c0ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564210d640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f563c145960 0x7f563c0a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564110b640 1 -- 172.21.15.26:0/3666793969 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c06b830 msgr2=0x7f563c0ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564110b640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c06b830 0x7f563c0ab320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564110b640 1 -- 172.21.15.26:0/3666793969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f563c145960 msgr2=0x7f563c0a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564110b640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f563c145960 0x7f563c0a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564110b640 1 -- 172.21.15.26:0/3666793969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c0aa040 con 0x7f563c073260 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.459+0000 7f564110b640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f563c073260 0x7f563c0a9460 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f562c00ebe0 tx=0x7f562c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.460+0000 7f56327fc640 1 -- 172.21.15.26:0/3666793969 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c017070 con 0x7f563c073260 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.460+0000 7f56327fc640 1 -- 172.21.15.26:0/3666793969 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c00cc30 con 0x7f563c073260 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.460+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c0aa320 con 0x7f563c073260 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.460+0000 7f56327fc640 1 -- 172.21.15.26:0/3666793969 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c013470 con 0x7f563c073260 2024-04-24T23:27:00.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.460+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c0ab860 con 0x7f563c073260 2024-04-24T23:27:00.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.461+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563c1f2320 con 0x7f563c073260 2024-04-24T23:27:00.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.461+0000 7f56327fc640 1 -- 172.21.15.26:0/3666793969 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f562c01a070 con 0x7f563c073260 2024-04-24T23:27:00.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.462+0000 7f56327fc640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f562006be30 0x7f562006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.462+0000 7f56327fc640 1 -- 172.21.15.26:0/3666793969 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f562c08b170 con 0x7f563c073260 2024-04-24T23:27:00.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.462+0000 7f564190c640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f562006be30 0x7f562006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.462+0000 7f564190c640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f562006be30 0x7f562006e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5638007fd0 tx=0x7f5638007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.466+0000 7f56327fc640 1 -- 172.21.15.26:0/3666793969 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f563c1f2320 con 0x7f563c073260 2024-04-24T23:27:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.687+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f563c13cef0 con 0x7f563c073260 2024-04-24T23:27:00.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.688+0000 7f56327fc640 1 -- 172.21.15.26:0/3666793969 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7f562c059430 con 0x7f563c073260 2024-04-24T23:27:00.689 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:00.689 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:27:00.689 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f562006be30 msgr2=0x7f562006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f562006be30 0x7f562006e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5638007fd0 tx=0x7f5638007e80 comp rx=0 tx=0).stop 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f563c073260 msgr2=0x7f563c0a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f563c073260 0x7f563c0a9460 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f562c00ebe0 tx=0x7f562c00c520 comp rx=0 tx=0).stop 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 shutdown_connections 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f562006be30 0x7f562006e2f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f563c145960 0x7f563c0a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f563c073260 0x7f563c0a9460 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 --2- 172.21.15.26:0/3666793969 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f563c06b830 0x7f563c0ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 >> 172.21.15.26:0/3666793969 conn(0x7f563c06e2c0 msgr2=0x7f563c144a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:00.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 shutdown_connections 2024-04-24T23:27:00.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.690+0000 7f5643b70640 1 -- 172.21.15.26:0/3666793969 wait complete. 2024-04-24T23:27:00.701 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'a', 'd'] 2024-04-24T23:27:00.702 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:27:00.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.800+0000 7f86de57e640 1 Processor -- start 2024-04-24T23:27:00.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.800+0000 7f86de57e640 1 -- start start 2024-04-24T23:27:00.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86de57e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f86d8145a80 0x7f86d80ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86de57e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86d815c5b0 0x7f86d806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86de57e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815ca40 0x7f86d806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86de57e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f86d80b3300 con 0x7f86d815c5b0 2024-04-24T23:27:00.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86de57e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f86d815cee0 con 0x7f86d8145a80 2024-04-24T23:27:00.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86de57e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f86d815d1c0 con 0x7f86d815ca40 2024-04-24T23:27:00.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86d77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86d815c5b0 0x7f86d806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86d77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86d815c5b0 0x7f86d806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52882/0 (socket says 172.21.15.26:52882) 2024-04-24T23:27:00.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86d77fe640 1 -- 172.21.15.26:0/3300134425 learned_addr learned my addr 172.21.15.26:0/3300134425 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:00.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86d7fff640 1 --2- 172.21.15.26:0/3300134425 >> v2:172.21.15.29:3300/0 conn(0x7f86d8145a80 0x7f86d80ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.801+0000 7f86dcb1b640 1 --2- 172.21.15.26:0/3300134425 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815ca40 0x7f86d806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.802+0000 7f86d77fe640 1 -- 172.21.15.26:0/3300134425 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815ca40 msgr2=0x7f86d806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.802+0000 7f86d77fe640 1 --2- 172.21.15.26:0/3300134425 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815ca40 0x7f86d806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.802+0000 7f86d77fe640 1 -- 172.21.15.26:0/3300134425 >> v2:172.21.15.29:3300/0 conn(0x7f86d8145a80 msgr2=0x7f86d80ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.802+0000 7f86d77fe640 1 --2- 172.21.15.26:0/3300134425 >> v2:172.21.15.29:3300/0 conn(0x7f86d8145a80 0x7f86d80ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.802+0000 7f86d77fe640 1 -- 172.21.15.26:0/3300134425 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d815d4a0 con 0x7f86d815c5b0 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.802+0000 7f86d77fe640 1 --2- 172.21.15.26:0/3300134425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d815c5b0 0x7f86d806d530 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f86d806f170 tx=0x7f86c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90140204e2d383c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.802+0000 7f86d6ffd640 1 -- 172.21.15.26:0/3300134425 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8037070 con 0x7f86d815c5b0 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86d6ffd640 1 -- 172.21.15.26:0/3300134425 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c8032440 con 0x7f86d815c5b0 2024-04-24T23:27:00.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86d6ffd640 1 -- 172.21.15.26:0/3300134425 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8032810 con 0x7f86d815c5b0 2024-04-24T23:27:00.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86de57e640 1 -- 172.21.15.26:0/3300134425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d815c5b0 msgr2=0x7f86d806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86de57e640 1 --2- 172.21.15.26:0/3300134425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d815c5b0 0x7f86d806d530 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f86d806f170 tx=0x7f86c802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86de57e640 1 -- 172.21.15.26:0/3300134425 shutdown_connections 2024-04-24T23:27:00.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86de57e640 1 --2- 172.21.15.26:0/3300134425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d815c5b0 0x7f86d806d530 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86de57e640 1 --2- 172.21.15.26:0/3300134425 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815ca40 0x7f86d806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86de57e640 1 --2- 172.21.15.26:0/3300134425 >> v2:172.21.15.29:3300/0 conn(0x7f86d8145a80 0x7f86d80ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86de57e640 1 -- 172.21.15.26:0/3300134425 >> 172.21.15.26:0/3300134425 conn(0x7f86d806b540 msgr2=0x7f86d806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86de57e640 1 -- 172.21.15.26:0/3300134425 shutdown_connections 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.803+0000 7f86de57e640 1 -- 172.21.15.26:0/3300134425 wait complete. 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.804+0000 7f86de57e640 1 Processor -- start 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86de57e640 1 -- start start 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86de57e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d8145a80 0x7f86d81f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86de57e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815c5b0 0x7f86d81508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86de57e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86d815ca40 0x7f86d8150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86de57e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86d815fd70 con 0x7f86d8145a80 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86de57e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86d815fbf0 con 0x7f86d815ca40 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86de57e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f86d815fef0 con 0x7f86d815c5b0 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d8145a80 0x7f86d81f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d8145a80 0x7f86d81f1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52898/0 (socket says 172.21.15.26:52898) 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86d7fff640 1 -- 172.21.15.26:0/4151069780 learned_addr learned my addr 172.21.15.26:0/4151069780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86dcb1b640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86d815ca40 0x7f86d8150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.805+0000 7f86d77fe640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815c5b0 0x7f86d81508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d7fff640 1 -- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815c5b0 msgr2=0x7f86d81508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d7fff640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815c5b0 0x7f86d81508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d7fff640 1 -- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86d815ca40 msgr2=0x7f86d8150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d7fff640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86d815ca40 0x7f86d8150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d7fff640 1 -- 172.21.15.26:0/4151069780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d814de20 con 0x7f86d8145a80 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86dcb1b640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86d815ca40 0x7f86d8150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d77fe640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815c5b0 0x7f86d81508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:00.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d7fff640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d8145a80 0x7f86d81f1e90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f86c000e890 tx=0x7f86c000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d4ff9640 1 -- 172.21.15.26:0/4151069780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0017070 con 0x7f86d8145a80 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d4ff9640 1 -- 172.21.15.26:0/4151069780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c000c940 con 0x7f86d8145a80 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d814e100 con 0x7f86d8145a80 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86d4ff9640 1 -- 172.21.15.26:0/4151069780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c000cd10 con 0x7f86d8145a80 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.806+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d8142c50 con 0x7f86d8145a80 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.807+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d80694c0 con 0x7f86d8145a80 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.808+0000 7f86d4ff9640 1 -- 172.21.15.26:0/4151069780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86c001a070 con 0x7f86d8145a80 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.808+0000 7f86d4ff9640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f86b806be30 0x7f86b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.808+0000 7f86d4ff9640 1 -- 172.21.15.26:0/4151069780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f86c008b160 con 0x7f86d8145a80 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.808+0000 7f86d77fe640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f86b806be30 0x7f86b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.809+0000 7f86d77fe640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f86b806be30 0x7f86b806e2f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f86c8034000 tx=0x7f86c802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:00.812+0000 7f86d4ff9640 1 -- 172.21.15.26:0/4151069780 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86c0059b60 con 0x7f86d8145a80 2024-04-24T23:27:00.874 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:27:00.873+0000 7f26a3749640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T23:27:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.065+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f86d80cdc10 con 0x7f86d8145a80 2024-04-24T23:27:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.066+0000 7f86d4ff9640 1 -- 172.21.15.26:0/4151069780 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1972 (secure 0 0 0) 0x7f86c0059460 con 0x7f86d8145a80 2024-04-24T23:27:01.067 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:01.067 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":390,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700753,"bytes_used":278085632,"bytes_avail":772816027648,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6824,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":94,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6299},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":6778},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-24T23:26:59.395553+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f86b806be30 msgr2=0x7f86b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f86b806be30 0x7f86b806e2f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f86c8034000 tx=0x7f86c802e000 comp rx=0 tx=0).stop 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d8145a80 msgr2=0x7f86d81f1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d8145a80 0x7f86d81f1e90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f86c000e890 tx=0x7f86c000ed50 comp rx=0 tx=0).stop 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 shutdown_connections 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f86b806be30 0x7f86b806e2f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f86d815ca40 0x7f86d8150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86d815c5b0 0x7f86d81508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 --2- 172.21.15.26:0/4151069780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d8145a80 0x7f86d81f1e90 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 >> 172.21.15.26:0/4151069780 conn(0x7f86d806b540 msgr2=0x7f86d8144a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 shutdown_connections 2024-04-24T23:27:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:01.068+0000 7f86de57e640 1 -- 172.21.15.26:0/4151069780 wait complete. 2024-04-24T23:27:01.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:27:02.080 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:02.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.193+0000 7f5e67d70640 1 Processor -- start 2024-04-24T23:27:02.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.193+0000 7f5e67d70640 1 -- start start 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e67d70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 0x7f5e600b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e67d70640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e600b4fd0 0x7f5e60072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e67d70640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5e600b0030 0x7f5e600b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e67d70640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5e6015e760 con 0x7f5e600b4fd0 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e67d70640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5e600725b0 con 0x7f5e600b0030 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e67d70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5e600b09a0 con 0x7f5e6015e380 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e6530b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e600b4fd0 0x7f5e60072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e6530b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e600b4fd0 0x7f5e60072a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52910/0 (socket says 172.21.15.26:52910) 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e6530b640 1 -- 172.21.15.26:0/3660639998 learned_addr learned my addr 172.21.15.26:0/3660639998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e65b0c640 1 --2- 172.21.15.26:0/3660639998 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 0x7f5e600b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.194+0000 7f5e6630d640 1 --2- 172.21.15.26:0/3660639998 >> v2:172.21.15.29:3300/0 conn(0x7f5e600b0030 0x7f5e600b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.195+0000 7f5e6530b640 1 -- 172.21.15.26:0/3660639998 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 msgr2=0x7f5e600b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.195+0000 7f5e6530b640 1 --2- 172.21.15.26:0/3660639998 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 0x7f5e600b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.195+0000 7f5e6530b640 1 -- 172.21.15.26:0/3660639998 >> v2:172.21.15.29:3300/0 conn(0x7f5e600b0030 msgr2=0x7f5e600b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.195+0000 7f5e6530b640 1 --2- 172.21.15.26:0/3660639998 >> v2:172.21.15.29:3300/0 conn(0x7f5e600b0030 0x7f5e600b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.195+0000 7f5e6530b640 1 -- 172.21.15.26:0/3660639998 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e600b5700 con 0x7f5e600b4fd0 2024-04-24T23:27:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.195+0000 7f5e6530b640 1 --2- 172.21.15.26:0/3660639998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 0x7f5e60072a40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f5e60072030 tx=0x7f5e5002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2644b3c2b7d92846 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.195+0000 7f5e64b0a640 1 -- 172.21.15.26:0/3660639998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50037070 con 0x7f5e600b4fd0 2024-04-24T23:27:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e64b0a640 1 -- 172.21.15.26:0/3660639998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e50032440 con 0x7f5e600b4fd0 2024-04-24T23:27:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e64b0a640 1 -- 172.21.15.26:0/3660639998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50032810 con 0x7f5e600b4fd0 2024-04-24T23:27:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e67d70640 1 -- 172.21.15.26:0/3660639998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 msgr2=0x7f5e60072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3660639998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 0x7f5e60072a40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f5e60072030 tx=0x7f5e5002a2a0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e67d70640 1 -- 172.21.15.26:0/3660639998 shutdown_connections 2024-04-24T23:27:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3660639998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 0x7f5e60072a40 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3660639998 >> v2:172.21.15.29:3300/0 conn(0x7f5e600b0030 0x7f5e600b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3660639998 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 0x7f5e600b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e67d70640 1 -- 172.21.15.26:0/3660639998 >> 172.21.15.26:0/3660639998 conn(0x7f5e6006e2c0 msgr2=0x7f5e6006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e67d70640 1 -- 172.21.15.26:0/3660639998 shutdown_connections 2024-04-24T23:27:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.196+0000 7f5e67d70640 1 -- 172.21.15.26:0/3660639998 wait complete. 2024-04-24T23:27:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.197+0000 7f5e67d70640 1 Processor -- start 2024-04-24T23:27:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.197+0000 7f5e67d70640 1 -- start start 2024-04-24T23:27:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e67d70640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e600b0030 0x7f5e6014d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e67d70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 0x7f5e601e8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e67d70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 0x7f5e601e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e67d70640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e600b5410 con 0x7f5e600b4fd0 2024-04-24T23:27:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e67d70640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60072240 con 0x7f5e600b0030 2024-04-24T23:27:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e67d70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5e60072f80 con 0x7f5e6015e380 2024-04-24T23:27:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e6530b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 0x7f5e601e8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e6530b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 0x7f5e601e8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52916/0 (socket says 172.21.15.26:52916) 2024-04-24T23:27:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e6530b640 1 -- 172.21.15.26:0/3950464821 learned_addr learned my addr 172.21.15.26:0/3950464821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e6630d640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 0x7f5e601e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.198+0000 7f5e65b0c640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e600b0030 0x7f5e6014d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e6530b640 1 -- 172.21.15.26:0/3950464821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 msgr2=0x7f5e601e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e6530b640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 0x7f5e601e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e6530b640 1 -- 172.21.15.26:0/3950464821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e600b0030 msgr2=0x7f5e6014d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e6530b640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e600b0030 0x7f5e6014d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e6530b640 1 -- 172.21.15.26:0/3950464821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e600d09d0 con 0x7f5e600b4fd0 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e6530b640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 0x7f5e601e8650 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5e50000c00 tx=0x7f5e500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e567fc640 1 -- 172.21.15.26:0/3950464821 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50041020 con 0x7f5e600b4fd0 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e567fc640 1 -- 172.21.15.26:0/3950464821 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5003d620 con 0x7f5e600b4fd0 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e567fc640 1 -- 172.21.15.26:0/3950464821 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50037040 con 0x7f5e600b4fd0 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e600ce450 con 0x7f5e600b4fd0 2024-04-24T23:27:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.199+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e600cea10 con 0x7f5e600b4fd0 2024-04-24T23:27:02.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.200+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e600694c0 con 0x7f5e600b4fd0 2024-04-24T23:27:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.201+0000 7f5e567fc640 1 -- 172.21.15.26:0/3950464821 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e50044020 con 0x7f5e600b4fd0 2024-04-24T23:27:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.201+0000 7f5e567fc640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e4406be30 0x7f5e4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.201+0000 7f5e567fc640 1 -- 172.21.15.26:0/3950464821 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f5e500ab530 con 0x7f5e600b4fd0 2024-04-24T23:27:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.201+0000 7f5e65b0c640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e4406be30 0x7f5e4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.202+0000 7f5e65b0c640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e4406be30 0x7f5e4406e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5e600d0500 tx=0x7f5e5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.206+0000 7f5e567fc640 1 -- 172.21.15.26:0/3950464821 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5e5004b030 con 0x7f5e600b4fd0 2024-04-24T23:27:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.428+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e600cebe0 con 0x7f5e600b4fd0 2024-04-24T23:27:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.428+0000 7f5e567fc640 1 -- 172.21.15.26:0/3950464821 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7f5e50079ea0 con 0x7f5e600b4fd0 2024-04-24T23:27:02.429 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:02.429 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:27:02.430 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:27:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e4406be30 msgr2=0x7f5e4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e4406be30 0x7f5e4406e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5e600d0500 tx=0x7f5e5c008040 comp rx=0 tx=0).stop 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 msgr2=0x7f5e601e8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 0x7f5e601e8650 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5e50000c00 tx=0x7f5e500046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 shutdown_connections 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5e4406be30 0x7f5e4406e2f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5e6015e380 0x7f5e601e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b4fd0 0x7f5e601e8650 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 --2- 172.21.15.26:0/3950464821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e600b0030 0x7f5e6014d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 >> 172.21.15.26:0/3950464821 conn(0x7f5e6006e2c0 msgr2=0x7f5e6006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 shutdown_connections 2024-04-24T23:27:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.430+0000 7f5e67d70640 1 -- 172.21.15.26:0/3950464821 wait complete. 2024-04-24T23:27:02.442 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'a', 'd'] 2024-04-24T23:27:02.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:27:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.541+0000 7f4576437640 1 Processor -- start 2024-04-24T23:27:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.541+0000 7f4576437640 1 -- start start 2024-04-24T23:27:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f4576437640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45700ce850 0x7f4570140700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f4576437640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4570140c40 0x7f45700711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f4576437640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45700cecd0 con 0x7f457013e090 2024-04-24T23:27:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f4576437640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4570155aa0 con 0x7f45700ce850 2024-04-24T23:27:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f4576437640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4570141070 con 0x7f4570140c40 2024-04-24T23:27:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f456f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45700ce850 0x7f4570140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f456ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f457013e090 0x7f457015bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40168/0 (socket says 172.21.15.26:40168) 2024-04-24T23:27:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f456ffff640 1 -- 172.21.15.26:0/1481435039 learned_addr learned my addr 172.21.15.26:0/1481435039 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f45749d4640 1 --2- 172.21.15.26:0/1481435039 >> v2:172.21.15.29:3300/0 conn(0x7f4570140c40 0x7f45700711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f456f7fe640 1 -- 172.21.15.26:0/1481435039 >> v2:172.21.15.29:3300/0 conn(0x7f4570140c40 msgr2=0x7f45700711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f456f7fe640 1 --2- 172.21.15.26:0/1481435039 >> v2:172.21.15.29:3300/0 conn(0x7f4570140c40 0x7f45700711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f456f7fe640 1 -- 172.21.15.26:0/1481435039 >> v1:172.21.15.26:6789/0 conn(0x7f457013e090 legacy=0x7f457015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f456f7fe640 1 -- 172.21.15.26:0/1481435039 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45700cef80 con 0x7f45700ce850 2024-04-24T23:27:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.542+0000 7f456effd640 1 -- 172.21.15.26:0/1481435039 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3770297522 0 0) 0x7f45700cecd0 con 0x7f457013e090 2024-04-24T23:27:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.543+0000 7f456f7fe640 1 --2- 172.21.15.26:0/1481435039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 0x7f4570140700 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4570072c60 tx=0x7f456002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a450f7c2ee32db75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.543+0000 7f456effd640 1 -- 172.21.15.26:0/1481435039 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560037070 con 0x7f45700ce850 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.543+0000 7f456effd640 1 -- 172.21.15.26:0/1481435039 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456002ae50 con 0x7f45700ce850 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.543+0000 7f456effd640 1 -- 172.21.15.26:0/1481435039 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560032660 con 0x7f45700ce850 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.543+0000 7f4576437640 1 -- 172.21.15.26:0/1481435039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 msgr2=0x7f4570140700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.543+0000 7f4576437640 1 --2- 172.21.15.26:0/1481435039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 0x7f4570140700 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4570072c60 tx=0x7f456002a140 comp rx=0 tx=0).stop 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.544+0000 7f4576437640 1 -- 172.21.15.26:0/1481435039 shutdown_connections 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.544+0000 7f4576437640 1 --2- 172.21.15.26:0/1481435039 >> v2:172.21.15.29:3300/0 conn(0x7f4570140c40 0x7f45700711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.544+0000 7f4576437640 1 --2- 172.21.15.26:0/1481435039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 0x7f4570140700 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.544+0000 7f4576437640 1 -- 172.21.15.26:0/1481435039 >> 172.21.15.26:0/1481435039 conn(0x7f457006b540 msgr2=0x7f457006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.544+0000 7f4576437640 1 -- 172.21.15.26:0/1481435039 shutdown_connections 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.544+0000 7f4576437640 1 -- 172.21.15.26:0/1481435039 wait complete. 2024-04-24T23:27:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.545+0000 7f4576437640 1 Processor -- start 2024-04-24T23:27:02.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.545+0000 7f4576437640 1 -- start start 2024-04-24T23:27:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f4576437640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 0x7f457013d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f4576437640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f457013e090 0x7f457013f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f4576437640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4570140c40 0x7f457013fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f4576437640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45700cf820 con 0x7f45700ce850 2024-04-24T23:27:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f4576437640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f45700cf540 con 0x7f4570140c40 2024-04-24T23:27:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f4576437640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f45700cfb00 con 0x7f457013e090 2024-04-24T23:27:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 0x7f457013d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 0x7f457013d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52936/0 (socket says 172.21.15.26:52936) 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456ffff640 1 -- 172.21.15.26:0/3624502831 learned_addr learned my addr 172.21.15.26:0/3624502831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f45749d4640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4570140c40 0x7f457013fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456f7fe640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f457013e090 0x7f457013f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456ffff640 1 -- 172.21.15.26:0/3624502831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f457013e090 msgr2=0x7f457013f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456ffff640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f457013e090 0x7f457013f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456ffff640 1 -- 172.21.15.26:0/3624502831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4570140c40 msgr2=0x7f457013fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456ffff640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4570140c40 0x7f457013fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456ffff640 1 -- 172.21.15.26:0/3624502831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457013dc20 con 0x7f45700ce850 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456ffff640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 0x7f457013d6e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f456400c430 tx=0x7f456400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.546+0000 7f456cff9640 1 -- 172.21.15.26:0/3624502831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45640074e0 con 0x7f45700ce850 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.547+0000 7f456cff9640 1 -- 172.21.15.26:0/3624502831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45640047e0 con 0x7f45700ce850 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.547+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4570141fd0 con 0x7f45700ce850 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.547+0000 7f456cff9640 1 -- 172.21.15.26:0/3624502831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564004bd0 con 0x7f45700ce850 2024-04-24T23:27:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.547+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701428f0 con 0x7f45700ce850 2024-04-24T23:27:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.548+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45700694c0 con 0x7f45700ce850 2024-04-24T23:27:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.548+0000 7f456cff9640 1 -- 172.21.15.26:0/3624502831 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4564011070 con 0x7f45700ce850 2024-04-24T23:27:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.549+0000 7f456cff9640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f455c06be30 0x7f455c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.549+0000 7f456cff9640 1 -- 172.21.15.26:0/3624502831 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f456408aab0 con 0x7f45700ce850 2024-04-24T23:27:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.549+0000 7f456f7fe640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f455c06be30 0x7f455c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.549+0000 7f456f7fe640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f455c06be30 0x7f455c06e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4560034000 tx=0x7f456002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.555+0000 7f456cff9640 1 -- 172.21.15.26:0/3624502831 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45640594b0 con 0x7f45700ce850 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.805+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f45700b2ff0 con 0x7f45700ce850 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.805+0000 7f456cff9640 1 -- 172.21.15.26:0/3624502831 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1984 (secure 0 0 0) 0x7f4564058db0 con 0x7f45700ce850 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":391,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700753,"bytes_used":278085632,"bytes_avail":772816027648,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6825,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":94,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6299},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":6778},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-24T23:27:01.395931+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f455c06be30 msgr2=0x7f455c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f455c06be30 0x7f455c06e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4560034000 tx=0x7f456002e000 comp rx=0 tx=0).stop 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 msgr2=0x7f457013d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 0x7f457013d6e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f456400c430 tx=0x7f456400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 shutdown_connections 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f455c06be30 0x7f455c06e2f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4570140c40 0x7f457013fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f457013e090 0x7f457013f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 --2- 172.21.15.26:0/3624502831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45700ce850 0x7f457013d6e0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.807+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 >> 172.21.15.26:0/3624502831 conn(0x7f457006b540 msgr2=0x7f45700710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.808+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 shutdown_connections 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:02.808+0000 7f4576437640 1 -- 172.21.15.26:0/3624502831 wait complete. 2024-04-24T23:27:02.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-04-24T23:27:03.822 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.934+0000 7f24a2316640 1 Processor -- start 2024-04-24T23:27:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.934+0000 7f24a2316640 1 -- start start 2024-04-24T23:27:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.935+0000 7f24a2316640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f249c06d350 0x7f249c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.935+0000 7f24a2316640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f249c0cd660 0x7f249c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.935+0000 7f24a2316640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f249c0cfbc0 con 0x7f249c145910 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.935+0000 7f24a2316640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f249c145cf0 con 0x7f249c0cd660 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.935+0000 7f24a2316640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f249c06d7f0 con 0x7f249c06d350 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.935+0000 7f24a08b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f249c0cd660 0x7f249c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.935+0000 7f249bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f249c145910 0x7f249c15d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59986/0 (socket says 172.21.15.26:59986) 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.935+0000 7f249bfff640 1 -- 172.21.15.26:0/3981670568 learned_addr learned my addr 172.21.15.26:0/3981670568 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.935+0000 7f249b7fe640 1 --2- 172.21.15.26:0/3981670568 >> v2:172.21.15.29:3300/0 conn(0x7f249c06d350 0x7f249c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.936+0000 7f24a08b3640 1 -- 172.21.15.26:0/3981670568 >> v2:172.21.15.29:3300/0 conn(0x7f249c06d350 msgr2=0x7f249c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.936+0000 7f24a08b3640 1 --2- 172.21.15.26:0/3981670568 >> v2:172.21.15.29:3300/0 conn(0x7f249c06d350 0x7f249c06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.936+0000 7f24a08b3640 1 -- 172.21.15.26:0/3981670568 >> v1:172.21.15.26:6789/0 conn(0x7f249c145910 legacy=0x7f249c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.936+0000 7f24a08b3640 1 -- 172.21.15.26:0/3981670568 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249c06dad0 con 0x7f249c0cd660 2024-04-24T23:27:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.936+0000 7f249affd640 1 -- 172.21.15.26:0/3981670568 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1361285136 0 0) 0x7f249c0cfbc0 con 0x7f249c145910 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.936+0000 7f24a08b3640 1 --2- 172.21.15.26:0/3981670568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0cd660 0x7f249c0cdaf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f2494009d90 tx=0x7f249402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0de5d4cea079ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f249affd640 1 -- 172.21.15.26:0/3981670568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494031600 con 0x7f249c0cd660 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f249affd640 1 -- 172.21.15.26:0/3981670568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24940317a0 con 0x7f249c0cd660 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f249affd640 1 -- 172.21.15.26:0/3981670568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494031b70 con 0x7f249c0cd660 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f24a2316640 1 -- 172.21.15.26:0/3981670568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0cd660 msgr2=0x7f249c0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f24a2316640 1 --2- 172.21.15.26:0/3981670568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0cd660 0x7f249c0cdaf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f2494009d90 tx=0x7f249402a610 comp rx=0 tx=0).stop 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f24a2316640 1 -- 172.21.15.26:0/3981670568 shutdown_connections 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f24a2316640 1 --2- 172.21.15.26:0/3981670568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c0cd660 0x7f249c0cdaf0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f24a2316640 1 --2- 172.21.15.26:0/3981670568 >> v2:172.21.15.29:3300/0 conn(0x7f249c06d350 0x7f249c06e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f24a2316640 1 -- 172.21.15.26:0/3981670568 >> 172.21.15.26:0/3981670568 conn(0x7f249c06b300 msgr2=0x7f249c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f24a2316640 1 -- 172.21.15.26:0/3981670568 shutdown_connections 2024-04-24T23:27:03.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.937+0000 7f24a2316640 1 -- 172.21.15.26:0/3981670568 wait complete. 2024-04-24T23:27:03.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.938+0000 7f24a2316640 1 Processor -- start 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.939+0000 7f24a2316640 1 -- start start 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.939+0000 7f24a2316640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c06d350 0x7f249c0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.939+0000 7f24a2316640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f249c0cd660 0x7f249c13ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.939+0000 7f24a2316640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f249c145910 0x7f249c0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.939+0000 7f24a2316640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f249c06e370 con 0x7f249c06d350 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.939+0000 7f24a2316640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f249c06e090 con 0x7f249c0cd660 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.939+0000 7f24a2316640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f249c15d770 con 0x7f249c145910 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c06d350 0x7f249c0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c06d350 0x7f249c0b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48530/0 (socket says 172.21.15.26:48530) 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249bfff640 1 -- 172.21.15.26:0/159002104 learned_addr learned my addr 172.21.15.26:0/159002104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249b7fe640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f249c0cd660 0x7f249c13ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:03.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f24a08b3640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f249c145910 0x7f249c0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249bfff640 1 -- 172.21.15.26:0/159002104 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f249c145910 msgr2=0x7f249c0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249bfff640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f249c145910 0x7f249c0b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249bfff640 1 -- 172.21.15.26:0/159002104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f249c0cd660 msgr2=0x7f249c13ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249bfff640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f249c0cd660 0x7f249c13ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249bfff640 1 -- 172.21.15.26:0/159002104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249c13ccf0 con 0x7f249c06d350 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f249bfff640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c06d350 0x7f249c0b2f80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c430 tx=0x7f248c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f2498ff9640 1 -- 172.21.15.26:0/159002104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c0074e0 con 0x7f249c06d350 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f2498ff9640 1 -- 172.21.15.26:0/159002104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f248c0047e0 con 0x7f249c06d350 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f2498ff9640 1 -- 172.21.15.26:0/159002104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c004bd0 con 0x7f249c06d350 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249c0b36e0 con 0x7f249c06d350 2024-04-24T23:27:03.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.940+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f249c0b3f80 con 0x7f249c06d350 2024-04-24T23:27:03.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.941+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249c0694c0 con 0x7f249c06d350 2024-04-24T23:27:03.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.942+0000 7f2498ff9640 1 -- 172.21.15.26:0/159002104 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f248c011070 con 0x7f249c06d350 2024-04-24T23:27:03.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.942+0000 7f2498ff9640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f248806be30 0x7f248806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:03.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.942+0000 7f2498ff9640 1 -- 172.21.15.26:0/159002104 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f248c08aa80 con 0x7f249c06d350 2024-04-24T23:27:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.942+0000 7f249b7fe640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f248806be30 0x7f248806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:03.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.943+0000 7f249b7fe640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f248806be30 0x7f248806e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f249c15ed80 tx=0x7f2490008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:03.947+0000 7f2498ff9640 1 -- 172.21.15.26:0/159002104 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f248c059480 con 0x7f249c06d350 2024-04-24T23:27:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.168+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f249c06b930 con 0x7f249c06d350 2024-04-24T23:27:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.168+0000 7f2498ff9640 1 -- 172.21.15.26:0/159002104 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7f248c058d80 con 0x7f249c06d350 2024-04-24T23:27:04.169 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:04.169 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:27:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:27:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f248806be30 msgr2=0x7f248806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f248806be30 0x7f248806e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f249c15ed80 tx=0x7f2490008040 comp rx=0 tx=0).stop 2024-04-24T23:27:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c06d350 msgr2=0x7f249c0b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c06d350 0x7f249c0b2f80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c430 tx=0x7f248c00c8f0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 shutdown_connections 2024-04-24T23:27:04.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f248806be30 0x7f248806e2f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f249c145910 0x7f249c0b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f249c0cd660 0x7f249c13ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 --2- 172.21.15.26:0/159002104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f249c06d350 0x7f249c0b2f80 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 >> 172.21.15.26:0/159002104 conn(0x7f249c06b300 msgr2=0x7f249c15eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:04.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 shutdown_connections 2024-04-24T23:27:04.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.170+0000 7f24a2316640 1 -- 172.21.15.26:0/159002104 wait complete. 2024-04-24T23:27:04.182 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'a', 'd'] 2024-04-24T23:27:04.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:27:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.281+0000 7f24915d5640 1 Processor -- start 2024-04-24T23:27:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.281+0000 7f24915d5640 1 -- start start 2024-04-24T23:27:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f24915d5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 0x7f248c0d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f24915d5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f248c06e010 0x7f248c15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f24915d5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f248c06f740 con 0x7f248c06dc30 2024-04-24T23:27:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f24915d5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f248c06fa20 con 0x7f248c06e010 2024-04-24T23:27:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f24915d5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f248c06fd00 con 0x7f248c09b110 2024-04-24T23:27:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f248b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f248c06e010 0x7f248c15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f248a7fc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 0x7f248c0d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f248affd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f248c06dc30 0x7f248c09aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:37776/0 (socket says 172.21.15.26:37776) 2024-04-24T23:27:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f248affd640 1 -- 172.21.15.26:0/2982983428 learned_addr learned my addr 172.21.15.26:0/2982983428 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.282+0000 7f248b7fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f248c06e010 0x7f248c15dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:57848/0 (socket says 172.21.15.26:57848) 2024-04-24T23:27:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.283+0000 7f248a7fc640 1 -- 172.21.15.26:0/2982983428 >> v2:172.21.15.29:3300/0 conn(0x7f248c06e010 msgr2=0x7f248c15dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.283+0000 7f248a7fc640 1 --2- 172.21.15.26:0/2982983428 >> v2:172.21.15.29:3300/0 conn(0x7f248c06e010 0x7f248c15dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.283+0000 7f248a7fc640 1 -- 172.21.15.26:0/2982983428 >> v1:172.21.15.29:6789/0 conn(0x7f248c06dc30 legacy=0x7f248c09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.283+0000 7f2489ffb640 1 -- 172.21.15.26:0/2982983428 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 667437857 0 0) 0x7f248c06f740 con 0x7f248c06dc30 2024-04-24T23:27:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.283+0000 7f248a7fc640 1 -- 172.21.15.26:0/2982983428 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c06ffe0 con 0x7f248c09b110 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.283+0000 7f248b7fe640 1 --2- 172.21.15.26:0/2982983428 >> v2:172.21.15.29:3300/0 conn(0x7f248c06e010 0x7f248c15dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f248a7fc640 1 --2- 172.21.15.26:0/2982983428 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 0x7f248c0d1180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24740097a0 tx=0x7f247402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd5cc99d383128 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f2489ffb640 1 -- 172.21.15.26:0/2982983428 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474037070 con 0x7f248c09b110 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f2489ffb640 1 -- 172.21.15.26:0/2982983428 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2474032440 con 0x7f248c09b110 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f2489ffb640 1 -- 172.21.15.26:0/2982983428 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474032810 con 0x7f248c09b110 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f24915d5640 1 -- 172.21.15.26:0/2982983428 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 msgr2=0x7f248c0d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f24915d5640 1 --2- 172.21.15.26:0/2982983428 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 0x7f248c0d1180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24740097a0 tx=0x7f247402a180 comp rx=0 tx=0).stop 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f24915d5640 1 -- 172.21.15.26:0/2982983428 shutdown_connections 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f24915d5640 1 --2- 172.21.15.26:0/2982983428 >> v2:172.21.15.29:3300/0 conn(0x7f248c06e010 0x7f248c15dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f24915d5640 1 --2- 172.21.15.26:0/2982983428 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 0x7f248c0d1180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f24915d5640 1 -- 172.21.15.26:0/2982983428 >> 172.21.15.26:0/2982983428 conn(0x7f248c0b3a60 msgr2=0x7f248c0b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f24915d5640 1 -- 172.21.15.26:0/2982983428 shutdown_connections 2024-04-24T23:27:04.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.284+0000 7f24915d5640 1 -- 172.21.15.26:0/2982983428 wait complete. 2024-04-24T23:27:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f24915d5640 1 Processor -- start 2024-04-24T23:27:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f24915d5640 1 -- start start 2024-04-24T23:27:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f24915d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248c06dc30 0x7f248c151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f24915d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f248c06e010 0x7f248c0b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f24915d5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 0x7f248c0b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f24915d5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f248c06f1a0 con 0x7f248c06dc30 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f24915d5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f248c0d0570 con 0x7f248c06e010 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f24915d5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f248c06f320 con 0x7f248c09b110 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f248affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248c06dc30 0x7f248c151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f248affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248c06dc30 0x7f248c151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48546/0 (socket says 172.21.15.26:48546) 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.286+0000 7f248affd640 1 -- 172.21.15.26:0/1902257300 learned_addr learned my addr 172.21.15.26:0/1902257300 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f248a7fc640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f248c06e010 0x7f248c0b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f248b7fe640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 0x7f248c0b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f248affd640 1 -- 172.21.15.26:0/1902257300 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 msgr2=0x7f248c0b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f248affd640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 0x7f248c0b70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f248affd640 1 -- 172.21.15.26:0/1902257300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f248c06e010 msgr2=0x7f248c0b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f248affd640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f248c06e010 0x7f248c0b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f248affd640 1 -- 172.21.15.26:0/1902257300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c0b5cb0 con 0x7f248c06dc30 2024-04-24T23:27:04.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f248affd640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248c06dc30 0x7f248c151d90 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f247c00c7c0 tx=0x7f247c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:04.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f24837fe640 1 -- 172.21.15.26:0/1902257300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c018070 con 0x7f248c06dc30 2024-04-24T23:27:04.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f24837fe640 1 -- 172.21.15.26:0/1902257300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c00ce10 con 0x7f248c06dc30 2024-04-24T23:27:04.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c0b5f90 con 0x7f248c06dc30 2024-04-24T23:27:04.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f24837fe640 1 -- 172.21.15.26:0/1902257300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c00a880 con 0x7f248c06dc30 2024-04-24T23:27:04.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.287+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c0c9290 con 0x7f248c06dc30 2024-04-24T23:27:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.288+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c0694c0 con 0x7f248c06dc30 2024-04-24T23:27:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.289+0000 7f24837fe640 1 -- 172.21.15.26:0/1902257300 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f247c01b070 con 0x7f248c06dc30 2024-04-24T23:27:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.289+0000 7f24837fe640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be30 0x7f246c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.289+0000 7f24837fe640 1 -- 172.21.15.26:0/1902257300 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f247c08a360 con 0x7f248c06dc30 2024-04-24T23:27:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.289+0000 7f248a7fc640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be30 0x7f246c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.290+0000 7f248a7fc640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be30 0x7f246c06e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2474034000 tx=0x7f247402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:04.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.294+0000 7f24837fe640 1 -- 172.21.15.26:0/1902257300 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f247c058710 con 0x7f248c06dc30 2024-04-24T23:27:04.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.550+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f248c0d1ab0 con 0x7f248c06dc30 2024-04-24T23:27:04.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.551+0000 7f24837fe640 1 -- 172.21.15.26:0/1902257300 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1984 (secure 0 0 0) 0x7f247c05ca10 con 0x7f248c06dc30 2024-04-24T23:27:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:04.552 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":393,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706268,"bytes_used":278093824,"bytes_avail":772816019456,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7507,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":94,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6299},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":6778},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-24T23:27:01.395931+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be30 msgr2=0x7f246c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be30 0x7f246c06e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2474034000 tx=0x7f247402e000 comp rx=0 tx=0).stop 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248c06dc30 msgr2=0x7f248c151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248c06dc30 0x7f248c151d90 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f247c00c7c0 tx=0x7f247c00cbc0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 shutdown_connections 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be30 0x7f246c06e2f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f248c09b110 0x7f248c0b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f248c06e010 0x7f248c0b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 --2- 172.21.15.26:0/1902257300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248c06dc30 0x7f248c151d90 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 >> 172.21.15.26:0/1902257300 conn(0x7f248c0b3a60 msgr2=0x7f248c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 shutdown_connections 2024-04-24T23:27:04.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:04.553+0000 7f24915d5640 1 -- 172.21.15.26:0/1902257300 wait complete. 2024-04-24T23:27:04.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-04-24T23:27:05.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.678+0000 7f1aa91e7640 1 Processor -- start 2024-04-24T23:27:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.678+0000 7f1aa91e7640 1 -- start start 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.678+0000 7f1aa91e7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1aa40cf870 0x7f1aa4072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.678+0000 7f1aa91e7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1aa415eb00 0x7f1aa415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.678+0000 7f1aa91e7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa415f450 con 0x7f1aa4143c40 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.678+0000 7f1aa91e7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa4071b70 con 0x7f1aa40cf870 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.678+0000 7f1aa91e7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa40cd0b0 con 0x7f1aa415eb00 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.679+0000 7f1a9bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1aa40cf870 0x7f1aa4072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.679+0000 7f1a9bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1aa40cf870 0x7f1aa4072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48560/0 (socket says 172.21.15.26:48560) 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.679+0000 7f1a9bfff640 1 -- 172.21.15.26:0/1940645532 learned_addr learned my addr 172.21.15.26:0/1940645532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.679+0000 7f1aa359e640 1 --2- 172.21.15.26:0/1940645532 >> v2:172.21.15.29:3300/0 conn(0x7f1aa415eb00 0x7f1aa415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.679+0000 7f1a9bfff640 1 -- 172.21.15.26:0/1940645532 >> v2:172.21.15.29:3300/0 conn(0x7f1aa415eb00 msgr2=0x7f1aa415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.679+0000 7f1a9bfff640 1 --2- 172.21.15.26:0/1940645532 >> v2:172.21.15.29:3300/0 conn(0x7f1aa415eb00 0x7f1aa415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.679+0000 7f1a9bfff640 1 -- 172.21.15.26:0/1940645532 >> v1:172.21.15.29:6789/0 conn(0x7f1aa4143c40 legacy=0x7f1aa40cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.679+0000 7f1a9bfff640 1 -- 172.21.15.26:0/1940645532 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa4071240 con 0x7f1aa40cf870 2024-04-24T23:27:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1a9bfff640 1 --2- 172.21.15.26:0/1940645532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa40cf870 0x7f1aa4072000 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4144d10 tx=0x7f1a8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130630a7aab01c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1aa259c640 1 -- 172.21.15.26:0/1940645532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c037070 con 0x7f1aa40cf870 2024-04-24T23:27:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1aa259c640 1 -- 172.21.15.26:0/1940645532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c032440 con 0x7f1aa40cf870 2024-04-24T23:27:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1aa259c640 1 -- 172.21.15.26:0/1940645532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c032810 con 0x7f1aa40cf870 2024-04-24T23:27:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1aa91e7640 1 -- 172.21.15.26:0/1940645532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa40cf870 msgr2=0x7f1aa4072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1aa91e7640 1 --2- 172.21.15.26:0/1940645532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa40cf870 0x7f1aa4072000 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4144d10 tx=0x7f1a8c02a270 comp rx=0 tx=0).stop 2024-04-24T23:27:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1aa91e7640 1 -- 172.21.15.26:0/1940645532 shutdown_connections 2024-04-24T23:27:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1aa91e7640 1 --2- 172.21.15.26:0/1940645532 >> v2:172.21.15.29:3300/0 conn(0x7f1aa415eb00 0x7f1aa415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1aa91e7640 1 --2- 172.21.15.26:0/1940645532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa40cf870 0x7f1aa4072000 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.680+0000 7f1aa91e7640 1 -- 172.21.15.26:0/1940645532 >> 172.21.15.26:0/1940645532 conn(0x7f1aa406b300 msgr2=0x7f1aa406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.681+0000 7f1aa91e7640 1 -- 172.21.15.26:0/1940645532 shutdown_connections 2024-04-24T23:27:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.681+0000 7f1aa91e7640 1 -- 172.21.15.26:0/1940645532 wait complete. 2024-04-24T23:27:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.682+0000 7f1aa91e7640 1 Processor -- start 2024-04-24T23:27:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.682+0000 7f1aa91e7640 1 -- start start 2024-04-24T23:27:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1aa91e7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1aa40cf870 0x7f1aa41318a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1aa91e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa4143c40 0x7f1aa415e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1aa91e7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa415eb00 0x7f1aa413d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1aa91e7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4144020 con 0x7f1aa4143c40 2024-04-24T23:27:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1aa91e7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4071800 con 0x7f1aa415eb00 2024-04-24T23:27:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1aa91e7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa4072540 con 0x7f1aa40cf870 2024-04-24T23:27:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1a9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa4143c40 0x7f1aa415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1a9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa4143c40 0x7f1aa415e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48562/0 (socket says 172.21.15.26:48562) 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1a9bfff640 1 -- 172.21.15.26:0/2470763227 learned_addr learned my addr 172.21.15.26:0/2470763227 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1aa2d9d640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1aa40cf870 0x7f1aa41318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1aa359e640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa415eb00 0x7f1aa413d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1a9bfff640 1 -- 172.21.15.26:0/2470763227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1aa40cf870 msgr2=0x7f1aa41318a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1a9bfff640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1aa40cf870 0x7f1aa41318a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1a9bfff640 1 -- 172.21.15.26:0/2470763227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa415eb00 msgr2=0x7f1aa413d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1a9bfff640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa415eb00 0x7f1aa413d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1a9bfff640 1 -- 172.21.15.26:0/2470763227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa413dc40 con 0x7f1aa4143c40 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.683+0000 7f1a9bfff640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa4143c40 0x7f1aa415e390 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c000c00 tx=0x7f1a8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.684+0000 7f1a9b7fe640 1 -- 172.21.15.26:0/2470763227 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c037040 con 0x7f1aa4143c40 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.684+0000 7f1a9b7fe640 1 -- 172.21.15.26:0/2470763227 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c042070 con 0x7f1aa4143c40 2024-04-24T23:27:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.684+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa41327d0 con 0x7f1aa4143c40 2024-04-24T23:27:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.684+0000 7f1a9b7fe640 1 -- 172.21.15.26:0/2470763227 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c030880 con 0x7f1aa4143c40 2024-04-24T23:27:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.684+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa4132210 con 0x7f1aa4143c40 2024-04-24T23:27:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.685+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa40694c0 con 0x7f1aa4143c40 2024-04-24T23:27:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.685+0000 7f1a9b7fe640 1 -- 172.21.15.26:0/2470763227 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a8c030a20 con 0x7f1aa4143c40 2024-04-24T23:27:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.685+0000 7f1a9b7fe640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a8406be30 0x7f1a8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.686+0000 7f1a9b7fe640 1 -- 172.21.15.26:0/2470763227 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f1a8c0ab570 con 0x7f1aa4143c40 2024-04-24T23:27:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.686+0000 7f1aa2d9d640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a8406be30 0x7f1a8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.686+0000 7f1aa2d9d640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a8406be30 0x7f1a8406e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40ccf40 tx=0x7f1a94009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.690+0000 7f1a9b7fe640 1 -- 172.21.15.26:0/2470763227 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a8c079920 con 0x7f1aa4143c40 2024-04-24T23:27:05.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.911+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1aa40b4d20 con 0x7f1aa4143c40 2024-04-24T23:27:05.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.912+0000 7f1a9b7fe640 1 -- 172.21.15.26:0/2470763227 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7f1a8c07dc20 con 0x7f1aa4143c40 2024-04-24T23:27:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:05.913 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:27:05.913 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a8406be30 msgr2=0x7f1a8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a8406be30 0x7f1a8406e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40ccf40 tx=0x7f1a94009040 comp rx=0 tx=0).stop 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa4143c40 msgr2=0x7f1aa415e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa4143c40 0x7f1aa415e390 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c000c00 tx=0x7f1a8c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 shutdown_connections 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1a8406be30 0x7f1a8406e2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1aa415eb00 0x7f1aa413d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1aa4143c40 0x7f1aa415e390 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 --2- 172.21.15.26:0/2470763227 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1aa40cf870 0x7f1aa41318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 >> 172.21.15.26:0/2470763227 conn(0x7f1aa406b300 msgr2=0x7f1aa413f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 shutdown_connections 2024-04-24T23:27:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:05.914+0000 7f1aa91e7640 1 -- 172.21.15.26:0/2470763227 wait complete. 2024-04-24T23:27:05.925 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'a', 'd'] 2024-04-24T23:27:05.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.025+0000 7f549a3ce640 1 Processor -- start 2024-04-24T23:27:06.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.025+0000 7f549a3ce640 1 -- start start 2024-04-24T23:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.025+0000 7f549a3ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f549413e090 0x7f54940ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.025+0000 7f549a3ce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f549415bfe0 0x7f5494140700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f549a3ce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54940cef80 con 0x7f5494140c40 2024-04-24T23:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f549a3ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5494155aa0 con 0x7f549413e090 2024-04-24T23:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f549a3ce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5494141070 con 0x7f549415bfe0 2024-04-24T23:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f5493fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f549413e090 0x7f54940ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f5493fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f549413e090 0x7f54940ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48570/0 (socket says 172.21.15.26:48570) 2024-04-24T23:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f5493fff640 1 -- 172.21.15.26:0/1648445882 learned_addr learned my addr 172.21.15.26:0/1648445882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:06.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f54937fe640 1 --2- 172.21.15.26:0/1648445882 >> v2:172.21.15.29:3300/0 conn(0x7f549415bfe0 0x7f5494140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f5493fff640 1 -- 172.21.15.26:0/1648445882 >> v2:172.21.15.29:3300/0 conn(0x7f549415bfe0 msgr2=0x7f5494140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:06.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f5493fff640 1 --2- 172.21.15.26:0/1648445882 >> v2:172.21.15.29:3300/0 conn(0x7f549415bfe0 0x7f5494140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f5493fff640 1 -- 172.21.15.26:0/1648445882 >> v1:172.21.15.29:6789/0 conn(0x7f5494140c40 legacy=0x7f549415eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:06.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.026+0000 7f5493fff640 1 -- 172.21.15.26:0/1648445882 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54940cf260 con 0x7f549413e090 2024-04-24T23:27:06.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f5493fff640 1 --2- 172.21.15.26:0/1648445882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549413e090 0x7f54940ce850 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f5494067930 tx=0x7f548402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d3a37145d36e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f5492ffd640 1 -- 172.21.15.26:0/1648445882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484037070 con 0x7f549413e090 2024-04-24T23:27:06.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f5492ffd640 1 -- 172.21.15.26:0/1648445882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5484032440 con 0x7f549413e090 2024-04-24T23:27:06.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f5492ffd640 1 -- 172.21.15.26:0/1648445882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484032810 con 0x7f549413e090 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f549a3ce640 1 -- 172.21.15.26:0/1648445882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549413e090 msgr2=0x7f54940ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f549a3ce640 1 --2- 172.21.15.26:0/1648445882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549413e090 0x7f54940ce850 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f5494067930 tx=0x7f548402a210 comp rx=0 tx=0).stop 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f549a3ce640 1 -- 172.21.15.26:0/1648445882 shutdown_connections 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f549a3ce640 1 --2- 172.21.15.26:0/1648445882 >> v2:172.21.15.29:3300/0 conn(0x7f549415bfe0 0x7f5494140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f549a3ce640 1 --2- 172.21.15.26:0/1648445882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549413e090 0x7f54940ce850 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.027+0000 7f549a3ce640 1 -- 172.21.15.26:0/1648445882 >> 172.21.15.26:0/1648445882 conn(0x7f549406b540 msgr2=0x7f549406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.028+0000 7f549a3ce640 1 -- 172.21.15.26:0/1648445882 shutdown_connections 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.028+0000 7f549a3ce640 1 -- 172.21.15.26:0/1648445882 wait complete. 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.029+0000 7f549a3ce640 1 Processor -- start 2024-04-24T23:27:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.029+0000 7f549a3ce640 1 -- start start 2024-04-24T23:27:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.029+0000 7f549a3ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549413e090 0x7f549413e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.029+0000 7f549a3ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5494140c40 0x7f549413fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f549a3ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f549415bfe0 0x7f54940b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f549a3ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f54940cfb00 con 0x7f5494140c40 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f549a3ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f54940cf820 con 0x7f549413e090 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f549a3ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f549415c460 con 0x7f549415bfe0 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f54937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5494140c40 0x7f549413fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f54937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5494140c40 0x7f549413fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48586/0 (socket says 172.21.15.26:48586) 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f54937fe640 1 -- 172.21.15.26:0/2697354987 learned_addr learned my addr 172.21.15.26:0/2697354987 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f5493fff640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549413e090 0x7f549413e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f549896b640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f549415bfe0 0x7f54940b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f54937fe640 1 -- 172.21.15.26:0/2697354987 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f549415bfe0 msgr2=0x7f54940b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f54937fe640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f549415bfe0 0x7f54940b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f54937fe640 1 -- 172.21.15.26:0/2697354987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549413e090 msgr2=0x7f549413e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f54937fe640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549413e090 0x7f549413e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f54937fe640 1 -- 172.21.15.26:0/2697354987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5494142460 con 0x7f5494140c40 2024-04-24T23:27:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f5493fff640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549413e090 0x7f549413e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f54937fe640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5494140c40 0x7f549413fb80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f548800d800 tx=0x7f548800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f5490ff9640 1 -- 172.21.15.26:0/2697354987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548800b6c0 con 0x7f5494140c40 2024-04-24T23:27:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f5490ff9640 1 -- 172.21.15.26:0/2697354987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5488019070 con 0x7f5494140c40 2024-04-24T23:27:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549413eae0 con 0x7f5494140c40 2024-04-24T23:27:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f5490ff9640 1 -- 172.21.15.26:0/2697354987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548800ba80 con 0x7f5494140c40 2024-04-24T23:27:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.030+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549413f380 con 0x7f5494140c40 2024-04-24T23:27:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.031+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54940694c0 con 0x7f5494140c40 2024-04-24T23:27:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.032+0000 7f5490ff9640 1 -- 172.21.15.26:0/2697354987 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f548800bc20 con 0x7f5494140c40 2024-04-24T23:27:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.032+0000 7f5490ff9640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f548006be30 0x7f548006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.032+0000 7f5490ff9640 1 -- 172.21.15.26:0/2697354987 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f548808b3e0 con 0x7f5494140c40 2024-04-24T23:27:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.032+0000 7f5493fff640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f548006be30 0x7f548006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.033+0000 7f5493fff640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f548006be30 0x7f548006e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5484034000 tx=0x7f548402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.037+0000 7f5490ff9640 1 -- 172.21.15.26:0/2697354987 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5488059cc0 con 0x7f5494140c40 2024-04-24T23:27:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.294+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f54940b2ff0 con 0x7f5494140c40 2024-04-24T23:27:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.294+0000 7f5490ff9640 1 -- 172.21.15.26:0/2697354987 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1870 (secure 0 0 0) 0x7f54880595c0 con 0x7f5494140c40 2024-04-24T23:27:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:06.295 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":395,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706268,"bytes_used":278093824,"bytes_avail":772816019456,"bytes_total":773094113280,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":94,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6299},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":6778},{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6790}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-24T23:27:05.396817+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:27:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f548006be30 msgr2=0x7f548006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f548006be30 0x7f548006e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5484034000 tx=0x7f548402e000 comp rx=0 tx=0).stop 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5494140c40 msgr2=0x7f549413fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5494140c40 0x7f549413fb80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f548800d800 tx=0x7f548800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 shutdown_connections 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f548006be30 0x7f548006e2f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f549415bfe0 0x7f54940b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5494140c40 0x7f549413fb80 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 --2- 172.21.15.26:0/2697354987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549413e090 0x7f549413e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.296+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 >> 172.21.15.26:0/2697354987 conn(0x7f549406b540 msgr2=0x7f5494071940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.297+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 shutdown_connections 2024-04-24T23:27:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.297+0000 7f549a3ce640 1 -- 172.21.15.26:0/2697354987 wait complete. 2024-04-24T23:27:06.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/d1/d2/d3]'} 2024-04-24T23:27:06.309 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:27:06.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:27:06.352 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:27:06.411 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:27:06.411 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:27:06.411 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:27:06.411 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:06.427 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:27:06.427 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:06.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.623+0000 7fdc2cb34640 1 Processor -- start 2024-04-24T23:27:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.623+0000 7fdc2cb34640 1 -- start start 2024-04-24T23:27:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.624+0000 7fdc2cb34640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdc2815e570 0x7fdc280d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.624+0000 7fdc2cb34640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc280d22e0 0x7fdc280d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.624+0000 7fdc2cb34640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc280d32f0 con 0x7fdc280b3130 2024-04-24T23:27:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.624+0000 7fdc2cb34640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdc280d1ab0 con 0x7fdc2815e570 2024-04-24T23:27:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.624+0000 7fdc2cb34640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdc280d2ae0 con 0x7fdc280d22e0 2024-04-24T23:27:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc2659c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdc2815e570 0x7fdc280d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc26d9d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc280d22e0 0x7fdc280d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc25d9b640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7fdc280b3130 0x7fdc280d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:37814/0 (socket says 172.21.15.26:37814) 2024-04-24T23:27:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc2659c640 1 --2- 172.21.15.26:0/1010950612 >> v2:172.21.15.29:3300/0 conn(0x7fdc2815e570 0x7fdc280d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:57874/0 (socket says 172.21.15.26:57874) 2024-04-24T23:27:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc25d9b640 1 -- 172.21.15.26:0/1010950612 learned_addr learned my addr 172.21.15.26:0/1010950612 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc2659c640 1 -- 172.21.15.26:0/1010950612 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc280d22e0 msgr2=0x7fdc280d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc2659c640 1 --2- 172.21.15.26:0/1010950612 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc280d22e0 0x7fdc280d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc2659c640 1 -- 172.21.15.26:0/1010950612 >> v1:172.21.15.29:6789/0 conn(0x7fdc280b3130 legacy=0x7fdc280d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc2659c640 1 -- 172.21.15.26:0/1010950612 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc280d0270 con 0x7fdc2815e570 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.625+0000 7fdc26d9d640 1 --2- 172.21.15.26:0/1010950612 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc280d22e0 0x7fdc280d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.626+0000 7fdc2659c640 1 --2- 172.21.15.26:0/1010950612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2815e570 0x7fdc280d4160 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdc200097a0 tx=0x7fdc2002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc910aebfac360b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.626+0000 7fdc2559a640 1 -- 172.21.15.26:0/1010950612 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20037070 con 0x7fdc2815e570 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.626+0000 7fdc2559a640 1 -- 172.21.15.26:0/1010950612 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc20032440 con 0x7fdc2815e570 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.626+0000 7fdc2559a640 1 -- 172.21.15.26:0/1010950612 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20032810 con 0x7fdc2815e570 2024-04-24T23:27:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.626+0000 7fdc2cb34640 1 -- 172.21.15.26:0/1010950612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2815e570 msgr2=0x7fdc280d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.627+0000 7fdc2cb34640 1 --2- 172.21.15.26:0/1010950612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2815e570 0x7fdc280d4160 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdc200097a0 tx=0x7fdc2002a180 comp rx=0 tx=0).stop 2024-04-24T23:27:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.627+0000 7fdc2cb34640 1 -- 172.21.15.26:0/1010950612 shutdown_connections 2024-04-24T23:27:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.627+0000 7fdc2cb34640 1 --2- 172.21.15.26:0/1010950612 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc280d22e0 0x7fdc280d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.627+0000 7fdc2cb34640 1 --2- 172.21.15.26:0/1010950612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc2815e570 0x7fdc280d4160 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.627+0000 7fdc2cb34640 1 -- 172.21.15.26:0/1010950612 >> 172.21.15.26:0/1010950612 conn(0x7fdc2808dbe0 msgr2=0x7fdc2809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.627+0000 7fdc2cb34640 1 -- 172.21.15.26:0/1010950612 shutdown_connections 2024-04-24T23:27:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.627+0000 7fdc2cb34640 1 -- 172.21.15.26:0/1010950612 wait complete. 2024-04-24T23:27:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.628+0000 7fdc2cb34640 1 Processor -- start 2024-04-24T23:27:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.628+0000 7fdc2cb34640 1 -- start start 2024-04-24T23:27:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2cb34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc280b3130 0x7fdc281526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2cb34640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc280d22e0 0x7fdc280b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2cb34640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc2815e570 0x7fdc280b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2cb34640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280cfa70 con 0x7fdc280b3130 2024-04-24T23:27:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2cb34640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280d3010 con 0x7fdc280d22e0 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2cb34640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdc280cfd50 con 0x7fdc2815e570 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc280b3130 0x7fdc281526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc280b3130 0x7fdc281526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48588/0 (socket says 172.21.15.26:48588) 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2659c640 1 -- 172.21.15.26:0/269507948 learned_addr learned my addr 172.21.15.26:0/269507948 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc25d9b640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc280d22e0 0x7fdc280b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2659c640 1 -- 172.21.15.26:0/269507948 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc2815e570 msgr2=0x7fdc280b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2659c640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc2815e570 0x7fdc280b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2659c640 1 -- 172.21.15.26:0/269507948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc280d22e0 msgr2=0x7fdc280b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2659c640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc280d22e0 0x7fdc280b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2659c640 1 -- 172.21.15.26:0/269507948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc281f7b40 con 0x7fdc280b3130 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2659c640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc280b3130 0x7fdc281526c0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fdc2002a1c0 tx=0x7fdc200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc16ffd640 1 -- 172.21.15.26:0/269507948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20041020 con 0x7fdc280b3130 2024-04-24T23:27:06.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc16ffd640 1 -- 172.21.15.26:0/269507948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc2003d6e0 con 0x7fdc280b3130 2024-04-24T23:27:06.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc281f7e20 con 0x7fdc280b3130 2024-04-24T23:27:06.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.629+0000 7fdc16ffd640 1 -- 172.21.15.26:0/269507948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20037040 con 0x7fdc280b3130 2024-04-24T23:27:06.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.630+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc281f83e0 con 0x7fdc280b3130 2024-04-24T23:27:06.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.631+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc280694c0 con 0x7fdc280b3130 2024-04-24T23:27:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.631+0000 7fdc16ffd640 1 -- 172.21.15.26:0/269507948 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc20044020 con 0x7fdc280b3130 2024-04-24T23:27:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.631+0000 7fdc16ffd640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc0806be60 0x7fdc0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.631+0000 7fdc16ffd640 1 -- 172.21.15.26:0/269507948 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fdc200ab5b0 con 0x7fdc280b3130 2024-04-24T23:27:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.631+0000 7fdc25d9b640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc0806be60 0x7fdc0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.632+0000 7fdc25d9b640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc0806be60 0x7fdc0806e320 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdc280b30e0 tx=0x7fdc1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.636+0000 7fdc16ffd640 1 -- 172.21.15.26:0/269507948 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdc2004b030 con 0x7fdc280b3130 2024-04-24T23:27:06.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.819+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc280b0e60 con 0x7fdc280b3130 2024-04-24T23:27:06.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.820+0000 7fdc16ffd640 1 -- 172.21.15.26:0/269507948 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16697 (secure 0 0 0) 0x7fdc20079fb0 con 0x7fdc280b3130 2024-04-24T23:27:06.821 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:06.821 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:26:38.480003+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:26:15.824299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:26:16.498456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.7","mappings":[{"from":6,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:06.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.822+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc0806be60 msgr2=0x7fdc0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.822+0000 7fdc2cb34640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc0806be60 0x7fdc0806e320 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdc280b30e0 tx=0x7fdc1c009040 comp rx=0 tx=0).stop 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.822+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc280b3130 msgr2=0x7fdc281526c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.822+0000 7fdc2cb34640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc280b3130 0x7fdc281526c0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fdc2002a1c0 tx=0x7fdc200046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.822+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 shutdown_connections 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.822+0000 7fdc2cb34640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdc0806be60 0x7fdc0806e320 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.822+0000 7fdc2cb34640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdc2815e570 0x7fdc280b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.823+0000 7fdc2cb34640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc280d22e0 0x7fdc280b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.823+0000 7fdc2cb34640 1 --2- 172.21.15.26:0/269507948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc280b3130 0x7fdc281526c0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.823+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 >> 172.21.15.26:0/269507948 conn(0x7fdc2808dbe0 msgr2=0x7fdc280ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.823+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 shutdown_connections 2024-04-24T23:27:06.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.823+0000 7fdc2cb34640 1 -- 172.21.15.26:0/269507948 wait complete. 2024-04-24T23:27:06.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:26:15.824299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:26:16.498456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:27:06.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:27:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.938+0000 7fdefeaee640 1 Processor -- start 2024-04-24T23:27:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.938+0000 7fdefeaee640 1 -- start start 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.939+0000 7fdefeaee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdef806b830 0x7fdef8161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.939+0000 7fdefeaee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef815e920 con 0x7fdef80d18b0 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.939+0000 7fdefeaee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef815eba0 con 0x7fdef80cc9b0 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.939+0000 7fdefeaee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdef815ee80 con 0x7fdef806b830 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.939+0000 7fdef7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdef80d18b0 0x7fdef806b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59992/0 (socket says 172.21.15.26:59992) 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.939+0000 7fdef7fff640 1 -- 172.21.15.26:0/2918900706 learned_addr learned my addr 172.21.15.26:0/2918900706 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.939+0000 7fdefd08b640 1 --2- 172.21.15.26:0/2918900706 >> v2:172.21.15.29:3300/0 conn(0x7fdef806b830 0x7fdef8161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.940+0000 7fdef77fe640 1 -- 172.21.15.26:0/2918900706 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114140075 0 0) 0x7fdeec001110 con 0x7fdef80d18b0 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.940+0000 7fdef77fe640 1 -- 172.21.15.26:0/2918900706 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdee0003520 con 0x7fdef80d18b0 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.940+0000 7fdef77fe640 1 -- 172.21.15.26:0/2918900706 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4105192813 0 0) 0x7fdef815eba0 con 0x7fdef80cc9b0 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.940+0000 7fdef77fe640 1 -- 172.21.15.26:0/2918900706 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdeec001110 con 0x7fdef80cc9b0 2024-04-24T23:27:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.940+0000 7fdefd08b640 1 -- 172.21.15.26:0/2918900706 >> v1:172.21.15.29:6789/0 conn(0x7fdef80cc9b0 legacy=0x7fdef80d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.940+0000 7fdefd08b640 1 -- 172.21.15.26:0/2918900706 >> v1:172.21.15.26:6789/0 conn(0x7fdef80d18b0 legacy=0x7fdef806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.940+0000 7fdefd08b640 1 -- 172.21.15.26:0/2918900706 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef815f160 con 0x7fdef806b830 2024-04-24T23:27:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdefd08b640 1 --2- 172.21.15.26:0/2918900706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdef806b830 0x7fdef8161f10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009d90 tx=0x7fdef002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0444a07e5a2404f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdef77fe640 1 -- 172.21.15.26:0/2918900706 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0007740 con 0x7fdef806b830 2024-04-24T23:27:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdef77fe640 1 -- 172.21.15.26:0/2918900706 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef00078e0 con 0x7fdef806b830 2024-04-24T23:27:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdef77fe640 1 -- 172.21.15.26:0/2918900706 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0007cb0 con 0x7fdef806b830 2024-04-24T23:27:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdefeaee640 1 -- 172.21.15.26:0/2918900706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdef806b830 msgr2=0x7fdef8161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdefeaee640 1 --2- 172.21.15.26:0/2918900706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdef806b830 0x7fdef8161f10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009d90 tx=0x7fdef002a6f0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdefeaee640 1 -- 172.21.15.26:0/2918900706 shutdown_connections 2024-04-24T23:27:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdefeaee640 1 --2- 172.21.15.26:0/2918900706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdef806b830 0x7fdef8161f10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdefeaee640 1 -- 172.21.15.26:0/2918900706 >> 172.21.15.26:0/2918900706 conn(0x7fdef80d0d20 msgr2=0x7fdef80d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdefeaee640 1 -- 172.21.15.26:0/2918900706 shutdown_connections 2024-04-24T23:27:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.941+0000 7fdefeaee640 1 -- 172.21.15.26:0/2918900706 wait complete. 2024-04-24T23:27:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.942+0000 7fdefeaee640 1 Processor -- start 2024-04-24T23:27:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.943+0000 7fdefeaee640 1 -- start start 2024-04-24T23:27:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.943+0000 7fdefeaee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef806b830 0x7fdef81529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.943+0000 7fdefeaee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdef80cc9b0 0x7fdef80b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.943+0000 7fdefeaee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdef80d18b0 0x7fdef80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.943+0000 7fdefeaee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdef8162450 con 0x7fdef806b830 2024-04-24T23:27:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.943+0000 7fdefeaee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdef80cec20 con 0x7fdef80cc9b0 2024-04-24T23:27:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.943+0000 7fdefeaee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdef81625d0 con 0x7fdef80d18b0 2024-04-24T23:27:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefc88a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef806b830 0x7fdef81529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefc88a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef806b830 0x7fdef81529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48598/0 (socket says 172.21.15.26:48598) 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefc88a640 1 -- 172.21.15.26:0/2315760734 learned_addr learned my addr 172.21.15.26:0/2315760734 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdef7fff640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdef80cc9b0 0x7fdef80b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefd08b640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdef80d18b0 0x7fdef80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefc88a640 1 -- 172.21.15.26:0/2315760734 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdef80d18b0 msgr2=0x7fdef80b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefc88a640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdef80d18b0 0x7fdef80b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefc88a640 1 -- 172.21.15.26:0/2315760734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdef80cc9b0 msgr2=0x7fdef80b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefc88a640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdef80cc9b0 0x7fdef80b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefc88a640 1 -- 172.21.15.26:0/2315760734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef81613a0 con 0x7fdef806b830 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefc88a640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef806b830 0x7fdef81529e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c500 tx=0x7fdee800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdef57fa640 1 -- 172.21.15.26:0/2315760734 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee80052d0 con 0x7fdef806b830 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdef57fa640 1 -- 172.21.15.26:0/2315760734 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee8005490 con 0x7fdef806b830 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef81610c0 con 0x7fdef806b830 2024-04-24T23:27:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdef57fa640 1 -- 172.21.15.26:0/2315760734 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8012870 con 0x7fdef806b830 2024-04-24T23:27:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.944+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef815a080 con 0x7fdef806b830 2024-04-24T23:27:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.945+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef8075610 con 0x7fdef806b830 2024-04-24T23:27:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.946+0000 7fdef57fa640 1 -- 172.21.15.26:0/2315760734 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdee8012a10 con 0x7fdef806b830 2024-04-24T23:27:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.946+0000 7fdef57fa640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdee406be30 0x7fdee406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.946+0000 7fdef7fff640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdee406be30 0x7fdee406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.947+0000 7fdef57fa640 1 -- 172.21.15.26:0/2315760734 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fdee808aa00 con 0x7fdef806b830 2024-04-24T23:27:06.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.947+0000 7fdef7fff640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdee406be30 0x7fdee406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdef806de90 tx=0x7fdeec007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:06.951+0000 7fdef57fa640 1 -- 172.21.15.26:0/2315760734 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdee8058d30 con 0x7fdef806b830 2024-04-24T23:27:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.137+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdef80cfef0 con 0x7fdef806b830 2024-04-24T23:27:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.137+0000 7fdef57fa640 1 -- 172.21.15.26:0/2315760734 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fdee805d030 con 0x7fdef806b830 2024-04-24T23:27:07.138 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdee406be30 msgr2=0x7fdee406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdee406be30 0x7fdee406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdef806de90 tx=0x7fdeec007d30 comp rx=0 tx=0).stop 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef806b830 msgr2=0x7fdef81529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef806b830 0x7fdef81529e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c500 tx=0x7fdee800c9c0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 shutdown_connections 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdee406be30 0x7fdee406e2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdef80d18b0 0x7fdef80b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdef80cc9b0 0x7fdef80b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 --2- 172.21.15.26:0/2315760734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdef806b830 0x7fdef81529e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 >> 172.21.15.26:0/2315760734 conn(0x7fdef80d0d20 msgr2=0x7fdef806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:07.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 shutdown_connections 2024-04-24T23:27:07.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.139+0000 7fdefeaee640 1 -- 172.21.15.26:0/2315760734 wait complete. 2024-04-24T23:27:07.151 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:27:07.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.254+0000 7f20f722c640 1 Processor -- start 2024-04-24T23:27:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.254+0000 7f20f722c640 1 -- start start 2024-04-24T23:27:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.254+0000 7f20f722c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.254+0000 7f20f722c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f00d2120 con 0x7f20f00d4c10 2024-04-24T23:27:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.254+0000 7f20f722c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f00b3740 con 0x7f20f015e750 2024-04-24T23:27:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.254+0000 7f20f722c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f20f00d1510 con 0x7f20f015e370 2024-04-24T23:27:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20f57c9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f20f00d4c10 0x7f20f00b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59994/0 (socket says 172.21.15.26:59994) 2024-04-24T23:27:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20f57c9640 1 -- 172.21.15.26:0/748628762 learned_addr learned my addr 172.21.15.26:0/748628762 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20f4fc8640 1 --2- 172.21.15.26:0/748628762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20ef7fe640 1 -- 172.21.15.26:0/748628762 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040462003 0 0) 0x7f20f00d2120 con 0x7f20f00d4c10 2024-04-24T23:27:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20ef7fe640 1 -- 172.21.15.26:0/748628762 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20e0003520 con 0x7f20f00d4c10 2024-04-24T23:27:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20ef7fe640 1 -- 172.21.15.26:0/748628762 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2971402999 0 0) 0x7f20f00b3740 con 0x7f20f015e750 2024-04-24T23:27:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20ef7fe640 1 -- 172.21.15.26:0/748628762 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20f00d2120 con 0x7f20f015e750 2024-04-24T23:27:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20ef7fe640 1 -- 172.21.15.26:0/748628762 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2571867479 0 0) 0x7f20e0003520 con 0x7f20f00d4c10 2024-04-24T23:27:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20f4fc8640 1 -- 172.21.15.26:0/748628762 >> v1:172.21.15.29:6789/0 conn(0x7f20f015e750 legacy=0x7f20f00b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20f4fc8640 1 -- 172.21.15.26:0/748628762 >> v1:172.21.15.26:6789/0 conn(0x7f20f00d4c10 legacy=0x7f20f00b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.255+0000 7f20f4fc8640 1 -- 172.21.15.26:0/748628762 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f00b3a20 con 0x7f20f015e370 2024-04-24T23:27:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.256+0000 7f20f4fc8640 1 --2- 172.21.15.26:0/748628762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00d2520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f20e40097d0 tx=0x7f20e402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8177b987308e065e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.256+0000 7f20ef7fe640 1 -- 172.21.15.26:0/748628762 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4037070 con 0x7f20f015e370 2024-04-24T23:27:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.256+0000 7f20ef7fe640 1 -- 172.21.15.26:0/748628762 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e402ac30 con 0x7f20f015e370 2024-04-24T23:27:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.256+0000 7f20ef7fe640 1 -- 172.21.15.26:0/748628762 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e80019f0 con 0x7f20f015e370 2024-04-24T23:27:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.257+0000 7f20f722c640 1 -- 172.21.15.26:0/748628762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 msgr2=0x7f20f00d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.257+0000 7f20f722c640 1 --2- 172.21.15.26:0/748628762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00d2520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f20e40097d0 tx=0x7f20e402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.257+0000 7f20f722c640 1 -- 172.21.15.26:0/748628762 shutdown_connections 2024-04-24T23:27:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.257+0000 7f20f722c640 1 --2- 172.21.15.26:0/748628762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00d2520 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.257+0000 7f20f722c640 1 -- 172.21.15.26:0/748628762 >> 172.21.15.26:0/748628762 conn(0x7f20f00d40f0 msgr2=0x7f20f00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.257+0000 7f20f722c640 1 -- 172.21.15.26:0/748628762 shutdown_connections 2024-04-24T23:27:07.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.257+0000 7f20f722c640 1 -- 172.21.15.26:0/748628762 wait complete. 2024-04-24T23:27:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.258+0000 7f20f722c640 1 Processor -- start 2024-04-24T23:27:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.258+0000 7f20f722c640 1 -- start start 2024-04-24T23:27:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f722c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20f00d4c10 0x7f20f01f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f722c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f722c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20f00c9860 0x7f20f00c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f722c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20f00cfca0 con 0x7f20f00c9860 2024-04-24T23:27:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f722c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20f00d1ed0 con 0x7f20f00d4c10 2024-04-24T23:27:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f722c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f20f00cfe20 con 0x7f20f015e370 2024-04-24T23:27:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f57c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20f00c9860 0x7f20f00c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f57c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20f00c9860 0x7f20f00c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48612/0 (socket says 172.21.15.26:48612) 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f57c9640 1 -- 172.21.15.26:0/295176436 learned_addr learned my addr 172.21.15.26:0/295176436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f4fc8640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20f00d4c10 0x7f20f01f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20effff640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f57c9640 1 -- 172.21.15.26:0/295176436 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 msgr2=0x7f20f00c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f57c9640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f57c9640 1 -- 172.21.15.26:0/295176436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20f00d4c10 msgr2=0x7f20f01f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f57c9640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20f00d4c10 0x7f20f01f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20f57c9640 1 -- 172.21.15.26:0/295176436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f00b9020 con 0x7f20f00c9860 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.259+0000 7f20effff640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.260+0000 7f20f57c9640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20f00c9860 0x7f20f00c9d10 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f20e8001f60 tx=0x7f20e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.260+0000 7f20ed7fa640 1 -- 172.21.15.26:0/295176436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8017070 con 0x7f20f00c9860 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.260+0000 7f20ed7fa640 1 -- 172.21.15.26:0/295176436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e8012400 con 0x7f20f00c9860 2024-04-24T23:27:07.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.260+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f00b9300 con 0x7f20f00c9860 2024-04-24T23:27:07.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.260+0000 7f20ed7fa640 1 -- 172.21.15.26:0/295176436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e80127f0 con 0x7f20f00c9860 2024-04-24T23:27:07.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.260+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f00cbe90 con 0x7f20f00c9860 2024-04-24T23:27:07.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.261+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f0075610 con 0x7f20f00c9860 2024-04-24T23:27:07.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.261+0000 7f20ed7fa640 1 -- 172.21.15.26:0/295176436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20e801a070 con 0x7f20f00c9860 2024-04-24T23:27:07.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.262+0000 7f20ed7fa640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f20d406be30 0x7f20d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.262+0000 7f20ed7fa640 1 -- 172.21.15.26:0/295176436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f20e808ad10 con 0x7f20f00c9860 2024-04-24T23:27:07.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.262+0000 7f20f4fc8640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f20d406be30 0x7f20d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.262+0000 7f20f4fc8640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f20d406be30 0x7f20d406e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f20e4034000 tx=0x7f20e402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.266+0000 7f20ed7fa640 1 -- 172.21.15.26:0/295176436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f20e8059030 con 0x7f20f00c9860 2024-04-24T23:27:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.452+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20f00b5380 con 0x7f20f00c9860 2024-04-24T23:27:07.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.452+0000 7f20ed7fa640 1 -- 172.21.15.26:0/295176436 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f20e805d330 con 0x7f20f00c9860 2024-04-24T23:27:07.453 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:27:07.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f20d406be30 msgr2=0x7f20d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f20d406be30 0x7f20d406e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f20e4034000 tx=0x7f20e402e000 comp rx=0 tx=0).stop 2024-04-24T23:27:07.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20f00c9860 msgr2=0x7f20f00c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20f00c9860 0x7f20f00c9d10 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f20e8001f60 tx=0x7f20e800cf90 comp rx=0 tx=0).stop 2024-04-24T23:27:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 shutdown_connections 2024-04-24T23:27:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f20d406be30 0x7f20d406e2f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20f00c9860 0x7f20f00c9d10 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f20f015e370 0x7f20f00c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 --2- 172.21.15.26:0/295176436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20f00d4c10 0x7f20f01f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 >> 172.21.15.26:0/295176436 conn(0x7f20f00d40f0 msgr2=0x7f20f006cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 shutdown_connections 2024-04-24T23:27:07.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.454+0000 7f20f722c640 1 -- 172.21.15.26:0/295176436 wait complete. 2024-04-24T23:27:07.466 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:27:07.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.569+0000 7fd8b2a76640 1 Processor -- start 2024-04-24T23:27:07.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.569+0000 7fd8b2a76640 1 -- start start 2024-04-24T23:27:07.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.570+0000 7fd8b2a76640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac0b3110 0x7fd8ac0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.570+0000 7fd8b2a76640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ac0d2950 con 0x7fd8ac0cfce0 2024-04-24T23:27:07.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.570+0000 7fd8b2a76640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ac0d2c30 con 0x7fd8ac15e610 2024-04-24T23:27:07.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.570+0000 7fd8b2a76640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac0b40b0 con 0x7fd8ac0b3110 2024-04-24T23:27:07.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.570+0000 7fd8b1013640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd8ac0cfce0 0x7fd8ac162280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60002/0 (socket says 172.21.15.26:60002) 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.570+0000 7fd8b1013640 1 -- 172.21.15.26:0/4092317945 learned_addr learned my addr 172.21.15.26:0/4092317945 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.570+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2989894667 0 0) 0x7fd8ac0d2950 con 0x7fd8ac0cfce0 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.570+0000 7fd8abfff640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac0b3110 0x7fd8ac0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.570+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd894003520 con 0x7fd8ac0cfce0 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1277862352 0 0) 0x7fd894003520 con 0x7fd8ac0cfce0 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac0b3110 msgr2=0x7fd8ac0cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac0b3110 0x7fd8ac0cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 >> v1:172.21.15.29:6789/0 conn(0x7fd8ac15e610 legacy=0x7fd8ac0b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac0d2f10 con 0x7fd8ac0cfce0 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fd8a40019f0 con 0x7fd8ac0cfce0 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 >> v1:172.21.15.26:6789/0 conn(0x7fd8ac0cfce0 legacy=0x7fd8ac162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8940039e0 0x7fd894003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac0b3110 0x7fd894004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd894004d40 0x7fd8940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac0b40b0 con 0x7fd894004d40 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd89400b9c0 con 0x7fd8940039e0 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd89400bbc0 con 0x7fd8ac0b3110 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8b1013640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd894004d40 0x7fd8940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8b0812640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8940039e0 0x7fd894003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8abfff640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac0b3110 0x7fd894004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8b1013640 1 -- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac0b3110 msgr2=0x7fd894004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8b1013640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac0b3110 0x7fd894004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8b1013640 1 -- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8940039e0 msgr2=0x7fd894003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8b1013640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8940039e0 0x7fd894003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.571+0000 7fd8b1013640 1 -- 172.21.15.26:0/4092317945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd894003520 con 0x7fd894004d40 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.572+0000 7fd8b1013640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd894004d40 0x7fd8940051c0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400a880 tx=0x7fd894005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfca1c65dd2c905 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.572+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400a4a0 con 0x7fd894004d40 2024-04-24T23:27:07.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.572+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a402f070 con 0x7fd894004d40 2024-04-24T23:27:07.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.572+0000 7fd8ab7fe640 1 -- 172.21.15.26:0/4092317945 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a402bce0 con 0x7fd894004d40 2024-04-24T23:27:07.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.573+0000 7fd8b2a76640 1 -- 172.21.15.26:0/4092317945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd894004d40 msgr2=0x7fd8940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.573+0000 7fd8b2a76640 1 --2- 172.21.15.26:0/4092317945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd894004d40 0x7fd8940051c0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400a880 tx=0x7fd894005830 comp rx=0 tx=0).stop 2024-04-24T23:27:07.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.573+0000 7fd8b1013640 1 -- 172.21.15.26:0/4092317945 reap_dead start 2024-04-24T23:27:07.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.573+0000 7fd8b2a76640 1 -- 172.21.15.26:0/4092317945 shutdown_connections 2024-04-24T23:27:07.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.573+0000 7fd8b2a76640 1 -- 172.21.15.26:0/4092317945 >> 172.21.15.26:0/4092317945 conn(0x7fd8ac0d4040 msgr2=0x7fd8ac0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:07.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.573+0000 7fd8b2a76640 1 -- 172.21.15.26:0/4092317945 shutdown_connections 2024-04-24T23:27:07.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.573+0000 7fd8b2a76640 1 -- 172.21.15.26:0/4092317945 wait complete. 2024-04-24T23:27:07.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.574+0000 7fd8b2a76640 1 Processor -- start 2024-04-24T23:27:07.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.574+0000 7fd8b2a76640 1 -- start start 2024-04-24T23:27:07.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b2a76640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac1f4160 0x7fd8ac0b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b2a76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8ac0b7170 0x7fd8ac0cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b2a76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1f4540 0x7fd8ac0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b2a76640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac0cec50 con 0x7fd8ac1f4540 2024-04-24T23:27:07.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b2a76640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac0cf280 con 0x7fd8ac0b7170 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b2a76640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac0d0160 con 0x7fd8ac1f4160 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b1013640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1f4540 0x7fd8ac0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b1013640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1f4540 0x7fd8ac0cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48624/0 (socket says 172.21.15.26:48624) 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b1013640 1 -- 172.21.15.26:0/2864356591 learned_addr learned my addr 172.21.15.26:0/2864356591 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8abfff640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8ac0b7170 0x7fd8ac0cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b0812640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac1f4160 0x7fd8ac0b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b1013640 1 -- 172.21.15.26:0/2864356591 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac1f4160 msgr2=0x7fd8ac0b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b1013640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac1f4160 0x7fd8ac0b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b1013640 1 -- 172.21.15.26:0/2864356591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8ac0b7170 msgr2=0x7fd8ac0cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b1013640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8ac0b7170 0x7fd8ac0cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b1013640 1 -- 172.21.15.26:0/2864356591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac0b75f0 con 0x7fd8ac1f4540 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.575+0000 7fd8b1013640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1f4540 0x7fd8ac0cd670 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd8a402b530 tx=0x7fd8a402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:07.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.576+0000 7fd8a97fa640 1 -- 172.21.15.26:0/2864356591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400a4a0 con 0x7fd8ac1f4540 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.576+0000 7fd8a97fa640 1 -- 172.21.15.26:0/2864356591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a40019c0 con 0x7fd8ac1f4540 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.576+0000 7fd8a97fa640 1 -- 172.21.15.26:0/2864356591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4034670 con 0x7fd8ac1f4540 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.576+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac0b78d0 con 0x7fd8ac1f4540 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.576+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac0b7e90 con 0x7fd8ac1f4540 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.577+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ac0696d0 con 0x7fd8ac1f4540 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.577+0000 7fd8a97fa640 1 -- 172.21.15.26:0/2864356591 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8a4034810 con 0x7fd8ac1f4540 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.578+0000 7fd8a97fa640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd89806be30 0x7fd89806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.578+0000 7fd8a97fa640 1 -- 172.21.15.26:0/2864356591 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fd8a40aadd0 con 0x7fd8ac1f4540 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.578+0000 7fd8b0812640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd89806be30 0x7fd89806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.578+0000 7fd8b0812640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd89806be30 0x7fd89806e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd89c000c00 tx=0x7fd89c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:07.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.582+0000 7fd8a97fa640 1 -- 172.21.15.26:0/2864356591 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd8ac0696d0 con 0x7fd8ac1f4540 2024-04-24T23:27:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.762+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd8ac075610 con 0x7fd8ac1f4540 2024-04-24T23:27:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.762+0000 7fd8a97fa640 1 -- 172.21.15.26:0/2864356591 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7fd8a40ac070 con 0x7fd8ac1f4540 2024-04-24T23:27:07.763 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd89806be30 msgr2=0x7fd89806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd89806be30 0x7fd89806e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd89c000c00 tx=0x7fd89c009040 comp rx=0 tx=0).stop 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1f4540 msgr2=0x7fd8ac0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1f4540 0x7fd8ac0cd670 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd8a402b530 tx=0x7fd8a402d070 comp rx=0 tx=0).stop 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 shutdown_connections 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd89806be30 0x7fd89806e2f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8ac1f4540 0x7fd8ac0cd670 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd8ac0b7170 0x7fd8ac0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 --2- 172.21.15.26:0/2864356591 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd8ac1f4160 0x7fd8ac0b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 >> 172.21.15.26:0/2864356591 conn(0x7fd8ac0d4040 msgr2=0x7fd8ac0b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:07.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 shutdown_connections 2024-04-24T23:27:07.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.764+0000 7fd8b2a76640 1 -- 172.21.15.26:0/2864356591 wait complete. 2024-04-24T23:27:07.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:27:07.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.877+0000 7fde2aafc640 1 Processor -- start 2024-04-24T23:27:07.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.877+0000 7fde2aafc640 1 -- start start 2024-04-24T23:27:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.877+0000 7fde2aafc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde2415e570 0x7fde240d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.877+0000 7fde2aafc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde240b3130 0x7fde240d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.877+0000 7fde2aafc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde240d22e0 0x7fde240d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.877+0000 7fde2aafc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fde240d1ab0 con 0x7fde240b3130 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.877+0000 7fde2aafc640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fde240d2ae0 con 0x7fde2415e570 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.877+0000 7fde2aafc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fde240d32f0 con 0x7fde240d22e0 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde23fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde240b3130 0x7fde240d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde23fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde240b3130 0x7fde240d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48640/0 (socket says 172.21.15.26:48640) 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde23fff640 1 -- 172.21.15.26:0/2655872809 learned_addr learned my addr 172.21.15.26:0/2655872809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde28898640 1 --2- 172.21.15.26:0/2655872809 >> v2:172.21.15.29:3300/0 conn(0x7fde2415e570 0x7fde240d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde29099640 1 --2- 172.21.15.26:0/2655872809 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde240d22e0 0x7fde240d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde23fff640 1 -- 172.21.15.26:0/2655872809 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde240d22e0 msgr2=0x7fde240d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde23fff640 1 --2- 172.21.15.26:0/2655872809 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde240d22e0 0x7fde240d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde23fff640 1 -- 172.21.15.26:0/2655872809 >> v2:172.21.15.29:3300/0 conn(0x7fde2415e570 msgr2=0x7fde240d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde23fff640 1 --2- 172.21.15.26:0/2655872809 >> v2:172.21.15.29:3300/0 conn(0x7fde2415e570 0x7fde240d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.878+0000 7fde23fff640 1 -- 172.21.15.26:0/2655872809 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde240d0270 con 0x7fde240b3130 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde23fff640 1 --2- 172.21.15.26:0/2655872809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240b3130 0x7fde240d46a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fde240d52d0 tx=0x7fde1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcb8d15260ae40f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:07.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde237fe640 1 -- 172.21.15.26:0/2655872809 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18037070 con 0x7fde240b3130 2024-04-24T23:27:07.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde237fe640 1 -- 172.21.15.26:0/2655872809 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde18032440 con 0x7fde240b3130 2024-04-24T23:27:07.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde237fe640 1 -- 172.21.15.26:0/2655872809 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18032810 con 0x7fde240b3130 2024-04-24T23:27:07.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde2aafc640 1 -- 172.21.15.26:0/2655872809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240b3130 msgr2=0x7fde240d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2655872809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240b3130 0x7fde240d46a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fde240d52d0 tx=0x7fde1802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde2aafc640 1 -- 172.21.15.26:0/2655872809 shutdown_connections 2024-04-24T23:27:07.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2655872809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240b3130 0x7fde240d46a0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2655872809 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde240d22e0 0x7fde240d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2655872809 >> v2:172.21.15.29:3300/0 conn(0x7fde2415e570 0x7fde240d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde2aafc640 1 -- 172.21.15.26:0/2655872809 >> 172.21.15.26:0/2655872809 conn(0x7fde2408dbe0 msgr2=0x7fde2409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:07.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde2aafc640 1 -- 172.21.15.26:0/2655872809 shutdown_connections 2024-04-24T23:27:07.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.879+0000 7fde2aafc640 1 -- 172.21.15.26:0/2655872809 wait complete. 2024-04-24T23:27:07.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.880+0000 7fde2aafc640 1 Processor -- start 2024-04-24T23:27:07.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.881+0000 7fde2aafc640 1 -- start start 2024-04-24T23:27:07.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.881+0000 7fde2aafc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde240b3130 0x7fde240cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.881+0000 7fde2aafc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240d22e0 0x7fde24158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.881+0000 7fde2aafc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2415e570 0x7fde241594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.881+0000 7fde2aafc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde240cfa70 con 0x7fde240d22e0 2024-04-24T23:27:07.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.881+0000 7fde2aafc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde240d3010 con 0x7fde240b3130 2024-04-24T23:27:07.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.881+0000 7fde2aafc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fde240cfd50 con 0x7fde2415e570 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240d22e0 0x7fde24158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240d22e0 0x7fde24158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48642/0 (socket says 172.21.15.26:48642) 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde23fff640 1 -- 172.21.15.26:0/2763074831 learned_addr learned my addr 172.21.15.26:0/2763074831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde29099640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2415e570 0x7fde241594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde28898640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde240b3130 0x7fde240cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde23fff640 1 -- 172.21.15.26:0/2763074831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2415e570 msgr2=0x7fde241594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde23fff640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2415e570 0x7fde241594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde23fff640 1 -- 172.21.15.26:0/2763074831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde240b3130 msgr2=0x7fde240cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde23fff640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde240b3130 0x7fde240cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde23fff640 1 -- 172.21.15.26:0/2763074831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde241e8010 con 0x7fde240d22e0 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde23fff640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240d22e0 0x7fde24158570 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fde1802a220 tx=0x7fde180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde217fa640 1 -- 172.21.15.26:0/2763074831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18041020 con 0x7fde240d22e0 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde217fa640 1 -- 172.21.15.26:0/2763074831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde180306c0 con 0x7fde240d22e0 2024-04-24T23:27:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde241e82f0 con 0x7fde240d22e0 2024-04-24T23:27:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde217fa640 1 -- 172.21.15.26:0/2763074831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18037040 con 0x7fde240d22e0 2024-04-24T23:27:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.882+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241e8b90 con 0x7fde240d22e0 2024-04-24T23:27:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.883+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde241e86e0 con 0x7fde240d22e0 2024-04-24T23:27:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.884+0000 7fde217fa640 1 -- 172.21.15.26:0/2763074831 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde18044020 con 0x7fde240d22e0 2024-04-24T23:27:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.884+0000 7fde217fa640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1006be30 0x7fde1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.885+0000 7fde217fa640 1 -- 172.21.15.26:0/2763074831 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fde180ab5f0 con 0x7fde240d22e0 2024-04-24T23:27:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.885+0000 7fde28898640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1006be30 0x7fde1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.885+0000 7fde28898640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1006be30 0x7fde1006e2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fde1c003aa0 tx=0x7fde1c009830 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:07.889+0000 7fde217fa640 1 -- 172.21.15.26:0/2763074831 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde1804b030 con 0x7fde240d22e0 2024-04-24T23:27:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.085+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde240d3af0 con 0x7fde240d22e0 2024-04-24T23:27:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.086+0000 7fde217fa640 1 -- 172.21.15.26:0/2763074831 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) ==== 80+0+0 (secure 0 0 0) 0x7fde240d3af0 con 0x7fde240d22e0 2024-04-24T23:27:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:27:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.087+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1006be30 msgr2=0x7fde1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.087+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1006be30 0x7fde1006e2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fde1c003aa0 tx=0x7fde1c009830 comp rx=0 tx=0).stop 2024-04-24T23:27:08.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.087+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240d22e0 msgr2=0x7fde24158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.087+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240d22e0 0x7fde24158570 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fde1802a220 tx=0x7fde180046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.088+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 shutdown_connections 2024-04-24T23:27:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.088+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde1006be30 0x7fde1006e2f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.088+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde2415e570 0x7fde241594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.088+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde240d22e0 0x7fde24158570 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.088+0000 7fde2aafc640 1 --2- 172.21.15.26:0/2763074831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde240b3130 0x7fde240cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.088+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 >> 172.21.15.26:0/2763074831 conn(0x7fde2408dbe0 msgr2=0x7fde240b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.088+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 shutdown_connections 2024-04-24T23:27:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.088+0000 7fde2aafc640 1 -- 172.21.15.26:0/2763074831 wait complete. 2024-04-24T23:27:08.099 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:27:08.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:08.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:27:08.128 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:27:08.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:08.128 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:27:08.186 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:27:08.186 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:27:08.186 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:08.205 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:27:08.205 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:27:08.205 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:27:08.206 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:08.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:27:08.242 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:27:08.242 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:27:08.242 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:08.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:27:08.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:08.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.382+0000 7ff86fc5d640 1 Processor -- start 2024-04-24T23:27:08.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.383+0000 7ff86fc5d640 1 -- start start 2024-04-24T23:27:08.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.383+0000 7ff86fc5d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff86815e570 0x7ff8680d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.383+0000 7ff86fc5d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680b3130 0x7ff8680d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.383+0000 7ff86fc5d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8680d32f0 con 0x7ff8680d22e0 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.383+0000 7ff86fc5d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff8680d1ab0 con 0x7ff86815e570 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.383+0000 7ff86fc5d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff8680d2ae0 con 0x7ff8680b3130 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.383+0000 7ff86d9f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff86815e570 0x7ff8680d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.383+0000 7ff86d9f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff86815e570 0x7ff8680d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48644/0 (socket says 172.21.15.26:48644) 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.383+0000 7ff86d9f9640 1 -- 172.21.15.26:0/3566552691 learned_addr learned my addr 172.21.15.26:0/3566552691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.384+0000 7ff86d9f9640 1 -- 172.21.15.26:0/3566552691 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680b3130 msgr2=0x7ff8680d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.384+0000 7ff86d9f9640 1 --2- 172.21.15.26:0/3566552691 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680b3130 0x7ff8680d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.384+0000 7ff86d9f9640 1 -- 172.21.15.26:0/3566552691 >> v1:172.21.15.29:6789/0 conn(0x7ff8680d22e0 legacy=0x7ff8680d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:08.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.384+0000 7ff86d9f9640 1 -- 172.21.15.26:0/3566552691 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8680d0270 con 0x7ff86815e570 2024-04-24T23:27:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.384+0000 7ff86d9f9640 1 --2- 172.21.15.26:0/3566552691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff86815e570 0x7ff8680d4160 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7ff864009870 tx=0x7ff86402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cff07ec62c8bc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86c9f7640 1 -- 172.21.15.26:0/3566552691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864037070 con 0x7ff86815e570 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86c9f7640 1 -- 172.21.15.26:0/3566552691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff864032440 con 0x7ff86815e570 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86c9f7640 1 -- 172.21.15.26:0/3566552691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864032810 con 0x7ff86815e570 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86fc5d640 1 -- 172.21.15.26:0/3566552691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff86815e570 msgr2=0x7ff8680d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86fc5d640 1 --2- 172.21.15.26:0/3566552691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff86815e570 0x7ff8680d4160 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7ff864009870 tx=0x7ff86402a210 comp rx=0 tx=0).stop 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86fc5d640 1 -- 172.21.15.26:0/3566552691 shutdown_connections 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86fc5d640 1 --2- 172.21.15.26:0/3566552691 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680b3130 0x7ff8680d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86fc5d640 1 --2- 172.21.15.26:0/3566552691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff86815e570 0x7ff8680d4160 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86fc5d640 1 -- 172.21.15.26:0/3566552691 >> 172.21.15.26:0/3566552691 conn(0x7ff86808dbe0 msgr2=0x7ff86809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86fc5d640 1 -- 172.21.15.26:0/3566552691 shutdown_connections 2024-04-24T23:27:08.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.385+0000 7ff86fc5d640 1 -- 172.21.15.26:0/3566552691 wait complete. 2024-04-24T23:27:08.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.386+0000 7ff86fc5d640 1 Processor -- start 2024-04-24T23:27:08.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86fc5d640 1 -- start start 2024-04-24T23:27:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86fc5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8680b3130 0x7ff86814f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86fc5d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680d22e0 0x7ff86815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86fc5d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff86815e570 0x7ff86815d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86fc5d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8680cfa70 con 0x7ff8680b3130 2024-04-24T23:27:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86fc5d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8680d3010 con 0x7ff86815e570 2024-04-24T23:27:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86fc5d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff8680cfd50 con 0x7ff8680d22e0 2024-04-24T23:27:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86d9f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8680b3130 0x7ff86814f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86d9f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8680b3130 0x7ff86814f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48656/0 (socket says 172.21.15.26:48656) 2024-04-24T23:27:08.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86d9f9640 1 -- 172.21.15.26:0/1449521077 learned_addr learned my addr 172.21.15.26:0/1449521077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:08.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86d1f8640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680d22e0 0x7ff86815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.387+0000 7ff86e1fa640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff86815e570 0x7ff86815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86d9f9640 1 -- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680d22e0 msgr2=0x7ff86815cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86d9f9640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680d22e0 0x7ff86815cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86d9f9640 1 -- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff86815e570 msgr2=0x7ff86815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86d9f9640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff86815e570 0x7ff86815d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86d9f9640 1 -- 172.21.15.26:0/1449521077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86814fc90 con 0x7ff8680b3130 2024-04-24T23:27:08.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86d1f8640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680d22e0 0x7ff86815cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:08.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86e1fa640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff86815e570 0x7ff86815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:08.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86d9f9640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8680b3130 0x7ff86814f750 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7ff864009870 tx=0x7ff8640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:08.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff85e7fc640 1 -- 172.21.15.26:0/1449521077 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864041070 con 0x7ff8680b3130 2024-04-24T23:27:08.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff85e7fc640 1 -- 172.21.15.26:0/1449521077 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86403c870 con 0x7ff8680b3130 2024-04-24T23:27:08.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff85e7fc640 1 -- 172.21.15.26:0/1449521077 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864037040 con 0x7ff8680b3130 2024-04-24T23:27:08.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8681590f0 con 0x7ff8680b3130 2024-04-24T23:27:08.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.388+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681596b0 con 0x7ff8680b3130 2024-04-24T23:27:08.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.389+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8680694c0 con 0x7ff8680b3130 2024-04-24T23:27:08.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.390+0000 7ff85e7fc640 1 -- 172.21.15.26:0/1449521077 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff864044070 con 0x7ff8680b3130 2024-04-24T23:27:08.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.390+0000 7ff85e7fc640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff84c06be30 0x7ff84c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.390+0000 7ff85e7fc640 1 -- 172.21.15.26:0/1449521077 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7ff8640abab0 con 0x7ff8680b3130 2024-04-24T23:27:08.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.390+0000 7ff86d1f8640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff84c06be30 0x7ff84c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.391+0000 7ff86d1f8640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff84c06be30 0x7ff84c06e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff8680b8680 tx=0x7ff858008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:08.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.394+0000 7ff85e7fc640 1 -- 172.21.15.26:0/1449521077 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8680d32f0 con 0x7ff8680b3130 2024-04-24T23:27:08.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.619+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8680d4650 con 0x7ff8680b3130 2024-04-24T23:27:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.620+0000 7ff85e7fc640 1 -- 172.21.15.26:0/1449521077 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7ff86407a540 con 0x7ff8680b3130 2024-04-24T23:27:08.621 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:08.621 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:27:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:27:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff84c06be30 msgr2=0x7ff84c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff84c06be30 0x7ff84c06e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff8680b8680 tx=0x7ff858008040 comp rx=0 tx=0).stop 2024-04-24T23:27:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8680b3130 msgr2=0x7ff86814f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8680b3130 0x7ff86814f750 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7ff864009870 tx=0x7ff8640046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 shutdown_connections 2024-04-24T23:27:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff84c06be30 0x7ff84c06e2f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff86815e570 0x7ff86815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8680d22e0 0x7ff86815cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 --2- 172.21.15.26:0/1449521077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8680b3130 0x7ff86814f750 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 >> 172.21.15.26:0/1449521077 conn(0x7ff86808dbe0 msgr2=0x7ff8680b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 shutdown_connections 2024-04-24T23:27:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.622+0000 7ff86fc5d640 1 -- 172.21.15.26:0/1449521077 wait complete. 2024-04-24T23:27:08.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.734+0000 7fbfd06ba640 1 Processor -- start 2024-04-24T23:27:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.734+0000 7fbfd06ba640 1 -- start start 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.735+0000 7fbfd06ba640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfc815e570 0x7fbfc80d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.735+0000 7fbfd06ba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc80b3130 0x7fbfc80d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.735+0000 7fbfd06ba640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc80d32f0 con 0x7fbfc80d22e0 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.735+0000 7fbfd06ba640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc80d1ab0 con 0x7fbfc815e570 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.735+0000 7fbfd06ba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc80d2ae0 con 0x7fbfc80b3130 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.735+0000 7fbfce456640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfc815e570 0x7fbfc80d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.735+0000 7fbfcec57640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbfc80d22e0 0x7fbfc80d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60008/0 (socket says 172.21.15.26:60008) 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.735+0000 7fbfcec57640 1 -- 172.21.15.26:0/3200655371 learned_addr learned my addr 172.21.15.26:0/3200655371 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.736+0000 7fbfcdc55640 1 --2- 172.21.15.26:0/3200655371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc80b3130 0x7fbfc80d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.736+0000 7fbfce456640 1 -- 172.21.15.26:0/3200655371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc80b3130 msgr2=0x7fbfc80d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.736+0000 7fbfce456640 1 --2- 172.21.15.26:0/3200655371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc80b3130 0x7fbfc80d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.736+0000 7fbfce456640 1 -- 172.21.15.26:0/3200655371 >> v1:172.21.15.26:6789/0 conn(0x7fbfc80d22e0 legacy=0x7fbfc80d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.736+0000 7fbfce456640 1 -- 172.21.15.26:0/3200655371 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc80d0270 con 0x7fbfc815e570 2024-04-24T23:27:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.736+0000 7fbfcd454640 1 -- 172.21.15.26:0/3200655371 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069138103 0 0) 0x7fbfc4001490 con 0x7fbfc80d22e0 2024-04-24T23:27:08.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.736+0000 7fbfcdc55640 1 --2- 172.21.15.26:0/3200655371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc80b3130 0x7fbfc80d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfce456640 1 --2- 172.21.15.26:0/3200655371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc815e570 0x7fbfc80d4160 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00097a0 tx=0x7fbfc002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9126e5c1dcd31083 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfcd454640 1 -- 172.21.15.26:0/3200655371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0037070 con 0x7fbfc815e570 2024-04-24T23:27:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfcd454640 1 -- 172.21.15.26:0/3200655371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc0032440 con 0x7fbfc815e570 2024-04-24T23:27:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfcd454640 1 -- 172.21.15.26:0/3200655371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0032810 con 0x7fbfc815e570 2024-04-24T23:27:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfd06ba640 1 -- 172.21.15.26:0/3200655371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc815e570 msgr2=0x7fbfc80d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfd06ba640 1 --2- 172.21.15.26:0/3200655371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc815e570 0x7fbfc80d4160 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00097a0 tx=0x7fbfc002a180 comp rx=0 tx=0).stop 2024-04-24T23:27:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfd06ba640 1 -- 172.21.15.26:0/3200655371 shutdown_connections 2024-04-24T23:27:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfd06ba640 1 --2- 172.21.15.26:0/3200655371 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc80b3130 0x7fbfc80d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfd06ba640 1 --2- 172.21.15.26:0/3200655371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc815e570 0x7fbfc80d4160 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfd06ba640 1 -- 172.21.15.26:0/3200655371 >> 172.21.15.26:0/3200655371 conn(0x7fbfc808dbe0 msgr2=0x7fbfc809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfd06ba640 1 -- 172.21.15.26:0/3200655371 shutdown_connections 2024-04-24T23:27:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.737+0000 7fbfd06ba640 1 -- 172.21.15.26:0/3200655371 wait complete. 2024-04-24T23:27:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.738+0000 7fbfd06ba640 1 Processor -- start 2024-04-24T23:27:08.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfd06ba640 1 -- start start 2024-04-24T23:27:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfd06ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc80b3130 0x7fbfc8152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfd06ba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfc80d22e0 0x7fbfc80cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfd06ba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc815e570 0x7fbfc80cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfd06ba640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc80cfa70 con 0x7fbfc80b3130 2024-04-24T23:27:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfd06ba640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc80d3010 con 0x7fbfc80d22e0 2024-04-24T23:27:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfd06ba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc80cfd50 con 0x7fbfc815e570 2024-04-24T23:27:08.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfce456640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc80b3130 0x7fbfc8152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfce456640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc80b3130 0x7fbfc8152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48680/0 (socket says 172.21.15.26:48680) 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfce456640 1 -- 172.21.15.26:0/1263620585 learned_addr learned my addr 172.21.15.26:0/1263620585 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfcec57640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc815e570 0x7fbfc80cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.739+0000 7fbfcdc55640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfc80d22e0 0x7fbfc80cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfce456640 1 -- 172.21.15.26:0/1263620585 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc815e570 msgr2=0x7fbfc80cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfce456640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc815e570 0x7fbfc80cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfce456640 1 -- 172.21.15.26:0/1263620585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfc80d22e0 msgr2=0x7fbfc80cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfce456640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfc80d22e0 0x7fbfc80cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfce456640 1 -- 172.21.15.26:0/1263620585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc8159720 con 0x7fbfc80b3130 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfce456640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc80b3130 0x7fbfc8152650 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0035040 tx=0x7fbfc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfbeffd640 1 -- 172.21.15.26:0/1263620585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0037040 con 0x7fbfc80b3130 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfbeffd640 1 -- 172.21.15.26:0/1263620585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc0030770 con 0x7fbfc80b3130 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfbeffd640 1 -- 172.21.15.26:0/1263620585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0030b80 con 0x7fbfc80b3130 2024-04-24T23:27:08.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc80b6df0 con 0x7fbfc80b3130 2024-04-24T23:27:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.740+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc80b73b0 con 0x7fbfc80b3130 2024-04-24T23:27:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.741+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc80694c0 con 0x7fbfc80b3130 2024-04-24T23:27:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.741+0000 7fbfbeffd640 1 -- 172.21.15.26:0/1263620585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbfc0044020 con 0x7fbfc80b3130 2024-04-24T23:27:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.742+0000 7fbfbeffd640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfac06be30 0x7fbfac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.742+0000 7fbfbeffd640 1 -- 172.21.15.26:0/1263620585 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fbfc00ab3a0 con 0x7fbfc80b3130 2024-04-24T23:27:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.742+0000 7fbfcdc55640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfac06be30 0x7fbfac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.742+0000 7fbfcdc55640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfac06be30 0x7fbfac06e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80cd060 tx=0x7fbfb8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:08.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.746+0000 7fbfbeffd640 1 -- 172.21.15.26:0/1263620585 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbfc0032e20 con 0x7fbfc80b3130 2024-04-24T23:27:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.929+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfc80d3a60 con 0x7fbfc80b3130 2024-04-24T23:27:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.930+0000 7fbfbeffd640 1 -- 172.21.15.26:0/1263620585 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16697 (secure 0 0 0) 0x7fbfc0079da0 con 0x7fbfc80b3130 2024-04-24T23:27:08.931 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:08.931 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:26:38.480003+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:26:15.824299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:26:16.498456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.7","mappings":[{"from":6,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfac06be30 msgr2=0x7fbfac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfac06be30 0x7fbfac06e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80cd060 tx=0x7fbfb8008040 comp rx=0 tx=0).stop 2024-04-24T23:27:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc80b3130 msgr2=0x7fbfc8152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc80b3130 0x7fbfc8152650 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0035040 tx=0x7fbfc00046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 shutdown_connections 2024-04-24T23:27:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbfac06be30 0x7fbfac06e2f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbfc815e570 0x7fbfc80cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbfc80d22e0 0x7fbfc80cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 --2- 172.21.15.26:0/1263620585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc80b3130 0x7fbfc8152650 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 >> 172.21.15.26:0/1263620585 conn(0x7fbfc808dbe0 msgr2=0x7fbfc815d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 shutdown_connections 2024-04-24T23:27:08.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:08.932+0000 7fbfd06ba640 1 -- 172.21.15.26:0/1263620585 wait complete. 2024-04-24T23:27:08.944 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:26:15.824299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:26:16.498456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:27:08.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:27:09.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.048+0000 7f6d3a015640 1 Processor -- start 2024-04-24T23:27:09.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.048+0000 7f6d3a015640 1 -- start start 2024-04-24T23:27:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.048+0000 7f6d3a015640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d3415e370 0x7f6d340d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.048+0000 7f6d3a015640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 0x7f6d340cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.048+0000 7f6d3a015640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d340d2120 con 0x7f6d3415e750 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.048+0000 7f6d3a015640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6d340d1510 con 0x7f6d3415e370 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.048+0000 7f6d3a015640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6d340d1ed0 con 0x7f6d340d4c10 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.049+0000 7f6d337fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d3415e370 0x7f6d340d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.049+0000 7f6d337fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d3415e370 0x7f6d340d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48682/0 (socket says 172.21.15.26:48682) 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.049+0000 7f6d337fe640 1 -- 172.21.15.26:0/3075524624 learned_addr learned my addr 172.21.15.26:0/3075524624 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.049+0000 7f6d337fe640 1 -- 172.21.15.26:0/3075524624 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 msgr2=0x7f6d340cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.049+0000 7f6d33fff640 1 --2- 172.21.15.26:0/3075524624 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 0x7f6d340cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.049+0000 7f6d337fe640 1 --2- 172.21.15.26:0/3075524624 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 0x7f6d340cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.049+0000 7f6d337fe640 1 -- 172.21.15.26:0/3075524624 >> v1:172.21.15.29:6789/0 conn(0x7f6d3415e750 legacy=0x7f6d340b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.049+0000 7f6d337fe640 1 -- 172.21.15.26:0/3075524624 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d340b3740 con 0x7f6d3415e370 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.049+0000 7f6d33fff640 1 --2- 172.21.15.26:0/3075524624 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 0x7f6d340cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d337fe640 1 --2- 172.21.15.26:0/3075524624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d3415e370 0x7f6d340d2520 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6d3408c7e0 tx=0x7f6d2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb603ef80a0e7cca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d327fc640 1 -- 172.21.15.26:0/3075524624 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24037070 con 0x7f6d3415e370 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d327fc640 1 -- 172.21.15.26:0/3075524624 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d24032440 con 0x7f6d3415e370 2024-04-24T23:27:09.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d327fc640 1 -- 172.21.15.26:0/3075524624 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24032810 con 0x7f6d3415e370 2024-04-24T23:27:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d3a015640 1 -- 172.21.15.26:0/3075524624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d3415e370 msgr2=0x7f6d340d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d3a015640 1 --2- 172.21.15.26:0/3075524624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d3415e370 0x7f6d340d2520 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6d3408c7e0 tx=0x7f6d2402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d3a015640 1 -- 172.21.15.26:0/3075524624 shutdown_connections 2024-04-24T23:27:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d3a015640 1 --2- 172.21.15.26:0/3075524624 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 0x7f6d340cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d3a015640 1 --2- 172.21.15.26:0/3075524624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d3415e370 0x7f6d340d2520 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d3a015640 1 -- 172.21.15.26:0/3075524624 >> 172.21.15.26:0/3075524624 conn(0x7f6d340d40f0 msgr2=0x7f6d340d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d3a015640 1 -- 172.21.15.26:0/3075524624 shutdown_connections 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.050+0000 7f6d3a015640 1 -- 172.21.15.26:0/3075524624 wait complete. 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.051+0000 7f6d3a015640 1 Processor -- start 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.052+0000 7f6d3a015640 1 -- start start 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.052+0000 7f6d3a015640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 0x7f6d340ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.052+0000 7f6d3a015640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d3415e370 0x7f6d340c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.052+0000 7f6d3a015640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d341f75a0 0x7f6d340b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.052+0000 7f6d3a015640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d340d2a60 con 0x7f6d341f75a0 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.052+0000 7f6d3a015640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6d340b3d00 con 0x7f6d3415e370 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.052+0000 7f6d3a015640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6d340d2be0 con 0x7f6d340d4c10 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d33fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d341f75a0 0x7f6d340b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d33fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d341f75a0 0x7f6d340b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48696/0 (socket says 172.21.15.26:48696) 2024-04-24T23:27:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d33fff640 1 -- 172.21.15.26:0/315768507 learned_addr learned my addr 172.21.15.26:0/315768507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d337fe640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 0x7f6d340ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d32ffd640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d3415e370 0x7f6d340c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d33fff640 1 -- 172.21.15.26:0/315768507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 msgr2=0x7f6d340ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d33fff640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 0x7f6d340ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d33fff640 1 -- 172.21.15.26:0/315768507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d3415e370 msgr2=0x7f6d340c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d33fff640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d3415e370 0x7f6d340c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d33fff640 1 -- 172.21.15.26:0/315768507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d340cc930 con 0x7f6d341f75a0 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d33fff640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d341f75a0 0x7f6d340b2610 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00cef0 tx=0x7f6d2c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d1bfff640 1 -- 172.21.15.26:0/315768507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c01a070 con 0x7f6d341f75a0 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d1bfff640 1 -- 172.21.15.26:0/315768507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2c00d860 con 0x7f6d341f75a0 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d341f7a50 con 0x7f6d341f75a0 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d1bfff640 1 -- 172.21.15.26:0/315768507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00dc50 con 0x7f6d341f75a0 2024-04-24T23:27:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.053+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d341f7fb0 con 0x7f6d341f75a0 2024-04-24T23:27:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.054+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d34075610 con 0x7f6d341f75a0 2024-04-24T23:27:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.055+0000 7f6d1bfff640 1 -- 172.21.15.26:0/315768507 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d2c01d070 con 0x7f6d341f75a0 2024-04-24T23:27:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.055+0000 7f6d1bfff640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d2006be30 0x7f6d2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.055+0000 7f6d1bfff640 1 -- 172.21.15.26:0/315768507 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f6d2c08b6d0 con 0x7f6d341f75a0 2024-04-24T23:27:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.055+0000 7f6d337fe640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d2006be30 0x7f6d2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.056+0000 7f6d337fe640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d2006be30 0x7f6d2006e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6d24034000 tx=0x7f6d2402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.059+0000 7f6d1bfff640 1 -- 172.21.15.26:0/315768507 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d341f7a50 con 0x7f6d341f75a0 2024-04-24T23:27:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.244+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d340b5290 con 0x7f6d341f75a0 2024-04-24T23:27:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.245+0000 7f6d1bfff640 1 -- 172.21.15.26:0/315768507 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f6d340cc930 con 0x7f6d341f75a0 2024-04-24T23:27:09.246 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d2006be30 msgr2=0x7f6d2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d2006be30 0x7f6d2006e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6d24034000 tx=0x7f6d2402e000 comp rx=0 tx=0).stop 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d341f75a0 msgr2=0x7f6d340b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d341f75a0 0x7f6d340b2610 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00cef0 tx=0x7f6d2c00a710 comp rx=0 tx=0).stop 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 shutdown_connections 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d2006be30 0x7f6d2006e2f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d341f75a0 0x7f6d340b2610 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d3415e370 0x7f6d340c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 --2- 172.21.15.26:0/315768507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d340d4c10 0x7f6d340ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 >> 172.21.15.26:0/315768507 conn(0x7f6d340d40f0 msgr2=0x7f6d3406cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 shutdown_connections 2024-04-24T23:27:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.247+0000 7f6d3a015640 1 -- 172.21.15.26:0/315768507 wait complete. 2024-04-24T23:27:09.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:27:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.361+0000 7fd26ae6b640 1 Processor -- start 2024-04-24T23:27:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.362+0000 7fd26ae6b640 1 -- start start 2024-04-24T23:27:09.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.362+0000 7fd26ae6b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2640d18b0 0x7fd26406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.362+0000 7fd26ae6b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd26415e920 con 0x7fd2640cc9b0 2024-04-24T23:27:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.362+0000 7fd26ae6b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd26415eba0 con 0x7fd26406b660 2024-04-24T23:27:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.362+0000 7fd26ae6b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd26406bab0 con 0x7fd2640d18b0 2024-04-24T23:27:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.363+0000 7fd263fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2640d18b0 0x7fd26406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.363+0000 7fd268c07640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd2640cc9b0 0x7fd2640d03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60022/0 (socket says 172.21.15.26:60022) 2024-04-24T23:27:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.363+0000 7fd268c07640 1 -- 172.21.15.26:0/3595311371 learned_addr learned my addr 172.21.15.26:0/3595311371 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.363+0000 7fd263fff640 1 -- 172.21.15.26:0/3595311371 >> v1:172.21.15.29:6789/0 conn(0x7fd26406b660 legacy=0x7fd264157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.363+0000 7fd263fff640 1 -- 172.21.15.26:0/3595311371 >> v1:172.21.15.26:6789/0 conn(0x7fd2640cc9b0 legacy=0x7fd2640d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.363+0000 7fd2637fe640 1 -- 172.21.15.26:0/3595311371 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3691120346 0 0) 0x7fd258001110 con 0x7fd2640cc9b0 2024-04-24T23:27:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.363+0000 7fd263fff640 1 -- 172.21.15.26:0/3595311371 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26415ee80 con 0x7fd2640d18b0 2024-04-24T23:27:09.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd263fff640 1 --2- 172.21.15.26:0/3595311371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2640d18b0 0x7fd26406b120 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd2500097a0 tx=0x7fd25002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66aeb8ae17041dde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd2637fe640 1 -- 172.21.15.26:0/3595311371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250037070 con 0x7fd2640d18b0 2024-04-24T23:27:09.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd2637fe640 1 -- 172.21.15.26:0/3595311371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd250032440 con 0x7fd2640d18b0 2024-04-24T23:27:09.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd2637fe640 1 -- 172.21.15.26:0/3595311371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250032810 con 0x7fd2640d18b0 2024-04-24T23:27:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd26ae6b640 1 -- 172.21.15.26:0/3595311371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2640d18b0 msgr2=0x7fd26406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd26ae6b640 1 --2- 172.21.15.26:0/3595311371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2640d18b0 0x7fd26406b120 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd2500097a0 tx=0x7fd25002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd26ae6b640 1 -- 172.21.15.26:0/3595311371 shutdown_connections 2024-04-24T23:27:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd26ae6b640 1 --2- 172.21.15.26:0/3595311371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2640d18b0 0x7fd26406b120 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd26ae6b640 1 -- 172.21.15.26:0/3595311371 >> 172.21.15.26:0/3595311371 conn(0x7fd2640d0d20 msgr2=0x7fd2640d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd26ae6b640 1 -- 172.21.15.26:0/3595311371 shutdown_connections 2024-04-24T23:27:09.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.364+0000 7fd26ae6b640 1 -- 172.21.15.26:0/3595311371 wait complete. 2024-04-24T23:27:09.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.365+0000 7fd26ae6b640 1 Processor -- start 2024-04-24T23:27:09.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd26ae6b640 1 -- start start 2024-04-24T23:27:09.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd26ae6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26406b660 0x7fd264156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd26ae6b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2640cc9b0 0x7fd2641609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd26ae6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2640d18b0 0x7fd264160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd26ae6b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2640cec20 con 0x7fd26406b660 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd26ae6b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd26415f440 con 0x7fd2640d18b0 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd26ae6b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd2641586d0 con 0x7fd2640cc9b0 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd268c07640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26406b660 0x7fd264156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd268c07640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26406b660 0x7fd264156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48720/0 (socket says 172.21.15.26:48720) 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.366+0000 7fd268c07640 1 -- 172.21.15.26:0/686912214 learned_addr learned my addr 172.21.15.26:0/686912214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd263fff640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2640cc9b0 0x7fd2641609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd269408640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2640d18b0 0x7fd264160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd268c07640 1 -- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2640cc9b0 msgr2=0x7fd2641609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd268c07640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2640cc9b0 0x7fd2641609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd268c07640 1 -- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2640d18b0 msgr2=0x7fd264160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd268c07640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2640d18b0 0x7fd264160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd268c07640 1 -- 172.21.15.26:0/686912214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd264072b70 con 0x7fd26406b660 2024-04-24T23:27:09.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd263fff640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2640cc9b0 0x7fd2641609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd269408640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2640d18b0 0x7fd264160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd268c07640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26406b660 0x7fd264156800 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c780 tx=0x7fd25800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd2617fa640 1 -- 172.21.15.26:0/686912214 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258018070 con 0x7fd26406b660 2024-04-24T23:27:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd2617fa640 1 -- 172.21.15.26:0/686912214 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2580072d0 con 0x7fd26406b660 2024-04-24T23:27:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26415bce0 con 0x7fd26406b660 2024-04-24T23:27:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd2617fa640 1 -- 172.21.15.26:0/686912214 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2580047e0 con 0x7fd26406b660 2024-04-24T23:27:09.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.367+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2640b3980 con 0x7fd26406b660 2024-04-24T23:27:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.368+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26415c300 con 0x7fd26406b660 2024-04-24T23:27:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.369+0000 7fd2617fa640 1 -- 172.21.15.26:0/686912214 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd25801b070 con 0x7fd26406b660 2024-04-24T23:27:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.369+0000 7fd2617fa640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd24806be60 0x7fd24806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.369+0000 7fd2617fa640 1 -- 172.21.15.26:0/686912214 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fd25808a140 con 0x7fd26406b660 2024-04-24T23:27:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.369+0000 7fd263fff640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd24806be60 0x7fd24806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.370+0000 7fd263fff640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd24806be60 0x7fd24806e320 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd250034000 tx=0x7fd25002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.373+0000 7fd2617fa640 1 -- 172.21.15.26:0/686912214 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd26415e920 con 0x7fd26406b660 2024-04-24T23:27:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.556+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd264075610 con 0x7fd26406b660 2024-04-24T23:27:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.557+0000 7fd2617fa640 1 -- 172.21.15.26:0/686912214 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7fd258058b40 con 0x7fd26406b660 2024-04-24T23:27:09.557 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd24806be60 msgr2=0x7fd24806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd24806be60 0x7fd24806e320 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd250034000 tx=0x7fd25002e000 comp rx=0 tx=0).stop 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26406b660 msgr2=0x7fd264156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26406b660 0x7fd264156800 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c780 tx=0x7fd25800cc40 comp rx=0 tx=0).stop 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 shutdown_connections 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd24806be60 0x7fd24806e320 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2640d18b0 0x7fd264160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2640cc9b0 0x7fd2641609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 --2- 172.21.15.26:0/686912214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26406b660 0x7fd264156800 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 >> 172.21.15.26:0/686912214 conn(0x7fd2640d0d20 msgr2=0x7fd2640b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 shutdown_connections 2024-04-24T23:27:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.559+0000 7fd26ae6b640 1 -- 172.21.15.26:0/686912214 wait complete. 2024-04-24T23:27:09.571 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:27:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.673+0000 7fd2c2f3b640 1 Processor -- start 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.673+0000 7fd2c2f3b640 1 -- start start 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.674+0000 7fd2c2f3b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2bc15e750 0x7fd2bc0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.674+0000 7fd2c2f3b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd2bc0d2c30 0x7fd2bc0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.674+0000 7fd2c2f3b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2bc0d2120 con 0x7fd2bc15e370 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.674+0000 7fd2c2f3b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd2bc0d1510 con 0x7fd2bc15e750 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.674+0000 7fd2c2f3b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd2bc0d1ed0 con 0x7fd2bc0d2c30 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.674+0000 7fd2bbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2bc15e750 0x7fd2bc0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.674+0000 7fd2c0cd7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd2bc15e370 0x7fd2bc0d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60028/0 (socket says 172.21.15.26:60028) 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.674+0000 7fd2c0cd7640 1 -- 172.21.15.26:0/482812698 learned_addr learned my addr 172.21.15.26:0/482812698 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.675+0000 7fd2bbfff640 1 -- 172.21.15.26:0/482812698 >> v2:172.21.15.29:3300/0 conn(0x7fd2bc0d2c30 msgr2=0x7fd2bc0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.675+0000 7fd2bbfff640 1 --2- 172.21.15.26:0/482812698 >> v2:172.21.15.29:3300/0 conn(0x7fd2bc0d2c30 0x7fd2bc0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.675+0000 7fd2bbfff640 1 -- 172.21.15.26:0/482812698 >> v1:172.21.15.26:6789/0 conn(0x7fd2bc15e370 legacy=0x7fd2bc0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.675+0000 7fd2bbfff640 1 -- 172.21.15.26:0/482812698 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc0b3740 con 0x7fd2bc15e750 2024-04-24T23:27:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.675+0000 7fd2bb7fe640 1 -- 172.21.15.26:0/482812698 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701264129 0 0) 0x7fd2bc0d2120 con 0x7fd2bc15e370 2024-04-24T23:27:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.675+0000 7fd2bbfff640 1 --2- 172.21.15.26:0/482812698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc15e750 0x7fd2bc0cec90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc08d9a0 tx=0x7fd2ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a83d40ebfa06d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2bb7fe640 1 -- 172.21.15.26:0/482812698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac037070 con 0x7fd2bc15e750 2024-04-24T23:27:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2bb7fe640 1 -- 172.21.15.26:0/482812698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac032440 con 0x7fd2bc15e750 2024-04-24T23:27:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2bb7fe640 1 -- 172.21.15.26:0/482812698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac032810 con 0x7fd2bc15e750 2024-04-24T23:27:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/482812698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc15e750 msgr2=0x7fd2bc0cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2c2f3b640 1 --2- 172.21.15.26:0/482812698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc15e750 0x7fd2bc0cec90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc08d9a0 tx=0x7fd2ac02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/482812698 shutdown_connections 2024-04-24T23:27:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2c2f3b640 1 --2- 172.21.15.26:0/482812698 >> v2:172.21.15.29:3300/0 conn(0x7fd2bc0d2c30 0x7fd2bc0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2c2f3b640 1 --2- 172.21.15.26:0/482812698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc15e750 0x7fd2bc0cec90 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/482812698 >> 172.21.15.26:0/482812698 conn(0x7fd2bc0d40f0 msgr2=0x7fd2bc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/482812698 shutdown_connections 2024-04-24T23:27:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.676+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/482812698 wait complete. 2024-04-24T23:27:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.677+0000 7fd2c2f3b640 1 Processor -- start 2024-04-24T23:27:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.677+0000 7fd2c2f3b640 1 -- start start 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.678+0000 7fd2c2f3b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2bc0d2c30 0x7fd2bc0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.678+0000 7fd2c2f3b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2bc15e370 0x7fd2bc0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.678+0000 7fd2c2f3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc0b9860 0x7fd2bc0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.678+0000 7fd2c2f3b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc0b5230 con 0x7fd2bc0b9860 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.678+0000 7fd2c2f3b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc0b3d00 con 0x7fd2bc0d2c30 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.678+0000 7fd2c2f3b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd2bc0b53b0 con 0x7fd2bc15e370 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.678+0000 7fd2c14d8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc0b9860 0x7fd2bc0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.678+0000 7fd2c14d8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc0b9860 0x7fd2bc0cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48746/0 (socket says 172.21.15.26:48746) 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.678+0000 7fd2c14d8640 1 -- 172.21.15.26:0/3695674696 learned_addr learned my addr 172.21.15.26:0/3695674696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2bbfff640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2bc15e370 0x7fd2bc0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c0cd7640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2bc0d2c30 0x7fd2bc0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c14d8640 1 -- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2bc15e370 msgr2=0x7fd2bc0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c14d8640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2bc15e370 0x7fd2bc0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c14d8640 1 -- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2bc0d2c30 msgr2=0x7fd2bc0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c14d8640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2bc0d2c30 0x7fd2bc0b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c14d8640 1 -- 172.21.15.26:0/3695674696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc0b8aa0 con 0x7fd2bc0b9860 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2bbfff640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2bc15e370 0x7fd2bc0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c0cd7640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2bc0d2c30 0x7fd2bc0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c14d8640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc0b9860 0x7fd2bc0cbab0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b410 tx=0x7fd2a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2b97fa640 1 -- 172.21.15.26:0/3695674696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4015020 con 0x7fd2bc0b9860 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2b97fa640 1 -- 172.21.15.26:0/3695674696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a4011850 con 0x7fd2bc0b9860 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2bc0b8d80 con 0x7fd2bc0b9860 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2b97fa640 1 -- 172.21.15.26:0/3695674696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4011c40 con 0x7fd2bc0b9860 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.679+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2bc0b9340 con 0x7fd2bc0b9860 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.680+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2bc075610 con 0x7fd2bc0b9860 2024-04-24T23:27:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.681+0000 7fd2b97fa640 1 -- 172.21.15.26:0/3695674696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2a4018020 con 0x7fd2bc0b9860 2024-04-24T23:27:09.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.681+0000 7fd2b97fa640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd29c06be60 0x7fd29c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.681+0000 7fd2b97fa640 1 -- 172.21.15.26:0/3695674696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fd2a408ad80 con 0x7fd2bc0b9860 2024-04-24T23:27:09.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.681+0000 7fd2c0cd7640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd29c06be60 0x7fd29c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.682+0000 7fd2c0cd7640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd29c06be60 0x7fd29c06e320 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40012e0 tx=0x7fd2b4007f90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.685+0000 7fd2b97fa640 1 -- 172.21.15.26:0/3695674696 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd2a401f020 con 0x7fd2bc0b9860 2024-04-24T23:27:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.866+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd2bc0b9cf0 con 0x7fd2bc0b9860 2024-04-24T23:27:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.867+0000 7fd2b97fa640 1 -- 172.21.15.26:0/3695674696 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7fd2a40172c0 con 0x7fd2bc0b9860 2024-04-24T23:27:09.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:27:09.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.868+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd29c06be60 msgr2=0x7fd29c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.868+0000 7fd2c2f3b640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd29c06be60 0x7fd29c06e320 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40012e0 tx=0x7fd2b4007f90 comp rx=0 tx=0).stop 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.868+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc0b9860 msgr2=0x7fd2bc0cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.868+0000 7fd2c2f3b640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc0b9860 0x7fd2bc0cbab0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b410 tx=0x7fd2a400b8d0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.869+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 shutdown_connections 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.869+0000 7fd2c2f3b640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd29c06be60 0x7fd29c06e320 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.869+0000 7fd2c2f3b640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2bc0b9860 0x7fd2bc0cbab0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.869+0000 7fd2c2f3b640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd2bc15e370 0x7fd2bc0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.869+0000 7fd2c2f3b640 1 --2- 172.21.15.26:0/3695674696 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2bc0d2c30 0x7fd2bc0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.869+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 >> 172.21.15.26:0/3695674696 conn(0x7fd2bc0d40f0 msgr2=0x7fd2bc15d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.869+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 shutdown_connections 2024-04-24T23:27:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.869+0000 7fd2c2f3b640 1 -- 172.21.15.26:0/3695674696 wait complete. 2024-04-24T23:27:09.880 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:09.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.980+0000 7f15fecbd640 1 Processor -- start 2024-04-24T23:27:09.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.980+0000 7f15fecbd640 1 -- start start 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.981+0000 7f15fecbd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f806e2f0 0x7f15f8165680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.981+0000 7f15fecbd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f80b2e80 con 0x7f15f815a860 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.981+0000 7f15fecbd640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f80b3120 con 0x7f15f815a480 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.981+0000 7f15fecbd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f15f80b33c0 con 0x7f15f806e2f0 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.981+0000 7f15f7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15f815a860 0x7f15f806dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60042/0 (socket says 172.21.15.26:60042) 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.981+0000 7f15f7fff640 1 -- 172.21.15.26:0/113687253 learned_addr learned my addr 172.21.15.26:0/113687253 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.981+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1139132897 0 0) 0x7f15f80b2e80 con 0x7f15f815a860 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.981+0000 7f15fd25a640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f806e2f0 0x7f15f8165680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.981+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e0003520 con 0x7f15f815a860 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2216303514 0 0) 0x7f15e0003520 con 0x7f15f815a860 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f806e2f0 msgr2=0x7f15f8165680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f806e2f0 0x7f15f8165680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 >> v1:172.21.15.29:6789/0 conn(0x7f15f815a480 legacy=0x7f15f80b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f80b36a0 con 0x7f15f815a860 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f15ec001670 con 0x7f15f815a860 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 >> v1:172.21.15.26:6789/0 conn(0x7f15f815a860 legacy=0x7f15f806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f806e2f0 0x7f15e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e0004d40 0x7f15e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15f80b33c0 con 0x7f15e0003f60 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f15e000bb50 con 0x7f15e0004d40 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f15e000bd50 con 0x7f15f806e2f0 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f7fff640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15fd25a640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e0004d40 0x7f15e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15fca59640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f806e2f0 0x7f15e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f7fff640 1 -- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f806e2f0 msgr2=0x7f15e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f7fff640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f806e2f0 0x7f15e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f7fff640 1 -- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e0004d40 msgr2=0x7f15e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f7fff640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15e0004d40 0x7f15e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.982+0000 7f15f7fff640 1 -- 172.21.15.26:0/113687253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e0003520 con 0x7f15e0003f60 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.983+0000 7f15f7fff640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15e0004800 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c240 tx=0x7f15e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcb14bf1454799e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.983+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec007d10 con 0x7f15e0003f60 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.983+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ec005ad0 con 0x7f15e0003f60 2024-04-24T23:27:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.983+0000 7f15f77fe640 1 -- 172.21.15.26:0/113687253 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec007070 con 0x7f15e0003f60 2024-04-24T23:27:09.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.984+0000 7f15fecbd640 1 -- 172.21.15.26:0/113687253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 msgr2=0x7f15e0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.984+0000 7f15fecbd640 1 --2- 172.21.15.26:0/113687253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15e0004800 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c240 tx=0x7f15e0005830 comp rx=0 tx=0).stop 2024-04-24T23:27:09.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.984+0000 7f15fd25a640 1 -- 172.21.15.26:0/113687253 reap_dead start 2024-04-24T23:27:09.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.984+0000 7f15fecbd640 1 -- 172.21.15.26:0/113687253 shutdown_connections 2024-04-24T23:27:09.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.984+0000 7f15fecbd640 1 -- 172.21.15.26:0/113687253 >> 172.21.15.26:0/113687253 conn(0x7f15f806b300 msgr2=0x7f15f806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.984+0000 7f15fecbd640 1 -- 172.21.15.26:0/113687253 shutdown_connections 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.984+0000 7f15fecbd640 1 -- 172.21.15.26:0/113687253 wait complete. 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.985+0000 7f15fecbd640 1 Processor -- start 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.985+0000 7f15fecbd640 1 -- start start 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fecbd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15f814a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fecbd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f8155830 0x7f15f81462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fecbd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f8146810 0x7f15f8150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fecbd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15f80b5000 con 0x7f15e0003f60 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fecbd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f15f80b3c60 con 0x7f15f8155830 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fecbd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f15f80b5180 con 0x7f15f8146810 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fca59640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15f814a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fca59640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15f814a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48766/0 (socket says 172.21.15.26:48766) 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fca59640 1 -- 172.21.15.26:0/1978525261 learned_addr learned my addr 172.21.15.26:0/1978525261 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15f7fff640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f8155830 0x7f15f81462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fd25a640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f8146810 0x7f15f8150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fca59640 1 -- 172.21.15.26:0/1978525261 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f8146810 msgr2=0x7f15f8150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fca59640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f8146810 0x7f15f8150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fca59640 1 -- 172.21.15.26:0/1978525261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f8155830 msgr2=0x7f15f81462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fca59640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f8155830 0x7f15f81462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fca59640 1 -- 172.21.15.26:0/1978525261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f8154790 con 0x7f15e0003f60 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fd25a640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f8146810 0x7f15f8150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.986+0000 7f15fca59640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15f814a7b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f15f000fbc0 tx=0x7f15f000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.987+0000 7f15f57fa640 1 -- 172.21.15.26:0/1978525261 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f000dc30 con 0x7f15e0003f60 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.987+0000 7f15f57fa640 1 -- 172.21.15.26:0/1978525261 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f0018070 con 0x7f15e0003f60 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.987+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f8154a70 con 0x7f15e0003f60 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.987+0000 7f15f57fa640 1 -- 172.21.15.26:0/1978525261 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0003970 con 0x7f15e0003f60 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.987+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f8155030 con 0x7f15e0003f60 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.988+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f80694c0 con 0x7f15e0003f60 2024-04-24T23:27:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.988+0000 7f15f57fa640 1 -- 172.21.15.26:0/1978525261 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15f0003b10 con 0x7f15e0003f60 2024-04-24T23:27:09.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.988+0000 7f15f57fa640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15e406be30 0x7f15e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:09.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.989+0000 7f15f57fa640 1 -- 172.21.15.26:0/1978525261 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f15f008b030 con 0x7f15e0003f60 2024-04-24T23:27:09.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.989+0000 7f15f7fff640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15e406be30 0x7f15e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:09.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.989+0000 7f15f7fff640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15e406be30 0x7f15e406e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c240 tx=0x7f15ec02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:09.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:09.993+0000 7f15f57fa640 1 -- 172.21.15.26:0/1978525261 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15f0059a30 con 0x7f15e0003f60 2024-04-24T23:27:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.214+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15f80b44a0 con 0x7f15e0003f60 2024-04-24T23:27:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.215+0000 7f15f57fa640 1 -- 172.21.15.26:0/1978525261 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7f15f0059330 con 0x7f15e0003f60 2024-04-24T23:27:10.216 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:10.216 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:27:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:27:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15e406be30 msgr2=0x7f15e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15e406be30 0x7f15e406e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c240 tx=0x7f15ec02e000 comp rx=0 tx=0).stop 2024-04-24T23:27:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 msgr2=0x7f15f814a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15f814a7b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f15f000fbc0 tx=0x7f15f000d520 comp rx=0 tx=0).stop 2024-04-24T23:27:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 shutdown_connections 2024-04-24T23:27:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f15e406be30 0x7f15e406e2f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f15f8146810 0x7f15f8150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f15f8155830 0x7f15f81462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 --2- 172.21.15.26:0/1978525261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e0003f60 0x7f15f814a7b0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 >> 172.21.15.26:0/1978525261 conn(0x7f15f806b300 msgr2=0x7f15f8072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 shutdown_connections 2024-04-24T23:27:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.217+0000 7f15fecbd640 1 -- 172.21.15.26:0/1978525261 wait complete. 2024-04-24T23:27:10.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:10.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.330+0000 7f34b029e640 1 Processor -- start 2024-04-24T23:27:10.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.330+0000 7f34b029e640 1 -- start start 2024-04-24T23:27:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.330+0000 7f34b029e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f34a815e380 0x7f34a80b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.330+0000 7f34b029e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34a80b4fd0 0x7f34a8072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34b029e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a80b5440 con 0x7f34a80b0030 2024-04-24T23:27:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34b029e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f34a815e760 con 0x7f34a80b4fd0 2024-04-24T23:27:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34b029e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f34a80725b0 con 0x7f34a815e380 2024-04-24T23:27:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34ad839640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34a80b4fd0 0x7f34a8072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34ae83b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f34a80b0030 0x7f34a80b0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60056/0 (socket says 172.21.15.26:60056) 2024-04-24T23:27:10.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34ae83b640 1 -- 172.21.15.26:0/1778776727 learned_addr learned my addr 172.21.15.26:0/1778776727 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34ae03a640 1 --2- 172.21.15.26:0/1778776727 >> v2:172.21.15.29:3300/0 conn(0x7f34a815e380 0x7f34a80b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34ad839640 1 -- 172.21.15.26:0/1778776727 >> v2:172.21.15.29:3300/0 conn(0x7f34a815e380 msgr2=0x7f34a80b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34ad839640 1 --2- 172.21.15.26:0/1778776727 >> v2:172.21.15.29:3300/0 conn(0x7f34a815e380 0x7f34a80b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34ad839640 1 -- 172.21.15.26:0/1778776727 >> v1:172.21.15.26:6789/0 conn(0x7f34a80b0030 legacy=0x7f34a80b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34ad038640 1 -- 172.21.15.26:0/1778776727 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2132902532 0 0) 0x7f34a80b5440 con 0x7f34a80b0030 2024-04-24T23:27:10.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.331+0000 7f34ad839640 1 -- 172.21.15.26:0/1778776727 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a80b5700 con 0x7f34a80b4fd0 2024-04-24T23:27:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.332+0000 7f34ad839640 1 --2- 172.21.15.26:0/1778776727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a80b4fd0 0x7f34a8072a40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f34a806f5f0 tx=0x7f34a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78ecc6c21e56657 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.332+0000 7f34ad038640 1 -- 172.21.15.26:0/1778776727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4037070 con 0x7f34a80b4fd0 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.332+0000 7f34ad038640 1 -- 172.21.15.26:0/1778776727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a4032440 con 0x7f34a80b4fd0 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.332+0000 7f34ad038640 1 -- 172.21.15.26:0/1778776727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4032810 con 0x7f34a80b4fd0 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.332+0000 7f34b029e640 1 -- 172.21.15.26:0/1778776727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a80b4fd0 msgr2=0x7f34a8072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.332+0000 7f34b029e640 1 --2- 172.21.15.26:0/1778776727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a80b4fd0 0x7f34a8072a40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f34a806f5f0 tx=0x7f34a402a230 comp rx=0 tx=0).stop 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.333+0000 7f34b029e640 1 -- 172.21.15.26:0/1778776727 shutdown_connections 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.333+0000 7f34b029e640 1 --2- 172.21.15.26:0/1778776727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a80b4fd0 0x7f34a8072a40 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.333+0000 7f34b029e640 1 --2- 172.21.15.26:0/1778776727 >> v2:172.21.15.29:3300/0 conn(0x7f34a815e380 0x7f34a80b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.333+0000 7f34b029e640 1 -- 172.21.15.26:0/1778776727 >> 172.21.15.26:0/1778776727 conn(0x7f34a806e2c0 msgr2=0x7f34a806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.333+0000 7f34b029e640 1 -- 172.21.15.26:0/1778776727 shutdown_connections 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.333+0000 7f34b029e640 1 -- 172.21.15.26:0/1778776727 wait complete. 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.334+0000 7f34b029e640 1 Processor -- start 2024-04-24T23:27:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.334+0000 7f34b029e640 1 -- start start 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.334+0000 7f34b029e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34a80b0030 0x7f34a80b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.334+0000 7f34b029e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34a80b4fd0 0x7f34a815c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34b029e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a815e380 0x7f34a814d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34b029e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8072f80 con 0x7f34a815e380 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34b029e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8072240 con 0x7f34a80b0030 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34b029e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f34a80ae530 con 0x7f34a80b4fd0 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae83b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a815e380 0x7f34a814d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae83b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a815e380 0x7f34a814d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48786/0 (socket says 172.21.15.26:48786) 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae83b640 1 -- 172.21.15.26:0/2972610716 learned_addr learned my addr 172.21.15.26:0/2972610716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ad839640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34a80b4fd0 0x7f34a815c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae03a640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34a80b0030 0x7f34a80b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae83b640 1 -- 172.21.15.26:0/2972610716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34a80b4fd0 msgr2=0x7f34a815c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae83b640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34a80b4fd0 0x7f34a815c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae83b640 1 -- 172.21.15.26:0/2972610716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34a80b0030 msgr2=0x7f34a80b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae83b640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34a80b0030 0x7f34a80b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae83b640 1 -- 172.21.15.26:0/2972610716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a81f7440 con 0x7f34a815e380 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34ae83b640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a815e380 0x7f34a814d800 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c7d0 tx=0x7f34a000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f349effd640 1 -- 172.21.15.26:0/2972610716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0018070 con 0x7f34a815e380 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f349effd640 1 -- 172.21.15.26:0/2972610716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a000a640 con 0x7f34a815e380 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a81f7720 con 0x7f34a815e380 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f349effd640 1 -- 172.21.15.26:0/2972610716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a000aa70 con 0x7f34a815e380 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.335+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a81f7ce0 con 0x7f34a815e380 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.336+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a80694c0 con 0x7f34a815e380 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.337+0000 7f349effd640 1 -- 172.21.15.26:0/2972610716 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34a001b070 con 0x7f34a815e380 2024-04-24T23:27:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.337+0000 7f349effd640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f348c06be30 0x7f348c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.337+0000 7f34ae03a640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f348c06be30 0x7f348c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.337+0000 7f349effd640 1 -- 172.21.15.26:0/2972610716 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f34a008a4f0 con 0x7f34a815e380 2024-04-24T23:27:10.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.338+0000 7f34ae03a640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f348c06be30 0x7f348c06e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3498009970 tx=0x7f3498008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:10.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.341+0000 7f349effd640 1 -- 172.21.15.26:0/2972610716 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f34a0058e60 con 0x7f34a815e380 2024-04-24T23:27:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.526+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34a806e9e0 con 0x7f34a815e380 2024-04-24T23:27:10.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.527+0000 7f349effd640 1 -- 172.21.15.26:0/2972610716 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16697 (secure 0 0 0) 0x7f34a0058760 con 0x7f34a815e380 2024-04-24T23:27:10.528 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:10.528 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:26:38.480003+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:26:15.824299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:26:16.498456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.7","mappings":[{"from":6,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f348c06be30 msgr2=0x7f348c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f348c06be30 0x7f348c06e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3498009970 tx=0x7f3498008040 comp rx=0 tx=0).stop 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a815e380 msgr2=0x7f34a814d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a815e380 0x7f34a814d800 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c7d0 tx=0x7f34a000cc90 comp rx=0 tx=0).stop 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 shutdown_connections 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f348c06be30 0x7f348c06e2f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a815e380 0x7f34a814d800 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34a80b4fd0 0x7f34a815c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 --2- 172.21.15.26:0/2972610716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34a80b0030 0x7f34a80b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 >> 172.21.15.26:0/2972610716 conn(0x7f34a806e2c0 msgr2=0x7f34a8161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 shutdown_connections 2024-04-24T23:27:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.529+0000 7f34b029e640 1 -- 172.21.15.26:0/2972610716 wait complete. 2024-04-24T23:27:10.541 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:27:10.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:10.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.642+0000 7feb1a761640 1 Processor -- start 2024-04-24T23:27:10.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.642+0000 7feb1a761640 1 -- start start 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.642+0000 7feb1a761640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb1406d350 0x7feb1406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.642+0000 7feb1a761640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb140cd660 0x7feb140cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.642+0000 7feb1a761640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb140cfbc0 con 0x7feb14145910 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.642+0000 7feb1a761640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7feb14145cf0 con 0x7feb1406d350 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.642+0000 7feb1a761640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feb1406d7f0 con 0x7feb140cd660 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.643+0000 7feb137fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7feb1406d350 0x7feb1406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.643+0000 7feb13fff640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7feb14145910 0x7feb1415d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:37884/0 (socket says 172.21.15.26:37884) 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.643+0000 7feb18cfe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb140cd660 0x7feb140cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.643+0000 7feb13fff640 1 -- 172.21.15.26:0/3847211487 learned_addr learned my addr 172.21.15.26:0/3847211487 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.643+0000 7feb18cfe640 1 -- 172.21.15.26:0/3847211487 >> v2:172.21.15.29:3300/0 conn(0x7feb1406d350 msgr2=0x7feb1406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.643+0000 7feb18cfe640 1 --2- 172.21.15.26:0/3847211487 >> v2:172.21.15.29:3300/0 conn(0x7feb1406d350 0x7feb1406e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.643+0000 7feb18cfe640 1 -- 172.21.15.26:0/3847211487 >> v1:172.21.15.29:6789/0 conn(0x7feb14145910 legacy=0x7feb1415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.643+0000 7feb18cfe640 1 -- 172.21.15.26:0/3847211487 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1406dad0 con 0x7feb140cd660 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.644+0000 7feb18cfe640 1 --2- 172.21.15.26:0/3847211487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb140cd660 0x7feb140cdaf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009dc0 tx=0x7feb0c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc334a26a5c83283 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.644+0000 7feb12ffd640 1 -- 172.21.15.26:0/3847211487 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c02ae60 con 0x7feb140cd660 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.644+0000 7feb12ffd640 1 -- 172.21.15.26:0/3847211487 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb0c007740 con 0x7feb140cd660 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.644+0000 7feb12ffd640 1 -- 172.21.15.26:0/3847211487 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c007b10 con 0x7feb140cd660 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.645+0000 7feb1a761640 1 -- 172.21.15.26:0/3847211487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb140cd660 msgr2=0x7feb140cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.645+0000 7feb1a761640 1 --2- 172.21.15.26:0/3847211487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb140cd660 0x7feb140cdaf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009dc0 tx=0x7feb0c02a680 comp rx=0 tx=0).stop 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.645+0000 7feb1a761640 1 -- 172.21.15.26:0/3847211487 shutdown_connections 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.645+0000 7feb1a761640 1 --2- 172.21.15.26:0/3847211487 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb140cd660 0x7feb140cdaf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.645+0000 7feb1a761640 1 --2- 172.21.15.26:0/3847211487 >> v2:172.21.15.29:3300/0 conn(0x7feb1406d350 0x7feb1406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.645+0000 7feb1a761640 1 -- 172.21.15.26:0/3847211487 >> 172.21.15.26:0/3847211487 conn(0x7feb1406b300 msgr2=0x7feb1406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.645+0000 7feb1a761640 1 -- 172.21.15.26:0/3847211487 shutdown_connections 2024-04-24T23:27:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.645+0000 7feb1a761640 1 -- 172.21.15.26:0/3847211487 wait complete. 2024-04-24T23:27:10.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.646+0000 7feb1a761640 1 Processor -- start 2024-04-24T23:27:10.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.646+0000 7feb1a761640 1 -- start start 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb1a761640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1406d350 0x7feb1413bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb1a761640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb140cd660 0x7feb1413e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb1a761640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb14145910 0x7feb141422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb1a761640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feb1406e370 con 0x7feb140cd660 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb1a761640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feb1406e090 con 0x7feb14145910 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb1a761640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feb1415d770 con 0x7feb1406d350 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb140cd660 0x7feb1413e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb140cd660 0x7feb1413e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48788/0 (socket says 172.21.15.26:48788) 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb137fe640 1 -- 172.21.15.26:0/3979261558 learned_addr learned my addr 172.21.15.26:0/3979261558 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb13fff640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1406d350 0x7feb1413bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb18cfe640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb14145910 0x7feb141422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb137fe640 1 -- 172.21.15.26:0/3979261558 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1406d350 msgr2=0x7feb1413bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb137fe640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1406d350 0x7feb1413bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb137fe640 1 -- 172.21.15.26:0/3979261558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb14145910 msgr2=0x7feb141422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb137fe640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb14145910 0x7feb141422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb137fe640 1 -- 172.21.15.26:0/3979261558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1413c2d0 con 0x7feb140cd660 2024-04-24T23:27:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb137fe640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb140cd660 0x7feb1413e510 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7feb0800e9b0 tx=0x7feb0800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.647+0000 7feb10ff9640 1 -- 172.21.15.26:0/3979261558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08017070 con 0x7feb140cd660 2024-04-24T23:27:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.648+0000 7feb10ff9640 1 -- 172.21.15.26:0/3979261558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb0800c9a0 con 0x7feb140cd660 2024-04-24T23:27:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.648+0000 7feb10ff9640 1 -- 172.21.15.26:0/3979261558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0800cd90 con 0x7feb140cd660 2024-04-24T23:27:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.648+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1413eff0 con 0x7feb140cd660 2024-04-24T23:27:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.648+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1413f5b0 con 0x7feb140cd660 2024-04-24T23:27:10.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.649+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb140694c0 con 0x7feb140cd660 2024-04-24T23:27:10.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.649+0000 7feb10ff9640 1 -- 172.21.15.26:0/3979261558 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb0801a070 con 0x7feb140cd660 2024-04-24T23:27:10.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.649+0000 7feb10ff9640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0006be30 0x7feb0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.650+0000 7feb10ff9640 1 -- 172.21.15.26:0/3979261558 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7feb0808aeb0 con 0x7feb140cd660 2024-04-24T23:27:10.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.650+0000 7feb13fff640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0006be30 0x7feb0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.650+0000 7feb13fff640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0006be30 0x7feb0006e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feb04000f60 tx=0x7feb04009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:10.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.654+0000 7feb10ff9640 1 -- 172.21.15.26:0/3979261558 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feb08059820 con 0x7feb140cd660 2024-04-24T23:27:10.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.875+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb1415f120 con 0x7feb140cd660 2024-04-24T23:27:10.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.876+0000 7feb10ff9640 1 -- 172.21.15.26:0/3979261558 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7feb08059120 con 0x7feb140cd660 2024-04-24T23:27:10.877 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:10.877 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:27:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0006be30 msgr2=0x7feb0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0006be30 0x7feb0006e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feb04000f60 tx=0x7feb04009040 comp rx=0 tx=0).stop 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb140cd660 msgr2=0x7feb1413e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb140cd660 0x7feb1413e510 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7feb0800e9b0 tx=0x7feb0800ee70 comp rx=0 tx=0).stop 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 shutdown_connections 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feb0006be30 0x7feb0006e2f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb14145910 0x7feb141422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb140cd660 0x7feb1413e510 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 --2- 172.21.15.26:0/3979261558 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feb1406d350 0x7feb1413bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 >> 172.21.15.26:0/3979261558 conn(0x7feb1406b300 msgr2=0x7feb141442f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:10.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 shutdown_connections 2024-04-24T23:27:10.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.878+0000 7feb1a761640 1 -- 172.21.15.26:0/3979261558 wait complete. 2024-04-24T23:27:10.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:10.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.990+0000 7ff65f8bd640 1 Processor -- start 2024-04-24T23:27:10.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.990+0000 7ff65f8bd640 1 -- start start 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65f8bd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff65806f4f0 0x7ff6580ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65f8bd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6580addb0 0x7ff658154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65f8bd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6580b0220 con 0x7ff65806f110 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65f8bd640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff6580af180 con 0x7ff65806f4f0 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65f8bd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff6580af460 con 0x7ff6580addb0 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65ce58640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff65806f4f0 0x7ff6580ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65d659640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff65806f110 0x7ff65815f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60058/0 (socket says 172.21.15.26:60058) 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65d659640 1 -- 172.21.15.26:0/1295430739 learned_addr learned my addr 172.21.15.26:0/1295430739 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65de5a640 1 --2- 172.21.15.26:0/1295430739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6580addb0 0x7ff658154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65ce58640 1 -- 172.21.15.26:0/1295430739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6580addb0 msgr2=0x7ff658154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65ce58640 1 --2- 172.21.15.26:0/1295430739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6580addb0 0x7ff658154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65ce58640 1 -- 172.21.15.26:0/1295430739 >> v1:172.21.15.26:6789/0 conn(0x7ff65806f110 legacy=0x7ff65815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.991+0000 7ff65ce58640 1 -- 172.21.15.26:0/1295430739 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6580af740 con 0x7ff65806f4f0 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.992+0000 7ff65ce58640 1 --2- 172.21.15.26:0/1295430739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f4f0 0x7ff6580ad870 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff6480097a0 tx=0x7ff64802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b50063f387a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.992+0000 7ff64ffff640 1 -- 172.21.15.26:0/1295430739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648037070 con 0x7ff65806f4f0 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.992+0000 7ff64ffff640 1 -- 172.21.15.26:0/1295430739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff648032440 con 0x7ff65806f4f0 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.992+0000 7ff64ffff640 1 -- 172.21.15.26:0/1295430739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648032810 con 0x7ff65806f4f0 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.992+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1295430739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f4f0 msgr2=0x7ff6580ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.992+0000 7ff65f8bd640 1 --2- 172.21.15.26:0/1295430739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f4f0 0x7ff6580ad870 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff6480097a0 tx=0x7ff64802a180 comp rx=0 tx=0).stop 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.993+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1295430739 shutdown_connections 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.993+0000 7ff65f8bd640 1 --2- 172.21.15.26:0/1295430739 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff6580addb0 0x7ff658154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.993+0000 7ff65f8bd640 1 --2- 172.21.15.26:0/1295430739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f4f0 0x7ff6580ad870 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.993+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1295430739 >> 172.21.15.26:0/1295430739 conn(0x7ff6580729a0 msgr2=0x7ff65806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.993+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1295430739 shutdown_connections 2024-04-24T23:27:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.993+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1295430739 wait complete. 2024-04-24T23:27:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.994+0000 7ff65f8bd640 1 Processor -- start 2024-04-24T23:27:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.994+0000 7ff65f8bd640 1 -- start start 2024-04-24T23:27:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.994+0000 7ff65f8bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f110 0x7ff6581474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.994+0000 7ff65f8bd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff65806f4f0 0x7ff6581e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.994+0000 7ff65f8bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6580addb0 0x7ff6580b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.994+0000 7ff65f8bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff6580aee50 con 0x7ff65806f110 2024-04-24T23:27:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.994+0000 7ff65f8bd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff6580afd00 con 0x7ff6580addb0 2024-04-24T23:27:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.994+0000 7ff65f8bd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff65806fb00 con 0x7ff65806f4f0 2024-04-24T23:27:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65d659640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f110 0x7ff6581474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65d659640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f110 0x7ff6581474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48792/0 (socket says 172.21.15.26:48792) 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65d659640 1 -- 172.21.15.26:0/1410100105 learned_addr learned my addr 172.21.15.26:0/1410100105 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65ce58640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff65806f4f0 0x7ff6581e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65d659640 1 -- 172.21.15.26:0/1410100105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff65806f4f0 msgr2=0x7ff6581e8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65de5a640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6580addb0 0x7ff6580b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65d659640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff65806f4f0 0x7ff6581e8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65d659640 1 -- 172.21.15.26:0/1410100105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6580addb0 msgr2=0x7ff6580b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65d659640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6580addb0 0x7ff6580b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65d659640 1 -- 172.21.15.26:0/1410100105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6581e9b70 con 0x7ff65806f110 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65d659640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f110 0x7ff6581474f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c430 tx=0x7ff65400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff64dffb640 1 -- 172.21.15.26:0/1410100105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654017070 con 0x7ff65806f110 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff64dffb640 1 -- 172.21.15.26:0/1410100105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff654005ce0 con 0x7ff65806f110 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6581e95b0 con 0x7ff65806f110 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.995+0000 7ff64dffb640 1 -- 172.21.15.26:0/1410100105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6540072f0 con 0x7ff65806f110 2024-04-24T23:27:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.996+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff658147f00 con 0x7ff65806f110 2024-04-24T23:27:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.997+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6580694c0 con 0x7ff65806f110 2024-04-24T23:27:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.997+0000 7ff64dffb640 1 -- 172.21.15.26:0/1410100105 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff65401a070 con 0x7ff65806f110 2024-04-24T23:27:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.997+0000 7ff64dffb640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff63c06be30 0x7ff63c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.997+0000 7ff64dffb640 1 -- 172.21.15.26:0/1410100105 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7ff65408a240 con 0x7ff65806f110 2024-04-24T23:27:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.997+0000 7ff65ce58640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff63c06be30 0x7ff63c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:10.998+0000 7ff65ce58640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff63c06be30 0x7ff63c06e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff648034000 tx=0x7ff64802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:11.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.002+0000 7ff64dffb640 1 -- 172.21.15.26:0/1410100105 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff654058c40 con 0x7ff65806f110 2024-04-24T23:27:11.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.221+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff658148270 con 0x7ff65806f110 2024-04-24T23:27:11.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.221+0000 7ff64dffb640 1 -- 172.21.15.26:0/1410100105 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5227 (secure 0 0 0) 0x7ff654058540 con 0x7ff65806f110 2024-04-24T23:27:11.222 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:11.222 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:26:17.523402+0000","modified":"2024-04-24T23:26:55.476419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6790,"mds_1":6778,"mds_2":6299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6299":{"gid":6299,"name":"d","rank":2,"incarnation":88,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6835/3727846607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":3727846607},{"type":"v1","addr":"172.21.15.29:6835","nonce":3727846607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6778":{"gid":6778,"name":"a","rank":1,"incarnation":86,"state":"up:active","state_seq":12,"addr":"172.21.15.26:6836/306957409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":306957409},{"type":"v1","addr":"172.21.15.26:6836","nonce":306957409}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6790":{"gid":6790,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":12,"addr":"172.21.15.29:6834/2253149800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2253149800},{"type":"v1","addr":"172.21.15.29:6834","nonce":2253149800}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6790,"qdb_cluster":[6790,6778,6299]},"id":6}]} 2024-04-24T23:27:11.223 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 94 2024-04-24T23:27:11.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff63c06be30 msgr2=0x7ff63c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff63c06be30 0x7ff63c06e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff648034000 tx=0x7ff64802e000 comp rx=0 tx=0).stop 2024-04-24T23:27:11.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f110 msgr2=0x7ff6581474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f110 0x7ff6581474f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c430 tx=0x7ff65400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 shutdown_connections 2024-04-24T23:27:11.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff63c06be30 0x7ff63c06e2f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6580addb0 0x7ff6580b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff65806f4f0 0x7ff6581e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 --2- 172.21.15.26:0/1410100105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65806f110 0x7ff6581474f0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 >> 172.21.15.26:0/1410100105 conn(0x7ff6580729a0 msgr2=0x7ff6580b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:11.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 shutdown_connections 2024-04-24T23:27:11.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.223+0000 7ff65f8bd640 1 -- 172.21.15.26:0/1410100105 wait complete. 2024-04-24T23:27:11.235 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:11.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.335+0000 7fde629a9640 1 Processor -- start 2024-04-24T23:27:11.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.335+0000 7fde629a9640 1 -- start start 2024-04-24T23:27:11.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.336+0000 7fde629a9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fde5c06f110 0x7fde5c15f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.336+0000 7fde629a9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde5c06f4f0 0x7fde5c0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.336+0000 7fde629a9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde5c15f750 con 0x7fde5c0addb0 2024-04-24T23:27:11.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.336+0000 7fde629a9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fde5c0b0220 con 0x7fde5c06f4f0 2024-04-24T23:27:11.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.336+0000 7fde629a9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fde5c0af180 con 0x7fde5c06f110 2024-04-24T23:27:11.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.336+0000 7fde5b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde5c06f4f0 0x7fde5c0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.336+0000 7fde60f46640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fde5c0addb0 0x7fde5c154a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60072/0 (socket says 172.21.15.26:60072) 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.336+0000 7fde60f46640 1 -- 172.21.15.26:0/4077199454 learned_addr learned my addr 172.21.15.26:0/4077199454 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.337+0000 7fde5b7fe640 1 -- 172.21.15.26:0/4077199454 >> v2:172.21.15.29:3300/0 conn(0x7fde5c06f110 msgr2=0x7fde5c15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.337+0000 7fde5bfff640 1 --2- 172.21.15.26:0/4077199454 >> v2:172.21.15.29:3300/0 conn(0x7fde5c06f110 0x7fde5c15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.337+0000 7fde5b7fe640 1 --2- 172.21.15.26:0/4077199454 >> v2:172.21.15.29:3300/0 conn(0x7fde5c06f110 0x7fde5c15f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.337+0000 7fde5affd640 1 -- 172.21.15.26:0/4077199454 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2903282813 0 0) 0x7fde54001490 con 0x7fde5c0addb0 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.337+0000 7fde5b7fe640 1 -- 172.21.15.26:0/4077199454 >> v1:172.21.15.26:6789/0 conn(0x7fde5c0addb0 legacy=0x7fde5c154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.337+0000 7fde5b7fe640 1 -- 172.21.15.26:0/4077199454 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5c0af460 con 0x7fde5c06f4f0 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.337+0000 7fde5bfff640 1 --2- 172.21.15.26:0/4077199454 >> v2:172.21.15.29:3300/0 conn(0x7fde5c06f110 0x7fde5c15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.337+0000 7fde5b7fe640 1 --2- 172.21.15.26:0/4077199454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 0x7fde5c0ad870 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fde5c072f30 tx=0x7fde5002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e026f73ebbebf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde5affd640 1 -- 172.21.15.26:0/4077199454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50037070 con 0x7fde5c06f4f0 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde5affd640 1 -- 172.21.15.26:0/4077199454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde50032440 con 0x7fde5c06f4f0 2024-04-24T23:27:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde5affd640 1 -- 172.21.15.26:0/4077199454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50032810 con 0x7fde5c06f4f0 2024-04-24T23:27:11.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde629a9640 1 -- 172.21.15.26:0/4077199454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 msgr2=0x7fde5c0ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde629a9640 1 --2- 172.21.15.26:0/4077199454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 0x7fde5c0ad870 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fde5c072f30 tx=0x7fde5002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde629a9640 1 -- 172.21.15.26:0/4077199454 shutdown_connections 2024-04-24T23:27:11.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde629a9640 1 --2- 172.21.15.26:0/4077199454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 0x7fde5c0ad870 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde629a9640 1 --2- 172.21.15.26:0/4077199454 >> v2:172.21.15.29:3300/0 conn(0x7fde5c06f110 0x7fde5c15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde629a9640 1 -- 172.21.15.26:0/4077199454 >> 172.21.15.26:0/4077199454 conn(0x7fde5c0729a0 msgr2=0x7fde5c06b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:11.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde629a9640 1 -- 172.21.15.26:0/4077199454 shutdown_connections 2024-04-24T23:27:11.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.338+0000 7fde629a9640 1 -- 172.21.15.26:0/4077199454 wait complete. 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.339+0000 7fde629a9640 1 Processor -- start 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.340+0000 7fde629a9640 1 -- start start 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.340+0000 7fde629a9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5c06f110 0x7fde5c1e9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.340+0000 7fde629a9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 0x7fde5c151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.340+0000 7fde629a9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde5c0addb0 0x7fde5c151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.340+0000 7fde629a9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c0afd00 con 0x7fde5c06f4f0 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.340+0000 7fde629a9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c0afa20 con 0x7fde5c06f110 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.340+0000 7fde629a9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fde5c0aee50 con 0x7fde5c0addb0 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.340+0000 7fde5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 0x7fde5c151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 0x7fde5c151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48800/0 (socket says 172.21.15.26:48800) 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde5b7fe640 1 -- 172.21.15.26:0/55447362 learned_addr learned my addr 172.21.15.26:0/55447362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde60f46640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde5c0addb0 0x7fde5c151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde5bfff640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5c06f110 0x7fde5c1e9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde5b7fe640 1 -- 172.21.15.26:0/55447362 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde5c0addb0 msgr2=0x7fde5c151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde5b7fe640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde5c0addb0 0x7fde5c151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde5b7fe640 1 -- 172.21.15.26:0/55447362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5c06f110 msgr2=0x7fde5c1e9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde5b7fe640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5c06f110 0x7fde5c1e9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde5b7fe640 1 -- 172.21.15.26:0/55447362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5c153d30 con 0x7fde5c06f4f0 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde5b7fe640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 0x7fde5c151870 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fde50009890 tx=0x7fde500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde58ff9640 1 -- 172.21.15.26:0/55447362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50037040 con 0x7fde5c06f4f0 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde58ff9640 1 -- 172.21.15.26:0/55447362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde50042070 con 0x7fde5c06f4f0 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde58ff9640 1 -- 172.21.15.26:0/55447362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50030a50 con 0x7fde5c06f4f0 2024-04-24T23:27:11.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5c154010 con 0x7fde5c06f4f0 2024-04-24T23:27:11.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.341+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde5c1545d0 con 0x7fde5c06f4f0 2024-04-24T23:27:11.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.342+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde5c1549b0 con 0x7fde5c06f4f0 2024-04-24T23:27:11.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.343+0000 7fde58ff9640 1 -- 172.21.15.26:0/55447362 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde50030bf0 con 0x7fde5c06f4f0 2024-04-24T23:27:11.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.343+0000 7fde58ff9640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde4806be30 0x7fde4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.343+0000 7fde58ff9640 1 -- 172.21.15.26:0/55447362 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7fde500ab830 con 0x7fde5c06f4f0 2024-04-24T23:27:11.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.343+0000 7fde5bfff640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde4806be30 0x7fde4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.344+0000 7fde5bfff640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde4806be30 0x7fde4806e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fde5c1f7310 tx=0x7fde4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.348+0000 7fde58ff9640 1 -- 172.21.15.26:0/55447362 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde5c1549b0 con 0x7fde5c06f4f0 2024-04-24T23:27:11.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.532+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde5c0694c0 con 0x7fde5c06f4f0 2024-04-24T23:27:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.534+0000 7fde58ff9640 1 -- 172.21.15.26:0/55447362 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16697 (secure 0 0 0) 0x7fde50079bf0 con 0x7fde5c06f4f0 2024-04-24T23:27:11.535 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:11.535 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:26:38.480003+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:26:15.824299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T23:26:16.498456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.7","mappings":[{"from":6,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.535+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde4806be30 msgr2=0x7fde4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.535+0000 7fde629a9640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde4806be30 0x7fde4806e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fde5c1f7310 tx=0x7fde4c008040 comp rx=0 tx=0).stop 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.535+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 msgr2=0x7fde5c151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.535+0000 7fde629a9640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 0x7fde5c151870 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fde50009890 tx=0x7fde500046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.536+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 shutdown_connections 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.536+0000 7fde629a9640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fde4806be30 0x7fde4806e2f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.536+0000 7fde629a9640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fde5c0addb0 0x7fde5c151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.536+0000 7fde629a9640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c06f4f0 0x7fde5c151870 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.536+0000 7fde629a9640 1 --2- 172.21.15.26:0/55447362 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde5c06f110 0x7fde5c1e9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.536+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 >> 172.21.15.26:0/55447362 conn(0x7fde5c0729a0 msgr2=0x7fde5c0b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.536+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 shutdown_connections 2024-04-24T23:27:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.536+0000 7fde629a9640 1 -- 172.21.15.26:0/55447362 wait complete. 2024-04-24T23:27:11.548 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:27:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.649+0000 7f1ec707d640 1 Processor -- start 2024-04-24T23:27:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.649+0000 7f1ec707d640 1 -- start start 2024-04-24T23:27:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec707d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ec0145980 0x7f1ec0073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec707d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec015c7c0 con 0x7f1ec006b830 2024-04-24T23:27:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec707d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec015caa0 con 0x7f1ec015d7e0 2024-04-24T23:27:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec707d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec0145d60 con 0x7f1ec0145980 2024-04-24T23:27:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec4e19640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ec0145980 0x7f1ec0073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec561a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ec006b830 0x7f1ec00721a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60080/0 (socket says 172.21.15.26:60080) 2024-04-24T23:27:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec561a640 1 -- 172.21.15.26:0/615135868 learned_addr learned my addr 172.21.15.26:0/615135868 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec4e19640 1 -- 172.21.15.26:0/615135868 >> v1:172.21.15.29:6789/0 conn(0x7f1ec015d7e0 legacy=0x7f1ec006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec4e19640 1 -- 172.21.15.26:0/615135868 >> v1:172.21.15.26:6789/0 conn(0x7f1ec006b830 legacy=0x7f1ec00721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ec4e19640 1 -- 172.21.15.26:0/615135868 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec015cd80 con 0x7f1ec0145980 2024-04-24T23:27:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.650+0000 7f1ebf7fe640 1 -- 172.21.15.26:0/615135868 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 773637325 0 0) 0x7f1ec015c7c0 con 0x7f1ec006b830 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.651+0000 7f1ec4e19640 1 --2- 172.21.15.26:0/615135868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec0145980 0x7f1ec0073240 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40097a0 tx=0x7f1eb402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a871795ac20ab5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.651+0000 7f1ebf7fe640 1 -- 172.21.15.26:0/615135868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4037070 con 0x7f1ec0145980 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.651+0000 7f1ebf7fe640 1 -- 172.21.15.26:0/615135868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb4032440 con 0x7f1ec0145980 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.651+0000 7f1ebf7fe640 1 -- 172.21.15.26:0/615135868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4032810 con 0x7f1ec0145980 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.652+0000 7f1ec707d640 1 -- 172.21.15.26:0/615135868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec0145980 msgr2=0x7f1ec0073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.652+0000 7f1ec707d640 1 --2- 172.21.15.26:0/615135868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec0145980 0x7f1ec0073240 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40097a0 tx=0x7f1eb402a230 comp rx=0 tx=0).stop 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.652+0000 7f1ec707d640 1 -- 172.21.15.26:0/615135868 shutdown_connections 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.652+0000 7f1ec707d640 1 --2- 172.21.15.26:0/615135868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec0145980 0x7f1ec0073240 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.652+0000 7f1ec707d640 1 -- 172.21.15.26:0/615135868 >> 172.21.15.26:0/615135868 conn(0x7f1ec006e2c0 msgr2=0x7f1ec006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.652+0000 7f1ec707d640 1 -- 172.21.15.26:0/615135868 shutdown_connections 2024-04-24T23:27:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.652+0000 7f1ec707d640 1 -- 172.21.15.26:0/615135868 wait complete. 2024-04-24T23:27:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.653+0000 7f1ec707d640 1 Processor -- start 2024-04-24T23:27:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.653+0000 7f1ec707d640 1 -- start start 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec707d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec006b830 0x7f1ec0142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec707d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ec0145980 0x7f1ec01438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec707d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ec015d7e0 0x7f1ec00cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec707d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00cdad0 con 0x7f1ec006b830 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec707d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00cd950 con 0x7f1ec0145980 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec707d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec00cdc50 con 0x7f1ec015d7e0 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec4e19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec006b830 0x7f1ec0142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec4e19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec006b830 0x7f1ec0142270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48818/0 (socket says 172.21.15.26:48818) 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec4e19640 1 -- 172.21.15.26:0/2529323880 learned_addr learned my addr 172.21.15.26:0/2529323880 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec4e19640 1 -- 172.21.15.26:0/2529323880 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ec015d7e0 msgr2=0x7f1ec00cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec4e19640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ec015d7e0 0x7f1ec00cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ebffff640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ec0145980 0x7f1ec01438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec4e19640 1 -- 172.21.15.26:0/2529323880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ec0145980 msgr2=0x7f1ec01438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec4e19640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ec0145980 0x7f1ec01438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec4e19640 1 -- 172.21.15.26:0/2529323880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec0143e30 con 0x7f1ec006b830 2024-04-24T23:27:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ebffff640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ec0145980 0x7f1ec01438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.654+0000 7f1ec4e19640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec006b830 0x7f1ec0142270 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4000c00 tx=0x7f1eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.655+0000 7f1ebd7fa640 1 -- 172.21.15.26:0/2529323880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4041070 con 0x7f1ec006b830 2024-04-24T23:27:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.655+0000 7f1ebd7fa640 1 -- 172.21.15.26:0/2529323880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb403c930 con 0x7f1ec006b830 2024-04-24T23:27:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.655+0000 7f1ebd7fa640 1 -- 172.21.15.26:0/2529323880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4037040 con 0x7f1ec006b830 2024-04-24T23:27:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.655+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec01428b0 con 0x7f1ec006b830 2024-04-24T23:27:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.655+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec01430f0 con 0x7f1ec006b830 2024-04-24T23:27:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.656+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec00694c0 con 0x7f1ec006b830 2024-04-24T23:27:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.656+0000 7f1ebd7fa640 1 -- 172.21.15.26:0/2529323880 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1eb4044070 con 0x7f1ec006b830 2024-04-24T23:27:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.657+0000 7f1ebd7fa640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ea406be30 0x7f1ea406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.657+0000 7f1ebd7fa640 1 -- 172.21.15.26:0/2529323880 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5767+0+0 (secure 0 0 0) 0x7f1eb40abb50 con 0x7f1ec006b830 2024-04-24T23:27:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.657+0000 7f1ebffff640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ea406be30 0x7f1ea406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.657+0000 7f1ebffff640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ea406be30 0x7f1ea406e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00d0d30 tx=0x7f1eac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.661+0000 7f1ebd7fa640 1 -- 172.21.15.26:0/2529323880 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ec01428b0 con 0x7f1ec006b830 2024-04-24T23:27:11.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.883+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1ec00b3700 con 0x7f1ec006b830 2024-04-24T23:27:11.884 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:27:11.884+0000 7f26a0f44640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:27:11.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.885+0000 7f1ebd7fa640 1 -- 172.21.15.26:0/2529323880 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) ==== 161+0+0 (secure 0 0 0) 0x7f1eb407a5e0 con 0x7f1ec006b830 2024-04-24T23:27:11.886 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.887+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ea406be30 msgr2=0x7f1ea406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.887+0000 7f1ec707d640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ea406be30 0x7f1ea406e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00d0d30 tx=0x7f1eac009040 comp rx=0 tx=0).stop 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.887+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec006b830 msgr2=0x7f1ec0142270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.887+0000 7f1ec707d640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec006b830 0x7f1ec0142270 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4000c00 tx=0x7f1eb40046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.888+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 shutdown_connections 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.888+0000 7f1ec707d640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1ea406be30 0x7f1ea406e2f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.888+0000 7f1ec707d640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1ec015d7e0 0x7f1ec00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.888+0000 7f1ec707d640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1ec0145980 0x7f1ec01438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.888+0000 7f1ec707d640 1 --2- 172.21.15.26:0/2529323880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec006b830 0x7f1ec0142270 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.888+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 >> 172.21.15.26:0/2529323880 conn(0x7f1ec006e2c0 msgr2=0x7f1ec00b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.888+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 shutdown_connections 2024-04-24T23:27:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:11.888+0000 7f1ec707d640 1 -- 172.21.15.26:0/2529323880 wait complete. 2024-04-24T23:27:11.894 INFO:tasks.ceph.mds.a.smithi026.stderr: -1225> 2024-04-24T23:26:51.401+0000 7f6800b1f640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1) n(v2 rc2024-04-24T23:26:34.115974+0000 4=0+4) (inest lock) (ifile excl) caps={6901=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6901 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x563badca3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:34.115974+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-04-24T23:26:34.115974+0000 4=0+4)","error_str":""},"return_code":-61} 2024-04-24T23:27:11.894 INFO:tasks.ceph.mds.a.smithi026.stderr: -1087> 2024-04-24T23:26:51.403+0000 7f680732c640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1) n(v1 rc2024-04-24T23:26:34.115974+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={6901=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x563bacd47080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:33.884977+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.883977+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-04-24T23:26:34.115974+0000 3=0+3)","error_str":""},"return_code":-61} 2024-04-24T23:27:11.899 INFO:tasks.ceph.mds.d.smithi029.stderr: -1122> 2024-04-24T23:26:51.402+0000 7f8211070640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-04-24T23:26:33.884977+0000 1=0+1) n(v0 rc2024-04-24T23:26:33.884977+0000 2=0+2) (inest lock) (ifile excl) caps={6901=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6901 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5632ac726580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.884977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:33.884977+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.884977+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-04-24T23:26:33.884977+0000 2=0+2)","error_str":""},"return_code":-61} 2024-04-24T23:27:11.899 INFO:tasks.ceph.mds.d.smithi029.stderr: -1102> 2024-04-24T23:26:51.402+0000 7f8211070640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={6901=pAsLsXsFsx/AsLsXsFsx@2},l=6901 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5632ac726b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-04-24T23:27:11.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:27:11.906 INFO:tasks.ceph.mds.b.smithi029.stderr: -2482> 2024-04-24T23:26:51.401+0000 7f43fc153640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1) n(v3 rc2024-04-24T23:26:34.115974+0000 6=0+6) (inest mix dirty) (ifile excl) caps={6901=pAsLsXsFsx/pAsLsXsFsxcral@4},l=6901 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x556ffeee3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:34.115974+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-04-24T23:26:34.115974+0000 6=0+6)","error_str":""},"return_code":-61} 2024-04-24T23:27:11.906 INFO:tasks.ceph.mds.b.smithi029.stderr: -2360> 2024-04-24T23:26:51.402+0000 7f4402960640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1) n(v4 rc2024-04-24T23:26:34.115974+0000 5=0+5) (inest sync->mix g=1,2) caps={6901=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x556ffeee3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:26:34.115974+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-04-24T23:26:33.882977+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-04-24T23:26:34.115974+0000 5=0+5)","error_str":""},"return_code":-61} 2024-04-24T23:27:11.920 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:27:11.929 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:27:11.935 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:27:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.003+0000 7f0306f42640 1 Processor -- start 2024-04-24T23:27:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.003+0000 7f0306f42640 1 -- start start 2024-04-24T23:27:12.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.004+0000 7f0306f42640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f030015adc0 0x7f03000b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:12.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.004+0000 7f0306f42640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f030015ef60 con 0x7f03000d0750 2024-04-24T23:27:12.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.004+0000 7f0306f42640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f030015f240 con 0x7f03000ad7b0 2024-04-24T23:27:12.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.004+0000 7f0306f42640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f030015f520 con 0x7f030015adc0 2024-04-24T23:27:12.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.004+0000 7f03054df640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f03000d0750 0x7f0300072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60094/0 (socket says 172.21.15.26:60094) 2024-04-24T23:27:12.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.004+0000 7f03054df640 1 -- 172.21.15.26:0/1923932780 learned_addr learned my addr 172.21.15.26:0/1923932780 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:12.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.004+0000 7f0304cde640 1 --2- 172.21.15.26:0/1923932780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f030015adc0 0x7f03000b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:12.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.005+0000 7f02ff7fe640 1 -- 172.21.15.26:0/1923932780 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 829479388 0 0) 0x7f030015ef60 con 0x7f03000d0750 2024-04-24T23:27:12.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.005+0000 7f02ff7fe640 1 -- 172.21.15.26:0/1923932780 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02ec003520 con 0x7f03000d0750 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.005+0000 7f02ff7fe640 1 -- 172.21.15.26:0/1923932780 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759176861 0 0) 0x7f030015f240 con 0x7f03000ad7b0 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.005+0000 7f0304cde640 1 -- 172.21.15.26:0/1923932780 >> v1:172.21.15.29:6789/0 conn(0x7f03000ad7b0 legacy=0x7f03000d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.005+0000 7f0304cde640 1 -- 172.21.15.26:0/1923932780 >> v1:172.21.15.26:6789/0 conn(0x7f03000d0750 legacy=0x7f0300072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.005+0000 7f0304cde640 1 -- 172.21.15.26:0/1923932780 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030015f800 con 0x7f030015adc0 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.005+0000 7f0304cde640 1 --2- 172.21.15.26:0/1923932780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f030015adc0 0x7f03000b4370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009dc0 tx=0x7f02f802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a8fe2ce2ea84ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f02ff7fe640 1 -- 172.21.15.26:0/1923932780 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f802ae60 con 0x7f030015adc0 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f02ff7fe640 1 -- 172.21.15.26:0/1923932780 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f8007740 con 0x7f030015adc0 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f02ff7fe640 1 -- 172.21.15.26:0/1923932780 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8007b10 con 0x7f030015adc0 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f0306f42640 1 -- 172.21.15.26:0/1923932780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f030015adc0 msgr2=0x7f03000b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f0306f42640 1 --2- 172.21.15.26:0/1923932780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f030015adc0 0x7f03000b4370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009dc0 tx=0x7f02f802a640 comp rx=0 tx=0).stop 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f0306f42640 1 -- 172.21.15.26:0/1923932780 shutdown_connections 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f0306f42640 1 --2- 172.21.15.26:0/1923932780 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f030015adc0 0x7f03000b4370 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f0306f42640 1 -- 172.21.15.26:0/1923932780 >> 172.21.15.26:0/1923932780 conn(0x7f030006b540 msgr2=0x7f030006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f0306f42640 1 -- 172.21.15.26:0/1923932780 shutdown_connections 2024-04-24T23:27:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.006+0000 7f0306f42640 1 -- 172.21.15.26:0/1923932780 wait complete. 2024-04-24T23:27:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.007+0000 7f0306f42640 1 Processor -- start 2024-04-24T23:27:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.007+0000 7f0306f42640 1 -- start start 2024-04-24T23:27:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f0306f42640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03000ad7b0 0x7f03001584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f0306f42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03000d0750 0x7f03001698c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f0306f42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030015adc0 0x7f0300169e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f0306f42640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f030015b1a0 con 0x7f030015adc0 2024-04-24T23:27:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f0306f42640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f03000b48b0 con 0x7f03000d0750 2024-04-24T23:27:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f0306f42640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f030015e9c0 con 0x7f03000ad7b0 2024-04-24T23:27:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f03054df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030015adc0 0x7f0300169e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f03054df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030015adc0 0x7f0300169e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48838/0 (socket says 172.21.15.26:48838) 2024-04-24T23:27:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f03054df640 1 -- 172.21.15.26:0/1197077744 learned_addr learned my addr 172.21.15.26:0/1197077744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.008+0000 7f0304cde640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03000ad7b0 0x7f03001584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f02fffff640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03000d0750 0x7f03001698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f03054df640 1 -- 172.21.15.26:0/1197077744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03000ad7b0 msgr2=0x7f03001584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f03054df640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03000ad7b0 0x7f03001584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f03054df640 1 -- 172.21.15.26:0/1197077744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03000d0750 msgr2=0x7f03001698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f03054df640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03000d0750 0x7f03001698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f03054df640 1 -- 172.21.15.26:0/1197077744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0300158cf0 con 0x7f030015adc0 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f02fffff640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03000d0750 0x7f03001698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f03054df640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030015adc0 0x7f0300169e00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f02f000c460 tx=0x7f02f000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f02fd7fa640 1 -- 172.21.15.26:0/1197077744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0017070 con 0x7f030015adc0 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f02fd7fa640 1 -- 172.21.15.26:0/1197077744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f0005ce0 con 0x7f030015adc0 2024-04-24T23:27:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0300158fd0 con 0x7f030015adc0 2024-04-24T23:27:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f02fd7fa640 1 -- 172.21.15.26:0/1197077744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0007310 con 0x7f030015adc0 2024-04-24T23:27:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.009+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0300159590 con 0x7f030015adc0 2024-04-24T23:27:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.010+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0300075610 con 0x7f030015adc0 2024-04-24T23:27:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.011+0000 7f02fd7fa640 1 -- 172.21.15.26:0/1197077744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02f001a070 con 0x7f030015adc0 2024-04-24T23:27:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.011+0000 7f02fd7fa640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02e006be30 0x7f02e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.011+0000 7f0304cde640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02e006be30 0x7f02e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.011+0000 7f02fd7fa640 1 -- 172.21.15.26:0/1197077744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6025+0+0 (secure 0 0 0) 0x7f02f008a320 con 0x7f030015adc0 2024-04-24T23:27:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.012+0000 7f0304cde640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02e006be30 0x7f02e006e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f02f8034000 tx=0x7f02f802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.016+0000 7f02fd7fa640 1 -- 172.21.15.26:0/1197077744 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02f0058540 con 0x7f030015adc0 2024-04-24T23:27:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.237+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f03000719f0 con 0x7f030015adc0 2024-04-24T23:27:12.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.887+0000 7f02fd7fa640 1 -- 172.21.15.26:0/1197077744 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) ==== 104+0+0 (secure 0 0 0) 0x7f02f005c840 con 0x7f030015adc0 2024-04-24T23:27:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02e006be30 msgr2=0x7f02e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02e006be30 0x7f02e006e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f02f8034000 tx=0x7f02f802f000 comp rx=0 tx=0).stop 2024-04-24T23:27:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030015adc0 msgr2=0x7f0300169e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030015adc0 0x7f0300169e00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f02f000c460 tx=0x7f02f000c920 comp rx=0 tx=0).stop 2024-04-24T23:27:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 shutdown_connections 2024-04-24T23:27:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02e006be30 0x7f02e006e2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030015adc0 0x7f0300169e00 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03000d0750 0x7f03001698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:12.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 --2- 172.21.15.26:0/1197077744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f03000ad7b0 0x7f03001584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 >> 172.21.15.26:0/1197077744 conn(0x7f030006b540 msgr2=0x7f03000cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 shutdown_connections 2024-04-24T23:27:12.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:12.891+0000 7f0306f42640 1 -- 172.21.15.26:0/1197077744 wait complete. 2024-04-24T23:27:12.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.059+0000 7fce96396640 1 Processor -- start 2024-04-24T23:27:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.059+0000 7fce96396640 1 -- start start 2024-04-24T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.060+0000 7fce96396640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9006de50 0x7fce900b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.060+0000 7fce96396640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce900cde40 con 0x7fce900b4f00 2024-04-24T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.060+0000 7fce96396640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce900b5380 con 0x7fce9006e230 2024-04-24T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.060+0000 7fce96396640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fce9015ace0 con 0x7fce9006de50 2024-04-24T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.060+0000 7fce8f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fce900b4f00 0x7fce90154670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60110/0 (socket says 172.21.15.26:60110) 2024-04-24T23:27:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.060+0000 7fce8f7fe640 1 -- 172.21.15.26:0/921920960 learned_addr learned my addr 172.21.15.26:0/921920960 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.060+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2225401665 0 0) 0x7fce80001110 con 0x7fce900b4f00 2024-04-24T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.060+0000 7fce8ffff640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9006de50 0x7fce900b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.060+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce7c005730 con 0x7fce900b4f00 2024-04-24T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 926546589 0 0) 0x7fce7c005730 con 0x7fce900b4f00 2024-04-24T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9006de50 msgr2=0x7fce900b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9006de50 0x7fce900b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 >> v1:172.21.15.29:6789/0 conn(0x7fce9006e230 legacy=0x7fce9015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9015afc0 con 0x7fce900b4f00 2024-04-24T23:27:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fce80001840 con 0x7fce900b4f00 2024-04-24T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 >> v1:172.21.15.26:6789/0 conn(0x7fce900b4f00 legacy=0x7fce90154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9006de50 0x7fce7c005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce7c006170 0x7fce7c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce7c006f50 0x7fce7c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce9015ace0 con 0x7fce7c006f50 2024-04-24T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c00dcb0 con 0x7fce7c006170 2024-04-24T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fce7c00deb0 con 0x7fce9006de50 2024-04-24T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce94933640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce7c006f50 0x7fce7c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8ffff640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9006de50 0x7fce7c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce8f7fe640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce7c006170 0x7fce7c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce94933640 1 -- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9006de50 msgr2=0x7fce7c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce94933640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9006de50 0x7fce7c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce94933640 1 -- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce7c006170 msgr2=0x7fce7c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce94933640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce7c006170 0x7fce7c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.061+0000 7fce94933640 1 -- 172.21.15.26:0/921920960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c005730 con 0x7fce7c006f50 2024-04-24T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.062+0000 7fce94933640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce7c006f50 0x7fce7c0073d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fce88001e50 tx=0x7fce7c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a9a1a40ec15902 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.062+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88001a70 con 0x7fce7c006f50 2024-04-24T23:27:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.062+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce88003970 con 0x7fce7c006f50 2024-04-24T23:27:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.062+0000 7fce8effd640 1 -- 172.21.15.26:0/921920960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88003d60 con 0x7fce7c006f50 2024-04-24T23:27:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.063+0000 7fce96396640 1 -- 172.21.15.26:0/921920960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce7c006f50 msgr2=0x7fce7c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.063+0000 7fce96396640 1 --2- 172.21.15.26:0/921920960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce7c006f50 0x7fce7c0073d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fce88001e50 tx=0x7fce7c007a40 comp rx=0 tx=0).stop 2024-04-24T23:27:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.063+0000 7fce94933640 1 -- 172.21.15.26:0/921920960 reap_dead start 2024-04-24T23:27:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.063+0000 7fce96396640 1 -- 172.21.15.26:0/921920960 shutdown_connections 2024-04-24T23:27:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.063+0000 7fce96396640 1 -- 172.21.15.26:0/921920960 >> 172.21.15.26:0/921920960 conn(0x7fce9006b300 msgr2=0x7fce9006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.063+0000 7fce96396640 1 -- 172.21.15.26:0/921920960 shutdown_connections 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.063+0000 7fce96396640 1 -- 172.21.15.26:0/921920960 wait complete. 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.064+0000 7fce96396640 1 Processor -- start 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.064+0000 7fce96396640 1 -- start start 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce96396640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce9014b200 0x7fce9014dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce96396640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce9014b5e0 0x7fce900afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce96396640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9014ba50 0x7fce900b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce96396640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce9015baa0 con 0x7fce9014b5e0 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce96396640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fce9015b580 con 0x7fce9014b200 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce96396640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fce90072bc0 con 0x7fce9014ba50 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce9014b5e0 0x7fce900afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce9014b5e0 0x7fce900afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48850/0 (socket says 172.21.15.26:48850) 2024-04-24T23:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce8f7fe640 1 -- 172.21.15.26:0/1101301865 learned_addr learned my addr 172.21.15.26:0/1101301865 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce8f7fe640 1 -- 172.21.15.26:0/1101301865 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9014ba50 msgr2=0x7fce900b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce8f7fe640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9014ba50 0x7fce900b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce8f7fe640 1 -- 172.21.15.26:0/1101301865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce9014b200 msgr2=0x7fce9014dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce8f7fe640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce9014b200 0x7fce9014dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce8f7fe640 1 -- 172.21.15.26:0/1101301865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9014e400 con 0x7fce9014b5e0 2024-04-24T23:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.065+0000 7fce8f7fe640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce9014b5e0 0x7fce900afb60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fce80001d50 tx=0x7fce80007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.066+0000 7fce8cff9640 1 -- 172.21.15.26:0/1101301865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8000f8b0 con 0x7fce9014b5e0 2024-04-24T23:27:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.066+0000 7fce8cff9640 1 -- 172.21.15.26:0/1101301865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce80018070 con 0x7fce9014b5e0 2024-04-24T23:27:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.066+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9014e6e0 con 0x7fce9014b5e0 2024-04-24T23:27:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.066+0000 7fce8cff9640 1 -- 172.21.15.26:0/1101301865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8000fcb0 con 0x7fce9014b5e0 2024-04-24T23:27:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.066+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9014eca0 con 0x7fce9014b5e0 2024-04-24T23:27:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.067+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce900694c0 con 0x7fce9014b5e0 2024-04-24T23:27:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.067+0000 7fce8cff9640 1 -- 172.21.15.26:0/1101301865 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce8001c410 con 0x7fce9014b5e0 2024-04-24T23:27:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.067+0000 7fce8cff9640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce7006be60 0x7fce7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.068+0000 7fce8cff9640 1 -- 172.21.15.26:0/1101301865 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6025+0+0 (secure 0 0 0) 0x7fce8008ad20 con 0x7fce9014b5e0 2024-04-24T23:27:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.068+0000 7fce8ffff640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce7006be60 0x7fce7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.068+0000 7fce8ffff640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce7006be60 0x7fce7006e320 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fce900b07f0 tx=0x7fce780095d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:13.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.072+0000 7fce8cff9640 1 -- 172.21.15.26:0/1101301865 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce900cde40 con 0x7fce9014b5e0 2024-04-24T23:27:13.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.257+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fce901e1ea0 con 0x7fce9014b5e0 2024-04-24T23:27:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.891+0000 7fce8cff9640 1 -- 172.21.15.26:0/1101301865 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) ==== 181+0+0 (secure 0 0 0) 0x7fce800594f0 con 0x7fce9014b5e0 2024-04-24T23:27:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:27:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce7006be60 msgr2=0x7fce7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce7006be60 0x7fce7006e320 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fce900b07f0 tx=0x7fce780095d0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce9014b5e0 msgr2=0x7fce900afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce9014b5e0 0x7fce900afb60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fce80001d50 tx=0x7fce80007720 comp rx=0 tx=0).stop 2024-04-24T23:27:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 shutdown_connections 2024-04-24T23:27:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce7006be60 0x7fce7006e320 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce9014ba50 0x7fce900b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce9014b5e0 0x7fce900afb60 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 --2- 172.21.15.26:0/1101301865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce9014b200 0x7fce9014dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 >> 172.21.15.26:0/1101301865 conn(0x7fce9006b300 msgr2=0x7fce901e23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 shutdown_connections 2024-04-24T23:27:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:13.895+0000 7fce96396640 1 -- 172.21.15.26:0/1101301865 wait complete. 2024-04-24T23:27:13.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:27:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.020+0000 7f6d2645d640 1 Processor -- start 2024-04-24T23:27:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.021+0000 7f6d2645d640 1 -- start start 2024-04-24T23:27:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.021+0000 7f6d2645d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d200d4630 0x7f6d200d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.021+0000 7f6d2645d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d200d27b0 con 0x7f6d2015e330 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.021+0000 7f6d2645d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d200cef00 con 0x7f6d2015e710 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.021+0000 7f6d2645d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6d200cf1e0 con 0x7f6d200d4630 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.021+0000 7f6d249fa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d200d4630 0x7f6d200d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.022+0000 7f6d249fa640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d200d4630 0x7f6d200d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52260/0 (socket says 172.21.15.26:52260) 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.022+0000 7f6d1ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6d2015e330 0x7f6d200d3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37668/0 (socket says 172.21.15.26:37668) 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.022+0000 7f6d249fa640 1 -- 172.21.15.26:0/455565200 learned_addr learned my addr 172.21.15.26:0/455565200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.022+0000 7f6d249fa640 1 -- 172.21.15.26:0/455565200 >> v1:172.21.15.29:6789/0 conn(0x7f6d2015e710 legacy=0x7f6d200b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.022+0000 7f6d249fa640 1 -- 172.21.15.26:0/455565200 >> v1:172.21.15.26:6789/0 conn(0x7f6d2015e330 legacy=0x7f6d200d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.022+0000 7f6d1effd640 1 -- 172.21.15.26:0/455565200 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3415906422 0 0) 0x7f6d200d27b0 con 0x7f6d2015e330 2024-04-24T23:27:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.022+0000 7f6d249fa640 1 -- 172.21.15.26:0/455565200 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d200cf4c0 con 0x7f6d200d4630 2024-04-24T23:27:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d249fa640 1 --2- 172.21.15.26:0/455565200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 0x7f6d200d20b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009e50 tx=0x7f6d1802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f46144f46a8ebfd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d1effd640 1 -- 172.21.15.26:0/455565200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18036070 con 0x7f6d200d4630 2024-04-24T23:27:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d1effd640 1 -- 172.21.15.26:0/455565200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d18031600 con 0x7f6d200d4630 2024-04-24T23:27:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d1effd640 1 -- 172.21.15.26:0/455565200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d180319f0 con 0x7f6d200d4630 2024-04-24T23:27:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d2645d640 1 -- 172.21.15.26:0/455565200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 msgr2=0x7f6d200d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d2645d640 1 --2- 172.21.15.26:0/455565200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 0x7f6d200d20b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009e50 tx=0x7f6d1802a7c0 comp rx=0 tx=0).stop 2024-04-24T23:27:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d2645d640 1 -- 172.21.15.26:0/455565200 shutdown_connections 2024-04-24T23:27:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d2645d640 1 --2- 172.21.15.26:0/455565200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 0x7f6d200d20b0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d2645d640 1 -- 172.21.15.26:0/455565200 >> 172.21.15.26:0/455565200 conn(0x7f6d2008dbc0 msgr2=0x7f6d2009d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d2645d640 1 -- 172.21.15.26:0/455565200 shutdown_connections 2024-04-24T23:27:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.023+0000 7f6d2645d640 1 -- 172.21.15.26:0/455565200 wait complete. 2024-04-24T23:27:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.025+0000 7f6d2645d640 1 Processor -- start 2024-04-24T23:27:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.025+0000 7f6d2645d640 1 -- start start 2024-04-24T23:27:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.025+0000 7f6d2645d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 0x7f6d2015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.025+0000 7f6d2645d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d2015e330 0x7f6d200cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.025+0000 7f6d2645d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d2015e710 0x7f6d20160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:14.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.025+0000 7f6d2645d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d200d1760 con 0x7f6d200d4630 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.025+0000 7f6d2645d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6d200d15e0 con 0x7f6d2015e710 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.025+0000 7f6d2645d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6d200d18e0 con 0x7f6d2015e330 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 0x7f6d2015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 0x7f6d2015d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52268/0 (socket says 172.21.15.26:52268) 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1ffff640 1 -- 172.21.15.26:0/298818642 learned_addr learned my addr 172.21.15.26:0/298818642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1ffff640 1 -- 172.21.15.26:0/298818642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d2015e330 msgr2=0x7f6d200cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1ffff640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d2015e330 0x7f6d200cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1ffff640 1 -- 172.21.15.26:0/298818642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d2015e710 msgr2=0x7f6d20160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1ffff640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d2015e710 0x7f6d20160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1ffff640 1 -- 172.21.15.26:0/298818642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2015db90 con 0x7f6d200d4630 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1ffff640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 0x7f6d2015d650 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000c4d0 tx=0x7f6d1000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1cff9640 1 -- 172.21.15.26:0/298818642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10018070 con 0x7f6d200d4630 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1cff9640 1 -- 172.21.15.26:0/298818642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d100074e0 con 0x7f6d200d4630 2024-04-24T23:27:14.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d200c9a40 con 0x7f6d200d4630 2024-04-24T23:27:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d1cff9640 1 -- 172.21.15.26:0/298818642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10004a20 con 0x7f6d200d4630 2024-04-24T23:27:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.026+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d200b71a0 con 0x7f6d200d4630 2024-04-24T23:27:14.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.027+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d200694c0 con 0x7f6d200d4630 2024-04-24T23:27:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.028+0000 7f6d1cff9640 1 -- 172.21.15.26:0/298818642 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d1001b070 con 0x7f6d200d4630 2024-04-24T23:27:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.028+0000 7f6d1cff9640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.028+0000 7f6d1cff9640 1 -- 172.21.15.26:0/298818642 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5587+0+0 (secure 0 0 0) 0x7f6d1008a0a0 con 0x7f6d200d4630 2024-04-24T23:27:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.028+0000 7f6d1f7fe640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.029+0000 7f6d1f7fe640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6d14000f60 tx=0x7f6d140079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.033+0000 7f6d1cff9640 1 -- 172.21.15.26:0/298818642 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d10058b50 con 0x7f6d200d4630 2024-04-24T23:27:14.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.218+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6d200d5060 con 0x7f6d200d4630 2024-04-24T23:27:14.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.893+0000 7f6d1cff9640 1 -- 172.21.15.26:0/298818642 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) ==== 169+0+0 (secure 0 0 0) 0x7f6d10058450 con 0x7f6d200d4630 2024-04-24T23:27:14.894 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-24T23:27:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d0c06be30 msgr2=0x7f6d0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6d14000f60 tx=0x7f6d140079b0 comp rx=0 tx=0).stop 2024-04-24T23:27:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 msgr2=0x7f6d2015d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 0x7f6d2015d650 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000c4d0 tx=0x7f6d1000c990 comp rx=0 tx=0).stop 2024-04-24T23:27:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 shutdown_connections 2024-04-24T23:27:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d2015e710 0x7f6d20160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6d2015e330 0x7f6d200cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 --2- 172.21.15.26:0/298818642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d200d4630 0x7f6d2015d650 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 >> 172.21.15.26:0/298818642 conn(0x7f6d2008dbc0 msgr2=0x7f6d200b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 shutdown_connections 2024-04-24T23:27:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:14.896+0000 7f6d2645d640 1 -- 172.21.15.26:0/298818642 wait complete. 2024-04-24T23:27:14.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:27:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.019+0000 7f02f38f5640 1 Processor -- start 2024-04-24T23:27:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.019+0000 7f02f38f5640 1 -- start start 2024-04-24T23:27:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.020+0000 7f02f38f5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f02ec0cdf30 0x7f02ec0ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.020+0000 7f02f38f5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02ec0b2580 con 0x7f02ec0b3150 2024-04-24T23:27:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.020+0000 7f02f38f5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02ec06c840 con 0x7f02ec0d1550 2024-04-24T23:27:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.020+0000 7f02f38f5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f02ec0d00d0 con 0x7f02ec0cdf30 2024-04-24T23:27:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.020+0000 7f02f1691640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f02ec0cdf30 0x7f02ec0ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.020+0000 7f02f0e90640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f02ec0b3150 0x7f02ec0b1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37674/0 (socket says 172.21.15.26:37674) 2024-04-24T23:27:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.020+0000 7f02f0e90640 1 -- 172.21.15.26:0/2211269529 learned_addr learned my addr 172.21.15.26:0/2211269529 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.020+0000 7f02f1691640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f02ec0cdf30 0x7f02ec0ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52274/0 (socket says 172.21.15.26:52274) 2024-04-24T23:27:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.021+0000 7f02f1691640 1 -- 172.21.15.26:0/2211269529 >> v1:172.21.15.29:6789/0 conn(0x7f02ec0d1550 legacy=0x7f02ec0b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.021+0000 7f02f1691640 1 -- 172.21.15.26:0/2211269529 >> v1:172.21.15.26:6789/0 conn(0x7f02ec0b3150 legacy=0x7f02ec0b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.021+0000 7f02f1691640 1 -- 172.21.15.26:0/2211269529 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec06cb20 con 0x7f02ec0cdf30 2024-04-24T23:27:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.021+0000 7f02ebfff640 1 -- 172.21.15.26:0/2211269529 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464899480 0 0) 0x7f02ec0b2580 con 0x7f02ec0b3150 2024-04-24T23:27:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.021+0000 7f02f1691640 1 --2- 172.21.15.26:0/2211269529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0cdf30 0x7f02ec0ce680 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f02e40097a0 tx=0x7f02e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f4aa65f3ee3f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02ebfff640 1 -- 172.21.15.26:0/2211269529 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4037070 con 0x7f02ec0cdf30 2024-04-24T23:27:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02ebfff640 1 -- 172.21.15.26:0/2211269529 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e4032440 con 0x7f02ec0cdf30 2024-04-24T23:27:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02ebfff640 1 -- 172.21.15.26:0/2211269529 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4032810 con 0x7f02ec0cdf30 2024-04-24T23:27:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02f38f5640 1 -- 172.21.15.26:0/2211269529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0cdf30 msgr2=0x7f02ec0ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02f38f5640 1 --2- 172.21.15.26:0/2211269529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0cdf30 0x7f02ec0ce680 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f02e40097a0 tx=0x7f02e402a230 comp rx=0 tx=0).stop 2024-04-24T23:27:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02f38f5640 1 -- 172.21.15.26:0/2211269529 shutdown_connections 2024-04-24T23:27:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02f38f5640 1 --2- 172.21.15.26:0/2211269529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0cdf30 0x7f02ec0ce680 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02f38f5640 1 -- 172.21.15.26:0/2211269529 >> 172.21.15.26:0/2211269529 conn(0x7f02ec0d1e00 msgr2=0x7f02ec0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02f38f5640 1 -- 172.21.15.26:0/2211269529 shutdown_connections 2024-04-24T23:27:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.022+0000 7f02f38f5640 1 -- 172.21.15.26:0/2211269529 wait complete. 2024-04-24T23:27:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.024+0000 7f02f38f5640 1 Processor -- start 2024-04-24T23:27:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.024+0000 7f02f38f5640 1 -- start start 2024-04-24T23:27:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.024+0000 7f02f38f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0b3150 0x7f02ec0b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.024+0000 7f02f38f5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ec0cdf30 0x7f02ec1f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.024+0000 7f02f38f5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ec0d1550 0x7f02ec0b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.024+0000 7f02f38f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec06d0e0 con 0x7f02ec0b3150 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.024+0000 7f02f38f5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec06c5f0 con 0x7f02ec0cdf30 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.024+0000 7f02f38f5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f02ec06d3c0 con 0x7f02ec0d1550 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.024+0000 7f02f1691640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0b3150 0x7f02ec0b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f1691640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0b3150 0x7f02ec0b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52280/0 (socket says 172.21.15.26:52280) 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f1691640 1 -- 172.21.15.26:0/878092600 learned_addr learned my addr 172.21.15.26:0/878092600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f0e90640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ec0cdf30 0x7f02ec1f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f1e92640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ec0d1550 0x7f02ec0b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f1691640 1 -- 172.21.15.26:0/878092600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ec0d1550 msgr2=0x7f02ec0b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f1691640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ec0d1550 0x7f02ec0b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f1691640 1 -- 172.21.15.26:0/878092600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ec0cdf30 msgr2=0x7f02ec1f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f1691640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ec0cdf30 0x7f02ec1f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f1691640 1 -- 172.21.15.26:0/878092600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec0b4cf0 con 0x7f02ec0b3150 2024-04-24T23:27:15.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f1691640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0b3150 0x7f02ec0b5a40 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f02e4000c00 tx=0x7f02e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02e9ffb640 1 -- 172.21.15.26:0/878092600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4041070 con 0x7f02ec0b3150 2024-04-24T23:27:15.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02e9ffb640 1 -- 172.21.15.26:0/878092600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e403c930 con 0x7f02ec0b3150 2024-04-24T23:27:15.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ec0b77a0 con 0x7f02ec0b3150 2024-04-24T23:27:15.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02e9ffb640 1 -- 172.21.15.26:0/878092600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4037040 con 0x7f02ec0b3150 2024-04-24T23:27:15.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.025+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ec0b5520 con 0x7f02ec0b3150 2024-04-24T23:27:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.026+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ec075610 con 0x7f02ec0b3150 2024-04-24T23:27:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.027+0000 7f02e9ffb640 1 -- 172.21.15.26:0/878092600 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02e4044070 con 0x7f02ec0b3150 2024-04-24T23:27:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.028+0000 7f02e9ffb640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02d806be30 0x7f02d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.028+0000 7f02f0e90640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02d806be30 0x7f02d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.028+0000 7f02e9ffb640 1 -- 172.21.15.26:0/878092600 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7f02e40ab930 con 0x7f02ec0b3150 2024-04-24T23:27:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.028+0000 7f02f0e90640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02d806be30 0x7f02d806e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000f60 tx=0x7f02e0007bb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.032+0000 7f02e9ffb640 1 -- 172.21.15.26:0/878092600 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02ec0b77a0 con 0x7f02ec0b3150 2024-04-24T23:27:15.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.215+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f02ec0b2980 con 0x7f02ec0b3150 2024-04-24T23:27:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.218+0000 7f02e9ffb640 1 -- 172.21.15.26:0/878092600 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f02e407a5b0 con 0x7f02ec0b3150 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02d806be30 msgr2=0x7f02d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02d806be30 0x7f02d806e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000f60 tx=0x7f02e0007bb0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0b3150 msgr2=0x7f02ec0b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0b3150 0x7f02ec0b5a40 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f02e4000c00 tx=0x7f02e40046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 shutdown_connections 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f02d806be30 0x7f02d806e2f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f02ec0d1550 0x7f02ec0b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f02ec0cdf30 0x7f02ec1f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 --2- 172.21.15.26:0/878092600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02ec0b3150 0x7f02ec0b5a40 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 >> 172.21.15.26:0/878092600 conn(0x7f02ec0d1e00 msgr2=0x7f02ec0b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 shutdown_connections 2024-04-24T23:27:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.220+0000 7f02f38f5640 1 -- 172.21.15.26:0/878092600 wait complete. 2024-04-24T23:27:15.232 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-04-24T23:27:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.332+0000 7ff13da6b640 1 Processor -- start 2024-04-24T23:27:15.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.332+0000 7ff13da6b640 1 -- start start 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.332+0000 7ff13da6b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 0x7ff1380d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.332+0000 7ff13da6b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff1380b3130 0x7ff1380d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.332+0000 7ff13da6b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1380d32f0 con 0x7ff1380d22e0 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.332+0000 7ff13da6b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff1380d1ab0 con 0x7ff1380b3130 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.332+0000 7ff13da6b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff1380d2ae0 con 0x7ff13815e570 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff137fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1380d22e0 0x7ff1380d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37682/0 (socket says 172.21.15.26:37682) 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff137fff640 1 -- 172.21.15.26:0/2227933580 learned_addr learned my addr 172.21.15.26:0/2227933580 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff136ffd640 1 --2- 172.21.15.26:0/2227933580 >> v2:172.21.15.29:3300/0 conn(0x7ff1380b3130 0x7ff1380d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1377fe640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 0x7ff1380d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898066313 0 0) 0x7ff130001490 con 0x7ff1380d22e0 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff120003520 con 0x7ff1380d22e0 2024-04-24T23:27:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 397025235 0 0) 0x7ff120003520 con 0x7ff1380d22e0 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 msgr2=0x7ff1380d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 0x7ff1380d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 >> v2:172.21.15.29:3300/0 conn(0x7ff1380b3130 msgr2=0x7ff1380d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 --2- 172.21.15.26:0/2227933580 >> v2:172.21.15.29:3300/0 conn(0x7ff1380b3130 0x7ff1380d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1380d0270 con 0x7ff1380d22e0 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7ff130001bc0 con 0x7ff1380d22e0 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff136ffd640 1 --2- 172.21.15.26:0/2227933580 >> v2:172.21.15.29:3300/0 conn(0x7ff1380b3130 0x7ff1380d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1377fe640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 0x7ff1380d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 >> v1:172.21.15.26:6789/0 conn(0x7ff1380d22e0 legacy=0x7ff1380d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1200039b0 0x7ff120003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff120004800 0x7ff120004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 0x7ff1200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1380d1ab0 con 0x7ff120004800 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1380d2ae0 con 0x7ff1200039b0 2024-04-24T23:27:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff12000b9c0 con 0x7ff13815e570 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.333+0000 7ff136ffd640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff120004800 0x7ff120004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.334+0000 7ff1377fe640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1200039b0 0x7ff120003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.334+0000 7ff137fff640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 0x7ff1200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.334+0000 7ff136ffd640 1 -- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 msgr2=0x7ff1200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.334+0000 7ff136ffd640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 0x7ff1200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.334+0000 7ff136ffd640 1 -- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1200039b0 msgr2=0x7ff120003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.334+0000 7ff136ffd640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1200039b0 0x7ff120003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.334+0000 7ff136ffd640 1 -- 172.21.15.26:0/2227933580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff120003520 con 0x7ff120004800 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.334+0000 7ff137fff640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815e570 0x7ff1200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.334+0000 7ff136ffd640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff120004800 0x7ff120004c80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009c90 tx=0x7ff120005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7086305d318b4d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c036070 con 0x7ff120004800 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c031400 con 0x7ff120004800 2024-04-24T23:27:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff1367fc640 1 -- 172.21.15.26:0/2227933580 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c009860 con 0x7ff120004800 2024-04-24T23:27:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff13da6b640 1 -- 172.21.15.26:0/2227933580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff120004800 msgr2=0x7ff120004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff13da6b640 1 --2- 172.21.15.26:0/2227933580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff120004800 0x7ff120004c80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009c90 tx=0x7ff120005830 comp rx=0 tx=0).stop 2024-04-24T23:27:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff137fff640 1 -- 172.21.15.26:0/2227933580 reap_dead start 2024-04-24T23:27:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff13da6b640 1 -- 172.21.15.26:0/2227933580 shutdown_connections 2024-04-24T23:27:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff13da6b640 1 -- 172.21.15.26:0/2227933580 >> 172.21.15.26:0/2227933580 conn(0x7ff13808dbe0 msgr2=0x7ff13809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff13da6b640 1 -- 172.21.15.26:0/2227933580 shutdown_connections 2024-04-24T23:27:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.335+0000 7ff13da6b640 1 -- 172.21.15.26:0/2227933580 wait complete. 2024-04-24T23:27:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.336+0000 7ff13da6b640 1 Processor -- start 2024-04-24T23:27:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff13da6b640 1 -- start start 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff13da6b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815cc40 0x7ff1380b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff13da6b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff13815d020 0x7ff13814f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff13da6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13814f750 0x7ff1380a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff13da6b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1380cfa70 con 0x7ff13814f750 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff13da6b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff1380d3010 con 0x7ff13815d020 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff13da6b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff1380cfd50 con 0x7ff13815cc40 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff137fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13814f750 0x7ff1380a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff137fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13814f750 0x7ff1380a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52302/0 (socket says 172.21.15.26:52302) 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff137fff640 1 -- 172.21.15.26:0/2648653281 learned_addr learned my addr 172.21.15.26:0/2648653281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff136ffd640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff13815d020 0x7ff13814f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.337+0000 7ff1377fe640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815cc40 0x7ff1380b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff137fff640 1 -- 172.21.15.26:0/2648653281 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815cc40 msgr2=0x7ff1380b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff137fff640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815cc40 0x7ff1380b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff137fff640 1 -- 172.21.15.26:0/2648653281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff13815d020 msgr2=0x7ff13814f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff137fff640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff13815d020 0x7ff13814f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff137fff640 1 -- 172.21.15.26:0/2648653281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1380a1bd0 con 0x7ff13814f750 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff137fff640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13814f750 0x7ff1380a0e60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff1380ba250 tx=0x7ff13000a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff11ffff640 1 -- 172.21.15.26:0/2648653281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13000abd0 con 0x7ff13814f750 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff11ffff640 1 -- 172.21.15.26:0/2648653281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff13000ad90 con 0x7ff13814f750 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff11ffff640 1 -- 172.21.15.26:0/2648653281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130013420 con 0x7ff13814f750 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1380a1eb0 con 0x7ff13814f750 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.338+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13809e250 con 0x7ff13814f750 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.339+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1380694c0 con 0x7ff13814f750 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.339+0000 7ff11ffff640 1 -- 172.21.15.26:0/2648653281 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff13001a020 con 0x7ff13814f750 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.340+0000 7ff11ffff640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff12406be60 0x7ff12406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.340+0000 7ff11ffff640 1 -- 172.21.15.26:0/2648653281 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7ff13008a900 con 0x7ff13814f750 2024-04-24T23:27:15.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.340+0000 7ff1377fe640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff12406be60 0x7ff12406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.340+0000 7ff1377fe640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff12406be60 0x7ff12406e320 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff128009ad0 tx=0x7ff128009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.344+0000 7ff11ffff640 1 -- 172.21.15.26:0/2648653281 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1380d32f0 con 0x7ff13814f750 2024-04-24T23:27:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.602+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) -- 0x7ff1380b8580 con 0x7ff13814f750 2024-04-24T23:27:15.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.899+0000 7ff11ffff640 1 -- 172.21.15.26:0/2648653281 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v335) ==== 154+0+0 (secure 0 0 0) 0x7ff130058e20 con 0x7ff13814f750 2024-04-24T23:27:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.901+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff12406be60 msgr2=0x7ff12406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.901+0000 7ff13da6b640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff12406be60 0x7ff12406e320 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff128009ad0 tx=0x7ff128009190 comp rx=0 tx=0).stop 2024-04-24T23:27:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13814f750 msgr2=0x7ff1380a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13814f750 0x7ff1380a0e60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff1380ba250 tx=0x7ff13000a180 comp rx=0 tx=0).stop 2024-04-24T23:27:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 shutdown_connections 2024-04-24T23:27:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff12406be60 0x7ff12406e320 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff13814f750 0x7ff1380a0e60 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff13815d020 0x7ff13814f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 --2- 172.21.15.26:0/2648653281 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff13815cc40 0x7ff1380b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 >> 172.21.15.26:0/2648653281 conn(0x7ff13808dbe0 msgr2=0x7ff13806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 shutdown_connections 2024-04-24T23:27:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:15.902+0000 7ff13da6b640 1 -- 172.21.15.26:0/2648653281 wait complete. 2024-04-24T23:27:15.914 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-04-24T23:27:15.914 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-04-24T23:27:15.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:16.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.015+0000 7fa100409640 1 Processor -- start 2024-04-24T23:27:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.015+0000 7fa100409640 1 -- start start 2024-04-24T23:27:16.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.016+0000 7fa100409640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa0f80d4870 0x7fa0f80b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.016+0000 7fa100409640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f80b3130 0x7fa0f80b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.016+0000 7fa100409640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f80d32f0 con 0x7fa0f815e570 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.016+0000 7fa100409640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f80d1ab0 con 0x7fa0f80d4870 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.016+0000 7fa100409640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f80d2ae0 con 0x7fa0f80b3130 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.016+0000 7fa0fe1a5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa0f815e570 0x7fa0f80d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37686/0 (socket says 172.21.15.26:37686) 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.016+0000 7fa0fe1a5640 1 -- 172.21.15.26:0/2882267478 learned_addr learned my addr 172.21.15.26:0/2882267478 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd9a4640 1 --2- 172.21.15.26:0/2882267478 >> v2:172.21.15.29:3300/0 conn(0x7fa0f80d4870 0x7fa0f80b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fe9a6640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f80b3130 0x7fa0f80b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710971237 0 0) 0x7fa0f0001110 con 0x7fa0f815e570 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0e0003520 con 0x7fa0f815e570 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 631091477 0 0) 0x7fa0e0003520 con 0x7fa0f815e570 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f80b3130 msgr2=0x7fa0f80b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f80b3130 0x7fa0f80b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 >> v2:172.21.15.29:3300/0 conn(0x7fa0f80d4870 msgr2=0x7fa0f80b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 --2- 172.21.15.26:0/2882267478 >> v2:172.21.15.29:3300/0 conn(0x7fa0f80d4870 0x7fa0f80b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f80d0270 con 0x7fa0f815e570 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fa0f0001860 con 0x7fa0f815e570 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 >> v1:172.21.15.26:6789/0 conn(0x7fa0f815e570 legacy=0x7fa0f80d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e00039b0 0x7fa0e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f80b3130 0x7fa0e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0e0004d40 0x7fa0e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80d1ab0 con 0x7fa0e00039b0 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80d2ae0 con 0x7fa0e0004d40 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e000b9c0 con 0x7fa0f80b3130 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fe1a5640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e00039b0 0x7fa0e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fd9a4640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f80b3130 0x7fa0e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.017+0000 7fa0fe9a6640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0e0004d40 0x7fa0e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.018+0000 7fa0fe1a5640 1 -- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f80b3130 msgr2=0x7fa0e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.018+0000 7fa0fe1a5640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f80b3130 0x7fa0e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.018+0000 7fa0fe1a5640 1 -- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0e0004d40 msgr2=0x7fa0e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.018+0000 7fa0fe1a5640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0e0004d40 0x7fa0e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.018+0000 7fa0fe1a5640 1 -- 172.21.15.26:0/2882267478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e0003520 con 0x7fa0e00039b0 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.018+0000 7fa0fe1a5640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e00039b0 0x7fa0e0003dd0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c490 tx=0x7fa0e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8731be9d891ec18 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.018+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000c0e0 con 0x7fa0e00039b0 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.019+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0034070 con 0x7fa0e00039b0 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.019+0000 7fa0fd1a3640 1 -- 172.21.15.26:0/2882267478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0005d30 con 0x7fa0e00039b0 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.019+0000 7fa100409640 1 -- 172.21.15.26:0/2882267478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e00039b0 msgr2=0x7fa0e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.019+0000 7fa100409640 1 --2- 172.21.15.26:0/2882267478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e00039b0 0x7fa0e0003dd0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c490 tx=0x7fa0e0005830 comp rx=0 tx=0).stop 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.019+0000 7fa0fe9a6640 1 -- 172.21.15.26:0/2882267478 reap_dead start 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.019+0000 7fa100409640 1 -- 172.21.15.26:0/2882267478 shutdown_connections 2024-04-24T23:27:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.019+0000 7fa100409640 1 -- 172.21.15.26:0/2882267478 >> 172.21.15.26:0/2882267478 conn(0x7fa0f808dbe0 msgr2=0x7fa0f809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.019+0000 7fa100409640 1 -- 172.21.15.26:0/2882267478 shutdown_connections 2024-04-24T23:27:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.019+0000 7fa100409640 1 -- 172.21.15.26:0/2882267478 wait complete. 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.020+0000 7fa100409640 1 Processor -- start 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.020+0000 7fa100409640 1 -- start start 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa100409640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0e00039b0 0x7fa0f81505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa100409640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f81e7f60 0x7fa0f81e8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa100409640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f81e8cc0 0x7fa0f81f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa100409640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80cfa70 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa100409640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80d3010 con 0x7fa0e00039b0 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa100409640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f80cfd50 con 0x7fa0f81e7f60 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa0fe9a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f81e8cc0 0x7fa0f81f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa0fe9a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f81e8cc0 0x7fa0f81f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52326/0 (socket says 172.21.15.26:52326) 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa0fe9a6640 1 -- 172.21.15.26:0/2217010603 learned_addr learned my addr 172.21.15.26:0/2217010603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa0fd9a4640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f81e7f60 0x7fa0f81e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.021+0000 7fa0fe1a5640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0e00039b0 0x7fa0f81505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa0fe9a6640 1 -- 172.21.15.26:0/2217010603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f81e7f60 msgr2=0x7fa0f81e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa0fe9a6640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f81e7f60 0x7fa0f81e8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa0fe9a6640 1 -- 172.21.15.26:0/2217010603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0e00039b0 msgr2=0x7fa0f81505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa0fe9a6640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0e00039b0 0x7fa0f81505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa0fe9a6640 1 -- 172.21.15.26:0/2217010603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f81e5300 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa0fe9a6640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f81e8cc0 0x7fa0f81f67f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fa0f808e4a0 tx=0x7fa0e800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa0f6ffd640 1 -- 172.21.15.26:0/2217010603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800fe10 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa0f6ffd640 1 -- 172.21.15.26:0/2217010603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e801d070 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f80cd040 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa0f6ffd640 1 -- 172.21.15.26:0/2217010603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8015670 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.022+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f80cd320 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.023+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f80694c0 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.023+0000 7fa0f6ffd640 1 -- 172.21.15.26:0/2217010603 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa0e8015810 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.024+0000 7fa0f6ffd640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0e406be30 0x7fa0e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.024+0000 7fa0f6ffd640 1 -- 172.21.15.26:0/2217010603 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7fa0e808b5f0 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.024+0000 7fa0fe1a5640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0e406be30 0x7fa0e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.024+0000 7fa0fe1a5640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0e406be30 0x7fa0e406e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0033000 tx=0x7fa0f002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.028+0000 7fa0f6ffd640 1 -- 172.21.15.26:0/2217010603 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa0e8010040 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.212+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0f80b12e0 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.213+0000 7fa0f6ffd640 1 -- 172.21.15.26:0/2217010603 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+13219 (secure 0 0 0) 0x7fa0e805a160 con 0x7fa0f81e8cc0 2024-04-24T23:27:16.214 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:16.214 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":88,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:27:14.892368+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/3727846607":"2024-04-25T23:27:11.884815+0000","172.21.15.29:6833/3727846607":"2024-04-25T23:27:11.884815+0000","172.21.15.29:6834/2253149800":"2024-04-25T23:27:11.884833+0000","172.21.15.29:6832/2253149800":"2024-04-25T23:27:11.884833+0000","172.21.15.26:6836/306957409":"2024-04-25T23:27:11.884828+0000","172.21.15.26:6834/306957409":"2024-04-25T23:27:11.884828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0e406be30 msgr2=0x7fa0e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0e406be30 0x7fa0e406e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0033000 tx=0x7fa0f002e000 comp rx=0 tx=0).stop 2024-04-24T23:27:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f81e8cc0 msgr2=0x7fa0f81f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f81e8cc0 0x7fa0f81f67f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fa0f808e4a0 tx=0x7fa0e800f580 comp rx=0 tx=0).stop 2024-04-24T23:27:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 shutdown_connections 2024-04-24T23:27:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa0e406be30 0x7fa0e406e2f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f81e8cc0 0x7fa0f81f67f0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa0f81e7f60 0x7fa0f81e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 --2- 172.21.15.26:0/2217010603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0e00039b0 0x7fa0f81505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 >> 172.21.15.26:0/2217010603 conn(0x7fa0f808dbe0 msgr2=0x7fa0f809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 shutdown_connections 2024-04-24T23:27:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.215+0000 7fa100409640 1 -- 172.21.15.26:0/2217010603 wait complete. 2024-04-24T23:27:16.228 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:27:16.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:27:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.332+0000 7f1e4aae7640 1 Processor -- start 2024-04-24T23:27:16.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.332+0000 7f1e4aae7640 1 -- start start 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.332+0000 7f1e4aae7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e4415e750 0x7f1e440cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.332+0000 7f1e4aae7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e440d2c30 0x7f1e440cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e4aae7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e440d2120 con 0x7f1e4415e370 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e4aae7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1e440d1510 con 0x7f1e440d2c30 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e4aae7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1e440d1ed0 con 0x7f1e4415e750 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e49084640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e440d2c30 0x7f1e440cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e48883640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1e4415e370 0x7f1e440d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37698/0 (socket says 172.21.15.26:37698) 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e48883640 1 -- 172.21.15.26:0/4021973696 learned_addr learned my addr 172.21.15.26:0/4021973696 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e43fff640 1 --2- 172.21.15.26:0/4021973696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e4415e750 0x7f1e440cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e49084640 1 -- 172.21.15.26:0/4021973696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e4415e750 msgr2=0x7f1e440cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e49084640 1 --2- 172.21.15.26:0/4021973696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e4415e750 0x7f1e440cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e49084640 1 -- 172.21.15.26:0/4021973696 >> v1:172.21.15.26:6789/0 conn(0x7f1e4415e370 legacy=0x7f1e440d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e437fe640 1 -- 172.21.15.26:0/4021973696 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852488074 0 0) 0x7f1e440d2120 con 0x7f1e4415e370 2024-04-24T23:27:16.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.333+0000 7f1e49084640 1 -- 172.21.15.26:0/4021973696 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e440b3740 con 0x7f1e440d2c30 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e49084640 1 --2- 172.21.15.26:0/4021973696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 0x7f1e440cf1d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f1e440d2050 tx=0x7f1e3c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7f0ef8756536eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e437fe640 1 -- 172.21.15.26:0/4021973696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c02ae60 con 0x7f1e440d2c30 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e437fe640 1 -- 172.21.15.26:0/4021973696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3c007740 con 0x7f1e440d2c30 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e437fe640 1 -- 172.21.15.26:0/4021973696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c007b10 con 0x7f1e440d2c30 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e4aae7640 1 -- 172.21.15.26:0/4021973696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 msgr2=0x7f1e440cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e4aae7640 1 --2- 172.21.15.26:0/4021973696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 0x7f1e440cf1d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f1e440d2050 tx=0x7f1e3c02a640 comp rx=0 tx=0).stop 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e4aae7640 1 -- 172.21.15.26:0/4021973696 shutdown_connections 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e4aae7640 1 --2- 172.21.15.26:0/4021973696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 0x7f1e440cf1d0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e4aae7640 1 --2- 172.21.15.26:0/4021973696 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e4415e750 0x7f1e440cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e4aae7640 1 -- 172.21.15.26:0/4021973696 >> 172.21.15.26:0/4021973696 conn(0x7f1e440d40f0 msgr2=0x7f1e440d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e4aae7640 1 -- 172.21.15.26:0/4021973696 shutdown_connections 2024-04-24T23:27:16.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.334+0000 7f1e4aae7640 1 -- 172.21.15.26:0/4021973696 wait complete. 2024-04-24T23:27:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.336+0000 7f1e4aae7640 1 Processor -- start 2024-04-24T23:27:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.336+0000 7f1e4aae7640 1 -- start start 2024-04-24T23:27:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.336+0000 7f1e4aae7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 0x7f1e440c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e4aae7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e4415e370 0x7f1e440ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e4aae7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e440b8f60 0x7f1e4415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e4aae7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e440b5230 con 0x7f1e440d2c30 2024-04-24T23:27:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e4aae7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e440b3d00 con 0x7f1e4415e370 2024-04-24T23:27:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e4aae7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1e440b53b0 con 0x7f1e440b8f60 2024-04-24T23:27:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e48883640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 0x7f1e440c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e48883640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 0x7f1e440c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52338/0 (socket says 172.21.15.26:52338) 2024-04-24T23:27:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e48883640 1 -- 172.21.15.26:0/770441460 learned_addr learned my addr 172.21.15.26:0/770441460 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e49084640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e440b8f60 0x7f1e4415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e43fff640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e4415e370 0x7f1e440ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e48883640 1 -- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e440b8f60 msgr2=0x7f1e4415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e48883640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e440b8f60 0x7f1e4415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e48883640 1 -- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e4415e370 msgr2=0x7f1e440ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e48883640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e4415e370 0x7f1e440ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e48883640 1 -- 172.21.15.26:0/770441460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e440c8e60 con 0x7f1e440d2c30 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e49084640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e440b8f60 0x7f1e4415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e43fff640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e4415e370 0x7f1e440ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e48883640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 0x7f1e440c8630 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c4d0 tx=0x7f1e3800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e417fa640 1 -- 172.21.15.26:0/770441460 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38018070 con 0x7f1e440d2c30 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e417fa640 1 -- 172.21.15.26:0/770441460 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e38007500 con 0x7f1e440d2c30 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e440c9140 con 0x7f1e440d2c30 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e417fa640 1 -- 172.21.15.26:0/770441460 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38004a20 con 0x7f1e440d2c30 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.337+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e440cbcb0 con 0x7f1e440d2c30 2024-04-24T23:27:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.338+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e440c9310 con 0x7f1e440d2c30 2024-04-24T23:27:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.339+0000 7f1e417fa640 1 -- 172.21.15.26:0/770441460 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e3801b070 con 0x7f1e440d2c30 2024-04-24T23:27:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.339+0000 7f1e417fa640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e2806be60 0x7f1e2806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.339+0000 7f1e417fa640 1 -- 172.21.15.26:0/770441460 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7f1e38089f60 con 0x7f1e440d2c30 2024-04-24T23:27:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.339+0000 7f1e43fff640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e2806be60 0x7f1e2806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.340+0000 7f1e43fff640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e2806be60 0x7f1e2806e320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1e440d0340 tx=0x7f1e30008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.344+0000 7f1e417fa640 1 -- 172.21.15.26:0/770441460 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e38058b60 con 0x7f1e440d2c30 2024-04-24T23:27:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.526+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e44075610 con 0x7f1e440d2c30 2024-04-24T23:27:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.527+0000 7f1e417fa640 1 -- 172.21.15.26:0/770441460 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f1e38058460 con 0x7f1e440d2c30 2024-04-24T23:27:16.528 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e2806be60 msgr2=0x7f1e2806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e2806be60 0x7f1e2806e320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1e440d0340 tx=0x7f1e30008040 comp rx=0 tx=0).stop 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 msgr2=0x7f1e440c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 0x7f1e440c8630 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c4d0 tx=0x7f1e3800c990 comp rx=0 tx=0).stop 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 shutdown_connections 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1e2806be60 0x7f1e2806e320 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1e440b8f60 0x7f1e4415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e4415e370 0x7f1e440ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 --2- 172.21.15.26:0/770441460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e440d2c30 0x7f1e440c8630 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 >> 172.21.15.26:0/770441460 conn(0x7f1e440d40f0 msgr2=0x7f1e4415d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 shutdown_connections 2024-04-24T23:27:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.529+0000 7f1e4aae7640 1 -- 172.21.15.26:0/770441460 wait complete. 2024-04-24T23:27:16.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-04-24T23:27:16.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.641+0000 7f4bba577640 1 Processor -- start 2024-04-24T23:27:16.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.641+0000 7f4bba577640 1 -- start start 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.642+0000 7f4bba577640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb415e220 0x7f4bb40cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.642+0000 7f4bba577640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4bb40d2db0 0x7f4bb40cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.642+0000 7f4bba577640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb40cf340 con 0x7f4bb40d2240 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.642+0000 7f4bba577640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb40cf620 con 0x7f4bb40d2db0 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.642+0000 7f4bba577640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb40cf900 con 0x7f4bb415e220 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.642+0000 7f4bb8b14640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4bb40d2db0 0x7f4bb40cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.642+0000 7f4bb8b14640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4bb40d2db0 0x7f4bb40cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52352/0 (socket says 172.21.15.26:52352) 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.642+0000 7f4bb8b14640 1 -- 172.21.15.26:0/685196054 learned_addr learned my addr 172.21.15.26:0/685196054 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.643+0000 7f4bb3fff640 1 --2- 172.21.15.26:0/685196054 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb415e220 0x7f4bb40cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.643+0000 7f4bb8b14640 1 -- 172.21.15.26:0/685196054 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb415e220 msgr2=0x7f4bb40cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:16.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.643+0000 7f4bb8b14640 1 --2- 172.21.15.26:0/685196054 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb415e220 0x7f4bb40cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.643+0000 7f4bb8b14640 1 -- 172.21.15.26:0/685196054 >> v1:172.21.15.29:6789/0 conn(0x7f4bb40d2240 legacy=0x7f4bb40d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.643+0000 7f4bb8b14640 1 -- 172.21.15.26:0/685196054 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb40cfbe0 con 0x7f4bb40d2db0 2024-04-24T23:27:16.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.643+0000 7f4bb8b14640 1 --2- 172.21.15.26:0/685196054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 0x7f4bb40cb370 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009e50 tx=0x7f4bac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99193274261c9bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bb2ffd640 1 -- 172.21.15.26:0/685196054 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac036070 con 0x7f4bb40d2db0 2024-04-24T23:27:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bb2ffd640 1 -- 172.21.15.26:0/685196054 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac031600 con 0x7f4bb40d2db0 2024-04-24T23:27:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bb2ffd640 1 -- 172.21.15.26:0/685196054 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0319d0 con 0x7f4bb40d2db0 2024-04-24T23:27:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bba577640 1 -- 172.21.15.26:0/685196054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 msgr2=0x7f4bb40cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bba577640 1 --2- 172.21.15.26:0/685196054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 0x7f4bb40cb370 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009e50 tx=0x7f4bac02a680 comp rx=0 tx=0).stop 2024-04-24T23:27:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bba577640 1 -- 172.21.15.26:0/685196054 shutdown_connections 2024-04-24T23:27:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bba577640 1 --2- 172.21.15.26:0/685196054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 0x7f4bb40cb370 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bba577640 1 --2- 172.21.15.26:0/685196054 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb415e220 0x7f4bb40cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bba577640 1 -- 172.21.15.26:0/685196054 >> 172.21.15.26:0/685196054 conn(0x7f4bb409d500 msgr2=0x7f4bb40d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bba577640 1 -- 172.21.15.26:0/685196054 shutdown_connections 2024-04-24T23:27:16.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.644+0000 7f4bba577640 1 -- 172.21.15.26:0/685196054 wait complete. 2024-04-24T23:27:16.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.645+0000 7f4bba577640 1 Processor -- start 2024-04-24T23:27:16.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bba577640 1 -- start start 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bba577640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb40d2240 0x7f4bb4152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bba577640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 0x7f4bb40b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bba577640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb415e220 0x7f4bb40b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bba577640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb415e600 con 0x7f4bb40d2db0 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bba577640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb40d1ee0 con 0x7f4bb415e220 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bba577640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb415e780 con 0x7f4bb40d2240 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 0x7f4bb40b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 0x7f4bb40b6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52368/0 (socket says 172.21.15.26:52368) 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bb37fe640 1 -- 172.21.15.26:0/1226129152 learned_addr learned my addr 172.21.15.26:0/1226129152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.646+0000 7f4bb3fff640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb40d2240 0x7f4bb4152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb8b14640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb415e220 0x7f4bb40b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb37fe640 1 -- 172.21.15.26:0/1226129152 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb40d2240 msgr2=0x7f4bb4152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb37fe640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb40d2240 0x7f4bb4152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb37fe640 1 -- 172.21.15.26:0/1226129152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb415e220 msgr2=0x7f4bb40b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb37fe640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb415e220 0x7f4bb40b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb37fe640 1 -- 172.21.15.26:0/1226129152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb415bce0 con 0x7f4bb40d2db0 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb37fe640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 0x7f4bb40b6de0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800c3c0 tx=0x7f4ba800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb0ff9640 1 -- 172.21.15.26:0/1226129152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80052d0 con 0x7f4bb40d2db0 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb0ff9640 1 -- 172.21.15.26:0/1226129152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba8005490 con 0x7f4bb40d2db0 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb40b7930 con 0x7f4bb40d2db0 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bb0ff9640 1 -- 172.21.15.26:0/1226129152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8007260 con 0x7f4bb40d2db0 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.647+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb40b7ef0 con 0x7f4bb40d2db0 2024-04-24T23:27:16.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.648+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb40694c0 con 0x7f4bb40d2db0 2024-04-24T23:27:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.649+0000 7f4bb0ff9640 1 -- 172.21.15.26:0/1226129152 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ba8007400 con 0x7f4bb40d2db0 2024-04-24T23:27:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.649+0000 7f4bb0ff9640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ba006be30 0x7f4ba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.649+0000 7f4bb0ff9640 1 -- 172.21.15.26:0/1226129152 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7f4ba808a040 con 0x7f4bb40d2db0 2024-04-24T23:27:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.649+0000 7f4bb3fff640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ba006be30 0x7f4ba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.650+0000 7f4bb3fff640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ba006be30 0x7f4ba006e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009b40 tx=0x7f4ba4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.654+0000 7f4bb0ff9640 1 -- 172.21.15.26:0/1226129152 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ba80585f0 con 0x7f4bb40d2db0 2024-04-24T23:27:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:16.906+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) -- 0x7f4bb40cb320 con 0x7f4bb40d2db0 2024-04-24T23:27:17.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.905+0000 7f4bb0ff9640 1 -- 172.21.15.26:0/1226129152 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v337) ==== 168+0+0 (secure 0 0 0) 0x7f4ba805c8f0 con 0x7f4bb40d2db0 2024-04-24T23:27:17.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ba006be30 msgr2=0x7f4ba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:17.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ba006be30 0x7f4ba006e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009b40 tx=0x7f4ba4009190 comp rx=0 tx=0).stop 2024-04-24T23:27:17.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 msgr2=0x7f4bb40b6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:17.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 0x7f4bb40b6de0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800c3c0 tx=0x7f4ba800c880 comp rx=0 tx=0).stop 2024-04-24T23:27:17.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 shutdown_connections 2024-04-24T23:27:17.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4ba006be30 0x7f4ba006e2f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4bb415e220 0x7f4bb40b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb40d2db0 0x7f4bb40b6de0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 --2- 172.21.15.26:0/1226129152 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4bb40d2240 0x7f4bb4152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 >> 172.21.15.26:0/1226129152 conn(0x7f4bb409d500 msgr2=0x7f4bb40cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:17.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 shutdown_connections 2024-04-24T23:27:17.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:17.908+0000 7f4bba577640 1 -- 172.21.15.26:0/1226129152 wait complete. 2024-04-24T23:27:17.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:27:18.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.032+0000 7fab2e2d2640 1 Processor -- start 2024-04-24T23:27:18.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.032+0000 7fab2e2d2640 1 -- start start 2024-04-24T23:27:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab2e2d2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fab2815be40 0x7fab280acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab2e2d2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab280b4ce0 0x7fab280d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab2e2d2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab280aeb10 con 0x7fab280d0580 2024-04-24T23:27:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab2e2d2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fab2809df20 con 0x7fab280b4ce0 2024-04-24T23:27:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab2e2d2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fab280aedf0 con 0x7fab2815be40 2024-04-24T23:27:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab277fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab280b4ce0 0x7fab280d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab2c86f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fab280d0580 0x7fab280b2c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37706/0 (socket says 172.21.15.26:37706) 2024-04-24T23:27:18.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab2c86f640 1 -- 172.21.15.26:0/3419187955 learned_addr learned my addr 172.21.15.26:0/3419187955 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:18.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab27fff640 1 --2- 172.21.15.26:0/3419187955 >> v2:172.21.15.29:3300/0 conn(0x7fab2815be40 0x7fab280acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab277fe640 1 -- 172.21.15.26:0/3419187955 >> v2:172.21.15.29:3300/0 conn(0x7fab2815be40 msgr2=0x7fab280acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab277fe640 1 --2- 172.21.15.26:0/3419187955 >> v2:172.21.15.29:3300/0 conn(0x7fab2815be40 0x7fab280acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab277fe640 1 -- 172.21.15.26:0/3419187955 >> v1:172.21.15.26:6789/0 conn(0x7fab280d0580 legacy=0x7fab280b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab26ffd640 1 -- 172.21.15.26:0/3419187955 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 874257834 0 0) 0x7fab280aeb10 con 0x7fab280d0580 2024-04-24T23:27:18.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.033+0000 7fab277fe640 1 -- 172.21.15.26:0/3419187955 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab280af0d0 con 0x7fab280b4ce0 2024-04-24T23:27:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.034+0000 7fab277fe640 1 --2- 172.21.15.26:0/3419187955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280b4ce0 0x7fab280d0040 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fab2806e400 tx=0x7fab1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e1fc574d762c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.034+0000 7fab26ffd640 1 -- 172.21.15.26:0/3419187955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18037070 con 0x7fab280b4ce0 2024-04-24T23:27:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.034+0000 7fab26ffd640 1 -- 172.21.15.26:0/3419187955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab18032440 con 0x7fab280b4ce0 2024-04-24T23:27:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.034+0000 7fab26ffd640 1 -- 172.21.15.26:0/3419187955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18032810 con 0x7fab280b4ce0 2024-04-24T23:27:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.035+0000 7fab2e2d2640 1 -- 172.21.15.26:0/3419187955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280b4ce0 msgr2=0x7fab280d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.035+0000 7fab2e2d2640 1 --2- 172.21.15.26:0/3419187955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280b4ce0 0x7fab280d0040 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fab2806e400 tx=0x7fab1802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.035+0000 7fab2e2d2640 1 -- 172.21.15.26:0/3419187955 shutdown_connections 2024-04-24T23:27:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.035+0000 7fab2e2d2640 1 --2- 172.21.15.26:0/3419187955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280b4ce0 0x7fab280d0040 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.035+0000 7fab2e2d2640 1 --2- 172.21.15.26:0/3419187955 >> v2:172.21.15.29:3300/0 conn(0x7fab2815be40 0x7fab280acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.035+0000 7fab2e2d2640 1 -- 172.21.15.26:0/3419187955 >> 172.21.15.26:0/3419187955 conn(0x7fab2806b540 msgr2=0x7fab2806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.035+0000 7fab2e2d2640 1 -- 172.21.15.26:0/3419187955 shutdown_connections 2024-04-24T23:27:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.035+0000 7fab2e2d2640 1 -- 172.21.15.26:0/3419187955 wait complete. 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.036+0000 7fab2e2d2640 1 Processor -- start 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.036+0000 7fab2e2d2640 1 -- start start 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab2e2d2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab280b4ce0 0x7fab28143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab2e2d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280d0580 0x7fab281536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab2e2d2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab2815be40 0x7fab2815e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab2e2d2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab2806cd10 con 0x7fab280d0580 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab2e2d2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fab2806cb90 con 0x7fab280b4ce0 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab2e2d2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fab2815d230 con 0x7fab2815be40 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280d0580 0x7fab281536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280d0580 0x7fab281536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52386/0 (socket says 172.21.15.26:52386) 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab277fe640 1 -- 172.21.15.26:0/630214772 learned_addr learned my addr 172.21.15.26:0/630214772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab27fff640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab280b4ce0 0x7fab28143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab2c86f640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab2815be40 0x7fab2815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab277fe640 1 -- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab2815be40 msgr2=0x7fab2815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab277fe640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab2815be40 0x7fab2815e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab277fe640 1 -- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab280b4ce0 msgr2=0x7fab28143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab277fe640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab280b4ce0 0x7fab28143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab277fe640 1 -- 172.21.15.26:0/630214772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab281f5470 con 0x7fab280d0580 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab27fff640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab280b4ce0 0x7fab28143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.037+0000 7fab2c86f640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab2815be40 0x7fab2815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.038+0000 7fab277fe640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280d0580 0x7fab281536f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fab18000c00 tx=0x7fab180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.038+0000 7fab24ff9640 1 -- 172.21.15.26:0/630214772 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18037050 con 0x7fab280d0580 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.038+0000 7fab24ff9640 1 -- 172.21.15.26:0/630214772 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab18038040 con 0x7fab280d0580 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.038+0000 7fab24ff9640 1 -- 172.21.15.26:0/630214772 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18004450 con 0x7fab280d0580 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.038+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab281f5710 con 0x7fab280d0580 2024-04-24T23:27:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.038+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab281f5cd0 con 0x7fab280d0580 2024-04-24T23:27:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.039+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab28075610 con 0x7fab280d0580 2024-04-24T23:27:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.039+0000 7fab24ff9640 1 -- 172.21.15.26:0/630214772 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab1803c6d0 con 0x7fab280d0580 2024-04-24T23:27:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.040+0000 7fab24ff9640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab1406be30 0x7fab1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.040+0000 7fab24ff9640 1 -- 172.21.15.26:0/630214772 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7fab180ab6b0 con 0x7fab280d0580 2024-04-24T23:27:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.040+0000 7fab27fff640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab1406be30 0x7fab1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.040+0000 7fab27fff640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab1406be30 0x7fab1406e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009a20 tx=0x7fab1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.044+0000 7fab24ff9640 1 -- 172.21.15.26:0/630214772 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fab1807a230 con 0x7fab280d0580 2024-04-24T23:27:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.226+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fab280b0b00 con 0x7fab280d0580 2024-04-24T23:27:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.229+0000 7fab24ff9640 1 -- 172.21.15.26:0/630214772 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fab18079b30 con 0x7fab280d0580 2024-04-24T23:27:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab1406be30 msgr2=0x7fab1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab1406be30 0x7fab1406e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009a20 tx=0x7fab1c008040 comp rx=0 tx=0).stop 2024-04-24T23:27:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280d0580 msgr2=0x7fab281536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280d0580 0x7fab281536f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fab18000c00 tx=0x7fab180046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 shutdown_connections 2024-04-24T23:27:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fab1406be30 0x7fab1406e2f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fab2815be40 0x7fab2815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab280d0580 0x7fab281536f0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 --2- 172.21.15.26:0/630214772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fab280b4ce0 0x7fab28143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 >> 172.21.15.26:0/630214772 conn(0x7fab2806b540 msgr2=0x7fab2806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 shutdown_connections 2024-04-24T23:27:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.231+0000 7fab2e2d2640 1 -- 172.21.15.26:0/630214772 wait complete. 2024-04-24T23:27:18.243 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:27:18.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:27:18.277 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:27:18.318 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:27:18.318 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:27:18.318 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:18.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:27:18.334 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:27:18.334 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:18.376 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:27:18.377 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:27:18.377 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:18.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:27:18.392 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:27:18.392 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:18.435 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:27:18.436 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:27:18.436 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.537+0000 7fceb502a640 1 Processor -- start 2024-04-24T23:27:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.537+0000 7fceb502a640 1 -- start start 2024-04-24T23:27:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.537+0000 7fceb502a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fceb0143c40 0x7fceb00cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.537+0000 7fceb502a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fceb00cc860 0x7fceb0072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.537+0000 7fceb502a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb00cd0b0 con 0x7fceb015eb00 2024-04-24T23:27:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.537+0000 7fceb502a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fceb0071b70 con 0x7fceb0143c40 2024-04-24T23:27:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.537+0000 7fceb502a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fceb0071240 con 0x7fceb00cc860 2024-04-24T23:27:18.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceaf59e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fceb015eb00 0x7fceb00b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37716/0 (socket says 172.21.15.26:37716) 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceaf59e640 1 -- 172.21.15.26:0/837012545 learned_addr learned my addr 172.21.15.26:0/837012545 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceae59c640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fceb00cc860 0x7fceb0072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceaed9d640 1 --2- 172.21.15.26:0/837012545 >> v2:172.21.15.29:3300/0 conn(0x7fceb0143c40 0x7fceb00cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2092539657 0 0) 0x7fceb00cd0b0 con 0x7fceb015eb00 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce98003520 con 0x7fceb015eb00 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3977480443 0 0) 0x7fce98003520 con 0x7fceb015eb00 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fceb00cc860 msgr2=0x7fceb0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fceb00cc860 0x7fceb0072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 >> v2:172.21.15.29:3300/0 conn(0x7fceb0143c40 msgr2=0x7fceb00cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 --2- 172.21.15.26:0/837012545 >> v2:172.21.15.29:3300/0 conn(0x7fceb0143c40 0x7fceb00cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb0071520 con 0x7fceb015eb00 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fcea00019f0 con 0x7fceb015eb00 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 >> v1:172.21.15.26:6789/0 conn(0x7fceb015eb00 legacy=0x7fceb00b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce980039b0 0x7fce98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce98004800 0x7fce98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fceb00cc860 0x7fce980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0071b70 con 0x7fce980039b0 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0071240 con 0x7fce98004800 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fce9800b9c0 con 0x7fceb00cc860 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceaed9d640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce980039b0 0x7fce98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceae59c640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce98004800 0x7fce98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.538+0000 7fceaf59e640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fceb00cc860 0x7fce980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceaed9d640 1 -- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fceb00cc860 msgr2=0x7fce980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceaed9d640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fceb00cc860 0x7fce980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceaed9d640 1 -- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce98004800 msgr2=0x7fce98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceaed9d640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce98004800 0x7fce98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceaed9d640 1 -- 172.21.15.26:0/837012545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce98003520 con 0x7fce980039b0 2024-04-24T23:27:18.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceae59c640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fce98004800 0x7fce98004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceaed9d640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce980039b0 0x7fce98003dd0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009ac0 tx=0x7fce98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf8f854130e24f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8036070 con 0x7fce980039b0 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea8031400 con 0x7fce980039b0 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.539+0000 7fceadd9b640 1 -- 172.21.15.26:0/837012545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8009690 con 0x7fce980039b0 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.540+0000 7fceb502a640 1 -- 172.21.15.26:0/837012545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce980039b0 msgr2=0x7fce98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.540+0000 7fceb502a640 1 --2- 172.21.15.26:0/837012545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce980039b0 0x7fce98003dd0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009ac0 tx=0x7fce98005830 comp rx=0 tx=0).stop 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.540+0000 7fceaf59e640 1 -- 172.21.15.26:0/837012545 reap_dead start 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.540+0000 7fceb502a640 1 -- 172.21.15.26:0/837012545 shutdown_connections 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.540+0000 7fceb502a640 1 -- 172.21.15.26:0/837012545 >> 172.21.15.26:0/837012545 conn(0x7fceb006b300 msgr2=0x7fceb006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.540+0000 7fceb502a640 1 -- 172.21.15.26:0/837012545 shutdown_connections 2024-04-24T23:27:18.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.540+0000 7fceb502a640 1 -- 172.21.15.26:0/837012545 wait complete. 2024-04-24T23:27:18.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.541+0000 7fceb502a640 1 Processor -- start 2024-04-24T23:27:18.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceb502a640 1 -- start start 2024-04-24T23:27:18.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceb502a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce980039b0 0x7fceb00cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceb502a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb01f3d60 0x7fceb015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceb502a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fceb01f41e0 0x7fceb013e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceb502a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00ccca0 con 0x7fceb01f3d60 2024-04-24T23:27:18.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceb502a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0144020 con 0x7fceb01f41e0 2024-04-24T23:27:18.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceb502a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fceb00cce20 con 0x7fce980039b0 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceae59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb01f3d60 0x7fceb015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceae59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb01f3d60 0x7fceb015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52400/0 (socket says 172.21.15.26:52400) 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.542+0000 7fceae59c640 1 -- 172.21.15.26:0/1982429265 learned_addr learned my addr 172.21.15.26:0/1982429265 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceaed9d640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce980039b0 0x7fceb00cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceaf59e640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fceb01f41e0 0x7fceb013e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceae59c640 1 -- 172.21.15.26:0/1982429265 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce980039b0 msgr2=0x7fceb00cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceae59c640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce980039b0 0x7fceb00cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceae59c640 1 -- 172.21.15.26:0/1982429265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fceb01f41e0 msgr2=0x7fceb013e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceae59c640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fceb01f41e0 0x7fceb013e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceae59c640 1 -- 172.21.15.26:0/1982429265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb00ca7e0 con 0x7fceb01f3d60 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceae59c640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb01f3d60 0x7fceb015c8a0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fcea4011700 tx=0x7fcea4011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fce977fe640 1 -- 172.21.15.26:0/1982429265 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea400f960 con 0x7fceb01f3d60 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fce977fe640 1 -- 172.21.15.26:0/1982429265 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea400fb00 con 0x7fceb01f3d60 2024-04-24T23:27:18.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fce977fe640 1 -- 172.21.15.26:0/1982429265 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4015460 con 0x7fceb01f3d60 2024-04-24T23:27:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb015d340 con 0x7fceb01f3d60 2024-04-24T23:27:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.543+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb00cad60 con 0x7fceb01f3d60 2024-04-24T23:27:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.544+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb00694c0 con 0x7fceb01f3d60 2024-04-24T23:27:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.544+0000 7fce977fe640 1 -- 172.21.15.26:0/1982429265 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcea4015600 con 0x7fceb01f3d60 2024-04-24T23:27:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.545+0000 7fce977fe640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce9c06be30 0x7fce9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.545+0000 7fce977fe640 1 -- 172.21.15.26:0/1982429265 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7fcea408b510 con 0x7fceb01f3d60 2024-04-24T23:27:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.545+0000 7fceaed9d640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce9c06be30 0x7fce9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.545+0000 7fceaed9d640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce9c06be30 0x7fce9c06e2f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcea8033000 tx=0x7fcea8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.549+0000 7fce977fe640 1 -- 172.21.15.26:0/1982429265 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcea4019030 con 0x7fceb01f3d60 2024-04-24T23:27:18.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.771+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fceb00b4d20 con 0x7fceb01f3d60 2024-04-24T23:27:18.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.772+0000 7fce977fe640 1 -- 172.21.15.26:0/1982429265 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+3863 (secure 0 0 0) 0x7fcea405a1a0 con 0x7fceb01f3d60 2024-04-24T23:27:18.773 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:18.773 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1815691075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1815691075},{"type":"v1","addr":"172.21.15.26:6837","nonce":1815691075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/140313803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":140313803},{"type":"v1","addr":"172.21.15.29:6835","nonce":140313803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/381516380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":381516380},{"type":"v1","addr":"172.21.15.29:6833","nonce":381516380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2496625589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2496625589},{"type":"v1","addr":"172.21.15.26:6836","nonce":2496625589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[]} 2024-04-24T23:27:18.773 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 96 2024-04-24T23:27:18.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce9c06be30 msgr2=0x7fce9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce9c06be30 0x7fce9c06e2f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcea8033000 tx=0x7fcea8008030 comp rx=0 tx=0).stop 2024-04-24T23:27:18.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb01f3d60 msgr2=0x7fceb015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb01f3d60 0x7fceb015c8a0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fcea4011700 tx=0x7fcea4011bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 shutdown_connections 2024-04-24T23:27:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fce9c06be30 0x7fce9c06e2f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fceb01f41e0 0x7fceb013e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb01f3d60 0x7fceb015c8a0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 --2- 172.21.15.26:0/1982429265 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fce980039b0 0x7fceb00cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 >> 172.21.15.26:0/1982429265 conn(0x7fceb006b300 msgr2=0x7fceb015d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 shutdown_connections 2024-04-24T23:27:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.774+0000 7fceb502a640 1 -- 172.21.15.26:0/1982429265 wait complete. 2024-04-24T23:27:18.786 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:27:18.786 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:27:18.786 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:27:18.787 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:27:18.787 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:27:18.787 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:27:18.787 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:27:18.787 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:27:18.787 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:27:18.787 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:27:18.788 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:27:18.788 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:27:18.788 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:27:18.788 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:27:18.788 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:27:18.788 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:27:18.796 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:27:18.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:27:18.797 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:27:18.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:27:18.800 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:27:18.800 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:27:18.801 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:27:18.801 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:27:18.839 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:27:18.839 DEBUG:teuthology.parallel:result is None 2024-04-24T23:27:18.840 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:27:18.840 DEBUG:teuthology.parallel:result is None 2024-04-24T23:27:18.842 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:27:18.843 DEBUG:teuthology.parallel:result is None 2024-04-24T23:27:18.843 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:27:18.843 DEBUG:teuthology.parallel:result is None 2024-04-24T23:27:18.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:27:18.915 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:27:18.916 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-24T23:27:18.926 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:27:18.926 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:27:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.951+0000 7f8fc2cba640 1 Processor -- start 2024-04-24T23:27:18.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.951+0000 7f8fc2cba640 1 -- start start 2024-04-24T23:27:18.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.952+0000 7f8fc2cba640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8fbc15e570 0x7f8fbc0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.952+0000 7f8fc2cba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc0d22e0 0x7f8fbc0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.952+0000 7f8fc2cba640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fbc0d32f0 con 0x7f8fbc0b3130 2024-04-24T23:27:18.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.952+0000 7f8fc2cba640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8fbc0d1ab0 con 0x7f8fbc15e570 2024-04-24T23:27:18.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.952+0000 7f8fc2cba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8fbc0d2ae0 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.952+0000 7f8fc0a56640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8fbc15e570 0x7f8fbc0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.952+0000 7f8fc0a56640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8fbc15e570 0x7f8fbc0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52450/0 (socket says 172.21.15.26:52450) 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.952+0000 7f8fc0a56640 1 -- 172.21.15.26:0/2557412191 learned_addr learned my addr 172.21.15.26:0/2557412191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.952+0000 7f8fc1257640 1 --2- 172.21.15.26:0/2557412191 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc0d22e0 0x7f8fbc0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.953+0000 7f8fc0a56640 1 -- 172.21.15.26:0/2557412191 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc0d22e0 msgr2=0x7f8fbc0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.953+0000 7f8fc0a56640 1 --2- 172.21.15.26:0/2557412191 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc0d22e0 0x7f8fbc0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.953+0000 7f8fc0a56640 1 -- 172.21.15.26:0/2557412191 >> v1:172.21.15.29:6789/0 conn(0x7f8fbc0b3130 legacy=0x7f8fbc0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.953+0000 7f8fc0a56640 1 -- 172.21.15.26:0/2557412191 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc0d0270 con 0x7f8fbc15e570 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.953+0000 7f8fc0a56640 1 --2- 172.21.15.26:0/2557412191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc15e570 0x7f8fbc0d4160 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009870 tx=0x7f8fb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d7e683b3620092 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.953+0000 7f8fbb7fe640 1 -- 172.21.15.26:0/2557412191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0037070 con 0x7f8fbc15e570 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.953+0000 7f8fbb7fe640 1 -- 172.21.15.26:0/2557412191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb0032440 con 0x7f8fbc15e570 2024-04-24T23:27:18.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.954+0000 7f8fbb7fe640 1 -- 172.21.15.26:0/2557412191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0032810 con 0x7f8fbc15e570 2024-04-24T23:27:18.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.954+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2557412191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc15e570 msgr2=0x7f8fbc0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.954+0000 7f8fc2cba640 1 --2- 172.21.15.26:0/2557412191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc15e570 0x7f8fbc0d4160 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009870 tx=0x7f8fb002a270 comp rx=0 tx=0).stop 2024-04-24T23:27:18.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.954+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2557412191 shutdown_connections 2024-04-24T23:27:18.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.954+0000 7f8fc2cba640 1 --2- 172.21.15.26:0/2557412191 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc0d22e0 0x7f8fbc0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.954+0000 7f8fc2cba640 1 --2- 172.21.15.26:0/2557412191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc15e570 0x7f8fbc0d4160 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.954+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2557412191 >> 172.21.15.26:0/2557412191 conn(0x7f8fbc08dbe0 msgr2=0x7f8fbc09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.954+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2557412191 shutdown_connections 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.954+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2557412191 wait complete. 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.955+0000 7f8fc2cba640 1 Processor -- start 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.955+0000 7f8fc2cba640 1 -- start start 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fc2cba640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fbc0b3130 0x7f8fbc14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fc2cba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc0d22e0 0x7f8fbc159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fc2cba640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc15e570 0x7f8fbc0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fc2cba640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc0cfa70 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fc2cba640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc0d3010 con 0x7f8fbc0b3130 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fc2cba640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8fbc0cfd50 con 0x7f8fbc15e570 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc0d22e0 0x7f8fbc159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc0d22e0 0x7f8fbc159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52456/0 (socket says 172.21.15.26:52456) 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fbbfff640 1 -- 172.21.15.26:0/2265273583 learned_addr learned my addr 172.21.15.26:0/2265273583 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fc1257640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc15e570 0x7f8fbc0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.956+0000 7f8fc0a56640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fbc0b3130 0x7f8fbc14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fbbfff640 1 -- 172.21.15.26:0/2265273583 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc15e570 msgr2=0x7f8fbc0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fbbfff640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc15e570 0x7f8fbc0b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fbbfff640 1 -- 172.21.15.26:0/2265273583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fbc0b3130 msgr2=0x7f8fbc14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fbbfff640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fbc0b3130 0x7f8fbc14f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fbbfff640 1 -- 172.21.15.26:0/2265273583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc0b7000 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fbbfff640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc0d22e0 0x7f8fbc159060 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800c400 tx=0x7f8fa800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fb97fa640 1 -- 172.21.15.26:0/2265273583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80052d0 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fb97fa640 1 -- 172.21.15.26:0/2265273583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa8005490 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc0b72e0 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fb97fa640 1 -- 172.21.15.26:0/2265273583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8007260 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.957+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc0b78a0 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.958+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc0694c0 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.958+0000 7f8fb97fa640 1 -- 172.21.15.26:0/2265273583 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fa8007400 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.959+0000 7f8fb97fa640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8fa006be30 0x7f8fa006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.959+0000 7f8fc0a56640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8fa006be30 0x7f8fa006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.959+0000 7f8fb97fa640 1 -- 172.21.15.26:0/2265273583 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7f8fa808a0a0 con 0x7f8fbc0d22e0 2024-04-24T23:27:18.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.960+0000 7f8fc0a56640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8fa006be30 0x7f8fa006e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0034000 tx=0x7f8fb002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:18.963+0000 7f8fb97fa640 1 -- 172.21.15.26:0/2265273583 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8fbc0d32f0 con 0x7f8fbc0d22e0 2024-04-24T23:27:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.160+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8fbc0b7a70 con 0x7f8fbc0d22e0 2024-04-24T23:27:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.916+0000 7f8fb97fa640 1 -- 172.21.15.26:0/2265273583 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v90) ==== 97+0+0 (secure 0 0 0) 0x7f8fa8058650 con 0x7f8fbc0d22e0 2024-04-24T23:27:19.917 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-04-24T23:27:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.919+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8fa006be30 msgr2=0x7f8fa006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.919+0000 7f8fc2cba640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8fa006be30 0x7f8fa006e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0034000 tx=0x7f8fb002e000 comp rx=0 tx=0).stop 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.919+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc0d22e0 msgr2=0x7f8fbc159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.919+0000 7f8fc2cba640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc0d22e0 0x7f8fbc159060 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800c400 tx=0x7f8fa800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.920+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 shutdown_connections 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.920+0000 7f8fc2cba640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8fa006be30 0x7f8fa006e2f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.920+0000 7f8fc2cba640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8fbc15e570 0x7f8fbc0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.920+0000 7f8fc2cba640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc0d22e0 0x7f8fbc159060 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.920+0000 7f8fc2cba640 1 --2- 172.21.15.26:0/2265273583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fbc0b3130 0x7f8fbc14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.920+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 >> 172.21.15.26:0/2265273583 conn(0x7f8fbc08dbe0 msgr2=0x7f8fbc0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.920+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 shutdown_connections 2024-04-24T23:27:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:19.920+0000 7f8fc2cba640 1 -- 172.21.15.26:0/2265273583 wait complete. 2024-04-24T23:27:19.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:27:20.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.042+0000 7f1b808a5640 1 Processor -- start 2024-04-24T23:27:20.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.042+0000 7f1b808a5640 1 -- start start 2024-04-24T23:27:20.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b808a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b7c0cf870 0x7f1b7c072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b808a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c15eb00 0x7f1b7c15ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b808a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7c0cd0b0 con 0x7f1b7c143c40 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b808a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1b7c071b70 con 0x7f1b7c0cf870 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b808a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7c15f470 con 0x7f1b7c15eb00 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b79d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b7c0cf870 0x7f1b7c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b79d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b7c0cf870 0x7f1b7c072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52468/0 (socket says 172.21.15.26:52468) 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b7a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1b7c143c40 0x7f1b7c0cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37732/0 (socket says 172.21.15.26:37732) 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b79d9b640 1 -- 172.21.15.26:0/3262111597 learned_addr learned my addr 172.21.15.26:0/3262111597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b7ad9d640 1 --2- 172.21.15.26:0/3262111597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c15eb00 0x7f1b7c15ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b79d9b640 1 -- 172.21.15.26:0/3262111597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c15eb00 msgr2=0x7f1b7c15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b79d9b640 1 --2- 172.21.15.26:0/3262111597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c15eb00 0x7f1b7c15ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b79d9b640 1 -- 172.21.15.26:0/3262111597 >> v1:172.21.15.26:6789/0 conn(0x7f1b7c143c40 legacy=0x7f1b7c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b79d9b640 1 -- 172.21.15.26:0/3262111597 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c071240 con 0x7f1b7c0cf870 2024-04-24T23:27:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b7959a640 1 -- 172.21.15.26:0/3262111597 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2630468099 0 0) 0x7f1b7c0cd0b0 con 0x7f1b7c143c40 2024-04-24T23:27:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.043+0000 7f1b7ad9d640 1 --2- 172.21.15.26:0/3262111597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c15eb00 0x7f1b7c15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.044+0000 7f1b79d9b640 1 --2- 172.21.15.26:0/3262111597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 0x7f1b7c072000 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000bb10 tx=0x7f1b7002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5866016bc1fb64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.044+0000 7f1b7959a640 1 -- 172.21.15.26:0/3262111597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70036020 con 0x7f1b7c0cf870 2024-04-24T23:27:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.044+0000 7f1b7959a640 1 -- 172.21.15.26:0/3262111597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b70032820 con 0x7f1b7c0cf870 2024-04-24T23:27:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.044+0000 7f1b7959a640 1 -- 172.21.15.26:0/3262111597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70032bf0 con 0x7f1b7c0cf870 2024-04-24T23:27:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.045+0000 7f1b808a5640 1 -- 172.21.15.26:0/3262111597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 msgr2=0x7f1b7c072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.045+0000 7f1b808a5640 1 --2- 172.21.15.26:0/3262111597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 0x7f1b7c072000 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000bb10 tx=0x7f1b7002c920 comp rx=0 tx=0).stop 2024-04-24T23:27:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.045+0000 7f1b808a5640 1 -- 172.21.15.26:0/3262111597 shutdown_connections 2024-04-24T23:27:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.045+0000 7f1b808a5640 1 --2- 172.21.15.26:0/3262111597 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c15eb00 0x7f1b7c15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.045+0000 7f1b808a5640 1 --2- 172.21.15.26:0/3262111597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 0x7f1b7c072000 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.045+0000 7f1b808a5640 1 -- 172.21.15.26:0/3262111597 >> 172.21.15.26:0/3262111597 conn(0x7f1b7c06b300 msgr2=0x7f1b7c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.045+0000 7f1b808a5640 1 -- 172.21.15.26:0/3262111597 shutdown_connections 2024-04-24T23:27:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.045+0000 7f1b808a5640 1 -- 172.21.15.26:0/3262111597 wait complete. 2024-04-24T23:27:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.046+0000 7f1b808a5640 1 Processor -- start 2024-04-24T23:27:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.046+0000 7f1b808a5640 1 -- start start 2024-04-24T23:27:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b808a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 0x7f1b7c1328f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b808a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c143c40 0x7f1b7c132110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b808a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c15eb00 0x7f1b7c1428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b808a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c144020 con 0x7f1b7c0cf870 2024-04-24T23:27:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b808a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c071800 con 0x7f1b7c15eb00 2024-04-24T23:27:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b808a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7c072540 con 0x7f1b7c143c40 2024-04-24T23:27:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 0x7f1b7c1328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 0x7f1b7c1328f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52470/0 (socket says 172.21.15.26:52470) 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7a59c640 1 -- 172.21.15.26:0/1522275731 learned_addr learned my addr 172.21.15.26:0/1522275731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b79d9b640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c143c40 0x7f1b7c132110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7ad9d640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c15eb00 0x7f1b7c1428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7a59c640 1 -- 172.21.15.26:0/1522275731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c143c40 msgr2=0x7f1b7c132110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7a59c640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c143c40 0x7f1b7c132110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7a59c640 1 -- 172.21.15.26:0/1522275731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c15eb00 msgr2=0x7f1b7c1428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7a59c640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c15eb00 0x7f1b7c1428c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7a59c640 1 -- 172.21.15.26:0/1522275731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c141b60 con 0x7f1b7c0cf870 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7ad9d640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c15eb00 0x7f1b7c1428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.047+0000 7f1b7a59c640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 0x7f1b7c1328f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c500 tx=0x7f1b6c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.048+0000 7f1b6affd640 1 -- 172.21.15.26:0/1522275731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c018070 con 0x7f1b7c0cf870 2024-04-24T23:27:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.048+0000 7f1b6affd640 1 -- 172.21.15.26:0/1522275731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c007580 con 0x7f1b7c0cf870 2024-04-24T23:27:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.048+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c143420 con 0x7f1b7c0cf870 2024-04-24T23:27:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.048+0000 7f1b6affd640 1 -- 172.21.15.26:0/1522275731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c004a20 con 0x7f1b7c0cf870 2024-04-24T23:27:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.048+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c141de0 con 0x7f1b7c0cf870 2024-04-24T23:27:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.049+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c0694c0 con 0x7f1b7c0cf870 2024-04-24T23:27:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.049+0000 7f1b6affd640 1 -- 172.21.15.26:0/1522275731 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b6c01b070 con 0x7f1b7c0cf870 2024-04-24T23:27:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.050+0000 7f1b6affd640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b5c06be30 0x7f1b5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.050+0000 7f1b6affd640 1 -- 172.21.15.26:0/1522275731 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4857+0+0 (secure 0 0 0) 0x7f1b6c089d80 con 0x7f1b7c0cf870 2024-04-24T23:27:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.050+0000 7f1b79d9b640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b5c06be30 0x7f1b5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.050+0000 7f1b79d9b640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b5c06be30 0x7f1b5c06e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1b70037000 tx=0x7f1b7003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.054+0000 7f1b6affd640 1 -- 172.21.15.26:0/1522275731 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b7c141b60 con 0x7f1b7c0cf870 2024-04-24T23:27:20.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.313+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1b7c0b4d20 con 0x7f1b7c0cf870 2024-04-24T23:27:20.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.313+0000 7f1b6affd640 1 -- 172.21.15.26:0/1522275731 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f1b6c058b10 con 0x7f1b7c0cf870 2024-04-24T23:27:20.314 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:20.314 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:20.314 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-04-24T23:27:20.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.314 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmg5ZUFBAAkapM9La3f+wSyLczFkm/0w==", 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPS3cHxAASacBQN+95Ds6N5668Je+DA==", 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:27:20.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmStpLGxAAz7molt8venundBs+CcFSbg==", 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylml0TBJhAAXQkjExbw1EXc90XlcAEBBw==", 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjkylmQt74DBAAZLk9JTH3qTKRouUawqNURw==", 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCmkylmi6IJCxAAWBnNl2y0XIUW/YoAIt6VIQ==", 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCpkylmRcgOCRAA79pdcH0OlB2VROLSRE4dcw==", 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCskylm8SopBxAArvUph95+zD5/0LmNM8WPIA==", 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCwkylmbHipDxAAVUI9OpiMFiM/HFo03fEzSA==", 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCzkylmDm4CDhAA1T/X+ywXEUCL96OGyi6S9Q==", 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC2kylmwWyVDBAAJsFFLltURqVqGLNjK9OCJQ==", 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-04-24T23:27:20.321 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5kylm/nLXChAA6kk0UI2hchkN2lhPO1sXHw==", 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ==", 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.323 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-04-24T23:27:20.323 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylm/BVuMxAAAdEb6dd3Q5eMc+I3oszOyg==", 2024-04-24T23:27:20.323 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.323 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:27:20.323 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:27:20.323 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmjnJGMBAAvwAmiHrkGkDFTeL/WZbdWw==", 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.324 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmPh3qOBAAa4cbQeF96mdpgNVb/l3xPg==", 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCgkylm1ZfsMxAAExymnmhW44TcXzRblp95UA==", 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-24T23:27:20.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmq+8yERAAMKjK+5f2Qr35ral8L2qJMw==", 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:20.326 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-04-24T23:27:20.327 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQChkylmqYXWDBAAnuAn2h4Gb9qSFBoAZX73qQ==", 2024-04-24T23:27:20.327 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-24T23:27:20.327 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-24T23:27:20.327 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-24T23:27:20.327 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-24T23:27:20.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:20.327 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-24T23:27:20.327 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:20.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b5c06be30 msgr2=0x7f1b5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b5c06be30 0x7f1b5c06e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1b70037000 tx=0x7f1b7003d040 comp rx=0 tx=0).stop 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 msgr2=0x7f1b7c1328f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 0x7f1b7c1328f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c500 tx=0x7f1b6c00c9c0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 shutdown_connections 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1b5c06be30 0x7f1b5c06e2f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b7c15eb00 0x7f1b7c1428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1b7c143c40 0x7f1b7c132110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 --2- 172.21.15.26:0/1522275731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7c0cf870 0x7f1b7c1328f0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 >> 172.21.15.26:0/1522275731 conn(0x7f1b7c06b300 msgr2=0x7f1b7c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 shutdown_connections 2024-04-24T23:27:20.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.315+0000 7f1b808a5640 1 -- 172.21.15.26:0/1522275731 wait complete. 2024-04-24T23:27:20.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.431+0000 7fc93eb1b640 1 Processor -- start 2024-04-24T23:27:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.431+0000 7fc93eb1b640 1 -- start start 2024-04-24T23:27:20.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc93eb1b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc9380b63e0 0x7fc938142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc93eb1b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc93806deb0 con 0x7fc938145890 2024-04-24T23:27:20.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc93eb1b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc93806e190 con 0x7fc938142310 2024-04-24T23:27:20.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc93eb1b640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc93806dc60 con 0x7fc9380b63e0 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc93d0b8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc938145890 0x7fc93806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37734/0 (socket says 172.21.15.26:37734) 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc93d0b8640 1 -- 172.21.15.26:0/3276908606 learned_addr learned my addr 172.21.15.26:0/3276908606 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc937fff640 1 --2- 172.21.15.26:0/3276908606 >> v2:172.21.15.29:3300/0 conn(0x7fc9380b63e0 0x7fc938142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420979960 0 0) 0x7fc93806deb0 con 0x7fc938145890 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc928003520 con 0x7fc938145890 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2863499990 0 0) 0x7fc928003520 con 0x7fc938145890 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 >> v2:172.21.15.29:3300/0 conn(0x7fc9380b63e0 msgr2=0x7fc938142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc9377fe640 1 --2- 172.21.15.26:0/3276908606 >> v2:172.21.15.29:3300/0 conn(0x7fc9380b63e0 0x7fc938142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 >> v1:172.21.15.29:6789/0 conn(0x7fc938142310 legacy=0x7fc9380b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93806e470 con 0x7fc938145890 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fc9300019f0 con 0x7fc938145890 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.432+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 >> v1:172.21.15.26:6789/0 conn(0x7fc938145890 legacy=0x7fc93806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc9377fe640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc9280039b0 0x7fc928003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc9377fe640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc928004800 0x7fc928004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc9377fe640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928005210 0x7fc92800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc93806dc60 con 0x7fc928004800 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc92800c140 con 0x7fc928005210 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc92800c340 con 0x7fc9280039b0 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc937fff640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc928004800 0x7fc928004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc93c8b7640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc9280039b0 0x7fc928003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc93d0b8640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928005210 0x7fc92800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc937fff640 1 -- 172.21.15.26:0/3276908606 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc9280039b0 msgr2=0x7fc928003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc937fff640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc9280039b0 0x7fc928003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc937fff640 1 -- 172.21.15.26:0/3276908606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928005210 msgr2=0x7fc92800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc937fff640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928005210 0x7fc92800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc937fff640 1 -- 172.21.15.26:0/3276908606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc928003520 con 0x7fc928004800 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.433+0000 7fc93d0b8640 1 -- 172.21.15.26:0/3276908606 reap_dead start 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc937fff640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc928004800 0x7fc928004c80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc924009ac0 tx=0x7fc928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7802c8e29d9bbc51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924036070 con 0x7fc928004800 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc924031400 con 0x7fc928004800 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc9377fe640 1 -- 172.21.15.26:0/3276908606 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924009690 con 0x7fc928004800 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc93eb1b640 1 -- 172.21.15.26:0/3276908606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc928004800 msgr2=0x7fc928004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc93eb1b640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc928004800 0x7fc928004c80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc924009ac0 tx=0x7fc928005830 comp rx=0 tx=0).stop 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc93eb1b640 1 -- 172.21.15.26:0/3276908606 shutdown_connections 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc93eb1b640 1 --2- 172.21.15.26:0/3276908606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc928004800 0x7fc928004c80 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc93eb1b640 1 -- 172.21.15.26:0/3276908606 >> 172.21.15.26:0/3276908606 conn(0x7fc9380729a0 msgr2=0x7fc93806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc93eb1b640 1 -- 172.21.15.26:0/3276908606 shutdown_connections 2024-04-24T23:27:20.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.434+0000 7fc93eb1b640 1 -- 172.21.15.26:0/3276908606 wait complete. 2024-04-24T23:27:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.436+0000 7fc93eb1b640 1 Processor -- start 2024-04-24T23:27:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.436+0000 7fc93eb1b640 1 -- start start 2024-04-24T23:27:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.436+0000 7fc93eb1b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928004800 0x7fc938140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.436+0000 7fc93eb1b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc93813eee0 0x7fc93813f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93eb1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9381f0d10 0x7fc9381f1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93eb1b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381426f0 con 0x7fc9381f0d10 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93eb1b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc93806eac0 con 0x7fc928004800 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93eb1b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc938142870 con 0x7fc93813eee0 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93d0b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9381f0d10 0x7fc9381f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93d0b8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9381f0d10 0x7fc9381f1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52478/0 (socket says 172.21.15.26:52478) 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93d0b8640 1 -- 172.21.15.26:0/1136226191 learned_addr learned my addr 172.21.15.26:0/1136226191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc937fff640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc93813eee0 0x7fc93813f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93c8b7640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928004800 0x7fc938140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93d0b8640 1 -- 172.21.15.26:0/1136226191 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc93813eee0 msgr2=0x7fc93813f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93d0b8640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc93813eee0 0x7fc93813f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93d0b8640 1 -- 172.21.15.26:0/1136226191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928004800 msgr2=0x7fc938140ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93d0b8640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928004800 0x7fc938140ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93d0b8640 1 -- 172.21.15.26:0/1136226191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9380a95e0 con 0x7fc9381f0d10 2024-04-24T23:27:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93c8b7640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928004800 0x7fc938140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93d0b8640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9381f0d10 0x7fc9381f1190 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fc938070350 tx=0x7fc930007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc9357fa640 1 -- 172.21.15.26:0/1136226191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93000ab80 con 0x7fc9381f0d10 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc9357fa640 1 -- 172.21.15.26:0/1136226191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc930018070 con 0x7fc9381f0d10 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9380a98c0 con 0x7fc9381f0d10 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc9357fa640 1 -- 172.21.15.26:0/1136226191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9300133f0 con 0x7fc9381f0d10 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.437+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9380a9e80 con 0x7fc9381f0d10 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.438+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc938067720 con 0x7fc9381f0d10 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.439+0000 7fc9357fa640 1 -- 172.21.15.26:0/1136226191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc930013590 con 0x7fc9381f0d10 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.439+0000 7fc9357fa640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc91c06be30 0x7fc91c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.439+0000 7fc9357fa640 1 -- 172.21.15.26:0/1136226191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4857+0+0 (secure 0 0 0) 0x7fc93008b0a0 con 0x7fc9381f0d10 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.439+0000 7fc93c8b7640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc91c06be30 0x7fc91c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.440+0000 7fc93c8b7640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc91c06be30 0x7fc91c06e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0035f0 tx=0x7fc92c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.443+0000 7fc9357fa640 1 -- 172.21.15.26:0/1136226191 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc930059e60 con 0x7fc9381f0d10 2024-04-24T23:27:20.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.627+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9380aa050 con 0x7fc9381f0d10 2024-04-24T23:27:20.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.628+0000 7fc9357fa640 1 -- 172.21.15.26:0/1136226191 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+12832 (secure 0 0 0) 0x7fc930059760 con 0x7fc9381f0d10 2024-04-24T23:27:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:20.629 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":92,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:27:19.924078+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc91c06be30 msgr2=0x7fc91c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc91c06be30 0x7fc91c06e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0035f0 tx=0x7fc92c009040 comp rx=0 tx=0).stop 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9381f0d10 msgr2=0x7fc9381f1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9381f0d10 0x7fc9381f1190 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fc938070350 tx=0x7fc930007bd0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 shutdown_connections 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc91c06be30 0x7fc91c06e2f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9381f0d10 0x7fc9381f1190 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc93813eee0 0x7fc93813f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 --2- 172.21.15.26:0/1136226191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc928004800 0x7fc938140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 >> 172.21.15.26:0/1136226191 conn(0x7fc9380729a0 msgr2=0x7fc938144780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 shutdown_connections 2024-04-24T23:27:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.630+0000 7fc93eb1b640 1 -- 172.21.15.26:0/1136226191 wait complete. 2024-04-24T23:27:20.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:27:20.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:27:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.745+0000 7fbb77b77640 1 Processor -- start 2024-04-24T23:27:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.745+0000 7fbb77b77640 1 -- start start 2024-04-24T23:27:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb77b77640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbb70158ae0 0x7fbb7006e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb77b77640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb70071720 0x7fbb70071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb77b77640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb7006b1c0 con 0x7fbb700d0ce0 2024-04-24T23:27:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb77b77640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbb700720b0 con 0x7fbb70158ae0 2024-04-24T23:27:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb77b77640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7006ec90 con 0x7fbb70071720 2024-04-24T23:27:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb75112640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbb700d0ce0 0x7fbb7015e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37736/0 (socket says 172.21.15.26:37736) 2024-04-24T23:27:20.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb75112640 1 -- 172.21.15.26:0/1115259727 learned_addr learned my addr 172.21.15.26:0/1115259727 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb76114640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb70071720 0x7fbb70071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb75913640 1 --2- 172.21.15.26:0/1115259727 >> v2:172.21.15.29:3300/0 conn(0x7fbb70158ae0 0x7fbb7006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.746+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240096428 0 0) 0x7fbb7006b1c0 con 0x7fbb700d0ce0 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb58003520 con 0x7fbb700d0ce0 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3783080934 0 0) 0x7fbb58003520 con 0x7fbb700d0ce0 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb70071720 msgr2=0x7fbb70071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb70071720 0x7fbb70071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 >> v2:172.21.15.29:3300/0 conn(0x7fbb70158ae0 msgr2=0x7fbb7006e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 --2- 172.21.15.26:0/1115259727 >> v2:172.21.15.29:3300/0 conn(0x7fbb70158ae0 0x7fbb7006e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7006ef70 con 0x7fbb700d0ce0 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fbb64001670 con 0x7fbb700d0ce0 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 >> v1:172.21.15.26:6789/0 conn(0x7fbb700d0ce0 legacy=0x7fbb7015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb580039b0 0x7fbb58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb70071720 0x7fbb58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb58004d40 0x7fbb580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbb700720b0 con 0x7fbb58004d40 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7006ec90 con 0x7fbb580039b0 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5800b9c0 con 0x7fbb70071720 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb76114640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb58004d40 0x7fbb580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb75913640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb580039b0 0x7fbb58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb75112640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb70071720 0x7fbb58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb76114640 1 -- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb70071720 msgr2=0x7fbb58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb76114640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb70071720 0x7fbb58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb76114640 1 -- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb580039b0 msgr2=0x7fbb58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb76114640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb580039b0 0x7fbb58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.747+0000 7fbb76114640 1 -- 172.21.15.26:0/1115259727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb58003520 con 0x7fbb58004d40 2024-04-24T23:27:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.748+0000 7fbb76114640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb58004d40 0x7fbb580051c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009fc0 tx=0x7fbb58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef54e7dd2134e483 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.748+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60009c10 con 0x7fbb58004d40 2024-04-24T23:27:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.748+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb60036070 con 0x7fbb58004d40 2024-04-24T23:27:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.748+0000 7fbb74911640 1 -- 172.21.15.26:0/1115259727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60031880 con 0x7fbb58004d40 2024-04-24T23:27:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.749+0000 7fbb77b77640 1 -- 172.21.15.26:0/1115259727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb58004d40 msgr2=0x7fbb580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.749+0000 7fbb77b77640 1 --2- 172.21.15.26:0/1115259727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb58004d40 0x7fbb580051c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009fc0 tx=0x7fbb58005830 comp rx=0 tx=0).stop 2024-04-24T23:27:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.749+0000 7fbb76114640 1 -- 172.21.15.26:0/1115259727 reap_dead start 2024-04-24T23:27:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.749+0000 7fbb77b77640 1 -- 172.21.15.26:0/1115259727 shutdown_connections 2024-04-24T23:27:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.749+0000 7fbb77b77640 1 -- 172.21.15.26:0/1115259727 >> 172.21.15.26:0/1115259727 conn(0x7fbb700b2970 msgr2=0x7fbb700d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.749+0000 7fbb77b77640 1 -- 172.21.15.26:0/1115259727 shutdown_connections 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.749+0000 7fbb77b77640 1 -- 172.21.15.26:0/1115259727 wait complete. 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.750+0000 7fbb77b77640 1 Processor -- start 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.750+0000 7fbb77b77640 1 -- start start 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb77b77640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb701461c0 0x7fbb70161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb77b77640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb70157e80 0x7fbb700ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb77b77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb700aee00 0x7fbb701f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb77b77640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7006f530 con 0x7fbb700aee00 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb77b77640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7015f060 con 0x7fbb70157e80 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb77b77640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbb700aff60 con 0x7fbb701461c0 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb76114640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb700aee00 0x7fbb701f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb76114640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb700aee00 0x7fbb701f78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52498/0 (socket says 172.21.15.26:52498) 2024-04-24T23:27:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb76114640 1 -- 172.21.15.26:0/36292408 learned_addr learned my addr 172.21.15.26:0/36292408 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb75913640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb701461c0 0x7fbb70161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb75112640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb70157e80 0x7fbb700ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb76114640 1 -- 172.21.15.26:0/36292408 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb701461c0 msgr2=0x7fbb70161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb76114640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb701461c0 0x7fbb70161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb76114640 1 -- 172.21.15.26:0/36292408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb70157e80 msgr2=0x7fbb700ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb76114640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb70157e80 0x7fbb700ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb76114640 1 -- 172.21.15.26:0/36292408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7015faf0 con 0x7fbb700aee00 2024-04-24T23:27:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb75112640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb70157e80 0x7fbb700ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.751+0000 7fbb76114640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb700aee00 0x7fbb701f78b0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbb60033040 tx=0x7fbb60009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.752+0000 7fbb6e7fc640 1 -- 172.21.15.26:0/36292408 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60009c10 con 0x7fbb700aee00 2024-04-24T23:27:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.752+0000 7fbb6e7fc640 1 -- 172.21.15.26:0/36292408 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb60031df0 con 0x7fbb700aee00 2024-04-24T23:27:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.752+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7015fdd0 con 0x7fbb700aee00 2024-04-24T23:27:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.752+0000 7fbb6e7fc640 1 -- 172.21.15.26:0/36292408 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60003bd0 con 0x7fbb700aee00 2024-04-24T23:27:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.752+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb70160390 con 0x7fbb700aee00 2024-04-24T23:27:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.753+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb70075610 con 0x7fbb700aee00 2024-04-24T23:27:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.753+0000 7fbb6e7fc640 1 -- 172.21.15.26:0/36292408 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb60005d10 con 0x7fbb700aee00 2024-04-24T23:27:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.753+0000 7fbb6e7fc640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbb5c06be60 0x7fbb5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.754+0000 7fbb6e7fc640 1 -- 172.21.15.26:0/36292408 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4857+0+0 (secure 0 0 0) 0x7fbb600abcb0 con 0x7fbb700aee00 2024-04-24T23:27:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.754+0000 7fbb75913640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbb5c06be60 0x7fbb5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.754+0000 7fbb75913640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbb5c06be60 0x7fbb5c06e320 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbb70158600 tx=0x7fbb68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.758+0000 7fbb6e7fc640 1 -- 172.21.15.26:0/36292408 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbb6004a060 con 0x7fbb700aee00 2024-04-24T23:27:20.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.940+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb70067580 con 0x7fbb700aee00 2024-04-24T23:27:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.941+0000 7fbb6e7fc640 1 -- 172.21.15.26:0/36292408 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fbb6007ab60 con 0x7fbb700aee00 2024-04-24T23:27:20.942 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbb5c06be60 msgr2=0x7fbb5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbb5c06be60 0x7fbb5c06e320 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbb70158600 tx=0x7fbb68008040 comp rx=0 tx=0).stop 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb700aee00 msgr2=0x7fbb701f78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb700aee00 0x7fbb701f78b0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbb60033040 tx=0x7fbb60009750 comp rx=0 tx=0).stop 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 shutdown_connections 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbb5c06be60 0x7fbb5c06e320 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb700aee00 0x7fbb701f78b0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbb70157e80 0x7fbb700ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 --2- 172.21.15.26:0/36292408 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbb701461c0 0x7fbb70161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 >> 172.21.15.26:0/36292408 conn(0x7fbb700b2970 msgr2=0x7fbb7006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 shutdown_connections 2024-04-24T23:27:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:20.943+0000 7fbb77b77640 1 -- 172.21.15.26:0/36292408 wait complete. 2024-04-24T23:27:20.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:27:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.054+0000 7f0a6c74f640 1 Processor -- start 2024-04-24T23:27:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.054+0000 7f0a6c74f640 1 -- start start 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.055+0000 7f0a6c74f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a64143c40 0x7f0a640cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.055+0000 7f0a6c74f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a6415eb00 0x7f0a6415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.055+0000 7f0a6c74f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a640cd0b0 con 0x7f0a640cc860 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.055+0000 7f0a6c74f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0a6415f490 con 0x7f0a6415eb00 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.055+0000 7f0a6c74f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0a64071b70 con 0x7f0a64143c40 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.056+0000 7f0a6acec640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a6415eb00 0x7f0a6415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.056+0000 7f0a69cea640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0a640cc860 0x7f0a64072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37740/0 (socket says 172.21.15.26:37740) 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.056+0000 7f0a69cea640 1 -- 172.21.15.26:0/3691563869 learned_addr learned my addr 172.21.15.26:0/3691563869 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.056+0000 7f0a6a4eb640 1 --2- 172.21.15.26:0/3691563869 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a64143c40 0x7f0a640cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.056+0000 7f0a6acec640 1 -- 172.21.15.26:0/3691563869 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a64143c40 msgr2=0x7f0a640cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.056+0000 7f0a6acec640 1 --2- 172.21.15.26:0/3691563869 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a64143c40 0x7f0a640cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.056+0000 7f0a6acec640 1 -- 172.21.15.26:0/3691563869 >> v1:172.21.15.26:6789/0 conn(0x7f0a640cc860 legacy=0x7f0a64072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.056+0000 7f0a6acec640 1 -- 172.21.15.26:0/3691563869 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a64071240 con 0x7f0a6415eb00 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a6acec640 1 --2- 172.21.15.26:0/3691563869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a6415eb00 0x7f0a6415ef50 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009d90 tx=0x7f0a5c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947cb64dbd590067 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a694e9640 1 -- 172.21.15.26:0/3691563869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c007740 con 0x7f0a6415eb00 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a694e9640 1 -- 172.21.15.26:0/3691563869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5c0078e0 con 0x7f0a6415eb00 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a694e9640 1 -- 172.21.15.26:0/3691563869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c007cb0 con 0x7f0a6415eb00 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3691563869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a6415eb00 msgr2=0x7f0a6415ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a6c74f640 1 --2- 172.21.15.26:0/3691563869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a6415eb00 0x7f0a6415ef50 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009d90 tx=0x7f0a5c02a660 comp rx=0 tx=0).stop 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3691563869 shutdown_connections 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a6c74f640 1 --2- 172.21.15.26:0/3691563869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a6415eb00 0x7f0a6415ef50 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a6c74f640 1 --2- 172.21.15.26:0/3691563869 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a64143c40 0x7f0a640cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3691563869 >> 172.21.15.26:0/3691563869 conn(0x7f0a6406b300 msgr2=0x7f0a6406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.057+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3691563869 shutdown_connections 2024-04-24T23:27:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.058+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3691563869 wait complete. 2024-04-24T23:27:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6c74f640 1 Processor -- start 2024-04-24T23:27:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6c74f640 1 -- start start 2024-04-24T23:27:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6c74f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a640cc860 0x7f0a6412c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6c74f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a64143c40 0x7f0a640c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6c74f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a6415eb00 0x7f0a640c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6c74f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64144020 con 0x7f0a640cc860 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6c74f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64071800 con 0x7f0a64143c40 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6c74f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f0a640ccca0 con 0x7f0a6415eb00 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6a4eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a640cc860 0x7f0a6412c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.059+0000 7f0a6a4eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a640cc860 0x7f0a6412c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52518/0 (socket says 172.21.15.26:52518) 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a6a4eb640 1 -- 172.21.15.26:0/3327747705 learned_addr learned my addr 172.21.15.26:0/3327747705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a6a4eb640 1 -- 172.21.15.26:0/3327747705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a6415eb00 msgr2=0x7f0a640c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a6a4eb640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a6415eb00 0x7f0a640c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a6a4eb640 1 -- 172.21.15.26:0/3327747705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a64143c40 msgr2=0x7f0a640c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a6a4eb640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a64143c40 0x7f0a640c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a6a4eb640 1 -- 172.21.15.26:0/3327747705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a640ca330 con 0x7f0a640cc860 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a6a4eb640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a640cc860 0x7f0a6412c5a0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400d830 tx=0x7f0a5400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a5affd640 1 -- 172.21.15.26:0/3327747705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5400b6c0 con 0x7f0a640cc860 2024-04-24T23:27:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a5affd640 1 -- 172.21.15.26:0/3327747705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a54019070 con 0x7f0a640cc860 2024-04-24T23:27:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a640ca610 con 0x7f0a640cc860 2024-04-24T23:27:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a5affd640 1 -- 172.21.15.26:0/3327747705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5400ba60 con 0x7f0a640cc860 2024-04-24T23:27:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.060+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a640c9fe0 con 0x7f0a640cc860 2024-04-24T23:27:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.061+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a640694c0 con 0x7f0a640cc860 2024-04-24T23:27:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.062+0000 7f0a5affd640 1 -- 172.21.15.26:0/3327747705 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a5401e460 con 0x7f0a640cc860 2024-04-24T23:27:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.062+0000 7f0a5affd640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a4806be60 0x7f0a4806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.062+0000 7f0a5affd640 1 -- 172.21.15.26:0/3327747705 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4857+0+0 (secure 0 0 0) 0x7f0a5408b1c0 con 0x7f0a640cc860 2024-04-24T23:27:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.062+0000 7f0a69cea640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a4806be60 0x7f0a4806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.062+0000 7f0a69cea640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a4806be60 0x7f0a4806e320 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0a60000f60 tx=0x7f0a60007a80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.066+0000 7f0a5affd640 1 -- 172.21.15.26:0/3327747705 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a54059ec0 con 0x7f0a640cc860 2024-04-24T23:27:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.252+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0a640b4d20 con 0x7f0a640cc860 2024-04-24T23:27:21.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.253+0000 7f0a5affd640 1 -- 172.21.15.26:0/3327747705 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) ==== 84+0+65 (secure 0 0 0) 0x7f0a540597c0 con 0x7f0a640cc860 2024-04-24T23:27:21.254 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:21.254 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-24T23:27:21.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-24T23:27:21.254 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-24T23:27:21.254 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-24T23:27:21.254 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:21.254 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-24T23:27:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a4806be60 msgr2=0x7f0a4806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a4806be60 0x7f0a4806e320 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0a60000f60 tx=0x7f0a60007a80 comp rx=0 tx=0).stop 2024-04-24T23:27:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a640cc860 msgr2=0x7f0a6412c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a640cc860 0x7f0a6412c5a0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400d830 tx=0x7f0a5400dcf0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 shutdown_connections 2024-04-24T23:27:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f0a4806be60 0x7f0a4806e320 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f0a6415eb00 0x7f0a640c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0a64143c40 0x7f0a640c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 --2- 172.21.15.26:0/3327747705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a640cc860 0x7f0a6412c5a0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 >> 172.21.15.26:0/3327747705 conn(0x7f0a6406b300 msgr2=0x7f0a6415e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 shutdown_connections 2024-04-24T23:27:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.255+0000 7f0a6c74f640 1 -- 172.21.15.26:0/3327747705 wait complete. 2024-04-24T23:27:21.266 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:27:21.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:27:21.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.371+0000 7f7dc539d640 1 Processor -- start 2024-04-24T23:27:21.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.371+0000 7f7dc539d640 1 -- start start 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.372+0000 7f7dc539d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7dc015e710 0x7f7dc00d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.372+0000 7f7dc539d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc00d27b0 con 0x7f7dc015e330 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.372+0000 7f7dc539d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc00cef00 con 0x7f7dc00d4630 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.372+0000 7f7dc539d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc00cf1e0 con 0x7f7dc015e710 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.372+0000 7f7dbeffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7dc015e330 0x7f7dc00d3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37746/0 (socket says 172.21.15.26:37746) 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.372+0000 7f7dbeffd640 1 -- 172.21.15.26:0/2605637853 learned_addr learned my addr 172.21.15.26:0/2605637853 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.372+0000 7f7dbe7fc640 1 --2- 172.21.15.26:0/2605637853 >> v2:172.21.15.29:3300/0 conn(0x7f7dc015e710 0x7f7dc00d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.372+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3343729501 0 0) 0x7f7dc00d27b0 con 0x7f7dc015e330 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dac003520 con 0x7f7dc015e330 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4288656859 0 0) 0x7f7dac003520 con 0x7f7dc015e330 2024-04-24T23:27:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 >> v2:172.21.15.29:3300/0 conn(0x7f7dc015e710 msgr2=0x7f7dc00d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 --2- 172.21.15.26:0/2605637853 >> v2:172.21.15.29:3300/0 conn(0x7f7dc015e710 0x7f7dc00d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 >> v1:172.21.15.29:6789/0 conn(0x7f7dc00d4630 legacy=0x7f7dc00b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc00cf4c0 con 0x7f7dc015e330 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f7db8001670 con 0x7f7dc015e330 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 >> v1:172.21.15.26:6789/0 conn(0x7f7dc015e330 legacy=0x7f7dc00d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dac003a00 0x7f7dac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7dac004800 0x7f7dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00cf1e0 con 0x7f7dac0051c0 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac00c0c0 con 0x7f7dac003a00 2024-04-24T23:27:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7dac00c2c0 con 0x7f7dac004800 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbf7fe640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbeffd640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dac003a00 0x7f7dac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbe7fc640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7dac004800 0x7f7dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbf7fe640 1 -- 172.21.15.26:0/2605637853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7dac004800 msgr2=0x7f7dac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbf7fe640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7dac004800 0x7f7dac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbf7fe640 1 -- 172.21.15.26:0/2605637853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dac003a00 msgr2=0x7f7dac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbf7fe640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dac003a00 0x7f7dac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbf7fe640 1 -- 172.21.15.26:0/2605637853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac003520 con 0x7f7dac0051c0 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.373+0000 7f7dbeffd640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dac003a00 0x7f7dac003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.374+0000 7f7dbf7fe640 1 -- 172.21.15.26:0/2605637853 reap_dead start 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.374+0000 7f7dbf7fe640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dac00b9c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7db0001e20 tx=0x7f7dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65a3cfde34e63a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.374+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8007c40 con 0x7f7dac0051c0 2024-04-24T23:27:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.374+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db002ae40 con 0x7f7dac0051c0 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.374+0000 7f7dbdffb640 1 -- 172.21.15.26:0/2605637853 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0001a70 con 0x7f7dac0051c0 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.375+0000 7f7dc539d640 1 -- 172.21.15.26:0/2605637853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 msgr2=0x7f7dac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.375+0000 7f7dc539d640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dac00b9c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7db0001e20 tx=0x7f7dac005830 comp rx=0 tx=0).stop 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.375+0000 7f7dc539d640 1 -- 172.21.15.26:0/2605637853 shutdown_connections 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.375+0000 7f7dc539d640 1 --2- 172.21.15.26:0/2605637853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dac00b9c0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.375+0000 7f7dc539d640 1 -- 172.21.15.26:0/2605637853 >> 172.21.15.26:0/2605637853 conn(0x7f7dc008dbc0 msgr2=0x7f7dc009d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.375+0000 7f7dc539d640 1 -- 172.21.15.26:0/2605637853 shutdown_connections 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.375+0000 7f7dc539d640 1 -- 172.21.15.26:0/2605637853 wait complete. 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.376+0000 7f7dc539d640 1 Processor -- start 2024-04-24T23:27:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.376+0000 7f7dc539d640 1 -- start start 2024-04-24T23:27:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dc539d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dc01614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dc539d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7dc0161a10 0x7f7dc014e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dc539d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dc015fe70 0x7f7dc014e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dc539d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00d2230 con 0x7f7dac0051c0 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dc539d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00d20b0 con 0x7f7dc015fe70 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dc539d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc00d23b0 con 0x7f7dc0161a10 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dc01614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dc01614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52532/0 (socket says 172.21.15.26:52532) 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbeffd640 1 -- 172.21.15.26:0/2470470284 learned_addr learned my addr 172.21.15.26:0/2470470284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbe7fc640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7dc0161a10 0x7f7dc014e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbf7fe640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dc015fe70 0x7f7dc014e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbeffd640 1 -- 172.21.15.26:0/2470470284 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7dc0161a10 msgr2=0x7f7dc014e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbeffd640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7dc0161a10 0x7f7dc014e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbeffd640 1 -- 172.21.15.26:0/2470470284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dc015fe70 msgr2=0x7f7dc014e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbeffd640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dc015fe70 0x7f7dc014e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbeffd640 1 -- 172.21.15.26:0/2470470284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc0160430 con 0x7f7dac0051c0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.377+0000 7f7dbeffd640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dc01614d0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7db8007290 tx=0x7f7db8007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.378+0000 7f7db77fe640 1 -- 172.21.15.26:0/2470470284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db800f9c0 con 0x7f7dac0051c0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.378+0000 7f7db77fe640 1 -- 172.21.15.26:0/2470470284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db8018070 con 0x7f7dac0051c0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.378+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc0160710 con 0x7f7dac0051c0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.378+0000 7f7db77fe640 1 -- 172.21.15.26:0/2470470284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db800fd60 con 0x7f7dac0051c0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.378+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc0160cd0 con 0x7f7dac0051c0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.379+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc00694c0 con 0x7f7dac0051c0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.379+0000 7f7db77fe640 1 -- 172.21.15.26:0/2470470284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7db801d460 con 0x7f7dac0051c0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.379+0000 7f7db77fe640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7da006be30 0x7f7da006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.380+0000 7f7db77fe640 1 -- 172.21.15.26:0/2470470284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4857+0+0 (secure 0 0 0) 0x7f7db808ac30 con 0x7f7dac0051c0 2024-04-24T23:27:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.380+0000 7f7dbe7fc640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7da006be30 0x7f7da006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.380+0000 7f7dbe7fc640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7da006be30 0x7f7da006e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7dc01513e0 tx=0x7f7da8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.384+0000 7f7db77fe640 1 -- 172.21.15.26:0/2470470284 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7db8059920 con 0x7f7dac0051c0 2024-04-24T23:27:21.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.568+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7dc00d5060 con 0x7f7dac0051c0 2024-04-24T23:27:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.918+0000 7f7db77fe640 1 -- 172.21.15.26:0/2470470284 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) ==== 138+0+0 (secure 0 0 0) 0x7f7db8059220 con 0x7f7dac0051c0 2024-04-24T23:27:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-24T23:27:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.920+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7da006be30 msgr2=0x7f7da006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.920+0000 7f7dc539d640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7da006be30 0x7f7da006e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7dc01513e0 tx=0x7f7da8008040 comp rx=0 tx=0).stop 2024-04-24T23:27:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.920+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 msgr2=0x7f7dc01614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.920+0000 7f7dc539d640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dc01614d0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7db8007290 tx=0x7f7db8007690 comp rx=0 tx=0).stop 2024-04-24T23:27:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.921+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 shutdown_connections 2024-04-24T23:27:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.921+0000 7f7dc539d640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7da006be30 0x7f7da006e2f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.921+0000 7f7dc539d640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7dc015fe70 0x7f7dc014e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.921+0000 7f7dc539d640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7dc0161a10 0x7f7dc014e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.921+0000 7f7dc539d640 1 --2- 172.21.15.26:0/2470470284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dac0051c0 0x7f7dc01614d0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.921+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 >> 172.21.15.26:0/2470470284 conn(0x7f7dc008dbc0 msgr2=0x7f7dc00b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.921+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 shutdown_connections 2024-04-24T23:27:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:21.921+0000 7f7dc539d640 1 -- 172.21.15.26:0/2470470284 wait complete. 2024-04-24T23:27:21.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:27:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.045+0000 7fd78cde2640 1 Processor -- start 2024-04-24T23:27:22.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.045+0000 7fd78cde2640 1 -- start start 2024-04-24T23:27:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.046+0000 7fd78cde2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd78815be40 0x7fd7880acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.046+0000 7fd78cde2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7880aeb10 con 0x7fd7880d0750 2024-04-24T23:27:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.046+0000 7fd78cde2640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7880aedf0 con 0x7fd7880b4ce0 2024-04-24T23:27:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.046+0000 7fd78cde2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd78809df20 con 0x7fd78815be40 2024-04-24T23:27:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd786d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd7880d0750 0x7fd78815c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37760/0 (socket says 172.21.15.26:37760) 2024-04-24T23:27:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd786d9d640 1 -- 172.21.15.26:0/2292903514 learned_addr learned my addr 172.21.15.26:0/2292903514 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78659c640 1 --2- 172.21.15.26:0/2292903514 >> v2:172.21.15.29:3300/0 conn(0x7fd78815be40 0x7fd7880acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 995767427 0 0) 0x7fd7880aeb10 con 0x7fd7880d0750 2024-04-24T23:27:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd774003520 con 0x7fd7880d0750 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1579311787 0 0) 0x7fd774003520 con 0x7fd7880d0750 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 >> v2:172.21.15.29:3300/0 conn(0x7fd78815be40 msgr2=0x7fd7880acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 --2- 172.21.15.26:0/2292903514 >> v2:172.21.15.29:3300/0 conn(0x7fd78815be40 0x7fd7880acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 >> v1:172.21.15.29:6789/0 conn(0x7fd7880b4ce0 legacy=0x7fd7880d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7880af0d0 con 0x7fd7880d0750 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fd780001670 con 0x7fd7880d0750 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78659c640 1 --2- 172.21.15.26:0/2292903514 >> v2:172.21.15.29:3300/0 conn(0x7fd78815be40 0x7fd7880acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 >> v1:172.21.15.26:6789/0 conn(0x7fd7880d0750 legacy=0x7fd78815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7740039b0 0x7fd774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd774004800 0x7fd774004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7740051c0 0x7fd77400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:22.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd77400c030 con 0x7fd774004800 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd77400c1e0 con 0x7fd7740039b0 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd77400c3e0 con 0x7fd7740051c0 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd785d9b640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd774004800 0x7fd774004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.047+0000 7fd78659c640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7740039b0 0x7fd774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.048+0000 7fd786d9d640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7740051c0 0x7fd77400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.048+0000 7fd785d9b640 1 -- 172.21.15.26:0/2292903514 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7740051c0 msgr2=0x7fd77400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.048+0000 7fd785d9b640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7740051c0 0x7fd77400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.048+0000 7fd785d9b640 1 -- 172.21.15.26:0/2292903514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7740039b0 msgr2=0x7fd774003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.048+0000 7fd785d9b640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7740039b0 0x7fd774003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.048+0000 7fd785d9b640 1 -- 172.21.15.26:0/2292903514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd774003520 con 0x7fd774004800 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.048+0000 7fd786d9d640 1 -- 172.21.15.26:0/2292903514 reap_dead start 2024-04-24T23:27:22.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.048+0000 7fd785d9b640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd774004800 0x7fd774004c80 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd770001750 tx=0x7fd774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ff908a62d96f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770001370 con 0x7fd774004800 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7700052d0 con 0x7fd774004800 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78559a640 1 -- 172.21.15.26:0/2292903514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77002abc0 con 0x7fd774004800 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78cde2640 1 -- 172.21.15.26:0/2292903514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd774004800 msgr2=0x7fd774004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78cde2640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd774004800 0x7fd774004c80 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd770001750 tx=0x7fd774005830 comp rx=0 tx=0).stop 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78cde2640 1 -- 172.21.15.26:0/2292903514 shutdown_connections 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78cde2640 1 --2- 172.21.15.26:0/2292903514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd774004800 0x7fd774004c80 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78cde2640 1 -- 172.21.15.26:0/2292903514 >> 172.21.15.26:0/2292903514 conn(0x7fd78806b540 msgr2=0x7fd78806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78cde2640 1 -- 172.21.15.26:0/2292903514 shutdown_connections 2024-04-24T23:27:22.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.049+0000 7fd78cde2640 1 -- 172.21.15.26:0/2292903514 wait complete. 2024-04-24T23:27:22.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd78cde2640 1 Processor -- start 2024-04-24T23:27:22.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd78cde2640 1 -- start start 2024-04-24T23:27:22.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd78cde2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd774004800 0x7fd7881f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:22.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd78cde2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd788151e50 0x7fd78814e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:22.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd78cde2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd78809cbd0 0x7fd78809d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:22.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd78cde2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd78806cd10 con 0x7fd788151e50 2024-04-24T23:27:22.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd78cde2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd78806cb90 con 0x7fd774004800 2024-04-24T23:27:22.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd78cde2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd7880acd30 con 0x7fd78809cbd0 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd785d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd788151e50 0x7fd78814e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd785d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd788151e50 0x7fd78814e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52548/0 (socket says 172.21.15.26:52548) 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd785d9b640 1 -- 172.21.15.26:0/4178993437 learned_addr learned my addr 172.21.15.26:0/4178993437 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.051+0000 7fd786d9d640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd78809cbd0 0x7fd78809d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd78659c640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd774004800 0x7fd7881f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd785d9b640 1 -- 172.21.15.26:0/4178993437 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd78809cbd0 msgr2=0x7fd78809d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd785d9b640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd78809cbd0 0x7fd78809d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd785d9b640 1 -- 172.21.15.26:0/4178993437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd774004800 msgr2=0x7fd7881f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd785d9b640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd774004800 0x7fd7881f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd785d9b640 1 -- 172.21.15.26:0/4178993437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd788152f60 con 0x7fd788151e50 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd785d9b640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd788151e50 0x7fd78814e8b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd770000c00 tx=0x7fd770001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd77effd640 1 -- 172.21.15.26:0/4178993437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77003f030 con 0x7fd788151e50 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd77effd640 1 -- 172.21.15.26:0/4178993437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd770005480 con 0x7fd788151e50 2024-04-24T23:27:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7881f4fb0 con 0x7fd788151e50 2024-04-24T23:27:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd77effd640 1 -- 172.21.15.26:0/4178993437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770037040 con 0x7fd788151e50 2024-04-24T23:27:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.052+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7881f5570 con 0x7fd788151e50 2024-04-24T23:27:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.053+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd788075610 con 0x7fd788151e50 2024-04-24T23:27:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.054+0000 7fd77effd640 1 -- 172.21.15.26:0/4178993437 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd770042070 con 0x7fd788151e50 2024-04-24T23:27:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.054+0000 7fd77effd640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76806be30 0x7fd76806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.054+0000 7fd78659c640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76806be30 0x7fd76806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.054+0000 7fd77effd640 1 -- 172.21.15.26:0/4178993437 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5227+0+0 (secure 0 0 0) 0x7fd7700aa4a0 con 0x7fd788151e50 2024-04-24T23:27:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.055+0000 7fd78659c640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76806be30 0x7fd76806e2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd778009fd0 tx=0x7fd77800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.059+0000 7fd77effd640 1 -- 172.21.15.26:0/4178993437 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd770078a70 con 0x7fd788151e50 2024-04-24T23:27:22.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.246+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd788158d80 con 0x7fd788151e50 2024-04-24T23:27:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.920+0000 7fd77effd640 1 -- 172.21.15.26:0/4178993437 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) ==== 170+0+0 (secure 0 0 0) 0x7fd77007cd70 con 0x7fd788151e50 2024-04-24T23:27:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-24T23:27:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.923+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76806be30 msgr2=0x7fd76806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.923+0000 7fd78cde2640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76806be30 0x7fd76806e2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd778009fd0 tx=0x7fd77800b040 comp rx=0 tx=0).stop 2024-04-24T23:27:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.923+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd788151e50 msgr2=0x7fd78814e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.923+0000 7fd78cde2640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd788151e50 0x7fd78814e8b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd770000c00 tx=0x7fd770001120 comp rx=0 tx=0).stop 2024-04-24T23:27:22.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.924+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 shutdown_connections 2024-04-24T23:27:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.924+0000 7fd78cde2640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76806be30 0x7fd76806e2f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.924+0000 7fd78cde2640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd78809cbd0 0x7fd78809d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.924+0000 7fd78cde2640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd788151e50 0x7fd78814e8b0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.924+0000 7fd78cde2640 1 --2- 172.21.15.26:0/4178993437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd774004800 0x7fd7881f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.924+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 >> 172.21.15.26:0/4178993437 conn(0x7fd78806b540 msgr2=0x7fd7880af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.924+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 shutdown_connections 2024-04-24T23:27:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:22.924+0000 7fd78cde2640 1 -- 172.21.15.26:0/4178993437 wait complete. 2024-04-24T23:27:22.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:27:23.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.057+0000 7fa155012640 1 Processor -- start 2024-04-24T23:27:23.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.057+0000 7fa155012640 1 -- start start 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.058+0000 7fa155012640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 0x7fa15006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.058+0000 7fa155012640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15015f700 con 0x7fa15015e310 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.058+0000 7fa155012640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1500d0310 con 0x7fa15006d180 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.058+0000 7fa155012640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa1500b3740 con 0x7fa15006c440 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.058+0000 7fa14ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa15015e310 0x7fa150162830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37770/0 (socket says 172.21.15.26:37770) 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.058+0000 7fa14ed9d640 1 -- 172.21.15.26:0/3041758432 learned_addr learned my addr 172.21.15.26:0/3041758432 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.059+0000 7fa14e59c640 1 --2- 172.21.15.26:0/3041758432 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 0x7fa15006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.059+0000 7fa14dd9b640 1 -- 172.21.15.26:0/3041758432 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3780770703 0 0) 0x7fa15015f700 con 0x7fa15015e310 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.059+0000 7fa14dd9b640 1 -- 172.21.15.26:0/3041758432 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa138003520 con 0x7fa15015e310 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.059+0000 7fa14dd9b640 1 -- 172.21.15.26:0/3041758432 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734128187 0 0) 0x7fa1500d0310 con 0x7fa15006d180 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.059+0000 7fa14e59c640 1 -- 172.21.15.26:0/3041758432 >> v1:172.21.15.29:6789/0 conn(0x7fa15006d180 legacy=0x7fa1500ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.059+0000 7fa14e59c640 1 -- 172.21.15.26:0/3041758432 >> v1:172.21.15.26:6789/0 conn(0x7fa15015e310 legacy=0x7fa150162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.059+0000 7fa14e59c640 1 -- 172.21.15.26:0/3041758432 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1500d05f0 con 0x7fa15006c440 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa14e59c640 1 --2- 172.21.15.26:0/3041758432 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 0x7fa15006cc40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1440097d0 tx=0x7fa14402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36394c3cfec11db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa14dd9b640 1 -- 172.21.15.26:0/3041758432 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144037070 con 0x7fa15006c440 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa14dd9b640 1 -- 172.21.15.26:0/3041758432 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa14402ac10 con 0x7fa15006c440 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa14dd9b640 1 -- 172.21.15.26:0/3041758432 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140001670 con 0x7fa15006c440 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa155012640 1 -- 172.21.15.26:0/3041758432 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 msgr2=0x7fa15006cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa155012640 1 --2- 172.21.15.26:0/3041758432 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 0x7fa15006cc40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1440097d0 tx=0x7fa14402a180 comp rx=0 tx=0).stop 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa155012640 1 -- 172.21.15.26:0/3041758432 shutdown_connections 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa155012640 1 --2- 172.21.15.26:0/3041758432 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 0x7fa15006cc40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa155012640 1 -- 172.21.15.26:0/3041758432 >> 172.21.15.26:0/3041758432 conn(0x7fa1500d1dd0 msgr2=0x7fa1500d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa155012640 1 -- 172.21.15.26:0/3041758432 shutdown_connections 2024-04-24T23:27:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.060+0000 7fa155012640 1 -- 172.21.15.26:0/3041758432 wait complete. 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.062+0000 7fa155012640 1 Processor -- start 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.062+0000 7fa155012640 1 -- start start 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa155012640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 0x7fa1501540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa155012640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa15006d180 0x7fa150154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa155012640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15015e310 0x7fa150154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa155012640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1500d0e90 con 0x7fa15006d180 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa155012640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa1500d0bb0 con 0x7fa15015e310 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa155012640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa1500d13b0 con 0x7fa15006c440 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa15006d180 0x7fa150154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa15006d180 0x7fa150154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52562/0 (socket says 172.21.15.26:52562) 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14e59c640 1 -- 172.21.15.26:0/1683633852 learned_addr learned my addr 172.21.15.26:0/1683633852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14ed9d640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 0x7fa1501540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14f59e640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15015e310 0x7fa150154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14e59c640 1 -- 172.21.15.26:0/1683633852 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 msgr2=0x7fa1501540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14e59c640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 0x7fa1501540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14e59c640 1 -- 172.21.15.26:0/1683633852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15015e310 msgr2=0x7fa150154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14e59c640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15015e310 0x7fa150154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14e59c640 1 -- 172.21.15.26:0/1683633852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1501532e0 con 0x7fa15006d180 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.063+0000 7fa14e59c640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa15006d180 0x7fa150154a80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa144009720 tx=0x7fa1440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.064+0000 7fa1377fe640 1 -- 172.21.15.26:0/1683633852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144037040 con 0x7fa15006d180 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.064+0000 7fa1377fe640 1 -- 172.21.15.26:0/1683633852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa144004020 con 0x7fa15006d180 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.064+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1501535c0 con 0x7fa15006d180 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.064+0000 7fa1377fe640 1 -- 172.21.15.26:0/1683633852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144004410 con 0x7fa15006d180 2024-04-24T23:27:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.064+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa150153b80 con 0x7fa15006d180 2024-04-24T23:27:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.065+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa150075610 con 0x7fa15006d180 2024-04-24T23:27:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.065+0000 7fa1377fe640 1 -- 172.21.15.26:0/1683633852 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa14403a040 con 0x7fa15006d180 2024-04-24T23:27:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.066+0000 7fa1377fe640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa13c06be30 0x7fa13c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.066+0000 7fa1377fe640 1 -- 172.21.15.26:0/1683633852 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5609+0+0 (secure 0 0 0) 0x7fa1440ab110 con 0x7fa15006d180 2024-04-24T23:27:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.066+0000 7fa14ed9d640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa13c06be30 0x7fa13c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.066+0000 7fa14ed9d640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa13c06be30 0x7fa13c06e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa140001600 tx=0x7fa140001570 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.070+0000 7fa1377fe640 1 -- 172.21.15.26:0/1683633852 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa14404b030 con 0x7fa15006d180 2024-04-24T23:27:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.295+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa1500b7c30 con 0x7fa15006d180 2024-04-24T23:27:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.940+0000 7fa1377fe640 1 -- 172.21.15.26:0/1683633852 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v99) ==== 174+0+0 (secure 0 0 0) 0x7fa144042030 con 0x7fa15006d180 2024-04-24T23:27:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 14 and data pool 15 2024-04-24T23:27:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.948+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa13c06be30 msgr2=0x7fa13c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.949+0000 7fa155012640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa13c06be30 0x7fa13c06e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa140001600 tx=0x7fa140001570 comp rx=0 tx=0).stop 2024-04-24T23:27:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.949+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa15006d180 msgr2=0x7fa150154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.949+0000 7fa155012640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa15006d180 0x7fa150154a80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa144009720 tx=0x7fa1440046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.953+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 shutdown_connections 2024-04-24T23:27:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.953+0000 7fa155012640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa13c06be30 0x7fa13c06e2f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.954+0000 7fa155012640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa15015e310 0x7fa150154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.954+0000 7fa155012640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa15006d180 0x7fa150154a80 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.954+0000 7fa155012640 1 --2- 172.21.15.26:0/1683633852 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa15006c440 0x7fa1501540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.954+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 >> 172.21.15.26:0/1683633852 conn(0x7fa1500d1dd0 msgr2=0x7fa15015c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.956+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 shutdown_connections 2024-04-24T23:27:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:23.957+0000 7fa155012640 1 -- 172.21.15.26:0/1683633852 wait complete. 2024-04-24T23:27:23.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.078+0000 7facecc31640 1 Processor -- start 2024-04-24T23:27:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.078+0000 7facecc31640 1 -- start start 2024-04-24T23:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7facecc31640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7face80b63e0 0x7face8142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7facecc31640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face8145890 0x7face80d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7facecc31640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face806deb0 con 0x7face8142310 2024-04-24T23:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7facecc31640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7face806dc60 con 0x7face80b63e0 2024-04-24T23:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7facecc31640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7face806eac0 con 0x7face8145890 2024-04-24T23:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7face6d9d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face8145890 0x7face80d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7face6d9d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face8145890 0x7face80d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:32904/0 (socket says 172.21.15.26:32904) 2024-04-24T23:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7face6d9d640 1 -- 172.21.15.26:0/1565950545 learned_addr learned my addr 172.21.15.26:0/1565950545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7facdffff640 1 --2- 172.21.15.26:0/1565950545 >> v2:172.21.15.29:3300/0 conn(0x7face80b63e0 0x7face8142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7face6d9d640 1 -- 172.21.15.26:0/1565950545 >> v2:172.21.15.29:3300/0 conn(0x7face80b63e0 msgr2=0x7face8142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7face6d9d640 1 --2- 172.21.15.26:0/1565950545 >> v2:172.21.15.29:3300/0 conn(0x7face80b63e0 0x7face8142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7face6d9d640 1 -- 172.21.15.26:0/1565950545 >> v1:172.21.15.29:6789/0 conn(0x7face8142310 legacy=0x7face80b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.079+0000 7face6d9d640 1 -- 172.21.15.26:0/1565950545 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face806e190 con 0x7face8145890 2024-04-24T23:27:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.080+0000 7face6d9d640 1 --2- 172.21.15.26:0/1565950545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face8145890 0x7face80d02a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7face0009e50 tx=0x7face002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb4d180877c110a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.080+0000 7face5d9b640 1 -- 172.21.15.26:0/1565950545 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0036070 con 0x7face8145890 2024-04-24T23:27:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.080+0000 7face5d9b640 1 -- 172.21.15.26:0/1565950545 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face0031600 con 0x7face8145890 2024-04-24T23:27:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.080+0000 7face5d9b640 1 -- 172.21.15.26:0/1565950545 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00319d0 con 0x7face8145890 2024-04-24T23:27:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.081+0000 7facecc31640 1 -- 172.21.15.26:0/1565950545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face8145890 msgr2=0x7face80d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.081+0000 7facecc31640 1 --2- 172.21.15.26:0/1565950545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face8145890 0x7face80d02a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7face0009e50 tx=0x7face002a680 comp rx=0 tx=0).stop 2024-04-24T23:27:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.081+0000 7facecc31640 1 -- 172.21.15.26:0/1565950545 shutdown_connections 2024-04-24T23:27:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.081+0000 7facecc31640 1 --2- 172.21.15.26:0/1565950545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face8145890 0x7face80d02a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.081+0000 7facecc31640 1 --2- 172.21.15.26:0/1565950545 >> v2:172.21.15.29:3300/0 conn(0x7face80b63e0 0x7face8142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.081+0000 7facecc31640 1 -- 172.21.15.26:0/1565950545 >> 172.21.15.26:0/1565950545 conn(0x7face80729a0 msgr2=0x7face806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.081+0000 7facecc31640 1 -- 172.21.15.26:0/1565950545 shutdown_connections 2024-04-24T23:27:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.081+0000 7facecc31640 1 -- 172.21.15.26:0/1565950545 wait complete. 2024-04-24T23:27:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.082+0000 7facecc31640 1 Processor -- start 2024-04-24T23:27:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.082+0000 7facecc31640 1 -- start start 2024-04-24T23:27:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7facecc31640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face80b63e0 0x7face813d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7facecc31640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7face8142310 0x7face81406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7facecc31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7face8145890 0x7face81f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7facecc31640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7face81426f0 con 0x7face8145890 2024-04-24T23:27:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7facecc31640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7face806e750 con 0x7face8142310 2024-04-24T23:27:24.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7facecc31640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7face8142870 con 0x7face80b63e0 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7face6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7face8145890 0x7face81f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7face6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7face8145890 0x7face81f2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60488/0 (socket says 172.21.15.26:60488) 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7face6d9d640 1 -- 172.21.15.26:0/2878932814 learned_addr learned my addr 172.21.15.26:0/2878932814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7facdffff640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7face8142310 0x7face81406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.083+0000 7face659c640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face80b63e0 0x7face813d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facdffff640 1 -- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face80b63e0 msgr2=0x7face813d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facdffff640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face80b63e0 0x7face813d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facdffff640 1 -- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7face8145890 msgr2=0x7face81f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facdffff640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7face8145890 0x7face81f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facdffff640 1 -- 172.21.15.26:0/2878932814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face81410d0 con 0x7face8142310 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7face6d9d640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7face8145890 0x7face81f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7face659c640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face80b63e0 0x7face813d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:24.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facdffff640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7face8142310 0x7face81406f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7facd000e880 tx=0x7facd000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facdeffd640 1 -- 172.21.15.26:0/2878932814 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0017070 con 0x7face8142310 2024-04-24T23:27:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facdeffd640 1 -- 172.21.15.26:0/2878932814 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd000c940 con 0x7face8142310 2024-04-24T23:27:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face813e0c0 con 0x7face8142310 2024-04-24T23:27:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facdeffd640 1 -- 172.21.15.26:0/2878932814 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd000cd30 con 0x7face8142310 2024-04-24T23:27:24.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.084+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face813e680 con 0x7face8142310 2024-04-24T23:27:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.085+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face8067720 con 0x7face8142310 2024-04-24T23:27:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.086+0000 7facdeffd640 1 -- 172.21.15.26:0/2878932814 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7facd001a070 con 0x7face8142310 2024-04-24T23:27:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.086+0000 7facdeffd640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7facc806be30 0x7facc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.086+0000 7facdeffd640 1 -- 172.21.15.26:0/2878932814 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7facd008b1a0 con 0x7face8142310 2024-04-24T23:27:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.086+0000 7face659c640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7facc806be30 0x7facc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.087+0000 7face659c640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7facc806be30 0x7facc806e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7facd8001b70 tx=0x7facd8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.091+0000 7facdeffd640 1 -- 172.21.15.26:0/2878932814 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7facd0059460 con 0x7face8142310 2024-04-24T23:27:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.277+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7face806c3a0 con 0x7face8142310 2024-04-24T23:27:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.278+0000 7facdeffd640 1 -- 172.21.15.26:0/2878932814 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16561 (secure 0 0 0) 0x7facd005d760 con 0x7face8142310 2024-04-24T23:27:24.280 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:24.280 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":95,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:27:23.296805+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:21.600058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:22.278230+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:24.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7facc806be30 msgr2=0x7facc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7facc806be30 0x7facc806e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7facd8001b70 tx=0x7facd8009040 comp rx=0 tx=0).stop 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7face8142310 msgr2=0x7face81406f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7face8142310 0x7face81406f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7facd000e880 tx=0x7facd000ed40 comp rx=0 tx=0).stop 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 shutdown_connections 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7facc806be30 0x7facc806e2f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7face8145890 0x7face81f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7face8142310 0x7face81406f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 --2- 172.21.15.26:0/2878932814 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7face80b63e0 0x7face813d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.280+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 >> 172.21.15.26:0/2878932814 conn(0x7face80729a0 msgr2=0x7face806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.281+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 shutdown_connections 2024-04-24T23:27:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.281+0000 7facecc31640 1 -- 172.21.15.26:0/2878932814 wait complete. 2024-04-24T23:27:24.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.393+0000 7f727ef10640 1 Processor -- start 2024-04-24T23:27:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.393+0000 7f727ef10640 1 -- start start 2024-04-24T23:27:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727ef10640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72780b3ac0 0x7f72780b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727ef10640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f72780ce320 0x7f72780ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727ef10640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7278156b40 con 0x7f7278156760 2024-04-24T23:27:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727ef10640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f72780cfed0 con 0x7f72780ce320 2024-04-24T23:27:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727ef10640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f72780d01b0 con 0x7f72780b3ac0 2024-04-24T23:27:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727d4ad640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f72780ce320 0x7f72780ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727d4ad640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f72780ce320 0x7f72780ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60506/0 (socket says 172.21.15.26:60506) 2024-04-24T23:27:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727d4ad640 1 -- 172.21.15.26:0/2988713926 learned_addr learned my addr 172.21.15.26:0/2988713926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f7277fff640 1 --2- 172.21.15.26:0/2988713926 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72780b3ac0 0x7f72780b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727d4ad640 1 -- 172.21.15.26:0/2988713926 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72780b3ac0 msgr2=0x7f72780b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727d4ad640 1 --2- 172.21.15.26:0/2988713926 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72780b3ac0 0x7f72780b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727d4ad640 1 -- 172.21.15.26:0/2988713926 >> v1:172.21.15.29:6789/0 conn(0x7f7278156760 legacy=0x7f727815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.394+0000 7f727d4ad640 1 -- 172.21.15.26:0/2988713926 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72780d0490 con 0x7f72780ce320 2024-04-24T23:27:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.395+0000 7f727d4ad640 1 --2- 172.21.15.26:0/2988713926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 0x7f72780ce750 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7270009e50 tx=0x7f727002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb84646af3cbebd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.395+0000 7f72777fe640 1 -- 172.21.15.26:0/2988713926 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270036070 con 0x7f72780ce320 2024-04-24T23:27:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.395+0000 7f72777fe640 1 -- 172.21.15.26:0/2988713926 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7270031600 con 0x7f72780ce320 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.395+0000 7f72777fe640 1 -- 172.21.15.26:0/2988713926 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72700319d0 con 0x7f72780ce320 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.396+0000 7f727ef10640 1 -- 172.21.15.26:0/2988713926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 msgr2=0x7f72780ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.396+0000 7f727ef10640 1 --2- 172.21.15.26:0/2988713926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 0x7f72780ce750 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7270009e50 tx=0x7f727002a710 comp rx=0 tx=0).stop 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.396+0000 7f727ef10640 1 -- 172.21.15.26:0/2988713926 shutdown_connections 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.396+0000 7f727ef10640 1 --2- 172.21.15.26:0/2988713926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 0x7f72780ce750 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.396+0000 7f727ef10640 1 --2- 172.21.15.26:0/2988713926 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72780b3ac0 0x7f72780b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.396+0000 7f727ef10640 1 -- 172.21.15.26:0/2988713926 >> 172.21.15.26:0/2988713926 conn(0x7f727806b120 msgr2=0x7f727806b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.396+0000 7f727ef10640 1 -- 172.21.15.26:0/2988713926 shutdown_connections 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.396+0000 7f727ef10640 1 -- 172.21.15.26:0/2988713926 wait complete. 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.397+0000 7f727ef10640 1 Processor -- start 2024-04-24T23:27:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.397+0000 7f727ef10640 1 -- start start 2024-04-24T23:27:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f727ef10640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72780b3ac0 0x7f72780b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f727ef10640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 0x7f72781535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f727ef10640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7278156760 0x7f7278153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f727ef10640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f72780cc7e0 con 0x7f72780ce320 2024-04-24T23:27:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f727ef10640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f72780d0a50 con 0x7f72780b3ac0 2024-04-24T23:27:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f727ef10640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f72780cfb30 con 0x7f7278156760 2024-04-24T23:27:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f7277fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 0x7f72781535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f7277fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 0x7f72781535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60508/0 (socket says 172.21.15.26:60508) 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f7277fff640 1 -- 172.21.15.26:0/2107951502 learned_addr learned my addr 172.21.15.26:0/2107951502 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f727d4ad640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7278156760 0x7f7278153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f727ccac640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72780b3ac0 0x7f72780b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f7277fff640 1 -- 172.21.15.26:0/2107951502 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7278156760 msgr2=0x7f7278153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f7277fff640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7278156760 0x7f7278153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f7277fff640 1 -- 172.21.15.26:0/2107951502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72780b3ac0 msgr2=0x7f72780b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f7277fff640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72780b3ac0 0x7f72780b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f7277fff640 1 -- 172.21.15.26:0/2107951502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72780c7450 con 0x7f72780ce320 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f727ccac640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72780b3ac0 0x7f72780b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.398+0000 7f7277fff640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 0x7f72781535c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f726400d830 tx=0x7f726400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.399+0000 7f72757fa640 1 -- 172.21.15.26:0/2107951502 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726400b6c0 con 0x7f72780ce320 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.399+0000 7f72757fa640 1 -- 172.21.15.26:0/2107951502 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7264019070 con 0x7f72780ce320 2024-04-24T23:27:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.399+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72780c7b30 con 0x7f72780ce320 2024-04-24T23:27:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.399+0000 7f72757fa640 1 -- 172.21.15.26:0/2107951502 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726400ba80 con 0x7f72780ce320 2024-04-24T23:27:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.399+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72780c80f0 con 0x7f72780ce320 2024-04-24T23:27:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.400+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72780694c0 con 0x7f72780ce320 2024-04-24T23:27:24.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.400+0000 7f72757fa640 1 -- 172.21.15.26:0/2107951502 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f726400bc20 con 0x7f72780ce320 2024-04-24T23:27:24.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.401+0000 7f72757fa640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f725c06be30 0x7f725c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.401+0000 7f72757fa640 1 -- 172.21.15.26:0/2107951502 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f726408b280 con 0x7f72780ce320 2024-04-24T23:27:24.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.401+0000 7f727ccac640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f725c06be30 0x7f725c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.401+0000 7f727ccac640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f725c06be30 0x7f725c06e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f726c001660 tx=0x7f726c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.405+0000 7f72757fa640 1 -- 172.21.15.26:0/2107951502 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7264059c30 con 0x7f72780ce320 2024-04-24T23:27:24.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.594+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72780ccb70 con 0x7f72780ce320 2024-04-24T23:27:24.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.595+0000 7f72757fa640 1 -- 172.21.15.26:0/2107951502 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16561 (secure 0 0 0) 0x7f7264059530 con 0x7f72780ce320 2024-04-24T23:27:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:24.596 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":95,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:27:23.296805+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:21.600058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:22.278230+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:24.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.597+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f725c06be30 msgr2=0x7f725c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.597+0000 7f727ef10640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f725c06be30 0x7f725c06e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f726c001660 tx=0x7f726c009040 comp rx=0 tx=0).stop 2024-04-24T23:27:24.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.597+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 msgr2=0x7f72781535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.597+0000 7f727ef10640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 0x7f72781535c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f726400d830 tx=0x7f726400dcf0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.598+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 shutdown_connections 2024-04-24T23:27:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.598+0000 7f727ef10640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f725c06be30 0x7f725c06e2f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.598+0000 7f727ef10640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7278156760 0x7f7278153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.598+0000 7f727ef10640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72780ce320 0x7f72781535c0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.598+0000 7f727ef10640 1 --2- 172.21.15.26:0/2107951502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72780b3ac0 0x7f72780b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.598+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 >> 172.21.15.26:0/2107951502 conn(0x7f727806b120 msgr2=0x7f727806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.598+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 shutdown_connections 2024-04-24T23:27:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.598+0000 7f727ef10640 1 -- 172.21.15.26:0/2107951502 wait complete. 2024-04-24T23:27:24.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:27:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.712+0000 7f7fdc1ee640 1 Processor -- start 2024-04-24T23:27:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.712+0000 7f7fdc1ee640 1 -- start start 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.713+0000 7f7fdc1ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd415e370 0x7f7fd40d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.713+0000 7f7fdc1ee640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd40d2120 con 0x7f7fd40d4c10 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.713+0000 7f7fdc1ee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd40b3740 con 0x7f7fd415e750 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.713+0000 7f7fdc1ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd40d1510 con 0x7f7fd415e370 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.713+0000 7f7fda78b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7fd40d4c10 0x7f7fd40b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58362/0 (socket says 172.21.15.26:58362) 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.713+0000 7f7fda78b640 1 -- 172.21.15.26:0/1528931393 learned_addr learned my addr 172.21.15.26:0/1528931393 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.713+0000 7f7fd9f8a640 1 --2- 172.21.15.26:0/1528931393 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd415e370 0x7f7fd40d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.714+0000 7f7fd8f88640 1 -- 172.21.15.26:0/1528931393 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1402005386 0 0) 0x7f7fd40d2120 con 0x7f7fd40d4c10 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.714+0000 7f7fd8f88640 1 -- 172.21.15.26:0/1528931393 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fbc003520 con 0x7f7fd40d4c10 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.714+0000 7f7fd8f88640 1 -- 172.21.15.26:0/1528931393 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2208169030 0 0) 0x7f7fd40b3740 con 0x7f7fd415e750 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.714+0000 7f7fd8f88640 1 -- 172.21.15.26:0/1528931393 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fd40d2120 con 0x7f7fd415e750 2024-04-24T23:27:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.714+0000 7f7fd9f8a640 1 -- 172.21.15.26:0/1528931393 >> v1:172.21.15.29:6789/0 conn(0x7f7fd415e750 legacy=0x7f7fd40b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.714+0000 7f7fd9f8a640 1 -- 172.21.15.26:0/1528931393 >> v1:172.21.15.26:6789/0 conn(0x7f7fd40d4c10 legacy=0x7f7fd40b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.714+0000 7f7fd9f8a640 1 -- 172.21.15.26:0/1528931393 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd40b3a20 con 0x7f7fd415e370 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fd9f8a640 1 --2- 172.21.15.26:0/1528931393 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd415e370 0x7f7fd40d2520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009dc0 tx=0x7f7fc402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da67273a6ae7b6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fd8f88640 1 -- 172.21.15.26:0/1528931393 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4007740 con 0x7f7fd415e370 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fd8f88640 1 -- 172.21.15.26:0/1528931393 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc40078e0 con 0x7f7fd415e370 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fd8f88640 1 -- 172.21.15.26:0/1528931393 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4007cb0 con 0x7f7fd415e370 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1528931393 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd415e370 msgr2=0x7f7fd40d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fdc1ee640 1 --2- 172.21.15.26:0/1528931393 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd415e370 0x7f7fd40d2520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009dc0 tx=0x7f7fc402a6d0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1528931393 shutdown_connections 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fdc1ee640 1 --2- 172.21.15.26:0/1528931393 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd415e370 0x7f7fd40d2520 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1528931393 >> 172.21.15.26:0/1528931393 conn(0x7f7fd40d40f0 msgr2=0x7f7fd40d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1528931393 shutdown_connections 2024-04-24T23:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.715+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1528931393 wait complete. 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fdc1ee640 1 Processor -- start 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fdc1ee640 1 -- start start 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fdc1ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40d4c10 0x7f7fd40cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fdc1ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd415e370 0x7f7fd41f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fdc1ee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd41f7c40 0x7f7fd41f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fdc1ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd40cfca0 con 0x7f7fd415e370 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fdc1ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd40d1ed0 con 0x7f7fd40d4c10 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fdc1ee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd40cfe20 con 0x7f7fd41f7c40 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fd9789640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd415e370 0x7f7fd41f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fd9789640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd415e370 0x7f7fd41f7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60522/0 (socket says 172.21.15.26:60522) 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.717+0000 7f7fd9789640 1 -- 172.21.15.26:0/1095662109 learned_addr learned my addr 172.21.15.26:0/1095662109 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:24.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fda78b640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd41f7c40 0x7f7fd41f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd9f8a640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40d4c10 0x7f7fd40cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd9789640 1 -- 172.21.15.26:0/1095662109 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd41f7c40 msgr2=0x7f7fd41f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd9789640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd41f7c40 0x7f7fd41f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd9789640 1 -- 172.21.15.26:0/1095662109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40d4c10 msgr2=0x7f7fd40cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd9789640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40d4c10 0x7f7fd40cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd9789640 1 -- 172.21.15.26:0/1095662109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd40cb920 con 0x7f7fd415e370 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd9789640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd415e370 0x7f7fd41f7700 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c530 tx=0x7f7fc800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd27fc640 1 -- 172.21.15.26:0/1095662109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80052d0 con 0x7f7fd415e370 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd27fc640 1 -- 172.21.15.26:0/1095662109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc8005490 con 0x7f7fd415e370 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd40cbc00 con 0x7f7fd415e370 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fd27fc640 1 -- 172.21.15.26:0/1095662109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8012870 con 0x7f7fd415e370 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.718+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd40cc1c0 con 0x7f7fd415e370 2024-04-24T23:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.719+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd41f8050 con 0x7f7fd415e370 2024-04-24T23:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.720+0000 7f7fd27fc640 1 -- 172.21.15.26:0/1095662109 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fc8012a10 con 0x7f7fd415e370 2024-04-24T23:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.720+0000 7f7fd27fc640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fc006be30 0x7f7fc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.720+0000 7f7fd27fc640 1 -- 172.21.15.26:0/1095662109 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f7fc808a940 con 0x7f7fd415e370 2024-04-24T23:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.720+0000 7f7fd9f8a640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fc006be30 0x7f7fc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.721+0000 7f7fd9f8a640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fc006be30 0x7f7fc006e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4034000 tx=0x7f7fc402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.725+0000 7f7fd27fc640 1 -- 172.21.15.26:0/1095662109 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7fd41f8050 con 0x7f7fd415e370 2024-04-24T23:27:24.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:24.952+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7fd4075610 con 0x7f7fd415e370 2024-04-24T23:27:25.920 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:27:25.919+0000 7f42b4d71640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:27:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.944+0000 7f7fd27fc640 1 -- 172.21.15.26:0/1095662109 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v102) ==== 118+0+0 (secure 0 0 0) 0x7f7fc8058cb0 con 0x7f7fd415e370 2024-04-24T23:27:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fc006be30 msgr2=0x7f7fc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fc006be30 0x7f7fc006e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4034000 tx=0x7f7fc402f000 comp rx=0 tx=0).stop 2024-04-24T23:27:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd415e370 msgr2=0x7f7fd41f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd415e370 0x7f7fd41f7700 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c530 tx=0x7f7fc800c9f0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 shutdown_connections 2024-04-24T23:27:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7fc006be30 0x7f7fc006e2f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7fd41f7c40 0x7f7fd41f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fd415e370 0x7f7fd41f7700 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 --2- 172.21.15.26:0/1095662109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7fd40d4c10 0x7f7fd40cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 >> 172.21.15.26:0/1095662109 conn(0x7f7fd40d40f0 msgr2=0x7f7fd40b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 shutdown_connections 2024-04-24T23:27:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:25.947+0000 7f7fdc1ee640 1 -- 172.21.15.26:0/1095662109 wait complete. 2024-04-24T23:27:25.961 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:27:25.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:27:26.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.071+0000 7fd78789a640 1 Processor -- start 2024-04-24T23:27:26.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.071+0000 7fd78789a640 1 -- start start 2024-04-24T23:27:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd78789a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7800b37e0 0x7fd7800b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd78789a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd780070f30 con 0x7fd78015e120 2024-04-24T23:27:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd78789a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd78006dcf0 con 0x7fd78006d5a0 2024-04-24T23:27:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd78789a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd7800d0750 con 0x7fd7800b37e0 2024-04-24T23:27:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd784e35640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7800b37e0 0x7fd7800b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd784e35640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7800b37e0 0x7fd7800b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60526/0 (socket says 172.21.15.26:60526) 2024-04-24T23:27:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd785e37640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd78015e120 0x7fd780150e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58374/0 (socket says 172.21.15.26:58374) 2024-04-24T23:27:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd784e35640 1 -- 172.21.15.26:0/1036991823 learned_addr learned my addr 172.21.15.26:0/1036991823 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd784e35640 1 -- 172.21.15.26:0/1036991823 >> v1:172.21.15.29:6789/0 conn(0x7fd78006d5a0 legacy=0x7fd7800d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd784e35640 1 -- 172.21.15.26:0/1036991823 >> v1:172.21.15.26:6789/0 conn(0x7fd78015e120 legacy=0x7fd780150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.072+0000 7fd784e35640 1 -- 172.21.15.26:0/1036991823 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78006dfd0 con 0x7fd7800b37e0 2024-04-24T23:27:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.073+0000 7fd784e35640 1 --2- 172.21.15.26:0/1036991823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7800b37e0 0x7fd7800b3c50 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd77000bb10 tx=0x7fd77002c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b049a82da65ca7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.073+0000 7fd777fff640 1 -- 172.21.15.26:0/1036991823 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770035070 con 0x7fd7800b37e0 2024-04-24T23:27:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.073+0000 7fd777fff640 1 -- 172.21.15.26:0/1036991823 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd770031830 con 0x7fd7800b37e0 2024-04-24T23:27:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.073+0000 7fd777fff640 1 -- 172.21.15.26:0/1036991823 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770031c00 con 0x7fd7800b37e0 2024-04-24T23:27:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.074+0000 7fd78789a640 1 -- 172.21.15.26:0/1036991823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7800b37e0 msgr2=0x7fd7800b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.074+0000 7fd78789a640 1 --2- 172.21.15.26:0/1036991823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7800b37e0 0x7fd7800b3c50 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd77000bb10 tx=0x7fd77002c9d0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.074+0000 7fd78789a640 1 -- 172.21.15.26:0/1036991823 shutdown_connections 2024-04-24T23:27:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.074+0000 7fd78789a640 1 --2- 172.21.15.26:0/1036991823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7800b37e0 0x7fd7800b3c50 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.074+0000 7fd78789a640 1 -- 172.21.15.26:0/1036991823 >> 172.21.15.26:0/1036991823 conn(0x7fd78006b540 msgr2=0x7fd78006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.074+0000 7fd78789a640 1 -- 172.21.15.26:0/1036991823 shutdown_connections 2024-04-24T23:27:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.074+0000 7fd78789a640 1 -- 172.21.15.26:0/1036991823 wait complete. 2024-04-24T23:27:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.075+0000 7fd78789a640 1 Processor -- start 2024-04-24T23:27:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.075+0000 7fd78789a640 1 -- start start 2024-04-24T23:27:26.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd78789a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd78006d5a0 0x7fd7800cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd78789a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7800b37e0 0x7fd780151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd78789a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78015e120 0x7fd78013e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd78789a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd78006d980 con 0x7fd78015e120 2024-04-24T23:27:26.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd78789a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd78006e590 con 0x7fd78006d5a0 2024-04-24T23:27:26.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd78789a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd780151570 con 0x7fd7800b37e0 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785e37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78015e120 0x7fd78013e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785e37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78015e120 0x7fd78013e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60542/0 (socket says 172.21.15.26:60542) 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785e37640 1 -- 172.21.15.26:0/1666285288 learned_addr learned my addr 172.21.15.26:0/1666285288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd784e35640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7800b37e0 0x7fd780151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785636640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd78006d5a0 0x7fd7800cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785e37640 1 -- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7800b37e0 msgr2=0x7fd780151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785e37640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7800b37e0 0x7fd780151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785e37640 1 -- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd78006d5a0 msgr2=0x7fd7800cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785e37640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd78006d5a0 0x7fd7800cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785e37640 1 -- 172.21.15.26:0/1666285288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd780152110 con 0x7fd78015e120 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.076+0000 7fd785636640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd78006d5a0 0x7fd7800cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.077+0000 7fd784e35640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7800b37e0 0x7fd780151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.077+0000 7fd785e37640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78015e120 0x7fd78013e690 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c760 tx=0x7fd77800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.077+0000 7fd775ffb640 1 -- 172.21.15.26:0/1666285288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778017070 con 0x7fd78015e120 2024-04-24T23:27:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.077+0000 7fd775ffb640 1 -- 172.21.15.26:0/1666285288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77800a640 con 0x7fd78015e120 2024-04-24T23:27:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.077+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78013ebd0 con 0x7fd78015e120 2024-04-24T23:27:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.077+0000 7fd775ffb640 1 -- 172.21.15.26:0/1666285288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77800a9f0 con 0x7fd78015e120 2024-04-24T23:27:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.077+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7800cd110 con 0x7fd78015e120 2024-04-24T23:27:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.078+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd780067a40 con 0x7fd78015e120 2024-04-24T23:27:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.079+0000 7fd775ffb640 1 -- 172.21.15.26:0/1666285288 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd77801a070 con 0x7fd78015e120 2024-04-24T23:27:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.079+0000 7fd775ffb640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76406be30 0x7fd76406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.079+0000 7fd775ffb640 1 -- 172.21.15.26:0/1666285288 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fd77808a2e0 con 0x7fd78015e120 2024-04-24T23:27:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.079+0000 7fd785636640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76406be30 0x7fd76406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.079+0000 7fd785636640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76406be30 0x7fd76406e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd77c001680 tx=0x7fd77c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.083+0000 7fd775ffb640 1 -- 172.21.15.26:0/1666285288 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd780067a40 con 0x7fd78015e120 2024-04-24T23:27:26.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.310+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd78015b510 con 0x7fd78015e120 2024-04-24T23:27:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.945+0000 7fd775ffb640 1 -- 172.21.15.26:0/1666285288 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v103) ==== 122+0+0 (secure 0 0 0) 0x7fd778058640 con 0x7fd78015e120 2024-04-24T23:27:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76406be30 msgr2=0x7fd76406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76406be30 0x7fd76406e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd77c001680 tx=0x7fd77c009040 comp rx=0 tx=0).stop 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78015e120 msgr2=0x7fd78013e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78015e120 0x7fd78013e690 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c760 tx=0x7fd77800cc20 comp rx=0 tx=0).stop 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 shutdown_connections 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd76406be30 0x7fd76406e2f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78015e120 0x7fd78013e690 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd7800b37e0 0x7fd780151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 --2- 172.21.15.26:0/1666285288 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd78006d5a0 0x7fd7800cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 >> 172.21.15.26:0/1666285288 conn(0x7fd78006b540 msgr2=0x7fd78006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.948+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 shutdown_connections 2024-04-24T23:27:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:26.949+0000 7fd78789a640 1 -- 172.21.15.26:0/1666285288 wait complete. 2024-04-24T23:27:26.963 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:27:26.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.069+0000 7f866dcf5640 1 Processor -- start 2024-04-24T23:27:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.070+0000 7f866dcf5640 1 -- start start 2024-04-24T23:27:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.070+0000 7f866dcf5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f866815e570 0x7f86680d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.070+0000 7f866dcf5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f86680d22e0 0x7f86680d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.070+0000 7f866dcf5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86680d32f0 con 0x7f86680b3130 2024-04-24T23:27:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.070+0000 7f866dcf5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f86680d1ab0 con 0x7f866815e570 2024-04-24T23:27:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.070+0000 7f866dcf5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f86680d2ae0 con 0x7f86680d22e0 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.071+0000 7f86677fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f866815e570 0x7f86680d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.071+0000 7f8666ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86680b3130 0x7f86680d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58376/0 (socket says 172.21.15.26:58376) 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.071+0000 7f8666ffd640 1 -- 172.21.15.26:0/3668888206 learned_addr learned my addr 172.21.15.26:0/3668888206 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.071+0000 7f8667fff640 1 --2- 172.21.15.26:0/3668888206 >> v2:172.21.15.29:3300/0 conn(0x7f86680d22e0 0x7f86680d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.071+0000 7f86677fe640 1 -- 172.21.15.26:0/3668888206 >> v2:172.21.15.29:3300/0 conn(0x7f86680d22e0 msgr2=0x7f86680d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.071+0000 7f86677fe640 1 --2- 172.21.15.26:0/3668888206 >> v2:172.21.15.29:3300/0 conn(0x7f86680d22e0 0x7f86680d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.071+0000 7f86677fe640 1 -- 172.21.15.26:0/3668888206 >> v1:172.21.15.26:6789/0 conn(0x7f86680b3130 legacy=0x7f86680d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.071+0000 7f86677fe640 1 -- 172.21.15.26:0/3668888206 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86680d0270 con 0x7f866815e570 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.071+0000 7f8667fff640 1 --2- 172.21.15.26:0/3668888206 >> v2:172.21.15.29:3300/0 conn(0x7f86680d22e0 0x7f86680d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f86677fe640 1 --2- 172.21.15.26:0/3668888206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866815e570 0x7f86680d4160 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f86680b6d60 tx=0x7f865802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0c4b16b1fc9126 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f86667fc640 1 -- 172.21.15.26:0/3668888206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658037070 con 0x7f866815e570 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f86667fc640 1 -- 172.21.15.26:0/3668888206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8658032440 con 0x7f866815e570 2024-04-24T23:27:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f86667fc640 1 -- 172.21.15.26:0/3668888206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658032810 con 0x7f866815e570 2024-04-24T23:27:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f866dcf5640 1 -- 172.21.15.26:0/3668888206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866815e570 msgr2=0x7f86680d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f866dcf5640 1 --2- 172.21.15.26:0/3668888206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866815e570 0x7f86680d4160 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f86680b6d60 tx=0x7f865802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f866dcf5640 1 -- 172.21.15.26:0/3668888206 shutdown_connections 2024-04-24T23:27:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f866dcf5640 1 --2- 172.21.15.26:0/3668888206 >> v2:172.21.15.29:3300/0 conn(0x7f86680d22e0 0x7f86680d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f866dcf5640 1 --2- 172.21.15.26:0/3668888206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866815e570 0x7f86680d4160 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f866dcf5640 1 -- 172.21.15.26:0/3668888206 >> 172.21.15.26:0/3668888206 conn(0x7f866808dbe0 msgr2=0x7f866809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f866dcf5640 1 -- 172.21.15.26:0/3668888206 shutdown_connections 2024-04-24T23:27:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.072+0000 7f866dcf5640 1 -- 172.21.15.26:0/3668888206 wait complete. 2024-04-24T23:27:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.074+0000 7f866dcf5640 1 Processor -- start 2024-04-24T23:27:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.074+0000 7f866dcf5640 1 -- start start 2024-04-24T23:27:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.074+0000 7f866dcf5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86680b3130 0x7f8668152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.074+0000 7f866dcf5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680d22e0 0x7f866815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f866dcf5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866815e570 0x7f866815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f866dcf5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86680cfa70 con 0x7f86680d22e0 2024-04-24T23:27:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f866dcf5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f86680d3010 con 0x7f866815e570 2024-04-24T23:27:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f866dcf5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f86680cfd50 con 0x7f86680b3130 2024-04-24T23:27:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8666ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680d22e0 0x7f866815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8666ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680d22e0 0x7f866815c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60560/0 (socket says 172.21.15.26:60560) 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8666ffd640 1 -- 172.21.15.26:0/2082556960 learned_addr learned my addr 172.21.15.26:0/2082556960 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8667fff640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866815e570 0x7f866815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f86677fe640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86680b3130 0x7f8668152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8666ffd640 1 -- 172.21.15.26:0/2082556960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86680b3130 msgr2=0x7f8668152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8666ffd640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86680b3130 0x7f8668152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8666ffd640 1 -- 172.21.15.26:0/2082556960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866815e570 msgr2=0x7f866815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8666ffd640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866815e570 0x7f866815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8666ffd640 1 -- 172.21.15.26:0/2082556960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86681f7520 con 0x7f86680d22e0 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f8666ffd640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680d22e0 0x7f866815c980 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f865c00c400 tx=0x7f865c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f864ffff640 1 -- 172.21.15.26:0/2082556960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c017070 con 0x7f86680d22e0 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f864ffff640 1 -- 172.21.15.26:0/2082556960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865c005ce0 con 0x7f86680d22e0 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86681f7800 con 0x7f86680d22e0 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f864ffff640 1 -- 172.21.15.26:0/2082556960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0072f0 con 0x7f86680d22e0 2024-04-24T23:27:27.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.075+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86681f7dc0 con 0x7f86680d22e0 2024-04-24T23:27:27.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.076+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86680694c0 con 0x7f86680d22e0 2024-04-24T23:27:27.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.077+0000 7f864ffff640 1 -- 172.21.15.26:0/2082556960 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f865c01a070 con 0x7f86680d22e0 2024-04-24T23:27:27.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.077+0000 7f864ffff640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f865406be30 0x7f865406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.078+0000 7f86677fe640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f865406be30 0x7f865406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.078+0000 7f864ffff640 1 -- 172.21.15.26:0/2082556960 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f865c08a1e0 con 0x7f86680d22e0 2024-04-24T23:27:27.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.078+0000 7f86677fe640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f865406be30 0x7f865406e2f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8658034000 tx=0x7f865802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:27.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.082+0000 7f864ffff640 1 -- 172.21.15.26:0/2082556960 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86680d32f0 con 0x7f86680d22e0 2024-04-24T23:27:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.302+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86680cf830 con 0x7f86680d22e0 2024-04-24T23:27:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.303+0000 7f864ffff640 1 -- 172.21.15.26:0/2082556960 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5185 (secure 0 0 0) 0x7f865c058540 con 0x7f86680d22e0 2024-04-24T23:27:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:27.304 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":103,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":7444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:26.311710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730]},"id":7}]} 2024-04-24T23:27:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 103 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.304+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f865406be30 msgr2=0x7f865406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.304+0000 7f866dcf5640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f865406be30 0x7f865406e2f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8658034000 tx=0x7f865802e000 comp rx=0 tx=0).stop 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.304+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680d22e0 msgr2=0x7f866815c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.304+0000 7f866dcf5640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680d22e0 0x7f866815c980 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f865c00c400 tx=0x7f865c00c8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.304+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 shutdown_connections 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.305+0000 7f866dcf5640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f865406be30 0x7f865406e2f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.305+0000 7f866dcf5640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f866815e570 0x7f866815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.305+0000 7f866dcf5640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680d22e0 0x7f866815c980 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.305+0000 7f866dcf5640 1 --2- 172.21.15.26:0/2082556960 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86680b3130 0x7f8668152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.305+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 >> 172.21.15.26:0/2082556960 conn(0x7f866808dbe0 msgr2=0x7f86680d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.305+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 shutdown_connections 2024-04-24T23:27:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.305+0000 7f866dcf5640 1 -- 172.21.15.26:0/2082556960 wait complete. 2024-04-24T23:27:27.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:27.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.417+0000 7f6221970640 1 Processor -- start 2024-04-24T23:27:27.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.417+0000 7f6221970640 1 -- start start 2024-04-24T23:27:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f6221970640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c143c40 0x7f621c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f6221970640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f621c15eb00 0x7f621c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f6221970640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621c0cd0b0 con 0x7f621c0cc860 2024-04-24T23:27:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f6221970640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f621c15f490 con 0x7f621c15eb00 2024-04-24T23:27:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f6221970640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f621c071b70 con 0x7f621c143c40 2024-04-24T23:27:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f621b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f621c15eb00 0x7f621c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f621b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f621c15eb00 0x7f621c15ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60576/0 (socket says 172.21.15.26:60576) 2024-04-24T23:27:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f621a7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f621c0cc860 0x7f621c072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58390/0 (socket says 172.21.15.26:58390) 2024-04-24T23:27:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f621b7fe640 1 -- 172.21.15.26:0/1815369571 learned_addr learned my addr 172.21.15.26:0/1815369571 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f621affd640 1 --2- 172.21.15.26:0/1815369571 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c143c40 0x7f621c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f621b7fe640 1 -- 172.21.15.26:0/1815369571 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c143c40 msgr2=0x7f621c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f621b7fe640 1 --2- 172.21.15.26:0/1815369571 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c143c40 0x7f621c0cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f621b7fe640 1 -- 172.21.15.26:0/1815369571 >> v1:172.21.15.26:6789/0 conn(0x7f621c0cc860 legacy=0x7f621c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f621b7fe640 1 -- 172.21.15.26:0/1815369571 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c071240 con 0x7f621c15eb00 2024-04-24T23:27:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.418+0000 7f6219ffb640 1 -- 172.21.15.26:0/1815369571 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436679712 0 0) 0x7f621c0cd0b0 con 0x7f621c0cc860 2024-04-24T23:27:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.419+0000 7f621b7fe640 1 --2- 172.21.15.26:0/1815369571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c15eb00 0x7f621c15ef50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f620c009e50 tx=0x7f620c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6ba7492dbc9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.419+0000 7f6219ffb640 1 -- 172.21.15.26:0/1815369571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c036070 con 0x7f621c15eb00 2024-04-24T23:27:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.419+0000 7f6219ffb640 1 -- 172.21.15.26:0/1815369571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c031600 con 0x7f621c15eb00 2024-04-24T23:27:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.419+0000 7f6219ffb640 1 -- 172.21.15.26:0/1815369571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c0319d0 con 0x7f621c15eb00 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.420+0000 7f6221970640 1 -- 172.21.15.26:0/1815369571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c15eb00 msgr2=0x7f621c15ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.420+0000 7f6221970640 1 --2- 172.21.15.26:0/1815369571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c15eb00 0x7f621c15ef50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f620c009e50 tx=0x7f620c02a710 comp rx=0 tx=0).stop 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.420+0000 7f6221970640 1 -- 172.21.15.26:0/1815369571 shutdown_connections 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.420+0000 7f6221970640 1 --2- 172.21.15.26:0/1815369571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c15eb00 0x7f621c15ef50 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.420+0000 7f6221970640 1 --2- 172.21.15.26:0/1815369571 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c143c40 0x7f621c0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.420+0000 7f6221970640 1 -- 172.21.15.26:0/1815369571 >> 172.21.15.26:0/1815369571 conn(0x7f621c06b300 msgr2=0x7f621c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.420+0000 7f6221970640 1 -- 172.21.15.26:0/1815369571 shutdown_connections 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.420+0000 7f6221970640 1 -- 172.21.15.26:0/1815369571 wait complete. 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.421+0000 7f6221970640 1 Processor -- start 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.421+0000 7f6221970640 1 -- start start 2024-04-24T23:27:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f6221970640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c0cc860 0x7f621c0c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f6221970640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f621c143c40 0x7f621c13bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f6221970640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c15eb00 0x7f621c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f6221970640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f621c144020 con 0x7f621c0cc860 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f6221970640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f621c071800 con 0x7f621c143c40 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f6221970640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f621c0ccca0 con 0x7f621c15eb00 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c0cc860 0x7f621c0c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c0cc860 0x7f621c0c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60582/0 (socket says 172.21.15.26:60582) 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621affd640 1 -- 172.21.15.26:0/91184369 learned_addr learned my addr 172.21.15.26:0/91184369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621b7fe640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c15eb00 0x7f621c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621a7fc640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f621c143c40 0x7f621c13bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621affd640 1 -- 172.21.15.26:0/91184369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c15eb00 msgr2=0x7f621c15c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621affd640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c15eb00 0x7f621c15c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621affd640 1 -- 172.21.15.26:0/91184369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f621c143c40 msgr2=0x7f621c13bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621affd640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f621c143c40 0x7f621c13bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621affd640 1 -- 172.21.15.26:0/91184369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c15b790 con 0x7f621c0cc860 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.422+0000 7f621b7fe640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c15eb00 0x7f621c15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.423+0000 7f621affd640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c0cc860 0x7f621c0c9db0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f621400d800 tx=0x7f621400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.423+0000 7f6213fff640 1 -- 172.21.15.26:0/91184369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621400b6c0 con 0x7f621c0cc860 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.423+0000 7f6213fff640 1 -- 172.21.15.26:0/91184369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6214019070 con 0x7f621c0cc860 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.423+0000 7f6213fff640 1 -- 172.21.15.26:0/91184369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621400ba80 con 0x7f621c0cc860 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.423+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c15ba70 con 0x7f621c0cc860 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.423+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c15c030 con 0x7f621c0cc860 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.424+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621c0694c0 con 0x7f621c0cc860 2024-04-24T23:27:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.424+0000 7f6213fff640 1 -- 172.21.15.26:0/91184369 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f621400bc20 con 0x7f621c0cc860 2024-04-24T23:27:27.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.425+0000 7f6213fff640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f61fc06be30 0x7f61fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.425+0000 7f6213fff640 1 -- 172.21.15.26:0/91184369 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f621408b330 con 0x7f621c0cc860 2024-04-24T23:27:27.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.425+0000 7f621a7fc640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f61fc06be30 0x7f61fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.425+0000 7f621a7fc640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f61fc06be30 0x7f61fc06e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f621c153480 tx=0x7f6204007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.429+0000 7f6213fff640 1 -- 172.21.15.26:0/91184369 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6214059ce0 con 0x7f621c0cc860 2024-04-24T23:27:27.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.613+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f621c0b4d20 con 0x7f621c0cc860 2024-04-24T23:27:27.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.614+0000 7f6213fff640 1 -- 172.21.15.26:0/91184369 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16548 (secure 0 0 0) 0x7f62140595e0 con 0x7f621c0cc860 2024-04-24T23:27:27.615 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:27.615 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:27:24.935708+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:21.600058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:22.278230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.616+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f61fc06be30 msgr2=0x7f61fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.616+0000 7f6221970640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f61fc06be30 0x7f61fc06e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f621c153480 tx=0x7f6204007b50 comp rx=0 tx=0).stop 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.616+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c0cc860 msgr2=0x7f621c0c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.616+0000 7f6221970640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c0cc860 0x7f621c0c9db0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f621400d800 tx=0x7f621400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.617+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 shutdown_connections 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.617+0000 7f6221970640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f61fc06be30 0x7f61fc06e2f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.617+0000 7f6221970640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f621c15eb00 0x7f621c15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.617+0000 7f6221970640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f621c143c40 0x7f621c13bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.617+0000 7f6221970640 1 --2- 172.21.15.26:0/91184369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f621c0cc860 0x7f621c0c9db0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.617+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 >> 172.21.15.26:0/91184369 conn(0x7f621c06b300 msgr2=0x7f621c142310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.617+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 shutdown_connections 2024-04-24T23:27:27.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.617+0000 7f6221970640 1 -- 172.21.15.26:0/91184369 wait complete. 2024-04-24T23:27:27.629 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:27:27.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:27:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.729+0000 7fbc7ada2640 1 Processor -- start 2024-04-24T23:27:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.730+0000 7fbc7ada2640 1 -- start start 2024-04-24T23:27:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.730+0000 7fbc7ada2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc7415e570 0x7fbc740d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.730+0000 7fbc7ada2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fbc740d22e0 0x7fbc740d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.730+0000 7fbc7ada2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc740d32f0 con 0x7fbc740b3130 2024-04-24T23:27:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.730+0000 7fbc7ada2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fbc740d1ab0 con 0x7fbc740d22e0 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.730+0000 7fbc7ada2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbc740d2ae0 con 0x7fbc7415e570 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.730+0000 7fbc73fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbc740b3130 0x7fbc740d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58398/0 (socket says 172.21.15.26:58398) 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.730+0000 7fbc73fff640 1 -- 172.21.15.26:0/940629087 learned_addr learned my addr 172.21.15.26:0/940629087 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc78b3e640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc7415e570 0x7fbc740d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc7933f640 1 --2- 172.21.15.26:0/940629087 >> v2:172.21.15.29:3300/0 conn(0x7fbc740d22e0 0x7fbc740d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648900075 0 0) 0x7fbc68001110 con 0x7fbc740b3130 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc5c003520 con 0x7fbc740b3130 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2969739700 0 0) 0x7fbc5c003520 con 0x7fbc740b3130 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc7415e570 msgr2=0x7fbc740d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc7415e570 0x7fbc740d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 >> v2:172.21.15.29:3300/0 conn(0x7fbc740d22e0 msgr2=0x7fbc740d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 --2- 172.21.15.26:0/940629087 >> v2:172.21.15.29:3300/0 conn(0x7fbc740d22e0 0x7fbc740d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc740d0270 con 0x7fbc740b3130 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fbc68001840 con 0x7fbc740b3130 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc78b3e640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc7415e570 0x7fbc740d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 >> v1:172.21.15.26:6789/0 conn(0x7fbc740b3130 legacy=0x7fbc740d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc7415e570 0x7fbc5c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc5c003ef0 0x7fbc5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc5c004d40 0x7fbc5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbc740d1ab0 con 0x7fbc5c003ef0 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc740d2ae0 con 0x7fbc5c004d40 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c00b9c0 con 0x7fbc7415e570 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc73fff640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc5c003ef0 0x7fbc5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc78b3e640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc7415e570 0x7fbc5c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.731+0000 7fbc7933f640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc5c004d40 0x7fbc5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.732+0000 7fbc73fff640 1 -- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc7415e570 msgr2=0x7fbc5c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.732+0000 7fbc73fff640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc7415e570 0x7fbc5c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.732+0000 7fbc73fff640 1 -- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc5c004d40 msgr2=0x7fbc5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.732+0000 7fbc73fff640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc5c004d40 0x7fbc5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.732+0000 7fbc73fff640 1 -- 172.21.15.26:0/940629087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c003520 con 0x7fbc5c003ef0 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.732+0000 7fbc73fff640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc5c003ef0 0x7fbc5c004800 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800c3e0 tx=0x7fbc5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f6e3f50806a516 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.732+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68032020 con 0x7fbc5c003ef0 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.732+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68005af0 con 0x7fbc5c003ef0 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.732+0000 7fbc737fe640 1 -- 172.21.15.26:0/940629087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6800c0e0 con 0x7fbc5c003ef0 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.733+0000 7fbc7ada2640 1 -- 172.21.15.26:0/940629087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc5c003ef0 msgr2=0x7fbc5c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.733+0000 7fbc7ada2640 1 --2- 172.21.15.26:0/940629087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc5c003ef0 0x7fbc5c004800 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800c3e0 tx=0x7fbc5c005830 comp rx=0 tx=0).stop 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.733+0000 7fbc7933f640 1 -- 172.21.15.26:0/940629087 reap_dead start 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.733+0000 7fbc7ada2640 1 -- 172.21.15.26:0/940629087 shutdown_connections 2024-04-24T23:27:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.733+0000 7fbc7ada2640 1 -- 172.21.15.26:0/940629087 >> 172.21.15.26:0/940629087 conn(0x7fbc7408dbe0 msgr2=0x7fbc7409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.733+0000 7fbc7ada2640 1 -- 172.21.15.26:0/940629087 shutdown_connections 2024-04-24T23:27:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.733+0000 7fbc7ada2640 1 -- 172.21.15.26:0/940629087 wait complete. 2024-04-24T23:27:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.734+0000 7fbc7ada2640 1 Processor -- start 2024-04-24T23:27:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.734+0000 7fbc7ada2640 1 -- start start 2024-04-24T23:27:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc7ada2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc5c003ef0 0x7fbc740cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc7ada2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc740cb630 0x7fbc74150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc7ada2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc740cdcb0 0x7fbc740ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc7ada2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbc740cfa70 con 0x7fbc740cdcb0 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc7ada2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbc740d3010 con 0x7fbc5c003ef0 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc7ada2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fbc740cfd50 con 0x7fbc740cb630 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc7933f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc740cdcb0 0x7fbc740ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc7933f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc740cdcb0 0x7fbc740ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60608/0 (socket says 172.21.15.26:60608) 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc7933f640 1 -- 172.21.15.26:0/1750248586 learned_addr learned my addr 172.21.15.26:0/1750248586 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc73fff640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc740cb630 0x7fbc74150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.735+0000 7fbc78b3e640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc5c003ef0 0x7fbc740cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc7933f640 1 -- 172.21.15.26:0/1750248586 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc740cb630 msgr2=0x7fbc74150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc7933f640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc740cb630 0x7fbc74150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc7933f640 1 -- 172.21.15.26:0/1750248586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc5c003ef0 msgr2=0x7fbc740cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc7933f640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc5c003ef0 0x7fbc740cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc7933f640 1 -- 172.21.15.26:0/1750248586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc740ccdb0 con 0x7fbc740cdcb0 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc7933f640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc740cdcb0 0x7fbc740ce130 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbc740ba250 tx=0x7fbc64011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc717fa640 1 -- 172.21.15.26:0/1750248586 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64019020 con 0x7fbc740cdcb0 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc717fa640 1 -- 172.21.15.26:0/1750248586 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc64004030 con 0x7fbc740cdcb0 2024-04-24T23:27:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc717fa640 1 -- 172.21.15.26:0/1750248586 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64004400 con 0x7fbc740cdcb0 2024-04-24T23:27:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc740cd090 con 0x7fbc740cdcb0 2024-04-24T23:27:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.736+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc74150b70 con 0x7fbc740cdcb0 2024-04-24T23:27:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.737+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc740694c0 con 0x7fbc740cdcb0 2024-04-24T23:27:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.738+0000 7fbc717fa640 1 -- 172.21.15.26:0/1750248586 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc6401c020 con 0x7fbc740cdcb0 2024-04-24T23:27:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.738+0000 7fbc717fa640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbc6006be60 0x7fbc6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.738+0000 7fbc717fa640 1 -- 172.21.15.26:0/1750248586 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fbc6408b1e0 con 0x7fbc740cdcb0 2024-04-24T23:27:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.738+0000 7fbc78b3e640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbc6006be60 0x7fbc6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.738+0000 7fbc78b3e640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbc6006be60 0x7fbc6006e320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c014430 tx=0x7fbc6c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.742+0000 7fbc717fa640 1 -- 172.21.15.26:0/1750248586 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbc64023020 con 0x7fbc740cdcb0 2024-04-24T23:27:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.920+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc740b0e60 con 0x7fbc6006be60 2024-04-24T23:27:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.926+0000 7fbc717fa640 1 -- 172.21.15.26:0/1750248586 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252085 (secure 0 0 0) 0x7fbc740b0e60 con 0x7fbc6006be60 2024-04-24T23:27:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:27.931 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:27:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbc6006be60 msgr2=0x7fbc6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbc6006be60 0x7fbc6006e320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c014430 tx=0x7fbc6c00b040 comp rx=0 tx=0).stop 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc740cdcb0 msgr2=0x7fbc740ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc740cdcb0 0x7fbc740ce130 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbc740ba250 tx=0x7fbc64011cf0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 shutdown_connections 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fbc6006be60 0x7fbc6006e320 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc740cdcb0 0x7fbc740ce130 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fbc740cb630 0x7fbc74150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 --2- 172.21.15.26:0/1750248586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbc5c003ef0 0x7fbc740cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 >> 172.21.15.26:0/1750248586 conn(0x7fbc7408dbe0 msgr2=0x7fbc7406d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 shutdown_connections 2024-04-24T23:27:27.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:27.929+0000 7fbc7ada2640 1 -- 172.21.15.26:0/1750248586 wait complete. 2024-04-24T23:27:27.941 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-04-24T23:27:27.400886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276616,"kb_used_data":10504,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754698104,"statfs":{"total":773094113280,"available":772810858496,"internally_reserved":0,"allocated":10756096,"data_stored":5312404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.481435"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168325+0000","last_change":"2024-04-24T23:27:23.949351+0000","last_active":"2024-04-24T23:27:26.168325+0000","last_peered":"2024-04-24T23:27:26.168325+0000","last_clean":"2024-04-24T23:27:26.168325+0000","last_became_active":"2024-04-24T23:27:23.949080+0000","last_became_peered":"2024-04-24T23:27:23.949080+0000","last_unstale":"2024-04-24T23:27:26.168325+0000","last_undegraded":"2024-04-24T23:27:26.168325+0000","last_fullsized":"2024-04-24T23:27:26.168325+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:44.770975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871849+0000","last_change":"2024-04-24T23:27:23.955385+0000","last_active":"2024-04-24T23:27:25.871849+0000","last_peered":"2024-04-24T23:27:25.871849+0000","last_clean":"2024-04-24T23:27:25.871849+0000","last_became_active":"2024-04-24T23:27:23.955066+0000","last_became_peered":"2024-04-24T23:27:23.955066+0000","last_unstale":"2024-04-24T23:27:25.871849+0000","last_undegraded":"2024-04-24T23:27:25.871849+0000","last_fullsized":"2024-04-24T23:27:25.871849+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:31.436625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168249+0000","last_change":"2024-04-24T23:27:23.956318+0000","last_active":"2024-04-24T23:27:26.168249+0000","last_peered":"2024-04-24T23:27:26.168249+0000","last_clean":"2024-04-24T23:27:26.168249+0000","last_became_active":"2024-04-24T23:27:23.956043+0000","last_became_peered":"2024-04-24T23:27:23.956043+0000","last_unstale":"2024-04-24T23:27:26.168249+0000","last_undegraded":"2024-04-24T23:27:26.168249+0000","last_fullsized":"2024-04-24T23:27:26.168249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:30.170440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943238+0000","last_change":"2024-04-24T23:27:23.940507+0000","last_active":"2024-04-24T23:27:24.943238+0000","last_peered":"2024-04-24T23:27:24.943238+0000","last_clean":"2024-04-24T23:27:24.943238+0000","last_became_active":"2024-04-24T23:27:23.940199+0000","last_became_peered":"2024-04-24T23:27:23.940199+0000","last_unstale":"2024-04-24T23:27:24.943238+0000","last_undegraded":"2024-04-24T23:27:24.943238+0000","last_fullsized":"2024-04-24T23:27:24.943238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:38.893817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480281+0000","last_change":"2024-04-24T23:27:23.949157+0000","last_active":"2024-04-24T23:27:25.480281+0000","last_peered":"2024-04-24T23:27:25.480281+0000","last_clean":"2024-04-24T23:27:25.480281+0000","last_became_active":"2024-04-24T23:27:23.948854+0000","last_became_peered":"2024-04-24T23:27:23.948854+0000","last_unstale":"2024-04-24T23:27:25.480281+0000","last_undegraded":"2024-04-24T23:27:25.480281+0000","last_fullsized":"2024-04-24T23:27:25.480281+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:26.577304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282337+0000","last_change":"2024-04-24T23:27:23.943103+0000","last_active":"2024-04-24T23:27:25.282337+0000","last_peered":"2024-04-24T23:27:25.282337+0000","last_clean":"2024-04-24T23:27:25.282337+0000","last_became_active":"2024-04-24T23:27:23.942803+0000","last_became_peered":"2024-04-24T23:27:23.942803+0000","last_unstale":"2024-04-24T23:27:25.282337+0000","last_undegraded":"2024-04-24T23:27:25.282337+0000","last_fullsized":"2024-04-24T23:27:25.282337+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:38.893817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282214+0000","last_change":"2024-04-24T23:27:23.943043+0000","last_active":"2024-04-24T23:27:25.282214+0000","last_peered":"2024-04-24T23:27:25.282214+0000","last_clean":"2024-04-24T23:27:25.282214+0000","last_became_active":"2024-04-24T23:27:23.942739+0000","last_became_peered":"2024-04-24T23:27:23.942739+0000","last_unstale":"2024-04-24T23:27:25.282214+0000","last_undegraded":"2024-04-24T23:27:25.282214+0000","last_fullsized":"2024-04-24T23:27:25.282214+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:49.313119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871369+0000","last_change":"2024-04-24T23:27:23.944197+0000","last_active":"2024-04-24T23:27:25.871369+0000","last_peered":"2024-04-24T23:27:25.871369+0000","last_clean":"2024-04-24T23:27:25.871369+0000","last_became_active":"2024-04-24T23:27:23.943689+0000","last_became_peered":"2024-04-24T23:27:23.943689+0000","last_unstale":"2024-04-24T23:27:25.871369+0000","last_undegraded":"2024-04-24T23:27:25.871369+0000","last_fullsized":"2024-04-24T23:27:25.871369+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:38.893817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372622+0000","last_change":"2024-04-24T23:27:23.957759+0000","last_active":"2024-04-24T23:27:26.372622+0000","last_peered":"2024-04-24T23:27:26.372622+0000","last_clean":"2024-04-24T23:27:26.372622+0000","last_became_active":"2024-04-24T23:27:23.956887+0000","last_became_peered":"2024-04-24T23:27:23.956887+0000","last_unstale":"2024-04-24T23:27:26.372622+0000","last_undegraded":"2024-04-24T23:27:26.372622+0000","last_fullsized":"2024-04-24T23:27:26.372622+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:47.149812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480616+0000","last_change":"2024-04-24T23:27:23.963044+0000","last_active":"2024-04-24T23:27:25.480616+0000","last_peered":"2024-04-24T23:27:25.480616+0000","last_clean":"2024-04-24T23:27:25.480616+0000","last_became_active":"2024-04-24T23:27:23.962730+0000","last_became_peered":"2024-04-24T23:27:23.962730+0000","last_unstale":"2024-04-24T23:27:25.480616+0000","last_undegraded":"2024-04-24T23:27:25.480616+0000","last_fullsized":"2024-04-24T23:27:25.480616+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:47.149812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282416+0000","last_change":"2024-04-24T23:27:23.951560+0000","last_active":"2024-04-24T23:27:25.282416+0000","last_peered":"2024-04-24T23:27:25.282416+0000","last_clean":"2024-04-24T23:27:25.282416+0000","last_became_active":"2024-04-24T23:27:23.950959+0000","last_became_peered":"2024-04-24T23:27:23.950959+0000","last_unstale":"2024-04-24T23:27:25.282416+0000","last_undegraded":"2024-04-24T23:27:25.282416+0000","last_fullsized":"2024-04-24T23:27:25.282416+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:14.428899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372396+0000","last_change":"2024-04-24T23:27:23.951454+0000","last_active":"2024-04-24T23:27:26.372396+0000","last_peered":"2024-04-24T23:27:26.372396+0000","last_clean":"2024-04-24T23:27:26.372396+0000","last_became_active":"2024-04-24T23:27:23.951071+0000","last_became_peered":"2024-04-24T23:27:23.951071+0000","last_unstale":"2024-04-24T23:27:26.372396+0000","last_undegraded":"2024-04-24T23:27:26.372396+0000","last_fullsized":"2024-04-24T23:27:26.372396+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:26.577304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943057+0000","last_change":"2024-04-24T23:27:23.949072+0000","last_active":"2024-04-24T23:27:24.943057+0000","last_peered":"2024-04-24T23:27:24.943057+0000","last_clean":"2024-04-24T23:27:24.943057+0000","last_became_active":"2024-04-24T23:27:23.948774+0000","last_became_peered":"2024-04-24T23:27:23.948774+0000","last_unstale":"2024-04-24T23:27:24.943057+0000","last_undegraded":"2024-04-24T23:27:24.943057+0000","last_fullsized":"2024-04-24T23:27:24.943057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:47.385604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107453+0000","last_change":"2024-04-24T23:27:23.956360+0000","last_active":"2024-04-24T23:27:25.107453+0000","last_peered":"2024-04-24T23:27:25.107453+0000","last_clean":"2024-04-24T23:27:25.107453+0000","last_became_active":"2024-04-24T23:27:23.956080+0000","last_became_peered":"2024-04-24T23:27:23.956080+0000","last_unstale":"2024-04-24T23:27:25.107453+0000","last_undegraded":"2024-04-24T23:27:25.107453+0000","last_fullsized":"2024-04-24T23:27:25.107453+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:54.764291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.942821+0000","last_change":"2024-04-24T23:27:23.945132+0000","last_active":"2024-04-24T23:27:24.942821+0000","last_peered":"2024-04-24T23:27:24.942821+0000","last_clean":"2024-04-24T23:27:24.942821+0000","last_became_active":"2024-04-24T23:27:23.944841+0000","last_became_peered":"2024-04-24T23:27:23.944841+0000","last_unstale":"2024-04-24T23:27:24.942821+0000","last_undegraded":"2024-04-24T23:27:24.942821+0000","last_fullsized":"2024-04-24T23:27:24.942821+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:25.533495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282184+0000","last_change":"2024-04-24T23:27:23.945180+0000","last_active":"2024-04-24T23:27:25.282184+0000","last_peered":"2024-04-24T23:27:25.282184+0000","last_clean":"2024-04-24T23:27:25.282184+0000","last_became_active":"2024-04-24T23:27:23.944812+0000","last_became_peered":"2024-04-24T23:27:23.944812+0000","last_unstale":"2024-04-24T23:27:25.282184+0000","last_undegraded":"2024-04-24T23:27:25.282184+0000","last_fullsized":"2024-04-24T23:27:25.282184+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:45.241737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.872009+0000","last_change":"2024-04-24T23:27:23.950146+0000","last_active":"2024-04-24T23:27:25.872009+0000","last_peered":"2024-04-24T23:27:25.872009+0000","last_clean":"2024-04-24T23:27:25.872009+0000","last_became_active":"2024-04-24T23:27:23.949831+0000","last_became_peered":"2024-04-24T23:27:23.949831+0000","last_unstale":"2024-04-24T23:27:25.872009+0000","last_undegraded":"2024-04-24T23:27:25.872009+0000","last_fullsized":"2024-04-24T23:27:25.872009+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:12.868499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943480+0000","last_change":"2024-04-24T23:27:23.948203+0000","last_active":"2024-04-24T23:27:24.943480+0000","last_peered":"2024-04-24T23:27:24.943480+0000","last_clean":"2024-04-24T23:27:24.943480+0000","last_became_active":"2024-04-24T23:27:23.947930+0000","last_became_peered":"2024-04-24T23:27:23.947930+0000","last_unstale":"2024-04-24T23:27:24.943480+0000","last_undegraded":"2024-04-24T23:27:24.943480+0000","last_fullsized":"2024-04-24T23:27:24.943480+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:59.743275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107701+0000","last_change":"2024-04-24T23:27:23.946009+0000","last_active":"2024-04-24T23:27:25.107701+0000","last_peered":"2024-04-24T23:27:25.107701+0000","last_clean":"2024-04-24T23:27:25.107701+0000","last_became_active":"2024-04-24T23:27:23.945665+0000","last_became_peered":"2024-04-24T23:27:23.945665+0000","last_unstale":"2024-04-24T23:27:25.107701+0000","last_undegraded":"2024-04-24T23:27:25.107701+0000","last_fullsized":"2024-04-24T23:27:25.107701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:23.547036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372528+0000","last_change":"2024-04-24T23:27:23.953430+0000","last_active":"2024-04-24T23:27:26.372528+0000","last_peered":"2024-04-24T23:27:26.372528+0000","last_clean":"2024-04-24T23:27:26.372528+0000","last_became_active":"2024-04-24T23:27:23.953111+0000","last_became_peered":"2024-04-24T23:27:23.953111+0000","last_unstale":"2024-04-24T23:27:26.372528+0000","last_undegraded":"2024-04-24T23:27:26.372528+0000","last_fullsized":"2024-04-24T23:27:26.372528+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:01.929662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107785+0000","last_change":"2024-04-24T23:27:23.948550+0000","last_active":"2024-04-24T23:27:25.107785+0000","last_peered":"2024-04-24T23:27:25.107785+0000","last_clean":"2024-04-24T23:27:25.107785+0000","last_became_active":"2024-04-24T23:27:23.948285+0000","last_became_peered":"2024-04-24T23:27:23.948285+0000","last_unstale":"2024-04-24T23:27:25.107785+0000","last_undegraded":"2024-04-24T23:27:25.107785+0000","last_fullsized":"2024-04-24T23:27:25.107785+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:02.851699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871413+0000","last_change":"2024-04-24T23:27:23.955648+0000","last_active":"2024-04-24T23:27:25.871413+0000","last_peered":"2024-04-24T23:27:25.871413+0000","last_clean":"2024-04-24T23:27:25.871413+0000","last_became_active":"2024-04-24T23:27:23.955150+0000","last_became_peered":"2024-04-24T23:27:23.955150+0000","last_unstale":"2024-04-24T23:27:25.871413+0000","last_undegraded":"2024-04-24T23:27:25.871413+0000","last_fullsized":"2024-04-24T23:27:25.871413+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:59.743275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871415+0000","last_change":"2024-04-24T23:27:23.954623+0000","last_active":"2024-04-24T23:27:25.871415+0000","last_peered":"2024-04-24T23:27:25.871415+0000","last_clean":"2024-04-24T23:27:25.871415+0000","last_became_active":"2024-04-24T23:27:23.954341+0000","last_became_peered":"2024-04-24T23:27:23.954341+0000","last_unstale":"2024-04-24T23:27:25.871415+0000","last_undegraded":"2024-04-24T23:27:25.871415+0000","last_fullsized":"2024-04-24T23:27:25.871415+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:16.893167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282308+0000","last_change":"2024-04-24T23:27:23.946051+0000","last_active":"2024-04-24T23:27:25.282308+0000","last_peered":"2024-04-24T23:27:25.282308+0000","last_clean":"2024-04-24T23:27:25.282308+0000","last_became_active":"2024-04-24T23:27:23.945694+0000","last_became_peered":"2024-04-24T23:27:23.945694+0000","last_unstale":"2024-04-24T23:27:25.282308+0000","last_undegraded":"2024-04-24T23:27:25.282308+0000","last_fullsized":"2024-04-24T23:27:25.282308+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:16.893167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480755+0000","last_change":"2024-04-24T23:27:23.951410+0000","last_active":"2024-04-24T23:27:25.480755+0000","last_peered":"2024-04-24T23:27:25.480755+0000","last_clean":"2024-04-24T23:27:25.480755+0000","last_became_active":"2024-04-24T23:27:23.951056+0000","last_became_peered":"2024-04-24T23:27:23.951056+0000","last_unstale":"2024-04-24T23:27:25.480755+0000","last_undegraded":"2024-04-24T23:27:25.480755+0000","last_fullsized":"2024-04-24T23:27:25.480755+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:31.921698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871874+0000","last_change":"2024-04-24T23:27:23.958551+0000","last_active":"2024-04-24T23:27:25.871874+0000","last_peered":"2024-04-24T23:27:25.871874+0000","last_clean":"2024-04-24T23:27:25.871874+0000","last_became_active":"2024-04-24T23:27:23.958241+0000","last_became_peered":"2024-04-24T23:27:23.958241+0000","last_unstale":"2024-04-24T23:27:25.871874+0000","last_undegraded":"2024-04-24T23:27:25.871874+0000","last_fullsized":"2024-04-24T23:27:25.871874+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:49.702454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282608+0000","last_change":"2024-04-24T23:27:23.948563+0000","last_active":"2024-04-24T23:27:25.282608+0000","last_peered":"2024-04-24T23:27:25.282608+0000","last_clean":"2024-04-24T23:27:25.282608+0000","last_became_active":"2024-04-24T23:27:23.948285+0000","last_became_peered":"2024-04-24T23:27:23.948285+0000","last_unstale":"2024-04-24T23:27:25.282608+0000","last_undegraded":"2024-04-24T23:27:25.282608+0000","last_fullsized":"2024-04-24T23:27:25.282608+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:49.702454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480417+0000","last_change":"2024-04-24T23:27:23.949786+0000","last_active":"2024-04-24T23:27:25.480417+0000","last_peered":"2024-04-24T23:27:25.480417+0000","last_clean":"2024-04-24T23:27:25.480417+0000","last_became_active":"2024-04-24T23:27:23.949164+0000","last_became_peered":"2024-04-24T23:27:23.949164+0000","last_unstale":"2024-04-24T23:27:25.480417+0000","last_undegraded":"2024-04-24T23:27:25.480417+0000","last_fullsized":"2024-04-24T23:27:25.480417+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:06.686817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871544+0000","last_change":"2024-04-24T23:27:23.960298+0000","last_active":"2024-04-24T23:27:25.871544+0000","last_peered":"2024-04-24T23:27:25.871544+0000","last_clean":"2024-04-24T23:27:25.871544+0000","last_became_active":"2024-04-24T23:27:23.959922+0000","last_became_peered":"2024-04-24T23:27:23.959922+0000","last_unstale":"2024-04-24T23:27:25.871544+0000","last_undegraded":"2024-04-24T23:27:25.871544+0000","last_fullsized":"2024-04-24T23:27:25.871544+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:14.428899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107456+0000","last_change":"2024-04-24T23:27:23.947590+0000","last_active":"2024-04-24T23:27:25.107456+0000","last_peered":"2024-04-24T23:27:25.107456+0000","last_clean":"2024-04-24T23:27:25.107456+0000","last_became_active":"2024-04-24T23:27:23.947269+0000","last_became_peered":"2024-04-24T23:27:23.947269+0000","last_unstale":"2024-04-24T23:27:25.107456+0000","last_undegraded":"2024-04-24T23:27:25.107456+0000","last_fullsized":"2024-04-24T23:27:25.107456+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:34.673219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.942959+0000","last_change":"2024-04-24T23:27:23.953836+0000","last_active":"2024-04-24T23:27:24.942959+0000","last_peered":"2024-04-24T23:27:24.942959+0000","last_clean":"2024-04-24T23:27:24.942959+0000","last_became_active":"2024-04-24T23:27:23.953525+0000","last_became_peered":"2024-04-24T23:27:23.953525+0000","last_unstale":"2024-04-24T23:27:24.942959+0000","last_undegraded":"2024-04-24T23:27:24.942959+0000","last_fullsized":"2024-04-24T23:27:24.942959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:45.241737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168035+0000","last_change":"2024-04-24T23:27:23.951539+0000","last_active":"2024-04-24T23:27:26.168035+0000","last_peered":"2024-04-24T23:27:26.168035+0000","last_clean":"2024-04-24T23:27:26.168035+0000","last_became_active":"2024-04-24T23:27:23.951225+0000","last_became_peered":"2024-04-24T23:27:23.951225+0000","last_unstale":"2024-04-24T23:27:26.168035+0000","last_undegraded":"2024-04-24T23:27:26.168035+0000","last_fullsized":"2024-04-24T23:27:26.168035+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:37.724643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107830+0000","last_change":"2024-04-24T23:27:23.946409+0000","last_active":"2024-04-24T23:27:25.107830+0000","last_peered":"2024-04-24T23:27:25.107830+0000","last_clean":"2024-04-24T23:27:25.107830+0000","last_became_active":"2024-04-24T23:27:23.945965+0000","last_became_peered":"2024-04-24T23:27:23.945965+0000","last_unstale":"2024-04-24T23:27:25.107830+0000","last_undegraded":"2024-04-24T23:27:25.107830+0000","last_fullsized":"2024-04-24T23:27:25.107830+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:03.028400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871993+0000","last_change":"2024-04-24T23:27:22.942606+0000","last_active":"2024-04-24T23:27:25.871993+0000","last_peered":"2024-04-24T23:27:25.871993+0000","last_clean":"2024-04-24T23:27:25.871993+0000","last_became_active":"2024-04-24T23:27:22.942273+0000","last_became_peered":"2024-04-24T23:27:22.942273+0000","last_unstale":"2024-04-24T23:27:25.871993+0000","last_undegraded":"2024-04-24T23:27:25.871993+0000","last_fullsized":"2024-04-24T23:27:25.871993+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:25.573128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943437+0000","last_change":"2024-04-24T23:27:23.953788+0000","last_active":"2024-04-24T23:27:24.943437+0000","last_peered":"2024-04-24T23:27:24.943437+0000","last_clean":"2024-04-24T23:27:24.943437+0000","last_became_active":"2024-04-24T23:27:23.953484+0000","last_became_peered":"2024-04-24T23:27:23.953484+0000","last_unstale":"2024-04-24T23:27:24.943437+0000","last_undegraded":"2024-04-24T23:27:24.943437+0000","last_fullsized":"2024-04-24T23:27:24.943437+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:16.893167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282464+0000","last_change":"2024-04-24T23:27:22.941320+0000","last_active":"2024-04-24T23:27:25.282464+0000","last_peered":"2024-04-24T23:27:25.282464+0000","last_clean":"2024-04-24T23:27:25.282464+0000","last_became_active":"2024-04-24T23:27:22.940992+0000","last_became_peered":"2024-04-24T23:27:22.940992+0000","last_unstale":"2024-04-24T23:27:25.282464+0000","last_undegraded":"2024-04-24T23:27:25.282464+0000","last_fullsized":"2024-04-24T23:27:25.282464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:27.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168356+0000","last_change":"2024-04-24T23:27:23.956266+0000","last_active":"2024-04-24T23:27:26.168356+0000","last_peered":"2024-04-24T23:27:26.168356+0000","last_clean":"2024-04-24T23:27:26.168356+0000","last_became_active":"2024-04-24T23:27:23.953605+0000","last_became_peered":"2024-04-24T23:27:23.953605+0000","last_unstale":"2024-04-24T23:27:26.168356+0000","last_undegraded":"2024-04-24T23:27:26.168356+0000","last_fullsized":"2024-04-24T23:27:26.168356+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:16.570967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871625+0000","last_change":"2024-04-24T23:27:22.945850+0000","last_active":"2024-04-24T23:27:25.871625+0000","last_peered":"2024-04-24T23:27:25.871625+0000","last_clean":"2024-04-24T23:27:25.871625+0000","last_became_active":"2024-04-24T23:27:22.945577+0000","last_became_peered":"2024-04-24T23:27:22.945577+0000","last_unstale":"2024-04-24T23:27:25.871625+0000","last_undegraded":"2024-04-24T23:27:25.871625+0000","last_fullsized":"2024-04-24T23:27:25.871625+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:09.433090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372554+0000","last_change":"2024-04-24T23:27:23.952642+0000","last_active":"2024-04-24T23:27:26.372554+0000","last_peered":"2024-04-24T23:27:26.372554+0000","last_clean":"2024-04-24T23:27:26.372554+0000","last_became_active":"2024-04-24T23:27:23.952409+0000","last_became_peered":"2024-04-24T23:27:23.952409+0000","last_unstale":"2024-04-24T23:27:26.372554+0000","last_undegraded":"2024-04-24T23:27:26.372554+0000","last_fullsized":"2024-04-24T23:27:26.372554+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:28.282192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871573+0000","last_change":"2024-04-24T23:27:22.944547+0000","last_active":"2024-04-24T23:27:25.871573+0000","last_peered":"2024-04-24T23:27:25.871573+0000","last_clean":"2024-04-24T23:27:25.871573+0000","last_became_active":"2024-04-24T23:27:22.943908+0000","last_became_peered":"2024-04-24T23:27:22.943908+0000","last_unstale":"2024-04-24T23:27:25.871573+0000","last_undegraded":"2024-04-24T23:27:25.871573+0000","last_fullsized":"2024-04-24T23:27:25.871573+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:09.273639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943924+0000","last_change":"2024-04-24T23:27:23.948127+0000","last_active":"2024-04-24T23:27:24.943924+0000","last_peered":"2024-04-24T23:27:24.943924+0000","last_clean":"2024-04-24T23:27:24.943924+0000","last_became_active":"2024-04-24T23:27:23.946659+0000","last_became_peered":"2024-04-24T23:27:23.946659+0000","last_unstale":"2024-04-24T23:27:24.943924+0000","last_undegraded":"2024-04-24T23:27:24.943924+0000","last_fullsized":"2024-04-24T23:27:24.943924+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:56.193488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168129+0000","last_change":"2024-04-24T23:27:22.945122+0000","last_active":"2024-04-24T23:27:26.168129+0000","last_peered":"2024-04-24T23:27:26.168129+0000","last_clean":"2024-04-24T23:27:26.168129+0000","last_became_active":"2024-04-24T23:27:22.944769+0000","last_became_peered":"2024-04-24T23:27:22.944769+0000","last_unstale":"2024-04-24T23:27:26.168129+0000","last_undegraded":"2024-04-24T23:27:26.168129+0000","last_fullsized":"2024-04-24T23:27:26.168129+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:37.426619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282289+0000","last_change":"2024-04-24T23:27:23.954439+0000","last_active":"2024-04-24T23:27:25.282289+0000","last_peered":"2024-04-24T23:27:25.282289+0000","last_clean":"2024-04-24T23:27:25.282289+0000","last_became_active":"2024-04-24T23:27:23.954166+0000","last_became_peered":"2024-04-24T23:27:23.954166+0000","last_unstale":"2024-04-24T23:27:25.282289+0000","last_undegraded":"2024-04-24T23:27:25.282289+0000","last_fullsized":"2024-04-24T23:27:25.282289+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:47.385604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871305+0000","last_change":"2024-04-24T23:27:22.949783+0000","last_active":"2024-04-24T23:27:25.871305+0000","last_peered":"2024-04-24T23:27:25.871305+0000","last_clean":"2024-04-24T23:27:25.871305+0000","last_became_active":"2024-04-24T23:27:22.949460+0000","last_became_peered":"2024-04-24T23:27:22.949460+0000","last_unstale":"2024-04-24T23:27:25.871305+0000","last_undegraded":"2024-04-24T23:27:25.871305+0000","last_fullsized":"2024-04-24T23:27:25.871305+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:47.345203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.942778+0000","last_change":"2024-04-24T23:27:23.945180+0000","last_active":"2024-04-24T23:27:24.942778+0000","last_peered":"2024-04-24T23:27:24.942778+0000","last_clean":"2024-04-24T23:27:24.942778+0000","last_became_active":"2024-04-24T23:27:23.944877+0000","last_became_peered":"2024-04-24T23:27:23.944877+0000","last_unstale":"2024-04-24T23:27:24.942778+0000","last_undegraded":"2024-04-24T23:27:24.942778+0000","last_fullsized":"2024-04-24T23:27:24.942778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:42.061586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480237+0000","last_change":"2024-04-24T23:27:22.939699+0000","last_active":"2024-04-24T23:27:25.480237+0000","last_peered":"2024-04-24T23:27:25.480237+0000","last_clean":"2024-04-24T23:27:25.480237+0000","last_became_active":"2024-04-24T23:27:22.939387+0000","last_became_peered":"2024-04-24T23:27:22.939387+0000","last_unstale":"2024-04-24T23:27:25.480237+0000","last_undegraded":"2024-04-24T23:27:25.480237+0000","last_fullsized":"2024-04-24T23:27:25.480237+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:34.272382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372138+0000","last_change":"2024-04-24T23:27:23.952219+0000","last_active":"2024-04-24T23:27:26.372138+0000","last_peered":"2024-04-24T23:27:26.372138+0000","last_clean":"2024-04-24T23:27:26.372138+0000","last_became_active":"2024-04-24T23:27:23.951937+0000","last_became_peered":"2024-04-24T23:27:23.951937+0000","last_unstale":"2024-04-24T23:27:26.372138+0000","last_undegraded":"2024-04-24T23:27:26.372138+0000","last_fullsized":"2024-04-24T23:27:26.372138+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.832665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.167967+0000","last_change":"2024-04-24T23:27:22.939303+0000","last_active":"2024-04-24T23:27:26.167967+0000","last_peered":"2024-04-24T23:27:26.167967+0000","last_clean":"2024-04-24T23:27:26.167967+0000","last_became_active":"2024-04-24T23:27:22.938906+0000","last_became_peered":"2024-04-24T23:27:22.938906+0000","last_unstale":"2024-04-24T23:27:26.167967+0000","last_undegraded":"2024-04-24T23:27:26.167967+0000","last_fullsized":"2024-04-24T23:27:26.167967+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:59.986254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.872025+0000","last_change":"2024-04-24T23:27:23.949157+0000","last_active":"2024-04-24T23:27:25.872025+0000","last_peered":"2024-04-24T23:27:25.872025+0000","last_clean":"2024-04-24T23:27:25.872025+0000","last_became_active":"2024-04-24T23:27:23.948868+0000","last_became_peered":"2024-04-24T23:27:23.948868+0000","last_unstale":"2024-04-24T23:27:25.872025+0000","last_undegraded":"2024-04-24T23:27:25.872025+0000","last_fullsized":"2024-04-24T23:27:25.872025+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:04.711226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168377+0000","last_change":"2024-04-24T23:27:22.933997+0000","last_active":"2024-04-24T23:27:26.168377+0000","last_peered":"2024-04-24T23:27:26.168377+0000","last_clean":"2024-04-24T23:27:26.168377+0000","last_became_active":"2024-04-24T23:27:22.929001+0000","last_became_peered":"2024-04-24T23:27:22.929001+0000","last_unstale":"2024-04-24T23:27:26.168377+0000","last_undegraded":"2024-04-24T23:27:26.168377+0000","last_fullsized":"2024-04-24T23:27:26.168377+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:33.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372621+0000","last_change":"2024-04-24T23:27:23.952088+0000","last_active":"2024-04-24T23:27:26.372621+0000","last_peered":"2024-04-24T23:27:26.372621+0000","last_clean":"2024-04-24T23:27:26.372621+0000","last_became_active":"2024-04-24T23:27:23.951816+0000","last_became_peered":"2024-04-24T23:27:23.951816+0000","last_unstale":"2024-04-24T23:27:26.372621+0000","last_undegraded":"2024-04-24T23:27:26.372621+0000","last_fullsized":"2024-04-24T23:27:26.372621+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:06.686817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943494+0000","last_change":"2024-04-24T23:27:22.939391+0000","last_active":"2024-04-24T23:27:24.943494+0000","last_peered":"2024-04-24T23:27:24.943494+0000","last_clean":"2024-04-24T23:27:24.943494+0000","last_became_active":"2024-04-24T23:27:22.939086+0000","last_became_peered":"2024-04-24T23:27:22.939086+0000","last_unstale":"2024-04-24T23:27:24.943494+0000","last_undegraded":"2024-04-24T23:27:24.943494+0000","last_fullsized":"2024-04-24T23:27:24.943494+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:47.345203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480383+0000","last_change":"2024-04-24T23:27:23.950557+0000","last_active":"2024-04-24T23:27:25.480383+0000","last_peered":"2024-04-24T23:27:25.480383+0000","last_clean":"2024-04-24T23:27:25.480383+0000","last_became_active":"2024-04-24T23:27:23.950284+0000","last_became_peered":"2024-04-24T23:27:23.950284+0000","last_unstale":"2024-04-24T23:27:25.480383+0000","last_undegraded":"2024-04-24T23:27:25.480383+0000","last_fullsized":"2024-04-24T23:27:25.480383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:28.282192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282650+0000","last_change":"2024-04-24T23:27:22.937754+0000","last_active":"2024-04-24T23:27:25.282650+0000","last_peered":"2024-04-24T23:27:25.282650+0000","last_clean":"2024-04-24T23:27:25.282650+0000","last_became_active":"2024-04-24T23:27:22.937437+0000","last_became_peered":"2024-04-24T23:27:22.937437+0000","last_unstale":"2024-04-24T23:27:25.282650+0000","last_undegraded":"2024-04-24T23:27:25.282650+0000","last_fullsized":"2024-04-24T23:27:25.282650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:25.573128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282382+0000","last_change":"2024-04-24T23:27:23.945284+0000","last_active":"2024-04-24T23:27:25.282382+0000","last_peered":"2024-04-24T23:27:25.282382+0000","last_clean":"2024-04-24T23:27:25.282382+0000","last_became_active":"2024-04-24T23:27:23.944960+0000","last_became_peered":"2024-04-24T23:27:23.944960+0000","last_unstale":"2024-04-24T23:27:25.282382+0000","last_undegraded":"2024-04-24T23:27:25.282382+0000","last_fullsized":"2024-04-24T23:27:25.282382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:59.743275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372489+0000","last_change":"2024-04-24T23:27:22.938175+0000","last_active":"2024-04-24T23:27:26.372489+0000","last_peered":"2024-04-24T23:27:26.372489+0000","last_clean":"2024-04-24T23:27:26.372489+0000","last_became_active":"2024-04-24T23:27:22.937843+0000","last_became_peered":"2024-04-24T23:27:22.937843+0000","last_unstale":"2024-04-24T23:27:26.372489+0000","last_undegraded":"2024-04-24T23:27:26.372489+0000","last_fullsized":"2024-04-24T23:27:26.372489+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:36.458168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943016+0000","last_change":"2024-04-24T23:27:23.943914+0000","last_active":"2024-04-24T23:27:24.943016+0000","last_peered":"2024-04-24T23:27:24.943016+0000","last_clean":"2024-04-24T23:27:24.943016+0000","last_became_active":"2024-04-24T23:27:23.943608+0000","last_became_peered":"2024-04-24T23:27:23.943608+0000","last_unstale":"2024-04-24T23:27:24.943016+0000","last_undegraded":"2024-04-24T23:27:24.943016+0000","last_fullsized":"2024-04-24T23:27:24.943016+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:49.313119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107842+0000","last_change":"2024-04-24T23:27:22.936410+0000","last_active":"2024-04-24T23:27:25.107842+0000","last_peered":"2024-04-24T23:27:25.107842+0000","last_clean":"2024-04-24T23:27:25.107842+0000","last_became_active":"2024-04-24T23:27:22.936121+0000","last_became_peered":"2024-04-24T23:27:22.936121+0000","last_unstale":"2024-04-24T23:27:25.107842+0000","last_undegraded":"2024-04-24T23:27:25.107842+0000","last_fullsized":"2024-04-24T23:27:25.107842+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:50.529750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168209+0000","last_change":"2024-04-24T23:27:23.957411+0000","last_active":"2024-04-24T23:27:26.168209+0000","last_peered":"2024-04-24T23:27:26.168209+0000","last_clean":"2024-04-24T23:27:26.168209+0000","last_became_active":"2024-04-24T23:27:23.957135+0000","last_became_peered":"2024-04-24T23:27:23.957135+0000","last_unstale":"2024-04-24T23:27:26.168209+0000","last_undegraded":"2024-04-24T23:27:26.168209+0000","last_fullsized":"2024-04-24T23:27:26.168209+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:40.309184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107707+0000","last_change":"2024-04-24T23:27:22.938749+0000","last_active":"2024-04-24T23:27:25.107707+0000","last_peered":"2024-04-24T23:27:25.107707+0000","last_clean":"2024-04-24T23:27:25.107707+0000","last_became_active":"2024-04-24T23:27:22.938464+0000","last_became_peered":"2024-04-24T23:27:22.938464+0000","last_unstale":"2024-04-24T23:27:25.107707+0000","last_undegraded":"2024-04-24T23:27:25.107707+0000","last_fullsized":"2024-04-24T23:27:25.107707+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:13.142519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.371978+0000","last_change":"2024-04-24T23:27:23.950613+0000","last_active":"2024-04-24T23:27:26.371978+0000","last_peered":"2024-04-24T23:27:26.371978+0000","last_clean":"2024-04-24T23:27:26.371978+0000","last_became_active":"2024-04-24T23:27:23.950351+0000","last_became_peered":"2024-04-24T23:27:23.950351+0000","last_unstale":"2024-04-24T23:27:26.371978+0000","last_undegraded":"2024-04-24T23:27:26.371978+0000","last_fullsized":"2024-04-24T23:27:26.371978+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:25.949155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372035+0000","last_change":"2024-04-24T23:27:22.940921+0000","last_active":"2024-04-24T23:27:26.372035+0000","last_peered":"2024-04-24T23:27:26.372035+0000","last_clean":"2024-04-24T23:27:26.372035+0000","last_became_active":"2024-04-24T23:27:22.940525+0000","last_became_peered":"2024-04-24T23:27:22.940525+0000","last_unstale":"2024-04-24T23:27:26.372035+0000","last_undegraded":"2024-04-24T23:27:26.372035+0000","last_fullsized":"2024-04-24T23:27:26.372035+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:58:57.449793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107447+0000","last_change":"2024-04-24T23:27:23.954529+0000","last_active":"2024-04-24T23:27:25.107447+0000","last_peered":"2024-04-24T23:27:25.107447+0000","last_clean":"2024-04-24T23:27:25.107447+0000","last_became_active":"2024-04-24T23:27:23.954272+0000","last_became_peered":"2024-04-24T23:27:23.954272+0000","last_unstale":"2024-04-24T23:27:25.107447+0000","last_undegraded":"2024-04-24T23:27:25.107447+0000","last_fullsized":"2024-04-24T23:27:25.107447+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:13.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168047+0000","last_change":"2024-04-24T23:27:22.940085+0000","last_active":"2024-04-24T23:27:26.168047+0000","last_peered":"2024-04-24T23:27:26.168047+0000","last_clean":"2024-04-24T23:27:26.168047+0000","last_became_active":"2024-04-24T23:27:22.939801+0000","last_became_peered":"2024-04-24T23:27:22.939801+0000","last_unstale":"2024-04-24T23:27:26.168047+0000","last_undegraded":"2024-04-24T23:27:26.168047+0000","last_fullsized":"2024-04-24T23:27:26.168047+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:45.557016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480755+0000","last_change":"2024-04-24T23:27:23.958289+0000","last_active":"2024-04-24T23:27:25.480755+0000","last_peered":"2024-04-24T23:27:25.480755+0000","last_clean":"2024-04-24T23:27:25.480755+0000","last_became_active":"2024-04-24T23:27:23.957984+0000","last_became_peered":"2024-04-24T23:27:23.957984+0000","last_unstale":"2024-04-24T23:27:25.480755+0000","last_undegraded":"2024-04-24T23:27:25.480755+0000","last_fullsized":"2024-04-24T23:27:25.480755+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:10.066682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372381+0000","last_change":"2024-04-24T23:27:22.937282+0000","last_active":"2024-04-24T23:27:26.372381+0000","last_peered":"2024-04-24T23:27:26.372381+0000","last_clean":"2024-04-24T23:27:26.372381+0000","last_became_active":"2024-04-24T23:27:22.936979+0000","last_became_peered":"2024-04-24T23:27:22.936979+0000","last_unstale":"2024-04-24T23:27:26.372381+0000","last_undegraded":"2024-04-24T23:27:26.372381+0000","last_fullsized":"2024-04-24T23:27:26.372381+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:11.873909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943923+0000","last_change":"2024-04-24T23:27:23.947260+0000","last_active":"2024-04-24T23:27:24.943923+0000","last_peered":"2024-04-24T23:27:24.943923+0000","last_clean":"2024-04-24T23:27:24.943923+0000","last_became_active":"2024-04-24T23:27:23.946898+0000","last_became_peered":"2024-04-24T23:27:23.946898+0000","last_unstale":"2024-04-24T23:27:24.943923+0000","last_undegraded":"2024-04-24T23:27:24.943923+0000","last_fullsized":"2024-04-24T23:27:24.943923+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:21.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480468+0000","last_change":"2024-04-24T23:27:22.937169+0000","last_active":"2024-04-24T23:27:25.480468+0000","last_peered":"2024-04-24T23:27:25.480468+0000","last_clean":"2024-04-24T23:27:25.480468+0000","last_became_active":"2024-04-24T23:27:22.936890+0000","last_became_peered":"2024-04-24T23:27:22.936890+0000","last_unstale":"2024-04-24T23:27:25.480468+0000","last_undegraded":"2024-04-24T23:27:25.480468+0000","last_fullsized":"2024-04-24T23:27:25.480468+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:58:57.449793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282591+0000","last_change":"2024-04-24T23:27:23.951783+0000","last_active":"2024-04-24T23:27:25.282591+0000","last_peered":"2024-04-24T23:27:25.282591+0000","last_clean":"2024-04-24T23:27:25.282591+0000","last_became_active":"2024-04-24T23:27:23.951300+0000","last_became_peered":"2024-04-24T23:27:23.951300+0000","last_unstale":"2024-04-24T23:27:25.282591+0000","last_undegraded":"2024-04-24T23:27:25.282591+0000","last_fullsized":"2024-04-24T23:27:25.282591+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:31.436625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871705+0000","last_change":"2024-04-24T23:27:22.944777+0000","last_active":"2024-04-24T23:27:25.871705+0000","last_peered":"2024-04-24T23:27:25.871705+0000","last_clean":"2024-04-24T23:27:25.871705+0000","last_became_active":"2024-04-24T23:27:22.944496+0000","last_became_peered":"2024-04-24T23:27:22.944496+0000","last_unstale":"2024-04-24T23:27:25.871705+0000","last_undegraded":"2024-04-24T23:27:25.871705+0000","last_fullsized":"2024-04-24T23:27:25.871705+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:24.944979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107653+0000","last_change":"2024-04-24T23:27:23.946716+0000","last_active":"2024-04-24T23:27:25.107653+0000","last_peered":"2024-04-24T23:27:25.107653+0000","last_clean":"2024-04-24T23:27:25.107653+0000","last_became_active":"2024-04-24T23:27:23.946417+0000","last_became_peered":"2024-04-24T23:27:23.946417+0000","last_unstale":"2024-04-24T23:27:25.107653+0000","last_undegraded":"2024-04-24T23:27:25.107653+0000","last_fullsized":"2024-04-24T23:27:25.107653+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:00.990431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871538+0000","last_change":"2024-04-24T23:27:22.945765+0000","last_active":"2024-04-24T23:27:25.871538+0000","last_peered":"2024-04-24T23:27:25.871538+0000","last_clean":"2024-04-24T23:27:25.871538+0000","last_became_active":"2024-04-24T23:27:22.945478+0000","last_became_peered":"2024-04-24T23:27:22.945478+0000","last_unstale":"2024-04-24T23:27:25.871538+0000","last_undegraded":"2024-04-24T23:27:25.871538+0000","last_fullsized":"2024-04-24T23:27:25.871538+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:27.278015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107785+0000","last_change":"2024-04-24T23:27:23.951034+0000","last_active":"2024-04-24T23:27:25.107785+0000","last_peered":"2024-04-24T23:27:25.107785+0000","last_clean":"2024-04-24T23:27:25.107785+0000","last_became_active":"2024-04-24T23:27:23.950722+0000","last_became_peered":"2024-04-24T23:27:23.950722+0000","last_unstale":"2024-04-24T23:27:25.107785+0000","last_undegraded":"2024-04-24T23:27:25.107785+0000","last_fullsized":"2024-04-24T23:27:25.107785+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:56.015884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943996+0000","last_change":"2024-04-24T23:27:22.926848+0000","last_active":"2024-04-24T23:27:24.943996+0000","last_peered":"2024-04-24T23:27:24.943996+0000","last_clean":"2024-04-24T23:27:24.943996+0000","last_became_active":"2024-04-24T23:27:22.926467+0000","last_became_peered":"2024-04-24T23:27:22.926467+0000","last_unstale":"2024-04-24T23:27:24.943996+0000","last_undegraded":"2024-04-24T23:27:24.943996+0000","last_fullsized":"2024-04-24T23:27:24.943996+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:58:57.449793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107432+0000","last_change":"2024-04-24T23:27:23.947820+0000","last_active":"2024-04-24T23:27:25.107432+0000","last_peered":"2024-04-24T23:27:25.107432+0000","last_clean":"2024-04-24T23:27:25.107432+0000","last_became_active":"2024-04-24T23:27:23.947537+0000","last_became_peered":"2024-04-24T23:27:23.947537+0000","last_unstale":"2024-04-24T23:27:25.107432+0000","last_undegraded":"2024-04-24T23:27:25.107432+0000","last_fullsized":"2024-04-24T23:27:25.107432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:25.248902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943644+0000","last_change":"2024-04-24T23:27:22.933151+0000","last_active":"2024-04-24T23:27:24.943644+0000","last_peered":"2024-04-24T23:27:24.943644+0000","last_clean":"2024-04-24T23:27:24.943644+0000","last_became_active":"2024-04-24T23:27:22.932855+0000","last_became_peered":"2024-04-24T23:27:22.932855+0000","last_unstale":"2024-04-24T23:27:24.943644+0000","last_undegraded":"2024-04-24T23:27:24.943644+0000","last_fullsized":"2024-04-24T23:27:24.943644+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:49.494758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480155+0000","last_change":"2024-04-24T23:27:23.962974+0000","last_active":"2024-04-24T23:27:25.480155+0000","last_peered":"2024-04-24T23:27:25.480155+0000","last_clean":"2024-04-24T23:27:25.480155+0000","last_became_active":"2024-04-24T23:27:23.962610+0000","last_became_peered":"2024-04-24T23:27:23.962610+0000","last_unstale":"2024-04-24T23:27:25.480155+0000","last_undegraded":"2024-04-24T23:27:25.480155+0000","last_fullsized":"2024-04-24T23:27:25.480155+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.832665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871252+0000","last_change":"2024-04-24T23:27:22.947404+0000","last_active":"2024-04-24T23:27:25.871252+0000","last_peered":"2024-04-24T23:27:25.871252+0000","last_clean":"2024-04-24T23:27:25.871252+0000","last_became_active":"2024-04-24T23:27:22.947148+0000","last_became_peered":"2024-04-24T23:27:22.947148+0000","last_unstale":"2024-04-24T23:27:25.871252+0000","last_undegraded":"2024-04-24T23:27:25.871252+0000","last_fullsized":"2024-04-24T23:27:25.871252+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:27.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.872048+0000","last_change":"2024-04-24T23:27:23.951304+0000","last_active":"2024-04-24T23:27:25.872048+0000","last_peered":"2024-04-24T23:27:25.872048+0000","last_clean":"2024-04-24T23:27:25.872048+0000","last_became_active":"2024-04-24T23:27:23.951017+0000","last_became_peered":"2024-04-24T23:27:23.951017+0000","last_unstale":"2024-04-24T23:27:25.872048+0000","last_undegraded":"2024-04-24T23:27:25.872048+0000","last_fullsized":"2024-04-24T23:27:25.872048+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:46.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.944120+0000","last_change":"2024-04-24T23:27:22.935055+0000","last_active":"2024-04-24T23:27:24.944120+0000","last_peered":"2024-04-24T23:27:24.944120+0000","last_clean":"2024-04-24T23:27:24.944120+0000","last_became_active":"2024-04-24T23:27:22.934759+0000","last_became_peered":"2024-04-24T23:27:22.934759+0000","last_unstale":"2024-04-24T23:27:24.944120+0000","last_undegraded":"2024-04-24T23:27:24.944120+0000","last_fullsized":"2024-04-24T23:27:24.944120+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:34.272382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168365+0000","last_change":"2024-04-24T23:27:23.948712+0000","last_active":"2024-04-24T23:27:26.168365+0000","last_peered":"2024-04-24T23:27:26.168365+0000","last_clean":"2024-04-24T23:27:26.168365+0000","last_became_active":"2024-04-24T23:27:23.948430+0000","last_became_peered":"2024-04-24T23:27:23.948430+0000","last_unstale":"2024-04-24T23:27:26.168365+0000","last_undegraded":"2024-04-24T23:27:26.168365+0000","last_fullsized":"2024-04-24T23:27:26.168365+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:50.498934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.872094+0000","last_change":"2024-04-24T23:27:22.945229+0000","last_active":"2024-04-24T23:27:25.872094+0000","last_peered":"2024-04-24T23:27:25.872094+0000","last_clean":"2024-04-24T23:27:25.872094+0000","last_became_active":"2024-04-24T23:27:22.944929+0000","last_became_peered":"2024-04-24T23:27:22.944929+0000","last_unstale":"2024-04-24T23:27:25.872094+0000","last_undegraded":"2024-04-24T23:27:25.872094+0000","last_fullsized":"2024-04-24T23:27:25.872094+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:00.925486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168490+0000","last_change":"2024-04-24T23:27:23.958334+0000","last_active":"2024-04-24T23:27:26.168490+0000","last_peered":"2024-04-24T23:27:26.168490+0000","last_clean":"2024-04-24T23:27:26.168490+0000","last_became_active":"2024-04-24T23:27:23.958032+0000","last_became_peered":"2024-04-24T23:27:23.958032+0000","last_unstale":"2024-04-24T23:27:26.168490+0000","last_undegraded":"2024-04-24T23:27:26.168490+0000","last_fullsized":"2024-04-24T23:27:26.168490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:35.276558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871735+0000","last_change":"2024-04-24T23:27:22.944225+0000","last_active":"2024-04-24T23:27:25.871735+0000","last_peered":"2024-04-24T23:27:25.871735+0000","last_clean":"2024-04-24T23:27:25.871735+0000","last_became_active":"2024-04-24T23:27:22.943272+0000","last_became_peered":"2024-04-24T23:27:22.943272+0000","last_unstale":"2024-04-24T23:27:25.871735+0000","last_undegraded":"2024-04-24T23:27:25.871735+0000","last_fullsized":"2024-04-24T23:27:25.871735+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:23:04.186183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480376+0000","last_change":"2024-04-24T23:27:23.947780+0000","last_active":"2024-04-24T23:27:25.480376+0000","last_peered":"2024-04-24T23:27:25.480376+0000","last_clean":"2024-04-24T23:27:25.480376+0000","last_became_active":"2024-04-24T23:27:23.947487+0000","last_became_peered":"2024-04-24T23:27:23.947487+0000","last_unstale":"2024-04-24T23:27:25.480376+0000","last_undegraded":"2024-04-24T23:27:25.480376+0000","last_fullsized":"2024-04-24T23:27:25.480376+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:01.929662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480447+0000","last_change":"2024-04-24T23:27:22.937169+0000","last_active":"2024-04-24T23:27:25.480447+0000","last_peered":"2024-04-24T23:27:25.480447+0000","last_clean":"2024-04-24T23:27:25.480447+0000","last_became_active":"2024-04-24T23:27:22.936615+0000","last_became_peered":"2024-04-24T23:27:22.936615+0000","last_unstale":"2024-04-24T23:27:25.480447+0000","last_undegraded":"2024-04-24T23:27:25.480447+0000","last_fullsized":"2024-04-24T23:27:25.480447+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:36.458168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107651+0000","last_change":"2024-04-24T23:27:23.950344+0000","last_active":"2024-04-24T23:27:25.107651+0000","last_peered":"2024-04-24T23:27:25.107651+0000","last_clean":"2024-04-24T23:27:25.107651+0000","last_became_active":"2024-04-24T23:27:23.949938+0000","last_became_peered":"2024-04-24T23:27:23.949938+0000","last_unstale":"2024-04-24T23:27:25.107651+0000","last_undegraded":"2024-04-24T23:27:25.107651+0000","last_fullsized":"2024-04-24T23:27:25.107651+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:46.561192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871625+0000","last_change":"2024-04-24T23:27:22.941877+0000","last_active":"2024-04-24T23:27:25.871625+0000","last_peered":"2024-04-24T23:27:25.871625+0000","last_clean":"2024-04-24T23:27:25.871625+0000","last_became_active":"2024-04-24T23:27:22.941539+0000","last_became_peered":"2024-04-24T23:27:22.941539+0000","last_unstale":"2024-04-24T23:27:25.871625+0000","last_undegraded":"2024-04-24T23:27:25.871625+0000","last_fullsized":"2024-04-24T23:27:25.871625+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:05.682641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372245+0000","last_change":"2024-04-24T23:27:23.948553+0000","last_active":"2024-04-24T23:27:26.372245+0000","last_peered":"2024-04-24T23:27:26.372245+0000","last_clean":"2024-04-24T23:27:26.372245+0000","last_became_active":"2024-04-24T23:27:23.947813+0000","last_became_peered":"2024-04-24T23:27:23.947813+0000","last_unstale":"2024-04-24T23:27:26.372245+0000","last_undegraded":"2024-04-24T23:27:26.372245+0000","last_fullsized":"2024-04-24T23:27:26.372245+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:48.349380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480013+0000","last_change":"2024-04-24T23:27:22.949767+0000","last_active":"2024-04-24T23:27:25.480013+0000","last_peered":"2024-04-24T23:27:25.480013+0000","last_clean":"2024-04-24T23:27:25.480013+0000","last_became_active":"2024-04-24T23:27:22.949531+0000","last_became_peered":"2024-04-24T23:27:22.949531+0000","last_unstale":"2024-04-24T23:27:25.480013+0000","last_undegraded":"2024-04-24T23:27:25.480013+0000","last_fullsized":"2024-04-24T23:27:25.480013+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:49.494758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107488+0000","last_change":"2024-04-24T23:27:23.953544+0000","last_active":"2024-04-24T23:27:25.107488+0000","last_peered":"2024-04-24T23:27:25.107488+0000","last_clean":"2024-04-24T23:27:25.107488+0000","last_became_active":"2024-04-24T23:27:23.953258+0000","last_became_peered":"2024-04-24T23:27:23.953258+0000","last_unstale":"2024-04-24T23:27:25.107488+0000","last_undegraded":"2024-04-24T23:27:25.107488+0000","last_fullsized":"2024-04-24T23:27:25.107488+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:38.430795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.942788+0000","last_change":"2024-04-24T23:27:22.947591+0000","last_active":"2024-04-24T23:27:24.942788+0000","last_peered":"2024-04-24T23:27:24.942788+0000","last_clean":"2024-04-24T23:27:24.942788+0000","last_became_active":"2024-04-24T23:27:22.947296+0000","last_became_peered":"2024-04-24T23:27:22.947296+0000","last_unstale":"2024-04-24T23:27:24.942788+0000","last_undegraded":"2024-04-24T23:27:24.942788+0000","last_fullsized":"2024-04-24T23:27:24.942788+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:27.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480206+0000","last_change":"2024-04-24T23:27:23.950390+0000","last_active":"2024-04-24T23:27:25.480206+0000","last_peered":"2024-04-24T23:27:25.480206+0000","last_clean":"2024-04-24T23:27:25.480206+0000","last_became_active":"2024-04-24T23:27:23.950094+0000","last_became_peered":"2024-04-24T23:27:23.950094+0000","last_unstale":"2024-04-24T23:27:25.480206+0000","last_undegraded":"2024-04-24T23:27:25.480206+0000","last_fullsized":"2024-04-24T23:27:25.480206+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:48.349380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107491+0000","last_change":"2024-04-24T23:27:22.943541+0000","last_active":"2024-04-24T23:27:25.107491+0000","last_peered":"2024-04-24T23:27:25.107491+0000","last_clean":"2024-04-24T23:27:25.107491+0000","last_became_active":"2024-04-24T23:27:22.942994+0000","last_became_peered":"2024-04-24T23:27:22.942994+0000","last_unstale":"2024-04-24T23:27:25.107491+0000","last_undegraded":"2024-04-24T23:27:25.107491+0000","last_fullsized":"2024-04-24T23:27:25.107491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:14.204966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107873+0000","last_change":"2024-04-24T23:27:23.951430+0000","last_active":"2024-04-24T23:27:25.107873+0000","last_peered":"2024-04-24T23:27:25.107873+0000","last_clean":"2024-04-24T23:27:25.107873+0000","last_became_active":"2024-04-24T23:27:23.951173+0000","last_became_peered":"2024-04-24T23:27:23.951173+0000","last_unstale":"2024-04-24T23:27:25.107873+0000","last_undegraded":"2024-04-24T23:27:25.107873+0000","last_fullsized":"2024-04-24T23:27:25.107873+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:08.139056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282432+0000","last_change":"2024-04-24T23:27:22.942215+0000","last_active":"2024-04-24T23:27:25.282432+0000","last_peered":"2024-04-24T23:27:25.282432+0000","last_clean":"2024-04-24T23:27:25.282432+0000","last_became_active":"2024-04-24T23:27:22.941971+0000","last_became_peered":"2024-04-24T23:27:22.941971+0000","last_unstale":"2024-04-24T23:27:25.282432+0000","last_undegraded":"2024-04-24T23:27:25.282432+0000","last_fullsized":"2024-04-24T23:27:25.282432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:47.345203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282404+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:27:25.282404+0000","last_peered":"2024-04-24T23:27:25.282404+0000","last_clean":"2024-04-24T23:27:25.282404+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:27:25.282404+0000","last_undegraded":"2024-04-24T23:27:25.282404+0000","last_fullsized":"2024-04-24T23:27:25.282404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37092,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334748,"statfs":{"total":96636764160,"available":96598781952,"internally_reserved":0,"allocated":1789952,"data_stored":1106336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":516508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":515926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1208320,"data_stored":517592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":517490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36512,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335328,"statfs":{"total":96636764160,"available":96599375872,"internally_reserved":0,"allocated":1191936,"data_stored":515890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36504,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335336,"statfs":{"total":96636764160,"available":96599384064,"internally_reserved":0,"allocated":1187840,"data_stored":515856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37084,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334756,"statfs":{"total":96636764160,"available":96598790144,"internally_reserved":0,"allocated":1785856,"data_stored":1106806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:27:27.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:27:28.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.082+0000 7f9e9d55a640 1 Processor -- start 2024-04-24T23:27:28.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.082+0000 7f9e9d55a640 1 -- start start 2024-04-24T23:27:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.083+0000 7f9e9d55a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e9806b830 0x7f9e98073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.083+0000 7f9e9d55a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e9815c7c0 con 0x7f9e98145980 2024-04-24T23:27:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.083+0000 7f9e9d55a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e9815caa0 con 0x7f9e9815d7e0 2024-04-24T23:27:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.083+0000 7f9e9d55a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9e98145d60 con 0x7f9e9806b830 2024-04-24T23:27:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.083+0000 7f9e96ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9e98145980 0x7f9e9806b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58400/0 (socket says 172.21.15.26:58400) 2024-04-24T23:27:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.083+0000 7f9e96ffd640 1 -- 172.21.15.26:0/3216664160 learned_addr learned my addr 172.21.15.26:0/3216664160 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.083+0000 7f9e967fc640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e9806b830 0x7f9e98073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.083+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504628300 0 0) 0x7f9e9815c7c0 con 0x7f9e98145980 2024-04-24T23:27:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.083+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e84003520 con 0x7f9e98145980 2024-04-24T23:27:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 928140738 0 0) 0x7f9e84003520 con 0x7f9e98145980 2024-04-24T23:27:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e9806b830 msgr2=0x7f9e98073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e9806b830 0x7f9e98073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 >> v1:172.21.15.29:6789/0 conn(0x7f9e9815d7e0 legacy=0x7f9e980721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9815cd80 con 0x7f9e98145980 2024-04-24T23:27:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f9e90001670 con 0x7f9e98145980 2024-04-24T23:27:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 >> v1:172.21.15.26:6789/0 conn(0x7f9e98145980 legacy=0x7f9e9806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e9806b830 0x7f9e840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e84004d40 0x7f9e840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98145d60 con 0x7f9e84003f60 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8400baa0 con 0x7f9e84004d40 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8400bca0 con 0x7f9e9806b830 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e967fc640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e96ffd640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e9806b830 0x7f9e840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e977fe640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e84004d40 0x7f9e840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e967fc640 1 -- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e9806b830 msgr2=0x7f9e840039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e967fc640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e9806b830 0x7f9e840039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e967fc640 1 -- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e84004d40 msgr2=0x7f9e840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e967fc640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e84004d40 0x7f9e840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.084+0000 7f9e967fc640 1 -- 172.21.15.26:0/3216664160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e84003520 con 0x7f9e84003f60 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.085+0000 7f9e967fc640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e84004800 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009ac0 tx=0x7f9e84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b40b0eefb90a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.085+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80036070 con 0x7f9e84003f60 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.085+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e80031400 con 0x7f9e84003f60 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.085+0000 7f9e95ffb640 1 -- 172.21.15.26:0/3216664160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80009690 con 0x7f9e84003f60 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.086+0000 7f9e9d55a640 1 -- 172.21.15.26:0/3216664160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 msgr2=0x7f9e84004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.086+0000 7f9e9d55a640 1 --2- 172.21.15.26:0/3216664160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e84004800 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009ac0 tx=0x7f9e84005830 comp rx=0 tx=0).stop 2024-04-24T23:27:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.086+0000 7f9e977fe640 1 -- 172.21.15.26:0/3216664160 reap_dead start 2024-04-24T23:27:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.086+0000 7f9e9d55a640 1 -- 172.21.15.26:0/3216664160 shutdown_connections 2024-04-24T23:27:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.086+0000 7f9e9d55a640 1 -- 172.21.15.26:0/3216664160 >> 172.21.15.26:0/3216664160 conn(0x7f9e9806e2c0 msgr2=0x7f9e9806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.086+0000 7f9e9d55a640 1 -- 172.21.15.26:0/3216664160 shutdown_connections 2024-04-24T23:27:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.086+0000 7f9e9d55a640 1 -- 172.21.15.26:0/3216664160 wait complete. 2024-04-24T23:27:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.087+0000 7f9e9d55a640 1 Processor -- start 2024-04-24T23:27:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.087+0000 7f9e9d55a640 1 -- start start 2024-04-24T23:27:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e9d55a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e98140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e9d55a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e981f2190 0x7f9e980c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e9d55a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e98141590 0x7f9e98144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e9d55a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e980733c0 con 0x7f9e84003f60 2024-04-24T23:27:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e9d55a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98073240 con 0x7f9e98141590 2024-04-24T23:27:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e9d55a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9e98073540 con 0x7f9e981f2190 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e96ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e98140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e96ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e98140e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60634/0 (socket says 172.21.15.26:60634) 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e96ffd640 1 -- 172.21.15.26:0/1451561682 learned_addr learned my addr 172.21.15.26:0/1451561682 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e967fc640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e981f2190 0x7f9e980c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e977fe640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e98141590 0x7f9e98144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e96ffd640 1 -- 172.21.15.26:0/1451561682 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e981f2190 msgr2=0x7f9e980c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e96ffd640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e981f2190 0x7f9e980c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e96ffd640 1 -- 172.21.15.26:0/1451561682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e98141590 msgr2=0x7f9e98144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e96ffd640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e98141590 0x7f9e98144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e96ffd640 1 -- 172.21.15.26:0/1451561682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e981413a0 con 0x7f9e84003f60 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e96ffd640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e98140e60 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f9e90007370 tx=0x7f9e90007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e8f7fe640 1 -- 172.21.15.26:0/1451561682 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000f940 con 0x7f9e84003f60 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e8f7fe640 1 -- 172.21.15.26:0/1451561682 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e90018070 con 0x7f9e84003f60 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.088+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e980ca000 con 0x7f9e84003f60 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.089+0000 7f9e8f7fe640 1 -- 172.21.15.26:0/1451561682 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000fd40 con 0x7f9e84003f60 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.089+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e980c74c0 con 0x7f9e84003f60 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.089+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e980694c0 con 0x7f9e84003f60 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.090+0000 7f9e8f7fe640 1 -- 172.21.15.26:0/1451561682 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e9001d460 con 0x7f9e84003f60 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.090+0000 7f9e8f7fe640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9e7806be30 0x7f9e7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.090+0000 7f9e8f7fe640 1 -- 172.21.15.26:0/1451561682 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f9e9008ade0 con 0x7f9e84003f60 2024-04-24T23:27:28.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.090+0000 7f9e967fc640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9e7806be30 0x7f9e7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.091+0000 7f9e967fc640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9e7806be30 0x7f9e7806e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9e80033000 tx=0x7f9e80008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:28.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.095+0000 7f9e8f7fe640 1 -- 172.21.15.26:0/1451561682 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9e90059820 con 0x7f9e84003f60 2024-04-24T23:27:28.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.273+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e9806b5f0 con 0x7f9e7806be30 2024-04-24T23:27:28.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.279+0000 7f9e8f7fe640 1 -- 172.21.15.26:0/1451561682 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252085 (secure 0 0 0) 0x7f9e9806b5f0 con 0x7f9e7806be30 2024-04-24T23:27:28.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:28.284 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.282+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9e7806be30 msgr2=0x7f9e7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.282+0000 7f9e9d55a640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9e7806be30 0x7f9e7806e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9e80033000 tx=0x7f9e80008030 comp rx=0 tx=0).stop 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.282+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 msgr2=0x7f9e98140e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.282+0000 7f9e9d55a640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e98140e60 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f9e90007370 tx=0x7f9e90007830 comp rx=0 tx=0).stop 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.283+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 shutdown_connections 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.283+0000 7f9e9d55a640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9e7806be30 0x7f9e7806e2f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.283+0000 7f9e9d55a640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e98141590 0x7f9e98144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.283+0000 7f9e9d55a640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9e981f2190 0x7f9e980c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.283+0000 7f9e9d55a640 1 --2- 172.21.15.26:0/1451561682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e84003f60 0x7f9e98140e60 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.283+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 >> 172.21.15.26:0/1451561682 conn(0x7f9e9806e2c0 msgr2=0x7f9e98143880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.283+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 shutdown_connections 2024-04-24T23:27:28.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.283+0000 7f9e9d55a640 1 -- 172.21.15.26:0/1451561682 wait complete. 2024-04-24T23:27:28.295 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-04-24T23:27:27.400886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276616,"kb_used_data":10504,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754698104,"statfs":{"total":773094113280,"available":772810858496,"internally_reserved":0,"allocated":10756096,"data_stored":5312404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.481435"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168325+0000","last_change":"2024-04-24T23:27:23.949351+0000","last_active":"2024-04-24T23:27:26.168325+0000","last_peered":"2024-04-24T23:27:26.168325+0000","last_clean":"2024-04-24T23:27:26.168325+0000","last_became_active":"2024-04-24T23:27:23.949080+0000","last_became_peered":"2024-04-24T23:27:23.949080+0000","last_unstale":"2024-04-24T23:27:26.168325+0000","last_undegraded":"2024-04-24T23:27:26.168325+0000","last_fullsized":"2024-04-24T23:27:26.168325+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:44.770975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871849+0000","last_change":"2024-04-24T23:27:23.955385+0000","last_active":"2024-04-24T23:27:25.871849+0000","last_peered":"2024-04-24T23:27:25.871849+0000","last_clean":"2024-04-24T23:27:25.871849+0000","last_became_active":"2024-04-24T23:27:23.955066+0000","last_became_peered":"2024-04-24T23:27:23.955066+0000","last_unstale":"2024-04-24T23:27:25.871849+0000","last_undegraded":"2024-04-24T23:27:25.871849+0000","last_fullsized":"2024-04-24T23:27:25.871849+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:31.436625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168249+0000","last_change":"2024-04-24T23:27:23.956318+0000","last_active":"2024-04-24T23:27:26.168249+0000","last_peered":"2024-04-24T23:27:26.168249+0000","last_clean":"2024-04-24T23:27:26.168249+0000","last_became_active":"2024-04-24T23:27:23.956043+0000","last_became_peered":"2024-04-24T23:27:23.956043+0000","last_unstale":"2024-04-24T23:27:26.168249+0000","last_undegraded":"2024-04-24T23:27:26.168249+0000","last_fullsized":"2024-04-24T23:27:26.168249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:30.170440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943238+0000","last_change":"2024-04-24T23:27:23.940507+0000","last_active":"2024-04-24T23:27:24.943238+0000","last_peered":"2024-04-24T23:27:24.943238+0000","last_clean":"2024-04-24T23:27:24.943238+0000","last_became_active":"2024-04-24T23:27:23.940199+0000","last_became_peered":"2024-04-24T23:27:23.940199+0000","last_unstale":"2024-04-24T23:27:24.943238+0000","last_undegraded":"2024-04-24T23:27:24.943238+0000","last_fullsized":"2024-04-24T23:27:24.943238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:38.893817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480281+0000","last_change":"2024-04-24T23:27:23.949157+0000","last_active":"2024-04-24T23:27:25.480281+0000","last_peered":"2024-04-24T23:27:25.480281+0000","last_clean":"2024-04-24T23:27:25.480281+0000","last_became_active":"2024-04-24T23:27:23.948854+0000","last_became_peered":"2024-04-24T23:27:23.948854+0000","last_unstale":"2024-04-24T23:27:25.480281+0000","last_undegraded":"2024-04-24T23:27:25.480281+0000","last_fullsized":"2024-04-24T23:27:25.480281+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:26.577304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282337+0000","last_change":"2024-04-24T23:27:23.943103+0000","last_active":"2024-04-24T23:27:25.282337+0000","last_peered":"2024-04-24T23:27:25.282337+0000","last_clean":"2024-04-24T23:27:25.282337+0000","last_became_active":"2024-04-24T23:27:23.942803+0000","last_became_peered":"2024-04-24T23:27:23.942803+0000","last_unstale":"2024-04-24T23:27:25.282337+0000","last_undegraded":"2024-04-24T23:27:25.282337+0000","last_fullsized":"2024-04-24T23:27:25.282337+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:38.893817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282214+0000","last_change":"2024-04-24T23:27:23.943043+0000","last_active":"2024-04-24T23:27:25.282214+0000","last_peered":"2024-04-24T23:27:25.282214+0000","last_clean":"2024-04-24T23:27:25.282214+0000","last_became_active":"2024-04-24T23:27:23.942739+0000","last_became_peered":"2024-04-24T23:27:23.942739+0000","last_unstale":"2024-04-24T23:27:25.282214+0000","last_undegraded":"2024-04-24T23:27:25.282214+0000","last_fullsized":"2024-04-24T23:27:25.282214+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:49.313119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871369+0000","last_change":"2024-04-24T23:27:23.944197+0000","last_active":"2024-04-24T23:27:25.871369+0000","last_peered":"2024-04-24T23:27:25.871369+0000","last_clean":"2024-04-24T23:27:25.871369+0000","last_became_active":"2024-04-24T23:27:23.943689+0000","last_became_peered":"2024-04-24T23:27:23.943689+0000","last_unstale":"2024-04-24T23:27:25.871369+0000","last_undegraded":"2024-04-24T23:27:25.871369+0000","last_fullsized":"2024-04-24T23:27:25.871369+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:38.893817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372622+0000","last_change":"2024-04-24T23:27:23.957759+0000","last_active":"2024-04-24T23:27:26.372622+0000","last_peered":"2024-04-24T23:27:26.372622+0000","last_clean":"2024-04-24T23:27:26.372622+0000","last_became_active":"2024-04-24T23:27:23.956887+0000","last_became_peered":"2024-04-24T23:27:23.956887+0000","last_unstale":"2024-04-24T23:27:26.372622+0000","last_undegraded":"2024-04-24T23:27:26.372622+0000","last_fullsized":"2024-04-24T23:27:26.372622+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:47.149812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480616+0000","last_change":"2024-04-24T23:27:23.963044+0000","last_active":"2024-04-24T23:27:25.480616+0000","last_peered":"2024-04-24T23:27:25.480616+0000","last_clean":"2024-04-24T23:27:25.480616+0000","last_became_active":"2024-04-24T23:27:23.962730+0000","last_became_peered":"2024-04-24T23:27:23.962730+0000","last_unstale":"2024-04-24T23:27:25.480616+0000","last_undegraded":"2024-04-24T23:27:25.480616+0000","last_fullsized":"2024-04-24T23:27:25.480616+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:47.149812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282416+0000","last_change":"2024-04-24T23:27:23.951560+0000","last_active":"2024-04-24T23:27:25.282416+0000","last_peered":"2024-04-24T23:27:25.282416+0000","last_clean":"2024-04-24T23:27:25.282416+0000","last_became_active":"2024-04-24T23:27:23.950959+0000","last_became_peered":"2024-04-24T23:27:23.950959+0000","last_unstale":"2024-04-24T23:27:25.282416+0000","last_undegraded":"2024-04-24T23:27:25.282416+0000","last_fullsized":"2024-04-24T23:27:25.282416+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:14.428899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372396+0000","last_change":"2024-04-24T23:27:23.951454+0000","last_active":"2024-04-24T23:27:26.372396+0000","last_peered":"2024-04-24T23:27:26.372396+0000","last_clean":"2024-04-24T23:27:26.372396+0000","last_became_active":"2024-04-24T23:27:23.951071+0000","last_became_peered":"2024-04-24T23:27:23.951071+0000","last_unstale":"2024-04-24T23:27:26.372396+0000","last_undegraded":"2024-04-24T23:27:26.372396+0000","last_fullsized":"2024-04-24T23:27:26.372396+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:26.577304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943057+0000","last_change":"2024-04-24T23:27:23.949072+0000","last_active":"2024-04-24T23:27:24.943057+0000","last_peered":"2024-04-24T23:27:24.943057+0000","last_clean":"2024-04-24T23:27:24.943057+0000","last_became_active":"2024-04-24T23:27:23.948774+0000","last_became_peered":"2024-04-24T23:27:23.948774+0000","last_unstale":"2024-04-24T23:27:24.943057+0000","last_undegraded":"2024-04-24T23:27:24.943057+0000","last_fullsized":"2024-04-24T23:27:24.943057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:47.385604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107453+0000","last_change":"2024-04-24T23:27:23.956360+0000","last_active":"2024-04-24T23:27:25.107453+0000","last_peered":"2024-04-24T23:27:25.107453+0000","last_clean":"2024-04-24T23:27:25.107453+0000","last_became_active":"2024-04-24T23:27:23.956080+0000","last_became_peered":"2024-04-24T23:27:23.956080+0000","last_unstale":"2024-04-24T23:27:25.107453+0000","last_undegraded":"2024-04-24T23:27:25.107453+0000","last_fullsized":"2024-04-24T23:27:25.107453+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:54.764291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.942821+0000","last_change":"2024-04-24T23:27:23.945132+0000","last_active":"2024-04-24T23:27:24.942821+0000","last_peered":"2024-04-24T23:27:24.942821+0000","last_clean":"2024-04-24T23:27:24.942821+0000","last_became_active":"2024-04-24T23:27:23.944841+0000","last_became_peered":"2024-04-24T23:27:23.944841+0000","last_unstale":"2024-04-24T23:27:24.942821+0000","last_undegraded":"2024-04-24T23:27:24.942821+0000","last_fullsized":"2024-04-24T23:27:24.942821+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:25.533495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282184+0000","last_change":"2024-04-24T23:27:23.945180+0000","last_active":"2024-04-24T23:27:25.282184+0000","last_peered":"2024-04-24T23:27:25.282184+0000","last_clean":"2024-04-24T23:27:25.282184+0000","last_became_active":"2024-04-24T23:27:23.944812+0000","last_became_peered":"2024-04-24T23:27:23.944812+0000","last_unstale":"2024-04-24T23:27:25.282184+0000","last_undegraded":"2024-04-24T23:27:25.282184+0000","last_fullsized":"2024-04-24T23:27:25.282184+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:45.241737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.872009+0000","last_change":"2024-04-24T23:27:23.950146+0000","last_active":"2024-04-24T23:27:25.872009+0000","last_peered":"2024-04-24T23:27:25.872009+0000","last_clean":"2024-04-24T23:27:25.872009+0000","last_became_active":"2024-04-24T23:27:23.949831+0000","last_became_peered":"2024-04-24T23:27:23.949831+0000","last_unstale":"2024-04-24T23:27:25.872009+0000","last_undegraded":"2024-04-24T23:27:25.872009+0000","last_fullsized":"2024-04-24T23:27:25.872009+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:12.868499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943480+0000","last_change":"2024-04-24T23:27:23.948203+0000","last_active":"2024-04-24T23:27:24.943480+0000","last_peered":"2024-04-24T23:27:24.943480+0000","last_clean":"2024-04-24T23:27:24.943480+0000","last_became_active":"2024-04-24T23:27:23.947930+0000","last_became_peered":"2024-04-24T23:27:23.947930+0000","last_unstale":"2024-04-24T23:27:24.943480+0000","last_undegraded":"2024-04-24T23:27:24.943480+0000","last_fullsized":"2024-04-24T23:27:24.943480+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:59.743275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107701+0000","last_change":"2024-04-24T23:27:23.946009+0000","last_active":"2024-04-24T23:27:25.107701+0000","last_peered":"2024-04-24T23:27:25.107701+0000","last_clean":"2024-04-24T23:27:25.107701+0000","last_became_active":"2024-04-24T23:27:23.945665+0000","last_became_peered":"2024-04-24T23:27:23.945665+0000","last_unstale":"2024-04-24T23:27:25.107701+0000","last_undegraded":"2024-04-24T23:27:25.107701+0000","last_fullsized":"2024-04-24T23:27:25.107701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:23.547036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372528+0000","last_change":"2024-04-24T23:27:23.953430+0000","last_active":"2024-04-24T23:27:26.372528+0000","last_peered":"2024-04-24T23:27:26.372528+0000","last_clean":"2024-04-24T23:27:26.372528+0000","last_became_active":"2024-04-24T23:27:23.953111+0000","last_became_peered":"2024-04-24T23:27:23.953111+0000","last_unstale":"2024-04-24T23:27:26.372528+0000","last_undegraded":"2024-04-24T23:27:26.372528+0000","last_fullsized":"2024-04-24T23:27:26.372528+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:01.929662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107785+0000","last_change":"2024-04-24T23:27:23.948550+0000","last_active":"2024-04-24T23:27:25.107785+0000","last_peered":"2024-04-24T23:27:25.107785+0000","last_clean":"2024-04-24T23:27:25.107785+0000","last_became_active":"2024-04-24T23:27:23.948285+0000","last_became_peered":"2024-04-24T23:27:23.948285+0000","last_unstale":"2024-04-24T23:27:25.107785+0000","last_undegraded":"2024-04-24T23:27:25.107785+0000","last_fullsized":"2024-04-24T23:27:25.107785+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:02.851699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871413+0000","last_change":"2024-04-24T23:27:23.955648+0000","last_active":"2024-04-24T23:27:25.871413+0000","last_peered":"2024-04-24T23:27:25.871413+0000","last_clean":"2024-04-24T23:27:25.871413+0000","last_became_active":"2024-04-24T23:27:23.955150+0000","last_became_peered":"2024-04-24T23:27:23.955150+0000","last_unstale":"2024-04-24T23:27:25.871413+0000","last_undegraded":"2024-04-24T23:27:25.871413+0000","last_fullsized":"2024-04-24T23:27:25.871413+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:59.743275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871415+0000","last_change":"2024-04-24T23:27:23.954623+0000","last_active":"2024-04-24T23:27:25.871415+0000","last_peered":"2024-04-24T23:27:25.871415+0000","last_clean":"2024-04-24T23:27:25.871415+0000","last_became_active":"2024-04-24T23:27:23.954341+0000","last_became_peered":"2024-04-24T23:27:23.954341+0000","last_unstale":"2024-04-24T23:27:25.871415+0000","last_undegraded":"2024-04-24T23:27:25.871415+0000","last_fullsized":"2024-04-24T23:27:25.871415+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:16.893167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282308+0000","last_change":"2024-04-24T23:27:23.946051+0000","last_active":"2024-04-24T23:27:25.282308+0000","last_peered":"2024-04-24T23:27:25.282308+0000","last_clean":"2024-04-24T23:27:25.282308+0000","last_became_active":"2024-04-24T23:27:23.945694+0000","last_became_peered":"2024-04-24T23:27:23.945694+0000","last_unstale":"2024-04-24T23:27:25.282308+0000","last_undegraded":"2024-04-24T23:27:25.282308+0000","last_fullsized":"2024-04-24T23:27:25.282308+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:16.893167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480755+0000","last_change":"2024-04-24T23:27:23.951410+0000","last_active":"2024-04-24T23:27:25.480755+0000","last_peered":"2024-04-24T23:27:25.480755+0000","last_clean":"2024-04-24T23:27:25.480755+0000","last_became_active":"2024-04-24T23:27:23.951056+0000","last_became_peered":"2024-04-24T23:27:23.951056+0000","last_unstale":"2024-04-24T23:27:25.480755+0000","last_undegraded":"2024-04-24T23:27:25.480755+0000","last_fullsized":"2024-04-24T23:27:25.480755+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:31.921698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871874+0000","last_change":"2024-04-24T23:27:23.958551+0000","last_active":"2024-04-24T23:27:25.871874+0000","last_peered":"2024-04-24T23:27:25.871874+0000","last_clean":"2024-04-24T23:27:25.871874+0000","last_became_active":"2024-04-24T23:27:23.958241+0000","last_became_peered":"2024-04-24T23:27:23.958241+0000","last_unstale":"2024-04-24T23:27:25.871874+0000","last_undegraded":"2024-04-24T23:27:25.871874+0000","last_fullsized":"2024-04-24T23:27:25.871874+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:49.702454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282608+0000","last_change":"2024-04-24T23:27:23.948563+0000","last_active":"2024-04-24T23:27:25.282608+0000","last_peered":"2024-04-24T23:27:25.282608+0000","last_clean":"2024-04-24T23:27:25.282608+0000","last_became_active":"2024-04-24T23:27:23.948285+0000","last_became_peered":"2024-04-24T23:27:23.948285+0000","last_unstale":"2024-04-24T23:27:25.282608+0000","last_undegraded":"2024-04-24T23:27:25.282608+0000","last_fullsized":"2024-04-24T23:27:25.282608+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:49.702454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480417+0000","last_change":"2024-04-24T23:27:23.949786+0000","last_active":"2024-04-24T23:27:25.480417+0000","last_peered":"2024-04-24T23:27:25.480417+0000","last_clean":"2024-04-24T23:27:25.480417+0000","last_became_active":"2024-04-24T23:27:23.949164+0000","last_became_peered":"2024-04-24T23:27:23.949164+0000","last_unstale":"2024-04-24T23:27:25.480417+0000","last_undegraded":"2024-04-24T23:27:25.480417+0000","last_fullsized":"2024-04-24T23:27:25.480417+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:06.686817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871544+0000","last_change":"2024-04-24T23:27:23.960298+0000","last_active":"2024-04-24T23:27:25.871544+0000","last_peered":"2024-04-24T23:27:25.871544+0000","last_clean":"2024-04-24T23:27:25.871544+0000","last_became_active":"2024-04-24T23:27:23.959922+0000","last_became_peered":"2024-04-24T23:27:23.959922+0000","last_unstale":"2024-04-24T23:27:25.871544+0000","last_undegraded":"2024-04-24T23:27:25.871544+0000","last_fullsized":"2024-04-24T23:27:25.871544+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:14.428899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107456+0000","last_change":"2024-04-24T23:27:23.947590+0000","last_active":"2024-04-24T23:27:25.107456+0000","last_peered":"2024-04-24T23:27:25.107456+0000","last_clean":"2024-04-24T23:27:25.107456+0000","last_became_active":"2024-04-24T23:27:23.947269+0000","last_became_peered":"2024-04-24T23:27:23.947269+0000","last_unstale":"2024-04-24T23:27:25.107456+0000","last_undegraded":"2024-04-24T23:27:25.107456+0000","last_fullsized":"2024-04-24T23:27:25.107456+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:34.673219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.942959+0000","last_change":"2024-04-24T23:27:23.953836+0000","last_active":"2024-04-24T23:27:24.942959+0000","last_peered":"2024-04-24T23:27:24.942959+0000","last_clean":"2024-04-24T23:27:24.942959+0000","last_became_active":"2024-04-24T23:27:23.953525+0000","last_became_peered":"2024-04-24T23:27:23.953525+0000","last_unstale":"2024-04-24T23:27:24.942959+0000","last_undegraded":"2024-04-24T23:27:24.942959+0000","last_fullsized":"2024-04-24T23:27:24.942959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:45.241737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168035+0000","last_change":"2024-04-24T23:27:23.951539+0000","last_active":"2024-04-24T23:27:26.168035+0000","last_peered":"2024-04-24T23:27:26.168035+0000","last_clean":"2024-04-24T23:27:26.168035+0000","last_became_active":"2024-04-24T23:27:23.951225+0000","last_became_peered":"2024-04-24T23:27:23.951225+0000","last_unstale":"2024-04-24T23:27:26.168035+0000","last_undegraded":"2024-04-24T23:27:26.168035+0000","last_fullsized":"2024-04-24T23:27:26.168035+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:37.724643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107830+0000","last_change":"2024-04-24T23:27:23.946409+0000","last_active":"2024-04-24T23:27:25.107830+0000","last_peered":"2024-04-24T23:27:25.107830+0000","last_clean":"2024-04-24T23:27:25.107830+0000","last_became_active":"2024-04-24T23:27:23.945965+0000","last_became_peered":"2024-04-24T23:27:23.945965+0000","last_unstale":"2024-04-24T23:27:25.107830+0000","last_undegraded":"2024-04-24T23:27:25.107830+0000","last_fullsized":"2024-04-24T23:27:25.107830+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:03.028400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871993+0000","last_change":"2024-04-24T23:27:22.942606+0000","last_active":"2024-04-24T23:27:25.871993+0000","last_peered":"2024-04-24T23:27:25.871993+0000","last_clean":"2024-04-24T23:27:25.871993+0000","last_became_active":"2024-04-24T23:27:22.942273+0000","last_became_peered":"2024-04-24T23:27:22.942273+0000","last_unstale":"2024-04-24T23:27:25.871993+0000","last_undegraded":"2024-04-24T23:27:25.871993+0000","last_fullsized":"2024-04-24T23:27:25.871993+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:25.573128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943437+0000","last_change":"2024-04-24T23:27:23.953788+0000","last_active":"2024-04-24T23:27:24.943437+0000","last_peered":"2024-04-24T23:27:24.943437+0000","last_clean":"2024-04-24T23:27:24.943437+0000","last_became_active":"2024-04-24T23:27:23.953484+0000","last_became_peered":"2024-04-24T23:27:23.953484+0000","last_unstale":"2024-04-24T23:27:24.943437+0000","last_undegraded":"2024-04-24T23:27:24.943437+0000","last_fullsized":"2024-04-24T23:27:24.943437+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:16.893167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282464+0000","last_change":"2024-04-24T23:27:22.941320+0000","last_active":"2024-04-24T23:27:25.282464+0000","last_peered":"2024-04-24T23:27:25.282464+0000","last_clean":"2024-04-24T23:27:25.282464+0000","last_became_active":"2024-04-24T23:27:22.940992+0000","last_became_peered":"2024-04-24T23:27:22.940992+0000","last_unstale":"2024-04-24T23:27:25.282464+0000","last_undegraded":"2024-04-24T23:27:25.282464+0000","last_fullsized":"2024-04-24T23:27:25.282464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:27.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168356+0000","last_change":"2024-04-24T23:27:23.956266+0000","last_active":"2024-04-24T23:27:26.168356+0000","last_peered":"2024-04-24T23:27:26.168356+0000","last_clean":"2024-04-24T23:27:26.168356+0000","last_became_active":"2024-04-24T23:27:23.953605+0000","last_became_peered":"2024-04-24T23:27:23.953605+0000","last_unstale":"2024-04-24T23:27:26.168356+0000","last_undegraded":"2024-04-24T23:27:26.168356+0000","last_fullsized":"2024-04-24T23:27:26.168356+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:16.570967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871625+0000","last_change":"2024-04-24T23:27:22.945850+0000","last_active":"2024-04-24T23:27:25.871625+0000","last_peered":"2024-04-24T23:27:25.871625+0000","last_clean":"2024-04-24T23:27:25.871625+0000","last_became_active":"2024-04-24T23:27:22.945577+0000","last_became_peered":"2024-04-24T23:27:22.945577+0000","last_unstale":"2024-04-24T23:27:25.871625+0000","last_undegraded":"2024-04-24T23:27:25.871625+0000","last_fullsized":"2024-04-24T23:27:25.871625+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:09.433090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372554+0000","last_change":"2024-04-24T23:27:23.952642+0000","last_active":"2024-04-24T23:27:26.372554+0000","last_peered":"2024-04-24T23:27:26.372554+0000","last_clean":"2024-04-24T23:27:26.372554+0000","last_became_active":"2024-04-24T23:27:23.952409+0000","last_became_peered":"2024-04-24T23:27:23.952409+0000","last_unstale":"2024-04-24T23:27:26.372554+0000","last_undegraded":"2024-04-24T23:27:26.372554+0000","last_fullsized":"2024-04-24T23:27:26.372554+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:28.282192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871573+0000","last_change":"2024-04-24T23:27:22.944547+0000","last_active":"2024-04-24T23:27:25.871573+0000","last_peered":"2024-04-24T23:27:25.871573+0000","last_clean":"2024-04-24T23:27:25.871573+0000","last_became_active":"2024-04-24T23:27:22.943908+0000","last_became_peered":"2024-04-24T23:27:22.943908+0000","last_unstale":"2024-04-24T23:27:25.871573+0000","last_undegraded":"2024-04-24T23:27:25.871573+0000","last_fullsized":"2024-04-24T23:27:25.871573+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:09.273639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943924+0000","last_change":"2024-04-24T23:27:23.948127+0000","last_active":"2024-04-24T23:27:24.943924+0000","last_peered":"2024-04-24T23:27:24.943924+0000","last_clean":"2024-04-24T23:27:24.943924+0000","last_became_active":"2024-04-24T23:27:23.946659+0000","last_became_peered":"2024-04-24T23:27:23.946659+0000","last_unstale":"2024-04-24T23:27:24.943924+0000","last_undegraded":"2024-04-24T23:27:24.943924+0000","last_fullsized":"2024-04-24T23:27:24.943924+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:56.193488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168129+0000","last_change":"2024-04-24T23:27:22.945122+0000","last_active":"2024-04-24T23:27:26.168129+0000","last_peered":"2024-04-24T23:27:26.168129+0000","last_clean":"2024-04-24T23:27:26.168129+0000","last_became_active":"2024-04-24T23:27:22.944769+0000","last_became_peered":"2024-04-24T23:27:22.944769+0000","last_unstale":"2024-04-24T23:27:26.168129+0000","last_undegraded":"2024-04-24T23:27:26.168129+0000","last_fullsized":"2024-04-24T23:27:26.168129+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:37.426619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282289+0000","last_change":"2024-04-24T23:27:23.954439+0000","last_active":"2024-04-24T23:27:25.282289+0000","last_peered":"2024-04-24T23:27:25.282289+0000","last_clean":"2024-04-24T23:27:25.282289+0000","last_became_active":"2024-04-24T23:27:23.954166+0000","last_became_peered":"2024-04-24T23:27:23.954166+0000","last_unstale":"2024-04-24T23:27:25.282289+0000","last_undegraded":"2024-04-24T23:27:25.282289+0000","last_fullsized":"2024-04-24T23:27:25.282289+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:47.385604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871305+0000","last_change":"2024-04-24T23:27:22.949783+0000","last_active":"2024-04-24T23:27:25.871305+0000","last_peered":"2024-04-24T23:27:25.871305+0000","last_clean":"2024-04-24T23:27:25.871305+0000","last_became_active":"2024-04-24T23:27:22.949460+0000","last_became_peered":"2024-04-24T23:27:22.949460+0000","last_unstale":"2024-04-24T23:27:25.871305+0000","last_undegraded":"2024-04-24T23:27:25.871305+0000","last_fullsized":"2024-04-24T23:27:25.871305+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:47.345203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.942778+0000","last_change":"2024-04-24T23:27:23.945180+0000","last_active":"2024-04-24T23:27:24.942778+0000","last_peered":"2024-04-24T23:27:24.942778+0000","last_clean":"2024-04-24T23:27:24.942778+0000","last_became_active":"2024-04-24T23:27:23.944877+0000","last_became_peered":"2024-04-24T23:27:23.944877+0000","last_unstale":"2024-04-24T23:27:24.942778+0000","last_undegraded":"2024-04-24T23:27:24.942778+0000","last_fullsized":"2024-04-24T23:27:24.942778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:42.061586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480237+0000","last_change":"2024-04-24T23:27:22.939699+0000","last_active":"2024-04-24T23:27:25.480237+0000","last_peered":"2024-04-24T23:27:25.480237+0000","last_clean":"2024-04-24T23:27:25.480237+0000","last_became_active":"2024-04-24T23:27:22.939387+0000","last_became_peered":"2024-04-24T23:27:22.939387+0000","last_unstale":"2024-04-24T23:27:25.480237+0000","last_undegraded":"2024-04-24T23:27:25.480237+0000","last_fullsized":"2024-04-24T23:27:25.480237+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:34.272382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372138+0000","last_change":"2024-04-24T23:27:23.952219+0000","last_active":"2024-04-24T23:27:26.372138+0000","last_peered":"2024-04-24T23:27:26.372138+0000","last_clean":"2024-04-24T23:27:26.372138+0000","last_became_active":"2024-04-24T23:27:23.951937+0000","last_became_peered":"2024-04-24T23:27:23.951937+0000","last_unstale":"2024-04-24T23:27:26.372138+0000","last_undegraded":"2024-04-24T23:27:26.372138+0000","last_fullsized":"2024-04-24T23:27:26.372138+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.832665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.167967+0000","last_change":"2024-04-24T23:27:22.939303+0000","last_active":"2024-04-24T23:27:26.167967+0000","last_peered":"2024-04-24T23:27:26.167967+0000","last_clean":"2024-04-24T23:27:26.167967+0000","last_became_active":"2024-04-24T23:27:22.938906+0000","last_became_peered":"2024-04-24T23:27:22.938906+0000","last_unstale":"2024-04-24T23:27:26.167967+0000","last_undegraded":"2024-04-24T23:27:26.167967+0000","last_fullsized":"2024-04-24T23:27:26.167967+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:59.986254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.872025+0000","last_change":"2024-04-24T23:27:23.949157+0000","last_active":"2024-04-24T23:27:25.872025+0000","last_peered":"2024-04-24T23:27:25.872025+0000","last_clean":"2024-04-24T23:27:25.872025+0000","last_became_active":"2024-04-24T23:27:23.948868+0000","last_became_peered":"2024-04-24T23:27:23.948868+0000","last_unstale":"2024-04-24T23:27:25.872025+0000","last_undegraded":"2024-04-24T23:27:25.872025+0000","last_fullsized":"2024-04-24T23:27:25.872025+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:04.711226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168377+0000","last_change":"2024-04-24T23:27:22.933997+0000","last_active":"2024-04-24T23:27:26.168377+0000","last_peered":"2024-04-24T23:27:26.168377+0000","last_clean":"2024-04-24T23:27:26.168377+0000","last_became_active":"2024-04-24T23:27:22.929001+0000","last_became_peered":"2024-04-24T23:27:22.929001+0000","last_unstale":"2024-04-24T23:27:26.168377+0000","last_undegraded":"2024-04-24T23:27:26.168377+0000","last_fullsized":"2024-04-24T23:27:26.168377+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:33.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372621+0000","last_change":"2024-04-24T23:27:23.952088+0000","last_active":"2024-04-24T23:27:26.372621+0000","last_peered":"2024-04-24T23:27:26.372621+0000","last_clean":"2024-04-24T23:27:26.372621+0000","last_became_active":"2024-04-24T23:27:23.951816+0000","last_became_peered":"2024-04-24T23:27:23.951816+0000","last_unstale":"2024-04-24T23:27:26.372621+0000","last_undegraded":"2024-04-24T23:27:26.372621+0000","last_fullsized":"2024-04-24T23:27:26.372621+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:06.686817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943494+0000","last_change":"2024-04-24T23:27:22.939391+0000","last_active":"2024-04-24T23:27:24.943494+0000","last_peered":"2024-04-24T23:27:24.943494+0000","last_clean":"2024-04-24T23:27:24.943494+0000","last_became_active":"2024-04-24T23:27:22.939086+0000","last_became_peered":"2024-04-24T23:27:22.939086+0000","last_unstale":"2024-04-24T23:27:24.943494+0000","last_undegraded":"2024-04-24T23:27:24.943494+0000","last_fullsized":"2024-04-24T23:27:24.943494+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:47.345203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480383+0000","last_change":"2024-04-24T23:27:23.950557+0000","last_active":"2024-04-24T23:27:25.480383+0000","last_peered":"2024-04-24T23:27:25.480383+0000","last_clean":"2024-04-24T23:27:25.480383+0000","last_became_active":"2024-04-24T23:27:23.950284+0000","last_became_peered":"2024-04-24T23:27:23.950284+0000","last_unstale":"2024-04-24T23:27:25.480383+0000","last_undegraded":"2024-04-24T23:27:25.480383+0000","last_fullsized":"2024-04-24T23:27:25.480383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:28.282192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282650+0000","last_change":"2024-04-24T23:27:22.937754+0000","last_active":"2024-04-24T23:27:25.282650+0000","last_peered":"2024-04-24T23:27:25.282650+0000","last_clean":"2024-04-24T23:27:25.282650+0000","last_became_active":"2024-04-24T23:27:22.937437+0000","last_became_peered":"2024-04-24T23:27:22.937437+0000","last_unstale":"2024-04-24T23:27:25.282650+0000","last_undegraded":"2024-04-24T23:27:25.282650+0000","last_fullsized":"2024-04-24T23:27:25.282650+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:25.573128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282382+0000","last_change":"2024-04-24T23:27:23.945284+0000","last_active":"2024-04-24T23:27:25.282382+0000","last_peered":"2024-04-24T23:27:25.282382+0000","last_clean":"2024-04-24T23:27:25.282382+0000","last_became_active":"2024-04-24T23:27:23.944960+0000","last_became_peered":"2024-04-24T23:27:23.944960+0000","last_unstale":"2024-04-24T23:27:25.282382+0000","last_undegraded":"2024-04-24T23:27:25.282382+0000","last_fullsized":"2024-04-24T23:27:25.282382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:59.743275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372489+0000","last_change":"2024-04-24T23:27:22.938175+0000","last_active":"2024-04-24T23:27:26.372489+0000","last_peered":"2024-04-24T23:27:26.372489+0000","last_clean":"2024-04-24T23:27:26.372489+0000","last_became_active":"2024-04-24T23:27:22.937843+0000","last_became_peered":"2024-04-24T23:27:22.937843+0000","last_unstale":"2024-04-24T23:27:26.372489+0000","last_undegraded":"2024-04-24T23:27:26.372489+0000","last_fullsized":"2024-04-24T23:27:26.372489+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:36.458168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943016+0000","last_change":"2024-04-24T23:27:23.943914+0000","last_active":"2024-04-24T23:27:24.943016+0000","last_peered":"2024-04-24T23:27:24.943016+0000","last_clean":"2024-04-24T23:27:24.943016+0000","last_became_active":"2024-04-24T23:27:23.943608+0000","last_became_peered":"2024-04-24T23:27:23.943608+0000","last_unstale":"2024-04-24T23:27:24.943016+0000","last_undegraded":"2024-04-24T23:27:24.943016+0000","last_fullsized":"2024-04-24T23:27:24.943016+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:49.313119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107842+0000","last_change":"2024-04-24T23:27:22.936410+0000","last_active":"2024-04-24T23:27:25.107842+0000","last_peered":"2024-04-24T23:27:25.107842+0000","last_clean":"2024-04-24T23:27:25.107842+0000","last_became_active":"2024-04-24T23:27:22.936121+0000","last_became_peered":"2024-04-24T23:27:22.936121+0000","last_unstale":"2024-04-24T23:27:25.107842+0000","last_undegraded":"2024-04-24T23:27:25.107842+0000","last_fullsized":"2024-04-24T23:27:25.107842+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:50.529750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168209+0000","last_change":"2024-04-24T23:27:23.957411+0000","last_active":"2024-04-24T23:27:26.168209+0000","last_peered":"2024-04-24T23:27:26.168209+0000","last_clean":"2024-04-24T23:27:26.168209+0000","last_became_active":"2024-04-24T23:27:23.957135+0000","last_became_peered":"2024-04-24T23:27:23.957135+0000","last_unstale":"2024-04-24T23:27:26.168209+0000","last_undegraded":"2024-04-24T23:27:26.168209+0000","last_fullsized":"2024-04-24T23:27:26.168209+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:40.309184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107707+0000","last_change":"2024-04-24T23:27:22.938749+0000","last_active":"2024-04-24T23:27:25.107707+0000","last_peered":"2024-04-24T23:27:25.107707+0000","last_clean":"2024-04-24T23:27:25.107707+0000","last_became_active":"2024-04-24T23:27:22.938464+0000","last_became_peered":"2024-04-24T23:27:22.938464+0000","last_unstale":"2024-04-24T23:27:25.107707+0000","last_undegraded":"2024-04-24T23:27:25.107707+0000","last_fullsized":"2024-04-24T23:27:25.107707+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:13.142519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.371978+0000","last_change":"2024-04-24T23:27:23.950613+0000","last_active":"2024-04-24T23:27:26.371978+0000","last_peered":"2024-04-24T23:27:26.371978+0000","last_clean":"2024-04-24T23:27:26.371978+0000","last_became_active":"2024-04-24T23:27:23.950351+0000","last_became_peered":"2024-04-24T23:27:23.950351+0000","last_unstale":"2024-04-24T23:27:26.371978+0000","last_undegraded":"2024-04-24T23:27:26.371978+0000","last_fullsized":"2024-04-24T23:27:26.371978+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:25.949155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372035+0000","last_change":"2024-04-24T23:27:22.940921+0000","last_active":"2024-04-24T23:27:26.372035+0000","last_peered":"2024-04-24T23:27:26.372035+0000","last_clean":"2024-04-24T23:27:26.372035+0000","last_became_active":"2024-04-24T23:27:22.940525+0000","last_became_peered":"2024-04-24T23:27:22.940525+0000","last_unstale":"2024-04-24T23:27:26.372035+0000","last_undegraded":"2024-04-24T23:27:26.372035+0000","last_fullsized":"2024-04-24T23:27:26.372035+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:58:57.449793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107447+0000","last_change":"2024-04-24T23:27:23.954529+0000","last_active":"2024-04-24T23:27:25.107447+0000","last_peered":"2024-04-24T23:27:25.107447+0000","last_clean":"2024-04-24T23:27:25.107447+0000","last_became_active":"2024-04-24T23:27:23.954272+0000","last_became_peered":"2024-04-24T23:27:23.954272+0000","last_unstale":"2024-04-24T23:27:25.107447+0000","last_undegraded":"2024-04-24T23:27:25.107447+0000","last_fullsized":"2024-04-24T23:27:25.107447+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:13.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168047+0000","last_change":"2024-04-24T23:27:22.940085+0000","last_active":"2024-04-24T23:27:26.168047+0000","last_peered":"2024-04-24T23:27:26.168047+0000","last_clean":"2024-04-24T23:27:26.168047+0000","last_became_active":"2024-04-24T23:27:22.939801+0000","last_became_peered":"2024-04-24T23:27:22.939801+0000","last_unstale":"2024-04-24T23:27:26.168047+0000","last_undegraded":"2024-04-24T23:27:26.168047+0000","last_fullsized":"2024-04-24T23:27:26.168047+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:45.557016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480755+0000","last_change":"2024-04-24T23:27:23.958289+0000","last_active":"2024-04-24T23:27:25.480755+0000","last_peered":"2024-04-24T23:27:25.480755+0000","last_clean":"2024-04-24T23:27:25.480755+0000","last_became_active":"2024-04-24T23:27:23.957984+0000","last_became_peered":"2024-04-24T23:27:23.957984+0000","last_unstale":"2024-04-24T23:27:25.480755+0000","last_undegraded":"2024-04-24T23:27:25.480755+0000","last_fullsized":"2024-04-24T23:27:25.480755+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:10.066682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372381+0000","last_change":"2024-04-24T23:27:22.937282+0000","last_active":"2024-04-24T23:27:26.372381+0000","last_peered":"2024-04-24T23:27:26.372381+0000","last_clean":"2024-04-24T23:27:26.372381+0000","last_became_active":"2024-04-24T23:27:22.936979+0000","last_became_peered":"2024-04-24T23:27:22.936979+0000","last_unstale":"2024-04-24T23:27:26.372381+0000","last_undegraded":"2024-04-24T23:27:26.372381+0000","last_fullsized":"2024-04-24T23:27:26.372381+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:11.873909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943923+0000","last_change":"2024-04-24T23:27:23.947260+0000","last_active":"2024-04-24T23:27:24.943923+0000","last_peered":"2024-04-24T23:27:24.943923+0000","last_clean":"2024-04-24T23:27:24.943923+0000","last_became_active":"2024-04-24T23:27:23.946898+0000","last_became_peered":"2024-04-24T23:27:23.946898+0000","last_unstale":"2024-04-24T23:27:24.943923+0000","last_undegraded":"2024-04-24T23:27:24.943923+0000","last_fullsized":"2024-04-24T23:27:24.943923+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:21.358487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480468+0000","last_change":"2024-04-24T23:27:22.937169+0000","last_active":"2024-04-24T23:27:25.480468+0000","last_peered":"2024-04-24T23:27:25.480468+0000","last_clean":"2024-04-24T23:27:25.480468+0000","last_became_active":"2024-04-24T23:27:22.936890+0000","last_became_peered":"2024-04-24T23:27:22.936890+0000","last_unstale":"2024-04-24T23:27:25.480468+0000","last_undegraded":"2024-04-24T23:27:25.480468+0000","last_fullsized":"2024-04-24T23:27:25.480468+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:58:57.449793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282591+0000","last_change":"2024-04-24T23:27:23.951783+0000","last_active":"2024-04-24T23:27:25.282591+0000","last_peered":"2024-04-24T23:27:25.282591+0000","last_clean":"2024-04-24T23:27:25.282591+0000","last_became_active":"2024-04-24T23:27:23.951300+0000","last_became_peered":"2024-04-24T23:27:23.951300+0000","last_unstale":"2024-04-24T23:27:25.282591+0000","last_undegraded":"2024-04-24T23:27:25.282591+0000","last_fullsized":"2024-04-24T23:27:25.282591+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:31.436625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871705+0000","last_change":"2024-04-24T23:27:22.944777+0000","last_active":"2024-04-24T23:27:25.871705+0000","last_peered":"2024-04-24T23:27:25.871705+0000","last_clean":"2024-04-24T23:27:25.871705+0000","last_became_active":"2024-04-24T23:27:22.944496+0000","last_became_peered":"2024-04-24T23:27:22.944496+0000","last_unstale":"2024-04-24T23:27:25.871705+0000","last_undegraded":"2024-04-24T23:27:25.871705+0000","last_fullsized":"2024-04-24T23:27:25.871705+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:24.944979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107653+0000","last_change":"2024-04-24T23:27:23.946716+0000","last_active":"2024-04-24T23:27:25.107653+0000","last_peered":"2024-04-24T23:27:25.107653+0000","last_clean":"2024-04-24T23:27:25.107653+0000","last_became_active":"2024-04-24T23:27:23.946417+0000","last_became_peered":"2024-04-24T23:27:23.946417+0000","last_unstale":"2024-04-24T23:27:25.107653+0000","last_undegraded":"2024-04-24T23:27:25.107653+0000","last_fullsized":"2024-04-24T23:27:25.107653+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:00.990431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871538+0000","last_change":"2024-04-24T23:27:22.945765+0000","last_active":"2024-04-24T23:27:25.871538+0000","last_peered":"2024-04-24T23:27:25.871538+0000","last_clean":"2024-04-24T23:27:25.871538+0000","last_became_active":"2024-04-24T23:27:22.945478+0000","last_became_peered":"2024-04-24T23:27:22.945478+0000","last_unstale":"2024-04-24T23:27:25.871538+0000","last_undegraded":"2024-04-24T23:27:25.871538+0000","last_fullsized":"2024-04-24T23:27:25.871538+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:27.278015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107785+0000","last_change":"2024-04-24T23:27:23.951034+0000","last_active":"2024-04-24T23:27:25.107785+0000","last_peered":"2024-04-24T23:27:25.107785+0000","last_clean":"2024-04-24T23:27:25.107785+0000","last_became_active":"2024-04-24T23:27:23.950722+0000","last_became_peered":"2024-04-24T23:27:23.950722+0000","last_unstale":"2024-04-24T23:27:25.107785+0000","last_undegraded":"2024-04-24T23:27:25.107785+0000","last_fullsized":"2024-04-24T23:27:25.107785+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:56.015884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943996+0000","last_change":"2024-04-24T23:27:22.926848+0000","last_active":"2024-04-24T23:27:24.943996+0000","last_peered":"2024-04-24T23:27:24.943996+0000","last_clean":"2024-04-24T23:27:24.943996+0000","last_became_active":"2024-04-24T23:27:22.926467+0000","last_became_peered":"2024-04-24T23:27:22.926467+0000","last_unstale":"2024-04-24T23:27:24.943996+0000","last_undegraded":"2024-04-24T23:27:24.943996+0000","last_fullsized":"2024-04-24T23:27:24.943996+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:58:57.449793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107432+0000","last_change":"2024-04-24T23:27:23.947820+0000","last_active":"2024-04-24T23:27:25.107432+0000","last_peered":"2024-04-24T23:27:25.107432+0000","last_clean":"2024-04-24T23:27:25.107432+0000","last_became_active":"2024-04-24T23:27:23.947537+0000","last_became_peered":"2024-04-24T23:27:23.947537+0000","last_unstale":"2024-04-24T23:27:25.107432+0000","last_undegraded":"2024-04-24T23:27:25.107432+0000","last_fullsized":"2024-04-24T23:27:25.107432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:25.248902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.943644+0000","last_change":"2024-04-24T23:27:22.933151+0000","last_active":"2024-04-24T23:27:24.943644+0000","last_peered":"2024-04-24T23:27:24.943644+0000","last_clean":"2024-04-24T23:27:24.943644+0000","last_became_active":"2024-04-24T23:27:22.932855+0000","last_became_peered":"2024-04-24T23:27:22.932855+0000","last_unstale":"2024-04-24T23:27:24.943644+0000","last_undegraded":"2024-04-24T23:27:24.943644+0000","last_fullsized":"2024-04-24T23:27:24.943644+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:49.494758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480155+0000","last_change":"2024-04-24T23:27:23.962974+0000","last_active":"2024-04-24T23:27:25.480155+0000","last_peered":"2024-04-24T23:27:25.480155+0000","last_clean":"2024-04-24T23:27:25.480155+0000","last_became_active":"2024-04-24T23:27:23.962610+0000","last_became_peered":"2024-04-24T23:27:23.962610+0000","last_unstale":"2024-04-24T23:27:25.480155+0000","last_undegraded":"2024-04-24T23:27:25.480155+0000","last_fullsized":"2024-04-24T23:27:25.480155+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:28.832665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871252+0000","last_change":"2024-04-24T23:27:22.947404+0000","last_active":"2024-04-24T23:27:25.871252+0000","last_peered":"2024-04-24T23:27:25.871252+0000","last_clean":"2024-04-24T23:27:25.871252+0000","last_became_active":"2024-04-24T23:27:22.947148+0000","last_became_peered":"2024-04-24T23:27:22.947148+0000","last_unstale":"2024-04-24T23:27:25.871252+0000","last_undegraded":"2024-04-24T23:27:25.871252+0000","last_fullsized":"2024-04-24T23:27:25.871252+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:27.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.872048+0000","last_change":"2024-04-24T23:27:23.951304+0000","last_active":"2024-04-24T23:27:25.872048+0000","last_peered":"2024-04-24T23:27:25.872048+0000","last_clean":"2024-04-24T23:27:25.872048+0000","last_became_active":"2024-04-24T23:27:23.951017+0000","last_became_peered":"2024-04-24T23:27:23.951017+0000","last_unstale":"2024-04-24T23:27:25.872048+0000","last_undegraded":"2024-04-24T23:27:25.872048+0000","last_fullsized":"2024-04-24T23:27:25.872048+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:46.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.944120+0000","last_change":"2024-04-24T23:27:22.935055+0000","last_active":"2024-04-24T23:27:24.944120+0000","last_peered":"2024-04-24T23:27:24.944120+0000","last_clean":"2024-04-24T23:27:24.944120+0000","last_became_active":"2024-04-24T23:27:22.934759+0000","last_became_peered":"2024-04-24T23:27:22.934759+0000","last_unstale":"2024-04-24T23:27:24.944120+0000","last_undegraded":"2024-04-24T23:27:24.944120+0000","last_fullsized":"2024-04-24T23:27:24.944120+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:34.272382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168365+0000","last_change":"2024-04-24T23:27:23.948712+0000","last_active":"2024-04-24T23:27:26.168365+0000","last_peered":"2024-04-24T23:27:26.168365+0000","last_clean":"2024-04-24T23:27:26.168365+0000","last_became_active":"2024-04-24T23:27:23.948430+0000","last_became_peered":"2024-04-24T23:27:23.948430+0000","last_unstale":"2024-04-24T23:27:26.168365+0000","last_undegraded":"2024-04-24T23:27:26.168365+0000","last_fullsized":"2024-04-24T23:27:26.168365+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:50.498934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.872094+0000","last_change":"2024-04-24T23:27:22.945229+0000","last_active":"2024-04-24T23:27:25.872094+0000","last_peered":"2024-04-24T23:27:25.872094+0000","last_clean":"2024-04-24T23:27:25.872094+0000","last_became_active":"2024-04-24T23:27:22.944929+0000","last_became_peered":"2024-04-24T23:27:22.944929+0000","last_unstale":"2024-04-24T23:27:25.872094+0000","last_undegraded":"2024-04-24T23:27:25.872094+0000","last_fullsized":"2024-04-24T23:27:25.872094+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:00.925486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.168490+0000","last_change":"2024-04-24T23:27:23.958334+0000","last_active":"2024-04-24T23:27:26.168490+0000","last_peered":"2024-04-24T23:27:26.168490+0000","last_clean":"2024-04-24T23:27:26.168490+0000","last_became_active":"2024-04-24T23:27:23.958032+0000","last_became_peered":"2024-04-24T23:27:23.958032+0000","last_unstale":"2024-04-24T23:27:26.168490+0000","last_undegraded":"2024-04-24T23:27:26.168490+0000","last_fullsized":"2024-04-24T23:27:26.168490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:35.276558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871735+0000","last_change":"2024-04-24T23:27:22.944225+0000","last_active":"2024-04-24T23:27:25.871735+0000","last_peered":"2024-04-24T23:27:25.871735+0000","last_clean":"2024-04-24T23:27:25.871735+0000","last_became_active":"2024-04-24T23:27:22.943272+0000","last_became_peered":"2024-04-24T23:27:22.943272+0000","last_unstale":"2024-04-24T23:27:25.871735+0000","last_undegraded":"2024-04-24T23:27:25.871735+0000","last_fullsized":"2024-04-24T23:27:25.871735+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:23:04.186183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480376+0000","last_change":"2024-04-24T23:27:23.947780+0000","last_active":"2024-04-24T23:27:25.480376+0000","last_peered":"2024-04-24T23:27:25.480376+0000","last_clean":"2024-04-24T23:27:25.480376+0000","last_became_active":"2024-04-24T23:27:23.947487+0000","last_became_peered":"2024-04-24T23:27:23.947487+0000","last_unstale":"2024-04-24T23:27:25.480376+0000","last_undegraded":"2024-04-24T23:27:25.480376+0000","last_fullsized":"2024-04-24T23:27:25.480376+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:01.929662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480447+0000","last_change":"2024-04-24T23:27:22.937169+0000","last_active":"2024-04-24T23:27:25.480447+0000","last_peered":"2024-04-24T23:27:25.480447+0000","last_clean":"2024-04-24T23:27:25.480447+0000","last_became_active":"2024-04-24T23:27:22.936615+0000","last_became_peered":"2024-04-24T23:27:22.936615+0000","last_unstale":"2024-04-24T23:27:25.480447+0000","last_undegraded":"2024-04-24T23:27:25.480447+0000","last_fullsized":"2024-04-24T23:27:25.480447+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:36.458168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107651+0000","last_change":"2024-04-24T23:27:23.950344+0000","last_active":"2024-04-24T23:27:25.107651+0000","last_peered":"2024-04-24T23:27:25.107651+0000","last_clean":"2024-04-24T23:27:25.107651+0000","last_became_active":"2024-04-24T23:27:23.949938+0000","last_became_peered":"2024-04-24T23:27:23.949938+0000","last_unstale":"2024-04-24T23:27:25.107651+0000","last_undegraded":"2024-04-24T23:27:25.107651+0000","last_fullsized":"2024-04-24T23:27:25.107651+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:46.561192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.871625+0000","last_change":"2024-04-24T23:27:22.941877+0000","last_active":"2024-04-24T23:27:25.871625+0000","last_peered":"2024-04-24T23:27:25.871625+0000","last_clean":"2024-04-24T23:27:25.871625+0000","last_became_active":"2024-04-24T23:27:22.941539+0000","last_became_peered":"2024-04-24T23:27:22.941539+0000","last_unstale":"2024-04-24T23:27:25.871625+0000","last_undegraded":"2024-04-24T23:27:25.871625+0000","last_fullsized":"2024-04-24T23:27:25.871625+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:05.682641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.372245+0000","last_change":"2024-04-24T23:27:23.948553+0000","last_active":"2024-04-24T23:27:26.372245+0000","last_peered":"2024-04-24T23:27:26.372245+0000","last_clean":"2024-04-24T23:27:26.372245+0000","last_became_active":"2024-04-24T23:27:23.947813+0000","last_became_peered":"2024-04-24T23:27:23.947813+0000","last_unstale":"2024-04-24T23:27:26.372245+0000","last_undegraded":"2024-04-24T23:27:26.372245+0000","last_fullsized":"2024-04-24T23:27:26.372245+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:48.349380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480013+0000","last_change":"2024-04-24T23:27:22.949767+0000","last_active":"2024-04-24T23:27:25.480013+0000","last_peered":"2024-04-24T23:27:25.480013+0000","last_clean":"2024-04-24T23:27:25.480013+0000","last_became_active":"2024-04-24T23:27:22.949531+0000","last_became_peered":"2024-04-24T23:27:22.949531+0000","last_unstale":"2024-04-24T23:27:25.480013+0000","last_undegraded":"2024-04-24T23:27:25.480013+0000","last_fullsized":"2024-04-24T23:27:25.480013+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:49.494758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107488+0000","last_change":"2024-04-24T23:27:23.953544+0000","last_active":"2024-04-24T23:27:25.107488+0000","last_peered":"2024-04-24T23:27:25.107488+0000","last_clean":"2024-04-24T23:27:25.107488+0000","last_became_active":"2024-04-24T23:27:23.953258+0000","last_became_peered":"2024-04-24T23:27:23.953258+0000","last_unstale":"2024-04-24T23:27:25.107488+0000","last_undegraded":"2024-04-24T23:27:25.107488+0000","last_fullsized":"2024-04-24T23:27:25.107488+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:38.430795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:24.942788+0000","last_change":"2024-04-24T23:27:22.947591+0000","last_active":"2024-04-24T23:27:24.942788+0000","last_peered":"2024-04-24T23:27:24.942788+0000","last_clean":"2024-04-24T23:27:24.942788+0000","last_became_active":"2024-04-24T23:27:22.947296+0000","last_became_peered":"2024-04-24T23:27:22.947296+0000","last_unstale":"2024-04-24T23:27:24.942788+0000","last_undegraded":"2024-04-24T23:27:24.942788+0000","last_fullsized":"2024-04-24T23:27:24.942788+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:27.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.480206+0000","last_change":"2024-04-24T23:27:23.950390+0000","last_active":"2024-04-24T23:27:25.480206+0000","last_peered":"2024-04-24T23:27:25.480206+0000","last_clean":"2024-04-24T23:27:25.480206+0000","last_became_active":"2024-04-24T23:27:23.950094+0000","last_became_peered":"2024-04-24T23:27:23.950094+0000","last_unstale":"2024-04-24T23:27:25.480206+0000","last_undegraded":"2024-04-24T23:27:25.480206+0000","last_fullsized":"2024-04-24T23:27:25.480206+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:48.349380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107491+0000","last_change":"2024-04-24T23:27:22.943541+0000","last_active":"2024-04-24T23:27:25.107491+0000","last_peered":"2024-04-24T23:27:25.107491+0000","last_clean":"2024-04-24T23:27:25.107491+0000","last_became_active":"2024-04-24T23:27:22.942994+0000","last_became_peered":"2024-04-24T23:27:22.942994+0000","last_unstale":"2024-04-24T23:27:25.107491+0000","last_undegraded":"2024-04-24T23:27:25.107491+0000","last_fullsized":"2024-04-24T23:27:25.107491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:14.204966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.107873+0000","last_change":"2024-04-24T23:27:23.951430+0000","last_active":"2024-04-24T23:27:25.107873+0000","last_peered":"2024-04-24T23:27:25.107873+0000","last_clean":"2024-04-24T23:27:25.107873+0000","last_became_active":"2024-04-24T23:27:23.951173+0000","last_became_peered":"2024-04-24T23:27:23.951173+0000","last_unstale":"2024-04-24T23:27:25.107873+0000","last_undegraded":"2024-04-24T23:27:25.107873+0000","last_fullsized":"2024-04-24T23:27:25.107873+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:22.918887+0000","last_clean_scrub_stamp":"2024-04-24T23:27:22.918887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:08.139056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282432+0000","last_change":"2024-04-24T23:27:22.942215+0000","last_active":"2024-04-24T23:27:25.282432+0000","last_peered":"2024-04-24T23:27:25.282432+0000","last_clean":"2024-04-24T23:27:25.282432+0000","last_became_active":"2024-04-24T23:27:22.941971+0000","last_became_peered":"2024-04-24T23:27:22.941971+0000","last_unstale":"2024-04-24T23:27:25.282432+0000","last_undegraded":"2024-04-24T23:27:25.282432+0000","last_fullsized":"2024-04-24T23:27:25.282432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:21.914710+0000","last_clean_scrub_stamp":"2024-04-24T23:27:21.914710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:44:47.345203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-24T23:27:25.282404+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:27:25.282404+0000","last_peered":"2024-04-24T23:27:25.282404+0000","last_clean":"2024-04-24T23:27:25.282404+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:27:25.282404+0000","last_undegraded":"2024-04-24T23:27:25.282404+0000","last_fullsized":"2024-04-24T23:27:25.282404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37092,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334748,"statfs":{"total":96636764160,"available":96598781952,"internally_reserved":0,"allocated":1789952,"data_stored":1106336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":516508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":515926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1208320,"data_stored":517592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":517490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36512,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335328,"statfs":{"total":96636764160,"available":96599375872,"internally_reserved":0,"allocated":1191936,"data_stored":515890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36504,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335336,"statfs":{"total":96636764160,"available":96599384064,"internally_reserved":0,"allocated":1187840,"data_stored":515856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37084,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334756,"statfs":{"total":96636764160,"available":96598790144,"internally_reserved":0,"allocated":1785856,"data_stored":1106806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:27:28.328 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:27:28.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:27:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.442+0000 7faaefc9e640 1 Processor -- start 2024-04-24T23:27:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.442+0000 7faaefc9e640 1 -- start start 2024-04-24T23:27:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.443+0000 7faaefc9e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7faae80ad7b0 0x7faae80d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.443+0000 7faaefc9e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae815ef60 con 0x7faae815e9c0 2024-04-24T23:27:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.443+0000 7faaefc9e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae815f240 con 0x7faae815adc0 2024-04-24T23:27:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.443+0000 7faaefc9e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7faae815f520 con 0x7faae80ad7b0 2024-04-24T23:27:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.443+0000 7faaee23b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faae815e9c0 0x7faae8072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58402/0 (socket says 172.21.15.26:58402) 2024-04-24T23:27:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.443+0000 7faaee23b640 1 -- 172.21.15.26:0/2749390597 learned_addr learned my addr 172.21.15.26:0/2749390597 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.443+0000 7faaed239640 1 --2- 172.21.15.26:0/2749390597 >> v2:172.21.15.29:3300/0 conn(0x7faae80ad7b0 0x7faae80d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.443+0000 7faaeca38640 1 -- 172.21.15.26:0/2749390597 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 188782971 0 0) 0x7faae815ef60 con 0x7faae815e9c0 2024-04-24T23:27:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.443+0000 7faaeca38640 1 -- 172.21.15.26:0/2749390597 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faad0003520 con 0x7faae815e9c0 2024-04-24T23:27:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.444+0000 7faaed239640 1 -- 172.21.15.26:0/2749390597 >> v1:172.21.15.29:6789/0 conn(0x7faae815adc0 legacy=0x7faae80b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.444+0000 7faaed239640 1 -- 172.21.15.26:0/2749390597 >> v1:172.21.15.26:6789/0 conn(0x7faae815e9c0 legacy=0x7faae8072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.444+0000 7faaed239640 1 -- 172.21.15.26:0/2749390597 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae815f800 con 0x7faae80ad7b0 2024-04-24T23:27:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.444+0000 7faaed239640 1 --2- 172.21.15.26:0/2749390597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80ad7b0 0x7faae80d0040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faadc0097d0 tx=0x7faadc02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b115072af2d7e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaeca38640 1 -- 172.21.15.26:0/2749390597 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc037070 con 0x7faae80ad7b0 2024-04-24T23:27:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaeca38640 1 -- 172.21.15.26:0/2749390597 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc02ae40 con 0x7faae80ad7b0 2024-04-24T23:27:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaeca38640 1 -- 172.21.15.26:0/2749390597 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc032660 con 0x7faae80ad7b0 2024-04-24T23:27:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaefc9e640 1 -- 172.21.15.26:0/2749390597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80ad7b0 msgr2=0x7faae80d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaefc9e640 1 --2- 172.21.15.26:0/2749390597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80ad7b0 0x7faae80d0040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faadc0097d0 tx=0x7faadc02a130 comp rx=0 tx=0).stop 2024-04-24T23:27:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaefc9e640 1 -- 172.21.15.26:0/2749390597 shutdown_connections 2024-04-24T23:27:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaefc9e640 1 --2- 172.21.15.26:0/2749390597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae80ad7b0 0x7faae80d0040 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaefc9e640 1 -- 172.21.15.26:0/2749390597 >> 172.21.15.26:0/2749390597 conn(0x7faae806b540 msgr2=0x7faae806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaefc9e640 1 -- 172.21.15.26:0/2749390597 shutdown_connections 2024-04-24T23:27:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.445+0000 7faaefc9e640 1 -- 172.21.15.26:0/2749390597 wait complete. 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.446+0000 7faaefc9e640 1 Processor -- start 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.446+0000 7faaefc9e640 1 -- start start 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaefc9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faae80ad7b0 0x7faae8140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaefc9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae815adc0 0x7faae8142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaefc9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae815e9c0 0x7faae8143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaefc9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faae80d0580 con 0x7faae815adc0 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaefc9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faae815b1a0 con 0x7faae815e9c0 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaefc9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faae80d0700 con 0x7faae80ad7b0 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaed239640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae815adc0 0x7faae8142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaed239640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae815adc0 0x7faae8142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60638/0 (socket says 172.21.15.26:60638) 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaed239640 1 -- 172.21.15.26:0/757090684 learned_addr learned my addr 172.21.15.26:0/757090684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaee23b640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae815e9c0 0x7faae8143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaeda3a640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faae80ad7b0 0x7faae8140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaed239640 1 -- 172.21.15.26:0/757090684 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faae80ad7b0 msgr2=0x7faae8140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaed239640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faae80ad7b0 0x7faae8140740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaed239640 1 -- 172.21.15.26:0/757090684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae815e9c0 msgr2=0x7faae8143430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.447+0000 7faaed239640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae815e9c0 0x7faae8143430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.448+0000 7faaed239640 1 -- 172.21.15.26:0/757090684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae809aa00 con 0x7faae815adc0 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.448+0000 7faaed239640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae815adc0 0x7faae8142ef0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7faadc0098f0 tx=0x7faadc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.448+0000 7faae67fc640 1 -- 172.21.15.26:0/757090684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc041020 con 0x7faae815adc0 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.448+0000 7faae67fc640 1 -- 172.21.15.26:0/757090684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc03c670 con 0x7faae815adc0 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.448+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae815cb80 con 0x7faae815adc0 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.448+0000 7faae67fc640 1 -- 172.21.15.26:0/757090684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc037040 con 0x7faae815adc0 2024-04-24T23:27:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.448+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae8141390 con 0x7faae815adc0 2024-04-24T23:27:28.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.449+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae8075610 con 0x7faae815adc0 2024-04-24T23:27:28.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.449+0000 7faae67fc640 1 -- 172.21.15.26:0/757090684 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faadc044020 con 0x7faae815adc0 2024-04-24T23:27:28.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.450+0000 7faae67fc640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad406be30 0x7faad406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.450+0000 7faae67fc640 1 -- 172.21.15.26:0/757090684 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7faadc0ab3d0 con 0x7faae815adc0 2024-04-24T23:27:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.450+0000 7faaeda3a640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad406be30 0x7faad406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.450+0000 7faaeda3a640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad406be30 0x7faad406e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faae0001780 tx=0x7faae0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.454+0000 7faae67fc640 1 -- 172.21.15.26:0/757090684 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faadc04b030 con 0x7faae815adc0 2024-04-24T23:27:28.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.709+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7faae815e730 con 0x7faae815adc0 2024-04-24T23:27:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.711+0000 7faae67fc640 1 -- 172.21.15.26:0/757090684 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7faae809aa00 con 0x7faae815adc0 2024-04-24T23:27:28.712 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-24T23:27:28.712 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQChkylmsqrFKhAAj4YMdfkIgaIrSWGhyze1oQ== 2024-04-24T23:27:28.712 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-04-24T23:27:28.712 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-04-24T23:27:28.712 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:27:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-04-24T23:27:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad406be30 msgr2=0x7faad406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad406be30 0x7faad406e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faae0001780 tx=0x7faae0009040 comp rx=0 tx=0).stop 2024-04-24T23:27:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae815adc0 msgr2=0x7faae8142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae815adc0 0x7faae8142ef0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7faadc0098f0 tx=0x7faadc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 shutdown_connections 2024-04-24T23:27:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faad406be30 0x7faad406e2f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faae815e9c0 0x7faae8143430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae815adc0 0x7faae8142ef0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 --2- 172.21.15.26:0/757090684 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faae80ad7b0 0x7faae8140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 >> 172.21.15.26:0/757090684 conn(0x7faae806b540 msgr2=0x7faae806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 shutdown_connections 2024-04-24T23:27:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.713+0000 7faaefc9e640 1 -- 172.21.15.26:0/757090684 wait complete. 2024-04-24T23:27:28.725 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.824+0000 7fdbdb4a5640 1 Processor -- start 2024-04-24T23:27:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.824+0000 7fdbdb4a5640 1 -- start start 2024-04-24T23:27:28.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbdb4a5640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdbd415e1c0 0x7fdbd40b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbdb4a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 0x7fdbd40ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbdb4a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd40733c0 con 0x7fdbd40af440 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbdb4a5640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd40add00 con 0x7fdbd415e1c0 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbdb4a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd40adfe0 con 0x7fdbd415e5a0 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbd9a42640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdbd40af440 0x7fdbd40afb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58414/0 (socket says 172.21.15.26:58414) 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbd9a42640 1 -- 172.21.15.26:0/258157497 learned_addr learned my addr 172.21.15.26:0/258157497 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237535561 0 0) 0x7fdbd40733c0 con 0x7fdbd40af440 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbbc003520 con 0x7fdbd40af440 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbd9241640 1 --2- 172.21.15.26:0/258157497 >> v2:172.21.15.29:3300/0 conn(0x7fdbd415e1c0 0x7fdbd40b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.825+0000 7fdbd8a40640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 0x7fdbd40ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 936353812 0 0) 0x7fdbbc003520 con 0x7fdbd40af440 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 msgr2=0x7fdbd40ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 0x7fdbd40ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 >> v2:172.21.15.29:3300/0 conn(0x7fdbd415e1c0 msgr2=0x7fdbd40b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 --2- 172.21.15.26:0/258157497 >> v2:172.21.15.29:3300/0 conn(0x7fdbd415e1c0 0x7fdbd40b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd40ae2c0 con 0x7fdbd40af440 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fdbcc001670 con 0x7fdbd40af440 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd8a40640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 0x7fdbd40ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 >> v1:172.21.15.26:6789/0 conn(0x7fdbd40af440 legacy=0x7fdbd40afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 0x7fdbbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbbc004d40 0x7fdbbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd40add00 con 0x7fdbbc0039b0 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd40adfe0 con 0x7fdbbc004d40 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdbbc00ba10 con 0x7fdbd415e5a0 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd9241640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd8a40640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 0x7fdbbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd9a42640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbbc004d40 0x7fdbbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd9241640 1 -- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 msgr2=0x7fdbbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd9241640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 0x7fdbbc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd9241640 1 -- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbbc004d40 msgr2=0x7fdbbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd9241640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbbc004d40 0x7fdbbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd9241640 1 -- 172.21.15.26:0/258157497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc003520 con 0x7fdbbc0039b0 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd8a40640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd415e5a0 0x7fdbbc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.826+0000 7fdbd9a42640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbbc004d40 0x7fdbbc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.827+0000 7fdbd9241640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbbc003dd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009fc0 tx=0x7fdbbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983db07a1325d8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.827+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4009c10 con 0x7fdbbc0039b0 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.827+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc4036070 con 0x7fdbbc0039b0 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.827+0000 7fdbd3fff640 1 -- 172.21.15.26:0/258157497 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc402d070 con 0x7fdbbc0039b0 2024-04-24T23:27:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.828+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/258157497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 msgr2=0x7fdbbc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.828+0000 7fdbdb4a5640 1 --2- 172.21.15.26:0/258157497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbbc003dd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009fc0 tx=0x7fdbbc005830 comp rx=0 tx=0).stop 2024-04-24T23:27:28.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.828+0000 7fdbd9a42640 1 -- 172.21.15.26:0/258157497 reap_dead start 2024-04-24T23:27:28.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.828+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/258157497 shutdown_connections 2024-04-24T23:27:28.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.828+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/258157497 >> 172.21.15.26:0/258157497 conn(0x7fdbd4072790 msgr2=0x7fdbd4072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:28.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.828+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/258157497 shutdown_connections 2024-04-24T23:27:28.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.828+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/258157497 wait complete. 2024-04-24T23:27:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.829+0000 7fdbdb4a5640 1 Processor -- start 2024-04-24T23:27:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.829+0000 7fdbdb4a5640 1 -- start start 2024-04-24T23:27:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbdb4a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbd4149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbdb4a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd41f4d00 0x7fdbd415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbdb4a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd40cfbe0 0x7fdbd40d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbdb4a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd40b03b0 con 0x7fdbbc0039b0 2024-04-24T23:27:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbdb4a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd40b0230 con 0x7fdbd40cfbe0 2024-04-24T23:27:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbdb4a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd40ae7e0 con 0x7fdbd41f4d00 2024-04-24T23:27:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9241640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbd4149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9241640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbd4149ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60652/0 (socket says 172.21.15.26:60652) 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9241640 1 -- 172.21.15.26:0/2023236704 learned_addr learned my addr 172.21.15.26:0/2023236704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9241640 1 -- 172.21.15.26:0/2023236704 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd41f4d00 msgr2=0x7fdbd415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd8a40640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd41f4d00 0x7fdbd415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9a42640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd40cfbe0 0x7fdbd40d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9241640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd41f4d00 0x7fdbd415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9241640 1 -- 172.21.15.26:0/2023236704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd40cfbe0 msgr2=0x7fdbd40d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9241640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd40cfbe0 0x7fdbd40d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9241640 1 -- 172.21.15.26:0/2023236704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd415d010 con 0x7fdbbc0039b0 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9a42640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd40cfbe0 0x7fdbd40d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd9241640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbd4149ca0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e230 tx=0x7fdbc4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd1ffb640 1 -- 172.21.15.26:0/2023236704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4009c10 con 0x7fdbbc0039b0 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd1ffb640 1 -- 172.21.15.26:0/2023236704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc4003b90 con 0x7fdbbc0039b0 2024-04-24T23:27:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd415d2f0 con 0x7fdbbc0039b0 2024-04-24T23:27:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.830+0000 7fdbd1ffb640 1 -- 172.21.15.26:0/2023236704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4004020 con 0x7fdbbc0039b0 2024-04-24T23:27:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.831+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd415d8b0 con 0x7fdbbc0039b0 2024-04-24T23:27:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.831+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd40694c0 con 0x7fdbbc0039b0 2024-04-24T23:27:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.832+0000 7fdbd1ffb640 1 -- 172.21.15.26:0/2023236704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdbc4005d10 con 0x7fdbbc0039b0 2024-04-24T23:27:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.832+0000 7fdbd1ffb640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdbc006be60 0x7fdbc006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.832+0000 7fdbd1ffb640 1 -- 172.21.15.26:0/2023236704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fdbc40ac360 con 0x7fdbbc0039b0 2024-04-24T23:27:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.832+0000 7fdbd8a40640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdbc006be60 0x7fdbc006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.833+0000 7fdbd8a40640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdbc006be60 0x7fdbc006e320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdbd41f50e0 tx=0x7fdbc8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:28.837+0000 7fdbd1ffb640 1 -- 172.21.15.26:0/2023236704 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdbc407ad90 con 0x7fdbbc0039b0 2024-04-24T23:27:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.056+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbd415ecd0 con 0x7fdbbc0039b0 2024-04-24T23:27:29.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.057+0000 7fdbd1ffb640 1 -- 172.21.15.26:0/2023236704 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5185 (secure 0 0 0) 0x7fdbc407a690 con 0x7fdbbc0039b0 2024-04-24T23:27:29.058 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:29.058 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":103,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":7444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:26.311710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730]},"id":7}]} 2024-04-24T23:27:29.058 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 103 2024-04-24T23:27:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdbc006be60 msgr2=0x7fdbc006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdbc006be60 0x7fdbc006e320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdbd41f50e0 tx=0x7fdbc8008040 comp rx=0 tx=0).stop 2024-04-24T23:27:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 msgr2=0x7fdbd4149ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbd4149ca0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e230 tx=0x7fdbc4009750 comp rx=0 tx=0).stop 2024-04-24T23:27:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 shutdown_connections 2024-04-24T23:27:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdbc006be60 0x7fdbc006e320 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbd40cfbe0 0x7fdbd40d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbd41f4d00 0x7fdbd415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 --2- 172.21.15.26:0/2023236704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc0039b0 0x7fdbd4149ca0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 >> 172.21.15.26:0/2023236704 conn(0x7fdbd4072790 msgr2=0x7fdbd40d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 shutdown_connections 2024-04-24T23:27:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.059+0000 7fdbdb4a5640 1 -- 172.21.15.26:0/2023236704 wait complete. 2024-04-24T23:27:29.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:27:23.297118+0000', 'modified': '2024-04-24T23:27:26.311710+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8730}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8730': {'gid': 8730, 'name': 'd', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/2225348762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 2225348762}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 2225348762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8730, 'qdb_cluster': [8730]} 2024-04-24T23:27:29.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:27:29.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:27:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.582+0000 7faa9643f640 1 Processor -- start 2024-04-24T23:27:29.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.582+0000 7faa9643f640 1 -- start start 2024-04-24T23:27:29.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa9643f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa90145910 0x7faa9006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa9643f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa900cfbc0 con 0x7faa9006d350 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa9643f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9006d7f0 con 0x7faa900cd660 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa9643f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa90145cf0 con 0x7faa90145910 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa8f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa9006d350 0x7faa9015d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58420/0 (socket says 172.21.15.26:58420) 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa8f7fe640 1 -- 172.21.15.26:0/2159793070 learned_addr learned my addr 172.21.15.26:0/2159793070 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa8ffff640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa90145910 0x7faa9006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4136750622 0 0) 0x7faa900cfbc0 con 0x7faa9006d350 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa78003520 con 0x7faa9006d350 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671104557 0 0) 0x7faa9006d7f0 con 0x7faa900cd660 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.583+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa900cfbc0 con 0x7faa900cd660 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3282300153 0 0) 0x7faa78003520 con 0x7faa9006d350 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa90145910 msgr2=0x7faa9006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa90145910 0x7faa9006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 >> v1:172.21.15.29:6789/0 conn(0x7faa900cd660 legacy=0x7faa9015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9006dad0 con 0x7faa9006d350 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7faa80001670 con 0x7faa9006d350 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 >> v1:172.21.15.26:6789/0 conn(0x7faa9006d350 legacy=0x7faa9015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa90145910 0x7faa78003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa780040d0 0x7faa78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78004d40 0x7faa780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa90145cf0 con 0x7faa780040d0 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800baa0 con 0x7faa78004d40 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa7800bca0 con 0x7faa90145910 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa780040d0 0x7faa78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8ffff640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa90145910 0x7faa78003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa949dc640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78004d40 0x7faa780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8f7fe640 1 -- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa90145910 msgr2=0x7faa78003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa90145910 0x7faa78003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8f7fe640 1 -- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78004d40 msgr2=0x7faa780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa78004d40 0x7faa780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.584+0000 7faa8f7fe640 1 -- 172.21.15.26:0/2159793070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa78003520 con 0x7faa780040d0 2024-04-24T23:27:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.585+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa780040d0 0x7faa78004800 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c190 tx=0x7faa78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f9f3fa401d3560 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.585+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80007cc0 con 0x7faa780040d0 2024-04-24T23:27:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.585+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa80005af0 con 0x7faa780040d0 2024-04-24T23:27:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.585+0000 7faa8effd640 1 -- 172.21.15.26:0/2159793070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80007070 con 0x7faa780040d0 2024-04-24T23:27:29.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.585+0000 7faa9643f640 1 -- 172.21.15.26:0/2159793070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa780040d0 msgr2=0x7faa78004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.585+0000 7faa9643f640 1 --2- 172.21.15.26:0/2159793070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa780040d0 0x7faa78004800 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c190 tx=0x7faa78005830 comp rx=0 tx=0).stop 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.585+0000 7faa949dc640 1 -- 172.21.15.26:0/2159793070 reap_dead start 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.586+0000 7faa9643f640 1 -- 172.21.15.26:0/2159793070 shutdown_connections 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.586+0000 7faa9643f640 1 -- 172.21.15.26:0/2159793070 >> 172.21.15.26:0/2159793070 conn(0x7faa9006b300 msgr2=0x7faa9006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.586+0000 7faa9643f640 1 -- 172.21.15.26:0/2159793070 shutdown_connections 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.586+0000 7faa9643f640 1 -- 172.21.15.26:0/2159793070 wait complete. 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.587+0000 7faa9643f640 1 Processor -- start 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.587+0000 7faa9643f640 1 -- start start 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa9643f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa780040d0 0x7faa901f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa9643f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa9013ca20 0x7faa9013e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa9643f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9013cea0 0x7faa9013ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa9643f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa9006e370 con 0x7faa9013cea0 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa9643f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa9006e090 con 0x7faa9013ca20 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa9643f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa9006e890 con 0x7faa780040d0 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa949dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9013cea0 0x7faa9013ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa949dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9013cea0 0x7faa9013ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60674/0 (socket says 172.21.15.26:60674) 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa949dc640 1 -- 172.21.15.26:0/2192020983 learned_addr learned my addr 172.21.15.26:0/2192020983 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa8ffff640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa780040d0 0x7faa901f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa9013ca20 0x7faa9013e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa949dc640 1 -- 172.21.15.26:0/2192020983 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa780040d0 msgr2=0x7faa901f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa949dc640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa780040d0 0x7faa901f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa949dc640 1 -- 172.21.15.26:0/2192020983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa9013ca20 msgr2=0x7faa9013e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa949dc640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa9013ca20 0x7faa9013e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa949dc640 1 -- 172.21.15.26:0/2192020983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9013d400 con 0x7faa9013cea0 2024-04-24T23:27:29.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa9013ca20 0x7faa9013e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa949dc640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9013cea0 0x7faa9013ec50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7faa8800f8d0 tx=0x7faa8800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.588+0000 7faa8cff9640 1 -- 172.21.15.26:0/2192020983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88019070 con 0x7faa9013cea0 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.589+0000 7faa8cff9640 1 -- 172.21.15.26:0/2192020983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88006e20 con 0x7faa9013cea0 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.589+0000 7faa9643f640 1 -- 172.21.15.26:0/2192020983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9013d6e0 con 0x7faa9013cea0 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.589+0000 7faa8cff9640 1 -- 172.21.15.26:0/2192020983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88003b70 con 0x7faa9013cea0 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.589+0000 7faa9643f640 1 -- 172.21.15.26:0/2192020983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9013dca0 con 0x7faa9013cea0 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.590+0000 7faa8cff9640 1 -- 172.21.15.26:0/2192020983 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa8801c070 con 0x7faa9013cea0 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.590+0000 7faa8cff9640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa7c06be30 0x7faa7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.591+0000 7faa8cff9640 1 -- 172.21.15.26:0/2192020983 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7faa8808a0e0 con 0x7faa9013cea0 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.591+0000 7faa8ffff640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa7c06be30 0x7faa7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.591+0000 7faa8ffff640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa7c06be30 0x7faa7c06e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faa84009a20 tx=0x7faa84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa9922d740 1 Processor -- start 2024-04-24T23:27:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa9922d740 1 -- start start 2024-04-24T23:27:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa9922d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d7e50 0x55f2371dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa9922d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371afb60 0x55f2371d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa9922d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d8770 0x55f2371db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa9922d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f237113f00 con 0x55f2371afb60 2024-04-24T23:27:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa9922d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f2370bc750 con 0x55f2371d7e50 2024-04-24T23:27:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa9922d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f23710fec0 con 0x55f2371d8770 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371afb60 0x55f2371d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371afb60 0x55f2371d8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60680/0 (socket says 172.21.15.26:60680) 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.592+0000 7faa8f7fe640 1 -- 172.21.15.26:0/1425538088 learned_addr learned my addr 172.21.15.26:0/1425538088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa8ffff640 1 --2- 172.21.15.26:0/1425538088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d7e50 0x55f2371dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa949dc640 1 --2- 172.21.15.26:0/1425538088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d8770 0x55f2371db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa8f7fe640 1 -- 172.21.15.26:0/1425538088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d8770 msgr2=0x55f2371db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/1425538088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d8770 0x55f2371db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa8f7fe640 1 -- 172.21.15.26:0/1425538088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d7e50 msgr2=0x55f2371dbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/1425538088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d7e50 0x55f2371dbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa8f7fe640 1 -- 172.21.15.26:0/1425538088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f236e282d0 con 0x55f2371afb60 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/1425538088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371afb60 0x55f2371d8230 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c1c0 tx=0x7faa80007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0e59e7a02873ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa75ffb640 1 -- 172.21.15.26:0/1425538088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80001470 con 0x55f2371afb60 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa75ffb640 1 -- 172.21.15.26:0/1425538088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa80043070 con 0x55f2371afb60 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.593+0000 7faa75ffb640 1 -- 172.21.15.26:0/1425538088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa800047e0 con 0x55f2371afb60 2024-04-24T23:27:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 -- 172.21.15.26:0/1425538088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371afb60 msgr2=0x55f2371d8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 --2- 172.21.15.26:0/1425538088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371afb60 0x55f2371d8230 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c1c0 tx=0x7faa80007000 comp rx=0 tx=0).stop 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 -- 172.21.15.26:0/1425538088 shutdown_connections 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 --2- 172.21.15.26:0/1425538088 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d8770 0x55f2371db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 --2- 172.21.15.26:0/1425538088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371afb60 0x55f2371d8230 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 --2- 172.21.15.26:0/1425538088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d7e50 0x55f2371dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 -- 172.21.15.26:0/1425538088 >> 172.21.15.26:0/1425538088 conn(0x55f2371ad760 msgr2=0x55f2371f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 -- 172.21.15.26:0/1425538088 shutdown_connections 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 -- 172.21.15.26:0/1425538088 wait complete. 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.594+0000 7faa9922d740 1 -- start start 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa9922d740 1 Processor -- start 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa9922d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d7e50 0x55f23720e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa9922d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 0x55f2371c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa9922d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d53a0 0x55f2371f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa9922d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2371abab0 con 0x55f2371d53a0 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa9922d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f2370cfcf0 con 0x55f2371d8770 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa9922d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f2371ab4f0 con 0x55f2371d7e50 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa9922d740 10 client.0 _ll_register_callbacks cb 0x55f2370cf6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa949dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d53a0 0x55f2371f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa949dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d53a0 0x55f2371f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60696/0 (socket says 172.21.15.26:60696) 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa949dc640 1 -- 172.21.15.26:0/991412545 learned_addr learned my addr 172.21.15.26:0/991412545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa8ffff640 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d7e50 0x55f23720e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.595+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 0x55f2371c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa949dc640 1 -- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d7e50 msgr2=0x55f23720e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa949dc640 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d7e50 0x55f23720e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa949dc640 1 -- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 msgr2=0x55f2371c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa949dc640 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 0x55f2371c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa949dc640 1 -- 172.21.15.26:0/991412545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f236ef26b0 con 0x55f2371d53a0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa949dc640 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d53a0 0x55f2371f7c50 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7faa880886c0 tx=0x7faa88058530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b5baff853b93ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa6effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa6effd640 1 -- 172.21.15.26:0/991412545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa880703e0 con 0x55f2371d53a0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa6effd640 1 -- 172.21.15.26:0/991412545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7faa88057870 con 0x55f2371d53a0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa9922d740 1 -- 172.21.15.26:0/991412545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa80043070 con 0x55f2371d53a0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa6effd640 1 -- 172.21.15.26:0/991412545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7faa88057a70 con 0x55f2371d53a0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa9922d740 10 client.8862 fetch_fsmap learned FSMap version 103 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa9922d740 1 -- 172.21.15.26:0/991412545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) -- 0x55f2371cee40 con 0x55f2371d53a0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa6effd640 1 -- 172.21.15.26:0/991412545 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa80043070 con 0x55f2371d53a0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa6effd640 1 -- 172.21.15.26:0/991412545 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 103) ==== 3494+0+0 (secure 0 0 0) 0x7faa88054170 con 0x55f2371d53a0 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa9922d740 10 client.8862 fetch_fsmap finished waiting for FSMap version 103 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa9922d740 10 client.8862 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa9922d740 20 client.8862 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa9922d740 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55f2371cc740 0x55f2371cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa9922d740 4 client.8862 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.596+0000 7faa9922d740 1 -- 172.21.15.26:0/991412545 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f237121440 con 0x55f2371cc740 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.597+0000 7faa8ffff640 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55f2371cc740 0x55f2371cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.597+0000 7faa8ffff640 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55f2371cc740 0x55f2371cdab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a0fc76a3bbef4a server_cookie=a821bb46dc97ef87 in_seq=0 out_seq=0 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.597+0000 7faa6effd640 10 client.8862 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa6effd640 1 -- 172.21.15.26:0/991412545 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f237121440 con 0x55f2371cc740 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa6effd640 10 client.8862 handle_command_reply: tid=0 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 client.8862 shutdown 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 -- 172.21.15.26:0/991412545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d53a0 msgr2=0x55f2371f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d53a0 0x55f2371f7c50 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7faa880886c0 tx=0x7faa88058530 comp rx=0 tx=0).stop 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 -- 172.21.15.26:0/991412545 shutdown_connections 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55f2371cc740 0x55f2371cdab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d53a0 0x55f2371f7c50 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 0x55f2371c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 --2- 172.21.15.26:0/991412545 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d7e50 0x55f23720e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa6effd640 0 client.8862 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 -- 172.21.15.26:0/991412545 >> 172.21.15.26:0/991412545 conn(0x55f2371afb60 msgr2=0x55f2371b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 -- 172.21.15.26:0/991412545 shutdown_connections 2024-04-24T23:27:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 1 -- 172.21.15.26:0/991412545 wait complete. 2024-04-24T23:27:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.598+0000 7faa9922d740 20 client.8862 trim_cache size 0 max 16384 2024-04-24T23:27:29.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.617+0000 7faa9922d740 1 Processor -- start 2024-04-24T23:27:29.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.617+0000 7faa9922d740 1 -- start start 2024-04-24T23:27:29.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa9922d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d53a0 0x55f2371deca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa9922d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d7e50 0x55f2371aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa9922d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d8770 0x55f2371aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa9922d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa900b4900 con 0x55f2371d8770 2024-04-24T23:27:29.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa9922d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faa9013a520 con 0x55f2371d53a0 2024-04-24T23:27:29.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa9922d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7faa900b42b0 con 0x55f2371d7e50 2024-04-24T23:27:29.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa949dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d8770 0x55f2371aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa949dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d8770 0x55f2371aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60698/0 (socket says 172.21.15.26:60698) 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa949dc640 1 -- 172.21.15.26:0/4255694627 learned_addr learned my addr 172.21.15.26:0/4255694627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa949dc640 1 -- 172.21.15.26:0/4255694627 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d7e50 msgr2=0x55f2371aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa949dc640 1 --2- 172.21.15.26:0/4255694627 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d7e50 0x55f2371aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa949dc640 1 -- 172.21.15.26:0/4255694627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d53a0 msgr2=0x55f2371deca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa949dc640 1 --2- 172.21.15.26:0/4255694627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d53a0 0x55f2371deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa949dc640 1 -- 172.21.15.26:0/4255694627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2371d0770 con 0x55f2371d8770 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa949dc640 1 --2- 172.21.15.26:0/4255694627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d8770 0x55f2371aa980 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7faa880566d0 tx=0x7faa88005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231991be3af90239 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa757fa640 1 -- 172.21.15.26:0/4255694627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88054170 con 0x55f2371d8770 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa757fa640 1 -- 172.21.15.26:0/4255694627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88070030 con 0x55f2371d8770 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.618+0000 7faa757fa640 1 -- 172.21.15.26:0/4255694627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa880703e0 con 0x55f2371d8770 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 -- 172.21.15.26:0/4255694627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d8770 msgr2=0x55f2371aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 --2- 172.21.15.26:0/4255694627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d8770 0x55f2371aa980 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7faa880566d0 tx=0x7faa88005070 comp rx=0 tx=0).stop 2024-04-24T23:27:29.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 -- 172.21.15.26:0/4255694627 shutdown_connections 2024-04-24T23:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 --2- 172.21.15.26:0/4255694627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d8770 0x55f2371aa980 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 --2- 172.21.15.26:0/4255694627 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d7e50 0x55f2371aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 --2- 172.21.15.26:0/4255694627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d53a0 0x55f2371deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 -- 172.21.15.26:0/4255694627 >> 172.21.15.26:0/4255694627 conn(0x55f2371afb60 msgr2=0x55f2371b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 -- 172.21.15.26:0/4255694627 shutdown_connections 2024-04-24T23:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 -- 172.21.15.26:0/4255694627 wait complete. 2024-04-24T23:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.619+0000 7faa9922d740 1 -- start start 2024-04-24T23:27:29.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.620+0000 7faa9922d740 1 Processor -- start 2024-04-24T23:27:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.620+0000 7faa9922d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d53a0 0x55f23725dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.620+0000 7faa9922d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d7e50 0x55f23725d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.620+0000 7faa9922d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 0x55f2371aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.620+0000 7faa9922d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2371cc5c0 con 0x55f2371d7e50 2024-04-24T23:27:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.620+0000 7faa9922d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f2371c86c0 con 0x55f2371d8770 2024-04-24T23:27:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.620+0000 7faa9922d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55f2371ccc10 con 0x55f2371d53a0 2024-04-24T23:27:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.620+0000 7faa9922d740 10 client.0 _ll_register_callbacks cb 0x7faa88057e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d7e50 0x55f23725d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d7e50 0x55f23725d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60702/0 (socket says 172.21.15.26:60702) 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8f7fe640 1 -- 172.21.15.26:0/1569258401 learned_addr learned my addr 172.21.15.26:0/1569258401 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8ffff640 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d53a0 0x55f23725dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa949dc640 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 0x55f2371aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8f7fe640 1 -- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d53a0 msgr2=0x55f23725dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d53a0 0x55f23725dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8f7fe640 1 -- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 msgr2=0x55f2371aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 0x55f2371aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8f7fe640 1 -- 172.21.15.26:0/1569258401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2371aa5c0 con 0x55f2371d7e50 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa949dc640 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 0x55f2371aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8ffff640 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d53a0 0x55f23725dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:29.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa8f7fe640 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d7e50 0x55f23725d570 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7faa800018f0 tx=0x7faa900cf020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6731786da4ee8a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa6effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa6effd640 1 -- 172.21.15.26:0/1569258401 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa800047e0 con 0x55f2371d7e50 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa6effd640 1 -- 172.21.15.26:0/1569258401 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7faa80004980 con 0x55f2371d7e50 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa9922d740 1 -- 172.21.15.26:0/1569258401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa88070030 con 0x55f2371d7e50 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa6effd640 1 -- 172.21.15.26:0/1569258401 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7faa80004b80 con 0x55f2371d7e50 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa9922d740 10 client.8868 fetch_fsmap learned FSMap version 103 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa9922d740 1 -- 172.21.15.26:0/1569258401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) -- 0x55f2371aab80 con 0x55f2371d7e50 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.621+0000 7faa6effd640 1 -- 172.21.15.26:0/1569258401 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88070030 con 0x55f2371d7e50 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa6effd640 1 -- 172.21.15.26:0/1569258401 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 103) ==== 3494+0+0 (secure 0 0 0) 0x7faa8003d040 con 0x55f2371d7e50 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa9922d740 10 client.8868 fetch_fsmap finished waiting for FSMap version 103 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa9922d740 10 client.8868 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa9922d740 20 client.8868 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa9922d740 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55f2371cdff0 0x55f2371ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa9922d740 4 client.8868 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa9922d740 1 -- 172.21.15.26:0/1569258401 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "status"}) -- 0x55f23711b790 con 0x55f2371cdff0 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa8ffff640 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55f2371cdff0 0x55f2371ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa6effd640 10 client.8868 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.622+0000 7faa8ffff640 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55f2371cdff0 0x55f2371ce3d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2022b4eb0cd07af server_cookie=3ad916c81969ef1f in_seq=0 out_seq=0 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa6effd640 1 -- 172.21.15.26:0/1569258401 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55f23711b790 con 0x55f2371cdff0 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa6effd640 10 client.8868 handle_command_reply: tid=0 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 client.8868 shutdown 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 -- 172.21.15.26:0/1569258401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d7e50 msgr2=0x55f23725d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d7e50 0x55f23725d570 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7faa800018f0 tx=0x7faa900cf020 comp rx=0 tx=0).stop 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 -- 172.21.15.26:0/1569258401 shutdown_connections 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55f2371cdff0 0x55f2371ce3d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f2371d8770 0x55f2371aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2371d7e50 0x55f23725d570 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa6effd640 0 client.8868 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 --2- 172.21.15.26:0/1569258401 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55f2371d53a0 0x55f23725dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 -- 172.21.15.26:0/1569258401 >> 172.21.15.26:0/1569258401 conn(0x55f2371afb60 msgr2=0x55f2371deca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 -- 172.21.15.26:0/1569258401 shutdown_connections 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 1 -- 172.21.15.26:0/1569258401 wait complete. 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9922d740 20 client.8868 trim_cache size 0 max 16384 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:27:29.630 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8730, 2024-04-24T23:27:29.631 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:27:29.631 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:27:29.631 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:27:29.631 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.6874788369999996, 2024-04-24T23:27:29.631 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 103, 2024-04-24T23:27:29.631 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 96, 2024-04-24T23:27:29.631 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 96, 2024-04-24T23:27:29.631 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.695694696 2024-04-24T23:27:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 -- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa7c06be30 msgr2=0x7faa7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa7c06be30 0x7faa7c06e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faa84009a20 tx=0x7faa84008040 comp rx=0 tx=0).stop 2024-04-24T23:27:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 -- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9013cea0 msgr2=0x7faa9013ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9013cea0 0x7faa9013ec50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7faa8800f8d0 tx=0x7faa8800fd90 comp rx=0 tx=0).stop 2024-04-24T23:27:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 -- 172.21.15.26:0/2192020983 shutdown_connections 2024-04-24T23:27:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7faa7c06be30 0x7faa7c06e2f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa9013cea0 0x7faa9013ec50 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa9013ca20 0x7faa9013e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 --2- 172.21.15.26:0/2192020983 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7faa780040d0 0x7faa901f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.623+0000 7faa9643f640 1 -- 172.21.15.26:0/2192020983 >> 172.21.15.26:0/2192020983 conn(0x7faa9006b300 msgr2=0x7faa901432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.624+0000 7faa9643f640 1 -- 172.21.15.26:0/2192020983 shutdown_connections 2024-04-24T23:27:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.624+0000 7faa9643f640 1 -- 172.21.15.26:0/2192020983 wait complete. 2024-04-24T23:27:29.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:27:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.736+0000 7f07fbf12640 1 Processor -- start 2024-04-24T23:27:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.736+0000 7f07fbf12640 1 -- start start 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07fbf12640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07f415e290 0x7f07f40d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07fbf12640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f406d050 con 0x7f07f40b3ac0 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07fbf12640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f40b5840 con 0x7f07f40b72f0 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07fbf12640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f07f40b5e50 con 0x7f07f415e290 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07f9cae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07f415e290 0x7f07f40d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07f94ad640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f07f40b3ac0 0x7f07f406c230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58428/0 (socket says 172.21.15.26:58428) 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07f94ad640 1 -- 172.21.15.26:0/3263737148 learned_addr learned my addr 172.21.15.26:0/3263737148 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07f9cae640 1 -- 172.21.15.26:0/3263737148 >> v1:172.21.15.29:6789/0 conn(0x7f07f40b72f0 legacy=0x7f07f406c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07f9cae640 1 -- 172.21.15.26:0/3263737148 >> v1:172.21.15.26:6789/0 conn(0x7f07f40b3ac0 legacy=0x7f07f406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07f8cac640 1 -- 172.21.15.26:0/3263737148 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805258822 0 0) 0x7f07f406d050 con 0x7f07f40b3ac0 2024-04-24T23:27:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.737+0000 7f07f9cae640 1 -- 172.21.15.26:0/3263737148 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f40cfb30 con 0x7f07f415e290 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.738+0000 7f07f9cae640 1 --2- 172.21.15.26:0/3263737148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415e290 0x7f07f40d1730 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f07f00097a0 tx=0x7f07f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a9c3f14e5b0462 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.738+0000 7f07f8cac640 1 -- 172.21.15.26:0/3263737148 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0037070 con 0x7f07f415e290 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.738+0000 7f07f8cac640 1 -- 172.21.15.26:0/3263737148 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f0032440 con 0x7f07f415e290 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.738+0000 7f07f8cac640 1 -- 172.21.15.26:0/3263737148 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0032810 con 0x7f07f415e290 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.739+0000 7f07fbf12640 1 -- 172.21.15.26:0/3263737148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415e290 msgr2=0x7f07f40d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.739+0000 7f07fbf12640 1 --2- 172.21.15.26:0/3263737148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415e290 0x7f07f40d1730 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f07f00097a0 tx=0x7f07f002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.739+0000 7f07fbf12640 1 -- 172.21.15.26:0/3263737148 shutdown_connections 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.739+0000 7f07fbf12640 1 --2- 172.21.15.26:0/3263737148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415e290 0x7f07f40d1730 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.739+0000 7f07fbf12640 1 -- 172.21.15.26:0/3263737148 >> 172.21.15.26:0/3263737148 conn(0x7f07f406b120 msgr2=0x7f07f406b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.739+0000 7f07fbf12640 1 -- 172.21.15.26:0/3263737148 shutdown_connections 2024-04-24T23:27:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.739+0000 7f07fbf12640 1 -- 172.21.15.26:0/3263737148 wait complete. 2024-04-24T23:27:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.740+0000 7f07fbf12640 1 Processor -- start 2024-04-24T23:27:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.740+0000 7f07fbf12640 1 -- start start 2024-04-24T23:27:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07fbf12640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f40b3ac0 0x7f07f40c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07fbf12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40b72f0 0x7f07f40c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07fbf12640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e290 0x7f07f40b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07fbf12640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f07f40b5ae0 con 0x7f07f40b72f0 2024-04-24T23:27:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07fbf12640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f07f40d00f0 con 0x7f07f415e290 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07fbf12640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f07f415e670 con 0x7f07f40b3ac0 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f94ad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40b72f0 0x7f07f40c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f94ad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40b72f0 0x7f07f40c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60734/0 (socket says 172.21.15.26:60734) 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f94ad640 1 -- 172.21.15.26:0/537002792 learned_addr learned my addr 172.21.15.26:0/537002792 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f9cae640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f40b3ac0 0x7f07f40c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07fa4af640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e290 0x7f07f40b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f94ad640 1 -- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f40b3ac0 msgr2=0x7f07f40c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f94ad640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f40b3ac0 0x7f07f40c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f94ad640 1 -- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e290 msgr2=0x7f07f40b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f94ad640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e290 0x7f07f40b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f94ad640 1 -- 172.21.15.26:0/537002792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f41f4480 con 0x7f07f40b72f0 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f9cae640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f40b3ac0 0x7f07f40c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07fa4af640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e290 0x7f07f40b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07f94ad640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40b72f0 0x7f07f40c6020 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c4d0 tx=0x7f07e400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.741+0000 7f07ea7fc640 1 -- 172.21.15.26:0/537002792 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4018070 con 0x7f07f40b72f0 2024-04-24T23:27:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.742+0000 7f07ea7fc640 1 -- 172.21.15.26:0/537002792 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e4007500 con 0x7f07f40b72f0 2024-04-24T23:27:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.742+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f41f4760 con 0x7f07f40b72f0 2024-04-24T23:27:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.742+0000 7f07ea7fc640 1 -- 172.21.15.26:0/537002792 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4004a20 con 0x7f07f40b72f0 2024-04-24T23:27:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.742+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f40c82b0 con 0x7f07f40b72f0 2024-04-24T23:27:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.743+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f40c84b0 con 0x7f07f40b72f0 2024-04-24T23:27:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.743+0000 7f07ea7fc640 1 -- 172.21.15.26:0/537002792 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07e401b070 con 0x7f07f40b72f0 2024-04-24T23:27:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.743+0000 7f07ea7fc640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d806be30 0x7f07d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.743+0000 7f07ea7fc640 1 -- 172.21.15.26:0/537002792 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f07e408a1e0 con 0x7f07f40b72f0 2024-04-24T23:27:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.743+0000 7f07f9cae640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d806be30 0x7f07d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.744+0000 7f07f9cae640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d806be30 0x7f07d806e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f07f0034000 tx=0x7f07f002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.748+0000 7f07ea7fc640 1 -- 172.21.15.26:0/537002792 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07f40c84b0 con 0x7f07f40b72f0 2024-04-24T23:27:29.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:27:29.817 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:27:29.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.973+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f07f40694c0 con 0x7f07f40b72f0 2024-04-24T23:27:29.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.973+0000 7f07ea7fc640 1 -- 172.21.15.26:0/537002792 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) ==== 53+0+73 (secure 0 0 0) 0x7f07e4058b90 con 0x7f07f40b72f0 2024-04-24T23:27:29.974 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:27:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d806be30 msgr2=0x7f07d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d806be30 0x7f07d806e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f07f0034000 tx=0x7f07f002e000 comp rx=0 tx=0).stop 2024-04-24T23:27:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40b72f0 msgr2=0x7f07f40c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40b72f0 0x7f07f40c6020 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c4d0 tx=0x7f07e400c990 comp rx=0 tx=0).stop 2024-04-24T23:27:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 shutdown_connections 2024-04-24T23:27:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f07d806be30 0x7f07d806e2f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f07f415e290 0x7f07f40b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40b72f0 0x7f07f40c6020 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 --2- 172.21.15.26:0/537002792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f07f40b3ac0 0x7f07f40c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 >> 172.21.15.26:0/537002792 conn(0x7f07f406b120 msgr2=0x7f07f40b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 shutdown_connections 2024-04-24T23:27:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:29.975+0000 7f07fbf12640 1 -- 172.21.15.26:0/537002792 wait complete. 2024-04-24T23:27:29.987 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:27:29.987 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:27:29.987 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-24T23:27:29.987 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:27:29.987 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:27:29.987 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:27:29.987 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:27:29.987 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:27:29.987 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:27:29.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:29.987 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-24T23:27:30.003 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:27:30.003 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-24T23:27:30.004 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42343sec preferred_lft 42343sec 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:27:30.005 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ccf2:1eff:fe9b:4749/64 scope link 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c2:ea:e0:db:00:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c0ea:e0ff:fedb:e0/64 scope link 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 1e:82:d0:fa:5b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::1c82:d0ff:fefa:5b3f/64 scope link 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.007 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:27:30.067 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-24T23:27:30.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.068 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-24T23:27:30.124 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-24T23:27:30.124 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:27:30.124 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:27:30.124 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:27:30.124 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:27:30.125 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:27:30.125 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-24T23:27:30.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:27:30.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:27:30.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:27:30.125 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-24T23:27:30.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:27:30.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:30.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.250 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:27:30.265 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:27:30.266 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:27:30.266 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:27:30.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.266 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-24T23:27:30.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:27:30.330 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:27:30.385 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:27:30.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:27:30.386 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:27:30.440 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:27:30.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:27:30.523 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:27:30.523 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:27:30.523 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:27:30.531 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:27:30.531 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:27:30.531 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:27:30.532 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:27:30.532 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:27:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:27:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:27:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-24T23:27:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.29:3300,172.21.15.29:3301" 2024-04-24T23:27:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:27:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790" 2024-04-24T23:27:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.29:6789,172.21.15.29:6790:/ 2024-04-24T23:27:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5a872e4b-97fd-4255-95a5-3ea24d1048cf" will pass to kernel 2024-04-24T23:27:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:27:30.538 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:27:30.538 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:27:30.538 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:27:30.538 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:27:30.539 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:27:30.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.539 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:27:30.604 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:27:30.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:27:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:30.707 INFO:teuthology.orchestra.run.smithi026.stdout:client7552 2024-04-24T23:27:30.707 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:27:30.707 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:30.723 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T23:27:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.866+0000 7fb795fa9640 1 Processor -- start 2024-04-24T23:27:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.866+0000 7fb795fa9640 1 -- start start 2024-04-24T23:27:30.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.867+0000 7fb795fa9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb79015e750 0x7fb7900cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.867+0000 7fb795fa9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb7900d2c30 0x7fb7900cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.867+0000 7fb795fa9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7900d2120 con 0x7fb79015e370 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.867+0000 7fb795fa9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb7900d1510 con 0x7fb7900d2c30 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.867+0000 7fb795fa9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb7900d1ed0 con 0x7fb79015e750 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.867+0000 7fb78ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb7900d2c30 0x7fb7900cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.867+0000 7fb78ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb7900d2c30 0x7fb7900cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60744/0 (socket says 172.21.15.26:60744) 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.867+0000 7fb78ffff640 1 -- 172.21.15.26:0/415884392 learned_addr learned my addr 172.21.15.26:0/415884392 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.868+0000 7fb78effd640 1 --2- 172.21.15.26:0/415884392 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb79015e750 0x7fb7900cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.868+0000 7fb78ffff640 1 -- 172.21.15.26:0/415884392 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb79015e750 msgr2=0x7fb7900cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.868+0000 7fb78ffff640 1 --2- 172.21.15.26:0/415884392 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb79015e750 0x7fb7900cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.868+0000 7fb78ffff640 1 -- 172.21.15.26:0/415884392 >> v1:172.21.15.29:6789/0 conn(0x7fb79015e370 legacy=0x7fb7900d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.868+0000 7fb78ffff640 1 -- 172.21.15.26:0/415884392 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7900b3740 con 0x7fb7900d2c30 2024-04-24T23:27:30.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.868+0000 7fb78ffff640 1 --2- 172.21.15.26:0/415884392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 0x7fb7900cf1d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fb7900d2050 tx=0x7fb78802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26505d5033316df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:30.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb78e7fc640 1 -- 172.21.15.26:0/415884392 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788036070 con 0x7fb7900d2c30 2024-04-24T23:27:30.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb78e7fc640 1 -- 172.21.15.26:0/415884392 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb788031600 con 0x7fb7900d2c30 2024-04-24T23:27:30.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb78e7fc640 1 -- 172.21.15.26:0/415884392 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7880319d0 con 0x7fb7900d2c30 2024-04-24T23:27:30.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb795fa9640 1 -- 172.21.15.26:0/415884392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 msgr2=0x7fb7900cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:30.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb795fa9640 1 --2- 172.21.15.26:0/415884392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 0x7fb7900cf1d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fb7900d2050 tx=0x7fb78802a710 comp rx=0 tx=0).stop 2024-04-24T23:27:30.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb795fa9640 1 -- 172.21.15.26:0/415884392 shutdown_connections 2024-04-24T23:27:30.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb795fa9640 1 --2- 172.21.15.26:0/415884392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 0x7fb7900cf1d0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb795fa9640 1 --2- 172.21.15.26:0/415884392 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb79015e750 0x7fb7900cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb795fa9640 1 -- 172.21.15.26:0/415884392 >> 172.21.15.26:0/415884392 conn(0x7fb7900d40f0 msgr2=0x7fb7900d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb795fa9640 1 -- 172.21.15.26:0/415884392 shutdown_connections 2024-04-24T23:27:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.869+0000 7fb795fa9640 1 -- 172.21.15.26:0/415884392 wait complete. 2024-04-24T23:27:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.870+0000 7fb795fa9640 1 Processor -- start 2024-04-24T23:27:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb795fa9640 1 -- start start 2024-04-24T23:27:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb795fa9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 0x7fb7901f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb795fa9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb79015e370 0x7fb79015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb795fa9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb7900b76f0 0x7fb7900ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb795fa9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7900b5230 con 0x7fb7900d2c30 2024-04-24T23:27:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb795fa9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb7900b3d00 con 0x7fb79015e370 2024-04-24T23:27:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb795fa9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb7900b53b0 con 0x7fb7900b76f0 2024-04-24T23:27:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb78f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 0x7fb7901f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:30.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb78f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 0x7fb7901f3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60758/0 (socket says 172.21.15.26:60758) 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb78f7fe640 1 -- 172.21.15.26:0/3225165795 learned_addr learned my addr 172.21.15.26:0/3225165795 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb78ffff640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb7900b76f0 0x7fb7900ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.871+0000 7fb78effd640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb79015e370 0x7fb79015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb78f7fe640 1 -- 172.21.15.26:0/3225165795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb7900b76f0 msgr2=0x7fb7900ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb78f7fe640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb7900b76f0 0x7fb7900ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb78f7fe640 1 -- 172.21.15.26:0/3225165795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb79015e370 msgr2=0x7fb79015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb78f7fe640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb79015e370 0x7fb79015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb78f7fe640 1 -- 172.21.15.26:0/3225165795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7900b7f70 con 0x7fb7900d2c30 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb78f7fe640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 0x7fb7901f3f10 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c430 tx=0x7fb78000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb777fff640 1 -- 172.21.15.26:0/3225165795 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7800052d0 con 0x7fb7900d2c30 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb777fff640 1 -- 172.21.15.26:0/3225165795 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780005490 con 0x7fb7900d2c30 2024-04-24T23:27:30.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb777fff640 1 -- 172.21.15.26:0/3225165795 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780007260 con 0x7fb7900d2c30 2024-04-24T23:27:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7900b8250 con 0x7fb7900d2c30 2024-04-24T23:27:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.872+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7900b8810 con 0x7fb7900d2c30 2024-04-24T23:27:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.873+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb790075610 con 0x7fb7900d2c30 2024-04-24T23:27:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.874+0000 7fb777fff640 1 -- 172.21.15.26:0/3225165795 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb780007400 con 0x7fb7900d2c30 2024-04-24T23:27:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.874+0000 7fb777fff640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb77c06be60 0x7fb77c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.874+0000 7fb777fff640 1 -- 172.21.15.26:0/3225165795 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fb78008a240 con 0x7fb7900d2c30 2024-04-24T23:27:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.874+0000 7fb78effd640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb77c06be60 0x7fb77c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.874+0000 7fb78effd640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb77c06be60 0x7fb77c06e320 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb7900c8070 tx=0x7fb784009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:30.878+0000 7fb777fff640 1 -- 172.21.15.26:0/3225165795 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb780058630 con 0x7fb7900d2c30 2024-04-24T23:27:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.108+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fb7900b89e0 con 0x7fb7900d2c30 2024-04-24T23:27:31.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.964+0000 7fb777fff640 1 -- 172.21.15.26:0/3225165795 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v104) ==== 105+0+0 (secure 0 0 0) 0x7fb78005c930 con 0x7fb7900d2c30 2024-04-24T23:27:31.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb77c06be60 msgr2=0x7fb77c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:31.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb77c06be60 0x7fb77c06e320 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb7900c8070 tx=0x7fb784009190 comp rx=0 tx=0).stop 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 msgr2=0x7fb7901f3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 0x7fb7901f3f10 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c430 tx=0x7fb78000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 shutdown_connections 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb77c06be60 0x7fb77c06e320 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb7900b76f0 0x7fb7900ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb79015e370 0x7fb79015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 --2- 172.21.15.26:0/3225165795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7900d2c30 0x7fb7901f3f10 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 >> 172.21.15.26:0/3225165795 conn(0x7fb7900d40f0 msgr2=0x7fb7900d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 shutdown_connections 2024-04-24T23:27:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:31.967+0000 7fb795fa9640 1 -- 172.21.15.26:0/3225165795 wait complete. 2024-04-24T23:27:31.988 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.094+0000 7f4a4576a640 1 Processor -- start 2024-04-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.094+0000 7f4a4576a640 1 -- start start 2024-04-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a4576a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f4a400cf870 0x7f4a40072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a4576a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a400cd0b0 con 0x7f4a40143c40 2024-04-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a4576a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a40071240 con 0x7f4a4015eb00 2024-04-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a4576a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f4a40071b70 con 0x7f4a400cf870 2024-04-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a3effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4a40143c40 0x7f4a400cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58434/0 (socket says 172.21.15.26:58434) 2024-04-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a3effd640 1 -- 172.21.15.26:0/602407517 learned_addr learned my addr 172.21.15.26:0/602407517 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a367fc640 1 --2- 172.21.15.26:0/602407517 >> v2:172.21.15.29:3300/0 conn(0x7f4a400cf870 0x7f4a40072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014906452 0 0) 0x7f4a400cd0b0 con 0x7f4a40143c40 2024-04-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a2c003520 con 0x7f4a40143c40 2024-04-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3962767959 0 0) 0x7f4a2c003520 con 0x7f4a40143c40 2024-04-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 >> v2:172.21.15.29:3300/0 conn(0x7f4a400cf870 msgr2=0x7f4a40072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.095+0000 7f4a3e7fc640 1 --2- 172.21.15.26:0/602407517 >> v2:172.21.15.29:3300/0 conn(0x7f4a400cf870 0x7f4a40072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 >> v1:172.21.15.29:6789/0 conn(0x7f4a4015eb00 legacy=0x7f4a4015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a40071520 con 0x7f4a40143c40 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f4a30001670 con 0x7f4a40143c40 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 >> v1:172.21.15.26:6789/0 conn(0x7f4a40143c40 legacy=0x7f4a400cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0039b0 0x7f4a2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c004800 0x7f4a2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c0051c0 0x7f4a2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40071b70 con 0x7f4a2c0051c0 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c00c060 con 0x7f4a2c0039b0 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c00c260 con 0x7f4a2c004800 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3f7fe640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c0051c0 0x7f4a2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3effd640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0039b0 0x7f4a2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a367fc640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c004800 0x7f4a2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3f7fe640 1 -- 172.21.15.26:0/602407517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c004800 msgr2=0x7f4a2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3f7fe640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c004800 0x7f4a2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3f7fe640 1 -- 172.21.15.26:0/602407517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0039b0 msgr2=0x7f4a2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3f7fe640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a2c0039b0 0x7f4a2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a3f7fe640 1 -- 172.21.15.26:0/602407517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c003520 con 0x7f4a2c0051c0 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.096+0000 7f4a367fc640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c004800 0x7f4a2c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.097+0000 7f4a3f7fe640 1 -- 172.21.15.26:0/602407517 reap_dead start 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.097+0000 7f4a3f7fe640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c0051c0 0x7f4a2c00b9c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f4a38001e50 tx=0x7f4a2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667e8f9e64150f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.097+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38001a70 con 0x7f4a2c0051c0 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.097+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a38003970 con 0x7f4a2c0051c0 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.097+0000 7f4a3e7fc640 1 -- 172.21.15.26:0/602407517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38003d60 con 0x7f4a2c0051c0 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.097+0000 7f4a4576a640 1 -- 172.21.15.26:0/602407517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c0051c0 msgr2=0x7f4a2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.097+0000 7f4a4576a640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c0051c0 0x7f4a2c00b9c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f4a38001e50 tx=0x7f4a2c005830 comp rx=0 tx=0).stop 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.097+0000 7f4a4576a640 1 -- 172.21.15.26:0/602407517 shutdown_connections 2024-04-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.097+0000 7f4a4576a640 1 --2- 172.21.15.26:0/602407517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c0051c0 0x7f4a2c00b9c0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.098+0000 7f4a4576a640 1 -- 172.21.15.26:0/602407517 >> 172.21.15.26:0/602407517 conn(0x7f4a4006b300 msgr2=0x7f4a4006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.098+0000 7f4a4576a640 1 -- 172.21.15.26:0/602407517 shutdown_connections 2024-04-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.098+0000 7f4a4576a640 1 -- 172.21.15.26:0/602407517 wait complete. 2024-04-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.099+0000 7f4a4576a640 1 Processor -- start 2024-04-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.099+0000 7f4a4576a640 1 -- start start 2024-04-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a4576a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c0051c0 0x7f4a4013ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a4576a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a4014d0b0 0x7f4a400a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a4576a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4014d530 0x7f4a400a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a4576a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4015ef10 con 0x7f4a4014d0b0 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a4576a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40144020 con 0x7f4a4014d530 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a4576a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4015f090 con 0x7f4a2c0051c0 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a4014d0b0 0x7f4a400a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a4014d0b0 0x7f4a400a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60770/0 (socket says 172.21.15.26:60770) 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a367fc640 1 -- 172.21.15.26:0/472209402 learned_addr learned my addr 172.21.15.26:0/472209402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a3effd640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c0051c0 0x7f4a4013ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a367fc640 1 -- 172.21.15.26:0/472209402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c0051c0 msgr2=0x7f4a4013ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a3f7fe640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4014d530 0x7f4a400a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a367fc640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c0051c0 0x7f4a4013ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a367fc640 1 -- 172.21.15.26:0/472209402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4014d530 msgr2=0x7f4a400a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a367fc640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4014d530 0x7f4a400a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a367fc640 1 -- 172.21.15.26:0/472209402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4015cd10 con 0x7f4a4014d0b0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a3f7fe640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4014d530 0x7f4a400a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a367fc640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a4014d0b0 0x7f4a400a9460 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f4a28011540 tx=0x7f4a28011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.100+0000 7f4a37fff640 1 -- 172.21.15.26:0/472209402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28015870 con 0x7f4a4014d0b0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.101+0000 7f4a37fff640 1 -- 172.21.15.26:0/472209402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a28015a10 con 0x7f4a4014d0b0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.101+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4015cff0 con 0x7f4a4014d0b0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.101+0000 7f4a37fff640 1 -- 172.21.15.26:0/472209402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28015dc0 con 0x7f4a4014d0b0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.101+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4015d5b0 con 0x7f4a4014d0b0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.102+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a400694c0 con 0x7f4a4014d0b0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.102+0000 7f4a37fff640 1 -- 172.21.15.26:0/472209402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a2800f460 con 0x7f4a4014d0b0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.102+0000 7f4a37fff640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4a2006be30 0x7f4a2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.103+0000 7f4a37fff640 1 -- 172.21.15.26:0/472209402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f4a2808b760 con 0x7f4a4014d0b0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.103+0000 7f4a3effd640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4a2006be30 0x7f4a2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.103+0000 7f4a3effd640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4a2006be30 0x7f4a2006e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4a30007f90 tx=0x7f4a30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.107+0000 7f4a37fff640 1 -- 172.21.15.26:0/472209402 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4a28019030 con 0x7f4a4014d0b0 2024-04-24T23:27:32.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.331+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a400b2930 con 0x7f4a4014d0b0 2024-04-24T23:27:32.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.332+0000 7f4a37fff640 1 -- 172.21.15.26:0/472209402 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5202 (secure 0 0 0) 0x7f4a2805a090 con 0x7f4a4014d0b0 2024-04-24T23:27:32.333 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:32.333 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":105,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":7444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:31.967420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":8730,"mds_1":8712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:creating","state_seq":1,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730]},"id":7}]} 2024-04-24T23:27:32.333 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 105 2024-04-24T23:27:32.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.333+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4a2006be30 msgr2=0x7f4a2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.333+0000 7f4a4576a640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4a2006be30 0x7f4a2006e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4a30007f90 tx=0x7f4a30009040 comp rx=0 tx=0).stop 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.333+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a4014d0b0 msgr2=0x7f4a400a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.333+0000 7f4a4576a640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a4014d0b0 0x7f4a400a9460 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f4a28011540 tx=0x7f4a28011a00 comp rx=0 tx=0).stop 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.334+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 shutdown_connections 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.334+0000 7f4a4576a640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4a2006be30 0x7f4a2006e2f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.334+0000 7f4a4576a640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4014d530 0x7f4a400a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.334+0000 7f4a4576a640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a4014d0b0 0x7f4a400a9460 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.334+0000 7f4a4576a640 1 --2- 172.21.15.26:0/472209402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4a2c0051c0 0x7f4a4013ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.334+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 >> 172.21.15.26:0/472209402 conn(0x7f4a4006b300 msgr2=0x7f4a400b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.334+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 shutdown_connections 2024-04-24T23:27:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:32.334+0000 7f4a4576a640 1 -- 172.21.15.26:0/472209402 wait complete. 2024-04-24T23:27:32.748 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:27:23.297118+0000', 'modified': '2024-04-24T23:27:31.967420+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 8730, 'mds_1': 8712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8712': {'gid': 8712, 'name': 'c', 'rank': 1, 'incarnation': 105, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.26:6835/3062763802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 3062763802}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3062763802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8730': {'gid': 8730, 'name': 'd', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/2225348762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 2225348762}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 2225348762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8730, 'qdb_cluster': [8730]} 2024-04-24T23:27:32.748 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8712:up:creating 2024-04-24T23:27:33.749 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.862+0000 7f360c94f640 1 Processor -- start 2024-04-24T23:27:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.862+0000 7f360c94f640 1 -- start start 2024-04-24T23:27:33.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360c94f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3608145910 0x7f360806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360c94f640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f360806d350 0x7f360815d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360c94f640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f360815d570 con 0x7f36080cd660 2024-04-24T23:27:33.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360c94f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3608145cf0 con 0x7f3608145910 2024-04-24T23:27:33.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360c94f640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f36080cfbc0 con 0x7f360806d350 2024-04-24T23:27:33.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3608145910 0x7f360806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3608145910 0x7f360806e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35056/0 (socket says 172.21.15.26:35056) 2024-04-24T23:27:33.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360659c640 1 -- 172.21.15.26:0/4286085896 learned_addr learned my addr 172.21.15.26:0/4286085896 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:33.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f35fffff640 1 --2- 172.21.15.26:0/4286085896 >> v2:172.21.15.29:3300/0 conn(0x7f360806d350 0x7f360815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360659c640 1 -- 172.21.15.26:0/4286085896 >> v2:172.21.15.29:3300/0 conn(0x7f360806d350 msgr2=0x7f360815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:33.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360659c640 1 --2- 172.21.15.26:0/4286085896 >> v2:172.21.15.29:3300/0 conn(0x7f360806d350 0x7f360815d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360659c640 1 -- 172.21.15.26:0/4286085896 >> v1:172.21.15.29:6789/0 conn(0x7f36080cd660 legacy=0x7f360815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.863+0000 7f360659c640 1 -- 172.21.15.26:0/4286085896 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360806d7f0 con 0x7f3608145910 2024-04-24T23:27:33.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.864+0000 7f360659c640 1 --2- 172.21.15.26:0/4286085896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3608145910 0x7f360806e890 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009870 tx=0x7f35f802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fbb2dafb49879f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:33.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.864+0000 7f3605d9b640 1 -- 172.21.15.26:0/4286085896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8037070 con 0x7f3608145910 2024-04-24T23:27:33.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.864+0000 7f3605d9b640 1 -- 172.21.15.26:0/4286085896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f8032440 con 0x7f3608145910 2024-04-24T23:27:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.864+0000 7f3605d9b640 1 -- 172.21.15.26:0/4286085896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8032810 con 0x7f3608145910 2024-04-24T23:27:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.865+0000 7f360c94f640 1 -- 172.21.15.26:0/4286085896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3608145910 msgr2=0x7f360806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.865+0000 7f360c94f640 1 --2- 172.21.15.26:0/4286085896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3608145910 0x7f360806e890 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009870 tx=0x7f35f802a210 comp rx=0 tx=0).stop 2024-04-24T23:27:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.865+0000 7f360c94f640 1 -- 172.21.15.26:0/4286085896 shutdown_connections 2024-04-24T23:27:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.865+0000 7f360c94f640 1 --2- 172.21.15.26:0/4286085896 >> v2:172.21.15.29:3300/0 conn(0x7f360806d350 0x7f360815d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.865+0000 7f360c94f640 1 --2- 172.21.15.26:0/4286085896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3608145910 0x7f360806e890 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.865+0000 7f360c94f640 1 -- 172.21.15.26:0/4286085896 >> 172.21.15.26:0/4286085896 conn(0x7f360806b300 msgr2=0x7f360806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.865+0000 7f360c94f640 1 -- 172.21.15.26:0/4286085896 shutdown_connections 2024-04-24T23:27:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.865+0000 7f360c94f640 1 -- 172.21.15.26:0/4286085896 wait complete. 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.866+0000 7f360c94f640 1 Processor -- start 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.866+0000 7f360c94f640 1 -- start start 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f360c94f640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f360806d350 0x7f360815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f360c94f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36080cd660 0x7f36080b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f360c94f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3608145910 0x7f3608141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f360c94f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f360806e090 con 0x7f36080cd660 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f360c94f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f360806ddb0 con 0x7f3608145910 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f360c94f640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f360806e370 con 0x7f360806d350 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f35fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36080cd660 0x7f36080b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f35fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36080cd660 0x7f36080b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35062/0 (socket says 172.21.15.26:35062) 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f35fffff640 1 -- 172.21.15.26:0/2915907201 learned_addr learned my addr 172.21.15.26:0/2915907201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f360659c640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f360806d350 0x7f360815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f3606d9d640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3608145910 0x7f3608141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.867+0000 7f35fffff640 1 -- 172.21.15.26:0/2915907201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f360806d350 msgr2=0x7f360815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f35fffff640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f360806d350 0x7f360815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f35fffff640 1 -- 172.21.15.26:0/2915907201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3608145910 msgr2=0x7f3608141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f35fffff640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3608145910 0x7f3608141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f35fffff640 1 -- 172.21.15.26:0/2915907201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36081e4690 con 0x7f36080cd660 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f360659c640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f360806d350 0x7f360815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f35fffff640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36080cd660 0x7f36080b4bb0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f35f000e9b0 tx=0x7f35f000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f35feffd640 1 -- 172.21.15.26:0/2915907201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0017070 con 0x7f36080cd660 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f35feffd640 1 -- 172.21.15.26:0/2915907201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f000c9a0 con 0x7f36080cd660 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f35feffd640 1 -- 172.21.15.26:0/2915907201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f000cd90 con 0x7f36080cd660 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36081e4970 con 0x7f36080cd660 2024-04-24T23:27:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.868+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3608159080 con 0x7f36080cd660 2024-04-24T23:27:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.869+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36080694c0 con 0x7f36080cd660 2024-04-24T23:27:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.869+0000 7f35feffd640 1 -- 172.21.15.26:0/2915907201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35f001a070 con 0x7f36080cd660 2024-04-24T23:27:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.870+0000 7f35feffd640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35e806be30 0x7f35e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.870+0000 7f35feffd640 1 -- 172.21.15.26:0/2915907201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f35f008b140 con 0x7f36080cd660 2024-04-24T23:27:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.870+0000 7f360659c640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35e806be30 0x7f35e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.870+0000 7f360659c640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35e806be30 0x7f35e806e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f35f8034000 tx=0x7f35f802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:33.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:33.874+0000 7f35feffd640 1 -- 172.21.15.26:0/2915907201 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f35f0059510 con 0x7f36080cd660 2024-04-24T23:27:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.097+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36080b17f0 con 0x7f36080cd660 2024-04-24T23:27:34.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.098+0000 7f35feffd640 1 -- 172.21.15.26:0/2915907201 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5225 (secure 0 0 0) 0x7f35f005d810 con 0x7f36080cd660 2024-04-24T23:27:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:34.099 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":108,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:33.973136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[7444,8712,8730]},"id":7}]} 2024-04-24T23:27:34.099 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 108 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35e806be30 msgr2=0x7f35e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35e806be30 0x7f35e806e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f35f8034000 tx=0x7f35f802e000 comp rx=0 tx=0).stop 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36080cd660 msgr2=0x7f36080b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36080cd660 0x7f36080b4bb0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f35f000e9b0 tx=0x7f35f000ee70 comp rx=0 tx=0).stop 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 shutdown_connections 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f35e806be30 0x7f35e806e2f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3608145910 0x7f3608141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36080cd660 0x7f36080b4bb0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 --2- 172.21.15.26:0/2915907201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f360806d350 0x7f360815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 >> 172.21.15.26:0/2915907201 conn(0x7f360806b300 msgr2=0x7f36080b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 shutdown_connections 2024-04-24T23:27:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.100+0000 7f360c94f640 1 -- 172.21.15.26:0/2915907201 wait complete. 2024-04-24T23:27:34.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:27:23.297118+0000', 'modified': '2024-04-24T23:27:33.973136+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 8730, 'mds_1': 8712, 'mds_2': 7444}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7444': {'gid': 7444, 'name': 'b', 'rank': 2, 'incarnation': 107, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.29:6834/2632309869', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6832', 'nonce': 2632309869}, {'type': 'v1', 'addr': '172.21.15.29:6834', 'nonce': 2632309869}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8712': {'gid': 8712, 'name': 'c', 'rank': 1, 'incarnation': 105, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6835/3062763802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 3062763802}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3062763802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8730': {'gid': 8730, 'name': 'd', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.29:6835/2225348762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6833', 'nonce': 2225348762}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 2225348762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8730, 'qdb_cluster': [7444, 8712, 8730]} 2024-04-24T23:27:34.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:27:34.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:27:34.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.620+0000 7fea3bff9640 1 Processor -- start 2024-04-24T23:27:34.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.621+0000 7fea3bff9640 1 -- start start 2024-04-24T23:27:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.621+0000 7fea3bff9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea340adf80 0x7fea34154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.621+0000 7fea3bff9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea340b0220 con 0x7fea3406f4f0 2024-04-24T23:27:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.621+0000 7fea3bff9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea340af180 con 0x7fea3406f110 2024-04-24T23:27:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.621+0000 7fea3bff9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fea340af460 con 0x7fea340adf80 2024-04-24T23:27:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.621+0000 7fea3a596640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea340adf80 0x7fea34154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.622+0000 7fea39594640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fea3406f4f0 0x7fea340ad870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44410/0 (socket says 172.21.15.26:44410) 2024-04-24T23:27:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.622+0000 7fea39594640 1 -- 172.21.15.26:0/243692741 learned_addr learned my addr 172.21.15.26:0/243692741 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.622+0000 7fea3a596640 1 -- 172.21.15.26:0/243692741 >> v1:172.21.15.29:6789/0 conn(0x7fea3406f110 legacy=0x7fea3415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.622+0000 7fea3a596640 1 -- 172.21.15.26:0/243692741 >> v1:172.21.15.26:6789/0 conn(0x7fea3406f4f0 legacy=0x7fea340ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.622+0000 7fea3a596640 1 -- 172.21.15.26:0/243692741 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea340af740 con 0x7fea340adf80 2024-04-24T23:27:34.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.622+0000 7fea38d93640 1 -- 172.21.15.26:0/243692741 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4101587793 0 0) 0x7fea340b0220 con 0x7fea3406f4f0 2024-04-24T23:27:34.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea3a596640 1 --2- 172.21.15.26:0/243692741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 0x7fea34154a50 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009d90 tx=0x7fea2c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b124baac0026bc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea38d93640 1 -- 172.21.15.26:0/243692741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c007740 con 0x7fea340adf80 2024-04-24T23:27:34.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea38d93640 1 -- 172.21.15.26:0/243692741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2c0078e0 con 0x7fea340adf80 2024-04-24T23:27:34.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea38d93640 1 -- 172.21.15.26:0/243692741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c007cb0 con 0x7fea340adf80 2024-04-24T23:27:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea3bff9640 1 -- 172.21.15.26:0/243692741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 msgr2=0x7fea34154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea3bff9640 1 --2- 172.21.15.26:0/243692741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 0x7fea34154a50 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009d90 tx=0x7fea2c02a660 comp rx=0 tx=0).stop 2024-04-24T23:27:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea3bff9640 1 -- 172.21.15.26:0/243692741 shutdown_connections 2024-04-24T23:27:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea3bff9640 1 --2- 172.21.15.26:0/243692741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 0x7fea34154a50 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea3bff9640 1 -- 172.21.15.26:0/243692741 >> 172.21.15.26:0/243692741 conn(0x7fea340729a0 msgr2=0x7fea3406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea3bff9640 1 -- 172.21.15.26:0/243692741 shutdown_connections 2024-04-24T23:27:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.623+0000 7fea3bff9640 1 -- 172.21.15.26:0/243692741 wait complete. 2024-04-24T23:27:34.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.624+0000 7fea3bff9640 1 Processor -- start 2024-04-24T23:27:34.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3bff9640 1 -- start start 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3bff9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea3406f110 0x7fea3415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3bff9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fea3406f4f0 0x7fea3414c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3bff9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 0x7fea3415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3bff9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea340aee50 con 0x7fea340adf80 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3bff9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fea340afd00 con 0x7fea3406f110 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3bff9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fea3406fb00 con 0x7fea3406f4f0 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3a596640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 0x7fea3415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3a596640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 0x7fea3415d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35074/0 (socket says 172.21.15.26:35074) 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea3a596640 1 -- 172.21.15.26:0/600109762 learned_addr learned my addr 172.21.15.26:0/600109762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea39594640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fea3406f4f0 0x7fea3414c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.625+0000 7fea39d95640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea3406f110 0x7fea3415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea3a596640 1 -- 172.21.15.26:0/600109762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fea3406f4f0 msgr2=0x7fea3414c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea3a596640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fea3406f4f0 0x7fea3414c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea3a596640 1 -- 172.21.15.26:0/600109762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea3406f110 msgr2=0x7fea3415b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea3a596640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea3406f110 0x7fea3415b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea3a596640 1 -- 172.21.15.26:0/600109762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3415a730 con 0x7fea340adf80 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea3a596640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 0x7fea3415d400 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009eb0 tx=0x7fea2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea2a7fc640 1 -- 172.21.15.26:0/600109762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c03f070 con 0x7fea340adf80 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea2a7fc640 1 -- 172.21.15.26:0/600109762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2c031c90 con 0x7fea340adf80 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea3bff9640 1 -- 172.21.15.26:0/600109762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3415aa10 con 0x7fea340adf80 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea2a7fc640 1 -- 172.21.15.26:0/600109762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c043630 con 0x7fea340adf80 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.626+0000 7fea3bff9640 1 -- 172.21.15.26:0/600109762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3415afd0 con 0x7fea340adf80 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.628+0000 7fea2a7fc640 1 -- 172.21.15.26:0/600109762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea2c03b410 con 0x7fea340adf80 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.628+0000 7fea2a7fc640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea1806be30 0x7fea1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.628+0000 7fea2a7fc640 1 -- 172.21.15.26:0/600109762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fea2c0ac270 con 0x7fea340adf80 2024-04-24T23:27:34.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.628+0000 7fea39d95640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea1806be30 0x7fea1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.629+0000 7fea39d95640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea1806be30 0x7fea1806e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fea30000f60 tx=0x7fea30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.629+0000 7fea3ede2740 1 Processor -- start 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.629+0000 7fea3ede2740 1 -- start start 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5540 0x55df2118a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea3ede2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55df210edf00 con 0x55df211d2b60 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea3ede2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55df21096750 con 0x55df211b5540 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea3ede2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55df210e9ec0 con 0x55df211b5c80 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39d95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39d95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211854f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35090/0 (socket says 172.21.15.26:35090) 2024-04-24T23:27:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39d95640 1 -- 172.21.15.26:0/1633144162 learned_addr learned my addr 172.21.15.26:0/1633144162 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea3a596640 1 --2- 172.21.15.26:0/1633144162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5540 0x55df2118a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39594640 1 --2- 172.21.15.26:0/1633144162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39d95640 1 -- 172.21.15.26:0/1633144162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 msgr2=0x55df211b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39d95640 1 --2- 172.21.15.26:0/1633144162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39d95640 1 -- 172.21.15.26:0/1633144162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5540 msgr2=0x55df2118a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39d95640 1 --2- 172.21.15.26:0/1633144162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5540 0x55df2118a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39d95640 1 -- 172.21.15.26:0/1633144162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df20e022d0 con 0x55df211d2b60 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.630+0000 7fea39d95640 1 --2- 172.21.15.26:0/1633144162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211854f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fea3000fbd0 tx=0x7fea30012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd30e570558e46fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea177fe640 1 -- 172.21.15.26:0/1633144162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30025070 con 0x55df211d2b60 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea177fe640 1 -- 172.21.15.26:0/1633144162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3001b050 con 0x55df211d2b60 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea177fe640 1 -- 172.21.15.26:0/1633144162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30020410 con 0x55df211d2b60 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea3ede2740 1 -- 172.21.15.26:0/1633144162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 msgr2=0x55df211854f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea3ede2740 1 --2- 172.21.15.26:0/1633144162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211854f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fea3000fbd0 tx=0x7fea30012000 comp rx=0 tx=0).stop 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea3ede2740 1 -- 172.21.15.26:0/1633144162 shutdown_connections 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea3ede2740 1 --2- 172.21.15.26:0/1633144162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5540 0x55df2118a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea3ede2740 1 --2- 172.21.15.26:0/1633144162 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea3ede2740 1 --2- 172.21.15.26:0/1633144162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211854f0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea3ede2740 1 -- 172.21.15.26:0/1633144162 >> 172.21.15.26:0/1633144162 conn(0x55df21187760 msgr2=0x55df211d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea3ede2740 1 -- 172.21.15.26:0/1633144162 shutdown_connections 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.631+0000 7fea3ede2740 1 -- 172.21.15.26:0/1633144162 wait complete. 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.632+0000 7fea3ede2740 1 -- start start 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.632+0000 7fea3ede2740 1 Processor -- start 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211c9000 0x55df211b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea3ede2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55df21189910 con 0x55df211d2b60 2024-04-24T23:27:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea3ede2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55df210a9cf0 con 0x55df211c9000 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea3ede2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55df21188d00 con 0x55df211b5c80 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea3ede2740 10 client.0 _ll_register_callbacks cb 0x55df210a96e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39594640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39d95640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea3a596640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211c9000 0x55df211b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39594640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35094/0 (socket says 172.21.15.26:35094) 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39594640 1 -- 172.21.15.26:0/3070279464 learned_addr learned my addr 172.21.15.26:0/3070279464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39d95640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:47178/0 (socket says 172.21.15.26:47178) 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39594640 1 -- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 msgr2=0x55df211b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39594640 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39594640 1 -- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211c9000 msgr2=0x55df211b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39594640 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211c9000 0x55df211b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39594640 1 -- 172.21.15.26:0/3070279464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df20ecc6b0 con 0x55df211d2b60 2024-04-24T23:27:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39d95640 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea39594640 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211c8ac0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fea24004e80 tx=0x7fea24001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a665612bfa0a5297 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.633+0000 7fea14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea14ff9640 1 -- 172.21.15.26:0/3070279464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24015650 con 0x55df211d2b60 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea14ff9640 1 -- 172.21.15.26:0/3070279464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fea24010040 con 0x55df211d2b60 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea3ede2740 1 -- 172.21.15.26:0/3070279464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3001b050 con 0x55df211d2b60 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea14ff9640 1 -- 172.21.15.26:0/3070279464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fea240157f0 con 0x55df211d2b60 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea3ede2740 10 client.8910 fetch_fsmap learned FSMap version 108 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea3ede2740 1 -- 172.21.15.26:0/3070279464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x55df211b75b0 con 0x55df211d2b60 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea14ff9640 1 -- 172.21.15.26:0/3070279464 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3001b050 con 0x55df211d2b60 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea14ff9640 1 -- 172.21.15.26:0/3070279464 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 108) ==== 3534+0+0 (secure 0 0 0) 0x7fea24015990 con 0x55df211d2b60 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea3ede2740 10 client.8910 fetch_fsmap finished waiting for FSMap version 108 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea3ede2740 10 client.8910 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea3ede2740 20 client.8910 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:34.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea3ede2740 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55df211b6b10 0x55df211e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea3ede2740 4 client.8910 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea3ede2740 1 -- 172.21.15.26:0/3070279464 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df210fb440 con 0x55df211b6b10 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.634+0000 7fea39d95640 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55df211b6b10 0x55df211e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.635+0000 7fea14ff9640 10 client.8910 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.635+0000 7fea39d95640 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55df211b6b10 0x55df211e09f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14fc4fbf2dbe6309 server_cookie=df7527df0b485acf in_seq=0 out_seq=0 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.635+0000 7fea14ff9640 1 -- 172.21.15.26:0/3070279464 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df210fb440 con 0x55df211b6b10 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.635+0000 7fea14ff9640 10 client.8910 handle_command_reply: tid=0 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 client.8910 shutdown 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 -- 172.21.15.26:0/3070279464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 msgr2=0x55df211c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211c8ac0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fea24004e80 tx=0x7fea24001680 comp rx=0 tx=0).stop 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 -- 172.21.15.26:0/3070279464 shutdown_connections 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55df211b6b10 0x55df211e09f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211c9000 0x55df211b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211d2b60 0x55df211c8ac0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea14ff9640 0 client.8910 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 --2- 172.21.15.26:0/3070279464 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211b5c80 0x55df211b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 -- 172.21.15.26:0/3070279464 >> 172.21.15.26:0/3070279464 conn(0x55df211b5540 msgr2=0x55df21184250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 -- 172.21.15.26:0/3070279464 shutdown_connections 2024-04-24T23:27:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 1 -- 172.21.15.26:0/3070279464 wait complete. 2024-04-24T23:27:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.636+0000 7fea3ede2740 20 client.8910 trim_cache size 0 max 16384 2024-04-24T23:27:34.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.654+0000 7fea3ede2740 1 Processor -- start 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.654+0000 7fea3ede2740 1 -- start start 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 0x55df21184440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 0x55df211aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea3ede2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea341ff8c0 con 0x55df211b5540 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea3ede2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fea341fff90 con 0x55df211b5c80 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea3ede2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fea341598a0 con 0x55df211c9000 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea39d95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea39d95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211e9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35096/0 (socket says 172.21.15.26:35096) 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea39d95640 1 -- 172.21.15.26:0/720001168 learned_addr learned my addr 172.21.15.26:0/720001168 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea3a596640 1 --2- 172.21.15.26:0/720001168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 0x55df211aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.655+0000 7fea39594640 1 --2- 172.21.15.26:0/720001168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 0x55df21184440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea39d95640 1 -- 172.21.15.26:0/720001168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 msgr2=0x55df211aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea39d95640 1 --2- 172.21.15.26:0/720001168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 0x55df211aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea39d95640 1 -- 172.21.15.26:0/720001168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 msgr2=0x55df21184440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea39d95640 1 --2- 172.21.15.26:0/720001168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 0x55df21184440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea39d95640 1 -- 172.21.15.26:0/720001168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df211ca6d0 con 0x55df211b5540 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea39d95640 1 --2- 172.21.15.26:0/720001168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211e9f30 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fea30020000 tx=0x7fea30015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64201f216138917c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea16ffd640 1 -- 172.21.15.26:0/720001168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30025040 con 0x55df211b5540 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea16ffd640 1 -- 172.21.15.26:0/720001168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea30032070 con 0x55df211b5540 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea16ffd640 1 -- 172.21.15.26:0/720001168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30033570 con 0x55df211b5540 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea3ede2740 1 -- 172.21.15.26:0/720001168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 msgr2=0x55df211e9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea3ede2740 1 --2- 172.21.15.26:0/720001168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211e9f30 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fea30020000 tx=0x7fea30015000 comp rx=0 tx=0).stop 2024-04-24T23:27:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea3ede2740 1 -- 172.21.15.26:0/720001168 shutdown_connections 2024-04-24T23:27:34.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea3ede2740 1 --2- 172.21.15.26:0/720001168 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 0x55df211aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea3ede2740 1 --2- 172.21.15.26:0/720001168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 0x55df21184440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea3ede2740 1 --2- 172.21.15.26:0/720001168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211e9f30 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea3ede2740 1 -- 172.21.15.26:0/720001168 >> 172.21.15.26:0/720001168 conn(0x55df2118b800 msgr2=0x55df2118c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.656+0000 7fea3ede2740 1 -- 172.21.15.26:0/720001168 shutdown_connections 2024-04-24T23:27:34.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.657+0000 7fea3ede2740 1 -- 172.21.15.26:0/720001168 wait complete. 2024-04-24T23:27:34.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.657+0000 7fea3ede2740 1 -- start start 2024-04-24T23:27:34.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.657+0000 7fea3ede2740 1 Processor -- start 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 0x55df21249e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3ede2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 0x55df211afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3ede2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55df211b6990 con 0x55df211b5540 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3ede2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55df211b6590 con 0x55df211b5c80 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3ede2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55df211b6fb0 con 0x55df211c9000 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3ede2740 10 client.0 _ll_register_callbacks cb 0x7fea24004ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea39d95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea39d95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211df340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35100/0 (socket says 172.21.15.26:35100) 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea39d95640 1 -- 172.21.15.26:0/607568330 learned_addr learned my addr 172.21.15.26:0/607568330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea39594640 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 0x55df21249e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3a596640 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 0x55df211afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3a596640 1 -- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 msgr2=0x55df21249e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3a596640 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 0x55df21249e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3a596640 1 -- 172.21.15.26:0/607568330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 msgr2=0x55df211df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3a596640 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211df340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea3a596640 1 -- 172.21.15.26:0/607568330 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55df21180810 con 0x55df211c9000 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.658+0000 7fea39d95640 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3a596640 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 0x55df211afb90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0aa880 tx=0x7fea2c0ac040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c67524594f8ff9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3301/0 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea14ff9640 1 -- 172.21.15.26:0/607568330 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c083250 con 0x55df211c9000 2024-04-24T23:27:34.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea14ff9640 1 -- 172.21.15.26:0/607568330 <== mon.2 v2:172.21.15.29:3301/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fea2c07a980 con 0x55df211c9000 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3ede2740 1 -- 172.21.15.26:0/607568330 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea30032070 con 0x55df211c9000 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea14ff9640 1 -- 172.21.15.26:0/607568330 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fea2c07ab80 con 0x55df211c9000 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3ede2740 10 client.6911 fetch_fsmap learned FSMap version 108 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3ede2740 1 -- 172.21.15.26:0/607568330 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x55df2124a410 con 0x55df211c9000 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea14ff9640 1 -- 172.21.15.26:0/607568330 <== mon.2 v2:172.21.15.29:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea30032070 con 0x55df211c9000 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea14ff9640 1 -- 172.21.15.26:0/607568330 <== mon.2 v2:172.21.15.29:3301/0 5 ==== fsmap(e 108) ==== 3534+0+0 (secure 0 0 0) 0x7fea2c082dc0 con 0x55df211c9000 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3ede2740 10 client.6911 fetch_fsmap finished waiting for FSMap version 108 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3ede2740 10 client.6911 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3ede2740 20 client.6911 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3ede2740 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55df211d2b60 0x55df211ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3ede2740 4 client.6911 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea3ede2740 1 -- 172.21.15.26:0/607568330 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "status"}) -- 0x55df210f5790 con 0x55df211d2b60 2024-04-24T23:27:34.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.659+0000 7fea39d95640 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55df211d2b60 0x55df211ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea14ff9640 10 client.6911 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea39d95640 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55df211d2b60 0x55df211ce910 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c65df229ea16da1b server_cookie=fe4c5bd49514618e in_seq=0 out_seq=0 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea14ff9640 1 -- 172.21.15.26:0/607568330 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55df210f5790 con 0x55df211d2b60 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea14ff9640 10 client.6911 handle_command_reply: tid=0 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 client.6911 shutdown 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 -- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 msgr2=0x55df211afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 0x55df211afb90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0aa880 tx=0x7fea2c0ac040 comp rx=0 tx=0).stop 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 -- 172.21.15.26:0/607568330 shutdown_connections 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55df211d2b60 0x55df211ce910 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55df211c9000 0x55df211afb90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea14ff9640 0 client.6911 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df211b5c80 0x55df21249e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 --2- 172.21.15.26:0/607568330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df211b5540 0x55df211df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 -- 172.21.15.26:0/607568330 >> 172.21.15.26:0/607568330 conn(0x55df2118b800 msgr2=0x55df211c6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 -- 172.21.15.26:0/607568330 shutdown_connections 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 1 -- 172.21.15.26:0/607568330 wait complete. 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.660+0000 7fea3ede2740 20 client.6911 trim_cache size 0 max 16384 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7444, 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:27:34.668 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 1.6873498899999999, 2024-04-24T23:27:34.669 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 108, 2024-04-24T23:27:34.669 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 96, 2024-04-24T23:27:34.669 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 96, 2024-04-24T23:27:34.669 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.733115494 2024-04-24T23:27:34.669 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 -- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea1806be30 msgr2=0x7fea1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea1806be30 0x7fea1806e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fea30000f60 tx=0x7fea30009040 comp rx=0 tx=0).stop 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 -- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 msgr2=0x7fea3415d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 0x7fea3415d400 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009eb0 tx=0x7fea2c004a40 comp rx=0 tx=0).stop 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 -- 172.21.15.26:0/600109762 shutdown_connections 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fea1806be30 0x7fea1806e2f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea340adf80 0x7fea3415d400 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fea3406f4f0 0x7fea3414c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 --2- 172.21.15.26:0/600109762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fea3406f110 0x7fea3415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 -- 172.21.15.26:0/600109762 >> 172.21.15.26:0/600109762 conn(0x7fea340729a0 msgr2=0x7fea340b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 -- 172.21.15.26:0/600109762 shutdown_connections 2024-04-24T23:27:34.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.661+0000 7fea3bff9640 1 -- 172.21.15.26:0/600109762 wait complete. 2024-04-24T23:27:34.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:27:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.773+0000 7fdbb0f68640 1 Processor -- start 2024-04-24T23:27:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.773+0000 7fdbb0f68640 1 -- start start 2024-04-24T23:27:34.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdbb0f68640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbac15e570 0x7fdbac0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdbb0f68640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fdbac0b3130 0x7fdbac0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdbb0f68640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbac0d32f0 con 0x7fdbac0d22e0 2024-04-24T23:27:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdbb0f68640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fdbac0d1ab0 con 0x7fdbac15e570 2024-04-24T23:27:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdbb0f68640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fdbac0d2ae0 con 0x7fdbac0b3130 2024-04-24T23:27:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdbaa59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbac15e570 0x7fdbac0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdbaa59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbac15e570 0x7fdbac0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35112/0 (socket says 172.21.15.26:35112) 2024-04-24T23:27:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdbaad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdbac0d22e0 0x7fdbac0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44414/0 (socket says 172.21.15.26:44414) 2024-04-24T23:27:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdbaa59c640 1 -- 172.21.15.26:0/1574589942 learned_addr learned my addr 172.21.15.26:0/1574589942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.774+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/1574589942 >> v2:172.21.15.29:3300/0 conn(0x7fdbac0b3130 0x7fdbac0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.775+0000 7fdbaa59c640 1 -- 172.21.15.26:0/1574589942 >> v2:172.21.15.29:3300/0 conn(0x7fdbac0b3130 msgr2=0x7fdbac0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.775+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/1574589942 >> v2:172.21.15.29:3300/0 conn(0x7fdbac0b3130 0x7fdbac0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.775+0000 7fdbaa59c640 1 -- 172.21.15.26:0/1574589942 >> v1:172.21.15.26:6789/0 conn(0x7fdbac0d22e0 legacy=0x7fdbac0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.775+0000 7fdbaa59c640 1 -- 172.21.15.26:0/1574589942 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbac0d0270 con 0x7fdbac15e570 2024-04-24T23:27:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.775+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/1574589942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 0x7fdbac0d4160 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fdbac08cae0 tx=0x7fdba402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e521a6d0741aa6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdba959a640 1 -- 172.21.15.26:0/1574589942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4037070 con 0x7fdbac15e570 2024-04-24T23:27:34.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdba959a640 1 -- 172.21.15.26:0/1574589942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba4032440 con 0x7fdbac15e570 2024-04-24T23:27:34.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdba959a640 1 -- 172.21.15.26:0/1574589942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4032810 con 0x7fdbac15e570 2024-04-24T23:27:34.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1574589942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 msgr2=0x7fdbac0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdbb0f68640 1 --2- 172.21.15.26:0/1574589942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 0x7fdbac0d4160 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fdbac08cae0 tx=0x7fdba402a270 comp rx=0 tx=0).stop 2024-04-24T23:27:34.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1574589942 shutdown_connections 2024-04-24T23:27:34.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdbb0f68640 1 --2- 172.21.15.26:0/1574589942 >> v2:172.21.15.29:3300/0 conn(0x7fdbac0b3130 0x7fdbac0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdbb0f68640 1 --2- 172.21.15.26:0/1574589942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 0x7fdbac0d4160 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1574589942 >> 172.21.15.26:0/1574589942 conn(0x7fdbac08dbe0 msgr2=0x7fdbac09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1574589942 shutdown_connections 2024-04-24T23:27:34.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.776+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1574589942 wait complete. 2024-04-24T23:27:34.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.778+0000 7fdbb0f68640 1 Processor -- start 2024-04-24T23:27:34.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.778+0000 7fdbb0f68640 1 -- start start 2024-04-24T23:27:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.778+0000 7fdbb0f68640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbac0b3130 0x7fdbac15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.778+0000 7fdbb0f68640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbac0d22e0 0x7fdbac0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.778+0000 7fdbb0f68640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 0x7fdbac0a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.778+0000 7fdbb0f68640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac0cfa70 con 0x7fdbac15e570 2024-04-24T23:27:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.778+0000 7fdbb0f68640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac0d3010 con 0x7fdbac0d22e0 2024-04-24T23:27:34.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.778+0000 7fdbb0f68640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdbac0cfd50 con 0x7fdbac0b3130 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 0x7fdbac0a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 0x7fdbac0a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35126/0 (socket says 172.21.15.26:35126) 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaad9d640 1 -- 172.21.15.26:0/1871519496 learned_addr learned my addr 172.21.15.26:0/1871519496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbac0d22e0 0x7fdbac0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbac0b3130 0x7fdbac15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaad9d640 1 -- 172.21.15.26:0/1871519496 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbac0b3130 msgr2=0x7fdbac15c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbac0b3130 0x7fdbac15c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaad9d640 1 -- 172.21.15.26:0/1871519496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbac0d22e0 msgr2=0x7fdbac0cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbac0d22e0 0x7fdbac0cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaad9d640 1 -- 172.21.15.26:0/1871519496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbac09e250 con 0x7fdbac15e570 2024-04-24T23:27:34.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbac0d22e0 0x7fdbac0cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 0x7fdbac0a0930 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00c7b0 tx=0x7fdb9c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdb9affd640 1 -- 172.21.15.26:0/1871519496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c017070 con 0x7fdbac15e570 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdb9affd640 1 -- 172.21.15.26:0/1871519496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c00a640 con 0x7fdbac15e570 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1871519496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbac09e530 con 0x7fdbac15e570 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdb9affd640 1 -- 172.21.15.26:0/1871519496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c00a9f0 con 0x7fdbac15e570 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.779+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1871519496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbac0cd460 con 0x7fdbac15e570 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.781+0000 7fdb9affd640 1 -- 172.21.15.26:0/1871519496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb9c01a070 con 0x7fdbac15e570 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.781+0000 7fdb9affd640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdb8c06be30 0x7fdb8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.782+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdb8c06be30 0x7fdb8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.782+0000 7fdb9affd640 1 -- 172.21.15.26:0/1871519496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fdb9c08a3b0 con 0x7fdbac15e570 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.782+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdb8c06be30 0x7fdb8c06e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdba4034000 tx=0x7fdba402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.782+0000 7fdbb3c72740 1 Processor -- start 2024-04-24T23:27:34.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbb3c72740 1 -- start start 2024-04-24T23:27:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 0x561a0b0504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b080c80 0x561a0b080000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080540 0x561a0b055350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561a0afb8f00 con 0x561a0b080540 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561a0af61750 con 0x561a0b080c80 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561a0afb4ec0 con 0x561a0b09db60 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbaad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080540 0x561a0b055350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbaad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080540 0x561a0b055350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35128/0 (socket says 172.21.15.26:35128) 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbaad9d640 1 -- 172.21.15.26:0/2026350754 learned_addr learned my addr 172.21.15.26:0/2026350754 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 0x561a0b0504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.783+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b080c80 0x561a0b080000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdbaad9d640 1 -- 172.21.15.26:0/2026350754 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 msgr2=0x561a0b0504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 0x561a0b0504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdbaad9d640 1 -- 172.21.15.26:0/2026350754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b080c80 msgr2=0x561a0b080000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b080c80 0x561a0b080000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdbaad9d640 1 -- 172.21.15.26:0/2026350754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0accd2d0 con 0x561a0b080540 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 0x561a0b0504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080540 0x561a0b055350 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c088820 tx=0x7fdb9c058360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0e18987eacf836 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdb8bfff640 1 -- 172.21.15.26:0/2026350754 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c061240 con 0x561a0b080540 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdb8bfff640 1 -- 172.21.15.26:0/2026350754 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c056a10 con 0x561a0b080540 2024-04-24T23:27:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.784+0000 7fdb8bfff640 1 -- 172.21.15.26:0/2026350754 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c054d80 con 0x561a0b080540 2024-04-24T23:27:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 -- 172.21.15.26:0/2026350754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080540 msgr2=0x561a0b055350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080540 0x561a0b055350 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c088820 tx=0x7fdb9c058360 comp rx=0 tx=0).stop 2024-04-24T23:27:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 -- 172.21.15.26:0/2026350754 shutdown_connections 2024-04-24T23:27:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080540 0x561a0b055350 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b080c80 0x561a0b080000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/2026350754 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 0x561a0b0504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 -- 172.21.15.26:0/2026350754 >> 172.21.15.26:0/2026350754 conn(0x561a0b052760 msgr2=0x561a0b09b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 -- 172.21.15.26:0/2026350754 shutdown_connections 2024-04-24T23:27:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 -- 172.21.15.26:0/2026350754 wait complete. 2024-04-24T23:27:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 -- start start 2024-04-24T23:27:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.785+0000 7fdbb3c72740 1 Processor -- start 2024-04-24T23:27:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 0x561a0b07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b09db60 0x561a0b093e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b04f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561a0b054910 con 0x561a0b09db60 2024-04-24T23:27:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561a0af74cf0 con 0x561a0b04f250 2024-04-24T23:27:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561a0b053d00 con 0x561a0b080c80 2024-04-24T23:27:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdbb3c72740 10 client.0 _ll_register_callbacks cb 0x561a0af746e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdba9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b09db60 0x561a0b093e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdba9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b09db60 0x561a0b093e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35142/0 (socket says 172.21.15.26:35142) 2024-04-24T23:27:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdba9d9b640 1 -- 172.21.15.26:0/3866040284 learned_addr learned my addr 172.21.15.26:0/3866040284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b04f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.786+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 0x561a0b07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdba9d9b640 1 -- 172.21.15.26:0/3866040284 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 msgr2=0x561a0b07b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 0x561a0b07b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdba9d9b640 1 -- 172.21.15.26:0/3866040284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 msgr2=0x561a0b04f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b04f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdba9d9b640 1 -- 172.21.15.26:0/3866040284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0ad976b0 con 0x561a0b09db60 2024-04-24T23:27:34.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b04f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:34.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b09db60 0x561a0b093e80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fdbac1563e0 tx=0x7fdba0012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8877ff2eda7c3d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdb897fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:34.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdb897fa640 1 -- 172.21.15.26:0/3866040284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0012e70 con 0x561a0b09db60 2024-04-24T23:27:34.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdb897fa640 1 -- 172.21.15.26:0/3866040284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fdba0004030 con 0x561a0b09db60 2024-04-24T23:27:34.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdbb3c72740 1 -- 172.21.15.26:0/3866040284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9c056a10 con 0x561a0b09db60 2024-04-24T23:27:34.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdb897fa640 1 -- 172.21.15.26:0/3866040284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fdba0004230 con 0x561a0b09db60 2024-04-24T23:27:34.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdbb3c72740 10 client.8931 fetch_fsmap learned FSMap version 108 2024-04-24T23:27:34.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdbb3c72740 1 -- 172.21.15.26:0/3866040284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x561a0b083d50 con 0x561a0b09db60 2024-04-24T23:27:34.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.787+0000 7fdb897fa640 1 -- 172.21.15.26:0/3866040284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c056a10 con 0x561a0b09db60 2024-04-24T23:27:34.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdb897fa640 1 -- 172.21.15.26:0/3866040284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 108) ==== 3534+0+0 (secure 0 0 0) 0x7fdba002a080 con 0x561a0b09db60 2024-04-24T23:27:34.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdbb3c72740 10 client.8931 fetch_fsmap finished waiting for FSMap version 108 2024-04-24T23:27:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdbb3c72740 10 client.8931 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:27:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdbb3c72740 20 client.8931 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x561a0b09ea20 0x561a0b09ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdbb3c72740 4 client.8931 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdbb3c72740 1 -- 172.21.15.26:0/3866040284 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a0afc6440 con 0x561a0b09ea20 2024-04-24T23:27:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x561a0b09ea20 0x561a0b09ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdb897fa640 10 client.8931 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.788+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x561a0b09ea20 0x561a0b09ee00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9f24ba79cc45706 server_cookie=54b1b5c2eb62b0d7 in_seq=0 out_seq=0 2024-04-24T23:27:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdb897fa640 1 -- 172.21.15.26:0/3866040284 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a0afc6440 con 0x561a0b09ea20 2024-04-24T23:27:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdb897fa640 10 client.8931 handle_command_reply: tid=0 2024-04-24T23:27:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 client.8931 shutdown 2024-04-24T23:27:34.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 -- 172.21.15.26:0/3866040284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b09db60 msgr2=0x561a0b093e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b09db60 0x561a0b093e80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fdbac1563e0 tx=0x7fdba0012700 comp rx=0 tx=0).stop 2024-04-24T23:27:34.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 -- 172.21.15.26:0/3866040284 shutdown_connections 2024-04-24T23:27:34.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x561a0b09ea20 0x561a0b09ee00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b04f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b09db60 0x561a0b093e80 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdb897fa640 0 client.8931 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:34.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/3866040284 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 0x561a0b07b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 -- 172.21.15.26:0/3866040284 >> 172.21.15.26:0/3866040284 conn(0x561a0b080540 msgr2=0x561a0b09b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 -- 172.21.15.26:0/3866040284 shutdown_connections 2024-04-24T23:27:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 1 -- 172.21.15.26:0/3866040284 wait complete. 2024-04-24T23:27:34.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.789+0000 7fdbb3c72740 20 client.8931 trim_cache size 0 max 16384 2024-04-24T23:27:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.807+0000 7fdbb3c72740 1 Processor -- start 2024-04-24T23:27:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.807+0000 7fdbb3c72740 1 -- start start 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b04f250 0x561a0b056530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 0x561a0b050060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b09db60 0x561a0b0505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac1ff9f0 con 0x561a0b04f250 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac2000c0 con 0x561a0b09db60 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdbac0cb410 con 0x561a0b080c80 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbaa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b04f250 0x561a0b056530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbaa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b04f250 0x561a0b056530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35156/0 (socket says 172.21.15.26:35156) 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbaa59c640 1 -- 172.21.15.26:0/261049201 learned_addr learned my addr 172.21.15.26:0/261049201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 0x561a0b050060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.808+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b09db60 0x561a0b0505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbaa59c640 1 -- 172.21.15.26:0/261049201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 msgr2=0x561a0b050060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 0x561a0b050060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbaa59c640 1 -- 172.21.15.26:0/261049201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b09db60 msgr2=0x561a0b0505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b09db60 0x561a0b0505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbaa59c640 1 -- 172.21.15.26:0/261049201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0b092d80 con 0x561a0b04f250 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 0x561a0b050060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b04f250 0x561a0b056530 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fdba40090b0 tx=0x7fdba402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff03f8e83f88c46d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdb8b7fe640 1 -- 172.21.15.26:0/261049201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4057070 con 0x561a0b04f250 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdb8b7fe640 1 -- 172.21.15.26:0/261049201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba4052dd0 con 0x561a0b04f250 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdb8b7fe640 1 -- 172.21.15.26:0/261049201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4037040 con 0x561a0b04f250 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbb3c72740 1 -- 172.21.15.26:0/261049201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b04f250 msgr2=0x561a0b056530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b04f250 0x561a0b056530 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fdba40090b0 tx=0x7fdba402c000 comp rx=0 tx=0).stop 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbb3c72740 1 -- 172.21.15.26:0/261049201 shutdown_connections 2024-04-24T23:27:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b09db60 0x561a0b0505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b080c80 0x561a0b050060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/261049201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b04f250 0x561a0b056530 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.809+0000 7fdbb3c72740 1 -- 172.21.15.26:0/261049201 >> 172.21.15.26:0/261049201 conn(0x561a0b080540 msgr2=0x561a0b04f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.810+0000 7fdbb3c72740 1 -- 172.21.15.26:0/261049201 shutdown_connections 2024-04-24T23:27:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.810+0000 7fdbb3c72740 1 -- 172.21.15.26:0/261049201 wait complete. 2024-04-24T23:27:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.810+0000 7fdbb3c72740 1 -- start start 2024-04-24T23:27:34.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.810+0000 7fdbb3c72740 1 Processor -- start 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b090990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080c80 0x561a0b114cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbb3c72740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 0x561a0b115950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561a0b083150 con 0x561a0b080c80 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561a0b091e60 con 0x561a0b04f250 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbb3c72740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561a0b092b30 con 0x561a0b09db60 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbb3c72740 10 client.0 _ll_register_callbacks cb 0x7fdba0014a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbaa59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b090990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdba9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080c80 0x561a0b114cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbaa59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b090990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:37236/0 (socket says 172.21.15.26:37236) 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbaa59c640 1 -- 172.21.15.26:0/780706999 learned_addr learned my addr 172.21.15.26:0/780706999 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbaad9d640 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 0x561a0b115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbaa59c640 1 -- 172.21.15.26:0/780706999 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 msgr2=0x561a0b115950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 0x561a0b115950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.811+0000 7fdbaa59c640 1 -- 172.21.15.26:0/780706999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080c80 msgr2=0x561a0b114cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080c80 0x561a0b114cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbaa59c640 1 -- 172.21.15.26:0/780706999 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0b081310 con 0x561a0b04f250 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080c80 0x561a0b114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbaa59c640 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b090990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdba402a2b0 tx=0x7fdba4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7b6566368ecace5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdb897fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdb897fa640 1 -- 172.21.15.26:0/780706999 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4061060 con 0x561a0b04f250 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdb897fa640 1 -- 172.21.15.26:0/780706999 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fdba40523b0 con 0x561a0b04f250 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbb3c72740 1 -- 172.21.15.26:0/780706999 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba4052dd0 con 0x561a0b04f250 2024-04-24T23:27:34.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdb897fa640 1 -- 172.21.15.26:0/780706999 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fdba4037040 con 0x561a0b04f250 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbb3c72740 10 client.7609 fetch_fsmap learned FSMap version 108 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbb3c72740 1 -- 172.21.15.26:0/780706999 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x561a0b0b5060 con 0x561a0b04f250 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdb897fa640 1 -- 172.21.15.26:0/780706999 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba4052dd0 con 0x561a0b04f250 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdb897fa640 1 -- 172.21.15.26:0/780706999 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 108) ==== 3534+0+0 (secure 0 0 0) 0x7fdba4046040 con 0x561a0b04f250 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbb3c72740 10 client.7609 fetch_fsmap finished waiting for FSMap version 108 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbb3c72740 10 client.7609 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbb3c72740 20 client.7609 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x561a0b0505b0 0x561a0b0a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbb3c72740 4 client.7609 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.812+0000 7fdbb3c72740 1 -- 172.21.15.26:0/780706999 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "status"}) -- 0x561a0b051bb0 con 0x561a0b0505b0 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x561a0b0505b0 0x561a0b0a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdb897fa640 10 client.7609 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdba9d9b640 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x561a0b0505b0 0x561a0b0a9700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec953a8cc01faac server_cookie=9540c10b47bd0a in_seq=0 out_seq=0 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdb897fa640 1 -- 172.21.15.26:0/780706999 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561a0b051bb0 con 0x561a0b0505b0 2024-04-24T23:27:34.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdb897fa640 10 client.7609 handle_command_reply: tid=0 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 client.7609 shutdown 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 -- 172.21.15.26:0/780706999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 msgr2=0x561a0b090990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b090990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdba402a2b0 tx=0x7fdba4004730 comp rx=0 tx=0).stop 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 -- 172.21.15.26:0/780706999 shutdown_connections 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x561a0b0505b0 0x561a0b0a9700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561a0b09db60 0x561a0b115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a0b080c80 0x561a0b114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 --2- 172.21.15.26:0/780706999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561a0b04f250 0x561a0b090990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdb897fa640 0 client.7609 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 -- 172.21.15.26:0/780706999 >> 172.21.15.26:0/780706999 conn(0x561a0b080540 msgr2=0x561a0b094510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 -- 172.21.15.26:0/780706999 shutdown_connections 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 1 -- 172.21.15.26:0/780706999 wait complete. 2024-04-24T23:27:34.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.813+0000 7fdbb3c72740 20 client.7609 trim_cache size 0 max 16384 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8712, 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 2.8453499390000001, 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 108, 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 96, 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 96, 2024-04-24T23:27:34.821 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.897454229999999 2024-04-24T23:27:34.822 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:34.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdb8c06be30 msgr2=0x7fdb8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdb8c06be30 0x7fdb8c06e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdba4034000 tx=0x7fdba402e000 comp rx=0 tx=0).stop 2024-04-24T23:27:34.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 msgr2=0x7fdbac0a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 0x7fdbac0a0930 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00c7b0 tx=0x7fdb9c00cc70 comp rx=0 tx=0).stop 2024-04-24T23:27:34.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1871519496 shutdown_connections 2024-04-24T23:27:34.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdb8c06be30 0x7fdb8c06e2f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbac15e570 0x7fdbac0a0930 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdbac0d22e0 0x7fdbac0cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 --2- 172.21.15.26:0/1871519496 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdbac0b3130 0x7fdbac15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1871519496 >> 172.21.15.26:0/1871519496 conn(0x7fdbac08dbe0 msgr2=0x7fdbac0b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1871519496 shutdown_connections 2024-04-24T23:27:34.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.814+0000 7fdbb0f68640 1 -- 172.21.15.26:0/1871519496 wait complete. 2024-04-24T23:27:34.826 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T23:27:34.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.927+0000 7f9dade93640 1 Processor -- start 2024-04-24T23:27:34.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.927+0000 7f9dade93640 1 -- start start 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.927+0000 7f9dade93640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9da80d4870 0x7f9da80b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.927+0000 7f9dade93640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da80d32f0 con 0x7f9da80b3130 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.927+0000 7f9dade93640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da80d0270 con 0x7f9da815e570 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.927+0000 7f9dade93640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9da80d1ab0 con 0x7f9da80d4870 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.928+0000 7f9d9effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9da80d4870 0x7f9da80b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.928+0000 7f9da7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9da80b3130 0x7f9da80b4640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44428/0 (socket says 172.21.15.26:44428) 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.928+0000 7f9da7fff640 1 -- 172.21.15.26:0/4041722517 learned_addr learned my addr 172.21.15.26:0/4041722517 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.928+0000 7f9d9effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9da80d4870 0x7f9da80b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35186/0 (socket says 172.21.15.26:35186) 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.928+0000 7f9d9effd640 1 -- 172.21.15.26:0/4041722517 >> v1:172.21.15.29:6789/0 conn(0x7f9da815e570 legacy=0x7f9da80d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.928+0000 7f9d9effd640 1 -- 172.21.15.26:0/4041722517 >> v1:172.21.15.26:6789/0 conn(0x7f9da80b3130 legacy=0x7f9da80b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.928+0000 7f9da6ffd640 1 -- 172.21.15.26:0/4041722517 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1172568680 0 0) 0x7f9da80d0270 con 0x7f9da815e570 2024-04-24T23:27:34.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.928+0000 7f9d9effd640 1 -- 172.21.15.26:0/4041722517 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da80d2d30 con 0x7f9da80d4870 2024-04-24T23:27:34.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9d9effd640 1 --2- 172.21.15.26:0/4041722517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da80d4870 0x7f9da80b4100 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f9d900097a0 tx=0x7f9d9002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8926e81c8e448b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9da6ffd640 1 -- 172.21.15.26:0/4041722517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90037070 con 0x7f9da80d4870 2024-04-24T23:27:34.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9da6ffd640 1 -- 172.21.15.26:0/4041722517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9002ad30 con 0x7f9da80d4870 2024-04-24T23:27:34.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9da6ffd640 1 -- 172.21.15.26:0/4041722517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90032660 con 0x7f9da80d4870 2024-04-24T23:27:34.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9dade93640 1 -- 172.21.15.26:0/4041722517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da80d4870 msgr2=0x7f9da80b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9dade93640 1 --2- 172.21.15.26:0/4041722517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da80d4870 0x7f9da80b4100 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f9d900097a0 tx=0x7f9d9002a0a0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9dade93640 1 -- 172.21.15.26:0/4041722517 shutdown_connections 2024-04-24T23:27:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9dade93640 1 --2- 172.21.15.26:0/4041722517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da80d4870 0x7f9da80b4100 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9dade93640 1 -- 172.21.15.26:0/4041722517 >> 172.21.15.26:0/4041722517 conn(0x7f9da808dbe0 msgr2=0x7f9da809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9dade93640 1 -- 172.21.15.26:0/4041722517 shutdown_connections 2024-04-24T23:27:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.929+0000 7f9dade93640 1 -- 172.21.15.26:0/4041722517 wait complete. 2024-04-24T23:27:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.930+0000 7f9dade93640 1 Processor -- start 2024-04-24T23:27:34.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.931+0000 7f9dade93640 1 -- start start 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.931+0000 7f9dade93640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9da80b3130 0x7f9da815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.931+0000 7f9dade93640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9da80d4870 0x7f9da80cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.931+0000 7f9dade93640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da815e570 0x7f9da81525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.931+0000 7f9dade93640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9da80cfa70 con 0x7f9da815e570 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.931+0000 7f9dade93640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9da80d2ae0 con 0x7f9da80b3130 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.931+0000 7f9dade93640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9da80cfd50 con 0x7f9da80d4870 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.931+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da815e570 0x7f9da81525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da815e570 0x7f9da81525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35196/0 (socket says 172.21.15.26:35196) 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da7fff640 1 -- 172.21.15.26:0/3512989233 learned_addr learned my addr 172.21.15.26:0/3512989233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9d9effd640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9da80d4870 0x7f9da80cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da77fe640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9da80b3130 0x7f9da815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da7fff640 1 -- 172.21.15.26:0/3512989233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9da80d4870 msgr2=0x7f9da80cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da7fff640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9da80d4870 0x7f9da80cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da7fff640 1 -- 172.21.15.26:0/3512989233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9da80b3130 msgr2=0x7f9da815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da7fff640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9da80b3130 0x7f9da815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da7fff640 1 -- 172.21.15.26:0/3512989233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da815a920 con 0x7f9da815e570 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da7fff640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da815e570 0x7f9da81525c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9da000cb40 tx=0x7f9da000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da4ff9640 1 -- 172.21.15.26:0/3512989233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0018070 con 0x7f9da815e570 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da4ff9640 1 -- 172.21.15.26:0/3512989233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da000a890 con 0x7f9da815e570 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9dade93640 1 -- 172.21.15.26:0/3512989233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da815ac00 con 0x7f9da815e570 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9da4ff9640 1 -- 172.21.15.26:0/3512989233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da000ac80 con 0x7f9da815e570 2024-04-24T23:27:34.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.932+0000 7f9dade93640 1 -- 172.21.15.26:0/3512989233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da815b1c0 con 0x7f9da815e570 2024-04-24T23:27:34.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.934+0000 7f9da4ff9640 1 -- 172.21.15.26:0/3512989233 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9da001b070 con 0x7f9da815e570 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.934+0000 7f9da4ff9640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9d8806be60 0x7f9d8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.934+0000 7f9da4ff9640 1 -- 172.21.15.26:0/3512989233 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f9da008a780 con 0x7f9da815e570 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.934+0000 7f9da77fe640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9d8806be60 0x7f9d8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.935+0000 7f9da77fe640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9d8806be60 0x7f9d8806e320 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9d98000f60 tx=0x7f9d98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.935+0000 7f9db0b9e740 1 Processor -- start 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.935+0000 7f9db0b9e740 1 -- start start 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b3b60 0x55bd739664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 0x55bd73996000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996540 0x55bd7396b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd738cef00 con 0x55bd73996540 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bd73877750 con 0x55bd73996c80 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55bd738caec0 con 0x55bd739b3b60 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996540 0x55bd7396b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9da77fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b3b60 0x55bd739664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996540 0x55bd7396b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35206/0 (socket says 172.21.15.26:35206) 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9da7fff640 1 -- 172.21.15.26:0/4094592808 learned_addr learned my addr 172.21.15.26:0/4094592808 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9d9effd640 1 --2- 172.21.15.26:0/4094592808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 0x55bd73996000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9da7fff640 1 -- 172.21.15.26:0/4094592808 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b3b60 msgr2=0x55bd739664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9da7fff640 1 --2- 172.21.15.26:0/4094592808 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b3b60 0x55bd739664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9da7fff640 1 -- 172.21.15.26:0/4094592808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 msgr2=0x55bd73996000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9da7fff640 1 --2- 172.21.15.26:0/4094592808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 0x55bd73996000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.936+0000 7f9da7fff640 1 -- 172.21.15.26:0/4094592808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd735e32d0 con 0x55bd73996540 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.937+0000 7f9da7fff640 1 --2- 172.21.15.26:0/4094592808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996540 0x55bd7396b350 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9da0088c20 tx=0x7f9da000aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e40e5645426136b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.937+0000 7f9d9d7fa640 1 -- 172.21.15.26:0/4094592808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0061610 con 0x55bd73996540 2024-04-24T23:27:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.937+0000 7f9d9d7fa640 1 -- 172.21.15.26:0/4094592808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0056de0 con 0x55bd73996540 2024-04-24T23:27:34.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.937+0000 7f9d9d7fa640 1 -- 172.21.15.26:0/4094592808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da005f070 con 0x55bd73996540 2024-04-24T23:27:34.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.937+0000 7f9db0b9e740 1 -- 172.21.15.26:0/4094592808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996540 msgr2=0x55bd7396b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.937+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/4094592808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996540 0x55bd7396b350 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9da0088c20 tx=0x7f9da000aee0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.938+0000 7f9db0b9e740 1 -- 172.21.15.26:0/4094592808 shutdown_connections 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.938+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/4094592808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996540 0x55bd7396b350 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.938+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/4094592808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 0x55bd73996000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.938+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/4094592808 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b3b60 0x55bd739664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.938+0000 7f9db0b9e740 1 -- 172.21.15.26:0/4094592808 >> 172.21.15.26:0/4094592808 conn(0x55bd73968760 msgr2=0x55bd739b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.938+0000 7f9db0b9e740 1 -- 172.21.15.26:0/4094592808 shutdown_connections 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.938+0000 7f9db0b9e740 1 -- 172.21.15.26:0/4094592808 wait complete. 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.938+0000 7f9db0b9e740 1 -- start start 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.938+0000 7f9db0b9e740 1 Processor -- start 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 0x55bd73986790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b3b60 0x55bd73998be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 0x55bd739b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd7396a910 con 0x55bd739b3b60 2024-04-24T23:27:34.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bd7388acf0 con 0x55bd73996c80 2024-04-24T23:27:34.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55bd73969d00 con 0x55bd739b2550 2024-04-24T23:27:34.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9db0b9e740 10 client.0 _ll_register_callbacks cb 0x55bd7388a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:34.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9d9effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b3b60 0x55bd73998be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 0x55bd739b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9da77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 0x55bd73986790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9d9effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b3b60 0x55bd73998be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35210/0 (socket says 172.21.15.26:35210) 2024-04-24T23:27:34.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9d9effd640 1 -- 172.21.15.26:0/1411044012 learned_addr learned my addr 172.21.15.26:0/1411044012 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9d9effd640 1 -- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 msgr2=0x55bd739b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9d9effd640 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 0x55bd739b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9d9effd640 1 -- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 msgr2=0x55bd73986790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9d9effd640 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 0x55bd73986790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.939+0000 7f9d9effd640 1 -- 172.21.15.26:0/1411044012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd736ad6b0 con 0x55bd739b3b60 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9d9effd640 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b3b60 0x55bd73998be0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f9d90035040 tx=0x7f9d90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544538fe482e121b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9d867fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9d867fc640 1 -- 172.21.15.26:0/1411044012 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90037040 con 0x55bd739b3b60 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9d867fc640 1 -- 172.21.15.26:0/1411044012 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f9d90003a20 con 0x55bd739b3b60 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1411044012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da0056de0 con 0x55bd739b3b60 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9d867fc640 1 -- 172.21.15.26:0/1411044012 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f9d900044d0 con 0x55bd739b3b60 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9db0b9e740 10 client.8952 fetch_fsmap learned FSMap version 108 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1411044012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x55bd739a9290 con 0x55bd739b3b60 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9d867fc640 1 -- 172.21.15.26:0/1411044012 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0056de0 con 0x55bd739b3b60 2024-04-24T23:27:34.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9d867fc640 1 -- 172.21.15.26:0/1411044012 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 108) ==== 3534+0+0 (secure 0 0 0) 0x7f9d90003c20 con 0x55bd739b3b60 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9db0b9e740 10 client.8952 fetch_fsmap finished waiting for FSMap version 108 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9db0b9e740 10 client.8952 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9db0b9e740 20 client.8952 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55bd73997640 0x55bd739a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9db0b9e740 4 client.8952 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1411044012 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd738dc440 con 0x55bd73997640 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.940+0000 7f9da77fe640 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55bd73997640 0x55bd739a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.941+0000 7f9d867fc640 10 client.8952 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.941+0000 7f9da77fe640 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55bd73997640 0x55bd739a8850 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b606c5e41540c8d server_cookie=feaab556f827c782 in_seq=0 out_seq=0 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.941+0000 7f9d867fc640 1 -- 172.21.15.26:0/1411044012 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd738dc440 con 0x55bd73997640 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.941+0000 7f9d867fc640 10 client.8952 handle_command_reply: tid=0 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.941+0000 7f9db0b9e740 1 client.8952 shutdown 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1411044012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b3b60 msgr2=0x55bd73998be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b3b60 0x55bd73998be0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f9d90035040 tx=0x7f9d90004730 comp rx=0 tx=0).stop 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1411044012 shutdown_connections 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55bd73997640 0x55bd739a8850 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 0x55bd739b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b3b60 0x55bd73998be0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9d867fc640 0 client.8952 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1411044012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73996c80 0x55bd73986790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1411044012 >> 172.21.15.26:0/1411044012 conn(0x55bd73996540 msgr2=0x55bd73992680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1411044012 shutdown_connections 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1411044012 wait complete. 2024-04-24T23:27:34.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.942+0000 7f9db0b9e740 20 client.8952 trim_cache size 0 max 16384 2024-04-24T23:27:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.960+0000 7f9db0b9e740 1 Processor -- start 2024-04-24T23:27:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.960+0000 7f9db0b9e740 1 -- start start 2024-04-24T23:27:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd73996c80 0x55bd739d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 0x55bd73991a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b2550 0x55bd739922a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81ff8c0 con 0x55bd739b2550 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81fff90 con 0x55bd73997640 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9da8159210 con 0x55bd73996c80 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b2550 0x55bd739922a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b2550 0x55bd739922a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35226/0 (socket says 172.21.15.26:35226) 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9d9effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 0x55bd73991a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9da7fff640 1 -- 172.21.15.26:0/1163768725 learned_addr learned my addr 172.21.15.26:0/1163768725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9da77fe640 1 --2- 172.21.15.26:0/1163768725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd73996c80 0x55bd739d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9da7fff640 1 -- 172.21.15.26:0/1163768725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd73996c80 msgr2=0x55bd739d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9da7fff640 1 --2- 172.21.15.26:0/1163768725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd73996c80 0x55bd739d4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9da7fff640 1 -- 172.21.15.26:0/1163768725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 msgr2=0x55bd73991a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9da7fff640 1 --2- 172.21.15.26:0/1163768725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 0x55bd73991a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9da7fff640 1 -- 172.21.15.26:0/1163768725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7399a330 con 0x55bd739b2550 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.961+0000 7f9d9effd640 1 --2- 172.21.15.26:0/1163768725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 0x55bd73991a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9da7fff640 1 --2- 172.21.15.26:0/1163768725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b2550 0x55bd739922a0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f9da0058ca0 tx=0x7f9da0054eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98da18cbc9be390 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9d9cff9640 1 -- 172.21.15.26:0/1163768725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0061610 con 0x55bd739b2550 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9d9cff9640 1 -- 172.21.15.26:0/1163768725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0055150 con 0x55bd739b2550 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9d9cff9640 1 -- 172.21.15.26:0/1163768725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da005f300 con 0x55bd739b2550 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1163768725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b2550 msgr2=0x55bd739922a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1163768725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b2550 0x55bd739922a0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f9da0058ca0 tx=0x7f9da0054eb0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1163768725 shutdown_connections 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1163768725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd739b2550 0x55bd739922a0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1163768725 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 0x55bd73991a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1163768725 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd73996c80 0x55bd739d4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1163768725 >> 172.21.15.26:0/1163768725 conn(0x55bd73996540 msgr2=0x55bd739865a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1163768725 shutdown_connections 2024-04-24T23:27:34.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.962+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1163768725 wait complete. 2024-04-24T23:27:34.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.963+0000 7f9db0b9e740 1 -- start start 2024-04-24T23:27:34.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.963+0000 7f9db0b9e740 1 Processor -- start 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996c80 0x55bd739a6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 0x55bd739d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9db0b9e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 0x55bd739d43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd739c4710 con 0x55bd73996c80 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bd739c54a0 con 0x55bd73997640 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9db0b9e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55bd73997a90 con 0x55bd739b2550 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9db0b9e740 10 client.0 _ll_register_callbacks cb 0x7f9d900041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996c80 0x55bd739a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996c80 0x55bd739a6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35236/0 (socket says 172.21.15.26:35236) 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da77fe640 1 -- 172.21.15.26:0/1414166734 learned_addr learned my addr 172.21.15.26:0/1414166734 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9d9effd640 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 0x55bd739d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da7fff640 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 0x55bd739d43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da77fe640 1 -- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 msgr2=0x55bd739d43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da77fe640 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 0x55bd739d43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da77fe640 1 -- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 msgr2=0x55bd739d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da77fe640 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 0x55bd739d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da77fe640 1 -- 172.21.15.26:0/1414166734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd73967620 con 0x55bd73996c80 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9da77fe640 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996c80 0x55bd739a6990 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800fed0 tx=0x7f9d98001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb427f13c843171e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9d867fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9d867fc640 1 -- 172.21.15.26:0/1414166734 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98014450 con 0x55bd73996c80 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9d867fc640 1 -- 172.21.15.26:0/1414166734 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f9d980145f0 con 0x55bd73996c80 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.964+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1414166734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da0055150 con 0x55bd73996c80 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9d867fc640 1 -- 172.21.15.26:0/1414166734 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f9d9803a050 con 0x55bd73996c80 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9db0b9e740 10 client.8958 fetch_fsmap learned FSMap version 108 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1414166734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x55bd739a6fd0 con 0x55bd73996c80 2024-04-24T23:27:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9d867fc640 1 -- 172.21.15.26:0/1414166734 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0055150 con 0x55bd73996c80 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9d867fc640 1 -- 172.21.15.26:0/1414166734 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 108) ==== 3534+0+0 (secure 0 0 0) 0x7f9d9803d020 con 0x55bd73996c80 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9db0b9e740 10 client.8958 fetch_fsmap finished waiting for FSMap version 108 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9db0b9e740 10 client.8958 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9db0b9e740 20 client.8958 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55bd739b3b60 0x55bd73997f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9db0b9e740 4 client.8958 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1414166734 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "status"}) -- 0x55bd738d6790 con 0x55bd739b3b60 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9d9effd640 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55bd739b3b60 0x55bd73997f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9d9effd640 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55bd739b3b60 0x55bd73997f20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4124ef74bb5ede2 server_cookie=e61ad95cc100e867 in_seq=0 out_seq=0 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.965+0000 7f9d867fc640 10 client.8958 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9d867fc640 1 -- 172.21.15.26:0/1414166734 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55bd738d6790 con 0x55bd739b3b60 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9d867fc640 10 client.8958 handle_command_reply: tid=0 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 client.8958 shutdown 2024-04-24T23:27:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1414166734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996c80 msgr2=0x55bd739a6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996c80 0x55bd739a6990 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800fed0 tx=0x7f9d98001080 comp rx=0 tx=0).stop 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1414166734 shutdown_connections 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55bd739b3b60 0x55bd73997f20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55bd739b2550 0x55bd739d43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9d867fc640 0 client.8958 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bd73997640 0x55bd739d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 --2- 172.21.15.26:0/1414166734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd73996c80 0x55bd739a6990 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1414166734 >> 172.21.15.26:0/1414166734 conn(0x55bd73996540 msgr2=0x55bd73961690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1414166734 shutdown_connections 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 1 -- 172.21.15.26:0/1414166734 wait complete. 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9db0b9e740 20 client.8958 trim_cache size 0 max 16384 2024-04-24T23:27:34.973 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "5a872e4b-97fd-4255-95a5-3ea24d1048cf", 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8730, 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 11.030548152, 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 108, 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 96, 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 96, 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 16.038762551000001 2024-04-24T23:27:34.974 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9dade93640 1 -- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9d8806be60 msgr2=0x7f9d8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9dade93640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9d8806be60 0x7f9d8806e320 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9d98000f60 tx=0x7f9d98009040 comp rx=0 tx=0).stop 2024-04-24T23:27:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9dade93640 1 -- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da815e570 msgr2=0x7f9da81525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9dade93640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da815e570 0x7f9da81525c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9da000cb40 tx=0x7f9da000a060 comp rx=0 tx=0).stop 2024-04-24T23:27:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9dade93640 1 -- 172.21.15.26:0/3512989233 shutdown_connections 2024-04-24T23:27:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9dade93640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f9d8806be60 0x7f9d8806e320 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9dade93640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9da815e570 0x7f9da81525c0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9dade93640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9da80d4870 0x7f9da80cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.966+0000 7f9dade93640 1 --2- 172.21.15.26:0/3512989233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9da80b3130 0x7f9da815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.967+0000 7f9dade93640 1 -- 172.21.15.26:0/3512989233 >> 172.21.15.26:0/3512989233 conn(0x7f9da808dbe0 msgr2=0x7f9da80d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.967+0000 7f9dade93640 1 -- 172.21.15.26:0/3512989233 shutdown_connections 2024-04-24T23:27:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:34.967+0000 7f9dade93640 1 -- 172.21.15.26:0/3512989233 wait complete. 2024-04-24T23:27:34.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.078+0000 7f132b4d4640 1 Processor -- start 2024-04-24T23:27:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.078+0000 7f132b4d4640 1 -- start start 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.079+0000 7f132b4d4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f1324145910 0x7f132406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.079+0000 7f132b4d4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13240cfbc0 con 0x7f13240cd660 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.079+0000 7f132b4d4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f132406d7f0 con 0x7f132406d350 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.079+0000 7f132b4d4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f1324145cf0 con 0x7f1324145910 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.079+0000 7f1329a71640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f13240cd660 0x7f132415d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44438/0 (socket says 172.21.15.26:44438) 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.079+0000 7f1329a71640 1 -- 172.21.15.26:0/3654600565 learned_addr learned my addr 172.21.15.26:0/3654600565 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1329270640 1 --2- 172.21.15.26:0/3654600565 >> v2:172.21.15.29:3300/0 conn(0x7f1324145910 0x7f132406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138736923 0 0) 0x7f13240cfbc0 con 0x7f13240cd660 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f130c003520 con 0x7f13240cd660 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3106197578 0 0) 0x7f130c003520 con 0x7f13240cd660 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 >> v2:172.21.15.29:3300/0 conn(0x7f1324145910 msgr2=0x7f132406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:35.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 --2- 172.21.15.26:0/3654600565 >> v2:172.21.15.29:3300/0 conn(0x7f1324145910 0x7f132406e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 >> v1:172.21.15.29:6789/0 conn(0x7f132406d350 legacy=0x7f132415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132406dad0 con 0x7f13240cd660 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f1314001670 con 0x7f13240cd660 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 >> v1:172.21.15.26:6789/0 conn(0x7f13240cd660 legacy=0x7f132415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130c0039b0 0x7f130c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c0043b0 0x7f130c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c005120 0x7f130c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1324145cf0 con 0x7f130c005120 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f130c00c060 con 0x7f130c0043b0 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f130c00c260 con 0x7f130c0039b0 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1329a71640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c005120 0x7f130c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1329270640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130c0039b0 0x7f130c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1328a6f640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c0043b0 0x7f130c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1329a71640 1 -- 172.21.15.26:0/3654600565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130c0039b0 msgr2=0x7f130c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1329a71640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f130c0039b0 0x7f130c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1329a71640 1 -- 172.21.15.26:0/3654600565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c0043b0 msgr2=0x7f130c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1329a71640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c0043b0 0x7f130c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.080+0000 7f1329a71640 1 -- 172.21.15.26:0/3654600565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c003520 con 0x7f130c005120 2024-04-24T23:27:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.081+0000 7f1329a71640 1 -- 172.21.15.26:0/3654600565 reap_dead start 2024-04-24T23:27:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.081+0000 7f1329a71640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c005120 0x7f130c00b9c0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f131400c240 tx=0x7f130c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9458a6a33ee83979 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.081+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314007d30 con 0x7f130c005120 2024-04-24T23:27:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.081+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1314005af0 con 0x7f130c005120 2024-04-24T23:27:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.081+0000 7f1323fff640 1 -- 172.21.15.26:0/3654600565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314007070 con 0x7f130c005120 2024-04-24T23:27:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.082+0000 7f132b4d4640 1 -- 172.21.15.26:0/3654600565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c005120 msgr2=0x7f130c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.082+0000 7f132b4d4640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c005120 0x7f130c00b9c0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f131400c240 tx=0x7f130c005830 comp rx=0 tx=0).stop 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.082+0000 7f132b4d4640 1 -- 172.21.15.26:0/3654600565 shutdown_connections 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.082+0000 7f132b4d4640 1 --2- 172.21.15.26:0/3654600565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f130c005120 0x7f130c00b9c0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.082+0000 7f132b4d4640 1 -- 172.21.15.26:0/3654600565 >> 172.21.15.26:0/3654600565 conn(0x7f132406b300 msgr2=0x7f132406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.082+0000 7f132b4d4640 1 -- 172.21.15.26:0/3654600565 shutdown_connections 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.082+0000 7f132b4d4640 1 -- 172.21.15.26:0/3654600565 wait complete. 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.083+0000 7f132b4d4640 1 Processor -- start 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.083+0000 7f132b4d4640 1 -- start start 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f132b4d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c005120 0x7f13240ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f132b4d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240ae7c0 0x7f13241f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f132b4d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13241f73e0 0x7f1324151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f132b4d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f132406e370 con 0x7f13240ae7c0 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f132b4d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f132406e090 con 0x7f130c005120 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f132b4d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f132406e890 con 0x7f13241f73e0 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1328a6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240ae7c0 0x7f13241f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1328a6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240ae7c0 0x7f13241f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35248/0 (socket says 172.21.15.26:35248) 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1328a6f640 1 -- 172.21.15.26:0/834413769 learned_addr learned my addr 172.21.15.26:0/834413769 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1329a71640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13241f73e0 0x7f1324151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1329270640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c005120 0x7f13240ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1328a6f640 1 -- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13241f73e0 msgr2=0x7f1324151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1328a6f640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13241f73e0 0x7f1324151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1328a6f640 1 -- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c005120 msgr2=0x7f13240ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1328a6f640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c005120 0x7f13240ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1328a6f640 1 -- 172.21.15.26:0/834413769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132413a0f0 con 0x7f13240ae7c0 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1329a71640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13241f73e0 0x7f1324151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1329270640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c005120 0x7f13240ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.084+0000 7f1328a6f640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240ae7c0 0x7f13241f6ea0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f131800f7c0 tx=0x7f131800fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.085+0000 7f1321ffb640 1 -- 172.21.15.26:0/834413769 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131800fe70 con 0x7f13240ae7c0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.085+0000 7f1321ffb640 1 -- 172.21.15.26:0/834413769 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1318018070 con 0x7f13240ae7c0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.085+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13240aec90 con 0x7f13240ae7c0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.085+0000 7f1321ffb640 1 -- 172.21.15.26:0/834413769 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131800d730 con 0x7f13240ae7c0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.085+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132413a370 con 0x7f13240ae7c0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.086+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13240694c0 con 0x7f13240ae7c0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.086+0000 7f1321ffb640 1 -- 172.21.15.26:0/834413769 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f131800d8d0 con 0x7f13240ae7c0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.086+0000 7f1321ffb640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f131006be30 0x7f131006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.087+0000 7f1321ffb640 1 -- 172.21.15.26:0/834413769 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f131808ad50 con 0x7f13240ae7c0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.087+0000 7f1329270640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f131006be30 0x7f131006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.087+0000 7f1329270640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f131006be30 0x7f131006e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f131c007440 tx=0x7f131c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.091+0000 7f1321ffb640 1 -- 172.21.15.26:0/834413769 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f131808e030 con 0x7f13240ae7c0 2024-04-24T23:27:35.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.313+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f132406b930 con 0x7f13240ae7c0 2024-04-24T23:27:35.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.314+0000 7f1321ffb640 1 -- 172.21.15.26:0/834413769 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5225 (secure 0 0 0) 0x7f132413a0f0 con 0x7f13240ae7c0 2024-04-24T23:27:35.315 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:35.315 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":108,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:33.973136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[7444,8712,8730]},"id":7}]} 2024-04-24T23:27:35.315 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 108 2024-04-24T23:27:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f131006be30 msgr2=0x7f131006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f131006be30 0x7f131006e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f131c007440 tx=0x7f131c00b040 comp rx=0 tx=0).stop 2024-04-24T23:27:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240ae7c0 msgr2=0x7f13241f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240ae7c0 0x7f13241f6ea0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f131800f7c0 tx=0x7f131800fc80 comp rx=0 tx=0).stop 2024-04-24T23:27:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 shutdown_connections 2024-04-24T23:27:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f131006be30 0x7f131006e2f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13241f73e0 0x7f1324151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240ae7c0 0x7f13241f6ea0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 --2- 172.21.15.26:0/834413769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f130c005120 0x7f13240ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 >> 172.21.15.26:0/834413769 conn(0x7f132406b300 msgr2=0x7f1324144aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 shutdown_connections 2024-04-24T23:27:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:35.316+0000 7f132b4d4640 1 -- 172.21.15.26:0/834413769 wait complete. 2024-04-24T23:27:35.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:35.328 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:27:35.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:35.353 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-04-24T23:27:35.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:35.414 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-04-24T23:27:35.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:35.471 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-04-24T23:27:35.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:35.528 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-04-24T23:27:35.583 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:27:35.708 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:27:35.709 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:27:35.709 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:27:37.710 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:27:37.840 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:27:37.840 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:27:37.840 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:27:39.842 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:27:39.973 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:27:39.974 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:27:39.974 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:27:41.975 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:27:42.105 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-04-24T23:27:42.105 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:27:42.105 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:27:44.107 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-04-24T23:27:44.237 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-04-24T23:27:44.238 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:27:44.238 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-04-24T23:27:44.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:27:44.368 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:27:44.369 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:27:44.369 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:27:46.370 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:27:46.500 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:27:46.501 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:27:46.501 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:27:48.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:27:48.633 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:27:48.633 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:27:48.634 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:27:50.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-04-24T23:27:50.764 INFO:teuthology.orchestra.run.smithi026.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-04-24T23:27:50.765 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-04-24T23:27:50.765 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:27:50.766 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-04-24T23:27:50.895 INFO:teuthology.orchestra.run.smithi029.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-04-24T23:27:50.896 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-04-24T23:27:50.896 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-04-24T23:27:50.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 flush journal 2024-04-24T23:27:51.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.002+0000 7fcd98c9a640 1 Processor -- start 2024-04-24T23:27:51.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.002+0000 7fcd98c9a640 1 -- start start 2024-04-24T23:27:51.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.003+0000 7fcd98c9a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fcd9415e370 0x7fcd940d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.003+0000 7fcd98c9a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd940d2120 con 0x7fcd9415e750 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.003+0000 7fcd98c9a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd940b3740 con 0x7fcd940d4c10 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.003+0000 7fcd98c9a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fcd940d1510 con 0x7fcd9415e370 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.003+0000 7fcd91d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcd9415e750 0x7fcd940b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57890/0 (socket says 172.21.15.26:57890) 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.003+0000 7fcd91d9b640 1 -- 172.21.15.26:0/1913578618 learned_addr learned my addr 172.21.15.26:0/1913578618 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.004+0000 7fcd9259c640 1 --2- 172.21.15.26:0/1913578618 >> v2:172.21.15.29:3300/0 conn(0x7fcd9415e370 0x7fcd940d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.004+0000 7fcd9159a640 1 -- 172.21.15.26:0/1913578618 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 323094884 0 0) 0x7fcd940d2120 con 0x7fcd9415e750 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.004+0000 7fcd9159a640 1 -- 172.21.15.26:0/1913578618 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd80003520 con 0x7fcd9415e750 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.004+0000 7fcd9159a640 1 -- 172.21.15.26:0/1913578618 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3411668416 0 0) 0x7fcd80003520 con 0x7fcd9415e750 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.004+0000 7fcd9259c640 1 -- 172.21.15.26:0/1913578618 >> v1:172.21.15.29:6789/0 conn(0x7fcd940d4c10 legacy=0x7fcd940b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.004+0000 7fcd9259c640 1 -- 172.21.15.26:0/1913578618 >> v1:172.21.15.26:6789/0 conn(0x7fcd9415e750 legacy=0x7fcd940b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.004+0000 7fcd9259c640 1 -- 172.21.15.26:0/1913578618 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd940b3a20 con 0x7fcd9415e370 2024-04-24T23:27:51.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.005+0000 7fcd9259c640 1 --2- 172.21.15.26:0/1913578618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd9415e370 0x7fcd940d2520 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0097a0 tx=0x7fcd8c02a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80347e9cebf8fdc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.005+0000 7fcd9159a640 1 -- 172.21.15.26:0/1913578618 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c037070 con 0x7fcd9415e370 2024-04-24T23:27:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.005+0000 7fcd9159a640 1 -- 172.21.15.26:0/1913578618 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c02ab60 con 0x7fcd9415e370 2024-04-24T23:27:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.005+0000 7fcd9159a640 1 -- 172.21.15.26:0/1913578618 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c001670 con 0x7fcd9415e370 2024-04-24T23:27:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.005+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1913578618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd9415e370 msgr2=0x7fcd940d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.005+0000 7fcd98c9a640 1 --2- 172.21.15.26:0/1913578618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd9415e370 0x7fcd940d2520 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0097a0 tx=0x7fcd8c02a0d0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.005+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1913578618 shutdown_connections 2024-04-24T23:27:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.005+0000 7fcd98c9a640 1 --2- 172.21.15.26:0/1913578618 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd9415e370 0x7fcd940d2520 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.005+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1913578618 >> 172.21.15.26:0/1913578618 conn(0x7fcd940d40f0 msgr2=0x7fcd940d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.006+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1913578618 shutdown_connections 2024-04-24T23:27:51.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.006+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1913578618 wait complete. 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.007+0000 7fcd98c9a640 1 Processor -- start 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.007+0000 7fcd98c9a640 1 -- start start 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.007+0000 7fcd98c9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940d4c10 0x7fcd940ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd98c9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd9415e370 0x7fcd94152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd98c9a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcd940ccc90 0x7fcd940cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd98c9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcd940cfca0 con 0x7fcd9415e370 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd98c9a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcd940d1ed0 con 0x7fcd940d4c10 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd98c9a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fcd940cfe20 con 0x7fcd940ccc90 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd9415e370 0x7fcd94152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd9415e370 0x7fcd94152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56114/0 (socket says 172.21.15.26:56114) 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd91d9b640 1 -- 172.21.15.26:0/1429761277 learned_addr learned my addr 172.21.15.26:0/1429761277 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd9259c640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940d4c10 0x7fcd940ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcd940ccc90 0x7fcd940cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd91d9b640 1 -- 172.21.15.26:0/1429761277 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcd940ccc90 msgr2=0x7fcd940cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd91d9b640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcd940ccc90 0x7fcd940cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd91d9b640 1 -- 172.21.15.26:0/1429761277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940d4c10 msgr2=0x7fcd940ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd91d9b640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940d4c10 0x7fcd940ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd91d9b640 1 -- 172.21.15.26:0/1429761277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd940c9810 con 0x7fcd9415e370 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcd940ccc90 0x7fcd940cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd91d9b640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd9415e370 0x7fcd94152ec0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c550 tx=0x7fcd7c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd8affd640 1 -- 172.21.15.26:0/1429761277 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c018070 con 0x7fcd9415e370 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd8affd640 1 -- 172.21.15.26:0/1429761277 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c0074e0 con 0x7fcd9415e370 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1429761277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd940c9af0 con 0x7fcd9415e370 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.008+0000 7fcd8affd640 1 -- 172.21.15.26:0/1429761277 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c004a20 con 0x7fcd9415e370 2024-04-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.009+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1429761277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd940ca0b0 con 0x7fcd9415e370 2024-04-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.010+0000 7fcd8affd640 1 -- 172.21.15.26:0/1429761277 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd7c004bc0 con 0x7fcd9415e370 2024-04-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.010+0000 7fcd8affd640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcd7406be30 0x7fcd7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.010+0000 7fcd8affd640 1 -- 172.21.15.26:0/1429761277 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fcd7c08a300 con 0x7fcd9415e370 2024-04-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.010+0000 7fcd9259c640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcd7406be30 0x7fcd7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.011+0000 7fcd9259c640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcd7406be30 0x7fcd7406e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c034000 tx=0x7fcd8c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.011+0000 7fcd9b9a2740 1 Processor -- start 2024-04-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd9b9a2740 1 -- start start 2024-04-24T23:27:51.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 0x55a9b0a6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a8c600 0x55a9b0a8cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a44a40 0x55a9b0a8acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a9b09a86b0 con 0x55a9b0a8c600 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a9b0950880 con 0x55a9b0a44a40 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a9b09a4670 con 0x55a9b0a6ed60 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a8c600 0x55a9b0a8cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd91d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a8c600 0x55a9b0a8cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56120/0 (socket says 172.21.15.26:56120) 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd91d9b640 1 -- 172.21.15.26:0/3770600042 learned_addr learned my addr 172.21.15.26:0/3770600042 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd9259c640 1 --2- 172.21.15.26:0/3770600042 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 0x55a9b0a6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.012+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/3770600042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a44a40 0x55a9b0a8acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.013+0000 7fcd91d9b640 1 -- 172.21.15.26:0/3770600042 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 msgr2=0x55a9b0a6f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.013+0000 7fcd91d9b640 1 --2- 172.21.15.26:0/3770600042 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 0x55a9b0a6f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.013+0000 7fcd91d9b640 1 -- 172.21.15.26:0/3770600042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a44a40 msgr2=0x55a9b0a8acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.013+0000 7fcd91d9b640 1 --2- 172.21.15.26:0/3770600042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a44a40 0x55a9b0a8acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.013+0000 7fcd91d9b640 1 -- 172.21.15.26:0/3770600042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b07867d0 con 0x55a9b0a8c600 2024-04-24T23:27:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.013+0000 7fcd91d9b640 1 --2- 172.21.15.26:0/3770600042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a8c600 0x55a9b0a8cb50 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c060b40 tx=0x7fcd7c057f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ae9456c6601e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.013+0000 7fcd73fff640 1 -- 172.21.15.26:0/3770600042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c056550 con 0x55a9b0a8c600 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.013+0000 7fcd73fff640 1 -- 172.21.15.26:0/3770600042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c0567b0 con 0x55a9b0a8c600 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.013+0000 7fcd73fff640 1 -- 172.21.15.26:0/3770600042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c057900 con 0x55a9b0a8c600 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3770600042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a8c600 msgr2=0x55a9b0a8cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/3770600042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a8c600 0x55a9b0a8cb50 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c060b40 tx=0x7fcd7c057f10 comp rx=0 tx=0).stop 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3770600042 shutdown_connections 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/3770600042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a44a40 0x55a9b0a8acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/3770600042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a8c600 0x55a9b0a8cb50 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/3770600042 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 0x55a9b0a6f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3770600042 >> 172.21.15.26:0/3770600042 conn(0x55a9b0a41600 msgr2=0x55a9b0a89b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3770600042 shutdown_connections 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3770600042 wait complete. 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 -- start start 2024-04-24T23:27:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.014+0000 7fcd9b9a2740 1 Processor -- start 2024-04-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.015+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 0x55a9b0a711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.015+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a8c600 0x55a9b0a81c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.015+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a71700 0x55a9b0a821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.015+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a9b0a457e0 con 0x55a9b0a71700 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.015+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a9b0963f50 con 0x55a9b0a8c600 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.015+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a9b0a3f530 con 0x55a9b0a6ed60 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.015+0000 7fcd9b9a2740 10 client.0 _ll_register_callbacks cb 0x55a9b099f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd92d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a71700 0x55a9b0a821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd9259c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 0x55a9b0a711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd92d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a71700 0x55a9b0a821a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56122/0 (socket says 172.21.15.26:56122) 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd92d9d640 1 -- 172.21.15.26:0/151148245 learned_addr learned my addr 172.21.15.26:0/151148245 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd9259c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 0x55a9b0a711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:48566/0 (socket says 172.21.15.26:48566) 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd91d9b640 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a8c600 0x55a9b0a81c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd92d9d640 1 -- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 msgr2=0x55a9b0a711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 0x55a9b0a711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd92d9d640 1 -- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a8c600 msgr2=0x55a9b0a81c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a8c600 0x55a9b0a81c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd92d9d640 1 -- 172.21.15.26:0/151148245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b06bc3f0 con 0x55a9b0a71700 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a71700 0x55a9b0a821a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fcd840116a0 tx=0x7fcd84011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e140d0bf47949f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.016+0000 7fcd717fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.017+0000 7fcd717fa640 1 -- 172.21.15.26:0/151148245 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84003970 con 0x55a9b0a71700 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.017+0000 7fcd717fa640 1 -- 172.21.15.26:0/151148245 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fcd84003b10 con 0x55a9b0a71700 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.017+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/151148245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd7c0567b0 con 0x55a9b0a71700 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.017+0000 7fcd717fa640 1 -- 172.21.15.26:0/151148245 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fcd84022050 con 0x55a9b0a71700 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.017+0000 7fcd9b9a2740 10 client.8976 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.017+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/151148245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55a9b067c0f0 con 0x55a9b0a71700 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.017+0000 7fcd717fa640 1 -- 172.21.15.26:0/151148245 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c0567b0 con 0x55a9b0a71700 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.017+0000 7fcd717fa640 1 -- 172.21.15.26:0/151148245 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7fcd84025020 con 0x55a9b0a71700 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.018+0000 7fcd9b9a2740 10 client.8976 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.018+0000 7fcd9b9a2740 10 client.8976 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.018+0000 7fcd9b9a2740 20 client.8976 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.018+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a9b0a832f0 0x55a9b0a83770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.018+0000 7fcd9b9a2740 4 client.8976 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.018+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/151148245 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9b09b5bf0 con 0x55a9b0a832f0 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.018+0000 7fcd9259c640 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a9b0a832f0 0x55a9b0a83770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.018+0000 7fcd9259c640 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a9b0a832f0 0x55a9b0a83770 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe1fb11d2ccf21d5 server_cookie=e9bc70327de607fd in_seq=0 out_seq=0 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.018+0000 7fcd717fa640 10 client.8976 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd717fa640 1 -- 172.21.15.26:0/151148245 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9b09b5bf0 con 0x55a9b0a832f0 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd717fa640 10 client.8976 handle_command_reply: tid=0 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 client.8976 shutdown 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/151148245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a71700 msgr2=0x55a9b0a821a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a71700 0x55a9b0a821a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fcd840116a0 tx=0x7fcd84011b60 comp rx=0 tx=0).stop 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/151148245 shutdown_connections 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a9b0a832f0 0x55a9b0a83770 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a71700 0x55a9b0a821a0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a8c600 0x55a9b0a81c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd717fa640 0 client.8976 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/151148245 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a6ed60 0x55a9b0a711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/151148245 >> 172.21.15.26:0/151148245 conn(0x55a9b0a44a40 msgr2=0x55a9b0a8d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/151148245 shutdown_connections 2024-04-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/151148245 wait complete. 2024-04-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.019+0000 7fcd9b9a2740 20 client.8976 trim_cache size 0 max 16384 2024-04-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.034+0000 7fcd9b9a2740 1 Processor -- start 2024-04-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.034+0000 7fcd9b9a2740 1 -- start start 2024-04-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 0x55a9b0a42910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 0x55a9b0a42e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a997d0 0x55a9b0a3e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcd94200730 con 0x55a9b0a997d0 2024-04-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcd941ffbb0 con 0x55a9b0a6ed60 2024-04-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fcd94200000 con 0x55a9b0a8c600 2024-04-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd92d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a997d0 0x55a9b0a3e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd92d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a997d0 0x55a9b0a3e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56130/0 (socket says 172.21.15.26:56130) 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd92d9d640 1 -- 172.21.15.26:0/3975482241 learned_addr learned my addr 172.21.15.26:0/3975482241 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd91d9b640 1 --2- 172.21.15.26:0/3975482241 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 0x55a9b0a42e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd92d9d640 1 -- 172.21.15.26:0/3975482241 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 msgr2=0x55a9b0a42e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/3975482241 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 0x55a9b0a42e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd92d9d640 1 -- 172.21.15.26:0/3975482241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 msgr2=0x55a9b0a42910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd9259c640 1 --2- 172.21.15.26:0/3975482241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 0x55a9b0a42910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/3975482241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 0x55a9b0a42910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd92d9d640 1 -- 172.21.15.26:0/3975482241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b0a6e830 con 0x55a9b0a997d0 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/3975482241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a997d0 0x55a9b0a3e260 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fcd840018c0 tx=0x7fcd8400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff42de07010efe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.035+0000 7fcd737fe640 1 -- 172.21.15.26:0/3975482241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84003ba0 con 0x55a9b0a997d0 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd737fe640 1 -- 172.21.15.26:0/3975482241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8401a070 con 0x55a9b0a997d0 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd737fe640 1 -- 172.21.15.26:0/3975482241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84006b30 con 0x55a9b0a997d0 2024-04-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3975482241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a997d0 msgr2=0x55a9b0a3e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/3975482241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a997d0 0x55a9b0a3e260 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fcd840018c0 tx=0x7fcd8400c000 comp rx=0 tx=0).stop 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3975482241 shutdown_connections 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/3975482241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a997d0 0x55a9b0a3e260 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/3975482241 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 0x55a9b0a42e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/3975482241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 0x55a9b0a42910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3975482241 >> 172.21.15.26:0/3975482241 conn(0x55a9b0a44a40 msgr2=0x55a9b0a6dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3975482241 shutdown_connections 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.036+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/3975482241 wait complete. 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.037+0000 7fcd9b9a2740 1 -- start start 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.037+0000 7fcd9b9a2740 1 Processor -- start 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 0x55a9b0b030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 0x55a9b0a64a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9b9a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a80ac0 0x55a9b0a64f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcd940cbab0 con 0x55a9b0a80ac0 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a9b0a44fc0 con 0x55a9b0a6ed60 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9b9a2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a9b0aa3510 con 0x55a9b0a8c600 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9b9a2740 10 client.0 _ll_register_callbacks cb 0x7fcd84001a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd92d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a80ac0 0x55a9b0a64f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd92d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a80ac0 0x55a9b0a64f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56136/0 (socket says 172.21.15.26:56136) 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd92d9d640 1 -- 172.21.15.26:0/648970587 learned_addr learned my addr 172.21.15.26:0/648970587 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd91d9b640 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 0x55a9b0a64a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9259c640 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 0x55a9b0b030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd92d9d640 1 -- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 msgr2=0x55a9b0a64a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 0x55a9b0a64a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd92d9d640 1 -- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 msgr2=0x55a9b0b030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 0x55a9b0b030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd92d9d640 1 -- 172.21.15.26:0/648970587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b0a68e50 con 0x55a9b0a80ac0 2024-04-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd92d9d640 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a80ac0 0x55a9b0a64f70 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fcd84001890 tx=0x7fcd84001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92d6b82a52795f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd717fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd717fa640 1 -- 172.21.15.26:0/648970587 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8400f7b0 con 0x55a9b0a80ac0 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd717fa640 1 -- 172.21.15.26:0/648970587 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fcd8400f950 con 0x55a9b0a80ac0 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/648970587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd8401a070 con 0x55a9b0a80ac0 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd717fa640 1 -- 172.21.15.26:0/648970587 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fcd8403d050 con 0x55a9b0a80ac0 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.038+0000 7fcd9b9a2740 10 client.8982 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/648970587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55a9b0a8dd60 con 0x55a9b0a80ac0 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd717fa640 1 -- 172.21.15.26:0/648970587 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8401a070 con 0x55a9b0a80ac0 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd717fa640 1 -- 172.21.15.26:0/648970587 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7fcd84040020 con 0x55a9b0a80ac0 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd9b9a2740 10 client.8982 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd9b9a2740 10 client.8982 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd9b9a2740 20 client.8982 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a9b0a7fb10 0x55a9b0a42b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd9b9a2740 4 client.8982 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/648970587 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9b09aff40 con 0x55a9b0a7fb10 2024-04-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd9259c640 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a9b0a7fb10 0x55a9b0a42b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd717fa640 10 client.8982 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.039+0000 7fcd9259c640 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a9b0a7fb10 0x55a9b0a42b50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f810bbe7c501d9a7 server_cookie=e4c57aa62edc22c8 in_seq=0 out_seq=0 2024-04-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd717fa640 1 -- 172.21.15.26:0/648970587 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9b09aff40 con 0x55a9b0a7fb10 2024-04-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd717fa640 10 client.8982 handle_command_reply: tid=0 2024-04-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd9b9a2740 1 client.8982 shutdown 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/648970587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a80ac0 msgr2=0x55a9b0a64f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a80ac0 0x55a9b0a64f70 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fcd84001890 tx=0x7fcd84001250 comp rx=0 tx=0).stop 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/648970587 shutdown_connections 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a9b0a7fb10 0x55a9b0a42b50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9b0a80ac0 0x55a9b0a64f70 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a9b0a8c600 0x55a9b0a64a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd9b9a2740 1 --2- 172.21.15.26:0/648970587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a9b0a6ed60 0x55a9b0b030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd717fa640 0 client.8982 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.050+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/648970587 >> 172.21.15.26:0/648970587 conn(0x55a9b0a44a40 msgr2=0x55a9b0a6dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/648970587 shutdown_connections 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd9b9a2740 1 -- 172.21.15.26:0/648970587 wait complete. 2024-04-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd9b9a2740 20 client.8982 trim_cache size 0 max 16384 2024-04-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcd7406be30 msgr2=0x7fcd7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcd7406be30 0x7fcd7406e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c034000 tx=0x7fcd8c02e000 comp rx=0 tx=0).stop 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd9415e370 msgr2=0x7fcd94152ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd9415e370 0x7fcd94152ec0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c550 tx=0x7fcd7c00ca10 comp rx=0 tx=0).stop 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1429761277 shutdown_connections 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcd7406be30 0x7fcd7406e2f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcd940ccc90 0x7fcd940cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd9415e370 0x7fcd94152ec0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 --2- 172.21.15.26:0/1429761277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd940d4c10 0x7fcd940ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1429761277 >> 172.21.15.26:0/1429761277 conn(0x7fcd940d40f0 msgr2=0x7fcd940d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1429761277 shutdown_connections 2024-04-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.051+0000 7fcd98c9a640 1 -- 172.21.15.26:0/1429761277 wait complete. 2024-04-24T23:27:51.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 flush journal 2024-04-24T23:27:51.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.166+0000 7fd00defc640 1 Processor -- start 2024-04-24T23:27:51.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.166+0000 7fd00defc640 1 -- start start 2024-04-24T23:27:51.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.167+0000 7fd00defc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd0080d0750 0x7fd008165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.167+0000 7fd00defc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00815ef60 con 0x7fd00815adc0 2024-04-24T23:27:51.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.167+0000 7fd00defc640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00815f240 con 0x7fd0080ad7b0 2024-04-24T23:27:51.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.167+0000 7fd00defc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd00815f520 con 0x7fd0080d0750 2024-04-24T23:27:51.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.167+0000 7fd0077fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd00815adc0 0x7fd0080b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57896/0 (socket says 172.21.15.26:57896) 2024-04-24T23:27:51.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.167+0000 7fd0077fe640 1 -- 172.21.15.26:0/3130769403 learned_addr learned my addr 172.21.15.26:0/3130769403 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:51.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.168+0000 7fd007fff640 1 --2- 172.21.15.26:0/3130769403 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd0080d0750 0x7fd008165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.168+0000 7fd0067fc640 1 -- 172.21.15.26:0/3130769403 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 474307538 0 0) 0x7fd00815ef60 con 0x7fd00815adc0 2024-04-24T23:27:51.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.168+0000 7fd0067fc640 1 -- 172.21.15.26:0/3130769403 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcff0003520 con 0x7fd00815adc0 2024-04-24T23:27:51.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.168+0000 7fd0067fc640 1 -- 172.21.15.26:0/3130769403 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4275822389 0 0) 0x7fd00815f240 con 0x7fd0080ad7b0 2024-04-24T23:27:51.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.168+0000 7fd007fff640 1 -- 172.21.15.26:0/3130769403 >> v1:172.21.15.29:6789/0 conn(0x7fd0080ad7b0 legacy=0x7fd0080d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.168+0000 7fd007fff640 1 -- 172.21.15.26:0/3130769403 >> v1:172.21.15.26:6789/0 conn(0x7fd00815adc0 legacy=0x7fd0080b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.168+0000 7fd007fff640 1 -- 172.21.15.26:0/3130769403 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00815f800 con 0x7fd0080d0750 2024-04-24T23:27:51.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd007fff640 1 --2- 172.21.15.26:0/3130769403 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd0080d0750 0x7fd008165d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd000009df0 tx=0x7fd00002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87b56c9f1133ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd0067fc640 1 -- 172.21.15.26:0/3130769403 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00002ae60 con 0x7fd0080d0750 2024-04-24T23:27:51.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd0067fc640 1 -- 172.21.15.26:0/3130769403 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd000007740 con 0x7fd0080d0750 2024-04-24T23:27:51.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd0067fc640 1 -- 172.21.15.26:0/3130769403 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000007b10 con 0x7fd0080d0750 2024-04-24T23:27:51.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd00defc640 1 -- 172.21.15.26:0/3130769403 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd0080d0750 msgr2=0x7fd008165d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd00defc640 1 --2- 172.21.15.26:0/3130769403 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd0080d0750 0x7fd008165d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd000009df0 tx=0x7fd00002a680 comp rx=0 tx=0).stop 2024-04-24T23:27:51.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd00defc640 1 -- 172.21.15.26:0/3130769403 shutdown_connections 2024-04-24T23:27:51.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd00defc640 1 --2- 172.21.15.26:0/3130769403 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd0080d0750 0x7fd008165d50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd00defc640 1 -- 172.21.15.26:0/3130769403 >> 172.21.15.26:0/3130769403 conn(0x7fd00806b540 msgr2=0x7fd00806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd00defc640 1 -- 172.21.15.26:0/3130769403 shutdown_connections 2024-04-24T23:27:51.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.169+0000 7fd00defc640 1 -- 172.21.15.26:0/3130769403 wait complete. 2024-04-24T23:27:51.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.171+0000 7fd00defc640 1 Processor -- start 2024-04-24T23:27:51.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.171+0000 7fd00defc640 1 -- start start 2024-04-24T23:27:51.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.171+0000 7fd00defc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0080ad7b0 0x7fd008153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.171+0000 7fd00defc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d0750 0x7fd0080ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.171+0000 7fd00defc640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd00815adc0 0x7fd0080cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.171+0000 7fd00defc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd00815e9c0 con 0x7fd0080d0750 2024-04-24T23:27:51.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.171+0000 7fd00defc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd00815b1a0 con 0x7fd0080ad7b0 2024-04-24T23:27:51.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.171+0000 7fd00defc640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd00815eb40 con 0x7fd00815adc0 2024-04-24T23:27:51.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd006ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d0750 0x7fd0080ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd006ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d0750 0x7fd0080ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56152/0 (socket says 172.21.15.26:56152) 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd006ffd640 1 -- 172.21.15.26:0/3017006134 learned_addr learned my addr 172.21.15.26:0/3017006134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd0077fe640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0080ad7b0 0x7fd008153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd007fff640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd00815adc0 0x7fd0080cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd006ffd640 1 -- 172.21.15.26:0/3017006134 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd00815adc0 msgr2=0x7fd0080cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd006ffd640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd00815adc0 0x7fd0080cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd006ffd640 1 -- 172.21.15.26:0/3017006134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0080ad7b0 msgr2=0x7fd008153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd006ffd640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0080ad7b0 0x7fd008153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd006ffd640 1 -- 172.21.15.26:0/3017006134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0081593b0 con 0x7fd0080d0750 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd006ffd640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d0750 0x7fd0080ccd90 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c550 tx=0x7fcffc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fcfeffff640 1 -- 172.21.15.26:0/3017006134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc017070 con 0x7fd0080d0750 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fcfeffff640 1 -- 172.21.15.26:0/3017006134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc0035f0 con 0x7fd0080d0750 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd00defc640 1 -- 172.21.15.26:0/3017006134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd008159690 con 0x7fd0080d0750 2024-04-24T23:27:51.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fcfeffff640 1 -- 172.21.15.26:0/3017006134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc0039e0 con 0x7fd0080d0750 2024-04-24T23:27:51.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.172+0000 7fd00defc640 1 -- 172.21.15.26:0/3017006134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd008159c50 con 0x7fd0080d0750 2024-04-24T23:27:51.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.174+0000 7fcfeffff640 1 -- 172.21.15.26:0/3017006134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcffc01a070 con 0x7fd0080d0750 2024-04-24T23:27:51.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.174+0000 7fcfeffff640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcff406be30 0x7fcff406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.174+0000 7fcfeffff640 1 -- 172.21.15.26:0/3017006134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fcffc08a180 con 0x7fd0080d0750 2024-04-24T23:27:51.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.174+0000 7fd0077fe640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcff406be30 0x7fcff406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.175+0000 7fd0077fe640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcff406be30 0x7fcff406e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcff8005880 tx=0x7fcff8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.175+0000 7fd010c09740 1 Processor -- start 2024-04-24T23:27:51.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.175+0000 7fd010c09740 1 -- start start 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd010c09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 0x5617879ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd010c09740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd010c09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879a3a40 0x5617879e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd010c09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5617879076b0 con 0x5617879a3a40 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd010c09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617878af880 con 0x5617879cdd60 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd010c09740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561787903670 con 0x5617879eb600 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd007fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879a3a40 0x5617879e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd007fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879a3a40 0x5617879e9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56162/0 (socket says 172.21.15.26:56162) 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd007fff640 1 -- 172.21.15.26:0/2336596474 learned_addr learned my addr 172.21.15.26:0/2336596474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd0077fe640 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 0x5617879ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd006ffd640 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd007fff640 1 -- 172.21.15.26:0/2336596474 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 msgr2=0x5617879ebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd007fff640 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879ebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd007fff640 1 -- 172.21.15.26:0/2336596474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 msgr2=0x5617879ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd007fff640 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 0x5617879ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd007fff640 1 -- 172.21.15.26:0/2336596474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617876e57d0 con 0x5617879a3a40 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.176+0000 7fd006ffd640 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fd007fff640 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879a3a40 0x5617879e9cc0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd000009df0 tx=0x7fd00002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb03cdc841a4398 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fcfed7fa640 1 -- 172.21.15.26:0/2336596474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000003b90 con 0x5617879a3a40 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fcfed7fa640 1 -- 172.21.15.26:0/2336596474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd000003d30 con 0x5617879a3a40 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fcfed7fa640 1 -- 172.21.15.26:0/2336596474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000004240 con 0x5617879a3a40 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fd010c09740 1 -- 172.21.15.26:0/2336596474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879a3a40 msgr2=0x5617879e9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fd010c09740 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879a3a40 0x5617879e9cc0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd000009df0 tx=0x7fd00002e000 comp rx=0 tx=0).stop 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fd010c09740 1 -- 172.21.15.26:0/2336596474 shutdown_connections 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fd010c09740 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879a3a40 0x5617879e9cc0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fd010c09740 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.177+0000 7fd010c09740 1 --2- 172.21.15.26:0/2336596474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 0x5617879ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.178+0000 7fd010c09740 1 -- 172.21.15.26:0/2336596474 >> 172.21.15.26:0/2336596474 conn(0x5617879a0600 msgr2=0x5617879e8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.178+0000 7fd010c09740 1 -- 172.21.15.26:0/2336596474 shutdown_connections 2024-04-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.178+0000 7fd010c09740 1 -- 172.21.15.26:0/2336596474 wait complete. 2024-04-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.178+0000 7fd010c09740 1 -- start start 2024-04-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.178+0000 7fd010c09740 1 Processor -- start 2024-04-24T23:27:51.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd010c09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 0x5617879d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd010c09740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd010c09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd010c09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5617879a47e0 con 0x5617879cef30 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd010c09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617878c2f50 con 0x5617879cdd60 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd010c09740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56178799e530 con 0x5617879eb600 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd010c09740 10 client.0 _ll_register_callbacks cb 0x5617878fe000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd007fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd006ffd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd007fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56164/0 (socket says 172.21.15.26:56164) 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd007fff640 1 -- 172.21.15.26:0/2191497102 learned_addr learned my addr 172.21.15.26:0/2191497102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd0077fe640 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 0x5617879d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd007fff640 1 -- 172.21.15.26:0/2191497102 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 msgr2=0x5617879f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd007fff640 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd007fff640 1 -- 172.21.15.26:0/2191497102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 msgr2=0x5617879d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd007fff640 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 0x5617879d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd007fff640 1 -- 172.21.15.26:0/2191497102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56178761b3f0 con 0x5617879cef30 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.179+0000 7fd006ffd640 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd007fff640 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879d0640 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd000009df0 tx=0x7fd000004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226027c02423640f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fcfe67fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2191497102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000004800 con 0x5617879cef30 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2191497102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fd000007540 con 0x5617879cef30 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd010c09740 1 -- 172.21.15.26:0/2191497102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd000003d30 con 0x5617879cef30 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2191497102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fd000007740 con 0x5617879cef30 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd010c09740 10 client.8994 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd010c09740 1 -- 172.21.15.26:0/2191497102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x5617875db0f0 con 0x5617879cef30 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2191497102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd000003d30 con 0x5617879cef30 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2191497102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7fd000044020 con 0x5617879cef30 2024-04-24T23:27:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd010c09740 10 client.8994 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd010c09740 10 client.8994 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd010c09740 20 client.8994 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd010c09740 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x5617879ed9a0 0x5617879eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd010c09740 4 client.8994 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd010c09740 1 -- 172.21.15.26:0/2191497102 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561787914bf0 con 0x5617879ed9a0 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.180+0000 7fd0077fe640 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x5617879ed9a0 0x5617879eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.181+0000 7fcfe67fc640 10 client.8994 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.181+0000 7fd0077fe640 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x5617879ed9a0 0x5617879eddb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e51da793675666a4 server_cookie=aebee998b4bb5004 in_seq=0 out_seq=0 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.181+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2191497102 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561787914bf0 con 0x5617879ed9a0 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.181+0000 7fcfe67fc640 10 client.8994 handle_command_reply: tid=0 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.181+0000 7fd010c09740 1 client.8994 shutdown 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.181+0000 7fd010c09740 1 -- 172.21.15.26:0/2191497102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 msgr2=0x5617879d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.181+0000 7fd010c09740 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879d0640 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd000009df0 tx=0x7fd000004ab0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fd010c09740 1 -- 172.21.15.26:0/2191497102 shutdown_connections 2024-04-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fd010c09740 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x5617879ed9a0 0x5617879eddb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fd010c09740 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879d0640 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fd010c09740 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879eb600 0x5617879f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fcfe67fc640 0 client.8994 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fd010c09740 1 --2- 172.21.15.26:0/2191497102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879cdd60 0x5617879d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fd010c09740 1 -- 172.21.15.26:0/2191497102 >> 172.21.15.26:0/2191497102 conn(0x5617879a3a40 msgr2=0x5617879ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fd010c09740 1 -- 172.21.15.26:0/2191497102 shutdown_connections 2024-04-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fd010c09740 1 -- 172.21.15.26:0/2191497102 wait complete. 2024-04-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.182+0000 7fd010c09740 20 client.8994 trim_cache size 0 max 16384 2024-04-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.196+0000 7fd010c09740 1 Processor -- start 2024-04-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.197+0000 7fd010c09740 1 -- start start 2024-04-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.197+0000 7fd010c09740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cdd60 0x561787a0d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.197+0000 7fd010c09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.197+0000 7fd010c09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 0x5617879a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.197+0000 7fd010c09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd008200420 con 0x5617879cef30 2024-04-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.197+0000 7fd010c09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd0081ff5f0 con 0x5617879eb600 2024-04-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.197+0000 7fd010c09740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd0081ffcf0 con 0x5617879cdd60 2024-04-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.197+0000 7fd006ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd006ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56172/0 (socket says 172.21.15.26:56172) 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd006ffd640 1 -- 172.21.15.26:0/1147901002 learned_addr learned my addr 172.21.15.26:0/1147901002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd007fff640 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 0x5617879a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd0077fe640 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cdd60 0x561787a0d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd006ffd640 1 -- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cdd60 msgr2=0x561787a0d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd006ffd640 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cdd60 0x561787a0d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd006ffd640 1 -- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 msgr2=0x5617879a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd006ffd640 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 0x5617879a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd006ffd640 1 -- 172.21.15.26:0/1147901002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617879cf310 con 0x5617879cef30 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd007fff640 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 0x5617879a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd0077fe640 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cdd60 0x561787a0d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fd006ffd640 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879a4d80 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fcffc058270 tx=0x7fcffc001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a600788039cf0c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fcfecff9640 1 -- 172.21.15.26:0/1147901002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc070020 con 0x5617879cef30 2024-04-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fcfecff9640 1 -- 172.21.15.26:0/1147901002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc056870 con 0x5617879cef30 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.198+0000 7fcfecff9640 1 -- 172.21.15.26:0/1147901002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc060e10 con 0x5617879cef30 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 -- 172.21.15.26:0/1147901002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 msgr2=0x5617879a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879a4d80 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fcffc058270 tx=0x7fcffc001080 comp rx=0 tx=0).stop 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 -- 172.21.15.26:0/1147901002 shutdown_connections 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 0x5617879a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cef30 0x5617879a4d80 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 --2- 172.21.15.26:0/1147901002 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cdd60 0x561787a0d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 -- 172.21.15.26:0/1147901002 >> 172.21.15.26:0/1147901002 conn(0x5617879a3a40 msgr2=0x5617879d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 -- 172.21.15.26:0/1147901002 shutdown_connections 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 -- 172.21.15.26:0/1147901002 wait complete. 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 -- start start 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.199+0000 7fd010c09740 1 Processor -- start 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd010c09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cdd60 0x5617879df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd010c09740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cef30 0x561787a62480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd010c09740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 0x5617879a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd010c09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd00809cbf0 con 0x5617879cdd60 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd010c09740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5617879ef060 con 0x5617879eb600 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd010c09740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5617879a1b70 con 0x5617879cef30 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd010c09740 10 client.0 _ll_register_callbacks cb 0x7fd00002ae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd0077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cdd60 0x5617879df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd0077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cdd60 0x5617879df130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56186/0 (socket says 172.21.15.26:56186) 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd0077fe640 1 -- 172.21.15.26:0/2575819904 learned_addr learned my addr 172.21.15.26:0/2575819904 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd006ffd640 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cef30 0x561787a62480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd007fff640 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 0x5617879a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd0077fe640 1 -- 172.21.15.26:0/2575819904 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cef30 msgr2=0x561787a62480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd0077fe640 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cef30 0x561787a62480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.200+0000 7fd0077fe640 1 -- 172.21.15.26:0/2575819904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 msgr2=0x5617879a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd0077fe640 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 0x5617879a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd0077fe640 1 -- 172.21.15.26:0/2575819904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617879f98c0 con 0x5617879cdd60 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd0077fe640 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cdd60 0x5617879df130 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fd0080d0420 tx=0x7fcff80010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da08dc998f97590 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fcfe67fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2575819904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8005030 con 0x5617879cdd60 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2575819904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fcff801e040 con 0x5617879cdd60 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd010c09740 1 -- 172.21.15.26:0/2575819904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcffc056870 con 0x5617879cdd60 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2575819904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fcff80051d0 con 0x5617879cdd60 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd010c09740 10 client.9000 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd010c09740 1 -- 172.21.15.26:0/2575819904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x5617879f9ba0 con 0x5617879cdd60 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2575819904 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc056870 con 0x5617879cdd60 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2575819904 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7fcff8005370 con 0x5617879cdd60 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd010c09740 10 client.9000 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd010c09740 10 client.9000 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd010c09740 20 client.9000 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd010c09740 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x5617879a52c0 0x561787a62ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd010c09740 4 client.9000 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd010c09740 1 -- 172.21.15.26:0/2575819904 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56178790ef40 con 0x5617879a52c0 2024-04-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.201+0000 7fd006ffd640 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x5617879a52c0 0x561787a62ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.202+0000 7fcfe67fc640 10 client.9000 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.202+0000 7fd006ffd640 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x5617879a52c0 0x561787a62ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f7991bc2eb8ac76 server_cookie=d7b29bc2bf593d1c in_seq=0 out_seq=0 2024-04-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.208+0000 7fcfe67fc640 1 -- 172.21.15.26:0/2575819904 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56178790ef40 con 0x5617879a52c0 2024-04-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.208+0000 7fcfe67fc640 10 client.9000 handle_command_reply: tid=0 2024-04-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 client.9000 shutdown 2024-04-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 -- 172.21.15.26:0/2575819904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cdd60 msgr2=0x5617879df130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cdd60 0x5617879df130 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fd0080d0420 tx=0x7fcff80010a0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 -- 172.21.15.26:0/2575819904 shutdown_connections 2024-04-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x5617879a52c0 0x561787a62ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5617879eb600 0x5617879a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5617879cef30 0x561787a62480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fcfe67fc640 0 client.9000 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 --2- 172.21.15.26:0/2575819904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617879cdd60 0x5617879df130 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 -- 172.21.15.26:0/2575819904 >> 172.21.15.26:0/2575819904 conn(0x5617879a3a40 msgr2=0x5617879dff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 -- 172.21.15.26:0/2575819904 shutdown_connections 2024-04-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 1 -- 172.21.15.26:0/2575819904 wait complete. 2024-04-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd010c09740 20 client.9000 trim_cache size 0 max 16384 2024-04-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd00defc640 1 -- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcff406be30 msgr2=0x7fcff406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd00defc640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcff406be30 0x7fcff406e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcff8005880 tx=0x7fcff8009040 comp rx=0 tx=0).stop 2024-04-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd00defc640 1 -- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d0750 msgr2=0x7fd0080ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.209+0000 7fd00defc640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d0750 0x7fd0080ccd90 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c550 tx=0x7fcffc00ca10 comp rx=0 tx=0).stop 2024-04-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.210+0000 7fd00defc640 1 -- 172.21.15.26:0/3017006134 shutdown_connections 2024-04-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.210+0000 7fd00defc640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcff406be30 0x7fcff406e2f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.210+0000 7fd00defc640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd00815adc0 0x7fd0080cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.210+0000 7fd00defc640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d0750 0x7fd0080ccd90 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.210+0000 7fd00defc640 1 --2- 172.21.15.26:0/3017006134 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd0080ad7b0 0x7fd008153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.210+0000 7fd00defc640 1 -- 172.21.15.26:0/3017006134 >> 172.21.15.26:0/3017006134 conn(0x7fd00806b540 msgr2=0x7fd00806cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.210+0000 7fd00defc640 1 -- 172.21.15.26:0/3017006134 shutdown_connections 2024-04-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.210+0000 7fd00defc640 1 -- 172.21.15.26:0/3017006134 wait complete. 2024-04-24T23:27:51.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 flush journal 2024-04-24T23:27:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.324+0000 7f21fc5f9640 1 Processor -- start 2024-04-24T23:27:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.324+0000 7f21fc5f9640 1 -- start start 2024-04-24T23:27:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.325+0000 7f21fc5f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f415e750 0x7f21f40cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.325+0000 7f21fc5f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21f40d2c30 0x7f21f40cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.325+0000 7f21fc5f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f40d2120 con 0x7f21f415e370 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.325+0000 7f21fc5f9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f21f40d1510 con 0x7f21f40d2c30 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.325+0000 7f21fc5f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f21f40d1ed0 con 0x7f21f415e750 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.325+0000 7f21fab96640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21f40d2c30 0x7f21f40cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.325+0000 7f21fa395640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21f415e370 0x7f21f40d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57906/0 (socket says 172.21.15.26:57906) 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.325+0000 7f21fa395640 1 -- 172.21.15.26:0/3385118089 learned_addr learned my addr 172.21.15.26:0/3385118089 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.326+0000 7f21f9b94640 1 --2- 172.21.15.26:0/3385118089 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f415e750 0x7f21f40cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.326+0000 7f21fab96640 1 -- 172.21.15.26:0/3385118089 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f415e750 msgr2=0x7f21f40cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.326+0000 7f21fab96640 1 --2- 172.21.15.26:0/3385118089 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f415e750 0x7f21f40cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.326+0000 7f21fab96640 1 -- 172.21.15.26:0/3385118089 >> v1:172.21.15.26:6789/0 conn(0x7f21f415e370 legacy=0x7f21f40d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.326+0000 7f21fab96640 1 -- 172.21.15.26:0/3385118089 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f40b3740 con 0x7f21f40d2c30 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.326+0000 7f21f9393640 1 -- 172.21.15.26:0/3385118089 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1857661434 0 0) 0x7f21f40d2120 con 0x7f21f415e370 2024-04-24T23:27:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.326+0000 7f21f9b94640 1 --2- 172.21.15.26:0/3385118089 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f415e750 0x7f21f40cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.326+0000 7f21fab96640 1 --2- 172.21.15.26:0/3385118089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f40d2c30 0x7f21f40cf1d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f21f40d2050 tx=0x7f21e402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2808e5f5a03fc5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21f9393640 1 -- 172.21.15.26:0/3385118089 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e402ae60 con 0x7f21f40d2c30 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21f9393640 1 -- 172.21.15.26:0/3385118089 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4007740 con 0x7f21f40d2c30 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21f9393640 1 -- 172.21.15.26:0/3385118089 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4007b10 con 0x7f21f40d2c30 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21fc5f9640 1 -- 172.21.15.26:0/3385118089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f40d2c30 msgr2=0x7f21f40cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21fc5f9640 1 --2- 172.21.15.26:0/3385118089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f40d2c30 0x7f21f40cf1d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f21f40d2050 tx=0x7f21e402a640 comp rx=0 tx=0).stop 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21fc5f9640 1 -- 172.21.15.26:0/3385118089 shutdown_connections 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21fc5f9640 1 --2- 172.21.15.26:0/3385118089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f40d2c30 0x7f21f40cf1d0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21fc5f9640 1 --2- 172.21.15.26:0/3385118089 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f415e750 0x7f21f40cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21fc5f9640 1 -- 172.21.15.26:0/3385118089 >> 172.21.15.26:0/3385118089 conn(0x7f21f40d40f0 msgr2=0x7f21f40d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21fc5f9640 1 -- 172.21.15.26:0/3385118089 shutdown_connections 2024-04-24T23:27:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.327+0000 7f21fc5f9640 1 -- 172.21.15.26:0/3385118089 wait complete. 2024-04-24T23:27:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.328+0000 7f21fc5f9640 1 Processor -- start 2024-04-24T23:27:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.328+0000 7f21fc5f9640 1 -- start start 2024-04-24T23:27:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21fc5f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21f40d2c30 0x7f21f40cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21fc5f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f415e370 0x7f21f40b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21fc5f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f40cd9a0 0x7f21f40c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21fc5f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21f40b5230 con 0x7f21f415e370 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21fc5f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f21f40b3d00 con 0x7f21f40d2c30 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21fc5f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f21f40b53b0 con 0x7f21f40cd9a0 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21f9b94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f415e370 0x7f21f40b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21f9b94640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f415e370 0x7f21f40b7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56204/0 (socket says 172.21.15.26:56204) 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21f9b94640 1 -- 172.21.15.26:0/4236985953 learned_addr learned my addr 172.21.15.26:0/4236985953 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21fab96640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f40cd9a0 0x7f21f40c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.329+0000 7f21fa395640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21f40d2c30 0x7f21f40cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21f9b94640 1 -- 172.21.15.26:0/4236985953 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f40cd9a0 msgr2=0x7f21f40c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21f9b94640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f40cd9a0 0x7f21f40c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21f9b94640 1 -- 172.21.15.26:0/4236985953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21f40d2c30 msgr2=0x7f21f40cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21f9b94640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21f40d2c30 0x7f21f40cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21f9b94640 1 -- 172.21.15.26:0/4236985953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f414ffe0 con 0x7f21f415e370 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21f9b94640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f415e370 0x7f21f40b7e40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c7f0 tx=0x7f21e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21f2ffd640 1 -- 172.21.15.26:0/4236985953 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8015070 con 0x7f21f415e370 2024-04-24T23:27:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21f2ffd640 1 -- 172.21.15.26:0/4236985953 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e8007820 con 0x7f21f415e370 2024-04-24T23:27:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21f2ffd640 1 -- 172.21.15.26:0/4236985953 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8007bd0 con 0x7f21f415e370 2024-04-24T23:27:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21fc5f9640 1 -- 172.21.15.26:0/4236985953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f40cc3c0 con 0x7f21f415e370 2024-04-24T23:27:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.330+0000 7f21fc5f9640 1 -- 172.21.15.26:0/4236985953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f40cc980 con 0x7f21f415e370 2024-04-24T23:27:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.332+0000 7f21f2ffd640 1 -- 172.21.15.26:0/4236985953 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21e8018070 con 0x7f21f415e370 2024-04-24T23:27:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.332+0000 7f21f2ffd640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21e006be30 0x7f21e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.332+0000 7f21f2ffd640 1 -- 172.21.15.26:0/4236985953 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f21e808afc0 con 0x7f21f415e370 2024-04-24T23:27:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.332+0000 7f21fa395640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21e006be30 0x7f21e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.333+0000 7f21fa395640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21e006be30 0x7f21e006e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f21ec000f60 tx=0x7f21ec007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.333+0000 7f21ff3e8740 1 Processor -- start 2024-04-24T23:27:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.333+0000 7f21ff3e8740 1 -- start start 2024-04-24T23:27:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c02a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 0x563c7c047b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7bfffa40 0x563c7c045cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563c7bf636b0 con 0x563c7c029d60 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563c7bf0b880 con 0x563c7c047600 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563c7bf5f670 con 0x563c7bfffa40 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21fa395640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c02a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21fa395640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c02a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56212/0 (socket says 172.21.15.26:56212) 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21fa395640 1 -- 172.21.15.26:0/610637209 learned_addr learned my addr 172.21.15.26:0/610637209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21fab96640 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7bfffa40 0x563c7c045cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21f9b94640 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 0x563c7c047b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21fa395640 1 -- 172.21.15.26:0/610637209 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7bfffa40 msgr2=0x563c7c045cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21fa395640 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7bfffa40 0x563c7c045cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21fa395640 1 -- 172.21.15.26:0/610637209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 msgr2=0x563c7c047b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21fa395640 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 0x563c7c047b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21fa395640 1 -- 172.21.15.26:0/610637209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c7bd417d0 con 0x563c7c029d60 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.334+0000 7f21f9b94640 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 0x563c7c047b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21fa395640 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c02a140 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005d80 tx=0x7f21ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd5183500ffe363 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21dbfff640 1 -- 172.21.15.26:0/610637209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0071a0 con 0x563c7c029d60 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21dbfff640 1 -- 172.21.15.26:0/610637209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec015110 con 0x563c7c029d60 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21dbfff640 1 -- 172.21.15.26:0/610637209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec015bc0 con 0x563c7c029d60 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21ff3e8740 1 -- 172.21.15.26:0/610637209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 msgr2=0x563c7c02a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c02a140 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005d80 tx=0x7f21ec012000 comp rx=0 tx=0).stop 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21ff3e8740 1 -- 172.21.15.26:0/610637209 shutdown_connections 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7bfffa40 0x563c7c045cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 0x563c7c047b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/610637209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c02a140 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.335+0000 7f21ff3e8740 1 -- 172.21.15.26:0/610637209 >> 172.21.15.26:0/610637209 conn(0x563c7bffc600 msgr2=0x563c7c044b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.336+0000 7f21ff3e8740 1 -- 172.21.15.26:0/610637209 shutdown_connections 2024-04-24T23:27:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.336+0000 7f21ff3e8740 1 -- 172.21.15.26:0/610637209 wait complete. 2024-04-24T23:27:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.336+0000 7f21ff3e8740 1 -- start start 2024-04-24T23:27:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.336+0000 7f21ff3e8740 1 Processor -- start 2024-04-24T23:27:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c029d60 0x563c7c02cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 0x563c7c03d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c03cd60 0x563c7c03d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563c7c0007e0 con 0x563c7c03cd60 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563c7bf1ef50 con 0x563c7c047600 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563c7bffa530 con 0x563c7c029d60 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21ff3e8740 10 client.0 _ll_register_callbacks cb 0x563c7bf5a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21fab96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c03cd60 0x563c7c03d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21fab96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c03cd60 0x563c7c03d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56224/0 (socket says 172.21.15.26:56224) 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21fab96640 1 -- 172.21.15.26:0/3161624934 learned_addr learned my addr 172.21.15.26:0/3161624934 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21fa395640 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c029d60 0x563c7c02cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21f9b94640 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 0x563c7c03d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21fab96640 1 -- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c029d60 msgr2=0x563c7c02cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21fab96640 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c029d60 0x563c7c02cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21fab96640 1 -- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 msgr2=0x563c7c03d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21fab96640 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 0x563c7c03d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.337+0000 7f21fab96640 1 -- 172.21.15.26:0/3161624934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7bc773f0 con 0x563c7c03cd60 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21fab96640 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c03cd60 0x563c7c03d7a0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009e80 tx=0x7f21e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d52a8b9063c32ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21d97fa640 1 -- 172.21.15.26:0/3161624934 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e403d430 con 0x563c7c03cd60 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21d97fa640 1 -- 172.21.15.26:0/3161624934 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f21e4047aa0 con 0x563c7c03cd60 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3161624934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ec015110 con 0x563c7c03cd60 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21d97fa640 1 -- 172.21.15.26:0/3161624934 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f21e4047ca0 con 0x563c7c03cd60 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21ff3e8740 10 client.9015 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3161624934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x563c7bc370f0 con 0x563c7c03cd60 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21d97fa640 1 -- 172.21.15.26:0/3161624934 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec015110 con 0x563c7c03cd60 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21d97fa640 1 -- 172.21.15.26:0/3161624934 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f21e404e050 con 0x563c7c03cd60 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21ff3e8740 10 client.9015 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21ff3e8740 10 client.9015 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21ff3e8740 20 client.9015 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x563c7c02b1b0 0x563c7c055390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21ff3e8740 4 client.9015 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3161624934 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c7bf70bf0 con 0x563c7c02b1b0 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.338+0000 7f21fa395640 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x563c7c02b1b0 0x563c7c055390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.339+0000 7f21fa395640 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x563c7c02b1b0 0x563c7c055390 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28570aca33af2911 server_cookie=14192d2ab16465b3 in_seq=0 out_seq=0 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.339+0000 7f21d97fa640 10 client.9015 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.339+0000 7f21d97fa640 1 -- 172.21.15.26:0/3161624934 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c7bf70bf0 con 0x563c7c02b1b0 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.339+0000 7f21d97fa640 10 client.9015 handle_command_reply: tid=0 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 client.9015 shutdown 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3161624934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c03cd60 msgr2=0x563c7c03d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c03cd60 0x563c7c03d7a0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009e80 tx=0x7f21e4004ab0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3161624934 shutdown_connections 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x563c7c02b1b0 0x563c7c055390 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c03cd60 0x563c7c03d7a0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c047600 0x563c7c03d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3161624934 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c029d60 0x563c7c02cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21d97fa640 0 client.9015 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3161624934 >> 172.21.15.26:0/3161624934 conn(0x563c7bfffa40 msgr2=0x563c7c048980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3161624934 shutdown_connections 2024-04-24T23:27:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3161624934 wait complete. 2024-04-24T23:27:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.340+0000 7f21ff3e8740 20 client.9015 trim_cache size 0 max 16384 2024-04-24T23:27:51.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.355+0000 7f21ff3e8740 1 Processor -- start 2024-04-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.355+0000 7f21ff3e8740 1 -- start start 2024-04-24T23:27:51.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c01ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c03cd60 0x563c7c000e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c047600 0x563c7c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4200690 con 0x563c7c029d60 2024-04-24T23:27:51.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41ff860 con 0x563c7c03cd60 2024-04-24T23:27:51.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f21f41fff60 con 0x563c7c047600 2024-04-24T23:27:51.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21fa395640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c01ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21fa395640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c01ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56234/0 (socket says 172.21.15.26:56234) 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21fa395640 1 -- 172.21.15.26:0/2918145616 learned_addr learned my addr 172.21.15.26:0/2918145616 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21fa395640 1 -- 172.21.15.26:0/2918145616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c047600 msgr2=0x563c7c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21fa395640 1 --2- 172.21.15.26:0/2918145616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c047600 0x563c7c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21fa395640 1 -- 172.21.15.26:0/2918145616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c03cd60 msgr2=0x563c7c000e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21fa395640 1 --2- 172.21.15.26:0/2918145616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c03cd60 0x563c7c000e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21fa395640 1 -- 172.21.15.26:0/2918145616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c7c03ba50 con 0x563c7c029d60 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21fa395640 1 --2- 172.21.15.26:0/2918145616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c01ad70 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f21ec014000 tx=0x7f21ec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c87ffb1373147b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21db7fe640 1 -- 172.21.15.26:0/2918145616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec010040 con 0x563c7c029d60 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21db7fe640 1 -- 172.21.15.26:0/2918145616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec015ac0 con 0x563c7c029d60 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.356+0000 7f21db7fe640 1 -- 172.21.15.26:0/2918145616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec02f0d0 con 0x563c7c029d60 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.357+0000 7f21ff3e8740 1 -- 172.21.15.26:0/2918145616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 msgr2=0x563c7c01ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.357+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/2918145616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c01ad70 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f21ec014000 tx=0x7f21ec013000 comp rx=0 tx=0).stop 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.357+0000 7f21ff3e8740 1 -- 172.21.15.26:0/2918145616 shutdown_connections 2024-04-24T23:27:51.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.357+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/2918145616 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c047600 0x563c7c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.357+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/2918145616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c03cd60 0x563c7c000e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.357+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/2918145616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c029d60 0x563c7c01ad70 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.357+0000 7f21ff3e8740 1 -- 172.21.15.26:0/2918145616 >> 172.21.15.26:0/2918145616 conn(0x563c7bfffa40 msgr2=0x563c7bffecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.357+0000 7f21ff3e8740 1 -- 172.21.15.26:0/2918145616 shutdown_connections 2024-04-24T23:27:51.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.357+0000 7f21ff3e8740 1 -- 172.21.15.26:0/2918145616 wait complete. 2024-04-24T23:27:51.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.358+0000 7f21ff3e8740 1 -- start start 2024-04-24T23:27:51.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.358+0000 7f21ff3e8740 1 Processor -- start 2024-04-24T23:27:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.358+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c029d60 0x563c7c03e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c03cd60 0x563c7c03a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21ff3e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c047600 0x563c7c0552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563c7bffae40 con 0x563c7c047600 2024-04-24T23:27:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f21f40b79b0 con 0x563c7c029d60 2024-04-24T23:27:51.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21ff3e8740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563c7c001350 con 0x563c7c03cd60 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21ff3e8740 10 client.0 _ll_register_callbacks cb 0x7f21e4009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fab96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c047600 0x563c7c0552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fab96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c047600 0x563c7c0552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56242/0 (socket says 172.21.15.26:56242) 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fab96640 1 -- 172.21.15.26:0/3937734443 learned_addr learned my addr 172.21.15.26:0/3937734443 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21f9b94640 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c03cd60 0x563c7c03a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fa395640 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c029d60 0x563c7c03e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fab96640 1 -- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c03cd60 msgr2=0x563c7c03a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fab96640 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c03cd60 0x563c7c03a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fab96640 1 -- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c029d60 msgr2=0x563c7c03e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fab96640 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c029d60 0x563c7c03e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fab96640 1 -- 172.21.15.26:0/3937734443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7c028a00 con 0x563c7c047600 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21fab96640 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c047600 0x563c7c0552a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f21e4002230 tx=0x7f21e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660b92e5eb85d967 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21d97fa640 1 -- 172.21.15.26:0/3937734443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4003990 con 0x563c7c047600 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21d97fa640 1 -- 172.21.15.26:0/3937734443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f21e4002030 con 0x563c7c047600 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3937734443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ec015ac0 con 0x563c7c047600 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21d97fa640 1 -- 172.21.15.26:0/3937734443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f21e4003b30 con 0x563c7c047600 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21ff3e8740 10 client.9021 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.359+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3937734443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x563c7c0292a0 con 0x563c7c047600 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21d97fa640 1 -- 172.21.15.26:0/3937734443 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec015ac0 con 0x563c7c047600 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21d97fa640 1 -- 172.21.15.26:0/3937734443 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f21e404e070 con 0x563c7c047600 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21ff3e8740 10 client.9021 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21ff3e8740 10 client.9021 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21ff3e8740 20 client.9021 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x563c7c0b4240 0x563c7c0557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21ff3e8740 4 client.9021 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:27:51.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3937734443 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c7bf6af40 con 0x563c7c0b4240 2024-04-24T23:27:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21fa395640 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x563c7c0b4240 0x563c7c0557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21d97fa640 10 client.9021 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.360+0000 7f21fa395640 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x563c7c0b4240 0x563c7c0557e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15db37847ec731df server_cookie=9cf8bcdda88e9ebb in_seq=0 out_seq=0 2024-04-24T23:27:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21d97fa640 1 -- 172.21.15.26:0/3937734443 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c7bf6af40 con 0x563c7c0b4240 2024-04-24T23:27:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21d97fa640 10 client.9021 handle_command_reply: tid=0 2024-04-24T23:27:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 client.9021 shutdown 2024-04-24T23:27:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3937734443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c047600 msgr2=0x563c7c0552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c047600 0x563c7c0552a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f21e4002230 tx=0x7f21e4004a40 comp rx=0 tx=0).stop 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3937734443 shutdown_connections 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x563c7c0b4240 0x563c7c0557e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c7c047600 0x563c7c0552a0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c7c03cd60 0x563c7c03a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 --2- 172.21.15.26:0/3937734443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c7c029d60 0x563c7c03e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21d97fa640 0 client.9021 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3937734443 >> 172.21.15.26:0/3937734443 conn(0x563c7bfffa40 msgr2=0x563c7bfffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3937734443 shutdown_connections 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 1 -- 172.21.15.26:0/3937734443 wait complete. 2024-04-24T23:27:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.367+0000 7f21ff3e8740 20 client.9021 trim_cache size 0 max 16384 2024-04-24T23:27:51.370 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "", 2024-04-24T23:27:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:27:51.370 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 -- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21e006be30 msgr2=0x7f21e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21e006be30 0x7f21e006e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f21ec000f60 tx=0x7f21ec007b50 comp rx=0 tx=0).stop 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 -- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f415e370 msgr2=0x7f21f40b7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f415e370 0x7f21f40b7e40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c7f0 tx=0x7f21e800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 -- 172.21.15.26:0/4236985953 shutdown_connections 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f21e006be30 0x7f21e006e2f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f21f40cd9a0 0x7f21f40c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21f415e370 0x7f21f40b7e40 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 --2- 172.21.15.26:0/4236985953 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21f40d2c30 0x7f21f40cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 -- 172.21.15.26:0/4236985953 >> 172.21.15.26:0/4236985953 conn(0x7f21f40d40f0 msgr2=0x7f21f40b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 -- 172.21.15.26:0/4236985953 shutdown_connections 2024-04-24T23:27:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51.368+0000 7f21fc5f9640 1 -- 172.21.15.26:0/4236985953 wait complete. 2024-04-24T23:27:51.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:51.380 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-04-24T23:27:51.401 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/ 2024-04-24T23:27:51.401 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4 2024-04-24T23:27:51.401 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5 2024-04-24T23:27:51.401 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6 2024-04-24T23:27:51.402 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7 2024-04-24T23:27:51.402 INFO:teuthology.orchestra.run.smithi026.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-04-24T23:27:51.402 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:27:51.402 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:27:51.402 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:27:51.402 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:27:51.402 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-04-24T23:27:51.402 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:27:51.508 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627778 2024-04-24T23:27:51.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:51.517 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:27:51.517 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:27:51.517 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:27:51.517 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:27:51.517 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-04-24T23:27:51.518 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:27:51.624 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627779 2024-04-24T23:27:51.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:51.633 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:27:51.634 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:27:51.634 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:27:51.634 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:27:51.634 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-04-24T23:27:51.634 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:27:51.741 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627780 2024-04-24T23:27:51.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:51.749 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:27:51.749 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:27:51.749 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:27:51.750 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:27:51.750 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-04-24T23:27:51.750 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:27:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627781 2024-04-24T23:27:51.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:51.865 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:27:51.865 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:27:51.865 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:27:51.865 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:27:51.865 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-04-24T23:27:51.865 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:27:51.972 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627782 2024-04-24T23:27:51.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:51.981 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-24T23:27:51.981 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-24T23:27:51.982 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-24T23:27:51.982 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-24T23:27:51.982 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-04-24T23:27:51.982 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-24T23:27:52.088 INFO:teuthology.orchestra.run.smithi026.stdout:1099511627783 2024-04-24T23:27:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:52.097 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-04-24T23:27:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.180+0000 7f382dc70f40 1 Processor -- start 2024-04-24T23:27:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.180+0000 7f382dc70f40 1 -- start start 2024-04-24T23:27:52.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382dc70f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f91a0a0 0x56352f88aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382dc70f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f88b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382dc70f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f873470 0x56352f889b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382dc70f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56352f847990 con 0x56352f91a950 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382dc70f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56352f7ec640 con 0x56352f91a0a0 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382dc70f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56352f91bd00 con 0x56352f873470 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382af70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f88b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382af70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f88b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56268/0 (socket says 172.21.15.26:56268) 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382af70640 1 -- 172.21.15.26:0/2638945599 learned_addr learned my addr 172.21.15.26:0/2638945599 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382bf72640 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f873470 0x56352f889b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382b771640 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f91a0a0 0x56352f88aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382af70640 1 -- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f873470 msgr2=0x56352f889b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382af70640 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f873470 0x56352f889b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382af70640 1 -- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f91a0a0 msgr2=0x56352f88aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382af70640 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f91a0a0 0x56352f88aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382af70640 1 -- 172.21.15.26:0/2638945599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56352f873da0 con 0x56352f91a950 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382bf72640 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f873470 0x56352f889b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382b771640 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f91a0a0 0x56352f88aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f382af70640 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f88b420 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3820009760 tx=0x7f3820005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3168e6a2f905ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f3829f6e640 1 -- 172.21.15.26:0/2638945599 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820008930 con 0x56352f91a950 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f3829f6e640 1 -- 172.21.15.26:0/2638945599 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f382001a070 con 0x56352f91a950 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.181+0000 7f3829f6e640 1 -- 172.21.15.26:0/2638945599 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820008cf0 con 0x56352f91a950 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 -- 172.21.15.26:0/2638945599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 msgr2=0x56352f88b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f88b420 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3820009760 tx=0x7f3820005a00 comp rx=0 tx=0).stop 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 -- 172.21.15.26:0/2638945599 shutdown_connections 2024-04-24T23:27:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f873470 0x56352f889b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f88b420 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 --2- 172.21.15.26:0/2638945599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f91a0a0 0x56352f88aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 -- 172.21.15.26:0/2638945599 >> 172.21.15.26:0/2638945599 conn(0x56352f845fd0 msgr2=0x56352f8439f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 -- 172.21.15.26:0/2638945599 shutdown_connections 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 -- 172.21.15.26:0/2638945599 wait complete. 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.182+0000 7f382dc70f40 1 Processor -- start 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382dc70f40 1 -- start start 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382dc70f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f873470 0x56352f9ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382dc70f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f91a0a0 0x56352f916dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382dc70f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f86f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382dc70f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56352f873b50 con 0x56352f91a950 2024-04-24T23:27:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382dc70f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56352f91a480 con 0x56352f873470 2024-04-24T23:27:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382dc70f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56352f874380 con 0x56352f91a0a0 2024-04-24T23:27:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382bf72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f86f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382bf72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f86f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56272/0 (socket says 172.21.15.26:56272) 2024-04-24T23:27:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382bf72640 1 -- 172.21.15.26:0/956523676 learned_addr learned my addr 172.21.15.26:0/956523676 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382af70640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f91a0a0 0x56352f916dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.183+0000 7f382b771640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f873470 0x56352f9ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f382bf72640 1 -- 172.21.15.26:0/956523676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f91a0a0 msgr2=0x56352f916dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f382bf72640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56352f91a0a0 0x56352f916dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f382bf72640 1 -- 172.21.15.26:0/956523676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f873470 msgr2=0x56352f9ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f382bf72640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56352f873470 0x56352f9ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f382bf72640 1 -- 172.21.15.26:0/956523676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56352f9acb90 con 0x56352f91a950 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f382bf72640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f86f7d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f38240260e0 tx=0x7f382404f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f3828f6c640 1 -- 172.21.15.26:0/956523676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824006980 con 0x56352f91a950 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f3828f6c640 1 -- 172.21.15.26:0/956523676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3824006070 con 0x56352f91a950 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f3828f6c640 1 -- 172.21.15.26:0/956523676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382404d970 con 0x56352f91a950 2024-04-24T23:27:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56352f917300 con 0x56352f91a950 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.184+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56352f9ae3c0 con 0x56352f91a950 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.185+0000 7f3828f6c640 1 -- 172.21.15.26:0/956523676 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3824060070 con 0x56352f91a950 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.186+0000 7f3828f6c640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f381806a810 0x7f381806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.186+0000 7f3828f6c640 1 -- 172.21.15.26:0/956523676 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f38240cbba0 con 0x56352f91a950 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.186+0000 7f382b771640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f381806a810 0x7f381806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.186+0000 7f382dc70f40 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x56352f86fd10 0x56352f9b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.186+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 --> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] -- osd_op(unknown.0.0:1 14.12 14:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56352f9b4150 con 0x56352f86fd10 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.186+0000 7f382af70640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x56352f86fd10 0x56352f9b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.186+0000 7f382b771640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f381806a810 0x7f381806ccd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f381c005400 tx=0x7f381c00c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.187+0000 7f382af70640 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x56352f86fd10 0x56352f9b3c10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382af70640 1 -- 172.21.15.26:0/956523676 <== osd.2 v2:172.21.15.26:6800/3780187135 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v96'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3820008940 con 0x56352f86fd10 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x56352f86fd10 msgr2=0x56352f9b3c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x56352f86fd10 0x56352f9b3c10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f381806a810 msgr2=0x7f381806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f381806a810 0x7f381806ccd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f381c005400 tx=0x7f381c00c040 comp rx=0 tx=0).stop 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 msgr2=0x56352f86f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 --2- 172.21.15.26:0/956523676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56352f91a950 0x56352f86f7d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f38240260e0 tx=0x7f382404f070 comp rx=0 tx=0).stop 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382bf72640 1 -- 172.21.15.26:0/956523676 reap_dead start 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 shutdown_connections 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 >> 172.21.15.26:0/956523676 conn(0x56352f845fd0 msgr2=0x56352f8439f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 shutdown_connections 2024-04-24T23:27:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.188+0000 7f382dc70f40 1 -- 172.21.15.26:0/956523676 wait complete. 2024-04-24T23:27:52.197 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-04-24T23:27:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.279+0000 7f59e4362f40 1 Processor -- start 2024-04-24T23:27:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.279+0000 7f59e4362f40 1 -- start start 2024-04-24T23:27:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e4362f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1192a2a90 0x55d119406390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e4362f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d1192a2e70 0x55d119408fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e4362f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192e7250 0x55d1193657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e4362f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d1192e79b0 con 0x55d1192e7250 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e4362f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d119247640 con 0x55d1192a2a90 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e4362f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d1192e7c90 con 0x55d1192a2e70 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e2664640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192e7250 0x55d1193657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e2664640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192e7250 0x55d1193657b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56290/0 (socket says 172.21.15.26:56290) 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e2664640 1 -- 172.21.15.26:0/3392762298 learned_addr learned my addr 172.21.15.26:0/3392762298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e1e63640 1 --2- 172.21.15.26:0/3392762298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1192a2a90 0x55d119406390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e1662640 1 --2- 172.21.15.26:0/3392762298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d1192a2e70 0x55d119408fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e2664640 1 -- 172.21.15.26:0/3392762298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d1192a2e70 msgr2=0x55d119408fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e2664640 1 --2- 172.21.15.26:0/3392762298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d1192a2e70 0x55d119408fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e2664640 1 -- 172.21.15.26:0/3392762298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1192a2a90 msgr2=0x55d119406390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e2664640 1 --2- 172.21.15.26:0/3392762298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1192a2a90 0x55d119406390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e2664640 1 -- 172.21.15.26:0/3392762298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1192d0240 con 0x55d1192e7250 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59e2664640 1 --2- 172.21.15.26:0/3392762298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192e7250 0x55d1193657b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f59d400c480 tx=0x7f59d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3e97bcf1c43ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59dbfff640 1 -- 172.21.15.26:0/3392762298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d40116b0 con 0x55d1192e7250 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59dbfff640 1 -- 172.21.15.26:0/3392762298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d4017070 con 0x55d1192e7250 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.280+0000 7f59dbfff640 1 -- 172.21.15.26:0/3392762298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4011a90 con 0x55d1192e7250 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.281+0000 7f59e4362f40 1 -- 172.21.15.26:0/3392762298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192e7250 msgr2=0x55d1193657b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.281+0000 7f59e4362f40 1 --2- 172.21.15.26:0/3392762298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192e7250 0x55d1193657b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f59d400c480 tx=0x7f59d400c940 comp rx=0 tx=0).stop 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.281+0000 7f59e4362f40 1 -- 172.21.15.26:0/3392762298 shutdown_connections 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.281+0000 7f59e4362f40 1 --2- 172.21.15.26:0/3392762298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192e7250 0x55d1193657b0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.281+0000 7f59e4362f40 1 --2- 172.21.15.26:0/3392762298 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d1192a2e70 0x55d119408fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.281+0000 7f59e4362f40 1 --2- 172.21.15.26:0/3392762298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1192a2a90 0x55d119406390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.281+0000 7f59e4362f40 1 -- 172.21.15.26:0/3392762298 >> 172.21.15.26:0/3392762298 conn(0x55d1192cab90 msgr2=0x55d1192cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.281+0000 7f59e4362f40 1 -- 172.21.15.26:0/3392762298 shutdown_connections 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.281+0000 7f59e4362f40 1 -- 172.21.15.26:0/3392762298 wait complete. 2024-04-24T23:27:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e4362f40 1 Processor -- start 2024-04-24T23:27:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e4362f40 1 -- start start 2024-04-24T23:27:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e4362f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1192a2a90 0x55d1193617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e4362f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192a2e70 0x55d119365790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e4362f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d1192e7250 0x55d119365cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e4362f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d1192d0520 con 0x55d1192a2e70 2024-04-24T23:27:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e4362f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d119373cb0 con 0x55d1192a2a90 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e4362f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55d119373f90 con 0x55d1192e7250 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e1662640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192a2e70 0x55d119365790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e1662640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192a2e70 0x55d119365790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56302/0 (socket says 172.21.15.26:56302) 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e1662640 1 -- 172.21.15.26:0/2147038677 learned_addr learned my addr 172.21.15.26:0/2147038677 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e1e63640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1192a2a90 0x55d1193617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.282+0000 7f59e2664640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d1192e7250 0x55d119365cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59e1662640 1 -- 172.21.15.26:0/2147038677 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d1192e7250 msgr2=0x55d119365cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59e1662640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55d1192e7250 0x55d119365cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59e1662640 1 -- 172.21.15.26:0/2147038677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1192a2a90 msgr2=0x55d1193617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59e1662640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d1192a2a90 0x55d1193617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59e1662640 1 -- 172.21.15.26:0/2147038677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d119406070 con 0x55d1192a2e70 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59e1662640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192a2e70 0x55d119365790 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f59cc006fd0 tx=0x7f59cc00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59daffd640 1 -- 172.21.15.26:0/2147038677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00f650 con 0x55d1192a2e70 2024-04-24T23:27:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59daffd640 1 -- 172.21.15.26:0/2147038677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59cc013030 con 0x55d1192a2e70 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d119406350 con 0x55d1192a2e70 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59daffd640 1 -- 172.21.15.26:0/2147038677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00f9f0 con 0x55d1192a2e70 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.283+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1193787a0 con 0x55d1192a2e70 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.285+0000 7f59daffd640 1 -- 172.21.15.26:0/2147038677 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59cc00fb90 con 0x55d1192a2e70 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.285+0000 7f59daffd640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59c406a810 0x7f59c406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.285+0000 7f59daffd640 1 -- 172.21.15.26:0/2147038677 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f59cc08a2b0 con 0x55d1192a2e70 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.285+0000 7f59e1e63640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59c406a810 0x7f59c406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.285+0000 7f59e4362f40 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55d1193637a0 0x55d119363bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.285+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 --> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] -- osd_op(unknown.0.0:1 14.12 14:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x55d1193640f0 con 0x55d1193637a0 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.285+0000 7f59e2664640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55d1193637a0 0x55d119363bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.285+0000 7f59e1e63640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59c406a810 0x7f59c406ccd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f59dc056000 tx=0x7f59dc04e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.286+0000 7f59e2664640 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55d1193637a0 0x55d119363bb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e2664640 1 -- 172.21.15.26:0/2147038677 <== osd.2 v2:172.21.15.26:6800/3780187135 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v96'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f59d4011910 con 0x55d1193637a0 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55d1193637a0 msgr2=0x55d119363bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x55d1193637a0 0x55d119363bb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59c406a810 msgr2=0x7f59c406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f59c406a810 0x7f59c406ccd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f59dc056000 tx=0x7f59dc04e000 comp rx=0 tx=0).stop 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192a2e70 msgr2=0x55d119365790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 --2- 172.21.15.26:0/2147038677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1192a2e70 0x55d119365790 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f59cc006fd0 tx=0x7f59cc00c730 comp rx=0 tx=0).stop 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e2664640 1 -- 172.21.15.26:0/2147038677 reap_dead start 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 shutdown_connections 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 >> 172.21.15.26:0/2147038677 conn(0x55d1192cab90 msgr2=0x55d1192cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 shutdown_connections 2024-04-24T23:27:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.287+0000 7f59e4362f40 1 -- 172.21.15.26:0/2147038677 wait complete. 2024-04-24T23:27:52.292 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-04-24T23:27:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.374+0000 7f7fa8f1bf40 1 Processor -- start 2024-04-24T23:27:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.374+0000 7f7fa8f1bf40 1 -- start start 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa8f1bf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645508d2fe0 0x5645508d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa8f1bf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2920 0x56455097c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa8f1bf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2d00 0x56455097edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa8f1bf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645508ac290 con 0x5645508f2d00 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa8f1bf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5645508a81b0 con 0x5645508d2fe0 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa8f1bf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5645508a8b80 con 0x5645508f2920 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa721d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2d00 0x56455097edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa721d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2d00 0x56455097edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56330/0 (socket says 172.21.15.26:56330) 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa721d640 1 -- 172.21.15.26:0/1146766376 learned_addr learned my addr 172.21.15.26:0/1146766376 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa6a1c640 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645508d2fe0 0x5645508d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa621b640 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2920 0x56455097c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa721d640 1 -- 172.21.15.26:0/1146766376 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2920 msgr2=0x56455097c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.375+0000 7f7fa721d640 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2920 0x56455097c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa721d640 1 -- 172.21.15.26:0/1146766376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645508d2fe0 msgr2=0x5645508d48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa721d640 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645508d2fe0 0x5645508d48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa721d640 1 -- 172.21.15.26:0/1146766376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645507d5cb0 con 0x5645508f2d00 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa621b640 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2920 0x56455097c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa721d640 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2d00 0x56455097edd0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000c4b0 tx=0x7f7fa000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a7de50f14a6471 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa5219640 1 -- 172.21.15.26:0/1146766376 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00116b0 con 0x5645508f2d00 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa5219640 1 -- 172.21.15.26:0/1146766376 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa0017070 con 0x5645508f2d00 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa5219640 1 -- 172.21.15.26:0/1146766376 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0011a90 con 0x5645508f2d00 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/1146766376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2d00 msgr2=0x56455097edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa8f1bf40 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2d00 0x56455097edd0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000c4b0 tx=0x7f7fa000c970 comp rx=0 tx=0).stop 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/1146766376 shutdown_connections 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa8f1bf40 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2d00 0x56455097edd0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa8f1bf40 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2920 0x56455097c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa8f1bf40 1 --2- 172.21.15.26:0/1146766376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645508d2fe0 0x5645508d48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/1146766376 >> 172.21.15.26:0/1146766376 conn(0x5645508d5bb0 msgr2=0x5645508d8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.376+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/1146766376 shutdown_connections 2024-04-24T23:27:52.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.377+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/1146766376 wait complete. 2024-04-24T23:27:52.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.377+0000 7f7fa8f1bf40 1 Processor -- start 2024-04-24T23:27:52.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.377+0000 7f7fa8f1bf40 1 -- start start 2024-04-24T23:27:52.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa8f1bf40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645508d2fe0 0x5645509709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa8f1bf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2920 0x56455097edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa8f1bf40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2d00 0x56455097f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa8f1bf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5645508f0190 con 0x5645508f2920 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa8f1bf40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5645508efeb0 con 0x5645508d2fe0 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa8f1bf40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x564550977e50 con 0x5645508f2d00 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa621b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2920 0x56455097edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa621b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2920 0x56455097edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56342/0 (socket says 172.21.15.26:56342) 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa621b640 1 -- 172.21.15.26:0/3191759185 learned_addr learned my addr 172.21.15.26:0/3191759185 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa721d640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2d00 0x56455097f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa6a1c640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645508d2fe0 0x5645509709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa621b640 1 -- 172.21.15.26:0/3191759185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2d00 msgr2=0x56455097f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa621b640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5645508f2d00 0x56455097f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa621b640 1 -- 172.21.15.26:0/3191759185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645508d2fe0 msgr2=0x5645509709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa621b640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5645508d2fe0 0x5645509709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa621b640 1 -- 172.21.15.26:0/3191759185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564550a0bf00 con 0x5645508f2920 2024-04-24T23:27:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.378+0000 7f7fa621b640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2920 0x56455097edd0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009e20 tx=0x7f7f9c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.379+0000 7f7f8ffff640 1 -- 172.21.15.26:0/3191759185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0143f0 con 0x5645508f2920 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.379+0000 7f7f8ffff640 1 -- 172.21.15.26:0/3191759185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9c019070 con 0x5645508f2920 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.379+0000 7f7f8ffff640 1 -- 172.21.15.26:0/3191759185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0147d0 con 0x5645508f2920 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.379+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564550971240 con 0x5645508f2920 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.379+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564550978540 con 0x5645508f2920 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.380+0000 7f7f8ffff640 1 -- 172.21.15.26:0/3191759185 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f9c014970 con 0x5645508f2920 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.380+0000 7f7f8ffff640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9406a840 0x7f7f9406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.380+0000 7f7fa6a1c640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9406a840 0x7f7f9406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.380+0000 7f7f8ffff640 1 -- 172.21.15.26:0/3191759185 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f7f9c089740 con 0x5645508f2920 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.381+0000 7f7fa8f1bf40 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5645508d5fb0 0x564550979420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.381+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- osd_op(unknown.0.0:1 14.5 14:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5645508aabf0 con 0x5645508d5fb0 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.381+0000 7f7fa721d640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5645508d5fb0 0x564550979420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.381+0000 7f7fa6a1c640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9406a840 0x7f7f9406cd00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x5645508f39e0 tx=0x7f7f98050000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.381+0000 7f7fa721d640 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5645508d5fb0 0x564550979420 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa721d640 1 -- 172.21.15.26:0/3191759185 <== osd.0 v2:172.21.15.26:6819/256768257 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v96'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7fa00118b0 con 0x5645508d5fb0 2024-04-24T23:27:52.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5645508d5fb0 msgr2=0x564550979420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x5645508d5fb0 0x564550979420 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9406a840 msgr2=0x7f7f9406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7f9406a840 0x7f7f9406cd00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x5645508f39e0 tx=0x7f7f98050000 comp rx=0 tx=0).stop 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2920 msgr2=0x56455097edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 --2- 172.21.15.26:0/3191759185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5645508f2920 0x56455097edd0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009e20 tx=0x7f7f9c009f10 comp rx=0 tx=0).stop 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa721d640 1 -- 172.21.15.26:0/3191759185 reap_dead start 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 shutdown_connections 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 >> 172.21.15.26:0/3191759185 conn(0x5645508d5bb0 msgr2=0x56455097fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 shutdown_connections 2024-04-24T23:27:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.383+0000 7f7fa8f1bf40 1 -- 172.21.15.26:0/3191759185 wait complete. 2024-04-24T23:27:52.387 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-04-24T23:27:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939d178f40 1 Processor -- start 2024-04-24T23:27:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939d178f40 1 -- start start 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939d178f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f93880030f0 0x556626c3d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939d178f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556626c67430 0x556626cfb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939d178f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626d16800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939d178f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556626be4640 con 0x556626d0c1c0 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939d178f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556626be8680 con 0x556626c67430 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939d178f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556626d14400 con 0x7f93880030f0 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939b47c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626d16800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939b47c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626d16800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56358/0 (socket says 172.21.15.26:56358) 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939b47c640 1 -- 172.21.15.26:0/1564448737 learned_addr learned my addr 172.21.15.26:0/1564448737 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.471+0000 7f939ac7b640 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f93880030f0 0x556626c3d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939a47a640 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556626c67430 0x556626cfb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939b47c640 1 -- 172.21.15.26:0/1564448737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f93880030f0 msgr2=0x556626c3d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939b47c640 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f93880030f0 0x556626c3d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939b47c640 1 -- 172.21.15.26:0/1564448737 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556626c67430 msgr2=0x556626cfb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939b47c640 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556626c67430 0x556626cfb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939b47c640 1 -- 172.21.15.26:0/1564448737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556626c3c0b0 con 0x556626d0c1c0 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939ac7b640 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f93880030f0 0x556626c3d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939b47c640 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626d16800 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f9394001770 tx=0x7f93940017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a647346683d930 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f9399478640 1 -- 172.21.15.26:0/1564448737 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939400f7d0 con 0x556626d0c1c0 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f9399478640 1 -- 172.21.15.26:0/1564448737 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939400f970 con 0x556626d0c1c0 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f9399478640 1 -- 172.21.15.26:0/1564448737 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939400fd60 con 0x556626d0c1c0 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939d178f40 1 -- 172.21.15.26:0/1564448737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 msgr2=0x556626d16800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939d178f40 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626d16800 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f9394001770 tx=0x7f93940017a0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939d178f40 1 -- 172.21.15.26:0/1564448737 shutdown_connections 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939d178f40 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626d16800 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939d178f40 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556626c67430 0x556626cfb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939d178f40 1 --2- 172.21.15.26:0/1564448737 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f93880030f0 0x556626c3d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.472+0000 7f939d178f40 1 -- 172.21.15.26:0/1564448737 >> 172.21.15.26:0/1564448737 conn(0x556626c67b90 msgr2=0x556626c3b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.473+0000 7f939d178f40 1 -- 172.21.15.26:0/1564448737 shutdown_connections 2024-04-24T23:27:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.473+0000 7f939d178f40 1 -- 172.21.15.26:0/1564448737 wait complete. 2024-04-24T23:27:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.473+0000 7f939d178f40 1 Processor -- start 2024-04-24T23:27:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.473+0000 7f939d178f40 1 -- start start 2024-04-24T23:27:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939d178f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556626c67430 0x556626dac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939d178f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626cf92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939d178f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93880030f0 0x556626cfa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939d178f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556626c3cea0 con 0x556626d0c1c0 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939d178f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556626c3c680 con 0x7f93880030f0 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939d178f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x556626c3bd60 con 0x556626c67430 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939a47a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626cf92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939a47a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626cf92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56374/0 (socket says 172.21.15.26:56374) 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939a47a640 1 -- 172.21.15.26:0/1003491539 learned_addr learned my addr 172.21.15.26:0/1003491539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939ac7b640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556626c67430 0x556626dac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939b47c640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93880030f0 0x556626cfa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939a47a640 1 -- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556626c67430 msgr2=0x556626dac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939a47a640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556626c67430 0x556626dac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939a47a640 1 -- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93880030f0 msgr2=0x556626cfa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939a47a640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93880030f0 0x556626cfa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939a47a640 1 -- 172.21.15.26:0/1003491539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556626dab000 con 0x556626d0c1c0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939ac7b640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556626c67430 0x556626dac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939b47c640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93880030f0 0x556626cfa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.474+0000 7f939a47a640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626cf92d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f9384025d70 tx=0x7f9384003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.475+0000 7f9393fff640 1 -- 172.21.15.26:0/1003491539 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384055040 con 0x556626d0c1c0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.475+0000 7f9393fff640 1 -- 172.21.15.26:0/1003491539 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93840518c0 con 0x556626d0c1c0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.475+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556626dab2e0 con 0x556626d0c1c0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.475+0000 7f9393fff640 1 -- 172.21.15.26:0/1003491539 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384051cb0 con 0x556626d0c1c0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.475+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556626dab8a0 con 0x556626d0c1c0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.476+0000 7f9393fff640 1 -- 172.21.15.26:0/1003491539 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9384058070 con 0x556626d0c1c0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.476+0000 7f9393fff640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f937c06a840 0x7f937c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.477+0000 7f939ac7b640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f937c06a840 0x7f937c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.477+0000 7f9393fff640 1 -- 172.21.15.26:0/1003491539 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f93840cd520 con 0x556626d0c1c0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.477+0000 7f939d178f40 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x556626d0c6d0 0x556626d0cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.477+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 --> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] -- osd_op(unknown.0.0:1 14.9 14:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x556626d0d020 con 0x556626d0c6d0 2024-04-24T23:27:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.477+0000 7f939ac7b640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f937c06a840 0x7f937c06cd00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f938c000f80 tx=0x7f938c0068f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.477+0000 7f939b47c640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x556626d0c6d0 0x556626d0cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.478+0000 7f939b47c640 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x556626d0c6d0 0x556626d0cae0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939b47c640 1 -- 172.21.15.26:0/1003491539 <== osd.5 v2:172.21.15.29:6800/1108214380 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v96'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f939400e470 con 0x556626d0c6d0 2024-04-24T23:27:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x556626d0c6d0 msgr2=0x556626d0cae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x556626d0c6d0 0x556626d0cae0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f937c06a840 msgr2=0x7f937c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f937c06a840 0x7f937c06cd00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f938c000f80 tx=0x7f938c0068f0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 msgr2=0x556626cf92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 --2- 172.21.15.26:0/1003491539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556626d0c1c0 0x556626cf92d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f9384025d70 tx=0x7f9384003020 comp rx=0 tx=0).stop 2024-04-24T23:27:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939b47c640 1 -- 172.21.15.26:0/1003491539 reap_dead start 2024-04-24T23:27:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 shutdown_connections 2024-04-24T23:27:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 >> 172.21.15.26:0/1003491539 conn(0x556626c67b90 msgr2=0x556626d112d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 shutdown_connections 2024-04-24T23:27:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.479+0000 7f939d178f40 1 -- 172.21.15.26:0/1003491539 wait complete. 2024-04-24T23:27:52.483 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-04-24T23:27:52.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.526+0000 7f4d9f030f40 1 Processor -- start 2024-04-24T23:27:52.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.526+0000 7f4d9f030f40 1 -- start start 2024-04-24T23:27:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.526+0000 7f4d9f030f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 0x56435cc4e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.526+0000 7f4d9f030f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5cbc0 0x56435cc60c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.526+0000 7f4d9f030f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56435cc5f1b0 0x56435cc5f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.526+0000 7f4d9f030f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56435cb30640 con 0x56435cc5cbc0 2024-04-24T23:27:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.526+0000 7f4d9f030f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56435cb34680 con 0x7f4d88002850 2024-04-24T23:27:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.526+0000 7f4d9f030f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56435cbd09b0 con 0x56435cc5f1b0 2024-04-24T23:27:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9c330640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5cbc0 0x56435cc60c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9c330640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5cbc0 0x56435cc60c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56396/0 (socket says 172.21.15.26:56396) 2024-04-24T23:27:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9c330640 1 -- 172.21.15.26:0/3027447460 learned_addr learned my addr 172.21.15.26:0/3027447460 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9cb31640 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 0x56435cc4e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9d332640 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56435cc5f1b0 0x56435cc5f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9c330640 1 -- 172.21.15.26:0/3027447460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56435cc5f1b0 msgr2=0x56435cc5f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9c330640 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56435cc5f1b0 0x56435cc5f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9c330640 1 -- 172.21.15.26:0/3027447460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 msgr2=0x56435cc4e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9c330640 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 0x56435cc4e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9c330640 1 -- 172.21.15.26:0/3027447460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435cc56b70 con 0x56435cc5cbc0 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9cb31640 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 0x56435cc4e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9c330640 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5cbc0 0x56435cc60c00 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f4d84046b80 tx=0x7f4d84050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897a65d1df7e3d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9b32e640 1 -- 172.21.15.26:0/3027447460 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84001630 con 0x56435cc5cbc0 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9b32e640 1 -- 172.21.15.26:0/3027447460 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d84066070 con 0x56435cc5cbc0 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.527+0000 7f4d9b32e640 1 -- 172.21.15.26:0/3027447460 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84061690 con 0x56435cc5cbc0 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 -- 172.21.15.26:0/3027447460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5cbc0 msgr2=0x56435cc60c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5cbc0 0x56435cc60c00 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f4d84046b80 tx=0x7f4d84050000 comp rx=0 tx=0).stop 2024-04-24T23:27:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 -- 172.21.15.26:0/3027447460 shutdown_connections 2024-04-24T23:27:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56435cc5f1b0 0x56435cc5f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5cbc0 0x56435cc60c00 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 --2- 172.21.15.26:0/3027447460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 0x56435cc4e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 -- 172.21.15.26:0/3027447460 >> 172.21.15.26:0/3027447460 conn(0x56435cbb3b90 msgr2=0x56435cbd2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 -- 172.21.15.26:0/3027447460 shutdown_connections 2024-04-24T23:27:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 -- 172.21.15.26:0/3027447460 wait complete. 2024-04-24T23:27:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 Processor -- start 2024-04-24T23:27:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.528+0000 7f4d9f030f40 1 -- start start 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9f030f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56435cc5cbc0 0x56435cc55960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9f030f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5f1b0 0x56435cbd3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9f030f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 0x56435cc442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9f030f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56435cbcfb70 con 0x56435cc5f1b0 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9f030f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56435cc56e50 con 0x7f4d88002850 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9f030f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x56435cbcfe50 con 0x56435cc5cbc0 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9c330640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5f1b0 0x56435cbd3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9c330640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5f1b0 0x56435cbd3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56398/0 (socket says 172.21.15.26:56398) 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9c330640 1 -- 172.21.15.26:0/2475184751 learned_addr learned my addr 172.21.15.26:0/2475184751 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9cb31640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56435cc5cbc0 0x56435cc55960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9d332640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 0x56435cc442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9c330640 1 -- 172.21.15.26:0/2475184751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56435cc5cbc0 msgr2=0x56435cc55960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9c330640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x56435cc5cbc0 0x56435cc55960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9c330640 1 -- 172.21.15.26:0/2475184751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 msgr2=0x56435cc442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9c330640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 0x56435cc442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9c330640 1 -- 172.21.15.26:0/2475184751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435cc51b20 con 0x56435cc5f1b0 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.529+0000 7f4d9d332640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d88002850 0x56435cc442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.530+0000 7f4d9c330640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5f1b0 0x56435cbd3130 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f4d84063040 tx=0x7f4d84001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.530+0000 7f4d9a32c640 1 -- 172.21.15.26:0/2475184751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84075070 con 0x56435cc5f1b0 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.530+0000 7f4d9a32c640 1 -- 172.21.15.26:0/2475184751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d84061810 con 0x56435cc5f1b0 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.530+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56435cc51d90 con 0x56435cc5f1b0 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.530+0000 7f4d9a32c640 1 -- 172.21.15.26:0/2475184751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84061c00 con 0x56435cc5f1b0 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.530+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435cc52350 con 0x56435cc5f1b0 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.531+0000 7f4d9a32c640 1 -- 172.21.15.26:0/2475184751 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d84078070 con 0x56435cc5f1b0 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.531+0000 7f4d9a32c640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7c06a840 0x7f4d7c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.532+0000 7f4d9a32c640 1 -- 172.21.15.26:0/2475184751 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f4d840cf4d0 con 0x56435cc5f1b0 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.532+0000 7f4d9cb31640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7c06a840 0x7f4d7c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.532+0000 7f4d9f030f40 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x56435cbd3670 0x56435cc52870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.532+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- osd_op(unknown.0.0:1 14.5 14:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56435cc52db0 con 0x56435cbd3670 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.532+0000 7f4d9d332640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x56435cbd3670 0x56435cc52870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.532+0000 7f4d9cb31640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7c06a840 0x7f4d7c06cd00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4d94013660 tx=0x7f4d94001730 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.532+0000 7f4d9d332640 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x56435cbd3670 0x56435cc52870 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9d332640 1 -- 172.21.15.26:0/2475184751 <== osd.0 v2:172.21.15.26:6819/256768257 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v96'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4d8c0028a0 con 0x56435cbd3670 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x56435cbd3670 msgr2=0x56435cc52870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x56435cbd3670 0x56435cc52870 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7c06a840 msgr2=0x7f4d7c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4d7c06a840 0x7f4d7c06cd00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4d94013660 tx=0x7f4d94001730 comp rx=0 tx=0).stop 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5f1b0 msgr2=0x56435cbd3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 --2- 172.21.15.26:0/2475184751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56435cc5f1b0 0x56435cbd3130 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f4d84063040 tx=0x7f4d84001a40 comp rx=0 tx=0).stop 2024-04-24T23:27:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9d332640 1 -- 172.21.15.26:0/2475184751 reap_dead start 2024-04-24T23:27:52.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 shutdown_connections 2024-04-24T23:27:52.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 >> 172.21.15.26:0/2475184751 conn(0x56435cbb3b90 msgr2=0x56435cc5f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 shutdown_connections 2024-04-24T23:27:52.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.534+0000 7f4d9f030f40 1 -- 172.21.15.26:0/2475184751 wait complete. 2024-04-24T23:27:52.539 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-04-24T23:27:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05bd017f40 1 Processor -- start 2024-04-24T23:27:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05bd017f40 1 -- start start 2024-04-24T23:27:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05bd017f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05a8002810 0x563c3cc39500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05bd017f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3cc4e210 0x563c3cc4ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05bd017f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c3cc50de0 0x563c3cc4b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05bd017f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563c3cb23640 con 0x563c3cc4e210 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05bd017f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563c3cb27680 con 0x563c3cc50de0 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05bd017f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563c3cb7b1b0 con 0x7f05a8002810 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05ba317640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3cc4e210 0x563c3cc4ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05ba317640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3cc4e210 0x563c3cc4ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56420/0 (socket says 172.21.15.26:56420) 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.623+0000 7f05ba317640 1 -- 172.21.15.26:0/3322625016 learned_addr learned my addr 172.21.15.26:0/3322625016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05bab18640 1 --2- 172.21.15.26:0/3322625016 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05a8002810 0x563c3cc39500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05ba317640 1 -- 172.21.15.26:0/3322625016 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05a8002810 msgr2=0x563c3cc39500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05ba317640 1 --2- 172.21.15.26:0/3322625016 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05a8002810 0x563c3cc39500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05ba317640 1 -- 172.21.15.26:0/3322625016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c3cc50de0 msgr2=0x563c3cc4b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05ba317640 1 --2- 172.21.15.26:0/3322625016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c3cc50de0 0x563c3cc4b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05ba317640 1 -- 172.21.15.26:0/3322625016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3cc51a70 con 0x563c3cc4e210 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05ba317640 1 --2- 172.21.15.26:0/3322625016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3cc4e210 0x563c3cc4ad20 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009760 tx=0x7f05a4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44025e4cdf29dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05b9315640 1 -- 172.21.15.26:0/3322625016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a40518c0 con 0x563c3cc4e210 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05b9315640 1 -- 172.21.15.26:0/3322625016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a4051a80 con 0x563c3cc4e210 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05b9315640 1 -- 172.21.15.26:0/3322625016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a405a450 con 0x563c3cc4e210 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05bd017f40 1 -- 172.21.15.26:0/3322625016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3cc4e210 msgr2=0x563c3cc4ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05bd017f40 1 --2- 172.21.15.26:0/3322625016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3cc4e210 0x563c3cc4ad20 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009760 tx=0x7f05a4051000 comp rx=0 tx=0).stop 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05bd017f40 1 -- 172.21.15.26:0/3322625016 shutdown_connections 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05bd017f40 1 --2- 172.21.15.26:0/3322625016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c3cc50de0 0x563c3cc4b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05bd017f40 1 --2- 172.21.15.26:0/3322625016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3cc4e210 0x563c3cc4ad20 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05bd017f40 1 --2- 172.21.15.26:0/3322625016 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f05a8002810 0x563c3cc39500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.624+0000 7f05bd017f40 1 -- 172.21.15.26:0/3322625016 >> 172.21.15.26:0/3322625016 conn(0x563c3cba8bb0 msgr2=0x563c3cba7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.625+0000 7f05bd017f40 1 -- 172.21.15.26:0/3322625016 shutdown_connections 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.625+0000 7f05bd017f40 1 -- 172.21.15.26:0/3322625016 wait complete. 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.625+0000 7f05bd017f40 1 Processor -- start 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.625+0000 7f05bd017f40 1 -- start start 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bd017f40 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c3cc4e210 0x563c3cc4adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bd017f40 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c3cc50de0 0x563c3cba9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bd017f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a8002810 0x563c3cba9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bd017f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563c3cc51d50 con 0x7f05a8002810 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bd017f40 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563c3cbaae10 con 0x563c3cc4e210 2024-04-24T23:27:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bd017f40 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563c3cc52030 con 0x563c3cc50de0 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bb319640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a8002810 0x563c3cba9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bb319640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a8002810 0x563c3cba9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56428/0 (socket says 172.21.15.26:56428) 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bb319640 1 -- 172.21.15.26:0/985094087 learned_addr learned my addr 172.21.15.26:0/985094087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05ba317640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c3cc50de0 0x563c3cba9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bab18640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c3cc4e210 0x563c3cc4adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bb319640 1 -- 172.21.15.26:0/985094087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c3cc50de0 msgr2=0x563c3cba9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bb319640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563c3cc50de0 0x563c3cba9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bb319640 1 -- 172.21.15.26:0/985094087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c3cc4e210 msgr2=0x563c3cc4adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bb319640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c3cc4e210 0x563c3cc4adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bb319640 1 -- 172.21.15.26:0/985094087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3cc534b0 con 0x7f05a8002810 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bab18640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563c3cc4e210 0x563c3cc4adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.626+0000 7f05bb319640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a8002810 0x563c3cba9ff0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x563c3cbc69e0 tx=0x7f05ac003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.627+0000 7f05b3fff640 1 -- 172.21.15.26:0/985094087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac011560 con 0x7f05a8002810 2024-04-24T23:27:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.627+0000 7f05b3fff640 1 -- 172.21.15.26:0/985094087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05ac017070 con 0x7f05a8002810 2024-04-24T23:27:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.627+0000 7f05b3fff640 1 -- 172.21.15.26:0/985094087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac011960 con 0x7f05a8002810 2024-04-24T23:27:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.627+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563c3cc53750 con 0x7f05a8002810 2024-04-24T23:27:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.627+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c3cc53d10 con 0x7f05a8002810 2024-04-24T23:27:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.628+0000 7f05b3fff640 1 -- 172.21.15.26:0/985094087 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05ac011b00 con 0x7f05a8002810 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.629+0000 7f05b3fff640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f059c06a810 0x7f059c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.629+0000 7f05b3fff640 1 -- 172.21.15.26:0/985094087 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f05ac089960 con 0x7f05a8002810 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.629+0000 7f05bab18640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f059c06a810 0x7f059c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.629+0000 7f05bd017f40 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x563c3ccde2d0 0x563c3ccde710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.629+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- osd_op(unknown.0.0:1 14.1d 14:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x563c3cbc4140 con 0x563c3ccde2d0 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.629+0000 7f05ba317640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x563c3ccde2d0 0x563c3ccde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.629+0000 7f05bab18640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f059c06a810 0x7f059c06ccd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f05b40014d0 tx=0x7f05b4001440 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.629+0000 7f05ba317640 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x563c3ccde2d0 0x563c3ccde710 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.630+0000 7f05ba317640 1 -- 172.21.15.26:0/985094087 <== osd.0 v2:172.21.15.26:6819/256768257 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v96'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f05a4057030 con 0x563c3ccde2d0 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x563c3ccde2d0 msgr2=0x563c3ccde710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x563c3ccde2d0 0x563c3ccde710 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f059c06a810 msgr2=0x7f059c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f059c06a810 0x7f059c06ccd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f05b40014d0 tx=0x7f05b4001440 comp rx=0 tx=0).stop 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a8002810 msgr2=0x563c3cba9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 --2- 172.21.15.26:0/985094087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a8002810 0x563c3cba9ff0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x563c3cbc69e0 tx=0x7f05ac003820 comp rx=0 tx=0).stop 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bb319640 1 -- 172.21.15.26:0/985094087 reap_dead start 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 shutdown_connections 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 >> 172.21.15.26:0/985094087 conn(0x563c3cba8bb0 msgr2=0x563c3cc38730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 shutdown_connections 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.631+0000 7f05bd017f40 1 -- 172.21.15.26:0/985094087 wait complete. 2024-04-24T23:27:52.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:52.638 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /d1/d2/d3 recursive,force 2024-04-24T23:27:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.741+0000 7f83538a5640 1 Processor -- start 2024-04-24T23:27:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.742+0000 7f83538a5640 1 -- start start 2024-04-24T23:27:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.742+0000 7f83538a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c15e710 0x7f834c0d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.742+0000 7f83538a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f834c0d4630 0x7f834c0d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.742+0000 7f83538a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834c0d27b0 con 0x7f834c15e330 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.742+0000 7f83538a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f834c0cef00 con 0x7f834c0d4630 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.742+0000 7f83538a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f834c0cf1e0 con 0x7f834c15e710 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.742+0000 7f8351e42640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f834c0d4630 0x7f834c0d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.742+0000 7f8351641640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f834c15e330 0x7f834c0d3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57964/0 (socket says 172.21.15.26:57964) 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.742+0000 7f8351641640 1 -- 172.21.15.26:0/570282910 learned_addr learned my addr 172.21.15.26:0/570282910 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.743+0000 7f8350e40640 1 --2- 172.21.15.26:0/570282910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c15e710 0x7f834c0d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.743+0000 7f8351e42640 1 -- 172.21.15.26:0/570282910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c15e710 msgr2=0x7f834c0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.743+0000 7f8351e42640 1 --2- 172.21.15.26:0/570282910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c15e710 0x7f834c0d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.743+0000 7f8351e42640 1 -- 172.21.15.26:0/570282910 >> v1:172.21.15.26:6789/0 conn(0x7f834c15e330 legacy=0x7f834c0d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.743+0000 7f8351e42640 1 -- 172.21.15.26:0/570282910 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834c0cf4c0 con 0x7f834c0d4630 2024-04-24T23:27:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.743+0000 7f8351e42640 1 --2- 172.21.15.26:0/570282910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c0d4630 0x7f834c0d15e0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f8344009dc0 tx=0x7f834402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc359692d38d8dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f8343fff640 1 -- 172.21.15.26:0/570282910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834402ae60 con 0x7f834c0d4630 2024-04-24T23:27:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f8343fff640 1 -- 172.21.15.26:0/570282910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8344007740 con 0x7f834c0d4630 2024-04-24T23:27:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f8343fff640 1 -- 172.21.15.26:0/570282910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344007b10 con 0x7f834c0d4630 2024-04-24T23:27:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f83538a5640 1 -- 172.21.15.26:0/570282910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c0d4630 msgr2=0x7f834c0d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f83538a5640 1 --2- 172.21.15.26:0/570282910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c0d4630 0x7f834c0d15e0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f8344009dc0 tx=0x7f834402a680 comp rx=0 tx=0).stop 2024-04-24T23:27:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f83538a5640 1 -- 172.21.15.26:0/570282910 shutdown_connections 2024-04-24T23:27:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f83538a5640 1 --2- 172.21.15.26:0/570282910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c0d4630 0x7f834c0d15e0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f83538a5640 1 --2- 172.21.15.26:0/570282910 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c15e710 0x7f834c0d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f83538a5640 1 -- 172.21.15.26:0/570282910 >> 172.21.15.26:0/570282910 conn(0x7f834c08dbc0 msgr2=0x7f834c09d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f83538a5640 1 -- 172.21.15.26:0/570282910 shutdown_connections 2024-04-24T23:27:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.744+0000 7f83538a5640 1 -- 172.21.15.26:0/570282910 wait complete. 2024-04-24T23:27:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.745+0000 7f83538a5640 1 Processor -- start 2024-04-24T23:27:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f83538a5640 1 -- start start 2024-04-24T23:27:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f83538a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c0d4630 0x7f834c0b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f83538a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f834c15e330 0x7f834c15f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f83538a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c15e710 0x7f834c152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f83538a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f834c0cfe40 con 0x7f834c15e710 2024-04-24T23:27:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f83538a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f834c0cfcc0 con 0x7f834c15e330 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f83538a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f834c0cffc0 con 0x7f834c0d4630 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f8351e42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c15e710 0x7f834c152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f8351e42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c15e710 0x7f834c152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56446/0 (socket says 172.21.15.26:56446) 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f8351e42640 1 -- 172.21.15.26:0/3089237398 learned_addr learned my addr 172.21.15.26:0/3089237398 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f8351641640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c0d4630 0x7f834c0b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.746+0000 7f8350e40640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f834c15e330 0x7f834c15f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f8351e42640 1 -- 172.21.15.26:0/3089237398 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c0d4630 msgr2=0x7f834c0b96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f8351e42640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c0d4630 0x7f834c0b96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f8351e42640 1 -- 172.21.15.26:0/3089237398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f834c15e330 msgr2=0x7f834c15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f8351e42640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f834c15e330 0x7f834c15f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f8351e42640 1 -- 172.21.15.26:0/3089237398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834c15fd50 con 0x7f834c15e710 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f8351e42640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c15e710 0x7f834c152a90 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8344009ee0 tx=0x7f8344004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f8341ffb640 1 -- 172.21.15.26:0/3089237398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834403f070 con 0x7f834c15e710 2024-04-24T23:27:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f8341ffb640 1 -- 172.21.15.26:0/3089237398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834402ae60 con 0x7f834c15e710 2024-04-24T23:27:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f83538a5640 1 -- 172.21.15.26:0/3089237398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834c160030 con 0x7f834c15e710 2024-04-24T23:27:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f8341ffb640 1 -- 172.21.15.26:0/3089237398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344009450 con 0x7f834c15e710 2024-04-24T23:27:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.747+0000 7f83538a5640 1 -- 172.21.15.26:0/3089237398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f834c1608d0 con 0x7f834c15e710 2024-04-24T23:27:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.749+0000 7f8341ffb640 1 -- 172.21.15.26:0/3089237398 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f834403b410 con 0x7f834c15e710 2024-04-24T23:27:52.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.749+0000 7f8341ffb640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f833006be30 0x7f833006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.749+0000 7f8341ffb640 1 -- 172.21.15.26:0/3089237398 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f83440ac310 con 0x7f834c15e710 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.749+0000 7f8351641640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f833006be30 0x7f833006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.750+0000 7f8351641640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f833006be30 0x7f833006e2f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f834c15d7b0 tx=0x7f8348007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.750+0000 7f83565ba740 1 Processor -- start 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.750+0000 7f83565ba740 1 -- start start 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f83565ba740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4ac71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f83565ba740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 0x557fc4aac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f83565ba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4aa8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f83565ba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557fc49e4490 con 0x557fc4aac580 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f83565ba740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557fc498cae0 con 0x557fc4aabc60 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f83565ba740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x557fc49e0450 con 0x557fc4abd680 2024-04-24T23:27:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351e42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4aa8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351e42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4aa8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56450/0 (socket says 172.21.15.26:56450) 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351e42640 1 -- 172.21.15.26:0/814918311 learned_addr learned my addr 172.21.15.26:0/814918311 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8350e40640 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 0x557fc4aac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351641640 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4ac71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351e42640 1 -- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 msgr2=0x557fc4aac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351e42640 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 0x557fc4aac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351e42640 1 -- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 msgr2=0x557fc4ac71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351e42640 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4ac71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351e42640 1 -- 172.21.15.26:0/814918311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc47c0a50 con 0x557fc4aac580 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8350e40640 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 0x557fc4aac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.751+0000 7f8351641640 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4ac71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f8351e42640 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4aa8840 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f83440aa8f0 tx=0x7f834407a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3162cee1d66e2949 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f8336ffd640 1 -- 172.21.15.26:0/814918311 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344083230 con 0x557fc4aac580 2024-04-24T23:27:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f8336ffd640 1 -- 172.21.15.26:0/814918311 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83440ac0c0 con 0x557fc4aac580 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f8336ffd640 1 -- 172.21.15.26:0/814918311 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344076d70 con 0x557fc4aac580 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f83565ba740 1 -- 172.21.15.26:0/814918311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 msgr2=0x557fc4aa8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f83565ba740 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4aa8840 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f83440aa8f0 tx=0x7f834407a060 comp rx=0 tx=0).stop 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f83565ba740 1 -- 172.21.15.26:0/814918311 shutdown_connections 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f83565ba740 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4aa8840 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f83565ba740 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 0x557fc4aac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f83565ba740 1 --2- 172.21.15.26:0/814918311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4ac71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f83565ba740 1 -- 172.21.15.26:0/814918311 >> 172.21.15.26:0/814918311 conn(0x557fc4a485e0 msgr2=0x557fc4ac6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f83565ba740 1 -- 172.21.15.26:0/814918311 shutdown_connections 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.752+0000 7f83565ba740 1 -- 172.21.15.26:0/814918311 wait complete. 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.753+0000 7f83565ba740 1 -- start start 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.753+0000 7f83565ba740 1 Processor -- start 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f83565ba740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aabc60 0x557fc4ac74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f83565ba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f83565ba740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4abd680 0x557fc4ac8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f83565ba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557fc49e4490 con 0x557fc4aac580 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f83565ba740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557fc49a0060 con 0x557fc4abd680 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f83565ba740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x557fc4a82fa0 con 0x557fc4aabc60 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f83565ba740 10 client.0 _ll_register_callbacks cb 0x557fc49db970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8350e40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8350e40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56466/0 (socket says 172.21.15.26:56466) 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8350e40640 1 -- 172.21.15.26:0/263964619 learned_addr learned my addr 172.21.15.26:0/263964619 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8351641640 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aabc60 0x557fc4ac74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8350e40640 1 -- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aabc60 msgr2=0x557fc4ac74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8350e40640 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aabc60 0x557fc4ac74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8350e40640 1 -- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4abd680 msgr2=0x557fc4ac8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8351e42640 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4abd680 0x557fc4ac8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8350e40640 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4abd680 0x557fc4ac8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8350e40640 1 -- 172.21.15.26:0/263964619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc46b83e0 con 0x557fc4aac580 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.754+0000 7f8350e40640 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac87f0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f833c012630 tx=0x7f833c012af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf2882a39c9dd9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f832bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f832bfff640 1 -- 172.21.15.26:0/263964619 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c01b020 con 0x557fc4aac580 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f832bfff640 1 -- 172.21.15.26:0/263964619 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f833c004240 con 0x557fc4aac580 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f83565ba740 1 -- 172.21.15.26:0/263964619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83440ac0c0 con 0x557fc4aac580 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f832bfff640 1 -- 172.21.15.26:0/263964619 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f833c025050 con 0x557fc4aac580 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f83565ba740 10 client.9093 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f83565ba740 1 -- 172.21.15.26:0/263964619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x557fc48211e0 con 0x557fc4aac580 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f832bfff640 1 -- 172.21.15.26:0/263964619 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83440ac0c0 con 0x557fc4aac580 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f832bfff640 1 -- 172.21.15.26:0/263964619 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f833c028020 con 0x557fc4aac580 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f83565ba740 10 client.9093 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f83565ba740 10 client.9093 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f83565ba740 20 client.9093 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:52.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f83565ba740 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x557fc4aa2820 0x557fc4ac4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f83565ba740 4 client.9093 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.755+0000 7f83565ba740 1 -- 172.21.15.26:0/263964619 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fc49f19d0 con 0x557fc4aa2820 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.756+0000 7f8351641640 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x557fc4aa2820 0x557fc4ac4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.756+0000 7f8351641640 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x557fc4aa2820 0x557fc4ac4620 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cdcf9e5ebc35aee server_cookie=19ae28560afc06da in_seq=0 out_seq=0 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.756+0000 7f832bfff640 10 client.9093 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f832bfff640 1 -- 172.21.15.26:0/263964619 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557fc49f19d0 con 0x557fc4aa2820 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f832bfff640 10 client.9093 handle_command_reply: tid=0 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 client.9093 shutdown 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 -- 172.21.15.26:0/263964619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 msgr2=0x557fc4ac87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac87f0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f833c012630 tx=0x7f833c012af0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 -- 172.21.15.26:0/263964619 shutdown_connections 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x557fc4aa2820 0x557fc4ac4620 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4abd680 0x557fc4ac8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac87f0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f832bfff640 0 client.9093 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 --2- 172.21.15.26:0/263964619 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aabc60 0x557fc4ac74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 -- 172.21.15.26:0/263964619 >> 172.21.15.26:0/263964619 conn(0x557fc4a485e0 msgr2=0x557fc4ac9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 -- 172.21.15.26:0/263964619 shutdown_connections 2024-04-24T23:27:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 1 -- 172.21.15.26:0/263964619 wait complete. 2024-04-24T23:27:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.757+0000 7f83565ba740 20 client.9093 trim_cache size 0 max 16384 2024-04-24T23:27:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.775+0000 7f83565ba740 1 Processor -- start 2024-04-24T23:27:52.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.775+0000 7f83565ba740 1 -- start start 2024-04-24T23:27:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.775+0000 7f83565ba740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4aaaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.775+0000 7f83565ba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.775+0000 7f83565ba740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 0x557fc4ac4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.775+0000 7f83565ba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f834c200980 con 0x557fc4aac580 2024-04-24T23:27:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.775+0000 7f83565ba740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f834c1ffb50 con 0x557fc4aabc60 2024-04-24T23:27:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.775+0000 7f83565ba740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f834c200250 con 0x557fc4abd680 2024-04-24T23:27:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8350e40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8350e40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56474/0 (socket says 172.21.15.26:56474) 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8350e40640 1 -- 172.21.15.26:0/3122338705 learned_addr learned my addr 172.21.15.26:0/3122338705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8351e42640 1 --2- 172.21.15.26:0/3122338705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 0x557fc4ac4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8351641640 1 --2- 172.21.15.26:0/3122338705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4aaaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8350e40640 1 -- 172.21.15.26:0/3122338705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 msgr2=0x557fc4ac4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8350e40640 1 --2- 172.21.15.26:0/3122338705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 0x557fc4ac4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8350e40640 1 -- 172.21.15.26:0/3122338705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 msgr2=0x557fc4aaaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8350e40640 1 --2- 172.21.15.26:0/3122338705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4aaaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8350e40640 1 -- 172.21.15.26:0/3122338705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc46914a0 con 0x557fc4aac580 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f8350e40640 1 --2- 172.21.15.26:0/3122338705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac9ff0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f833c007010 tx=0x7f833c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ccbe4d329219ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f83367fc640 1 -- 172.21.15.26:0/3122338705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c01b070 con 0x557fc4aac580 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f83367fc640 1 -- 172.21.15.26:0/3122338705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c0100c0 con 0x557fc4aac580 2024-04-24T23:27:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.776+0000 7f83367fc640 1 -- 172.21.15.26:0/3122338705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c003830 con 0x557fc4aac580 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 -- 172.21.15.26:0/3122338705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 msgr2=0x557fc4ac9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 --2- 172.21.15.26:0/3122338705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac9ff0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f833c007010 tx=0x7f833c018000 comp rx=0 tx=0).stop 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 -- 172.21.15.26:0/3122338705 shutdown_connections 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 --2- 172.21.15.26:0/3122338705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4abd680 0x557fc4ac4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 --2- 172.21.15.26:0/3122338705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4aac580 0x557fc4ac9ff0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 --2- 172.21.15.26:0/3122338705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4aaaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 -- 172.21.15.26:0/3122338705 >> 172.21.15.26:0/3122338705 conn(0x557fc4a485e0 msgr2=0x557fc4ad9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 -- 172.21.15.26:0/3122338705 shutdown_connections 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 -- 172.21.15.26:0/3122338705 wait complete. 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 -- start start 2024-04-24T23:27:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.777+0000 7f83565ba740 1 Processor -- start 2024-04-24T23:27:52.782 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:27:52.782+0000 7f42b0d69640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-04-24T23:27:35.347023+0000 1=0+1) n(v3 rc2024-04-24T23:27:35.581019+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7552=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7552 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x558526e71b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.347023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.581019+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.347023+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-04-24T23:27:35.581019+0000 6=0+6)","error_str":""},"return_code":-61} 2024-04-24T23:27:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.778+0000 7f83565ba740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4aca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.778+0000 7f83565ba740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aac580 0x557fc4ac68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.778+0000 7f83565ba740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4abd680 0x557fc4aacf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.778+0000 7f83565ba740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f834c1febb0 con 0x557fc4abd680 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.778+0000 7f83565ba740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557fc4a81c60 con 0x557fc4aabc60 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.778+0000 7f83565ba740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x557fc4b41e10 con 0x557fc4aac580 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.778+0000 7f83565ba740 10 client.0 _ll_register_callbacks cb 0x7f833c0075f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.778+0000 7f8351e42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4abd680 0x557fc4aacf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8351e42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4abd680 0x557fc4aacf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56478/0 (socket says 172.21.15.26:56478) 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8351e42640 1 -- 172.21.15.26:0/4041326413 learned_addr learned my addr 172.21.15.26:0/4041326413 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8350e40640 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aac580 0x557fc4ac68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8351641640 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4aca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8351e42640 1 -- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aac580 msgr2=0x557fc4ac68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8351e42640 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aac580 0x557fc4ac68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8351e42640 1 -- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 msgr2=0x557fc4aca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8351e42640 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4aca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8351e42640 1 -- 172.21.15.26:0/4041326413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc4abcd40 con 0x557fc4abd680 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f8351e42640 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4abd680 0x557fc4aacf20 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f834407d040 tx=0x7f8344009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de501d874260b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f832bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f832bfff640 1 -- 172.21.15.26:0/4041326413 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344083230 con 0x557fc4abd680 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f83565ba740 1 -- 172.21.15.26:0/4041326413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f833c0100c0 con 0x557fc4abd680 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f832bfff640 1 -- 172.21.15.26:0/4041326413 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f834407a960 con 0x557fc4abd680 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f832bfff640 1 -- 172.21.15.26:0/4041326413 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f834407ab60 con 0x557fc4abd680 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f83565ba740 10 client.9099 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f83565ba740 1 -- 172.21.15.26:0/4041326413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x557fc4ac3440 con 0x557fc4abd680 2024-04-24T23:27:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f832bfff640 1 -- 172.21.15.26:0/4041326413 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c0100c0 con 0x557fc4abd680 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.779+0000 7f832bfff640 1 -- 172.21.15.26:0/4041326413 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f834404c030 con 0x557fc4abd680 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.780+0000 7f83565ba740 10 client.9099 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.780+0000 7f83565ba740 10 client.9099 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.780+0000 7f83565ba740 20 client.9099 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.780+0000 7f83565ba740 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x557fc4b26620 0x557fc4aad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.780+0000 7f83565ba740 4 client.9099 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.780+0000 7f83565ba740 1 -- 172.21.15.26:0/4041326413 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x557fc4a7d0d0 con 0x557fc4b26620 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.780+0000 7f8351641640 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x557fc4b26620 0x557fc4aad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.780+0000 7f8351641640 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x557fc4b26620 0x557fc4aad460 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4145d28e18eed099 server_cookie=20f7473d90efb564 in_seq=0 out_seq=0 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.780+0000 7f832bfff640 10 client.9099 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f832bfff640 1 -- 172.21.15.26:0/4041326413 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557fc4a7d0d0 con 0x557fc4b26620 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f832bfff640 10 client.9099 handle_command_reply: tid=0 2024-04-24T23:27:52.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 client.9099 shutdown 2024-04-24T23:27:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 -- 172.21.15.26:0/4041326413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4abd680 msgr2=0x557fc4aacf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4abd680 0x557fc4aacf20 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f834407d040 tx=0x7f8344009750 comp rx=0 tx=0).stop 2024-04-24T23:27:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 -- 172.21.15.26:0/4041326413 shutdown_connections 2024-04-24T23:27:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x557fc4b26620 0x557fc4aad460 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fc4abd680 0x557fc4aacf20 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x557fc4aac580 0x557fc4ac68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f832bfff640 0 client.9099 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 --2- 172.21.15.26:0/4041326413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fc4aabc60 0x557fc4aca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 -- 172.21.15.26:0/4041326413 >> 172.21.15.26:0/4041326413 conn(0x557fc4a485e0 msgr2=0x557fc4ac0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 -- 172.21.15.26:0/4041326413 shutdown_connections 2024-04-24T23:27:52.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 1 -- 172.21.15.26:0/4041326413 wait complete. 2024-04-24T23:27:52.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.781+0000 7f83565ba740 20 client.9099 trim_cache size 0 max 16384 2024-04-24T23:27:52.790 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-04-24T23:27:52.782+0000 7fb852d4e640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1) n(v2 rc2024-04-24T23:27:35.581019+0000 4=0+4) (inest lock) (ifile excl) caps={7552=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7552 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x5572738d1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.581019+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-04-24T23:27:35.581019+0000 4=0+4)","error_str":""},"return_code":-61} 2024-04-24T23:27:52.790 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2024-04-24T23:27:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:27:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2024-04-24T23:27:52.791 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:52.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 -- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f833006be30 msgr2=0x7f833006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f833006be30 0x7f833006e2f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f834c15d7b0 tx=0x7f8348007e80 comp rx=0 tx=0).stop 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 -- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c15e710 msgr2=0x7f834c152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c15e710 0x7f834c152a90 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8344009ee0 tx=0x7f8344004a40 comp rx=0 tx=0).stop 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 -- 172.21.15.26:0/3089237398 shutdown_connections 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f833006be30 0x7f833006e2f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834c15e710 0x7f834c152a90 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f834c15e330 0x7f834c15f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 --2- 172.21.15.26:0/3089237398 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f834c0d4630 0x7f834c0b96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 -- 172.21.15.26:0/3089237398 >> 172.21.15.26:0/3089237398 conn(0x7f834c08dbc0 msgr2=0x7f834c0b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 -- 172.21.15.26:0/3089237398 shutdown_connections 2024-04-24T23:27:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.782+0000 7f83538a5640 1 -- 172.21.15.26:0/3089237398 wait complete. 2024-04-24T23:27:52.792 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-04-24T23:27:52.783+0000 7fb85955b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1) n(v1 rc2024-04-24T23:27:35.581019+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7552=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55727295b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.350023+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-04-24T23:27:35.581019+0000 3=0+3)","error_str":""},"return_code":-61} 2024-04-24T23:27:52.793 INFO:tasks.ceph.mds.b.smithi029.stderr:2024-04-24T23:27:52.782+0000 7f02efa72640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-04-24T23:27:35.350023+0000 1=0+1) n(v0 rc2024-04-24T23:27:35.350023+0000 2=0+2) (inest lock) (ifile excl) caps={7552=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7552 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55fcf25d3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.350023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.350023+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.350023+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-04-24T23:27:35.350023+0000 2=0+2)","error_str":""},"return_code":-61} 2024-04-24T23:27:52.793 INFO:tasks.ceph.mds.b.smithi029.stderr:2024-04-24T23:27:52.783+0000 7f02efa72640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={7552=pAsLsXsFsx/AsLsXsFsx@2},l=7552 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55fcf25d3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-04-24T23:27:52.793 INFO:tasks.ceph.mds.d.smithi029.stderr:2024-04-24T23:27:52.783+0000 7f42b7576640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-04-24T23:27:35.348023+0000 1=0+1) n(v4 rc2024-04-24T23:27:35.581019+0000 5=0+5) (inest sync->mix g=1,2) caps={7552=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x558526e72100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.348023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.581019+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.348023+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-04-24T23:27:35.581019+0000 5=0+5)","error_str":""},"return_code":-61} 2024-04-24T23:27:52.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:52.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-24T23:27:52.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.896+0000 7fd9d186d640 1 Processor -- start 2024-04-24T23:27:52.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.896+0000 7fd9d186d640 1 -- start start 2024-04-24T23:27:52.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9d186d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9cc15e370 0x7fd9cc0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9d186d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e750 0x7fd9cc0d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9d186d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fd9cc0d4c10 0x7fd9cc0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9d186d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd9cc0d1510 con 0x7fd9cc15e370 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9d186d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fd9cc0d1ed0 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9d186d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc0d2120 con 0x7fd9cc15e750 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9caffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9cc15e370 0x7fd9cc0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9caffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9cc15e370 0x7fd9cc0d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56492/0 (socket says 172.21.15.26:56492) 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9caffd640 1 -- 172.21.15.26:0/3904963056 learned_addr learned my addr 172.21.15.26:0/3904963056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.897+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/3904963056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e750 0x7fd9cc0d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9cb7fe640 1 --2- 172.21.15.26:0/3904963056 >> v2:172.21.15.29:3300/0 conn(0x7fd9cc0d4c10 0x7fd9cc0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9caffd640 1 -- 172.21.15.26:0/3904963056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e750 msgr2=0x7fd9cc0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9caffd640 1 --2- 172.21.15.26:0/3904963056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e750 0x7fd9cc0d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9caffd640 1 -- 172.21.15.26:0/3904963056 >> v2:172.21.15.29:3300/0 conn(0x7fd9cc0d4c10 msgr2=0x7fd9cc0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9caffd640 1 --2- 172.21.15.26:0/3904963056 >> v2:172.21.15.29:3300/0 conn(0x7fd9cc0d4c10 0x7fd9cc0b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9caffd640 1 -- 172.21.15.26:0/3904963056 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc0b3740 con 0x7fd9cc15e370 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9caffd640 1 --2- 172.21.15.26:0/3904963056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc15e370 0x7fd9cc0d2520 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc161f80 tx=0x7fd9c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a854bb6c84696cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9c9ffb640 1 -- 172.21.15.26:0/3904963056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4037070 con 0x7fd9cc15e370 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9c9ffb640 1 -- 172.21.15.26:0/3904963056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c4032440 con 0x7fd9cc15e370 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.898+0000 7fd9c9ffb640 1 -- 172.21.15.26:0/3904963056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4032810 con 0x7fd9cc15e370 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.899+0000 7fd9d186d640 1 -- 172.21.15.26:0/3904963056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc15e370 msgr2=0x7fd9cc0d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.899+0000 7fd9d186d640 1 --2- 172.21.15.26:0/3904963056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc15e370 0x7fd9cc0d2520 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc161f80 tx=0x7fd9c402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.899+0000 7fd9d186d640 1 -- 172.21.15.26:0/3904963056 shutdown_connections 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.899+0000 7fd9d186d640 1 --2- 172.21.15.26:0/3904963056 >> v2:172.21.15.29:3300/0 conn(0x7fd9cc0d4c10 0x7fd9cc0b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.899+0000 7fd9d186d640 1 --2- 172.21.15.26:0/3904963056 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e750 0x7fd9cc0d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.899+0000 7fd9d186d640 1 --2- 172.21.15.26:0/3904963056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc15e370 0x7fd9cc0d2520 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.899+0000 7fd9d186d640 1 -- 172.21.15.26:0/3904963056 >> 172.21.15.26:0/3904963056 conn(0x7fd9cc0d40f0 msgr2=0x7fd9cc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.899+0000 7fd9d186d640 1 -- 172.21.15.26:0/3904963056 shutdown_connections 2024-04-24T23:27:52.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.899+0000 7fd9d186d640 1 -- 172.21.15.26:0/3904963056 wait complete. 2024-04-24T23:27:52.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.900+0000 7fd9d186d640 1 Processor -- start 2024-04-24T23:27:52.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.900+0000 7fd9d186d640 1 -- start start 2024-04-24T23:27:52.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9d186d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc0d4c10 0x7fd9cc14e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9d186d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e370 0x7fd9cc0ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9d186d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9cc0b89e0 0x7fd9cc14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9d186d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc0b4760 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9d186d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc0b3d00 con 0x7fd9cc0b89e0 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9d186d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc0b48e0 con 0x7fd9cc15e370 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc0d4c10 0x7fd9cc14e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc0d4c10 0x7fd9cc14e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56500/0 (socket says 172.21.15.26:56500) 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9caffd640 1 -- 172.21.15.26:0/4173420024 learned_addr learned my addr 172.21.15.26:0/4173420024 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e370 0x7fd9cc0ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.901+0000 7fd9cb7fe640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9cc0b89e0 0x7fd9cc14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9caffd640 1 -- 172.21.15.26:0/4173420024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e370 msgr2=0x7fd9cc0ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9caffd640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e370 0x7fd9cc0ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9caffd640 1 -- 172.21.15.26:0/4173420024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9cc0b89e0 msgr2=0x7fd9cc14d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9caffd640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9cc0b89e0 0x7fd9cc14d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9caffd640 1 -- 172.21.15.26:0/4173420024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc0b6e80 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9caffd640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc0d4c10 0x7fd9cc14e0c0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd9c402a220 tx=0x7fd9c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9c3fff640 1 -- 172.21.15.26:0/4173420024 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4041020 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9c3fff640 1 -- 172.21.15.26:0/4173420024 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c40306c0 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9c3fff640 1 -- 172.21.15.26:0/4173420024 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4037040 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9d186d640 1 -- 172.21.15.26:0/4173420024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc0b9260 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.902+0000 7fd9d186d640 1 -- 172.21.15.26:0/4173420024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc0b9820 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.903+0000 7fd9c3fff640 1 -- 172.21.15.26:0/4173420024 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9c4044020 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.904+0000 7fd9c3fff640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd9ac06be30 0x7fd9ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.904+0000 7fd9c3fff640 1 -- 172.21.15.26:0/4173420024 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fd9c40ab520 con 0x7fd9cc0d4c10 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.904+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd9ac06be30 0x7fd9ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.904+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd9ac06be30 0x7fd9ac06e2f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0ce6c0 tx=0x7fd9b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.905+0000 7fd9d4577740 1 Processor -- start 2024-04-24T23:27:52.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.905+0000 7fd9d4577740 1 -- start start 2024-04-24T23:27:52.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214a93510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ac3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ac3560 0x55a214a98370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9d4577740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2149fc330 con 0x55a214ae0b80 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9d4577740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a2149a4ab0 con 0x55a214ac3560 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9d4577740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a2149f82f0 con 0x55a214ac3ca0 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214a93510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214a93510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56504/0 (socket says 172.21.15.26:56504) 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9caffd640 1 -- 172.21.15.26:0/2812295829 learned_addr learned my addr 172.21.15.26:0/2812295829 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ac3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9cb7fe640 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ac3560 0x55a214a98370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9caffd640 1 -- 172.21.15.26:0/2812295829 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 msgr2=0x55a214ac3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9caffd640 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ac3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9caffd640 1 -- 172.21.15.26:0/2812295829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ac3560 msgr2=0x55a214a98370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9caffd640 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ac3560 0x55a214a98370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9caffd640 1 -- 172.21.15.26:0/2812295829 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2147da7d0 con 0x55a214ae0b80 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9cb7fe640 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ac3560 0x55a214a98370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9caffd640 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214a93510 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40a9a20 tx=0x7fd9c4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6533dc32cae61c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9c17fa640 1 -- 172.21.15.26:0/2812295829 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4082440 con 0x55a214ae0b80 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9c17fa640 1 -- 172.21.15.26:0/2812295829 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c40800b0 con 0x55a214ae0b80 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.906+0000 7fd9c17fa640 1 -- 172.21.15.26:0/2812295829 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4075f80 con 0x55a214ae0b80 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.907+0000 7fd9d4577740 1 -- 172.21.15.26:0/2812295829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 msgr2=0x55a214a93510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.907+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214a93510 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40a9a20 tx=0x7fd9c4009580 comp rx=0 tx=0).stop 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.907+0000 7fd9d4577740 1 -- 172.21.15.26:0/2812295829 shutdown_connections 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.907+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ac3560 0x55a214a98370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.907+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ac3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.907+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2812295829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214a93510 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.907+0000 7fd9d4577740 1 -- 172.21.15.26:0/2812295829 >> 172.21.15.26:0/2812295829 conn(0x55a214a95780 msgr2=0x55a214ade020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.907+0000 7fd9d4577740 1 -- 172.21.15.26:0/2812295829 shutdown_connections 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.907+0000 7fd9d4577740 1 -- 172.21.15.26:0/2812295829 wait complete. 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.908+0000 7fd9d4577740 1 -- start start 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.908+0000 7fd9d4577740 1 Processor -- start 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ad4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ae0b80 0x55a214ad63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ab0ae0 0x55a214ad6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9d4577740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2149fc330 con 0x55a214ab0ae0 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9d4577740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a2149b7f40 con 0x55a214ae0b80 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9d4577740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a214a97930 con 0x55a214ac3ca0 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9d4577740 10 client.0 _ll_register_callbacks cb 0x55a21470fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ab0ae0 0x55a214ad6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ad4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ab0ae0 0x55a214ad6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56506/0 (socket says 172.21.15.26:56506) 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9cb7fe640 1 -- 172.21.15.26:0/860751309 learned_addr learned my addr 172.21.15.26:0/860751309 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ae0b80 0x55a214ad63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9caffd640 1 -- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ae0b80 msgr2=0x55a214ad63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9caffd640 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ae0b80 0x55a214ad63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9caffd640 1 -- 172.21.15.26:0/860751309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ab0ae0 msgr2=0x55a214ad6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9caffd640 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ab0ae0 0x55a214ad6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9caffd640 1 -- 172.21.15.26:0/860751309 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55a2146d00f0 con 0x55a214ac3ca0 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9cb7fe640 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ab0ae0 0x55a214ad6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9caffd640 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ad4ce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009630 tx=0x7fd9c40ab070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f33b3d1d9f698ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.909+0000 7fd9aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3301/0 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/860751309 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40a9070 con 0x55a214ac3ca0 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/860751309 <== mon.2 v2:172.21.15.29:3301/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fd9c4081fb0 con 0x55a214ac3ca0 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9d4577740 1 -- 172.21.15.26:0/860751309 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c40800b0 con 0x55a214ac3ca0 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/860751309 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fd9c40821d0 con 0x55a214ac3ca0 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9d4577740 10 client.7043 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9d4577740 1 -- 172.21.15.26:0/860751309 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55a21483af60 con 0x55a214ac3ca0 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/860751309 <== mon.2 v2:172.21.15.29:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c40800b0 con 0x55a214ac3ca0 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/860751309 <== mon.2 v2:172.21.15.29:3301/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7fd9c403a030 con 0x55a214ac3ca0 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9d4577740 10 client.7043 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9d4577740 10 client.7043 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9d4577740 20 client.7043 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9d4577740 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a214ac6600 0x55a214ac6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9d4577740 4 client.7043 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9d4577740 1 -- 172.21.15.26:0/860751309 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a214a09870 con 0x55a214ac6600 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.910+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a214ac6600 0x55a214ac6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.911+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a214ac6600 0x55a214ac6a50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ed99198e06a2f0 server_cookie=c0fa574fe3100d8 in_seq=0 out_seq=0 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.911+0000 7fd9aa7fc640 10 client.7043 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.911+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/860751309 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a214a09870 con 0x55a214ac6600 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.911+0000 7fd9aa7fc640 10 client.7043 handle_command_reply: tid=0 2024-04-24T23:27:52.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.911+0000 7fd9d4577740 1 client.7043 shutdown 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 -- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 msgr2=0x55a214ad4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ad4ce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009630 tx=0x7fd9c40ab070 comp rx=0 tx=0).stop 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 -- 172.21.15.26:0/860751309 shutdown_connections 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a214ac6600 0x55a214ac6a50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ab0ae0 0x55a214ad6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ae0b80 0x55a214ad63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 --2- 172.21.15.26:0/860751309 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ad4ce0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9aa7fc640 0 client.7043 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 -- 172.21.15.26:0/860751309 >> 172.21.15.26:0/860751309 conn(0x55a214ac3560 msgr2=0x55a214a922b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 -- 172.21.15.26:0/860751309 shutdown_connections 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 1 -- 172.21.15.26:0/860751309 wait complete. 2024-04-24T23:27:52.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.912+0000 7fd9d4577740 20 client.7043 trim_cache size 0 max 16384 2024-04-24T23:27:52.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.929+0000 7fd9d4577740 1 Processor -- start 2024-04-24T23:27:52.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.929+0000 7fd9d4577740 1 -- start start 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 0x55a214ad6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ac3ca0 0x55a214a96fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ae0b80 0x55a214aedd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9d4577740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a214adef00 con 0x55a214ac3ca0 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9d4577740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc2008d0 con 0x55a214ab0ae0 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9d4577740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc1ffaa0 con 0x55a214ae0b80 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9ca7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ac3ca0 0x55a214a96fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9ca7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ac3ca0 0x55a214a96fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56508/0 (socket says 172.21.15.26:56508) 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9ca7fc640 1 -- 172.21.15.26:0/786609745 learned_addr learned my addr 172.21.15.26:0/786609745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9cb7fe640 1 --2- 172.21.15.26:0/786609745 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ae0b80 0x55a214aedd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9caffd640 1 --2- 172.21.15.26:0/786609745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 0x55a214ad6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9ca7fc640 1 -- 172.21.15.26:0/786609745 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ae0b80 msgr2=0x55a214aedd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/786609745 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ae0b80 0x55a214aedd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9ca7fc640 1 -- 172.21.15.26:0/786609745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 msgr2=0x55a214ad6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/786609745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 0x55a214ad6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9ca7fc640 1 -- 172.21.15.26:0/786609745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2146a91b0 con 0x55a214ac3ca0 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.930+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/786609745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ac3ca0 0x55a214a96fe0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc151930 tx=0x7fd9b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3b5332e2bb949e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9c0ff9640 1 -- 172.21.15.26:0/786609745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4031070 con 0x55a214ac3ca0 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9c0ff9640 1 -- 172.21.15.26:0/786609745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b401d040 con 0x55a214ac3ca0 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9c0ff9640 1 -- 172.21.15.26:0/786609745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b401c070 con 0x55a214ac3ca0 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9d4577740 1 -- 172.21.15.26:0/786609745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ac3ca0 msgr2=0x55a214a96fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9d4577740 1 --2- 172.21.15.26:0/786609745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ac3ca0 0x55a214a96fe0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc151930 tx=0x7fd9b4016000 comp rx=0 tx=0).stop 2024-04-24T23:27:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9d4577740 1 -- 172.21.15.26:0/786609745 shutdown_connections 2024-04-24T23:27:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9d4577740 1 --2- 172.21.15.26:0/786609745 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ae0b80 0x55a214aedd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9d4577740 1 --2- 172.21.15.26:0/786609745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ac3ca0 0x55a214a96fe0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9d4577740 1 --2- 172.21.15.26:0/786609745 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 0x55a214ad6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9d4577740 1 -- 172.21.15.26:0/786609745 >> 172.21.15.26:0/786609745 conn(0x55a214ac3560 msgr2=0x55a214addf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9d4577740 1 -- 172.21.15.26:0/786609745 shutdown_connections 2024-04-24T23:27:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.931+0000 7fd9d4577740 1 -- 172.21.15.26:0/786609745 wait complete. 2024-04-24T23:27:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.932+0000 7fd9d4577740 1 -- start start 2024-04-24T23:27:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.932+0000 7fd9d4577740 1 Processor -- start 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 0x55a214b3dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ad6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9d4577740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214b5b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9d4577740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1f80e0 con 0x55a214ae0b80 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9d4577740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc2001a0 con 0x55a214ab0ae0 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9d4577740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55a214a8db90 con 0x55a214ac3ca0 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9d4577740 10 client.0 _ll_register_callbacks cb 0x7fd9c4090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214b5b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214b5b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56518/0 (socket says 172.21.15.26:56518) 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9cb7fe640 1 -- 172.21.15.26:0/2746300874 learned_addr learned my addr 172.21.15.26:0/2746300874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9ca7fc640 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ad6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9caffd640 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 0x55a214b3dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9cb7fe640 1 -- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 msgr2=0x55a214ad6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9cb7fe640 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ad6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9cb7fe640 1 -- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 msgr2=0x55a214b3dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9cb7fe640 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 0x55a214b3dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9cb7fe640 1 -- 172.21.15.26:0/2746300874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a214b5b8b0 con 0x55a214ae0b80 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9cb7fe640 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214b5b370 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc007770 tx=0x7fd9bc007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec0bd328489f67b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/2746300874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc01d030 con 0x55a214ae0b80 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/2746300874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fd9bc012040 con 0x55a214ae0b80 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.933+0000 7fd9d4577740 1 -- 172.21.15.26:0/2746300874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b401d040 con 0x55a214ae0b80 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/2746300874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fd9bc016a50 con 0x55a214ae0b80 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9d4577740 10 client.9117 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9d4577740 1 -- 172.21.15.26:0/2746300874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55a214b57fb0 con 0x55a214ae0b80 2024-04-24T23:27:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/2746300874 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b401d040 con 0x55a214ae0b80 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/2746300874 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7fd9bc02c080 con 0x55a214ae0b80 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9d4577740 10 client.9117 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9d4577740 10 client.9117 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9d4577740 20 client.9117 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a214b584d0 0x55a214affa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9d4577740 4 client.9117 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9d4577740 1 -- 172.21.15.26:0/2746300874 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x55a214a03bc0 con 0x55a214b584d0 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9caffd640 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a214b584d0 0x55a214affa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9caffd640 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a214b584d0 0x55a214affa40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b06b8d36885c7768 server_cookie=5c89bd56bbb5a5ab in_seq=0 out_seq=0 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.934+0000 7fd9aa7fc640 10 client.9117 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9aa7fc640 1 -- 172.21.15.26:0/2746300874 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a214a03bc0 con 0x55a214b584d0 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9aa7fc640 10 client.9117 handle_command_reply: tid=0 2024-04-24T23:27:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 client.9117 shutdown 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 -- 172.21.15.26:0/2746300874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 msgr2=0x55a214b5b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214b5b370 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc007770 tx=0x7fd9bc007c30 comp rx=0 tx=0).stop 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 -- 172.21.15.26:0/2746300874 shutdown_connections 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55a214b584d0 0x55a214affa40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a214ae0b80 0x55a214b5b370 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55a214ac3ca0 0x55a214ad6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9aa7fc640 0 client.9117 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 --2- 172.21.15.26:0/2746300874 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a214ab0ae0 0x55a214b3dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 -- 172.21.15.26:0/2746300874 >> 172.21.15.26:0/2746300874 conn(0x55a214ac3560 msgr2=0x55a214a92440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 -- 172.21.15.26:0/2746300874 shutdown_connections 2024-04-24T23:27:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 1 -- 172.21.15.26:0/2746300874 wait complete. 2024-04-24T23:27:52.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.935+0000 7fd9d4577740 20 client.9117 trim_cache size 0 max 16384 2024-04-24T23:27:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:27:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 -- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd9ac06be30 msgr2=0x7fd9ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd9ac06be30 0x7fd9ac06e2f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0ce6c0 tx=0x7fd9b4008040 comp rx=0 tx=0).stop 2024-04-24T23:27:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 -- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc0d4c10 msgr2=0x7fd9cc14e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc0d4c10 0x7fd9cc14e0c0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd9c402a220 tx=0x7fd9c40046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 -- 172.21.15.26:0/4173420024 shutdown_connections 2024-04-24T23:27:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fd9ac06be30 0x7fd9ac06e2f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9cc0b89e0 0x7fd9cc14d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fd9cc15e370 0x7fd9cc0ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 --2- 172.21.15.26:0/4173420024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9cc0d4c10 0x7fd9cc14e0c0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 -- 172.21.15.26:0/4173420024 >> 172.21.15.26:0/4173420024 conn(0x7fd9cc0d40f0 msgr2=0x7fd9cc0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 -- 172.21.15.26:0/4173420024 shutdown_connections 2024-04-24T23:27:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:52.936+0000 7fd9d186d640 1 -- 172.21.15.26:0/4173420024 wait complete. 2024-04-24T23:27:52.947 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:27:52.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-04-24T23:27:53.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.051+0000 7f312ec72640 1 Processor -- start 2024-04-24T23:27:53.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.052+0000 7f312ec72640 1 -- start start 2024-04-24T23:27:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.052+0000 7f312ec72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e120 0x7f312806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.052+0000 7f312ec72640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f312815e500 0x7f31280b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.052+0000 7f312ec72640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3128070f30 con 0x7f31280b3d20 2024-04-24T23:27:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.052+0000 7f312ec72640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f312806dcf0 con 0x7f312815e500 2024-04-24T23:27:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.052+0000 7f312ec72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f312806dfd0 con 0x7f312815e120 2024-04-24T23:27:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.053+0000 7f312ca0e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e120 0x7f312806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.053+0000 7f3127fff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f312815e500 0x7f31280b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.053+0000 7f312ca0e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e120 0x7f312806d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:48994/0 (socket says 172.21.15.26:48994) 2024-04-24T23:27:53.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.053+0000 7f312ca0e640 1 -- 172.21.15.26:0/547928179 learned_addr learned my addr 172.21.15.26:0/547928179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.053+0000 7f312d20f640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f31280b3d20 0x7f3128150e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:59486/0 (socket says 172.21.15.26:59486) 2024-04-24T23:27:53.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.053+0000 7f312ca0e640 1 -- 172.21.15.26:0/547928179 >> v2:172.21.15.29:3300/0 conn(0x7f312815e500 msgr2=0x7f31280b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.053+0000 7f312ca0e640 1 --2- 172.21.15.26:0/547928179 >> v2:172.21.15.29:3300/0 conn(0x7f312815e500 0x7f31280b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.053+0000 7f312ca0e640 1 -- 172.21.15.26:0/547928179 >> v1:172.21.15.29:6789/0 conn(0x7f31280b3d20 legacy=0x7f3128150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.053+0000 7f312ca0e640 1 -- 172.21.15.26:0/547928179 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312806e2b0 con 0x7f312815e120 2024-04-24T23:27:53.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.054+0000 7f312ca0e640 1 --2- 172.21.15.26:0/547928179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e120 0x7f312806d5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f311c00bb00 tx=0x7f311c02c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd1fd6473600991 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.054+0000 7f31277fe640 1 -- 172.21.15.26:0/547928179 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c03a070 con 0x7f312815e120 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.054+0000 7f31277fe640 1 -- 172.21.15.26:0/547928179 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c032480 con 0x7f312815e120 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.054+0000 7f31277fe640 1 -- 172.21.15.26:0/547928179 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c032850 con 0x7f312815e120 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.055+0000 7f312ec72640 1 -- 172.21.15.26:0/547928179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e120 msgr2=0x7f312806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.055+0000 7f312ec72640 1 --2- 172.21.15.26:0/547928179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e120 0x7f312806d5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f311c00bb00 tx=0x7f311c02c6e0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.055+0000 7f312ec72640 1 -- 172.21.15.26:0/547928179 shutdown_connections 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.055+0000 7f312ec72640 1 --2- 172.21.15.26:0/547928179 >> v2:172.21.15.29:3300/0 conn(0x7f312815e500 0x7f31280b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.055+0000 7f312ec72640 1 --2- 172.21.15.26:0/547928179 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e120 0x7f312806d5a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.055+0000 7f312ec72640 1 -- 172.21.15.26:0/547928179 >> 172.21.15.26:0/547928179 conn(0x7f312806b540 msgr2=0x7f312806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.055+0000 7f312ec72640 1 -- 172.21.15.26:0/547928179 shutdown_connections 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.055+0000 7f312ec72640 1 -- 172.21.15.26:0/547928179 wait complete. 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.056+0000 7f312ec72640 1 Processor -- start 2024-04-24T23:27:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.056+0000 7f312ec72640 1 -- start start 2024-04-24T23:27:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f312ec72640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31280b3d20 0x7f312813c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f312ec72640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312815e120 0x7f31280af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f312ec72640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e500 0x7f312814b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f312ec72640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f312815e940 con 0x7f312815e120 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f312ec72640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3128151570 con 0x7f31280b3d20 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f312ec72640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f312809af10 con 0x7f312815e500 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f3127fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312815e120 0x7f31280af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f3127fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312815e120 0x7f31280af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56532/0 (socket says 172.21.15.26:56532) 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f3127fff640 1 -- 172.21.15.26:0/880819733 learned_addr learned my addr 172.21.15.26:0/880819733 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f312d20f640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e500 0x7f312814b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f312ca0e640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31280b3d20 0x7f312813c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f3127fff640 1 -- 172.21.15.26:0/880819733 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e500 msgr2=0x7f312814b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f3127fff640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e500 0x7f312814b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f3127fff640 1 -- 172.21.15.26:0/880819733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31280b3d20 msgr2=0x7f312813c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f3127fff640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31280b3d20 0x7f312813c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f3127fff640 1 -- 172.21.15.26:0/880819733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31280b02f0 con 0x7f312815e120 2024-04-24T23:27:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.057+0000 7f3127fff640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312815e120 0x7f31280af8a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f311400e870 tx=0x7f311400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.058+0000 7f31257fa640 1 -- 172.21.15.26:0/880819733 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114017070 con 0x7f312815e120 2024-04-24T23:27:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.058+0000 7f31257fa640 1 -- 172.21.15.26:0/880819733 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311400c940 con 0x7f312815e120 2024-04-24T23:27:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.058+0000 7f312ec72640 1 -- 172.21.15.26:0/880819733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31280b05d0 con 0x7f312815e120 2024-04-24T23:27:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.058+0000 7f31257fa640 1 -- 172.21.15.26:0/880819733 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311400cd30 con 0x7f312815e120 2024-04-24T23:27:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.058+0000 7f312ec72640 1 -- 172.21.15.26:0/880819733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312813cab0 con 0x7f312815e120 2024-04-24T23:27:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.059+0000 7f31257fa640 1 -- 172.21.15.26:0/880819733 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f311401a070 con 0x7f312815e120 2024-04-24T23:27:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.060+0000 7f31257fa640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f310c06be60 0x7f310c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.060+0000 7f312ca0e640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f310c06be60 0x7f310c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.060+0000 7f31257fa640 1 -- 172.21.15.26:0/880819733 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f311408b0a0 con 0x7f312815e120 2024-04-24T23:27:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.060+0000 7f312ca0e640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f310c06be60 0x7f310c06e320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f311c037000 tx=0x7f311c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.061+0000 7f313197f740 1 Processor -- start 2024-04-24T23:27:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.061+0000 7f313197f740 1 -- start start 2024-04-24T23:27:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f313197f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b4310 0x55b2b878c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f313197f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b46f0 0x55b2b87b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f313197f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b70b0 0x55b2b878a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f313197f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b2b86f06b0 con 0x55b2b87b4310 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f313197f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b2b8698880 con 0x55b2b87b70b0 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f313197f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b2b86ec670 con 0x55b2b87b46f0 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f312ca0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b4310 0x55b2b878c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f312ca0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b4310 0x55b2b878c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56536/0 (socket says 172.21.15.26:56536) 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f312ca0e640 1 -- 172.21.15.26:0/3213019844 learned_addr learned my addr 172.21.15.26:0/3213019844 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f312ca0e640 1 -- 172.21.15.26:0/3213019844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b46f0 msgr2=0x55b2b87b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f312ca0e640 1 --2- 172.21.15.26:0/3213019844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b46f0 0x55b2b87b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f312ca0e640 1 -- 172.21.15.26:0/3213019844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b70b0 msgr2=0x55b2b878a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f312ca0e640 1 --2- 172.21.15.26:0/3213019844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b70b0 0x55b2b878a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f312ca0e640 1 -- 172.21.15.26:0/3213019844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b84ce7d0 con 0x55b2b87b4310 2024-04-24T23:27:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f312ca0e640 1 --2- 172.21.15.26:0/3213019844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b4310 0x55b2b878c970 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f311c00bbf0 tx=0x7f311c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8877a1e72a70594c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f31127fc640 1 -- 172.21.15.26:0/3213019844 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c03a040 con 0x55b2b87b4310 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f31127fc640 1 -- 172.21.15.26:0/3213019844 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c032d30 con 0x55b2b87b4310 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.062+0000 7f31127fc640 1 -- 172.21.15.26:0/3213019844 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c04a630 con 0x55b2b87b4310 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 -- 172.21.15.26:0/3213019844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b4310 msgr2=0x55b2b878c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 --2- 172.21.15.26:0/3213019844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b4310 0x55b2b878c970 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f311c00bbf0 tx=0x7f311c032000 comp rx=0 tx=0).stop 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 -- 172.21.15.26:0/3213019844 shutdown_connections 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 --2- 172.21.15.26:0/3213019844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b70b0 0x55b2b878a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 --2- 172.21.15.26:0/3213019844 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b46f0 0x55b2b87b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 --2- 172.21.15.26:0/3213019844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b4310 0x55b2b878c970 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 -- 172.21.15.26:0/3213019844 >> 172.21.15.26:0/3213019844 conn(0x55b2b8789600 msgr2=0x55b2b87d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 -- 172.21.15.26:0/3213019844 shutdown_connections 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 -- 172.21.15.26:0/3213019844 wait complete. 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.063+0000 7f313197f740 1 -- start start 2024-04-24T23:27:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.064+0000 7f313197f740 1 Processor -- start 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.064+0000 7f313197f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.064+0000 7f313197f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b70b0 0x55b2b87b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.064+0000 7f313197f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 0x55b2b87b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.064+0000 7f313197f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b2b86abf50 con 0x55b2b87b46f0 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.064+0000 7f313197f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b2b87cf870 con 0x55b2b87b70b0 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.064+0000 7f313197f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b2b87b9c20 con 0x55b2b87b3920 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.064+0000 7f313197f740 10 client.0 _ll_register_callbacks cb 0x55b2b86e7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312ca0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312ca0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56540/0 (socket says 172.21.15.26:56540) 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312ca0e640 1 -- 172.21.15.26:0/2909776440 learned_addr learned my addr 172.21.15.26:0/2909776440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312d20f640 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 0x55b2b87b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f3127fff640 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b70b0 0x55b2b87b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312ca0e640 1 -- 172.21.15.26:0/2909776440 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 msgr2=0x55b2b87b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312ca0e640 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 0x55b2b87b8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312ca0e640 1 -- 172.21.15.26:0/2909776440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b70b0 msgr2=0x55b2b87b89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312ca0e640 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b70b0 0x55b2b87b89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312ca0e640 1 -- 172.21.15.26:0/2909776440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b84043f0 con 0x55b2b87b46f0 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f312ca0e640 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2cd0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f311c032ac0 tx=0x7f311c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb8aa77ac9ec8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f31077fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f31077fe640 1 -- 172.21.15.26:0/2909776440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c03a040 con 0x55b2b87b46f0 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f31077fe640 1 -- 172.21.15.26:0/2909776440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f311c03f410 con 0x55b2b87b46f0 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f313197f740 1 -- 172.21.15.26:0/2909776440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f311c032d30 con 0x55b2b87b46f0 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f31077fe640 1 -- 172.21.15.26:0/2909776440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f311c064050 con 0x55b2b87b46f0 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f313197f740 10 client.9126 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.065+0000 7f313197f740 1 -- 172.21.15.26:0/2909776440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55b2b83c40f0 con 0x55b2b87b46f0 2024-04-24T23:27:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f31077fe640 1 -- 172.21.15.26:0/2909776440 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c032d30 con 0x55b2b87b46f0 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f31077fe640 1 -- 172.21.15.26:0/2909776440 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f311c067020 con 0x55b2b87b46f0 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f313197f740 10 client.9126 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f313197f740 10 client.9126 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f313197f740 20 client.9126 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f313197f740 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x55b2b87e9ef0 0x55b2b87ea300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f313197f740 4 client.9126 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f313197f740 1 -- 172.21.15.26:0/2909776440 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b86fdbf0 con 0x55b2b87e9ef0 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f3127fff640 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x55b2b87e9ef0 0x55b2b87ea300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f31077fe640 10 client.9126 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.066+0000 7f3127fff640 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x55b2b87e9ef0 0x55b2b87ea300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d10120a9fe23e3d server_cookie=9603e5fadbd250fa in_seq=0 out_seq=0 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f31077fe640 1 -- 172.21.15.26:0/2909776440 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2b86fdbf0 con 0x55b2b87e9ef0 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f31077fe640 10 client.9126 handle_command_reply: tid=0 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 client.9126 shutdown 2024-04-24T23:27:53.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 -- 172.21.15.26:0/2909776440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b46f0 msgr2=0x55b2b87b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2cd0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f311c032ac0 tx=0x7f311c02e000 comp rx=0 tx=0).stop 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 -- 172.21.15.26:0/2909776440 shutdown_connections 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x55b2b87e9ef0 0x55b2b87ea300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 0x55b2b87b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b70b0 0x55b2b87b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 --2- 172.21.15.26:0/2909776440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2cd0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f31077fe640 0 client.9126 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 -- 172.21.15.26:0/2909776440 >> 172.21.15.26:0/2909776440 conn(0x55b2b87b4310 msgr2=0x55b2b8786c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 -- 172.21.15.26:0/2909776440 shutdown_connections 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 1 -- 172.21.15.26:0/2909776440 wait complete. 2024-04-24T23:27:53.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.067+0000 7f313197f740 20 client.9126 trim_cache size 0 max 16384 2024-04-24T23:27:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.085+0000 7f313197f740 1 Processor -- start 2024-04-24T23:27:53.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.085+0000 7f313197f740 1 -- start start 2024-04-24T23:27:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.085+0000 7f313197f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b3920 0x55b2b8786450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.085+0000 7f313197f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b46f0 0x55b2b87d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.085+0000 7f313197f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b70b0 0x55b2b87d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.085+0000 7f313197f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b2b87b7910 con 0x55b2b87b3920 2024-04-24T23:27:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.085+0000 7f313197f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31281fc510 con 0x55b2b87b46f0 2024-04-24T23:27:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.085+0000 7f313197f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f312809c6a0 con 0x55b2b87b70b0 2024-04-24T23:27:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f312ca0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b3920 0x55b2b8786450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f312ca0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b3920 0x55b2b8786450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56556/0 (socket says 172.21.15.26:56556) 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f312ca0e640 1 -- 172.21.15.26:0/418546301 learned_addr learned my addr 172.21.15.26:0/418546301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f312ca0e640 1 -- 172.21.15.26:0/418546301 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b70b0 msgr2=0x55b2b87d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f312ca0e640 1 --2- 172.21.15.26:0/418546301 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b70b0 0x55b2b87d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f312ca0e640 1 -- 172.21.15.26:0/418546301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b46f0 msgr2=0x55b2b87d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f312ca0e640 1 --2- 172.21.15.26:0/418546301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b46f0 0x55b2b87d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f312ca0e640 1 -- 172.21.15.26:0/418546301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b87d22b0 con 0x55b2b87b3920 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f312ca0e640 1 --2- 172.21.15.26:0/418546301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b3920 0x55b2b8786450 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f311c02cc50 tx=0x7f311c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4323dc8f423c39cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f3111ffb640 1 -- 172.21.15.26:0/418546301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c043070 con 0x55b2b87b3920 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f3111ffb640 1 -- 172.21.15.26:0/418546301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c032d30 con 0x55b2b87b3920 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.086+0000 7f3111ffb640 1 -- 172.21.15.26:0/418546301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c03a070 con 0x55b2b87b3920 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 -- 172.21.15.26:0/418546301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b3920 msgr2=0x55b2b8786450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 --2- 172.21.15.26:0/418546301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b3920 0x55b2b8786450 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f311c02cc50 tx=0x7f311c049000 comp rx=0 tx=0).stop 2024-04-24T23:27:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 -- 172.21.15.26:0/418546301 shutdown_connections 2024-04-24T23:27:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 --2- 172.21.15.26:0/418546301 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b70b0 0x55b2b87d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 --2- 172.21.15.26:0/418546301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b46f0 0x55b2b87d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 --2- 172.21.15.26:0/418546301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b3920 0x55b2b8786450 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 -- 172.21.15.26:0/418546301 >> 172.21.15.26:0/418546301 conn(0x55b2b87b4310 msgr2=0x55b2b87b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 -- 172.21.15.26:0/418546301 shutdown_connections 2024-04-24T23:27:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 -- 172.21.15.26:0/418546301 wait complete. 2024-04-24T23:27:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 -- start start 2024-04-24T23:27:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.087+0000 7f313197f740 1 Processor -- start 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f313197f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 0x55b2b87b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f313197f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f313197f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b70b0 0x55b2b884f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f313197f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31281f4020 con 0x55b2b87b70b0 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f313197f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31281fbde0 con 0x55b2b87b46f0 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f313197f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55b2b87d1180 con 0x55b2b87b3920 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f313197f740 10 client.0 _ll_register_callbacks cb 0x7f3114009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f312d20f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b70b0 0x55b2b884f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f312d20f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b70b0 0x55b2b884f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56566/0 (socket says 172.21.15.26:56566) 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f312d20f640 1 -- 172.21.15.26:0/90559067 learned_addr learned my addr 172.21.15.26:0/90559067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f312ca0e640 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 0x55b2b87b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.088+0000 7f3127fff640 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f312d20f640 1 -- 172.21.15.26:0/90559067 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 msgr2=0x55b2b87b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f312d20f640 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 0x55b2b87b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f312d20f640 1 -- 172.21.15.26:0/90559067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b46f0 msgr2=0x55b2b87b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f312d20f640 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f312d20f640 1 -- 172.21.15.26:0/90559067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b87864f0 con 0x55b2b87b70b0 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f312d20f640 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b70b0 0x55b2b884f0f0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f31200014e0 tx=0x7f3120007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5d534a2a9b9df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f31077fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f31077fe640 1 -- 172.21.15.26:0/90559067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31200145e0 con 0x55b2b87b70b0 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f31077fe640 1 -- 172.21.15.26:0/90559067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f312000c780 con 0x55b2b87b70b0 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f313197f740 1 -- 172.21.15.26:0/90559067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f311c032d30 con 0x55b2b87b70b0 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f31077fe640 1 -- 172.21.15.26:0/90559067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f312000c980 con 0x55b2b87b70b0 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f313197f740 10 client.9132 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f313197f740 1 -- 172.21.15.26:0/90559067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55b2b87d0860 con 0x55b2b87b70b0 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f31077fe640 1 -- 172.21.15.26:0/90559067 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c032d30 con 0x55b2b87b70b0 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f31077fe640 1 -- 172.21.15.26:0/90559067 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f312000cb20 con 0x55b2b87b70b0 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f313197f740 10 client.9132 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f313197f740 10 client.9132 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.089+0000 7f313197f740 20 client.9132 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x55b2b87e9150 0x55b2b87d4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 4 client.9132 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 -- 172.21.15.26:0/90559067 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2b86f7f40 con 0x55b2b87e9150 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f312ca0e640 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x55b2b87e9150 0x55b2b87d4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f31077fe640 10 client.9132 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f312ca0e640 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x55b2b87e9150 0x55b2b87d4c80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93f13bf3356b522f server_cookie=921ca35be3e1ae03 in_seq=0 out_seq=0 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f31077fe640 1 -- 172.21.15.26:0/90559067 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55b2b86f7f40 con 0x55b2b87e9150 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f31077fe640 10 client.9132 handle_command_reply: tid=0 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 client.9132 shutdown 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 -- 172.21.15.26:0/90559067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b70b0 msgr2=0x55b2b884f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b70b0 0x55b2b884f0f0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f31200014e0 tx=0x7f3120007430 comp rx=0 tx=0).stop 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 -- 172.21.15.26:0/90559067 shutdown_connections 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x55b2b87e9150 0x55b2b87d4c80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b2b87b70b0 0x55b2b884f0f0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b2b87b46f0 0x55b2b87b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f31077fe640 0 client.9132 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 --2- 172.21.15.26:0/90559067 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55b2b87b3920 0x55b2b87b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 -- 172.21.15.26:0/90559067 >> 172.21.15.26:0/90559067 conn(0x55b2b87b4310 msgr2=0x55b2b87cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 -- 172.21.15.26:0/90559067 shutdown_connections 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.090+0000 7f313197f740 1 -- 172.21.15.26:0/90559067 wait complete. 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f313197f740 20 client.9132 trim_cache size 0 max 16384 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 -- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f310c06be60 msgr2=0x7f310c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f310c06be60 0x7f310c06e320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f311c037000 tx=0x7f311c030000 comp rx=0 tx=0).stop 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 -- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312815e120 msgr2=0x7f31280af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312815e120 0x7f31280af8a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f311400e870 tx=0x7f311400ed30 comp rx=0 tx=0).stop 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 -- 172.21.15.26:0/880819733 shutdown_connections 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f310c06be60 0x7f310c06e320 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f312815e500 0x7f312814b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312815e120 0x7f31280af8a0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 --2- 172.21.15.26:0/880819733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31280b3d20 0x7f312813c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 -- 172.21.15.26:0/880819733 >> 172.21.15.26:0/880819733 conn(0x7f312806b540 msgr2=0x7f31280adf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 -- 172.21.15.26:0/880819733 shutdown_connections 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.091+0000 7f312ec72640 1 -- 172.21.15.26:0/880819733 wait complete. 2024-04-24T23:27:53.102 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:27:53.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-04-24T23:27:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.205+0000 7f6655321640 1 Processor -- start 2024-04-24T23:27:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.205+0000 7f6655321640 1 -- start start 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f6655321640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6650158ae0 0x7f665006e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f6655321640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6650071720 0x7f6650071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f6655321640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665006b1c0 con 0x7f66500d0ce0 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f6655321640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6650072070 con 0x7f6650071720 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f6655321640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f665006ec90 con 0x7f6650158ae0 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f664f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6650071720 0x7f6650071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f664f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6650071720 0x7f6650071b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56574/0 (socket says 172.21.15.26:56574) 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f664f7fe640 1 -- 172.21.15.26:0/81584855 learned_addr learned my addr 172.21.15.26:0/81584855 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f664effd640 1 --2- 172.21.15.26:0/81584855 >> v2:172.21.15.29:3300/0 conn(0x7f6650158ae0 0x7f665006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f664f7fe640 1 -- 172.21.15.26:0/81584855 >> v2:172.21.15.29:3300/0 conn(0x7f6650158ae0 msgr2=0x7f665006e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f664f7fe640 1 --2- 172.21.15.26:0/81584855 >> v2:172.21.15.29:3300/0 conn(0x7f6650158ae0 0x7f665006e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f664f7fe640 1 -- 172.21.15.26:0/81584855 >> v1:172.21.15.29:6789/0 conn(0x7f66500d0ce0 legacy=0x7f665015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f664f7fe640 1 -- 172.21.15.26:0/81584855 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665006ef70 con 0x7f6650071720 2024-04-24T23:27:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.206+0000 7f664effd640 1 --2- 172.21.15.26:0/81584855 >> v2:172.21.15.29:3300/0 conn(0x7f6650158ae0 0x7f665006e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.207+0000 7f664f7fe640 1 --2- 172.21.15.26:0/81584855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650071720 0x7f6650071b30 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f66500cf530 tx=0x7f664002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418af93e8fb85165 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.207+0000 7f664dffb640 1 -- 172.21.15.26:0/81584855 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640036070 con 0x7f6650071720 2024-04-24T23:27:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.207+0000 7f664dffb640 1 -- 172.21.15.26:0/81584855 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6640031600 con 0x7f6650071720 2024-04-24T23:27:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.207+0000 7f664dffb640 1 -- 172.21.15.26:0/81584855 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66400319d0 con 0x7f6650071720 2024-04-24T23:27:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.208+0000 7f6655321640 1 -- 172.21.15.26:0/81584855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650071720 msgr2=0x7f6650071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.208+0000 7f6655321640 1 --2- 172.21.15.26:0/81584855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650071720 0x7f6650071b30 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f66500cf530 tx=0x7f664002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.208+0000 7f6655321640 1 -- 172.21.15.26:0/81584855 shutdown_connections 2024-04-24T23:27:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.208+0000 7f6655321640 1 --2- 172.21.15.26:0/81584855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650071720 0x7f6650071b30 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.208+0000 7f6655321640 1 --2- 172.21.15.26:0/81584855 >> v2:172.21.15.29:3300/0 conn(0x7f6650158ae0 0x7f665006e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.208+0000 7f6655321640 1 -- 172.21.15.26:0/81584855 >> 172.21.15.26:0/81584855 conn(0x7f66500b2970 msgr2=0x7f66500d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.208+0000 7f6655321640 1 -- 172.21.15.26:0/81584855 shutdown_connections 2024-04-24T23:27:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.208+0000 7f6655321640 1 -- 172.21.15.26:0/81584855 wait complete. 2024-04-24T23:27:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.209+0000 7f6655321640 1 Processor -- start 2024-04-24T23:27:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.209+0000 7f6655321640 1 -- start start 2024-04-24T23:27:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f6655321640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6650071720 0x7f66500aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f6655321640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66500d0ce0 0x7f6650146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f6655321640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650158ae0 0x7f66500b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f6655321640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f665006f530 con 0x7f6650158ae0 2024-04-24T23:27:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f6655321640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f665015f060 con 0x7f6650071720 2024-04-24T23:27:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f6655321640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f66500aff60 con 0x7f66500d0ce0 2024-04-24T23:27:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650158ae0 0x7f66500b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650158ae0 0x7f66500b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56584/0 (socket says 172.21.15.26:56584) 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664f7fe640 1 -- 172.21.15.26:0/150400215 learned_addr learned my addr 172.21.15.26:0/150400215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664e7fc640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66500d0ce0 0x7f6650146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664effd640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6650071720 0x7f66500aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664f7fe640 1 -- 172.21.15.26:0/150400215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66500d0ce0 msgr2=0x7f6650146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664f7fe640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66500d0ce0 0x7f6650146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664f7fe640 1 -- 172.21.15.26:0/150400215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6650071720 msgr2=0x7f66500aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664f7fe640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6650071720 0x7f66500aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.210+0000 7f664f7fe640 1 -- 172.21.15.26:0/150400215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66500add50 con 0x7f6650158ae0 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.211+0000 7f664f7fe640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650158ae0 0x7f66500b52b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6640009f40 tx=0x7f6640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.211+0000 7f66377fe640 1 -- 172.21.15.26:0/150400215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640036050 con 0x7f6650158ae0 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.211+0000 7f66377fe640 1 -- 172.21.15.26:0/150400215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6640042070 con 0x7f6650158ae0 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.211+0000 7f6655321640 1 -- 172.21.15.26:0/150400215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66500ae030 con 0x7f6650158ae0 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.211+0000 7f66377fe640 1 -- 172.21.15.26:0/150400215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640009510 con 0x7f6650158ae0 2024-04-24T23:27:53.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.211+0000 7f6655321640 1 -- 172.21.15.26:0/150400215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66500ae5f0 con 0x7f6650158ae0 2024-04-24T23:27:53.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.213+0000 7f66377fe640 1 -- 172.21.15.26:0/150400215 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6640007bd0 con 0x7f6650158ae0 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.213+0000 7f66377fe640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f663c06be30 0x7f663c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.213+0000 7f66377fe640 1 -- 172.21.15.26:0/150400215 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f66400abd20 con 0x7f6650158ae0 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.213+0000 7f664effd640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f663c06be30 0x7f663c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.214+0000 7f664effd640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f663c06be30 0x7f663c06e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6648006fd0 tx=0x7f6648008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.214+0000 7f665802a740 1 Processor -- start 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.214+0000 7f665802a740 1 -- start start 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f665802a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d557140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f665802a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d574b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f665802a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d52ca40 0x55627d572cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f665802a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55627d4906b0 con 0x55627d52ca40 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f665802a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55627d438880 con 0x55627d556d60 2024-04-24T23:27:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f665802a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55627d48c670 con 0x55627d574600 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d52ca40 0x55627d572cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d52ca40 0x55627d572cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56596/0 (socket says 172.21.15.26:56596) 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664f7fe640 1 -- 172.21.15.26:0/1560790642 learned_addr learned my addr 172.21.15.26:0/1560790642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664e7fc640 1 --2- 172.21.15.26:0/1560790642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d574b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664effd640 1 --2- 172.21.15.26:0/1560790642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d557140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664f7fe640 1 -- 172.21.15.26:0/1560790642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 msgr2=0x55627d574b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664f7fe640 1 --2- 172.21.15.26:0/1560790642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d574b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664f7fe640 1 -- 172.21.15.26:0/1560790642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 msgr2=0x55627d557140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664f7fe640 1 --2- 172.21.15.26:0/1560790642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d557140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664f7fe640 1 -- 172.21.15.26:0/1560790642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55627d26e7d0 con 0x55627d52ca40 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.215+0000 7f664f7fe640 1 --2- 172.21.15.26:0/1560790642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d52ca40 0x55627d572cc0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f6640082680 tx=0x7f6640079930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb00fbbf8f20dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f6634ff9640 1 -- 172.21.15.26:0/1560790642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664007a6d0 con 0x55627d52ca40 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f6634ff9640 1 -- 172.21.15.26:0/1560790642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6640092020 con 0x55627d52ca40 2024-04-24T23:27:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f6634ff9640 1 -- 172.21.15.26:0/1560790642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640079fd0 con 0x55627d52ca40 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f665802a740 1 -- 172.21.15.26:0/1560790642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d52ca40 msgr2=0x55627d572cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f665802a740 1 --2- 172.21.15.26:0/1560790642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d52ca40 0x55627d572cc0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f6640082680 tx=0x7f6640079930 comp rx=0 tx=0).stop 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f665802a740 1 -- 172.21.15.26:0/1560790642 shutdown_connections 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f665802a740 1 --2- 172.21.15.26:0/1560790642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d52ca40 0x55627d572cc0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f665802a740 1 --2- 172.21.15.26:0/1560790642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d574b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f665802a740 1 --2- 172.21.15.26:0/1560790642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d557140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f665802a740 1 -- 172.21.15.26:0/1560790642 >> 172.21.15.26:0/1560790642 conn(0x55627d529600 msgr2=0x55627d571b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f665802a740 1 -- 172.21.15.26:0/1560790642 shutdown_connections 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.216+0000 7f665802a740 1 -- 172.21.15.26:0/1560790642 wait complete. 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.217+0000 7f665802a740 1 -- start start 2024-04-24T23:27:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.217+0000 7f665802a740 1 Processor -- start 2024-04-24T23:27:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f665802a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d545570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f665802a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d574600 0x55627d55a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f665802a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d55a600 0x55627d526260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f665802a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55627d4906b0 con 0x55627d574600 2024-04-24T23:27:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f665802a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55627d44bf50 con 0x55627d556d60 2024-04-24T23:27:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f665802a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55627d52d7e0 con 0x55627d55a600 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f665802a740 10 client.0 _ll_register_callbacks cb 0x55627d487000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d574600 0x55627d55a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d574600 0x55627d55a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56606/0 (socket says 172.21.15.26:56606) 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664e7fc640 1 -- 172.21.15.26:0/2944950723 learned_addr learned my addr 172.21.15.26:0/2944950723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664effd640 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d545570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664f7fe640 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d55a600 0x55627d526260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664e7fc640 1 -- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d55a600 msgr2=0x55627d526260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664e7fc640 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d55a600 0x55627d526260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664e7fc640 1 -- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 msgr2=0x55627d545570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664e7fc640 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d545570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664e7fc640 1 -- 172.21.15.26:0/2944950723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55627d1a43f0 con 0x55627d574600 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f664e7fc640 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d574600 0x55627d55a0c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f6644001e80 tx=0x7f66440116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71a2cc9127f3a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f662dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.218+0000 7f662dffb640 1 -- 172.21.15.26:0/2944950723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644005320 con 0x55627d574600 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f662dffb640 1 -- 172.21.15.26:0/2944950723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f6644010040 con 0x55627d574600 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f665802a740 1 -- 172.21.15.26:0/2944950723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6640092020 con 0x55627d574600 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f662dffb640 1 -- 172.21.15.26:0/2944950723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f6644007030 con 0x55627d574600 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f665802a740 10 client.9144 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f665802a740 1 -- 172.21.15.26:0/2944950723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55627d1640f0 con 0x55627d574600 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f662dffb640 1 -- 172.21.15.26:0/2944950723 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6640092020 con 0x55627d574600 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f662dffb640 1 -- 172.21.15.26:0/2944950723 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f66440071d0 con 0x55627d574600 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f665802a740 10 client.9144 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f665802a740 10 client.9144 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f665802a740 20 client.9144 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f665802a740 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55627d555570 0x55627d555990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f665802a740 4 client.9144 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f665802a740 1 -- 172.21.15.26:0/2944950723 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55627d49dbf0 con 0x55627d555570 2024-04-24T23:27:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.219+0000 7f664effd640 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55627d555570 0x55627d555990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.220+0000 7f664effd640 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55627d555570 0x55627d555990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=208edd3d848310f9 server_cookie=f98dea5e41f7ecef in_seq=0 out_seq=0 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.220+0000 7f662dffb640 10 client.9144 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.220+0000 7f662dffb640 1 -- 172.21.15.26:0/2944950723 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55627d49dbf0 con 0x55627d555570 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.220+0000 7f662dffb640 10 client.9144 handle_command_reply: tid=0 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.220+0000 7f665802a740 1 client.9144 shutdown 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.220+0000 7f665802a740 1 -- 172.21.15.26:0/2944950723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d574600 msgr2=0x55627d55a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d574600 0x55627d55a0c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f6644001e80 tx=0x7f66440116e0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 1 -- 172.21.15.26:0/2944950723 shutdown_connections 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55627d555570 0x55627d555990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d55a600 0x55627d526260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d574600 0x55627d55a0c0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 1 --2- 172.21.15.26:0/2944950723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d545570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 1 -- 172.21.15.26:0/2944950723 >> 172.21.15.26:0/2944950723 conn(0x55627d52ca40 msgr2=0x55627d5750f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f662dffb640 0 client.9144 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:53.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 1 -- 172.21.15.26:0/2944950723 shutdown_connections 2024-04-24T23:27:53.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 1 -- 172.21.15.26:0/2944950723 wait complete. 2024-04-24T23:27:53.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.221+0000 7f665802a740 20 client.9144 trim_cache size 0 max 16384 2024-04-24T23:27:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.239+0000 7f665802a740 1 Processor -- start 2024-04-24T23:27:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.239+0000 7f665802a740 1 -- start start 2024-04-24T23:27:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.239+0000 7f665802a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d556d60 0x55627d556790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.239+0000 7f665802a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d55a600 0x55627d52ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.239+0000 7f665802a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d52d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.239+0000 7f665802a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f66502008f0 con 0x55627d556d60 2024-04-24T23:27:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.239+0000 7f665802a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66501ffd10 con 0x55627d55a600 2024-04-24T23:27:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.239+0000 7f665802a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f66502001c0 con 0x55627d574600 2024-04-24T23:27:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d556d60 0x55627d556790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d556d60 0x55627d556790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56620/0 (socket says 172.21.15.26:56620) 2024-04-24T23:27:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664effd640 1 -- 172.21.15.26:0/1742094356 learned_addr learned my addr 172.21.15.26:0/1742094356 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664effd640 1 -- 172.21.15.26:0/1742094356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 msgr2=0x55627d52d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664f7fe640 1 --2- 172.21.15.26:0/1742094356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d52d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664effd640 1 --2- 172.21.15.26:0/1742094356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d52d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664e7fc640 1 --2- 172.21.15.26:0/1742094356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d55a600 0x55627d52ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664effd640 1 -- 172.21.15.26:0/1742094356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d55a600 msgr2=0x55627d52ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664effd640 1 --2- 172.21.15.26:0/1742094356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d55a600 0x55627d52ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664effd640 1 -- 172.21.15.26:0/1742094356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55627d553890 con 0x55627d556d60 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f664effd640 1 --2- 172.21.15.26:0/1742094356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d556d60 0x55627d556790 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f664800f9c0 tx=0x7f6648015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49c108af197a5d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f662ffff640 1 -- 172.21.15.26:0/1742094356 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664801b040 con 0x55627d556d60 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f662ffff640 1 -- 172.21.15.26:0/1742094356 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664801c040 con 0x55627d556d60 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.240+0000 7f662ffff640 1 -- 172.21.15.26:0/1742094356 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648014b60 con 0x55627d556d60 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 -- 172.21.15.26:0/1742094356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d556d60 msgr2=0x55627d556790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 --2- 172.21.15.26:0/1742094356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d556d60 0x55627d556790 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f664800f9c0 tx=0x7f6648015000 comp rx=0 tx=0).stop 2024-04-24T23:27:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 -- 172.21.15.26:0/1742094356 shutdown_connections 2024-04-24T23:27:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 --2- 172.21.15.26:0/1742094356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d52d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 --2- 172.21.15.26:0/1742094356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d55a600 0x55627d52ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 --2- 172.21.15.26:0/1742094356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d556d60 0x55627d556790 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 -- 172.21.15.26:0/1742094356 >> 172.21.15.26:0/1742094356 conn(0x55627d52ca40 msgr2=0x55627d545380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 -- 172.21.15.26:0/1742094356 shutdown_connections 2024-04-24T23:27:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 -- 172.21.15.26:0/1742094356 wait complete. 2024-04-24T23:27:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 -- start start 2024-04-24T23:27:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.241+0000 7f665802a740 1 Processor -- start 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f665802a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d582f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f665802a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d55a600 0x55627d52ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f665802a740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d52d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f665802a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f66500b3f70 con 0x55627d55a600 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f665802a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55627d52a810 con 0x55627d556d60 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f665802a740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55627d567620 con 0x55627d574600 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f665802a740 10 client.0 _ll_register_callbacks cb 0x7f6644007af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d55a600 0x55627d52ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d55a600 0x55627d52ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56622/0 (socket says 172.21.15.26:56622) 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664e7fc640 1 -- 172.21.15.26:0/3395288492 learned_addr learned my addr 172.21.15.26:0/3395288492 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664f7fe640 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d52d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664effd640 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d582f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664e7fc640 1 -- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 msgr2=0x55627d52d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664e7fc640 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d52d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664e7fc640 1 -- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 msgr2=0x55627d582f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664e7fc640 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d582f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.242+0000 7f664e7fc640 1 -- 172.21.15.26:0/3395288492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55627d557f50 con 0x55627d55a600 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f664e7fc640 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d55a600 0x55627d52ce80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f6644007ba0 tx=0x7f6644000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da87ab99180a9785 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f662dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f662dffb640 1 -- 172.21.15.26:0/3395288492 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644011be0 con 0x55627d55a600 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f662dffb640 1 -- 172.21.15.26:0/3395288492 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f6644010040 con 0x55627d55a600 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f665802a740 1 -- 172.21.15.26:0/3395288492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f664801c040 con 0x55627d55a600 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f662dffb640 1 -- 172.21.15.26:0/3395288492 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f66440069e0 con 0x55627d55a600 2024-04-24T23:27:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f665802a740 10 client.9150 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f665802a740 1 -- 172.21.15.26:0/3395288492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55627d55a320 con 0x55627d55a600 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f662dffb640 1 -- 172.21.15.26:0/3395288492 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664801c040 con 0x55627d55a600 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f662dffb640 1 -- 172.21.15.26:0/3395288492 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f6644031050 con 0x55627d55a600 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f665802a740 10 client.9150 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f665802a740 10 client.9150 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f665802a740 20 client.9150 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f665802a740 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55627d5eb7b0 0x55627d5ebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f665802a740 4 client.9150 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f665802a740 1 -- 172.21.15.26:0/3395288492 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55627d497f40 con 0x55627d5eb7b0 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.243+0000 7f664effd640 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55627d5eb7b0 0x55627d5ebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f662dffb640 10 client.9150 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f664effd640 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55627d5eb7b0 0x55627d5ebbc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3795f8cdcfe3f94 server_cookie=acd3686e238f3843 in_seq=0 out_seq=0 2024-04-24T23:27:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f662dffb640 1 -- 172.21.15.26:0/3395288492 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55627d497f40 con 0x55627d5eb7b0 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f662dffb640 10 client.9150 handle_command_reply: tid=0 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 client.9150 shutdown 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 -- 172.21.15.26:0/3395288492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d55a600 msgr2=0x55627d52ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d55a600 0x55627d52ce80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f6644007ba0 tx=0x7f6644000ed0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 -- 172.21.15.26:0/3395288492 shutdown_connections 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55627d5eb7b0 0x55627d5ebbc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55627d574600 0x55627d52d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55627d55a600 0x55627d52ce80 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 --2- 172.21.15.26:0/3395288492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55627d556d60 0x55627d582f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f662dffb640 0 client.9150 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 -- 172.21.15.26:0/3395288492 >> 172.21.15.26:0/3395288492 conn(0x55627d52ca40 msgr2=0x55627d52c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 -- 172.21.15.26:0/3395288492 shutdown_connections 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 1 -- 172.21.15.26:0/3395288492 wait complete. 2024-04-24T23:27:53.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.244+0000 7f665802a740 20 client.9150 trim_cache size 0 max 16384 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:53.252 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:53.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 -- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f663c06be30 msgr2=0x7f663c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f663c06be30 0x7f663c06e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6648006fd0 tx=0x7f6648008040 comp rx=0 tx=0).stop 2024-04-24T23:27:53.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 -- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650158ae0 msgr2=0x7f66500b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650158ae0 0x7f66500b52b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6640009f40 tx=0x7f6640004a40 comp rx=0 tx=0).stop 2024-04-24T23:27:53.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 -- 172.21.15.26:0/150400215 shutdown_connections 2024-04-24T23:27:53.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f663c06be30 0x7f663c06e2f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6650158ae0 0x7f66500b52b0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f66500d0ce0 0x7f6650146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 --2- 172.21.15.26:0/150400215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6650071720 0x7f66500aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 -- 172.21.15.26:0/150400215 >> 172.21.15.26:0/150400215 conn(0x7f66500b2970 msgr2=0x7f665015dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 -- 172.21.15.26:0/150400215 shutdown_connections 2024-04-24T23:27:53.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.245+0000 7f6655321640 1 -- 172.21.15.26:0/150400215 wait complete. 2024-04-24T23:27:53.256 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:27:53.256 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:27:53.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-24T23:27:53.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.358+0000 7f6fa219d640 1 Processor -- start 2024-04-24T23:27:53.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.358+0000 7f6fa219d640 1 -- start start 2024-04-24T23:27:53.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.359+0000 7f6fa219d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.359+0000 7f6fa219d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f9c0d4c10 0x7f6f9c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.359+0000 7f6fa219d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f9c0d2120 con 0x7f6f9c15e750 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.359+0000 7f6fa219d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f6f9c0d1510 con 0x7f6f9c0d4c10 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.359+0000 7f6fa219d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9c0d1ed0 con 0x7f6f9c15e370 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.360+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.360+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:49082/0 (socket says 172.21.15.26:49082) 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.360+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/2395183974 learned_addr learned my addr 172.21.15.26:0/2395183974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.360+0000 7f6f9bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f6f9c0d4c10 0x7f6f9c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.360+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/2395183974 >> v2:172.21.15.29:3300/0 conn(0x7f6f9c0d4c10 msgr2=0x7f6f9c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.360+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/2395183974 >> v2:172.21.15.29:3300/0 conn(0x7f6f9c0d4c10 0x7f6f9c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.360+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/2395183974 >> v1:172.21.15.29:6789/0 conn(0x7f6f9c15e750 legacy=0x7f6f9c0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.360+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/2395183974 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c0b3740 con 0x7f6f9c15e370 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/2395183974 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0d2520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009870 tx=0x7f6f8c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab2aeaab708054d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6f9a7fc640 1 -- 172.21.15.26:0/2395183974 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c037070 con 0x7f6f9c15e370 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6f9a7fc640 1 -- 172.21.15.26:0/2395183974 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f8c032440 con 0x7f6f9c15e370 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6f9a7fc640 1 -- 172.21.15.26:0/2395183974 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c032810 con 0x7f6f9c15e370 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6fa219d640 1 -- 172.21.15.26:0/2395183974 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 msgr2=0x7f6f9c0d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6fa219d640 1 --2- 172.21.15.26:0/2395183974 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0d2520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009870 tx=0x7f6f8c02a270 comp rx=0 tx=0).stop 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6fa219d640 1 -- 172.21.15.26:0/2395183974 shutdown_connections 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6fa219d640 1 --2- 172.21.15.26:0/2395183974 >> v2:172.21.15.29:3300/0 conn(0x7f6f9c0d4c10 0x7f6f9c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6fa219d640 1 --2- 172.21.15.26:0/2395183974 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0d2520 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.361+0000 7f6fa219d640 1 -- 172.21.15.26:0/2395183974 >> 172.21.15.26:0/2395183974 conn(0x7f6f9c0d40f0 msgr2=0x7f6f9c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.362+0000 7f6fa219d640 1 -- 172.21.15.26:0/2395183974 shutdown_connections 2024-04-24T23:27:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.362+0000 7f6fa219d640 1 -- 172.21.15.26:0/2395183974 wait complete. 2024-04-24T23:27:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.363+0000 7f6fa219d640 1 Processor -- start 2024-04-24T23:27:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.363+0000 7f6fa219d640 1 -- start start 2024-04-24T23:27:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.363+0000 7f6fa219d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0d4c10 0x7f6f9c0ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6fa219d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6fa219d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f9c150420 0x7f6f9c0b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6fa219d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c0d2a60 con 0x7f6f9c150420 2024-04-24T23:27:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6fa219d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c0b3d00 con 0x7f6f9c0d4c10 2024-04-24T23:27:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6fa219d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9c0d2be0 con 0x7f6f9c15e370 2024-04-24T23:27:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f9c150420 0x7f6f9c0b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f9c150420 0x7f6f9c0b89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56626/0 (socket says 172.21.15.26:56626) 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9bfff640 1 -- 172.21.15.26:0/3891617422 learned_addr learned my addr 172.21.15.26:0/3891617422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9affd640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0d4c10 0x7f6f9c0ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9bfff640 1 -- 172.21.15.26:0/3891617422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 msgr2=0x7f6f9c0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9bfff640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9bfff640 1 -- 172.21.15.26:0/3891617422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0d4c10 msgr2=0x7f6f9c0ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9bfff640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0d4c10 0x7f6f9c0ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9bfff640 1 -- 172.21.15.26:0/3891617422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c0caaa0 con 0x7f6f9c150420 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f9bfff640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f9c150420 0x7f6f9c0b89d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400efc0 tx=0x7f6f9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f83fff640 1 -- 172.21.15.26:0/3891617422 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94018020 con 0x7f6f9c150420 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f83fff640 1 -- 172.21.15.26:0/3891617422 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f94014480 con 0x7f6f9c150420 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6fa219d640 1 -- 172.21.15.26:0/3891617422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9c0c8880 con 0x7f6f9c150420 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6f83fff640 1 -- 172.21.15.26:0/3891617422 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94014870 con 0x7f6f9c150420 2024-04-24T23:27:53.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.364+0000 7f6fa219d640 1 -- 172.21.15.26:0/3891617422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9c0c8e40 con 0x7f6f9c150420 2024-04-24T23:27:53.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.366+0000 7f6f83fff640 1 -- 172.21.15.26:0/3891617422 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f94005d10 con 0x7f6f9c150420 2024-04-24T23:27:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.366+0000 7f6f83fff640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f8806be30 0x7f6f8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.367+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f8806be30 0x7f6f8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.367+0000 7f6f83fff640 1 -- 172.21.15.26:0/3891617422 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f6f9408b860 con 0x7f6f9c150420 2024-04-24T23:27:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.367+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f8806be30 0x7f6f8806e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c034000 tx=0x7f6f8c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.368+0000 7f6fa4f81740 1 Processor -- start 2024-04-24T23:27:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.368+0000 7f6fa4f81740 1 -- start start 2024-04-24T23:27:53.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.368+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc8b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.368+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 0x55ed7cca8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.368+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc60a40 0x55ed7cca6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.368+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ed7cbc46b0 con 0x55ed7cc60a40 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.368+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ed7cb6c880 con 0x55ed7cc8ad60 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.368+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ed7cbc0670 con 0x55ed7cca8600 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc60a40 0x55ed7cca6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc60a40 0x55ed7cca6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56642/0 (socket says 172.21.15.26:56642) 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9bfff640 1 -- 172.21.15.26:0/2846394295 learned_addr learned my addr 172.21.15.26:0/2846394295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9affd640 1 --2- 172.21.15.26:0/2846394295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 0x55ed7cca8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/2846394295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc8b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9bfff640 1 -- 172.21.15.26:0/2846394295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 msgr2=0x55ed7cca8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9bfff640 1 --2- 172.21.15.26:0/2846394295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 0x55ed7cca8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9bfff640 1 -- 172.21.15.26:0/2846394295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc8ad60 msgr2=0x55ed7cc8b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9bfff640 1 --2- 172.21.15.26:0/2846394295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc8b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9bfff640 1 -- 172.21.15.26:0/2846394295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7c9a27d0 con 0x55ed7cc60a40 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f9bfff640 1 --2- 172.21.15.26:0/2846394295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc60a40 0x55ed7cca6cc0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6f940621a0 tx=0x7f6f9405dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4bbeaace77002f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f817fa640 1 -- 172.21.15.26:0/2846394295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94071020 con 0x55ed7cc60a40 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f817fa640 1 -- 172.21.15.26:0/2846394295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f94056210 con 0x55ed7cc60a40 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.369+0000 7f6f817fa640 1 -- 172.21.15.26:0/2846394295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94062480 con 0x55ed7cc60a40 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2846394295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc60a40 msgr2=0x55ed7cca6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2846394295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc60a40 0x55ed7cca6cc0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6f940621a0 tx=0x7f6f9405dea0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2846394295 shutdown_connections 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2846394295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc60a40 0x55ed7cca6cc0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2846394295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 0x55ed7cca8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2846394295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc8b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2846394295 >> 172.21.15.26:0/2846394295 conn(0x55ed7cc5d600 msgr2=0x55ed7cca5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2846394295 shutdown_connections 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2846394295 wait complete. 2024-04-24T23:27:53.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.370+0000 7f6fa4f81740 1 -- start start 2024-04-24T23:27:53.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6fa4f81740 1 Processor -- start 2024-04-24T23:27:53.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 0x55ed7cc5a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc8e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ed7cc617e0 con 0x55ed7cc8ad60 2024-04-24T23:27:53.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ed7cb7ff50 con 0x55ed7cc9cc20 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ed7cc5b530 con 0x55ed7cca8600 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6fa4f81740 10 client.0 _ll_register_callbacks cb 0x55ed7cbbb000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56650/0 (socket says 172.21.15.26:56650) 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.371+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/1742739328 learned_addr learned my addr 172.21.15.26:0/1742739328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f9affd640 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 0x55ed7cc5a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f9bfff640 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc8e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 msgr2=0x55ed7cc5a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 0x55ed7cc5a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc9cc20 msgr2=0x55ed7cc8e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc8e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/1742739328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7c8d83f0 con 0x55ed7cc8ad60 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9ef30 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c000c00 tx=0x7f6f8c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf62f49455fb8d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/1742739328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c037040 con 0x55ed7cc8ad60 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/1742739328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f6f8c04a670 con 0x55ed7cc8ad60 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1742739328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f94056210 con 0x55ed7cc8ad60 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/1742739328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f6f8c04a870 con 0x55ed7cc8ad60 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6fa4f81740 10 client.9159 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1742739328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55ed7c8980f0 con 0x55ed7cc8ad60 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.372+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/1742739328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f94056210 con 0x55ed7cc8ad60 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/1742739328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f6f8c04aa10 con 0x55ed7cc8ad60 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6fa4f81740 10 client.9159 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6fa4f81740 10 client.9159 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6fa4f81740 20 client.9159 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55ed7cca6fc0 0x55ed7cca73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6fa4f81740 4 client.9159 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1742739328 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed7cbd1bf0 con 0x55ed7cca6fc0 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6f9affd640 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55ed7cca6fc0 0x55ed7cca73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6f9affd640 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55ed7cca6fc0 0x55ed7cca73e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32deb8242c9c5de2 server_cookie=47a2e42c8fda0a71 in_seq=0 out_seq=0 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.373+0000 7f6f7a7fc640 10 client.9159 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/1742739328 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed7cbd1bf0 con 0x55ed7cca6fc0 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6f7a7fc640 10 client.9159 handle_command_reply: tid=0 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 client.9159 shutdown 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1742739328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 msgr2=0x55ed7cc9ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9ef30 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c000c00 tx=0x7f6f8c02c070 comp rx=0 tx=0).stop 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1742739328 shutdown_connections 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55ed7cca6fc0 0x55ed7cca73e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc8e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cca8600 0x55ed7cc5a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6f7a7fc640 0 client.9159 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1742739328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9ef30 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1742739328 >> 172.21.15.26:0/1742739328 conn(0x55ed7cc60a40 msgr2=0x55ed7cca90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1742739328 shutdown_connections 2024-04-24T23:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1742739328 wait complete. 2024-04-24T23:27:53.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.374+0000 7f6fa4f81740 20 client.9159 trim_cache size 0 max 16384 2024-04-24T23:27:53.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.392+0000 7f6fa4f81740 1 Processor -- start 2024-04-24T23:27:53.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.392+0000 7f6fa4f81740 1 -- start start 2024-04-24T23:27:53.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cc8ad60 0x55ed7cc86c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc89d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cca8600 0x55ed7cc8a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c200850 con 0x55ed7cc9cc20 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c1ffa20 con 0x55ed7cca8600 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9c200120 con 0x55ed7cc8ad60 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6f9affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc89d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6f9affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc89d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56666/0 (socket says 172.21.15.26:56666) 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6f9affd640 1 -- 172.21.15.26:0/1315390651 learned_addr learned my addr 172.21.15.26:0/1315390651 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6f9affd640 1 -- 172.21.15.26:0/1315390651 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cc8ad60 msgr2=0x55ed7cc86c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6f9affd640 1 --2- 172.21.15.26:0/1315390651 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cc8ad60 0x55ed7cc86c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6f9affd640 1 -- 172.21.15.26:0/1315390651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cca8600 msgr2=0x55ed7cc8a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6f9affd640 1 --2- 172.21.15.26:0/1315390651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cca8600 0x55ed7cc8a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6f9affd640 1 -- 172.21.15.26:0/1315390651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7cca9ff0 con 0x55ed7cc9cc20 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.393+0000 7f6f9affd640 1 --2- 172.21.15.26:0/1315390651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc89d50 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x55ed7cca79e0 tx=0x7f6f90007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d715abd64335dc9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6f80ff9640 1 -- 172.21.15.26:0/1315390651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90020030 con 0x55ed7cc9cc20 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6f80ff9640 1 -- 172.21.15.26:0/1315390651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f900074d0 con 0x55ed7cc9cc20 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6f80ff9640 1 -- 172.21.15.26:0/1315390651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f900076a0 con 0x55ed7cc9cc20 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1315390651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc9cc20 msgr2=0x55ed7cc89d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1315390651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc89d50 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x55ed7cca79e0 tx=0x7f6f90007e50 comp rx=0 tx=0).stop 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1315390651 shutdown_connections 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1315390651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cca8600 0x55ed7cc8a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1315390651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc9cc20 0x55ed7cc89d50 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/1315390651 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cc8ad60 0x55ed7cc86c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1315390651 >> 172.21.15.26:0/1315390651 conn(0x55ed7cc60a40 msgr2=0x55ed7cc5a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1315390651 shutdown_connections 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.394+0000 7f6fa4f81740 1 -- 172.21.15.26:0/1315390651 wait complete. 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.395+0000 7f6fa4f81740 1 -- start start 2024-04-24T23:27:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.395+0000 7f6fa4f81740 1 Processor -- start 2024-04-24T23:27:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cc9cc20 0x55ed7cc89d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6fa4f81740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cca8600 0x55ed7cc8a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ed7cb4a490 con 0x55ed7cc8ad60 2024-04-24T23:27:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c0b79c0 con 0x55ed7cca8600 2024-04-24T23:27:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6fa4f81740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55ed7ccbfc50 con 0x55ed7cc9cc20 2024-04-24T23:27:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6fa4f81740 10 client.0 _ll_register_callbacks cb 0x7f6f940095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56672/0 (socket says 172.21.15.26:56672) 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/2994394384 learned_addr learned my addr 172.21.15.26:0/2994394384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cc9cc20 msgr2=0x55ed7cc89d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cc9cc20 0x55ed7cc89d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cca8600 msgr2=0x55ed7cc8a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cca8600 0x55ed7cc8a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f9b7fe640 1 -- 172.21.15.26:0/2994394384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7ccb45a0 con 0x55ed7cc8ad60 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f9b7fe640 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9eea0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c035040 tx=0x7f6f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acca67b00593c74e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/2994394384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c040030 con 0x55ed7cc8ad60 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/2994394384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f6f8c03c850 con 0x55ed7cc8ad60 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.396+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2994394384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f900074d0 con 0x55ed7cc8ad60 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/2994394384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f6f8c037040 con 0x55ed7cc8ad60 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6fa4f81740 10 client.9165 fetch_fsmap learned FSMap version 110 2024-04-24T23:27:53.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2994394384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55ed7cd061f0 con 0x55ed7cc8ad60 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/2994394384 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f900074d0 con 0x55ed7cc8ad60 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/2994394384 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 110) ==== 3546+0+0 (secure 0 0 0) 0x7f6f8c03ca50 con 0x55ed7cc8ad60 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6fa4f81740 10 client.9165 fetch_fsmap finished waiting for FSMap version 110 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6fa4f81740 10 client.9165 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6fa4f81740 20 client.9165 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55ed7cc9f3e0 0x55ed7cc5eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6fa4f81740 4 client.9165 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2994394384 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed7cbcbf40 con 0x55ed7cc9f3e0 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.397+0000 7f6f9affd640 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55ed7cc9f3e0 0x55ed7cc5eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6f9affd640 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55ed7cc9f3e0 0x55ed7cc5eca0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad438b5da5d9e071 server_cookie=2dc1202b0d49ffa9 in_seq=0 out_seq=0 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6f7a7fc640 10 client.9165 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6f7a7fc640 1 -- 172.21.15.26:0/2994394384 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55ed7cbcbf40 con 0x55ed7cc9f3e0 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6f7a7fc640 10 client.9165 handle_command_reply: tid=0 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 client.9165 shutdown 2024-04-24T23:27:53.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2994394384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 msgr2=0x55ed7cc9eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9eea0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c035040 tx=0x7f6f8c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2994394384 shutdown_connections 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55ed7cc9f3e0 0x55ed7cc5eca0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ed7cca8600 0x55ed7cc8a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55ed7cc9cc20 0x55ed7cc89d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 --2- 172.21.15.26:0/2994394384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed7cc8ad60 0x55ed7cc9eea0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6f7a7fc640 0 client.9165 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2994394384 >> 172.21.15.26:0/2994394384 conn(0x55ed7cc60a40 msgr2=0x55ed7cc60e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2994394384 shutdown_connections 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 1 -- 172.21.15.26:0/2994394384 wait complete. 2024-04-24T23:27:53.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa4f81740 20 client.9165 trim_cache size 0 max 16384 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:53.405 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:53.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa219d640 1 -- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f8806be30 msgr2=0x7f6f8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa219d640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f8806be30 0x7f6f8806e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c034000 tx=0x7f6f8c02e000 comp rx=0 tx=0).stop 2024-04-24T23:27:53.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa219d640 1 -- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f9c150420 msgr2=0x7f6f9c0b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.398+0000 7f6fa219d640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f9c150420 0x7f6f9c0b89d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400efc0 tx=0x7f6f9400c490 comp rx=0 tx=0).stop 2024-04-24T23:27:53.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.399+0000 7f6fa219d640 1 -- 172.21.15.26:0/3891617422 shutdown_connections 2024-04-24T23:27:53.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.399+0000 7f6fa219d640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f6f8806be30 0x7f6f8806e2f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.399+0000 7f6fa219d640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f9c150420 0x7f6f9c0b89d0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.399+0000 7f6fa219d640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f6f9c15e370 0x7f6f9c0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.399+0000 7f6fa219d640 1 --2- 172.21.15.26:0/3891617422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6f9c0d4c10 0x7f6f9c0ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.399+0000 7f6fa219d640 1 -- 172.21.15.26:0/3891617422 >> 172.21.15.26:0/3891617422 conn(0x7f6f9c0d40f0 msgr2=0x7f6f9c0cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.399+0000 7f6fa219d640 1 -- 172.21.15.26:0/3891617422 shutdown_connections 2024-04-24T23:27:53.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.399+0000 7f6fa219d640 1 -- 172.21.15.26:0/3891617422 wait complete. 2024-04-24T23:27:53.410 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:27:53.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:53.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.510+0000 7fb0d79e7640 1 Processor -- start 2024-04-24T23:27:53.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.510+0000 7fb0d79e7640 1 -- start start 2024-04-24T23:27:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d79e7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0d0143c40 0x7fb0d00cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d79e7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 0x7fb0d0072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d79e7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d00cd0b0 con 0x7fb0d015eb00 2024-04-24T23:27:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d79e7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d0071b70 con 0x7fb0d0143c40 2024-04-24T23:27:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d79e7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d0071240 con 0x7fb0d00cc860 2024-04-24T23:27:53.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d5783640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0d0143c40 0x7fb0d00cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d5f84640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb0d015eb00 0x7fb0d00b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57970/0 (socket says 172.21.15.26:57970) 2024-04-24T23:27:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d5783640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0d0143c40 0x7fb0d00cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56684/0 (socket says 172.21.15.26:56684) 2024-04-24T23:27:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d5f84640 1 -- 172.21.15.26:0/1344023427 learned_addr learned my addr 172.21.15.26:0/1344023427 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d4f82640 1 --2- 172.21.15.26:0/1344023427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 0x7fb0d0072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d5783640 1 -- 172.21.15.26:0/1344023427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 msgr2=0x7fb0d0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d5783640 1 --2- 172.21.15.26:0/1344023427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 0x7fb0d0072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d5783640 1 -- 172.21.15.26:0/1344023427 >> v1:172.21.15.26:6789/0 conn(0x7fb0d015eb00 legacy=0x7fb0d00b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.511+0000 7fb0d5783640 1 -- 172.21.15.26:0/1344023427 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d0071520 con 0x7fb0d0143c40 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.512+0000 7fb0d5783640 1 --2- 172.21.15.26:0/1344023427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 0x7fb0d00cf870 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80097a0 tx=0x7fb0c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c2dfc44cd6954d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.512+0000 7fb0cffff640 1 -- 172.21.15.26:0/1344023427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8037070 con 0x7fb0d0143c40 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.512+0000 7fb0cffff640 1 -- 172.21.15.26:0/1344023427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c8032440 con 0x7fb0d0143c40 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.512+0000 7fb0cffff640 1 -- 172.21.15.26:0/1344023427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8032810 con 0x7fb0d0143c40 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.513+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1344023427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 msgr2=0x7fb0d00cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.513+0000 7fb0d79e7640 1 --2- 172.21.15.26:0/1344023427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 0x7fb0d00cf870 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80097a0 tx=0x7fb0c802a180 comp rx=0 tx=0).stop 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.513+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1344023427 shutdown_connections 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.513+0000 7fb0d79e7640 1 --2- 172.21.15.26:0/1344023427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 0x7fb0d0072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.513+0000 7fb0d79e7640 1 --2- 172.21.15.26:0/1344023427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 0x7fb0d00cf870 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.513+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1344023427 >> 172.21.15.26:0/1344023427 conn(0x7fb0d006b300 msgr2=0x7fb0d006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.513+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1344023427 shutdown_connections 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.513+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1344023427 wait complete. 2024-04-24T23:27:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.514+0000 7fb0d79e7640 1 Processor -- start 2024-04-24T23:27:53.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.514+0000 7fb0d79e7640 1 -- start start 2024-04-24T23:27:53.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d79e7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 0x7fb0d013cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d79e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 0x7fb0d006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d79e7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0d015eb00 0x7fb0d01392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d79e7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0072540 con 0x7fb0d0143c40 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d79e7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0144020 con 0x7fb0d015eb00 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d79e7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d013d330 con 0x7fb0d00cc860 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d4f82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 0x7fb0d006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d4f82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 0x7fb0d006de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56690/0 (socket says 172.21.15.26:56690) 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d4f82640 1 -- 172.21.15.26:0/1708451711 learned_addr learned my addr 172.21.15.26:0/1708451711 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d5f84640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0d015eb00 0x7fb0d01392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d5783640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 0x7fb0d013cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d4f82640 1 -- 172.21.15.26:0/1708451711 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 msgr2=0x7fb0d013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d4f82640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 0x7fb0d013cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d4f82640 1 -- 172.21.15.26:0/1708451711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0d015eb00 msgr2=0x7fb0d01392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d4f82640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0d015eb00 0x7fb0d01392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d4f82640 1 -- 172.21.15.26:0/1708451711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d00cc2b0 con 0x7fb0d0143c40 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d5f84640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0d015eb00 0x7fb0d01392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.515+0000 7fb0d4f82640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 0x7fb0d006de50 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400e9b0 tx=0x7fb0c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.516+0000 7fb0cdffb640 1 -- 172.21.15.26:0/1708451711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4017070 con 0x7fb0d0143c40 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.516+0000 7fb0cdffb640 1 -- 172.21.15.26:0/1708451711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c400c9a0 con 0x7fb0d0143c40 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.516+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d00cb210 con 0x7fb0d0143c40 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.516+0000 7fb0cdffb640 1 -- 172.21.15.26:0/1708451711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400cd90 con 0x7fb0d0143c40 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.516+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d00cbab0 con 0x7fb0d0143c40 2024-04-24T23:27:53.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.517+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d00694c0 con 0x7fb0d0143c40 2024-04-24T23:27:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.517+0000 7fb0cdffb640 1 -- 172.21.15.26:0/1708451711 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb0c401a070 con 0x7fb0d0143c40 2024-04-24T23:27:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.518+0000 7fb0cdffb640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0bc06be30 0x7fb0bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.518+0000 7fb0cdffb640 1 -- 172.21.15.26:0/1708451711 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fb0c408ad50 con 0x7fb0d0143c40 2024-04-24T23:27:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.518+0000 7fb0d5783640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0bc06be30 0x7fb0bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.518+0000 7fb0d5783640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0bc06be30 0x7fb0bc06e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8034000 tx=0x7fb0c802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.522+0000 7fb0cdffb640 1 -- 172.21.15.26:0/1708451711 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb0c408e030 con 0x7fb0d0143c40 2024-04-24T23:27:53.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.749+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0d00b4d20 con 0x7fb0d0143c40 2024-04-24T23:27:53.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.750+0000 7fb0cdffb640 1 -- 172.21.15.26:0/1708451711 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5229 (secure 0 0 0) 0x7fb0c4059140 con 0x7fb0d0143c40 2024-04-24T23:27:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:53.751 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:48.920245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[7444,8712,8730]},"id":7}]} 2024-04-24T23:27:53.751 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 110 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0bc06be30 msgr2=0x7fb0bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0bc06be30 0x7fb0bc06e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8034000 tx=0x7fb0c802e000 comp rx=0 tx=0).stop 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 msgr2=0x7fb0d006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 0x7fb0d006de50 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400e9b0 tx=0x7fb0c400ee70 comp rx=0 tx=0).stop 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 shutdown_connections 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb0bc06be30 0x7fb0bc06e2f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb0d015eb00 0x7fb0d01392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d0143c40 0x7fb0d006de50 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 --2- 172.21.15.26:0/1708451711 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb0d00cc860 0x7fb0d013cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 >> 172.21.15.26:0/1708451711 conn(0x7fb0d006b300 msgr2=0x7fb0d013f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 shutdown_connections 2024-04-24T23:27:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.752+0000 7fb0d79e7640 1 -- 172.21.15.26:0/1708451711 wait complete. 2024-04-24T23:27:53.763 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:27:53.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:27:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.863+0000 7f122e5a0640 1 Processor -- start 2024-04-24T23:27:53.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.863+0000 7f122e5a0640 1 -- start start 2024-04-24T23:27:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.864+0000 7f122e5a0640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280b05d0 0x7f12280b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.864+0000 7f122e5a0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12280b0a10 con 0x7f122815e1b0 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.864+0000 7f122e5a0640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12280af530 con 0x7f122806c230 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.864+0000 7f122e5a0640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f12280b0140 con 0x7f12280b05d0 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.864+0000 7f122d59e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f122815e1b0 0x7f12280b1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33912/0 (socket says 172.21.15.26:33912) 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.864+0000 7f122d59e640 1 -- 172.21.15.26:0/2413287443 learned_addr learned my addr 172.21.15.26:0/2413287443 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.864+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2859345129 0 0) 0x7f1224001110 con 0x7f122815e1b0 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.864+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1214003520 con 0x7f122815e1b0 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.864+0000 7f122cd9d640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280b05d0 0x7f12280b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1889362539 0 0) 0x7f1214003520 con 0x7f122815e1b0 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280b05d0 msgr2=0x7f12280b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280b05d0 0x7f12280b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 >> v1:172.21.15.29:6789/0 conn(0x7f122806c230 legacy=0x7f122806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12280af810 con 0x7f122815e1b0 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f1224001840 con 0x7f122815e1b0 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 >> v1:172.21.15.26:6789/0 conn(0x7f122815e1b0 legacy=0x7f12280b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12140039e0 0x7f1214003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f1214004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280b05d0 0x7f12140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12280b0140 con 0x7f1214004800 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f121400ba70 con 0x7f12140039e0 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f121400bc70 con 0x7f12280b05d0 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f122cd9d640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f1214004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f122d59e640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12140039e0 0x7f1214003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f122dd9f640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280b05d0 0x7f12140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f122cd9d640 1 -- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280b05d0 msgr2=0x7f12140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f122cd9d640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280b05d0 0x7f12140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f122cd9d640 1 -- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12140039e0 msgr2=0x7f1214003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f122cd9d640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12140039e0 0x7f1214003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f122cd9d640 1 -- 172.21.15.26:0/2413287443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1214003520 con 0x7f1214004800 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.865+0000 7f122d59e640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12140039e0 0x7f1214003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.866+0000 7f122cd9d640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f1214004c80 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f1218009a90 tx=0x7f1214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dc18c4464c72e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.866+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218036070 con 0x7f1214004800 2024-04-24T23:27:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.866+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1218031400 con 0x7f1214004800 2024-04-24T23:27:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.866+0000 7f121ffff640 1 -- 172.21.15.26:0/2413287443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218009690 con 0x7f1214004800 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.866+0000 7f122e5a0640 1 -- 172.21.15.26:0/2413287443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 msgr2=0x7f1214004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.866+0000 7f122e5a0640 1 --2- 172.21.15.26:0/2413287443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f1214004c80 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f1218009a90 tx=0x7f1214005830 comp rx=0 tx=0).stop 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.867+0000 7f122dd9f640 1 -- 172.21.15.26:0/2413287443 reap_dead start 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.867+0000 7f122e5a0640 1 -- 172.21.15.26:0/2413287443 shutdown_connections 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.867+0000 7f122e5a0640 1 -- 172.21.15.26:0/2413287443 >> 172.21.15.26:0/2413287443 conn(0x7f12280d1c50 msgr2=0x7f12280d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.867+0000 7f122e5a0640 1 -- 172.21.15.26:0/2413287443 shutdown_connections 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.867+0000 7f122e5a0640 1 -- 172.21.15.26:0/2413287443 wait complete. 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.868+0000 7f122e5a0640 1 Processor -- start 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.868+0000 7f122e5a0640 1 -- start start 2024-04-24T23:27:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122e5a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f122815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122e5a0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1228159d20 0x7f12281f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122e5a0640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280c9d70 0x7f12280ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122e5a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12280b0e80 con 0x7f1214004800 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122e5a0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12280afdd0 con 0x7f1228159d20 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122e5a0640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f12280b1000 con 0x7f12280c9d70 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122d59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f122815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122d59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f122815a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50970/0 (socket says 172.21.15.26:50970) 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122d59e640 1 -- 172.21.15.26:0/1252333118 learned_addr learned my addr 172.21.15.26:0/1252333118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122dd9f640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280c9d70 0x7f12280ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122cd9d640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1228159d20 0x7f12281f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122d59e640 1 -- 172.21.15.26:0/1252333118 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280c9d70 msgr2=0x7f12280ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122d59e640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280c9d70 0x7f12280ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122d59e640 1 -- 172.21.15.26:0/1252333118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1228159d20 msgr2=0x7f12281f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:53.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122d59e640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1228159d20 0x7f12281f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122d59e640 1 -- 172.21.15.26:0/1252333118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12281f8470 con 0x7f1214004800 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122cd9d640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1228159d20 0x7f12281f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122d59e640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f122815a1b0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f1224007360 tx=0x7f1224007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f121dffb640 1 -- 172.21.15.26:0/1252333118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224005e20 con 0x7f1214004800 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f121dffb640 1 -- 172.21.15.26:0/1252333118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1224018070 con 0x7f1214004800 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f121dffb640 1 -- 172.21.15.26:0/1252333118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122400eaf0 con 0x7f1214004800 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.869+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12280d0a00 con 0x7f1214004800 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.870+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12281f7eb0 con 0x7f1214004800 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.871+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1228075610 con 0x7f1214004800 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.871+0000 7f121dffb640 1 -- 172.21.15.26:0/1252333118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f122400ec90 con 0x7f1214004800 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.871+0000 7f121dffb640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f120c06be30 0x7f120c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.871+0000 7f121dffb640 1 -- 172.21.15.26:0/1252333118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f122408ade0 con 0x7f1214004800 2024-04-24T23:27:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.871+0000 7f122cd9d640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f120c06be30 0x7f120c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.872+0000 7f122cd9d640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f120c06be30 0x7f120c06e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1218033000 tx=0x7f1218008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:53.876+0000 7f121dffb640 1 -- 172.21.15.26:0/1252333118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f12280b0a10 con 0x7f1214004800 2024-04-24T23:27:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.135+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1228161750 con 0x7f1214004800 2024-04-24T23:27:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.135+0000 7f121dffb640 1 -- 172.21.15.26:0/1252333118 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7f1224059820 con 0x7f1214004800 2024-04-24T23:27:54.136 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:54.136 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":443,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700179,"bytes_used":287752192,"bytes_avail":772806361088,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":10664,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":111,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-24T23:27:15.398834+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:27:54.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f120c06be30 msgr2=0x7f120c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:54.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f120c06be30 0x7f120c06e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1218033000 tx=0x7f1218008030 comp rx=0 tx=0).stop 2024-04-24T23:27:54.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 msgr2=0x7f122815a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:54.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f122815a1b0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f1224007360 tx=0x7f1224007820 comp rx=0 tx=0).stop 2024-04-24T23:27:54.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 shutdown_connections 2024-04-24T23:27:54.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f120c06be30 0x7f120c06e2f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f12280c9d70 0x7f12280ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1228159d20 0x7f12281f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 --2- 172.21.15.26:0/1252333118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1214004800 0x7f122815a1b0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 >> 172.21.15.26:0/1252333118 conn(0x7f12280d1c50 msgr2=0x7f1228160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:54.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 shutdown_connections 2024-04-24T23:27:54.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:54.137+0000 7f122e5a0640 1 -- 172.21.15.26:0/1252333118 wait complete. 2024-04-24T23:27:54.149 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-24T23:27:55.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.262+0000 7f3eef44d640 1 Processor -- start 2024-04-24T23:27:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.263+0000 7f3eef44d640 1 -- start start 2024-04-24T23:27:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.263+0000 7f3eef44d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3ee815a480 0x7f3ee80b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.263+0000 7f3eef44d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ee80b2e80 con 0x7f3ee806e2f0 2024-04-24T23:27:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.263+0000 7f3eef44d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ee80b3120 con 0x7f3ee815a860 2024-04-24T23:27:55.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.263+0000 7f3eef44d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f3ee80b33c0 con 0x7f3ee815a480 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3eed9ea640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3ee806e2f0 0x7f3ee8071070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33920/0 (socket says 172.21.15.26:33920) 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3eed9ea640 1 -- 172.21.15.26:0/791423220 learned_addr learned my addr 172.21.15.26:0/791423220 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3eed1e9640 1 --2- 172.21.15.26:0/791423220 >> v2:172.21.15.29:3300/0 conn(0x7f3ee815a480 0x7f3ee80b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3535402447 0 0) 0x7f3ee80b2e80 con 0x7f3ee806e2f0 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ed4003520 con 0x7f3ee806e2f0 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560036807 0 0) 0x7f3ee80b3120 con 0x7f3ee815a860 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ee80b2e80 con 0x7f3ee815a860 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2630846352 0 0) 0x7f3ed4003520 con 0x7f3ee806e2f0 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 >> v2:172.21.15.29:3300/0 conn(0x7f3ee815a480 msgr2=0x7f3ee80b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 --2- 172.21.15.26:0/791423220 >> v2:172.21.15.29:3300/0 conn(0x7f3ee815a480 0x7f3ee80b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 >> v1:172.21.15.29:6789/0 conn(0x7f3ee815a860 legacy=0x7f3ee806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee80b36a0 con 0x7f3ee806e2f0 2024-04-24T23:27:55.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f3ee0001a10 con 0x7f3ee806e2f0 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3eed1e9640 1 --2- 172.21.15.26:0/791423220 >> v2:172.21.15.29:3300/0 conn(0x7f3ee815a480 0x7f3ee80b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 >> v1:172.21.15.26:6789/0 conn(0x7f3ee806e2f0 legacy=0x7f3ee8071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4003b40 0x7f3ed4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ed4004800 0x7f3ed4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ed40051c0 0x7f3ed400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee80b33c0 con 0x7f3ed4003b40 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed400c0f0 con 0x7f3ed4004800 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed400c2f0 con 0x7f3ed40051c0 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.264+0000 7f3eed1e9640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4003b40 0x7f3ed4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.265+0000 7f3eec9e8640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ed4004800 0x7f3ed4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.265+0000 7f3eed9ea640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ed40051c0 0x7f3ed400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.265+0000 7f3eed1e9640 1 -- 172.21.15.26:0/791423220 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ed40051c0 msgr2=0x7f3ed400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.265+0000 7f3eed1e9640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ed40051c0 0x7f3ed400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.265+0000 7f3eed1e9640 1 -- 172.21.15.26:0/791423220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ed4004800 msgr2=0x7f3ed4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:55.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.265+0000 7f3eed1e9640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ed4004800 0x7f3ed4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.265+0000 7f3eed1e9640 1 -- 172.21.15.26:0/791423220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed4003520 con 0x7f3ed4003b40 2024-04-24T23:27:55.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.265+0000 7f3eed9ea640 1 -- 172.21.15.26:0/791423220 reap_dead start 2024-04-24T23:27:55.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3eed1e9640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4003b40 0x7f3ed4003f40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009c90 tx=0x7f3ed4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296485fce79fdfe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:55.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4036070 con 0x7f3ed4003b40 2024-04-24T23:27:55.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee4031400 con 0x7f3ed4003b40 2024-04-24T23:27:55.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3edffff640 1 -- 172.21.15.26:0/791423220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4009860 con 0x7f3ed4003b40 2024-04-24T23:27:55.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3eef44d640 1 -- 172.21.15.26:0/791423220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4003b40 msgr2=0x7f3ed4003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3eef44d640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4003b40 0x7f3ed4003f40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009c90 tx=0x7f3ed4005830 comp rx=0 tx=0).stop 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3eef44d640 1 -- 172.21.15.26:0/791423220 shutdown_connections 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3eef44d640 1 --2- 172.21.15.26:0/791423220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ed4003b40 0x7f3ed4003f40 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3eef44d640 1 -- 172.21.15.26:0/791423220 >> 172.21.15.26:0/791423220 conn(0x7f3ee806b300 msgr2=0x7f3ee806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3eef44d640 1 -- 172.21.15.26:0/791423220 shutdown_connections 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.266+0000 7f3eef44d640 1 -- 172.21.15.26:0/791423220 wait complete. 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.268+0000 7f3eef44d640 1 Processor -- start 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.268+0000 7f3eef44d640 1 -- start start 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.268+0000 7f3eef44d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ed4003b40 0x7f3ee81e4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.268+0000 7f3eef44d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ee815d4e0 0x7f3ee80b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.268+0000 7f3eef44d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee80b0cf0 0x7f3ee81492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.268+0000 7f3eef44d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee80b48f0 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.268+0000 7f3eef44d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee80b3c60 con 0x7f3ee815d4e0 2024-04-24T23:27:55.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.268+0000 7f3eef44d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee80b4a70 con 0x7f3ed4003b40 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.268+0000 7f3eed9ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee80b0cf0 0x7f3ee81492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eed9ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee80b0cf0 0x7f3ee81492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50986/0 (socket says 172.21.15.26:50986) 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eed9ea640 1 -- 172.21.15.26:0/1207435902 learned_addr learned my addr 172.21.15.26:0/1207435902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eed1e9640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ed4003b40 0x7f3ee81e4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eec9e8640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ee815d4e0 0x7f3ee80b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eed9ea640 1 -- 172.21.15.26:0/1207435902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ed4003b40 msgr2=0x7f3ee81e4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eed9ea640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ed4003b40 0x7f3ee81e4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eed9ea640 1 -- 172.21.15.26:0/1207435902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ee815d4e0 msgr2=0x7f3ee80b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eed9ea640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ee815d4e0 0x7f3ee80b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eed9ea640 1 -- 172.21.15.26:0/1207435902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee81e4670 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eed9ea640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee80b0cf0 0x7f3ee81492b0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f3ee815acb0 tx=0x7f3ee0007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eddffb640 1 -- 172.21.15.26:0/1207435902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000aa10 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eddffb640 1 -- 172.21.15.26:0/1207435902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee0018070 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eddffb640 1 -- 172.21.15.26:0/1207435902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000ae30 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.269+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee81f4850 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.270+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee81f4e10 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.271+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee815eb60 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.271+0000 7f3eddffb640 1 -- 172.21.15.26:0/1207435902 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ee001d460 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.271+0000 7f3eddffb640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.272+0000 7f3eddffb640 1 -- 172.21.15.26:0/1207435902 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f3ee008b1c0 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.272+0000 7f3eed1e9640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.272+0000 7f3eed1e9640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4033000 tx=0x7f3ee4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:55.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.276+0000 7f3eddffb640 1 -- 172.21.15.26:0/1207435902 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ee815eb60 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.497+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ee80694c0 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.497+0000 7f3eddffb640 1 -- 172.21.15.26:0/1207435902 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5231 (secure 0 0 0) 0x7f3ee0059c00 con 0x7f3ee80b0cf0 2024-04-24T23:27:55.498 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:55.498 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":111,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:53.920076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:27:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 111 2024-04-24T23:27:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ecc06be30 msgr2=0x7f3ecc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4033000 tx=0x7f3ee4008030 comp rx=0 tx=0).stop 2024-04-24T23:27:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee80b0cf0 msgr2=0x7f3ee81492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee80b0cf0 0x7f3ee81492b0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f3ee815acb0 tx=0x7f3ee0007b10 comp rx=0 tx=0).stop 2024-04-24T23:27:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 shutdown_connections 2024-04-24T23:27:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee80b0cf0 0x7f3ee81492b0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ee815d4e0 0x7f3ee80b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 --2- 172.21.15.26:0/1207435902 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3ed4003b40 0x7f3ee81e4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 >> 172.21.15.26:0/1207435902 conn(0x7f3ee806b300 msgr2=0x7f3ee80cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 shutdown_connections 2024-04-24T23:27:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.499+0000 7f3eef44d640 1 -- 172.21.15.26:0/1207435902 wait complete. 2024-04-24T23:27:55.511 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:27:55.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:27:55.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.611+0000 7f3eb7b21640 1 Processor -- start 2024-04-24T23:27:55.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.611+0000 7f3eb7b21640 1 -- start start 2024-04-24T23:27:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb7b21640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb00ce040 0x7f3eb006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb7b21640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb015cee0 con 0x7f3eb0145a80 2024-04-24T23:27:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb7b21640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb015d1c0 con 0x7f3eb006da70 2024-04-24T23:27:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb7b21640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb015ccc0 con 0x7f3eb00ce040 2024-04-24T23:27:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb58bd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3eb0145a80 0x7f3eb015c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33936/0 (socket says 172.21.15.26:33936) 2024-04-24T23:27:55.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb58bd640 1 -- 172.21.15.26:0/1853062132 learned_addr learned my addr 172.21.15.26:0/1853062132 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 735033184 0 0) 0x7f3eb015cee0 con 0x7f3eb0145a80 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb50bc640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb00ce040 0x7f3eb006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e9c003520 con 0x7f3eb0145a80 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1401375322 0 0) 0x7f3e9c003520 con 0x7f3eb0145a80 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb00ce040 msgr2=0x7f3eb006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb00ce040 0x7f3eb006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 >> v1:172.21.15.29:6789/0 conn(0x7f3eb006da70 legacy=0x7f3eb00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb015d4a0 con 0x7f3eb0145a80 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f3eac001670 con 0x7f3eb0145a80 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 >> v1:172.21.15.26:6789/0 conn(0x7f3eb0145a80 legacy=0x7f3eb015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb00ce040 0x7f3e9c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e9c003f60 0x7f3e9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c004d40 0x7f3e9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb015ccc0 con 0x7f3e9c004d40 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c00baa0 con 0x7f3e9c003f60 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.612+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9c00bca0 con 0x7f3eb00ce040 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.613+0000 7f3eb60be640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c004d40 0x7f3e9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.613+0000 7f3eb58bd640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb00ce040 0x7f3e9c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.613+0000 7f3eb50bc640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e9c003f60 0x7f3e9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.613+0000 7f3eb60be640 1 -- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb00ce040 msgr2=0x7f3e9c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.613+0000 7f3eb60be640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb00ce040 0x7f3e9c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.613+0000 7f3eb60be640 1 -- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e9c003f60 msgr2=0x7f3e9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.613+0000 7f3eb60be640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3e9c003f60 0x7f3e9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.613+0000 7f3eb60be640 1 -- 172.21.15.26:0/1853062132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9c003520 con 0x7f3e9c004d40 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb60be640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c004d40 0x7f3e9c0051c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8001e50 tx=0x7f3e9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c169c09e0625de8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8001a70 con 0x7f3e9c004d40 2024-04-24T23:27:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea8003970 con 0x7f3e9c004d40 2024-04-24T23:27:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb48bb640 1 -- 172.21.15.26:0/1853062132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8003d60 con 0x7f3e9c004d40 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1853062132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c004d40 msgr2=0x7f3e9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb7b21640 1 --2- 172.21.15.26:0/1853062132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e9c004d40 0x7f3e9c0051c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8001e50 tx=0x7f3e9c005830 comp rx=0 tx=0).stop 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb60be640 1 -- 172.21.15.26:0/1853062132 reap_dead start 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1853062132 shutdown_connections 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1853062132 >> 172.21.15.26:0/1853062132 conn(0x7f3eb006b540 msgr2=0x7f3eb006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1853062132 shutdown_connections 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.614+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1853062132 wait complete. 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb7b21640 1 Processor -- start 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb7b21640 1 -- start start 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb7b21640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb013d100 0x7f3eb00aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb7b21640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb015b7f0 0x7f3eb0071be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb7b21640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb015bc70 0x7f3eb0072120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb7b21640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb00b3300 con 0x7f3eb013d100 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb7b21640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb006d530 con 0x7f3eb015bc70 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb7b21640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb006d6b0 con 0x7f3eb015b7f0 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb58bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb013d100 0x7f3eb00aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb58bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb013d100 0x7f3eb00aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51002/0 (socket says 172.21.15.26:51002) 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb58bd640 1 -- 172.21.15.26:0/1163227201 learned_addr learned my addr 172.21.15.26:0/1163227201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.616+0000 7f3eb50bc640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb015b7f0 0x7f3eb0071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3eb60be640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb015bc70 0x7f3eb0072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3eb58bd640 1 -- 172.21.15.26:0/1163227201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb015b7f0 msgr2=0x7f3eb0071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3eb58bd640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb015b7f0 0x7f3eb0071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3eb58bd640 1 -- 172.21.15.26:0/1163227201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb015bc70 msgr2=0x7f3eb0072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3eb58bd640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb015bc70 0x7f3eb0072120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3eb58bd640 1 -- 172.21.15.26:0/1163227201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb00ad870 con 0x7f3eb013d100 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3eb58bd640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb013d100 0x7f3eb00aebf0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f3eac007340 tx=0x7f3eac007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3ea67fc640 1 -- 172.21.15.26:0/1163227201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac00f940 con 0x7f3eb013d100 2024-04-24T23:27:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3ea67fc640 1 -- 172.21.15.26:0/1163227201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eac018070 con 0x7f3eb013d100 2024-04-24T23:27:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb00adb50 con 0x7f3eb013d100 2024-04-24T23:27:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3ea67fc640 1 -- 172.21.15.26:0/1163227201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac00fd20 con 0x7f3eb013d100 2024-04-24T23:27:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.617+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb00ae110 con 0x7f3eb013d100 2024-04-24T23:27:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.618+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb00694c0 con 0x7f3eb013d100 2024-04-24T23:27:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.618+0000 7f3ea67fc640 1 -- 172.21.15.26:0/1163227201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3eac01d460 con 0x7f3eb013d100 2024-04-24T23:27:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.619+0000 7f3ea67fc640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9406be30 0x7f3e9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.619+0000 7f3eb50bc640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9406be30 0x7f3e9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.619+0000 7f3ea67fc640 1 -- 172.21.15.26:0/1163227201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f3eac08aed0 con 0x7f3eb013d100 2024-04-24T23:27:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.619+0000 7f3eb50bc640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9406be30 0x7f3e9406e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0071a80 tx=0x7f3ea0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.623+0000 7f3ea67fc640 1 -- 172.21.15.26:0/1163227201 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3eac059910 con 0x7f3eb013d100 2024-04-24T23:27:55.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.876+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3eb00b53f0 con 0x7f3eb013d100 2024-04-24T23:27:55.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.876+0000 7f3ea67fc640 1 -- 172.21.15.26:0/1163227201 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7f3eac059210 con 0x7f3eb013d100 2024-04-24T23:27:55.877 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:55.877 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":444,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700179,"bytes_used":287752192,"bytes_avail":772806361088,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":10664,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":111,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-24T23:27:15.398834+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9406be30 msgr2=0x7f3e9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9406be30 0x7f3e9406e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0071a80 tx=0x7f3ea0009190 comp rx=0 tx=0).stop 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb013d100 msgr2=0x7f3eb00aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb013d100 0x7f3eb00aebf0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f3eac007340 tx=0x7f3eac007740 comp rx=0 tx=0).stop 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 shutdown_connections 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3e9406be30 0x7f3e9406e2f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3eb015bc70 0x7f3eb0072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3eb015b7f0 0x7f3eb0071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 --2- 172.21.15.26:0/1163227201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3eb013d100 0x7f3eb00aebf0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 >> 172.21.15.26:0/1163227201 conn(0x7f3eb006b540 msgr2=0x7f3eb00b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 shutdown_connections 2024-04-24T23:27:55.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:55.879+0000 7f3eb7b21640 1 -- 172.21.15.26:0/1163227201 wait complete. 2024-04-24T23:27:55.890 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-24T23:27:56.885 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:27:56.884+0000 7f26a3749640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T23:27:56.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:57.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.003+0000 7f5078fee640 1 Processor -- start 2024-04-24T23:27:57.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.003+0000 7f5078fee640 1 -- start start 2024-04-24T23:27:57.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.004+0000 7f5078fee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 0x7f50740d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.004+0000 7f5078fee640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f50740b3130 0x7f50740d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.004+0000 7f5078fee640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50740d32f0 con 0x7f50740d22e0 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.004+0000 7f5078fee640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f50740d1ab0 con 0x7f50740b3130 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.004+0000 7f5078fee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f50740d2ae0 con 0x7f507415e570 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.005+0000 7f5071d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f50740b3130 0x7f50740d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.005+0000 7f507259c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 0x7f50740d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.005+0000 7f5072d9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f50740d22e0 0x7f50740d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:42608/0 (socket says 172.21.15.26:42608) 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.005+0000 7f5072d9d640 1 -- 172.21.15.26:0/2558135913 learned_addr learned my addr 172.21.15.26:0/2558135913 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.005+0000 7f5071d9b640 1 -- 172.21.15.26:0/2558135913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 msgr2=0x7f50740d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.005+0000 7f5071d9b640 1 --2- 172.21.15.26:0/2558135913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 0x7f50740d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.005+0000 7f5071d9b640 1 -- 172.21.15.26:0/2558135913 >> v1:172.21.15.29:6789/0 conn(0x7f50740d22e0 legacy=0x7f50740d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.005+0000 7f5071d9b640 1 -- 172.21.15.26:0/2558135913 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50740d0270 con 0x7f50740b3130 2024-04-24T23:27:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.005+0000 7f507259c640 1 --2- 172.21.15.26:0/2558135913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 0x7f50740d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.006+0000 7f5071d9b640 1 --2- 172.21.15.26:0/2558135913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 0x7f50740d46a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f505c0097a0 tx=0x7f505c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33adeaa4ef80fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.006+0000 7f507159a640 1 -- 172.21.15.26:0/2558135913 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c037070 con 0x7f50740b3130 2024-04-24T23:27:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.006+0000 7f507159a640 1 -- 172.21.15.26:0/2558135913 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505c032440 con 0x7f50740b3130 2024-04-24T23:27:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.006+0000 7f507159a640 1 -- 172.21.15.26:0/2558135913 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c032810 con 0x7f50740b3130 2024-04-24T23:27:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.007+0000 7f5078fee640 1 -- 172.21.15.26:0/2558135913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 msgr2=0x7f50740d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.007+0000 7f5078fee640 1 --2- 172.21.15.26:0/2558135913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 0x7f50740d46a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f505c0097a0 tx=0x7f505c02a180 comp rx=0 tx=0).stop 2024-04-24T23:27:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.007+0000 7f5078fee640 1 -- 172.21.15.26:0/2558135913 shutdown_connections 2024-04-24T23:27:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.007+0000 7f5078fee640 1 --2- 172.21.15.26:0/2558135913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 0x7f50740d46a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.007+0000 7f5078fee640 1 --2- 172.21.15.26:0/2558135913 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 0x7f50740d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.007+0000 7f5078fee640 1 -- 172.21.15.26:0/2558135913 >> 172.21.15.26:0/2558135913 conn(0x7f507408dbe0 msgr2=0x7f507409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.007+0000 7f5078fee640 1 -- 172.21.15.26:0/2558135913 shutdown_connections 2024-04-24T23:27:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.007+0000 7f5078fee640 1 -- 172.21.15.26:0/2558135913 wait complete. 2024-04-24T23:27:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.008+0000 7f5078fee640 1 Processor -- start 2024-04-24T23:27:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.008+0000 7f5078fee640 1 -- start start 2024-04-24T23:27:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5078fee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 0x7f5074150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5078fee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50740d22e0 0x7f507415fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5078fee640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 0x7f50740b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5078fee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f50740cfa70 con 0x7f50740d22e0 2024-04-24T23:27:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5078fee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f50740d3010 con 0x7f50740b3130 2024-04-24T23:27:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5078fee640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f50740cfd50 con 0x7f507415e570 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5071d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50740d22e0 0x7f507415fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5071d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50740d22e0 0x7f507415fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51006/0 (socket says 172.21.15.26:51006) 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f507259c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 0x7f5074150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5071d9b640 1 -- 172.21.15.26:0/2637824593 learned_addr learned my addr 172.21.15.26:0/2637824593 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5072d9d640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 0x7f50740b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5071d9b640 1 -- 172.21.15.26:0/2637824593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 msgr2=0x7f50740b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5071d9b640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 0x7f50740b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5071d9b640 1 -- 172.21.15.26:0/2637824593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 msgr2=0x7f5074150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5071d9b640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 0x7f5074150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5071d9b640 1 -- 172.21.15.26:0/2637824593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507415eba0 con 0x7f50740d22e0 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f507259c640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 0x7f5074150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.009+0000 7f5071d9b640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50740d22e0 0x7f507415fc40 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f505c0098c0 tx=0x7f505c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.010+0000 7f506affd640 1 -- 172.21.15.26:0/2637824593 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c037040 con 0x7f50740d22e0 2024-04-24T23:27:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.010+0000 7f506affd640 1 -- 172.21.15.26:0/2637824593 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505c030500 con 0x7f50740d22e0 2024-04-24T23:27:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.010+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507415ee80 con 0x7f50740d22e0 2024-04-24T23:27:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.010+0000 7f506affd640 1 -- 172.21.15.26:0/2637824593 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c0308d0 con 0x7f50740d22e0 2024-04-24T23:27:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.010+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507415f440 con 0x7f50740d22e0 2024-04-24T23:27:57.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.011+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50740694c0 con 0x7f50740d22e0 2024-04-24T23:27:57.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.011+0000 7f506affd640 1 -- 172.21.15.26:0/2637824593 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f505c044020 con 0x7f50740d22e0 2024-04-24T23:27:57.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.012+0000 7f506affd640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f505406be30 0x7f505406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.012+0000 7f506affd640 1 -- 172.21.15.26:0/2637824593 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f505c0aafe0 con 0x7f50740d22e0 2024-04-24T23:27:57.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.012+0000 7f507259c640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f505406be30 0x7f505406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.012+0000 7f507259c640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f505406be30 0x7f505406e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f506c009560 tx=0x7f506c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.016+0000 7f506affd640 1 -- 172.21.15.26:0/2637824593 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f505c0792e0 con 0x7f50740d22e0 2024-04-24T23:27:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.237+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50740ceea0 con 0x7f50740d22e0 2024-04-24T23:27:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.237+0000 7f506affd640 1 -- 172.21.15.26:0/2637824593 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5233 (secure 0 0 0) 0x7f505c07d5e0 con 0x7f50740d22e0 2024-04-24T23:27:57.238 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:57.238 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":112,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:55.982635+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":5,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[7444,8712,8730]},"id":7}]} 2024-04-24T23:27:57.239 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 112 2024-04-24T23:27:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f505406be30 msgr2=0x7f505406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f505406be30 0x7f505406e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f506c009560 tx=0x7f506c009190 comp rx=0 tx=0).stop 2024-04-24T23:27:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50740d22e0 msgr2=0x7f507415fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50740d22e0 0x7f507415fc40 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f505c0098c0 tx=0x7f505c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 shutdown_connections 2024-04-24T23:27:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f505406be30 0x7f505406e2f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f507415e570 0x7f50740b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50740d22e0 0x7f507415fc40 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 --2- 172.21.15.26:0/2637824593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f50740b3130 0x7f5074150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.239+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 >> 172.21.15.26:0/2637824593 conn(0x7f507408dbe0 msgr2=0x7f507406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.240+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 shutdown_connections 2024-04-24T23:27:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.240+0000 7f5078fee640 1 -- 172.21.15.26:0/2637824593 wait complete. 2024-04-24T23:27:57.251 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:27:57.251 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:27:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.350+0000 7f1fe88f7640 1 Processor -- start 2024-04-24T23:27:57.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.350+0000 7f1fe88f7640 1 -- start start 2024-04-24T23:27:57.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fe88f7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 0x7f1fe415f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fe88f7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe415f850 con 0x7f1fe406d760 2024-04-24T23:27:57.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fe88f7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe4162000 con 0x7f1fe406dbc0 2024-04-24T23:27:57.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fe88f7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe415c270 con 0x7f1fe415be90 2024-04-24T23:27:57.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fe259c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 0x7f1fe415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fd9d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1fe406d760 0x7f1fe4162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33952/0 (socket says 172.21.15.26:33952) 2024-04-24T23:27:57.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fd9d9b640 1 -- 172.21.15.26:0/702292295 learned_addr learned my addr 172.21.15.26:0/702292295 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:57.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fe1d9b640 1 -- 172.21.15.26:0/702292295 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331205032 0 0) 0x7f1fcc001110 con 0x7f1fe406d760 2024-04-24T23:27:57.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fe259c640 1 -- 172.21.15.26:0/702292295 >> v1:172.21.15.29:6789/0 conn(0x7f1fe406dbc0 legacy=0x7f1fe40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fe259c640 1 -- 172.21.15.26:0/702292295 >> v1:172.21.15.26:6789/0 conn(0x7f1fe406d760 legacy=0x7f1fe4162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.351+0000 7f1fe259c640 1 -- 172.21.15.26:0/702292295 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe4071a70 con 0x7f1fe415be90 2024-04-24T23:27:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.352+0000 7f1fe259c640 1 --2- 172.21.15.26:0/702292295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 0x7f1fe415f310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009750 tx=0x7f1fd402a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfdd38e6ea162b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.352+0000 7f1fe1d9b640 1 -- 172.21.15.26:0/702292295 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4037070 con 0x7f1fe415be90 2024-04-24T23:27:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.352+0000 7f1fe1d9b640 1 -- 172.21.15.26:0/702292295 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd402ad20 con 0x7f1fe415be90 2024-04-24T23:27:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.352+0000 7f1fe1d9b640 1 -- 172.21.15.26:0/702292295 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4032660 con 0x7f1fe415be90 2024-04-24T23:27:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.352+0000 7f1fe88f7640 1 -- 172.21.15.26:0/702292295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 msgr2=0x7f1fe415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.352+0000 7f1fe88f7640 1 --2- 172.21.15.26:0/702292295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 0x7f1fe415f310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009750 tx=0x7f1fd402a010 comp rx=0 tx=0).stop 2024-04-24T23:27:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.353+0000 7f1fe88f7640 1 -- 172.21.15.26:0/702292295 shutdown_connections 2024-04-24T23:27:57.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.353+0000 7f1fe88f7640 1 --2- 172.21.15.26:0/702292295 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 0x7f1fe415f310 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.353+0000 7f1fe88f7640 1 -- 172.21.15.26:0/702292295 >> 172.21.15.26:0/702292295 conn(0x7f1fe40ccff0 msgr2=0x7f1fe40cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:57.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.353+0000 7f1fe88f7640 1 -- 172.21.15.26:0/702292295 shutdown_connections 2024-04-24T23:27:57.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.353+0000 7f1fe88f7640 1 -- 172.21.15.26:0/702292295 wait complete. 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.354+0000 7f1fe88f7640 1 Processor -- start 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.354+0000 7f1fe88f7640 1 -- start start 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fe88f7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe406d760 0x7f1fe4154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fe88f7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe406dbc0 0x7f1fe415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fe88f7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 0x7f1fe4161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fe88f7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4071740 con 0x7f1fe406dbc0 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fe88f7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4072030 con 0x7f1fe406d760 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fe88f7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe40b3650 con 0x7f1fe415be90 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fd9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe406dbc0 0x7f1fe415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fd9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe406dbc0 0x7f1fe415fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51018/0 (socket says 172.21.15.26:51018) 2024-04-24T23:27:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fd9d9b640 1 -- 172.21.15.26:0/2802320970 learned_addr learned my addr 172.21.15.26:0/2802320970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fe2d9d640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 0x7f1fe4161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fe259c640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe406d760 0x7f1fe4154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fd9d9b640 1 -- 172.21.15.26:0/2802320970 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 msgr2=0x7f1fe4161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fd9d9b640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 0x7f1fe4161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fd9d9b640 1 -- 172.21.15.26:0/2802320970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe406d760 msgr2=0x7f1fe4154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fd9d9b640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe406d760 0x7f1fe4154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.355+0000 7f1fd9d9b640 1 -- 172.21.15.26:0/2802320970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe4154680 con 0x7f1fe406dbc0 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.356+0000 7f1fd9d9b640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe406dbc0 0x7f1fe415fa30 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00c6d0 tx=0x7f1fcc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.356+0000 7f1fdb7fe640 1 -- 172.21.15.26:0/2802320970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc018070 con 0x7f1fe406dbc0 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.356+0000 7f1fdb7fe640 1 -- 172.21.15.26:0/2802320970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fcc0072d0 con 0x7f1fe406dbc0 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.356+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe41604f0 con 0x7f1fe406dbc0 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.356+0000 7f1fdb7fe640 1 -- 172.21.15.26:0/2802320970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0047e0 con 0x7f1fe406dbc0 2024-04-24T23:27:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.356+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe4160ab0 con 0x7f1fe406dbc0 2024-04-24T23:27:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.357+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe40694c0 con 0x7f1fe406dbc0 2024-04-24T23:27:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.357+0000 7f1fdb7fe640 1 -- 172.21.15.26:0/2802320970 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fcc01b070 con 0x7f1fe406dbc0 2024-04-24T23:27:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.358+0000 7f1fdb7fe640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc406be30 0x7f1fc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.358+0000 7f1fdb7fe640 1 -- 172.21.15.26:0/2802320970 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f1fcc08a180 con 0x7f1fe406dbc0 2024-04-24T23:27:57.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.358+0000 7f1fe259c640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc406be30 0x7f1fc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.358+0000 7f1fe259c640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc406be30 0x7f1fc406e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4034000 tx=0x7f1fd402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:57.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.362+0000 7f1fdb7fe640 1 -- 172.21.15.26:0/2802320970 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1fe4154680 con 0x7f1fe406dbc0 2024-04-24T23:27:57.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.619+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1fe41515f0 con 0x7f1fe406dbc0 2024-04-24T23:27:57.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.619+0000 7f1fdb7fe640 1 -- 172.21.15.26:0/2802320970 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1886 (secure 0 0 0) 0x7f1fe415f850 con 0x7f1fe406dbc0 2024-04-24T23:27:57.620 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:57.620 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"2 MDSs report damaged metadata","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":446,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700179,"bytes_used":287752192,"bytes_avail":772806361088,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":10066,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":112,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-24T23:27:55.407920+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:27:57.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc406be30 msgr2=0x7f1fc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc406be30 0x7f1fc406e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4034000 tx=0x7f1fd402e000 comp rx=0 tx=0).stop 2024-04-24T23:27:57.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe406dbc0 msgr2=0x7f1fe415fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe406dbc0 0x7f1fe415fa30 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00c6d0 tx=0x7f1fcc00cb90 comp rx=0 tx=0).stop 2024-04-24T23:27:57.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 shutdown_connections 2024-04-24T23:27:57.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1fc406be30 0x7f1fc406e2f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1fe415be90 0x7f1fe4161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fe406dbc0 0x7f1fe415fa30 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 --2- 172.21.15.26:0/2802320970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1fe406d760 0x7f1fe4154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 >> 172.21.15.26:0/2802320970 conn(0x7f1fe40ccff0 msgr2=0x7f1fe415efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:57.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 shutdown_connections 2024-04-24T23:27:57.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:57.621+0000 7f1fe88f7640 1 -- 172.21.15.26:0/2802320970 wait complete. 2024-04-24T23:27:57.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:27:58.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.747+0000 7f4052de7640 1 Processor -- start 2024-04-24T23:27:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.747+0000 7f4052de7640 1 -- start start 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.747+0000 7f4052de7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f404c143c40 0x7f404c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.747+0000 7f4052de7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f404c0cc860 0x7f404c072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4052de7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f404c0cd0b0 con 0x7f404c15eb00 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4052de7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f404c071b70 con 0x7f404c143c40 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4052de7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f404c071240 con 0x7f404c0cc860 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4050b83640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f404c143c40 0x7f404c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4051384640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f404c15eb00 0x7f404c15ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33956/0 (socket says 172.21.15.26:33956) 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4051384640 1 -- 172.21.15.26:0/4267050850 learned_addr learned my addr 172.21.15.26:0/4267050850 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f404bfff640 1 --2- 172.21.15.26:0/4267050850 >> v2:172.21.15.29:3300/0 conn(0x7f404c0cc860 0x7f404c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4050b83640 1 -- 172.21.15.26:0/4267050850 >> v2:172.21.15.29:3300/0 conn(0x7f404c0cc860 msgr2=0x7f404c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4050b83640 1 --2- 172.21.15.26:0/4267050850 >> v2:172.21.15.29:3300/0 conn(0x7f404c0cc860 0x7f404c072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4050b83640 1 -- 172.21.15.26:0/4267050850 >> v1:172.21.15.26:6789/0 conn(0x7f404c15eb00 legacy=0x7f404c15ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.748+0000 7f4050b83640 1 -- 172.21.15.26:0/4267050850 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c071520 con 0x7f404c143c40 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.749+0000 7f4050b83640 1 --2- 172.21.15.26:0/4267050850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c143c40 0x7f404c0cf870 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f404c06db50 tx=0x7f404002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ab4dc5c2544848 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.749+0000 7f404b7fe640 1 -- 172.21.15.26:0/4267050850 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040037070 con 0x7f404c143c40 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.749+0000 7f404b7fe640 1 -- 172.21.15.26:0/4267050850 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4040032440 con 0x7f404c143c40 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.749+0000 7f404b7fe640 1 -- 172.21.15.26:0/4267050850 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040032810 con 0x7f404c143c40 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.749+0000 7f4052de7640 1 -- 172.21.15.26:0/4267050850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c143c40 msgr2=0x7f404c0cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.749+0000 7f4052de7640 1 --2- 172.21.15.26:0/4267050850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c143c40 0x7f404c0cf870 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f404c06db50 tx=0x7f404002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.750+0000 7f4052de7640 1 -- 172.21.15.26:0/4267050850 shutdown_connections 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.750+0000 7f4052de7640 1 --2- 172.21.15.26:0/4267050850 >> v2:172.21.15.29:3300/0 conn(0x7f404c0cc860 0x7f404c072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.750+0000 7f4052de7640 1 --2- 172.21.15.26:0/4267050850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c143c40 0x7f404c0cf870 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.750+0000 7f4052de7640 1 -- 172.21.15.26:0/4267050850 >> 172.21.15.26:0/4267050850 conn(0x7f404c06b300 msgr2=0x7f404c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.750+0000 7f4052de7640 1 -- 172.21.15.26:0/4267050850 shutdown_connections 2024-04-24T23:27:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.750+0000 7f4052de7640 1 -- 172.21.15.26:0/4267050850 wait complete. 2024-04-24T23:27:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.751+0000 7f4052de7640 1 Processor -- start 2024-04-24T23:27:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.751+0000 7f4052de7640 1 -- start start 2024-04-24T23:27:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4052de7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f404c0cc860 0x7f404c138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4052de7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f404c143c40 0x7f404c0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4052de7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c15eb00 0x7f404c1e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4052de7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f404c072540 con 0x7f404c15eb00 2024-04-24T23:27:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4052de7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f404c144020 con 0x7f404c143c40 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4052de7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f404c0b04b0 con 0x7f404c0cc860 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4051384640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c15eb00 0x7f404c1e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4051384640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c15eb00 0x7f404c1e60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51036/0 (socket says 172.21.15.26:51036) 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4051384640 1 -- 172.21.15.26:0/530761847 learned_addr learned my addr 172.21.15.26:0/530761847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f404bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f404c143c40 0x7f404c0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4050b83640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f404c0cc860 0x7f404c138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4051384640 1 -- 172.21.15.26:0/530761847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f404c0cc860 msgr2=0x7f404c138d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4051384640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f404c0cc860 0x7f404c138d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4051384640 1 -- 172.21.15.26:0/530761847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f404c143c40 msgr2=0x7f404c0afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4051384640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f404c143c40 0x7f404c0afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4051384640 1 -- 172.21.15.26:0/530761847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c0b0790 con 0x7f404c15eb00 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f404bfff640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f404c143c40 0x7f404c0afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.752+0000 7f4051384640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c15eb00 0x7f404c1e60a0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f404400cab0 tx=0x7f404400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.753+0000 7f40497fa640 1 -- 172.21.15.26:0/530761847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044017070 con 0x7f404c15eb00 2024-04-24T23:27:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.753+0000 7f40497fa640 1 -- 172.21.15.26:0/530761847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404400ac30 con 0x7f404c15eb00 2024-04-24T23:27:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.753+0000 7f40497fa640 1 -- 172.21.15.26:0/530761847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044012400 con 0x7f404c15eb00 2024-04-24T23:27:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.753+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c0b0a70 con 0x7f404c15eb00 2024-04-24T23:27:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.753+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c0b1030 con 0x7f404c15eb00 2024-04-24T23:27:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.754+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c0694c0 con 0x7f404c15eb00 2024-04-24T23:27:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.754+0000 7f40497fa640 1 -- 172.21.15.26:0/530761847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f404401a070 con 0x7f404c15eb00 2024-04-24T23:27:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.755+0000 7f40497fa640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f403006be30 0x7f403006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.755+0000 7f4050b83640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f403006be30 0x7f403006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.755+0000 7f40497fa640 1 -- 172.21.15.26:0/530761847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f404408a9b0 con 0x7f404c15eb00 2024-04-24T23:27:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.755+0000 7f4050b83640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f403006be30 0x7f403006e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4040034000 tx=0x7f404002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:58.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.759+0000 7f40497fa640 1 -- 172.21.15.26:0/530761847 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4044058d10 con 0x7f404c15eb00 2024-04-24T23:27:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.981+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f404c0b1200 con 0x7f404c15eb00 2024-04-24T23:27:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.981+0000 7f40497fa640 1 -- 172.21.15.26:0/530761847 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7f404405d010 con 0x7f404c15eb00 2024-04-24T23:27:58.982 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:58.982 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:27:58.983 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:27:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f403006be30 msgr2=0x7f403006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f403006be30 0x7f403006e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4040034000 tx=0x7f404002e000 comp rx=0 tx=0).stop 2024-04-24T23:27:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c15eb00 msgr2=0x7f404c1e60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c15eb00 0x7f404c1e60a0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f404400cab0 tx=0x7f404400cf70 comp rx=0 tx=0).stop 2024-04-24T23:27:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 shutdown_connections 2024-04-24T23:27:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f403006be30 0x7f403006e2f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c15eb00 0x7f404c1e60a0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f404c143c40 0x7f404c0afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 --2- 172.21.15.26:0/530761847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f404c0cc860 0x7f404c138d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.983+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 >> 172.21.15.26:0/530761847 conn(0x7f404c06b300 msgr2=0x7f404c0ae080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.984+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 shutdown_connections 2024-04-24T23:27:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:58.984+0000 7f4052de7640 1 -- 172.21.15.26:0/530761847 wait complete. 2024-04-24T23:27:58.995 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:27:58.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:27:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.095+0000 7fb63e15a640 1 Processor -- start 2024-04-24T23:27:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.095+0000 7fb63e15a640 1 -- start start 2024-04-24T23:27:59.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb63e15a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fb638145a80 0x7fb6380ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb63e15a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb63815ca40 0x7fb6380b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb63e15a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb63815cee0 con 0x7fb63815c5b0 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb63e15a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb6380b3300 con 0x7fb63815ca40 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb63e15a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fb63815d1c0 con 0x7fb638145a80 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb637fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb63815ca40 0x7fb6380b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb636ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb63815c5b0 0x7fb63806d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33966/0 (socket says 172.21.15.26:33966) 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb636ffd640 1 -- 172.21.15.26:0/4126761016 learned_addr learned my addr 172.21.15.26:0/4126761016 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb6377fe640 1 --2- 172.21.15.26:0/4126761016 >> v2:172.21.15.29:3300/0 conn(0x7fb638145a80 0x7fb6380ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb637fff640 1 -- 172.21.15.26:0/4126761016 >> v2:172.21.15.29:3300/0 conn(0x7fb638145a80 msgr2=0x7fb6380ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb637fff640 1 --2- 172.21.15.26:0/4126761016 >> v2:172.21.15.29:3300/0 conn(0x7fb638145a80 0x7fb6380ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb637fff640 1 -- 172.21.15.26:0/4126761016 >> v1:172.21.15.26:6789/0 conn(0x7fb63815c5b0 legacy=0x7fb63806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.096+0000 7fb637fff640 1 -- 172.21.15.26:0/4126761016 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63815d4a0 con 0x7fb63815ca40 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.097+0000 7fb637fff640 1 --2- 172.21.15.26:0/4126761016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63815ca40 0x7fb6380b3790 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fb63806f170 tx=0x7fb63002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d411755e3dee99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.097+0000 7fb6367fc640 1 -- 172.21.15.26:0/4126761016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630007740 con 0x7fb63815ca40 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.097+0000 7fb6367fc640 1 -- 172.21.15.26:0/4126761016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6300078e0 con 0x7fb63815ca40 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.097+0000 7fb6367fc640 1 -- 172.21.15.26:0/4126761016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630007cb0 con 0x7fb63815ca40 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.098+0000 7fb63e15a640 1 -- 172.21.15.26:0/4126761016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63815ca40 msgr2=0x7fb6380b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.098+0000 7fb63e15a640 1 --2- 172.21.15.26:0/4126761016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63815ca40 0x7fb6380b3790 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fb63806f170 tx=0x7fb63002a5f0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.098+0000 7fb63e15a640 1 -- 172.21.15.26:0/4126761016 shutdown_connections 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.098+0000 7fb63e15a640 1 --2- 172.21.15.26:0/4126761016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63815ca40 0x7fb6380b3790 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.098+0000 7fb63e15a640 1 --2- 172.21.15.26:0/4126761016 >> v2:172.21.15.29:3300/0 conn(0x7fb638145a80 0x7fb6380ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.098+0000 7fb63e15a640 1 -- 172.21.15.26:0/4126761016 >> 172.21.15.26:0/4126761016 conn(0x7fb63806b540 msgr2=0x7fb63806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.098+0000 7fb63e15a640 1 -- 172.21.15.26:0/4126761016 shutdown_connections 2024-04-24T23:27:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.098+0000 7fb63e15a640 1 -- 172.21.15.26:0/4126761016 wait complete. 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.099+0000 7fb63e15a640 1 Processor -- start 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.099+0000 7fb63e15a640 1 -- start start 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb63e15a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb638145a80 0x7fb63815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb63e15a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63815c5b0 0x7fb638071c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb63e15a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb63815ca40 0x7fb63813a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb63e15a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb63806ddc0 con 0x7fb638145a80 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb63e15a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb63806dc40 con 0x7fb63815c5b0 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb63e15a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb6380b3cd0 con 0x7fb63815ca40 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb6377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb638145a80 0x7fb63815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb6377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb638145a80 0x7fb63815b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51054/0 (socket says 172.21.15.26:51054) 2024-04-24T23:27:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb636ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63815c5b0 0x7fb638071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb6377fe640 1 -- 172.21.15.26:0/105911328 learned_addr learned my addr 172.21.15.26:0/105911328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb637fff640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb63815ca40 0x7fb63813a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb6377fe640 1 -- 172.21.15.26:0/105911328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb63815ca40 msgr2=0x7fb63813a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb6377fe640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb63815ca40 0x7fb63813a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb6377fe640 1 -- 172.21.15.26:0/105911328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63815c5b0 msgr2=0x7fb638071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb6377fe640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63815c5b0 0x7fb638071c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.100+0000 7fb6377fe640 1 -- 172.21.15.26:0/105911328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6381e6630 con 0x7fb638145a80 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.101+0000 7fb6377fe640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb638145a80 0x7fb63815b370 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fb62800d800 tx=0x7fb62800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.101+0000 7fb61ffff640 1 -- 172.21.15.26:0/105911328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62800b6c0 con 0x7fb638145a80 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.101+0000 7fb61ffff640 1 -- 172.21.15.26:0/105911328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb628019070 con 0x7fb638145a80 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.101+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6381e6910 con 0x7fb638145a80 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.101+0000 7fb61ffff640 1 -- 172.21.15.26:0/105911328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62800ba80 con 0x7fb638145a80 2024-04-24T23:27:59.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.101+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6381e6ed0 con 0x7fb638145a80 2024-04-24T23:27:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.102+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6381e7310 con 0x7fb638145a80 2024-04-24T23:27:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.102+0000 7fb61ffff640 1 -- 172.21.15.26:0/105911328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb62800bc20 con 0x7fb638145a80 2024-04-24T23:27:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.103+0000 7fb61ffff640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb62406be30 0x7fb62406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.103+0000 7fb61ffff640 1 -- 172.21.15.26:0/105911328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fb62808b160 con 0x7fb638145a80 2024-04-24T23:27:59.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.103+0000 7fb636ffd640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb62406be30 0x7fb62406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.103+0000 7fb636ffd640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb62406be30 0x7fb62406e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb62c005880 tx=0x7fb62c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.107+0000 7fb61ffff640 1 -- 172.21.15.26:0/105911328 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb628059550 con 0x7fb638145a80 2024-04-24T23:27:59.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.359+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb6380694c0 con 0x7fb638145a80 2024-04-24T23:27:59.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.360+0000 7fb61ffff640 1 -- 172.21.15.26:0/105911328 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1968 (secure 0 0 0) 0x7fb62805d850 con 0x7fb638145a80 2024-04-24T23:27:59.361 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:27:59.361 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":448,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700753,"bytes_used":287752192,"bytes_avail":772806361088,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":10152,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":113,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-24T23:27:57.408420+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:27:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb62406be30 msgr2=0x7fb62406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb62406be30 0x7fb62406e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb62c005880 tx=0x7fb62c009040 comp rx=0 tx=0).stop 2024-04-24T23:27:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb638145a80 msgr2=0x7fb63815b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb638145a80 0x7fb63815b370 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fb62800d800 tx=0x7fb62800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 shutdown_connections 2024-04-24T23:27:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb62406be30 0x7fb62406e2f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb63815ca40 0x7fb63813a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb63815c5b0 0x7fb638071c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 --2- 172.21.15.26:0/105911328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb638145a80 0x7fb63815b370 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 >> 172.21.15.26:0/105911328 conn(0x7fb63806b540 msgr2=0x7fb6381603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 shutdown_connections 2024-04-24T23:27:59.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.362+0000 7fb63e15a640 1 -- 172.21.15.26:0/105911328 wait complete. 2024-04-24T23:27:59.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:27:59.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:59.375 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub abort 2024-04-24T23:27:59.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.479+0000 7f6904ba2640 1 Processor -- start 2024-04-24T23:27:59.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.479+0000 7f6904ba2640 1 -- start start 2024-04-24T23:27:59.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f6904ba2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69000d0680 0x7f69000d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f6904ba2640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f69000d4b60 0x7f69000d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f6904ba2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69000cfe80 con 0x7f69000cec10 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f6904ba2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f69000cfc30 con 0x7f69000d0680 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f6904ba2640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f69000b6e90 con 0x7f69000d4b60 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f68fe59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69000d0680 0x7f69000d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f68fed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f69000cec10 0x7f69000d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33970/0 (socket says 172.21.15.26:33970) 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f68fed9d640 1 -- 172.21.15.26:0/2383966573 learned_addr learned my addr 172.21.15.26:0/2383966573 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f68fe59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69000d0680 0x7f69000d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51064/0 (socket says 172.21.15.26:51064) 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f68fdd9b640 1 --2- 172.21.15.26:0/2383966573 >> v2:172.21.15.29:3300/0 conn(0x7f69000d4b60 0x7f69000d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f68fe59c640 1 -- 172.21.15.26:0/2383966573 >> v2:172.21.15.29:3300/0 conn(0x7f69000d4b60 msgr2=0x7f69000d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f68fe59c640 1 --2- 172.21.15.26:0/2383966573 >> v2:172.21.15.29:3300/0 conn(0x7f69000d4b60 0x7f69000d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f68fe59c640 1 -- 172.21.15.26:0/2383966573 >> v1:172.21.15.26:6789/0 conn(0x7f69000cec10 legacy=0x7f69000d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.480+0000 7f68fe59c640 1 -- 172.21.15.26:0/2383966573 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69000b70e0 con 0x7f69000d0680 2024-04-24T23:27:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.481+0000 7f68fe59c640 1 --2- 172.21.15.26:0/2383966573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 0x7f69000d0a60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f690008c500 tx=0x7f68f002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3e7ed37af899e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.481+0000 7f68fd59a640 1 -- 172.21.15.26:0/2383966573 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0037070 con 0x7f69000d0680 2024-04-24T23:27:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.481+0000 7f68fd59a640 1 -- 172.21.15.26:0/2383966573 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f0032440 con 0x7f69000d0680 2024-04-24T23:27:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.481+0000 7f68fd59a640 1 -- 172.21.15.26:0/2383966573 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0032810 con 0x7f69000d0680 2024-04-24T23:27:59.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.482+0000 7f6904ba2640 1 -- 172.21.15.26:0/2383966573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 msgr2=0x7f69000d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.482+0000 7f6904ba2640 1 --2- 172.21.15.26:0/2383966573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 0x7f69000d0a60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f690008c500 tx=0x7f68f002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.482+0000 7f6904ba2640 1 -- 172.21.15.26:0/2383966573 shutdown_connections 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.482+0000 7f6904ba2640 1 --2- 172.21.15.26:0/2383966573 >> v2:172.21.15.29:3300/0 conn(0x7f69000d4b60 0x7f69000d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.482+0000 7f6904ba2640 1 --2- 172.21.15.26:0/2383966573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 0x7f69000d0a60 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.482+0000 7f6904ba2640 1 -- 172.21.15.26:0/2383966573 >> 172.21.15.26:0/2383966573 conn(0x7f69000d4040 msgr2=0x7f69000d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.482+0000 7f6904ba2640 1 -- 172.21.15.26:0/2383966573 shutdown_connections 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.482+0000 7f6904ba2640 1 -- 172.21.15.26:0/2383966573 wait complete. 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.483+0000 7f6904ba2640 1 Processor -- start 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.483+0000 7f6904ba2640 1 -- start start 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f6904ba2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69000cec10 0x7f6900150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f6904ba2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 0x7f69000a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f6904ba2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69000d4b60 0x7f69000a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f6904ba2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69000b8a70 con 0x7f69000d0680 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f6904ba2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f69000b88f0 con 0x7f69000d4b60 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f6904ba2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f69000b8bf0 con 0x7f69000cec10 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 0x7f69000a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 0x7f69000a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51070/0 (socket says 172.21.15.26:51070) 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fdd9b640 1 -- 172.21.15.26:0/2538745854 learned_addr learned my addr 172.21.15.26:0/2538745854 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fe59c640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69000cec10 0x7f6900150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fed9d640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69000d4b60 0x7f69000a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fdd9b640 1 -- 172.21.15.26:0/2538745854 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69000cec10 msgr2=0x7f6900150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fdd9b640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69000cec10 0x7f6900150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fdd9b640 1 -- 172.21.15.26:0/2538745854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69000d4b60 msgr2=0x7f69000a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fdd9b640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69000d4b60 0x7f69000a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fdd9b640 1 -- 172.21.15.26:0/2538745854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6900153f90 con 0x7f69000d0680 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.484+0000 7f68fed9d640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69000d4b60 0x7f69000a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.485+0000 7f68fdd9b640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 0x7f69000a0b80 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f68f400d5f0 tx=0x7f68f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.485+0000 7f68eeffd640 1 -- 172.21.15.26:0/2538745854 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f40125f0 con 0x7f69000d0680 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.485+0000 7f68eeffd640 1 -- 172.21.15.26:0/2538745854 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f4019070 con 0x7f69000d0680 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.485+0000 7f68eeffd640 1 -- 172.21.15.26:0/2538745854 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f40129d0 con 0x7f69000d0680 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.485+0000 7f6904ba2640 1 -- 172.21.15.26:0/2538745854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6900151240 con 0x7f69000d0680 2024-04-24T23:27:59.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.485+0000 7f6904ba2640 1 -- 172.21.15.26:0/2538745854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6900151800 con 0x7f69000d0680 2024-04-24T23:27:59.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.487+0000 7f68eeffd640 1 -- 172.21.15.26:0/2538745854 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68f4012b70 con 0x7f69000d0680 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.487+0000 7f68eeffd640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68e006be30 0x7f68e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.487+0000 7f68eeffd640 1 -- 172.21.15.26:0/2538745854 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f68f408b130 con 0x7f69000d0680 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.487+0000 7f68fe59c640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68e006be30 0x7f68e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.488+0000 7f68fe59c640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68e006be30 0x7f68e006e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f68f0034000 tx=0x7f68f002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.488+0000 7f6907993740 1 Processor -- start 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.488+0000 7f6907993740 1 -- start start 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f6907993740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252735b80 0x5562526e8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f6907993740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252718020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f6907993740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718560 0x5562526ed370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f6907993740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556252651330 con 0x556252718ca0 2024-04-24T23:27:59.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f6907993740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5562525f9ab0 con 0x556252735b80 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f6907993740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55625264d2f0 con 0x556252718560 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252718020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fdd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252718020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51086/0 (socket says 172.21.15.26:51086) 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fdd9b640 1 -- 172.21.15.26:0/783463792 learned_addr learned my addr 172.21.15.26:0/783463792 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fe59c640 1 --2- 172.21.15.26:0/783463792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252735b80 0x5562526e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fed9d640 1 --2- 172.21.15.26:0/783463792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718560 0x5562526ed370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fdd9b640 1 -- 172.21.15.26:0/783463792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718560 msgr2=0x5562526ed370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fdd9b640 1 --2- 172.21.15.26:0/783463792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718560 0x5562526ed370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fdd9b640 1 -- 172.21.15.26:0/783463792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252735b80 msgr2=0x5562526e8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fdd9b640 1 --2- 172.21.15.26:0/783463792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252735b80 0x5562526e8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.489+0000 7f68fdd9b640 1 -- 172.21.15.26:0/783463792 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55625242f7d0 con 0x556252718ca0 2024-04-24T23:27:59.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f68fdd9b640 1 --2- 172.21.15.26:0/783463792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252718020 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f68f4089660 tx=0x7f68f4059b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd91955e4b875875 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f68dffff640 1 -- 172.21.15.26:0/783463792 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4071020 con 0x556252718ca0 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f68dffff640 1 -- 172.21.15.26:0/783463792 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f405d820 con 0x556252718ca0 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f68dffff640 1 -- 172.21.15.26:0/783463792 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4062050 con 0x556252718ca0 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f6907993740 1 -- 172.21.15.26:0/783463792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 msgr2=0x556252718020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f6907993740 1 --2- 172.21.15.26:0/783463792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252718020 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f68f4089660 tx=0x7f68f4059b70 comp rx=0 tx=0).stop 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f6907993740 1 -- 172.21.15.26:0/783463792 shutdown_connections 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f6907993740 1 --2- 172.21.15.26:0/783463792 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718560 0x5562526ed370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f6907993740 1 --2- 172.21.15.26:0/783463792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252718020 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f6907993740 1 --2- 172.21.15.26:0/783463792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252735b80 0x5562526e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.490+0000 7f6907993740 1 -- 172.21.15.26:0/783463792 >> 172.21.15.26:0/783463792 conn(0x5562526ea780 msgr2=0x556252733020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.491+0000 7f6907993740 1 -- 172.21.15.26:0/783463792 shutdown_connections 2024-04-24T23:27:59.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.491+0000 7f6907993740 1 -- 172.21.15.26:0/783463792 wait complete. 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.491+0000 7f6907993740 1 -- start start 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.491+0000 7f6907993740 1 Processor -- start 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f6907993740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252718ca0 0x55625271ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f6907993740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252735b80 0x556252719f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f6907993740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55625271a4a0 0x556252713810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f6907993740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5562526ec930 con 0x55625271a4a0 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f6907993740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55625260cf40 con 0x556252718ca0 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f6907993740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5562526ebd20 con 0x556252735b80 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f6907993740 10 client.0 _ll_register_callbacks cb 0x556252364be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f68fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55625271a4a0 0x556252713810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f68fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55625271a4a0 0x556252713810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51088/0 (socket says 172.21.15.26:51088) 2024-04-24T23:27:59.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f68fed9d640 1 -- 172.21.15.26:0/3314716447 learned_addr learned my addr 172.21.15.26:0/3314716447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f68fdd9b640 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252735b80 0x556252719f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.492+0000 7f68fe59c640 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252718ca0 0x55625271ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68fed9d640 1 -- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252735b80 msgr2=0x556252719f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68fed9d640 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252735b80 0x556252719f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68fed9d640 1 -- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252718ca0 msgr2=0x55625271ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68fed9d640 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252718ca0 0x55625271ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68fed9d640 1 -- 172.21.15.26:0/3314716447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562523250f0 con 0x55625271a4a0 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68fed9d640 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55625271a4a0 0x556252713810 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f68f8011940 tx=0x7f68f8011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382d3b096249aa2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3314716447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f800fa80 con 0x55625271a4a0 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3314716447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f68f800fc20 con 0x55625271a4a0 2024-04-24T23:27:59.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f6907993740 1 -- 172.21.15.26:0/3314716447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f405d820 con 0x55625271a4a0 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3314716447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f68f8022050 con 0x55625271a4a0 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f6907993740 10 client.9234 fetch_fsmap learned FSMap version 113 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f6907993740 1 -- 172.21.15.26:0/3314716447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55625248ff60 con 0x55625271a4a0 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3314716447 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f405d820 con 0x55625271a4a0 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3314716447 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f68f8025020 con 0x55625271a4a0 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f6907993740 10 client.9234 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f6907993740 10 client.9234 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.493+0000 7f6907993740 20 client.9234 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.494+0000 7f6907993740 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55625272b7c0 0x556252734500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.494+0000 7f6907993740 4 client.9234 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.494+0000 7f6907993740 1 -- 172.21.15.26:0/3314716447 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55625265e870 con 0x55625272b7c0 2024-04-24T23:27:59.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.494+0000 7f68fe59c640 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55625272b7c0 0x556252734500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.494+0000 7f68fe59c640 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55625272b7c0 0x556252734500 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41aa33d593ba862b server_cookie=bade28f0b32df589 in_seq=0 out_seq=0 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.494+0000 7f68dd7fa640 10 client.9234 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3314716447 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55625265e870 con 0x55625272b7c0 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f68dd7fa640 10 client.9234 handle_command_reply: tid=0 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 client.9234 shutdown 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 -- 172.21.15.26:0/3314716447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55625271a4a0 msgr2=0x556252713810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55625271a4a0 0x556252713810 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f68f8011940 tx=0x7f68f8011e00 comp rx=0 tx=0).stop 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 -- 172.21.15.26:0/3314716447 shutdown_connections 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55625272b7c0 0x556252734500 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55625271a4a0 0x556252713810 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252735b80 0x556252719f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f68dd7fa640 0 client.9234 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:59.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 --2- 172.21.15.26:0/3314716447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556252718ca0 0x55625271ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 -- 172.21.15.26:0/3314716447 >> 172.21.15.26:0/3314716447 conn(0x556252718560 msgr2=0x556252736440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 -- 172.21.15.26:0/3314716447 shutdown_connections 2024-04-24T23:27:59.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 1 -- 172.21.15.26:0/3314716447 wait complete. 2024-04-24T23:27:59.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.495+0000 7f6907993740 20 client.9234 trim_cache size 0 max 16384 2024-04-24T23:27:59.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.512+0000 7f6907993740 1 Processor -- start 2024-04-24T23:27:59.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f6907993740 1 -- start start 2024-04-24T23:27:59.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f6907993740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252731e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f6907993740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55625271a4a0 0x556252736440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f6907993740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252735b80 0x556252736980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f6907993740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69002009a0 con 0x556252718ca0 2024-04-24T23:27:59.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f6907993740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f69001ffb40 con 0x55625271a4a0 2024-04-24T23:27:59.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f6907993740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f6900200270 con 0x556252735b80 2024-04-24T23:27:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f68fe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252731e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f68fe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252731e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51096/0 (socket says 172.21.15.26:51096) 2024-04-24T23:27:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.513+0000 7f68fe59c640 1 -- 172.21.15.26:0/3551497333 learned_addr learned my addr 172.21.15.26:0/3551497333 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.514+0000 7f68fe59c640 1 -- 172.21.15.26:0/3551497333 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252735b80 msgr2=0x556252736980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.514+0000 7f68fe59c640 1 --2- 172.21.15.26:0/3551497333 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252735b80 0x556252736980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.514+0000 7f68fe59c640 1 -- 172.21.15.26:0/3551497333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55625271a4a0 msgr2=0x556252736440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.514+0000 7f68fe59c640 1 --2- 172.21.15.26:0/3551497333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55625271a4a0 0x556252736440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.514+0000 7f68fe59c640 1 -- 172.21.15.26:0/3551497333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562522fe1b0 con 0x556252718ca0 2024-04-24T23:27:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.514+0000 7f68fe59c640 1 --2- 172.21.15.26:0/3551497333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252731e50 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009560 tx=0x7f68f002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940b5dc29c46051f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.514+0000 7f68df7fe640 1 -- 172.21.15.26:0/3551497333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0037040 con 0x556252718ca0 2024-04-24T23:27:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.514+0000 7f68df7fe640 1 -- 172.21.15.26:0/3551497333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f0056070 con 0x556252718ca0 2024-04-24T23:27:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.514+0000 7f68df7fe640 1 -- 172.21.15.26:0/3551497333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0051d50 con 0x556252718ca0 2024-04-24T23:27:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 -- 172.21.15.26:0/3551497333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 msgr2=0x556252731e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 --2- 172.21.15.26:0/3551497333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252731e50 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009560 tx=0x7f68f002c000 comp rx=0 tx=0).stop 2024-04-24T23:27:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 -- 172.21.15.26:0/3551497333 shutdown_connections 2024-04-24T23:27:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 --2- 172.21.15.26:0/3551497333 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252735b80 0x556252736980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 --2- 172.21.15.26:0/3551497333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55625271a4a0 0x556252736440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 --2- 172.21.15.26:0/3551497333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252718ca0 0x556252731e50 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 -- 172.21.15.26:0/3551497333 >> 172.21.15.26:0/3551497333 conn(0x556252718560 msgr2=0x55625271b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 -- 172.21.15.26:0/3551497333 shutdown_connections 2024-04-24T23:27:59.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 -- 172.21.15.26:0/3551497333 wait complete. 2024-04-24T23:27:59.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 -- start start 2024-04-24T23:27:59.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.515+0000 7f6907993740 1 Processor -- start 2024-04-24T23:27:59.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f6907993740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718ca0 0x5562527abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f6907993740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55625271a4a0 0x556252736440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f6907993740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252735b80 0x556252736980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f6907993740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556252729bd0 con 0x556252735b80 2024-04-24T23:27:59.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f6907993740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f69000cab00 con 0x55625271a4a0 2024-04-24T23:27:59.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f6907993740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5562525f46c0 con 0x556252718ca0 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f6907993740 10 client.0 _ll_register_callbacks cb 0x7f68f8001970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f68fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252735b80 0x556252736980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f68fed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252735b80 0x556252736980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51108/0 (socket says 172.21.15.26:51108) 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f68fed9d640 1 -- 172.21.15.26:0/3192408521 learned_addr learned my addr 172.21.15.26:0/3192408521 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f68fe59c640 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718ca0 0x5562527abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.516+0000 7f68fdd9b640 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55625271a4a0 0x556252736440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68fed9d640 1 -- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718ca0 msgr2=0x5562527abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68fed9d640 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718ca0 0x5562527abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68fed9d640 1 -- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55625271a4a0 msgr2=0x556252736440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68fed9d640 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55625271a4a0 0x556252736440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68fed9d640 1 -- 172.21.15.26:0/3192408521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562526e3790 con 0x556252735b80 2024-04-24T23:27:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68fed9d640 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252735b80 0x556252736980 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f68f8001720 tx=0x7f68f8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c28f83e5ae1852b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3192408521 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8015880 con 0x556252735b80 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3192408521 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f68f8015a20 con 0x556252735b80 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f6907993740 1 -- 172.21.15.26:0/3192408521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f0056070 con 0x556252735b80 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3192408521 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f68f800f5a0 con 0x556252735b80 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f6907993740 10 client.9240 fetch_fsmap learned FSMap version 113 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f6907993740 1 -- 172.21.15.26:0/3192408521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x5562527ac5d0 con 0x556252735b80 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.517+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3192408521 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f0056070 con 0x556252735b80 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3192408521 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f68f8025050 con 0x556252735b80 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f6907993740 10 client.9240 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f6907993740 10 client.9240 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f6907993740 20 client.9240 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f6907993740 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5562527293c0 0x556252719340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f6907993740 4 client.9240 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f6907993740 1 -- 172.21.15.26:0/3192408521 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x556252658bc0 con 0x5562527293c0 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f68fe59c640 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5562527293c0 0x556252719340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f68fe59c640 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5562527293c0 0x556252719340 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ca48c027b6af2e server_cookie=9e2b1856c5ebfbef in_seq=0 out_seq=0 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f68dd7fa640 10 client.9240 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.518+0000 7f68dd7fa640 1 -- 172.21.15.26:0/3192408521 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556252658bc0 con 0x5562527293c0 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f68dd7fa640 10 client.9240 handle_command_reply: tid=0 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 client.9240 shutdown 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 -- 172.21.15.26:0/3192408521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252735b80 msgr2=0x556252736980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252735b80 0x556252736980 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f68f8001720 tx=0x7f68f8001250 comp rx=0 tx=0).stop 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 -- 172.21.15.26:0/3192408521 shutdown_connections 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5562527293c0 0x556252719340 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556252735b80 0x556252736980 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55625271a4a0 0x556252736440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 --2- 172.21.15.26:0/3192408521 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x556252718ca0 0x5562527abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f68dd7fa640 0 client.9240 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 -- 172.21.15.26:0/3192408521 >> 172.21.15.26:0/3192408521 conn(0x556252718560 msgr2=0x5562526ebd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 -- 172.21.15.26:0/3192408521 shutdown_connections 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 1 -- 172.21.15.26:0/3192408521 wait complete. 2024-04-24T23:27:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6907993740 20 client.9240 trim_cache size 0 max 16384 2024-04-24T23:27:59.528 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:59.528 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:27:59.528 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:59.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 -- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68e006be30 msgr2=0x7f68e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68e006be30 0x7f68e006e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f68f0034000 tx=0x7f68f002e000 comp rx=0 tx=0).stop 2024-04-24T23:27:59.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 -- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 msgr2=0x7f69000a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 0x7f69000a0b80 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f68f400d5f0 tx=0x7f68f400dab0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 -- 172.21.15.26:0/2538745854 shutdown_connections 2024-04-24T23:27:59.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f68e006be30 0x7f68e006e2f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69000d4b60 0x7f69000a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69000d0680 0x7f69000a0b80 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 --2- 172.21.15.26:0/2538745854 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f69000cec10 0x7f6900150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.519+0000 7f6904ba2640 1 -- 172.21.15.26:0/2538745854 >> 172.21.15.26:0/2538745854 conn(0x7f69000d4040 msgr2=0x7f69000d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.520+0000 7f6904ba2640 1 -- 172.21.15.26:0/2538745854 shutdown_connections 2024-04-24T23:27:59.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.520+0000 7f6904ba2640 1 -- 172.21.15.26:0/2538745854 wait complete. 2024-04-24T23:27:59.531 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:27:59.531 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-04-24T23:27:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.636+0000 7f3513e9e640 1 Processor -- start 2024-04-24T23:27:59.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.636+0000 7f3513e9e640 1 -- start start 2024-04-24T23:27:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3513e9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c15e750 0x7f350c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3513e9e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f350c0d2c30 0x7f350c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3513e9e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350c0d2120 con 0x7f350c15e370 2024-04-24T23:27:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3513e9e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f350c0d1510 con 0x7f350c0d2c30 2024-04-24T23:27:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3513e9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f350c0d1ed0 con 0x7f350c15e750 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3511c3a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f350c15e370 0x7f350c0d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33982/0 (socket says 172.21.15.26:33982) 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3511c3a640 1 -- 172.21.15.26:0/1270706154 learned_addr learned my addr 172.21.15.26:0/1270706154 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3511439640 1 --2- 172.21.15.26:0/1270706154 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c15e750 0x7f350c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f351243b640 1 --2- 172.21.15.26:0/1270706154 >> v2:172.21.15.29:3300/0 conn(0x7f350c0d2c30 0x7f350c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3510c38640 1 -- 172.21.15.26:0/1270706154 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845346810 0 0) 0x7f350c0d2120 con 0x7f350c15e370 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.637+0000 7f3510c38640 1 -- 172.21.15.26:0/1270706154 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34f8003520 con 0x7f350c15e370 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.638+0000 7f3510c38640 1 -- 172.21.15.26:0/1270706154 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2914495489 0 0) 0x7f34f8003520 con 0x7f350c15e370 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.638+0000 7f3511439640 1 -- 172.21.15.26:0/1270706154 >> v2:172.21.15.29:3300/0 conn(0x7f350c0d2c30 msgr2=0x7f350c0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.638+0000 7f3511439640 1 --2- 172.21.15.26:0/1270706154 >> v2:172.21.15.29:3300/0 conn(0x7f350c0d2c30 0x7f350c0cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.638+0000 7f3511439640 1 -- 172.21.15.26:0/1270706154 >> v1:172.21.15.26:6789/0 conn(0x7f350c15e370 legacy=0x7f350c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.638+0000 7f3511439640 1 -- 172.21.15.26:0/1270706154 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c0b3740 con 0x7f350c15e750 2024-04-24T23:27:59.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.638+0000 7f3511439640 1 --2- 172.21.15.26:0/1270706154 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c15e750 0x7f350c0cec90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f350c0d2050 tx=0x7f34fc02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83a443759c02f30a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.639+0000 7f3510c38640 1 -- 172.21.15.26:0/1270706154 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc037070 con 0x7f350c15e750 2024-04-24T23:27:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.639+0000 7f3510c38640 1 -- 172.21.15.26:0/1270706154 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc02aba0 con 0x7f350c15e750 2024-04-24T23:27:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.639+0000 7f3510c38640 1 -- 172.21.15.26:0/1270706154 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508001670 con 0x7f350c15e750 2024-04-24T23:27:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.639+0000 7f3513e9e640 1 -- 172.21.15.26:0/1270706154 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c15e750 msgr2=0x7f350c0cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.639+0000 7f3513e9e640 1 --2- 172.21.15.26:0/1270706154 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c15e750 0x7f350c0cec90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f350c0d2050 tx=0x7f34fc02a110 comp rx=0 tx=0).stop 2024-04-24T23:27:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.639+0000 7f3513e9e640 1 -- 172.21.15.26:0/1270706154 shutdown_connections 2024-04-24T23:27:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.639+0000 7f3513e9e640 1 --2- 172.21.15.26:0/1270706154 >> v2:172.21.15.29:3300/0 conn(0x7f350c0d2c30 0x7f350c0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.639+0000 7f3513e9e640 1 --2- 172.21.15.26:0/1270706154 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c15e750 0x7f350c0cec90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.639+0000 7f3513e9e640 1 -- 172.21.15.26:0/1270706154 >> 172.21.15.26:0/1270706154 conn(0x7f350c0d40f0 msgr2=0x7f350c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.640+0000 7f3513e9e640 1 -- 172.21.15.26:0/1270706154 shutdown_connections 2024-04-24T23:27:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.640+0000 7f3513e9e640 1 -- 172.21.15.26:0/1270706154 wait complete. 2024-04-24T23:27:59.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.641+0000 7f3513e9e640 1 Processor -- start 2024-04-24T23:27:59.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.641+0000 7f3513e9e640 1 -- start start 2024-04-24T23:27:59.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.641+0000 7f3513e9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c0d2c30 0x7f350c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3513e9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f350c15e370 0x7f350c0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3513e9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f350c0b8eb0 0x7f350c0b9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3513e9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f350c0b5230 con 0x7f350c15e370 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3513e9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f350c0b3d00 con 0x7f350c0b8eb0 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3513e9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f350c0b53b0 con 0x7f350c0d2c30 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511439640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f350c15e370 0x7f350c0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511439640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f350c15e370 0x7f350c0ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51118/0 (socket says 172.21.15.26:51118) 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511439640 1 -- 172.21.15.26:0/3525235166 learned_addr learned my addr 172.21.15.26:0/3525235166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511c3a640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c0d2c30 0x7f350c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f351243b640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f350c0b8eb0 0x7f350c0b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511439640 1 -- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c0d2c30 msgr2=0x7f350c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511439640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c0d2c30 0x7f350c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511439640 1 -- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f350c0b8eb0 msgr2=0x7f350c0b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511439640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f350c0b8eb0 0x7f350c0b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511439640 1 -- 172.21.15.26:0/3525235166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c0cd330 con 0x7f350c15e370 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511c3a640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c0d2c30 0x7f350c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f351243b640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f350c0b8eb0 0x7f350c0b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f3511439640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f350c15e370 0x7f350c0ccdf0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009890 tx=0x7f34fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f35027fc640 1 -- 172.21.15.26:0/3525235166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc041020 con 0x7f350c15e370 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.642+0000 7f35027fc640 1 -- 172.21.15.26:0/3525235166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc03d640 con 0x7f350c15e370 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.643+0000 7f3513e9e640 1 -- 172.21.15.26:0/3525235166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350c0cb560 con 0x7f350c15e370 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.643+0000 7f35027fc640 1 -- 172.21.15.26:0/3525235166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc037040 con 0x7f350c15e370 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.643+0000 7f3513e9e640 1 -- 172.21.15.26:0/3525235166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350c0cbb20 con 0x7f350c15e370 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.644+0000 7f35027fc640 1 -- 172.21.15.26:0/3525235166 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34fc044020 con 0x7f350c15e370 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.645+0000 7f35027fc640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34f006be30 0x7f34f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.645+0000 7f35027fc640 1 -- 172.21.15.26:0/3525235166 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f34fc0ab3f0 con 0x7f350c15e370 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.645+0000 7f3511c3a640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34f006be30 0x7f34f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.645+0000 7f3511c3a640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34f006be30 0x7f34f006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3508005880 tx=0x7f3508009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.646+0000 7f3516bac740 1 Processor -- start 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.646+0000 7f3516bac740 1 -- start start 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3516bac740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d83f7e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3516bac740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f9bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3516bac740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f53a40 0x558d83f99cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3516bac740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558d83eb76b0 con 0x558d83f9b600 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3516bac740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558d83e5f880 con 0x558d83f53a40 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3516bac740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x558d83eb3670 con 0x558d83f7dd60 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511439640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f9bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511439640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f9bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51120/0 (socket says 172.21.15.26:51120) 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511439640 1 -- 172.21.15.26:0/749453182 learned_addr learned my addr 172.21.15.26:0/749453182 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511c3a640 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d83f7e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511439640 1 -- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 msgr2=0x558d83f7e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511439640 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d83f7e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f351243b640 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f53a40 0x558d83f99cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511439640 1 -- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f53a40 msgr2=0x558d83f99cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511439640 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f53a40 0x558d83f99cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511439640 1 -- 172.21.15.26:0/749453182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d83c957d0 con 0x558d83f9b600 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511c3a640 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d83f7e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f351243b640 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f53a40 0x558d83f99cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f3511439640 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f9bb50 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0a9940 tx=0x7f34fc009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997a06d2cec65af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f34ef7fe640 1 -- 172.21.15.26:0/749453182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc02aba0 con 0x558d83f9b600 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f34ef7fe640 1 -- 172.21.15.26:0/749453182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc079da0 con 0x558d83f9b600 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.647+0000 7f34ef7fe640 1 -- 172.21.15.26:0/749453182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0796a0 con 0x558d83f9b600 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.648+0000 7f3516bac740 1 -- 172.21.15.26:0/749453182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 msgr2=0x558d83f9bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.648+0000 7f3516bac740 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f9bb50 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0a9940 tx=0x7f34fc009520 comp rx=0 tx=0).stop 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.648+0000 7f3516bac740 1 -- 172.21.15.26:0/749453182 shutdown_connections 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.648+0000 7f3516bac740 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f53a40 0x558d83f99cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.648+0000 7f3516bac740 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f9bb50 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.648+0000 7f3516bac740 1 --2- 172.21.15.26:0/749453182 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d83f7e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.648+0000 7f3516bac740 1 -- 172.21.15.26:0/749453182 >> 172.21.15.26:0/749453182 conn(0x558d83f50600 msgr2=0x558d83f98b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.648+0000 7f3516bac740 1 -- 172.21.15.26:0/749453182 shutdown_connections 2024-04-24T23:27:59.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.648+0000 7f3516bac740 1 -- 172.21.15.26:0/749453182 wait complete. 2024-04-24T23:27:59.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.649+0000 7f3516bac740 1 -- start start 2024-04-24T23:27:59.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.649+0000 7f3516bac740 1 Processor -- start 2024-04-24T23:27:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3516bac740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f7dd60 0x558d83f910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3516bac740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 0x558d83fac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3516bac740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f92900 0x558d83f92d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3516bac740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558d83f547e0 con 0x558d83f7dd60 2024-04-24T23:27:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3516bac740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558d83e72f50 con 0x558d83f9b600 2024-04-24T23:27:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3516bac740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x558d83f4e530 con 0x558d83f92900 2024-04-24T23:27:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3516bac740 10 client.0 _ll_register_callbacks cb 0x558d83eae000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511c3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f7dd60 0x558d83f910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511c3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f7dd60 0x558d83f910a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51124/0 (socket says 172.21.15.26:51124) 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511c3a640 1 -- 172.21.15.26:0/595231507 learned_addr learned my addr 172.21.15.26:0/595231507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f351243b640 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f92900 0x558d83f92d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511439640 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 0x558d83fac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511c3a640 1 -- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f92900 msgr2=0x558d83f92d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511c3a640 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f92900 0x558d83f92d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511c3a640 1 -- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 msgr2=0x558d83fac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511c3a640 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 0x558d83fac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511c3a640 1 -- 172.21.15.26:0/595231507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d83bcb3f0 con 0x558d83f7dd60 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511439640 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 0x558d83fac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f351243b640 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f92900 0x558d83f92d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f3511c3a640 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f7dd60 0x558d83f910a0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3508007480 tx=0x7f35080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06f33a3a794fe0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f34ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f34ecff9640 1 -- 172.21.15.26:0/595231507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508012820 con 0x558d83f7dd60 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.650+0000 7f34ecff9640 1 -- 172.21.15.26:0/595231507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f3508020a80 con 0x558d83f7dd60 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3516bac740 1 -- 172.21.15.26:0/595231507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34fc079da0 con 0x558d83f7dd60 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f34ecff9640 1 -- 172.21.15.26:0/595231507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f3508020c80 con 0x558d83f7dd60 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3516bac740 10 client.9252 fetch_fsmap learned FSMap version 113 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3516bac740 1 -- 172.21.15.26:0/595231507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x558d83b8b0f0 con 0x558d83f7dd60 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f34ecff9640 1 -- 172.21.15.26:0/595231507 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc079da0 con 0x558d83f7dd60 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f34ecff9640 1 -- 172.21.15.26:0/595231507 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f35080129c0 con 0x558d83f7dd60 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3516bac740 10 client.9252 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3516bac740 10 client.9252 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3516bac740 20 client.9252 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3516bac740 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x558d83f7f620 0x558d83fa8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3516bac740 4 client.9252 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3516bac740 1 -- 172.21.15.26:0/595231507 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d83ec4bf0 con 0x558d83f7f620 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f3511439640 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x558d83f7f620 0x558d83fa8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.651+0000 7f34ecff9640 10 client.9252 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.652+0000 7f3511439640 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x558d83f7f620 0x558d83fa8f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c552aa74573703a server_cookie=dfb64ab4f2efad48 in_seq=0 out_seq=0 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.652+0000 7f34ecff9640 1 -- 172.21.15.26:0/595231507 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d83ec4bf0 con 0x558d83f7f620 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.652+0000 7f34ecff9640 10 client.9252 handle_command_reply: tid=0 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.652+0000 7f3516bac740 1 client.9252 shutdown 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 -- 172.21.15.26:0/595231507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f7dd60 msgr2=0x558d83f910a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f7dd60 0x558d83f910a0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3508007480 tx=0x7f35080057f0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 -- 172.21.15.26:0/595231507 shutdown_connections 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x558d83f7f620 0x558d83fa8f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f92900 0x558d83f92d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 0x558d83fac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f34ecff9640 0 client.9252 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 --2- 172.21.15.26:0/595231507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f7dd60 0x558d83f910a0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 -- 172.21.15.26:0/595231507 >> 172.21.15.26:0/595231507 conn(0x558d83f53a40 msgr2=0x558d83f9c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 -- 172.21.15.26:0/595231507 shutdown_connections 2024-04-24T23:27:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 1 -- 172.21.15.26:0/595231507 wait complete. 2024-04-24T23:27:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.653+0000 7f3516bac740 20 client.9252 trim_cache size 0 max 16384 2024-04-24T23:27:59.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.671+0000 7f3516bac740 1 Processor -- start 2024-04-24T23:27:59.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.671+0000 7f3516bac740 1 -- start start 2024-04-24T23:27:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.671+0000 7f3516bac740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d83f54f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.671+0000 7f3516bac740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f92900 0x558d83f52920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.671+0000 7f3516bac740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f52e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.671+0000 7f3516bac740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f350c200770 con 0x558d83f9b600 2024-04-24T23:27:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.671+0000 7f3516bac740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f350c1ff910 con 0x558d83f92900 2024-04-24T23:27:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.671+0000 7f3516bac740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f350c200040 con 0x558d83f7dd60 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f351243b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f52e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f351243b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f52e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51136/0 (socket says 172.21.15.26:51136) 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f351243b640 1 -- 172.21.15.26:0/3923285719 learned_addr learned my addr 172.21.15.26:0/3923285719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f3511c3a640 1 --2- 172.21.15.26:0/3923285719 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d83f54f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f3511439640 1 --2- 172.21.15.26:0/3923285719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f92900 0x558d83f52920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f351243b640 1 -- 172.21.15.26:0/3923285719 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 msgr2=0x558d83f54f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f351243b640 1 --2- 172.21.15.26:0/3923285719 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d83f54f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f351243b640 1 -- 172.21.15.26:0/3923285719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f92900 msgr2=0x558d83f52920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f351243b640 1 --2- 172.21.15.26:0/3923285719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f92900 0x558d83f52920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f351243b640 1 -- 172.21.15.26:0/3923285719 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d83f48f00 con 0x558d83f9b600 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f351243b640 1 --2- 172.21.15.26:0/3923285719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f52e60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f35040076b0 tx=0x7f3504009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20338009a9688393 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f34eeffd640 1 -- 172.21.15.26:0/3923285719 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350401b020 con 0x558d83f9b600 2024-04-24T23:27:59.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f34eeffd640 1 -- 172.21.15.26:0/3923285719 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3504017870 con 0x558d83f9b600 2024-04-24T23:27:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.672+0000 7f34eeffd640 1 -- 172.21.15.26:0/3923285719 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504017c20 con 0x558d83f9b600 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.673+0000 7f3516bac740 1 -- 172.21.15.26:0/3923285719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 msgr2=0x558d83f52e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.673+0000 7f3516bac740 1 --2- 172.21.15.26:0/3923285719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f52e60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f35040076b0 tx=0x7f3504009750 comp rx=0 tx=0).stop 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.673+0000 7f3516bac740 1 -- 172.21.15.26:0/3923285719 shutdown_connections 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.673+0000 7f3516bac740 1 --2- 172.21.15.26:0/3923285719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f9b600 0x558d83f52e60 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.673+0000 7f3516bac740 1 --2- 172.21.15.26:0/3923285719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f92900 0x558d83f52920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.673+0000 7f3516bac740 1 --2- 172.21.15.26:0/3923285719 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d83f54f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.673+0000 7f3516bac740 1 -- 172.21.15.26:0/3923285719 >> 172.21.15.26:0/3923285719 conn(0x558d83f53a40 msgr2=0x558d83fbca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.673+0000 7f3516bac740 1 -- 172.21.15.26:0/3923285719 shutdown_connections 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.673+0000 7f3516bac740 1 -- 172.21.15.26:0/3923285719 wait complete. 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.674+0000 7f3516bac740 1 -- start start 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.674+0000 7f3516bac740 1 Processor -- start 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3516bac740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d84011e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3516bac740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f92900 0x558d83f54e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3516bac740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 0x558d83f55350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3516bac740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558d83f54380 con 0x558d83f92900 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3516bac740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f350c0c86a0 con 0x558d83f9b600 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3516bac740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x558d83f905f0 con 0x558d83f7dd60 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3516bac740 10 client.0 _ll_register_callbacks cb 0x7f35080014e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511439640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f92900 0x558d83f54e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511439640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f92900 0x558d83f54e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51140/0 (socket says 172.21.15.26:51140) 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511439640 1 -- 172.21.15.26:0/3175978729 learned_addr learned my addr 172.21.15.26:0/3175978729 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511c3a640 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d84011e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f351243b640 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 0x558d83f55350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511439640 1 -- 172.21.15.26:0/3175978729 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 msgr2=0x558d84011e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511439640 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d84011e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511439640 1 -- 172.21.15.26:0/3175978729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 msgr2=0x558d83f55350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511439640 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 0x558d83f55350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511439640 1 -- 172.21.15.26:0/3175978729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d83f4d390 con 0x558d83f92900 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f3511439640 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f92900 0x558d83f54e10 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f34fc081d50 tx=0x7f34fc0ab000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635fdfac492d59e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.675+0000 7f34ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f34ecff9640 1 -- 172.21.15.26:0/3175978729 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc02aba0 con 0x558d83f92900 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f34ecff9640 1 -- 172.21.15.26:0/3175978729 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f34fc0ab0e0 con 0x558d83f92900 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3516bac740 1 -- 172.21.15.26:0/3175978729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3504017870 con 0x558d83f92900 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f34ecff9640 1 -- 172.21.15.26:0/3175978729 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f34fc079da0 con 0x558d83f92900 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3516bac740 10 client.9258 fetch_fsmap learned FSMap version 113 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3516bac740 1 -- 172.21.15.26:0/3175978729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x558d83f4dc30 con 0x558d83f92900 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f34ecff9640 1 -- 172.21.15.26:0/3175978729 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3504017870 con 0x558d83f92900 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f34ecff9640 1 -- 172.21.15.26:0/3175978729 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f34fc03a030 con 0x558d83f92900 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3516bac740 10 client.9258 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3516bac740 10 client.9258 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:27:59.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3516bac740 20 client.9258 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3516bac740 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x558d83fa8150 0x558d83faf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3516bac740 4 client.9258 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3516bac740 1 -- 172.21.15.26:0/3175978729 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d83f50f10 con 0x558d83fa8150 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.676+0000 7f3511c3a640 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x558d83fa8150 0x558d83faf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f34ecff9640 10 client.9258 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3511c3a640 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x558d83fa8150 0x558d83faf2f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4dc3ee79bddffd6 server_cookie=afba0c688023d126 in_seq=0 out_seq=0 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f34ecff9640 1 -- 172.21.15.26:0/3175978729 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x558d83f50f10 con 0x558d83fa8150 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f34ecff9640 10 client.9258 handle_command_reply: tid=0 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 client.9258 shutdown 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 -- 172.21.15.26:0/3175978729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f92900 msgr2=0x558d83f54e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f92900 0x558d83f54e10 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f34fc081d50 tx=0x7f34fc0ab000 comp rx=0 tx=0).stop 2024-04-24T23:27:59.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 -- 172.21.15.26:0/3175978729 shutdown_connections 2024-04-24T23:27:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x558d83fa8150 0x558d83faf2f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558d83f9b600 0x558d83f55350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f34ecff9640 0 client.9258 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:27:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558d83f92900 0x558d83f54e10 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 --2- 172.21.15.26:0/3175978729 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x558d83f7dd60 0x558d84011e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 -- 172.21.15.26:0/3175978729 >> 172.21.15.26:0/3175978729 conn(0x558d83f53a40 msgr2=0x558d83f7a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 -- 172.21.15.26:0/3175978729 shutdown_connections 2024-04-24T23:27:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 1 -- 172.21.15.26:0/3175978729 wait complete. 2024-04-24T23:27:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.677+0000 7f3516bac740 20 client.9258 trim_cache size 0 max 16384 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:59.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 -- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34f006be30 msgr2=0x7f34f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34f006be30 0x7f34f006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3508005880 tx=0x7f3508009040 comp rx=0 tx=0).stop 2024-04-24T23:27:59.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 -- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f350c15e370 msgr2=0x7f350c0ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f350c15e370 0x7f350c0ccdf0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009890 tx=0x7f34fc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 -- 172.21.15.26:0/3525235166 shutdown_connections 2024-04-24T23:27:59.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f34f006be30 0x7f34f006e2f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f350c0b8eb0 0x7f350c0b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f350c15e370 0x7f350c0ccdf0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 --2- 172.21.15.26:0/3525235166 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f350c0d2c30 0x7f350c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 -- 172.21.15.26:0/3525235166 >> 172.21.15.26:0/3525235166 conn(0x7f350c0d40f0 msgr2=0x7f350c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 -- 172.21.15.26:0/3525235166 shutdown_connections 2024-04-24T23:27:59.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.678+0000 7f3513e9e640 1 -- 172.21.15.26:0/3525235166 wait complete. 2024-04-24T23:27:59.689 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:27:59.689 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:27:59.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-04-24T23:27:59.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.795+0000 7f3531427640 1 Processor -- start 2024-04-24T23:27:59.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.795+0000 7f3531427640 1 -- start start 2024-04-24T23:27:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f3531427640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f352c15e750 0x7f352c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f3531427640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 0x7f352c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f3531427640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c0d2120 con 0x7f352c15e370 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f3531427640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f352c0d1510 con 0x7f352c15e750 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f3531427640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f352c0d1ed0 con 0x7f352c0d2c30 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f352affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f352c15e370 0x7f352c0d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33986/0 (socket says 172.21.15.26:33986) 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f352a7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f352c15e750 0x7f352c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f352affd640 1 -- 172.21.15.26:0/2950805508 learned_addr learned my addr 172.21.15.26:0/2950805508 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f352b7fe640 1 --2- 172.21.15.26:0/2950805508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 0x7f352c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f352a7fc640 1 -- 172.21.15.26:0/2950805508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 msgr2=0x7f352c0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f352a7fc640 1 --2- 172.21.15.26:0/2950805508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 0x7f352c0cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f352a7fc640 1 -- 172.21.15.26:0/2950805508 >> v1:172.21.15.26:6789/0 conn(0x7f352c15e370 legacy=0x7f352c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f352a7fc640 1 -- 172.21.15.26:0/2950805508 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c0b3740 con 0x7f352c15e750 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f3529ffb640 1 -- 172.21.15.26:0/2950805508 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575928225 0 0) 0x7f352c0d2120 con 0x7f352c15e370 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.796+0000 7f352b7fe640 1 --2- 172.21.15.26:0/2950805508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 0x7f352c0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.797+0000 7f352a7fc640 1 --2- 172.21.15.26:0/2950805508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e750 0x7f352c0cec90 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f35140097a0 tx=0x7f351402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee9c5812fa4cdb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.798+0000 7f3529ffb640 1 -- 172.21.15.26:0/2950805508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514037070 con 0x7f352c15e750 2024-04-24T23:27:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.798+0000 7f3529ffb640 1 -- 172.21.15.26:0/2950805508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3514032440 con 0x7f352c15e750 2024-04-24T23:27:59.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.798+0000 7f3529ffb640 1 -- 172.21.15.26:0/2950805508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514032810 con 0x7f352c15e750 2024-04-24T23:27:59.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.798+0000 7f3531427640 1 -- 172.21.15.26:0/2950805508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e750 msgr2=0x7f352c0cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.798+0000 7f3531427640 1 --2- 172.21.15.26:0/2950805508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e750 0x7f352c0cec90 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f35140097a0 tx=0x7f351402a180 comp rx=0 tx=0).stop 2024-04-24T23:27:59.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.799+0000 7f3531427640 1 -- 172.21.15.26:0/2950805508 shutdown_connections 2024-04-24T23:27:59.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.799+0000 7f3531427640 1 --2- 172.21.15.26:0/2950805508 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 0x7f352c0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.799+0000 7f3531427640 1 --2- 172.21.15.26:0/2950805508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e750 0x7f352c0cec90 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.799+0000 7f3531427640 1 -- 172.21.15.26:0/2950805508 >> 172.21.15.26:0/2950805508 conn(0x7f352c0d40f0 msgr2=0x7f352c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.799+0000 7f3531427640 1 -- 172.21.15.26:0/2950805508 shutdown_connections 2024-04-24T23:27:59.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.799+0000 7f3531427640 1 -- 172.21.15.26:0/2950805508 wait complete. 2024-04-24T23:27:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.800+0000 7f3531427640 1 Processor -- start 2024-04-24T23:27:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.800+0000 7f3531427640 1 -- start start 2024-04-24T23:27:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.801+0000 7f3531427640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 0x7f352c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.801+0000 7f3531427640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e370 0x7f352c0b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.801+0000 7f3531427640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f352c0b9270 0x7f352c1f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.801+0000 7f3531427640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f352c0b5230 con 0x7f352c15e370 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.801+0000 7f3531427640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f352c0b3d00 con 0x7f352c0b9270 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.801+0000 7f3531427640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f352c0b53b0 con 0x7f352c0d2c30 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e370 0x7f352c0b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e370 0x7f352c0b8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51160/0 (socket says 172.21.15.26:51160) 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352a7fc640 1 -- 172.21.15.26:0/2954153031 learned_addr learned my addr 172.21.15.26:0/2954153031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352affd640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 0x7f352c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352b7fe640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f352c0b9270 0x7f352c1f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352a7fc640 1 -- 172.21.15.26:0/2954153031 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 msgr2=0x7f352c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352a7fc640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 0x7f352c153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352a7fc640 1 -- 172.21.15.26:0/2954153031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f352c0b9270 msgr2=0x7f352c1f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352a7fc640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f352c0b9270 0x7f352c1f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352a7fc640 1 -- 172.21.15.26:0/2954153031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c1f4b10 con 0x7f352c15e370 2024-04-24T23:27:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352b7fe640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f352c0b9270 0x7f352c1f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:59.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f352a7fc640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e370 0x7f352c0b8d30 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3514035040 tx=0x7f35140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f35237fe640 1 -- 172.21.15.26:0/2954153031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514037040 con 0x7f352c15e370 2024-04-24T23:27:59.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f3531427640 1 -- 172.21.15.26:0/2954153031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352c0b71d0 con 0x7f352c15e370 2024-04-24T23:27:59.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f35237fe640 1 -- 172.21.15.26:0/2954153031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3514030770 con 0x7f352c15e370 2024-04-24T23:27:59.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f35237fe640 1 -- 172.21.15.26:0/2954153031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514030b80 con 0x7f352c15e370 2024-04-24T23:27:59.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.802+0000 7f3531427640 1 -- 172.21.15.26:0/2954153031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352c0b7790 con 0x7f352c15e370 2024-04-24T23:27:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.804+0000 7f35237fe640 1 -- 172.21.15.26:0/2954153031 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3514044020 con 0x7f352c15e370 2024-04-24T23:27:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.805+0000 7f35237fe640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f350c06be30 0x7f350c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.805+0000 7f35237fe640 1 -- 172.21.15.26:0/2954153031 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f35140ab3a0 con 0x7f352c15e370 2024-04-24T23:27:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.805+0000 7f352affd640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f350c06be30 0x7f350c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.805+0000 7f352affd640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f350c06be30 0x7f350c06e2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f351c000f60 tx=0x7f351c007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.806+0000 7f353420d740 1 Processor -- start 2024-04-24T23:27:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.806+0000 7f353420d740 1 -- start start 2024-04-24T23:27:59.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.807+0000 7f353420d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f5b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f353420d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f353420d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f58aa40 0x55c78f5d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f353420d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c78f4ee6b0 con 0x55c78f5d2600 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f353420d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c78f496880 con 0x55c78f5b4d60 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f353420d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c78f4ea670 con 0x55c78f58aa40 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5d2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51170/0 (socket says 172.21.15.26:51170) 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352a7fc640 1 -- 172.21.15.26:0/1606180642 learned_addr learned my addr 172.21.15.26:0/1606180642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352b7fe640 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f58aa40 0x55c78f5d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352affd640 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f5b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352a7fc640 1 -- 172.21.15.26:0/1606180642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f58aa40 msgr2=0x55c78f5d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352a7fc640 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f58aa40 0x55c78f5d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352a7fc640 1 -- 172.21.15.26:0/1606180642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 msgr2=0x55c78f5b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352a7fc640 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f5b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352a7fc640 1 -- 172.21.15.26:0/1606180642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78f2cc7d0 con 0x55c78f5d2600 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352affd640 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f5b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f352a7fc640 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5d2b50 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f351402a1c0 tx=0x7f35140791d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953c3ae2758d5198 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f3520ff9640 1 -- 172.21.15.26:0/1606180642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351407d980 con 0x55c78f5d2600 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f3520ff9640 1 -- 172.21.15.26:0/1606180642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35140821b0 con 0x55c78f5d2600 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.808+0000 7f3520ff9640 1 -- 172.21.15.26:0/1606180642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514030d20 con 0x55c78f5d2600 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.809+0000 7f353420d740 1 -- 172.21.15.26:0/1606180642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 msgr2=0x55c78f5d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.809+0000 7f353420d740 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5d2b50 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f351402a1c0 tx=0x7f35140791d0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.809+0000 7f353420d740 1 -- 172.21.15.26:0/1606180642 shutdown_connections 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.809+0000 7f353420d740 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f58aa40 0x55c78f5d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.809+0000 7f353420d740 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5d2b50 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.809+0000 7f353420d740 1 --2- 172.21.15.26:0/1606180642 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f5b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.809+0000 7f353420d740 1 -- 172.21.15.26:0/1606180642 >> 172.21.15.26:0/1606180642 conn(0x55c78f587600 msgr2=0x55c78f5cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.809+0000 7f353420d740 1 -- 172.21.15.26:0/1606180642 shutdown_connections 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.809+0000 7f353420d740 1 -- 172.21.15.26:0/1606180642 wait complete. 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.810+0000 7f353420d740 1 -- start start 2024-04-24T23:27:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.810+0000 7f353420d740 1 Processor -- start 2024-04-24T23:27:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f353420d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5b4d60 0x55c78f5b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f353420d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f353420d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5c7160 0x55c78f5b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f353420d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c78f58b7e0 con 0x55c78f5d2600 2024-04-24T23:27:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f353420d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c78f4a9f50 con 0x55c78f5c7160 2024-04-24T23:27:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f353420d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c78f585530 con 0x55c78f5b4d60 2024-04-24T23:27:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51172/0 (socket says 172.21.15.26:51172) 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352a7fc640 1 -- 172.21.15.26:0/2604034517 learned_addr learned my addr 172.21.15.26:0/2604034517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352affd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5b4d60 0x55c78f5b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f353420d740 10 client.0 _ll_register_callbacks cb 0x55c78f4e5000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352b7fe640 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5c7160 0x55c78f5b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352a7fc640 1 -- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5b4d60 msgr2=0x55c78f5b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352a7fc640 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5b4d60 0x55c78f5b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352a7fc640 1 -- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5c7160 msgr2=0x55c78f5b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352a7fc640 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5c7160 0x55c78f5b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352a7fc640 1 -- 172.21.15.26:0/2604034517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78f2023f0 con 0x55c78f5d2600 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f352a7fc640 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5c6c20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f35140a98a0 tx=0x7f3514009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b470a789ce4fd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f3509ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f3509ffb640 1 -- 172.21.15.26:0/2604034517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514030d20 con 0x55c78f5d2600 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f3509ffb640 1 -- 172.21.15.26:0/2604034517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f35140ab0b0 con 0x55c78f5d2600 2024-04-24T23:27:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f353420d740 1 -- 172.21.15.26:0/2604034517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35140821b0 con 0x55c78f5d2600 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.811+0000 7f3509ffb640 1 -- 172.21.15.26:0/2604034517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f3514075c40 con 0x55c78f5d2600 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f353420d740 10 client.9273 fetch_fsmap learned FSMap version 113 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f353420d740 1 -- 172.21.15.26:0/2604034517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55c78f1c20f0 con 0x55c78f5d2600 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f3509ffb640 1 -- 172.21.15.26:0/2604034517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3514079830 con 0x55c78f5d2600 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f3509ffb640 1 -- 172.21.15.26:0/2604034517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f351403a030 con 0x55c78f5d2600 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f353420d740 10 client.9273 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f353420d740 10 client.9273 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f353420d740 20 client.9273 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f353420d740 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55c78f5e3c50 0x55c78f5e4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f353420d740 4 client.9273 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f353420d740 1 -- 172.21.15.26:0/2604034517 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f35140821b0 con 0x55c78f5e3c50 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.812+0000 7f352affd640 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55c78f5e3c50 0x55c78f5e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.813+0000 7f352affd640 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55c78f5e3c50 0x55c78f5e4060 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21fa7c41fffe573b server_cookie=b574ae93a912b874 in_seq=0 out_seq=0 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.813+0000 7f3509ffb640 10 client.9273 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.813+0000 7f3509ffb640 1 -- 172.21.15.26:0/2604034517 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f35140821b0 con 0x55c78f5e3c50 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.813+0000 7f3509ffb640 10 client.9273 handle_command_reply: tid=0 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.813+0000 7f353420d740 1 client.9273 shutdown 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 -- 172.21.15.26:0/2604034517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 msgr2=0x55c78f5c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5c6c20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f35140a98a0 tx=0x7f3514009190 comp rx=0 tx=0).stop 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 -- 172.21.15.26:0/2604034517 shutdown_connections 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55c78f5e3c50 0x55c78f5e4060 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5c7160 0x55c78f5b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5d2600 0x55c78f5c6c20 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 --2- 172.21.15.26:0/2604034517 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5b4d60 0x55c78f5b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f3509ffb640 0 client.9273 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 -- 172.21.15.26:0/2604034517 >> 172.21.15.26:0/2604034517 conn(0x55c78f58aa40 msgr2=0x55c78f5d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 -- 172.21.15.26:0/2604034517 shutdown_connections 2024-04-24T23:27:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 1 -- 172.21.15.26:0/2604034517 wait complete. 2024-04-24T23:27:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.814+0000 7f353420d740 20 client.9273 trim_cache size 0 max 16384 2024-04-24T23:27:59.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.832+0000 7f353420d740 1 Processor -- start 2024-04-24T23:27:59.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.832+0000 7f353420d740 1 -- start start 2024-04-24T23:27:59.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f353420d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f58bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f353420d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f5e4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f353420d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 0x55c78f5b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f353420d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f352c200780 con 0x55c78f5c7160 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f353420d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1ff920 con 0x55c78f5b4d60 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f353420d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f352c200050 con 0x55c78f5d2600 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f5e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f5e4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51180/0 (socket says 172.21.15.26:51180) 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352a7fc640 1 -- 172.21.15.26:0/4156530795 learned_addr learned my addr 172.21.15.26:0/4156530795 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352b7fe640 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 0x55c78f5b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352affd640 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f58bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352a7fc640 1 -- 172.21.15.26:0/4156530795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 msgr2=0x55c78f5b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352a7fc640 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 0x55c78f5b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352a7fc640 1 -- 172.21.15.26:0/4156530795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 msgr2=0x55c78f58bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352a7fc640 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f58bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352a7fc640 1 -- 172.21.15.26:0/4156530795 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78f5b54b0 con 0x55c78f5c7160 2024-04-24T23:27:59.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.833+0000 7f352affd640 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f58bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:59.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f352a7fc640 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f5e4060 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f351403b040 tx=0x7f3514008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb95d673e64163b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f350bfff640 1 -- 172.21.15.26:0/4156530795 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351409c020 con 0x55c78f5c7160 2024-04-24T23:27:59.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f350bfff640 1 -- 172.21.15.26:0/4156530795 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35140821b0 con 0x55c78f5c7160 2024-04-24T23:27:59.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f350bfff640 1 -- 172.21.15.26:0/4156530795 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514075de0 con 0x55c78f5c7160 2024-04-24T23:27:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f353420d740 1 -- 172.21.15.26:0/4156530795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 msgr2=0x55c78f5e4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f353420d740 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f5e4060 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f351403b040 tx=0x7f3514008000 comp rx=0 tx=0).stop 2024-04-24T23:27:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f353420d740 1 -- 172.21.15.26:0/4156530795 shutdown_connections 2024-04-24T23:27:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f353420d740 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 0x55c78f5b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f353420d740 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f5e4060 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.834+0000 7f353420d740 1 --2- 172.21.15.26:0/4156530795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f58bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.835+0000 7f353420d740 1 -- 172.21.15.26:0/4156530795 >> 172.21.15.26:0/4156530795 conn(0x55c78f58aa40 msgr2=0x55c78f589f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.835+0000 7f353420d740 1 -- 172.21.15.26:0/4156530795 shutdown_connections 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.835+0000 7f353420d740 1 -- 172.21.15.26:0/4156530795 wait complete. 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.835+0000 7f353420d740 1 -- start start 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.835+0000 7f353420d740 1 Processor -- start 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f353420d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f589390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f353420d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f649020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f353420d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 0x55c78f589ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f353420d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c78f5b54b0 con 0x55c78f5c7160 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f353420d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f352c0c89c0 con 0x55c78f5b4d60 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f353420d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55c78f5851b0 con 0x55c78f5d2600 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f353420d740 10 client.0 _ll_register_callbacks cb 0x7f3514090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f649020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f649020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51188/0 (socket says 172.21.15.26:51188) 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352a7fc640 1 -- 172.21.15.26:0/1030507507 learned_addr learned my addr 172.21.15.26:0/1030507507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352affd640 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f589390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352b7fe640 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 0x55c78f589ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352a7fc640 1 -- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 msgr2=0x55c78f589ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352a7fc640 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 0x55c78f589ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352a7fc640 1 -- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 msgr2=0x55c78f589390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352a7fc640 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f589390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352a7fc640 1 -- 172.21.15.26:0/1030507507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78f5f3c10 con 0x55c78f5c7160 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352affd640 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f589390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352b7fe640 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 0x55c78f589ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f352a7fc640 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f649020 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f351407db40 tx=0x7f3514009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d22aac2e985e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.836+0000 7f3509ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:59.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f3509ffb640 1 -- 172.21.15.26:0/1030507507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351409c070 con 0x55c78f5c7160 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f3509ffb640 1 -- 172.21.15.26:0/1030507507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f3514079a40 con 0x55c78f5c7160 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f353420d740 1 -- 172.21.15.26:0/1030507507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35140821b0 con 0x55c78f5c7160 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f3509ffb640 1 -- 172.21.15.26:0/1030507507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f3514075de0 con 0x55c78f5c7160 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f353420d740 10 client.9279 fetch_fsmap learned FSMap version 113 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f353420d740 1 -- 172.21.15.26:0/1030507507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55c78f5f3c10 con 0x55c78f5c7160 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f3509ffb640 1 -- 172.21.15.26:0/1030507507 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35140821b0 con 0x55c78f5c7160 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f3509ffb640 1 -- 172.21.15.26:0/1030507507 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f3514078a20 con 0x55c78f5c7160 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f353420d740 10 client.9279 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f353420d740 10 client.9279 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f353420d740 20 client.9279 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f353420d740 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55c78f649560 0x55c78f58a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f353420d740 4 client.9279 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f353420d740 1 -- 172.21.15.26:0/1030507507 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c78f4fbbf0 con 0x55c78f649560 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.837+0000 7f352affd640 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55c78f649560 0x55c78f58a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f3509ffb640 10 client.9279 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f352affd640 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55c78f649560 0x55c78f58a3e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6661f7fa3a384d87 server_cookie=1c9e2d3736dcd78d in_seq=0 out_seq=0 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f3509ffb640 1 -- 172.21.15.26:0/1030507507 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55c78f4fbbf0 con 0x55c78f649560 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f3509ffb640 10 client.9279 handle_command_reply: tid=0 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 client.9279 shutdown 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 -- 172.21.15.26:0/1030507507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 msgr2=0x55c78f649020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f649020 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f351407db40 tx=0x7f3514009190 comp rx=0 tx=0).stop 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 -- 172.21.15.26:0/1030507507 shutdown_connections 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55c78f649560 0x55c78f58a3e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55c78f5d2600 0x55c78f589ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c78f5c7160 0x55c78f649020 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 --2- 172.21.15.26:0/1030507507 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c78f5b4d60 0x55c78f589390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f3509ffb640 0 client.9279 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 -- 172.21.15.26:0/1030507507 >> 172.21.15.26:0/1030507507 conn(0x55c78f58aa40 msgr2=0x55c78f5c7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 -- 172.21.15.26:0/1030507507 shutdown_connections 2024-04-24T23:27:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 1 -- 172.21.15.26:0/1030507507 wait complete. 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.838+0000 7f353420d740 20 client.9279 trim_cache size 0 max 16384 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:27:59.845 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:27:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 -- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f350c06be30 msgr2=0x7f350c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f350c06be30 0x7f350c06e2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f351c000f60 tx=0x7f351c007d50 comp rx=0 tx=0).stop 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 -- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e370 msgr2=0x7f352c0b8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e370 0x7f352c0b8d30 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3514035040 tx=0x7f35140046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 -- 172.21.15.26:0/2954153031 shutdown_connections 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f350c06be30 0x7f350c06e2f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f352c0b9270 0x7f352c1f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352c15e370 0x7f352c0b8d30 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 --2- 172.21.15.26:0/2954153031 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f352c0d2c30 0x7f352c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 -- 172.21.15.26:0/2954153031 >> 172.21.15.26:0/2954153031 conn(0x7f352c0d40f0 msgr2=0x7f352c06d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 -- 172.21.15.26:0/2954153031 shutdown_connections 2024-04-24T23:27:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.839+0000 7f3531427640 1 -- 172.21.15.26:0/2954153031 wait complete. 2024-04-24T23:27:59.850 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:27:59.851 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:27:59.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-24T23:27:59.899 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:27:59.901 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:27:59.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.954+0000 7f452849a640 1 Processor -- start 2024-04-24T23:27:59.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.954+0000 7f452849a640 1 -- start start 2024-04-24T23:27:59.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.955+0000 7f452849a640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f452015e370 0x7f45200d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.955+0000 7f452849a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45200d4c10 0x7f45200cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.955+0000 7f452849a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45200d2120 con 0x7f452015e750 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.955+0000 7f452849a640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f45200d1510 con 0x7f452015e370 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.955+0000 7f452849a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f45200d1ed0 con 0x7f45200d4c10 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.955+0000 7f4525a35640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f452015e750 0x7f45200b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33992/0 (socket says 172.21.15.26:33992) 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4525a35640 1 -- 172.21.15.26:0/2507644823 learned_addr learned my addr 172.21.15.26:0/2507644823 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4526a37640 1 --2- 172.21.15.26:0/2507644823 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45200d4c10 0x7f45200cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4526236640 1 --2- 172.21.15.26:0/2507644823 >> v2:172.21.15.29:3300/0 conn(0x7f452015e370 0x7f45200d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4525234640 1 -- 172.21.15.26:0/2507644823 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151602177 0 0) 0x7f45200d2120 con 0x7f452015e750 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4525234640 1 -- 172.21.15.26:0/2507644823 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f450c003520 con 0x7f452015e750 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4526a37640 1 -- 172.21.15.26:0/2507644823 >> v2:172.21.15.29:3300/0 conn(0x7f452015e370 msgr2=0x7f45200d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4526a37640 1 --2- 172.21.15.26:0/2507644823 >> v2:172.21.15.29:3300/0 conn(0x7f452015e370 0x7f45200d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4526a37640 1 -- 172.21.15.26:0/2507644823 >> v1:172.21.15.26:6789/0 conn(0x7f452015e750 legacy=0x7f45200b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4526a37640 1 -- 172.21.15.26:0/2507644823 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45200b3740 con 0x7f45200d4c10 2024-04-24T23:27:59.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.956+0000 7f4526236640 1 --2- 172.21.15.26:0/2507644823 >> v2:172.21.15.29:3300/0 conn(0x7f452015e370 0x7f45200d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:59.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.957+0000 7f4526a37640 1 --2- 172.21.15.26:0/2507644823 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45200d4c10 0x7f45200cfca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4518009d90 tx=0x7f451802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be2b0296fad7cef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.957+0000 7f4525234640 1 -- 172.21.15.26:0/2507644823 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518007740 con 0x7f45200d4c10 2024-04-24T23:27:59.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.957+0000 7f4525234640 1 -- 172.21.15.26:0/2507644823 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45180078e0 con 0x7f45200d4c10 2024-04-24T23:27:59.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.957+0000 7f4525234640 1 -- 172.21.15.26:0/2507644823 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518007cb0 con 0x7f45200d4c10 2024-04-24T23:27:59.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.958+0000 7f452849a640 1 -- 172.21.15.26:0/2507644823 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45200d4c10 msgr2=0x7f45200cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.958+0000 7f452849a640 1 --2- 172.21.15.26:0/2507644823 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45200d4c10 0x7f45200cfca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4518009d90 tx=0x7f451802a660 comp rx=0 tx=0).stop 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.958+0000 7f452849a640 1 -- 172.21.15.26:0/2507644823 shutdown_connections 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.958+0000 7f452849a640 1 --2- 172.21.15.26:0/2507644823 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f45200d4c10 0x7f45200cfca0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.958+0000 7f452849a640 1 --2- 172.21.15.26:0/2507644823 >> v2:172.21.15.29:3300/0 conn(0x7f452015e370 0x7f45200d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.958+0000 7f452849a640 1 -- 172.21.15.26:0/2507644823 >> 172.21.15.26:0/2507644823 conn(0x7f45200d40f0 msgr2=0x7f45200d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.958+0000 7f452849a640 1 -- 172.21.15.26:0/2507644823 shutdown_connections 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.958+0000 7f452849a640 1 -- 172.21.15.26:0/2507644823 wait complete. 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.959+0000 7f452849a640 1 Processor -- start 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.959+0000 7f452849a640 1 -- start start 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f452849a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45200d4c10 0x7f45200b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f452849a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f452015e370 0x7f452015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f452849a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4520160a20 0x7f45200b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f452849a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45200d2a60 con 0x7f452015e370 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f452849a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f45200b3d00 con 0x7f45200d4c10 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f452849a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f45200d2be0 con 0x7f4520160a20 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4525a35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f452015e370 0x7f452015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4525a35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f452015e370 0x7f452015b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51200/0 (socket says 172.21.15.26:51200) 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4526a37640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4520160a20 0x7f45200b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4525a35640 1 -- 172.21.15.26:0/4128347111 learned_addr learned my addr 172.21.15.26:0/4128347111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4526236640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45200d4c10 0x7f45200b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4525a35640 1 -- 172.21.15.26:0/4128347111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4520160a20 msgr2=0x7f45200b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4525a35640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4520160a20 0x7f45200b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4525a35640 1 -- 172.21.15.26:0/4128347111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45200d4c10 msgr2=0x7f45200b95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4525a35640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45200d4c10 0x7f45200b95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4525a35640 1 -- 172.21.15.26:0/4128347111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452015fd00 con 0x7f452015e370 2024-04-24T23:27:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.960+0000 7f4525a35640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f452015e370 0x7f452015b690 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f45100010e0 tx=0x7f451000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.961+0000 7f4516ffd640 1 -- 172.21.15.26:0/4128347111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510017070 con 0x7f452015e370 2024-04-24T23:27:59.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.961+0000 7f4516ffd640 1 -- 172.21.15.26:0/4128347111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4510005d00 con 0x7f452015e370 2024-04-24T23:27:59.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.961+0000 7f452849a640 1 -- 172.21.15.26:0/4128347111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4520160f20 con 0x7f452015e370 2024-04-24T23:27:59.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.961+0000 7f4516ffd640 1 -- 172.21.15.26:0/4128347111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510007310 con 0x7f452015e370 2024-04-24T23:27:59.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.961+0000 7f452849a640 1 -- 172.21.15.26:0/4128347111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201614e0 con 0x7f452015e370 2024-04-24T23:27:59.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.962+0000 7f4516ffd640 1 -- 172.21.15.26:0/4128347111 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f451001a070 con 0x7f452015e370 2024-04-24T23:27:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.963+0000 7f4516ffd640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f450406be30 0x7f450406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.963+0000 7f4516ffd640 1 -- 172.21.15.26:0/4128347111 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f451008a210 con 0x7f452015e370 2024-04-24T23:27:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.963+0000 7f4526236640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f450406be30 0x7f450406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.963+0000 7f4526236640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f450406be30 0x7f450406e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f451c009560 tx=0x7f451c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.965+0000 7f452b1a5740 1 Processor -- start 2024-04-24T23:27:59.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.965+0000 7f452b1a5740 1 -- start start 2024-04-24T23:27:59.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.966+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 0x55662a55a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.966+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a577b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.966+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a52fa40 0x55662a575cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.966+0000 7f452b1a5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55662a4936b0 con 0x55662a52fa40 2024-04-24T23:27:59.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.966+0000 7f452b1a5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55662a43b880 con 0x55662a559d60 2024-04-24T23:27:59.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.966+0000 7f452b1a5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55662a48f670 con 0x55662a577600 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526a37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a52fa40 0x55662a575cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526a37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a52fa40 0x55662a575cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51216/0 (socket says 172.21.15.26:51216) 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526a37640 1 -- 172.21.15.26:0/2851794463 learned_addr learned my addr 172.21.15.26:0/2851794463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4525a35640 1 --2- 172.21.15.26:0/2851794463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a577b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526236640 1 --2- 172.21.15.26:0/2851794463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 0x55662a55a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526a37640 1 -- 172.21.15.26:0/2851794463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 msgr2=0x55662a577b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526a37640 1 --2- 172.21.15.26:0/2851794463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a577b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526a37640 1 -- 172.21.15.26:0/2851794463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 msgr2=0x55662a55a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526a37640 1 --2- 172.21.15.26:0/2851794463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 0x55662a55a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526a37640 1 -- 172.21.15.26:0/2851794463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662a2717d0 con 0x55662a52fa40 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4526a37640 1 --2- 172.21.15.26:0/2851794463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a52fa40 0x55662a575cc0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4518009d90 tx=0x7f451802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4831299606b0549 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4503fff640 1 -- 172.21.15.26:0/2851794463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518007740 con 0x55662a52fa40 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4503fff640 1 -- 172.21.15.26:0/2851794463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45180078e0 con 0x55662a52fa40 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.967+0000 7f4503fff640 1 -- 172.21.15.26:0/2851794463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518007c90 con 0x55662a52fa40 2024-04-24T23:27:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.968+0000 7f452b1a5740 1 -- 172.21.15.26:0/2851794463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a52fa40 msgr2=0x55662a575cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.968+0000 7f452b1a5740 1 --2- 172.21.15.26:0/2851794463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a52fa40 0x55662a575cc0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4518009d90 tx=0x7f451802e000 comp rx=0 tx=0).stop 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.968+0000 7f452b1a5740 1 -- 172.21.15.26:0/2851794463 shutdown_connections 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.968+0000 7f452b1a5740 1 --2- 172.21.15.26:0/2851794463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a52fa40 0x55662a575cc0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.968+0000 7f452b1a5740 1 --2- 172.21.15.26:0/2851794463 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a577b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.968+0000 7f452b1a5740 1 --2- 172.21.15.26:0/2851794463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 0x55662a55a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.968+0000 7f452b1a5740 1 -- 172.21.15.26:0/2851794463 >> 172.21.15.26:0/2851794463 conn(0x55662a52c600 msgr2=0x55662a574b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.968+0000 7f452b1a5740 1 -- 172.21.15.26:0/2851794463 shutdown_connections 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.968+0000 7f452b1a5740 1 -- 172.21.15.26:0/2851794463 wait complete. 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.969+0000 7f452b1a5740 1 -- start start 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.969+0000 7f452b1a5740 1 Processor -- start 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a55bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a585290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 0x55662a56e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f452b1a5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55662a5307e0 con 0x55662a559d60 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f452b1a5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55662a44ef50 con 0x55662a556920 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f452b1a5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55662a52a530 con 0x55662a577600 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f452b1a5740 10 client.0 _ll_register_callbacks cb 0x55662a48a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4526236640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a55bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4526236640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a55bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51224/0 (socket says 172.21.15.26:51224) 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4526236640 1 -- 172.21.15.26:0/1058867377 learned_addr learned my addr 172.21.15.26:0/1058867377 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4525a35640 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a585290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4526a37640 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 0x55662a56e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4526236640 1 -- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 msgr2=0x55662a585290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4526236640 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a585290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4526236640 1 -- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 msgr2=0x55662a56e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4526236640 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 0x55662a56e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.970+0000 7f4526236640 1 -- 172.21.15.26:0/1058867377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662a1a73f0 con 0x55662a559d60 2024-04-24T23:27:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f4526236640 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a55bcf0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f451c011fb0 tx=0x7f451c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cd656f2a48ccad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f45017fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f45017fa640 1 -- 172.21.15.26:0/1058867377 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c027070 con 0x55662a559d60 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f45017fa640 1 -- 172.21.15.26:0/1058867377 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f451c014440 con 0x55662a559d60 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f452b1a5740 1 -- 172.21.15.26:0/1058867377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45180078e0 con 0x55662a559d60 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f45017fa640 1 -- 172.21.15.26:0/1058867377 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f451c032050 con 0x55662a559d60 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f452b1a5740 10 client.9291 fetch_fsmap learned FSMap version 113 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f452b1a5740 1 -- 172.21.15.26:0/1058867377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55662a1670f0 con 0x55662a559d60 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f45017fa640 1 -- 172.21.15.26:0/1058867377 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45180078e0 con 0x55662a559d60 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f45017fa640 1 -- 172.21.15.26:0/1058867377 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f451c035020 con 0x55662a559d60 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f452b1a5740 10 client.9291 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f452b1a5740 10 client.9291 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f452b1a5740 20 client.9291 populate_metadata read hostname 'smithi026' 2024-04-24T23:27:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f452b1a5740 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55662a56c400 0x55662a588880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f452b1a5740 4 client.9291 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f452b1a5740 1 -- 172.21.15.26:0/1058867377 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55662a4a0bf0 con 0x55662a56c400 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.971+0000 7f4525a35640 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55662a56c400 0x55662a588880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.972+0000 7f4525a35640 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55662a56c400 0x55662a588880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288c2cc3736da7e7 server_cookie=17193c2d971706ff in_seq=0 out_seq=0 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.972+0000 7f45017fa640 10 client.9291 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.972+0000 7f45017fa640 1 -- 172.21.15.26:0/1058867377 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55662a4a0bf0 con 0x55662a56c400 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.972+0000 7f45017fa640 10 client.9291 handle_command_reply: tid=0 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.972+0000 7f452b1a5740 1 client.9291 shutdown 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 -- 172.21.15.26:0/1058867377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 msgr2=0x55662a55bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a55bcf0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f451c011fb0 tx=0x7f451c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 -- 172.21.15.26:0/1058867377 shutdown_connections 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55662a56c400 0x55662a588880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 0x55662a56e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a585290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f45017fa640 0 client.9291 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:27:59.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 --2- 172.21.15.26:0/1058867377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a55bcf0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:59.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 -- 172.21.15.26:0/1058867377 >> 172.21.15.26:0/1058867377 conn(0x55662a52fa40 msgr2=0x55662a5780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:59.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 -- 172.21.15.26:0/1058867377 shutdown_connections 2024-04-24T23:27:59.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 1 -- 172.21.15.26:0/1058867377 wait complete. 2024-04-24T23:27:59.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:27:59.973+0000 7f452b1a5740 20 client.9291 trim_cache size 0 max 16384 2024-04-24T23:28:00.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.008+0000 7f452b1a5740 1 Processor -- start 2024-04-24T23:28:00.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.008+0000 7f452b1a5740 1 -- start start 2024-04-24T23:28:00.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.008+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 0x55662a585b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.008+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a598d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.008+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a599260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.008+0000 7f452b1a5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4520200450 con 0x55662a559d60 2024-04-24T23:28:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.008+0000 7f452b1a5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f45201ff670 con 0x55662a556920 2024-04-24T23:28:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.008+0000 7f452b1a5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f45201ffd20 con 0x55662a577600 2024-04-24T23:28:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4525a35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a598d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4525a35640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a598d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51226/0 (socket says 172.21.15.26:51226) 2024-04-24T23:28:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4525a35640 1 -- 172.21.15.26:0/2155733207 learned_addr learned my addr 172.21.15.26:0/2155733207 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4526a37640 1 --2- 172.21.15.26:0/2155733207 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a599260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4526236640 1 --2- 172.21.15.26:0/2155733207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 0x55662a585b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4525a35640 1 -- 172.21.15.26:0/2155733207 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 msgr2=0x55662a599260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4525a35640 1 --2- 172.21.15.26:0/2155733207 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a599260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4525a35640 1 -- 172.21.15.26:0/2155733207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 msgr2=0x55662a585b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4525a35640 1 --2- 172.21.15.26:0/2155733207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 0x55662a585b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4525a35640 1 -- 172.21.15.26:0/2155733207 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662a5705c0 con 0x55662a559d60 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f4525a35640 1 --2- 172.21.15.26:0/2155733207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a598d20 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f451008a1c0 tx=0x7f451001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1e242a6b3aa28b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f45037fe640 1 -- 172.21.15.26:0/2155733207 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100567d0 con 0x55662a559d60 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f45037fe640 1 -- 172.21.15.26:0/2155733207 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4510070020 con 0x55662a559d60 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.009+0000 7f45037fe640 1 -- 172.21.15.26:0/2155733207 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100074b0 con 0x55662a559d60 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 -- 172.21.15.26:0/2155733207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 msgr2=0x55662a598d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 --2- 172.21.15.26:0/2155733207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a598d20 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f451008a1c0 tx=0x7f451001e040 comp rx=0 tx=0).stop 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 -- 172.21.15.26:0/2155733207 shutdown_connections 2024-04-24T23:28:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 --2- 172.21.15.26:0/2155733207 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a577600 0x55662a599260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 --2- 172.21.15.26:0/2155733207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a559d60 0x55662a598d20 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 --2- 172.21.15.26:0/2155733207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a556920 0x55662a585b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 -- 172.21.15.26:0/2155733207 >> 172.21.15.26:0/2155733207 conn(0x55662a52fa40 msgr2=0x55662a585120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 -- 172.21.15.26:0/2155733207 shutdown_connections 2024-04-24T23:28:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 -- 172.21.15.26:0/2155733207 wait complete. 2024-04-24T23:28:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.010+0000 7f452b1a5740 1 -- start start 2024-04-24T23:28:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.011+0000 7f452b1a5740 1 Processor -- start 2024-04-24T23:28:00.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.011+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a556920 0x55662a5eb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.011+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 0x55662a52f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f452b1a5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a577600 0x55662a529260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f452b1a5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55662a52aea0 con 0x55662a577600 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f452b1a5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f45201f7ed0 con 0x55662a559d60 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f452b1a5740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55662a5eecf0 con 0x55662a556920 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f452b1a5740 10 client.0 _ll_register_callbacks cb 0x7f451c011a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526a37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a577600 0x55662a529260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526a37640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a577600 0x55662a529260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51238/0 (socket says 172.21.15.26:51238) 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526a37640 1 -- 172.21.15.26:0/231256994 learned_addr learned my addr 172.21.15.26:0/231256994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526236640 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a556920 0x55662a5eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4525a35640 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 0x55662a52f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526a37640 1 -- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a556920 msgr2=0x55662a5eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526a37640 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a556920 0x55662a5eb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526a37640 1 -- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 msgr2=0x55662a52f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526a37640 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 0x55662a52f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526a37640 1 -- 172.21.15.26:0/231256994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662a56c990 con 0x55662a577600 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f4526a37640 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a577600 0x55662a529260 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4518000f80 tx=0x7f4518004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f7c658101fba2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f45017fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f45017fa640 1 -- 172.21.15.26:0/231256994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451802ace0 con 0x55662a577600 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f45017fa640 1 -- 172.21.15.26:0/231256994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f4518004580 con 0x55662a577600 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.012+0000 7f452b1a5740 1 -- 172.21.15.26:0/231256994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4510070020 con 0x55662a577600 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f45017fa640 1 -- 172.21.15.26:0/231256994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f451805e050 con 0x55662a577600 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f452b1a5740 10 client.9297 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f452b1a5740 1 -- 172.21.15.26:0/231256994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55662a56a600 con 0x55662a577600 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f45017fa640 1 -- 172.21.15.26:0/231256994 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4510070020 con 0x55662a577600 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f45017fa640 1 -- 172.21.15.26:0/231256994 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f4518061020 con 0x55662a577600 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f452b1a5740 10 client.9297 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f452b1a5740 10 client.9297 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f452b1a5740 20 client.9297 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f452b1a5740 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55662a5297a0 0x55662a5ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f452b1a5740 4 client.9297 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f452b1a5740 1 -- 172.21.15.26:0/231256994 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55662a49af40 con 0x55662a5297a0 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.013+0000 7f4526236640 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55662a5297a0 0x55662a5ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f45017fa640 10 client.9297 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f4526236640 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55662a5297a0 0x55662a5ee100 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb60861b2ec67af server_cookie=df28fd5efb844fdb in_seq=0 out_seq=0 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f45017fa640 1 -- 172.21.15.26:0/231256994 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55662a49af40 con 0x55662a5297a0 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f45017fa640 10 client.9297 handle_command_reply: tid=0 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 client.9297 shutdown 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 -- 172.21.15.26:0/231256994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a577600 msgr2=0x55662a529260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a577600 0x55662a529260 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4518000f80 tx=0x7f4518004ab0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 -- 172.21.15.26:0/231256994 shutdown_connections 2024-04-24T23:28:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55662a5297a0 0x55662a5ee100 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55662a577600 0x55662a529260 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55662a559d60 0x55662a52f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f45017fa640 0 client.9297 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 --2- 172.21.15.26:0/231256994 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55662a556920 0x55662a5eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 -- 172.21.15.26:0/231256994 >> 172.21.15.26:0/231256994 conn(0x55662a52fa40 msgr2=0x55662a555d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 -- 172.21.15.26:0/231256994 shutdown_connections 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 1 -- 172.21.15.26:0/231256994 wait complete. 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.014+0000 7f452b1a5740 20 client.9297 trim_cache size 0 max 16384 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:00.021 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:28:00.022 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:28:00.022 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:28:00.022 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:28:00.022 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:28:00.022 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:28:00.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:00.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:00.022 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 -- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f450406be30 msgr2=0x7f450406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f450406be30 0x7f450406e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f451c009560 tx=0x7f451c009190 comp rx=0 tx=0).stop 2024-04-24T23:28:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 -- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f452015e370 msgr2=0x7f452015b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f452015e370 0x7f452015b690 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f45100010e0 tx=0x7f451000c890 comp rx=0 tx=0).stop 2024-04-24T23:28:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 -- 172.21.15.26:0/4128347111 shutdown_connections 2024-04-24T23:28:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f450406be30 0x7f450406e2f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4520160a20 0x7f45200b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f452015e370 0x7f452015b690 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 --2- 172.21.15.26:0/4128347111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f45200d4c10 0x7f45200b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 -- 172.21.15.26:0/4128347111 >> 172.21.15.26:0/4128347111 conn(0x7f45200d40f0 msgr2=0x7f45200b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 -- 172.21.15.26:0/4128347111 shutdown_connections 2024-04-24T23:28:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.015+0000 7f452849a640 1 -- 172.21.15.26:0/4128347111 wait complete. 2024-04-24T23:28:00.026 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:28:00.026 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:28:00.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-04-24T23:28:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.169+0000 7f248508c640 1 Processor -- start 2024-04-24T23:28:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.169+0000 7f248508c640 1 -- start start 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f248508c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f24800d2c30 0x7f24800cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f248508c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24800d2120 con 0x7f248015e370 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f248508c640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24800b3740 con 0x7f248015e750 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f248508c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f24800d1510 con 0x7f24800d2c30 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f248015e370 0x7f24800d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33996/0 (socket says 172.21.15.26:33996) 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247ed9d640 1 -- 172.21.15.26:0/1545683366 learned_addr learned my addr 172.21.15.26:0/1545683366 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3846135827 0 0) 0x7f24800d2120 con 0x7f248015e370 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2468003520 con 0x7f248015e370 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247f59e640 1 --2- 172.21.15.26:0/1545683366 >> v2:172.21.15.29:3300/0 conn(0x7f24800d2c30 0x7f24800cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3511829286 0 0) 0x7f2468003520 con 0x7f248015e370 2024-04-24T23:28:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 >> v2:172.21.15.29:3300/0 conn(0x7f24800d2c30 msgr2=0x7f24800cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247dd9b640 1 --2- 172.21.15.26:0/1545683366 >> v2:172.21.15.29:3300/0 conn(0x7f24800d2c30 0x7f24800cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 >> v1:172.21.15.29:6789/0 conn(0x7f248015e750 legacy=0x7f24800cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24800b3a20 con 0x7f248015e370 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.170+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f2474001670 con 0x7f248015e370 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 >> v1:172.21.15.26:6789/0 conn(0x7f248015e370 legacy=0x7f24800d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247dd9b640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f24680039b0 0x7f2468003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247dd9b640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2468004800 0x7f2468004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247dd9b640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24680051c0 0x7f246800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f246800c030 con 0x7f24680051c0 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f246800c1e0 con 0x7f2468004800 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f246800c3e0 con 0x7f24680039b0 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247f59e640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24680051c0 0x7f246800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247ed9d640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f24680039b0 0x7f2468003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247e59c640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2468004800 0x7f2468004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247f59e640 1 -- 172.21.15.26:0/1545683366 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f24680039b0 msgr2=0x7f2468003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247f59e640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f24680039b0 0x7f2468003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247f59e640 1 -- 172.21.15.26:0/1545683366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2468004800 msgr2=0x7f2468004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247f59e640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2468004800 0x7f2468004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.171+0000 7f247f59e640 1 -- 172.21.15.26:0/1545683366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2468003520 con 0x7f24680051c0 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.172+0000 7f247f59e640 1 -- 172.21.15.26:0/1545683366 reap_dead start 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.172+0000 7f247f59e640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24680051c0 0x7f246800b9c0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f2478009fc0 tx=0x7f2468005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3cc7b1c1c2418f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.172+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478035020 con 0x7f24680051c0 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.172+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2478003bd0 con 0x7f24680051c0 2024-04-24T23:28:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.172+0000 7f247dd9b640 1 -- 172.21.15.26:0/1545683366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478009c10 con 0x7f24680051c0 2024-04-24T23:28:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.173+0000 7f248508c640 1 -- 172.21.15.26:0/1545683366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24680051c0 msgr2=0x7f246800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.173+0000 7f248508c640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24680051c0 0x7f246800b9c0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f2478009fc0 tx=0x7f2468005830 comp rx=0 tx=0).stop 2024-04-24T23:28:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.173+0000 7f248508c640 1 -- 172.21.15.26:0/1545683366 shutdown_connections 2024-04-24T23:28:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.173+0000 7f248508c640 1 --2- 172.21.15.26:0/1545683366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24680051c0 0x7f246800b9c0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.173+0000 7f248508c640 1 -- 172.21.15.26:0/1545683366 >> 172.21.15.26:0/1545683366 conn(0x7f24800d40f0 msgr2=0x7f24800d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.173+0000 7f248508c640 1 -- 172.21.15.26:0/1545683366 shutdown_connections 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.173+0000 7f248508c640 1 -- 172.21.15.26:0/1545683366 wait complete. 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.174+0000 7f248508c640 1 Processor -- start 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.174+0000 7f248508c640 1 -- start start 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f248508c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24680051c0 0x7f24800c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f248508c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f24801f7c40 0x7f248015a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f248508c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24801f7310 0x7f2480149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f248508c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24800d4c10 con 0x7f24801f7310 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f248508c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f24800d1ed0 con 0x7f24680051c0 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f248508c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f24800d4d90 con 0x7f24801f7c40 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f247f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24801f7310 0x7f2480149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f247f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24801f7310 0x7f2480149400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51246/0 (socket says 172.21.15.26:51246) 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f247f59e640 1 -- 172.21.15.26:0/476484211 learned_addr learned my addr 172.21.15.26:0/476484211 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f247e59c640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f24801f7c40 0x7f248015a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.175+0000 7f247ed9d640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24680051c0 0x7f24800c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f247f59e640 1 -- 172.21.15.26:0/476484211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f24801f7c40 msgr2=0x7f248015a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f247f59e640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f24801f7c40 0x7f248015a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f247f59e640 1 -- 172.21.15.26:0/476484211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24680051c0 msgr2=0x7f24800c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f247f59e640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24680051c0 0x7f24800c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f247f59e640 1 -- 172.21.15.26:0/476484211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248015abe0 con 0x7f24801f7310 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f247f59e640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24801f7310 0x7f2480149400 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f247802af10 tx=0x7f2478009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f24677fe640 1 -- 172.21.15.26:0/476484211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478040070 con 0x7f24801f7310 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f24677fe640 1 -- 172.21.15.26:0/476484211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2478032e80 con 0x7f24801f7310 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f248508c640 1 -- 172.21.15.26:0/476484211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24800cab50 con 0x7f24801f7310 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f24677fe640 1 -- 172.21.15.26:0/476484211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478035030 con 0x7f24801f7310 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.176+0000 7f248508c640 1 -- 172.21.15.26:0/476484211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24800ca870 con 0x7f24801f7310 2024-04-24T23:28:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.178+0000 7f24677fe640 1 -- 172.21.15.26:0/476484211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2478043070 con 0x7f24801f7310 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.178+0000 7f24677fe640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be60 0x7f246c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.179+0000 7f24677fe640 1 -- 172.21.15.26:0/476484211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f24780abde0 con 0x7f24801f7310 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.179+0000 7f247ed9d640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be60 0x7f246c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.179+0000 7f247ed9d640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be60 0x7f246c06e320 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f248006c2e0 tx=0x7f24740078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.179+0000 7f2487d98740 1 Processor -- start 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.179+0000 7f2487d98740 1 -- start start 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f2487d98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdbb0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f2487d98740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdbcdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f2487d98740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdb85a40 0x563fcdbcbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f2487d98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563fcdae96b0 con 0x563fcdbafd60 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f2487d98740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563fcda91880 con 0x563fcdb85a40 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f2487d98740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563fcdae5670 con 0x563fcdbcd600 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdbb0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdbb0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51252/0 (socket says 172.21.15.26:51252) 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247ed9d640 1 -- 172.21.15.26:0/2939752082 learned_addr learned my addr 172.21.15.26:0/2939752082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247e59c640 1 --2- 172.21.15.26:0/2939752082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdbcdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247f59e640 1 --2- 172.21.15.26:0/2939752082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdb85a40 0x563fcdbcbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247ed9d640 1 -- 172.21.15.26:0/2939752082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 msgr2=0x563fcdbcdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247ed9d640 1 --2- 172.21.15.26:0/2939752082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdbcdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247ed9d640 1 -- 172.21.15.26:0/2939752082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdb85a40 msgr2=0x563fcdbcbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247ed9d640 1 --2- 172.21.15.26:0/2939752082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdb85a40 0x563fcdbcbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.180+0000 7f247ed9d640 1 -- 172.21.15.26:0/2939752082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fcd8c77d0 con 0x563fcdbafd60 2024-04-24T23:28:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.181+0000 7f247ed9d640 1 --2- 172.21.15.26:0/2939752082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdbb0140 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f2474007f70 tx=0x7f247400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c991e917ee780830 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.181+0000 7f2464ff9640 1 -- 172.21.15.26:0/2939752082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474028070 con 0x563fcdbafd60 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.181+0000 7f2464ff9640 1 -- 172.21.15.26:0/2939752082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24740159c0 con 0x563fcdbafd60 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.181+0000 7f2464ff9640 1 -- 172.21.15.26:0/2939752082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474015d70 con 0x563fcdbafd60 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 -- 172.21.15.26:0/2939752082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 msgr2=0x563fcdbb0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 --2- 172.21.15.26:0/2939752082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdbb0140 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f2474007f70 tx=0x7f247400c000 comp rx=0 tx=0).stop 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 -- 172.21.15.26:0/2939752082 shutdown_connections 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 --2- 172.21.15.26:0/2939752082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdb85a40 0x563fcdbcbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 --2- 172.21.15.26:0/2939752082 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdbcdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 --2- 172.21.15.26:0/2939752082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdbb0140 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 -- 172.21.15.26:0/2939752082 >> 172.21.15.26:0/2939752082 conn(0x563fcdb82600 msgr2=0x563fcdbcab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 -- 172.21.15.26:0/2939752082 shutdown_connections 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 -- 172.21.15.26:0/2939752082 wait complete. 2024-04-24T23:28:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 -- start start 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.182+0000 7f2487d98740 1 Processor -- start 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f2487d98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdba16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f2487d98740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdbb2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f2487d98740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbc1c20 0x563fcdbdb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f2487d98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563fcdb867e0 con 0x563fcdbafd60 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f2487d98740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563fcdaa4f50 con 0x563fcdbc1c20 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f2487d98740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563fcdb80530 con 0x563fcdbcd600 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f2487d98740 10 client.0 _ll_register_callbacks cb 0x563fcdae0000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f247ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdba16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f247ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdba16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51268/0 (socket says 172.21.15.26:51268) 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f247ed9d640 1 -- 172.21.15.26:0/3397145399 learned_addr learned my addr 172.21.15.26:0/3397145399 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f247f59e640 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbc1c20 0x563fcdbdb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.183+0000 7f247e59c640 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdbb2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f247ed9d640 1 -- 172.21.15.26:0/3397145399 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 msgr2=0x563fcdbb2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f247ed9d640 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdbb2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f247ed9d640 1 -- 172.21.15.26:0/3397145399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbc1c20 msgr2=0x563fcdbdb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f247ed9d640 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbc1c20 0x563fcdbdb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f247ed9d640 1 -- 172.21.15.26:0/3397145399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fcd7fd3f0 con 0x563fcdbafd60 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f247ed9d640 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdba16e0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f2474000ff0 tx=0x7f24800d22f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95186fe69a8a00f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f245dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f245dffb640 1 -- 172.21.15.26:0/3397145399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474032060 con 0x563fcdbafd60 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f245dffb640 1 -- 172.21.15.26:0/3397145399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f24740247f0 con 0x563fcdbafd60 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f2487d98740 1 -- 172.21.15.26:0/3397145399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24740159c0 con 0x563fcdbafd60 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f245dffb640 1 -- 172.21.15.26:0/3397145399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f2474038050 con 0x563fcdbafd60 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f2487d98740 10 client.9309 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f2487d98740 1 -- 172.21.15.26:0/3397145399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x563fcd7bd0f0 con 0x563fcdbafd60 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f245dffb640 1 -- 172.21.15.26:0/3397145399 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24740159c0 con 0x563fcdbafd60 2024-04-24T23:28:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f245dffb640 1 -- 172.21.15.26:0/3397145399 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f247403b020 con 0x563fcdbafd60 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f2487d98740 10 client.9309 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f2487d98740 10 client.9309 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f2487d98740 20 client.9309 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.184+0000 7f2487d98740 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x563fcdbb0b80 0x563fcdbb0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.185+0000 7f2487d98740 4 client.9309 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.185+0000 7f2487d98740 1 -- 172.21.15.26:0/3397145399 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fcdaf6bf0 con 0x563fcdbb0b80 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.185+0000 7f247e59c640 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x563fcdbb0b80 0x563fcdbb0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.185+0000 7f247e59c640 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x563fcdbb0b80 0x563fcdbb0ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd685e3ee868eecf server_cookie=4b64a09249427422 in_seq=0 out_seq=0 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.185+0000 7f245dffb640 10 client.9309 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.185+0000 7f245dffb640 1 -- 172.21.15.26:0/3397145399 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fcdaf6bf0 con 0x563fcdbb0b80 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.185+0000 7f245dffb640 10 client.9309 handle_command_reply: tid=0 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 client.9309 shutdown 2024-04-24T23:28:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 -- 172.21.15.26:0/3397145399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 msgr2=0x563fcdba16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdba16e0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f2474000ff0 tx=0x7f24800d22f0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 -- 172.21.15.26:0/3397145399 shutdown_connections 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x563fcdbb0b80 0x563fcdbb0ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbc1c20 0x563fcdbdb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdbb2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f245dffb640 0 client.9309 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 --2- 172.21.15.26:0/3397145399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdba16e0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 -- 172.21.15.26:0/3397145399 >> 172.21.15.26:0/3397145399 conn(0x563fcdb85a40 msgr2=0x563fcdbce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 -- 172.21.15.26:0/3397145399 shutdown_connections 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 1 -- 172.21.15.26:0/3397145399 wait complete. 2024-04-24T23:28:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.186+0000 7f2487d98740 20 client.9309 trim_cache size 0 max 16384 2024-04-24T23:28:00.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f2487d98740 1 Processor -- start 2024-04-24T23:28:00.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f2487d98740 1 -- start start 2024-04-24T23:28:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f2487d98740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbafd60 0x563fcdb7f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f2487d98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbc1c20 0x563fcdb83910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f2487d98740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdb83e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f2487d98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2480200ef0 con 0x563fcdbc1c20 2024-04-24T23:28:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f2487d98740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2480200090 con 0x563fcdbafd60 2024-04-24T23:28:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f2487d98740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f24802007c0 con 0x563fcdbcd600 2024-04-24T23:28:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f247e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbc1c20 0x563fcdb83910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f247e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbc1c20 0x563fcdb83910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51270/0 (socket says 172.21.15.26:51270) 2024-04-24T23:28:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.204+0000 7f247e59c640 1 -- 172.21.15.26:0/951573981 learned_addr learned my addr 172.21.15.26:0/951573981 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f247f59e640 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdb83e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f247ed9d640 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbafd60 0x563fcdb7f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f247e59c640 1 -- 172.21.15.26:0/951573981 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 msgr2=0x563fcdb83e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f247e59c640 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdb83e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f247e59c640 1 -- 172.21.15.26:0/951573981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbafd60 msgr2=0x563fcdb7f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f247e59c640 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbafd60 0x563fcdb7f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f247e59c640 1 -- 172.21.15.26:0/951573981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fcdbc1a50 con 0x563fcdbc1c20 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f247ed9d640 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbafd60 0x563fcdb7f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f247e59c640 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbc1c20 0x563fcdb83910 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f24800ce480 tx=0x7f2470001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0038c92122f6da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f245ffff640 1 -- 172.21.15.26:0/951573981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470024070 con 0x563fcdbc1c20 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f245ffff640 1 -- 172.21.15.26:0/951573981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24700190d0 con 0x563fcdbc1c20 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.205+0000 7f245ffff640 1 -- 172.21.15.26:0/951573981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470007230 con 0x563fcdbc1c20 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 -- 172.21.15.26:0/951573981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbc1c20 msgr2=0x563fcdb83910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbc1c20 0x563fcdb83910 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f24800ce480 tx=0x7f2470001120 comp rx=0 tx=0).stop 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 -- 172.21.15.26:0/951573981 shutdown_connections 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbcd600 0x563fcdb83e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbc1c20 0x563fcdb83910 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 --2- 172.21.15.26:0/951573981 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbafd60 0x563fcdb7f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 -- 172.21.15.26:0/951573981 >> 172.21.15.26:0/951573981 conn(0x563fcdb85a40 msgr2=0x563fcdbb2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 -- 172.21.15.26:0/951573981 shutdown_connections 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 -- 172.21.15.26:0/951573981 wait complete. 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 -- start start 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.206+0000 7f2487d98740 1 Processor -- start 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f2487d98740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdb7bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f2487d98740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbc1c20 0x563fcdbcbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f2487d98740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbcd600 0x563fcdbcc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f2487d98740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24800ca540 con 0x563fcdbafd60 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f2487d98740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563fcdbc12e0 con 0x563fcdbcd600 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f2487d98740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x563fcdb7f570 con 0x563fcdbc1c20 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f2487d98740 10 client.0 _ll_register_callbacks cb 0x7f2474026070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f247ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdb7bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f247ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdb7bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51274/0 (socket says 172.21.15.26:51274) 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f247ed9d640 1 -- 172.21.15.26:0/2115562451 learned_addr learned my addr 172.21.15.26:0/2115562451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f247e59c640 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbc1c20 0x563fcdbcbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.207+0000 7f247f59e640 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbcd600 0x563fcdbcc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f247ed9d640 1 -- 172.21.15.26:0/2115562451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbc1c20 msgr2=0x563fcdbcbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f247ed9d640 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbc1c20 0x563fcdbcbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f247ed9d640 1 -- 172.21.15.26:0/2115562451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbcd600 msgr2=0x563fcdbcc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f247ed9d640 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbcd600 0x563fcdbcc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f247ed9d640 1 -- 172.21.15.26:0/2115562451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fcdb7f7c0 con 0x563fcdbafd60 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f247ed9d640 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdb7bbc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f2474007f70 tx=0x7f2474006030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586f97402aa91b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f245dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f245dffb640 1 -- 172.21.15.26:0/2115562451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474028040 con 0x563fcdbafd60 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f245dffb640 1 -- 172.21.15.26:0/2115562451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f24740158c0 con 0x563fcdbafd60 2024-04-24T23:28:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f2487d98740 1 -- 172.21.15.26:0/2115562451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24700190d0 con 0x563fcdbafd60 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f245dffb640 1 -- 172.21.15.26:0/2115562451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f2474049050 con 0x563fcdbafd60 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f2487d98740 10 client.9315 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f2487d98740 1 -- 172.21.15.26:0/2115562451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x563fcdb86f80 con 0x563fcdbafd60 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f245dffb640 1 -- 172.21.15.26:0/2115562451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24700190d0 con 0x563fcdbafd60 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f245dffb640 1 -- 172.21.15.26:0/2115562451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f247404c020 con 0x563fcdbafd60 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f2487d98740 10 client.9315 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f2487d98740 10 client.9315 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f2487d98740 20 client.9315 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f2487d98740 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x563fcdc44f40 0x563fcdbc0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f2487d98740 4 client.9315 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.208+0000 7f2487d98740 1 -- 172.21.15.26:0/2115562451 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fcdaf0f40 con 0x563fcdc44f40 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f247e59c640 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x563fcdc44f40 0x563fcdbc0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f245dffb640 10 client.9315 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:28:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f247e59c640 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x563fcdc44f40 0x563fcdbc0500 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b1ef3602d7d18de server_cookie=6f3358f76b632c3 in_seq=0 out_seq=0 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f245dffb640 1 -- 172.21.15.26:0/2115562451 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x563fcdaf0f40 con 0x563fcdc44f40 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f245dffb640 10 client.9315 handle_command_reply: tid=0 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 client.9315 shutdown 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 -- 172.21.15.26:0/2115562451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 msgr2=0x563fcdb7bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdb7bbc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f2474007f70 tx=0x7f2474006030 comp rx=0 tx=0).stop 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 -- 172.21.15.26:0/2115562451 shutdown_connections 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x563fcdc44f40 0x563fcdbc0500 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fcdbcd600 0x563fcdbcc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x563fcdbc1c20 0x563fcdbcbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 --2- 172.21.15.26:0/2115562451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fcdbafd60 0x563fcdb7bbc0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f245dffb640 0 client.9315 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 -- 172.21.15.26:0/2115562451 >> 172.21.15.26:0/2115562451 conn(0x563fcdb85a40 msgr2=0x563fcdbc9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 -- 172.21.15.26:0/2115562451 shutdown_connections 2024-04-24T23:28:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.209+0000 7f2487d98740 1 -- 172.21.15.26:0/2115562451 wait complete. 2024-04-24T23:28:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f2487d98740 20 client.9315 trim_cache size 0 max 16384 2024-04-24T23:28:00.220 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:00.220 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:28:00.220 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:28:00.220 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:28:00.220 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:28:00.220 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:28:00.220 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:28:00.221 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:00.221 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:00.221 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 -- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be60 msgr2=0x7f246c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be60 0x7f246c06e320 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f248006c2e0 tx=0x7f24740078f0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 -- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24801f7310 msgr2=0x7f2480149400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24801f7310 0x7f2480149400 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f247802af10 tx=0x7f2478009750 comp rx=0 tx=0).stop 2024-04-24T23:28:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 -- 172.21.15.26:0/476484211 shutdown_connections 2024-04-24T23:28:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f246c06be60 0x7f246c06e320 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24801f7310 0x7f2480149400 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f24801f7c40 0x7f248015a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 --2- 172.21.15.26:0/476484211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24680051c0 0x7f24800c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 -- 172.21.15.26:0/476484211 >> 172.21.15.26:0/476484211 conn(0x7f24800d40f0 msgr2=0x7f24800b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 -- 172.21.15.26:0/476484211 shutdown_connections 2024-04-24T23:28:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.210+0000 7f248508c640 1 -- 172.21.15.26:0/476484211 wait complete. 2024-04-24T23:28:00.223 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:28:00.223 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:28:00.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-04-24T23:28:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.326+0000 7f7a6e9c8640 1 Processor -- start 2024-04-24T23:28:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.326+0000 7f7a6e9c8640 1 -- start start 2024-04-24T23:28:00.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a6e9c8640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a6815e370 0x7f7a680d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a6e9c8640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7a6815e750 0x7f7a680d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a6e9c8640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a680d2120 con 0x7f7a680d4c10 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a6e9c8640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7a680d1510 con 0x7f7a6815e750 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a6e9c8640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7a680d1ed0 con 0x7f7a6815e370 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a6cf65640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7a680d4c10 0x7f7a680b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34004/0 (socket says 172.21.15.26:34004) 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a6cf65640 1 -- 172.21.15.26:0/2356862238 learned_addr learned my addr 172.21.15.26:0/2356862238 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a67fff640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a6815e370 0x7f7a680d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1287502694 0 0) 0x7f7a680d2120 con 0x7f7a680d4c10 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a677fe640 1 --2- 172.21.15.26:0/2356862238 >> v2:172.21.15.29:3300/0 conn(0x7f7a6815e750 0x7f7a680d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a50003520 con 0x7f7a680d4c10 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1809533578 0 0) 0x7f7a50003520 con 0x7f7a680d4c10 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a6815e370 msgr2=0x7f7a680d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a66ffd640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a6815e370 0x7f7a680d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 >> v2:172.21.15.29:3300/0 conn(0x7f7a6815e750 msgr2=0x7f7a680d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a66ffd640 1 --2- 172.21.15.26:0/2356862238 >> v2:172.21.15.29:3300/0 conn(0x7f7a6815e750 0x7f7a680d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a680b3740 con 0x7f7a680d4c10 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.327+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f7a600019f0 con 0x7f7a680d4c10 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 >> v1:172.21.15.26:6789/0 conn(0x7f7a680d4c10 legacy=0x7f7a680b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a66ffd640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a500039b0 0x7f7a50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a66ffd640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a6815e370 0x7f7a50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a66ffd640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a50004d40 0x7f7a500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a680d1ed0 con 0x7f7a50004d40 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5000b9c0 con 0x7f7a500039b0 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5000bb40 con 0x7f7a6815e370 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a50004d40 0x7f7a500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a67fff640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a500039b0 0x7f7a50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a677fe640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a6815e370 0x7f7a50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a6cf65640 1 -- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a6815e370 msgr2=0x7f7a50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a6815e370 0x7f7a50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a6cf65640 1 -- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a500039b0 msgr2=0x7f7a50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a500039b0 0x7f7a50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.328+0000 7f7a6cf65640 1 -- 172.21.15.26:0/2356862238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a50003520 con 0x7f7a50004d40 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.329+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a50004d40 0x7f7a500051c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000a920 tx=0x7f7a50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475454a845b3c053 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.329+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000a4c0 con 0x7f7a50004d40 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.329+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a6002f070 con 0x7f7a50004d40 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.329+0000 7f7a66ffd640 1 -- 172.21.15.26:0/2356862238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6002bd00 con 0x7f7a50004d40 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.330+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/2356862238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a50004d40 msgr2=0x7f7a500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.330+0000 7f7a6e9c8640 1 --2- 172.21.15.26:0/2356862238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a50004d40 0x7f7a500051c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000a920 tx=0x7f7a50005830 comp rx=0 tx=0).stop 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.330+0000 7f7a6cf65640 1 -- 172.21.15.26:0/2356862238 reap_dead start 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.330+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/2356862238 shutdown_connections 2024-04-24T23:28:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.330+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/2356862238 >> 172.21.15.26:0/2356862238 conn(0x7f7a680d40f0 msgr2=0x7f7a680d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.330+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/2356862238 shutdown_connections 2024-04-24T23:28:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.330+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/2356862238 wait complete. 2024-04-24T23:28:00.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.332+0000 7f7a6e9c8640 1 Processor -- start 2024-04-24T23:28:00.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.332+0000 7f7a6e9c8640 1 -- start start 2024-04-24T23:28:00.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a6e9c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a68150550 0x7f7a681f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a6e9c8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a680b1ca0 0x7f7a680b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a6e9c8640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a680cbaf0 0x7f7a680cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a6e9c8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a680d2520 con 0x7f7a68150550 2024-04-24T23:28:00.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a6e9c8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7a680b3d00 con 0x7f7a680b1ca0 2024-04-24T23:28:00.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a6e9c8640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7a680d26a0 con 0x7f7a680cbaf0 2024-04-24T23:28:00.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a68150550 0x7f7a681f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a68150550 0x7f7a681f4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51292/0 (socket says 172.21.15.26:51292) 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a67fff640 1 -- 172.21.15.26:0/1655606427 learned_addr learned my addr 172.21.15.26:0/1655606427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a680cbaf0 0x7f7a680cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a680b1ca0 0x7f7a680b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a67fff640 1 -- 172.21.15.26:0/1655606427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a680cbaf0 msgr2=0x7f7a680cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a67fff640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a680cbaf0 0x7f7a680cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a67fff640 1 -- 172.21.15.26:0/1655606427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a680b1ca0 msgr2=0x7f7a680b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a67fff640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a680b1ca0 0x7f7a680b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.333+0000 7f7a67fff640 1 -- 172.21.15.26:0/1655606427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6815ff00 con 0x7f7a68150550 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.334+0000 7f7a67fff640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a68150550 0x7f7a681f4280 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0118e0 tx=0x7f7a5c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.334+0000 7f7a64ff9640 1 -- 172.21.15.26:0/1655606427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00faf0 con 0x7f7a68150550 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.334+0000 7f7a64ff9640 1 -- 172.21.15.26:0/1655606427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5c00fc90 con 0x7f7a68150550 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.334+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/1655606427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a681601e0 con 0x7f7a68150550 2024-04-24T23:28:00.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.334+0000 7f7a64ff9640 1 -- 172.21.15.26:0/1655606427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c014440 con 0x7f7a68150550 2024-04-24T23:28:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.334+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/1655606427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a681607a0 con 0x7f7a68150550 2024-04-24T23:28:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.335+0000 7f7a64ff9640 1 -- 172.21.15.26:0/1655606427 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a5c012040 con 0x7f7a68150550 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.336+0000 7f7a64ff9640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7a5406be30 0x7f7a5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.336+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7a5406be30 0x7f7a5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.336+0000 7f7a64ff9640 1 -- 172.21.15.26:0/1655606427 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f7a5c08b670 con 0x7f7a68150550 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.337+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7a5406be30 0x7f7a5406e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7a680ce7a0 tx=0x7f7a58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.337+0000 7f7a717b2740 1 Processor -- start 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.337+0000 7f7a717b2740 1 -- start start 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d0ed60 0x565525d0f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d2c600 0x565525d2cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525ce4a40 0x565525d2acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a717b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565525c486b0 con 0x565525ce4a40 2024-04-24T23:28:00.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a717b2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x565525bf0880 con 0x565525d0ed60 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a717b2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x565525c44670 con 0x565525d2c600 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a6cf65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525ce4a40 0x565525d2acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a6cf65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525ce4a40 0x565525d2acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51304/0 (socket says 172.21.15.26:51304) 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a6cf65640 1 -- 172.21.15.26:0/3692786546 learned_addr learned my addr 172.21.15.26:0/3692786546 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a677fe640 1 --2- 172.21.15.26:0/3692786546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d2c600 0x565525d2cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a6cf65640 1 -- 172.21.15.26:0/3692786546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d2c600 msgr2=0x565525d2cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/3692786546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d2c600 0x565525d2cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a67fff640 1 --2- 172.21.15.26:0/3692786546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d0ed60 0x565525d0f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a6cf65640 1 -- 172.21.15.26:0/3692786546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d0ed60 msgr2=0x565525d0f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/3692786546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d0ed60 0x565525d0f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a6cf65640 1 -- 172.21.15.26:0/3692786546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565525a267d0 con 0x565525ce4a40 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/3692786546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525ce4a40 0x565525d2acc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000aa40 tx=0x7f7a6002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d952d330340dbcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.338+0000 7f7a4dffb640 1 -- 172.21.15.26:0/3692786546 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60043070 con 0x565525ce4a40 2024-04-24T23:28:00.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a4dffb640 1 -- 172.21.15.26:0/3692786546 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a6002f050 con 0x565525ce4a40 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a4dffb640 1 -- 172.21.15.26:0/3692786546 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600019f0 con 0x565525ce4a40 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a717b2740 1 -- 172.21.15.26:0/3692786546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525ce4a40 msgr2=0x565525d2acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a717b2740 1 --2- 172.21.15.26:0/3692786546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525ce4a40 0x565525d2acc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000aa40 tx=0x7f7a6002d000 comp rx=0 tx=0).stop 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a717b2740 1 -- 172.21.15.26:0/3692786546 shutdown_connections 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a717b2740 1 --2- 172.21.15.26:0/3692786546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525ce4a40 0x565525d2acc0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a717b2740 1 --2- 172.21.15.26:0/3692786546 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d2c600 0x565525d2cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a717b2740 1 --2- 172.21.15.26:0/3692786546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d0ed60 0x565525d0f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a717b2740 1 -- 172.21.15.26:0/3692786546 >> 172.21.15.26:0/3692786546 conn(0x565525ce1600 msgr2=0x565525d29b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a717b2740 1 -- 172.21.15.26:0/3692786546 shutdown_connections 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.339+0000 7f7a717b2740 1 -- 172.21.15.26:0/3692786546 wait complete. 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.340+0000 7f7a717b2740 1 -- start start 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.340+0000 7f7a717b2740 1 Processor -- start 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d20c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 0x565525d39ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a717b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565525ce57e0 con 0x565525d0ed60 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a717b2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x565525c03f50 con 0x565525d2c600 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a717b2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x565525cdf530 con 0x565525d0b930 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d20c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a717b2740 10 client.0 _ll_register_callbacks cb 0x565525c3f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d20c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51320/0 (socket says 172.21.15.26:51320) 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a67fff640 1 -- 172.21.15.26:0/1237284188 learned_addr learned my addr 172.21.15.26:0/1237284188 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 0x565525d39ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a67fff640 1 -- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 msgr2=0x565525d39ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a67fff640 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 0x565525d39ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a67fff640 1 -- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 msgr2=0x565525d239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a67fff640 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a67fff640 1 -- 172.21.15.26:0/1237284188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56552595c3f0 con 0x565525d0ed60 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a67fff640 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d20c20 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c061fd0 tx=0x7f7a5c059a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c31765975dc60f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.341+0000 7f7a46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1237284188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c071020 con 0x565525d0ed60 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1237284188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f7a5c062270 con 0x565525d0ed60 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a717b2740 1 -- 172.21.15.26:0/1237284188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6002f050 con 0x565525d0ed60 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1237284188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c05abc0 con 0x565525d0ed60 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a717b2740 10 client.9327 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a717b2740 1 -- 172.21.15.26:0/1237284188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x56552591c0f0 con 0x565525d0ed60 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1237284188 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a6002f050 con 0x565525d0ed60 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1237284188 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f7a5c025030 con 0x565525d0ed60 2024-04-24T23:28:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a717b2740 10 client.9327 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a717b2740 10 client.9327 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a717b2740 20 client.9327 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x565525d11260 0x565525d116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a717b2740 4 client.9327 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a717b2740 1 -- 172.21.15.26:0/1237284188 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565525c55bf0 con 0x565525d11260 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.342+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x565525d11260 0x565525d116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.343+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x565525d11260 0x565525d116f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb35a79d924ad93f server_cookie=711da7285f1dc77b in_seq=0 out_seq=0 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.343+0000 7f7a46ffd640 10 client.9327 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.343+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1237284188 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565525c55bf0 con 0x565525d11260 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.343+0000 7f7a46ffd640 10 client.9327 handle_command_reply: tid=0 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.343+0000 7f7a717b2740 1 client.9327 shutdown 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 -- 172.21.15.26:0/1237284188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 msgr2=0x565525d20c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d20c20 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c061fd0 tx=0x7f7a5c059a00 comp rx=0 tx=0).stop 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 -- 172.21.15.26:0/1237284188 shutdown_connections 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x565525d11260 0x565525d116f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 0x565525d39ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1237284188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d20c20 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 -- 172.21.15.26:0/1237284188 >> 172.21.15.26:0/1237284188 conn(0x565525ce4a40 msgr2=0x565525d2d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a46ffd640 0 client.9327 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 -- 172.21.15.26:0/1237284188 shutdown_connections 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 1 -- 172.21.15.26:0/1237284188 wait complete. 2024-04-24T23:28:00.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.344+0000 7f7a717b2740 20 client.9327 trim_cache size 0 max 16384 2024-04-24T23:28:00.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.361+0000 7f7a717b2740 1 Processor -- start 2024-04-24T23:28:00.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.361+0000 7f7a717b2740 1 -- start start 2024-04-24T23:28:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.362+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 0x565525ce5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.362+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d24360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.362+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.362+0000 7f7a717b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565525cde5d0 con 0x565525d0ed60 2024-04-24T23:28:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.362+0000 7f7a717b2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68201320 con 0x565525d2c600 2024-04-24T23:28:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.362+0000 7f7a717b2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7a68200540 con 0x565525d0b930 2024-04-24T23:28:00.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.362+0000 7f7a677fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d24360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.362+0000 7f7a677fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d24360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51332/0 (socket says 172.21.15.26:51332) 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.362+0000 7f7a677fe640 1 -- 172.21.15.26:0/4266895673 learned_addr learned my addr 172.21.15.26:0/4266895673 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a677fe640 1 -- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 msgr2=0x565525ce5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a67fff640 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 0x565525ce5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a677fe640 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 0x565525ce5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a677fe640 1 -- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 msgr2=0x565525d248a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a677fe640 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d248a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a677fe640 1 -- 172.21.15.26:0/4266895673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565525cdfe50 con 0x565525d0ed60 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a67fff640 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 0x565525ce5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a677fe640 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d24360 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7a580147e0 tx=0x7f7a58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55af88f717d55ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a4d7fa640 1 -- 172.21.15.26:0/4266895673 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a580097f0 con 0x565525d0ed60 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a4d7fa640 1 -- 172.21.15.26:0/4266895673 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a58033070 con 0x565525d0ed60 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a4d7fa640 1 -- 172.21.15.26:0/4266895673 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5802d610 con 0x565525d0ed60 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a717b2740 1 -- 172.21.15.26:0/4266895673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 msgr2=0x565525d24360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.363+0000 7f7a717b2740 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d24360 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7a580147e0 tx=0x7f7a58017000 comp rx=0 tx=0).stop 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.364+0000 7f7a717b2740 1 -- 172.21.15.26:0/4266895673 shutdown_connections 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.364+0000 7f7a717b2740 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.364+0000 7f7a717b2740 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0ed60 0x565525d24360 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.364+0000 7f7a717b2740 1 --2- 172.21.15.26:0/4266895673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0b930 0x565525ce5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.364+0000 7f7a717b2740 1 -- 172.21.15.26:0/4266895673 >> 172.21.15.26:0/4266895673 conn(0x565525ce4a40 msgr2=0x565525d08a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.364+0000 7f7a717b2740 1 -- 172.21.15.26:0/4266895673 shutdown_connections 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.364+0000 7f7a717b2740 1 -- 172.21.15.26:0/4266895673 wait complete. 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.364+0000 7f7a717b2740 1 -- start start 2024-04-24T23:28:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.364+0000 7f7a717b2740 1 Processor -- start 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0b930 0x565525ce2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0ed60 0x565525d29ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a717b2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d2a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a717b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6815f7f0 con 0x565525d0b930 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a717b2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68200bf0 con 0x565525d2c600 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a717b2740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x565525d22510 con 0x565525d0ed60 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a717b2740 10 client.0 _ll_register_callbacks cb 0x7f7a5c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0b930 0x565525ce2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0b930 0x565525ce2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51344/0 (socket says 172.21.15.26:51344) 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a67fff640 1 -- 172.21.15.26:0/1604893672 learned_addr learned my addr 172.21.15.26:0/1604893672 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0ed60 0x565525d29ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.365+0000 7f7a6cf65640 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d2a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a67fff640 1 -- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0ed60 msgr2=0x565525d29ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a67fff640 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0ed60 0x565525d29ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a67fff640 1 -- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 msgr2=0x565525d2a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a67fff640 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d2a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a67fff640 1 -- 172.21.15.26:0/1604893672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565525d248e0 con 0x565525d0b930 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a67fff640 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0b930 0x565525ce2910 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00f790 tx=0x7f7a5c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b56eb52618eaee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1604893672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c07d060 con 0x565525d0b930 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1604893672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f7a5c058800 con 0x565525d0b930 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a717b2740 1 -- 172.21.15.26:0/1604893672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a58033070 con 0x565525d0b930 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1604893672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c071040 con 0x565525d0b930 2024-04-24T23:28:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a717b2740 10 client.9333 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a717b2740 1 -- 172.21.15.26:0/1604893672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x565525d250a0 con 0x565525d0b930 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1604893672 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a58033070 con 0x565525d0b930 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1604893672 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f7a5c025050 con 0x565525d0b930 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a717b2740 10 client.9333 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a717b2740 10 client.9333 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a717b2740 20 client.9333 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x565525ce5560 0x565525d0fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a717b2740 4 client.9333 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.366+0000 7f7a717b2740 1 -- 172.21.15.26:0/1604893672 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565525c4ff40 con 0x565525ce5560 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x565525ce5560 0x565525d0fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a677fe640 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x565525ce5560 0x565525d0fa20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3060092ca0752148 server_cookie=c3a925f21d532724 in_seq=0 out_seq=0 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a46ffd640 10 client.9333 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a46ffd640 1 -- 172.21.15.26:0/1604893672 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x565525c4ff40 con 0x565525ce5560 2024-04-24T23:28:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a46ffd640 10 client.9333 handle_command_reply: tid=0 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a717b2740 1 client.9333 shutdown 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a717b2740 1 -- 172.21.15.26:0/1604893672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0b930 msgr2=0x565525ce2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0b930 0x565525ce2910 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00f790 tx=0x7f7a5c05c040 comp rx=0 tx=0).stop 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a717b2740 1 -- 172.21.15.26:0/1604893672 shutdown_connections 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x565525ce5560 0x565525d0fa20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x565525d2c600 0x565525d2a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x565525d0ed60 0x565525d29ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a46ffd640 0 client.9333 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a717b2740 1 --2- 172.21.15.26:0/1604893672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565525d0b930 0x565525ce2910 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.367+0000 7f7a717b2740 1 -- 172.21.15.26:0/1604893672 >> 172.21.15.26:0/1604893672 conn(0x565525ce4a40 msgr2=0x565525d0dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a717b2740 1 -- 172.21.15.26:0/1604893672 shutdown_connections 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a717b2740 1 -- 172.21.15.26:0/1604893672 wait complete. 2024-04-24T23:28:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a717b2740 20 client.9333 trim_cache size 0 max 16384 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:00.375 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:00.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7a5406be30 msgr2=0x7f7a5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7a5406be30 0x7f7a5406e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7a680ce7a0 tx=0x7f7a58008040 comp rx=0 tx=0).stop 2024-04-24T23:28:00.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a68150550 msgr2=0x7f7a681f4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a68150550 0x7f7a681f4280 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0118e0 tx=0x7f7a5c011da0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/1655606427 shutdown_connections 2024-04-24T23:28:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7a5406be30 0x7f7a5406e2f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7a680cbaf0 0x7f7a680cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a680b1ca0 0x7f7a680b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 --2- 172.21.15.26:0/1655606427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a68150550 0x7f7a681f4280 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/1655606427 >> 172.21.15.26:0/1655606427 conn(0x7f7a680d40f0 msgr2=0x7f7a6815c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/1655606427 shutdown_connections 2024-04-24T23:28:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.368+0000 7f7a6e9c8640 1 -- 172.21.15.26:0/1655606427 wait complete. 2024-04-24T23:28:00.379 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:28:00.380 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:28:00.380 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-24T23:28:00.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.482+0000 7f43a5753640 1 Processor -- start 2024-04-24T23:28:00.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.482+0000 7f43a5753640 1 -- start start 2024-04-24T23:28:00.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f43a5753640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f43a015e370 0x7f43a00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f43a5753640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43a015e750 0x7f43a00d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f43a5753640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a00d2120 con 0x7f43a00d4c10 2024-04-24T23:28:00.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f43a5753640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f43a00d1510 con 0x7f43a015e750 2024-04-24T23:28:00.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f43a5753640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f43a00d1ed0 con 0x7f43a015e370 2024-04-24T23:28:00.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f439e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43a015e750 0x7f43a00d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f439f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f43a00d4c10 0x7f43a00b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34008/0 (socket says 172.21.15.26:34008) 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f439f7fe640 1 -- 172.21.15.26:0/1949463927 learned_addr learned my addr 172.21.15.26:0/1949463927 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f439effd640 1 --2- 172.21.15.26:0/1949463927 >> v2:172.21.15.29:3300/0 conn(0x7f43a015e370 0x7f43a00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f439e7fc640 1 -- 172.21.15.26:0/1949463927 >> v2:172.21.15.29:3300/0 conn(0x7f43a015e370 msgr2=0x7f43a00d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f439e7fc640 1 --2- 172.21.15.26:0/1949463927 >> v2:172.21.15.29:3300/0 conn(0x7f43a015e370 0x7f43a00d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f439e7fc640 1 -- 172.21.15.26:0/1949463927 >> v1:172.21.15.26:6789/0 conn(0x7f43a00d4c10 legacy=0x7f43a00b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.484+0000 7f439e7fc640 1 -- 172.21.15.26:0/1949463927 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a00b3740 con 0x7f43a015e750 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.483+0000 7f439dffb640 1 -- 172.21.15.26:0/1949463927 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079295424 0 0) 0x7f43a00d2120 con 0x7f43a00d4c10 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.484+0000 7f439e7fc640 1 --2- 172.21.15.26:0/1949463927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e750 0x7f43a00d2a60 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f43a008c7e0 tx=0x7f439402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20b1550a31f1e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.484+0000 7f439dffb640 1 -- 172.21.15.26:0/1949463927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394037070 con 0x7f43a015e750 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.484+0000 7f439dffb640 1 -- 172.21.15.26:0/1949463927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4394032440 con 0x7f43a015e750 2024-04-24T23:28:00.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.484+0000 7f439dffb640 1 -- 172.21.15.26:0/1949463927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394032810 con 0x7f43a015e750 2024-04-24T23:28:00.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.485+0000 7f43a5753640 1 -- 172.21.15.26:0/1949463927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e750 msgr2=0x7f43a00d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.485+0000 7f43a5753640 1 --2- 172.21.15.26:0/1949463927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e750 0x7f43a00d2a60 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f43a008c7e0 tx=0x7f439402a1a0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.485+0000 7f43a5753640 1 -- 172.21.15.26:0/1949463927 shutdown_connections 2024-04-24T23:28:00.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.485+0000 7f43a5753640 1 --2- 172.21.15.26:0/1949463927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e750 0x7f43a00d2a60 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.485+0000 7f43a5753640 1 --2- 172.21.15.26:0/1949463927 >> v2:172.21.15.29:3300/0 conn(0x7f43a015e370 0x7f43a00d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.485+0000 7f43a5753640 1 -- 172.21.15.26:0/1949463927 >> 172.21.15.26:0/1949463927 conn(0x7f43a00d40f0 msgr2=0x7f43a00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.485+0000 7f43a5753640 1 -- 172.21.15.26:0/1949463927 shutdown_connections 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.485+0000 7f43a5753640 1 -- 172.21.15.26:0/1949463927 wait complete. 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.486+0000 7f43a5753640 1 Processor -- start 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.486+0000 7f43a5753640 1 -- start start 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f43a5753640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d4c10 0x7f43a015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f43a5753640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e370 0x7f43a0156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f43a5753640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a00c8e00 0x7f43a00b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f43a5753640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f43a00cfca0 con 0x7f43a015e370 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f43a5753640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43a00b3d00 con 0x7f43a00c8e00 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f43a5753640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f43a00cfe20 con 0x7f43a00d4c10 2024-04-24T23:28:00.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f439e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e370 0x7f43a0156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f439e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e370 0x7f43a0156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51370/0 (socket says 172.21.15.26:51370) 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f439e7fc640 1 -- 172.21.15.26:0/766160308 learned_addr learned my addr 172.21.15.26:0/766160308 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f439f7fe640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a00c8e00 0x7f43a00b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.487+0000 7f439effd640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d4c10 0x7f43a015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f439e7fc640 1 -- 172.21.15.26:0/766160308 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d4c10 msgr2=0x7f43a015eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f439e7fc640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d4c10 0x7f43a015eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f439e7fc640 1 -- 172.21.15.26:0/766160308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a00c8e00 msgr2=0x7f43a00b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f439e7fc640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a00c8e00 0x7f43a00b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f439e7fc640 1 -- 172.21.15.26:0/766160308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a00b7b50 con 0x7f43a015e370 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f439e7fc640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e370 0x7f43a0156f40 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4394000c00 tx=0x7f43940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f43877fe640 1 -- 172.21.15.26:0/766160308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394037040 con 0x7f43a015e370 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f43877fe640 1 -- 172.21.15.26:0/766160308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4394042070 con 0x7f43a015e370 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f43a5753640 1 -- 172.21.15.26:0/766160308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a00b7dc0 con 0x7f43a015e370 2024-04-24T23:28:00.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f43877fe640 1 -- 172.21.15.26:0/766160308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394030920 con 0x7f43a015e370 2024-04-24T23:28:00.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.488+0000 7f43a5753640 1 -- 172.21.15.26:0/766160308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a0157480 con 0x7f43a015e370 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.490+0000 7f43877fe640 1 -- 172.21.15.26:0/766160308 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4394030ac0 con 0x7f43a015e370 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.490+0000 7f43877fe640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438c06be30 0x7f438c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.490+0000 7f43877fe640 1 -- 172.21.15.26:0/766160308 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f43940ab650 con 0x7f43a015e370 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.490+0000 7f439effd640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438c06be30 0x7f438c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.491+0000 7f439effd640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438c06be30 0x7f438c06e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4390009b40 tx=0x7f4390009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.491+0000 7f43a845d740 1 Processor -- start 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.491+0000 7f43a845d740 1 -- start start 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f43a845d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 0x55fdb7725140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f43a845d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7742600 0x55fdb7742b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f43a845d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb76faa40 0x55fdb7740cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f43a845d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fdb765e6b0 con 0x55fdb76faa40 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f43a845d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fdb7606880 con 0x55fdb7724d60 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f43a845d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55fdb765a670 con 0x55fdb7742600 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb76faa40 0x55fdb7740cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb76faa40 0x55fdb7740cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51376/0 (socket says 172.21.15.26:51376) 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439f7fe640 1 -- 172.21.15.26:0/1273953148 learned_addr learned my addr 172.21.15.26:0/1273953148 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439effd640 1 --2- 172.21.15.26:0/1273953148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 0x55fdb7725140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439e7fc640 1 --2- 172.21.15.26:0/1273953148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7742600 0x55fdb7742b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439f7fe640 1 -- 172.21.15.26:0/1273953148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7742600 msgr2=0x55fdb7742b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439f7fe640 1 --2- 172.21.15.26:0/1273953148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7742600 0x55fdb7742b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439f7fe640 1 -- 172.21.15.26:0/1273953148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 msgr2=0x55fdb7725140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439f7fe640 1 --2- 172.21.15.26:0/1273953148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 0x55fdb7725140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439f7fe640 1 -- 172.21.15.26:0/1273953148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb743c7d0 con 0x55fdb76faa40 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.492+0000 7f439f7fe640 1 --2- 172.21.15.26:0/1273953148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb76faa40 0x55fdb7740cc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f4398001c30 tx=0x7f43980020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bc2d1a017ba71c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f4384ff9640 1 -- 172.21.15.26:0/1273953148 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398007850 con 0x55fdb76faa40 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f4384ff9640 1 -- 172.21.15.26:0/1273953148 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4398018070 con 0x55fdb76faa40 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f4384ff9640 1 -- 172.21.15.26:0/1273953148 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398013630 con 0x55fdb76faa40 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f43a845d740 1 -- 172.21.15.26:0/1273953148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb76faa40 msgr2=0x55fdb7740cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f43a845d740 1 --2- 172.21.15.26:0/1273953148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb76faa40 0x55fdb7740cc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f4398001c30 tx=0x7f43980020f0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f43a845d740 1 -- 172.21.15.26:0/1273953148 shutdown_connections 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f43a845d740 1 --2- 172.21.15.26:0/1273953148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb76faa40 0x55fdb7740cc0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f43a845d740 1 --2- 172.21.15.26:0/1273953148 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7742600 0x55fdb7742b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f43a845d740 1 --2- 172.21.15.26:0/1273953148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 0x55fdb7725140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f43a845d740 1 -- 172.21.15.26:0/1273953148 >> 172.21.15.26:0/1273953148 conn(0x55fdb76f7600 msgr2=0x55fdb773fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.493+0000 7f43a845d740 1 -- 172.21.15.26:0/1273953148 shutdown_connections 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.494+0000 7f43a845d740 1 -- 172.21.15.26:0/1273953148 wait complete. 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.494+0000 7f43a845d740 1 -- start start 2024-04-24T23:28:00.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.494+0000 7f43a845d740 1 Processor -- start 2024-04-24T23:28:00.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f43a845d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 0x55fdb7728950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f43a845d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7753720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f43a845d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 0x55fdb7753c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f43a845d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fdb76fb7e0 con 0x55fdb7742600 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f43a845d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fdb7619f50 con 0x55fdb7724d60 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f43a845d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55fdb76f5530 con 0x55fdb774fd70 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f43a845d740 10 client.0 _ll_register_callbacks cb 0x55fdb7655000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7753720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7753720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51382/0 (socket says 172.21.15.26:51382) 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439e7fc640 1 -- 172.21.15.26:0/3852059593 learned_addr learned my addr 172.21.15.26:0/3852059593 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439effd640 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 0x55fdb7728950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439f7fe640 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 0x55fdb7753c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439e7fc640 1 -- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 msgr2=0x55fdb7753c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439e7fc640 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 0x55fdb7753c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439e7fc640 1 -- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 msgr2=0x55fdb7728950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439e7fc640 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 0x55fdb7728950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.495+0000 7f439e7fc640 1 -- 172.21.15.26:0/3852059593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb73723f0 con 0x55fdb7742600 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f439e7fc640 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7753720 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f43940a9b20 tx=0x7f4394032500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5149827c9d06c777 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f437dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f437dffb640 1 -- 172.21.15.26:0/3852059593 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43940824e0 con 0x55fdb7742600 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f437dffb640 1 -- 172.21.15.26:0/3852059593 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f43940800b0 con 0x55fdb7742600 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f43a845d740 1 -- 172.21.15.26:0/3852059593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4398018070 con 0x55fdb7742600 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f437dffb640 1 -- 172.21.15.26:0/3852059593 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f4394076020 con 0x55fdb7742600 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f43a845d740 10 client.9348 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f43a845d740 1 -- 172.21.15.26:0/3852059593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55fdb73320f0 con 0x55fdb7742600 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f437dffb640 1 -- 172.21.15.26:0/3852059593 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4398018070 con 0x55fdb7742600 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f437dffb640 1 -- 172.21.15.26:0/3852059593 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f439403a070 con 0x55fdb7742600 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f43a845d740 10 client.9348 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:00.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f43a845d740 10 client.9348 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f43a845d740 20 client.9348 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f43a845d740 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55fdb7738d70 0x55fdb7739180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f43a845d740 4 client.9348 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f43a845d740 1 -- 172.21.15.26:0/3852059593 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdb766bbf0 con 0x55fdb7738d70 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.496+0000 7f439effd640 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55fdb7738d70 0x55fdb7739180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.497+0000 7f439effd640 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55fdb7738d70 0x55fdb7739180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7481691c6633b77d server_cookie=7f18f1c080ed2dc2 in_seq=0 out_seq=0 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.497+0000 7f437dffb640 10 client.9348 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f437dffb640 1 -- 172.21.15.26:0/3852059593 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fdb766bbf0 con 0x55fdb7738d70 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f437dffb640 10 client.9348 handle_command_reply: tid=0 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 client.9348 shutdown 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 -- 172.21.15.26:0/3852059593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 msgr2=0x55fdb7753720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7753720 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f43940a9b20 tx=0x7f4394032500 comp rx=0 tx=0).stop 2024-04-24T23:28:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 -- 172.21.15.26:0/3852059593 shutdown_connections 2024-04-24T23:28:00.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55fdb7738d70 0x55fdb7739180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 0x55fdb7753c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f437dffb640 0 client.9348 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:00.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7753720 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 --2- 172.21.15.26:0/3852059593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7724d60 0x55fdb7728950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 -- 172.21.15.26:0/3852059593 >> 172.21.15.26:0/3852059593 conn(0x55fdb76faa40 msgr2=0x55fdb773f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 -- 172.21.15.26:0/3852059593 shutdown_connections 2024-04-24T23:28:00.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 1 -- 172.21.15.26:0/3852059593 wait complete. 2024-04-24T23:28:00.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.498+0000 7f43a845d740 20 client.9348 trim_cache size 0 max 16384 2024-04-24T23:28:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.515+0000 7f43a845d740 1 Processor -- start 2024-04-24T23:28:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.515+0000 7f43a845d740 1 -- start start 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.516+0000 7f43a845d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7724d60 0x55fdb77266b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.516+0000 7f43a845d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7736d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.516+0000 7f43a845d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb774fd70 0x55fdb773e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.516+0000 7f43a845d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0200640 con 0x55fdb7742600 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.516+0000 7f43a845d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43a00ca760 con 0x55fdb774fd70 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.516+0000 7f43a845d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f43a01fff10 con 0x55fdb7724d60 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.516+0000 7f439e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7736d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.516+0000 7f439e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7736d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51390/0 (socket says 172.21.15.26:51390) 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.516+0000 7f439e7fc640 1 -- 172.21.15.26:0/3471230356 learned_addr learned my addr 172.21.15.26:0/3471230356 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f439effd640 1 --2- 172.21.15.26:0/3471230356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7724d60 0x55fdb77266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f439e7fc640 1 -- 172.21.15.26:0/3471230356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7724d60 msgr2=0x55fdb77266b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f439e7fc640 1 --2- 172.21.15.26:0/3471230356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7724d60 0x55fdb77266b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f439f7fe640 1 --2- 172.21.15.26:0/3471230356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb774fd70 0x55fdb773e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f439e7fc640 1 -- 172.21.15.26:0/3471230356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb774fd70 msgr2=0x55fdb773e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f439e7fc640 1 --2- 172.21.15.26:0/3471230356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb774fd70 0x55fdb773e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f439e7fc640 1 -- 172.21.15.26:0/3471230356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb77394f0 con 0x55fdb7742600 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f439e7fc640 1 --2- 172.21.15.26:0/3471230356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7736d30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f43940796b0 tx=0x7f4394049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addd4386fb86598e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f437ffff640 1 -- 172.21.15.26:0/3471230356 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439409d070 con 0x55fdb7742600 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f437ffff640 1 -- 172.21.15.26:0/3471230356 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43940800b0 con 0x55fdb7742600 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f437ffff640 1 -- 172.21.15.26:0/3471230356 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43940824e0 con 0x55fdb7742600 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f43a845d740 1 -- 172.21.15.26:0/3471230356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 msgr2=0x55fdb7736d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.517+0000 7f43a845d740 1 --2- 172.21.15.26:0/3471230356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7736d30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f43940796b0 tx=0x7f4394049000 comp rx=0 tx=0).stop 2024-04-24T23:28:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.518+0000 7f43a845d740 1 -- 172.21.15.26:0/3471230356 shutdown_connections 2024-04-24T23:28:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.518+0000 7f43a845d740 1 --2- 172.21.15.26:0/3471230356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb774fd70 0x55fdb773e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.518+0000 7f43a845d740 1 --2- 172.21.15.26:0/3471230356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7742600 0x55fdb7736d30 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.518+0000 7f43a845d740 1 --2- 172.21.15.26:0/3471230356 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb7724d60 0x55fdb77266b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.518+0000 7f43a845d740 1 -- 172.21.15.26:0/3471230356 >> 172.21.15.26:0/3471230356 conn(0x55fdb76faa40 msgr2=0x55fdb7763f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.518+0000 7f43a845d740 1 -- 172.21.15.26:0/3471230356 shutdown_connections 2024-04-24T23:28:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.518+0000 7f43a845d740 1 -- 172.21.15.26:0/3471230356 wait complete. 2024-04-24T23:28:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.518+0000 7f43a845d740 1 -- start start 2024-04-24T23:28:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.518+0000 7f43a845d740 1 Processor -- start 2024-04-24T23:28:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f43a845d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7724d60 0x55fdb773e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f43a845d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7742600 0x55fdb7721310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f43a845d740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 0x55fdb773eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f43a845d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fdb7724900 con 0x55fdb7724d60 2024-04-24T23:28:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f43a845d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43a00c8600 con 0x55fdb7742600 2024-04-24T23:28:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f43a845d740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55fdb773a7a0 con 0x55fdb774fd70 2024-04-24T23:28:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f43a845d740 10 client.0 _ll_register_callbacks cb 0x7f4394009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7724d60 0x55fdb773e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7724d60 0x55fdb773e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51402/0 (socket says 172.21.15.26:51402) 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439effd640 1 -- 172.21.15.26:0/3250683187 learned_addr learned my addr 172.21.15.26:0/3250683187 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439e7fc640 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7742600 0x55fdb7721310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439f7fe640 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 0x55fdb773eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439effd640 1 -- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 msgr2=0x55fdb773eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439effd640 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 0x55fdb773eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439effd640 1 -- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7742600 msgr2=0x55fdb7721310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439effd640 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7742600 0x55fdb7721310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.519+0000 7f439effd640 1 -- 172.21.15.26:0/3250683187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb7721850 con 0x55fdb7724d60 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f439effd640 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7724d60 0x55fdb773e2e0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x55fdb7726680 tx=0x7f4390004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdb126911bfbea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f437dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f437dffb640 1 -- 172.21.15.26:0/3250683187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390030060 con 0x55fdb7724d60 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f437dffb640 1 -- 172.21.15.26:0/3250683187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f439002bb50 con 0x55fdb7724d60 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f43a845d740 1 -- 172.21.15.26:0/3250683187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43940800b0 con 0x55fdb7724d60 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f437dffb640 1 -- 172.21.15.26:0/3250683187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f439002bd50 con 0x55fdb7724d60 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f43a845d740 10 client.9354 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f43a845d740 1 -- 172.21.15.26:0/3250683187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55fdb774e450 con 0x55fdb7724d60 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f437dffb640 1 -- 172.21.15.26:0/3250683187 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43940800b0 con 0x55fdb7724d60 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f437dffb640 1 -- 172.21.15.26:0/3250683187 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f4390039080 con 0x55fdb7724d60 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f43a845d740 10 client.9354 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f43a845d740 10 client.9354 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f43a845d740 20 client.9354 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f43a845d740 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55fdb77b9cb0 0x55fdb773f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f43a845d740 4 client.9354 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f43a845d740 1 -- 172.21.15.26:0/3250683187 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fdb7665f40 con 0x55fdb77b9cb0 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.520+0000 7f439e7fc640 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55fdb77b9cb0 0x55fdb773f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f439e7fc640 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55fdb77b9cb0 0x55fdb773f2e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb4d8e2d546a2a2 server_cookie=d10208e1f5259e35 in_seq=0 out_seq=0 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f437dffb640 10 client.9354 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f437dffb640 1 -- 172.21.15.26:0/3250683187 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55fdb7665f40 con 0x55fdb77b9cb0 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f437dffb640 10 client.9354 handle_command_reply: tid=0 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 client.9354 shutdown 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 -- 172.21.15.26:0/3250683187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7724d60 msgr2=0x55fdb773e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7724d60 0x55fdb773e2e0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x55fdb7726680 tx=0x7f4390004730 comp rx=0 tx=0).stop 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 -- 172.21.15.26:0/3250683187 shutdown_connections 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x55fdb77b9cb0 0x55fdb773f2e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fdb774fd70 0x55fdb773eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fdb7742600 0x55fdb7721310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f437dffb640 0 client.9354 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 --2- 172.21.15.26:0/3250683187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdb7724d60 0x55fdb773e2e0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 -- 172.21.15.26:0/3250683187 >> 172.21.15.26:0/3250683187 conn(0x55fdb76faa40 msgr2=0x55fdb7727df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 -- 172.21.15.26:0/3250683187 shutdown_connections 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 1 -- 172.21.15.26:0/3250683187 wait complete. 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.521+0000 7f43a845d740 20 client.9354 trim_cache size 0 max 16384 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:28:00.529 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:28:00.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:28:00.530 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:28:00.530 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:00.530 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:00.530 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 -- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438c06be30 msgr2=0x7f438c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438c06be30 0x7f438c06e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4390009b40 tx=0x7f4390009190 comp rx=0 tx=0).stop 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 -- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e370 msgr2=0x7f43a0156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e370 0x7f43a0156f40 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4394000c00 tx=0x7f43940046c0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 -- 172.21.15.26:0/766160308 shutdown_connections 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438c06be30 0x7f438c06e2f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a00c8e00 0x7f43a00b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a015e370 0x7f43a0156f40 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 --2- 172.21.15.26:0/766160308 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d4c10 0x7f43a015eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 -- 172.21.15.26:0/766160308 >> 172.21.15.26:0/766160308 conn(0x7f43a00d40f0 msgr2=0x7f43a015d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 -- 172.21.15.26:0/766160308 shutdown_connections 2024-04-24T23:28:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.522+0000 7f43a5753640 1 -- 172.21.15.26:0/766160308 wait complete. 2024-04-24T23:28:00.533 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:28:00.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:00.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.633+0000 7efc18cda640 1 Processor -- start 2024-04-24T23:28:00.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.633+0000 7efc18cda640 1 -- start start 2024-04-24T23:28:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.634+0000 7efc18cda640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 0x7efc140b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.634+0000 7efc18cda640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc140d32f0 con 0x7efc1415e570 2024-04-24T23:28:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.634+0000 7efc18cda640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc140d0270 con 0x7efc140b3130 2024-04-24T23:28:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.634+0000 7efc18cda640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc140d1ab0 con 0x7efc140d4870 2024-04-24T23:28:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.634+0000 7efc1259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efc1415e570 0x7efc140d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34018/0 (socket says 172.21.15.26:34018) 2024-04-24T23:28:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.634+0000 7efc1259c640 1 -- 172.21.15.26:0/1595968946 learned_addr learned my addr 172.21.15.26:0/1595968946 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.634+0000 7efc0bfff640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 0x7efc140b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59072372 0 0) 0x7efc04001110 con 0x7efc1415e570 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc00003520 con 0x7efc1415e570 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1693861842 0 0) 0x7efc00003520 con 0x7efc1415e570 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 msgr2=0x7efc140b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 0x7efc140b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 >> v1:172.21.15.29:6789/0 conn(0x7efc140b3130 legacy=0x7efc140b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc140d2d30 con 0x7efc1415e570 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7efc04001840 con 0x7efc1415e570 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc0bfff640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 0x7efc140b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 >> v1:172.21.15.26:6789/0 conn(0x7efc1415e570 legacy=0x7efc140d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc000039e0 0x7efc00003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc00004390 0x7efc00004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 0x7efc00005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc140d1ab0 con 0x7efc00004390 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc0000b9c0 con 0x7efc000039e0 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc0000bb70 con 0x7efc140d4870 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc0bfff640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc00004390 0x7efc00004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc1259c640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc000039e0 0x7efc00003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc12d9d640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 0x7efc00005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc0bfff640 1 -- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 msgr2=0x7efc00005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc0bfff640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 0x7efc00005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc0bfff640 1 -- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc000039e0 msgr2=0x7efc00003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc0bfff640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc000039e0 0x7efc00003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc0bfff640 1 -- 172.21.15.26:0/1595968946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc00003520 con 0x7efc00004390 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.635+0000 7efc12d9d640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc140d4870 0x7efc00005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.636+0000 7efc0bfff640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc00004390 0x7efc00004860 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009c90 tx=0x7efc00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860b1530eb0e8834 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.636+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc036070 con 0x7efc00004390 2024-04-24T23:28:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.636+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbfc031400 con 0x7efc00004390 2024-04-24T23:28:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.636+0000 7efc11d9b640 1 -- 172.21.15.26:0/1595968946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc009860 con 0x7efc00004390 2024-04-24T23:28:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.637+0000 7efc18cda640 1 -- 172.21.15.26:0/1595968946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc00004390 msgr2=0x7efc00004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.637+0000 7efc18cda640 1 --2- 172.21.15.26:0/1595968946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc00004390 0x7efc00004860 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009c90 tx=0x7efc00005830 comp rx=0 tx=0).stop 2024-04-24T23:28:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.637+0000 7efc12d9d640 1 -- 172.21.15.26:0/1595968946 reap_dead start 2024-04-24T23:28:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.637+0000 7efc18cda640 1 -- 172.21.15.26:0/1595968946 shutdown_connections 2024-04-24T23:28:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.637+0000 7efc18cda640 1 -- 172.21.15.26:0/1595968946 >> 172.21.15.26:0/1595968946 conn(0x7efc1408dbe0 msgr2=0x7efc1409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.637+0000 7efc18cda640 1 -- 172.21.15.26:0/1595968946 shutdown_connections 2024-04-24T23:28:00.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.637+0000 7efc18cda640 1 -- 172.21.15.26:0/1595968946 wait complete. 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.638+0000 7efc18cda640 1 Processor -- start 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.638+0000 7efc18cda640 1 -- start start 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc18cda640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc00004390 0x7efc1415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc18cda640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc140cbfd0 0x7efc1415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc18cda640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc1414f4f0 0x7efc1415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc18cda640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc140cfa70 con 0x7efc140cbfd0 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc18cda640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc140d2ae0 con 0x7efc00004390 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc18cda640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc140cfd50 con 0x7efc1414f4f0 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc140cbfd0 0x7efc1415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc140cbfd0 0x7efc1415be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51418/0 (socket says 172.21.15.26:51418) 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc0bfff640 1 -- 172.21.15.26:0/2886087119 learned_addr learned my addr 172.21.15.26:0/2886087119 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc12d9d640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc1414f4f0 0x7efc1415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.639+0000 7efc1259c640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc00004390 0x7efc1415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc0bfff640 1 -- 172.21.15.26:0/2886087119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc1414f4f0 msgr2=0x7efc1415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc0bfff640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc1414f4f0 0x7efc1415cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc0bfff640 1 -- 172.21.15.26:0/2886087119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc00004390 msgr2=0x7efc1415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc0bfff640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc00004390 0x7efc1415ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc0bfff640 1 -- 172.21.15.26:0/2886087119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1415b360 con 0x7efc140cbfd0 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc0bfff640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc140cbfd0 0x7efc1415be00 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7efbfc000c00 tx=0x7efbfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc0affd640 1 -- 172.21.15.26:0/2886087119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc040070 con 0x7efc140cbfd0 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc0affd640 1 -- 172.21.15.26:0/2886087119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbfc0093e0 con 0x7efc140cbfd0 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc140cc7b0 con 0x7efc140cbfd0 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc0affd640 1 -- 172.21.15.26:0/2886087119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc036040 con 0x7efc140cbfd0 2024-04-24T23:28:00.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.640+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1415b600 con 0x7efc140cbfd0 2024-04-24T23:28:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.641+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc140694c0 con 0x7efc140cbfd0 2024-04-24T23:28:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.642+0000 7efc0affd640 1 -- 172.21.15.26:0/2886087119 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efbfc043070 con 0x7efc140cbfd0 2024-04-24T23:28:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.642+0000 7efc0affd640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbf406be60 0x7efbf406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.642+0000 7efc0affd640 1 -- 172.21.15.26:0/2886087119 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7efbfc0ab310 con 0x7efc140cbfd0 2024-04-24T23:28:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.642+0000 7efc1259c640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbf406be60 0x7efbf406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.642+0000 7efc1259c640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbf406be60 0x7efbf406e320 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efc040017a0 tx=0x7efc04007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.650+0000 7efc0affd640 1 -- 172.21.15.26:0/2886087119 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efbfc0796a0 con 0x7efc140cbfd0 2024-04-24T23:28:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.869+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc1406c910 con 0x7efc140cbfd0 2024-04-24T23:28:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.870+0000 7efc0affd640 1 -- 172.21.15.26:0/2886087119 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7efbfc07d9a0 con 0x7efc140cbfd0 2024-04-24T23:28:00.870 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:00.871 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.871+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbf406be60 msgr2=0x7efbf406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.871+0000 7efc18cda640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbf406be60 0x7efbf406e320 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efc040017a0 tx=0x7efc04007d40 comp rx=0 tx=0).stop 2024-04-24T23:28:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.871+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc140cbfd0 msgr2=0x7efc1415be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.871+0000 7efc18cda640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc140cbfd0 0x7efc1415be00 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7efbfc000c00 tx=0x7efbfc004730 comp rx=0 tx=0).stop 2024-04-24T23:28:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.871+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 shutdown_connections 2024-04-24T23:28:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.871+0000 7efc18cda640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efbf406be60 0x7efbf406e320 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.871+0000 7efc18cda640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc1414f4f0 0x7efc1415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.871+0000 7efc18cda640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc140cbfd0 0x7efc1415be00 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.872+0000 7efc18cda640 1 --2- 172.21.15.26:0/2886087119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc00004390 0x7efc1415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.872+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 >> 172.21.15.26:0/2886087119 conn(0x7efc1408dbe0 msgr2=0x7efc1406d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.872+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 shutdown_connections 2024-04-24T23:28:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.872+0000 7efc18cda640 1 -- 172.21.15.26:0/2886087119 wait complete. 2024-04-24T23:28:00.883 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:28:00.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:28:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.983+0000 7f40ce990640 1 Processor -- start 2024-04-24T23:28:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.983+0000 7f40ce990640 1 -- start start 2024-04-24T23:28:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40ce990640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c80ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40ce990640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40c815c5b0 0x7f40c806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40ce990640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40c815cee0 con 0x7f40c815ca10 2024-04-24T23:28:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40ce990640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f40c80b3300 con 0x7f40c815c5b0 2024-04-24T23:28:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40ce990640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f40c815d1c0 con 0x7f40c8145a80 2024-04-24T23:28:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40c7fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c80ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40c77fe640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f40c815c5b0 0x7f40c806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40c7fff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c80ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:42542/0 (socket says 172.21.15.26:42542) 2024-04-24T23:28:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40c7fff640 1 -- 172.21.15.26:0/129660648 learned_addr learned my addr 172.21.15.26:0/129660648 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40c7fff640 1 -- 172.21.15.26:0/129660648 >> v2:172.21.15.29:3300/0 conn(0x7f40c815c5b0 msgr2=0x7f40c806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40c7fff640 1 --2- 172.21.15.26:0/129660648 >> v2:172.21.15.29:3300/0 conn(0x7f40c815c5b0 0x7f40c806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40c7fff640 1 -- 172.21.15.26:0/129660648 >> v1:172.21.15.29:6789/0 conn(0x7f40c815ca10 legacy=0x7f40c815fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.984+0000 7f40c7fff640 1 -- 172.21.15.26:0/129660648 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c815d4a0 con 0x7f40c8145a80 2024-04-24T23:28:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.985+0000 7f40c7fff640 1 --2- 172.21.15.26:0/129660648 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c80ce040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f40b80098a0 tx=0x7f40b802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b894a3036a64b409 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.985+0000 7f40c6ffd640 1 -- 172.21.15.26:0/129660648 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8037070 con 0x7f40c8145a80 2024-04-24T23:28:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.985+0000 7f40c6ffd640 1 -- 172.21.15.26:0/129660648 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b8032440 con 0x7f40c8145a80 2024-04-24T23:28:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.985+0000 7f40c6ffd640 1 -- 172.21.15.26:0/129660648 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8032810 con 0x7f40c8145a80 2024-04-24T23:28:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.986+0000 7f40ce990640 1 -- 172.21.15.26:0/129660648 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 msgr2=0x7f40c80ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.986+0000 7f40ce990640 1 --2- 172.21.15.26:0/129660648 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c80ce040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f40b80098a0 tx=0x7f40b802a290 comp rx=0 tx=0).stop 2024-04-24T23:28:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.986+0000 7f40ce990640 1 -- 172.21.15.26:0/129660648 shutdown_connections 2024-04-24T23:28:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.986+0000 7f40ce990640 1 --2- 172.21.15.26:0/129660648 >> v2:172.21.15.29:3300/0 conn(0x7f40c815c5b0 0x7f40c806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.986+0000 7f40ce990640 1 --2- 172.21.15.26:0/129660648 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c80ce040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.986+0000 7f40ce990640 1 -- 172.21.15.26:0/129660648 >> 172.21.15.26:0/129660648 conn(0x7f40c806b540 msgr2=0x7f40c806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.986+0000 7f40ce990640 1 -- 172.21.15.26:0/129660648 shutdown_connections 2024-04-24T23:28:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.986+0000 7f40ce990640 1 -- 172.21.15.26:0/129660648 wait complete. 2024-04-24T23:28:00.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.987+0000 7f40ce990640 1 Processor -- start 2024-04-24T23:28:00.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.987+0000 7f40ce990640 1 -- start start 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40ce990640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c814b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40ce990640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40c815c5b0 0x7f40c8142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40ce990640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40c815ca10 0x7f40c8143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40ce990640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f40c80b4920 con 0x7f40c815c5b0 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40ce990640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f40c80b47a0 con 0x7f40c815ca10 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40ce990640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f40c80b4aa0 con 0x7f40c8145a80 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40c815c5b0 0x7f40c8142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40c815c5b0 0x7f40c8142d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51426/0 (socket says 172.21.15.26:51426) 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40c77fe640 1 -- 172.21.15.26:0/2045373956 learned_addr learned my addr 172.21.15.26:0/2045373956 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40c7fff640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.988+0000 7f40ccf2d640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40c815ca10 0x7f40c8143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c7fff640 1 -- 172.21.15.26:0/2045373956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40c815ca10 msgr2=0x7f40c8143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c7fff640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40c815ca10 0x7f40c8143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c7fff640 1 -- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40c815c5b0 msgr2=0x7f40c8142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c7fff640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40c815c5b0 0x7f40c8142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c7fff640 1 -- 172.21.15.26:0/2045373956 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c814ba60 con 0x7f40c8145a80 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c77fe640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40c815c5b0 0x7f40c8142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c7fff640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c814b520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f40b8031000 tx=0x7f40b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c4ff9640 1 -- 172.21.15.26:0/2045373956 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8041020 con 0x7f40c8145a80 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c4ff9640 1 -- 172.21.15.26:0/2045373956 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b8030660 con 0x7f40c8145a80 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c8143260 con 0x7f40c8145a80 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40c4ff9640 1 -- 172.21.15.26:0/2045373956 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8037040 con 0x7f40c8145a80 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.989+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c813af40 con 0x7f40c8145a80 2024-04-24T23:28:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.990+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c80694c0 con 0x7f40c8145a80 2024-04-24T23:28:00.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.991+0000 7f40c4ff9640 1 -- 172.21.15.26:0/2045373956 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40b8044020 con 0x7f40c8145a80 2024-04-24T23:28:00.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.991+0000 7f40c4ff9640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f40b406be60 0x7f40b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.991+0000 7f40c4ff9640 1 -- 172.21.15.26:0/2045373956 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f40b80ab5e0 con 0x7f40c8145a80 2024-04-24T23:28:00.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.991+0000 7f40c77fe640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f40b406be60 0x7f40b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.992+0000 7f40c77fe640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f40b406be60 0x7f40b406e320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f40c81399b0 tx=0x7f40bc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:00.996+0000 7f40c4ff9640 1 -- 172.21.15.26:0/2045373956 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f40b804b030 con 0x7f40c8145a80 2024-04-24T23:28:01.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.248+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f40c8144e50 con 0x7f40c8145a80 2024-04-24T23:28:01.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.248+0000 7f40c4ff9640 1 -- 172.21.15.26:0/2045373956 <== mon.2 v2:172.21.15.29:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1967 (secure 0 0 0) 0x7f40b8079920 con 0x7f40c8145a80 2024-04-24T23:28:01.249 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:01.249 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":450,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700753,"bytes_used":287752192,"bytes_avail":772806361088,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6739,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":113,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-24T23:27:57.408420+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:28:01.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f40b406be60 msgr2=0x7f40b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f40b406be60 0x7f40b406e320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f40c81399b0 tx=0x7f40bc009190 comp rx=0 tx=0).stop 2024-04-24T23:28:01.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 msgr2=0x7f40c814b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c814b520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f40b8031000 tx=0x7f40b80046c0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 shutdown_connections 2024-04-24T23:28:01.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f40b406be60 0x7f40b406e320 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f40c815ca10 0x7f40c8143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40c815c5b0 0x7f40c8142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 --2- 172.21.15.26:0/2045373956 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f40c8145a80 0x7f40c814b520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 >> 172.21.15.26:0/2045373956 conn(0x7f40c806b540 msgr2=0x7f40c80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 shutdown_connections 2024-04-24T23:28:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.250+0000 7f40ce990640 1 -- 172.21.15.26:0/2045373956 wait complete. 2024-04-24T23:28:01.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:28:01.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:01.262 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-24T23:28:01.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.364+0000 7f3f7cf7e640 1 Processor -- start 2024-04-24T23:28:01.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.364+0000 7f3f7cf7e640 1 -- start start 2024-04-24T23:28:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.365+0000 7f3f7cf7e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f7815e370 0x7f3f780d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.365+0000 7f3f7cf7e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f780d4c10 0x7f3f780cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.365+0000 7f3f7cf7e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f780d2120 con 0x7f3f7815e750 2024-04-24T23:28:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.365+0000 7f3f7cf7e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3f780d1510 con 0x7f3f780d4c10 2024-04-24T23:28:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.365+0000 7f3f7cf7e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3f780d1ed0 con 0x7f3f7815e370 2024-04-24T23:28:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.365+0000 7f3f76d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f780d4c10 0x7f3f780cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.365+0000 7f3f76d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f780d4c10 0x7f3f780cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51432/0 (socket says 172.21.15.26:51432) 2024-04-24T23:28:01.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.365+0000 7f3f76d9d640 1 -- 172.21.15.26:0/1563635018 learned_addr learned my addr 172.21.15.26:0/1563635018 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.366+0000 7f3f7659c640 1 --2- 172.21.15.26:0/1563635018 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f7815e370 0x7f3f780d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.366+0000 7f3f76d9d640 1 -- 172.21.15.26:0/1563635018 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f7815e370 msgr2=0x7f3f780d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.366+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/1563635018 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f7815e370 0x7f3f780d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.366+0000 7f3f76d9d640 1 -- 172.21.15.26:0/1563635018 >> v1:172.21.15.29:6789/0 conn(0x7f3f7815e750 legacy=0x7f3f780b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.366+0000 7f3f76d9d640 1 -- 172.21.15.26:0/1563635018 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f780b3740 con 0x7f3f780d4c10 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.366+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/1563635018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f780d4c10 0x7f3f780cfca0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009e50 tx=0x7f3f6802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb060a90a72a0df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7559a640 1 -- 172.21.15.26:0/1563635018 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68036070 con 0x7f3f780d4c10 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7559a640 1 -- 172.21.15.26:0/1563635018 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f68031600 con 0x7f3f780d4c10 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7559a640 1 -- 172.21.15.26:0/1563635018 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680319d0 con 0x7f3f780d4c10 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/1563635018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f780d4c10 msgr2=0x7f3f780cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7cf7e640 1 --2- 172.21.15.26:0/1563635018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f780d4c10 0x7f3f780cfca0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009e50 tx=0x7f3f6802a710 comp rx=0 tx=0).stop 2024-04-24T23:28:01.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/1563635018 shutdown_connections 2024-04-24T23:28:01.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7cf7e640 1 --2- 172.21.15.26:0/1563635018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f780d4c10 0x7f3f780cfca0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7cf7e640 1 --2- 172.21.15.26:0/1563635018 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f7815e370 0x7f3f780d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/1563635018 >> 172.21.15.26:0/1563635018 conn(0x7f3f780d40f0 msgr2=0x7f3f780d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/1563635018 shutdown_connections 2024-04-24T23:28:01.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.367+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/1563635018 wait complete. 2024-04-24T23:28:01.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f7cf7e640 1 Processor -- start 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f7cf7e640 1 -- start start 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f7cf7e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f780d4c10 0x7f3f781f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f7cf7e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f7815e370 0x7f3f780cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f7cf7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f780c9400 0x7f3f780cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f7cf7e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f780d2a60 con 0x7f3f7815e370 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f7cf7e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f780b3d00 con 0x7f3f780c9400 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f7cf7e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3f780d2be0 con 0x7f3f780d4c10 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f75d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f7815e370 0x7f3f780cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f75d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f7815e370 0x7f3f780cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51446/0 (socket says 172.21.15.26:51446) 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.369+0000 7f3f75d9b640 1 -- 172.21.15.26:0/2132133584 learned_addr learned my addr 172.21.15.26:0/2132133584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f7659c640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f780d4c10 0x7f3f781f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f780c9400 0x7f3f780cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f75d9b640 1 -- 172.21.15.26:0/2132133584 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f780d4c10 msgr2=0x7f3f781f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f780d4c10 0x7f3f781f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f75d9b640 1 -- 172.21.15.26:0/2132133584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f780c9400 msgr2=0x7f3f780cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f780c9400 0x7f3f780cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f75d9b640 1 -- 172.21.15.26:0/2132133584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f780c9d90 con 0x7f3f7815e370 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f7815e370 0x7f3f780cc770 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000c3c0 tx=0x7f3f6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f6effd640 1 -- 172.21.15.26:0/2132133584 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600052d0 con 0x7f3f7815e370 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f6effd640 1 -- 172.21.15.26:0/2132133584 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f60005490 con 0x7f3f7815e370 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/2132133584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f780ca070 con 0x7f3f7815e370 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f6effd640 1 -- 172.21.15.26:0/2132133584 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60007260 con 0x7f3f7815e370 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.370+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/2132133584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f780ca910 con 0x7f3f7815e370 2024-04-24T23:28:01.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.372+0000 7f3f6effd640 1 -- 172.21.15.26:0/2132133584 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f60007400 con 0x7f3f7815e370 2024-04-24T23:28:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.372+0000 7f3f6effd640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f5806be30 0x7f3f5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.372+0000 7f3f6effd640 1 -- 172.21.15.26:0/2132133584 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f3f6008a340 con 0x7f3f7815e370 2024-04-24T23:28:01.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.372+0000 7f3f7659c640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f5806be30 0x7f3f5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.373+0000 7f3f7659c640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f5806be30 0x7f3f5806e2f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3f780d0a00 tx=0x7f3f70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.373+0000 7f3f7fd6f740 1 Processor -- start 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.373+0000 7f3f7fd6f740 1 -- start start 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9b2d60 0x561cac9b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9d0600 0x561cac9d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac988a40 0x561cac9cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561cac8ec6b0 con 0x561cac988a40 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561cac894880 con 0x561cac9b2d60 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561cac8e8670 con 0x561cac9d0600 2024-04-24T23:28:01.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f76d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac988a40 0x561cac9cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f76d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac988a40 0x561cac9cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51462/0 (socket says 172.21.15.26:51462) 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f76d9d640 1 -- 172.21.15.26:0/2039961092 learned_addr learned my addr 172.21.15.26:0/2039961092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f7659c640 1 --2- 172.21.15.26:0/2039961092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9b2d60 0x561cac9b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/2039961092 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9d0600 0x561cac9d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f76d9d640 1 -- 172.21.15.26:0/2039961092 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9d0600 msgr2=0x561cac9d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/2039961092 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9d0600 0x561cac9d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f76d9d640 1 -- 172.21.15.26:0/2039961092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9b2d60 msgr2=0x561cac9b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/2039961092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9b2d60 0x561cac9b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f76d9d640 1 -- 172.21.15.26:0/2039961092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cac6ca7d0 con 0x561cac988a40 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/2039961092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac988a40 0x561cac9cecc0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009e50 tx=0x7f3f6802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362c7caa6ca803a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f57fff640 1 -- 172.21.15.26:0/2039961092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68036040 con 0x561cac988a40 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f57fff640 1 -- 172.21.15.26:0/2039961092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6803c660 con 0x561cac988a40 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.374+0000 7f3f57fff640 1 -- 172.21.15.26:0/2039961092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6803ca50 con 0x561cac988a40 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.375+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/2039961092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac988a40 msgr2=0x561cac9cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.375+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/2039961092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac988a40 0x561cac9cecc0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009e50 tx=0x7f3f6802e000 comp rx=0 tx=0).stop 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.375+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/2039961092 shutdown_connections 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.375+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/2039961092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac988a40 0x561cac9cecc0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.375+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/2039961092 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9d0600 0x561cac9d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.375+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/2039961092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9b2d60 0x561cac9b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.375+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/2039961092 >> 172.21.15.26:0/2039961092 conn(0x561cac985600 msgr2=0x561cac9cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.375+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/2039961092 shutdown_connections 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.375+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/2039961092 wait complete. 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.376+0000 7f3f7fd6f740 1 -- start start 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.376+0000 7f3f7fd6f740 1 Processor -- start 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac9e1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9d0600 0x561cac9ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9de090 0x561cac9b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561cac9897e0 con 0x561cac9de090 2024-04-24T23:28:01.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561cac8a7f50 con 0x561cac9d0600 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561cac983530 con 0x561cac9b2d60 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f7fd6f740 10 client.0 _ll_register_callbacks cb 0x561cac8e3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f76d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9de090 0x561cac9b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f76d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9de090 0x561cac9b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51472/0 (socket says 172.21.15.26:51472) 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f76d9d640 1 -- 172.21.15.26:0/1400932194 learned_addr learned my addr 172.21.15.26:0/1400932194 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f7659c640 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac9e1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9d0600 0x561cac9ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f76d9d640 1 -- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 msgr2=0x561cac9e1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac9e1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f76d9d640 1 -- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9d0600 msgr2=0x561cac9ddb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9d0600 0x561cac9ddb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f76d9d640 1 -- 172.21.15.26:0/1400932194 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cac6003f0 con 0x561cac9de090 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9de090 0x561cac9b4f20 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f3f6802ad00 tx=0x7f3f6802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecd12ec77f837c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.377+0000 7f3f557fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f557fa640 1 -- 172.21.15.26:0/1400932194 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68036050 con 0x561cac9de090 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f557fa640 1 -- 172.21.15.26:0/1400932194 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f3f6803ca10 con 0x561cac9de090 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1400932194 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f6803c660 con 0x561cac9de090 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f557fa640 1 -- 172.21.15.26:0/1400932194 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f3f68053050 con 0x561cac9de090 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7fd6f740 10 client.9375 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1400932194 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x561cac5c00f0 con 0x561cac9de090 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f557fa640 1 -- 172.21.15.26:0/1400932194 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6803c660 con 0x561cac9de090 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f557fa640 1 -- 172.21.15.26:0/1400932194 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f3f68056020 con 0x561cac9de090 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7fd6f740 10 client.9375 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7fd6f740 10 client.9375 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7fd6f740 20 client.9375 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x561cac9ceaf0 0x561cac9cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7fd6f740 4 client.9375 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1400932194 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cac8f9bf0 con 0x561cac9ceaf0 2024-04-24T23:28:01.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.378+0000 7f3f7659c640 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x561cac9ceaf0 0x561cac9cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.379+0000 7f3f557fa640 10 client.9375 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.379+0000 7f3f7659c640 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x561cac9ceaf0 0x561cac9cef00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13abbda111ae895e server_cookie=96f6e9717cd6f5b4 in_seq=0 out_seq=0 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.379+0000 7f3f557fa640 1 -- 172.21.15.26:0/1400932194 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561cac8f9bf0 con 0x561cac9ceaf0 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.379+0000 7f3f557fa640 10 client.9375 handle_command_reply: tid=0 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.379+0000 7f3f7fd6f740 1 client.9375 shutdown 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1400932194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9de090 msgr2=0x561cac9b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9de090 0x561cac9b4f20 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f3f6802ad00 tx=0x7f3f6802c000 comp rx=0 tx=0).stop 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1400932194 shutdown_connections 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x561cac9ceaf0 0x561cac9cef00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9de090 0x561cac9b4f20 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9d0600 0x561cac9ddb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1400932194 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac9e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f557fa640 0 client.9375 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1400932194 >> 172.21.15.26:0/1400932194 conn(0x561cac988a40 msgr2=0x561cac9d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1400932194 shutdown_connections 2024-04-24T23:28:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1400932194 wait complete. 2024-04-24T23:28:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.380+0000 7f3f7fd6f740 20 client.9375 trim_cache size 0 max 16384 2024-04-24T23:28:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.397+0000 7f3f7fd6f740 1 Processor -- start 2024-04-24T23:28:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.397+0000 7f3f7fd6f740 1 -- start start 2024-04-24T23:28:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac989e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac9dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 0x561cac988e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f78200720 con 0x561cac9d0600 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781ff920 con 0x561cac9de090 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f3f78200020 con 0x561cac9b2d60 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f75d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac9dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f75d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac9dd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51474/0 (socket says 172.21.15.26:51474) 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f75d9b640 1 -- 172.21.15.26:0/1141446705 learned_addr learned my addr 172.21.15.26:0/1141446705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/1141446705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 0x561cac988e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f7659c640 1 --2- 172.21.15.26:0/1141446705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac989e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f75d9b640 1 -- 172.21.15.26:0/1141446705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 msgr2=0x561cac989e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/1141446705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac989e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f75d9b640 1 -- 172.21.15.26:0/1141446705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 msgr2=0x561cac988e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/1141446705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 0x561cac988e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f75d9b640 1 -- 172.21.15.26:0/1141446705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cac9aca50 con 0x561cac9d0600 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/1141446705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac9dd750 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3f60007810 tx=0x7f3f6008a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c161dd08fa806e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f577fe640 1 -- 172.21.15.26:0/1141446705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600611d0 con 0x561cac9d0600 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f577fe640 1 -- 172.21.15.26:0/1141446705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f600568f0 con 0x561cac9d0600 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.398+0000 7f3f577fe640 1 -- 172.21.15.26:0/1141446705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60054c60 con 0x561cac9d0600 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1141446705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 msgr2=0x561cac9dd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1141446705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac9dd750 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3f60007810 tx=0x7f3f6008a040 comp rx=0 tx=0).stop 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1141446705 shutdown_connections 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1141446705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 0x561cac988e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1141446705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac9dd750 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/1141446705 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac989e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1141446705 >> 172.21.15.26:0/1141446705 conn(0x561cac988a40 msgr2=0x561cac9f18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1141446705 shutdown_connections 2024-04-24T23:28:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/1141446705 wait complete. 2024-04-24T23:28:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.399+0000 7f3f7fd6f740 1 -- start start 2024-04-24T23:28:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.400+0000 7f3f7fd6f740 1 Processor -- start 2024-04-24T23:28:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.400+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac9c5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.400+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac988e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.400+0000 7f3f7fd6f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 0x561cac989360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.400+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781f8000 con 0x561cac9d0600 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.400+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561cac9b4200 con 0x561cac9de090 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.400+0000 7f3f7fd6f740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x561cac9822f0 con 0x561cac9b2d60 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f7fd6f740 10 client.0 _ll_register_callbacks cb 0x7f3f70009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f75d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac988e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f75d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac988e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51486/0 (socket says 172.21.15.26:51486) 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f75d9b640 1 -- 172.21.15.26:0/3793212281 learned_addr learned my addr 172.21.15.26:0/3793212281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f7659c640 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac9c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 0x561cac989360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f75d9b640 1 -- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 msgr2=0x561cac9c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac9c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f75d9b640 1 -- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 msgr2=0x561cac989360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 0x561cac989360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f75d9b640 1 -- 172.21.15.26:0/3793212281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cac982540 con 0x561cac9d0600 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f76d9d640 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 0x561cac989360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f75d9b640 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac988e20 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000c390 tx=0x7f3f60000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5499b1ef1ab714a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f557fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f557fa640 1 -- 172.21.15.26:0/3793212281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600611d0 con 0x561cac9d0600 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f557fa640 1 -- 172.21.15.26:0/3793212281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f3f60054c60 con 0x561cac9d0600 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/3793212281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f600568f0 con 0x561cac9d0600 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f557fa640 1 -- 172.21.15.26:0/3793212281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f3f6005db30 con 0x561cac9d0600 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f7fd6f740 10 client.9381 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/3793212281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x561cac9b4e40 con 0x561cac9d0600 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.401+0000 7f3f557fa640 1 -- 172.21.15.26:0/3793212281 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f600568f0 con 0x561cac9d0600 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f557fa640 1 -- 172.21.15.26:0/3793212281 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f3f6005dcd0 con 0x561cac9d0600 2024-04-24T23:28:01.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f7fd6f740 10 client.9381 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f7fd6f740 10 client.9381 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f7fd6f740 20 client.9381 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x561cac9b3a20 0x561cac9d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f7fd6f740 4 client.9381 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/3793212281 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x561cac8f3f40 con 0x561cac9b3a20 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f7659c640 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x561cac9b3a20 0x561cac9d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f557fa640 10 client.9381 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.402+0000 7f3f7659c640 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x561cac9b3a20 0x561cac9d0e30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde2e9ea814d6d40 server_cookie=90db501e99672472 in_seq=0 out_seq=0 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f557fa640 1 -- 172.21.15.26:0/3793212281 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561cac8f3f40 con 0x561cac9b3a20 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f557fa640 10 client.9381 handle_command_reply: tid=0 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 client.9381 shutdown 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/3793212281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 msgr2=0x561cac988e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac988e20 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000c390 tx=0x7f3f60000ed0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/3793212281 shutdown_connections 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x561cac9b3a20 0x561cac9d0e30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cac9de090 0x561cac989360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cac9d0600 0x561cac988e20 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 --2- 172.21.15.26:0/3793212281 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x561cac9b2d60 0x561cac9c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f557fa640 0 client.9381 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/3793212281 >> 172.21.15.26:0/3793212281 conn(0x561cac988a40 msgr2=0x561cac987ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/3793212281 shutdown_connections 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 1 -- 172.21.15.26:0/3793212281 wait complete. 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7fd6f740 20 client.9381 trim_cache size 0 max 16384 2024-04-24T23:28:01.410 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:01.411 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0 2024-04-24T23:28:01.411 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f5806be30 msgr2=0x7f3f5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f5806be30 0x7f3f5806e2f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3f780d0a00 tx=0x7f3f70008040 comp rx=0 tx=0).stop 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f7815e370 msgr2=0x7f3f780cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f7815e370 0x7f3f780cc770 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000c3c0 tx=0x7f3f6000c880 comp rx=0 tx=0).stop 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/2132133584 shutdown_connections 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f3f5806be30 0x7f3f5806e2f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f780c9400 0x7f3f780cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f7815e370 0x7f3f780cc770 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 --2- 172.21.15.26:0/2132133584 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3f780d4c10 0x7f3f781f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.403+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/2132133584 >> 172.21.15.26:0/2132133584 conn(0x7f3f780d40f0 msgr2=0x7f3f7806c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.404+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/2132133584 shutdown_connections 2024-04-24T23:28:01.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.404+0000 7f3f7cf7e640 1 -- 172.21.15.26:0/2132133584 wait complete. 2024-04-24T23:28:01.415 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:28:01.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-04-24T23:28:01.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.517+0000 7f2053d53640 1 Processor -- start 2024-04-24T23:28:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.517+0000 7f2053d53640 1 -- start start 2024-04-24T23:28:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2053d53640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f204c06d5a0 0x7f204c0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2053d53640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c0b3d20 0x7f204c150e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2053d53640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f204c070f30 con 0x7f204c15e120 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2053d53640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f204c06dcf0 con 0x7f204c06d5a0 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2053d53640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f204c06dfd0 con 0x7f204c0b3d20 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2051aef640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f204c15e120 0x7f204c15e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34024/0 (socket says 172.21.15.26:34024) 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2051aef640 1 -- 172.21.15.26:0/332547958 learned_addr learned my addr 172.21.15.26:0/332547958 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f20522f0640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c0b3d20 0x7f204c150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f20512ee640 1 --2- 172.21.15.26:0/332547958 >> v2:172.21.15.29:3300/0 conn(0x7f204c06d5a0 0x7f204c0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447713644 0 0) 0x7f204c070f30 con 0x7f204c15e120 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2038003520 con 0x7f204c15e120 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.518+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2317350861 0 0) 0x7f2038003520 con 0x7f204c15e120 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c0b3d20 msgr2=0x7f204c150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c0b3d20 0x7f204c150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 >> v2:172.21.15.29:3300/0 conn(0x7f204c06d5a0 msgr2=0x7f204c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 --2- 172.21.15.26:0/332547958 >> v2:172.21.15.29:3300/0 conn(0x7f204c06d5a0 0x7f204c0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c06e2b0 con 0x7f204c15e120 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f20522f0640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c0b3d20 0x7f204c150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f2048001670 con 0x7f204c15e120 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 >> v1:172.21.15.26:6789/0 conn(0x7f204c15e120 legacy=0x7f204c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c0b3d20 0x7f20380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2038003ef0 0x7f2038004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2038004d40 0x7f20380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f204c06dcf0 con 0x7f2038004d40 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f204c06dfd0 con 0x7f2038003ef0 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f203800b9c0 con 0x7f204c0b3d20 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f20522f0640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2038004d40 0x7f20380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f2051aef640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c0b3d20 0x7f20380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f20512ee640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2038003ef0 0x7f2038004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f20522f0640 1 -- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c0b3d20 msgr2=0x7f20380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f20522f0640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c0b3d20 0x7f20380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f20522f0640 1 -- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2038003ef0 msgr2=0x7f2038004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f20522f0640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2038003ef0 0x7f2038004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.519+0000 7f20522f0640 1 -- 172.21.15.26:0/332547958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2038003520 con 0x7f2038004d40 2024-04-24T23:28:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.520+0000 7f20522f0640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2038004d40 0x7f20380051c0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f204400aee0 tx=0x7f2038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eda3be3f7e933e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.520+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204400f140 con 0x7f2038004d40 2024-04-24T23:28:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.520+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204400a2d0 con 0x7f2038004d40 2024-04-24T23:28:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.520+0000 7f2050aed640 1 -- 172.21.15.26:0/332547958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204402d760 con 0x7f2038004d40 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.520+0000 7f2053d53640 1 -- 172.21.15.26:0/332547958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2038004d40 msgr2=0x7f20380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.520+0000 7f2053d53640 1 --2- 172.21.15.26:0/332547958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2038004d40 0x7f20380051c0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f204400aee0 tx=0x7f2038005830 comp rx=0 tx=0).stop 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.520+0000 7f20522f0640 1 -- 172.21.15.26:0/332547958 reap_dead start 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.520+0000 7f2053d53640 1 -- 172.21.15.26:0/332547958 shutdown_connections 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.520+0000 7f2053d53640 1 -- 172.21.15.26:0/332547958 >> 172.21.15.26:0/332547958 conn(0x7f204c06b540 msgr2=0x7f204c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.521+0000 7f2053d53640 1 -- 172.21.15.26:0/332547958 shutdown_connections 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.521+0000 7f2053d53640 1 -- 172.21.15.26:0/332547958 wait complete. 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.522+0000 7f2053d53640 1 Processor -- start 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.522+0000 7f2053d53640 1 -- start start 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.522+0000 7f2053d53640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c14a990 0x7f204c156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.522+0000 7f2053d53640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f204c1572a0 0x7f204c09ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.522+0000 7f2053d53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204c09b1b0 0x7f204c155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.522+0000 7f2053d53640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f204c0d0040 con 0x7f204c09b1b0 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.522+0000 7f2053d53640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f204c06da10 con 0x7f204c1572a0 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.522+0000 7f2053d53640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f204c0d01c0 con 0x7f204c14a990 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20522f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204c09b1b0 0x7f204c155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20522f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204c09b1b0 0x7f204c155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51502/0 (socket says 172.21.15.26:51502) 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20522f0640 1 -- 172.21.15.26:0/2420259575 learned_addr learned my addr 172.21.15.26:0/2420259575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f2051aef640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c14a990 0x7f204c156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20512ee640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f204c1572a0 0x7f204c09ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20522f0640 1 -- 172.21.15.26:0/2420259575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c14a990 msgr2=0x7f204c156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20522f0640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c14a990 0x7f204c156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20522f0640 1 -- 172.21.15.26:0/2420259575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f204c1572a0 msgr2=0x7f204c09ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20522f0640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f204c1572a0 0x7f204c09ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20522f0640 1 -- 172.21.15.26:0/2420259575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c09ba10 con 0x7f204c09b1b0 2024-04-24T23:28:01.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20522f0640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204c09b1b0 0x7f204c155050 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f204402d250 tx=0x7f204400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20427fc640 1 -- 172.21.15.26:0/2420259575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204400ab60 con 0x7f204c09b1b0 2024-04-24T23:28:01.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20427fc640 1 -- 172.21.15.26:0/2420259575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204400a0d0 con 0x7f204c09b1b0 2024-04-24T23:28:01.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f2053d53640 1 -- 172.21.15.26:0/2420259575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c09bcf0 con 0x7f204c09b1b0 2024-04-24T23:28:01.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f20427fc640 1 -- 172.21.15.26:0/2420259575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044003bb0 con 0x7f204c09b1b0 2024-04-24T23:28:01.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.523+0000 7f2053d53640 1 -- 172.21.15.26:0/2420259575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c09c2b0 con 0x7f204c09b1b0 2024-04-24T23:28:01.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.525+0000 7f20427fc640 1 -- 172.21.15.26:0/2420259575 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2044046030 con 0x7f204c09b1b0 2024-04-24T23:28:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.525+0000 7f20427fc640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f203006bfd0 0x7f203006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.526+0000 7f2051aef640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f203006bfd0 0x7f203006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.526+0000 7f20427fc640 1 -- 172.21.15.26:0/2420259575 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f20440ad260 con 0x7f204c09b1b0 2024-04-24T23:28:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.526+0000 7f2051aef640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f203006bfd0 0x7f203006e490 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2048004840 tx=0x7f20480078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.527+0000 7f2056a5e740 1 Processor -- start 2024-04-24T23:28:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.527+0000 7f2056a5e740 1 -- start start 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.527+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe77140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.527+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 0x562d1fe94b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.527+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe4ca40 0x562d1fe92cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.527+0000 7f2056a5e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562d1fdb06b0 con 0x562d1fe76d60 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.527+0000 7f2056a5e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562d1fd58880 con 0x562d1fe4ca40 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.527+0000 7f2056a5e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562d1fdac670 con 0x562d1fe94600 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f2051aef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe77140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f2051aef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe77140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51508/0 (socket says 172.21.15.26:51508) 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f2051aef640 1 -- 172.21.15.26:0/62007567 learned_addr learned my addr 172.21.15.26:0/62007567 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f20522f0640 1 --2- 172.21.15.26:0/62007567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe4ca40 0x562d1fe92cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f20512ee640 1 --2- 172.21.15.26:0/62007567 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 0x562d1fe94b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f2051aef640 1 -- 172.21.15.26:0/62007567 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 msgr2=0x562d1fe94b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f2051aef640 1 --2- 172.21.15.26:0/62007567 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 0x562d1fe94b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f2051aef640 1 -- 172.21.15.26:0/62007567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe4ca40 msgr2=0x562d1fe92cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f2051aef640 1 --2- 172.21.15.26:0/62007567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe4ca40 0x562d1fe92cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f2051aef640 1 -- 172.21.15.26:0/62007567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1fb8e7d0 con 0x562d1fe76d60 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f2051aef640 1 --2- 172.21.15.26:0/62007567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe77140 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2048007c40 tx=0x7f204800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90c56fd4e18af60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f202f7fe640 1 -- 172.21.15.26:0/62007567 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048028070 con 0x562d1fe76d60 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f202f7fe640 1 -- 172.21.15.26:0/62007567 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2048015d50 con 0x562d1fe76d60 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.528+0000 7f202f7fe640 1 -- 172.21.15.26:0/62007567 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20480215c0 con 0x562d1fe76d60 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 -- 172.21.15.26:0/62007567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 msgr2=0x562d1fe77140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 --2- 172.21.15.26:0/62007567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe77140 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2048007c40 tx=0x7f204800c000 comp rx=0 tx=0).stop 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 -- 172.21.15.26:0/62007567 shutdown_connections 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 --2- 172.21.15.26:0/62007567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe4ca40 0x562d1fe92cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 --2- 172.21.15.26:0/62007567 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 0x562d1fe94b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 --2- 172.21.15.26:0/62007567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe77140 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 -- 172.21.15.26:0/62007567 >> 172.21.15.26:0/62007567 conn(0x562d1fe49600 msgr2=0x562d1fe91b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 -- 172.21.15.26:0/62007567 shutdown_connections 2024-04-24T23:28:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 -- 172.21.15.26:0/62007567 wait complete. 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 -- start start 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.529+0000 7f2056a5e740 1 Processor -- start 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe61a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 0x562d1fe78d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 0x562d1fea5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2056a5e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562d1fe4d7e0 con 0x562d1fe76d60 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2056a5e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562d1fd6bf50 con 0x562d1fea55c0 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2056a5e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562d1fe47530 con 0x562d1fe94600 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2056a5e740 10 client.0 _ll_register_callbacks cb 0x562d1fda7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2051aef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe61a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2051aef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe61a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51514/0 (socket says 172.21.15.26:51514) 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.530+0000 7f2051aef640 1 -- 172.21.15.26:0/3905619754 learned_addr learned my addr 172.21.15.26:0/3905619754 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f20522f0640 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 0x562d1fea5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f20512ee640 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 0x562d1fe78d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2051aef640 1 -- 172.21.15.26:0/3905619754 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 msgr2=0x562d1fe78d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2051aef640 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 0x562d1fe78d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2051aef640 1 -- 172.21.15.26:0/3905619754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 msgr2=0x562d1fea5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2051aef640 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 0x562d1fea5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2051aef640 1 -- 172.21.15.26:0/3905619754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1fac43f0 con 0x562d1fe76d60 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f20522f0640 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 0x562d1fea5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2051aef640 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe61a70 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f2048024a70 tx=0x7f2048001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285691a1f58cab81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f202cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f202cff9640 1 -- 172.21.15.26:0/3905619754 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048032060 con 0x562d1fe76d60 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f202cff9640 1 -- 172.21.15.26:0/3905619754 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f2048024c50 con 0x562d1fe76d60 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2056a5e740 1 -- 172.21.15.26:0/3905619754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2048015d50 con 0x562d1fe76d60 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f202cff9640 1 -- 172.21.15.26:0/3905619754 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f2048038050 con 0x562d1fe76d60 2024-04-24T23:28:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2056a5e740 10 client.9393 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2056a5e740 1 -- 172.21.15.26:0/3905619754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x562d1fa840f0 con 0x562d1fe76d60 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f202cff9640 1 -- 172.21.15.26:0/3905619754 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2048015d50 con 0x562d1fe76d60 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f202cff9640 1 -- 172.21.15.26:0/3905619754 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f204803b020 con 0x562d1fe76d60 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.531+0000 7f2056a5e740 10 client.9393 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f2056a5e740 10 client.9393 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f2056a5e740 20 client.9393 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f2056a5e740 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x562d1fe648c0 0x562d1fe64ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f2056a5e740 4 client.9393 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f2056a5e740 1 -- 172.21.15.26:0/3905619754 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d1fdbdbf0 con 0x562d1fe648c0 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f20512ee640 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x562d1fe648c0 0x562d1fe64ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f20512ee640 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x562d1fe648c0 0x562d1fe64ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=979d0330ac07663a server_cookie=db55884e2b7d03aa in_seq=0 out_seq=0 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f202cff9640 10 client.9393 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f202cff9640 1 -- 172.21.15.26:0/3905619754 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d1fdbdbf0 con 0x562d1fe648c0 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.532+0000 7f202cff9640 10 client.9393 handle_command_reply: tid=0 2024-04-24T23:28:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 client.9393 shutdown 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 -- 172.21.15.26:0/3905619754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 msgr2=0x562d1fe61a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe61a70 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f2048024a70 tx=0x7f2048001a40 comp rx=0 tx=0).stop 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 -- 172.21.15.26:0/3905619754 shutdown_connections 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x562d1fe648c0 0x562d1fe64ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 0x562d1fea5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe94600 0x562d1fe78d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 --2- 172.21.15.26:0/3905619754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe76d60 0x562d1fe61a70 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f202cff9640 0 client.9393 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 -- 172.21.15.26:0/3905619754 >> 172.21.15.26:0/3905619754 conn(0x562d1fe4ca40 msgr2=0x562d1fe95980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 -- 172.21.15.26:0/3905619754 shutdown_connections 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 1 -- 172.21.15.26:0/3905619754 wait complete. 2024-04-24T23:28:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.533+0000 7f2056a5e740 20 client.9393 trim_cache size 0 max 16384 2024-04-24T23:28:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f2056a5e740 1 Processor -- start 2024-04-24T23:28:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f2056a5e740 1 -- start start 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 0x562d1fea48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe94600 0x562d1ff01e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 0x562d1ff02390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f2056a5e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1fcf90 con 0x562d1fe94600 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f2056a5e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1fc130 con 0x562d1fea55c0 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f2056a5e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f204c1fc860 con 0x562d1fe76d60 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f20512ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe94600 0x562d1ff01e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f20512ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe94600 0x562d1ff01e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51526/0 (socket says 172.21.15.26:51526) 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.551+0000 7f20512ee640 1 -- 172.21.15.26:0/1224141121 learned_addr learned my addr 172.21.15.26:0/1224141121 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f2051aef640 1 --2- 172.21.15.26:0/1224141121 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 0x562d1fea48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f20522f0640 1 --2- 172.21.15.26:0/1224141121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 0x562d1ff02390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f20512ee640 1 -- 172.21.15.26:0/1224141121 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 msgr2=0x562d1fea48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f20512ee640 1 --2- 172.21.15.26:0/1224141121 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 0x562d1fea48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f20512ee640 1 -- 172.21.15.26:0/1224141121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 msgr2=0x562d1ff02390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f20512ee640 1 --2- 172.21.15.26:0/1224141121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 0x562d1ff02390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f20512ee640 1 -- 172.21.15.26:0/1224141121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1fe94ab0 con 0x562d1fe94600 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f20512ee640 1 --2- 172.21.15.26:0/1224141121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe94600 0x562d1ff01e50 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f203c009f50 tx=0x7f203c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d2ad334589d4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f202effd640 1 -- 172.21.15.26:0/1224141121 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c0224e0 con 0x562d1fe94600 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f202effd640 1 -- 172.21.15.26:0/1224141121 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203c027070 con 0x562d1fe94600 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.552+0000 7f202effd640 1 -- 172.21.15.26:0/1224141121 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c0228e0 con 0x562d1fe94600 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 -- 172.21.15.26:0/1224141121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe94600 msgr2=0x562d1ff01e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1224141121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe94600 0x562d1ff01e50 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f203c009f50 tx=0x7f203c009190 comp rx=0 tx=0).stop 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 -- 172.21.15.26:0/1224141121 shutdown_connections 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1224141121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fea55c0 0x562d1ff02390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1224141121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fe94600 0x562d1ff01e50 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1224141121 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 0x562d1fea48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 -- 172.21.15.26:0/1224141121 >> 172.21.15.26:0/1224141121 conn(0x562d1fe4ca40 msgr2=0x562d1fe46260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 -- 172.21.15.26:0/1224141121 shutdown_connections 2024-04-24T23:28:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 -- 172.21.15.26:0/1224141121 wait complete. 2024-04-24T23:28:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 -- start start 2024-04-24T23:28:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.553+0000 7f2056a5e740 1 Processor -- start 2024-04-24T23:28:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 0x562d1fe78680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe94600 0x562d1fe60d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f2056a5e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fea55c0 0x562d1fea77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f2056a5e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562d1feb4c20 con 0x562d1fea55c0 2024-04-24T23:28:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f2056a5e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1fb250 con 0x562d1fe94600 2024-04-24T23:28:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f2056a5e740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x562d1fe432a0 con 0x562d1fe76d60 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f2056a5e740 10 client.0 _ll_register_callbacks cb 0x7f2048026070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f20522f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fea55c0 0x562d1fea77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f20522f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fea55c0 0x562d1fea77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51528/0 (socket says 172.21.15.26:51528) 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f20522f0640 1 -- 172.21.15.26:0/1217450681 learned_addr learned my addr 172.21.15.26:0/1217450681 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f20512ee640 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe94600 0x562d1fe60d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f2051aef640 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 0x562d1fe78680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f20522f0640 1 -- 172.21.15.26:0/1217450681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 msgr2=0x562d1fe78680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.554+0000 7f20522f0640 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 0x562d1fe78680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f20522f0640 1 -- 172.21.15.26:0/1217450681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe94600 msgr2=0x562d1fe60d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f20522f0640 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe94600 0x562d1fe60d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f20522f0640 1 -- 172.21.15.26:0/1217450681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1fe61e20 con 0x562d1fea55c0 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2051aef640 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 0x562d1fe78680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f20522f0640 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fea55c0 0x562d1fea77d0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f204400aee0 tx=0x7f204407b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f011dfe08abe60c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f202cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f202cff9640 1 -- 172.21.15.26:0/1217450681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204407b5d0 con 0x562d1fea55c0 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f202cff9640 1 -- 172.21.15.26:0/1217450681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f2044094e20 con 0x562d1fea55c0 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2056a5e740 1 -- 172.21.15.26:0/1217450681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f203c027070 con 0x562d1fea55c0 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f202cff9640 1 -- 172.21.15.26:0/1217450681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f204407f980 con 0x562d1fea55c0 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2056a5e740 10 client.9399 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2056a5e740 1 -- 172.21.15.26:0/1217450681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x562d1fe77ee0 con 0x562d1fea55c0 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f202cff9640 1 -- 172.21.15.26:0/1217450681 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203c027070 con 0x562d1fea55c0 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f202cff9640 1 -- 172.21.15.26:0/1217450681 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7f204407b830 con 0x562d1fea55c0 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2056a5e740 10 client.9399 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2056a5e740 10 client.9399 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2056a5e740 20 client.9399 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x562d1fe87fa0 0x562d1fea7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2056a5e740 4 client.9399 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2056a5e740 1 -- 172.21.15.26:0/1217450681 --> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562d1fe8da20 con 0x562d1fe87fa0 2024-04-24T23:28:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.555+0000 7f2051aef640 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x562d1fe87fa0 0x562d1fea7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2051aef640 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x562d1fe87fa0 0x562d1fea7d10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=353ac6a0f6a28770 server_cookie=cc840b78b00d5c9a in_seq=0 out_seq=0 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f202cff9640 10 client.9399 ms_handle_connect on v2:172.21.15.26:6834/3062763802 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f202cff9640 1 -- 172.21.15.26:0/1217450681 <== mds.1 v2:172.21.15.26:6834/3062763802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x562d1fe8da20 con 0x562d1fe87fa0 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f202cff9640 10 client.9399 handle_command_reply: tid=0 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 client.9399 shutdown 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 -- 172.21.15.26:0/1217450681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fea55c0 msgr2=0x562d1fea77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fea55c0 0x562d1fea77d0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f204400aee0 tx=0x7f204407b1d0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 -- 172.21.15.26:0/1217450681 shutdown_connections 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.26:6834/3062763802,v1:172.21.15.26:6835/3062763802] conn(0x562d1fe87fa0 0x562d1fea7d10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562d1fea55c0 0x562d1fea77d0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d1fe94600 0x562d1fe60d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f202cff9640 0 client.9399 ms_handle_reset on v2:172.21.15.26:6834/3062763802 2024-04-24T23:28:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 --2- 172.21.15.26:0/1217450681 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x562d1fe76d60 0x562d1fe78680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 -- 172.21.15.26:0/1217450681 >> 172.21.15.26:0/1217450681 conn(0x562d1fe4ca40 msgr2=0x562d1fe63730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 -- 172.21.15.26:0/1217450681 shutdown_connections 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 1 -- 172.21.15.26:0/1217450681 wait complete. 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.556+0000 7f2056a5e740 20 client.9399 trim_cache size 0 max 16384 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:28:01.563 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:28:01.564 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:01.564 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:01.564 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 -- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f203006bfd0 msgr2=0x7f203006e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f203006bfd0 0x7f203006e490 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2048004840 tx=0x7f20480078f0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 -- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204c09b1b0 msgr2=0x7f204c155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204c09b1b0 0x7f204c155050 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f204402d250 tx=0x7f204400a860 comp rx=0 tx=0).stop 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 -- 172.21.15.26:0/2420259575 shutdown_connections 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f203006bfd0 0x7f203006e490 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204c09b1b0 0x7f204c155050 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f204c1572a0 0x7f204c09ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 --2- 172.21.15.26:0/2420259575 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f204c14a990 0x7f204c156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 -- 172.21.15.26:0/2420259575 >> 172.21.15.26:0/2420259575 conn(0x7f204c06b540 msgr2=0x7f204c15b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 -- 172.21.15.26:0/2420259575 shutdown_connections 2024-04-24T23:28:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.557+0000 7f2053d53640 1 -- 172.21.15.26:0/2420259575 wait complete. 2024-04-24T23:28:01.568 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:28:01.568 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:28:01.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-04-24T23:28:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.671+0000 7efd913ce640 1 Processor -- start 2024-04-24T23:28:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.671+0000 7efd913ce640 1 -- start start 2024-04-24T23:28:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd913ce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efd8c0afc20 0x7efd8c071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd913ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c071c60 0x7efd8c165d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd913ce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd8c0b5410 con 0x7efd8c15c270 2024-04-24T23:28:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd913ce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efd8c0ce4b0 con 0x7efd8c0afc20 2024-04-24T23:28:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd913ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c15c650 con 0x7efd8c071c60 2024-04-24T23:28:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd8b7fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c071c60 0x7efd8c165d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd8a7fc640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efd8c0afc20 0x7efd8c071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd8affd640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7efd8c15c270 0x7efd8c0d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:42670/0 (socket says 172.21.15.26:42670) 2024-04-24T23:28:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd8affd640 1 -- 172.21.15.26:0/3323298005 learned_addr learned my addr 172.21.15.26:0/3323298005 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd89ffb640 1 -- 172.21.15.26:0/3323298005 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495111530 0 0) 0x7efd8c0b5410 con 0x7efd8c15c270 2024-04-24T23:28:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd89ffb640 1 -- 172.21.15.26:0/3323298005 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd74003520 con 0x7efd8c15c270 2024-04-24T23:28:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd8a7fc640 1 -- 172.21.15.26:0/3323298005 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c071c60 msgr2=0x7efd8c165d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/3323298005 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c071c60 0x7efd8c165d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd8a7fc640 1 -- 172.21.15.26:0/3323298005 >> v1:172.21.15.29:6789/0 conn(0x7efd8c15c270 legacy=0x7efd8c0d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.672+0000 7efd8a7fc640 1 -- 172.21.15.26:0/3323298005 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c0b56f0 con 0x7efd8c0afc20 2024-04-24T23:28:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.673+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/3323298005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd8c0afc20 0x7efd8c071720 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efd800097d0 tx=0x7efd8002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b96de11ad40cbc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.673+0000 7efd89ffb640 1 -- 172.21.15.26:0/3323298005 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80037070 con 0x7efd8c0afc20 2024-04-24T23:28:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.673+0000 7efd89ffb640 1 -- 172.21.15.26:0/3323298005 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd80032440 con 0x7efd8c0afc20 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.673+0000 7efd89ffb640 1 -- 172.21.15.26:0/3323298005 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800327f0 con 0x7efd8c0afc20 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.674+0000 7efd913ce640 1 -- 172.21.15.26:0/3323298005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd8c0afc20 msgr2=0x7efd8c071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.674+0000 7efd913ce640 1 --2- 172.21.15.26:0/3323298005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd8c0afc20 0x7efd8c071720 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efd800097d0 tx=0x7efd8002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.674+0000 7efd913ce640 1 -- 172.21.15.26:0/3323298005 shutdown_connections 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.674+0000 7efd913ce640 1 --2- 172.21.15.26:0/3323298005 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c071c60 0x7efd8c165d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.674+0000 7efd913ce640 1 --2- 172.21.15.26:0/3323298005 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd8c0afc20 0x7efd8c071720 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.674+0000 7efd913ce640 1 -- 172.21.15.26:0/3323298005 >> 172.21.15.26:0/3323298005 conn(0x7efd8c0b2970 msgr2=0x7efd8c0d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.674+0000 7efd913ce640 1 -- 172.21.15.26:0/3323298005 shutdown_connections 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.674+0000 7efd913ce640 1 -- 172.21.15.26:0/3323298005 wait complete. 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.675+0000 7efd913ce640 1 Processor -- start 2024-04-24T23:28:01.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.675+0000 7efd913ce640 1 -- start start 2024-04-24T23:28:01.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd913ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8c071c60 0x7efd8c1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd913ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd8c0afc20 0x7efd8c15df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd913ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c15c270 0x7efd8c143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd913ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c06c650 con 0x7efd8c071c60 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd913ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c06b120 con 0x7efd8c0afc20 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd913ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c15e4a0 con 0x7efd8c15c270 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8c071c60 0x7efd8c1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8c071c60 0x7efd8c1544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51542/0 (socket says 172.21.15.26:51542) 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8affd640 1 -- 172.21.15.26:0/3186986668 learned_addr learned my addr 172.21.15.26:0/3186986668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd8c0afc20 0x7efd8c15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8b7fe640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c15c270 0x7efd8c143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8affd640 1 -- 172.21.15.26:0/3186986668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c15c270 msgr2=0x7efd8c143f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8affd640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c15c270 0x7efd8c143f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8affd640 1 -- 172.21.15.26:0/3186986668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd8c0afc20 msgr2=0x7efd8c15df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8affd640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd8c0afc20 0x7efd8c15df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.676+0000 7efd8affd640 1 -- 172.21.15.26:0/3186986668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c15e6d0 con 0x7efd8c071c60 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.677+0000 7efd8affd640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8c071c60 0x7efd8c1544d0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c390 tx=0x7efd7c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.677+0000 7efd737fe640 1 -- 172.21.15.26:0/3186986668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c017070 con 0x7efd8c071c60 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.677+0000 7efd737fe640 1 -- 172.21.15.26:0/3186986668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c005ce0 con 0x7efd8c071c60 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.677+0000 7efd913ce640 1 -- 172.21.15.26:0/3186986668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c15e9b0 con 0x7efd8c071c60 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.677+0000 7efd737fe640 1 -- 172.21.15.26:0/3186986668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0072f0 con 0x7efd8c071c60 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.677+0000 7efd913ce640 1 -- 172.21.15.26:0/3186986668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c15ef70 con 0x7efd8c071c60 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.678+0000 7efd737fe640 1 -- 172.21.15.26:0/3186986668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd7c01a070 con 0x7efd8c071c60 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.679+0000 7efd737fe640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd7806be30 0x7efd7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.679+0000 7efd737fe640 1 -- 172.21.15.26:0/3186986668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7efd7c08a330 con 0x7efd8c071c60 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.679+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd7806be30 0x7efd7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.679+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd7806be30 0x7efd7806e2f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efd80034000 tx=0x7efd8002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.680+0000 7efd941bc740 1 Processor -- start 2024-04-24T23:28:01.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.680+0000 7efd941bc740 1 -- start start 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.680+0000 7efd941bc740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 0x55fcf261b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.680+0000 7efd941bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2638600 0x55fcf2638b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.680+0000 7efd941bc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf25f0a40 0x55fcf2636cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.680+0000 7efd941bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fcf25546b0 con 0x55fcf2638600 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.680+0000 7efd941bc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fcf24fc880 con 0x55fcf25f0a40 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd941bc740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55fcf2550670 con 0x55fcf261ad60 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2638600 0x55fcf2638b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2638600 0x55fcf2638b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51554/0 (socket says 172.21.15.26:51554) 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8a7fc640 1 -- 172.21.15.26:0/993324504 learned_addr learned my addr 172.21.15.26:0/993324504 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8affd640 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 0x55fcf261b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8b7fe640 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf25f0a40 0x55fcf2636cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8a7fc640 1 -- 172.21.15.26:0/993324504 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 msgr2=0x55fcf261b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 0x55fcf261b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8a7fc640 1 -- 172.21.15.26:0/993324504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf25f0a40 msgr2=0x55fcf2636cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf25f0a40 0x55fcf2636cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8a7fc640 1 -- 172.21.15.26:0/993324504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf23327d0 con 0x55fcf2638600 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8b7fe640 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf25f0a40 0x55fcf2636cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2638600 0x55fcf2638b50 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7efd8002a7b0 tx=0x7efd8002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf806fcd14594b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd70ff9640 1 -- 172.21.15.26:0/993324504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80003ae0 con 0x55fcf2638600 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd70ff9640 1 -- 172.21.15.26:0/993324504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8004d070 con 0x55fcf2638600 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.681+0000 7efd70ff9640 1 -- 172.21.15.26:0/993324504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8003e450 con 0x55fcf2638600 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.682+0000 7efd941bc740 1 -- 172.21.15.26:0/993324504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2638600 msgr2=0x55fcf2638b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.682+0000 7efd941bc740 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2638600 0x55fcf2638b50 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7efd8002a7b0 tx=0x7efd8002c000 comp rx=0 tx=0).stop 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.682+0000 7efd941bc740 1 -- 172.21.15.26:0/993324504 shutdown_connections 2024-04-24T23:28:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.682+0000 7efd941bc740 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf25f0a40 0x55fcf2636cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.682+0000 7efd941bc740 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2638600 0x55fcf2638b50 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.682+0000 7efd941bc740 1 --2- 172.21.15.26:0/993324504 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 0x55fcf261b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.682+0000 7efd941bc740 1 -- 172.21.15.26:0/993324504 >> 172.21.15.26:0/993324504 conn(0x55fcf25ed600 msgr2=0x55fcf2635b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.682+0000 7efd941bc740 1 -- 172.21.15.26:0/993324504 shutdown_connections 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.682+0000 7efd941bc740 1 -- 172.21.15.26:0/993324504 wait complete. 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd941bc740 1 -- start start 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd941bc740 1 Processor -- start 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd941bc740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 0x55fcf2649a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd941bc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf2638600 0x55fcf2649f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd941bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf261db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd941bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fcf25f17e0 con 0x55fcf2616a10 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd941bc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fcf250ff50 con 0x55fcf2638600 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd941bc740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55fcf25eb530 con 0x55fcf261ad60 2024-04-24T23:28:01.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd941bc740 10 client.0 _ll_register_callbacks cb 0x55fcf254b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd8b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf261db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd8b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf261db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51560/0 (socket says 172.21.15.26:51560) 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.683+0000 7efd8b7fe640 1 -- 172.21.15.26:0/2910934785 learned_addr learned my addr 172.21.15.26:0/2910934785 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd8affd640 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 0x55fcf2649a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf2638600 0x55fcf2649f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd8b7fe640 1 -- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 msgr2=0x55fcf2649a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd8b7fe640 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 0x55fcf2649a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd8b7fe640 1 -- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf2638600 msgr2=0x55fcf2649f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd8b7fe640 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf2638600 0x55fcf2649f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd8b7fe640 1 -- 172.21.15.26:0/2910934785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf22683f0 con 0x55fcf2616a10 2024-04-24T23:28:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd8b7fe640 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf261db20 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7efd84004350 tx=0x7efd84009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18a5e43603f3170 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd69ffb640 1 -- 172.21.15.26:0/2910934785 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84007b60 con 0x55fcf2616a10 2024-04-24T23:28:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd69ffb640 1 -- 172.21.15.26:0/2910934785 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7efd84007d00 con 0x55fcf2616a10 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd941bc740 1 -- 172.21.15.26:0/2910934785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd8004d070 con 0x55fcf2616a10 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd69ffb640 1 -- 172.21.15.26:0/2910934785 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7efd84015420 con 0x55fcf2616a10 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd941bc740 10 client.9408 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd941bc740 1 -- 172.21.15.26:0/2910934785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55fcf22280f0 con 0x55fcf2616a10 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.684+0000 7efd69ffb640 1 -- 172.21.15.26:0/2910934785 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8004d070 con 0x55fcf2616a10 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd69ffb640 1 -- 172.21.15.26:0/2910934785 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7efd84025080 con 0x55fcf2616a10 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd941bc740 10 client.9408 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd941bc740 10 client.9408 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd941bc740 20 client.9408 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd941bc740 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55fcf261e060 0x55fcf261e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd941bc740 4 client.9408 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd941bc740 1 -- 172.21.15.26:0/2910934785 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcf2561bf0 con 0x55fcf261e060 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd8affd640 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55fcf261e060 0x55fcf261e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd69ffb640 10 client.9408 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.685+0000 7efd8affd640 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55fcf261e060 0x55fcf261e440 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70be404271e2569d server_cookie=174bc2e400a15b74 in_seq=0 out_seq=0 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd69ffb640 1 -- 172.21.15.26:0/2910934785 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fcf2561bf0 con 0x55fcf261e060 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd69ffb640 10 client.9408 handle_command_reply: tid=0 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 client.9408 shutdown 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 -- 172.21.15.26:0/2910934785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 msgr2=0x55fcf261db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf261db20 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7efd84004350 tx=0x7efd84009f90 comp rx=0 tx=0).stop 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 -- 172.21.15.26:0/2910934785 shutdown_connections 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55fcf261e060 0x55fcf261e440 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf261db20 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd69ffb640 0 client.9408 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf2638600 0x55fcf2649f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 --2- 172.21.15.26:0/2910934785 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf261ad60 0x55fcf2649a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 -- 172.21.15.26:0/2910934785 >> 172.21.15.26:0/2910934785 conn(0x55fcf25f0a40 msgr2=0x55fcf2639980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 -- 172.21.15.26:0/2910934785 shutdown_connections 2024-04-24T23:28:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 1 -- 172.21.15.26:0/2910934785 wait complete. 2024-04-24T23:28:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.686+0000 7efd941bc740 20 client.9408 trim_cache size 0 max 16384 2024-04-24T23:28:01.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.704+0000 7efd941bc740 1 Processor -- start 2024-04-24T23:28:01.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.704+0000 7efd941bc740 1 -- start start 2024-04-24T23:28:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd941bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf25f1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd941bc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf261ad60 0x55fcf2657d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd941bc740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf2638600 0x55fcf2658240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd941bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1fe280 con 0x55fcf2616a10 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd941bc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1fd4a0 con 0x55fcf261ad60 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd941bc740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c1fdb50 con 0x55fcf2638600 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd8affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf25f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd8affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf25f1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51574/0 (socket says 172.21.15.26:51574) 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd8affd640 1 -- 172.21.15.26:0/3955318726 learned_addr learned my addr 172.21.15.26:0/3955318726 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/3955318726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf261ad60 0x55fcf2657d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd8affd640 1 -- 172.21.15.26:0/3955318726 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf2638600 msgr2=0x55fcf2658240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd8affd640 1 --2- 172.21.15.26:0/3955318726 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf2638600 0x55fcf2658240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd8affd640 1 -- 172.21.15.26:0/3955318726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf261ad60 msgr2=0x55fcf2657d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd8affd640 1 --2- 172.21.15.26:0/3955318726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf261ad60 0x55fcf2657d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.705+0000 7efd8affd640 1 -- 172.21.15.26:0/3955318726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf2617b10 con 0x55fcf2616a10 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd8affd640 1 --2- 172.21.15.26:0/3955318726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf25f1010 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0584e0 tx=0x7efd7c088730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaa9f90a2fc22a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd6bfff640 1 -- 172.21.15.26:0/3955318726 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c056850 con 0x55fcf2616a10 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd6bfff640 1 -- 172.21.15.26:0/3955318726 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c08a0b0 con 0x55fcf2616a10 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd6bfff640 1 -- 172.21.15.26:0/3955318726 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c007490 con 0x55fcf2616a10 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd941bc740 1 -- 172.21.15.26:0/3955318726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 msgr2=0x55fcf25f1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd941bc740 1 --2- 172.21.15.26:0/3955318726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf25f1010 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0584e0 tx=0x7efd7c088730 comp rx=0 tx=0).stop 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd941bc740 1 -- 172.21.15.26:0/3955318726 shutdown_connections 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd941bc740 1 --2- 172.21.15.26:0/3955318726 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf2638600 0x55fcf2658240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd941bc740 1 --2- 172.21.15.26:0/3955318726 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf261ad60 0x55fcf2657d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd941bc740 1 --2- 172.21.15.26:0/3955318726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf2616a10 0x55fcf25f1010 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd941bc740 1 -- 172.21.15.26:0/3955318726 >> 172.21.15.26:0/3955318726 conn(0x55fcf25f0a40 msgr2=0x55fcf2637120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd941bc740 1 -- 172.21.15.26:0/3955318726 shutdown_connections 2024-04-24T23:28:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.706+0000 7efd941bc740 1 -- 172.21.15.26:0/3955318726 wait complete. 2024-04-24T23:28:01.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.707+0000 7efd941bc740 1 -- start start 2024-04-24T23:28:01.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.707+0000 7efd941bc740 1 Processor -- start 2024-04-24T23:28:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd941bc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf2616a10 0x55fcf26b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd941bc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf261ad60 0x55fcf2633fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd941bc740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf2638600 0x55fcf2634510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd941bc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fcf25f1e10 con 0x55fcf261ad60 2024-04-24T23:28:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd941bc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c15d370 con 0x55fcf2616a10 2024-04-24T23:28:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd941bc740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x55fcf262c6e0 con 0x55fcf2638600 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd941bc740 10 client.0 _ll_register_callbacks cb 0x7efd84009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd8a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf261ad60 0x55fcf2633fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd8a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf261ad60 0x55fcf2633fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51588/0 (socket says 172.21.15.26:51588) 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd8a7fc640 1 -- 172.21.15.26:0/1298307947 learned_addr learned my addr 172.21.15.26:0/1298307947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd8a7fc640 1 -- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf2638600 msgr2=0x55fcf2634510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf2638600 0x55fcf2634510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd8a7fc640 1 -- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf2616a10 msgr2=0x55fcf26b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf2616a10 0x55fcf26b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd8a7fc640 1 -- 172.21.15.26:0/1298307947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf2659500 con 0x55fcf261ad60 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd8a7fc640 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf261ad60 0x55fcf2633fd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7efd8002a7b0 tx=0x7efd80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1759f83dcdfe00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd69ffb640 1 -- 172.21.15.26:0/1298307947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8003e860 con 0x55fcf261ad60 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd69ffb640 1 -- 172.21.15.26:0/1298307947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7efd8003ea00 con 0x55fcf261ad60 2024-04-24T23:28:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.708+0000 7efd941bc740 1 -- 172.21.15.26:0/1298307947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd7c08a0b0 con 0x55fcf261ad60 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd69ffb640 1 -- 172.21.15.26:0/1298307947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7efd80061050 con 0x55fcf261ad60 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd941bc740 10 client.9414 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd941bc740 1 -- 172.21.15.26:0/1298307947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55fcf26b5300 con 0x55fcf261ad60 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd69ffb640 1 -- 172.21.15.26:0/1298307947 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c08a0b0 con 0x55fcf261ad60 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd69ffb640 1 -- 172.21.15.26:0/1298307947 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7efd80064020 con 0x55fcf261ad60 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd941bc740 10 client.9414 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd941bc740 10 client.9414 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.b 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd941bc740 20 client.9414 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd941bc740 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55fcf263cff0 0x55fcf2657d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd941bc740 4 client.9414 mds_command: new command op to 7444 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd941bc740 1 -- 172.21.15.26:0/1298307947 --> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fcf255bf40 con 0x55fcf263cff0 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.709+0000 7efd8affd640 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55fcf263cff0 0x55fcf2657d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd69ffb640 10 client.9414 ms_handle_connect on v2:172.21.15.29:6832/2632309869 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd8affd640 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55fcf263cff0 0x55fcf2657d00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2bac830b66ed73ec server_cookie=791b799e912f07c9 in_seq=0 out_seq=0 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd69ffb640 1 -- 172.21.15.26:0/1298307947 <== mds.2 v2:172.21.15.29:6832/2632309869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55fcf255bf40 con 0x55fcf263cff0 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd69ffb640 10 client.9414 handle_command_reply: tid=0 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 client.9414 shutdown 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 -- 172.21.15.26:0/1298307947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf261ad60 msgr2=0x55fcf2633fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf261ad60 0x55fcf2633fd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7efd8002a7b0 tx=0x7efd80004730 comp rx=0 tx=0).stop 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 -- 172.21.15.26:0/1298307947 shutdown_connections 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:6832/2632309869,v1:172.21.15.29:6834/2632309869] conn(0x55fcf263cff0 0x55fcf2657d00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x55fcf2638600 0x55fcf2634510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fcf261ad60 0x55fcf2633fd0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd69ffb640 0 client.9414 ms_handle_reset on v2:172.21.15.29:6832/2632309869 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 --2- 172.21.15.26:0/1298307947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fcf2616a10 0x55fcf26b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 -- 172.21.15.26:0/1298307947 >> 172.21.15.26:0/1298307947 conn(0x55fcf25f0a40 msgr2=0x55fcf26499a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 -- 172.21.15.26:0/1298307947 shutdown_connections 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 1 -- 172.21.15.26:0/1298307947 wait complete. 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd941bc740 20 client.9414 trim_cache size 0 max 16384 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:28:01.717 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:28:01.718 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:28:01.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:28:01.718 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:28:01.718 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:01.718 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:01.718 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd913ce640 1 -- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd7806be30 msgr2=0x7efd7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd913ce640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd7806be30 0x7efd7806e2f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efd80034000 tx=0x7efd8002e000 comp rx=0 tx=0).stop 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd913ce640 1 -- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8c071c60 msgr2=0x7efd8c1544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.710+0000 7efd913ce640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8c071c60 0x7efd8c1544d0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c390 tx=0x7efd7c00c850 comp rx=0 tx=0).stop 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.711+0000 7efd913ce640 1 -- 172.21.15.26:0/3186986668 shutdown_connections 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.711+0000 7efd913ce640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efd7806be30 0x7efd7806e2f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.711+0000 7efd913ce640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efd8c15c270 0x7efd8c143f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.711+0000 7efd913ce640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efd8c0afc20 0x7efd8c15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.711+0000 7efd913ce640 1 --2- 172.21.15.26:0/3186986668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8c071c60 0x7efd8c1544d0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.711+0000 7efd913ce640 1 -- 172.21.15.26:0/3186986668 >> 172.21.15.26:0/3186986668 conn(0x7efd8c0b2970 msgr2=0x7efd8c0cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.711+0000 7efd913ce640 1 -- 172.21.15.26:0/3186986668 shutdown_connections 2024-04-24T23:28:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.711+0000 7efd913ce640 1 -- 172.21.15.26:0/3186986668 wait complete. 2024-04-24T23:28:01.722 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:28:01.722 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-24T23:28:01.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-24T23:28:01.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.824+0000 7fb383790640 1 Processor -- start 2024-04-24T23:28:01.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.824+0000 7fb383790640 1 -- start start 2024-04-24T23:28:01.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.825+0000 7fb383790640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb37c0af5f0 0x7fb37c06e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.825+0000 7fb383790640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37c0c6ef0 con 0x7fb37c06d240 2024-04-24T23:28:01.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.825+0000 7fb383790640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37c0711e0 con 0x7fb37c0cd120 2024-04-24T23:28:01.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.825+0000 7fb383790640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb37c0c69e0 con 0x7fb37c0af5f0 2024-04-24T23:28:01.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.825+0000 7fb380d2b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb37c0af5f0 0x7fb37c06e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.825+0000 7fb381d2d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb37c06d240 0x7fb37c06d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34032/0 (socket says 172.21.15.26:34032) 2024-04-24T23:28:01.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.825+0000 7fb380d2b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb37c0af5f0 0x7fb37c06e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51598/0 (socket says 172.21.15.26:51598) 2024-04-24T23:28:01.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.825+0000 7fb381d2d640 1 -- 172.21.15.26:0/3197909982 learned_addr learned my addr 172.21.15.26:0/3197909982 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:01.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.826+0000 7fb380d2b640 1 -- 172.21.15.26:0/3197909982 >> v1:172.21.15.29:6789/0 conn(0x7fb37c0cd120 legacy=0x7fb37c0c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.826+0000 7fb380d2b640 1 -- 172.21.15.26:0/3197909982 >> v1:172.21.15.26:6789/0 conn(0x7fb37c06d240 legacy=0x7fb37c06d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.826+0000 7fb380d2b640 1 -- 172.21.15.26:0/3197909982 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c0c6c10 con 0x7fb37c0af5f0 2024-04-24T23:28:01.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.826+0000 7fb373fff640 1 -- 172.21.15.26:0/3197909982 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1835982248 0 0) 0x7fb37c0c6ef0 con 0x7fb37c06d240 2024-04-24T23:28:01.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.826+0000 7fb380d2b640 1 --2- 172.21.15.26:0/3197909982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0af5f0 0x7fb37c06e990 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0097a0 tx=0x7fb36c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e39eb1de00fb7ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.826+0000 7fb373fff640 1 -- 172.21.15.26:0/3197909982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c037070 con 0x7fb37c0af5f0 2024-04-24T23:28:01.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.827+0000 7fb373fff640 1 -- 172.21.15.26:0/3197909982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c032440 con 0x7fb37c0af5f0 2024-04-24T23:28:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.827+0000 7fb373fff640 1 -- 172.21.15.26:0/3197909982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c02ae60 con 0x7fb37c0af5f0 2024-04-24T23:28:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.827+0000 7fb383790640 1 -- 172.21.15.26:0/3197909982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0af5f0 msgr2=0x7fb37c06e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.827+0000 7fb383790640 1 --2- 172.21.15.26:0/3197909982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0af5f0 0x7fb37c06e990 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0097a0 tx=0x7fb36c02a110 comp rx=0 tx=0).stop 2024-04-24T23:28:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.827+0000 7fb383790640 1 -- 172.21.15.26:0/3197909982 shutdown_connections 2024-04-24T23:28:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.827+0000 7fb383790640 1 --2- 172.21.15.26:0/3197909982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0af5f0 0x7fb37c06e990 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.827+0000 7fb383790640 1 -- 172.21.15.26:0/3197909982 >> 172.21.15.26:0/3197909982 conn(0x7fb37c0b1c80 msgr2=0x7fb37c0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.827+0000 7fb383790640 1 -- 172.21.15.26:0/3197909982 shutdown_connections 2024-04-24T23:28:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.827+0000 7fb383790640 1 -- 172.21.15.26:0/3197909982 wait complete. 2024-04-24T23:28:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.828+0000 7fb383790640 1 Processor -- start 2024-04-24T23:28:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.828+0000 7fb383790640 1 -- start start 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb383790640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb37c06d240 0x7fb37c156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb383790640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0af5f0 0x7fb37c0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb383790640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0cd120 0x7fb37c0ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb383790640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c0cfd10 con 0x7fb37c0cd120 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb383790640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c0afa50 con 0x7fb37c0af5f0 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb383790640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb37c0cee80 con 0x7fb37c06d240 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb381d2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0cd120 0x7fb37c0ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb381d2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0cd120 0x7fb37c0ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51602/0 (socket says 172.21.15.26:51602) 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb381d2d640 1 -- 172.21.15.26:0/2695049744 learned_addr learned my addr 172.21.15.26:0/2695049744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb380d2b640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0af5f0 0x7fb37c0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.829+0000 7fb38152c640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb37c06d240 0x7fb37c156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb381d2d640 1 -- 172.21.15.26:0/2695049744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb37c06d240 msgr2=0x7fb37c156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb381d2d640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb37c06d240 0x7fb37c156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb381d2d640 1 -- 172.21.15.26:0/2695049744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0af5f0 msgr2=0x7fb37c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb381d2d640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0af5f0 0x7fb37c0adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb381d2d640 1 -- 172.21.15.26:0/2695049744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c1f7b30 con 0x7fb37c0cd120 2024-04-24T23:28:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb38152c640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb37c06d240 0x7fb37c156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb381d2d640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0cd120 0x7fb37c0ae4f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c800 tx=0x7fb37800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb371ffb640 1 -- 172.21.15.26:0/2695049744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378018070 con 0x7fb37c0cd120 2024-04-24T23:28:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb371ffb640 1 -- 172.21.15.26:0/2695049744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb378004d80 con 0x7fb37c0cd120 2024-04-24T23:28:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb383790640 1 -- 172.21.15.26:0/2695049744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c1f7e10 con 0x7fb37c0cd120 2024-04-24T23:28:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb371ffb640 1 -- 172.21.15.26:0/2695049744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37800a880 con 0x7fb37c0cd120 2024-04-24T23:28:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.830+0000 7fb383790640 1 -- 172.21.15.26:0/2695049744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c1f83d0 con 0x7fb37c0cd120 2024-04-24T23:28:01.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.832+0000 7fb371ffb640 1 -- 172.21.15.26:0/2695049744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb37801b070 con 0x7fb37c0cd120 2024-04-24T23:28:01.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.832+0000 7fb371ffb640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb36006be30 0x7fb36006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.832+0000 7fb371ffb640 1 -- 172.21.15.26:0/2695049744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fb37808a410 con 0x7fb37c0cd120 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.832+0000 7fb38152c640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb36006be30 0x7fb36006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.833+0000 7fb38152c640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb36006be30 0x7fb36006e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb37c15ffc0 tx=0x7fb374009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.833+0000 7fb386580740 1 Processor -- start 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.833+0000 7fb386580740 1 -- start start 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb386580740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c8d60 0x5556c00c9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb386580740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb386580740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c009ea40 0x5556c00e4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb386580740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5556c00026b0 con 0x5556c00e6600 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb386580740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5556bffaa880 con 0x5556c009ea40 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb386580740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5556bfffe670 con 0x5556c00c8d60 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb380d2b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb380d2b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00e6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51606/0 (socket says 172.21.15.26:51606) 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb380d2b640 1 -- 172.21.15.26:0/2988969124 learned_addr learned my addr 172.21.15.26:0/2988969124 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb38152c640 1 --2- 172.21.15.26:0/2988969124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c8d60 0x5556c00c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb381d2d640 1 --2- 172.21.15.26:0/2988969124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c009ea40 0x5556c00e4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb380d2b640 1 -- 172.21.15.26:0/2988969124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c8d60 msgr2=0x5556c00c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb380d2b640 1 --2- 172.21.15.26:0/2988969124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c8d60 0x5556c00c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb380d2b640 1 -- 172.21.15.26:0/2988969124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c009ea40 msgr2=0x5556c00e4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb380d2b640 1 --2- 172.21.15.26:0/2988969124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c009ea40 0x5556c00e4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.834+0000 7fb380d2b640 1 -- 172.21.15.26:0/2988969124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556bfde07d0 con 0x5556c00e6600 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb380d2b640 1 --2- 172.21.15.26:0/2988969124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00e6b50 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0097a0 tx=0x7fb36c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c7e35ee3231264 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb366ffd640 1 -- 172.21.15.26:0/2988969124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c041070 con 0x5556c00e6600 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb366ffd640 1 -- 172.21.15.26:0/2988969124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c032740 con 0x5556c00e6600 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb366ffd640 1 -- 172.21.15.26:0/2988969124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c037040 con 0x5556c00e6600 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb386580740 1 -- 172.21.15.26:0/2988969124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 msgr2=0x5556c00e6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb386580740 1 --2- 172.21.15.26:0/2988969124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00e6b50 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0097a0 tx=0x7fb36c02c000 comp rx=0 tx=0).stop 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb386580740 1 -- 172.21.15.26:0/2988969124 shutdown_connections 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb386580740 1 --2- 172.21.15.26:0/2988969124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c009ea40 0x5556c00e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb386580740 1 --2- 172.21.15.26:0/2988969124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00e6b50 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb386580740 1 --2- 172.21.15.26:0/2988969124 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c8d60 0x5556c00c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb386580740 1 -- 172.21.15.26:0/2988969124 >> 172.21.15.26:0/2988969124 conn(0x5556c009b600 msgr2=0x5556c00e3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb386580740 1 -- 172.21.15.26:0/2988969124 shutdown_connections 2024-04-24T23:28:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.835+0000 7fb386580740 1 -- 172.21.15.26:0/2988969124 wait complete. 2024-04-24T23:28:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.836+0000 7fb386580740 1 -- start start 2024-04-24T23:28:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.836+0000 7fb386580740 1 Processor -- start 2024-04-24T23:28:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb386580740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 0x5556c00cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb386580740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00bb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb386580740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c00dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb386580740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5556c009f7e0 con 0x5556c00e6600 2024-04-24T23:28:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb386580740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5556bffbdf50 con 0x5556c00c8d60 2024-04-24T23:28:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb386580740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5556c0099530 con 0x5556c00c9f80 2024-04-24T23:28:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb386580740 10 client.0 _ll_register_callbacks cb 0x5556bfff9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb380d2b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00bb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb380d2b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00bb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51618/0 (socket says 172.21.15.26:51618) 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb380d2b640 1 -- 172.21.15.26:0/842998395 learned_addr learned my addr 172.21.15.26:0/842998395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb381d2d640 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c00dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb38152c640 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 0x5556c00cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb380d2b640 1 -- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 msgr2=0x5556c00dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb380d2b640 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c00dd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb380d2b640 1 -- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 msgr2=0x5556c00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb380d2b640 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 0x5556c00cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.837+0000 7fb380d2b640 1 -- 172.21.15.26:0/842998395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556bfd163f0 con 0x5556c00e6600 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb380d2b640 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00bb1f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009890 tx=0x7fb36c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac56f0964e5ab03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb35bfff640 1 -- 172.21.15.26:0/842998395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c041030 con 0x5556c00e6600 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb35bfff640 1 -- 172.21.15.26:0/842998395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fb36c032b00 con 0x5556c00e6600 2024-04-24T23:28:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb386580740 1 -- 172.21.15.26:0/842998395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36c032740 con 0x5556c00e6600 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb35bfff640 1 -- 172.21.15.26:0/842998395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fb36c04f050 con 0x5556c00e6600 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb386580740 10 client.9429 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb386580740 1 -- 172.21.15.26:0/842998395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x5556bfcd60f0 con 0x5556c00e6600 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb35bfff640 1 -- 172.21.15.26:0/842998395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c032740 con 0x5556c00e6600 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb35bfff640 1 -- 172.21.15.26:0/842998395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7fb36c052020 con 0x5556c00e6600 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb386580740 10 client.9429 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb386580740 10 client.9429 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb386580740 20 client.9429 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb386580740 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5556c00dbd50 0x5556c00dc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb386580740 4 client.9429 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:28:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.838+0000 7fb386580740 1 -- 172.21.15.26:0/842998395 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556c000fbf0 con 0x5556c00dbd50 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.839+0000 7fb38152c640 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5556c00dbd50 0x5556c00dc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.839+0000 7fb38152c640 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5556c00dbd50 0x5556c00dc180 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f31b4cf53e6d0e8 server_cookie=c155c571c6d77e1c in_seq=0 out_seq=0 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.839+0000 7fb35bfff640 10 client.9429 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb35bfff640 1 -- 172.21.15.26:0/842998395 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556c000fbf0 con 0x5556c00dbd50 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb35bfff640 10 client.9429 handle_command_reply: tid=0 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 client.9429 shutdown 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 -- 172.21.15.26:0/842998395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 msgr2=0x5556c00bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00bb1f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009890 tx=0x7fb36c004730 comp rx=0 tx=0).stop 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 -- 172.21.15.26:0/842998395 shutdown_connections 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5556c00dbd50 0x5556c00dc180 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c00dd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c00bb1f0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb35bfff640 0 client.9429 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 --2- 172.21.15.26:0/842998395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 0x5556c00cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 -- 172.21.15.26:0/842998395 >> 172.21.15.26:0/842998395 conn(0x5556c009ea40 msgr2=0x5556c00e7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 -- 172.21.15.26:0/842998395 shutdown_connections 2024-04-24T23:28:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 1 -- 172.21.15.26:0/842998395 wait complete. 2024-04-24T23:28:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.840+0000 7fb386580740 20 client.9429 trim_cache size 0 max 16384 2024-04-24T23:28:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.857+0000 7fb386580740 1 Processor -- start 2024-04-24T23:28:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb386580740 1 -- start start 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb386580740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00c8d60 0x5556c00e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb386580740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c009c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb386580740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00e6600 0x5556c009ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb386580740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c2007f0 con 0x5556c00c8d60 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb386580740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1ff990 con 0x5556c00e6600 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb386580740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb37c2000c0 con 0x5556c00c9f80 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb38152c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00c8d60 0x5556c00e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb38152c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00c8d60 0x5556c00e4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51632/0 (socket says 172.21.15.26:51632) 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb38152c640 1 -- 172.21.15.26:0/612884062 learned_addr learned my addr 172.21.15.26:0/612884062 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb381d2d640 1 --2- 172.21.15.26:0/612884062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00e6600 0x5556c009ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.858+0000 7fb380d2b640 1 --2- 172.21.15.26:0/612884062 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c009c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb38152c640 1 -- 172.21.15.26:0/612884062 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 msgr2=0x5556c009c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb38152c640 1 --2- 172.21.15.26:0/612884062 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c009c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb38152c640 1 -- 172.21.15.26:0/612884062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00e6600 msgr2=0x5556c009ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb38152c640 1 --2- 172.21.15.26:0/612884062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00e6600 0x5556c009ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb38152c640 1 -- 172.21.15.26:0/612884062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c00952a0 con 0x5556c00c8d60 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb38152c640 1 --2- 172.21.15.26:0/612884062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00c8d60 0x5556c00e4ec0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x5556c00dc990 tx=0x7fb374013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca62132eb05f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb3667fc640 1 -- 172.21.15.26:0/612884062 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3740272e0 con 0x5556c00c8d60 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb3667fc640 1 -- 172.21.15.26:0/612884062 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb374027480 con 0x5556c00c8d60 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb3667fc640 1 -- 172.21.15.26:0/612884062 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374027830 con 0x5556c00c8d60 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb386580740 1 -- 172.21.15.26:0/612884062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00c8d60 msgr2=0x5556c00e4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb386580740 1 --2- 172.21.15.26:0/612884062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00c8d60 0x5556c00e4ec0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x5556c00dc990 tx=0x7fb374013000 comp rx=0 tx=0).stop 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb386580740 1 -- 172.21.15.26:0/612884062 shutdown_connections 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb386580740 1 --2- 172.21.15.26:0/612884062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00e6600 0x5556c009ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb386580740 1 --2- 172.21.15.26:0/612884062 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c009c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb386580740 1 --2- 172.21.15.26:0/612884062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00c8d60 0x5556c00e4ec0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb386580740 1 -- 172.21.15.26:0/612884062 >> 172.21.15.26:0/612884062 conn(0x5556c009ea40 msgr2=0x5556c009ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb386580740 1 -- 172.21.15.26:0/612884062 shutdown_connections 2024-04-24T23:28:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.859+0000 7fb386580740 1 -- 172.21.15.26:0/612884062 wait complete. 2024-04-24T23:28:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.860+0000 7fb386580740 1 -- start start 2024-04-24T23:28:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.860+0000 7fb386580740 1 Processor -- start 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb386580740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 0x5556c009d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb386580740 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c0155530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb386580740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb386580740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5556c00dab50 con 0x5556c00e6600 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb386580740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1febd0 con 0x5556c00c8d60 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb386580740 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x5556bffcfec0 con 0x5556c00c9f80 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb386580740 10 client.0 _ll_register_callbacks cb 0x7fb378054a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb381d2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb381d2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c015dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51648/0 (socket says 172.21.15.26:51648) 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb381d2d640 1 -- 172.21.15.26:0/1244805014 learned_addr learned my addr 172.21.15.26:0/1244805014 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb38152c640 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 0x5556c009d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb380d2b640 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb381d2d640 1 -- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 msgr2=0x5556c0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb381d2d640 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c0155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb381d2d640 1 -- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 msgr2=0x5556c009d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb381d2d640 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 0x5556c009d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.861+0000 7fb381d2d640 1 -- 172.21.15.26:0/1244805014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c0150780 con 0x5556c00e6600 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb381d2d640 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c015dea0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb378058710 tx=0x7fb378088810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386bdd67817e8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb35bfff640 1 -- 172.21.15.26:0/1244805014 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378061360 con 0x5556c00e6600 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb35bfff640 1 -- 172.21.15.26:0/1244805014 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fb378057bd0 con 0x5556c00e6600 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb386580740 1 -- 172.21.15.26:0/1244805014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb374027480 con 0x5556c00e6600 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb35bfff640 1 -- 172.21.15.26:0/1244805014 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fb378057dd0 con 0x5556c00e6600 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb386580740 10 client.9435 fetch_fsmap learned FSMap version 113 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb386580740 1 -- 172.21.15.26:0/1244805014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x5556c0155a70 con 0x5556c00e6600 2024-04-24T23:28:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb35bfff640 1 -- 172.21.15.26:0/1244805014 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb374027480 con 0x5556c00e6600 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb35bfff640 1 -- 172.21.15.26:0/1244805014 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 113) ==== 3550+0+0 (secure 0 0 0) 0x7fb3780544d0 con 0x5556c00e6600 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb386580740 10 client.9435 fetch_fsmap finished waiting for FSMap version 113 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb386580740 10 client.9435 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb386580740 20 client.9435 populate_metadata read hostname 'smithi026' 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb386580740 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5556c015e3e0 0x5556c00dc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb386580740 4 client.9435 mds_command: new command op to 8730 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb386580740 1 -- 172.21.15.26:0/1244805014 --> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556c0009f40 con 0x5556c015e3e0 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.862+0000 7fb38152c640 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5556c015e3e0 0x5556c00dc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb35bfff640 10 client.9435 ms_handle_connect on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb38152c640 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5556c015e3e0 0x5556c00dc9c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77bc8afa17d3ec0 server_cookie=2c2b413e66e4882c in_seq=0 out_seq=0 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb35bfff640 1 -- 172.21.15.26:0/1244805014 <== mds.0 v2:172.21.15.29:6833/2225348762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5556c0009f40 con 0x5556c015e3e0 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb35bfff640 10 client.9435 handle_command_reply: tid=0 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 client.9435 shutdown 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 -- 172.21.15.26:0/1244805014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 msgr2=0x5556c015dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c015dea0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb378058710 tx=0x7fb378088810 comp rx=0 tx=0).stop 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 -- 172.21.15.26:0/1244805014 shutdown_connections 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:6833/2225348762,v1:172.21.15.29:6835/2225348762] conn(0x5556c015e3e0 0x5556c00dc9c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556c00e6600 0x5556c015dea0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x5556c00c9f80 0x5556c0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb35bfff640 0 client.9435 ms_handle_reset on v2:172.21.15.29:6833/2225348762 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 --2- 172.21.15.26:0/1244805014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5556c00c8d60 0x5556c009d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 -- 172.21.15.26:0/1244805014 >> 172.21.15.26:0/1244805014 conn(0x5556c009ea40 msgr2=0x5556c009ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 -- 172.21.15.26:0/1244805014 shutdown_connections 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 1 -- 172.21.15.26:0/1244805014 wait complete. 2024-04-24T23:28:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.863+0000 7fb386580740 20 client.9435 trim_cache size 0 max 16384 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "1163022b-502e-4b34-8ce2-d72741e7eef7": { 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/d1/d2/d3", 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "1163022b-502e-4b34-8ce2-d72741e7eef7", 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-24T23:28:01.871 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 -- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb36006be30 msgr2=0x7fb36006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb36006be30 0x7fb36006e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb37c15ffc0 tx=0x7fb374009040 comp rx=0 tx=0).stop 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 -- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0cd120 msgr2=0x7fb37c0ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0cd120 0x7fb37c0ae4f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c800 tx=0x7fb37800ccc0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 -- 172.21.15.26:0/2695049744 shutdown_connections 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb36006be30 0x7fb36006e2f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb37c0cd120 0x7fb37c0ae4f0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb37c0af5f0 0x7fb37c0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 --2- 172.21.15.26:0/2695049744 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb37c06d240 0x7fb37c156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 -- 172.21.15.26:0/2695049744 >> 172.21.15.26:0/2695049744 conn(0x7fb37c0b1c80 msgr2=0x7fb37c0c4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 -- 172.21.15.26:0/2695049744 shutdown_connections 2024-04-24T23:28:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.864+0000 7fb383790640 1 -- 172.21.15.26:0/2695049744 wait complete. 2024-04-24T23:28:01.875 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:28:01.876 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.975+0000 7fa99cb70640 1 Processor -- start 2024-04-24T23:28:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.976+0000 7fa99cb70640 1 -- start start 2024-04-24T23:28:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.976+0000 7fa99cb70640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa99815c2e0 0x7fa9980cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.976+0000 7fa99cb70640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa9980cdbc0 0x7fa9980ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.976+0000 7fa99cb70640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9980d0af0 con 0x7fa9980ceed0 2024-04-24T23:28:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.976+0000 7fa99cb70640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa9980cf320 con 0x7fa9980cdbc0 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.976+0000 7fa99cb70640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa9980d0dd0 con 0x7fa99815c2e0 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.977+0000 7fa98ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa9980cdbc0 0x7fa9980ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.977+0000 7fa996d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa9980ceed0 0x7fa9980b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34038/0 (socket says 172.21.15.26:34038) 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.977+0000 7fa996d9d640 1 -- 172.21.15.26:0/3360057493 learned_addr learned my addr 172.21.15.26:0/3360057493 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.977+0000 7fa99659c640 1 --2- 172.21.15.26:0/3360057493 >> v2:172.21.15.29:3300/0 conn(0x7fa99815c2e0 0x7fa9980cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.977+0000 7fa98ffff640 1 -- 172.21.15.26:0/3360057493 >> v2:172.21.15.29:3300/0 conn(0x7fa99815c2e0 msgr2=0x7fa9980cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.977+0000 7fa98ffff640 1 --2- 172.21.15.26:0/3360057493 >> v2:172.21.15.29:3300/0 conn(0x7fa99815c2e0 0x7fa9980cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.977+0000 7fa98ffff640 1 -- 172.21.15.26:0/3360057493 >> v1:172.21.15.26:6789/0 conn(0x7fa9980ceed0 legacy=0x7fa9980b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.977+0000 7fa98ffff640 1 -- 172.21.15.26:0/3360057493 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9980ccba0 con 0x7fa9980cdbc0 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.977+0000 7fa995d9b640 1 -- 172.21.15.26:0/3360057493 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3931534408 0 0) 0x7fa9980d0af0 con 0x7fa9980ceed0 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa98ffff640 1 --2- 172.21.15.26:0/3360057493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9980cdbc0 0x7fa9980ce990 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cf650 tx=0x7fa98002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e883d65eace48a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa995d9b640 1 -- 172.21.15.26:0/3360057493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980037070 con 0x7fa9980cdbc0 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa995d9b640 1 -- 172.21.15.26:0/3360057493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa980032440 con 0x7fa9980cdbc0 2024-04-24T23:28:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa995d9b640 1 -- 172.21.15.26:0/3360057493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980032810 con 0x7fa9980cdbc0 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa99cb70640 1 -- 172.21.15.26:0/3360057493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9980cdbc0 msgr2=0x7fa9980ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa99cb70640 1 --2- 172.21.15.26:0/3360057493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9980cdbc0 0x7fa9980ce990 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cf650 tx=0x7fa98002a1a0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa99cb70640 1 -- 172.21.15.26:0/3360057493 shutdown_connections 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa99cb70640 1 --2- 172.21.15.26:0/3360057493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9980cdbc0 0x7fa9980ce990 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa99cb70640 1 --2- 172.21.15.26:0/3360057493 >> v2:172.21.15.29:3300/0 conn(0x7fa99815c2e0 0x7fa9980cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa99cb70640 1 -- 172.21.15.26:0/3360057493 >> 172.21.15.26:0/3360057493 conn(0x7fa9980d1f80 msgr2=0x7fa9980d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa99cb70640 1 -- 172.21.15.26:0/3360057493 shutdown_connections 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.978+0000 7fa99cb70640 1 -- 172.21.15.26:0/3360057493 wait complete. 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.980+0000 7fa99cb70640 1 Processor -- start 2024-04-24T23:28:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.980+0000 7fa99cb70640 1 -- start start 2024-04-24T23:28:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.980+0000 7fa99cb70640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9980cdbc0 0x7fa99815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa99cb70640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa9980ceed0 0x7fa9980c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa99cb70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99815c2e0 0x7fa9980c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa99cb70640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9980b1f70 con 0x7fa99815c2e0 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa99cb70640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa9980cd160 con 0x7fa9980cdbc0 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa99cb70640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa9980b20f0 con 0x7fa9980ceed0 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa996d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99815c2e0 0x7fa9980c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa996d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99815c2e0 0x7fa9980c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51658/0 (socket says 172.21.15.26:51658) 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa996d9d640 1 -- 172.21.15.26:0/1127078314 learned_addr learned my addr 172.21.15.26:0/1127078314 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa99659c640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9980cdbc0 0x7fa99815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa98ffff640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa9980ceed0 0x7fa9980c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa996d9d640 1 -- 172.21.15.26:0/1127078314 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa9980ceed0 msgr2=0x7fa9980c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa996d9d640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa9980ceed0 0x7fa9980c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa996d9d640 1 -- 172.21.15.26:0/1127078314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9980cdbc0 msgr2=0x7fa99815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa996d9d640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9980cdbc0 0x7fa99815e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa996d9d640 1 -- 172.21.15.26:0/1127078314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99815e8d0 con 0x7fa99815c2e0 2024-04-24T23:28:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa98ffff640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa9980ceed0 0x7fa9980c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa996d9d640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99815c2e0 0x7fa9980c7820 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa99000c7a0 tx=0x7fa99000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa98effd640 1 -- 172.21.15.26:0/1127078314 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990018070 con 0x7fa99815c2e0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa98effd640 1 -- 172.21.15.26:0/1127078314 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99000a5e0 con 0x7fa99815c2e0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99815ebb0 con 0x7fa99815c2e0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa98effd640 1 -- 172.21.15.26:0/1127078314 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99000aa10 con 0x7fa99815c2e0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.981+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9980c6ae0 con 0x7fa99815c2e0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.982+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9980681e0 con 0x7fa99815c2e0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.983+0000 7fa98effd640 1 -- 172.21.15.26:0/1127078314 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa99001b070 con 0x7fa99815c2e0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.983+0000 7fa98effd640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa97806be30 0x7fa97806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.983+0000 7fa98effd640 1 -- 172.21.15.26:0/1127078314 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fa99008a330 con 0x7fa99815c2e0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.983+0000 7fa99659c640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa97806be30 0x7fa97806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.984+0000 7fa99659c640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa97806be30 0x7fa97806e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa988009aa0 tx=0x7fa988009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:01.988+0000 7fa98effd640 1 -- 172.21.15.26:0/1127078314 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa990058720 con 0x7fa99815c2e0 2024-04-24T23:28:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.209+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9980c86a0 con 0x7fa99815c2e0 2024-04-24T23:28:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.209+0000 7fa98effd640 1 -- 172.21.15.26:0/1127078314 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7fa99005ca20 con 0x7fa99815c2e0 2024-04-24T23:28:02.210 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:02.210 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa97806be30 msgr2=0x7fa97806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa97806be30 0x7fa97806e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa988009aa0 tx=0x7fa988009190 comp rx=0 tx=0).stop 2024-04-24T23:28:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99815c2e0 msgr2=0x7fa9980c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99815c2e0 0x7fa9980c7820 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa99000c7a0 tx=0x7fa99000cc60 comp rx=0 tx=0).stop 2024-04-24T23:28:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 shutdown_connections 2024-04-24T23:28:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa97806be30 0x7fa97806e2f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99815c2e0 0x7fa9980c7820 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa9980ceed0 0x7fa9980c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 --2- 172.21.15.26:0/1127078314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9980cdbc0 0x7fa99815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 >> 172.21.15.26:0/1127078314 conn(0x7fa9980d1f80 msgr2=0x7fa9980b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 shutdown_connections 2024-04-24T23:28:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.211+0000 7fa99cb70640 1 -- 172.21.15.26:0/1127078314 wait complete. 2024-04-24T23:28:02.223 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:28:02.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:28:02.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.322+0000 7fb1ddc15640 1 Processor -- start 2024-04-24T23:28:02.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.322+0000 7fb1ddc15640 1 -- start start 2024-04-24T23:28:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.322+0000 7fb1ddc15640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d80cd1a0 0x7fb1d80cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.322+0000 7fb1ddc15640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d8162360 con 0x7fb1d80cf970 2024-04-24T23:28:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.322+0000 7fb1ddc15640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d8162640 con 0x7fb1d80cbf70 2024-04-24T23:28:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.322+0000 7fb1ddc15640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d8162c90 con 0x7fb1d80cd1a0 2024-04-24T23:28:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb1d80cf970 0x7fb1d80afb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34052/0 (socket says 172.21.15.26:34052) 2024-04-24T23:28:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d7fff640 1 -- 172.21.15.26:0/270101982 learned_addr learned my addr 172.21.15.26:0/270101982 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d6ffd640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d80cd1a0 0x7fb1d80cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 612322117 0 0) 0x7fb1d8162360 con 0x7fb1d80cf970 2024-04-24T23:28:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1c4003520 con 0x7fb1d80cf970 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2612072037 0 0) 0x7fb1c4003520 con 0x7fb1d80cf970 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d80cd1a0 msgr2=0x7fb1d80cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d80cd1a0 0x7fb1d80cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 >> v1:172.21.15.29:6789/0 conn(0x7fb1d80cbf70 legacy=0x7fb1d80cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d8162920 con 0x7fb1d80cf970 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fb1d00019f0 con 0x7fb1d80cf970 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 >> v1:172.21.15.26:6789/0 conn(0x7fb1d80cf970 legacy=0x7fb1d80afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1c40039e0 0x7fb1c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c40043b0 0x7fb1c4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d80cd1a0 0x7fb1c40050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8162c90 con 0x7fb1c40043b0 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c400b9f0 con 0x7fb1c40039e0 2024-04-24T23:28:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c400bbf0 con 0x7fb1d80cd1a0 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d6ffd640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c40043b0 0x7fb1c4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.323+0000 7fb1d77fe640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1c40039e0 0x7fb1c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.324+0000 7fb1d7fff640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d80cd1a0 0x7fb1c40050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.324+0000 7fb1d6ffd640 1 -- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d80cd1a0 msgr2=0x7fb1c40050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.324+0000 7fb1d6ffd640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d80cd1a0 0x7fb1c40050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.324+0000 7fb1d6ffd640 1 -- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1c40039e0 msgr2=0x7fb1c4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.324+0000 7fb1d6ffd640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1c40039e0 0x7fb1c4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.324+0000 7fb1d6ffd640 1 -- 172.21.15.26:0/270101982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c4003520 con 0x7fb1c40043b0 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.324+0000 7fb1d6ffd640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c40043b0 0x7fb1c4004880 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009a90 tx=0x7fb1c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdcc4b5907bc6ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0036070 con 0x7fb1c40043b0 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0031400 con 0x7fb1c40043b0 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1d67fc640 1 -- 172.21.15.26:0/270101982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0009690 con 0x7fb1c40043b0 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1ddc15640 1 -- 172.21.15.26:0/270101982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c40043b0 msgr2=0x7fb1c4004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1ddc15640 1 --2- 172.21.15.26:0/270101982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c40043b0 0x7fb1c4004880 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009a90 tx=0x7fb1c4005830 comp rx=0 tx=0).stop 2024-04-24T23:28:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1d7fff640 1 -- 172.21.15.26:0/270101982 reap_dead start 2024-04-24T23:28:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1ddc15640 1 -- 172.21.15.26:0/270101982 shutdown_connections 2024-04-24T23:28:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1ddc15640 1 -- 172.21.15.26:0/270101982 >> 172.21.15.26:0/270101982 conn(0x7fb1d8072740 msgr2=0x7fb1d8072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1ddc15640 1 -- 172.21.15.26:0/270101982 shutdown_connections 2024-04-24T23:28:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.325+0000 7fb1ddc15640 1 -- 172.21.15.26:0/270101982 wait complete. 2024-04-24T23:28:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.326+0000 7fb1ddc15640 1 Processor -- start 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1ddc15640 1 -- start start 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1ddc15640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1c40043b0 0x7fb1d80c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1ddc15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1d80c4110 0x7fb1d80b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1ddc15640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d815a6c0 0x7fb1d80b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1ddc15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d80cc350 con 0x7fb1d80c4110 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1ddc15640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d80ca770 con 0x7fb1c40043b0 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1ddc15640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d80cc4d0 con 0x7fb1d815a6c0 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1d6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1d80c4110 0x7fb1d80b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1d6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1d80c4110 0x7fb1d80b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51684/0 (socket says 172.21.15.26:51684) 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.327+0000 7fb1d6ffd640 1 -- 172.21.15.26:0/1290627223 learned_addr learned my addr 172.21.15.26:0/1290627223 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1d77fe640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1c40043b0 0x7fb1d80c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1d7fff640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d815a6c0 0x7fb1d80b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1d6ffd640 1 -- 172.21.15.26:0/1290627223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d815a6c0 msgr2=0x7fb1d80b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1d6ffd640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d815a6c0 0x7fb1d80b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1d6ffd640 1 -- 172.21.15.26:0/1290627223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1c40043b0 msgr2=0x7fb1d80c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1d6ffd640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1c40043b0 0x7fb1d80c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1d6ffd640 1 -- 172.21.15.26:0/1290627223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d8073330 con 0x7fb1d80c4110 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1d6ffd640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1d80c4110 0x7fb1d80b4c40 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009a90 tx=0x7fb1c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1cffff640 1 -- 172.21.15.26:0/1290627223 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0040020 con 0x7fb1d80c4110 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1cffff640 1 -- 172.21.15.26:0/1290627223 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0003bc0 con 0x7fb1d80c4110 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d8073050 con 0x7fb1d80c4110 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1cffff640 1 -- 172.21.15.26:0/1290627223 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0036040 con 0x7fb1d80c4110 2024-04-24T23:28:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.328+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d80b4160 con 0x7fb1d80c4110 2024-04-24T23:28:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.329+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d80694c0 con 0x7fb1d80c4110 2024-04-24T23:28:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.329+0000 7fb1cffff640 1 -- 172.21.15.26:0/1290627223 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1c0043020 con 0x7fb1d80c4110 2024-04-24T23:28:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.330+0000 7fb1cffff640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb1b806be30 0x7fb1b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.330+0000 7fb1cffff640 1 -- 172.21.15.26:0/1290627223 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fb1c00aafc0 con 0x7fb1d80c4110 2024-04-24T23:28:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.330+0000 7fb1d77fe640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb1b806be30 0x7fb1b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.330+0000 7fb1d77fe640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb1b806be30 0x7fb1b806e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8003640 tx=0x7fb1c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:02.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.334+0000 7fb1cffff640 1 -- 172.21.15.26:0/1290627223 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb1c004a070 con 0x7fb1d80c4110 2024-04-24T23:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.587+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb1d80ca4c0 con 0x7fb1d80c4110 2024-04-24T23:28:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.588+0000 7fb1cffff640 1 -- 172.21.15.26:0/1290627223 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1967 (secure 0 0 0) 0x7fb1c00044e0 con 0x7fb1d80c4110 2024-04-24T23:28:02.589 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:02.589 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":451,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700753,"bytes_used":287752192,"bytes_avail":772806361088,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6739,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":113,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-24T23:27:57.408420+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:28:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.589+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb1b806be30 msgr2=0x7fb1b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.589+0000 7fb1ddc15640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb1b806be30 0x7fb1b806e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8003640 tx=0x7fb1c8009040 comp rx=0 tx=0).stop 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.589+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1d80c4110 msgr2=0x7fb1d80b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.589+0000 7fb1ddc15640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1d80c4110 0x7fb1d80b4c40 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009a90 tx=0x7fb1c0004730 comp rx=0 tx=0).stop 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.590+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 shutdown_connections 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.590+0000 7fb1ddc15640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb1b806be30 0x7fb1b806e2f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.590+0000 7fb1ddc15640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb1d815a6c0 0x7fb1d80b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.590+0000 7fb1ddc15640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1d80c4110 0x7fb1d80b4c40 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.590+0000 7fb1ddc15640 1 --2- 172.21.15.26:0/1290627223 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb1c40043b0 0x7fb1d80c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.590+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 >> 172.21.15.26:0/1290627223 conn(0x7fb1d8072740 msgr2=0x7fb1d815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.590+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 shutdown_connections 2024-04-24T23:28:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:02.590+0000 7fb1ddc15640 1 -- 172.21.15.26:0/1290627223 wait complete. 2024-04-24T23:28:02.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:28:03.602 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:03.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.715+0000 7f2398389640 1 Processor -- start 2024-04-24T23:28:03.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.715+0000 7f2398389640 1 -- start start 2024-04-24T23:28:03.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2398389640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2390145910 0x7f239006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:03.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2398389640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f239006d350 0x7f239015d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:03.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2398389640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f23900cd660 0x7f23900cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:03.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2398389640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f23900cdff0 con 0x7f239006d350 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2398389640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2390145cf0 con 0x7f23900cd660 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2398389640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f239015d570 con 0x7f2390145910 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2395924640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f239006d350 0x7f239015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2395924640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f239006d350 0x7f239015d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43964/0 (socket says 172.21.15.26:43964) 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2395924640 1 -- 172.21.15.26:0/2646657673 learned_addr learned my addr 172.21.15.26:0/2646657673 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2396125640 1 --2- 172.21.15.26:0/2646657673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2390145910 0x7f239006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2396926640 1 --2- 172.21.15.26:0/2646657673 >> v2:172.21.15.29:3300/0 conn(0x7f23900cd660 0x7f23900cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2395924640 1 -- 172.21.15.26:0/2646657673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2390145910 msgr2=0x7f239006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2395924640 1 --2- 172.21.15.26:0/2646657673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2390145910 0x7f239006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2395924640 1 -- 172.21.15.26:0/2646657673 >> v2:172.21.15.29:3300/0 conn(0x7f23900cd660 msgr2=0x7f23900cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2395924640 1 --2- 172.21.15.26:0/2646657673 >> v2:172.21.15.29:3300/0 conn(0x7f23900cd660 0x7f23900cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2395924640 1 -- 172.21.15.26:0/2646657673 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23900cfbc0 con 0x7f239006d350 2024-04-24T23:28:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.716+0000 7f2396125640 1 --2- 172.21.15.26:0/2646657673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2390145910 0x7f239006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.717+0000 7f2395924640 1 --2- 172.21.15.26:0/2646657673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239006d350 0x7f239015d030 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f23900710b0 tx=0x7f238002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c047d89c58d081 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.717+0000 7f2395123640 1 -- 172.21.15.26:0/2646657673 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380037070 con 0x7f239006d350 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.717+0000 7f2395123640 1 -- 172.21.15.26:0/2646657673 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2380032440 con 0x7f239006d350 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.717+0000 7f2395123640 1 -- 172.21.15.26:0/2646657673 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380032810 con 0x7f239006d350 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.718+0000 7f2398389640 1 -- 172.21.15.26:0/2646657673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239006d350 msgr2=0x7f239015d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.718+0000 7f2398389640 1 --2- 172.21.15.26:0/2646657673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239006d350 0x7f239015d030 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f23900710b0 tx=0x7f238002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.718+0000 7f2398389640 1 -- 172.21.15.26:0/2646657673 shutdown_connections 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.718+0000 7f2398389640 1 --2- 172.21.15.26:0/2646657673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239006d350 0x7f239015d030 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.718+0000 7f2398389640 1 --2- 172.21.15.26:0/2646657673 >> v2:172.21.15.29:3300/0 conn(0x7f23900cd660 0x7f23900cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.718+0000 7f2398389640 1 --2- 172.21.15.26:0/2646657673 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2390145910 0x7f239006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.718+0000 7f2398389640 1 -- 172.21.15.26:0/2646657673 >> 172.21.15.26:0/2646657673 conn(0x7f239006b300 msgr2=0x7f239006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.718+0000 7f2398389640 1 -- 172.21.15.26:0/2646657673 shutdown_connections 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.718+0000 7f2398389640 1 -- 172.21.15.26:0/2646657673 wait complete. 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.719+0000 7f2398389640 1 Processor -- start 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.719+0000 7f2398389640 1 -- start start 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2398389640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f239006d350 0x7f239013c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2398389640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23900cd660 0x7f239013f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2398389640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2390145910 0x7f239013fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2398389640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f239006ddb0 con 0x7f2390145910 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2398389640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f239006dad0 con 0x7f23900cd660 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2398389640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f239006e090 con 0x7f239006d350 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2396926640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2390145910 0x7f239013fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2396926640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2390145910 0x7f239013fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43978/0 (socket says 172.21.15.26:43978) 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2396926640 1 -- 172.21.15.26:0/2605949860 learned_addr learned my addr 172.21.15.26:0/2605949860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2396125640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f239006d350 0x7f239013c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2395924640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23900cd660 0x7f239013f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2396926640 1 -- 172.21.15.26:0/2605949860 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f239006d350 msgr2=0x7f239013c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2396926640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f239006d350 0x7f239013c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2396926640 1 -- 172.21.15.26:0/2605949860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23900cd660 msgr2=0x7f239013f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2396926640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23900cd660 0x7f239013f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.720+0000 7f2396926640 1 -- 172.21.15.26:0/2605949860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239015aed0 con 0x7f2390145910 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.721+0000 7f2396926640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2390145910 0x7f239013fa10 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2388007e60 tx=0x7f238800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.721+0000 7f2386ffd640 1 -- 172.21.15.26:0/2605949860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388014620 con 0x7f2390145910 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.721+0000 7f2386ffd640 1 -- 172.21.15.26:0/2605949860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2388019070 con 0x7f2390145910 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.721+0000 7f2386ffd640 1 -- 172.21.15.26:0/2605949860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388014a20 con 0x7f2390145910 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.721+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239015a350 con 0x7f2390145910 2024-04-24T23:28:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.721+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239015a910 con 0x7f2390145910 2024-04-24T23:28:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.722+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23900694c0 con 0x7f2390145910 2024-04-24T23:28:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.722+0000 7f2386ffd640 1 -- 172.21.15.26:0/2605949860 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2388014bc0 con 0x7f2390145910 2024-04-24T23:28:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.723+0000 7f2386ffd640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f237406be30 0x7f237406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.723+0000 7f2386ffd640 1 -- 172.21.15.26:0/2605949860 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f238808b640 con 0x7f2390145910 2024-04-24T23:28:03.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.723+0000 7f2396125640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f237406be30 0x7f237406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:03.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.723+0000 7f2396125640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f237406be30 0x7f237406e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f238c006fd0 tx=0x7f238c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:03.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.727+0000 7f2386ffd640 1 -- 172.21.15.26:0/2605949860 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2388059ff0 con 0x7f2390145910 2024-04-24T23:28:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.951+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2390153970 con 0x7f2390145910 2024-04-24T23:28:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.951+0000 7f2386ffd640 1 -- 172.21.15.26:0/2605949860 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7f23880598f0 con 0x7f2390145910 2024-04-24T23:28:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:03.952 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f237406be30 msgr2=0x7f237406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f237406be30 0x7f237406e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f238c006fd0 tx=0x7f238c008040 comp rx=0 tx=0).stop 2024-04-24T23:28:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2390145910 msgr2=0x7f239013fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2390145910 0x7f239013fa10 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2388007e60 tx=0x7f238800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 shutdown_connections 2024-04-24T23:28:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f237406be30 0x7f237406e2f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2390145910 0x7f239013fa10 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23900cd660 0x7f239013f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 --2- 172.21.15.26:0/2605949860 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f239006d350 0x7f239013c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 >> 172.21.15.26:0/2605949860 conn(0x7f239006b300 msgr2=0x7f23900b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.953+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 shutdown_connections 2024-04-24T23:28:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:03.954+0000 7f2398389640 1 -- 172.21.15.26:0/2605949860 wait complete. 2024-04-24T23:28:03.965 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:28:03.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:28:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.066+0000 7f82fe1a5640 1 Processor -- start 2024-04-24T23:28:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.066+0000 7f82fe1a5640 1 -- start start 2024-04-24T23:28:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82fe1a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82f80b4190 0x7f82f80ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82fe1a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80cd0f0 0x7f82f80b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82fe1a5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f80b1f40 con 0x7f82f815c240 2024-04-24T23:28:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82fe1a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f82f80cd8c0 con 0x7f82f80b4190 2024-04-24T23:28:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82fe1a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80cdba0 con 0x7f82f80cd0f0 2024-04-24T23:28:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82f6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82f80b4190 0x7f82f80ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82f6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82f80b4190 0x7f82f80ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43992/0 (socket says 172.21.15.26:43992) 2024-04-24T23:28:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82f6ffd640 1 -- 172.21.15.26:0/538533561 learned_addr learned my addr 172.21.15.26:0/538533561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82f7fff640 1 --2- 172.21.15.26:0/538533561 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80cd0f0 0x7f82f80b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82f6ffd640 1 -- 172.21.15.26:0/538533561 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80cd0f0 msgr2=0x7f82f80b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82f6ffd640 1 --2- 172.21.15.26:0/538533561 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80cd0f0 0x7f82f80b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82f6ffd640 1 -- 172.21.15.26:0/538533561 >> v1:172.21.15.29:6789/0 conn(0x7f82f815c240 legacy=0x7f82f80b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.067+0000 7f82f6ffd640 1 -- 172.21.15.26:0/538533561 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80cde80 con 0x7f82f80b4190 2024-04-24T23:28:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.068+0000 7f82f6ffd640 1 --2- 172.21.15.26:0/538533561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f80b4190 0x7f82f80ccbb0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009870 tx=0x7f82ec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41064b5b8462e43f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.068+0000 7f82f67fc640 1 -- 172.21.15.26:0/538533561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec037070 con 0x7f82f80b4190 2024-04-24T23:28:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.068+0000 7f82f67fc640 1 -- 172.21.15.26:0/538533561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec032440 con 0x7f82f80b4190 2024-04-24T23:28:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.068+0000 7f82f67fc640 1 -- 172.21.15.26:0/538533561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec032810 con 0x7f82f80b4190 2024-04-24T23:28:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.069+0000 7f82fe1a5640 1 -- 172.21.15.26:0/538533561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f80b4190 msgr2=0x7f82f80ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.069+0000 7f82fe1a5640 1 --2- 172.21.15.26:0/538533561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f80b4190 0x7f82f80ccbb0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009870 tx=0x7f82ec02a210 comp rx=0 tx=0).stop 2024-04-24T23:28:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.069+0000 7f82fe1a5640 1 -- 172.21.15.26:0/538533561 shutdown_connections 2024-04-24T23:28:04.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.069+0000 7f82fe1a5640 1 --2- 172.21.15.26:0/538533561 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80cd0f0 0x7f82f80b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.069+0000 7f82fe1a5640 1 --2- 172.21.15.26:0/538533561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f80b4190 0x7f82f80ccbb0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.069+0000 7f82fe1a5640 1 -- 172.21.15.26:0/538533561 >> 172.21.15.26:0/538533561 conn(0x7f82f809b350 msgr2=0x7f82f808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:04.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.069+0000 7f82fe1a5640 1 -- 172.21.15.26:0/538533561 shutdown_connections 2024-04-24T23:28:04.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.069+0000 7f82fe1a5640 1 -- 172.21.15.26:0/538533561 wait complete. 2024-04-24T23:28:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.070+0000 7f82fe1a5640 1 Processor -- start 2024-04-24T23:28:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.070+0000 7f82fe1a5640 1 -- start start 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82fe1a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80b4190 0x7f82f80cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82fe1a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82f80cd0f0 0x7f82f815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82fe1a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f815c240 0x7f82f81f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82fe1a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80cd560 con 0x7f82f815c240 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82fe1a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80c5b10 con 0x7f82f80cd0f0 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82fe1a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f82f815c620 con 0x7f82f80b4190 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f815c240 0x7f82f81f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f815c240 0x7f82f81f1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44008/0 (socket says 172.21.15.26:44008) 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f7fff640 1 -- 172.21.15.26:0/2349813059 learned_addr learned my addr 172.21.15.26:0/2349813059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f77fe640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80b4190 0x7f82f80cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f6ffd640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82f80cd0f0 0x7f82f815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f7fff640 1 -- 172.21.15.26:0/2349813059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80b4190 msgr2=0x7f82f80cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f7fff640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80b4190 0x7f82f80cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f7fff640 1 -- 172.21.15.26:0/2349813059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82f80cd0f0 msgr2=0x7f82f815a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f7fff640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82f80cd0f0 0x7f82f815a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f7fff640 1 -- 172.21.15.26:0/2349813059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f81f22a0 con 0x7f82f815c240 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.071+0000 7f82f6ffd640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82f80cd0f0 0x7f82f815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.072+0000 7f82f7fff640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f815c240 0x7f82f81f1a70 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f82f0007e40 tx=0x7f82f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.072+0000 7f82dffff640 1 -- 172.21.15.26:0/2349813059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0014620 con 0x7f82f815c240 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.072+0000 7f82dffff640 1 -- 172.21.15.26:0/2349813059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f0019070 con 0x7f82f815c240 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.072+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f81f2580 con 0x7f82f815c240 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.072+0000 7f82dffff640 1 -- 172.21.15.26:0/2349813059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0014a20 con 0x7f82f815c240 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.072+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f80b4df0 con 0x7f82f815c240 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.073+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f80681e0 con 0x7f82f815c240 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.073+0000 7f82dffff640 1 -- 172.21.15.26:0/2349813059 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82f0014bc0 con 0x7f82f815c240 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.074+0000 7f82dffff640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82e406be30 0x7f82e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.074+0000 7f82dffff640 1 -- 172.21.15.26:0/2349813059 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f82f008b770 con 0x7f82f815c240 2024-04-24T23:28:04.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.074+0000 7f82f77fe640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82e406be30 0x7f82e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:04.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.074+0000 7f82f77fe640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82e406be30 0x7f82e406e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f82e8004ea0 tx=0x7f82e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:04.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.078+0000 7f82dffff640 1 -- 172.21.15.26:0/2349813059 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82f005a1b0 con 0x7f82f815c240 2024-04-24T23:28:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.338+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f82f80b39d0 con 0x7f82f815c240 2024-04-24T23:28:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.339+0000 7f82dffff640 1 -- 172.21.15.26:0/2349813059 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1967 (secure 0 0 0) 0x7f82f0059ab0 con 0x7f82f815c240 2024-04-24T23:28:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:04.340 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":453,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706268,"bytes_used":287760384,"bytes_avail":772806352896,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7422,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":113,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-24T23:27:57.408420+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:28:04.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82e406be30 msgr2=0x7f82e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:04.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82e406be30 0x7f82e406e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f82e8004ea0 tx=0x7f82e8009040 comp rx=0 tx=0).stop 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f815c240 msgr2=0x7f82f81f1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f815c240 0x7f82f81f1a70 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f82f0007e40 tx=0x7f82f000e4b0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 shutdown_connections 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f82e406be30 0x7f82e406e2f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82f815c240 0x7f82f81f1a70 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f82f80cd0f0 0x7f82f815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 --2- 172.21.15.26:0/2349813059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f82f80b4190 0x7f82f80cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 >> 172.21.15.26:0/2349813059 conn(0x7f82f809b350 msgr2=0x7f82f80d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 shutdown_connections 2024-04-24T23:28:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:04.341+0000 7f82fe1a5640 1 -- 172.21.15.26:0/2349813059 wait complete. 2024-04-24T23:28:04.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:28:05.354 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:05.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.467+0000 7feedb2a5640 1 Processor -- start 2024-04-24T23:28:05.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.467+0000 7feedb2a5640 1 -- start start 2024-04-24T23:28:05.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feedb2a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed40cc300 0x7feed40b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feedb2a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed40b46e0 con 0x7feed40cc6e0 2024-04-24T23:28:05.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feedb2a5640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed40b49c0 con 0x7feed40726a0 2024-04-24T23:28:05.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feedb2a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7feed40b4ca0 con 0x7feed40cc300 2024-04-24T23:28:05.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feed9041640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed40cc300 0x7feed40b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feed8840640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feed40cc6e0 0x7feed4071f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42772/0 (socket says 172.21.15.26:42772) 2024-04-24T23:28:05.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feed8840640 1 -- 172.21.15.26:0/4049253982 learned_addr learned my addr 172.21.15.26:0/4049253982 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:05.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feed9041640 1 -- 172.21.15.26:0/4049253982 >> v1:172.21.15.29:6789/0 conn(0x7feed40726a0 legacy=0x7feed406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:05.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feed9041640 1 -- 172.21.15.26:0/4049253982 >> v1:172.21.15.26:6789/0 conn(0x7feed40cc6e0 legacy=0x7feed4071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feecbfff640 1 -- 172.21.15.26:0/4049253982 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505371412 0 0) 0x7feed40b46e0 con 0x7feed40cc6e0 2024-04-24T23:28:05.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.468+0000 7feed9041640 1 -- 172.21.15.26:0/4049253982 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed40b43d0 con 0x7feed40cc300 2024-04-24T23:28:05.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.469+0000 7feed9041640 1 --2- 172.21.15.26:0/4049253982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 0x7feed40b3e90 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7feed00097a0 tx=0x7feed002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d088565c0d0706 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:05.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.469+0000 7feecbfff640 1 -- 172.21.15.26:0/4049253982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0037070 con 0x7feed40cc300 2024-04-24T23:28:05.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.469+0000 7feecbfff640 1 -- 172.21.15.26:0/4049253982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feed0032440 con 0x7feed40cc300 2024-04-24T23:28:05.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.469+0000 7feecbfff640 1 -- 172.21.15.26:0/4049253982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0032810 con 0x7feed40cc300 2024-04-24T23:28:05.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.470+0000 7feedb2a5640 1 -- 172.21.15.26:0/4049253982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 msgr2=0x7feed40b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.470+0000 7feedb2a5640 1 --2- 172.21.15.26:0/4049253982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 0x7feed40b3e90 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7feed00097a0 tx=0x7feed002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.470+0000 7feedb2a5640 1 -- 172.21.15.26:0/4049253982 shutdown_connections 2024-04-24T23:28:05.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.470+0000 7feedb2a5640 1 --2- 172.21.15.26:0/4049253982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 0x7feed40b3e90 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.470+0000 7feedb2a5640 1 -- 172.21.15.26:0/4049253982 >> 172.21.15.26:0/4049253982 conn(0x7feed406b300 msgr2=0x7feed406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:05.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.470+0000 7feedb2a5640 1 -- 172.21.15.26:0/4049253982 shutdown_connections 2024-04-24T23:28:05.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.470+0000 7feedb2a5640 1 -- 172.21.15.26:0/4049253982 wait complete. 2024-04-24T23:28:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.471+0000 7feedb2a5640 1 Processor -- start 2024-04-24T23:28:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.471+0000 7feedb2a5640 1 -- start start 2024-04-24T23:28:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feedb2a5640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40726a0 0x7feed41417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feedb2a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 0x7feed4143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feedb2a5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40cc6e0 0x7feed4141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feedb2a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feed406e100 con 0x7feed40cc300 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feedb2a5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feed4163b00 con 0x7feed40cc6e0 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feedb2a5640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7feed406e280 con 0x7feed40726a0 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feed8840640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 0x7feed4143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feed8840640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 0x7feed4143fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44026/0 (socket says 172.21.15.26:44026) 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feed8840640 1 -- 172.21.15.26:0/3390569665 learned_addr learned my addr 172.21.15.26:0/3390569665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feed9842640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40cc6e0 0x7feed4141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feed9041640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40726a0 0x7feed41417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.472+0000 7feed8840640 1 -- 172.21.15.26:0/3390569665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40726a0 msgr2=0x7feed41417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feed8840640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40726a0 0x7feed41417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feed8840640 1 -- 172.21.15.26:0/3390569665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40cc6e0 msgr2=0x7feed4141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feed8840640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40cc6e0 0x7feed4141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feed8840640 1 -- 172.21.15.26:0/3390569665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed4144a20 con 0x7feed40cc300 2024-04-24T23:28:05.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feed8840640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 0x7feed4143fa0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7feec400c430 tx=0x7feec400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:05.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feec9ffb640 1 -- 172.21.15.26:0/3390569665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec40074e0 con 0x7feed40cc300 2024-04-24T23:28:05.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feec9ffb640 1 -- 172.21.15.26:0/3390569665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec40047e0 con 0x7feed40cc300 2024-04-24T23:28:05.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed4144d00 con 0x7feed40cc300 2024-04-24T23:28:05.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feec9ffb640 1 -- 172.21.15.26:0/3390569665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4004bd0 con 0x7feed40cc300 2024-04-24T23:28:05.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.473+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41571c0 con 0x7feed40cc300 2024-04-24T23:28:05.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.474+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed40b4190 con 0x7feed40cc300 2024-04-24T23:28:05.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.475+0000 7feec9ffb640 1 -- 172.21.15.26:0/3390569665 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feec4011070 con 0x7feed40cc300 2024-04-24T23:28:05.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.475+0000 7feec9ffb640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feeb806be60 0x7feeb806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.475+0000 7feec9ffb640 1 -- 172.21.15.26:0/3390569665 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7feec408aa00 con 0x7feed40cc300 2024-04-24T23:28:05.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.475+0000 7feed9041640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feeb806be60 0x7feeb806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.476+0000 7feed9041640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feeb806be60 0x7feeb806e320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feed0034000 tx=0x7feed002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:05.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.480+0000 7feec9ffb640 1 -- 172.21.15.26:0/3390569665 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feed40b4190 con 0x7feed40cc300 2024-04-24T23:28:05.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.701+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feed40694c0 con 0x7feed40cc300 2024-04-24T23:28:05.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.702+0000 7feec9ffb640 1 -- 172.21.15.26:0/3390569665 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7feec400e280 con 0x7feed40cc300 2024-04-24T23:28:05.702 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:05.703 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:05.703 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.703+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feeb806be60 msgr2=0x7feeb806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feeb806be60 0x7feeb806e320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feed0034000 tx=0x7feed002e000 comp rx=0 tx=0).stop 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 msgr2=0x7feed4143fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 0x7feed4143fa0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7feec400c430 tx=0x7feec400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 shutdown_connections 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7feeb806be60 0x7feeb806e320 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feed40cc6e0 0x7feed4141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed40cc300 0x7feed4143fa0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 --2- 172.21.15.26:0/3390569665 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7feed40726a0 0x7feed41417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 >> 172.21.15.26:0/3390569665 conn(0x7feed406b300 msgr2=0x7feed4158200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 shutdown_connections 2024-04-24T23:28:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.704+0000 7feedb2a5640 1 -- 172.21.15.26:0/3390569665 wait complete. 2024-04-24T23:28:05.715 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:28:05.715 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:28:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.814+0000 7fa70042a640 1 Processor -- start 2024-04-24T23:28:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.814+0000 7fa70042a640 1 -- start start 2024-04-24T23:28:05.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa70042a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f80ce850 0x7fa6f8140700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa70042a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6f8140c40 0x7fa6f80711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa70042a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f80cecd0 con 0x7fa6f813e090 2024-04-24T23:28:05.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa70042a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f8155aa0 con 0x7fa6f8140c40 2024-04-24T23:28:05.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa70042a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f80cef80 con 0x7fa6f80ce850 2024-04-24T23:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa6fe9c7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6f8140c40 0x7fa6f80711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa6fe1c6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa6f813e090 0x7fa6f815bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42776/0 (socket says 172.21.15.26:42776) 2024-04-24T23:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa6fe1c6640 1 -- 172.21.15.26:0/1129281881 learned_addr learned my addr 172.21.15.26:0/1129281881 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa6fd9c5640 1 --2- 172.21.15.26:0/1129281881 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f80ce850 0x7fa6f8140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa6fe9c7640 1 -- 172.21.15.26:0/1129281881 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f80ce850 msgr2=0x7fa6f8140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa6fe9c7640 1 --2- 172.21.15.26:0/1129281881 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f80ce850 0x7fa6f8140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa6fe9c7640 1 -- 172.21.15.26:0/1129281881 >> v1:172.21.15.26:6789/0 conn(0x7fa6f813e090 legacy=0x7fa6f815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa6fe9c7640 1 -- 172.21.15.26:0/1129281881 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f80cf260 con 0x7fa6f8140c40 2024-04-24T23:28:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.815+0000 7fa6fd1c4640 1 -- 172.21.15.26:0/1129281881 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165879024 0 0) 0x7fa6f80cecd0 con 0x7fa6f813e090 2024-04-24T23:28:05.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.816+0000 7fa6fd9c5640 1 --2- 172.21.15.26:0/1129281881 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f80ce850 0x7fa6f8140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:05.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.816+0000 7fa6fe9c7640 1 --2- 172.21.15.26:0/1129281881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 0x7fa6f80711b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f6eb8d21ea86fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.816+0000 7fa6fd1c4640 1 -- 172.21.15.26:0/1129281881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4031600 con 0x7fa6f8140c40 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.816+0000 7fa6fd1c4640 1 -- 172.21.15.26:0/1129281881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f40317a0 con 0x7fa6f8140c40 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.816+0000 7fa6fd1c4640 1 -- 172.21.15.26:0/1129281881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4031b70 con 0x7fa6f8140c40 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.817+0000 7fa70042a640 1 -- 172.21.15.26:0/1129281881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 msgr2=0x7fa6f80711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.817+0000 7fa70042a640 1 --2- 172.21.15.26:0/1129281881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 0x7fa6f80711b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f402a5d0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.817+0000 7fa70042a640 1 -- 172.21.15.26:0/1129281881 shutdown_connections 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.817+0000 7fa70042a640 1 --2- 172.21.15.26:0/1129281881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 0x7fa6f80711b0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.817+0000 7fa70042a640 1 --2- 172.21.15.26:0/1129281881 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f80ce850 0x7fa6f8140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.817+0000 7fa70042a640 1 -- 172.21.15.26:0/1129281881 >> 172.21.15.26:0/1129281881 conn(0x7fa6f806b540 msgr2=0x7fa6f806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.817+0000 7fa70042a640 1 -- 172.21.15.26:0/1129281881 shutdown_connections 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.817+0000 7fa70042a640 1 -- 172.21.15.26:0/1129281881 wait complete. 2024-04-24T23:28:05.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.818+0000 7fa70042a640 1 Processor -- start 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.818+0000 7fa70042a640 1 -- start start 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa70042a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6f80ce850 0x7fa6f80b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa70042a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f813e090 0x7fa6f81f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa70042a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 0x7fa6f81f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa70042a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80cfb00 con 0x7fa6f8140c40 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa70042a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80cf820 con 0x7fa6f80ce850 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa70042a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f80716f0 con 0x7fa6f813e090 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fe9c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 0x7fa6f81f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fe9c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 0x7fa6f81f1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44034/0 (socket says 172.21.15.26:44034) 2024-04-24T23:28:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fe9c7640 1 -- 172.21.15.26:0/2690326543 learned_addr learned my addr 172.21.15.26:0/2690326543 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fd9c5640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f813e090 0x7fa6f81f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fe1c6640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6f80ce850 0x7fa6f80b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fe9c7640 1 -- 172.21.15.26:0/2690326543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f813e090 msgr2=0x7fa6f81f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fe9c7640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f813e090 0x7fa6f81f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fe9c7640 1 -- 172.21.15.26:0/2690326543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6f80ce850 msgr2=0x7fa6f80b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fe9c7640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6f80ce850 0x7fa6f80b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.819+0000 7fa6fe9c7640 1 -- 172.21.15.26:0/2690326543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f81f7260 con 0x7fa6f8140c40 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.820+0000 7fa6fe9c7640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 0x7fa6f81f1080 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.820+0000 7fa6eeffd640 1 -- 172.21.15.26:0/2690326543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f403f070 con 0x7fa6f8140c40 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.820+0000 7fa6eeffd640 1 -- 172.21.15.26:0/2690326543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f4030be0 con 0x7fa6f8140c40 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.820+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f81f7540 con 0x7fa6f8140c40 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.820+0000 7fa6eeffd640 1 -- 172.21.15.26:0/2690326543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4009450 con 0x7fa6f8140c40 2024-04-24T23:28:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.820+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f81f7b00 con 0x7fa6f8140c40 2024-04-24T23:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.821+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f80694c0 con 0x7fa6f8140c40 2024-04-24T23:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.821+0000 7fa6eeffd640 1 -- 172.21.15.26:0/2690326543 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6f4003990 con 0x7fa6f8140c40 2024-04-24T23:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.822+0000 7fa6eeffd640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6dc06be30 0x7fa6dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.822+0000 7fa6eeffd640 1 -- 172.21.15.26:0/2690326543 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fa6f40abe80 con 0x7fa6f8140c40 2024-04-24T23:28:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.822+0000 7fa6fe1c6640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6dc06be30 0x7fa6dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.822+0000 7fa6fe1c6640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6dc06be30 0x7fa6dc06e2f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0000f60 tx=0x7fa6f0007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:05.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:05.826+0000 7fa6eeffd640 1 -- 172.21.15.26:0/2690326543 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa6f407a8c0 con 0x7fa6f8140c40 2024-04-24T23:28:05.885 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:28:05.884+0000 7f26a3749640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T23:28:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.078+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa6f80b4340 con 0x7fa6f8140c40 2024-04-24T23:28:06.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.079+0000 7fa6eeffd640 1 -- 172.21.15.26:0/2690326543 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1906 (secure 0 0 0) 0x7fa6f407a1c0 con 0x7fa6f8140c40 2024-04-24T23:28:06.079 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:06.080 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":455,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706268,"bytes_used":287760384,"bytes_avail":772806352896,"bytes_total":773094113280,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":113,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-24T23:28:05.410167+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:28:06.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6dc06be30 msgr2=0x7fa6dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6dc06be30 0x7fa6dc06e2f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0000f60 tx=0x7fa6f0007b50 comp rx=0 tx=0).stop 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 msgr2=0x7fa6f81f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 0x7fa6f81f1080 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f4004a40 comp rx=0 tx=0).stop 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 shutdown_connections 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa6dc06be30 0x7fa6dc06e2f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8140c40 0x7fa6f81f1080 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa6f813e090 0x7fa6f81f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 --2- 172.21.15.26:0/2690326543 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6f80ce850 0x7fa6f80b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 >> 172.21.15.26:0/2690326543 conn(0x7fa6f806b540 msgr2=0x7fa6f80b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 shutdown_connections 2024-04-24T23:28:06.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:06.081+0000 7fa70042a640 1 -- 172.21.15.26:0/2690326543 wait complete. 2024-04-24T23:28:06.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:28:07.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.206+0000 7f73d2ae7640 1 Processor -- start 2024-04-24T23:28:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.206+0000 7f73d2ae7640 1 -- start start 2024-04-24T23:28:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.207+0000 7f73d2ae7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc06d350 0x7f73cc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.207+0000 7f73d2ae7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73cc0cfbc0 con 0x7f73cc0cd660 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.207+0000 7f73d2ae7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73cc06d7f0 con 0x7f73cc145910 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.207+0000 7f73d2ae7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f73cc145cf0 con 0x7f73cc06d350 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.207+0000 7f73d1084640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f73cc0cd660 0x7f73cc15d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42792/0 (socket says 172.21.15.26:42792) 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.207+0000 7f73d1084640 1 -- 172.21.15.26:0/1654426059 learned_addr learned my addr 172.21.15.26:0/1654426059 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.207+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3380303196 0 0) 0x7f73cc0cfbc0 con 0x7f73cc0cd660 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.207+0000 7f73cbfff640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc06d350 0x7f73cc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.207+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73bc003520 con 0x7f73cc0cd660 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2030455495 0 0) 0x7f73bc003520 con 0x7f73cc0cd660 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc06d350 msgr2=0x7f73cc06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc06d350 0x7f73cc06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 >> v1:172.21.15.29:6789/0 conn(0x7f73cc145910 legacy=0x7f73cc15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc06dad0 con 0x7f73cc0cd660 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f73c40019f0 con 0x7f73cc0cd660 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 >> v1:172.21.15.26:6789/0 conn(0x7f73cc0cd660 legacy=0x7f73cc15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73bc0039e0 0x7f73bc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc004390 0x7f73bc004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc06d350 0x7f73bc0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc145cf0 con 0x7f73bc004390 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc00b9f0 con 0x7f73bc0039e0 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f73bc00bbf0 con 0x7f73cc06d350 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cbfff640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc004390 0x7f73bc004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73d0883640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73bc0039e0 0x7f73bc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73d1084640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc06d350 0x7f73bc0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cbfff640 1 -- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc06d350 msgr2=0x7f73bc0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cbfff640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc06d350 0x7f73bc0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cbfff640 1 -- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73bc0039e0 msgr2=0x7f73bc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cbfff640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73bc0039e0 0x7f73bc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.208+0000 7f73cbfff640 1 -- 172.21.15.26:0/1654426059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc003520 con 0x7f73bc004390 2024-04-24T23:28:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.209+0000 7f73cbfff640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc004390 0x7f73bc004860 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009ac0 tx=0x7f73bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6ef0b7d34173a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.209+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8036070 con 0x7f73bc004390 2024-04-24T23:28:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.209+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b8031400 con 0x7f73bc004390 2024-04-24T23:28:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.209+0000 7f73cb7fe640 1 -- 172.21.15.26:0/1654426059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8009690 con 0x7f73bc004390 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.210+0000 7f73d2ae7640 1 -- 172.21.15.26:0/1654426059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc004390 msgr2=0x7f73bc004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.210+0000 7f73d2ae7640 1 --2- 172.21.15.26:0/1654426059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73bc004390 0x7f73bc004860 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009ac0 tx=0x7f73bc005830 comp rx=0 tx=0).stop 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.210+0000 7f73d1084640 1 -- 172.21.15.26:0/1654426059 reap_dead start 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.210+0000 7f73d2ae7640 1 -- 172.21.15.26:0/1654426059 shutdown_connections 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.210+0000 7f73d2ae7640 1 -- 172.21.15.26:0/1654426059 >> 172.21.15.26:0/1654426059 conn(0x7f73cc06b300 msgr2=0x7f73cc06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.210+0000 7f73d2ae7640 1 -- 172.21.15.26:0/1654426059 shutdown_connections 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.210+0000 7f73d2ae7640 1 -- 172.21.15.26:0/1654426059 wait complete. 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.211+0000 7f73d2ae7640 1 Processor -- start 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.211+0000 7f73d2ae7640 1 -- start start 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d2ae7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73bc004390 0x7f73cc1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d2ae7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc153fd0 0x7f73cc141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d2ae7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73cc1529f0 0x7f73cc141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d2ae7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc06e370 con 0x7f73cc1529f0 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d2ae7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc06e090 con 0x7f73bc004390 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d2ae7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f73cc06e890 con 0x7f73cc153fd0 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d1084640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73cc1529f0 0x7f73cc141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d1084640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73cc1529f0 0x7f73cc141d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44052/0 (socket says 172.21.15.26:44052) 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d1084640 1 -- 172.21.15.26:0/3918990413 learned_addr learned my addr 172.21.15.26:0/3918990413 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d1084640 1 -- 172.21.15.26:0/3918990413 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc153fd0 msgr2=0x7f73cc141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d1084640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc153fd0 0x7f73cc141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d0883640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73bc004390 0x7f73cc1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d1084640 1 -- 172.21.15.26:0/3918990413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73bc004390 msgr2=0x7f73cc1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d1084640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73bc004390 0x7f73cc1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.212+0000 7f73d1084640 1 -- 172.21.15.26:0/3918990413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc152f30 con 0x7f73cc1529f0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.213+0000 7f73d1084640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73cc1529f0 0x7f73cc141d60 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f73c4001560 tx=0x7f73c4007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.213+0000 7f73c97fa640 1 -- 172.21.15.26:0/3918990413 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c400ac80 con 0x7f73cc1529f0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.213+0000 7f73c97fa640 1 -- 172.21.15.26:0/3918990413 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c4019070 con 0x7f73cc1529f0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.213+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc153210 con 0x7f73cc1529f0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.213+0000 7f73c97fa640 1 -- 172.21.15.26:0/3918990413 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4013630 con 0x7f73cc1529f0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.213+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc1537d0 con 0x7f73cc1529f0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.214+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73cc0694c0 con 0x7f73cc1529f0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.214+0000 7f73c97fa640 1 -- 172.21.15.26:0/3918990413 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73c40137d0 con 0x7f73cc1529f0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.215+0000 7f73c97fa640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f73b006be30 0x7f73b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.215+0000 7f73c97fa640 1 -- 172.21.15.26:0/3918990413 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f73c408b490 con 0x7f73cc1529f0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.215+0000 7f73d0883640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f73b006be30 0x7f73b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.215+0000 7f73d0883640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f73b006be30 0x7f73b006e2f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f73c0000c00 tx=0x7f73c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.219+0000 7f73c97fa640 1 -- 172.21.15.26:0/3918990413 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f73c4059e40 con 0x7f73cc1529f0 2024-04-24T23:28:07.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.443+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73cc0cd100 con 0x7f73cc1529f0 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.444+0000 7f73c97fa640 1 -- 172.21.15.26:0/3918990413 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7f73c4059740 con 0x7f73cc1529f0 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:07.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f73b006be30 msgr2=0x7f73b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f73b006be30 0x7f73b006e2f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f73c0000c00 tx=0x7f73c0009040 comp rx=0 tx=0).stop 2024-04-24T23:28:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73cc1529f0 msgr2=0x7f73cc141d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73cc1529f0 0x7f73cc141d60 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f73c4001560 tx=0x7f73c4007b20 comp rx=0 tx=0).stop 2024-04-24T23:28:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 shutdown_connections 2024-04-24T23:28:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f73b006be30 0x7f73b006e2f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73cc1529f0 0x7f73cc141d60 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f73cc153fd0 0x7f73cc141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 --2- 172.21.15.26:0/3918990413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73bc004390 0x7f73cc1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 >> 172.21.15.26:0/3918990413 conn(0x7f73cc06b300 msgr2=0x7f73cc143770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 shutdown_connections 2024-04-24T23:28:07.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.446+0000 7f73d2ae7640 1 -- 172.21.15.26:0/3918990413 wait complete. 2024-04-24T23:28:07.458 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:28:07.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:28:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.556+0000 7f31a739d640 1 Processor -- start 2024-04-24T23:28:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.556+0000 7f31a739d640 1 -- start start 2024-04-24T23:28:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.557+0000 7f31a739d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a00d4860 0x7f31a00cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.557+0000 7f31a739d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a00d1fb0 con 0x7f31a00d1270 2024-04-24T23:28:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.557+0000 7f31a739d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a00cff00 con 0x7f31a0162cb0 2024-04-24T23:28:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.557+0000 7f31a739d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f31a00b7140 con 0x7f31a00d4860 2024-04-24T23:28:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.557+0000 7f31a4938640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f31a00d1270 0x7f31a00b7c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42802/0 (socket says 172.21.15.26:42802) 2024-04-24T23:28:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.557+0000 7f31a4938640 1 -- 172.21.15.26:0/835344330 learned_addr learned my addr 172.21.15.26:0/835344330 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f31a593a640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a00d4860 0x7f31a00cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231956735 0 0) 0x7f31a00d1fb0 con 0x7f31a00d1270 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3188003520 con 0x7f31a00d1270 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2964047320 0 0) 0x7f31a00cff00 con 0x7f31a0162cb0 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31a00d1fb0 con 0x7f31a0162cb0 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1987691048 0 0) 0x7f3188003520 con 0x7f31a00d1270 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a00d4860 msgr2=0x7f31a00cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a00d4860 0x7f31a00cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 >> v1:172.21.15.29:6789/0 conn(0x7f31a0162cb0 legacy=0x7f31a00d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a00b7420 con 0x7f31a00d1270 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f3194001670 con 0x7f31a00d1270 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 >> v1:172.21.15.26:6789/0 conn(0x7f31a00d1270 legacy=0x7f31a00b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f3188003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3188004800 0x7f3188004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a00d4860 0x7f31880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31a00b7140 con 0x7f3188003b70 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f318800ba70 con 0x7f3188004800 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f318800bc40 con 0x7f31a00d4860 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f31a5139640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f3188003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f31a593a640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a00d4860 0x7f31880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f31a4938640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3188004800 0x7f3188004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f31a5139640 1 -- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a00d4860 msgr2=0x7f31880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f31a5139640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a00d4860 0x7f31880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f31a5139640 1 -- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3188004800 msgr2=0x7f3188004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.558+0000 7f31a5139640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3188004800 0x7f3188004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.559+0000 7f31a5139640 1 -- 172.21.15.26:0/835344330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3188003520 con 0x7f3188003b70 2024-04-24T23:28:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.559+0000 7f31a5139640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f3188003f70 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f31980018d0 tx=0x7f3188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbb7d65955ea50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.559+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31980014f0 con 0x7f3188003b70 2024-04-24T23:28:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.559+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31980052d0 con 0x7f3188003b70 2024-04-24T23:28:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.559+0000 7f319ffff640 1 -- 172.21.15.26:0/835344330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319802ac30 con 0x7f3188003b70 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.560+0000 7f31a739d640 1 -- 172.21.15.26:0/835344330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 msgr2=0x7f3188003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.560+0000 7f31a739d640 1 --2- 172.21.15.26:0/835344330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f3188003f70 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f31980018d0 tx=0x7f3188005830 comp rx=0 tx=0).stop 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.560+0000 7f31a593a640 1 -- 172.21.15.26:0/835344330 reap_dead start 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.560+0000 7f31a739d640 1 -- 172.21.15.26:0/835344330 shutdown_connections 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.560+0000 7f31a739d640 1 -- 172.21.15.26:0/835344330 >> 172.21.15.26:0/835344330 conn(0x7f31a009d500 msgr2=0x7f31a00d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.560+0000 7f31a739d640 1 -- 172.21.15.26:0/835344330 shutdown_connections 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.560+0000 7f31a739d640 1 -- 172.21.15.26:0/835344330 wait complete. 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.561+0000 7f31a739d640 1 Processor -- start 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.561+0000 7f31a739d640 1 -- start start 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a739d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f31a0159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a739d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a01e8e40 0x7f31a01e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a739d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31a015da90 0x7f31a015df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a739d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31a00cfcc0 con 0x7f3188003b70 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a739d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31a00cfb40 con 0x7f31a015da90 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a739d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f31a00b8330 con 0x7f31a01e8e40 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a5139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f31a0159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a5139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f31a0159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44076/0 (socket says 172.21.15.26:44076) 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a5139640 1 -- 172.21.15.26:0/3834151689 learned_addr learned my addr 172.21.15.26:0/3834151689 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a4938640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a01e8e40 0x7f31a01e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a593a640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31a015da90 0x7f31a015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a5139640 1 -- 172.21.15.26:0/3834151689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a01e8e40 msgr2=0x7f31a01e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a5139640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a01e8e40 0x7f31a01e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a5139640 1 -- 172.21.15.26:0/3834151689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31a015da90 msgr2=0x7f31a015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a5139640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31a015da90 0x7f31a015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.562+0000 7f31a5139640 1 -- 172.21.15.26:0/3834151689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a00c8070 con 0x7f3188003b70 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.563+0000 7f31a5139640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f31a0159e40 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3198005040 tx=0x7f31980012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.563+0000 7f319dffb640 1 -- 172.21.15.26:0/3834151689 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319803f020 con 0x7f3188003b70 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.563+0000 7f319dffb640 1 -- 172.21.15.26:0/3834151689 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3198006e50 con 0x7f3188003b70 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.563+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a00c8310 con 0x7f3188003b70 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.563+0000 7f319dffb640 1 -- 172.21.15.26:0/3834151689 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198037040 con 0x7f3188003b70 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.563+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a00c88d0 con 0x7f3188003b70 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.564+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a00694c0 con 0x7f3188003b70 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.564+0000 7f319dffb640 1 -- 172.21.15.26:0/3834151689 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3198042020 con 0x7f3188003b70 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.565+0000 7f319dffb640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f318c06be30 0x7f318c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.565+0000 7f319dffb640 1 -- 172.21.15.26:0/3834151689 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f31980aa860 con 0x7f3188003b70 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.565+0000 7f31a4938640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f318c06be30 0x7f318c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.565+0000 7f31a4938640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f318c06be30 0x7f318c06e2f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f31a01eac20 tx=0x7f3194001e20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.569+0000 7f319dffb640 1 -- 172.21.15.26:0/3834151689 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3198049030 con 0x7f3188003b70 2024-04-24T23:28:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.824+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f31a00d0940 con 0x7f3188003b70 2024-04-24T23:28:07.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.825+0000 7f319dffb640 1 -- 172.21.15.26:0/3834151689 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1906 (secure 0 0 0) 0x7f31980aa070 con 0x7f3188003b70 2024-04-24T23:28:07.826 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:07.826 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":456,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706268,"bytes_used":287760384,"bytes_avail":772806352896,"bytes_total":773094113280,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":113,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-24T23:28:05.410167+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-04-24T23:28:07.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f318c06be30 msgr2=0x7f318c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f318c06be30 0x7f318c06e2f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f31a01eac20 tx=0x7f3194001e20 comp rx=0 tx=0).stop 2024-04-24T23:28:07.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 msgr2=0x7f31a0159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f31a0159e40 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3198005040 tx=0x7f31980012d0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 shutdown_connections 2024-04-24T23:28:07.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f318c06be30 0x7f318c06e2f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31a015da90 0x7f31a015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f31a01e8e40 0x7f31a01e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 --2- 172.21.15.26:0/3834151689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3188003b70 0x7f31a0159e40 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 >> 172.21.15.26:0/3834151689 conn(0x7f31a009d500 msgr2=0x7f31a00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 shutdown_connections 2024-04-24T23:28:07.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:07.827+0000 7f31a739d640 1 -- 172.21.15.26:0/3834151689 wait complete. 2024-04-24T23:28:07.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-04-24T23:28:08.840 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:08.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.954+0000 7f19c8390640 1 Processor -- start 2024-04-24T23:28:08.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.954+0000 7f19c8390640 1 -- start start 2024-04-24T23:28:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.955+0000 7f19c8390640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f19c00d4870 0x7f19c00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.955+0000 7f19c8390640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19c00b3130 0x7f19c00b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.955+0000 7f19c8390640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19c00d32f0 con 0x7f19c015e570 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.955+0000 7f19c8390640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f19c00d1ab0 con 0x7f19c00b3130 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.955+0000 7f19c8390640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f19c00d2ae0 con 0x7f19c00d4870 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.956+0000 7f19c692d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19c00b3130 0x7f19c00b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.956+0000 7f19c612c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f19c015e570 0x7f19c00d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42804/0 (socket says 172.21.15.26:42804) 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.956+0000 7f19c612c640 1 -- 172.21.15.26:0/992601692 learned_addr learned my addr 172.21.15.26:0/992601692 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.956+0000 7f19c592b640 1 --2- 172.21.15.26:0/992601692 >> v2:172.21.15.29:3300/0 conn(0x7f19c00d4870 0x7f19c00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.956+0000 7f19c692d640 1 -- 172.21.15.26:0/992601692 >> v2:172.21.15.29:3300/0 conn(0x7f19c00d4870 msgr2=0x7f19c00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.956+0000 7f19c692d640 1 --2- 172.21.15.26:0/992601692 >> v2:172.21.15.29:3300/0 conn(0x7f19c00d4870 0x7f19c00b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.956+0000 7f19c692d640 1 -- 172.21.15.26:0/992601692 >> v1:172.21.15.26:6789/0 conn(0x7f19c015e570 legacy=0x7f19c00d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.956+0000 7f19c692d640 1 -- 172.21.15.26:0/992601692 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c00d0270 con 0x7f19c00b3130 2024-04-24T23:28:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.956+0000 7f19c592b640 1 --2- 172.21.15.26:0/992601692 >> v2:172.21.15.29:3300/0 conn(0x7f19c00d4870 0x7f19c00b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.957+0000 7f19c692d640 1 --2- 172.21.15.26:0/992601692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 0x7f19c00b4640 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f19c008cae0 tx=0x7f19b802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cfdf9d8f85fa0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.957+0000 7f19c512a640 1 -- 172.21.15.26:0/992601692 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8007740 con 0x7f19c00b3130 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.957+0000 7f19c512a640 1 -- 172.21.15.26:0/992601692 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b80078e0 con 0x7f19c00b3130 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.957+0000 7f19c512a640 1 -- 172.21.15.26:0/992601692 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8007cb0 con 0x7f19c00b3130 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.957+0000 7f19c8390640 1 -- 172.21.15.26:0/992601692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 msgr2=0x7f19c00b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.957+0000 7f19c8390640 1 --2- 172.21.15.26:0/992601692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 0x7f19c00b4640 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f19c008cae0 tx=0x7f19b802a5f0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.958+0000 7f19c8390640 1 -- 172.21.15.26:0/992601692 shutdown_connections 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.958+0000 7f19c8390640 1 --2- 172.21.15.26:0/992601692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 0x7f19c00b4640 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.958+0000 7f19c8390640 1 --2- 172.21.15.26:0/992601692 >> v2:172.21.15.29:3300/0 conn(0x7f19c00d4870 0x7f19c00b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.958+0000 7f19c8390640 1 -- 172.21.15.26:0/992601692 >> 172.21.15.26:0/992601692 conn(0x7f19c008dbe0 msgr2=0x7f19c009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.958+0000 7f19c8390640 1 -- 172.21.15.26:0/992601692 shutdown_connections 2024-04-24T23:28:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.958+0000 7f19c8390640 1 -- 172.21.15.26:0/992601692 wait complete. 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.959+0000 7f19c8390640 1 Processor -- start 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.959+0000 7f19c8390640 1 -- start start 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c8390640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 0x7f19c01f8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c8390640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c00d4870 0x7f19c015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c8390640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c015e570 0x7f19c00b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c8390640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19c00cfa70 con 0x7f19c00b3130 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c8390640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19c00d3010 con 0x7f19c00d4870 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c8390640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f19c00cfd50 con 0x7f19c015e570 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c612c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 0x7f19c01f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c612c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 0x7f19c01f8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44084/0 (socket says 172.21.15.26:44084) 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c612c640 1 -- 172.21.15.26:0/1586865685 learned_addr learned my addr 172.21.15.26:0/1586865685 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c592b640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c00d4870 0x7f19c015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c692d640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c015e570 0x7f19c00b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c612c640 1 -- 172.21.15.26:0/1586865685 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c015e570 msgr2=0x7f19c00b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c612c640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c015e570 0x7f19c00b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c612c640 1 -- 172.21.15.26:0/1586865685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c00d4870 msgr2=0x7f19c015b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c612c640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c00d4870 0x7f19c015b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c612c640 1 -- 172.21.15.26:0/1586865685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c00b7600 con 0x7f19c00b3130 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c692d640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c015e570 0x7f19c00b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c612c640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 0x7f19c01f8910 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00c3c0 tx=0x7f19bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19b6ffd640 1 -- 172.21.15.26:0/1586865685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc017070 con 0x7f19c00b3130 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19b6ffd640 1 -- 172.21.15.26:0/1586865685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19bc005ce0 con 0x7f19c00b3130 2024-04-24T23:28:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c00b78e0 con 0x7f19c00b3130 2024-04-24T23:28:08.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.960+0000 7f19b6ffd640 1 -- 172.21.15.26:0/1586865685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc0072f0 con 0x7f19c00b3130 2024-04-24T23:28:08.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.961+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c00b72f0 con 0x7f19c00b3130 2024-04-24T23:28:08.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.961+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c00694c0 con 0x7f19c00b3130 2024-04-24T23:28:08.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.962+0000 7f19b6ffd640 1 -- 172.21.15.26:0/1586865685 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19bc01a070 con 0x7f19c00b3130 2024-04-24T23:28:08.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.962+0000 7f19b6ffd640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19a406be30 0x7f19a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.962+0000 7f19b6ffd640 1 -- 172.21.15.26:0/1586865685 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f19bc08a080 con 0x7f19c00b3130 2024-04-24T23:28:08.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.962+0000 7f19c592b640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19a406be30 0x7f19a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.963+0000 7f19c592b640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19a406be30 0x7f19a406e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f19c01f8e50 tx=0x7f19b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:08.967+0000 7f19b6ffd640 1 -- 172.21.15.26:0/1586865685 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f19c00d32f0 con 0x7f19c00b3130 2024-04-24T23:28:09.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.191+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19c00caeb0 con 0x7f19c00b3130 2024-04-24T23:28:09.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.191+0000 7f19b6ffd640 1 -- 172.21.15.26:0/1586865685 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7f19bc058470 con 0x7f19c00b3130 2024-04-24T23:28:09.193 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:09.193 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:09.193 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:09.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.193+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19a406be30 msgr2=0x7f19a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.193+0000 7f19c8390640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19a406be30 0x7f19a406e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f19c01f8e50 tx=0x7f19b0008040 comp rx=0 tx=0).stop 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.193+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 msgr2=0x7f19c01f8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.193+0000 7f19c8390640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 0x7f19c01f8910 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00c3c0 tx=0x7f19bc00c880 comp rx=0 tx=0).stop 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.193+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 shutdown_connections 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.194+0000 7f19c8390640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f19a406be30 0x7f19a406e2f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.194+0000 7f19c8390640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f19c015e570 0x7f19c00b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.194+0000 7f19c8390640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f19c00d4870 0x7f19c015b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.194+0000 7f19c8390640 1 --2- 172.21.15.26:0/1586865685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19c00b3130 0x7f19c01f8910 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.194+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 >> 172.21.15.26:0/1586865685 conn(0x7f19c008dbe0 msgr2=0x7f19c006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.194+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 shutdown_connections 2024-04-24T23:28:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.194+0000 7f19c8390640 1 -- 172.21.15.26:0/1586865685 wait complete. 2024-04-24T23:28:09.207 DEBUG:tasks.cephfs.test_multimds_misc:current active=['d', 'c', 'b'] 2024-04-24T23:28:09.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:28:09.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.306+0000 7f4b97ad4640 1 Processor -- start 2024-04-24T23:28:09.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.306+0000 7f4b97ad4640 1 -- start start 2024-04-24T23:28:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b97ad4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b900ce850 0x7f4b90140700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b97ad4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b900cecd0 con 0x7f4b90140c40 2024-04-24T23:28:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b97ad4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b900cef80 con 0x7f4b9013e090 2024-04-24T23:28:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b97ad4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4b90155aa0 con 0x7f4b900ce850 2024-04-24T23:28:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b9506f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b900ce850 0x7f4b90140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b96071640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4b90140c40 0x7f4b900711b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42808/0 (socket says 172.21.15.26:42808) 2024-04-24T23:28:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b96071640 1 -- 172.21.15.26:0/3444235776 learned_addr learned my addr 172.21.15.26:0/3444235776 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b9506f640 1 -- 172.21.15.26:0/3444235776 >> v1:172.21.15.29:6789/0 conn(0x7f4b9013e090 legacy=0x7f4b9015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b9506f640 1 -- 172.21.15.26:0/3444235776 >> v1:172.21.15.26:6789/0 conn(0x7f4b90140c40 legacy=0x7f4b900711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b9506f640 1 -- 172.21.15.26:0/3444235776 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b900cf260 con 0x7f4b900ce850 2024-04-24T23:28:09.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.307+0000 7f4b9486e640 1 -- 172.21.15.26:0/3444235776 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934934974 0 0) 0x7f4b900cecd0 con 0x7f4b90140c40 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.308+0000 7f4b9506f640 1 --2- 172.21.15.26:0/3444235776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b900ce850 0x7f4b90140700 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4b800097a0 tx=0x7f4b8002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c73ffbfd568f567 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.308+0000 7f4b9486e640 1 -- 172.21.15.26:0/3444235776 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80037070 con 0x7f4b900ce850 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.308+0000 7f4b9486e640 1 -- 172.21.15.26:0/3444235776 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8002ae50 con 0x7f4b900ce850 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.308+0000 7f4b9486e640 1 -- 172.21.15.26:0/3444235776 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80032660 con 0x7f4b900ce850 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.309+0000 7f4b97ad4640 1 -- 172.21.15.26:0/3444235776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b900ce850 msgr2=0x7f4b90140700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.309+0000 7f4b97ad4640 1 --2- 172.21.15.26:0/3444235776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b900ce850 0x7f4b90140700 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4b800097a0 tx=0x7f4b8002a140 comp rx=0 tx=0).stop 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.309+0000 7f4b97ad4640 1 -- 172.21.15.26:0/3444235776 shutdown_connections 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.309+0000 7f4b97ad4640 1 --2- 172.21.15.26:0/3444235776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b900ce850 0x7f4b90140700 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.309+0000 7f4b97ad4640 1 -- 172.21.15.26:0/3444235776 >> 172.21.15.26:0/3444235776 conn(0x7f4b9006b540 msgr2=0x7f4b9006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.309+0000 7f4b97ad4640 1 -- 172.21.15.26:0/3444235776 shutdown_connections 2024-04-24T23:28:09.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.309+0000 7f4b97ad4640 1 -- 172.21.15.26:0/3444235776 wait complete. 2024-04-24T23:28:09.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.310+0000 7f4b97ad4640 1 Processor -- start 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.310+0000 7f4b97ad4640 1 -- start start 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b97ad4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b900ce850 0x7f4b90137610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b97ad4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b9013e090 0x7f4b9015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b97ad4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b90140c40 0x7f4b9015d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b97ad4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b900cf820 con 0x7f4b9013e090 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b97ad4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90141070 con 0x7f4b90140c40 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b97ad4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f4b900cfb00 con 0x7f4b900ce850 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b9506f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b9013e090 0x7f4b9015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b9506f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b9013e090 0x7f4b9015cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44100/0 (socket says 172.21.15.26:44100) 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b9506f640 1 -- 172.21.15.26:0/2470583592 learned_addr learned my addr 172.21.15.26:0/2470583592 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b96071640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b90140c40 0x7f4b9015d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b95870640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b900ce850 0x7f4b90137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b9506f640 1 -- 172.21.15.26:0/2470583592 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b900ce850 msgr2=0x7f4b90137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b9506f640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b900ce850 0x7f4b90137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b9506f640 1 -- 172.21.15.26:0/2470583592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b90140c40 msgr2=0x7f4b9015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b9506f640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b90140c40 0x7f4b9015d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b9506f640 1 -- 172.21.15.26:0/2470583592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b90137b50 con 0x7f4b9013e090 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.311+0000 7f4b9506f640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b9013e090 0x7f4b9015cec0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f4b800098c0 tx=0x7f4b80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.312+0000 7f4b867fc640 1 -- 172.21.15.26:0/2470583592 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80048070 con 0x7f4b9013e090 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.312+0000 7f4b867fc640 1 -- 172.21.15.26:0/2470583592 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b80039600 con 0x7f4b9013e090 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.312+0000 7f4b867fc640 1 -- 172.21.15.26:0/2470583592 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80037040 con 0x7f4b9013e090 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.312+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b90138bf0 con 0x7f4b9013e090 2024-04-24T23:28:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.312+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b901391b0 con 0x7f4b9013e090 2024-04-24T23:28:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.313+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b90140700 con 0x7f4b9013e090 2024-04-24T23:28:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.313+0000 7f4b867fc640 1 -- 172.21.15.26:0/2470583592 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b8004b070 con 0x7f4b9013e090 2024-04-24T23:28:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.313+0000 7f4b867fc640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4b7406be30 0x7f4b7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.314+0000 7f4b867fc640 1 -- 172.21.15.26:0/2470583592 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f4b800acc70 con 0x7f4b9013e090 2024-04-24T23:28:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.314+0000 7f4b95870640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4b7406be30 0x7f4b7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.314+0000 7f4b95870640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4b7406be30 0x7f4b7406e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c004ea0 tx=0x7f4b8c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.318+0000 7f4b867fc640 1 -- 172.21.15.26:0/2470583592 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4b90140700 con 0x7f4b9013e090 2024-04-24T23:28:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.570+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4b900b37b0 con 0x7f4b9013e090 2024-04-24T23:28:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.571+0000 7f4b867fc640 1 -- 172.21.15.26:0/2470583592 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1824 (secure 0 0 0) 0x7f4b8007b610 con 0x7f4b9013e090 2024-04-24T23:28:09.572 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:09.572 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":458,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714000841,"num_in_osds":8,"osd_in_since":1714000833,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706268,"bytes_used":287760384,"bytes_avail":772806352896,"bytes_total":773094113280,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":113,"id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"b","status":"up:active","gid":7444},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8712},{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8730}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-24T23:28:07.410642+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T23:28:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4b7406be30 msgr2=0x7f4b7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4b7406be30 0x7f4b7406e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c004ea0 tx=0x7f4b8c009040 comp rx=0 tx=0).stop 2024-04-24T23:28:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b9013e090 msgr2=0x7f4b9015cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b9013e090 0x7f4b9015cec0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f4b800098c0 tx=0x7f4b80004730 comp rx=0 tx=0).stop 2024-04-24T23:28:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 shutdown_connections 2024-04-24T23:28:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f4b7406be30 0x7f4b7406e2f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b90140c40 0x7f4b9015d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b9013e090 0x7f4b9015cec0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 --2- 172.21.15.26:0/2470583592 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f4b900ce850 0x7f4b90137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 >> 172.21.15.26:0/2470583592 conn(0x7f4b9006b540 msgr2=0x7f4b9006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 shutdown_connections 2024-04-24T23:28:09.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.573+0000 7f4b97ad4640 1 -- 172.21.15.26:0/2470583592 wait complete. 2024-04-24T23:28:09.585 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-24T23:28:09.585 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:28:09.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:28:09.633 DEBUG:teuthology.orchestra.run.smithi029:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:28:09.683 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:28:09.683 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:28:09.683 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:28:09.683 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:09.699 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:28:09.699 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:09.755 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:28:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.895+0000 7f380e31d640 1 Processor -- start 2024-04-24T23:28:09.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.895+0000 7f380e31d640 1 -- start start 2024-04-24T23:28:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380e31d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a480 0x7f38080b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380e31d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f380806e2f0 0x7f3808071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380e31d640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38080b2e80 con 0x7f380815a860 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380e31d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f38080b3120 con 0x7f380806e2f0 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380e31d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f38080b33c0 con 0x7f380815a480 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380c8ba640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f380806e2f0 0x7f3808071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380c8ba640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f380806e2f0 0x7f3808071070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44108/0 (socket says 172.21.15.26:44108) 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380c8ba640 1 -- 172.21.15.26:0/1314421352 learned_addr learned my addr 172.21.15.26:0/1314421352 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380c8ba640 1 -- 172.21.15.26:0/1314421352 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a480 msgr2=0x7f38080b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f3807fff640 1 --2- 172.21.15.26:0/1314421352 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a480 0x7f38080b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380c8ba640 1 --2- 172.21.15.26:0/1314421352 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a480 0x7f38080b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380c8ba640 1 -- 172.21.15.26:0/1314421352 >> v1:172.21.15.29:6789/0 conn(0x7f380815a860 legacy=0x7f380806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.896+0000 7f380c8ba640 1 -- 172.21.15.26:0/1314421352 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38080b36a0 con 0x7f380806e2f0 2024-04-24T23:28:09.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.897+0000 7f380c8ba640 1 --2- 172.21.15.26:0/1314421352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 0x7f3808071070 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f380000d150 tx=0x7f380002c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f43928d3dc9e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.897+0000 7f3806ffd640 1 -- 172.21.15.26:0/1314421352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800039070 con 0x7f380806e2f0 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.897+0000 7f3806ffd640 1 -- 172.21.15.26:0/1314421352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3800009600 con 0x7f380806e2f0 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.897+0000 7f3806ffd640 1 -- 172.21.15.26:0/1314421352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000099d0 con 0x7f380806e2f0 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.898+0000 7f380e31d640 1 -- 172.21.15.26:0/1314421352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 msgr2=0x7f3808071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.898+0000 7f380e31d640 1 --2- 172.21.15.26:0/1314421352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 0x7f3808071070 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f380000d150 tx=0x7f380002c840 comp rx=0 tx=0).stop 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.898+0000 7f380e31d640 1 -- 172.21.15.26:0/1314421352 shutdown_connections 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.898+0000 7f380e31d640 1 --2- 172.21.15.26:0/1314421352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 0x7f3808071070 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.898+0000 7f380e31d640 1 --2- 172.21.15.26:0/1314421352 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a480 0x7f38080b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.898+0000 7f380e31d640 1 -- 172.21.15.26:0/1314421352 >> 172.21.15.26:0/1314421352 conn(0x7f380806b300 msgr2=0x7f380806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.898+0000 7f380e31d640 1 -- 172.21.15.26:0/1314421352 shutdown_connections 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.898+0000 7f380e31d640 1 -- 172.21.15.26:0/1314421352 wait complete. 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.899+0000 7f380e31d640 1 Processor -- start 2024-04-24T23:28:09.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.899+0000 7f380e31d640 1 -- start start 2024-04-24T23:28:09.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f380e31d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 0x7f38081484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f380e31d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380815a480 0x7f3808157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f380e31d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a860 0x7f3808157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f380e31d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38080b4e30 con 0x7f380806e2f0 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f380e31d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38080b3c60 con 0x7f380815a480 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f380e31d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f38080b4fb0 con 0x7f380815a860 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f3807fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 0x7f38081484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f3807fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 0x7f38081484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44120/0 (socket says 172.21.15.26:44120) 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f3807fff640 1 -- 172.21.15.26:0/2800329466 learned_addr learned my addr 172.21.15.26:0/2800329466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f38077fe640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380815a480 0x7f3808157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.900+0000 7f380c8ba640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a860 0x7f3808157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f3807fff640 1 -- 172.21.15.26:0/2800329466 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a860 msgr2=0x7f3808157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f3807fff640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a860 0x7f3808157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f3807fff640 1 -- 172.21.15.26:0/2800329466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380815a480 msgr2=0x7f3808157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f3807fff640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380815a480 0x7f3808157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f3807fff640 1 -- 172.21.15.26:0/2800329466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3808151c10 con 0x7f380806e2f0 2024-04-24T23:28:09.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f3807fff640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 0x7f38081484f0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c7f0 tx=0x7f37f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f3804ff9640 1 -- 172.21.15.26:0/2800329466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0015070 con 0x7f380806e2f0 2024-04-24T23:28:09.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f3804ff9640 1 -- 172.21.15.26:0/2800329466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f0004240 con 0x7f380806e2f0 2024-04-24T23:28:09.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38081585e0 con 0x7f380806e2f0 2024-04-24T23:28:09.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f3804ff9640 1 -- 172.21.15.26:0/2800329466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0007620 con 0x7f380806e2f0 2024-04-24T23:28:09.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.901+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3808158ba0 con 0x7f380806e2f0 2024-04-24T23:28:09.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.902+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38080694c0 con 0x7f380806e2f0 2024-04-24T23:28:09.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.902+0000 7f3804ff9640 1 -- 172.21.15.26:0/2800329466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37f00079d0 con 0x7f380806e2f0 2024-04-24T23:28:09.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.903+0000 7f3804ff9640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e806be30 0x7f37e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.903+0000 7f3804ff9640 1 -- 172.21.15.26:0/2800329466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f37f008aea0 con 0x7f380806e2f0 2024-04-24T23:28:09.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.903+0000 7f38077fe640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e806be30 0x7f37e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.903+0000 7f38077fe640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e806be30 0x7f37e806e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f38080b16e0 tx=0x7f37f8007e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:09.907+0000 7f3804ff9640 1 -- 172.21.15.26:0/2800329466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f37f0059160 con 0x7f380806e2f0 2024-04-24T23:28:10.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.093+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38080b44a0 con 0x7f380806e2f0 2024-04-24T23:28:10.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.094+0000 7f3804ff9640 1 -- 172.21.15.26:0/2800329466 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16548 (secure 0 0 0) 0x7f37f005d460 con 0x7f380806e2f0 2024-04-24T23:28:10.095 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:10.095 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:27:24.935708+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:21.600058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:22.278230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:28:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e806be30 msgr2=0x7f37e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e806be30 0x7f37e806e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f38080b16e0 tx=0x7f37f8007e30 comp rx=0 tx=0).stop 2024-04-24T23:28:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 msgr2=0x7f38081484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 0x7f38081484f0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c7f0 tx=0x7f37f000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 shutdown_connections 2024-04-24T23:28:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f37e806be30 0x7f37e806e2f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f380815a860 0x7f3808157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f380815a480 0x7f3808157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 --2- 172.21.15.26:0/2800329466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f380806e2f0 0x7f38081484f0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 >> 172.21.15.26:0/2800329466 conn(0x7f380806b300 msgr2=0x7f38080cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 shutdown_connections 2024-04-24T23:28:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.096+0000 7f380e31d640 1 -- 172.21.15.26:0/2800329466 wait complete. 2024-04-24T23:28:10.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:21.600058+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:22.278230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:28:10.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:28:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.212+0000 7f5dbb789640 1 Processor -- start 2024-04-24T23:28:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.213+0000 7f5dbb789640 1 -- start start 2024-04-24T23:28:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.213+0000 7f5dbb789640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5db40d0ce0 0x7f5db406e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.213+0000 7f5dbb789640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4071720 0x7f5db4071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.213+0000 7f5dbb789640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db406b1c0 con 0x7f5db4158ae0 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.213+0000 7f5dbb789640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5db40720b0 con 0x7f5db40d0ce0 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.213+0000 7f5dbb789640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5db406ec90 con 0x7f5db4071720 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.214+0000 7f5db9d26640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4071720 0x7f5db4071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.214+0000 7f5db9d26640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4071720 0x7f5db4071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:47720/0 (socket says 172.21.15.26:47720) 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.214+0000 7f5db9d26640 1 -- 172.21.15.26:0/1055775115 learned_addr learned my addr 172.21.15.26:0/1055775115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.214+0000 7f5db8d24640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5db40d0ce0 0x7f5db406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.214+0000 7f5db9d26640 1 -- 172.21.15.26:0/1055775115 >> v2:172.21.15.29:3300/0 conn(0x7f5db40d0ce0 msgr2=0x7f5db406e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.214+0000 7f5db9d26640 1 --2- 172.21.15.26:0/1055775115 >> v2:172.21.15.29:3300/0 conn(0x7f5db40d0ce0 0x7f5db406e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.214+0000 7f5db9d26640 1 -- 172.21.15.26:0/1055775115 >> v1:172.21.15.29:6789/0 conn(0x7f5db4158ae0 legacy=0x7f5db415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.214+0000 7f5db9d26640 1 -- 172.21.15.26:0/1055775115 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db406ef70 con 0x7f5db4071720 2024-04-24T23:28:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5db9d26640 1 --2- 172.21.15.26:0/1055775115 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4071720 0x7f5db4071b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009870 tx=0x7f5dac02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ef793de643c5db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5db3fff640 1 -- 172.21.15.26:0/1055775115 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac037070 con 0x7f5db4071720 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5db3fff640 1 -- 172.21.15.26:0/1055775115 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac032440 con 0x7f5db4071720 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5db3fff640 1 -- 172.21.15.26:0/1055775115 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac032810 con 0x7f5db4071720 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5dbb789640 1 -- 172.21.15.26:0/1055775115 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4071720 msgr2=0x7f5db4071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5dbb789640 1 --2- 172.21.15.26:0/1055775115 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4071720 0x7f5db4071b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009870 tx=0x7f5dac02a1c0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5dbb789640 1 -- 172.21.15.26:0/1055775115 shutdown_connections 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5dbb789640 1 --2- 172.21.15.26:0/1055775115 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4071720 0x7f5db4071b70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5dbb789640 1 --2- 172.21.15.26:0/1055775115 >> v2:172.21.15.29:3300/0 conn(0x7f5db40d0ce0 0x7f5db406e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.215+0000 7f5dbb789640 1 -- 172.21.15.26:0/1055775115 >> 172.21.15.26:0/1055775115 conn(0x7f5db40b2970 msgr2=0x7f5db40d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.216+0000 7f5dbb789640 1 -- 172.21.15.26:0/1055775115 shutdown_connections 2024-04-24T23:28:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.216+0000 7f5dbb789640 1 -- 172.21.15.26:0/1055775115 wait complete. 2024-04-24T23:28:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.217+0000 7f5dbb789640 1 Processor -- start 2024-04-24T23:28:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.217+0000 7f5dbb789640 1 -- start start 2024-04-24T23:28:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.217+0000 7f5dbb789640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5db4071720 0x7f5db4158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.217+0000 7f5dbb789640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5db40d0ce0 0x7f5db414ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.217+0000 7f5dbb789640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4158ae0 0x7f5db40b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.217+0000 7f5dbb789640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5db406f530 con 0x7f5db4071720 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.217+0000 7f5dbb789640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5db415f060 con 0x7f5db40d0ce0 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.217+0000 7f5dbb789640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5db40aff60 con 0x7f5db4158ae0 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9525640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5db4071720 0x7f5db4158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9525640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5db4071720 0x7f5db4158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44126/0 (socket says 172.21.15.26:44126) 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9525640 1 -- 172.21.15.26:0/3082402103 learned_addr learned my addr 172.21.15.26:0/3082402103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9d26640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4158ae0 0x7f5db40b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db8d24640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5db40d0ce0 0x7f5db414ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9525640 1 -- 172.21.15.26:0/3082402103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4158ae0 msgr2=0x7f5db40b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9525640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4158ae0 0x7f5db40b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9525640 1 -- 172.21.15.26:0/3082402103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5db40d0ce0 msgr2=0x7f5db414ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9525640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5db40d0ce0 0x7f5db414ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9525640 1 -- 172.21.15.26:0/3082402103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db4154480 con 0x7f5db4071720 2024-04-24T23:28:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db9525640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5db4071720 0x7f5db4158ec0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c760 tx=0x7f5da400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db1ffb640 1 -- 172.21.15.26:0/3082402103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4016020 con 0x7f5db4071720 2024-04-24T23:28:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db1ffb640 1 -- 172.21.15.26:0/3082402103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da4006d50 con 0x7f5db4071720 2024-04-24T23:28:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db41546f0 con 0x7f5db4071720 2024-04-24T23:28:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5db1ffb640 1 -- 172.21.15.26:0/3082402103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4003bb0 con 0x7f5db4071720 2024-04-24T23:28:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.218+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db4154cb0 con 0x7f5db4071720 2024-04-24T23:28:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.219+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db4075610 con 0x7f5db4071720 2024-04-24T23:28:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.220+0000 7f5db1ffb640 1 -- 172.21.15.26:0/3082402103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5da400a850 con 0x7f5db4071720 2024-04-24T23:28:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.220+0000 7f5db1ffb640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5da006be30 0x7f5da006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.220+0000 7f5db1ffb640 1 -- 172.21.15.26:0/3082402103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f5da408a980 con 0x7f5db4071720 2024-04-24T23:28:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.220+0000 7f5db8d24640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5da006be30 0x7f5da006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.221+0000 7f5db8d24640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5da006be30 0x7f5da006e2f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5db415ccb0 tx=0x7f5da8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.224+0000 7f5db1ffb640 1 -- 172.21.15.26:0/3082402103 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5da4020020 con 0x7f5db4071720 2024-04-24T23:28:10.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.405+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5db40b5b40 con 0x7f5db4071720 2024-04-24T23:28:10.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.406+0000 7f5db1ffb640 1 -- 172.21.15.26:0/3082402103 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f5da4058cf0 con 0x7f5db4071720 2024-04-24T23:28:10.406 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:28:10.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5da006be30 msgr2=0x7f5da006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5da006be30 0x7f5da006e2f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5db415ccb0 tx=0x7f5da8009190 comp rx=0 tx=0).stop 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5db4071720 msgr2=0x7f5db4158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5db4071720 0x7f5db4158ec0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c760 tx=0x7f5da400cc20 comp rx=0 tx=0).stop 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 shutdown_connections 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5da006be30 0x7f5da006e2f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5db4158ae0 0x7f5db40b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5db40d0ce0 0x7f5db414ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 --2- 172.21.15.26:0/3082402103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5db4071720 0x7f5db4158ec0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 >> 172.21.15.26:0/3082402103 conn(0x7f5db40b2970 msgr2=0x7f5db4157eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 shutdown_connections 2024-04-24T23:28:10.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.408+0000 7f5dbb789640 1 -- 172.21.15.26:0/3082402103 wait complete. 2024-04-24T23:28:10.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:28:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.522+0000 7f1380a38640 1 Processor -- start 2024-04-24T23:28:10.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.522+0000 7f1380a38640 1 -- start start 2024-04-24T23:28:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f1380a38640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c158ae0 0x7f137c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f1380a38640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f137c06b1c0 con 0x7f137c071720 2024-04-24T23:28:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f1380a38640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f137c06ec90 con 0x7f137c0d0ce0 2024-04-24T23:28:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f1380a38640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f137c06ef70 con 0x7f137c158ae0 2024-04-24T23:28:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f137ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f137c071720 0x7f137c06b670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42822/0 (socket says 172.21.15.26:42822) 2024-04-24T23:28:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f137ad9d640 1 -- 172.21.15.26:0/1075802874 learned_addr learned my addr 172.21.15.26:0/1075802874 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f137a59c640 1 --2- 172.21.15.26:0/1075802874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c158ae0 0x7f137c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f1379d9b640 1 -- 172.21.15.26:0/1075802874 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 312270861 0 0) 0x7f137c06b1c0 con 0x7f137c071720 2024-04-24T23:28:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f1379d9b640 1 -- 172.21.15.26:0/1075802874 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1368003520 con 0x7f137c071720 2024-04-24T23:28:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f1379d9b640 1 -- 172.21.15.26:0/1075802874 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4277079291 0 0) 0x7f137c06ec90 con 0x7f137c0d0ce0 2024-04-24T23:28:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f1379d9b640 1 -- 172.21.15.26:0/1075802874 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f137c06b1c0 con 0x7f137c0d0ce0 2024-04-24T23:28:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f137a59c640 1 -- 172.21.15.26:0/1075802874 >> v1:172.21.15.29:6789/0 conn(0x7f137c0d0ce0 legacy=0x7f137c15e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f137a59c640 1 -- 172.21.15.26:0/1075802874 >> v1:172.21.15.26:6789/0 conn(0x7f137c071720 legacy=0x7f137c06b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.523+0000 7f137a59c640 1 -- 172.21.15.26:0/1075802874 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c06f250 con 0x7f137c158ae0 2024-04-24T23:28:10.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.524+0000 7f137a59c640 1 --2- 172.21.15.26:0/1075802874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c158ae0 0x7f137c06e620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f136c0097d0 tx=0x7f136c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8438ad2d392e431 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.524+0000 7f1379d9b640 1 -- 172.21.15.26:0/1075802874 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c037070 con 0x7f137c158ae0 2024-04-24T23:28:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.524+0000 7f1379d9b640 1 -- 172.21.15.26:0/1075802874 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c032440 con 0x7f137c158ae0 2024-04-24T23:28:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.524+0000 7f1379d9b640 1 -- 172.21.15.26:0/1075802874 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c0327f0 con 0x7f137c158ae0 2024-04-24T23:28:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.525+0000 7f1380a38640 1 -- 172.21.15.26:0/1075802874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c158ae0 msgr2=0x7f137c06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.525+0000 7f1380a38640 1 --2- 172.21.15.26:0/1075802874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c158ae0 0x7f137c06e620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f136c0097d0 tx=0x7f136c02a180 comp rx=0 tx=0).stop 2024-04-24T23:28:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.525+0000 7f1380a38640 1 -- 172.21.15.26:0/1075802874 shutdown_connections 2024-04-24T23:28:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.525+0000 7f1380a38640 1 --2- 172.21.15.26:0/1075802874 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c158ae0 0x7f137c06e620 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.525+0000 7f1380a38640 1 -- 172.21.15.26:0/1075802874 >> 172.21.15.26:0/1075802874 conn(0x7f137c0b2970 msgr2=0x7f137c0d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.525+0000 7f1380a38640 1 -- 172.21.15.26:0/1075802874 shutdown_connections 2024-04-24T23:28:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.525+0000 7f1380a38640 1 -- 172.21.15.26:0/1075802874 wait complete. 2024-04-24T23:28:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.526+0000 7f1380a38640 1 Processor -- start 2024-04-24T23:28:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.526+0000 7f1380a38640 1 -- start start 2024-04-24T23:28:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f1380a38640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c071720 0x7f137c14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f1380a38640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f137c0d0ce0 0x7f137c15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f1380a38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f137c158ae0 0x7f137c160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f1380a38640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f137c0aff60 con 0x7f137c0d0ce0 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f1380a38640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f137c06f530 con 0x7f137c158ae0 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f1380a38640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f137c0b00e0 con 0x7f137c071720 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f1371d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f137c0d0ce0 0x7f137c15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f1371d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f137c0d0ce0 0x7f137c15b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44140/0 (socket says 172.21.15.26:44140) 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f1371d9b640 1 -- 172.21.15.26:0/1946413958 learned_addr learned my addr 172.21.15.26:0/1946413958 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f137a59c640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c071720 0x7f137c14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.527+0000 7f137ad9d640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f137c158ae0 0x7f137c160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f1371d9b640 1 -- 172.21.15.26:0/1946413958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c071720 msgr2=0x7f137c14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f1371d9b640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c071720 0x7f137c14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f1371d9b640 1 -- 172.21.15.26:0/1946413958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f137c158ae0 msgr2=0x7f137c160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f1371d9b640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f137c158ae0 0x7f137c160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f1371d9b640 1 -- 172.21.15.26:0/1946413958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c159d20 con 0x7f137c0d0ce0 2024-04-24T23:28:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f137a59c640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c071720 0x7f137c14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f1371d9b640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f137c0d0ce0 0x7f137c15b720 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f136400c490 tx=0x7f136400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f13737fe640 1 -- 172.21.15.26:0/1946413958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640052d0 con 0x7f137c0d0ce0 2024-04-24T23:28:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f13737fe640 1 -- 172.21.15.26:0/1946413958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1364005470 con 0x7f137c0d0ce0 2024-04-24T23:28:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137c15a000 con 0x7f137c0d0ce0 2024-04-24T23:28:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f13737fe640 1 -- 172.21.15.26:0/1946413958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364012850 con 0x7f137c0d0ce0 2024-04-24T23:28:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.528+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137c15a5c0 con 0x7f137c0d0ce0 2024-04-24T23:28:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.529+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137c075610 con 0x7f137c0d0ce0 2024-04-24T23:28:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.529+0000 7f13737fe640 1 -- 172.21.15.26:0/1946413958 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13640129f0 con 0x7f137c0d0ce0 2024-04-24T23:28:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.530+0000 7f13737fe640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f135c06be30 0x7f135c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.530+0000 7f137a59c640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f135c06be30 0x7f135c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.530+0000 7f13737fe640 1 -- 172.21.15.26:0/1946413958 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f136408a9c0 con 0x7f137c0d0ce0 2024-04-24T23:28:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.530+0000 7f137a59c640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f135c06be30 0x7f135c06e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f136c034000 tx=0x7f136c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.534+0000 7f13737fe640 1 -- 172.21.15.26:0/1946413958 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f136408ce20 con 0x7f137c0d0ce0 2024-04-24T23:28:10.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.718+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f137c072370 con 0x7f137c0d0ce0 2024-04-24T23:28:10.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.718+0000 7f13737fe640 1 -- 172.21.15.26:0/1946413958 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f136405f030 con 0x7f137c0d0ce0 2024-04-24T23:28:10.719 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:28:10.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f135c06be30 msgr2=0x7f135c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f135c06be30 0x7f135c06e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f136c034000 tx=0x7f136c02e000 comp rx=0 tx=0).stop 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f137c0d0ce0 msgr2=0x7f137c15b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f137c0d0ce0 0x7f137c15b720 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f136400c490 tx=0x7f136400c950 comp rx=0 tx=0).stop 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 shutdown_connections 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f135c06be30 0x7f135c06e2f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f137c158ae0 0x7f137c160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f137c0d0ce0 0x7f137c15b720 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 --2- 172.21.15.26:0/1946413958 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f137c071720 0x7f137c14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.720+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 >> 172.21.15.26:0/1946413958 conn(0x7f137c0b2970 msgr2=0x7f137c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.721+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 shutdown_connections 2024-04-24T23:28:10.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.721+0000 7f1380a38640 1 -- 172.21.15.26:0/1946413958 wait complete. 2024-04-24T23:28:10.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:28:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.836+0000 7fec5e2ca640 1 Processor -- start 2024-04-24T23:28:10.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.836+0000 7fec5e2ca640 1 -- start start 2024-04-24T23:28:10.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.836+0000 7fec5e2ca640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 0x7fec580d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5e2ca640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec5815e9c0 0x7fec580d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5e2ca640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec5815ef60 con 0x7fec5815adc0 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5e2ca640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fec5815f240 con 0x7fec5815e9c0 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5e2ca640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fec5815f520 con 0x7fec580ad7b0 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5c867640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec5815e9c0 0x7fec580d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5c867640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec5815e9c0 0x7fec580d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44152/0 (socket says 172.21.15.26:44152) 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5c867640 1 -- 172.21.15.26:0/2880552715 learned_addr learned my addr 172.21.15.26:0/2880552715 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec577fe640 1 --2- 172.21.15.26:0/2880552715 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 0x7fec580d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5c867640 1 -- 172.21.15.26:0/2880552715 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 msgr2=0x7fec580d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5c867640 1 --2- 172.21.15.26:0/2880552715 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 0x7fec580d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5c867640 1 -- 172.21.15.26:0/2880552715 >> v1:172.21.15.29:6789/0 conn(0x7fec5815adc0 legacy=0x7fec580b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.837+0000 7fec5c867640 1 -- 172.21.15.26:0/2880552715 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5815f800 con 0x7fec5815e9c0 2024-04-24T23:28:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec5c867640 1 --2- 172.21.15.26:0/2880552715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815e9c0 0x7fec580d0580 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fec50009e50 tx=0x7fec5002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488d79305dea2a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec56ffd640 1 -- 172.21.15.26:0/2880552715 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50036070 con 0x7fec5815e9c0 2024-04-24T23:28:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec56ffd640 1 -- 172.21.15.26:0/2880552715 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec50031600 con 0x7fec5815e9c0 2024-04-24T23:28:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec56ffd640 1 -- 172.21.15.26:0/2880552715 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec500319d0 con 0x7fec5815e9c0 2024-04-24T23:28:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec5e2ca640 1 -- 172.21.15.26:0/2880552715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815e9c0 msgr2=0x7fec580d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec5e2ca640 1 --2- 172.21.15.26:0/2880552715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815e9c0 0x7fec580d0580 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fec50009e50 tx=0x7fec5002a680 comp rx=0 tx=0).stop 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec5e2ca640 1 -- 172.21.15.26:0/2880552715 shutdown_connections 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec5e2ca640 1 --2- 172.21.15.26:0/2880552715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815e9c0 0x7fec580d0580 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec5e2ca640 1 --2- 172.21.15.26:0/2880552715 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 0x7fec580d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec5e2ca640 1 -- 172.21.15.26:0/2880552715 >> 172.21.15.26:0/2880552715 conn(0x7fec5806b540 msgr2=0x7fec5806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.838+0000 7fec5e2ca640 1 -- 172.21.15.26:0/2880552715 shutdown_connections 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.839+0000 7fec5e2ca640 1 -- 172.21.15.26:0/2880552715 wait complete. 2024-04-24T23:28:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.840+0000 7fec5e2ca640 1 Processor -- start 2024-04-24T23:28:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.840+0000 7fec5e2ca640 1 -- start start 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5e2ca640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 0x7fec58148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5e2ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815adc0 0x7fec581522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5e2ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5815e9c0 0x7fec58152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5e2ca640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec58072130 con 0x7fec5815adc0 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5e2ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fec5815b1a0 con 0x7fec5815e9c0 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5e2ca640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fec580722b0 con 0x7fec580ad7b0 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815adc0 0x7fec581522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815adc0 0x7fec581522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44164/0 (socket says 172.21.15.26:44164) 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec577fe640 1 -- 172.21.15.26:0/1634878242 learned_addr learned my addr 172.21.15.26:0/1634878242 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec57fff640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 0x7fec58148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5c867640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5815e9c0 0x7fec58152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5c867640 1 -- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 msgr2=0x7fec58148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5c867640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 0x7fec58148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5c867640 1 -- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815adc0 msgr2=0x7fec581522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5c867640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815adc0 0x7fec581522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5c867640 1 -- 172.21.15.26:0/1634878242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec58151c40 con 0x7fec5815e9c0 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec577fe640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815adc0 0x7fec581522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec57fff640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 0x7fec58148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5c867640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5815e9c0 0x7fec58152800 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fec50030000 tx=0x7fec50004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec54ff9640 1 -- 172.21.15.26:0/1634878242 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50036050 con 0x7fec5815e9c0 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec54ff9640 1 -- 172.21.15.26:0/1634878242 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec50042070 con 0x7fec5815e9c0 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.841+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec580b1ec0 con 0x7fec5815e9c0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.842+0000 7fec54ff9640 1 -- 172.21.15.26:0/1634878242 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50009510 con 0x7fec5815e9c0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.842+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec580cd2b0 con 0x7fec5815e9c0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.842+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58075610 con 0x7fec5815e9c0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.843+0000 7fec54ff9640 1 -- 172.21.15.26:0/1634878242 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec50007c10 con 0x7fec5815e9c0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.843+0000 7fec54ff9640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4406be30 0x7fec4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.844+0000 7fec54ff9640 1 -- 172.21.15.26:0/1634878242 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fec500abe10 con 0x7fec5815e9c0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.844+0000 7fec57fff640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4406be30 0x7fec4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.844+0000 7fec57fff640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4406be30 0x7fec4406e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fec48001b70 tx=0x7fec48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:10.851+0000 7fec54ff9640 1 -- 172.21.15.26:0/1634878242 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec58151c40 con 0x7fec5815e9c0 2024-04-24T23:28:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.036+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec580719f0 con 0x7fec5815e9c0 2024-04-24T23:28:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.037+0000 7fec54ff9640 1 -- 172.21.15.26:0/1634878242 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7fec5007a0d0 con 0x7fec5815e9c0 2024-04-24T23:28:11.038 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:28:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4406be30 msgr2=0x7fec4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4406be30 0x7fec4406e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fec48001b70 tx=0x7fec48009040 comp rx=0 tx=0).stop 2024-04-24T23:28:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5815e9c0 msgr2=0x7fec58152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5815e9c0 0x7fec58152800 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fec50030000 tx=0x7fec50004a40 comp rx=0 tx=0).stop 2024-04-24T23:28:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 shutdown_connections 2024-04-24T23:28:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fec4406be30 0x7fec4406e2f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5815e9c0 0x7fec58152800 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5815adc0 0x7fec581522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 --2- 172.21.15.26:0/1634878242 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fec580ad7b0 0x7fec58148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 >> 172.21.15.26:0/1634878242 conn(0x7fec5806b540 msgr2=0x7fec58072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:11.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 shutdown_connections 2024-04-24T23:28:11.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.039+0000 7fec5e2ca640 1 -- 172.21.15.26:0/1634878242 wait complete. 2024-04-24T23:28:11.051 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:28:11.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.152+0000 7f72e86cd640 1 Processor -- start 2024-04-24T23:28:11.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.152+0000 7f72e86cd640 1 -- start start 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e86cd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f72e0145910 0x7f72e006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e86cd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 0x7f72e015d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e86cd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72e015d570 con 0x7f72e00cd660 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e86cd640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f72e0145cf0 con 0x7f72e0145910 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e86cd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f72e00cfbc0 con 0x7f72e006d350 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e6c6a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f72e00cd660 0x7f72e015d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42838/0 (socket says 172.21.15.26:42838) 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e6c6a640 1 -- 172.21.15.26:0/1947793084 learned_addr learned my addr 172.21.15.26:0/1947793084 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e5c68640 1 --2- 172.21.15.26:0/1947793084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 0x7f72e015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e6469640 1 --2- 172.21.15.26:0/1947793084 >> v2:172.21.15.29:3300/0 conn(0x7f72e0145910 0x7f72e006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e5467640 1 -- 172.21.15.26:0/1947793084 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2556430971 0 0) 0x7f72d8001490 con 0x7f72e00cd660 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.153+0000 7f72e5467640 1 -- 172.21.15.26:0/1947793084 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72cc003520 con 0x7f72e00cd660 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.154+0000 7f72e5c68640 1 -- 172.21.15.26:0/1947793084 >> v2:172.21.15.29:3300/0 conn(0x7f72e0145910 msgr2=0x7f72e006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.154+0000 7f72e5c68640 1 --2- 172.21.15.26:0/1947793084 >> v2:172.21.15.29:3300/0 conn(0x7f72e0145910 0x7f72e006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.154+0000 7f72e5c68640 1 -- 172.21.15.26:0/1947793084 >> v1:172.21.15.26:6789/0 conn(0x7f72e00cd660 legacy=0x7f72e015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.154+0000 7f72e5c68640 1 -- 172.21.15.26:0/1947793084 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e006d7f0 con 0x7f72e006d350 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.154+0000 7f72e5c68640 1 --2- 172.21.15.26:0/1947793084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 0x7f72e015d030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f72d00097a0 tx=0x7f72d002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f869e37dfe1bc5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.154+0000 7f72e5467640 1 -- 172.21.15.26:0/1947793084 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0037070 con 0x7f72e006d350 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.154+0000 7f72e5467640 1 -- 172.21.15.26:0/1947793084 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d0032440 con 0x7f72e006d350 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.154+0000 7f72e5467640 1 -- 172.21.15.26:0/1947793084 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0032810 con 0x7f72e006d350 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.155+0000 7f72e86cd640 1 -- 172.21.15.26:0/1947793084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 msgr2=0x7f72e015d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.155+0000 7f72e86cd640 1 --2- 172.21.15.26:0/1947793084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 0x7f72e015d030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f72d00097a0 tx=0x7f72d002a180 comp rx=0 tx=0).stop 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.155+0000 7f72e86cd640 1 -- 172.21.15.26:0/1947793084 shutdown_connections 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.155+0000 7f72e86cd640 1 --2- 172.21.15.26:0/1947793084 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 0x7f72e015d030 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.155+0000 7f72e86cd640 1 --2- 172.21.15.26:0/1947793084 >> v2:172.21.15.29:3300/0 conn(0x7f72e0145910 0x7f72e006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.155+0000 7f72e86cd640 1 -- 172.21.15.26:0/1947793084 >> 172.21.15.26:0/1947793084 conn(0x7f72e006b300 msgr2=0x7f72e006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.155+0000 7f72e86cd640 1 -- 172.21.15.26:0/1947793084 shutdown_connections 2024-04-24T23:28:11.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.155+0000 7f72e86cd640 1 -- 172.21.15.26:0/1947793084 wait complete. 2024-04-24T23:28:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.156+0000 7f72e86cd640 1 Processor -- start 2024-04-24T23:28:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.156+0000 7f72e86cd640 1 -- start start 2024-04-24T23:28:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e86cd640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 0x7f72e013f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e86cd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72e00cd660 0x7f72e01f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e86cd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72e0145910 0x7f72e0143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e86cd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f72e006e090 con 0x7f72e00cd660 2024-04-24T23:28:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e86cd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f72e006ddb0 con 0x7f72e0145910 2024-04-24T23:28:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e86cd640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f72e006e370 con 0x7f72e006d350 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e5c68640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72e00cd660 0x7f72e01f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e5c68640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72e00cd660 0x7f72e01f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44168/0 (socket says 172.21.15.26:44168) 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e5c68640 1 -- 172.21.15.26:0/1155907672 learned_addr learned my addr 172.21.15.26:0/1155907672 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e6469640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 0x7f72e013f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e6c6a640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72e0145910 0x7f72e0143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e5c68640 1 -- 172.21.15.26:0/1155907672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 msgr2=0x7f72e013f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e5c68640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 0x7f72e013f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e5c68640 1 -- 172.21.15.26:0/1155907672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72e0145910 msgr2=0x7f72e0143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e5c68640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72e0145910 0x7f72e0143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.157+0000 7f72e5c68640 1 -- 172.21.15.26:0/1155907672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e013fce0 con 0x7f72e00cd660 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.158+0000 7f72e5c68640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72e00cd660 0x7f72e01f4600 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f72d00098c0 tx=0x7f72d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.158+0000 7f72d6ffd640 1 -- 172.21.15.26:0/1155907672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0041030 con 0x7f72e00cd660 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.158+0000 7f72d6ffd640 1 -- 172.21.15.26:0/1155907672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d003d4e0 con 0x7f72e00cd660 2024-04-24T23:28:11.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.158+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e013ffc0 con 0x7f72e00cd660 2024-04-24T23:28:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.158+0000 7f72d6ffd640 1 -- 172.21.15.26:0/1155907672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0037040 con 0x7f72e00cd660 2024-04-24T23:28:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.158+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e00addc0 con 0x7f72e00cd660 2024-04-24T23:28:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.159+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e00694c0 con 0x7f72e00cd660 2024-04-24T23:28:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.159+0000 7f72d6ffd640 1 -- 172.21.15.26:0/1155907672 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72d0044070 con 0x7f72e00cd660 2024-04-24T23:28:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.160+0000 7f72d6ffd640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72c406be30 0x7f72c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.160+0000 7f72d6ffd640 1 -- 172.21.15.26:0/1155907672 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f72d00ab3a0 con 0x7f72e00cd660 2024-04-24T23:28:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.160+0000 7f72e6469640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72c406be30 0x7f72c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.160+0000 7f72e6469640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72c406be30 0x7f72c406e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009aa0 tx=0x7f72dc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.164+0000 7f72d6ffd640 1 -- 172.21.15.26:0/1155907672 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f72d0079710 con 0x7f72e00cd660 2024-04-24T23:28:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.360+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f72e00ce0c0 con 0x7f72e00cd660 2024-04-24T23:28:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.360+0000 7f72d6ffd640 1 -- 172.21.15.26:0/1155907672 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) ==== 80+0+0 (secure 0 0 0) 0x7f72d007da10 con 0x7f72e00cd660 2024-04-24T23:28:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-24T23:28:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72c406be30 msgr2=0x7f72c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72c406be30 0x7f72c406e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009aa0 tx=0x7f72dc009190 comp rx=0 tx=0).stop 2024-04-24T23:28:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72e00cd660 msgr2=0x7f72e01f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72e00cd660 0x7f72e01f4600 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f72d00098c0 tx=0x7f72d00046c0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 shutdown_connections 2024-04-24T23:28:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f72c406be30 0x7f72c406e2f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f72e0145910 0x7f72e0143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72e00cd660 0x7f72e01f4600 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 --2- 172.21.15.26:0/1155907672 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f72e006d350 0x7f72e013f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 >> 172.21.15.26:0/1155907672 conn(0x7f72e006b300 msgr2=0x7f72e00b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 shutdown_connections 2024-04-24T23:28:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.362+0000 7f72e86cd640 1 -- 172.21.15.26:0/1155907672 wait complete. 2024-04-24T23:28:11.374 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:28:11.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:11.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:28:11.402 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:28:11.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:11.403 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:28:11.458 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:28:11.458 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:28:11.458 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:11.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:28:11.475 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:28:11.475 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:28:11.475 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:11.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:28:11.513 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T23:28:11.513 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:28:11.514 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:11.530 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:28:11.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.654+0000 7efe84dce640 1 Processor -- start 2024-04-24T23:28:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.654+0000 7efe84dce640 1 -- start start 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe84dce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80145910 0x7efe8006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe84dce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efe800cd660 0x7efe800cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe84dce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe800cfbc0 con 0x7efe8006d350 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe84dce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efe800cdff0 con 0x7efe800cd660 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe84dce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efe80145cf0 con 0x7efe80145910 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe77fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efe8006d350 0x7efe8015d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42854/0 (socket says 172.21.15.26:42854) 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe77fff640 1 -- 172.21.15.26:0/1636531717 learned_addr learned my addr 172.21.15.26:0/1636531717 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 264898210 0 0) 0x7efe800cfbc0 con 0x7efe8006d350 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe6c003520 con 0x7efe8006d350 2024-04-24T23:28:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7e59c640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80145910 0x7efe8006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7ed9d640 1 --2- 172.21.15.26:0/1636531717 >> v2:172.21.15.29:3300/0 conn(0x7efe800cd660 0x7efe800cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3791683768 0 0) 0x7efe6c003520 con 0x7efe8006d350 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80145910 msgr2=0x7efe8006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80145910 0x7efe8006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 >> v2:172.21.15.29:3300/0 conn(0x7efe800cd660 msgr2=0x7efe800cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 --2- 172.21.15.26:0/1636531717 >> v2:172.21.15.29:3300/0 conn(0x7efe800cd660 0x7efe800cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8006d7f0 con 0x7efe8006d350 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7efe68001670 con 0x7efe8006d350 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.655+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 >> v1:172.21.15.26:6789/0 conn(0x7efe8006d350 legacy=0x7efe8015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe7dd9b640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe6c0039b0 0x7efe6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe7dd9b640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe7dd9b640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80145910 0x7efe6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe800cdff0 con 0x7efe6c004800 2024-04-24T23:28:11.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efe80145cf0 con 0x7efe6c0039b0 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c00b9c0 con 0x7efe80145910 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe77fff640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe7e59c640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe6c0039b0 0x7efe6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe7ed9d640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80145910 0x7efe6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe77fff640 1 -- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80145910 msgr2=0x7efe6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe77fff640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80145910 0x7efe6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe77fff640 1 -- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe6c0039b0 msgr2=0x7efe6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe77fff640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe6c0039b0 0x7efe6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe77fff640 1 -- 172.21.15.26:0/1636531717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c003520 con 0x7efe6c004800 2024-04-24T23:28:11.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.656+0000 7efe7e59c640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe6c0039b0 0x7efe6c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe77fff640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe6c004c80 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7efe6800c290 tx=0x7efe6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2217a2a7f5abde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68007d50 con 0x7efe6c004800 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe68005ad0 con 0x7efe6c004800 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe7dd9b640 1 -- 172.21.15.26:0/1636531717 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68007070 con 0x7efe6c004800 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe84dce640 1 -- 172.21.15.26:0/1636531717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 msgr2=0x7efe6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe84dce640 1 --2- 172.21.15.26:0/1636531717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe6c004c80 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7efe6800c290 tx=0x7efe6c005830 comp rx=0 tx=0).stop 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe7ed9d640 1 -- 172.21.15.26:0/1636531717 reap_dead start 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe84dce640 1 -- 172.21.15.26:0/1636531717 shutdown_connections 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe84dce640 1 -- 172.21.15.26:0/1636531717 >> 172.21.15.26:0/1636531717 conn(0x7efe8006b300 msgr2=0x7efe8006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe84dce640 1 -- 172.21.15.26:0/1636531717 shutdown_connections 2024-04-24T23:28:11.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.657+0000 7efe84dce640 1 -- 172.21.15.26:0/1636531717 wait complete. 2024-04-24T23:28:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.659+0000 7efe84dce640 1 Processor -- start 2024-04-24T23:28:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.659+0000 7efe84dce640 1 -- start start 2024-04-24T23:28:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.659+0000 7efe84dce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe8015b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.659+0000 7efe84dce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80127c50 0x7efe80128b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.659+0000 7efe84dce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe801f16c0 0x7efe801f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.659+0000 7efe84dce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe8006e090 con 0x7efe6c004800 2024-04-24T23:28:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.659+0000 7efe84dce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efe8006ddb0 con 0x7efe801f16c0 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.659+0000 7efe84dce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efe8006e370 con 0x7efe80127c50 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe8015b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe8015b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44198/0 (socket says 172.21.15.26:44198) 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7e59c640 1 -- 172.21.15.26:0/4235044143 learned_addr learned my addr 172.21.15.26:0/4235044143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7ed9d640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe801f16c0 0x7efe801f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7e59c640 1 -- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80127c50 msgr2=0x7efe80128b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe77fff640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80127c50 0x7efe80128b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7e59c640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80127c50 0x7efe80128b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7e59c640 1 -- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe801f16c0 msgr2=0x7efe801f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7e59c640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe801f16c0 0x7efe801f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7e59c640 1 -- 172.21.15.26:0/4235044143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe801290b0 con 0x7efe6c004800 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7ed9d640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe801f16c0 0x7efe801f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe77fff640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80127c50 0x7efe80128b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe7e59c640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe8015b9b0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7efe70011a70 tx=0x7efe70011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe76ffd640 1 -- 172.21.15.26:0/4235044143 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7000fbb0 con 0x7efe6c004800 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe76ffd640 1 -- 172.21.15.26:0/4235044143 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7000fd50 con 0x7efe6c004800 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8013d920 con 0x7efe6c004800 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe76ffd640 1 -- 172.21.15.26:0/4235044143 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70014640 con 0x7efe6c004800 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.660+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8013dee0 con 0x7efe6c004800 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.661+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe800694c0 con 0x7efe6c004800 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.662+0000 7efe76ffd640 1 -- 172.21.15.26:0/4235044143 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe70012040 con 0x7efe6c004800 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.662+0000 7efe76ffd640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe6006be30 0x7efe6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.662+0000 7efe76ffd640 1 -- 172.21.15.26:0/4235044143 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7efe7008b520 con 0x7efe6c004800 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.662+0000 7efe77fff640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe6006be30 0x7efe6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:11.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.663+0000 7efe77fff640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe6006be30 0x7efe6006e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7efe68033000 tx=0x7efe68009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:11.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.666+0000 7efe76ffd640 1 -- 172.21.15.26:0/4235044143 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe70019030 con 0x7efe6c004800 2024-04-24T23:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.892+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe8013e2c0 con 0x7efe6c004800 2024-04-24T23:28:11.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.893+0000 7efe76ffd640 1 -- 172.21.15.26:0/4235044143 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7efe70059f60 con 0x7efe6c004800 2024-04-24T23:28:11.893 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:11.894 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:11.894 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:11.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.894+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe6006be30 msgr2=0x7efe6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.894+0000 7efe84dce640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe6006be30 0x7efe6006e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7efe68033000 tx=0x7efe68009030 comp rx=0 tx=0).stop 2024-04-24T23:28:11.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.894+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 msgr2=0x7efe8015b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:11.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.894+0000 7efe84dce640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe8015b9b0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7efe70011a70 tx=0x7efe70011f30 comp rx=0 tx=0).stop 2024-04-24T23:28:11.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.895+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 shutdown_connections 2024-04-24T23:28:11.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.895+0000 7efe84dce640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efe6006be30 0x7efe6006e2f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.895+0000 7efe84dce640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efe801f16c0 0x7efe801f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.895+0000 7efe84dce640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efe80127c50 0x7efe80128b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.895+0000 7efe84dce640 1 --2- 172.21.15.26:0/4235044143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6c004800 0x7efe8015b9b0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:11.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.895+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 >> 172.21.15.26:0/4235044143 conn(0x7efe8006b300 msgr2=0x7efe800b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:11.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.895+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 shutdown_connections 2024-04-24T23:28:11.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:11.895+0000 7efe84dce640 1 -- 172.21.15.26:0/4235044143 wait complete. 2024-04-24T23:28:11.907 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:28:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.010+0000 7f39afa76640 1 Processor -- start 2024-04-24T23:28:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.010+0000 7f39afa76640 1 -- start start 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39afa76640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f39a815a860 0x7f39a806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39afa76640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a806e120 0x7f39a8071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39afa76640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a80b2e80 con 0x7f39a815a480 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39afa76640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f39a806e550 con 0x7f39a815a860 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39afa76640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f39a80b3120 con 0x7f39a806e120 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39ad011640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f39a815a860 0x7f39a806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39ad011640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f39a815a860 0x7f39a806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44214/0 (socket says 172.21.15.26:44214) 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39ad011640 1 -- 172.21.15.26:0/708041534 learned_addr learned my addr 172.21.15.26:0/708041534 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39ae013640 1 --2- 172.21.15.26:0/708041534 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a806e120 0x7f39a8071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39ad011640 1 -- 172.21.15.26:0/708041534 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a806e120 msgr2=0x7f39a8071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39ad011640 1 --2- 172.21.15.26:0/708041534 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a806e120 0x7f39a8071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39ad011640 1 -- 172.21.15.26:0/708041534 >> v1:172.21.15.29:6789/0 conn(0x7f39a815a480 legacy=0x7f39a80b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.011+0000 7f39ad011640 1 -- 172.21.15.26:0/708041534 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80b33c0 con 0x7f39a815a860 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39ad011640 1 --2- 172.21.15.26:0/708041534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a815a860 0x7f39a806dbe0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f3998009870 tx=0x7f399802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b8750a7938f6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39ac810640 1 -- 172.21.15.26:0/708041534 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998037070 con 0x7f39a815a860 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39ac810640 1 -- 172.21.15.26:0/708041534 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3998032440 con 0x7f39a815a860 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39ac810640 1 -- 172.21.15.26:0/708041534 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998032810 con 0x7f39a815a860 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39afa76640 1 -- 172.21.15.26:0/708041534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a815a860 msgr2=0x7f39a806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39afa76640 1 --2- 172.21.15.26:0/708041534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a815a860 0x7f39a806dbe0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f3998009870 tx=0x7f399802a270 comp rx=0 tx=0).stop 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39afa76640 1 -- 172.21.15.26:0/708041534 shutdown_connections 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39afa76640 1 --2- 172.21.15.26:0/708041534 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a806e120 0x7f39a8071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39afa76640 1 --2- 172.21.15.26:0/708041534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a815a860 0x7f39a806dbe0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.012+0000 7f39afa76640 1 -- 172.21.15.26:0/708041534 >> 172.21.15.26:0/708041534 conn(0x7f39a806b300 msgr2=0x7f39a806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.013+0000 7f39afa76640 1 -- 172.21.15.26:0/708041534 shutdown_connections 2024-04-24T23:28:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.013+0000 7f39afa76640 1 -- 172.21.15.26:0/708041534 wait complete. 2024-04-24T23:28:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.014+0000 7f39afa76640 1 Processor -- start 2024-04-24T23:28:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.014+0000 7f39afa76640 1 -- start start 2024-04-24T23:28:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.014+0000 7f39afa76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a806e120 0x7f39a814fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.014+0000 7f39afa76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a815a480 0x7f39a8143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.014+0000 7f39afa76640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a815a860 0x7f39a8143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.014+0000 7f39afa76640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f39a80b3c60 con 0x7f39a806e120 2024-04-24T23:28:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.014+0000 7f39afa76640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f39a80b3980 con 0x7f39a815a480 2024-04-24T23:28:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.014+0000 7f39afa76640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f39a80b5000 con 0x7f39a815a860 2024-04-24T23:28:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad812640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a806e120 0x7f39a814fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad812640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a806e120 0x7f39a814fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44230/0 (socket says 172.21.15.26:44230) 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad812640 1 -- 172.21.15.26:0/3966533439 learned_addr learned my addr 172.21.15.26:0/3966533439 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ae013640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a815a860 0x7f39a8143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad011640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a815a480 0x7f39a8143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad812640 1 -- 172.21.15.26:0/3966533439 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a815a860 msgr2=0x7f39a8143b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad812640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a815a860 0x7f39a8143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad812640 1 -- 172.21.15.26:0/3966533439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a815a480 msgr2=0x7f39a8143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad812640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a815a480 0x7f39a8143610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad812640 1 -- 172.21.15.26:0/3966533439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a814d780 con 0x7f39a806e120 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39ad812640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a806e120 0x7f39a814fcd0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c430 tx=0x7f39a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f399e7fc640 1 -- 172.21.15.26:0/3966533439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4017070 con 0x7f39a806e120 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f399e7fc640 1 -- 172.21.15.26:0/3966533439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a40052d0 con 0x7f39a806e120 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a814da60 con 0x7f39a806e120 2024-04-24T23:28:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f399e7fc640 1 -- 172.21.15.26:0/3966533439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4003850 con 0x7f39a806e120 2024-04-24T23:28:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.015+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a8150210 con 0x7f39a806e120 2024-04-24T23:28:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.016+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a80694c0 con 0x7f39a806e120 2024-04-24T23:28:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.017+0000 7f399e7fc640 1 -- 172.21.15.26:0/3966533439 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39a401a070 con 0x7f39a806e120 2024-04-24T23:28:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.017+0000 7f399e7fc640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f398c06be30 0x7f398c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.017+0000 7f399e7fc640 1 -- 172.21.15.26:0/3966533439 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f39a408a220 con 0x7f39a806e120 2024-04-24T23:28:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.017+0000 7f39ad011640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f398c06be30 0x7f398c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.018+0000 7f39ad011640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f398c06be30 0x7f398c06e2f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3998034000 tx=0x7f399802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.021+0000 7f399e7fc640 1 -- 172.21.15.26:0/3966533439 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f39a4058bd0 con 0x7f39a806e120 2024-04-24T23:28:12.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.205+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39a80b44a0 con 0x7f39a806e120 2024-04-24T23:28:12.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.206+0000 7f399e7fc640 1 -- 172.21.15.26:0/3966533439 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16548 (secure 0 0 0) 0x7f39a40584d0 con 0x7f39a806e120 2024-04-24T23:28:12.207 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:27:24.935708+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:21.600058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:22.278230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:28:12.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f398c06be30 msgr2=0x7f398c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f398c06be30 0x7f398c06e2f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3998034000 tx=0x7f399802e000 comp rx=0 tx=0).stop 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a806e120 msgr2=0x7f39a814fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a806e120 0x7f39a814fcd0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c430 tx=0x7f39a400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 shutdown_connections 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f398c06be30 0x7f398c06e2f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f39a815a860 0x7f39a8143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a815a480 0x7f39a8143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 --2- 172.21.15.26:0/3966533439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a806e120 0x7f39a814fcd0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.208+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 >> 172.21.15.26:0/3966533439 conn(0x7f39a806b300 msgr2=0x7f39a80b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.209+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 shutdown_connections 2024-04-24T23:28:12.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.209+0000 7f39afa76640 1 -- 172.21.15.26:0/3966533439 wait complete. 2024-04-24T23:28:12.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:20:43.358101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:21.600058+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:22.278230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:28:12.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:28:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.323+0000 7f30d7bf9640 1 Processor -- start 2024-04-24T23:28:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.323+0000 7f30d7bf9640 1 -- start start 2024-04-24T23:28:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.324+0000 7f30d7bf9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f30d0158ae0 0x7f30d006e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.324+0000 7f30d7bf9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d006b1c0 con 0x7f30d0071720 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.324+0000 7f30d7bf9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d006ec90 con 0x7f30d00d0ce0 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.324+0000 7f30d7bf9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f30d006ef70 con 0x7f30d0158ae0 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.324+0000 7f30d6196640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f30d0071720 0x7f30d0071b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42860/0 (socket says 172.21.15.26:42860) 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.324+0000 7f30d6196640 1 -- 172.21.15.26:0/2164142998 learned_addr learned my addr 172.21.15.26:0/2164142998 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.324+0000 7f30d5995640 1 --2- 172.21.15.26:0/2164142998 >> v2:172.21.15.29:3300/0 conn(0x7f30d0158ae0 0x7f30d006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.325+0000 7f30d4993640 1 -- 172.21.15.26:0/2164142998 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3870705298 0 0) 0x7f30d006b1c0 con 0x7f30d0071720 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.325+0000 7f30d4993640 1 -- 172.21.15.26:0/2164142998 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30bc003520 con 0x7f30d0071720 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.325+0000 7f30d5995640 1 -- 172.21.15.26:0/2164142998 >> v1:172.21.15.29:6789/0 conn(0x7f30d00d0ce0 legacy=0x7f30d015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.325+0000 7f30d4993640 1 -- 172.21.15.26:0/2164142998 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1549492026 0 0) 0x7f30d006ec90 con 0x7f30d00d0ce0 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.325+0000 7f30d5995640 1 -- 172.21.15.26:0/2164142998 >> v1:172.21.15.26:6789/0 conn(0x7f30d0071720 legacy=0x7f30d0071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.325+0000 7f30d5995640 1 -- 172.21.15.26:0/2164142998 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d006f250 con 0x7f30d0158ae0 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d5995640 1 --2- 172.21.15.26:0/2164142998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d0158ae0 0x7f30d006e620 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0097a0 tx=0x7f30cc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c3475345f4fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d4993640 1 -- 172.21.15.26:0/2164142998 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc037070 con 0x7f30d0158ae0 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d4993640 1 -- 172.21.15.26:0/2164142998 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30cc02ab90 con 0x7f30d0158ae0 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d4993640 1 -- 172.21.15.26:0/2164142998 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c80019f0 con 0x7f30d0158ae0 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d7bf9640 1 -- 172.21.15.26:0/2164142998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d0158ae0 msgr2=0x7f30d006e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d7bf9640 1 --2- 172.21.15.26:0/2164142998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d0158ae0 0x7f30d006e620 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0097a0 tx=0x7f30cc02a100 comp rx=0 tx=0).stop 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d7bf9640 1 -- 172.21.15.26:0/2164142998 shutdown_connections 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d7bf9640 1 --2- 172.21.15.26:0/2164142998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d0158ae0 0x7f30d006e620 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d7bf9640 1 -- 172.21.15.26:0/2164142998 >> 172.21.15.26:0/2164142998 conn(0x7f30d00b2970 msgr2=0x7f30d00d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.326+0000 7f30d7bf9640 1 -- 172.21.15.26:0/2164142998 shutdown_connections 2024-04-24T23:28:12.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.327+0000 7f30d7bf9640 1 -- 172.21.15.26:0/2164142998 wait complete. 2024-04-24T23:28:12.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d7bf9640 1 Processor -- start 2024-04-24T23:28:12.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d7bf9640 1 -- start start 2024-04-24T23:28:12.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d7bf9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d0071720 0x7f30d015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d7bf9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d00d0ce0 0x7f30d0149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d7bf9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f30d0158ae0 0x7f30d01f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d7bf9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30d00aff60 con 0x7f30d0071720 2024-04-24T23:28:12.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d7bf9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30d006f530 con 0x7f30d00d0ce0 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d7bf9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f30d00b00e0 con 0x7f30d0158ae0 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d5995640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d0071720 0x7f30d015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d5995640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d0071720 0x7f30d015b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44236/0 (socket says 172.21.15.26:44236) 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.328+0000 7f30d5995640 1 -- 172.21.15.26:0/1506332314 learned_addr learned my addr 172.21.15.26:0/1506332314 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d5194640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d00d0ce0 0x7f30d0149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d6196640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f30d0158ae0 0x7f30d01f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d5995640 1 -- 172.21.15.26:0/1506332314 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f30d0158ae0 msgr2=0x7f30d01f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d5995640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f30d0158ae0 0x7f30d01f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d5995640 1 -- 172.21.15.26:0/1506332314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d00d0ce0 msgr2=0x7f30d0149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d5995640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d00d0ce0 0x7f30d0149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d5995640 1 -- 172.21.15.26:0/1506332314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d015bd20 con 0x7f30d0071720 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d5995640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d0071720 0x7f30d015b7e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0097a0 tx=0x7f30cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30c67fc640 1 -- 172.21.15.26:0/1506332314 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc037040 con 0x7f30d0071720 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30c67fc640 1 -- 172.21.15.26:0/1506332314 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30cc004030 con 0x7f30d0071720 2024-04-24T23:28:12.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d014a480 con 0x7f30d0071720 2024-04-24T23:28:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30c67fc640 1 -- 172.21.15.26:0/1506332314 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc004400 con 0x7f30d0071720 2024-04-24T23:28:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.329+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d00ae1e0 con 0x7f30d0071720 2024-04-24T23:28:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.330+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d0075610 con 0x7f30d0071720 2024-04-24T23:28:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.331+0000 7f30c67fc640 1 -- 172.21.15.26:0/1506332314 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30cc03a040 con 0x7f30d0071720 2024-04-24T23:28:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.331+0000 7f30c67fc640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f30b406be30 0x7f30b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.331+0000 7f30c67fc640 1 -- 172.21.15.26:0/1506332314 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f30cc0ab2f0 con 0x7f30d0071720 2024-04-24T23:28:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.331+0000 7f30d5194640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f30b406be30 0x7f30b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.332+0000 7f30d5194640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f30b406be30 0x7f30b406e2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f30d015e2f0 tx=0x7f30c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.335+0000 7f30c67fc640 1 -- 172.21.15.26:0/1506332314 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f30cc042020 con 0x7f30d0071720 2024-04-24T23:28:12.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.517+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30d0067580 con 0x7f30d0071720 2024-04-24T23:28:12.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.518+0000 7f30c67fc640 1 -- 172.21.15.26:0/1506332314 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f30cc079770 con 0x7f30d0071720 2024-04-24T23:28:12.519 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-24T23:28:12.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f30b406be30 msgr2=0x7f30b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f30b406be30 0x7f30b406e2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f30d015e2f0 tx=0x7f30c0009040 comp rx=0 tx=0).stop 2024-04-24T23:28:12.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d0071720 msgr2=0x7f30d015b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d0071720 0x7f30d015b7e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0097a0 tx=0x7f30cc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 shutdown_connections 2024-04-24T23:28:12.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f30b406be30 0x7f30b406e2f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f30d0158ae0 0x7f30d01f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30d00d0ce0 0x7f30d0149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 --2- 172.21.15.26:0/1506332314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d0071720 0x7f30d015b7e0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 >> 172.21.15.26:0/1506332314 conn(0x7f30d00b2970 msgr2=0x7f30d0071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:12.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 shutdown_connections 2024-04-24T23:28:12.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.520+0000 7f30d7bf9640 1 -- 172.21.15.26:0/1506332314 wait complete. 2024-04-24T23:28:12.532 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:28:12.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.635+0000 7f43a4fa9640 1 Processor -- start 2024-04-24T23:28:12.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.635+0000 7f43a4fa9640 1 -- start start 2024-04-24T23:28:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f43a4fa9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f43a015be40 0x7f43a00acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f43a4fa9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43a00d0750 0x7f43a015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f43a4fa9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a00aeb10 con 0x7f43a00b4ce0 2024-04-24T23:28:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f43a4fa9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f43a009df20 con 0x7f43a00d0750 2024-04-24T23:28:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f43a4fa9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f43a00aedf0 con 0x7f43a015be40 2024-04-24T23:28:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43a00d0750 0x7f43a015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439dd9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f43a00b4ce0 0x7f43a00d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42868/0 (socket says 172.21.15.26:42868) 2024-04-24T23:28:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439dd9b640 1 -- 172.21.15.26:0/2033350432 learned_addr learned my addr 172.21.15.26:0/2033350432 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439e59c640 1 --2- 172.21.15.26:0/2033350432 >> v2:172.21.15.29:3300/0 conn(0x7f43a015be40 0x7f43a00acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439ed9d640 1 -- 172.21.15.26:0/2033350432 >> v2:172.21.15.29:3300/0 conn(0x7f43a015be40 msgr2=0x7f43a00acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439d59a640 1 -- 172.21.15.26:0/2033350432 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107662682 0 0) 0x7f43a00aeb10 con 0x7f43a00b4ce0 2024-04-24T23:28:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439ed9d640 1 --2- 172.21.15.26:0/2033350432 >> v2:172.21.15.29:3300/0 conn(0x7f43a015be40 0x7f43a00acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439ed9d640 1 -- 172.21.15.26:0/2033350432 >> v1:172.21.15.26:6789/0 conn(0x7f43a00b4ce0 legacy=0x7f43a00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439ed9d640 1 -- 172.21.15.26:0/2033350432 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a00af0d0 con 0x7f43a00d0750 2024-04-24T23:28:12.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.636+0000 7f439e59c640 1 --2- 172.21.15.26:0/2033350432 >> v2:172.21.15.29:3300/0 conn(0x7f43a015be40 0x7f43a00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.637+0000 7f439ed9d640 1 --2- 172.21.15.26:0/2033350432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00d0750 0x7f43a015c220 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f43a006e400 tx=0x7f439802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b264c74ea3218c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.637+0000 7f439d59a640 1 -- 172.21.15.26:0/2033350432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398007740 con 0x7f43a00d0750 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.637+0000 7f439d59a640 1 -- 172.21.15.26:0/2033350432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43980078e0 con 0x7f43a00d0750 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.637+0000 7f439d59a640 1 -- 172.21.15.26:0/2033350432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398007cb0 con 0x7f43a00d0750 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.638+0000 7f43a4fa9640 1 -- 172.21.15.26:0/2033350432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00d0750 msgr2=0x7f43a015c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.638+0000 7f43a4fa9640 1 --2- 172.21.15.26:0/2033350432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00d0750 0x7f43a015c220 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f43a006e400 tx=0x7f439802a680 comp rx=0 tx=0).stop 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.638+0000 7f43a4fa9640 1 -- 172.21.15.26:0/2033350432 shutdown_connections 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.638+0000 7f43a4fa9640 1 --2- 172.21.15.26:0/2033350432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00d0750 0x7f43a015c220 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.638+0000 7f43a4fa9640 1 --2- 172.21.15.26:0/2033350432 >> v2:172.21.15.29:3300/0 conn(0x7f43a015be40 0x7f43a00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.638+0000 7f43a4fa9640 1 -- 172.21.15.26:0/2033350432 >> 172.21.15.26:0/2033350432 conn(0x7f43a006b540 msgr2=0x7f43a006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:12.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.638+0000 7f43a4fa9640 1 -- 172.21.15.26:0/2033350432 shutdown_connections 2024-04-24T23:28:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.638+0000 7f43a4fa9640 1 -- 172.21.15.26:0/2033350432 wait complete. 2024-04-24T23:28:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.639+0000 7f43a4fa9640 1 Processor -- start 2024-04-24T23:28:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.639+0000 7f43a4fa9640 1 -- start start 2024-04-24T23:28:12.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f43a4fa9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00b4ce0 0x7f43a01ede30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f43a4fa9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d0750 0x7f43a0072710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f43a4fa9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a015be40 0x7f43a01f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f43a4fa9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f43a006cd10 con 0x7f43a00b4ce0 2024-04-24T23:28:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f43a4fa9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43a006cb90 con 0x7f43a015be40 2024-04-24T23:28:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f43a4fa9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f43a015c760 con 0x7f43a00d0750 2024-04-24T23:28:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00b4ce0 0x7f43a01ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00b4ce0 0x7f43a01ede30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44258/0 (socket says 172.21.15.26:44258) 2024-04-24T23:28:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439e59c640 1 -- 172.21.15.26:0/3855934367 learned_addr learned my addr 172.21.15.26:0/3855934367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439ed9d640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a015be40 0x7f43a01f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439dd9b640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d0750 0x7f43a0072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439e59c640 1 -- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d0750 msgr2=0x7f43a0072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439e59c640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d0750 0x7f43a0072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439e59c640 1 -- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a015be40 msgr2=0x7f43a01f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439e59c640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a015be40 0x7f43a01f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439e59c640 1 -- 172.21.15.26:0/3855934367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a01f82a0 con 0x7f43a00b4ce0 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439ed9d640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a015be40 0x7f43a01f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.640+0000 7f439dd9b640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d0750 0x7f43a0072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.641+0000 7f439e59c640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00b4ce0 0x7f43a01ede30 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f439000c7f0 tx=0x7f439000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.641+0000 7f438effd640 1 -- 172.21.15.26:0/3855934367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390015070 con 0x7f43a00b4ce0 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.641+0000 7f438effd640 1 -- 172.21.15.26:0/3855934367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4390007820 con 0x7f43a00b4ce0 2024-04-24T23:28:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.641+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a00b2340 con 0x7f43a00b4ce0 2024-04-24T23:28:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.641+0000 7f438effd640 1 -- 172.21.15.26:0/3855934367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390007bd0 con 0x7f43a00b4ce0 2024-04-24T23:28:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.641+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a00b2900 con 0x7f43a00b4ce0 2024-04-24T23:28:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.642+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a00ae3e0 con 0x7f43a00b4ce0 2024-04-24T23:28:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.642+0000 7f438effd640 1 -- 172.21.15.26:0/3855934367 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4390018070 con 0x7f43a00b4ce0 2024-04-24T23:28:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.643+0000 7f438effd640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438006be30 0x7f438006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.643+0000 7f438effd640 1 -- 172.21.15.26:0/3855934367 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f439008af10 con 0x7f43a00b4ce0 2024-04-24T23:28:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.643+0000 7f439dd9b640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438006be30 0x7f438006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.643+0000 7f439dd9b640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438006be30 0x7f438006e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f43a00ce800 tx=0x7f4394009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.647+0000 7f438effd640 1 -- 172.21.15.26:0/3855934367 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f43a00ae3e0 con 0x7f43a00b4ce0 2024-04-24T23:28:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.835+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f43a0075610 con 0x7f43a00b4ce0 2024-04-24T23:28:12.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.836+0000 7f438effd640 1 -- 172.21.15.26:0/3855934367 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f4390059950 con 0x7f43a00b4ce0 2024-04-24T23:28:12.837 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438006be30 msgr2=0x7f438006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438006be30 0x7f438006e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f43a00ce800 tx=0x7f4394009040 comp rx=0 tx=0).stop 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00b4ce0 msgr2=0x7f43a01ede30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00b4ce0 0x7f43a01ede30 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f439000c7f0 tx=0x7f439000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 shutdown_connections 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f438006be30 0x7f438006e2f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f43a015be40 0x7f43a01f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f43a00d0750 0x7f43a0072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 --2- 172.21.15.26:0/3855934367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a00b4ce0 0x7f43a01ede30 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 >> 172.21.15.26:0/3855934367 conn(0x7f43a006b540 msgr2=0x7f43a015a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 shutdown_connections 2024-04-24T23:28:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.838+0000 7f43a4fa9640 1 -- 172.21.15.26:0/3855934367 wait complete. 2024-04-24T23:28:12.849 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:28:12.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.952+0000 7efc2a0d4640 1 Processor -- start 2024-04-24T23:28:12.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.952+0000 7efc2a0d4640 1 -- start start 2024-04-24T23:28:12.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.953+0000 7efc2a0d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc2415c270 0x7efc240cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.953+0000 7efc2a0d4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc2415c650 0x7efc240d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.953+0000 7efc2a0d4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc240cf050 con 0x7efc240b2950 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.953+0000 7efc2a0d4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7efc240cee00 con 0x7efc2415c650 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.953+0000 7efc2a0d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc240d28e0 con 0x7efc2415c270 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.954+0000 7efc237fe640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc2415c270 0x7efc240cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.954+0000 7efc22ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc2415c650 0x7efc240d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.954+0000 7efc22ffd640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7efc2415c650 0x7efc240d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:36238/0 (socket says 172.21.15.26:36238) 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.954+0000 7efc22ffd640 1 -- 172.21.15.26:0/1750546840 learned_addr learned my addr 172.21.15.26:0/1750546840 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.954+0000 7efc237fe640 1 -- 172.21.15.26:0/1750546840 >> v2:172.21.15.29:3300/0 conn(0x7efc2415c650 msgr2=0x7efc240d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.954+0000 7efc237fe640 1 --2- 172.21.15.26:0/1750546840 >> v2:172.21.15.29:3300/0 conn(0x7efc2415c650 0x7efc240d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.954+0000 7efc237fe640 1 -- 172.21.15.26:0/1750546840 >> v1:172.21.15.29:6789/0 conn(0x7efc240b2950 legacy=0x7efc240c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.954+0000 7efc237fe640 1 -- 172.21.15.26:0/1750546840 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc240cdd60 con 0x7efc2415c270 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc237fe640 1 --2- 172.21.15.26:0/1750546840 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc2415c270 0x7efc240cfd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efc140097a0 tx=0x7efc1402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49833219e2911b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc227fc640 1 -- 172.21.15.26:0/1750546840 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14037070 con 0x7efc2415c270 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc227fc640 1 -- 172.21.15.26:0/1750546840 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1402ae50 con 0x7efc2415c270 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc227fc640 1 -- 172.21.15.26:0/1750546840 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14032660 con 0x7efc2415c270 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc2a0d4640 1 -- 172.21.15.26:0/1750546840 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc2415c270 msgr2=0x7efc240cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc2a0d4640 1 --2- 172.21.15.26:0/1750546840 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc2415c270 0x7efc240cfd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efc140097a0 tx=0x7efc1402a100 comp rx=0 tx=0).stop 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc2a0d4640 1 -- 172.21.15.26:0/1750546840 shutdown_connections 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc2a0d4640 1 --2- 172.21.15.26:0/1750546840 >> v2:172.21.15.29:3300/0 conn(0x7efc2415c650 0x7efc240d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc2a0d4640 1 --2- 172.21.15.26:0/1750546840 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc2415c270 0x7efc240cfd50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc2a0d4640 1 -- 172.21.15.26:0/1750546840 >> 172.21.15.26:0/1750546840 conn(0x7efc240d1dc0 msgr2=0x7efc240d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:12.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc2a0d4640 1 -- 172.21.15.26:0/1750546840 shutdown_connections 2024-04-24T23:28:12.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.955+0000 7efc2a0d4640 1 -- 172.21.15.26:0/1750546840 wait complete. 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.957+0000 7efc2a0d4640 1 Processor -- start 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.957+0000 7efc2a0d4640 1 -- start start 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.957+0000 7efc2a0d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc240b2950 0x7efc24147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.957+0000 7efc2a0d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc2415c270 0x7efc240c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.957+0000 7efc2a0d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc2415c650 0x7efc240c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.957+0000 7efc2a0d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc240b2e80 con 0x7efc2415c650 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.957+0000 7efc2a0d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc240cd9e0 con 0x7efc2415c270 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc2a0d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc240b3000 con 0x7efc240b2950 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc2415c650 0x7efc240c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc2415c650 0x7efc240c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44264/0 (socket says 172.21.15.26:44264) 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc23fff640 1 -- 172.21.15.26:0/3316759802 learned_addr learned my addr 172.21.15.26:0/3316759802 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:12.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc237fe640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc240b2950 0x7efc24147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc22ffd640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc2415c270 0x7efc240c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc23fff640 1 -- 172.21.15.26:0/3316759802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc240b2950 msgr2=0x7efc24147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc23fff640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc240b2950 0x7efc24147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc23fff640 1 -- 172.21.15.26:0/3316759802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc2415c270 msgr2=0x7efc240c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc23fff640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc2415c270 0x7efc240c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc23fff640 1 -- 172.21.15.26:0/3316759802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2414a1c0 con 0x7efc2415c650 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc23fff640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc2415c650 0x7efc240c76a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00c6b0 tx=0x7efc1c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc0bfff640 1 -- 172.21.15.26:0/3316759802 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c003b70 con 0x7efc2415c650 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc0bfff640 1 -- 172.21.15.26:0/3316759802 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1c003d30 con 0x7efc2415c650 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc241e5b50 con 0x7efc2415c650 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc0bfff640 1 -- 172.21.15.26:0/3316759802 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c005310 con 0x7efc2415c650 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.958+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241e6110 con 0x7efc2415c650 2024-04-24T23:28:12.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.959+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc240d21a0 con 0x7efc2415c650 2024-04-24T23:28:12.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.960+0000 7efc0bfff640 1 -- 172.21.15.26:0/3316759802 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc1c0054b0 con 0x7efc2415c650 2024-04-24T23:28:12.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.960+0000 7efc0bfff640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc1006be30 0x7efc1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:12.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.961+0000 7efc237fe640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc1006be30 0x7efc1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:12.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.961+0000 7efc0bfff640 1 -- 172.21.15.26:0/3316759802 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7efc1c08ab70 con 0x7efc2415c650 2024-04-24T23:28:12.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.961+0000 7efc237fe640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc1006be30 0x7efc1006e2f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efc14034000 tx=0x7efc1402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:12.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:12.965+0000 7efc0bfff640 1 -- 172.21.15.26:0/3316759802 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc240d21a0 con 0x7efc2415c650 2024-04-24T23:28:13.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.148+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efc24075610 con 0x7efc2415c650 2024-04-24T23:28:13.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.149+0000 7efc0bfff640 1 -- 172.21.15.26:0/3316759802 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7efc1c058ed0 con 0x7efc2415c650 2024-04-24T23:28:13.150 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-24T23:28:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc1006be30 msgr2=0x7efc1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc1006be30 0x7efc1006e2f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efc14034000 tx=0x7efc1402e000 comp rx=0 tx=0).stop 2024-04-24T23:28:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc2415c650 msgr2=0x7efc240c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc2415c650 0x7efc240c76a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00c6b0 tx=0x7efc1c00cb70 comp rx=0 tx=0).stop 2024-04-24T23:28:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 shutdown_connections 2024-04-24T23:28:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc1006be30 0x7efc1006e2f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc2415c650 0x7efc240c76a0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc2415c270 0x7efc240c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 --2- 172.21.15.26:0/3316759802 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc240b2950 0x7efc24147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 >> 172.21.15.26:0/3316759802 conn(0x7efc240d1dc0 msgr2=0x7efc240c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 shutdown_connections 2024-04-24T23:28:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.151+0000 7efc2a0d4640 1 -- 172.21.15.26:0/3316759802 wait complete. 2024-04-24T23:28:13.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:13.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.263+0000 7fefb88ca640 1 Processor -- start 2024-04-24T23:28:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.263+0000 7fefb88ca640 1 -- start start 2024-04-24T23:28:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb88ca640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb88ca640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fefb40b3ac0 0x7fefb40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb88ca640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb415d660 con 0x7fefb40d0ee0 2024-04-24T23:28:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb88ca640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fefb4156b40 con 0x7fefb40b3ac0 2024-04-24T23:28:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb88ca640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fefb40cfed0 con 0x7fefb4156760 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb259c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefa9d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fefb40b3ac0 0x7fefb40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefa9d9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fefb40b3ac0 0x7fefb40b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:36262/0 (socket says 172.21.15.26:36262) 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb259c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:47852/0 (socket says 172.21.15.26:47852) 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefa9d9b640 1 -- 172.21.15.26:0/1066908930 learned_addr learned my addr 172.21.15.26:0/1066908930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb1d9b640 1 -- 172.21.15.26:0/1066908930 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 837618595 0 0) 0x7fefac001490 con 0x7fefb40d0ee0 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb1d9b640 1 -- 172.21.15.26:0/1066908930 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefa0003520 con 0x7fefb40d0ee0 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefa9d9b640 1 -- 172.21.15.26:0/1066908930 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 msgr2=0x7fefb415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefa9d9b640 1 --2- 172.21.15.26:0/1066908930 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefa9d9b640 1 -- 172.21.15.26:0/1066908930 >> v1:172.21.15.29:6789/0 conn(0x7fefb40d0ee0 legacy=0x7fefb40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefa9d9b640 1 -- 172.21.15.26:0/1066908930 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb40d01b0 con 0x7fefb40b3ac0 2024-04-24T23:28:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.264+0000 7fefb259c640 1 --2- 172.21.15.26:0/1066908930 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.265+0000 7fefa9d9b640 1 --2- 172.21.15.26:0/1066908930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40b3ac0 0x7fefb40b3f40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009870 tx=0x7fef9c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c8196678da8d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.265+0000 7fefb1d9b640 1 -- 172.21.15.26:0/1066908930 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c037070 con 0x7fefb40b3ac0 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.265+0000 7fefb1d9b640 1 -- 172.21.15.26:0/1066908930 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9c032440 con 0x7fefb40b3ac0 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.265+0000 7fefb1d9b640 1 -- 172.21.15.26:0/1066908930 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c032810 con 0x7fefb40b3ac0 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.266+0000 7fefb88ca640 1 -- 172.21.15.26:0/1066908930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40b3ac0 msgr2=0x7fefb40b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.266+0000 7fefb88ca640 1 --2- 172.21.15.26:0/1066908930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40b3ac0 0x7fefb40b3f40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009870 tx=0x7fef9c02a250 comp rx=0 tx=0).stop 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.266+0000 7fefb88ca640 1 -- 172.21.15.26:0/1066908930 shutdown_connections 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.266+0000 7fefb88ca640 1 --2- 172.21.15.26:0/1066908930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40b3ac0 0x7fefb40b3f40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.266+0000 7fefb88ca640 1 --2- 172.21.15.26:0/1066908930 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.266+0000 7fefb88ca640 1 -- 172.21.15.26:0/1066908930 >> 172.21.15.26:0/1066908930 conn(0x7fefb406b120 msgr2=0x7fefb406b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.266+0000 7fefb88ca640 1 -- 172.21.15.26:0/1066908930 shutdown_connections 2024-04-24T23:28:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.266+0000 7fefb88ca640 1 -- 172.21.15.26:0/1066908930 wait complete. 2024-04-24T23:28:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.267+0000 7fefb88ca640 1 Processor -- start 2024-04-24T23:28:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.267+0000 7fefb88ca640 1 -- start start 2024-04-24T23:28:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb88ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb40b3ac0 0x7fefb40cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb88ca640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40d0ee0 0x7fefb40c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb88ca640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb40ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb88ca640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fefb40d0a50 con 0x7fefb40b3ac0 2024-04-24T23:28:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb88ca640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fefb40d0770 con 0x7fefb40d0ee0 2024-04-24T23:28:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb88ca640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fefb40cc7e0 con 0x7fefb4156760 2024-04-24T23:28:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb40b3ac0 0x7fefb40cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb40b3ac0 0x7fefb40cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44278/0 (socket says 172.21.15.26:44278) 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb259c640 1 -- 172.21.15.26:0/106950775 learned_addr learned my addr 172.21.15.26:0/106950775 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefb2d9d640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb40ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.268+0000 7fefa9d9b640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40d0ee0 0x7fefb40c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefb259c640 1 -- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 msgr2=0x7fefb40ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefb259c640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb40ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefb259c640 1 -- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40d0ee0 msgr2=0x7fefb40c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefb259c640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40d0ee0 0x7fefb40c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefb259c640 1 -- 172.21.15.26:0/106950775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb4158000 con 0x7fefb40b3ac0 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefb2d9d640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb40ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefa9d9b640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40d0ee0 0x7fefb40c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefb259c640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb40b3ac0 0x7fefb40cbde0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e960 tx=0x7fefa400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefab7fe640 1 -- 172.21.15.26:0/106950775 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4017070 con 0x7fefb40b3ac0 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefab7fe640 1 -- 172.21.15.26:0/106950775 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa400ca00 con 0x7fefb40b3ac0 2024-04-24T23:28:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefab7fe640 1 -- 172.21.15.26:0/106950775 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa400cdd0 con 0x7fefb40b3ac0 2024-04-24T23:28:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb4158340 con 0x7fefb40b3ac0 2024-04-24T23:28:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.269+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb4158b80 con 0x7fefb40b3ac0 2024-04-24T23:28:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.270+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb40694c0 con 0x7fefb40b3ac0 2024-04-24T23:28:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.271+0000 7fefab7fe640 1 -- 172.21.15.26:0/106950775 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefa401a070 con 0x7fefb40b3ac0 2024-04-24T23:28:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.271+0000 7fefab7fe640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef9406be30 0x7fef9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.271+0000 7fefab7fe640 1 -- 172.21.15.26:0/106950775 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fefa408ace0 con 0x7fefb40b3ac0 2024-04-24T23:28:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.271+0000 7fefa9d9b640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef9406be30 0x7fef9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.272+0000 7fefa9d9b640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef9406be30 0x7fef9406e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fef9c034000 tx=0x7fef9c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.275+0000 7fefab7fe640 1 -- 172.21.15.26:0/106950775 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fefb4158340 con 0x7fefb40b3ac0 2024-04-24T23:28:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.498+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefb40b2e00 con 0x7fefb40b3ac0 2024-04-24T23:28:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.499+0000 7fefab7fe640 1 -- 172.21.15.26:0/106950775 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7fefa4059720 con 0x7fefb40b3ac0 2024-04-24T23:28:13.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:13.500 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:13.500 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef9406be30 msgr2=0x7fef9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef9406be30 0x7fef9406e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fef9c034000 tx=0x7fef9c02e000 comp rx=0 tx=0).stop 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb40b3ac0 msgr2=0x7fefb40cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb40b3ac0 0x7fefb40cbde0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e960 tx=0x7fefa400ee20 comp rx=0 tx=0).stop 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 shutdown_connections 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fef9406be30 0x7fef9406e2f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fefb4156760 0x7fefb40ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fefb40d0ee0 0x7fefb40c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 --2- 172.21.15.26:0/106950775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb40b3ac0 0x7fefb40cbde0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 >> 172.21.15.26:0/106950775 conn(0x7fefb406b120 msgr2=0x7fefb415cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:13.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 shutdown_connections 2024-04-24T23:28:13.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.501+0000 7fefb88ca640 1 -- 172.21.15.26:0/106950775 wait complete. 2024-04-24T23:28:13.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:28:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.613+0000 7fae790a7640 1 Processor -- start 2024-04-24T23:28:13.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.613+0000 7fae790a7640 1 -- start start 2024-04-24T23:28:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.614+0000 7fae790a7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fae7415e570 0x7fae740d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.614+0000 7fae790a7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae740d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.614+0000 7fae790a7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae740d32f0 con 0x7fae740d22e0 2024-04-24T23:28:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.614+0000 7fae790a7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fae740d1ab0 con 0x7fae7415e570 2024-04-24T23:28:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.614+0000 7fae790a7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fae740d2ae0 con 0x7fae740b3130 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.614+0000 7fae7359e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae740d22e0 0x7fae740d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42880/0 (socket says 172.21.15.26:42880) 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.614+0000 7fae7359e640 1 -- 172.21.15.26:0/3703263980 learned_addr learned my addr 172.21.15.26:0/3703263980 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.614+0000 7fae72d9d640 1 --2- 172.21.15.26:0/3703263980 >> v2:172.21.15.29:3300/0 conn(0x7fae7415e570 0x7fae740d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.614+0000 7fae7259c640 1 --2- 172.21.15.26:0/3703263980 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae740d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.615+0000 7fae71d9b640 1 -- 172.21.15.26:0/3703263980 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1998897907 0 0) 0x7fae64001490 con 0x7fae740d22e0 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.615+0000 7fae71d9b640 1 -- 172.21.15.26:0/3703263980 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae5c003520 con 0x7fae740d22e0 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.615+0000 7fae72d9d640 1 -- 172.21.15.26:0/3703263980 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 msgr2=0x7fae740d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.615+0000 7fae72d9d640 1 --2- 172.21.15.26:0/3703263980 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae740d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.615+0000 7fae72d9d640 1 -- 172.21.15.26:0/3703263980 >> v1:172.21.15.26:6789/0 conn(0x7fae740d22e0 legacy=0x7fae740d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.615+0000 7fae72d9d640 1 -- 172.21.15.26:0/3703263980 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae740d0270 con 0x7fae7415e570 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.615+0000 7fae7259c640 1 --2- 172.21.15.26:0/3703263980 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae740d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.615+0000 7fae72d9d640 1 --2- 172.21.15.26:0/3703263980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 0x7fae740d4160 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097a0 tx=0x7fae6c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8423e3fe900d7627 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae71d9b640 1 -- 172.21.15.26:0/3703263980 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c037070 con 0x7fae7415e570 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae71d9b640 1 -- 172.21.15.26:0/3703263980 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6c032440 con 0x7fae7415e570 2024-04-24T23:28:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae71d9b640 1 -- 172.21.15.26:0/3703263980 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0327f0 con 0x7fae7415e570 2024-04-24T23:28:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae790a7640 1 -- 172.21.15.26:0/3703263980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 msgr2=0x7fae740d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae790a7640 1 --2- 172.21.15.26:0/3703263980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 0x7fae740d4160 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097a0 tx=0x7fae6c02a180 comp rx=0 tx=0).stop 2024-04-24T23:28:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae790a7640 1 -- 172.21.15.26:0/3703263980 shutdown_connections 2024-04-24T23:28:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae790a7640 1 --2- 172.21.15.26:0/3703263980 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae740d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae790a7640 1 --2- 172.21.15.26:0/3703263980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 0x7fae740d4160 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae790a7640 1 -- 172.21.15.26:0/3703263980 >> 172.21.15.26:0/3703263980 conn(0x7fae7408dbe0 msgr2=0x7fae7409d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae790a7640 1 -- 172.21.15.26:0/3703263980 shutdown_connections 2024-04-24T23:28:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.616+0000 7fae790a7640 1 -- 172.21.15.26:0/3703263980 wait complete. 2024-04-24T23:28:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.618+0000 7fae790a7640 1 Processor -- start 2024-04-24T23:28:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.618+0000 7fae790a7640 1 -- start start 2024-04-24T23:28:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.618+0000 7fae790a7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae74155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.618+0000 7fae790a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740d22e0 0x7fae741526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae790a7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 0x7fae740a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae790a7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae740cfa70 con 0x7fae740d22e0 2024-04-24T23:28:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae790a7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fae740d3010 con 0x7fae7415e570 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae790a7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fae740cfd50 con 0x7fae740b3130 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740d22e0 0x7fae741526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740d22e0 0x7fae741526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44284/0 (socket says 172.21.15.26:44284) 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7259c640 1 -- 172.21.15.26:0/3252031384 learned_addr learned my addr 172.21.15.26:0/3252031384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae72d9d640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae74155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7359e640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 0x7fae740a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7259c640 1 -- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 msgr2=0x7fae74155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7259c640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae74155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7259c640 1 -- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 msgr2=0x7fae740a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7259c640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 0x7fae740a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7259c640 1 -- 172.21.15.26:0/3252031384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae740b7e40 con 0x7fae740d22e0 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae72d9d640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae74155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7359e640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 0x7fae740a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:13.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae7259c640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740d22e0 0x7fae741526e0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fae6800ec10 tx=0x7fae6800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3252031384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68017070 con 0x7fae740d22e0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3252031384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6800cc30 con 0x7fae740d22e0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae740b8120 con 0x7fae740d22e0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3252031384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68013470 con 0x7fae740d22e0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.619+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae740b86e0 con 0x7fae740d22e0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.620+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae740694c0 con 0x7fae740d22e0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.621+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3252031384 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae6801a070 con 0x7fae740d22e0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.621+0000 7fae5b7fe640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.621+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3252031384 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fae6808b120 con 0x7fae740d22e0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.621+0000 7fae72d9d640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.622+0000 7fae72d9d640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fae6c034000 tx=0x7fae6c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.626+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3252031384 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fae68059ad0 con 0x7fae740d22e0 2024-04-24T23:28:13.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.811+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae740d14b0 con 0x7fae740d22e0 2024-04-24T23:28:13.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.812+0000 7fae5b7fe640 1 -- 172.21.15.26:0/3252031384 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16548 (secure 0 0 0) 0x7fae680593d0 con 0x7fae740d22e0 2024-04-24T23:28:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:13.813 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:27:24.935708+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:21.600058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:22.278230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:28:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 msgr2=0x7fae6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fae6c034000 tx=0x7fae6c02e000 comp rx=0 tx=0).stop 2024-04-24T23:28:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740d22e0 msgr2=0x7fae741526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740d22e0 0x7fae741526e0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fae6800ec10 tx=0x7fae6800c520 comp rx=0 tx=0).stop 2024-04-24T23:28:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 shutdown_connections 2024-04-24T23:28:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fae6006be60 0x7fae6006e320 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae7415e570 0x7fae740a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae740d22e0 0x7fae741526e0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 --2- 172.21.15.26:0/3252031384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fae740b3130 0x7fae74155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 >> 172.21.15.26:0/3252031384 conn(0x7fae7408dbe0 msgr2=0x7fae740d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 shutdown_connections 2024-04-24T23:28:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.814+0000 7fae790a7640 1 -- 172.21.15.26:0/3252031384 wait complete. 2024-04-24T23:28:13.827 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:28:13.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.927+0000 7fb2cacb3640 1 Processor -- start 2024-04-24T23:28:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.927+0000 7fb2cacb3640 1 -- start start 2024-04-24T23:28:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2cacb3640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 0x7fb2c415f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2cacb3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2c406f4f0 0x7fb2c40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2cacb3640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c415f750 con 0x7fb2c40addb0 2024-04-24T23:28:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2cacb3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c40ae220 con 0x7fb2c406f4f0 2024-04-24T23:28:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2cacb3640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c40b0220 con 0x7fb2c406f110 2024-04-24T23:28:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2c3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2c406f4f0 0x7fb2c40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2c3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2c406f4f0 0x7fb2c40ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59864/0 (socket says 172.21.15.26:59864) 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2c3fff640 1 -- 172.21.15.26:0/4003258697 learned_addr learned my addr 172.21.15.26:0/4003258697 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2c8a4f640 1 --2- 172.21.15.26:0/4003258697 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 0x7fb2c415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2c3fff640 1 -- 172.21.15.26:0/4003258697 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 msgr2=0x7fb2c415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2c3fff640 1 --2- 172.21.15.26:0/4003258697 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 0x7fb2c415f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2c3fff640 1 -- 172.21.15.26:0/4003258697 >> v1:172.21.15.29:6789/0 conn(0x7fb2c40addb0 legacy=0x7fb2c4154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.928+0000 7fb2c3fff640 1 -- 172.21.15.26:0/4003258697 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c40af180 con 0x7fb2c406f4f0 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.929+0000 7fb2c3fff640 1 --2- 172.21.15.26:0/4003258697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c406f4f0 0x7fb2c40ad870 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009870 tx=0x7fb2b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3851dcc62bdbc7b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.929+0000 7fb2c37fe640 1 -- 172.21.15.26:0/4003258697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0037070 con 0x7fb2c406f4f0 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.929+0000 7fb2c37fe640 1 -- 172.21.15.26:0/4003258697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b0032440 con 0x7fb2c406f4f0 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.929+0000 7fb2c37fe640 1 -- 172.21.15.26:0/4003258697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0032810 con 0x7fb2c406f4f0 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.930+0000 7fb2cacb3640 1 -- 172.21.15.26:0/4003258697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c406f4f0 msgr2=0x7fb2c40ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.930+0000 7fb2cacb3640 1 --2- 172.21.15.26:0/4003258697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c406f4f0 0x7fb2c40ad870 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009870 tx=0x7fb2b002a270 comp rx=0 tx=0).stop 2024-04-24T23:28:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.930+0000 7fb2cacb3640 1 -- 172.21.15.26:0/4003258697 shutdown_connections 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.930+0000 7fb2cacb3640 1 --2- 172.21.15.26:0/4003258697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c406f4f0 0x7fb2c40ad870 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.930+0000 7fb2cacb3640 1 --2- 172.21.15.26:0/4003258697 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 0x7fb2c415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.930+0000 7fb2cacb3640 1 -- 172.21.15.26:0/4003258697 >> 172.21.15.26:0/4003258697 conn(0x7fb2c40729a0 msgr2=0x7fb2c406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.930+0000 7fb2cacb3640 1 -- 172.21.15.26:0/4003258697 shutdown_connections 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.930+0000 7fb2cacb3640 1 -- 172.21.15.26:0/4003258697 wait complete. 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.931+0000 7fb2cacb3640 1 Processor -- start 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.931+0000 7fb2cacb3640 1 -- start start 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2cacb3640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 0x7fb2c40c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2cacb3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c406f4f0 0x7fb2c40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2cacb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40addb0 0x7fb2c40cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2cacb3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c40afa20 con 0x7fb2c40addb0 2024-04-24T23:28:13.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2cacb3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c40af740 con 0x7fb2c406f4f0 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2cacb3640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c40afd00 con 0x7fb2c406f110 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c9250640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40addb0 0x7fb2c40cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c9250640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40addb0 0x7fb2c40cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59878/0 (socket says 172.21.15.26:59878) 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c9250640 1 -- 172.21.15.26:0/1190420222 learned_addr learned my addr 172.21.15.26:0/1190420222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c8a4f640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 0x7fb2c40c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c3fff640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c406f4f0 0x7fb2c40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c9250640 1 -- 172.21.15.26:0/1190420222 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 msgr2=0x7fb2c40c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c9250640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 0x7fb2c40c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c9250640 1 -- 172.21.15.26:0/1190420222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c406f4f0 msgr2=0x7fb2c40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c9250640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c406f4f0 0x7fb2c40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.932+0000 7fb2c9250640 1 -- 172.21.15.26:0/1190420222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c40c8700 con 0x7fb2c40addb0 2024-04-24T23:28:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.933+0000 7fb2c9250640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40addb0 0x7fb2c40cb040 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c6b0 tx=0x7fb2bc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.933+0000 7fb2c17fa640 1 -- 172.21.15.26:0/1190420222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc003b70 con 0x7fb2c40addb0 2024-04-24T23:28:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.933+0000 7fb2c17fa640 1 -- 172.21.15.26:0/1190420222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc003d30 con 0x7fb2c40addb0 2024-04-24T23:28:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.933+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c40c89e0 con 0x7fb2c40addb0 2024-04-24T23:28:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.933+0000 7fb2c17fa640 1 -- 172.21.15.26:0/1190420222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc005310 con 0x7fb2c40addb0 2024-04-24T23:28:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.933+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c40c8fa0 con 0x7fb2c40addb0 2024-04-24T23:28:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.934+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c40c9ce0 con 0x7fb2c40addb0 2024-04-24T23:28:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.934+0000 7fb2c17fa640 1 -- 172.21.15.26:0/1190420222 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2bc0054b0 con 0x7fb2c40addb0 2024-04-24T23:28:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.935+0000 7fb2c17fa640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb2a806be30 0x7fb2a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.935+0000 7fb2c17fa640 1 -- 172.21.15.26:0/1190420222 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fb2bc08aae0 con 0x7fb2c40addb0 2024-04-24T23:28:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.935+0000 7fb2c8a4f640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb2a806be30 0x7fb2a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.935+0000 7fb2c8a4f640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb2a806be30 0x7fb2a806e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006fd0 tx=0x7fb2b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:13.939+0000 7fb2c17fa640 1 -- 172.21.15.26:0/1190420222 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb2c415f750 con 0x7fb2c40addb0 2024-04-24T23:28:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.163+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2c40694c0 con 0x7fb2c40addb0 2024-04-24T23:28:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.163+0000 7fb2c17fa640 1 -- 172.21.15.26:0/1190420222 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7fb2bc058ed0 con 0x7fb2c40addb0 2024-04-24T23:28:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:14.164 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:14.165 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb2a806be30 msgr2=0x7fb2a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb2a806be30 0x7fb2a806e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006fd0 tx=0x7fb2b8008040 comp rx=0 tx=0).stop 2024-04-24T23:28:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40addb0 msgr2=0x7fb2c40cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40addb0 0x7fb2c40cb040 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c6b0 tx=0x7fb2bc00cb70 comp rx=0 tx=0).stop 2024-04-24T23:28:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 shutdown_connections 2024-04-24T23:28:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb2a806be30 0x7fb2a806e2f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40addb0 0x7fb2c40cb040 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2c406f4f0 0x7fb2c40cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 --2- 172.21.15.26:0/1190420222 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb2c406f110 0x7fb2c40c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 >> 172.21.15.26:0/1190420222 conn(0x7fb2c40729a0 msgr2=0x7fb2c40b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 shutdown_connections 2024-04-24T23:28:14.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.165+0000 7fb2cacb3640 1 -- 172.21.15.26:0/1190420222 wait complete. 2024-04-24T23:28:14.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:14.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.277+0000 7f3627645640 1 Processor -- start 2024-04-24T23:28:14.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.277+0000 7f3627645640 1 -- start start 2024-04-24T23:28:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f3627645640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f3620145910 0x7f362006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f3627645640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36200cd660 0x7f36200cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f3627645640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36200cfbc0 con 0x7f362006d350 2024-04-24T23:28:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f3627645640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3620145cf0 con 0x7f36200cd660 2024-04-24T23:28:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f3627645640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f362006d7f0 con 0x7f3620145910 2024-04-24T23:28:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f3625be2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36200cd660 0x7f36200cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f3625be2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36200cd660 0x7f36200cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59884/0 (socket says 172.21.15.26:59884) 2024-04-24T23:28:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f3625be2640 1 -- 172.21.15.26:0/2760732798 learned_addr learned my addr 172.21.15.26:0/2760732798 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f3624be0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f362006d350 0x7f362015d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52886/0 (socket says 172.21.15.26:52886) 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.278+0000 7f36253e1640 1 --2- 172.21.15.26:0/2760732798 >> v2:172.21.15.29:3300/0 conn(0x7f3620145910 0x7f362006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.279+0000 7f3625be2640 1 -- 172.21.15.26:0/2760732798 >> v2:172.21.15.29:3300/0 conn(0x7f3620145910 msgr2=0x7f362006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.279+0000 7f3625be2640 1 --2- 172.21.15.26:0/2760732798 >> v2:172.21.15.29:3300/0 conn(0x7f3620145910 0x7f362006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.279+0000 7f3625be2640 1 -- 172.21.15.26:0/2760732798 >> v1:172.21.15.26:6789/0 conn(0x7f362006d350 legacy=0x7f362015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.279+0000 7f3625be2640 1 -- 172.21.15.26:0/2760732798 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362006dad0 con 0x7f36200cd660 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.279+0000 7f3617fff640 1 -- 172.21.15.26:0/2760732798 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 540354180 0 0) 0x7f36200cfbc0 con 0x7f362006d350 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.279+0000 7f3625be2640 1 --2- 172.21.15.26:0/2760732798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36200cd660 0x7f36200cdaf0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f361c009e50 tx=0x7f361c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf609c1361ce7d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.279+0000 7f3617fff640 1 -- 172.21.15.26:0/2760732798 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c036070 con 0x7f36200cd660 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.279+0000 7f3617fff640 1 -- 172.21.15.26:0/2760732798 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c031600 con 0x7f36200cd660 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.280+0000 7f3617fff640 1 -- 172.21.15.26:0/2760732798 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c0319d0 con 0x7f36200cd660 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.280+0000 7f3627645640 1 -- 172.21.15.26:0/2760732798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36200cd660 msgr2=0x7f36200cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.280+0000 7f3627645640 1 --2- 172.21.15.26:0/2760732798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36200cd660 0x7f36200cdaf0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f361c009e50 tx=0x7f361c02a6d0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.280+0000 7f3627645640 1 -- 172.21.15.26:0/2760732798 shutdown_connections 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.280+0000 7f3627645640 1 --2- 172.21.15.26:0/2760732798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36200cd660 0x7f36200cdaf0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.280+0000 7f3627645640 1 --2- 172.21.15.26:0/2760732798 >> v2:172.21.15.29:3300/0 conn(0x7f3620145910 0x7f362006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.280+0000 7f3627645640 1 -- 172.21.15.26:0/2760732798 >> 172.21.15.26:0/2760732798 conn(0x7f362006b300 msgr2=0x7f362006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:14.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.280+0000 7f3627645640 1 -- 172.21.15.26:0/2760732798 shutdown_connections 2024-04-24T23:28:14.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.280+0000 7f3627645640 1 -- 172.21.15.26:0/2760732798 wait complete. 2024-04-24T23:28:14.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.281+0000 7f3627645640 1 Processor -- start 2024-04-24T23:28:14.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.281+0000 7f3627645640 1 -- start start 2024-04-24T23:28:14.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f3627645640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362006d350 0x7f36201f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f3627645640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36200cd660 0x7f3620156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f3627645640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3620145910 0x7f36201f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f3627645640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f362006e370 con 0x7f362006d350 2024-04-24T23:28:14.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f3627645640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f362006e090 con 0x7f36200cd660 2024-04-24T23:28:14.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f3627645640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f362015d770 con 0x7f3620145910 2024-04-24T23:28:14.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f36253e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362006d350 0x7f36201f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f36253e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362006d350 0x7f36201f7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59894/0 (socket says 172.21.15.26:59894) 2024-04-24T23:28:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f36253e1640 1 -- 172.21.15.26:0/2132678369 learned_addr learned my addr 172.21.15.26:0/2132678369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f3624be0640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36200cd660 0x7f3620156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f3625be2640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3620145910 0x7f36201f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f36253e1640 1 -- 172.21.15.26:0/2132678369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3620145910 msgr2=0x7f36201f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f36253e1640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3620145910 0x7f36201f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f36253e1640 1 -- 172.21.15.26:0/2132678369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36200cd660 msgr2=0x7f3620156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.282+0000 7f36253e1640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36200cd660 0x7f3620156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.283+0000 7f36253e1640 1 -- 172.21.15.26:0/2132678369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36201f4890 con 0x7f362006d350 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.283+0000 7f3625be2640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3620145910 0x7f36201f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.283+0000 7f36253e1640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362006d350 0x7f36201f7b40 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f361800e8a0 tx=0x7f361800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.283+0000 7f3615ffb640 1 -- 172.21.15.26:0/2132678369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618017070 con 0x7f362006d350 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.283+0000 7f3615ffb640 1 -- 172.21.15.26:0/2132678369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361800c940 con 0x7f362006d350 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.283+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3620156fc0 con 0x7f362006d350 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.283+0000 7f3615ffb640 1 -- 172.21.15.26:0/2132678369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361800cd30 con 0x7f362006d350 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.283+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36201f45b0 con 0x7f362006d350 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.284+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36201f7770 con 0x7f362006d350 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.285+0000 7f3615ffb640 1 -- 172.21.15.26:0/2132678369 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f361801a070 con 0x7f362006d350 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.285+0000 7f3615ffb640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f360406be30 0x7f360406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.285+0000 7f3615ffb640 1 -- 172.21.15.26:0/2132678369 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f361808ad10 con 0x7f362006d350 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.285+0000 7f3624be0640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f360406be30 0x7f360406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.285+0000 7f3624be0640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f360406be30 0x7f360406e2f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f36200af490 tx=0x7f3610007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.289+0000 7f3615ffb640 1 -- 172.21.15.26:0/2132678369 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f36180596c0 con 0x7f362006d350 2024-04-24T23:28:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.512+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36200694c0 con 0x7f362006d350 2024-04-24T23:28:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.512+0000 7f3615ffb640 1 -- 172.21.15.26:0/2132678369 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5234 (secure 0 0 0) 0x7f3618058fc0 con 0x7f362006d350 2024-04-24T23:28:14.513 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:14.513 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":113,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:23.297118+0000","modified":"2024-04-24T23:27:56.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8730,"mds_1":8712,"mds_2":7444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7444":{"gid":7444,"name":"b","rank":2,"incarnation":107,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6834/2632309869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2632309869},{"type":"v1","addr":"172.21.15.29:6834","nonce":2632309869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8712":{"gid":8712,"name":"c","rank":1,"incarnation":105,"state":"up:active","state_seq":11,"addr":"172.21.15.26:6835/3062763802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3062763802},{"type":"v1","addr":"172.21.15.26:6835","nonce":3062763802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8730":{"gid":8730,"name":"d","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6835/2225348762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6833","nonce":2225348762},{"type":"v1","addr":"172.21.15.29:6835","nonce":2225348762}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8730,"qdb_cluster":[8730,8712,7444]},"id":7}]} 2024-04-24T23:28:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 113 2024-04-24T23:28:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f360406be30 msgr2=0x7f360406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f360406be30 0x7f360406e2f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f36200af490 tx=0x7f3610007b50 comp rx=0 tx=0).stop 2024-04-24T23:28:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362006d350 msgr2=0x7f36201f7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362006d350 0x7f36201f7b40 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f361800e8a0 tx=0x7f361800ed60 comp rx=0 tx=0).stop 2024-04-24T23:28:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 shutdown_connections 2024-04-24T23:28:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f360406be30 0x7f360406e2f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f3620145910 0x7f36201f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f36200cd660 0x7f3620156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 --2- 172.21.15.26:0/2132678369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362006d350 0x7f36201f7b40 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 >> 172.21.15.26:0/2132678369 conn(0x7f362006b300 msgr2=0x7f3620144910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:14.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 shutdown_connections 2024-04-24T23:28:14.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.514+0000 7f3627645640 1 -- 172.21.15.26:0/2132678369 wait complete. 2024-04-24T23:28:14.527 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:28:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.627+0000 7ff8c4909640 1 Processor -- start 2024-04-24T23:28:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.627+0000 7ff8c4909640 1 -- start start 2024-04-24T23:28:14.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.628+0000 7ff8c4909640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8c0145910 0x7ff8c006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.628+0000 7ff8c4909640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff8c006d350 0x7ff8c015d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.628+0000 7ff8c4909640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c015d570 con 0x7ff8c00cd660 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.628+0000 7ff8c4909640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c0145cf0 con 0x7ff8c0145910 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.628+0000 7ff8c4909640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c00cfbc0 con 0x7ff8c006d350 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.628+0000 7ff8be59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8c0145910 0x7ff8c006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.628+0000 7ff8bed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff8c00cd660 0x7ff8c015d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52892/0 (socket says 172.21.15.26:52892) 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.628+0000 7ff8be59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8c0145910 0x7ff8c006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59902/0 (socket says 172.21.15.26:59902) 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.628+0000 7ff8bed9d640 1 -- 172.21.15.26:0/3607773856 learned_addr learned my addr 172.21.15.26:0/3607773856 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.629+0000 7ff8bdd9b640 1 --2- 172.21.15.26:0/3607773856 >> v2:172.21.15.29:3300/0 conn(0x7ff8c006d350 0x7ff8c015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.629+0000 7ff8be59c640 1 -- 172.21.15.26:0/3607773856 >> v2:172.21.15.29:3300/0 conn(0x7ff8c006d350 msgr2=0x7ff8c015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.629+0000 7ff8be59c640 1 --2- 172.21.15.26:0/3607773856 >> v2:172.21.15.29:3300/0 conn(0x7ff8c006d350 0x7ff8c015d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.629+0000 7ff8be59c640 1 -- 172.21.15.26:0/3607773856 >> v1:172.21.15.26:6789/0 conn(0x7ff8c00cd660 legacy=0x7ff8c015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.629+0000 7ff8be59c640 1 -- 172.21.15.26:0/3607773856 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c006d7f0 con 0x7ff8c0145910 2024-04-24T23:28:14.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.629+0000 7ff8bd59a640 1 -- 172.21.15.26:0/3607773856 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3352490716 0 0) 0x7ff8b0001490 con 0x7ff8c00cd660 2024-04-24T23:28:14.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.629+0000 7ff8bdd9b640 1 --2- 172.21.15.26:0/3607773856 >> v2:172.21.15.29:3300/0 conn(0x7ff8c006d350 0x7ff8c015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.629+0000 7ff8be59c640 1 --2- 172.21.15.26:0/3607773856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 0x7ff8c006e890 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80097a0 tx=0x7ff8b802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80fe6d9d7bd1302 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.629+0000 7ff8bd59a640 1 -- 172.21.15.26:0/3607773856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8037070 con 0x7ff8c0145910 2024-04-24T23:28:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8bd59a640 1 -- 172.21.15.26:0/3607773856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b802ae50 con 0x7ff8c0145910 2024-04-24T23:28:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8bd59a640 1 -- 172.21.15.26:0/3607773856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8032660 con 0x7ff8c0145910 2024-04-24T23:28:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8c4909640 1 -- 172.21.15.26:0/3607773856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 msgr2=0x7ff8c006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8c4909640 1 --2- 172.21.15.26:0/3607773856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 0x7ff8c006e890 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80097a0 tx=0x7ff8b802a140 comp rx=0 tx=0).stop 2024-04-24T23:28:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8c4909640 1 -- 172.21.15.26:0/3607773856 shutdown_connections 2024-04-24T23:28:14.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8c4909640 1 --2- 172.21.15.26:0/3607773856 >> v2:172.21.15.29:3300/0 conn(0x7ff8c006d350 0x7ff8c015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8c4909640 1 --2- 172.21.15.26:0/3607773856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 0x7ff8c006e890 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8c4909640 1 -- 172.21.15.26:0/3607773856 >> 172.21.15.26:0/3607773856 conn(0x7ff8c006b300 msgr2=0x7ff8c006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:14.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8c4909640 1 -- 172.21.15.26:0/3607773856 shutdown_connections 2024-04-24T23:28:14.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.630+0000 7ff8c4909640 1 -- 172.21.15.26:0/3607773856 wait complete. 2024-04-24T23:28:14.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.631+0000 7ff8c4909640 1 Processor -- start 2024-04-24T23:28:14.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8c4909640 1 -- start start 2024-04-24T23:28:14.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8c4909640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8c006d350 0x7ff8c00aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8c4909640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8c00cd660 0x7ff8c0158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8c4909640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 0x7ff8c013d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8c4909640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c006e090 con 0x7ff8c0145910 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8c4909640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c006ddb0 con 0x7ff8c006d350 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8c4909640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c006e370 con 0x7ff8c00cd660 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8bed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 0x7ff8c013d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8bed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 0x7ff8c013d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59918/0 (socket says 172.21.15.26:59918) 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8bed9d640 1 -- 172.21.15.26:0/1294678054 learned_addr learned my addr 172.21.15.26:0/1294678054 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8be59c640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8c006d350 0x7ff8c00aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.632+0000 7ff8bdd9b640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8c00cd660 0x7ff8c0158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8bed9d640 1 -- 172.21.15.26:0/1294678054 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8c00cd660 msgr2=0x7ff8c0158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8bed9d640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8c00cd660 0x7ff8c0158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8bed9d640 1 -- 172.21.15.26:0/1294678054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8c006d350 msgr2=0x7ff8c00aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8bed9d640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8c006d350 0x7ff8c00aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8bed9d640 1 -- 172.21.15.26:0/1294678054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c00adc30 con 0x7ff8c0145910 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8bed9d640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 0x7ff8c013d480 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c7f0 tx=0x7ff8b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1294678054 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0018070 con 0x7ff8c0145910 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1294678054 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b000a5e0 con 0x7ff8c0145910 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c00adf10 con 0x7ff8c0145910 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1294678054 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000aa10 con 0x7ff8c0145910 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.633+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c00ae4d0 con 0x7ff8c0145910 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.634+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c00694c0 con 0x7ff8c0145910 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.634+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1294678054 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8b001b070 con 0x7ff8c0145910 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.635+0000 7ff8b6ffd640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff8a006be60 0x7ff8a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.635+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1294678054 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7ff8b008a3a0 con 0x7ff8c0145910 2024-04-24T23:28:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.635+0000 7ff8be59c640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff8a006be60 0x7ff8a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.635+0000 7ff8be59c640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff8a006be60 0x7ff8a006e320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8034000 tx=0x7ff8b802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.639+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1294678054 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8c015d570 con 0x7ff8c0145910 2024-04-24T23:28:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.822+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8c015f690 con 0x7ff8c0145910 2024-04-24T23:28:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.823+0000 7ff8b6ffd640 1 -- 172.21.15.26:0/1294678054 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16548 (secure 0 0 0) 0x7ff8b0058700 con 0x7ff8c0145910 2024-04-24T23:28:14.824 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:14.824 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:27:24.935708+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:21.600058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:22.278230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:28:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff8a006be60 msgr2=0x7ff8a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff8a006be60 0x7ff8a006e320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8034000 tx=0x7ff8b802e000 comp rx=0 tx=0).stop 2024-04-24T23:28:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 msgr2=0x7ff8c013d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 0x7ff8c013d480 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c7f0 tx=0x7ff8b000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 shutdown_connections 2024-04-24T23:28:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff8a006be60 0x7ff8a006e320 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c0145910 0x7ff8c013d480 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff8c00cd660 0x7ff8c0158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 --2- 172.21.15.26:0/1294678054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8c006d350 0x7ff8c00aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 >> 172.21.15.26:0/1294678054 conn(0x7ff8c006b300 msgr2=0x7ff8c006b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 shutdown_connections 2024-04-24T23:28:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.825+0000 7ff8c4909640 1 -- 172.21.15.26:0/1294678054 wait complete. 2024-04-24T23:28:14.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:28:14.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.938+0000 7f91c697e640 1 Processor -- start 2024-04-24T23:28:14.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.938+0000 7f91c697e640 1 -- start start 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.939+0000 7f91c697e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c015e570 0x7f91c00d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.939+0000 7f91c697e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c00d32f0 con 0x7f91c00b3130 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.939+0000 7f91c697e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c00d1600 con 0x7f91c00d22e0 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.939+0000 7f91c697e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f91c00d1ab0 con 0x7f91c015e570 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.939+0000 7f91bf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f91c00b3130 0x7f91c00d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52908/0 (socket says 172.21.15.26:52908) 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.939+0000 7f91bf7fe640 1 -- 172.21.15.26:0/261817914 learned_addr learned my addr 172.21.15.26:0/261817914 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.939+0000 7f91bffff640 1 --2- 172.21.15.26:0/261817914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c015e570 0x7f91c00d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.940+0000 7f91beffd640 1 -- 172.21.15.26:0/261817914 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 524795624 0 0) 0x7f91b0001110 con 0x7f91c00b3130 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.940+0000 7f91beffd640 1 -- 172.21.15.26:0/261817914 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91a8003520 con 0x7f91c00b3130 2024-04-24T23:28:14.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.940+0000 7f91beffd640 1 -- 172.21.15.26:0/261817914 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2991394741 0 0) 0x7f91c00d1600 con 0x7f91c00d22e0 2024-04-24T23:28:14.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.940+0000 7f91bffff640 1 -- 172.21.15.26:0/261817914 >> v1:172.21.15.29:6789/0 conn(0x7f91c00d22e0 legacy=0x7f91c00d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.940+0000 7f91bffff640 1 -- 172.21.15.26:0/261817914 >> v1:172.21.15.26:6789/0 conn(0x7f91c00b3130 legacy=0x7f91c00d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.940+0000 7f91bffff640 1 -- 172.21.15.26:0/261817914 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c00d0270 con 0x7f91c015e570 2024-04-24T23:28:14.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91bffff640 1 --2- 172.21.15.26:0/261817914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c015e570 0x7f91c00d4160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91b40097d0 tx=0x7f91b402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43002d88755b4e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:14.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91beffd640 1 -- 172.21.15.26:0/261817914 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4037070 con 0x7f91c015e570 2024-04-24T23:28:14.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91beffd640 1 -- 172.21.15.26:0/261817914 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b4032440 con 0x7f91c015e570 2024-04-24T23:28:14.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91beffd640 1 -- 172.21.15.26:0/261817914 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40327f0 con 0x7f91c015e570 2024-04-24T23:28:14.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91c697e640 1 -- 172.21.15.26:0/261817914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c015e570 msgr2=0x7f91c00d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91c697e640 1 --2- 172.21.15.26:0/261817914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c015e570 0x7f91c00d4160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91b40097d0 tx=0x7f91b402a210 comp rx=0 tx=0).stop 2024-04-24T23:28:14.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91c697e640 1 -- 172.21.15.26:0/261817914 shutdown_connections 2024-04-24T23:28:14.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91c697e640 1 --2- 172.21.15.26:0/261817914 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c015e570 0x7f91c00d4160 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91c697e640 1 -- 172.21.15.26:0/261817914 >> 172.21.15.26:0/261817914 conn(0x7f91c008dbe0 msgr2=0x7f91c009d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:14.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.941+0000 7f91c697e640 1 -- 172.21.15.26:0/261817914 shutdown_connections 2024-04-24T23:28:14.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.942+0000 7f91c697e640 1 -- 172.21.15.26:0/261817914 wait complete. 2024-04-24T23:28:14.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.943+0000 7f91c697e640 1 Processor -- start 2024-04-24T23:28:14.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.943+0000 7f91c697e640 1 -- start start 2024-04-24T23:28:14.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.943+0000 7f91c697e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c00b3130 0x7f91c00ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c697e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91c00d22e0 0x7f91c00b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c697e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91c015e570 0x7f91c00b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c697e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f91c00d3010 con 0x7f91c015e570 2024-04-24T23:28:14.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c697e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f91c00d2ae0 con 0x7f91c00d22e0 2024-04-24T23:28:14.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c697e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f91c00cfa70 con 0x7f91c00b3130 2024-04-24T23:28:14.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c4f1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91c015e570 0x7f91c00b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c4f1b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91c015e570 0x7f91c00b72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59922/0 (socket says 172.21.15.26:59922) 2024-04-24T23:28:14.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c4f1b640 1 -- 172.21.15.26:0/1181531501 learned_addr learned my addr 172.21.15.26:0/1181531501 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:14.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91bffff640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c00b3130 0x7f91c00ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91bf7fe640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91c00d22e0 0x7f91c00b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c4f1b640 1 -- 172.21.15.26:0/1181531501 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c00b3130 msgr2=0x7f91c00ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c4f1b640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c00b3130 0x7f91c00ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c4f1b640 1 -- 172.21.15.26:0/1181531501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91c00d22e0 msgr2=0x7f91c00b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:14.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c4f1b640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91c00d22e0 0x7f91c00b6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:14.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c4f1b640 1 -- 172.21.15.26:0/1181531501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c00cac20 con 0x7f91c015e570 2024-04-24T23:28:14.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91bffff640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c00b3130 0x7f91c00ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:14.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c4f1b640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91c015e570 0x7f91c00b72f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f91b800c760 tx=0x7f91b800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:14.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91bcff9640 1 -- 172.21.15.26:0/1181531501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8003da0 con 0x7f91c015e570 2024-04-24T23:28:14.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91bcff9640 1 -- 172.21.15.26:0/1181531501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b8005070 con 0x7f91c015e570 2024-04-24T23:28:14.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91bcff9640 1 -- 172.21.15.26:0/1181531501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8005460 con 0x7f91c015e570 2024-04-24T23:28:14.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c00b9c10 con 0x7f91c015e570 2024-04-24T23:28:14.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.944+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c00cc2f0 con 0x7f91c015e570 2024-04-24T23:28:14.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.945+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c00694c0 con 0x7f91c015e570 2024-04-24T23:28:14.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.946+0000 7f91bcff9640 1 -- 172.21.15.26:0/1181531501 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91b8005600 con 0x7f91c015e570 2024-04-24T23:28:14.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.946+0000 7f91bcff9640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:14.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.946+0000 7f91bcff9640 1 -- 172.21.15.26:0/1181531501 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f91b808adb0 con 0x7f91c015e570 2024-04-24T23:28:14.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.946+0000 7f91bffff640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:14.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.947+0000 7f91bffff640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f91b4034000 tx=0x7f91b402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:14.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:14.951+0000 7f91bcff9640 1 -- 172.21.15.26:0/1181531501 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91b8059760 con 0x7f91c015e570 2024-04-24T23:28:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.176+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f91c006ca60 con 0x7f91c015e570 2024-04-24T23:28:15.177 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-24T23:28:15.177+0000 7f26a0f44640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:28:15.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.178+0000 7f91bcff9640 1 -- 172.21.15.26:0/1181531501 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) ==== 161+0+0 (secure 0 0 0) 0x7f91b8059060 con 0x7f91c015e570 2024-04-24T23:28:15.179 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 msgr2=0x7f91ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f91b4034000 tx=0x7f91b402e000 comp rx=0 tx=0).stop 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91c015e570 msgr2=0x7f91c00b72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91c015e570 0x7f91c00b72f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f91b800c760 tx=0x7f91b800cc20 comp rx=0 tx=0).stop 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 shutdown_connections 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91c015e570 0x7f91c00b72f0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91c00d22e0 0x7f91c00b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 --2- 172.21.15.26:0/1181531501 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f91c00b3130 0x7f91c00ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 >> 172.21.15.26:0/1181531501 conn(0x7f91c008dbe0 msgr2=0x7f91c006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 shutdown_connections 2024-04-24T23:28:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.181+0000 7f91c697e640 1 -- 172.21.15.26:0/1181531501 wait complete. 2024-04-24T23:28:15.187 INFO:tasks.ceph.mds.c.smithi026.stderr: -1379> 2024-04-24T23:27:52.782+0000 7fb852d4e640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1) n(v2 rc2024-04-24T23:27:35.581019+0000 4=0+4) (inest lock) (ifile excl) caps={7552=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7552 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x5572738d1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.581019+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-04-24T23:27:35.581019+0000 4=0+4)","error_str":""},"return_code":-61} 2024-04-24T23:28:15.187 INFO:tasks.ceph.mds.c.smithi026.stderr: -1241> 2024-04-24T23:27:52.783+0000 7fb85955b640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1) n(v1 rc2024-04-24T23:27:35.581019+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7552=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55727295b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.350023+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.349023+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-04-24T23:27:35.581019+0000 3=0+3)","error_str":""},"return_code":-61} 2024-04-24T23:28:15.188 INFO:tasks.ceph.mds.b.smithi029.stderr: -1277> 2024-04-24T23:27:52.782+0000 7f02efa72640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-04-24T23:27:35.350023+0000 1=0+1) n(v0 rc2024-04-24T23:27:35.350023+0000 2=0+2) (inest lock) (ifile excl) caps={7552=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7552 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55fcf25d3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.350023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.350023+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.350023+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-04-24T23:27:35.350023+0000 2=0+2)","error_str":""},"return_code":-61} 2024-04-24T23:28:15.188 INFO:tasks.ceph.mds.b.smithi029.stderr: -1257> 2024-04-24T23:27:52.783+0000 7f02efa72640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={7552=pAsLsXsFsx/AsLsXsFsx@2},l=7552 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55fcf25d3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-04-24T23:28:15.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:28:15.196 INFO:tasks.ceph.mds.d.smithi029.stderr: -2748> 2024-04-24T23:27:52.782+0000 7f42b0d69640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-04-24T23:27:35.347023+0000 1=0+1) n(v3 rc2024-04-24T23:27:35.581019+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7552=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7552 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x558526e71b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.347023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.581019+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.347023+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-04-24T23:27:35.581019+0000 6=0+6)","error_str":""},"return_code":-61} 2024-04-24T23:28:15.196 INFO:tasks.ceph.mds.d.smithi029.stderr: -2630> 2024-04-24T23:27:52.783+0000 7f42b7576640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-04-24T23:27:35.348023+0000 1=0+1) n(v4 rc2024-04-24T23:27:35.581019+0000 5=0+5) (inest sync->mix g=1,2) caps={7552=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x558526e72100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T23:27:35.348023+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T23:27:35.581019+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-04-24T23:27:35.348023+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-04-24T23:27:35.581019+0000 5=0+5)","error_str":""},"return_code":-61} 2024-04-24T23:28:15.213 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-24T23:28:15.220 INFO:tasks.ceph.mds.b.smithi029.stdout:starting mds.b at 2024-04-24T23:28:15.228 INFO:tasks.ceph.mds.d.smithi029.stdout:starting mds.d at 2024-04-24T23:28:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.296+0000 7f44fd3e7640 1 Processor -- start 2024-04-24T23:28:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.297+0000 7f44fd3e7640 1 -- start start 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.297+0000 7f44fd3e7640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f44f815e370 0x7f44f80d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.297+0000 7f44fd3e7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f80d2120 con 0x7f44f815e750 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.297+0000 7f44fd3e7640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f80b3740 con 0x7f44f80d4c10 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.297+0000 7f44fd3e7640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f44f80d1510 con 0x7f44f815e370 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.297+0000 7f44f67fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f44f815e750 0x7f44f80b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52912/0 (socket says 172.21.15.26:52912) 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f67fc640 1 -- 172.21.15.26:0/3639851752 learned_addr learned my addr 172.21.15.26:0/3639851752 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 304625345 0 0) 0x7f44f80d2120 con 0x7f44f815e750 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44e0003520 con 0x7f44f815e750 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f6ffd640 1 --2- 172.21.15.26:0/3639851752 >> v2:172.21.15.29:3300/0 conn(0x7f44f815e370 0x7f44f80d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2210720611 0 0) 0x7f44e0003520 con 0x7f44f815e750 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 >> v2:172.21.15.29:3300/0 conn(0x7f44f815e370 msgr2=0x7f44f80d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:15.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 --2- 172.21.15.26:0/3639851752 >> v2:172.21.15.29:3300/0 conn(0x7f44f815e370 0x7f44f80d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 >> v1:172.21.15.29:6789/0 conn(0x7f44f80d4c10 legacy=0x7f44f80b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f80b3a20 con 0x7f44f815e750 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f44ec001670 con 0x7f44f815e750 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 >> v1:172.21.15.26:6789/0 conn(0x7f44f815e750 legacy=0x7f44f80b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e00039b0 0x7f44e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e0004800 0x7f44e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44e00051c0 0x7f44e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f44e000c030 con 0x7f44e0004800 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44e000c1e0 con 0x7f44e00039b0 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f44e000c3e0 con 0x7f44e00051c0 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f67fc640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e0004800 0x7f44e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:15.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f6ffd640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e00039b0 0x7f44e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.298+0000 7f44f77fe640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44e00051c0 0x7f44e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f67fc640 1 -- 172.21.15.26:0/3639851752 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44e00051c0 msgr2=0x7f44e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f67fc640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44e00051c0 0x7f44e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f67fc640 1 -- 172.21.15.26:0/3639851752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e00039b0 msgr2=0x7f44e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f67fc640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e00039b0 0x7f44e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f67fc640 1 -- 172.21.15.26:0/3639851752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e0003520 con 0x7f44e0004800 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f77fe640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44e00051c0 0x7f44e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f77fe640 1 -- 172.21.15.26:0/3639851752 reap_dead start 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f67fc640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e0004800 0x7f44e0004c80 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c240 tx=0x7f44e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494b261824389cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec007cf0 con 0x7f44e0004800 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ec0058d0 con 0x7f44e0004800 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.299+0000 7f44f5ffb640 1 -- 172.21.15.26:0/3639851752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec005d20 con 0x7f44e0004800 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.300+0000 7f44fd3e7640 1 -- 172.21.15.26:0/3639851752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e0004800 msgr2=0x7f44e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.300+0000 7f44fd3e7640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e0004800 0x7f44e0004c80 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c240 tx=0x7f44e0005830 comp rx=0 tx=0).stop 2024-04-24T23:28:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.300+0000 7f44fd3e7640 1 -- 172.21.15.26:0/3639851752 shutdown_connections 2024-04-24T23:28:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.300+0000 7f44fd3e7640 1 --2- 172.21.15.26:0/3639851752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e0004800 0x7f44e0004c80 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.300+0000 7f44fd3e7640 1 -- 172.21.15.26:0/3639851752 >> 172.21.15.26:0/3639851752 conn(0x7f44f80d40f0 msgr2=0x7f44f80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.300+0000 7f44fd3e7640 1 -- 172.21.15.26:0/3639851752 shutdown_connections 2024-04-24T23:28:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.300+0000 7f44fd3e7640 1 -- 172.21.15.26:0/3639851752 wait complete. 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.301+0000 7f44fd3e7640 1 Processor -- start 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.301+0000 7f44fd3e7640 1 -- start start 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44fd3e7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e0004800 0x7f44f8161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44fd3e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f8149c80 0x7f44f8146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44fd3e7640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44f81f3470 0x7f44f815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44fd3e7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f44f80d2520 con 0x7f44f8149c80 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44fd3e7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44f80d1ed0 con 0x7f44e0004800 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44fd3e7640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f44f80d26a0 con 0x7f44f81f3470 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44f67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f8149c80 0x7f44f8146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44f67fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f8149c80 0x7f44f8146f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59964/0 (socket says 172.21.15.26:59964) 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44f67fc640 1 -- 172.21.15.26:0/1026575236 learned_addr learned my addr 172.21.15.26:0/1026575236 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:15.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44f77fe640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44f81f3470 0x7f44f815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44f6ffd640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e0004800 0x7f44f8161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44f67fc640 1 -- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44f81f3470 msgr2=0x7f44f815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44f67fc640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44f81f3470 0x7f44f815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44f67fc640 1 -- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e0004800 msgr2=0x7f44f8161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.302+0000 7f44f67fc640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e0004800 0x7f44f8161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.303+0000 7f44f67fc640 1 -- 172.21.15.26:0/1026575236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f8161d90 con 0x7f44f8149c80 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.303+0000 7f44f77fe640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44f81f3470 0x7f44f815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.303+0000 7f44f6ffd640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e0004800 0x7f44f8161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.303+0000 7f44f67fc640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f8149c80 0x7f44f8146f90 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c240 tx=0x7f44f80b3bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.303+0000 7f44df7fe640 1 -- 172.21.15.26:0/1026575236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec007cf0 con 0x7f44f8149c80 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.303+0000 7f44df7fe640 1 -- 172.21.15.26:0/1026575236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ec041070 con 0x7f44f8149c80 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.303+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f80cc470 con 0x7f44f8149c80 2024-04-24T23:28:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.303+0000 7f44df7fe640 1 -- 172.21.15.26:0/1026575236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec001c30 con 0x7f44f8149c80 2024-04-24T23:28:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.303+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f80cc9f0 con 0x7f44f8149c80 2024-04-24T23:28:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.304+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f8075610 con 0x7f44f8149c80 2024-04-24T23:28:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.304+0000 7f44df7fe640 1 -- 172.21.15.26:0/1026575236 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44ec03c480 con 0x7f44f8149c80 2024-04-24T23:28:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.305+0000 7f44df7fe640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f44e406be30 0x7f44e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.305+0000 7f44df7fe640 1 -- 172.21.15.26:0/1026575236 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5967+0+0 (secure 0 0 0) 0x7f44ec0ab0d0 con 0x7f44f8149c80 2024-04-24T23:28:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.305+0000 7f44f6ffd640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f44e406be30 0x7f44e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.305+0000 7f44f6ffd640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f44e406be30 0x7f44e406e2f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f44f80cae40 tx=0x7f44e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:15.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.309+0000 7f44df7fe640 1 -- 172.21.15.26:0/1026575236 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f44ec079a10 con 0x7f44f8149c80 2024-04-24T23:28:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:15.533+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f44f80d06f0 con 0x7f44f8149c80 2024-04-24T23:28:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.182+0000 7f44df7fe640 1 -- 172.21.15.26:0/1026575236 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) ==== 104+0+0 (secure 0 0 0) 0x7f44ec079310 con 0x7f44f8149c80 2024-04-24T23:28:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f44e406be30 msgr2=0x7f44e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f44e406be30 0x7f44e406e2f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f44f80cae40 tx=0x7f44e8008040 comp rx=0 tx=0).stop 2024-04-24T23:28:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f8149c80 msgr2=0x7f44f8146f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f8149c80 0x7f44f8146f90 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c240 tx=0x7f44f80b3bf0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 shutdown_connections 2024-04-24T23:28:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f44e406be30 0x7f44e406e2f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f44f81f3470 0x7f44f815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f8149c80 0x7f44f8146f90 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 --2- 172.21.15.26:0/1026575236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44e0004800 0x7f44f8161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 >> 172.21.15.26:0/1026575236 conn(0x7f44f80d40f0 msgr2=0x7f44f80b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:16.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 shutdown_connections 2024-04-24T23:28:16.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.185+0000 7f44fd3e7640 1 -- 172.21.15.26:0/1026575236 wait complete. 2024-04-24T23:28:16.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:28:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.347+0000 7f8c06990640 1 Processor -- start 2024-04-24T23:28:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.347+0000 7f8c06990640 1 -- start start 2024-04-24T23:28:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.348+0000 7f8c06990640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c0015e2a0 0x7f8c000cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.348+0000 7f8c06990640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8c000d4510 0x7f8c000b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.348+0000 7f8c06990640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c000cf170 con 0x7f8c000b3f00 2024-04-24T23:28:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.348+0000 7f8c06990640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8c000d1c80 con 0x7f8c0015e2a0 2024-04-24T23:28:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.348+0000 7f8c06990640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f8c000b49c0 con 0x7f8c000d4510 2024-04-24T23:28:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.348+0000 7f8bfffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c0015e2a0 0x7f8c000cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.348+0000 7f8bfffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8c0015e2a0 0x7f8c000cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59966/0 (socket says 172.21.15.26:59966) 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.348+0000 7f8bfffff640 1 -- 172.21.15.26:0/569283193 learned_addr learned my addr 172.21.15.26:0/569283193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.349+0000 7f8c04f2d640 1 --2- 172.21.15.26:0/569283193 >> v2:172.21.15.29:3300/0 conn(0x7f8c000d4510 0x7f8c000b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.349+0000 7f8bfffff640 1 -- 172.21.15.26:0/569283193 >> v2:172.21.15.29:3300/0 conn(0x7f8c000d4510 msgr2=0x7f8c000b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.349+0000 7f8bfffff640 1 --2- 172.21.15.26:0/569283193 >> v2:172.21.15.29:3300/0 conn(0x7f8c000d4510 0x7f8c000b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.349+0000 7f8bfffff640 1 -- 172.21.15.26:0/569283193 >> v1:172.21.15.29:6789/0 conn(0x7f8c000b3f00 legacy=0x7f8c000d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.349+0000 7f8bfffff640 1 -- 172.21.15.26:0/569283193 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c000b4ca0 con 0x7f8c0015e2a0 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.349+0000 7f8bfffff640 1 --2- 172.21.15.26:0/569283193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c0015e2a0 0x7f8c000cec30 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f8c000cb100 tx=0x7f8bf002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a916646755ffd8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8bfeffd640 1 -- 172.21.15.26:0/569283193 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0037070 con 0x7f8c0015e2a0 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8bfeffd640 1 -- 172.21.15.26:0/569283193 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf0032440 con 0x7f8c0015e2a0 2024-04-24T23:28:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8bfeffd640 1 -- 172.21.15.26:0/569283193 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0032810 con 0x7f8c0015e2a0 2024-04-24T23:28:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8c06990640 1 -- 172.21.15.26:0/569283193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c0015e2a0 msgr2=0x7f8c000cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8c06990640 1 --2- 172.21.15.26:0/569283193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c0015e2a0 0x7f8c000cec30 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f8c000cb100 tx=0x7f8bf002a200 comp rx=0 tx=0).stop 2024-04-24T23:28:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8c06990640 1 -- 172.21.15.26:0/569283193 shutdown_connections 2024-04-24T23:28:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8c06990640 1 --2- 172.21.15.26:0/569283193 >> v2:172.21.15.29:3300/0 conn(0x7f8c000d4510 0x7f8c000b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8c06990640 1 --2- 172.21.15.26:0/569283193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c0015e2a0 0x7f8c000cec30 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8c06990640 1 -- 172.21.15.26:0/569283193 >> 172.21.15.26:0/569283193 conn(0x7f8c0008db40 msgr2=0x7f8c0009d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8c06990640 1 -- 172.21.15.26:0/569283193 shutdown_connections 2024-04-24T23:28:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.350+0000 7f8c06990640 1 -- 172.21.15.26:0/569283193 wait complete. 2024-04-24T23:28:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.351+0000 7f8c06990640 1 Processor -- start 2024-04-24T23:28:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8c06990640 1 -- start start 2024-04-24T23:28:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8c06990640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c000b3f00 0x7f8c000cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8c06990640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c000d4510 0x7f8c0015e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8c06990640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c0015e2a0 0x7f8c0015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8c06990640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c000d0bd0 con 0x7f8c000d4510 2024-04-24T23:28:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8c06990640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0015e680 con 0x7f8c0015e2a0 2024-04-24T23:28:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8c06990640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f8c000d0d50 con 0x7f8c000b3f00 2024-04-24T23:28:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8bff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c000d4510 0x7f8c0015e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8bff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c000d4510 0x7f8c0015e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59970/0 (socket says 172.21.15.26:59970) 2024-04-24T23:28:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.352+0000 7f8bff7fe640 1 -- 172.21.15.26:0/1956205268 learned_addr learned my addr 172.21.15.26:0/1956205268 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8c04f2d640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c0015e2a0 0x7f8c0015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bfffff640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c000b3f00 0x7f8c000cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bff7fe640 1 -- 172.21.15.26:0/1956205268 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c000b3f00 msgr2=0x7f8c000cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bff7fe640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c000b3f00 0x7f8c000cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bff7fe640 1 -- 172.21.15.26:0/1956205268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c0015e2a0 msgr2=0x7f8c0015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bff7fe640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c0015e2a0 0x7f8c0015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bff7fe640 1 -- 172.21.15.26:0/1956205268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c000c8850 con 0x7f8c000d4510 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bff7fe640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c000d4510 0x7f8c0015e800 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400c430 tx=0x7f8bf400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bfcff9640 1 -- 172.21.15.26:0/1956205268 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4017070 con 0x7f8c000d4510 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bfcff9640 1 -- 172.21.15.26:0/1956205268 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf40052d0 con 0x7f8c000d4510 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c000c8b30 con 0x7f8c000d4510 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8bfcff9640 1 -- 172.21.15.26:0/1956205268 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4003850 con 0x7f8c000d4510 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.353+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c000c90f0 con 0x7f8c000d4510 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.354+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c000694c0 con 0x7f8c000d4510 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.355+0000 7f8bfcff9640 1 -- 172.21.15.26:0/1956205268 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bf401a070 con 0x7f8c000d4510 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.355+0000 7f8bfcff9640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bec06be30 0x7f8bec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.355+0000 7f8bfcff9640 1 -- 172.21.15.26:0/1956205268 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5967+0+0 (secure 0 0 0) 0x7f8bf408a1f0 con 0x7f8c000d4510 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.355+0000 7f8bfffff640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bec06be30 0x7f8bec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.356+0000 7f8bfffff640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bec06be30 0x7f8bec06e2f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0034000 tx=0x7f8bf002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.360+0000 7f8bfcff9640 1 -- 172.21.15.26:0/1956205268 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8bf40584e0 con 0x7f8c000d4510 2024-04-24T23:28:16.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:16.543+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8c000b3b10 con 0x7f8c000d4510 2024-04-24T23:28:17.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.184+0000 7f8bfcff9640 1 -- 172.21.15.26:0/1956205268 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7f8bf405c7e0 con 0x7f8c000d4510 2024-04-24T23:28:17.185 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:28:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bec06be30 msgr2=0x7f8bec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bec06be30 0x7f8bec06e2f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0034000 tx=0x7f8bf002e000 comp rx=0 tx=0).stop 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c000d4510 msgr2=0x7f8c0015e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c000d4510 0x7f8c0015e800 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400c430 tx=0x7f8bf400c8f0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 shutdown_connections 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f8bec06be30 0x7f8bec06e2f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c0015e2a0 0x7f8c0015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c000d4510 0x7f8c0015e800 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 --2- 172.21.15.26:0/1956205268 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f8c000b3f00 0x7f8c000cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 >> 172.21.15.26:0/1956205268 conn(0x7f8c0008db40 msgr2=0x7f8c0006d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 shutdown_connections 2024-04-24T23:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.188+0000 7f8c06990640 1 -- 172.21.15.26:0/1956205268 wait complete. 2024-04-24T23:28:17.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:28:17.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.309+0000 7fa93ab9e640 1 Processor -- start 2024-04-24T23:28:17.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.309+0000 7fa93ab9e640 1 -- start start 2024-04-24T23:28:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa93ab9e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fa93415df50 0x7fa9340d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa93ab9e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9340cee60 con 0x7fa9340ce9e0 2024-04-24T23:28:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa93ab9e640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9340cf140 con 0x7fa9340d0f40 2024-04-24T23:28:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa93ab9e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fa9340cff00 con 0x7fa93415df50 2024-04-24T23:28:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa93939b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa9340ce9e0 0x7fa9340d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52920/0 (socket says 172.21.15.26:52920) 2024-04-24T23:28:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa93939b640 1 -- 172.21.15.26:0/1656758423 learned_addr learned my addr 172.21.15.26:0/1656758423 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464471130 0 0) 0x7fa9340cee60 con 0x7fa9340ce9e0 2024-04-24T23:28:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa91c003520 con 0x7fa9340ce9e0 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa939b9c640 1 --2- 172.21.15.26:0/1656758423 >> v2:172.21.15.29:3300/0 conn(0x7fa93415df50 0x7fa9340d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1472464715 0 0) 0x7fa91c003520 con 0x7fa9340ce9e0 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 >> v2:172.21.15.29:3300/0 conn(0x7fa93415df50 msgr2=0x7fa9340d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 --2- 172.21.15.26:0/1656758423 >> v2:172.21.15.29:3300/0 conn(0x7fa93415df50 0x7fa9340d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 >> v1:172.21.15.29:6789/0 conn(0x7fa9340d0f40 legacy=0x7fa934168000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9340cf420 con 0x7fa9340ce9e0 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7fa928001670 con 0x7fa9340ce9e0 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 >> v1:172.21.15.26:6789/0 conn(0x7fa9340ce9e0 legacy=0x7fa9340d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa91c0039b0 0x7fa91c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa91c004800 0x7fa91c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0051c0 0x7fa91c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9340cff00 con 0x7fa91c0039b0 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c00c110 con 0x7fa91c004800 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa91c00c310 con 0x7fa91c0051c0 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.310+0000 7fa939b9c640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa91c0039b0 0x7fa91c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa93939b640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa91c004800 0x7fa91c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa93a39d640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0051c0 0x7fa91c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa939b9c640 1 -- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0051c0 msgr2=0x7fa91c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa939b9c640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0051c0 0x7fa91c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa939b9c640 1 -- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa91c004800 msgr2=0x7fa91c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa939b9c640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa91c004800 0x7fa91c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa939b9c640 1 -- 172.21.15.26:0/1656758423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c003520 con 0x7fa91c0039b0 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa93939b640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa91c004800 0x7fa91c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:17.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa93a39d640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0051c0 0x7fa91c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa93a39d640 1 -- 172.21.15.26:0/1656758423 reap_dead start 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.311+0000 7fa939b9c640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa91c0039b0 0x7fa91c003dd0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00ba30 tx=0x7fa91c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeaded2bfea6d316 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c030070 con 0x7fa91c0039b0 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92c02ade0 con 0x7fa91c0039b0 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa938b9a640 1 -- 172.21.15.26:0/1656758423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00b600 con 0x7fa91c0039b0 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa93ab9e640 1 -- 172.21.15.26:0/1656758423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa91c0039b0 msgr2=0x7fa91c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa93ab9e640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa91c0039b0 0x7fa91c003dd0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00ba30 tx=0x7fa91c005830 comp rx=0 tx=0).stop 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa93ab9e640 1 -- 172.21.15.26:0/1656758423 shutdown_connections 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa93ab9e640 1 --2- 172.21.15.26:0/1656758423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa91c0039b0 0x7fa91c003dd0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa93ab9e640 1 -- 172.21.15.26:0/1656758423 >> 172.21.15.26:0/1656758423 conn(0x7fa93409d1d0 msgr2=0x7fa9340d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa93ab9e640 1 -- 172.21.15.26:0/1656758423 shutdown_connections 2024-04-24T23:28:17.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.312+0000 7fa93ab9e640 1 -- 172.21.15.26:0/1656758423 wait complete. 2024-04-24T23:28:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93ab9e640 1 Processor -- start 2024-04-24T23:28:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93ab9e640 1 -- start start 2024-04-24T23:28:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93ab9e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0039b0 0x7fa9340b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93ab9e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9340b6a80 0x7fa9340b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93ab9e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa934146be0 0x7fa934147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93ab9e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa93415e330 con 0x7fa934146be0 2024-04-24T23:28:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93ab9e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa9340cfa70 con 0x7fa9340b6a80 2024-04-24T23:28:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93ab9e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa93415e4b0 con 0x7fa91c0039b0 2024-04-24T23:28:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93a39d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa934146be0 0x7fa934147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93a39d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa934146be0 0x7fa934147060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59992/0 (socket says 172.21.15.26:59992) 2024-04-24T23:28:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.314+0000 7fa93a39d640 1 -- 172.21.15.26:0/3074714783 learned_addr learned my addr 172.21.15.26:0/3074714783 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa93939b640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9340b6a80 0x7fa9340b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa939b9c640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0039b0 0x7fa9340b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa93a39d640 1 -- 172.21.15.26:0/3074714783 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0039b0 msgr2=0x7fa9340b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa93a39d640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0039b0 0x7fa9340b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa93a39d640 1 -- 172.21.15.26:0/3074714783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9340b6a80 msgr2=0x7fa9340b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa93a39d640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9340b6a80 0x7fa9340b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa93a39d640 1 -- 172.21.15.26:0/3074714783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9341f8a80 con 0x7fa934146be0 2024-04-24T23:28:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa93a39d640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa934146be0 0x7fa934147060 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fa9340b3680 tx=0x7fa92400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa9327fc640 1 -- 172.21.15.26:0/3074714783 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924003970 con 0x7fa934146be0 2024-04-24T23:28:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa9327fc640 1 -- 172.21.15.26:0/3074714783 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa924018070 con 0x7fa934146be0 2024-04-24T23:28:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9340b8a70 con 0x7fa934146be0 2024-04-24T23:28:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa9327fc640 1 -- 172.21.15.26:0/3074714783 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924003d50 con 0x7fa934146be0 2024-04-24T23:28:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.315+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9340b8d10 con 0x7fa934146be0 2024-04-24T23:28:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.316+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa934075610 con 0x7fa934146be0 2024-04-24T23:28:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.317+0000 7fa9327fc640 1 -- 172.21.15.26:0/3074714783 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9240050d0 con 0x7fa934146be0 2024-04-24T23:28:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.317+0000 7fa9327fc640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa92006be60 0x7fa92006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.317+0000 7fa9327fc640 1 -- 172.21.15.26:0/3074714783 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5529+0+0 (secure 0 0 0) 0x7fa92408a5f0 con 0x7fa934146be0 2024-04-24T23:28:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.317+0000 7fa939b9c640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa92006be60 0x7fa92006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.317+0000 7fa939b9c640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa92006be60 0x7fa92006e320 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00ba00 tx=0x7fa92c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.321+0000 7fa9327fc640 1 -- 172.21.15.26:0/3074714783 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa924022020 con 0x7fa934146be0 2024-04-24T23:28:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:17.512+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa9340d2210 con 0x7fa934146be0 2024-04-24T23:28:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.189+0000 7fa9327fc640 1 -- 172.21.15.26:0/3074714783 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7fa924058a00 con 0x7fa934146be0 2024-04-24T23:28:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-24T23:28:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa92006be60 msgr2=0x7fa92006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa92006be60 0x7fa92006e320 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00ba00 tx=0x7fa92c030000 comp rx=0 tx=0).stop 2024-04-24T23:28:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa934146be0 msgr2=0x7fa934147060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa934146be0 0x7fa934147060 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fa9340b3680 tx=0x7fa92400eb90 comp rx=0 tx=0).stop 2024-04-24T23:28:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 shutdown_connections 2024-04-24T23:28:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa92006be60 0x7fa92006e320 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa934146be0 0x7fa934147060 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9340b6a80 0x7fa9340b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 --2- 172.21.15.26:0/3074714783 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa91c0039b0 0x7fa9340b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.193+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 >> 172.21.15.26:0/3074714783 conn(0x7fa93409d1d0 msgr2=0x7fa93406c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:18.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.194+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 shutdown_connections 2024-04-24T23:28:18.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.194+0000 7fa93ab9e640 1 -- 172.21.15.26:0/3074714783 wait complete. 2024-04-24T23:28:18.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T23:28:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.320+0000 7fdd22a79640 1 Processor -- start 2024-04-24T23:28:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.320+0000 7fdd22a79640 1 -- start start 2024-04-24T23:28:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.321+0000 7fdd22a79640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd1c071ef0 0x7fdd1c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.321+0000 7fdd22a79640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c06b120 0x7fdd1c06b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.321+0000 7fdd22a79640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd1c072520 con 0x7fdd1c145b30 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.321+0000 7fdd22a79640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fdd1c0cec20 con 0x7fdd1c071ef0 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.321+0000 7fdd22a79640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1c072800 con 0x7fdd1c06b120 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.321+0000 7fdd1bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd1c071ef0 0x7fdd1c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.321+0000 7fdd1bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd1c071ef0 0x7fdd1c0add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60004/0 (socket says 172.21.15.26:60004) 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.321+0000 7fdd1bfff640 1 -- 172.21.15.26:0/77141396 learned_addr learned my addr 172.21.15.26:0/77141396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.322+0000 7fdd21016640 1 --2- 172.21.15.26:0/77141396 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c06b120 0x7fdd1c06b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.322+0000 7fdd1bfff640 1 -- 172.21.15.26:0/77141396 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c06b120 msgr2=0x7fdd1c06b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.322+0000 7fdd1bfff640 1 --2- 172.21.15.26:0/77141396 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c06b120 0x7fdd1c06b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.322+0000 7fdd1bfff640 1 -- 172.21.15.26:0/77141396 >> v1:172.21.15.29:6789/0 conn(0x7fdd1c145b30 legacy=0x7fdd1c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.322+0000 7fdd1bfff640 1 -- 172.21.15.26:0/77141396 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c072ae0 con 0x7fdd1c071ef0 2024-04-24T23:28:18.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.322+0000 7fdd1bfff640 1 --2- 172.21.15.26:0/77141396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c071ef0 0x7fdd1c0add40 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009870 tx=0x7fdd1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d811a99efeead315 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd1b7fe640 1 -- 172.21.15.26:0/77141396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10037070 con 0x7fdd1c071ef0 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd1b7fe640 1 -- 172.21.15.26:0/77141396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd10032440 con 0x7fdd1c071ef0 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd1b7fe640 1 -- 172.21.15.26:0/77141396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10032810 con 0x7fdd1c071ef0 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd22a79640 1 -- 172.21.15.26:0/77141396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c071ef0 msgr2=0x7fdd1c0add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd22a79640 1 --2- 172.21.15.26:0/77141396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c071ef0 0x7fdd1c0add40 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009870 tx=0x7fdd1002a270 comp rx=0 tx=0).stop 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd22a79640 1 -- 172.21.15.26:0/77141396 shutdown_connections 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd22a79640 1 --2- 172.21.15.26:0/77141396 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c06b120 0x7fdd1c06b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd22a79640 1 --2- 172.21.15.26:0/77141396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c071ef0 0x7fdd1c0add40 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd22a79640 1 -- 172.21.15.26:0/77141396 >> 172.21.15.26:0/77141396 conn(0x7fdd1c0d0d20 msgr2=0x7fdd1c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd22a79640 1 -- 172.21.15.26:0/77141396 shutdown_connections 2024-04-24T23:28:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.323+0000 7fdd22a79640 1 -- 172.21.15.26:0/77141396 wait complete. 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.325+0000 7fdd22a79640 1 Processor -- start 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.325+0000 7fdd22a79640 1 -- start start 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.325+0000 7fdd22a79640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd1c06b120 0x7fdd1c15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.325+0000 7fdd22a79640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c071ef0 0x7fdd1c15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.325+0000 7fdd22a79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c145b30 0x7fdd1c159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.325+0000 7fdd22a79640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c0cf230 con 0x7fdd1c145b30 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.325+0000 7fdd22a79640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c0cf0b0 con 0x7fdd1c06b120 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.325+0000 7fdd22a79640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1c0cf3b0 con 0x7fdd1c071ef0 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd21016640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c145b30 0x7fdd1c159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd21016640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c145b30 0x7fdd1c159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60008/0 (socket says 172.21.15.26:60008) 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd21016640 1 -- 172.21.15.26:0/1339061989 learned_addr learned my addr 172.21.15.26:0/1339061989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd1bfff640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c071ef0 0x7fdd1c15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd20815640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd1c06b120 0x7fdd1c15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd21016640 1 -- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c071ef0 msgr2=0x7fdd1c15a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd21016640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c071ef0 0x7fdd1c15a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd21016640 1 -- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd1c06b120 msgr2=0x7fdd1c15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd21016640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd1c06b120 0x7fdd1c15c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd21016640 1 -- 172.21.15.26:0/1339061989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c1f8080 con 0x7fdd1c145b30 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd20815640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd1c06b120 0x7fdd1c15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd1bfff640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c071ef0 0x7fdd1c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd21016640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c145b30 0x7fdd1c159ba0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fdd14007e40 tx=0x7fdd1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd197fa640 1 -- 172.21.15.26:0/1339061989 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14014620 con 0x7fdd1c145b30 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd197fa640 1 -- 172.21.15.26:0/1339061989 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd14019070 con 0x7fdd1c145b30 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c1f8360 con 0x7fdd1c145b30 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd197fa640 1 -- 172.21.15.26:0/1339061989 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14014a20 con 0x7fdd1c145b30 2024-04-24T23:28:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.326+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c1f8920 con 0x7fdd1c145b30 2024-04-24T23:28:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.327+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1c075610 con 0x7fdd1c145b30 2024-04-24T23:28:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.328+0000 7fdd197fa640 1 -- 172.21.15.26:0/1339061989 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd14014bc0 con 0x7fdd1c145b30 2024-04-24T23:28:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.328+0000 7fdd197fa640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdd0806be30 0x7fdd0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.328+0000 7fdd197fa640 1 -- 172.21.15.26:0/1339061989 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7fdd1408b440 con 0x7fdd1c145b30 2024-04-24T23:28:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.328+0000 7fdd20815640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdd0806be30 0x7fdd0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.329+0000 7fdd20815640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdd0806be30 0x7fdd0806e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c001660 tx=0x7fdd0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.333+0000 7fdd197fa640 1 -- 172.21.15.26:0/1339061989 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd1405a0d0 con 0x7fdd1c145b30 2024-04-24T23:28:18.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.520+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fdd1c15f9d0 con 0x7fdd1c145b30 2024-04-24T23:28:18.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.524+0000 7fdd197fa640 1 -- 172.21.15.26:0/1339061989 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fdd140599d0 con 0x7fdd1c145b30 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdd0806be30 msgr2=0x7fdd0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdd0806be30 0x7fdd0806e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c001660 tx=0x7fdd0c009040 comp rx=0 tx=0).stop 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c145b30 msgr2=0x7fdd1c159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c145b30 0x7fdd1c159ba0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fdd14007e40 tx=0x7fdd1400e4b0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 shutdown_connections 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fdd0806be30 0x7fdd0806e2f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd1c145b30 0x7fdd1c159ba0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fdd1c071ef0 0x7fdd1c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 --2- 172.21.15.26:0/1339061989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd1c06b120 0x7fdd1c15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 >> 172.21.15.26:0/1339061989 conn(0x7fdd1c0d0d20 msgr2=0x7fdd1c143120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:18.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 shutdown_connections 2024-04-24T23:28:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.526+0000 7fdd22a79640 1 -- 172.21.15.26:0/1339061989 wait complete. 2024-04-24T23:28:18.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-04-24T23:28:18.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.638+0000 7fe672190640 1 Processor -- start 2024-04-24T23:28:18.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.638+0000 7fe672190640 1 -- start start 2024-04-24T23:28:18.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.639+0000 7fe672190640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe66c15e380 0x7fe66c0b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.639+0000 7fe672190640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c0b5440 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.639+0000 7fe672190640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c0b5700 con 0x7fe66c0b0030 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.639+0000 7fe672190640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c15e760 con 0x7fe66c15e380 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.639+0000 7fe66affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe66c0b4fd0 0x7fe66c072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52926/0 (socket says 172.21.15.26:52926) 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.639+0000 7fe66affd640 1 -- 172.21.15.26:0/2055841128 learned_addr learned my addr 172.21.15.26:0/2055841128 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.639+0000 7fe66b7fe640 1 --2- 172.21.15.26:0/2055841128 >> v2:172.21.15.29:3300/0 conn(0x7fe66c15e380 0x7fe66c0b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.639+0000 7fe66a7fc640 1 -- 172.21.15.26:0/2055841128 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4144729465 0 0) 0x7fe66c0b5440 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.640+0000 7fe66a7fc640 1 -- 172.21.15.26:0/2055841128 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe654003520 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.640+0000 7fe66b7fe640 1 -- 172.21.15.26:0/2055841128 >> v1:172.21.15.29:6789/0 conn(0x7fe66c0b0030 legacy=0x7fe66c0b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.640+0000 7fe66b7fe640 1 -- 172.21.15.26:0/2055841128 >> v1:172.21.15.26:6789/0 conn(0x7fe66c0b4fd0 legacy=0x7fe66c072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.640+0000 7fe66b7fe640 1 -- 172.21.15.26:0/2055841128 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c0b59e0 con 0x7fe66c15e380 2024-04-24T23:28:18.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.640+0000 7fe66b7fe640 1 --2- 172.21.15.26:0/2055841128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe66c15e380 0x7fe66c0b4530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0097a0 tx=0x7fe65c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88afb416ebf42b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe66a7fc640 1 -- 172.21.15.26:0/2055841128 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c037070 con 0x7fe66c15e380 2024-04-24T23:28:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe66a7fc640 1 -- 172.21.15.26:0/2055841128 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65c032640 con 0x7fe66c15e380 2024-04-24T23:28:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe66a7fc640 1 -- 172.21.15.26:0/2055841128 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c032a10 con 0x7fe66c15e380 2024-04-24T23:28:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe672190640 1 -- 172.21.15.26:0/2055841128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe66c15e380 msgr2=0x7fe66c0b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe672190640 1 --2- 172.21.15.26:0/2055841128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe66c15e380 0x7fe66c0b4530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0097a0 tx=0x7fe65c02a2c0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe672190640 1 -- 172.21.15.26:0/2055841128 shutdown_connections 2024-04-24T23:28:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe672190640 1 --2- 172.21.15.26:0/2055841128 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe66c15e380 0x7fe66c0b4530 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe672190640 1 -- 172.21.15.26:0/2055841128 >> 172.21.15.26:0/2055841128 conn(0x7fe66c06e2c0 msgr2=0x7fe66c06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe672190640 1 -- 172.21.15.26:0/2055841128 shutdown_connections 2024-04-24T23:28:18.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.641+0000 7fe672190640 1 -- 172.21.15.26:0/2055841128 wait complete. 2024-04-24T23:28:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.642+0000 7fe672190640 1 Processor -- start 2024-04-24T23:28:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe672190640 1 -- start start 2024-04-24T23:28:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe672190640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe66c0b0030 0x7fe66c0c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe672190640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66c0b4fd0 0x7fe66c158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe672190640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe66c15e380 0x7fe66c15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe672190640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0ae530 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe672190640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0725b0 con 0x7fe66c15e380 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe672190640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c0ae6b0 con 0x7fe66c0b0030 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe66affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66c0b4fd0 0x7fe66c158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe66affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66c0b4fd0 0x7fe66c158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60020/0 (socket says 172.21.15.26:60020) 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.643+0000 7fe66affd640 1 -- 172.21.15.26:0/2847060890 learned_addr learned my addr 172.21.15.26:0/2847060890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe66bfff640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe66c15e380 0x7fe66c15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe66b7fe640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe66c0b0030 0x7fe66c0c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe66affd640 1 -- 172.21.15.26:0/2847060890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe66c0b0030 msgr2=0x7fe66c0c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe66affd640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe66c0b0030 0x7fe66c0c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe66affd640 1 -- 172.21.15.26:0/2847060890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe66c15e380 msgr2=0x7fe66c15fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe66affd640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe66c15e380 0x7fe66c15fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe66affd640 1 -- 172.21.15.26:0/2847060890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c0c7280 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe66affd640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66c0b4fd0 0x7fe66c158cb0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe66000c500 tx=0x7fe66000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe653fff640 1 -- 172.21.15.26:0/2847060890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660017070 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe653fff640 1 -- 172.21.15.26:0/2847060890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe660005ce0 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c1591f0 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe653fff640 1 -- 172.21.15.26:0/2847060890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660007330 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.644+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c0b1b70 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.645+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c0694c0 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.646+0000 7fe653fff640 1 -- 172.21.15.26:0/2847060890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe66001a070 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.646+0000 7fe653fff640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe65806be60 0x7fe65806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.646+0000 7fe66b7fe640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe65806be60 0x7fe65806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.646+0000 7fe653fff640 1 -- 172.21.15.26:0/2847060890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7fe660089fa0 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.647+0000 7fe66b7fe640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe65806be60 0x7fe65806e320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe65c034000 tx=0x7fe65c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.650+0000 7fe653fff640 1 -- 172.21.15.26:0/2847060890 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe660058cf0 con 0x7fe66c0b4fd0 2024-04-24T23:28:18.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:18.904+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) -- 0x7fe66c06d240 con 0x7fe66c0b4fd0 2024-04-24T23:28:19.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.190+0000 7fe653fff640 1 -- 172.21.15.26:0/2847060890 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v391) ==== 165+0+0 (secure 0 0 0) 0x7fe6600585f0 con 0x7fe66c0b4fd0 2024-04-24T23:28:19.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe65806be60 msgr2=0x7fe65806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe65806be60 0x7fe65806e320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe65c034000 tx=0x7fe65c02e000 comp rx=0 tx=0).stop 2024-04-24T23:28:19.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66c0b4fd0 msgr2=0x7fe66c158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66c0b4fd0 0x7fe66c158cb0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe66000c500 tx=0x7fe66000c9c0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 shutdown_connections 2024-04-24T23:28:19.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe65806be60 0x7fe65806e320 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe66c15e380 0x7fe66c15fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66c0b4fd0 0x7fe66c158cb0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 --2- 172.21.15.26:0/2847060890 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe66c0b0030 0x7fe66c0c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 >> 172.21.15.26:0/2847060890 conn(0x7fe66c06e2c0 msgr2=0x7fe66c15d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:19.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.192+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 shutdown_connections 2024-04-24T23:28:19.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:19.193+0000 7fe672190640 1 -- 172.21.15.26:0/2847060890 wait complete. 2024-04-24T23:28:19.205 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-04-24T23:28:19.205 INFO:tasks.cephfs_test_runner: 2024-04-24T23:28:19.205 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-24T23:28:19.205 INFO:tasks.cephfs_test_runner:Ran 6 tests in 415.071s 2024-04-24T23:28:19.205 INFO:tasks.cephfs_test_runner: 2024-04-24T23:28:19.205 INFO:tasks.cephfs_test_runner:OK 2024-04-24T23:28:19.206 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-24T23:28:19.216 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-24T23:28:19.230 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-24T23:28:19.230 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:28:19.230 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:19.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:28:19.251 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-24T23:28:19.251 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:19.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056881,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T23:28:19.306 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:28:19.306 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:19.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:28:19.326 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-04-24T23:28:19.326 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:19.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T23:28:19.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.387 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:28:19.403 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:28:19.403 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:28:19.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.404 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:28:19.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.470 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:28:19.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.536 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-24T23:28:19.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.641 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-24T23:28:19.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.656 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-24T23:28:19.724 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-04-24T23:28:19.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:28:19.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.725 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-24T23:28:19.745 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T23:28:19.745 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:28:19.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.745 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:28:19.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.817 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:28:19.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.886 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-04-24T23:28:19.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.978 DEBUG:teuthology.orchestra.run.smithi029:> ip netns list 2024-04-24T23:28:19.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:19.994 DEBUG:teuthology.orchestra.run.smithi029:> sudo ip link delete ceph-brx 2024-04-24T23:28:20.064 INFO:teuthology.orchestra.run.smithi029.stderr:Cannot find device "ceph-brx" 2024-04-24T23:28:20.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:28:20.065 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-24T23:28:20.077 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T23:28:20.077 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:28:20.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.181+0000 7f864ff42640 1 Processor -- start 2024-04-24T23:28:20.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.181+0000 7f864ff42640 1 -- start start 2024-04-24T23:28:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864ff42640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f8648145910 0x7f864806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864ff42640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86480cd660 0x7f86480cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864ff42640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86480cfbc0 con 0x7f864806d350 2024-04-24T23:28:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864ff42640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8648145cf0 con 0x7f86480cd660 2024-04-24T23:28:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864ff42640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f864806d7f0 con 0x7f8648145910 2024-04-24T23:28:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864e4df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86480cd660 0x7f86480cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864e4df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86480cd660 0x7f86480cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60024/0 (socket says 172.21.15.26:60024) 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864e4df640 1 -- 172.21.15.26:0/2807232270 learned_addr learned my addr 172.21.15.26:0/2807232270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864dcde640 1 --2- 172.21.15.26:0/2807232270 >> v2:172.21.15.29:3300/0 conn(0x7f8648145910 0x7f864806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864e4df640 1 -- 172.21.15.26:0/2807232270 >> v2:172.21.15.29:3300/0 conn(0x7f8648145910 msgr2=0x7f864806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864e4df640 1 --2- 172.21.15.26:0/2807232270 >> v2:172.21.15.29:3300/0 conn(0x7f8648145910 0x7f864806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864e4df640 1 -- 172.21.15.26:0/2807232270 >> v1:172.21.15.29:6789/0 conn(0x7f864806d350 legacy=0x7f864815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.182+0000 7f864e4df640 1 -- 172.21.15.26:0/2807232270 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864806dad0 con 0x7f86480cd660 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.183+0000 7f864e4df640 1 --2- 172.21.15.26:0/2807232270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86480cd660 0x7f86480cdaf0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f8638009e50 tx=0x7f863802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d75b0aadd90664 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.183+0000 7f864ccdc640 1 -- 172.21.15.26:0/2807232270 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638036070 con 0x7f86480cd660 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.183+0000 7f864ccdc640 1 -- 172.21.15.26:0/2807232270 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8638031600 con 0x7f86480cd660 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.183+0000 7f864ccdc640 1 -- 172.21.15.26:0/2807232270 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86380319d0 con 0x7f86480cd660 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.183+0000 7f864ff42640 1 -- 172.21.15.26:0/2807232270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86480cd660 msgr2=0x7f86480cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.183+0000 7f864ff42640 1 --2- 172.21.15.26:0/2807232270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86480cd660 0x7f86480cdaf0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f8638009e50 tx=0x7f863802a6d0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.183+0000 7f864ff42640 1 -- 172.21.15.26:0/2807232270 shutdown_connections 2024-04-24T23:28:20.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.183+0000 7f864ff42640 1 --2- 172.21.15.26:0/2807232270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86480cd660 0x7f86480cdaf0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.183+0000 7f864ff42640 1 --2- 172.21.15.26:0/2807232270 >> v2:172.21.15.29:3300/0 conn(0x7f8648145910 0x7f864806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.184+0000 7f864ff42640 1 -- 172.21.15.26:0/2807232270 >> 172.21.15.26:0/2807232270 conn(0x7f864806b300 msgr2=0x7f864806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.184+0000 7f864ff42640 1 -- 172.21.15.26:0/2807232270 shutdown_connections 2024-04-24T23:28:20.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.184+0000 7f864ff42640 1 -- 172.21.15.26:0/2807232270 wait complete. 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.185+0000 7f864ff42640 1 Processor -- start 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.185+0000 7f864ff42640 1 -- start start 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.185+0000 7f864ff42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864806d350 0x7f8648138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.185+0000 7f864ff42640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86480cd660 0x7f8648159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.185+0000 7f864ff42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8648145910 0x7f864813d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.185+0000 7f864ff42640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f864806e370 con 0x7f8648145910 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.185+0000 7f864ff42640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f864806e090 con 0x7f864806d350 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864ff42640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f864815d770 con 0x7f86480cd660 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864e4df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8648145910 0x7f864813d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864e4df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8648145910 0x7f864813d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60040/0 (socket says 172.21.15.26:60040) 2024-04-24T23:28:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864e4df640 1 -- 172.21.15.26:0/3254650369 learned_addr learned my addr 172.21.15.26:0/3254650369 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864d4dd640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86480cd660 0x7f8648159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864dcde640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864806d350 0x7f8648138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864e4df640 1 -- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86480cd660 msgr2=0x7f8648159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864e4df640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86480cd660 0x7f8648159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864e4df640 1 -- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864806d350 msgr2=0x7f8648138fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864e4df640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864806d350 0x7f8648138fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864e4df640 1 -- 172.21.15.26:0/3254650369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8648140990 con 0x7f8648145910 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864dcde640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864806d350 0x7f8648138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864d4dd640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86480cd660 0x7f8648159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864e4df640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8648145910 0x7f864813d410 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f8638034040 tx=0x7f8638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f86467fc640 1 -- 172.21.15.26:0/3254650369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638036050 con 0x7f8648145910 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f86467fc640 1 -- 172.21.15.26:0/3254650369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8638042070 con 0x7f8648145910 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8648140c70 con 0x7f8648145910 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f86467fc640 1 -- 172.21.15.26:0/3254650369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638009510 con 0x7f8648145910 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.186+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8648141230 con 0x7f8648145910 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.187+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86480694c0 con 0x7f8648145910 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.188+0000 7f86467fc640 1 -- 172.21.15.26:0/3254650369 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8638007bd0 con 0x7f8648145910 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.188+0000 7f86467fc640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f863406be30 0x7f863406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.188+0000 7f86467fc640 1 -- 172.21.15.26:0/3254650369 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f86380abb20 con 0x7f8648145910 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.188+0000 7f864dcde640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f863406be30 0x7f863406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.189+0000 7f864dcde640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f863406be30 0x7f863406e2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8640006fd0 tx=0x7f8640008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.193+0000 7f86467fc640 1 -- 172.21.15.26:0/3254650369 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8638045360 con 0x7f8648145910 2024-04-24T23:28:20.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.372+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f864806c130 con 0x7f863406be30 2024-04-24T23:28:20.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.373+0000 7f86467fc640 1 -- 172.21.15.26:0/3254650369 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13484 (secure 0 0 0) 0x7f864806c130 con 0x7f863406be30 2024-04-24T23:28:20.373 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:20.374 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:28:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f863406be30 msgr2=0x7f863406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f863406be30 0x7f863406e2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8640006fd0 tx=0x7f8640008040 comp rx=0 tx=0).stop 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8648145910 msgr2=0x7f864813d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8648145910 0x7f864813d410 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f8638034040 tx=0x7f8638004a40 comp rx=0 tx=0).stop 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 shutdown_connections 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f863406be30 0x7f863406e2f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8648145910 0x7f864813d410 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f86480cd660 0x7f8648159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 --2- 172.21.15.26:0/3254650369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f864806d350 0x7f8648138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 >> 172.21.15.26:0/3254650369 conn(0x7f864806b300 msgr2=0x7f864815cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 shutdown_connections 2024-04-24T23:28:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.375+0000 7f864ff42640 1 -- 172.21.15.26:0/3254650369 wait complete. 2024-04-24T23:28:20.386 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-24T23:28:19.412816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281152,"kb_used_data":10912,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754693568,"statfs":{"total":773094113280,"available":772806213632,"internally_reserved":0,"allocated":11173888,"data_stored":5603194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-122105,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-256,"num_write_kb":-207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001236"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T23:28:17.358319+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:28:17.358319+0000","last_peered":"2024-04-24T23:28:17.358319+0000","last_clean":"2024-04-24T23:28:17.358319+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:28:17.358319+0000","last_undegraded":"2024-04-24T23:28:17.358319+0000","last_fullsized":"2024-04-24T23:28:17.358319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37112,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334728,"statfs":{"total":96636764160,"available":96598761472,"internally_reserved":0,"allocated":1814528,"data_stored":1113455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1253376,"data_stored":560863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":522531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":620471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36616,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335224,"statfs":{"total":96636764160,"available":96599269376,"internally_reserved":0,"allocated":1306624,"data_stored":604292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36516,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335324,"statfs":{"total":96636764160,"available":96599371776,"internally_reserved":0,"allocated":1204224,"data_stored":522315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":545398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37108,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334732,"statfs":{"total":96636764160,"available":96598765568,"internally_reserved":0,"allocated":1810432,"data_stored":1113869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:28:20.387 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.487+0000 7fc4a037e640 1 Processor -- start 2024-04-24T23:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.487+0000 7fc4a037e640 1 -- start start 2024-04-24T23:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc4a037e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc498143c40 0x7fc4980cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc4a037e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4980cc860 0x7fc498072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc4a037e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc49815eb00 0x7fc49815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc4a037e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc49815f490 con 0x7fc4980cc860 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc4a037e640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc498071b70 con 0x7fc49815eb00 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc4a037e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc4980cd0b0 con 0x7fc498143c40 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49d919640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4980cc860 0x7fc498072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49d919640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4980cc860 0x7fc498072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60056/0 (socket says 172.21.15.26:60056) 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49d919640 1 -- 172.21.15.26:0/3105852153 learned_addr learned my addr 172.21.15.26:0/3105852153 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49e91b640 1 --2- 172.21.15.26:0/3105852153 >> v2:172.21.15.29:3300/0 conn(0x7fc49815eb00 0x7fc49815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49e11a640 1 --2- 172.21.15.26:0/3105852153 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc498143c40 0x7fc4980cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49d919640 1 -- 172.21.15.26:0/3105852153 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc498143c40 msgr2=0x7fc4980cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49d919640 1 --2- 172.21.15.26:0/3105852153 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc498143c40 0x7fc4980cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49d919640 1 -- 172.21.15.26:0/3105852153 >> v2:172.21.15.29:3300/0 conn(0x7fc49815eb00 msgr2=0x7fc49815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49d919640 1 --2- 172.21.15.26:0/3105852153 >> v2:172.21.15.29:3300/0 conn(0x7fc49815eb00 0x7fc49815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49d919640 1 -- 172.21.15.26:0/3105852153 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc498071240 con 0x7fc4980cc860 2024-04-24T23:28:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.488+0000 7fc49e11a640 1 --2- 172.21.15.26:0/3105852153 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc498143c40 0x7fc4980cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.489+0000 7fc49d919640 1 --2- 172.21.15.26:0/3105852153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4980cc860 0x7fc498072000 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fc49806d450 tx=0x7fc48802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7f253938d0e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.489+0000 7fc49d118640 1 -- 172.21.15.26:0/3105852153 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488037070 con 0x7fc4980cc860 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.489+0000 7fc49d118640 1 -- 172.21.15.26:0/3105852153 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc488032440 con 0x7fc4980cc860 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.489+0000 7fc49d118640 1 -- 172.21.15.26:0/3105852153 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488032810 con 0x7fc4980cc860 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.489+0000 7fc4a037e640 1 -- 172.21.15.26:0/3105852153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4980cc860 msgr2=0x7fc498072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.489+0000 7fc4a037e640 1 --2- 172.21.15.26:0/3105852153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4980cc860 0x7fc498072000 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fc49806d450 tx=0x7fc48802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.490+0000 7fc4a037e640 1 -- 172.21.15.26:0/3105852153 shutdown_connections 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.490+0000 7fc4a037e640 1 --2- 172.21.15.26:0/3105852153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4980cc860 0x7fc498072000 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.490+0000 7fc4a037e640 1 --2- 172.21.15.26:0/3105852153 >> v2:172.21.15.29:3300/0 conn(0x7fc49815eb00 0x7fc49815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.490+0000 7fc4a037e640 1 --2- 172.21.15.26:0/3105852153 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc498143c40 0x7fc4980cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.490+0000 7fc4a037e640 1 -- 172.21.15.26:0/3105852153 >> 172.21.15.26:0/3105852153 conn(0x7fc49806b300 msgr2=0x7fc49806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.490+0000 7fc4a037e640 1 -- 172.21.15.26:0/3105852153 shutdown_connections 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.490+0000 7fc4a037e640 1 -- 172.21.15.26:0/3105852153 wait complete. 2024-04-24T23:28:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.491+0000 7fc4a037e640 1 Processor -- start 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.491+0000 7fc4a037e640 1 -- start start 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc4a037e640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc4980cc860 0x7fc4981f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc4a037e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc498143c40 0x7fc498139fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc4a037e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49815eb00 0x7fc49813c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc4a037e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc498144020 con 0x7fc498143c40 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc4a037e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc498071800 con 0x7fc49815eb00 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc4a037e640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc498072540 con 0x7fc4980cc860 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49d919640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc498143c40 0x7fc498139fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49d919640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc498143c40 0x7fc498139fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60062/0 (socket says 172.21.15.26:60062) 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49d919640 1 -- 172.21.15.26:0/1054171101 learned_addr learned my addr 172.21.15.26:0/1054171101 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49e91b640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49815eb00 0x7fc49813c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49e11a640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc4980cc860 0x7fc4981f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49d919640 1 -- 172.21.15.26:0/1054171101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc4980cc860 msgr2=0x7fc4981f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49d919640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc4980cc860 0x7fc4981f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49d919640 1 -- 172.21.15.26:0/1054171101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49815eb00 msgr2=0x7fc49813c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49d919640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49815eb00 0x7fc49813c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49d919640 1 -- 172.21.15.26:0/1054171101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49813a910 con 0x7fc498143c40 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc49d919640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc498143c40 0x7fc498139fc0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fc48802a220 tx=0x7fc4880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc48effd640 1 -- 172.21.15.26:0/1054171101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488037040 con 0x7fc498143c40 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc48effd640 1 -- 172.21.15.26:0/1054171101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc488042070 con 0x7fc498143c40 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.492+0000 7fc48effd640 1 -- 172.21.15.26:0/1054171101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488030880 con 0x7fc498143c40 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.493+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49813abf0 con 0x7fc498143c40 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.493+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4980c93a0 con 0x7fc498143c40 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.494+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49813b110 con 0x7fc498143c40 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.494+0000 7fc48effd640 1 -- 172.21.15.26:0/1054171101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc488030a20 con 0x7fc498143c40 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.494+0000 7fc48effd640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc47c06be30 0x7fc47c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.495+0000 7fc48effd640 1 -- 172.21.15.26:0/1054171101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7fc4880ab4c0 con 0x7fc498143c40 2024-04-24T23:28:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.495+0000 7fc49e11a640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc47c06be30 0x7fc47c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.495+0000 7fc49e11a640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc47c06be30 0x7fc47c06e2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc494006fd0 tx=0x7fc494008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.499+0000 7fc48effd640 1 -- 172.21.15.26:0/1054171101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc49813b110 con 0x7fc498143c40 2024-04-24T23:28:20.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.678+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4980694c0 con 0x7fc47c06be30 2024-04-24T23:28:20.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.679+0000 7fc48effd640 1 -- 172.21.15.26:0/1054171101 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13484 (secure 0 0 0) 0x7fc4980694c0 con 0x7fc47c06be30 2024-04-24T23:28:20.679 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:20.680 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc47c06be30 msgr2=0x7fc47c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc47c06be30 0x7fc47c06e2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc494006fd0 tx=0x7fc494008040 comp rx=0 tx=0).stop 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc498143c40 msgr2=0x7fc498139fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc498143c40 0x7fc498139fc0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fc48802a220 tx=0x7fc4880046c0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 shutdown_connections 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc47c06be30 0x7fc47c06e2f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc49815eb00 0x7fc49813c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc498143c40 0x7fc498139fc0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 --2- 172.21.15.26:0/1054171101 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc4980cc860 0x7fc4981f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 >> 172.21.15.26:0/1054171101 conn(0x7fc49806b300 msgr2=0x7fc49815be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 shutdown_connections 2024-04-24T23:28:20.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.681+0000 7fc4a037e640 1 -- 172.21.15.26:0/1054171101 wait complete. 2024-04-24T23:28:20.693 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-24T23:28:19.412816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281152,"kb_used_data":10912,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754693568,"statfs":{"total":773094113280,"available":772806213632,"internally_reserved":0,"allocated":11173888,"data_stored":5603194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-122105,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-256,"num_write_kb":-207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001236"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T23:28:17.358319+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:28:17.358319+0000","last_peered":"2024-04-24T23:28:17.358319+0000","last_clean":"2024-04-24T23:28:17.358319+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:28:17.358319+0000","last_undegraded":"2024-04-24T23:28:17.358319+0000","last_fullsized":"2024-04-24T23:28:17.358319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37112,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334728,"statfs":{"total":96636764160,"available":96598761472,"internally_reserved":0,"allocated":1814528,"data_stored":1113455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1253376,"data_stored":560863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":522531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":620471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36616,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335224,"statfs":{"total":96636764160,"available":96599269376,"internally_reserved":0,"allocated":1306624,"data_stored":604292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36516,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335324,"statfs":{"total":96636764160,"available":96599371776,"internally_reserved":0,"allocated":1204224,"data_stored":522315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":545398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37108,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334732,"statfs":{"total":96636764160,"available":96598765568,"internally_reserved":0,"allocated":1810432,"data_stored":1113869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:28:20.693 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T23:28:20.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:28:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.793+0000 7fe1491a2640 1 Processor -- start 2024-04-24T23:28:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.793+0000 7fe1491a2640 1 -- start start 2024-04-24T23:28:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe1491a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe144143c40 0x7fe1440cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe1491a2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1440cc860 0x7fe144072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe1491a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1440cd0b0 con 0x7fe14415eb00 2024-04-24T23:28:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe1491a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe144071b70 con 0x7fe144143c40 2024-04-24T23:28:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe1491a2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe144071240 con 0x7fe1440cc860 2024-04-24T23:28:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe142d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe144143c40 0x7fe1440cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe142d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe144143c40 0x7fe1440cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60064/0 (socket says 172.21.15.26:60064) 2024-04-24T23:28:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe142d9d640 1 -- 172.21.15.26:0/388129791 learned_addr learned my addr 172.21.15.26:0/388129791 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe14359e640 1 --1- 172.21.15.26:0/388129791 >> v1:172.21.15.26:6789/0 conn(0x7fe14415eb00 0x7fe1440b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52934/0 (socket says 172.21.15.26:52934) 2024-04-24T23:28:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe14259c640 1 --2- 172.21.15.26:0/388129791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1440cc860 0x7fe144072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe142d9d640 1 -- 172.21.15.26:0/388129791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1440cc860 msgr2=0x7fe144072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe142d9d640 1 --2- 172.21.15.26:0/388129791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1440cc860 0x7fe144072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe142d9d640 1 -- 172.21.15.26:0/388129791 >> v1:172.21.15.26:6789/0 conn(0x7fe14415eb00 legacy=0x7fe1440b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe141d9b640 1 -- 172.21.15.26:0/388129791 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 648898765 0 0) 0x7fe1440cd0b0 con 0x7fe14415eb00 2024-04-24T23:28:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.794+0000 7fe142d9d640 1 -- 172.21.15.26:0/388129791 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe144071520 con 0x7fe144143c40 2024-04-24T23:28:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.795+0000 7fe142d9d640 1 --2- 172.21.15.26:0/388129791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 0x7fe1440cf870 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009e50 tx=0x7fe13c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b7c1d8f95162dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.795+0000 7fe141d9b640 1 -- 172.21.15.26:0/388129791 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c036070 con 0x7fe144143c40 2024-04-24T23:28:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.795+0000 7fe141d9b640 1 -- 172.21.15.26:0/388129791 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c031600 con 0x7fe144143c40 2024-04-24T23:28:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.795+0000 7fe141d9b640 1 -- 172.21.15.26:0/388129791 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0319d0 con 0x7fe144143c40 2024-04-24T23:28:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.796+0000 7fe1491a2640 1 -- 172.21.15.26:0/388129791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 msgr2=0x7fe1440cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.796+0000 7fe1491a2640 1 --2- 172.21.15.26:0/388129791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 0x7fe1440cf870 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009e50 tx=0x7fe13c02a710 comp rx=0 tx=0).stop 2024-04-24T23:28:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.796+0000 7fe1491a2640 1 -- 172.21.15.26:0/388129791 shutdown_connections 2024-04-24T23:28:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.796+0000 7fe1491a2640 1 --2- 172.21.15.26:0/388129791 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe1440cc860 0x7fe144072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.796+0000 7fe1491a2640 1 --2- 172.21.15.26:0/388129791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 0x7fe1440cf870 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.796+0000 7fe1491a2640 1 -- 172.21.15.26:0/388129791 >> 172.21.15.26:0/388129791 conn(0x7fe14406b300 msgr2=0x7fe14406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.796+0000 7fe1491a2640 1 -- 172.21.15.26:0/388129791 shutdown_connections 2024-04-24T23:28:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.796+0000 7fe1491a2640 1 -- 172.21.15.26:0/388129791 wait complete. 2024-04-24T23:28:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.797+0000 7fe1491a2640 1 Processor -- start 2024-04-24T23:28:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.797+0000 7fe1491a2640 1 -- start start 2024-04-24T23:28:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe1491a2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1440cc860 0x7fe14413d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe1491a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 0x7fe1440a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe1491a2640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe14415eb00 0x7fe1440b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe1491a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe144072540 con 0x7fe144143c40 2024-04-24T23:28:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe1491a2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe144144020 con 0x7fe1440cc860 2024-04-24T23:28:20.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe1491a2640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe1440cace0 con 0x7fe14415eb00 2024-04-24T23:28:20.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe14259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 0x7fe1440a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe14259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 0x7fe1440a9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60078/0 (socket says 172.21.15.26:60078) 2024-04-24T23:28:20.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe14259c640 1 -- 172.21.15.26:0/56780731 learned_addr learned my addr 172.21.15.26:0/56780731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:20.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe142d9d640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1440cc860 0x7fe14413d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.798+0000 7fe14359e640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe14415eb00 0x7fe1440b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe14259c640 1 -- 172.21.15.26:0/56780731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe14415eb00 msgr2=0x7fe1440b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe14259c640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe14415eb00 0x7fe1440b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe14259c640 1 -- 172.21.15.26:0/56780731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1440cc860 msgr2=0x7fe14413d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe14259c640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1440cc860 0x7fe14413d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe14259c640 1 -- 172.21.15.26:0/56780731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1440cafc0 con 0x7fe144143c40 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe14359e640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe14415eb00 0x7fe1440b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe14259c640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 0x7fe1440a9dd0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe13800d830 tx=0x7fe13800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe12b7fe640 1 -- 172.21.15.26:0/56780731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13800b6c0 con 0x7fe144143c40 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe12b7fe640 1 -- 172.21.15.26:0/56780731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe138019070 con 0x7fe144143c40 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1440cb2a0 con 0x7fe144143c40 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe12b7fe640 1 -- 172.21.15.26:0/56780731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13800ba80 con 0x7fe144143c40 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.799+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1440aa310 con 0x7fe144143c40 2024-04-24T23:28:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.800+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1440694c0 con 0x7fe144143c40 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.801+0000 7fe12b7fe640 1 -- 172.21.15.26:0/56780731 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe13800bc20 con 0x7fe144143c40 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.801+0000 7fe12b7fe640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe13006be30 0x7fe13006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.801+0000 7fe12b7fe640 1 -- 172.21.15.26:0/56780731 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7fe13808afa0 con 0x7fe144143c40 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.801+0000 7fe142d9d640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe13006be30 0x7fe13006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.802+0000 7fe142d9d640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe13006be30 0x7fe13006e2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe13c033000 tx=0x7fe13c009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.805+0000 7fe12b7fe640 1 -- 172.21.15.26:0/56780731 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe138059c30 con 0x7fe144143c40 2024-04-24T23:28:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.985+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1440b4d20 con 0x7fe13006be30 2024-04-24T23:28:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.986+0000 7fe12b7fe640 1 -- 172.21.15.26:0/56780731 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13484 (secure 0 0 0) 0x7fe1440b4d20 con 0x7fe13006be30 2024-04-24T23:28:20.987 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:28:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe13006be30 msgr2=0x7fe13006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe13006be30 0x7fe13006e2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe13c033000 tx=0x7fe13c009000 comp rx=0 tx=0).stop 2024-04-24T23:28:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 msgr2=0x7fe1440a9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 0x7fe1440a9dd0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe13800d830 tx=0x7fe13800dcf0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 shutdown_connections 2024-04-24T23:28:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe13006be30 0x7fe13006e2f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe14415eb00 0x7fe1440b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe144143c40 0x7fe1440a9dd0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 --2- 172.21.15.26:0/56780731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1440cc860 0x7fe14413d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 >> 172.21.15.26:0/56780731 conn(0x7fe14406b300 msgr2=0x7fe14406b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 shutdown_connections 2024-04-24T23:28:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:20.988+0000 7fe1491a2640 1 -- 172.21.15.26:0/56780731 wait complete. 2024-04-24T23:28:21.000 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-24T23:28:19.412816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281152,"kb_used_data":10912,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754693568,"statfs":{"total":773094113280,"available":772806213632,"internally_reserved":0,"allocated":11173888,"data_stored":5603194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-122105,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-256,"num_write_kb":-207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001236"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T23:28:17.358319+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:28:17.358319+0000","last_peered":"2024-04-24T23:28:17.358319+0000","last_clean":"2024-04-24T23:28:17.358319+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:28:17.358319+0000","last_undegraded":"2024-04-24T23:28:17.358319+0000","last_fullsized":"2024-04-24T23:28:17.358319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37112,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334728,"statfs":{"total":96636764160,"available":96598761472,"internally_reserved":0,"allocated":1814528,"data_stored":1113455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1253376,"data_stored":560863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":522531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":620471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36616,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335224,"statfs":{"total":96636764160,"available":96599269376,"internally_reserved":0,"allocated":1306624,"data_stored":604292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36516,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335324,"statfs":{"total":96636764160,"available":96599371776,"internally_reserved":0,"allocated":1204224,"data_stored":522315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1245184,"data_stored":545398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37108,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334732,"statfs":{"total":96636764160,"available":96598765568,"internally_reserved":0,"allocated":1810432,"data_stored":1113869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:28:21.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:28:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.102+0000 7ff984b42640 1 Processor -- start 2024-04-24T23:28:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.102+0000 7ff984b42640 1 -- start start 2024-04-24T23:28:21.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.103+0000 7ff984b42640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 0x7ff9800b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.103+0000 7ff984b42640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff9800732b0 0x7ff98015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.103+0000 7ff984b42640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9800b09a0 con 0x7ff98015cb30 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.103+0000 7ff984b42640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7ff9800ad170 con 0x7ff9800732b0 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.103+0000 7ff984b42640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff9800ad730 con 0x7ff98015c210 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97dd9b640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7ff9800732b0 0x7ff98015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97e59c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 0x7ff9800b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97ed9d640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7ff98015cb30 0x7ff9800aed50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:45334/0 (socket says 172.21.15.26:45334) 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97ed9d640 1 -- 172.21.15.26:0/3187414670 learned_addr learned my addr 172.21.15.26:0/3187414670 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97d59a640 1 -- 172.21.15.26:0/3187414670 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198794562 0 0) 0x7ff9800b09a0 con 0x7ff98015cb30 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97d59a640 1 -- 172.21.15.26:0/3187414670 --> v1:172.21.15.29:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff96c003520 con 0x7ff98015cb30 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97e59c640 1 -- 172.21.15.26:0/3187414670 >> v2:172.21.15.29:3300/0 conn(0x7ff9800732b0 msgr2=0x7ff98015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97e59c640 1 --2- 172.21.15.26:0/3187414670 >> v2:172.21.15.29:3300/0 conn(0x7ff9800732b0 0x7ff98015c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97e59c640 1 -- 172.21.15.26:0/3187414670 >> v1:172.21.15.29:6789/0 conn(0x7ff98015cb30 legacy=0x7ff9800aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97e59c640 1 -- 172.21.15.26:0/3187414670 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9800b0c80 con 0x7ff98015c210 2024-04-24T23:28:21.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.104+0000 7ff97dd9b640 1 --2- 172.21.15.26:0/3187414670 >> v2:172.21.15.29:3300/0 conn(0x7ff9800732b0 0x7ff98015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.105+0000 7ff97e59c640 1 --2- 172.21.15.26:0/3187414670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 0x7ff9800b1650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff970009d90 tx=0x7ff97002a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ace9d91fa5b9f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.105+0000 7ff97d59a640 1 -- 172.21.15.26:0/3187414670 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970007740 con 0x7ff98015c210 2024-04-24T23:28:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.105+0000 7ff97d59a640 1 -- 172.21.15.26:0/3187414670 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9700078e0 con 0x7ff98015c210 2024-04-24T23:28:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.105+0000 7ff97d59a640 1 -- 172.21.15.26:0/3187414670 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970007c90 con 0x7ff98015c210 2024-04-24T23:28:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.105+0000 7ff984b42640 1 -- 172.21.15.26:0/3187414670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 msgr2=0x7ff9800b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.105+0000 7ff984b42640 1 --2- 172.21.15.26:0/3187414670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 0x7ff9800b1650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff970009d90 tx=0x7ff97002a530 comp rx=0 tx=0).stop 2024-04-24T23:28:21.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.106+0000 7ff984b42640 1 -- 172.21.15.26:0/3187414670 shutdown_connections 2024-04-24T23:28:21.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.106+0000 7ff984b42640 1 --2- 172.21.15.26:0/3187414670 >> v2:172.21.15.29:3300/0 conn(0x7ff9800732b0 0x7ff98015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.106+0000 7ff984b42640 1 --2- 172.21.15.26:0/3187414670 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 0x7ff9800b1650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.106+0000 7ff984b42640 1 -- 172.21.15.26:0/3187414670 >> 172.21.15.26:0/3187414670 conn(0x7ff980072790 msgr2=0x7ff980072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.106+0000 7ff984b42640 1 -- 172.21.15.26:0/3187414670 shutdown_connections 2024-04-24T23:28:21.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.106+0000 7ff984b42640 1 -- 172.21.15.26:0/3187414670 wait complete. 2024-04-24T23:28:21.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.107+0000 7ff984b42640 1 Processor -- start 2024-04-24T23:28:21.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.107+0000 7ff984b42640 1 -- start start 2024-04-24T23:28:21.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff984b42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800732b0 0x7ff9801467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff984b42640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 0x7ff9801e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff984b42640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98015cb30 0x7ff9801e6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff984b42640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800d0f60 con 0x7ff9800732b0 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff984b42640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800b1b90 con 0x7ff98015cb30 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff984b42640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff98015cfc0 con 0x7ff98015c210 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800732b0 0x7ff9801467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800732b0 0x7ff9801467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60094/0 (socket says 172.21.15.26:60094) 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97e59c640 1 -- 172.21.15.26:0/337985024 learned_addr learned my addr 172.21.15.26:0/337985024 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97ed9d640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98015cb30 0x7ff9801e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97dd9b640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 0x7ff9801e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97e59c640 1 -- 172.21.15.26:0/337985024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 msgr2=0x7ff9801e6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97e59c640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 0x7ff9801e6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97e59c640 1 -- 172.21.15.26:0/337985024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98015cb30 msgr2=0x7ff9801e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97e59c640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98015cb30 0x7ff9801e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97e59c640 1 -- 172.21.15.26:0/337985024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9801e5290 con 0x7ff9800732b0 2024-04-24T23:28:21.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.108+0000 7ff97e59c640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800732b0 0x7ff9801467b0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7ff970009eb0 tx=0x7ff970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.109+0000 7ff976ffd640 1 -- 172.21.15.26:0/337985024 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970040070 con 0x7ff9800732b0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.109+0000 7ff976ffd640 1 -- 172.21.15.26:0/337985024 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff970031c40 con 0x7ff9800732b0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.109+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9801e5570 con 0x7ff9800732b0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.109+0000 7ff976ffd640 1 -- 172.21.15.26:0/337985024 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970009450 con 0x7ff9800732b0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.109+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff980154660 con 0x7ff9800732b0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.110+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9800694c0 con 0x7ff9800732b0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.110+0000 7ff976ffd640 1 -- 172.21.15.26:0/337985024 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff970039040 con 0x7ff9800732b0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.110+0000 7ff976ffd640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff96006be30 0x7ff96006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.111+0000 7ff976ffd640 1 -- 172.21.15.26:0/337985024 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7ff9700ac290 con 0x7ff9800732b0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.111+0000 7ff97dd9b640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff96006be30 0x7ff96006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.111+0000 7ff97dd9b640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff96006be30 0x7ff96006e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff968009560 tx=0x7ff968009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.115+0000 7ff976ffd640 1 -- 172.21.15.26:0/337985024 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff97007a9f0 con 0x7ff9800732b0 2024-04-24T23:28:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.300+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff98015dcd0 con 0x7ff9800732b0 2024-04-24T23:28:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.301+0000 7ff976ffd640 1 -- 172.21.15.26:0/337985024 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+13221 (secure 0 0 0) 0x7ff97007ecf0 con 0x7ff9800732b0 2024-04-24T23:28:21.302 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:21.302 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":99,"fsid":"5a872e4b-97fd-4255-95a5-3ea24d1048cf","created":"2024-04-24T23:20:31.033039+0000","modified":"2024-04-24T23:28:18.185781+0000","last_up_change":"2024-04-24T23:20:41.659811+0000","last_in_change":"2024-04-24T23:20:33.243284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:20:43.358101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dcaba0fb-1e8d-46d0-aa7b-244f65b6f71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6821","nonce":256768257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6823","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6825","nonce":256768257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6831","nonce":256768257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6827","nonce":256768257},{"type":"v1","addr":"172.21.15.26:6829","nonce":256768257}]},"public_addr":"172.21.15.26:6821/256768257","cluster_addr":"172.21.15.26:6825/256768257","heartbeat_back_addr":"172.21.15.26:6831/256768257","heartbeat_front_addr":"172.21.15.26:6829/256768257","state":["exists","up"]},{"osd":1,"uuid":"6a144c00-9fcd-45b9-a22b-6964b76e7d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6817","nonce":1500554534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6820","nonce":1500554534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6828","nonce":1500554534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1500554534},{"type":"v1","addr":"172.21.15.26:6824","nonce":1500554534}]},"public_addr":"172.21.15.26:6817/1500554534","cluster_addr":"172.21.15.26:6820/1500554534","heartbeat_back_addr":"172.21.15.26:6828/1500554534","heartbeat_front_addr":"172.21.15.26:6824/1500554534","state":["exists","up"]},{"osd":2,"uuid":"e43870f1-4c8c-4cde-8dcb-3eb9e80cd853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6801","nonce":3780187135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6803","nonce":3780187135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6807","nonce":3780187135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3780187135},{"type":"v1","addr":"172.21.15.26:6805","nonce":3780187135}]},"public_addr":"172.21.15.26:6801/3780187135","cluster_addr":"172.21.15.26:6803/3780187135","heartbeat_back_addr":"172.21.15.26:6807/3780187135","heartbeat_front_addr":"172.21.15.26:6805/3780187135","state":["exists","up"]},{"osd":3,"uuid":"64ec8a53-51a8-4500-a714-36c1a8f342ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6809","nonce":2035350946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6811","nonce":2035350946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6815","nonce":2035350946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2035350946},{"type":"v1","addr":"172.21.15.26:6813","nonce":2035350946}]},"public_addr":"172.21.15.26:6809/2035350946","cluster_addr":"172.21.15.26:6811/2035350946","heartbeat_back_addr":"172.21.15.26:6815/2035350946","heartbeat_front_addr":"172.21.15.26:6813/2035350946","state":["exists","up"]},{"osd":4,"uuid":"fe83d1a2-0532-4aad-aba5-c02850051d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6809","nonce":3208387101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6811","nonce":3208387101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6820","nonce":3208387101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3208387101},{"type":"v1","addr":"172.21.15.29:6816","nonce":3208387101}]},"public_addr":"172.21.15.29:6809/3208387101","cluster_addr":"172.21.15.29:6811/3208387101","heartbeat_back_addr":"172.21.15.29:6820/3208387101","heartbeat_front_addr":"172.21.15.29:6816/3208387101","state":["exists","up"]},{"osd":5,"uuid":"6f2cff4a-8ea8-42ac-9f50-f870906ae05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6801","nonce":1108214380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6803","nonce":1108214380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6807","nonce":1108214380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":1108214380},{"type":"v1","addr":"172.21.15.29:6805","nonce":1108214380}]},"public_addr":"172.21.15.29:6801/1108214380","cluster_addr":"172.21.15.29:6803/1108214380","heartbeat_back_addr":"172.21.15.29:6807/1108214380","heartbeat_front_addr":"172.21.15.29:6805/1108214380","state":["exists","up"]},{"osd":6,"uuid":"f684c2b3-ae8a-43da-9091-d7c4dacbdd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6813","nonce":744093613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6815","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6817","nonce":744093613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6823","nonce":744093613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6819","nonce":744093613},{"type":"v1","addr":"172.21.15.29:6821","nonce":744093613}]},"public_addr":"172.21.15.29:6813/744093613","cluster_addr":"172.21.15.29:6817/744093613","heartbeat_back_addr":"172.21.15.29:6823/744093613","heartbeat_front_addr":"172.21.15.29:6821/744093613","state":["exists","up"]},{"osd":7,"uuid":"6b4fa289-00aa-481d-a9c1-87634fa1d532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6825","nonce":2604664408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6827","nonce":2604664408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6831","nonce":2604664408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":2604664408},{"type":"v1","addr":"172.21.15.29:6829","nonce":2604664408}]},"public_addr":"172.21.15.29:6825/2604664408","cluster_addr":"172.21.15.29:6827/2604664408","heartbeat_back_addr":"172.21.15.29:6831/2604664408","heartbeat_front_addr":"172.21.15.29:6829/2604664408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.449848+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.388600+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.355643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.361924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.460429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.344919+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.464536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:20:38.405017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.29:6835/2225348762":"2024-04-25T23:28:15.178186+0000","172.21.15.29:6833/2225348762":"2024-04-25T23:28:15.178186+0000","172.21.15.29:6834/2632309869":"2024-04-25T23:28:15.178169+0000","172.21.15.29:6832/2632309869":"2024-04-25T23:28:15.178169+0000","172.21.15.26:6835/3062763802":"2024-04-25T23:28:15.178182+0000","172.21.15.26:6834/3062763802":"2024-04-25T23:28:15.178182+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:28:21.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff96006be30 msgr2=0x7ff96006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff96006be30 0x7ff96006e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff968009560 tx=0x7ff968009190 comp rx=0 tx=0).stop 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800732b0 msgr2=0x7ff9801467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800732b0 0x7ff9801467b0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7ff970009eb0 tx=0x7ff970004a40 comp rx=0 tx=0).stop 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 shutdown_connections 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff96006be30 0x7ff96006e2f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff98015cb30 0x7ff9801e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff98015c210 0x7ff9801e6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 --2- 172.21.15.26:0/337985024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800732b0 0x7ff9801467b0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 >> 172.21.15.26:0/337985024 conn(0x7ff980072790 msgr2=0x7ff98006e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 shutdown_connections 2024-04-24T23:28:21.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:21.303+0000 7ff984b42640 1 -- 172.21.15.26:0/337985024 wait complete. 2024-04-24T23:28:22.317 INFO:tasks.ceph:Scrubbing osd.0 2024-04-24T23:28:22.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:28:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.435+0000 7f27431f9640 1 Processor -- start 2024-04-24T23:28:22.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.435+0000 7f27431f9640 1 -- start start 2024-04-24T23:28:22.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.436+0000 7f27431f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c141160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.436+0000 7f27431f9640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f273c15ee60 0x7f273c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.436+0000 7f27431f9640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f273c0cf050 con 0x7f273c15b490 2024-04-24T23:28:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.436+0000 7f27431f9640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f273c0cf330 con 0x7f273c15ee60 2024-04-24T23:28:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.436+0000 7f27431f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f273c0cf610 con 0x7f273c15ea80 2024-04-24T23:28:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.436+0000 7f273bfff640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f273c15ee60 0x7f273c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.436+0000 7f2740f95640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.436+0000 7f2741796640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f273c15b490 0x7f273c15b920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:45346/0 (socket says 172.21.15.26:45346) 2024-04-24T23:28:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.436+0000 7f2741796640 1 -- 172.21.15.26:0/3955949402 learned_addr learned my addr 172.21.15.26:0/3955949402 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.437+0000 7f273bfff640 1 -- 172.21.15.26:0/3955949402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 msgr2=0x7f273c141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.437+0000 7f273bfff640 1 --2- 172.21.15.26:0/3955949402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.437+0000 7f273bfff640 1 -- 172.21.15.26:0/3955949402 >> v1:172.21.15.29:6789/0 conn(0x7f273c15b490 legacy=0x7f273c15b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.437+0000 7f273bfff640 1 -- 172.21.15.26:0/3955949402 --> v2:172.21.15.29:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c0cf8f0 con 0x7f273c15ee60 2024-04-24T23:28:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.437+0000 7f273b7fe640 1 -- 172.21.15.26:0/3955949402 <== mon.1 v1:172.21.15.29:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2405382058 0 0) 0x7f273c0cf050 con 0x7f273c15b490 2024-04-24T23:28:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.437+0000 7f2740f95640 1 --2- 172.21.15.26:0/3955949402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.437+0000 7f273bfff640 1 --2- 172.21.15.26:0/3955949402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f273c15ee60 0x7f273c0ce740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f27280097a0 tx=0x7f272802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6431122e927ebbf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f273b7fe640 1 -- 172.21.15.26:0/3955949402 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728037070 con 0x7f273c15ee60 2024-04-24T23:28:22.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f273b7fe640 1 -- 172.21.15.26:0/3955949402 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2728032440 con 0x7f273c15ee60 2024-04-24T23:28:22.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f273b7fe640 1 -- 172.21.15.26:0/3955949402 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728032810 con 0x7f273c15ee60 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f27431f9640 1 -- 172.21.15.26:0/3955949402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f273c15ee60 msgr2=0x7f273c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f27431f9640 1 --2- 172.21.15.26:0/3955949402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f273c15ee60 0x7f273c0ce740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f27280097a0 tx=0x7f272802a180 comp rx=0 tx=0).stop 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f27431f9640 1 -- 172.21.15.26:0/3955949402 shutdown_connections 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f27431f9640 1 --2- 172.21.15.26:0/3955949402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f273c15ee60 0x7f273c0ce740 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f27431f9640 1 --2- 172.21.15.26:0/3955949402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f27431f9640 1 -- 172.21.15.26:0/3955949402 >> 172.21.15.26:0/3955949402 conn(0x7f273c06b300 msgr2=0x7f273c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f27431f9640 1 -- 172.21.15.26:0/3955949402 shutdown_connections 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.438+0000 7f27431f9640 1 -- 172.21.15.26:0/3955949402 wait complete. 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.439+0000 7f27431f9640 1 Processor -- start 2024-04-24T23:28:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.439+0000 7f27431f9640 1 -- start start 2024-04-24T23:28:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.440+0000 7f27431f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f273c15b490 0x7f273c153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.440+0000 7f27431f9640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.440+0000 7f27431f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c15ee60 0x7f273c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.440+0000 7f27431f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f273c15f2a0 con 0x7f273c15ee60 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.440+0000 7f27431f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f273c0cec80 con 0x7f273c15b490 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.440+0000 7f27431f9640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f273c153470 con 0x7f273c15ea80 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f2741796640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c15ee60 0x7f273c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f273bfff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f2740f95640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f273c15b490 0x7f273c153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f2741796640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c15ee60 0x7f273c15d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60110/0 (socket says 172.21.15.26:60110) 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f273bfff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c152f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:35280/0 (socket says 172.21.15.26:35280) 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f2741796640 1 -- 172.21.15.26:0/325084341 learned_addr learned my addr 172.21.15.26:0/325084341 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f273bfff640 1 -- 172.21.15.26:0/325084341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f273c15b490 msgr2=0x7f273c153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f273bfff640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f273c15b490 0x7f273c153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f273bfff640 1 -- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c15ee60 msgr2=0x7f273c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f273bfff640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c15ee60 0x7f273c15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f273bfff640 1 -- 172.21.15.26:0/325084341 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c153db0 con 0x7f273c15ea80 2024-04-24T23:28:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f2741796640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c15ee60 0x7f273c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f273bfff640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c152f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f273c06f1b0 tx=0x7f27280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f27397fa640 1 -- 172.21.15.26:0/325084341 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728041030 con 0x7f273c15ea80 2024-04-24T23:28:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f27397fa640 1 -- 172.21.15.26:0/325084341 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272803d420 con 0x7f273c15ea80 2024-04-24T23:28:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c154090 con 0x7f273c15ea80 2024-04-24T23:28:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f27397fa640 1 -- 172.21.15.26:0/325084341 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728037040 con 0x7f273c15ea80 2024-04-24T23:28:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.441+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c154650 con 0x7f273c15ea80 2024-04-24T23:28:22.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.442+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f273c0694c0 con 0x7f273c15ea80 2024-04-24T23:28:22.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.443+0000 7f27397fa640 1 -- 172.21.15.26:0/325084341 <== mon.2 v2:172.21.15.29:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2728044070 con 0x7f273c15ea80 2024-04-24T23:28:22.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.443+0000 7f27397fa640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f272006be30 0x7f272006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.444+0000 7f27397fa640 1 -- 172.21.15.26:0/325084341 <== mon.2 v2:172.21.15.29:3301/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f27280ab180 con 0x7f273c15ea80 2024-04-24T23:28:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.444+0000 7f2740f95640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f272006be30 0x7f272006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.444+0000 7f27397fa640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7f2720071e30 0x7f2720074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.444+0000 7f27397fa640 1 -- 172.21.15.26:0/325084341 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f272800af30 con 0x7f2720071e30 2024-04-24T23:28:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.444+0000 7f27397fa640 1 -- 172.21.15.26:0/325084341 <== mon.2 v2:172.21.15.29:3301/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f27280b2050 con 0x7f273c15ea80 2024-04-24T23:28:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.444+0000 7f2741796640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7f2720071e30 0x7f2720074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.444+0000 7f2740f95640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f272006be30 0x7f272006e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2730009560 tx=0x7f2730009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.444+0000 7f2741796640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7f2720071e30 0x7f2720074250 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.445+0000 7f27397fa640 1 -- 172.21.15.26:0/325084341 <== osd.0 v2:172.21.15.26:6819/256768257 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f272800af30 con 0x7f2720071e30 2024-04-24T23:28:22.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.458+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 --> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f273c13dc40 con 0x7f2720071e30 2024-04-24T23:28:22.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27397fa640 1 -- 172.21.15.26:0/325084341 <== osd.0 v2:172.21.15.26:6819/256768257 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f273c13dc40 con 0x7f2720071e30 2024-04-24T23:28:22.460 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:22.460 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T23:28:22.460 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7f2720071e30 msgr2=0x7f2720074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6819/256768257,v1:172.21.15.26:6821/256768257] conn(0x7f2720071e30 0x7f2720074250 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f272006be30 msgr2=0x7f272006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f272006be30 0x7f272006e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2730009560 tx=0x7f2730009190 comp rx=0 tx=0).stop 2024-04-24T23:28:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 msgr2=0x7f273c152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 --2- 172.21.15.26:0/325084341 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f273c15ea80 0x7f273c152f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f273c06f1b0 tx=0x7f27280046c0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f2741796640 1 -- 172.21.15.26:0/325084341 reap_dead start 2024-04-24T23:28:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 shutdown_connections 2024-04-24T23:28:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 >> 172.21.15.26:0/325084341 conn(0x7f273c06b300 msgr2=0x7f273c159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 shutdown_connections 2024-04-24T23:28:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.459+0000 7f27431f9640 1 -- 172.21.15.26:0/325084341 wait complete. 2024-04-24T23:28:22.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-24T23:28:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.572+0000 7f1395963640 1 Processor -- start 2024-04-24T23:28:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.572+0000 7f1395963640 1 -- start start 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f1395963640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f139015eb00 0x7f139015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f1395963640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139015f450 con 0x7f1390143c40 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f1395963640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13900cd0b0 con 0x7f13900cf870 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f1395963640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1390071b70 con 0x7f139015eb00 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f138f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f139015eb00 0x7f139015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f138effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1390143c40 0x7f13900cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52944/0 (socket says 172.21.15.26:52944) 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f138effd640 1 -- 172.21.15.26:0/2309667923 learned_addr learned my addr 172.21.15.26:0/2309667923 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f138f7fe640 1 -- 172.21.15.26:0/2309667923 >> v1:172.21.15.29:6789/0 conn(0x7f13900cf870 legacy=0x7f1390072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f138f7fe640 1 -- 172.21.15.26:0/2309667923 >> v1:172.21.15.26:6789/0 conn(0x7f1390143c40 legacy=0x7f13900cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.573+0000 7f138f7fe640 1 -- 172.21.15.26:0/2309667923 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1390071240 con 0x7f139015eb00 2024-04-24T23:28:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.574+0000 7f138f7fe640 1 --2- 172.21.15.26:0/2309667923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 0x7f139015ef10 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f13800097a0 tx=0x7f138002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd439dce4f69825 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.574+0000 7f138dffb640 1 -- 172.21.15.26:0/2309667923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380037070 con 0x7f139015eb00 2024-04-24T23:28:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.574+0000 7f138dffb640 1 -- 172.21.15.26:0/2309667923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1380032440 con 0x7f139015eb00 2024-04-24T23:28:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.574+0000 7f138dffb640 1 -- 172.21.15.26:0/2309667923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380032810 con 0x7f139015eb00 2024-04-24T23:28:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.575+0000 7f1395963640 1 -- 172.21.15.26:0/2309667923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 msgr2=0x7f139015ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.575+0000 7f1395963640 1 --2- 172.21.15.26:0/2309667923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 0x7f139015ef10 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f13800097a0 tx=0x7f138002a230 comp rx=0 tx=0).stop 2024-04-24T23:28:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.575+0000 7f1395963640 1 -- 172.21.15.26:0/2309667923 shutdown_connections 2024-04-24T23:28:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.575+0000 7f1395963640 1 --2- 172.21.15.26:0/2309667923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 0x7f139015ef10 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.575+0000 7f1395963640 1 -- 172.21.15.26:0/2309667923 >> 172.21.15.26:0/2309667923 conn(0x7f139006b300 msgr2=0x7f139006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.575+0000 7f1395963640 1 -- 172.21.15.26:0/2309667923 shutdown_connections 2024-04-24T23:28:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.575+0000 7f1395963640 1 -- 172.21.15.26:0/2309667923 wait complete. 2024-04-24T23:28:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.576+0000 7f1395963640 1 Processor -- start 2024-04-24T23:28:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.576+0000 7f1395963640 1 -- start start 2024-04-24T23:28:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f1395963640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13900cf870 0x7f13901f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f1395963640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1390143c40 0x7f139015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f1395963640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 0x7f1390138330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f1395963640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1390144020 con 0x7f139015eb00 2024-04-24T23:28:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f1395963640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1390071800 con 0x7f1390143c40 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f1395963640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f13900b2e80 con 0x7f13900cf870 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 0x7f1390138330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 0x7f1390138330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60114/0 (socket says 172.21.15.26:60114) 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138f7fe640 1 -- 172.21.15.26:0/3617874761 learned_addr learned my addr 172.21.15.26:0/3617874761 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138effd640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13900cf870 0x7f13901f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138e7fc640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1390143c40 0x7f139015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138f7fe640 1 -- 172.21.15.26:0/3617874761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13900cf870 msgr2=0x7f13901f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138f7fe640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13900cf870 0x7f13901f18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138f7fe640 1 -- 172.21.15.26:0/3617874761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1390143c40 msgr2=0x7f139015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138f7fe640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1390143c40 0x7f139015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138f7fe640 1 -- 172.21.15.26:0/3617874761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139015d020 con 0x7f139015eb00 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138e7fc640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1390143c40 0x7f139015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.577+0000 7f138f7fe640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 0x7f1390138330 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f1380035040 tx=0x7f13800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.578+0000 7f1377fff640 1 -- 172.21.15.26:0/3617874761 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380037040 con 0x7f139015eb00 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.578+0000 7f1377fff640 1 -- 172.21.15.26:0/3617874761 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1380042070 con 0x7f139015eb00 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.578+0000 7f1377fff640 1 -- 172.21.15.26:0/3617874761 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380030b50 con 0x7f139015eb00 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.578+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139015d300 con 0x7f139015eb00 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.578+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13901406f0 con 0x7f139015eb00 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.579+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13900694c0 con 0x7f139015eb00 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.579+0000 7f1377fff640 1 -- 172.21.15.26:0/3617874761 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1380030cf0 con 0x7f139015eb00 2024-04-24T23:28:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.580+0000 7f1377fff640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f137c06be30 0x7f137c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.580+0000 7f138effd640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f137c06be30 0x7f137c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.580+0000 7f1377fff640 1 -- 172.21.15.26:0/3617874761 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f13800ab8b0 con 0x7f139015eb00 2024-04-24T23:28:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.580+0000 7f138effd640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f137c06be30 0x7f137c06e2f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1388005c00 tx=0x7f1388009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.584+0000 7f1377fff640 1 -- 172.21.15.26:0/3617874761 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1380032e20 con 0x7f139015eb00 2024-04-24T23:28:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.769+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f139006ba20 con 0x7f137c06be30 2024-04-24T23:28:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.769+0000 7f1377fff640 1 -- 172.21.15.26:0/3617874761 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f139006ba20 con 0x7f137c06be30 2024-04-24T23:28:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 0 to deep-scrub 2024-04-24T23:28:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f137c06be30 msgr2=0x7f137c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f137c06be30 0x7f137c06e2f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1388005c00 tx=0x7f1388009040 comp rx=0 tx=0).stop 2024-04-24T23:28:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 msgr2=0x7f1390138330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 0x7f1390138330 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f1380035040 tx=0x7f13800046c0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 shutdown_connections 2024-04-24T23:28:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f137c06be30 0x7f137c06e2f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f139015eb00 0x7f1390138330 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1390143c40 0x7f139015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 --2- 172.21.15.26:0/3617874761 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f13900cf870 0x7f13901f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 >> 172.21.15.26:0/3617874761 conn(0x7f139006b300 msgr2=0x7f139013f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 shutdown_connections 2024-04-24T23:28:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.771+0000 7f1395963640 1 -- 172.21.15.26:0/3617874761 wait complete. 2024-04-24T23:28:22.783 INFO:tasks.ceph:Scrubbing osd.1 2024-04-24T23:28:22.783 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:28:22.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.887+0000 7f974122c640 1 Processor -- start 2024-04-24T23:28:22.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.887+0000 7f974122c640 1 -- start start 2024-04-24T23:28:22.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f974122c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f973c15ea80 0x7f973c141160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f974122c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f973c15ee60 0x7f973c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f974122c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f973c15b490 0x7f973c15b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f974122c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f973c0cf050 con 0x7f973c15b490 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f974122c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f973c0cf330 con 0x7f973c15ea80 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f974122c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f973c0cf610 con 0x7f973c15ee60 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973b59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f973c15b490 0x7f973c15b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973b59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f973c15b490 0x7f973c15b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60122/0 (socket says 172.21.15.26:60122) 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973b59e640 1 -- 172.21.15.26:0/3330589905 learned_addr learned my addr 172.21.15.26:0/3330589905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973ad9d640 1 --2- 172.21.15.26:0/3330589905 >> v2:172.21.15.29:3300/0 conn(0x7f973c15ea80 0x7f973c141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973a59c640 1 --2- 172.21.15.26:0/3330589905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f973c15ee60 0x7f973c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973b59e640 1 -- 172.21.15.26:0/3330589905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f973c15ee60 msgr2=0x7f973c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973b59e640 1 --2- 172.21.15.26:0/3330589905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f973c15ee60 0x7f973c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973b59e640 1 -- 172.21.15.26:0/3330589905 >> v2:172.21.15.29:3300/0 conn(0x7f973c15ea80 msgr2=0x7f973c141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973b59e640 1 --2- 172.21.15.26:0/3330589905 >> v2:172.21.15.29:3300/0 conn(0x7f973c15ea80 0x7f973c141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973b59e640 1 -- 172.21.15.26:0/3330589905 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c0cf8f0 con 0x7f973c15b490 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.888+0000 7f973ad9d640 1 --2- 172.21.15.26:0/3330589905 >> v2:172.21.15.29:3300/0 conn(0x7f973c15ea80 0x7f973c141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.889+0000 7f973b59e640 1 --2- 172.21.15.26:0/3330589905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 0x7f973c15b920 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f973c06ee00 tx=0x7f972c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182e361d230d177c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.889+0000 7f9739d9b640 1 -- 172.21.15.26:0/3330589905 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c036070 con 0x7f973c15b490 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.889+0000 7f9739d9b640 1 -- 172.21.15.26:0/3330589905 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972c031600 con 0x7f973c15b490 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.889+0000 7f9739d9b640 1 -- 172.21.15.26:0/3330589905 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c0319d0 con 0x7f973c15b490 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.890+0000 7f974122c640 1 -- 172.21.15.26:0/3330589905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 msgr2=0x7f973c15b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.890+0000 7f974122c640 1 --2- 172.21.15.26:0/3330589905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 0x7f973c15b920 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f973c06ee00 tx=0x7f972c02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.890+0000 7f974122c640 1 -- 172.21.15.26:0/3330589905 shutdown_connections 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.890+0000 7f974122c640 1 --2- 172.21.15.26:0/3330589905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 0x7f973c15b920 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.890+0000 7f974122c640 1 --2- 172.21.15.26:0/3330589905 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f973c15ee60 0x7f973c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.890+0000 7f974122c640 1 --2- 172.21.15.26:0/3330589905 >> v2:172.21.15.29:3300/0 conn(0x7f973c15ea80 0x7f973c141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.890+0000 7f974122c640 1 -- 172.21.15.26:0/3330589905 >> 172.21.15.26:0/3330589905 conn(0x7f973c06b300 msgr2=0x7f973c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:22.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.890+0000 7f974122c640 1 -- 172.21.15.26:0/3330589905 shutdown_connections 2024-04-24T23:28:22.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.890+0000 7f974122c640 1 -- 172.21.15.26:0/3330589905 wait complete. 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.891+0000 7f974122c640 1 Processor -- start 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.891+0000 7f974122c640 1 -- start start 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f974122c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 0x7f973c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f974122c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f973c15ea80 0x7f973c13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f974122c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f973c15ee60 0x7f973c1e7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f974122c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f973c15f2a0 con 0x7f973c15b490 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f974122c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f973c0cec80 con 0x7f973c15ee60 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f974122c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f973c15a290 con 0x7f973c15ea80 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 0x7f973c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 0x7f973c15e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60138/0 (socket says 172.21.15.26:60138) 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973ad9d640 1 -- 172.21.15.26:0/1575003539 learned_addr learned my addr 172.21.15.26:0/1575003539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973b59e640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f973c15ee60 0x7f973c1e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973a59c640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f973c15ea80 0x7f973c13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973ad9d640 1 -- 172.21.15.26:0/1575003539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f973c15ea80 msgr2=0x7f973c13d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973ad9d640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f973c15ea80 0x7f973c13d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973ad9d640 1 -- 172.21.15.26:0/1575003539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f973c15ee60 msgr2=0x7f973c1e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973ad9d640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f973c15ee60 0x7f973c1e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973ad9d640 1 -- 172.21.15.26:0/1575003539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c15e7b0 con 0x7f973c15b490 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.892+0000 7f973b59e640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f973c15ee60 0x7f973c1e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.893+0000 7f973ad9d640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 0x7f973c15e270 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f973400e840 tx=0x7f973400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.893+0000 7f97337fe640 1 -- 172.21.15.26:0/1575003539 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973400f040 con 0x7f973c15b490 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.893+0000 7f97337fe640 1 -- 172.21.15.26:0/1575003539 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9734004030 con 0x7f973c15b490 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.893+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973c15d1d0 con 0x7f973c15b490 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.893+0000 7f97337fe640 1 -- 172.21.15.26:0/1575003539 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734004400 con 0x7f973c15b490 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.893+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973c15d790 con 0x7f973c15b490 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.894+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f973c0694c0 con 0x7f973c15b490 2024-04-24T23:28:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.894+0000 7f97337fe640 1 -- 172.21.15.26:0/1575003539 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f973400c670 con 0x7f973c15b490 2024-04-24T23:28:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.895+0000 7f97337fe640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971c06be30 0x7f971c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.895+0000 7f97337fe640 1 -- 172.21.15.26:0/1575003539 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f973408a860 con 0x7f973c15b490 2024-04-24T23:28:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.895+0000 7f973a59c640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971c06be30 0x7f971c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.895+0000 7f97337fe640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f971c071e30 0x7f971c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.895+0000 7f97337fe640 1 -- 172.21.15.26:0/1575003539 --> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f973400fe50 con 0x7f971c071e30 2024-04-24T23:28:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.895+0000 7f97337fe640 1 -- 172.21.15.26:0/1575003539 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f9734091050 con 0x7f973c15b490 2024-04-24T23:28:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.895+0000 7f973b59e640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f971c071e30 0x7f971c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.895+0000 7f973a59c640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971c06be30 0x7f971c06e2f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9724006fd0 tx=0x7f9724008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.895+0000 7f973b59e640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f971c071e30 0x7f971c074250 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:22.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.896+0000 7f97337fe640 1 -- 172.21.15.26:0/1575003539 <== osd.1 v2:172.21.15.26:6816/1500554534 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f973400fe50 con 0x7f971c071e30 2024-04-24T23:28:22.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.909+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 --> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f973c072f00 con 0x7f971c071e30 2024-04-24T23:28:22.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.909+0000 7f97337fe640 1 -- 172.21.15.26:0/1575003539 <== osd.1 v2:172.21.15.26:6816/1500554534 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f973c072f00 con 0x7f971c071e30 2024-04-24T23:28:22.910 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:22.910 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T23:28:22.910 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:22.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f971c071e30 msgr2=0x7f971c074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6816/1500554534,v1:172.21.15.26:6817/1500554534] conn(0x7f971c071e30 0x7f971c074250 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971c06be30 msgr2=0x7f971c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f971c06be30 0x7f971c06e2f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9724006fd0 tx=0x7f9724008040 comp rx=0 tx=0).stop 2024-04-24T23:28:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 msgr2=0x7f973c15e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 --2- 172.21.15.26:0/1575003539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c15b490 0x7f973c15e270 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f973400e840 tx=0x7f973400ed00 comp rx=0 tx=0).stop 2024-04-24T23:28:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f973b59e640 1 -- 172.21.15.26:0/1575003539 reap_dead start 2024-04-24T23:28:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 shutdown_connections 2024-04-24T23:28:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 >> 172.21.15.26:0/1575003539 conn(0x7f973c06b300 msgr2=0x7f973c0721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 shutdown_connections 2024-04-24T23:28:22.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:22.910+0000 7f974122c640 1 -- 172.21.15.26:0/1575003539 wait complete. 2024-04-24T23:28:22.921 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-24T23:28:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.022+0000 7f962b288640 1 Processor -- start 2024-04-24T23:28:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.022+0000 7f962b288640 1 -- start start 2024-04-24T23:28:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.023+0000 7f962b288640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f96240cf870 0x7f9624072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.023+0000 7f962b288640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f962415eb00 0x7f962415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.023+0000 7f962b288640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f962415f450 con 0x7f9624143c40 2024-04-24T23:28:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.023+0000 7f962b288640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9624071b70 con 0x7f962415eb00 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.023+0000 7f962b288640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f96240cd0b0 con 0x7f96240cf870 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.023+0000 7f9629825640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f962415eb00 0x7f962415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.023+0000 7f9629024640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9624143c40 0x7f96240cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52946/0 (socket says 172.21.15.26:52946) 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.023+0000 7f9629024640 1 -- 172.21.15.26:0/3604998257 learned_addr learned my addr 172.21.15.26:0/3604998257 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.023+0000 7f9629825640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f962415eb00 0x7f962415ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60150/0 (socket says 172.21.15.26:60150) 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.024+0000 7f9629825640 1 -- 172.21.15.26:0/3604998257 >> v2:172.21.15.29:3300/0 conn(0x7f96240cf870 msgr2=0x7f9624072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.024+0000 7f9628823640 1 --2- 172.21.15.26:0/3604998257 >> v2:172.21.15.29:3300/0 conn(0x7f96240cf870 0x7f9624072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.024+0000 7f961bfff640 1 -- 172.21.15.26:0/3604998257 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 697436709 0 0) 0x7f961c001110 con 0x7f9624143c40 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.024+0000 7f9629825640 1 --2- 172.21.15.26:0/3604998257 >> v2:172.21.15.29:3300/0 conn(0x7f96240cf870 0x7f9624072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.024+0000 7f9629825640 1 -- 172.21.15.26:0/3604998257 >> v1:172.21.15.26:6789/0 conn(0x7f9624143c40 legacy=0x7f96240cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.024+0000 7f9629825640 1 -- 172.21.15.26:0/3604998257 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9624071240 con 0x7f962415eb00 2024-04-24T23:28:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.024+0000 7f9629825640 1 --2- 172.21.15.26:0/3604998257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f962415eb00 0x7f962415ef10 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f962406db50 tx=0x7f962002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1272f1d126f82dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f961bfff640 1 -- 172.21.15.26:0/3604998257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620007740 con 0x7f962415eb00 2024-04-24T23:28:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f961bfff640 1 -- 172.21.15.26:0/3604998257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96200078e0 con 0x7f962415eb00 2024-04-24T23:28:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f961bfff640 1 -- 172.21.15.26:0/3604998257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620007cb0 con 0x7f962415eb00 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f962b288640 1 -- 172.21.15.26:0/3604998257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f962415eb00 msgr2=0x7f962415ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f962b288640 1 --2- 172.21.15.26:0/3604998257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f962415eb00 0x7f962415ef10 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f962406db50 tx=0x7f962002a680 comp rx=0 tx=0).stop 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f962b288640 1 -- 172.21.15.26:0/3604998257 shutdown_connections 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f962b288640 1 --2- 172.21.15.26:0/3604998257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f962415eb00 0x7f962415ef10 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f962b288640 1 --2- 172.21.15.26:0/3604998257 >> v2:172.21.15.29:3300/0 conn(0x7f96240cf870 0x7f9624072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f962b288640 1 -- 172.21.15.26:0/3604998257 >> 172.21.15.26:0/3604998257 conn(0x7f962406b300 msgr2=0x7f962406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f962b288640 1 -- 172.21.15.26:0/3604998257 shutdown_connections 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.025+0000 7f962b288640 1 -- 172.21.15.26:0/3604998257 wait complete. 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.026+0000 7f962b288640 1 Processor -- start 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.026+0000 7f962b288640 1 -- start start 2024-04-24T23:28:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f962b288640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240cf870 0x7f962413f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f962b288640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9624143c40 0x7f962415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f962b288640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962415eb00 0x7f96240ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f962b288640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9624144020 con 0x7f96240cf870 2024-04-24T23:28:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f962b288640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9624071800 con 0x7f962415eb00 2024-04-24T23:28:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f962b288640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f9624072540 con 0x7f9624143c40 2024-04-24T23:28:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f9629024640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240cf870 0x7f962413f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f9629024640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240cf870 0x7f962413f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60156/0 (socket says 172.21.15.26:60156) 2024-04-24T23:28:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f9629024640 1 -- 172.21.15.26:0/866151120 learned_addr learned my addr 172.21.15.26:0/866151120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f9629825640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962415eb00 0x7f96240ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.027+0000 7f9628823640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9624143c40 0x7f962415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f9629024640 1 -- 172.21.15.26:0/866151120 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9624143c40 msgr2=0x7f962415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f9629024640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9624143c40 0x7f962415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f9629024640 1 -- 172.21.15.26:0/866151120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962415eb00 msgr2=0x7f96240ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f9629024640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962415eb00 0x7f96240ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f9629024640 1 -- 172.21.15.26:0/866151120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962415d0a0 con 0x7f96240cf870 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f9629024640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240cf870 0x7f962413f650 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f961c00c7b0 tx=0x7f961c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f9619ffb640 1 -- 172.21.15.26:0/866151120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c018070 con 0x7f96240cf870 2024-04-24T23:28:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f9619ffb640 1 -- 172.21.15.26:0/866151120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961c0072d0 con 0x7f96240cf870 2024-04-24T23:28:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f9619ffb640 1 -- 172.21.15.26:0/866151120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c0047e0 con 0x7f96240cf870 2024-04-24T23:28:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962415d360 con 0x7f96240cf870 2024-04-24T23:28:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.028+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962415d920 con 0x7f96240cf870 2024-04-24T23:28:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.029+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96240694c0 con 0x7f96240cf870 2024-04-24T23:28:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.030+0000 7f9619ffb640 1 -- 172.21.15.26:0/866151120 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f961c004980 con 0x7f96240cf870 2024-04-24T23:28:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.030+0000 7f9619ffb640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f960806be30 0x7f960806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.030+0000 7f9619ffb640 1 -- 172.21.15.26:0/866151120 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f961c089e60 con 0x7f96240cf870 2024-04-24T23:28:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.030+0000 7f9628823640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f960806be30 0x7f960806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.031+0000 7f9628823640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f960806be30 0x7f960806e2f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f962415f130 tx=0x7f9614008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.034+0000 7f9619ffb640 1 -- 172.21.15.26:0/866151120 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f962415f450 con 0x7f96240cf870 2024-04-24T23:28:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.221+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f962415e7c0 con 0x7f960806be30 2024-04-24T23:28:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.221+0000 7f9619ffb640 1 -- 172.21.15.26:0/866151120 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f962415e7c0 con 0x7f960806be30 2024-04-24T23:28:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 1 to deep-scrub 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f960806be30 msgr2=0x7f960806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f960806be30 0x7f960806e2f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f962415f130 tx=0x7f9614008040 comp rx=0 tx=0).stop 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240cf870 msgr2=0x7f962413f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240cf870 0x7f962413f650 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f961c00c7b0 tx=0x7f961c00cc70 comp rx=0 tx=0).stop 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 shutdown_connections 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f960806be30 0x7f960806e2f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f962415eb00 0x7f96240ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f9624143c40 0x7f962415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 --2- 172.21.15.26:0/866151120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96240cf870 0x7f962413f650 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 >> 172.21.15.26:0/866151120 conn(0x7f962406b300 msgr2=0x7f962406dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 shutdown_connections 2024-04-24T23:28:23.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.224+0000 7f962b288640 1 -- 172.21.15.26:0/866151120 wait complete. 2024-04-24T23:28:23.235 INFO:tasks.ceph:Scrubbing osd.2 2024-04-24T23:28:23.235 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:28:23.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.339+0000 7f34beb29640 1 Processor -- start 2024-04-24T23:28:23.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.339+0000 7f34beb29640 1 -- start start 2024-04-24T23:28:23.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.340+0000 7f34beb29640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f34b81608e0 0x7f34b80b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.340+0000 7f34beb29640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b80cd9f0 con 0x7f34b815f590 2024-04-24T23:28:23.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.340+0000 7f34beb29640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b80cdcd0 con 0x7f34b815fcd0 2024-04-24T23:28:23.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.340+0000 7f34beb29640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f34b806f8f0 con 0x7f34b81608e0 2024-04-24T23:28:23.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.340+0000 7f34bc8c5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f34b815f590 0x7f34b806f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52958/0 (socket says 172.21.15.26:52958) 2024-04-24T23:28:23.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.340+0000 7f34bc8c5640 1 -- 172.21.15.26:0/3404078432 learned_addr learned my addr 172.21.15.26:0/3404078432 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.340+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 688098368 0 0) 0x7f34b80cd9f0 con 0x7f34b815f590 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.340+0000 7f34b7fff640 1 --2- 172.21.15.26:0/3404078432 >> v2:172.21.15.29:3300/0 conn(0x7f34b81608e0 0x7f34b80b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.340+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34a4003520 con 0x7f34b815f590 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 803220159 0 0) 0x7f34a4003520 con 0x7f34b815f590 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 >> v2:172.21.15.29:3300/0 conn(0x7f34b81608e0 msgr2=0x7f34b80b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 --2- 172.21.15.26:0/3404078432 >> v2:172.21.15.29:3300/0 conn(0x7f34b81608e0 0x7f34b80b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 >> v1:172.21.15.29:6789/0 conn(0x7f34b815fcd0 legacy=0x7f34b80d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b80cfc40 con 0x7f34b815f590 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3767943540 0 0) 0x7f34b00019f0 con 0x7f34b815f590 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 >> v1:172.21.15.26:6789/0 conn(0x7f34b815f590 legacy=0x7f34b806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34a4004800 0x7f34a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34a40051c0 0x7f34a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34b806f8f0 con 0x7f34a40039b0 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34a400c160 con 0x7f34a40051c0 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f34a400c360 con 0x7f34a4004800 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34bc8c5640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34b7fff640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34a4004800 0x7f34a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34bd0c6640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34a40051c0 0x7f34a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34bc8c5640 1 -- 172.21.15.26:0/3404078432 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34a4004800 msgr2=0x7f34a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34bc8c5640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34a4004800 0x7f34a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34bc8c5640 1 -- 172.21.15.26:0/3404078432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34a40051c0 msgr2=0x7f34a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34bc8c5640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34a40051c0 0x7f34a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34bc8c5640 1 -- 172.21.15.26:0/3404078432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a4003520 con 0x7f34a40039b0 2024-04-24T23:28:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.341+0000 7f34bd0c6640 1 -- 172.21.15.26:0/3404078432 reap_dead start 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.342+0000 7f34bc8c5640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34a4003dd0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f34b000a850 tx=0x7f34a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994df10532dfe4e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.342+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b000a4a0 con 0x7f34a40039b0 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.342+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b002f070 con 0x7f34a40039b0 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.342+0000 7f34b77fe640 1 -- 172.21.15.26:0/3404078432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b002bd20 con 0x7f34a40039b0 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.343+0000 7f34beb29640 1 -- 172.21.15.26:0/3404078432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 msgr2=0x7f34a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.343+0000 7f34beb29640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34a4003dd0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f34b000a850 tx=0x7f34a4005830 comp rx=0 tx=0).stop 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.343+0000 7f34beb29640 1 -- 172.21.15.26:0/3404078432 shutdown_connections 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.343+0000 7f34beb29640 1 --2- 172.21.15.26:0/3404078432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34a4003dd0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.343+0000 7f34beb29640 1 -- 172.21.15.26:0/3404078432 >> 172.21.15.26:0/3404078432 conn(0x7f34b80b39e0 msgr2=0x7f34b80b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.343+0000 7f34beb29640 1 -- 172.21.15.26:0/3404078432 shutdown_connections 2024-04-24T23:28:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.343+0000 7f34beb29640 1 -- 172.21.15.26:0/3404078432 wait complete. 2024-04-24T23:28:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.344+0000 7f34beb29640 1 Processor -- start 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.344+0000 7f34beb29640 1 -- start start 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34beb29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34b80c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34beb29640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34b814aff0 0x7f34b8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34beb29640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34b814b470 0x7f34b815a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34beb29640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34b80cd5d0 con 0x7f34a40039b0 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34beb29640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34b815f970 con 0x7f34b814b470 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34beb29640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f34b80cd750 con 0x7f34b814aff0 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bc8c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34b80c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bc8c5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34b80c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60180/0 (socket says 172.21.15.26:60180) 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bc8c5640 1 -- 172.21.15.26:0/905417929 learned_addr learned my addr 172.21.15.26:0/905417929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bd0c6640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34b814b470 0x7f34b815a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34b7fff640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34b814aff0 0x7f34b8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bc8c5640 1 -- 172.21.15.26:0/905417929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34b814aff0 msgr2=0x7f34b8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bc8c5640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f34b814aff0 0x7f34b8159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bc8c5640 1 -- 172.21.15.26:0/905417929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34b814b470 msgr2=0x7f34b815a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bc8c5640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34b814b470 0x7f34b815a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bc8c5640 1 -- 172.21.15.26:0/905417929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b80c7060 con 0x7f34a40039b0 2024-04-24T23:28:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34bc8c5640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34b80c68c0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f34b002b530 tx=0x7f34b002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34b57fa640 1 -- 172.21.15.26:0/905417929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0045070 con 0x7f34a40039b0 2024-04-24T23:28:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34b57fa640 1 -- 172.21.15.26:0/905417929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b0040680 con 0x7f34a40039b0 2024-04-24T23:28:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.345+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b80c7340 con 0x7f34a40039b0 2024-04-24T23:28:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.346+0000 7f34b57fa640 1 -- 172.21.15.26:0/905417929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0040a70 con 0x7f34a40039b0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.346+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b80c7900 con 0x7f34a40039b0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.346+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34b80c6e00 con 0x7f34a40039b0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.347+0000 7f34b57fa640 1 -- 172.21.15.26:0/905417929 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34b0048070 con 0x7f34a40039b0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.347+0000 7f34b57fa640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f349806be60 0x7f349806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.348+0000 7f34b57fa640 1 -- 172.21.15.26:0/905417929 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f34b00ab880 con 0x7f34a40039b0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.348+0000 7f34b7fff640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f349806be60 0x7f349806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.348+0000 7f34b57fa640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7f3498071e60 0x7f3498074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.348+0000 7f34b57fa640 1 -- 172.21.15.26:0/905417929 --> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34b00059c0 con 0x7f3498071e60 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.348+0000 7f34b57fa640 1 -- 172.21.15.26:0/905417929 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f34b00abd10 con 0x7f34a40039b0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.348+0000 7f34bd0c6640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7f3498071e60 0x7f3498074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.348+0000 7f34b7fff640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f349806be60 0x7f349806e320 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f34b80cf4d0 tx=0x7f34a0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.348+0000 7f34bd0c6640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7f3498071e60 0x7f3498074280 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.349+0000 7f34b57fa640 1 -- 172.21.15.26:0/905417929 <== osd.2 v2:172.21.15.26:6800/3780187135 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34b00059c0 con 0x7f3498071e60 2024-04-24T23:28:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.362+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 --> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f34b80694c0 con 0x7f3498071e60 2024-04-24T23:28:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.362+0000 7f34b57fa640 1 -- 172.21.15.26:0/905417929 <== osd.2 v2:172.21.15.26:6800/3780187135 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f34b80694c0 con 0x7f3498071e60 2024-04-24T23:28:23.364 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:23.364 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T23:28:23.364 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.362+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7f3498071e60 msgr2=0x7f3498074280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.362+0000 7f34beb29640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6800/3780187135,v1:172.21.15.26:6801/3780187135] conn(0x7f3498071e60 0x7f3498074280 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.363+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f349806be60 msgr2=0x7f349806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.363+0000 7f34beb29640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f349806be60 0x7f349806e320 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f34b80cf4d0 tx=0x7f34a0009190 comp rx=0 tx=0).stop 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.363+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 msgr2=0x7f34b80c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.363+0000 7f34beb29640 1 --2- 172.21.15.26:0/905417929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a40039b0 0x7f34b80c68c0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f34b002b530 tx=0x7f34b002d070 comp rx=0 tx=0).stop 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.363+0000 7f34bd0c6640 1 -- 172.21.15.26:0/905417929 reap_dead start 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.363+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 shutdown_connections 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.363+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 >> 172.21.15.26:0/905417929 conn(0x7f34b80b39e0 msgr2=0x7f34b80b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.363+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 shutdown_connections 2024-04-24T23:28:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.363+0000 7f34beb29640 1 -- 172.21.15.26:0/905417929 wait complete. 2024-04-24T23:28:23.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-24T23:28:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.476+0000 7fc6bb570640 1 Processor -- start 2024-04-24T23:28:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.476+0000 7fc6bb570640 1 -- start start 2024-04-24T23:28:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6bb570640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b4143c40 0x7fc6b40cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6bb570640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fc6b40cc860 0x7fc6b4072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6bb570640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6b415eb00 0x7fc6b415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6bb570640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc6b415f490 con 0x7fc6b415eb00 2024-04-24T23:28:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6bb570640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fc6b4071b70 con 0x7fc6b40cc860 2024-04-24T23:28:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6bb570640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b40cd0b0 con 0x7fc6b4143c40 2024-04-24T23:28:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b9b0d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6b415eb00 0x7fc6b415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b9b0d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6b415eb00 0x7fc6b415ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60184/0 (socket says 172.21.15.26:60184) 2024-04-24T23:28:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b9b0d640 1 -- 172.21.15.26:0/636752110 learned_addr learned my addr 172.21.15.26:0/636752110 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b8b0b640 1 --2- 172.21.15.26:0/636752110 >> v2:172.21.15.29:3300/0 conn(0x7fc6b40cc860 0x7fc6b4072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b930c640 1 --2- 172.21.15.26:0/636752110 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b4143c40 0x7fc6b40cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b9b0d640 1 -- 172.21.15.26:0/636752110 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b4143c40 msgr2=0x7fc6b40cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b9b0d640 1 --2- 172.21.15.26:0/636752110 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b4143c40 0x7fc6b40cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b9b0d640 1 -- 172.21.15.26:0/636752110 >> v2:172.21.15.29:3300/0 conn(0x7fc6b40cc860 msgr2=0x7fc6b4072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b9b0d640 1 --2- 172.21.15.26:0/636752110 >> v2:172.21.15.29:3300/0 conn(0x7fc6b40cc860 0x7fc6b4072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b9b0d640 1 -- 172.21.15.26:0/636752110 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b4071240 con 0x7fc6b415eb00 2024-04-24T23:28:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.477+0000 7fc6b930c640 1 --2- 172.21.15.26:0/636752110 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b4143c40 0x7fc6b40cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.478+0000 7fc6b9b0d640 1 --2- 172.21.15.26:0/636752110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 0x7fc6b415ef50 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fc6b406ee60 tx=0x7fc6ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa4bb0751543ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.478+0000 7fc6b3fff640 1 -- 172.21.15.26:0/636752110 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac036070 con 0x7fc6b415eb00 2024-04-24T23:28:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.478+0000 7fc6b3fff640 1 -- 172.21.15.26:0/636752110 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6ac031600 con 0x7fc6b415eb00 2024-04-24T23:28:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.478+0000 7fc6b3fff640 1 -- 172.21.15.26:0/636752110 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0319d0 con 0x7fc6b415eb00 2024-04-24T23:28:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.478+0000 7fc6bb570640 1 -- 172.21.15.26:0/636752110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 msgr2=0x7fc6b415ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.478+0000 7fc6bb570640 1 --2- 172.21.15.26:0/636752110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 0x7fc6b415ef50 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fc6b406ee60 tx=0x7fc6ac02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.479+0000 7fc6bb570640 1 -- 172.21.15.26:0/636752110 shutdown_connections 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.479+0000 7fc6bb570640 1 --2- 172.21.15.26:0/636752110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 0x7fc6b415ef50 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.479+0000 7fc6bb570640 1 --2- 172.21.15.26:0/636752110 >> v2:172.21.15.29:3300/0 conn(0x7fc6b40cc860 0x7fc6b4072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.479+0000 7fc6bb570640 1 --2- 172.21.15.26:0/636752110 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b4143c40 0x7fc6b40cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.479+0000 7fc6bb570640 1 -- 172.21.15.26:0/636752110 >> 172.21.15.26:0/636752110 conn(0x7fc6b406b300 msgr2=0x7fc6b406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.479+0000 7fc6bb570640 1 -- 172.21.15.26:0/636752110 shutdown_connections 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.479+0000 7fc6bb570640 1 -- 172.21.15.26:0/636752110 wait complete. 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.480+0000 7fc6bb570640 1 Processor -- start 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.480+0000 7fc6bb570640 1 -- start start 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.480+0000 7fc6bb570640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b40cc860 0x7fc6b413aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.480+0000 7fc6bb570640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6b4143c40 0x7fc6b4139a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.480+0000 7fc6bb570640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 0x7fc6b4141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.480+0000 7fc6bb570640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4144020 con 0x7fc6b415eb00 2024-04-24T23:28:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6bb570640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4071800 con 0x7fc6b4143c40 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6bb570640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b4072540 con 0x7fc6b40cc860 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b9b0d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 0x7fc6b4141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b9b0d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 0x7fc6b4141820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60196/0 (socket says 172.21.15.26:60196) 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b9b0d640 1 -- 172.21.15.26:0/3176798157 learned_addr learned my addr 172.21.15.26:0/3176798157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b8b0b640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6b4143c40 0x7fc6b4139a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b9b0d640 1 -- 172.21.15.26:0/3176798157 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b40cc860 msgr2=0x7fc6b413aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b9b0d640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b40cc860 0x7fc6b413aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b9b0d640 1 -- 172.21.15.26:0/3176798157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6b4143c40 msgr2=0x7fc6b4139a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b9b0d640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6b4143c40 0x7fc6b4139a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b9b0d640 1 -- 172.21.15.26:0/3176798157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b413c120 con 0x7fc6b415eb00 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b9b0d640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 0x7fc6b4141820 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac02ac90 tx=0x7fc6ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b1ffb640 1 -- 172.21.15.26:0/3176798157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac03f020 con 0x7fc6b415eb00 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b1ffb640 1 -- 172.21.15.26:0/3176798157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6ac037040 con 0x7fc6b415eb00 2024-04-24T23:28:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b413b080 con 0x7fc6b415eb00 2024-04-24T23:28:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6b1ffb640 1 -- 172.21.15.26:0/3176798157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac036070 con 0x7fc6b415eb00 2024-04-24T23:28:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.481+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b413b640 con 0x7fc6b415eb00 2024-04-24T23:28:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.483+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b40ccca0 con 0x7fc6b415eb00 2024-04-24T23:28:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.483+0000 7fc6b1ffb640 1 -- 172.21.15.26:0/3176798157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc6ac042020 con 0x7fc6b415eb00 2024-04-24T23:28:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.483+0000 7fc6b1ffb640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc6a006be30 0x7fc6a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.484+0000 7fc6b930c640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc6a006be30 0x7fc6a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.484+0000 7fc6b1ffb640 1 -- 172.21.15.26:0/3176798157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7fc6ac0ab060 con 0x7fc6b415eb00 2024-04-24T23:28:23.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.484+0000 7fc6b930c640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc6a006be30 0x7fc6a006e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4006fd0 tx=0x7fc6a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.488+0000 7fc6b1ffb640 1 -- 172.21.15.26:0/3176798157 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc6b40ccca0 con 0x7fc6b415eb00 2024-04-24T23:28:23.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.672+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fc6b40ce1f0 con 0x7fc6a006be30 2024-04-24T23:28:23.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.672+0000 7fc6b1ffb640 1 -- 172.21.15.26:0/3176798157 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc6b40ce1f0 con 0x7fc6a006be30 2024-04-24T23:28:23.673 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 2 to deep-scrub 2024-04-24T23:28:23.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc6a006be30 msgr2=0x7fc6a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc6a006be30 0x7fc6a006e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4006fd0 tx=0x7fc6a4008040 comp rx=0 tx=0).stop 2024-04-24T23:28:23.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 msgr2=0x7fc6b4141820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 0x7fc6b4141820 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac02ac90 tx=0x7fc6ac004a40 comp rx=0 tx=0).stop 2024-04-24T23:28:23.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 shutdown_connections 2024-04-24T23:28:23.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fc6a006be30 0x7fc6a006e2f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6b415eb00 0x7fc6b4141820 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6b4143c40 0x7fc6b4139a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 --2- 172.21.15.26:0/3176798157 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fc6b40cc860 0x7fc6b413aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 >> 172.21.15.26:0/3176798157 conn(0x7fc6b406b300 msgr2=0x7fc6b415ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 shutdown_connections 2024-04-24T23:28:23.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.674+0000 7fc6bb570640 1 -- 172.21.15.26:0/3176798157 wait complete. 2024-04-24T23:28:23.686 INFO:tasks.ceph:Scrubbing osd.3 2024-04-24T23:28:23.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:28:23.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.790+0000 7f7eb22d4640 1 Processor -- start 2024-04-24T23:28:23.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.790+0000 7f7eb22d4640 1 -- start start 2024-04-24T23:28:23.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eb22d4640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f7eac06de50 0x7f7eac0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eb22d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7eac0b4f00 0x7f7eac154670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eb22d4640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eac0cde40 con 0x7f7eac06e230 2024-04-24T23:28:23.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eb22d4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7eac0b5380 con 0x7f7eac0b4f00 2024-04-24T23:28:23.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eb22d4640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f7eac15ace0 con 0x7f7eac06de50 2024-04-24T23:28:23.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eab7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7eac0b4f00 0x7f7eac154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eab7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7eac0b4f00 0x7f7eac154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58266/0 (socket says 172.21.15.26:58266) 2024-04-24T23:28:23.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eab7fe640 1 -- 172.21.15.26:0/3397192704 learned_addr learned my addr 172.21.15.26:0/3397192704 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:23.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eabfff640 1 --2- 172.21.15.26:0/3397192704 >> v2:172.21.15.29:3300/0 conn(0x7f7eac06de50 0x7f7eac0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eab7fe640 1 -- 172.21.15.26:0/3397192704 >> v2:172.21.15.29:3300/0 conn(0x7f7eac06de50 msgr2=0x7f7eac0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:23.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eab7fe640 1 --2- 172.21.15.26:0/3397192704 >> v2:172.21.15.29:3300/0 conn(0x7f7eac06de50 0x7f7eac0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eab7fe640 1 -- 172.21.15.26:0/3397192704 >> v1:172.21.15.29:6789/0 conn(0x7f7eac06e230 legacy=0x7f7eac070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.791+0000 7f7eab7fe640 1 -- 172.21.15.26:0/3397192704 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac15afc0 con 0x7f7eac0b4f00 2024-04-24T23:28:23.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.792+0000 7f7eab7fe640 1 --2- 172.21.15.26:0/3397192704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 0x7f7eac154670 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f7eac072180 tx=0x7f7e9c02b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8586e88921ffa71c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.792+0000 7f7eaaffd640 1 -- 172.21.15.26:0/3397192704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c038070 con 0x7f7eac0b4f00 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.792+0000 7f7eaaffd640 1 -- 172.21.15.26:0/3397192704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c033640 con 0x7f7eac0b4f00 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.792+0000 7f7eaaffd640 1 -- 172.21.15.26:0/3397192704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c033a10 con 0x7f7eac0b4f00 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.793+0000 7f7eb22d4640 1 -- 172.21.15.26:0/3397192704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 msgr2=0x7f7eac154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.793+0000 7f7eb22d4640 1 --2- 172.21.15.26:0/3397192704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 0x7f7eac154670 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f7eac072180 tx=0x7f7e9c02b380 comp rx=0 tx=0).stop 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.793+0000 7f7eb22d4640 1 -- 172.21.15.26:0/3397192704 shutdown_connections 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.793+0000 7f7eb22d4640 1 --2- 172.21.15.26:0/3397192704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 0x7f7eac154670 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.793+0000 7f7eb22d4640 1 --2- 172.21.15.26:0/3397192704 >> v2:172.21.15.29:3300/0 conn(0x7f7eac06de50 0x7f7eac0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.793+0000 7f7eb22d4640 1 -- 172.21.15.26:0/3397192704 >> 172.21.15.26:0/3397192704 conn(0x7f7eac06b300 msgr2=0x7f7eac06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.793+0000 7f7eb22d4640 1 -- 172.21.15.26:0/3397192704 shutdown_connections 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.793+0000 7f7eb22d4640 1 -- 172.21.15.26:0/3397192704 wait complete. 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.794+0000 7f7eb22d4640 1 Processor -- start 2024-04-24T23:28:23.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.794+0000 7f7eb22d4640 1 -- start start 2024-04-24T23:28:23.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb22d4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac06de50 0x7f7eac14d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb22d4640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7eac06e230 0x7f7eac1e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb22d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 0x7f7eac1e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb22d4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac15baa0 con 0x7f7eac0b4f00 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb22d4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac15b580 con 0x7f7eac06de50 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb22d4640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac072bc0 con 0x7f7eac06e230 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb0871640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 0x7f7eac1e34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb0871640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 0x7f7eac1e34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58268/0 (socket says 172.21.15.26:58268) 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb0871640 1 -- 172.21.15.26:0/184815957 learned_addr learned my addr 172.21.15.26:0/184815957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eabfff640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac06de50 0x7f7eac14d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eab7fe640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7eac06e230 0x7f7eac1e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb0871640 1 -- 172.21.15.26:0/184815957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7eac06e230 msgr2=0x7f7eac1e2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb0871640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f7eac06e230 0x7f7eac1e2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb0871640 1 -- 172.21.15.26:0/184815957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac06de50 msgr2=0x7f7eac14d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb0871640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eac06de50 0x7f7eac14d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb0871640 1 -- 172.21.15.26:0/184815957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac14d6c0 con 0x7f7eac0b4f00 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.795+0000 7f7eb0871640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 0x7f7eac1e34f0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c730 tx=0x7f7ea400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.796+0000 7f7ea8ff9640 1 -- 172.21.15.26:0/184815957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4003b70 con 0x7f7eac0b4f00 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.796+0000 7f7ea8ff9640 1 -- 172.21.15.26:0/184815957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea4003d10 con 0x7f7eac0b4f00 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.796+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac0ab710 con 0x7f7eac0b4f00 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.796+0000 7f7ea8ff9640 1 -- 172.21.15.26:0/184815957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400aa90 con 0x7f7eac0b4f00 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.796+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac0abcd0 con 0x7f7eac0b4f00 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.796+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7eac0694c0 con 0x7f7eac0b4f00 2024-04-24T23:28:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.797+0000 7f7ea8ff9640 1 -- 172.21.15.26:0/184815957 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ea400ac30 con 0x7f7eac0b4f00 2024-04-24T23:28:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.797+0000 7f7ea8ff9640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7e9806be30 0x7f7e9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.798+0000 7f7ea8ff9640 1 -- 172.21.15.26:0/184815957 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f7ea408a950 con 0x7f7eac0b4f00 2024-04-24T23:28:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.798+0000 7f7eabfff640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7e9806be30 0x7f7e9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.798+0000 7f7ea8ff9640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f7e98071e30 0x7f7e98074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.798+0000 7f7ea8ff9640 1 -- 172.21.15.26:0/184815957 --> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ea4010e50 con 0x7f7e98071e30 2024-04-24T23:28:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.798+0000 7f7ea8ff9640 1 -- 172.21.15.26:0/184815957 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f7ea408f2c0 con 0x7f7eac0b4f00 2024-04-24T23:28:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.798+0000 7f7eab7fe640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f7e98071e30 0x7f7e98074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.798+0000 7f7eabfff640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7e9806be30 0x7f7e9806e2f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006fd0 tx=0x7f7ea0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.798+0000 7f7eab7fe640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f7e98071e30 0x7f7e98074250 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.799+0000 7f7ea8ff9640 1 -- 172.21.15.26:0/184815957 <== osd.3 v2:172.21.15.26:6808/2035350946 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ea4010e50 con 0x7f7e98071e30 2024-04-24T23:28:23.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.812+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 --> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7eac0ac160 con 0x7f7e98071e30 2024-04-24T23:28:23.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.812+0000 7f7ea8ff9640 1 -- 172.21.15.26:0/184815957 <== osd.3 v2:172.21.15.26:6808/2035350946 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7eac0ac160 con 0x7f7e98071e30 2024-04-24T23:28:23.815 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:23.815 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T23:28:23.815 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.812+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f7e98071e30 msgr2=0x7f7e98074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.812+0000 7f7eb22d4640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6808/2035350946,v1:172.21.15.26:6809/2035350946] conn(0x7f7e98071e30 0x7f7e98074250 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.813+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7e9806be30 msgr2=0x7f7e9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.813+0000 7f7eb22d4640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f7e9806be30 0x7f7e9806e2f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006fd0 tx=0x7f7ea0008040 comp rx=0 tx=0).stop 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.813+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 msgr2=0x7f7eac1e34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.813+0000 7f7eb22d4640 1 --2- 172.21.15.26:0/184815957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eac0b4f00 0x7f7eac1e34f0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c730 tx=0x7f7ea400cbf0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.813+0000 7f7eb0871640 1 -- 172.21.15.26:0/184815957 reap_dead start 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.813+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 shutdown_connections 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.813+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 >> 172.21.15.26:0/184815957 conn(0x7f7eac06b300 msgr2=0x7f7eac153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.813+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 shutdown_connections 2024-04-24T23:28:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.813+0000 7f7eb22d4640 1 -- 172.21.15.26:0/184815957 wait complete. 2024-04-24T23:28:23.824 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-24T23:28:23.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.925+0000 7f2929942640 1 Processor -- start 2024-04-24T23:28:23.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.925+0000 7f2929942640 1 -- start start 2024-04-24T23:28:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.925+0000 7f2929942640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f292415e290 0x7f29240d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f2929942640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292406d050 con 0x7f29240b72f0 2024-04-24T23:28:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f2929942640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29240b5840 con 0x7f29240b3ac0 2024-04-24T23:28:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f2929942640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f29240b5e50 con 0x7f292415e290 2024-04-24T23:28:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f2922ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f292415e290 0x7f29240d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f29237fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f29240b72f0 0x7f292406c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45508/0 (socket says 172.21.15.26:45508) 2024-04-24T23:28:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f29237fe640 1 -- 172.21.15.26:0/3004615222 learned_addr learned my addr 172.21.15.26:0/3004615222 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f2922ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f292415e290 0x7f29240d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58282/0 (socket says 172.21.15.26:58282) 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f2922ffd640 1 -- 172.21.15.26:0/3004615222 >> v1:172.21.15.29:6789/0 conn(0x7f29240b3ac0 legacy=0x7f292406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f2922ffd640 1 -- 172.21.15.26:0/3004615222 >> v1:172.21.15.26:6789/0 conn(0x7f29240b72f0 legacy=0x7f292406c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.926+0000 7f2922ffd640 1 -- 172.21.15.26:0/3004615222 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29240cfb30 con 0x7f292415e290 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f2922ffd640 1 --2- 172.21.15.26:0/3004615222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f292415e290 0x7f29240d1730 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f29140097a0 tx=0x7f291402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b24b8d70334c6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f29227fc640 1 -- 172.21.15.26:0/3004615222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914037070 con 0x7f292415e290 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f29227fc640 1 -- 172.21.15.26:0/3004615222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914032440 con 0x7f292415e290 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f29227fc640 1 -- 172.21.15.26:0/3004615222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914032810 con 0x7f292415e290 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f2929942640 1 -- 172.21.15.26:0/3004615222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f292415e290 msgr2=0x7f29240d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f2929942640 1 --2- 172.21.15.26:0/3004615222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f292415e290 0x7f29240d1730 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f29140097a0 tx=0x7f291402a230 comp rx=0 tx=0).stop 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f2929942640 1 -- 172.21.15.26:0/3004615222 shutdown_connections 2024-04-24T23:28:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f2929942640 1 --2- 172.21.15.26:0/3004615222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f292415e290 0x7f29240d1730 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f2929942640 1 -- 172.21.15.26:0/3004615222 >> 172.21.15.26:0/3004615222 conn(0x7f292406b120 msgr2=0x7f292406b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.927+0000 7f2929942640 1 -- 172.21.15.26:0/3004615222 shutdown_connections 2024-04-24T23:28:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.928+0000 7f2929942640 1 -- 172.21.15.26:0/3004615222 wait complete. 2024-04-24T23:28:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.929+0000 7f2929942640 1 Processor -- start 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.929+0000 7f2929942640 1 -- start start 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2929942640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29240b3ac0 0x7f29241f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2929942640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29240b72f0 0x7f29240b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2929942640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f292415e290 0x7f29240b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2929942640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f29240b5ae0 con 0x7f29240b3ac0 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2929942640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f29240d00f0 con 0x7f29240b72f0 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2929942640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f292415e670 con 0x7f292415e290 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2922ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29240b3ac0 0x7f29241f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2922ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29240b3ac0 0x7f29241f33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58292/0 (socket says 172.21.15.26:58292) 2024-04-24T23:28:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2922ffd640 1 -- 172.21.15.26:0/3344343847 learned_addr learned my addr 172.21.15.26:0/3344343847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f291a7fc640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29240b72f0 0x7f29240b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f29237fe640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f292415e290 0x7f29240b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2922ffd640 1 -- 172.21.15.26:0/3344343847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f292415e290 msgr2=0x7f29240b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2922ffd640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f292415e290 0x7f29240b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2922ffd640 1 -- 172.21.15.26:0/3344343847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29240b72f0 msgr2=0x7f29240b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2922ffd640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29240b72f0 0x7f29240b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2922ffd640 1 -- 172.21.15.26:0/3344343847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29241f38f0 con 0x7f29240b3ac0 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f291a7fc640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29240b72f0 0x7f29240b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2922ffd640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29240b3ac0 0x7f29241f33b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f2914035040 tx=0x7f29140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2920ff9640 1 -- 172.21.15.26:0/3344343847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914037040 con 0x7f29240b3ac0 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2920ff9640 1 -- 172.21.15.26:0/3344343847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914030770 con 0x7f29240b3ac0 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29240cb090 con 0x7f29240b3ac0 2024-04-24T23:28:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2920ff9640 1 -- 172.21.15.26:0/3344343847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914030b80 con 0x7f29240b3ac0 2024-04-24T23:28:23.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.930+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29240ca5b0 con 0x7f29240b3ac0 2024-04-24T23:28:23.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.931+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29240694c0 con 0x7f29240b3ac0 2024-04-24T23:28:23.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.932+0000 7f2920ff9640 1 -- 172.21.15.26:0/3344343847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2914044020 con 0x7f29240b3ac0 2024-04-24T23:28:23.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.932+0000 7f2920ff9640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f290406be30 0x7f290406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:23.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.932+0000 7f2920ff9640 1 -- 172.21.15.26:0/3344343847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f29140ab1b0 con 0x7f29240b3ac0 2024-04-24T23:28:23.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.932+0000 7f291a7fc640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f290406be30 0x7f290406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:23.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.933+0000 7f291a7fc640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f290406be30 0x7f290406e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f29240b3910 tx=0x7f290c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:23.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:23.937+0000 7f2920ff9640 1 -- 172.21.15.26:0/3344343847 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f29241f38f0 con 0x7f29240b3ac0 2024-04-24T23:28:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.120+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f29240ca080 con 0x7f290406be30 2024-04-24T23:28:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.120+0000 7f2920ff9640 1 -- 172.21.15.26:0/3344343847 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f29240ca080 con 0x7f290406be30 2024-04-24T23:28:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 3 to deep-scrub 2024-04-24T23:28:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f290406be30 msgr2=0x7f290406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f290406be30 0x7f290406e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f29240b3910 tx=0x7f290c009040 comp rx=0 tx=0).stop 2024-04-24T23:28:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29240b3ac0 msgr2=0x7f29241f33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29240b3ac0 0x7f29241f33b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f2914035040 tx=0x7f29140046c0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 shutdown_connections 2024-04-24T23:28:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f290406be30 0x7f290406e2f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f292415e290 0x7f29240b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f29240b72f0 0x7f29240b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 --2- 172.21.15.26:0/3344343847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29240b3ac0 0x7f29241f33b0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 >> 172.21.15.26:0/3344343847 conn(0x7f292406b120 msgr2=0x7f29240c8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 shutdown_connections 2024-04-24T23:28:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.122+0000 7f2929942640 1 -- 172.21.15.26:0/3344343847 wait complete. 2024-04-24T23:28:24.134 INFO:tasks.ceph:Scrubbing osd.4 2024-04-24T23:28:24.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:28:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.241+0000 7fb39d46b640 1 Processor -- start 2024-04-24T23:28:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.241+0000 7fb39d46b640 1 -- start start 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb39d46b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3980721b0 0x7fb398165e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb39d46b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3980cf050 con 0x7fb398141160 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb39d46b640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3980cf330 con 0x7fb39815ea80 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb39d46b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb3980cf610 con 0x7fb3980721b0 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb3977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3980721b0 0x7fb398165e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb3977fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3980721b0 0x7fb398165e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58294/0 (socket says 172.21.15.26:58294) 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb3977fe640 1 -- 172.21.15.26:0/3577806272 learned_addr learned my addr 172.21.15.26:0/3577806272 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb38ffff640 1 --1- 172.21.15.26:0/3577806272 >> v1:172.21.15.26:6789/0 conn(0x7fb398141160 0x7fb3980ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45520/0 (socket says 172.21.15.26:45520) 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb3977fe640 1 -- 172.21.15.26:0/3577806272 >> v1:172.21.15.29:6789/0 conn(0x7fb39815ea80 legacy=0x7fb39815ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb3977fe640 1 -- 172.21.15.26:0/3577806272 >> v1:172.21.15.26:6789/0 conn(0x7fb398141160 legacy=0x7fb3980ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.242+0000 7fb3977fe640 1 -- 172.21.15.26:0/3577806272 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3980cf8f0 con 0x7fb3980721b0 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.243+0000 7fb3977fe640 1 --2- 172.21.15.26:0/3577806272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 0x7fb398165e10 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fb390009e50 tx=0x7fb39002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd72c42fbd94b984 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.243+0000 7fb3967fc640 1 -- 172.21.15.26:0/3577806272 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390036070 con 0x7fb3980721b0 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.243+0000 7fb3967fc640 1 -- 172.21.15.26:0/3577806272 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb390031600 con 0x7fb3980721b0 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.243+0000 7fb3967fc640 1 -- 172.21.15.26:0/3577806272 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3900319d0 con 0x7fb3980721b0 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.244+0000 7fb39d46b640 1 -- 172.21.15.26:0/3577806272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 msgr2=0x7fb398165e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.244+0000 7fb39d46b640 1 --2- 172.21.15.26:0/3577806272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 0x7fb398165e10 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fb390009e50 tx=0x7fb39002a760 comp rx=0 tx=0).stop 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.244+0000 7fb39d46b640 1 -- 172.21.15.26:0/3577806272 shutdown_connections 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.244+0000 7fb39d46b640 1 --2- 172.21.15.26:0/3577806272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 0x7fb398165e10 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.244+0000 7fb39d46b640 1 -- 172.21.15.26:0/3577806272 >> 172.21.15.26:0/3577806272 conn(0x7fb39806b300 msgr2=0x7fb39806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.244+0000 7fb39d46b640 1 -- 172.21.15.26:0/3577806272 shutdown_connections 2024-04-24T23:28:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.244+0000 7fb39d46b640 1 -- 172.21.15.26:0/3577806272 wait complete. 2024-04-24T23:28:24.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.245+0000 7fb39d46b640 1 Processor -- start 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.245+0000 7fb39d46b640 1 -- start start 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb39d46b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 0x7fb39815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb39d46b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb398141160 0x7fb398152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb39d46b640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb39815ea80 0x7fb3981534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb39d46b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb39813e210 con 0x7fb3980721b0 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb39d46b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb39813e090 con 0x7fb398141160 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb39d46b640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fb39806dbe0 con 0x7fb39815ea80 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb396ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 0x7fb39815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb396ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 0x7fb39815a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58304/0 (socket says 172.21.15.26:58304) 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb396ffd640 1 -- 172.21.15.26:0/3392303533 learned_addr learned my addr 172.21.15.26:0/3392303533 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb396ffd640 1 -- 172.21.15.26:0/3392303533 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb39815ea80 msgr2=0x7fb3981534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb396ffd640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fb39815ea80 0x7fb3981534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb396ffd640 1 -- 172.21.15.26:0/3392303533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb398141160 msgr2=0x7fb398152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb396ffd640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb398141160 0x7fb398152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb396ffd640 1 -- 172.21.15.26:0/3392303533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39813f650 con 0x7fb3980721b0 2024-04-24T23:28:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb396ffd640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 0x7fb39815a640 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b3f0 tx=0x7fb38800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb38f7fe640 1 -- 172.21.15.26:0/3392303533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388015020 con 0x7fb3980721b0 2024-04-24T23:28:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb38f7fe640 1 -- 172.21.15.26:0/3392303533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb388011830 con 0x7fb3980721b0 2024-04-24T23:28:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb398153b30 con 0x7fb3980721b0 2024-04-24T23:28:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb38f7fe640 1 -- 172.21.15.26:0/3392303533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388011c20 con 0x7fb3980721b0 2024-04-24T23:28:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.246+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3981540f0 con 0x7fb3980721b0 2024-04-24T23:28:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.247+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb3981547a0 con 0x7fb3980721b0 2024-04-24T23:28:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.248+0000 7fb38f7fe640 1 -- 172.21.15.26:0/3392303533 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb388018020 con 0x7fb3980721b0 2024-04-24T23:28:24.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.248+0000 7fb38f7fe640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb37806be30 0x7fb37806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.248+0000 7fb38f7fe640 1 -- 172.21.15.26:0/3392303533 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7fb38808a8e0 con 0x7fb3980721b0 2024-04-24T23:28:24.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.248+0000 7fb38ffff640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb37806be30 0x7fb37806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.249+0000 7fb38f7fe640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7fb378071e30 0x7fb378074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.249+0000 7fb38f7fe640 1 -- 172.21.15.26:0/3392303533 --> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb378074900 con 0x7fb378071e30 2024-04-24T23:28:24.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.249+0000 7fb38f7fe640 1 -- 172.21.15.26:0/3392303533 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fb38808f2c0 con 0x7fb3980721b0 2024-04-24T23:28:24.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.249+0000 7fb38ffff640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb37806be30 0x7fb37806e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb380000f60 tx=0x7fb380007a00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.249+0000 7fb3977fe640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7fb378071e30 0x7fb378074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.249+0000 7fb3977fe640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7fb378071e30 0x7fb378074250 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.250+0000 7fb38f7fe640 1 -- 172.21.15.26:0/3392303533 <== osd.4 v2:172.21.15.29:6808/3208387101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb378074900 con 0x7fb378071e30 2024-04-24T23:28:24.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.263+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 --> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb3980694c0 con 0x7fb378071e30 2024-04-24T23:28:24.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.264+0000 7fb38f7fe640 1 -- 172.21.15.26:0/3392303533 <== osd.4 v2:172.21.15.29:6808/3208387101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb3980694c0 con 0x7fb378071e30 2024-04-24T23:28:24.265 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:24.265 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T23:28:24.265 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:24.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7fb378071e30 msgr2=0x7fb378074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.29:6808/3208387101,v1:172.21.15.29:6809/3208387101] conn(0x7fb378071e30 0x7fb378074250 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb37806be30 msgr2=0x7fb37806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fb37806be30 0x7fb37806e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb380000f60 tx=0x7fb380007a00 comp rx=0 tx=0).stop 2024-04-24T23:28:24.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 msgr2=0x7fb39815a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 --2- 172.21.15.26:0/3392303533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3980721b0 0x7fb39815a640 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b3f0 tx=0x7fb38800b8b0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb3977fe640 1 -- 172.21.15.26:0/3392303533 reap_dead start 2024-04-24T23:28:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 shutdown_connections 2024-04-24T23:28:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 >> 172.21.15.26:0/3392303533 conn(0x7fb39806b300 msgr2=0x7fb3980b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 shutdown_connections 2024-04-24T23:28:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.265+0000 7fb39d46b640 1 -- 172.21.15.26:0/3392303533 wait complete. 2024-04-24T23:28:24.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-24T23:28:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.377+0000 7fa140184640 1 Processor -- start 2024-04-24T23:28:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.377+0000 7fa140184640 1 -- start start 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa140184640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa1380cdd70 0x7fa13806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa140184640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1380cfbc0 con 0x7fa13806d350 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa140184640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13806d7f0 con 0x7fa138145910 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa140184640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa138145cf0 con 0x7fa1380cdd70 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa13e721640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa1380cdd70 0x7fa13806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa13d71f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa13806d350 0x7fa1380cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45522/0 (socket says 172.21.15.26:45522) 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa13d71f640 1 -- 172.21.15.26:0/226624127 learned_addr learned my addr 172.21.15.26:0/226624127 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa13e721640 1 -- 172.21.15.26:0/226624127 >> v1:172.21.15.29:6789/0 conn(0x7fa138145910 legacy=0x7fa13815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa13e721640 1 -- 172.21.15.26:0/226624127 >> v1:172.21.15.26:6789/0 conn(0x7fa13806d350 legacy=0x7fa1380cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.378+0000 7fa13e721640 1 -- 172.21.15.26:0/226624127 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13806dad0 con 0x7fa1380cdd70 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.379+0000 7fa13e721640 1 --2- 172.21.15.26:0/226624127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 0x7fa13806e890 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa130009dc0 tx=0x7fa13002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324c7baf2d67047a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.379+0000 7fa13cf1e640 1 -- 172.21.15.26:0/226624127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130007740 con 0x7fa1380cdd70 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.379+0000 7fa13cf1e640 1 -- 172.21.15.26:0/226624127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1300078e0 con 0x7fa1380cdd70 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.379+0000 7fa13cf1e640 1 -- 172.21.15.26:0/226624127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130007cb0 con 0x7fa1380cdd70 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.380+0000 7fa140184640 1 -- 172.21.15.26:0/226624127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 msgr2=0x7fa13806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.380+0000 7fa140184640 1 --2- 172.21.15.26:0/226624127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 0x7fa13806e890 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa130009dc0 tx=0x7fa13002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.380+0000 7fa140184640 1 -- 172.21.15.26:0/226624127 shutdown_connections 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.380+0000 7fa140184640 1 --2- 172.21.15.26:0/226624127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 0x7fa13806e890 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.380+0000 7fa140184640 1 -- 172.21.15.26:0/226624127 >> 172.21.15.26:0/226624127 conn(0x7fa13806b300 msgr2=0x7fa13806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.380+0000 7fa140184640 1 -- 172.21.15.26:0/226624127 shutdown_connections 2024-04-24T23:28:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.380+0000 7fa140184640 1 -- 172.21.15.26:0/226624127 wait complete. 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.381+0000 7fa140184640 1 Processor -- start 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.381+0000 7fa140184640 1 -- start start 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa140184640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa13806d350 0x7fa138158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa140184640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 0x7fa138141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa140184640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa138145910 0x7fa13813c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa140184640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa13806e370 con 0x7fa1380cdd70 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa140184640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa13806e090 con 0x7fa13806d350 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa140184640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fa13815d770 con 0x7fa138145910 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13d71f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 0x7fa138141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13d71f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 0x7fa138141c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58330/0 (socket says 172.21.15.26:58330) 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13d71f640 1 -- 172.21.15.26:0/4172453164 learned_addr learned my addr 172.21.15.26:0/4172453164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13e721640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa138145910 0x7fa13813c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13df20640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa13806d350 0x7fa138158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13d71f640 1 -- 172.21.15.26:0/4172453164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa138145910 msgr2=0x7fa13813c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13d71f640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa138145910 0x7fa13813c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13d71f640 1 -- 172.21.15.26:0/4172453164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa13806d350 msgr2=0x7fa138158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13d71f640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa13806d350 0x7fa138158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13d71f640 1 -- 172.21.15.26:0/4172453164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa138159060 con 0x7fa1380cdd70 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa13d71f640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 0x7fa138141c80 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c4d0 tx=0x7fa12800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa12e7fc640 1 -- 172.21.15.26:0/4172453164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128017070 con 0x7fa1380cdd70 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa12e7fc640 1 -- 172.21.15.26:0/4172453164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa128005ce0 con 0x7fa1380cdd70 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa138140be0 con 0x7fa1380cdd70 2024-04-24T23:28:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa12e7fc640 1 -- 172.21.15.26:0/4172453164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128007270 con 0x7fa1380cdd70 2024-04-24T23:28:24.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.382+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1381411a0 con 0x7fa1380cdd70 2024-04-24T23:28:24.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.383+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1380694c0 con 0x7fa1380cdd70 2024-04-24T23:28:24.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.384+0000 7fa12e7fc640 1 -- 172.21.15.26:0/4172453164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa12801a070 con 0x7fa1380cdd70 2024-04-24T23:28:24.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.384+0000 7fa12e7fc640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa11c06be30 0x7fa11c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.384+0000 7fa12e7fc640 1 -- 172.21.15.26:0/4172453164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7fa128089ed0 con 0x7fa1380cdd70 2024-04-24T23:28:24.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.384+0000 7fa13df20640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa11c06be30 0x7fa11c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.385+0000 7fa13df20640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa11c06be30 0x7fa11c06e2f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa134000f60 tx=0x7fa134009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.389+0000 7fa12e7fc640 1 -- 172.21.15.26:0/4172453164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1280583e0 con 0x7fa1380cdd70 2024-04-24T23:28:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.570+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fa13806b9c0 con 0x7fa11c06be30 2024-04-24T23:28:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.571+0000 7fa12e7fc640 1 -- 172.21.15.26:0/4172453164 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa13806b9c0 con 0x7fa11c06be30 2024-04-24T23:28:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 4 to deep-scrub 2024-04-24T23:28:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.572+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa11c06be30 msgr2=0x7fa11c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.572+0000 7fa140184640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa11c06be30 0x7fa11c06e2f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa134000f60 tx=0x7fa134009040 comp rx=0 tx=0).stop 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.572+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 msgr2=0x7fa138141c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.572+0000 7fa140184640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 0x7fa138141c80 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c4d0 tx=0x7fa12800c990 comp rx=0 tx=0).stop 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.573+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 shutdown_connections 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.573+0000 7fa140184640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fa11c06be30 0x7fa11c06e2f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.573+0000 7fa140184640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fa138145910 0x7fa13813c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.573+0000 7fa140184640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1380cdd70 0x7fa138141c80 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.573+0000 7fa140184640 1 --2- 172.21.15.26:0/4172453164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa13806d350 0x7fa138158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.573+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 >> 172.21.15.26:0/4172453164 conn(0x7fa13806b300 msgr2=0x7fa138144550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.573+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 shutdown_connections 2024-04-24T23:28:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.573+0000 7fa140184640 1 -- 172.21.15.26:0/4172453164 wait complete. 2024-04-24T23:28:24.584 INFO:tasks.ceph:Scrubbing osd.5 2024-04-24T23:28:24.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:28:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.688+0000 7f56982b3640 1 Processor -- start 2024-04-24T23:28:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.688+0000 7f56982b3640 1 -- start start 2024-04-24T23:28:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.689+0000 7f56982b3640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f569015e710 0x7f56900d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.689+0000 7f56982b3640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f56900d4630 0x7f56900d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.689+0000 7f56982b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56900d27b0 con 0x7f569015e330 2024-04-24T23:28:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.689+0000 7f56982b3640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f56900cef00 con 0x7f56900d4630 2024-04-24T23:28:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.689+0000 7f56982b3640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f56900cf1e0 con 0x7f569015e710 2024-04-24T23:28:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f569604f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f569015e330 0x7f56900d3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45530/0 (socket says 172.21.15.26:45530) 2024-04-24T23:28:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f569604f640 1 -- 172.21.15.26:0/1131739954 learned_addr learned my addr 172.21.15.26:0/1131739954 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f5696850640 1 --2- 172.21.15.26:0/1131739954 >> v2:172.21.15.29:3300/0 conn(0x7f56900d4630 0x7f56900d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f569584e640 1 --2- 172.21.15.26:0/1131739954 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f569015e710 0x7f56900d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f569504d640 1 -- 172.21.15.26:0/1131739954 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345311970 0 0) 0x7f56900d27b0 con 0x7f569015e330 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f569504d640 1 -- 172.21.15.26:0/1131739954 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f567c003520 con 0x7f569015e330 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f569584e640 1 -- 172.21.15.26:0/1131739954 >> v2:172.21.15.29:3300/0 conn(0x7f56900d4630 msgr2=0x7f56900d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f569584e640 1 --2- 172.21.15.26:0/1131739954 >> v2:172.21.15.29:3300/0 conn(0x7f56900d4630 0x7f56900d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f569584e640 1 -- 172.21.15.26:0/1131739954 >> v1:172.21.15.26:6789/0 conn(0x7f569015e330 legacy=0x7f56900d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.690+0000 7f569584e640 1 -- 172.21.15.26:0/1131739954 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56900cf4c0 con 0x7f569015e710 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.691+0000 7f569584e640 1 --2- 172.21.15.26:0/1131739954 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f569015e710 0x7f56900d20b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f56800097d0 tx=0x7f568002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479355e5158cd69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.691+0000 7f569504d640 1 -- 172.21.15.26:0/1131739954 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680037070 con 0x7f569015e710 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.691+0000 7f569504d640 1 -- 172.21.15.26:0/1131739954 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5680032440 con 0x7f569015e710 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.691+0000 7f569504d640 1 -- 172.21.15.26:0/1131739954 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56800327f0 con 0x7f569015e710 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.691+0000 7f56982b3640 1 -- 172.21.15.26:0/1131739954 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f569015e710 msgr2=0x7f56900d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.691+0000 7f56982b3640 1 --2- 172.21.15.26:0/1131739954 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f569015e710 0x7f56900d20b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f56800097d0 tx=0x7f568002a1c0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.692+0000 7f56982b3640 1 -- 172.21.15.26:0/1131739954 shutdown_connections 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.692+0000 7f56982b3640 1 --2- 172.21.15.26:0/1131739954 >> v2:172.21.15.29:3300/0 conn(0x7f56900d4630 0x7f56900d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.692+0000 7f56982b3640 1 --2- 172.21.15.26:0/1131739954 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f569015e710 0x7f56900d20b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.692+0000 7f56982b3640 1 -- 172.21.15.26:0/1131739954 >> 172.21.15.26:0/1131739954 conn(0x7f569008dbc0 msgr2=0x7f569009d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.692+0000 7f56982b3640 1 -- 172.21.15.26:0/1131739954 shutdown_connections 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.692+0000 7f56982b3640 1 -- 172.21.15.26:0/1131739954 wait complete. 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.693+0000 7f56982b3640 1 Processor -- start 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.693+0000 7f56982b3640 1 -- start start 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.693+0000 7f56982b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56900d4630 0x7f5690160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.693+0000 7f56982b3640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f569015e330 0x7f569014f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.693+0000 7f56982b3640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f569015e710 0x7f56900c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.693+0000 7f56982b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56900cfe40 con 0x7f56900d4630 2024-04-24T23:28:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.693+0000 7f56982b3640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f56900cfcc0 con 0x7f569015e330 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.693+0000 7f56982b3640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f56900cffc0 con 0x7f569015e710 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f569604f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56900d4630 0x7f5690160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f569604f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56900d4630 0x7f5690160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58344/0 (socket says 172.21.15.26:58344) 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f569604f640 1 -- 172.21.15.26:0/2797772091 learned_addr learned my addr 172.21.15.26:0/2797772091 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f569604f640 1 -- 172.21.15.26:0/2797772091 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f569015e710 msgr2=0x7f56900c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f569604f640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f569015e710 0x7f56900c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f569604f640 1 -- 172.21.15.26:0/2797772091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f569015e330 msgr2=0x7f569014f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f569604f640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f569015e330 0x7f569014f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f569604f640 1 -- 172.21.15.26:0/2797772091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569015f5a0 con 0x7f56900d4630 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f569604f640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56900d4630 0x7f5690160640 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c3e0 tx=0x7f568c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f5686ffd640 1 -- 172.21.15.26:0/2797772091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c017070 con 0x7f56900d4630 2024-04-24T23:28:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f5686ffd640 1 -- 172.21.15.26:0/2797772091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f568c005ce0 con 0x7f56900d4630 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569015f880 con 0x7f56900d4630 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f5686ffd640 1 -- 172.21.15.26:0/2797772091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c0072d0 con 0x7f56900d4630 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.694+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569015fe40 con 0x7f56900d4630 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.695+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f56900694c0 con 0x7f56900d4630 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.696+0000 7f5686ffd640 1 -- 172.21.15.26:0/2797772091 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f568c01a070 con 0x7f56900d4630 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.696+0000 7f5686ffd640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f567406be30 0x7f567406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.696+0000 7f5686ffd640 1 -- 172.21.15.26:0/2797772091 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f568c089f90 con 0x7f56900d4630 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.696+0000 7f569584e640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f567406be30 0x7f567406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.697+0000 7f5686ffd640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f5674071e30 0x7f5674074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.697+0000 7f5686ffd640 1 -- 172.21.15.26:0/2797772091 --> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f568c010e50 con 0x7f5674071e30 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.697+0000 7f5686ffd640 1 -- 172.21.15.26:0/2797772091 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f568c08f560 con 0x7f56900d4630 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.697+0000 7f569584e640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f567406be30 0x7f567406e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5680034000 tx=0x7f568002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.697+0000 7f5696850640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f5674071e30 0x7f5674074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.697+0000 7f5696850640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f5674071e30 0x7f5674074250 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.698+0000 7f5686ffd640 1 -- 172.21.15.26:0/2797772091 <== osd.5 v2:172.21.15.29:6800/1108214380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f568c010e50 con 0x7f5674071e30 2024-04-24T23:28:24.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.711+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 --> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f56900d5060 con 0x7f5674071e30 2024-04-24T23:28:24.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f5686ffd640 1 -- 172.21.15.26:0/2797772091 <== osd.5 v2:172.21.15.29:6800/1108214380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f56900d5060 con 0x7f5674071e30 2024-04-24T23:28:24.713 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:24.713 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T23:28:24.714 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f5674071e30 msgr2=0x7f5674074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.29:6800/1108214380,v1:172.21.15.29:6801/1108214380] conn(0x7f5674071e30 0x7f5674074250 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f567406be30 msgr2=0x7f567406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f567406be30 0x7f567406e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5680034000 tx=0x7f568002c000 comp rx=0 tx=0).stop 2024-04-24T23:28:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56900d4630 msgr2=0x7f5690160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 --2- 172.21.15.26:0/2797772091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56900d4630 0x7f5690160640 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c3e0 tx=0x7f568c00c8a0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f5696850640 1 -- 172.21.15.26:0/2797772091 reap_dead start 2024-04-24T23:28:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 shutdown_connections 2024-04-24T23:28:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 >> 172.21.15.26:0/2797772091 conn(0x7f569008dbc0 msgr2=0x7f56900b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 shutdown_connections 2024-04-24T23:28:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.712+0000 7f56982b3640 1 -- 172.21.15.26:0/2797772091 wait complete. 2024-04-24T23:28:24.723 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-24T23:28:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.824+0000 7fe8be5ce640 1 Processor -- start 2024-04-24T23:28:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.824+0000 7fe8be5ce640 1 -- start start 2024-04-24T23:28:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8be5ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8b806f110 0x7fe8b815f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8be5ce640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7fe8b80adf80 0x7fe8b8154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8be5ce640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b815f750 con 0x7fe8b806f4f0 2024-04-24T23:28:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8be5ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b80b0220 con 0x7fe8b806f110 2024-04-24T23:28:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8be5ce640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b80af180 con 0x7fe8b80adf80 2024-04-24T23:28:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8b7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8b806f110 0x7fe8b815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8b7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8b806f110 0x7fe8b815f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58358/0 (socket says 172.21.15.26:58358) 2024-04-24T23:28:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8b7fff640 1 -- 172.21.15.26:0/3313882694 learned_addr learned my addr 172.21.15.26:0/3313882694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8b7fff640 1 -- 172.21.15.26:0/3313882694 >> v2:172.21.15.29:3300/0 conn(0x7fe8b80adf80 msgr2=0x7fe8b8154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8b7fff640 1 --2- 172.21.15.26:0/3313882694 >> v2:172.21.15.29:3300/0 conn(0x7fe8b80adf80 0x7fe8b8154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8b7fff640 1 -- 172.21.15.26:0/3313882694 >> v1:172.21.15.29:6789/0 conn(0x7fe8b806f4f0 legacy=0x7fe8b80ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.825+0000 7fe8b7fff640 1 -- 172.21.15.26:0/3313882694 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b80af460 con 0x7fe8b806f110 2024-04-24T23:28:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.826+0000 7fe8b7fff640 1 --2- 172.21.15.26:0/3313882694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f110 0x7fe8b815f210 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80b3ca0 tx=0x7fe8a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80119d3408b7b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.826+0000 7fe8b6ffd640 1 -- 172.21.15.26:0/3313882694 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8037070 con 0x7fe8b806f110 2024-04-24T23:28:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.826+0000 7fe8b6ffd640 1 -- 172.21.15.26:0/3313882694 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a8032440 con 0x7fe8b806f110 2024-04-24T23:28:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.826+0000 7fe8b6ffd640 1 -- 172.21.15.26:0/3313882694 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8032810 con 0x7fe8b806f110 2024-04-24T23:28:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.827+0000 7fe8be5ce640 1 -- 172.21.15.26:0/3313882694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f110 msgr2=0x7fe8b815f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.827+0000 7fe8be5ce640 1 --2- 172.21.15.26:0/3313882694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f110 0x7fe8b815f210 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80b3ca0 tx=0x7fe8a802a1a0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.827+0000 7fe8be5ce640 1 -- 172.21.15.26:0/3313882694 shutdown_connections 2024-04-24T23:28:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.827+0000 7fe8be5ce640 1 --2- 172.21.15.26:0/3313882694 >> v2:172.21.15.29:3300/0 conn(0x7fe8b80adf80 0x7fe8b8154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.827+0000 7fe8be5ce640 1 --2- 172.21.15.26:0/3313882694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f110 0x7fe8b815f210 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.827+0000 7fe8be5ce640 1 -- 172.21.15.26:0/3313882694 >> 172.21.15.26:0/3313882694 conn(0x7fe8b80729a0 msgr2=0x7fe8b806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.827+0000 7fe8be5ce640 1 -- 172.21.15.26:0/3313882694 shutdown_connections 2024-04-24T23:28:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.827+0000 7fe8be5ce640 1 -- 172.21.15.26:0/3313882694 wait complete. 2024-04-24T23:28:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.828+0000 7fe8be5ce640 1 Processor -- start 2024-04-24T23:28:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.828+0000 7fe8be5ce640 1 -- start start 2024-04-24T23:28:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8be5ce640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8b806f110 0x7fe8b81451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8be5ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f4f0 0x7fe8b8145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8be5ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8b80adf80 0x7fe8b815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8be5ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b80afd00 con 0x7fe8b806f4f0 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8be5ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b80afa20 con 0x7fe8b80adf80 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8be5ce640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b80aee50 con 0x7fe8b806f110 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f4f0 0x7fe8b8145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f4f0 0x7fe8b8145c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58362/0 (socket says 172.21.15.26:58362) 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8b77fe640 1 -- 172.21.15.26:0/344373676 learned_addr learned my addr 172.21.15.26:0/344373676 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8bcb6b640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8b80adf80 0x7fe8b815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8b7fff640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8b806f110 0x7fe8b81451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8b77fe640 1 -- 172.21.15.26:0/344373676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8b806f110 msgr2=0x7fe8b81451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8b77fe640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8b806f110 0x7fe8b81451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8b77fe640 1 -- 172.21.15.26:0/344373676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8b80adf80 msgr2=0x7fe8b815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8b77fe640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8b80adf80 0x7fe8b815b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.829+0000 7fe8b77fe640 1 -- 172.21.15.26:0/344373676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b8159070 con 0x7fe8b806f4f0 2024-04-24T23:28:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.830+0000 7fe8b77fe640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f4f0 0x7fe8b8145c90 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c400 tx=0x7fe8ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.830+0000 7fe8b4ff9640 1 -- 172.21.15.26:0/344373676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac017070 con 0x7fe8b806f4f0 2024-04-24T23:28:24.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.830+0000 7fe8b4ff9640 1 -- 172.21.15.26:0/344373676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ac0035f0 con 0x7fe8b806f4f0 2024-04-24T23:28:24.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.830+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b8159350 con 0x7fe8b806f4f0 2024-04-24T23:28:24.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.830+0000 7fe8b4ff9640 1 -- 172.21.15.26:0/344373676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0039e0 con 0x7fe8b806f4f0 2024-04-24T23:28:24.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.830+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b8145710 con 0x7fe8b806f4f0 2024-04-24T23:28:24.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.831+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b80694c0 con 0x7fe8b806f4f0 2024-04-24T23:28:24.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.831+0000 7fe8b4ff9640 1 -- 172.21.15.26:0/344373676 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8ac01a070 con 0x7fe8b806f4f0 2024-04-24T23:28:24.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.832+0000 7fe8b4ff9640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8a406be30 0x7fe8a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.832+0000 7fe8b4ff9640 1 -- 172.21.15.26:0/344373676 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7fe8ac089e60 con 0x7fe8b806f4f0 2024-04-24T23:28:24.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.832+0000 7fe8b7fff640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8a406be30 0x7fe8a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.832+0000 7fe8b7fff640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8a406be30 0x7fe8a406e2f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8034000 tx=0x7fe8a802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:24.836+0000 7fe8b4ff9640 1 -- 172.21.15.26:0/344373676 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe8ac0010c0 con 0x7fe8b806f4f0 2024-04-24T23:28:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.018+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fe8b815f440 con 0x7fe8a406be30 2024-04-24T23:28:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.018+0000 7fe8b4ff9640 1 -- 172.21.15.26:0/344373676 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe8b815f440 con 0x7fe8a406be30 2024-04-24T23:28:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 5 to deep-scrub 2024-04-24T23:28:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8a406be30 msgr2=0x7fe8a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8a406be30 0x7fe8a406e2f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8034000 tx=0x7fe8a802e000 comp rx=0 tx=0).stop 2024-04-24T23:28:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f4f0 msgr2=0x7fe8b8145c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f4f0 0x7fe8b8145c90 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c400 tx=0x7fe8ac00c8c0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 shutdown_connections 2024-04-24T23:28:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fe8a406be30 0x7fe8a406e2f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8b80adf80 0x7fe8b815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b806f4f0 0x7fe8b8145c90 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 --2- 172.21.15.26:0/344373676 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fe8b806f110 0x7fe8b81451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 >> 172.21.15.26:0/344373676 conn(0x7fe8b80729a0 msgr2=0x7fe8b806c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 shutdown_connections 2024-04-24T23:28:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.020+0000 7fe8be5ce640 1 -- 172.21.15.26:0/344373676 wait complete. 2024-04-24T23:28:25.032 INFO:tasks.ceph:Scrubbing osd.6 2024-04-24T23:28:25.032 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:28:25.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.136+0000 7f5abb65d640 1 Processor -- start 2024-04-24T23:28:25.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.136+0000 7f5abb65d640 1 -- start start 2024-04-24T23:28:25.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5abb65d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ab415ea80 0x7f5ab4141160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5abb65d640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f5ab415ee60 0x7f5ab40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5abb65d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab40cf050 con 0x7f5ab415b490 2024-04-24T23:28:25.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5abb65d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab40cf330 con 0x7f5ab415ea80 2024-04-24T23:28:25.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5abb65d640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab40cf610 con 0x7f5ab415ee60 2024-04-24T23:28:25.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5ab93f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ab415ea80 0x7f5ab4141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5ab93f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ab415ea80 0x7f5ab4141160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58372/0 (socket says 172.21.15.26:58372) 2024-04-24T23:28:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5ab93f9640 1 -- 172.21.15.26:0/2796055464 learned_addr learned my addr 172.21.15.26:0/2796055464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5ab8bf8640 1 --2- 172.21.15.26:0/2796055464 >> v2:172.21.15.29:3300/0 conn(0x7f5ab415ee60 0x7f5ab40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5ab93f9640 1 -- 172.21.15.26:0/2796055464 >> v2:172.21.15.29:3300/0 conn(0x7f5ab415ee60 msgr2=0x7f5ab40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5ab93f9640 1 --2- 172.21.15.26:0/2796055464 >> v2:172.21.15.29:3300/0 conn(0x7f5ab415ee60 0x7f5ab40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5ab93f9640 1 -- 172.21.15.26:0/2796055464 >> v1:172.21.15.26:6789/0 conn(0x7f5ab415b490 legacy=0x7f5ab415b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.137+0000 7f5ab93f9640 1 -- 172.21.15.26:0/2796055464 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40cf8f0 con 0x7f5ab415ea80 2024-04-24T23:28:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5ab93f9640 1 --2- 172.21.15.26:0/2796055464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ea80 0x7f5ab4141160 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f5ab406f180 tx=0x7f5ab002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e56f3dda3ef3413 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5aabfff640 1 -- 172.21.15.26:0/2796055464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0036020 con 0x7f5ab415ea80 2024-04-24T23:28:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5aabfff640 1 -- 172.21.15.26:0/2796055464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0032640 con 0x7f5ab415ea80 2024-04-24T23:28:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5aabfff640 1 -- 172.21.15.26:0/2796055464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0032a10 con 0x7f5ab415ea80 2024-04-24T23:28:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5abb65d640 1 -- 172.21.15.26:0/2796055464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ea80 msgr2=0x7f5ab4141160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5abb65d640 1 --2- 172.21.15.26:0/2796055464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ea80 0x7f5ab4141160 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f5ab406f180 tx=0x7f5ab002c940 comp rx=0 tx=0).stop 2024-04-24T23:28:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5abb65d640 1 -- 172.21.15.26:0/2796055464 shutdown_connections 2024-04-24T23:28:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5abb65d640 1 --2- 172.21.15.26:0/2796055464 >> v2:172.21.15.29:3300/0 conn(0x7f5ab415ee60 0x7f5ab40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5abb65d640 1 --2- 172.21.15.26:0/2796055464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ea80 0x7f5ab4141160 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.138+0000 7f5abb65d640 1 -- 172.21.15.26:0/2796055464 >> 172.21.15.26:0/2796055464 conn(0x7f5ab406b300 msgr2=0x7f5ab406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.139+0000 7f5abb65d640 1 -- 172.21.15.26:0/2796055464 shutdown_connections 2024-04-24T23:28:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.139+0000 7f5abb65d640 1 -- 172.21.15.26:0/2796055464 wait complete. 2024-04-24T23:28:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.140+0000 7f5abb65d640 1 Processor -- start 2024-04-24T23:28:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.140+0000 7f5abb65d640 1 -- start start 2024-04-24T23:28:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.140+0000 7f5abb65d640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5ab415b490 0x7f5ab41f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.140+0000 7f5abb65d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ab415ea80 0x7f5ab413f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.140+0000 7f5abb65d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ee60 0x7f5ab40aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.140+0000 7f5abb65d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40cec80 con 0x7f5ab415ee60 2024-04-24T23:28:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.140+0000 7f5abb65d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab415f2d0 con 0x7f5ab415ea80 2024-04-24T23:28:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.140+0000 7f5abb65d640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab40cee00 con 0x7f5ab415b490 2024-04-24T23:28:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab9bfa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ee60 0x7f5ab40aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab9bfa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ee60 0x7f5ab40aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58384/0 (socket says 172.21.15.26:58384) 2024-04-24T23:28:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab9bfa640 1 -- 172.21.15.26:0/2517771287 learned_addr learned my addr 172.21.15.26:0/2517771287 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab8bf8640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ab415ea80 0x7f5ab413f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab93f9640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5ab415b490 0x7f5ab41f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab9bfa640 1 -- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5ab415b490 msgr2=0x7f5ab41f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab9bfa640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f5ab415b490 0x7f5ab41f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab9bfa640 1 -- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ab415ea80 msgr2=0x7f5ab413f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab9bfa640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ab415ea80 0x7f5ab413f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab9bfa640 1 -- 172.21.15.26:0/2517771287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab413f970 con 0x7f5ab415ee60 2024-04-24T23:28:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5ab9bfa640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ee60 0x7f5ab40aebc0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c7d0 tx=0x7f5aac00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5aa9ffb640 1 -- 172.21.15.26:0/2517771287 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac003b70 con 0x7f5ab415ee60 2024-04-24T23:28:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5aa9ffb640 1 -- 172.21.15.26:0/2517771287 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aac003d30 con 0x7f5ab415ee60 2024-04-24T23:28:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5aa9ffb640 1 -- 172.21.15.26:0/2517771287 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00aa90 con 0x7f5ab415ee60 2024-04-24T23:28:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.141+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab413fc50 con 0x7f5ab415ee60 2024-04-24T23:28:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.142+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab4140210 con 0x7f5ab415ee60 2024-04-24T23:28:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.142+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ab40694c0 con 0x7f5ab415ee60 2024-04-24T23:28:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.143+0000 7f5aa9ffb640 1 -- 172.21.15.26:0/2517771287 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5aac00ac30 con 0x7f5ab415ee60 2024-04-24T23:28:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.143+0000 7f5aa9ffb640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5a9806be30 0x7f5a9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.143+0000 7f5aa9ffb640 1 -- 172.21.15.26:0/2517771287 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f5aac08aa00 con 0x7f5ab415ee60 2024-04-24T23:28:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.143+0000 7f5ab93f9640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5a9806be30 0x7f5a9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.144+0000 7f5aa9ffb640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f5a98071e30 0x7f5a98074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.144+0000 7f5aa9ffb640 1 -- 172.21.15.26:0/2517771287 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5aac010e50 con 0x7f5a98071e30 2024-04-24T23:28:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.144+0000 7f5aa9ffb640 1 -- 172.21.15.26:0/2517771287 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f5aac08ce60 con 0x7f5ab415ee60 2024-04-24T23:28:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.144+0000 7f5ab8bf8640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f5a98071e30 0x7f5a98074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.144+0000 7f5ab93f9640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5a9806be30 0x7f5a9806e2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0037000 tx=0x7f5ab003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.144+0000 7f5ab8bf8640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f5a98071e30 0x7f5a98074250 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.145+0000 7f5aa9ffb640 1 -- 172.21.15.26:0/2517771287 <== osd.6 v2:172.21.15.29:6812/744093613 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5aac010e50 con 0x7f5a98071e30 2024-04-24T23:28:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.158+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 --> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5ab4072f00 con 0x7f5a98071e30 2024-04-24T23:28:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.159+0000 7f5aa9ffb640 1 -- 172.21.15.26:0/2517771287 <== osd.6 v2:172.21.15.29:6812/744093613 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5ab4072f00 con 0x7f5a98071e30 2024-04-24T23:28:25.160 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:25.160 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T23:28:25.160 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:25.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.159+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f5a98071e30 msgr2=0x7f5a98074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.159+0000 7f5abb65d640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.29:6812/744093613,v1:172.21.15.29:6813/744093613] conn(0x7f5a98071e30 0x7f5a98074250 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.160+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5a9806be30 msgr2=0x7f5a9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.160+0000 7f5abb65d640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f5a9806be30 0x7f5a9806e2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0037000 tx=0x7f5ab003d040 comp rx=0 tx=0).stop 2024-04-24T23:28:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.160+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ee60 msgr2=0x7f5ab40aebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.160+0000 7f5abb65d640 1 --2- 172.21.15.26:0/2517771287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ab415ee60 0x7f5ab40aebc0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c7d0 tx=0x7f5aac00cc90 comp rx=0 tx=0).stop 2024-04-24T23:28:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.160+0000 7f5ab9bfa640 1 -- 172.21.15.26:0/2517771287 reap_dead start 2024-04-24T23:28:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.160+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 shutdown_connections 2024-04-24T23:28:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.160+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 >> 172.21.15.26:0/2517771287 conn(0x7f5ab406b300 msgr2=0x7f5ab40b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.160+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 shutdown_connections 2024-04-24T23:28:25.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.160+0000 7f5abb65d640 1 -- 172.21.15.26:0/2517771287 wait complete. 2024-04-24T23:28:25.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-24T23:28:25.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.272+0000 7f1f2249a640 1 Processor -- start 2024-04-24T23:28:25.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.272+0000 7f1f2249a640 1 -- start start 2024-04-24T23:28:25.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.273+0000 7f1f2249a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f1c15a860 0x7f1f1c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.273+0000 7f1f2249a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1c0b2e80 con 0x7f1f1c06e120 2024-04-24T23:28:25.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.273+0000 7f1f2249a640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1c0b3120 con 0x7f1f1c15a480 2024-04-24T23:28:25.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.273+0000 7f1f2249a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1f1c06da60 con 0x7f1f1c15a860 2024-04-24T23:28:25.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.273+0000 7f1f1b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f1c15a860 0x7f1f1c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.273+0000 7f1f1b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f1c15a860 0x7f1f1c06dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58388/0 (socket says 172.21.15.26:58388) 2024-04-24T23:28:25.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.273+0000 7f1f1b7fe640 1 -- 172.21.15.26:0/3217796167 learned_addr learned my addr 172.21.15.26:0/3217796167 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:25.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.274+0000 7f1f1b7fe640 1 -- 172.21.15.26:0/3217796167 >> v1:172.21.15.29:6789/0 conn(0x7f1f1c15a480 legacy=0x7f1f1c0b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.274+0000 7f1f1b7fe640 1 -- 172.21.15.26:0/3217796167 >> v1:172.21.15.26:6789/0 conn(0x7f1f1c06e120 legacy=0x7f1f1c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.274+0000 7f1f1b7fe640 1 -- 172.21.15.26:0/3217796167 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c0b33c0 con 0x7f1f1c15a860 2024-04-24T23:28:25.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.274+0000 7f1f1affd640 1 -- 172.21.15.26:0/3217796167 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 768304459 0 0) 0x7f1f1c0b2e80 con 0x7f1f1c06e120 2024-04-24T23:28:25.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.274+0000 7f1f1b7fe640 1 --2- 172.21.15.26:0/3217796167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c15a860 0x7f1f1c06dbe0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009870 tx=0x7f1f0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6af229fad5b4ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.274+0000 7f1f1affd640 1 -- 172.21.15.26:0/3217796167 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c037070 con 0x7f1f1c15a860 2024-04-24T23:28:25.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.274+0000 7f1f1affd640 1 -- 172.21.15.26:0/3217796167 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0c032440 con 0x7f1f1c15a860 2024-04-24T23:28:25.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.274+0000 7f1f1affd640 1 -- 172.21.15.26:0/3217796167 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c032810 con 0x7f1f1c15a860 2024-04-24T23:28:25.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.275+0000 7f1f2249a640 1 -- 172.21.15.26:0/3217796167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c15a860 msgr2=0x7f1f1c06dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.275+0000 7f1f2249a640 1 --2- 172.21.15.26:0/3217796167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c15a860 0x7f1f1c06dbe0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009870 tx=0x7f1f0c02a270 comp rx=0 tx=0).stop 2024-04-24T23:28:25.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.275+0000 7f1f2249a640 1 -- 172.21.15.26:0/3217796167 shutdown_connections 2024-04-24T23:28:25.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.275+0000 7f1f2249a640 1 --2- 172.21.15.26:0/3217796167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c15a860 0x7f1f1c06dbe0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.275+0000 7f1f2249a640 1 -- 172.21.15.26:0/3217796167 >> 172.21.15.26:0/3217796167 conn(0x7f1f1c06b300 msgr2=0x7f1f1c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:25.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.275+0000 7f1f2249a640 1 -- 172.21.15.26:0/3217796167 shutdown_connections 2024-04-24T23:28:25.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.275+0000 7f1f2249a640 1 -- 172.21.15.26:0/3217796167 wait complete. 2024-04-24T23:28:25.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.276+0000 7f1f2249a640 1 Processor -- start 2024-04-24T23:28:25.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.276+0000 7f1f2249a640 1 -- start start 2024-04-24T23:28:25.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f2249a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c06e120 0x7f1f1c138180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f2249a640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1f1c15a480 0x7f1f1c1ef200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f2249a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f1c15a860 0x7f1f1c1ef740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f2249a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c0b3980 con 0x7f1f1c06e120 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f2249a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c06e550 con 0x7f1f1c15a860 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f2249a640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1c0b3c60 con 0x7f1f1c15a480 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f1bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c06e120 0x7f1f1c138180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f1bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c06e120 0x7f1f1c138180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58404/0 (socket says 172.21.15.26:58404) 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f1bfff640 1 -- 172.21.15.26:0/1077084728 learned_addr learned my addr 172.21.15.26:0/1077084728 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f20a37640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f1c15a860 0x7f1f1c1ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.277+0000 7f1f1b7fe640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1f1c15a480 0x7f1f1c1ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f1bfff640 1 -- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1f1c15a480 msgr2=0x7f1f1c1ef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f1bfff640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1f1c15a480 0x7f1f1c1ef200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f1bfff640 1 -- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f1c15a860 msgr2=0x7f1f1c1ef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f1bfff640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f1c15a860 0x7f1f1c1ef740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f1bfff640 1 -- 172.21.15.26:0/1077084728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c1386c0 con 0x7f1f1c06e120 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f20a37640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f1c15a860 0x7f1f1c1ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f1b7fe640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1f1c15a480 0x7f1f1c1ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f1bfff640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c06e120 0x7f1f1c138180 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400c500 tx=0x7f1f0400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f18ff9640 1 -- 172.21.15.26:0/1077084728 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f040052d0 con 0x7f1f1c06e120 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f18ff9640 1 -- 172.21.15.26:0/1077084728 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f04005470 con 0x7f1f1c06e120 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c157650 con 0x7f1f1c06e120 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f18ff9640 1 -- 172.21.15.26:0/1077084728 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04012850 con 0x7f1f1c06e120 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.278+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c15c360 con 0x7f1f1c06e120 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.279+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1c0694c0 con 0x7f1f1c06e120 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.280+0000 7f1f18ff9640 1 -- 172.21.15.26:0/1077084728 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f040129f0 con 0x7f1f1c06e120 2024-04-24T23:28:25.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.280+0000 7f1f18ff9640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1efc06be30 0x7f1efc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.280+0000 7f1f18ff9640 1 -- 172.21.15.26:0/1077084728 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f1f0408a830 con 0x7f1f1c06e120 2024-04-24T23:28:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.280+0000 7f1f1b7fe640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1efc06be30 0x7f1efc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.281+0000 7f1f1b7fe640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1efc06be30 0x7f1efc06e2f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c034000 tx=0x7f1f0c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.284+0000 7f1f18ff9640 1 -- 172.21.15.26:0/1077084728 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f04059440 con 0x7f1f1c06e120 2024-04-24T23:28:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.471+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f1f1c15f190 con 0x7f1efc06be30 2024-04-24T23:28:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.471+0000 7f1f18ff9640 1 -- 172.21.15.26:0/1077084728 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1f1c15f190 con 0x7f1efc06be30 2024-04-24T23:28:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 6 to deep-scrub 2024-04-24T23:28:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1efc06be30 msgr2=0x7f1efc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1efc06be30 0x7f1efc06e2f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c034000 tx=0x7f1f0c02e000 comp rx=0 tx=0).stop 2024-04-24T23:28:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c06e120 msgr2=0x7f1f1c138180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c06e120 0x7f1f1c138180 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400c500 tx=0x7f1f0400c9c0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 shutdown_connections 2024-04-24T23:28:25.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f1efc06be30 0x7f1efc06e2f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1f1c15a860 0x7f1f1c1ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f1f1c15a480 0x7f1f1c1ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 --2- 172.21.15.26:0/1077084728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f1c06e120 0x7f1f1c138180 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.473+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 >> 172.21.15.26:0/1077084728 conn(0x7f1f1c06b300 msgr2=0x7f1f1c147290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:25.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.474+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 shutdown_connections 2024-04-24T23:28:25.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.474+0000 7f1f2249a640 1 -- 172.21.15.26:0/1077084728 wait complete. 2024-04-24T23:28:25.486 INFO:tasks.ceph:Scrubbing osd.7 2024-04-24T23:28:25.486 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-24T23:28:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.590+0000 7ff92ef84640 1 Processor -- start 2024-04-24T23:28:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.590+0000 7ff92ef84640 1 -- start start 2024-04-24T23:28:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92ef84640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9281663e0 0x7ff9280b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92ef84640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9281667c0 0x7ff9280b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92ef84640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9280b3a60 con 0x7ff9280b3430 2024-04-24T23:28:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92ef84640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff928166fa0 con 0x7ff9281663e0 2024-04-24T23:28:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92ef84640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff9280b3d40 con 0x7ff9281667c0 2024-04-24T23:28:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92cd20640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9281663e0 0x7ff9280b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92cd20640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9281663e0 0x7ff9280b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58414/0 (socket says 172.21.15.26:58414) 2024-04-24T23:28:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92cd20640 1 -- 172.21.15.26:0/112046895 learned_addr learned my addr 172.21.15.26:0/112046895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92cd20640 1 -- 172.21.15.26:0/112046895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9281667c0 msgr2=0x7ff9280b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92cd20640 1 --2- 172.21.15.26:0/112046895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9281667c0 0x7ff9280b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92cd20640 1 -- 172.21.15.26:0/112046895 >> v1:172.21.15.29:6789/0 conn(0x7ff9280b3430 legacy=0x7ff92814ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.591+0000 7ff92cd20640 1 -- 172.21.15.26:0/112046895 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9280b4020 con 0x7ff9281663e0 2024-04-24T23:28:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.592+0000 7ff92cd20640 1 --2- 172.21.15.26:0/112046895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9281663e0 0x7ff9280b4ae0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7ff920009e50 tx=0x7ff92002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6522a17ef4a85bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.592+0000 7ff9277fe640 1 -- 172.21.15.26:0/112046895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920036070 con 0x7ff9281663e0 2024-04-24T23:28:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.592+0000 7ff9277fe640 1 -- 172.21.15.26:0/112046895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff920031600 con 0x7ff9281663e0 2024-04-24T23:28:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.592+0000 7ff9277fe640 1 -- 172.21.15.26:0/112046895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9200319d0 con 0x7ff9281663e0 2024-04-24T23:28:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.593+0000 7ff92ef84640 1 -- 172.21.15.26:0/112046895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9281663e0 msgr2=0x7ff9280b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.593+0000 7ff92ef84640 1 --2- 172.21.15.26:0/112046895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9281663e0 0x7ff9280b4ae0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7ff920009e50 tx=0x7ff92002a6c0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.593+0000 7ff92ef84640 1 -- 172.21.15.26:0/112046895 shutdown_connections 2024-04-24T23:28:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.593+0000 7ff92ef84640 1 --2- 172.21.15.26:0/112046895 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9281667c0 0x7ff9280b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.593+0000 7ff92ef84640 1 --2- 172.21.15.26:0/112046895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9281663e0 0x7ff9280b4ae0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.593+0000 7ff92ef84640 1 -- 172.21.15.26:0/112046895 >> 172.21.15.26:0/112046895 conn(0x7ff92806b300 msgr2=0x7ff92806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.593+0000 7ff92ef84640 1 -- 172.21.15.26:0/112046895 shutdown_connections 2024-04-24T23:28:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.593+0000 7ff92ef84640 1 -- 172.21.15.26:0/112046895 wait complete. 2024-04-24T23:28:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.594+0000 7ff92ef84640 1 Processor -- start 2024-04-24T23:28:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.594+0000 7ff92ef84640 1 -- start start 2024-04-24T23:28:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92ef84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9280b3430 0x7ff928141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92ef84640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9281663e0 0x7ff928145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92ef84640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9281667c0 0x7ff9281462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92ef84640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff92814c150 con 0x7ff9280b3430 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92ef84640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9280b45e0 con 0x7ff9281667c0 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92ef84640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7ff92814c2d0 con 0x7ff9281663e0 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92cd20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9280b3430 0x7ff928141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92cd20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9280b3430 0x7ff928141970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58426/0 (socket says 172.21.15.26:58426) 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92cd20640 1 -- 172.21.15.26:0/1377341986 learned_addr learned my addr 172.21.15.26:0/1377341986 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92cd20640 1 -- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9281663e0 msgr2=0x7ff928145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92cd20640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7ff9281663e0 0x7ff928145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92cd20640 1 -- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9281667c0 msgr2=0x7ff9281462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92d521640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9281667c0 0x7ff9281462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92cd20640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9281667c0 0x7ff9281462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92cd20640 1 -- 172.21.15.26:0/1377341986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9281635f0 con 0x7ff9280b3430 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.595+0000 7ff92d521640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9281667c0 0x7ff9281462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.596+0000 7ff92cd20640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9280b3430 0x7ff928141970 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7ff920034040 tx=0x7ff920004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.596+0000 7ff9257fa640 1 -- 172.21.15.26:0/1377341986 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920041020 con 0x7ff9280b3430 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.596+0000 7ff9257fa640 1 -- 172.21.15.26:0/1377341986 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff920007800 con 0x7ff9280b3430 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.596+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9281638d0 con 0x7ff9280b3430 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.596+0000 7ff9257fa640 1 -- 172.21.15.26:0/1377341986 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920036040 con 0x7ff9280b3430 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.596+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff928163e90 con 0x7ff9280b3430 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.597+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9281633a0 con 0x7ff9280b3430 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.597+0000 7ff9257fa640 1 -- 172.21.15.26:0/1377341986 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff920044020 con 0x7ff9280b3430 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.598+0000 7ff9257fa640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff91406be30 0x7ff91406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.598+0000 7ff9257fa640 1 -- 172.21.15.26:0/1377341986 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7ff9200abc20 con 0x7ff9280b3430 2024-04-24T23:28:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.598+0000 7ff927fff640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff91406be30 0x7ff91406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.598+0000 7ff9257fa640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7ff914071e30 0x7ff914074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.598+0000 7ff9257fa640 1 -- 172.21.15.26:0/1377341986 --> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff920031600 con 0x7ff914071e30 2024-04-24T23:28:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.598+0000 7ff9257fa640 1 -- 172.21.15.26:0/1377341986 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7ff9200b02a0 con 0x7ff9280b3430 2024-04-24T23:28:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.598+0000 7ff927fff640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff91406be30 0x7ff91406e2f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff9280af080 tx=0x7ff91c006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.598+0000 7ff92d521640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7ff914071e30 0x7ff914074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.599+0000 7ff92d521640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7ff914071e30 0x7ff914074250 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.599+0000 7ff9257fa640 1 -- 172.21.15.26:0/1377341986 <== osd.7 v2:172.21.15.29:6824/2604664408 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff920031600 con 0x7ff914071e30 2024-04-24T23:28:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.613+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 --> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff9280694c0 con 0x7ff914071e30 2024-04-24T23:28:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.613+0000 7ff9257fa640 1 -- 172.21.15.26:0/1377341986 <== osd.7 v2:172.21.15.29:6824/2604664408 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff9280694c0 con 0x7ff914071e30 2024-04-24T23:28:25.614 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-24T23:28:25.614 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T23:28:25.614 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-24T23:28:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7ff914071e30 msgr2=0x7ff914074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.29:6824/2604664408,v1:172.21.15.29:6825/2604664408] conn(0x7ff914071e30 0x7ff914074250 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff91406be30 msgr2=0x7ff91406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7ff91406be30 0x7ff91406e2f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff9280af080 tx=0x7ff91c006bf0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9280b3430 msgr2=0x7ff928141970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 --2- 172.21.15.26:0/1377341986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9280b3430 0x7ff928141970 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7ff920034040 tx=0x7ff920004a40 comp rx=0 tx=0).stop 2024-04-24T23:28:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92d521640 1 -- 172.21.15.26:0/1377341986 reap_dead start 2024-04-24T23:28:25.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 shutdown_connections 2024-04-24T23:28:25.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 >> 172.21.15.26:0/1377341986 conn(0x7ff92806b300 msgr2=0x7ff928149c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:25.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 shutdown_connections 2024-04-24T23:28:25.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.614+0000 7ff92ef84640 1 -- 172.21.15.26:0/1377341986 wait complete. 2024-04-24T23:28:25.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-24T23:28:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.728+0000 7f93026e1640 1 Processor -- start 2024-04-24T23:28:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.728+0000 7f93026e1640 1 -- start start 2024-04-24T23:28:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.728+0000 7f93026e1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc15e570 0x7f92fc0d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.728+0000 7f93026e1640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f92fc0d22e0 0x7f92fc0d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.728+0000 7f93026e1640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92fc0d32f0 con 0x7f92fc0b3130 2024-04-24T23:28:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.728+0000 7f93026e1640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f92fc0d1ab0 con 0x7f92fc0d22e0 2024-04-24T23:28:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.728+0000 7f93026e1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f92fc0d2ae0 con 0x7f92fc15e570 2024-04-24T23:28:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc15e570 0x7f92fc0d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f9300c7e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f92fc0d22e0 0x7f92fc0d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc15e570 0x7f92fc0d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3301/0 says I am v2:172.21.15.26:51830/0 (socket says 172.21.15.26:51830) 2024-04-24T23:28:25.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f92fb7fe640 1 --1- >> v1:172.21.15.29:6789/0 conn(0x7f92fc0b3130 0x7f92fc0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.29:6789/0 says I am v1:172.21.15.26:56754/0 (socket says 172.21.15.26:56754) 2024-04-24T23:28:25.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f9300c7e640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f92fc0d22e0 0x7f92fc0d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.26:49616/0 (socket says 172.21.15.26:49616) 2024-04-24T23:28:25.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f92fbfff640 1 -- 172.21.15.26:0/614257858 learned_addr learned my addr 172.21.15.26:0/614257858 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f92fbfff640 1 -- 172.21.15.26:0/614257858 >> v2:172.21.15.29:3300/0 conn(0x7f92fc0d22e0 msgr2=0x7f92fc0d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f92fbfff640 1 --2- 172.21.15.26:0/614257858 >> v2:172.21.15.29:3300/0 conn(0x7f92fc0d22e0 0x7f92fc0d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f92fbfff640 1 -- 172.21.15.26:0/614257858 >> v1:172.21.15.29:6789/0 conn(0x7f92fc0b3130 legacy=0x7f92fc0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f92fbfff640 1 -- 172.21.15.26:0/614257858 --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc0d0270 con 0x7f92fc15e570 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.729+0000 7f9300c7e640 1 --2- 172.21.15.26:0/614257858 >> v2:172.21.15.29:3300/0 conn(0x7f92fc0d22e0 0x7f92fc0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.730+0000 7f92fbfff640 1 --2- 172.21.15.26:0/614257858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc15e570 0x7f92fc0d4160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009870 tx=0x7f92ec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19715b285c6a6ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.730+0000 7f92faffd640 1 -- 172.21.15.26:0/614257858 <== mon.2 v2:172.21.15.29:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec037070 con 0x7f92fc15e570 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.730+0000 7f92faffd640 1 -- 172.21.15.26:0/614257858 <== mon.2 v2:172.21.15.29:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec032440 con 0x7f92fc15e570 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.730+0000 7f92faffd640 1 -- 172.21.15.26:0/614257858 <== mon.2 v2:172.21.15.29:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec032810 con 0x7f92fc15e570 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.731+0000 7f93026e1640 1 -- 172.21.15.26:0/614257858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc15e570 msgr2=0x7f92fc0d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.731+0000 7f93026e1640 1 --2- 172.21.15.26:0/614257858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc15e570 0x7f92fc0d4160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009870 tx=0x7f92ec02a270 comp rx=0 tx=0).stop 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.731+0000 7f93026e1640 1 -- 172.21.15.26:0/614257858 shutdown_connections 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.731+0000 7f93026e1640 1 --2- 172.21.15.26:0/614257858 >> v2:172.21.15.29:3300/0 conn(0x7f92fc0d22e0 0x7f92fc0d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.731+0000 7f93026e1640 1 --2- 172.21.15.26:0/614257858 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc15e570 0x7f92fc0d4160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.731+0000 7f93026e1640 1 -- 172.21.15.26:0/614257858 >> 172.21.15.26:0/614257858 conn(0x7f92fc08dbe0 msgr2=0x7f92fc09d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.731+0000 7f93026e1640 1 -- 172.21.15.26:0/614257858 shutdown_connections 2024-04-24T23:28:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.731+0000 7f93026e1640 1 -- 172.21.15.26:0/614257858 wait complete. 2024-04-24T23:28:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.732+0000 7f93026e1640 1 Processor -- start 2024-04-24T23:28:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.732+0000 7f93026e1640 1 -- start start 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f93026e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc0b3130 0x7f92fc1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f93026e1640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc0d22e0 0x7f92fc06ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f93026e1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92fc15e570 0x7f92fc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f93026e1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc0cfa70 con 0x7f92fc0b3130 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f93026e1640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc0d3010 con 0x7f92fc15e570 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f93026e1640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f92fc0cfd50 con 0x7f92fc0d22e0 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc0b3130 0x7f92fc1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc0b3130 0x7f92fc1581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58428/0 (socket says 172.21.15.26:58428) 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f92fbfff640 1 -- 172.21.15.26:0/2943858935 learned_addr learned my addr 172.21.15.26:0/2943858935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc0d22e0 0x7f92fc06ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f9300c7e640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92fc15e570 0x7f92fc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f92fbfff640 1 -- 172.21.15.26:0/2943858935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc0d22e0 msgr2=0x7f92fc06ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f92fbfff640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc0d22e0 0x7f92fc06ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f92fbfff640 1 -- 172.21.15.26:0/2943858935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92fc15e570 msgr2=0x7f92fc06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f92fbfff640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92fc15e570 0x7f92fc06d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.733+0000 7f92fbfff640 1 -- 172.21.15.26:0/2943858935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc0cb4f0 con 0x7f92fc0b3130 2024-04-24T23:28:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.734+0000 7f92fbfff640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc0b3130 0x7f92fc1581d0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.734+0000 7f92f8ff9640 1 -- 172.21.15.26:0/2943858935 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec037040 con 0x7f92fc0b3130 2024-04-24T23:28:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.734+0000 7f92f8ff9640 1 -- 172.21.15.26:0/2943858935 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec042070 con 0x7f92fc0b3130 2024-04-24T23:28:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.734+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc0cb7d0 con 0x7f92fc0b3130 2024-04-24T23:28:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.734+0000 7f92f8ff9640 1 -- 172.21.15.26:0/2943858935 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec030860 con 0x7f92fc0b3130 2024-04-24T23:28:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.734+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc0cbd90 con 0x7f92fc0b3130 2024-04-24T23:28:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.735+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc0cb280 con 0x7f92fc0b3130 2024-04-24T23:28:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.735+0000 7f92f8ff9640 1 -- 172.21.15.26:0/2943858935 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92ec030a00 con 0x7f92fc0b3130 2024-04-24T23:28:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.736+0000 7f92f8ff9640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f92e806be30 0x7f92e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.736+0000 7f92f8ff9640 1 -- 172.21.15.26:0/2943858935 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f92ec0ab4f0 con 0x7f92fc0b3130 2024-04-24T23:28:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.736+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f92e806be30 0x7f92e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.736+0000 7f92fb7fe640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f92e806be30 0x7f92e806e2f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f92fc158ea0 tx=0x7f92f0007cc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:25.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.740+0000 7f92f8ff9640 1 -- 172.21.15.26:0/2943858935 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f92ec045360 con 0x7f92fc0b3130 2024-04-24T23:28:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.924+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f92fc0694c0 con 0x7f92e806be30 2024-04-24T23:28:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.924+0000 7f92f8ff9640 1 -- 172.21.15.26:0/2943858935 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f92fc0694c0 con 0x7f92e806be30 2024-04-24T23:28:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 7 to deep-scrub 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f92e806be30 msgr2=0x7f92e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f92e806be30 0x7f92e806e2f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f92fc158ea0 tx=0x7f92f0007cc0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc0b3130 msgr2=0x7f92fc1581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc0b3130 0x7f92fc1581d0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec0046c0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 shutdown_connections 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f92e806be30 0x7f92e806e2f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f92fc15e570 0x7f92fc06d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f92fc0d22e0 0x7f92fc06ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 --2- 172.21.15.26:0/2943858935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92fc0b3130 0x7f92fc1581d0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:25.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 >> 172.21.15.26:0/2943858935 conn(0x7f92fc08dbe0 msgr2=0x7f92fc0b8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:25.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 shutdown_connections 2024-04-24T23:28:25.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:25.926+0000 7f93026e1640 1 -- 172.21.15.26:0/2943858935 wait complete. 2024-04-24T23:28:25.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:28:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.038+0000 7f41d5658640 1 Processor -- start 2024-04-24T23:28:26.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.038+0000 7f41d5658640 1 -- start start 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41d5658640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 0x7f41d00cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41d5658640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41d00d4a30 0x7f41d00d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41d5658640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d015e750 con 0x7f41d00d0ef0 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41d5658640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f41d00d2640 con 0x7f41d00d4a30 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41d5658640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f41d00cfcc0 con 0x7f41d00d0b10 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41ce7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41d00d4a30 0x7f41d00d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41cf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41d00d0ef0 0x7f41d00cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45576/0 (socket says 172.21.15.26:45576) 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41cf7fe640 1 -- 172.21.15.26:0/1206971384 learned_addr learned my addr 172.21.15.26:0/1206971384 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41ceffd640 1 --2- 172.21.15.26:0/1206971384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 0x7f41d00cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41ce7fc640 1 -- 172.21.15.26:0/1206971384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 msgr2=0x7f41d00cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41ce7fc640 1 --2- 172.21.15.26:0/1206971384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 0x7f41d00cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41ce7fc640 1 -- 172.21.15.26:0/1206971384 >> v1:172.21.15.26:6789/0 conn(0x7f41d00d0ef0 legacy=0x7f41d00cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41ce7fc640 1 -- 172.21.15.26:0/1206971384 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d00d1d30 con 0x7f41d00d4a30 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41cdffb640 1 -- 172.21.15.26:0/1206971384 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244229382 0 0) 0x7f41c8001490 con 0x7f41d00d0ef0 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.039+0000 7f41ceffd640 1 --2- 172.21.15.26:0/1206971384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 0x7f41d00cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.040+0000 7f41ce7fc640 1 --2- 172.21.15.26:0/1206971384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 0x7f41d00d2ad0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f41c40097a0 tx=0x7f41c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54edcf363df921e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.040+0000 7f41cdffb640 1 -- 172.21.15.26:0/1206971384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4037070 con 0x7f41d00d4a30 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.040+0000 7f41cdffb640 1 -- 172.21.15.26:0/1206971384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c4032440 con 0x7f41d00d4a30 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.040+0000 7f41cdffb640 1 -- 172.21.15.26:0/1206971384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4032810 con 0x7f41d00d4a30 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.041+0000 7f41d5658640 1 -- 172.21.15.26:0/1206971384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 msgr2=0x7f41d00d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.041+0000 7f41d5658640 1 --2- 172.21.15.26:0/1206971384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 0x7f41d00d2ad0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f41c40097a0 tx=0x7f41c402a180 comp rx=0 tx=0).stop 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.041+0000 7f41d5658640 1 -- 172.21.15.26:0/1206971384 shutdown_connections 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.041+0000 7f41d5658640 1 --2- 172.21.15.26:0/1206971384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 0x7f41d00d2ad0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.041+0000 7f41d5658640 1 --2- 172.21.15.26:0/1206971384 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 0x7f41d00cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.041+0000 7f41d5658640 1 -- 172.21.15.26:0/1206971384 >> 172.21.15.26:0/1206971384 conn(0x7f41d009d500 msgr2=0x7f41d00d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:26.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.041+0000 7f41d5658640 1 -- 172.21.15.26:0/1206971384 shutdown_connections 2024-04-24T23:28:26.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.041+0000 7f41d5658640 1 -- 172.21.15.26:0/1206971384 wait complete. 2024-04-24T23:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.042+0000 7f41d5658640 1 Processor -- start 2024-04-24T23:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.042+0000 7f41d5658640 1 -- start start 2024-04-24T23:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41d5658640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 0x7f41d00a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41d5658640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00d0ef0 0x7f41d00a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41d5658640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 0x7f41d0151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41d5658640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41d00d22d0 con 0x7f41d00d4a30 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41d5658640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f41d00d1ff0 con 0x7f41d00d0ef0 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41d5658640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f41d00d3010 con 0x7f41d00d0b10 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41cf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 0x7f41d0151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41cf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 0x7f41d0151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58444/0 (socket says 172.21.15.26:58444) 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41cf7fe640 1 -- 172.21.15.26:0/1205919641 learned_addr learned my addr 172.21.15.26:0/1205919641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41ce7fc640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00d0ef0 0x7f41d00a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41ceffd640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 0x7f41d00a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41cf7fe640 1 -- 172.21.15.26:0/1205919641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 msgr2=0x7f41d00a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41cf7fe640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 0x7f41d00a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41cf7fe640 1 -- 172.21.15.26:0/1205919641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00d0ef0 msgr2=0x7f41d00a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41cf7fe640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00d0ef0 0x7f41d00a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41cf7fe640 1 -- 172.21.15.26:0/1205919641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d009f090 con 0x7f41d00d4a30 2024-04-24T23:28:26.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41ce7fc640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00d0ef0 0x7f41d00a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.043+0000 7f41cf7fe640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 0x7f41d0151010 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c730 tx=0x7f41c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.044+0000 7f41b77fe640 1 -- 172.21.15.26:0/1205919641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8018070 con 0x7f41d00d4a30 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.044+0000 7f41b77fe640 1 -- 172.21.15.26:0/1205919641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c800ce40 con 0x7f41d00d4a30 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.044+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d009f370 con 0x7f41d00d4a30 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.044+0000 7f41b77fe640 1 -- 172.21.15.26:0/1205919641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c800a860 con 0x7f41d00d4a30 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.044+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d009f930 con 0x7f41d00d4a30 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.045+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d00694c0 con 0x7f41d00d4a30 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.045+0000 7f41b77fe640 1 -- 172.21.15.26:0/1205919641 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41c801b070 con 0x7f41d00d4a30 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.046+0000 7f41b77fe640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41bc06be30 0x7f41bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.046+0000 7f41b77fe640 1 -- 172.21.15.26:0/1205919641 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f41c808a190 con 0x7f41d00d4a30 2024-04-24T23:28:26.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.046+0000 7f41ceffd640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41bc06be30 0x7f41bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.046+0000 7f41ceffd640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41bc06be30 0x7f41bc06e2f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f41c000a950 tx=0x7f41c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:26.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.050+0000 7f41b77fe640 1 -- 172.21.15.26:0/1205919641 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f41d015e750 con 0x7f41d00d4a30 2024-04-24T23:28:26.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.231+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41d00b53f0 con 0x7f41bc06be30 2024-04-24T23:28:26.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.231+0000 7f41b77fe640 1 -- 172.21.15.26:0/1205919641 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13431 (secure 0 0 0) 0x7f41d00b53f0 con 0x7f41bc06be30 2024-04-24T23:28:26.232 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:26.232 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:28:26.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41bc06be30 msgr2=0x7f41bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41bc06be30 0x7f41bc06e2f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f41c000a950 tx=0x7f41c0008040 comp rx=0 tx=0).stop 2024-04-24T23:28:26.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 msgr2=0x7f41d0151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 0x7f41d0151010 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c730 tx=0x7f41c800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 shutdown_connections 2024-04-24T23:28:26.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f41bc06be30 0x7f41bc06e2f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d00d4a30 0x7f41d0151010 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f41d00d0ef0 0x7f41d00a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 --2- 172.21.15.26:0/1205919641 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f41d00d0b10 0x7f41d00a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 >> 172.21.15.26:0/1205919641 conn(0x7f41d009d500 msgr2=0x7f41d00cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:26.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 shutdown_connections 2024-04-24T23:28:26.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:26.233+0000 7f41d5658640 1 -- 172.21.15.26:0/1205919641 wait complete. 2024-04-24T23:28:26.245 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-24T23:28:25.413745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":281036,"kb_used_data":10792,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754693684,"statfs":{"total":773094113280,"available":772806332416,"internally_reserved":0,"allocated":11051008,"data_stored":5454280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.227852"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":314,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T23:28:19.633694+0000","last_change":"2024-04-24T23:20:44.373724+0000","last_active":"2024-04-24T23:28:19.633694+0000","last_peered":"2024-04-24T23:28:19.633694+0000","last_clean":"2024-04-24T23:28:19.633694+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:28:19.633694+0000","last_undegraded":"2024-04-24T23:28:19.633694+0000","last_fullsized":"2024-04-24T23:28:19.633694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:20:43.358551+0000","last_clean_scrub_stamp":"2024-04-24T23:20:43.358551+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:02.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37124,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334716,"statfs":{"total":96636764160,"available":96598749184,"internally_reserved":0,"allocated":1826816,"data_stored":1124561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36544,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335296,"statfs":{"total":96636764160,"available":96599343104,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36548,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335292,"statfs":{"total":96636764160,"available":96599339008,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36544,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335296,"statfs":{"total":96636764160,"available":96599343104,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37124,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334716,"statfs":{"total":96636764160,"available":96598749184,"internally_reserved":0,"allocated":1826816,"data_stored":1124561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:28:26.248 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-24T23:20:43.358551+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=20, tm_sec=43, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T23:28:26.248 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-24T23:28:29.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:28:29.972 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:28:46.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:28:46.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.362+0000 7f2c50a1c640 1 Processor -- start 2024-04-24T23:28:46.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.362+0000 7f2c50a1c640 1 -- start start 2024-04-24T23:28:46.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c50a1c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2c4c143c40 0x7f2c4c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c50a1c640 1 --2- >> v2:172.21.15.29:3300/0 conn(0x7f2c4c0cc860 0x7f2c4c072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c50a1c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4c0cd0b0 con 0x7f2c4c15eb00 2024-04-24T23:28:46.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c50a1c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4c071b70 con 0x7f2c4c143c40 2024-04-24T23:28:46.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c50a1c640 1 -- --> v2:172.21.15.29:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4c071240 con 0x7f2c4c0cc860 2024-04-24T23:28:46.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c4a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2c4c143c40 0x7f2c4c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c4ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2c4c15eb00 0x7f2c4c15ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56402/0 (socket says 172.21.15.26:56402) 2024-04-24T23:28:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c4ad9d640 1 -- 172.21.15.26:0/1311019794 learned_addr learned my addr 172.21.15.26:0/1311019794 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c41d9b640 1 --2- 172.21.15.26:0/1311019794 >> v2:172.21.15.29:3300/0 conn(0x7f2c4c0cc860 0x7f2c4c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c4a59c640 1 -- 172.21.15.26:0/1311019794 >> v2:172.21.15.29:3300/0 conn(0x7f2c4c0cc860 msgr2=0x7f2c4c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c4a59c640 1 --2- 172.21.15.26:0/1311019794 >> v2:172.21.15.29:3300/0 conn(0x7f2c4c0cc860 0x7f2c4c072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c4a59c640 1 -- 172.21.15.26:0/1311019794 >> v1:172.21.15.26:6789/0 conn(0x7f2c4c15eb00 legacy=0x7f2c4c15ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c49d9b640 1 -- 172.21.15.26:0/1311019794 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 235748131 0 0) 0x7f2c4c0cd0b0 con 0x7f2c4c15eb00 2024-04-24T23:28:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.363+0000 7f2c4a59c640 1 -- 172.21.15.26:0/1311019794 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c071520 con 0x7f2c4c143c40 2024-04-24T23:28:46.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.364+0000 7f2c4a59c640 1 --2- 172.21.15.26:0/1311019794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c143c40 0x7f2c4c0cf870 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c06db50 tx=0x7f2c3c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6391e526710f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.364+0000 7f2c49d9b640 1 -- 172.21.15.26:0/1311019794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c037070 con 0x7f2c4c143c40 2024-04-24T23:28:46.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.364+0000 7f2c49d9b640 1 -- 172.21.15.26:0/1311019794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c3c032440 con 0x7f2c4c143c40 2024-04-24T23:28:46.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.364+0000 7f2c49d9b640 1 -- 172.21.15.26:0/1311019794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c032810 con 0x7f2c4c143c40 2024-04-24T23:28:46.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.365+0000 7f2c50a1c640 1 -- 172.21.15.26:0/1311019794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c143c40 msgr2=0x7f2c4c0cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.365+0000 7f2c50a1c640 1 --2- 172.21.15.26:0/1311019794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c143c40 0x7f2c4c0cf870 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c06db50 tx=0x7f2c3c02a1a0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.365+0000 7f2c50a1c640 1 -- 172.21.15.26:0/1311019794 shutdown_connections 2024-04-24T23:28:46.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.365+0000 7f2c50a1c640 1 --2- 172.21.15.26:0/1311019794 >> v2:172.21.15.29:3300/0 conn(0x7f2c4c0cc860 0x7f2c4c072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.365+0000 7f2c50a1c640 1 --2- 172.21.15.26:0/1311019794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c143c40 0x7f2c4c0cf870 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.365+0000 7f2c50a1c640 1 -- 172.21.15.26:0/1311019794 >> 172.21.15.26:0/1311019794 conn(0x7f2c4c06b300 msgr2=0x7f2c4c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:46.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.365+0000 7f2c50a1c640 1 -- 172.21.15.26:0/1311019794 shutdown_connections 2024-04-24T23:28:46.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.365+0000 7f2c50a1c640 1 -- 172.21.15.26:0/1311019794 wait complete. 2024-04-24T23:28:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.366+0000 7f2c50a1c640 1 Processor -- start 2024-04-24T23:28:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.366+0000 7f2c50a1c640 1 -- start start 2024-04-24T23:28:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.367+0000 7f2c50a1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c4c0cc860 0x7f2c4c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.367+0000 7f2c50a1c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c4c143c40 0x7f2c4c155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.367+0000 7f2c50a1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c15eb00 0x7f2c4c13ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.367+0000 7f2c50a1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c072540 con 0x7f2c4c15eb00 2024-04-24T23:28:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.367+0000 7f2c50a1c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c144020 con 0x7f2c4c0cc860 2024-04-24T23:28:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.367+0000 7f2c50a1c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4c156210 con 0x7f2c4c143c40 2024-04-24T23:28:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.367+0000 7f2c4ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c15eb00 0x7f2c4c13ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.367+0000 7f2c4ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c15eb00 0x7f2c4c13ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38480/0 (socket says 172.21.15.26:38480) 2024-04-24T23:28:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.367+0000 7f2c4ad9d640 1 -- 172.21.15.26:0/2970635312 learned_addr learned my addr 172.21.15.26:0/2970635312 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c4a59c640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c4c0cc860 0x7f2c4c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c41d9b640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c4c143c40 0x7f2c4c155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c4ad9d640 1 -- 172.21.15.26:0/2970635312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c4c143c40 msgr2=0x7f2c4c155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c4ad9d640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c4c143c40 0x7f2c4c155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c4ad9d640 1 -- 172.21.15.26:0/2970635312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c4c0cc860 msgr2=0x7f2c4c15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c4ad9d640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c4c0cc860 0x7f2c4c15a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c4ad9d640 1 -- 172.21.15.26:0/2970635312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c140610 con 0x7f2c4c15eb00 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c41d9b640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c4c143c40 0x7f2c4c155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c4ad9d640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c15eb00 0x7f2c4c13ce20 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400c700 tx=0x7f2c4400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c437fe640 1 -- 172.21.15.26:0/2970635312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44018070 con 0x7f2c4c15eb00 2024-04-24T23:28:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c437fe640 1 -- 172.21.15.26:0/2970635312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4400ce10 con 0x7f2c4c15eb00 2024-04-24T23:28:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4c1408f0 con 0x7f2c4c15eb00 2024-04-24T23:28:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c437fe640 1 -- 172.21.15.26:0/2970635312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400a860 con 0x7f2c4c15eb00 2024-04-24T23:28:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.368+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4c140eb0 con 0x7f2c4c15eb00 2024-04-24T23:28:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.369+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4c0694c0 con 0x7f2c4c15eb00 2024-04-24T23:28:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.370+0000 7f2c437fe640 1 -- 172.21.15.26:0/2970635312 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c4401b070 con 0x7f2c4c15eb00 2024-04-24T23:28:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.370+0000 7f2c437fe640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.370+0000 7f2c437fe640 1 -- 172.21.15.26:0/2970635312 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7f2c4408a0f0 con 0x7f2c4c15eb00 2024-04-24T23:28:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.370+0000 7f2c4a59c640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.371+0000 7f2c4a59c640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c034000 tx=0x7f2c3c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.375+0000 7f2c437fe640 1 -- 172.21.15.26:0/2970635312 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c44058730 con 0x7f2c4c15eb00 2024-04-24T23:28:46.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.554+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 --> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c4c06b780 con 0x7f2c2c06be30 2024-04-24T23:28:46.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.555+0000 7f2c437fe640 1 -- 172.21.15.26:0/2970635312 <== mgr.4110 v2:172.21.15.26:6832/3792722438 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13375 (secure 0 0 0) 0x7f2c4c06b780 con 0x7f2c2c06be30 2024-04-24T23:28:46.556 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:46.556 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-24T23:28:46.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c2c06be30 msgr2=0x7f2c2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c034000 tx=0x7f2c3c02e000 comp rx=0 tx=0).stop 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c15eb00 msgr2=0x7f2c4c13ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c15eb00 0x7f2c4c13ce20 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400c700 tx=0x7f2c4400cbc0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 shutdown_connections 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c15eb00 0x7f2c4c13ce20 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7f2c4c143c40 0x7f2c4c155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 --2- 172.21.15.26:0/2970635312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c4c0cc860 0x7f2c4c15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 >> 172.21.15.26:0/2970635312 conn(0x7f2c4c06b300 msgr2=0x7f2c4c166760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 shutdown_connections 2024-04-24T23:28:46.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.557+0000 7f2c50a1c640 1 -- 172.21.15.26:0/2970635312 wait complete. 2024-04-24T23:28:46.568 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-04-24T23:28:45.417533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":281028,"kb_used_data":10792,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754693692,"statfs":{"total":773094113280,"available":772806340608,"internally_reserved":0,"allocated":11051008,"data_stored":5454280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002308"},"pg_stats":[{"pgid":"1.0","version":"99'119","reported_seq":325,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T23:28:26.818991+0000","last_change":"2024-04-24T23:28:26.819026+0000","last_active":"2024-04-24T23:28:26.818991+0000","last_peered":"2024-04-24T23:28:26.818991+0000","last_clean":"2024-04-24T23:28:26.818991+0000","last_became_active":"2024-04-24T23:20:44.372736+0000","last_became_peered":"2024-04-24T23:20:44.372736+0000","last_unstale":"2024-04-24T23:28:26.818991+0000","last_undegraded":"2024-04-24T23:28:26.818991+0000","last_fullsized":"2024-04-24T23:28:26.818991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'119","last_scrub_stamp":"2024-04-24T23:28:26.818903+0000","last_deep_scrub":"99'119","last_deep_scrub_stamp":"2024-04-24T23:28:26.818903+0000","last_clean_scrub_stamp":"2024-04-24T23:28:26.818903+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:33.511654+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574947,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37120,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334720,"statfs":{"total":96636764160,"available":96598753280,"internally_reserved":0,"allocated":1826816,"data_stored":1124561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36544,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335296,"statfs":{"total":96636764160,"available":96599343104,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36548,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335292,"statfs":{"total":96636764160,"available":96599339008,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36544,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335296,"statfs":{"total":96636764160,"available":96599343104,"internally_reserved":0,"allocated":1232896,"data_stored":534193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37120,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334720,"statfs":{"total":96636764160,"available":96598753280,"internally_reserved":0,"allocated":1826816,"data_stored":1124561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:28:46.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-24T23:28:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.669+0000 7efc8b476640 1 Processor -- start 2024-04-24T23:28:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.669+0000 7efc8b476640 1 -- start start 2024-04-24T23:28:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.670+0000 7efc8b476640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc840d0750 0x7efc84165d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.670+0000 7efc8b476640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8415ef60 con 0x7efc8415adc0 2024-04-24T23:28:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.670+0000 7efc8b476640 1 -- --> v1:172.21.15.29:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8415f240 con 0x7efc840ad7b0 2024-04-24T23:28:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.670+0000 7efc8b476640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7efc8415f520 con 0x7efc840d0750 2024-04-24T23:28:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.670+0000 7efc89a13640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc840d0750 0x7efc84165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.670+0000 7efc89212640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efc8415adc0 0x7efc840b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56414/0 (socket says 172.21.15.26:56414) 2024-04-24T23:28:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.670+0000 7efc89a13640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc840d0750 0x7efc84165d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38484/0 (socket says 172.21.15.26:38484) 2024-04-24T23:28:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.670+0000 7efc89212640 1 -- 172.21.15.26:0/2127259856 learned_addr learned my addr 172.21.15.26:0/2127259856 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-24T23:28:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.670+0000 7efc89a13640 1 -- 172.21.15.26:0/2127259856 >> v1:172.21.15.29:6789/0 conn(0x7efc840ad7b0 legacy=0x7efc840d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.671+0000 7efc89a13640 1 -- 172.21.15.26:0/2127259856 >> v1:172.21.15.26:6789/0 conn(0x7efc8415adc0 legacy=0x7efc840b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.671+0000 7efc89a13640 1 -- 172.21.15.26:0/2127259856 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8415f800 con 0x7efc840d0750 2024-04-24T23:28:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.671+0000 7efc7bfff640 1 -- 172.21.15.26:0/2127259856 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1991559456 0 0) 0x7efc8415ef60 con 0x7efc8415adc0 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.671+0000 7efc89a13640 1 --2- 172.21.15.26:0/2127259856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc840d0750 0x7efc84165d50 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7efc80009dc0 tx=0x7efc8002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a0a9736b2a7325 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.671+0000 7efc7bfff640 1 -- 172.21.15.26:0/2127259856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80007740 con 0x7efc840d0750 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.671+0000 7efc7bfff640 1 -- 172.21.15.26:0/2127259856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc800078e0 con 0x7efc840d0750 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.671+0000 7efc7bfff640 1 -- 172.21.15.26:0/2127259856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80007cb0 con 0x7efc840d0750 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.672+0000 7efc8b476640 1 -- 172.21.15.26:0/2127259856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc840d0750 msgr2=0x7efc84165d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.672+0000 7efc8b476640 1 --2- 172.21.15.26:0/2127259856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc840d0750 0x7efc84165d50 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7efc80009dc0 tx=0x7efc8002a640 comp rx=0 tx=0).stop 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.672+0000 7efc8b476640 1 -- 172.21.15.26:0/2127259856 shutdown_connections 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.672+0000 7efc8b476640 1 --2- 172.21.15.26:0/2127259856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc840d0750 0x7efc84165d50 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.672+0000 7efc8b476640 1 -- 172.21.15.26:0/2127259856 >> 172.21.15.26:0/2127259856 conn(0x7efc8406b540 msgr2=0x7efc8406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.672+0000 7efc8b476640 1 -- 172.21.15.26:0/2127259856 shutdown_connections 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.672+0000 7efc8b476640 1 -- 172.21.15.26:0/2127259856 wait complete. 2024-04-24T23:28:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.673+0000 7efc8b476640 1 Processor -- start 2024-04-24T23:28:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.673+0000 7efc8b476640 1 -- start start 2024-04-24T23:28:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc8b476640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc840ad7b0 0x7efc841476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc8b476640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc840d0750 0x7efc840af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc8b476640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc8415adc0 0x7efc840af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc8b476640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc8415e9c0 con 0x7efc8415adc0 2024-04-24T23:28:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc8b476640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc8415b1a0 con 0x7efc840d0750 2024-04-24T23:28:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc8b476640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7efc8415eb40 con 0x7efc840ad7b0 2024-04-24T23:28:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89a13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc8415adc0 0x7efc840af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89a13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc8415adc0 0x7efc840af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38486/0 (socket says 172.21.15.26:38486) 2024-04-24T23:28:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89a13640 1 -- 172.21.15.26:0/803651402 learned_addr learned my addr 172.21.15.26:0/803651402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89212640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc840ad7b0 0x7efc841476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc88a11640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc840d0750 0x7efc840af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89a13640 1 -- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc840ad7b0 msgr2=0x7efc841476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89a13640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc840ad7b0 0x7efc841476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89a13640 1 -- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc840d0750 msgr2=0x7efc840af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89a13640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc840d0750 0x7efc840af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89a13640 1 -- 172.21.15.26:0/803651402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc84147bf0 con 0x7efc8415adc0 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc89212640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc840ad7b0 0x7efc841476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.674+0000 7efc88a11640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc840d0750 0x7efc840af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.675+0000 7efc89a13640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc8415adc0 0x7efc840af580 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7efc80009ee0 tx=0x7efc80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.675+0000 7efc79ffb640 1 -- 172.21.15.26:0/803651402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80040070 con 0x7efc8415adc0 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.675+0000 7efc79ffb640 1 -- 172.21.15.26:0/803651402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc80031c40 con 0x7efc8415adc0 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.675+0000 7efc79ffb640 1 -- 172.21.15.26:0/803651402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80003990 con 0x7efc8415adc0 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.675+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8409ccc0 con 0x7efc8415adc0 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.675+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8409d280 con 0x7efc8415adc0 2024-04-24T23:28:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.676+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc84075610 con 0x7efc8415adc0 2024-04-24T23:28:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.676+0000 7efc79ffb640 1 -- 172.21.15.26:0/803651402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc80039040 con 0x7efc8415adc0 2024-04-24T23:28:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.677+0000 7efc79ffb640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc6806be60 0x7efc6806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.677+0000 7efc79ffb640 1 -- 172.21.15.26:0/803651402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7efc800abe20 con 0x7efc8415adc0 2024-04-24T23:28:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.677+0000 7efc89212640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc6806be60 0x7efc6806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.677+0000 7efc89212640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc6806be60 0x7efc6806e320 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efc7c000f60 tx=0x7efc7c007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.681+0000 7efc79ffb640 1 -- 172.21.15.26:0/803651402 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc84147bf0 con 0x7efc8415adc0 2024-04-24T23:28:46.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.866+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7efc8415e730 con 0x7efc8415adc0 2024-04-24T23:28:46.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.869+0000 7efc79ffb640 1 -- 172.21.15.26:0/803651402 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7efc8002ae20 con 0x7efc8415adc0 2024-04-24T23:28:46.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.870+0000 7efc79ffb640 1 -- 172.21.15.26:0/803651402 <== mon.0 v2:172.21.15.26:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efc8007aa90 con 0x7efc8415adc0 2024-04-24T23:28:46.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc6806be60 msgr2=0x7efc6806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc6806be60 0x7efc6806e320 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efc7c000f60 tx=0x7efc7c007ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc8415adc0 msgr2=0x7efc840af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc8415adc0 0x7efc840af580 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7efc80009ee0 tx=0x7efc80004a40 comp rx=0 tx=0).stop 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 shutdown_connections 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7efc6806be60 0x7efc6806e320 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc8415adc0 0x7efc840af580 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc840d0750 0x7efc840af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 --2- 172.21.15.26:0/803651402 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7efc840ad7b0 0x7efc841476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 >> 172.21.15.26:0/803651402 conn(0x7efc8406b540 msgr2=0x7efc8406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 shutdown_connections 2024-04-24T23:28:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.872+0000 7efc8b476640 1 -- 172.21.15.26:0/803651402 wait complete. 2024-04-24T23:28:46.885 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:28:46.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:46.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.985+0000 7fcf1dc1c640 1 Processor -- start 2024-04-24T23:28:46.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.985+0000 7fcf1dc1c640 1 -- start start 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf1dc1c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf1815a480 0x7fcf180b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf1dc1c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1806e2f0 0x7fcf18071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf1dc1c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf180b2e80 con 0x7fcf1815a860 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf1dc1c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fcf180b3120 con 0x7fcf1815a480 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf1dc1c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fcf180b33c0 con 0x7fcf1806e2f0 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf177fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf1815a480 0x7fcf180b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf177fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf1815a480 0x7fcf180b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38498/0 (socket says 172.21.15.26:38498) 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf177fe640 1 -- 172.21.15.26:0/3901298090 learned_addr learned my addr 172.21.15.26:0/3901298090 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf16ffd640 1 --1- 172.21.15.26:0/3901298090 >> v1:172.21.15.26:6789/0 conn(0x7fcf1815a860 0x7fcf1806dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56430/0 (socket says 172.21.15.26:56430) 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.986+0000 7fcf17fff640 1 --2- 172.21.15.26:0/3901298090 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1806e2f0 0x7fcf18071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.987+0000 7fcf177fe640 1 -- 172.21.15.26:0/3901298090 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1806e2f0 msgr2=0x7fcf18071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:28:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.987+0000 7fcf177fe640 1 --2- 172.21.15.26:0/3901298090 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1806e2f0 0x7fcf18071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.987+0000 7fcf177fe640 1 -- 172.21.15.26:0/3901298090 >> v1:172.21.15.26:6789/0 conn(0x7fcf1815a860 legacy=0x7fcf1806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.987+0000 7fcf177fe640 1 -- 172.21.15.26:0/3901298090 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf180b36a0 con 0x7fcf1815a480 2024-04-24T23:28:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.987+0000 7fcf17fff640 1 --2- 172.21.15.26:0/3901298090 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1806e2f0 0x7fcf18071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.987+0000 7fcf177fe640 1 --2- 172.21.15.26:0/3901298090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 0x7fcf180b48f0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009870 tx=0x7fcf0802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38993eb847654830 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.987+0000 7fcf167fc640 1 -- 172.21.15.26:0/3901298090 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08037070 con 0x7fcf1815a480 2024-04-24T23:28:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.987+0000 7fcf167fc640 1 -- 172.21.15.26:0/3901298090 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fcf08032440 con 0x7fcf1815a480 2024-04-24T23:28:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.987+0000 7fcf167fc640 1 -- 172.21.15.26:0/3901298090 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08032830 con 0x7fcf1815a480 2024-04-24T23:28:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.988+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/3901298090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 msgr2=0x7fcf180b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.988+0000 7fcf1dc1c640 1 --2- 172.21.15.26:0/3901298090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 0x7fcf180b48f0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009870 tx=0x7fcf0802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.988+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/3901298090 shutdown_connections 2024-04-24T23:28:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.988+0000 7fcf1dc1c640 1 --2- 172.21.15.26:0/3901298090 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1806e2f0 0x7fcf18071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.988+0000 7fcf1dc1c640 1 --2- 172.21.15.26:0/3901298090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 0x7fcf180b48f0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.988+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/3901298090 >> 172.21.15.26:0/3901298090 conn(0x7fcf1806b300 msgr2=0x7fcf1806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.988+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/3901298090 shutdown_connections 2024-04-24T23:28:46.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.988+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/3901298090 wait complete. 2024-04-24T23:28:46.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.989+0000 7fcf1dc1c640 1 Processor -- start 2024-04-24T23:28:46.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.989+0000 7fcf1dc1c640 1 -- start start 2024-04-24T23:28:46.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf1dc1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf1806e2f0 0x7fcf18144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf1dc1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 0x7fcf181452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf1dc1c640 1 --2- >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1815a860 0x7fcf181466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf1dc1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf180b4e30 con 0x7fcf1815a480 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf1dc1c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcf180b3c60 con 0x7fcf1806e2f0 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf1dc1c640 1 -- --> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] -- mon_getmap magic: 0 -- 0x7fcf180b4fb0 con 0x7fcf1815a860 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 0x7fcf181452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 0x7fcf181452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38508/0 (socket says 172.21.15.26:38508) 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf16ffd640 1 -- 172.21.15.26:0/538796020 learned_addr learned my addr 172.21.15.26:0/538796020 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf177fe640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf1806e2f0 0x7fcf18144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.990+0000 7fcf17fff640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1815a860 0x7fcf181466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcf16ffd640 1 -- 172.21.15.26:0/538796020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1815a860 msgr2=0x7fcf181466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcf16ffd640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1815a860 0x7fcf181466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcf16ffd640 1 -- 172.21.15.26:0/538796020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf1806e2f0 msgr2=0x7fcf18144db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcf16ffd640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf1806e2f0 0x7fcf18144db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcf16ffd640 1 -- 172.21.15.26:0/538796020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf181695a0 con 0x7fcf1815a480 2024-04-24T23:28:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcf16ffd640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 0x7fcf181452f0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c4d0 tx=0x7fcf0c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcefffff640 1 -- 172.21.15.26:0/538796020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c018070 con 0x7fcf1815a480 2024-04-24T23:28:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcefffff640 1 -- 172.21.15.26:0/538796020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fcf0c007480 con 0x7fcf1815a480 2024-04-24T23:28:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf18168740 con 0x7fcf1815a480 2024-04-24T23:28:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcefffff640 1 -- 172.21.15.26:0/538796020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0049e0 con 0x7fcf1815a480 2024-04-24T23:28:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.991+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf18168d00 con 0x7fcf1815a480 2024-04-24T23:28:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.992+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf18067c30 con 0x7fcf1815a480 2024-04-24T23:28:46.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.993+0000 7fcefffff640 1 -- 172.21.15.26:0/538796020 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf0c01b070 con 0x7fcf1815a480 2024-04-24T23:28:46.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.993+0000 7fcefffff640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcf0406be30 0x7fcf0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.993+0000 7fcefffff640 1 -- 172.21.15.26:0/538796020 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5115+0+0 (secure 0 0 0) 0x7fcf0c089f70 con 0x7fcf1815a480 2024-04-24T23:28:46.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.993+0000 7fcf177fe640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcf0406be30 0x7fcf0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.993+0000 7fcf177fe640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcf0406be30 0x7fcf0406e2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fcf08034000 tx=0x7fcf0802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:46.997+0000 7fcefffff640 1 -- 172.21.15.26:0/538796020 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf18067c30 con 0x7fcf1815a480 2024-04-24T23:28:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.218+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf180b2970 con 0x7fcf1815a480 2024-04-24T23:28:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.219+0000 7fcefffff640 1 -- 172.21.15.26:0/538796020 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+3867 (secure 0 0 0) 0x7fcf0c058490 con 0x7fcf1815a480 2024-04-24T23:28:47.220 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-24T23:28:47.220 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":115,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3372521943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3372521943},{"type":"v1","addr":"172.21.15.26:6837","nonce":3372521943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":7400,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/416738894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":416738894},{"type":"v1","addr":"172.21.15.29:6833","nonce":416738894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":7403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/369041117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":369041117},{"type":"v1","addr":"172.21.15.29:6835","nonce":369041117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2294542474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2294542474},{"type":"v1","addr":"172.21.15.26:6835","nonce":2294542474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115}],"filesystems":[]} 2024-04-24T23:28:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 115 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcf0406be30 msgr2=0x7fcf0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcf0406be30 0x7fcf0406e2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fcf08034000 tx=0x7fcf0802e000 comp rx=0 tx=0).stop 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 msgr2=0x7fcf181452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 0x7fcf181452f0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c4d0 tx=0x7fcf0c00c990 comp rx=0 tx=0).stop 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 shutdown_connections 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:6832/3792722438,v1:172.21.15.26:6833/3792722438] conn(0x7fcf0406be30 0x7fcf0406e2f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.29:3301/0,v1:172.21.15.29:6790/0] conn(0x7fcf1815a860 0x7fcf181466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf1815a480 0x7fcf181452f0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 --2- 172.21.15.26:0/538796020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf1806e2f0 0x7fcf18144db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 >> 172.21.15.26:0/538796020 conn(0x7fcf1806b300 msgr2=0x7fcf180cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 shutdown_connections 2024-04-24T23:28:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-24T23:28:47.221+0000 7fcf1dc1c640 1 -- 172.21.15.26:0/538796020 wait complete. 2024-04-24T23:28:47.233 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-24T23:28:47.233 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-24T23:28:47.233 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:28:47.233 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.242 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:28:47.242 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:28:47.242 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.294 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:28:47.295 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:28:47.295 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.307 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:28:47.307 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:28:47.307 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.360 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:28:47.360 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-24T23:28:47.361 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-24T23:28:47.361 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.389 INFO:tasks.ceph.osd.0:Stopped 2024-04-24T23:28:47.389 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-24T23:28:47.389 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.461 INFO:tasks.ceph.osd.1:Stopped 2024-04-24T23:28:47.461 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-24T23:28:47.461 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.528 INFO:tasks.ceph.osd.2:Stopped 2024-04-24T23:28:47.528 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-24T23:28:47.528 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.599 INFO:tasks.ceph.osd.3:Stopped 2024-04-24T23:28:47.599 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-24T23:28:47.599 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.635 INFO:tasks.ceph.osd.4:Stopped 2024-04-24T23:28:47.635 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-24T23:28:47.635 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.707 INFO:tasks.ceph.osd.5:Stopped 2024-04-24T23:28:47.707 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-24T23:28:47.707 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.778 INFO:tasks.ceph.osd.6:Stopped 2024-04-24T23:28:47.778 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-24T23:28:47.778 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.849 INFO:tasks.ceph.osd.7:Stopped 2024-04-24T23:28:47.849 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-24T23:28:47.849 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-24T23:28:47.850 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.876 INFO:tasks.ceph.mgr.y:Stopped 2024-04-24T23:28:47.876 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-24T23:28:47.876 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.920 INFO:tasks.ceph.mgr.x:Stopped 2024-04-24T23:28:47.920 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-24T23:28:47.920 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-24T23:28:47.921 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.932 INFO:tasks.ceph.mon.a:Stopped 2024-04-24T23:28:47.932 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-24T23:28:47.932 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:47.973 INFO:tasks.ceph.mon.b:Stopped 2024-04-24T23:28:47.973 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-24T23:28:47.973 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:48.025 INFO:tasks.ceph.mon.c:Stopped 2024-04-24T23:28:48.025 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-24T23:28:48.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Scrub error on inode' | head -n 1 2024-04-24T23:28:48.068 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:28:48.069 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-24T23:28:48.247 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:28:48.247 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-24T23:28:48.370 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:28:48.370 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-24T23:28:48.435 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:28:48.436 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-24T23:28:48.484 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:28:48.485 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-24T23:28:48.630 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:28:48.630 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-24T23:28:48.700 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:28:48.700 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-24T23:28:48.750 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:28:48.750 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-24T23:28:48.817 INFO:tasks.ceph:Archiving mon data... 2024-04-24T23:28:48.818 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/data/mon.a.tgz 2024-04-24T23:28:48.818 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-04-24T23:28:48.835 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.2DSJC17rnk 2024-04-24T23:28:48.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2DSJC17rnk 2024-04-24T23:28:48.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.2DSJC17rnk 2024-04-24T23:28:49.080 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.2DSJC17rnk is 443KB 2024-04-24T23:28:49.180 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.2DSJC17rnk 2024-04-24T23:28:49.195 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/data/mon.b.tgz 2024-04-24T23:28:49.196 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-04-24T23:28:49.214 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.6RVPFLkEAf 2024-04-24T23:28:49.215 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.6RVPFLkEAf 2024-04-24T23:28:49.392 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.6RVPFLkEAf 2024-04-24T23:28:49.477 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.6RVPFLkEAf is 451KB 2024-04-24T23:28:49.575 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.6RVPFLkEAf 2024-04-24T23:28:49.592 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/data/mon.c.tgz 2024-04-24T23:28:49.592 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-04-24T23:28:49.645 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.tqqrBKjMvA 2024-04-24T23:28:49.646 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.tqqrBKjMvA 2024-04-24T23:28:49.799 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.tqqrBKjMvA 2024-04-24T23:28:49.882 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.tqqrBKjMvA is 451KB 2024-04-24T23:28:49.995 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.tqqrBKjMvA 2024-04-24T23:28:50.012 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-24T23:28:50.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T23:28:50.015 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T23:28:50.098 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-24T23:28:50.131 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-24T23:28:50.131 INFO:tasks.ceph:Archiving crash dumps... 2024-04-24T23:28:50.133 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/remote/smithi026/crash 2024-04-24T23:28:50.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T23:28:50.166 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/remote/smithi029/crash 2024-04-24T23:28:50.167 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T23:28:50.195 INFO:tasks.ceph:Shutting down logrotate 2024-04-24T23:28:50.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:28:50.211 DEBUG:teuthology.orchestra.run.smithi029:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:28:50.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-24T23:28:50.317 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-24T23:28:50.341 INFO:tasks.ceph:Compressing logs... 2024-04-24T23:28:50.342 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:28:50.360 DEBUG:teuthology.orchestra.run.smithi029:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:28:50.383 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62470.log 2024-04-24T23:28:50.383 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58603.log 2024-04-24T23:28:50.383 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66369.log 2024-04-24T23:28:50.438 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34797.log 2024-04-24T23:28:50.438 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T23:28:50.438 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34921.log 2024-04-24T23:28:50.438 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34797.log.gz 2024-04-24T23:28:50.438 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40415.log 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34877.log 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34921.log.gz 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.1.40415.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40415.log.gz 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34877.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34877.log.gz 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40420.log 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34837.log 2024-04-24T23:28:50.439 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.1.40420.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.1.40420.log.gz 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr: -- /var/log/ceph/ceph-client.3.40815.log 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34837.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34837.log.gz 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.3.40815.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.40815.log.gz 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35008.log 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35048.log 2024-04-24T23:28:50.440 INFO:teuthology.orchestra.run.smithi029.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35008.log.gz 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35048.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35048.log.gz 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.40819.log 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34968.log 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.3.40819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.40819.log.gz 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35088.log 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34968.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34968.log.gz 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35088.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35088.log.gz 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-24T23:28:50.441 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mds.d.log: 94.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-24T23:28:50.574 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mon.b.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65886.log 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58603.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62470.log.gz 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66369.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58603.log.gz 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58353.log 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66369.log.gz 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58784.log 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65886.log.gz 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62976.log 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58784.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58353.log.gz 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54230.log 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52207.log 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58784.log.gz 2024-04-24T23:28:50.688 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62976.log.gz 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58178.log 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54230.log.gz 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70675.log 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50500.log 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.52207.log.gz 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70675.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58178.log.gz 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40721.log 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70675.log.gz 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46466.log 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42065.log 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50500.log.gz 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67951.log 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40721.log.gz 2024-04-24T23:28:50.689 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46466.log.gz 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65844.log 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71275.log 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67951.log.gz 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65634.log 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42065.log.gz 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65844.log.gz 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50416.log 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56679.log 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71275.log.gz 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50416.log: gzip -5 --verbose -- 89.6% /var/log/ceph/ceph-client.admin.48242.log 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65634.log.gz 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71065.log 2024-04-24T23:28:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67699.log 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42552.log 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56679.log.gz 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67699.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.48242.log.gz 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71065.log.gz 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66285.log 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67699.log.gz 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.42552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58890.log 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42552.log.gz 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52300.log 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66285.log.gz/var/log/ceph/ceph-client.admin.58890.log: 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54869.log 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52300.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58890.log.gz 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50063.log 2024-04-24T23:28:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46508.log 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50063.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54869.log.gz 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48625.log 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52300.log.gz 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50063.log.gz 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73400.log 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46508.log.gz 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69308.log 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52366.log 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48625.log.gz 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73400.log.gz 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73500.log 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.69308.log.gz 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53885.log 2024-04-24T23:28:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45948.log 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52366.log.gz 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53885.log: gzip 88.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.73500.log.gz -- 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.41223.log 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53707.log 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53885.log.gz 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45948.log.gz 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72014.log 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41223.log.gz 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58932.log 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72014.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.53707.log.gz /var/log/ceph/ceph-client.admin.45325.log 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58932.log: gzip 88.8% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.72014.log.gz /var/log/ceph/ceph-client.admin.53978.log 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.693 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58932.log.gz 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71317.log 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45325.log.gz 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44293.log 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72140.log 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53978.log.gz 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44293.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71317.log.gz 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55823.log 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44293.log.gz 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71497.log 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.72140.log.gz -5 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65532.log 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55823.log: /var/log/ceph/ceph-client.admin.71497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50752.log 2024-04-24T23:28:50.694 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71497.log.gz 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46680.log 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55823.log.gz 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50752.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.65532.log.gz --verbose 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.66723.log 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50752.log.gz 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55163.log 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46680.log.gz 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73316.log 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7%/var/log/ceph/ceph-client.admin.55163.log: -- replaced with /var/log/ceph/ceph-client.admin.66723.log.gz 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61655.log 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73316.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55163.log.gz 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58518.log 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61655.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73316.log.gz 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39732.log 2024-04-24T23:28:50.695 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61655.log.gz 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57636.log 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58518.log.gz 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.42999.log 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57636.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39732.log.gz 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49097.log 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57636.log.gz 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.2.42999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63102.log 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.42999.log.gz 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45199.log 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63102.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49097.log.gz 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53497.log 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45199.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63102.log.gz 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73358.log 2024-04-24T23:28:50.696 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45199.log.gz 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70057.log 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73358.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53497.log.gz 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46550.log 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73358.log.gz 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40199.log 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70057.log.gz 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44569.log 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40199.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46550.log.gz 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61291.log 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44569.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40199.log.gz 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72285.log 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44569.log.gz 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70015.log 2024-04-24T23:28:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72285.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61291.log.gz 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65058.log 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70015.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72285.log.gz 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70015.log.gz 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57552.log 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51364.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65058.log.gz 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45703.log 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70915.log 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65490.log 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57552.log.gz 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45703.log.gz 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67060.log 2024-04-24T23:28:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66892.log 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67060.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65490.log.gz 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49223.log 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.70915.log.gz 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67060.log.gz 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41435.log 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66850.log 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66892.log.gz 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41435.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.49223.log.gz 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.56613.log 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41435.log.gz 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49013.log 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56613.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66850.log.gz 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40115.log 2024-04-24T23:28:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68593.log 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49013.log.gz 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40115.log: gzip -5 90.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.56613.log.gz 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.52498.log 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40115.log.gz 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47907.log 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63402.log 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45535.log 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68593.log.gz 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.63402.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52498.log.gz 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63402.log.gz 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47907.log.gz 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60903.log 2024-04-24T23:28:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.45073.log -- replaced with /var/log/ceph/ceph-client.admin.45535.log.gz 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46327.log 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64560.log 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60903.log.gz 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46327.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45073.log.gz 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69889.log 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46327.log.gz 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45367.log 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64393.log 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45367.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69889.log.gz 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64560.log.gz 2024-04-24T23:28:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44379.log 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45367.log.gz 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62257.log 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47148.log 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64393.log.gz 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57080.log 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44379.log.gz 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.47148.log: gzip -5 88.2% --verbose -- /var/log/ceph/ceph-client.admin.53395.log -- replaced with /var/log/ceph/ceph-client.admin.62257.log.gz 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47148.log.gz 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51214.log 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57080.log.gz 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70632.log 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44251.log 2024-04-24T23:28:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51214.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53395.log.gz 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65676.log 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51214.log.gz 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44251.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70632.log.gz 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62724.log 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45827.log 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44251.log.gz 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62724.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65676.log.gz 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43678.log 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62724.log.gz 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62001.log 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45827.log: /var/log/ceph/ceph-client.admin.43678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70717.log 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45827.log.gz 2024-04-24T23:28:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60987.log 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43678.log.gz 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62001.log.gz 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48400.log 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58742.log 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60987.log.gz 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48400.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70717.log.gz 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58742.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.48400.log.gz 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71972.log 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58742.log.gz 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44014.log 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.71972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66621.log 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62341.log 2024-04-24T23:28:50.704 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71972.log.gz 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66621.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.44014.log.gz --verbose 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.48971.log 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62341.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66621.log.gz 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71930.log 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62341.log.gz 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72560.log 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48971.log.gz 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49139.log 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57342.log 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71930.log.gz 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49139.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72560.log.gz 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67867.log 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49139.log.gz 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40157.log 2024-04-24T23:28:50.705 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55459.log 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57342.log.gz 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40157.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67867.log.gz 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41561.log 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40157.log.gz 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.55459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51893.log 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55459.log.gz 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41561.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41561.log.gz 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67235.log 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60024.log 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60331.log 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60024.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67235.log.gz 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51893.log.gz 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43762.log 2024-04-24T23:28:50.706 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60024.log.gz 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40893.log 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65592.log 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43762.log.gz 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40893.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60331.log.gz 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44336.log 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40893.log.gz 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55541.log 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65592.log.gz 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44336.log: /var/log/ceph/ceph-client.admin.73190.log 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55541.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44336.log.gz 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43930.log 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73190.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.55541.log.gz -- 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.40903.log 2024-04-24T23:28:50.707 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73190.log.gz 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72812.log 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40903.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43930.log.gz -5 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40409.log 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40903.log.gz 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57678.log 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65274.log 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72812.log.gz 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40409.log.gz 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54487.log 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57678.log.gz 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70500.log 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54487.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T23:28:50.708 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.65274.log.gz 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70500.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54487.log.gz 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63834.log 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71149.log 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.63834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67741.log 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71149.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70500.log.gz 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63834.log.gz 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67485.log 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71149.log.gz 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66934.log 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67741.log: /var/log/ceph/ceph-client.admin.67485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67443.log 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67741.log.gz 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66934.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67485.log.gz 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54146.log 2024-04-24T23:28:50.709 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66934.log.gz 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67993.log 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67443.log: /var/log/ceph/ceph-client.admin.54146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53175.log 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67443.log.gz 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67993.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54146.log.gz 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49368.log 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67993.log.gz 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39943.log 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53175.log: /var/log/ceph/ceph-client.admin.49368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68659.log 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53175.log.gz 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49368.log.gz 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43342.log 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64153.log 2024-04-24T23:28:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68659.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39943.log.gz 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52828.log 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64153.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43342.log.gz 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.68659.log.gz 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67615.log 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71233.log 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52828.log.gz 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.64153.log.gz 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44653.log 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67615.log.gz 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43174.log 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71233.log: /var/log/ceph/ceph-client.admin.44653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44098.log 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71233.log.gz 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44653.log.gz 2024-04-24T23:28:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66537.log 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43174.log: /var/log/ceph/ceph-client.admin.44098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40892.log 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43174.log.gz 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66537.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44098.log.gz 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54104.log 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66537.log.gz 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41855.log 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47370.log 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40892.log.gz 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47436.log 89.1% 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54104.log.gz 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41855.log.gz 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62096.log 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68527.log 2024-04-24T23:28:50.712 INFO:teuthology.orchestra.run.smithi026.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47370.log.gz 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.62096.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50794.log 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47436.log.gz 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69234.log 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68527.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62096.log.gz 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41981.log 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50794.log.gz 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71762.log 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.68527.log.gz 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5/var/log/ceph/ceph-client.admin.41981.log: --verbose -- /var/log/ceph/ceph-client.admin.50710.log 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.69234.log.gz 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71762.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41981.log.gz 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46596.log 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71762.log.gz 2024-04-24T23:28:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56206.log 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46596.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.50710.log.gz 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.73542.log 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56206.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46596.log.gz 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68934.log 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52870.log 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73542.log.gz 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56206.log.gz 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57164.log 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64430.log 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52870.log: /var/log/ceph/ceph-client.admin.57164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55889.log 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52870.log.gz 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.68934.log.gz 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57164.log.gz 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56409.log 2024-04-24T23:28:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55889.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64430.log.gz -5 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48887.log 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40878.log 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55889.log.gz 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48887.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.56409.log.gz 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49181.log 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48887.log.gz 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72369.log 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68101.log 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40878.log.gz 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72369.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49181.log.gz 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57926.log 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72369.log.gz 2024-04-24T23:28:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.68101.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54743.log 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57926.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68101.log.gz 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34442.log 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68725.log 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57926.log.gz 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.34442.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54743.log.gz 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45990.log 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34442.log.gz 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43468.log 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45990.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68725.log.gz 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43468.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45990.log.gz 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43300.log 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43468.log.gz 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46764.log 2024-04-24T23:28:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42203.log 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43300.log.gz 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53791.log 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42203.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46764.log.gz 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46159.log 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42203.log.gz 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52996.log 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53791.log: /var/log/ceph/ceph-client.admin.46159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69607.log 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53791.log.gz 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52996.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46159.log.gz 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65340.log 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69607.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52996.log.gz 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47568.log 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57884.log 2024-04-24T23:28:50.717 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69607.log.gz 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43636.log 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.65340.log.gz 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57884.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47568.log.gz 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49494.log 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57884.log.gz 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43636.log: /var/log/ceph/ceph-client.admin.65981.log 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56557.log 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49494.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43636.log.gz 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69739.log 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49494.log.gz 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40241.log 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65981.log.gz 2024-04-24T23:28:50.718 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69739.log: gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-client.admin.57248.log 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56557.log.gz 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40637.log 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40241.log: /var/log/ceph/ceph-client.admin.57248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52912.log 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69739.log.gz 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40241.log.gz 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57248.log.gz 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72854.log 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61459.log 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40637.log.gz 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52912.log.gz 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59958.log 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61459.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72854.log.gz 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69541.log 2024-04-24T23:28:50.719 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61459.log.gz 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48929.log 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59958.log: /var/log/ceph/ceph-client.admin.69541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66327.log 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41239.log 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.59958.log.gz 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48929.log.gz 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66327.log: gzip -5 --verbose 91.3% -- /var/log/ceph/ceph-client.admin.48437.log -- replaced with /var/log/ceph/ceph-client.admin.69541.log.gz 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41239.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66327.log.gz 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43258.log 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68035.log 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48437.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41239.log.gz 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55691.log 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48437.log.gz 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47310.log 2024-04-24T23:28:50.720 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43258.log.gz 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49536.log 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47310.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68035.log.gz 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49536.log: 91.0% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47310.log.gz -- replaced with /var/log/ceph/ceph-client.admin.55691.log.gz 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46285.log 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mds.c.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49536.log.gz 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51094.log 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71541.log 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46285.log.gz 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.51094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66065.log 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51094.log.gz 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62176.log 2024-04-24T23:28:50.721 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71541.log: /var/log/ceph/ceph-client.admin.66065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58476.log 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71541.log.gz 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66065.log.gz 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55592.log 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54911.log 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58476.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62176.log.gz 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45745.log/var/log/ceph/ceph-client.admin.55592.log: 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58476.log.gz 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49978.log 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55592.log.gz 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45745.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54911.log.gz 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67400.log 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45745.log.gz 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61501.log 2024-04-24T23:28:50.722 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49978.log: /var/log/ceph/ceph-client.admin.67400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65208.log 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49978.log.gz 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61249.log 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67400.log.gz 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65208.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61501.log.gz 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72243.log 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63636.log 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72243.log: gzip 91.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.53581.log 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61249.log.gz 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65208.log.gz 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72243.log.gz 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69001.log 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58136.log 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53581.log.gz 2024-04-24T23:28:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58691.log 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.63636.log.gz 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58136.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58691.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58136.log.gz 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72327.log 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69001.log.gz 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58691.log.gz 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63768.log 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72327.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71888.log 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72327.log.gz 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59310.log 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63768.log: /var/log/ceph/ceph-client.admin.71888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65100.log 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59310.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71888.log.gz 2024-04-24T23:28:50.724 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63768.log.gz 89.2% 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59310.log.gz 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49452.log 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41238.log 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8%/var/log/ceph/ceph-client.admin.49452.log: -- replaced with /var/log/ceph/ceph-client.admin.65100.log.gz 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69382.log 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69847.log 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41238.log: /var/log/ceph/ceph-client.admin.69382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39774.log 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69847.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41238.log.gz 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.69382.log.gz 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69847.log.gz 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63398.log 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39774.log: -- /var/log/ceph/ceph-client.admin.45493.log 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.63398.log: 0.0% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39774.log.gz 2024-04-24T23:28:50.725 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.0.63398.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39651.log 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45493.log: -- /var/log/ceph/ceph-client.admin.59058.log 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39651.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64021.log 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45493.log.gz 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39651.log.gz 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50105.log 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64021.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59058.log.gz 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63210.log 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45451.log 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9%/var/log/ceph/ceph-client.admin.63210.log: -- replaced with /var/log/ceph/ceph-client.admin.64021.log.gz/var/log/ceph/ceph-client.admin.50105.log: 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63210.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69973.log 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50105.log.gz 2024-04-24T23:28:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59826.log 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69973.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45451.log.gz 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57426.log 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4%/var/log/ceph/ceph-client.admin.59826.log: -- replaced with /var/log/ceph/ceph-client.admin.69973.log.gz 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52244.log 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59826.log.gz 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57426.log.gz 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40073.log 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51430.log 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2%/var/log/ceph/ceph-client.admin.40073.log: -- replaced with /var/log/ceph/ceph-client.admin.52244.log.gz 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40073.log.gz 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64504.log 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54613.log 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64504.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.51430.log.gz 2024-04-24T23:28:50.727 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64504.log.gz 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68311.log 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53133.log 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54613.log.gz 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.68311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68311.log.gz 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55646.log 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39463.log 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53133.log: /var/log/ceph/ceph-client.admin.55646.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53133.log.gz 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55646.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48474.log 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39463.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39463.log.gz 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66023.log 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41603.log 2024-04-24T23:28:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48474.log: /var/log/ceph/ceph-client.admin.66023.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.48474.log.gz 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66023.log.gz 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57968.log 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72770.log 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41603.log: /var/log/ceph/ceph-client.admin.57968.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41603.log.gz 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57968.log.gz 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61917.log 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72770.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72770.log.gz 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71846.log 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68440.log 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61917.log: /var/log/ceph/ceph-client.admin.71846.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61917.log.gz 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71846.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46075.log 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68440.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68440.log.gz 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43384.log 2024-04-24T23:28:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48326.log 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46075.log: /var/log/ceph/ceph-client.admin.43384.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46075.log.gz 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43384.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70348.log 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48326.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.48326.log.gz 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58647.log 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43888.log 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70348.log: /var/log/ceph/ceph-client.admin.58647.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58647.log.gz 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.70348.log.gz 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45283.log 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61833.log 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43888.log: /var/log/ceph/ceph-client.admin.45283.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43888.log.gz 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45283.log.gz 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54062.log 2024-04-24T23:28:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61833.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61833.log.gz 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54571.log 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40469.log 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54062.log: /var/log/ceph/ceph-client.admin.54571.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54062.log.gz 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54571.log.gz 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58974.log 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40469.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40469.log.gz 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64824.log 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67155.log 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58974.log: /var/log/ceph/ceph-client.admin.64824.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58974.log.gz 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65718.log 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0%/var/log/ceph/ceph-client.admin.67155.log: -- replaced with /var/log/ceph/ceph-client.admin.64824.log.gz 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67155.log.gz 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42691.log 2024-04-24T23:28:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49757.log 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65718.log: /var/log/ceph/ceph-client.admin.42691.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65718.log.gz 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42691.log.gz 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59892.log 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49757.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49757.log.gz 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59610.log 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50668.log 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59892.log: /var/log/ceph/ceph-client.0.59610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59610.log.gz 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59892.log.gz 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50584.log 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41645.log 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50668.log: /var/log/ceph/ceph-client.admin.50584.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50668.log.gz 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50584.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55271.log 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41645.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41645.log.gz 2024-04-24T23:28:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73022.log 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64626.log 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55271.log: /var/log/ceph/ceph-client.admin.73022.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55271.log.gz 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73022.log.gz 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42557.log 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64626.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.64626.log.gz 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43216.log 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58052.log 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.42557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42557.log.gz 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65406.log 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43216.log.gz 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58052.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58052.log.gz 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59784.log 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43594.log 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65406.log: /var/log/ceph/ceph-client.admin.59784.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65406.log.gz 2024-04-24T23:28:50.733 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59784.log.gz 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58826.log 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50020.log 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43594.log: /var/log/ceph/ceph-client.admin.58826.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43594.log.gz 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58826.log.gz 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50020.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50020.log.gz 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46415.log 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39985.log 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.46415.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46415.log.gz 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71584.log 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39985.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39985.log.gz 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63060.log 2024-04-24T23:28:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71584.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71584.log.gz 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69673.log 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63060.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63060.log.gz 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48092.log 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69673.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.69673.log.gz 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45619.log 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48092.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.48092.log.gz 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70434.log 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45619.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45619.log.gz 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53833.log 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70434.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70434.log.gz 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61417.log 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53833.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53833.log.gz 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43426.log 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61417.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61417.log.gz 2024-04-24T23:28:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51256.log 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43426.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43426.log.gz 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61791.log 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51256.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51256.log.gz 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69931.log 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61791.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61791.log.gz 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65760.log 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69931.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69931.log.gz 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69805.log 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65760.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65760.log.gz 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45031.log 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69805.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69805.log.gz 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54529.log 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45031.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45031.log.gz 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39555.log 2024-04-24T23:28:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54529.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54529.log.gz 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70849.log 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39555.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39555.log.gz 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68307.log 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70849.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.70849.log.gz 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28381.log 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.68307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68307.log.gz 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44611.log 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.tmp-client.admin.28381.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28381.log.gz 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69271.log 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44611.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44611.log.gz 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63144.log 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69271.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.69271.log.gz 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44527.log 2024-04-24T23:28:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63144.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63144.log.gz 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39859.log 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44527.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44527.log.gz 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50626.log 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39859.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39859.log.gz 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64319.log 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50626.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50626.log.gz 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53623.log 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64319.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.64319.log.gz 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63549.log 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53623.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53623.log.gz 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70981.log 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63549.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63549.log.gz 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67358.log 2024-04-24T23:28:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70981.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70981.log.gz 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64974.log 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67358.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67358.log.gz 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62556.log 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64974.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64974.log.gz 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72602.log 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62556.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62556.log.gz 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41234.log 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72602.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72602.log.gz 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63018.log 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41234.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41234.log.gz 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67529.log 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63018.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63018.log.gz 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67529.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67529.log.gz 2024-04-24T23:28:50.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54953.log 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40553.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57594.log 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54953.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54953.log.gz 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59184.log 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57594.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57594.log.gz 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66411.log 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59184.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59184.log.gz 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60529.log 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66411.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66411.log.gz 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39817.log 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60529.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60529.log.gz 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73106.log 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39817.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39817.log.gz 2024-04-24T23:28:50.740 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67316.log 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73106.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73106.log.gz 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58561.log 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67316.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67316.log.gz 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52762.log 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58561.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58561.log.gz 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52059.log 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52762.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52762.log.gz 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52059.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52059.log.gz 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59100.log 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41519.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62299.log 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59100.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59100.log.gz 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59142.log 2024-04-24T23:28:50.741 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62299.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62299.log.gz 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72896.log 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59142.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59142.log.gz 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44827.log 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72896.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72896.log.gz 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56446.log 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44827.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44827.log.gz 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62384.log 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56446.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56446.log.gz 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43132.log 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62384.log: 91.6% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62384.log.gz 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56520.log 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66766.log 2024-04-24T23:28:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43132.log: /var/log/ceph/ceph-client.admin.56520.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43132.log.gz 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60397.log 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr: 91.9% 89.9% -- replaced with /var/log/ceph/ceph-client.admin.56520.log.gz 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45157.log 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66766.log.gz/var/log/ceph/ceph-client.admin.60397.log: 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60784.log 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59606.log 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60784.log: 91.0% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45157.log.gz 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60397.log.gz 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59352.log 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.59606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47268.log 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59606.log.gz 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60784.log.gz 2024-04-24T23:28:50.743 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62640.log 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58273.log 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47268.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.59352.log.gz 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62640.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66808.log 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47268.log.gz 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62640.log.gz 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63594.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58273.log.gz 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67018.log 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66808.log.gz 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73274.log 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63594.log.gz 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61375.log 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67018.log: /var/log/ceph/ceph-client.admin.73274.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67018.log.gz 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73274.log.gz 2024-04-24T23:28:50.744 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64890.log 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53259.log 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47841.log 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61375.log.gz 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2%/var/log/ceph/ceph-client.admin.53259.log: -- replaced with /var/log/ceph/ceph-client.admin.64890.log.gz 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58010.log 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61333.log 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53259.log.gz 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69345.log 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58010.log.gz 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61333.log: -- replaced with /var/log/ceph/ceph-client.admin.47841.log.gz 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66243.log 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61333.log.gz 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46033.log 2024-04-24T23:28:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51496.log 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.69345.log.gz 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46033.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66243.log.gz 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68485.log 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41477.log 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46033.log.gz 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51496.log.gz 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68485.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68485.log.gz 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56961.log 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44911.log 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56961.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41477.log.gz 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65016.log 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46848.log 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44911.log: 90.9%/var/log/ceph/ceph-client.admin.65016.log: -- replaced with /var/log/ceph/ceph-client.admin.56961.log.gz 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44911.log.gz 2024-04-24T23:28:50.746 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65016.log.gz 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49936.log 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53936.log 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46848.log.gz 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56829.log 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49936.log.gz 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71804.log 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53936.log.gz 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39901.log 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52096.log 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.56829.log.gz 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71804.log.gz 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45241.log 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72686.log 2024-04-24T23:28:50.747 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39901.log.gz 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45241.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.52096.log.gz 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45241.log.gz 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40900.log 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56140.log 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72686.log.gz 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41222.log 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40900.log.gz 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46722.log 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56140.log: /var/log/ceph/ceph-client.admin.41222.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41222.log.gz 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56140.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43804.log 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71107.log 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5%/var/log/ceph/ceph-client.admin.43804.log: -- replaced with /var/log/ceph/ceph-client.admin.46722.log.gz 2024-04-24T23:28:50.748 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47502.log 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43804.log.gz 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64087.log 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71107.log: /var/log/ceph/ceph-client.admin.47502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71023.log 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64087.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71107.log.gz 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47502.log.gz 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64087.log.gz 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64692.log 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50902.log 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71023.log.gz 2024-04-24T23:28:50.749 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72938.log 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62934.log 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50902.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.64692.log.gz 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72938.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50902.log.gz 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72938.log.gz 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45906.log 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48751.log 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62934.log.gz 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61959.log 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45906.log.gz 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72056.log 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48751.log.gz 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64356.log 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44785.log 2024-04-24T23:28:50.750 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5%/var/log/ceph/ceph-client.admin.72056.log: -- replaced with /var/log/ceph/ceph-client.admin.61959.log.gz 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64356.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72056.log.gz 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.64356.log.gz 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41224.log 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44785.log: /var/log/ceph/ceph-client.admin.72728.log 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41224.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49410.log 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44785.log.gz 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41224.log.gz 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40367.log 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55121.log 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2%/var/log/ceph/ceph-client.admin.40367.log: -- replaced with /var/log/ceph/ceph-client.admin.72728.log.gz 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49410.log.gz 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40367.log.gz 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70142.log 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52432.log 2024-04-24T23:28:50.751 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55121.log.gz 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52432.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70142.log.gz 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70275.log 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52432.log.gz 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53665.log 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr: 89.8%/var/log/ceph/ceph-client.admin.70275.log: -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71455.log 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50244.log 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53665.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70275.log.gz 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71455.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53665.log.gz 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41939.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71455.log.gz 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40901.log 2024-04-24T23:28:50.752 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50244.log.gz 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49055.log 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41939.log.gz/var/log/ceph/ceph-client.admin.40901.log: 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50328.log 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40901.log.gz 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57122.log 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49055.log: /var/log/ceph/ceph-client.admin.50328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43720.log 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57122.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49055.log.gz 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50328.log.gz 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57122.log.gz 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73064.log 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67783.log 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43720.log.gz 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70566.log 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73064.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48363.log 2024-04-24T23:28:50.753 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61707.log 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70566.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67783.log.gz 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48363.log: gzip -5 90.3% -- replaced with /var/log/ceph/ceph-client.admin.48363.log.gz 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43846.log 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45577.log 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70566.log.gz 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61707.log.gz 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67825.log 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43846.log: /var/log/ceph/ceph-client.admin.45577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53539.log 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0%/var/log/ceph/ceph-client.admin.67825.log: -- replaced with /var/log/ceph/ceph-client.admin.43846.log.gz 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45577.log.gz 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67825.log.gz 2024-04-24T23:28:50.754 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46371.log 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52133.log 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46371.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53539.log.gz 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40679.log 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46371.log.gz 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53437.log 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52133.log: /var/log/ceph/ceph-client.admin.40679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46201.log 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53437.log: 89.8% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40679.log.gz 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52133.log.gz 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53437.log.gz 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57206.log 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46201.log: -- /var/log/ceph/ceph-client.admin.56372.log 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69475.log 88.8% 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46201.log.gz 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57206.log.gz 2024-04-24T23:28:50.755 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52954.log 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56372.log: /var/log/ceph/ceph-client.admin.69475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61585.log 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52954.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.56372.log.gz 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52954.log.gz 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69475.log.gz 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53301.log 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41687.log 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61585.log.gz 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45409.log 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53301.log.gz 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49317.log 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.42995.log 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41687.log.gz 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45409.log: /var/log/ceph/ceph-client.admin.49317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72644.log 2024-04-24T23:28:50.756 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45409.log.gz 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.2.42995.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49317.log.gz 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.42995.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61071.log 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.72644.log: /var/log/ceph/ceph-client.admin.62766.log 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61071.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46117.log 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72644.log.gz 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61071.log.gz 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62426.log 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50458.log 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62766.log.gz 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62426.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46117.log.gz 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62426.log.gz 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50193.log 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48289.log 2024-04-24T23:28:50.757 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50193.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50458.log.gz 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59688.log 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50193.log.gz 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49620.log 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48289.log: /var/log/ceph/ceph-client.admin.59688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49265.log 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.48289.log.gz 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59688.log.gz 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49620.log: gzip 88.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.41226.log 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49620.log.gz 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53217.log 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43552.log 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49265.log.gz 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53217.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41226.log.gz 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56763.log 2024-04-24T23:28:50.758 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43552.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53217.log.gz 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr: -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43552.log.gz 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.54020.log 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56763.log: /var/log/ceph/ceph-client.admin.67657.log 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50149.log 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54020.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56763.log.gz/var/log/ceph/ceph-client.admin.67657.log: 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54020.log.gz 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66149.log 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67657.log.gz 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50149.log: -- /var/log/ceph/ceph-client.admin.51172.log 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66149.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50149.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62514.log 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.66149.log.gz 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61543.log 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51172.log: /var/log/ceph/ceph-client.admin.62514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72098.log 2024-04-24T23:28:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61543.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51172.log.gz 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62514.log.gz 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61543.log.gz 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59226.log 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51090.log 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72098.log.gz 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67573.log 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59226.log.gz 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.51090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51090.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54405.log 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46956.log 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.67573.log.gz --verbose 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.55205.log 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54405.log: gzip -5/var/log/ceph/ceph-client.admin.46956.log: --verbose -- /var/log/ceph/ceph-client.admin.59739.log 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55205.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46956.log.gz 2024-04-24T23:28:50.760 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54405.log.gz 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54325.log 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55205.log.gz 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60463.log 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59739.log.gz 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57468.log 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54325.log: /var/log/ceph/ceph-client.admin.60463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46806.log 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57468.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54325.log.gz 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57468.log.gzgzip 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50542.log 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60463.log.gz 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52696.log 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46806.log: /var/log/ceph/ceph-client.admin.50542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54699.log 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46806.log.gz 2024-04-24T23:28:50.761 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52696.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50542.log.gz 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54995.log 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54785.log 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52696.log.gz 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54699.log.gz 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71359.log 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9%/var/log/ceph/ceph-client.admin.54785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48583.log 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54995.log.gz 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54785.log.gz 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63702.log 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71359.log: /var/log/ceph/ceph-client.admin.48583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66579.log 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7%/var/log/ceph/ceph-client.admin.63702.log: -- replaced with /var/log/ceph/ceph-client.admin.71359.log.gz 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48583.log.gz 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51298.log 2024-04-24T23:28:50.762 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66579.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63702.log.gz 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61029.log 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66579.log.gz 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60719.log 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51298.log: /var/log/ceph/ceph-client.admin.61029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34732.log 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60719.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61029.log.gz 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61749.log 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.51298.log.gz 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.34732.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60719.log.gz 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71626.log 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34732.log.gz 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57762.log 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61749.log: /var/log/ceph/ceph-client.admin.71626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41897.log 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57762.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71626.log.gz 2024-04-24T23:28:50.763 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61749.log.gz 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57762.log.gz 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62808.log 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57720.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41897.log.gz 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65448.log 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62808.log.gz 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49837.log 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57720.log.gz 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41236.log 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65448.log: /var/log/ceph/ceph-client.admin.49837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56483.log 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41236.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49837.log.gz 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65448.log.gz 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41236.log.gz 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60654.log 2024-04-24T23:28:50.764 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56483.log: /var/log/ceph/ceph-client.admin.45661.log 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59268.log 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.56483.log.gz 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72980.log 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45661.log: /var/log/ceph/ceph-client.admin.59268.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60654.log.gz 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45661.log.gz 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59268.log.gz 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44056.log 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53038.log 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr:gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.72980.log.gz --verbose 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.60589.log 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44056.log: gzip/var/log/ceph/ceph-client.admin.53038.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55037.log 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60589.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53038.log.gz 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44056.log.gz 2024-04-24T23:28:50.765 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50286.log 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60589.log.gz 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68389.log 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55037.log.gz/var/log/ceph/ceph-client.admin.50286.log: 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40511.log 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50286.log.gz 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57510.log 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68389.log: /var/log/ceph/ceph-client.admin.40511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57510.log: 88.8% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40511.log.gz 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68389.log.gz 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57510.log.gz 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50836.log 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43972.log: -- /var/log/ceph/ceph-client.admin.66976.log 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50836.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48158.log 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-04-24T23:28:50.766 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-24T23:28:50.767 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1%/var/log/ceph/ceph-client.admin.48158.log: /var/log/ceph/ceph-client.admin.66976.log: -- replaced with /var/log/ceph/ceph-client.admin.50836.log.gz 2024-04-24T23:28:50.767 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41771.log 2024-04-24T23:28:50.767 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66976.log.gz 2024-04-24T23:28:50.767 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8%/var/log/ceph/ceph-mds.a.log: -- replaced with /var/log/ceph/ceph-client.admin.48158.log.gz 2024-04-24T23:28:50.767 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46638.log 2024-04-24T23:28:50.767 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61875.log 2024-04-24T23:28:50.767 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41771.log.gz 2024-04-24T23:28:50.767 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46638.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46638.log.gz 2024-04-24T23:28:50.768 INFO:teuthology.orchestra.run.smithi029.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-24T23:28:50.768 INFO:teuthology.orchestra.run.smithi029.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-24T23:28:50.768 INFO:teuthology.orchestra.run.smithi029.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-24T23:28:50.816 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44989.log 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71191.log 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61875.log.gz 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44466.log 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71191.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44989.log.gz 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71191.log.gz 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72200.log 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69419.log 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72200.log: /var/log/ceph/ceph-client.admin.44466.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44466.log.gz 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72200.log.gz 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70783.log 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62682.log 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69419.log: /var/log/ceph/ceph-client.admin.70783.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.69419.log.gz 2024-04-24T23:28:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51761.log 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70783.log.gz 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40896.log 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2%/var/log/ceph/ceph-client.admin.51761.log: -- replaced with /var/log/ceph/ceph-client.admin.62682.log.gz 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51761.log.gz 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46890.log 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44209.log 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40896.log.gz 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58094.log 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44209.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44209.log.gz 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46890.log.gz 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57840.log 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52630.log 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58094.log: /var/log/ceph/ceph-client.admin.57840.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58094.log.gz 2024-04-24T23:28:50.818 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57840.log.gz 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45115.log 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65802.log 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45115.log: /var/log/ceph/ceph-client.admin.52630.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45115.log.gz 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr: 90.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55079.log 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52630.log.gz 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40595.log 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65802.log.gz 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63480.log 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55079.log.gz 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56895.log 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63480.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40595.log.gz 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64932.log 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63480.log.gz 2024-04-24T23:28:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61165.log 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64932.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64932.log.gz 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56895.log.gz 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49578.log 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50374.log 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61165.log.gz 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47775.log 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49578.log.gz 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67909.log 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47775.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50374.log.gz 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48845.log 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47775.log.gz/var/log/ceph/ceph-client.admin.67909.log: 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57384.log 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67909.log.gz 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55757.log 2024-04-24T23:28:50.820 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48845.log.gz 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73148.log 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57384.log.gz/var/log/ceph/ceph-client.admin.55757.log: 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42023.log 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55757.log.gz 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47144.log 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6%/var/log/ceph/ceph-client.admin.42023.log: -- replaced with /var/log/ceph/ceph-client.admin.73148.log.gz 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70099.log 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42023.log.gz 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.47144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64758.log 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47144.log.gz 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71720.log 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70099.log.gz 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41729.log 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71720.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64758.log.gz 2024-04-24T23:28:50.821 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71720.log.gz 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43510.log 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51827.log 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41729.log.gz 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70208.log 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51827.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43510.log.gz 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62892.log 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51827.log.gz 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54827.log 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62892.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62892.log.gz 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70208.log.gz 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41813.log 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62598.log 2024-04-24T23:28:50.822 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54827.log.gz 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40902.log 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62598.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41813.log.gz 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55463.log 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62598.log.gz 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44869.log 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40902.log: /var/log/ceph/ceph-client.0.55463.log: 0.0% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40902.log.gz 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.0.55463.log.gz 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66453.log 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52564.log 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44869.log.gz 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49662.log 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52564.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66453.log.gz 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58434.log 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52564.log.gz 2024-04-24T23:28:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54655.log 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58434.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49662.log.gz 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58434.log.gz 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65142.log 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73232.log 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65142.log: /var/log/ceph/ceph-client.admin.54655.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54655.log.gz 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44743.log 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65142.log.gz 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40325.log 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44743.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73232.log.gz 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73460.log 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44743.log.gz 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5/var/log/ceph/ceph-client.admin.40325.log: --verbose -- /var/log/ceph/ceph-client.admin.46243.log 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40325.log.gz 2024-04-24T23:28:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73460.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73460.log.gz 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.54188.log 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40283.log 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46243.log: /var/log/ceph/ceph-client.admin.54188.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46243.log.gz 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62850.log 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54188.log.gz/var/log/ceph/ceph-client.admin.40283.log: 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66107.log 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40283.log.gz 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59418.log 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62850.log: /var/log/ceph/ceph-client.admin.66107.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62850.log.gz 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66107.log.gz 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66495.log 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59016.log 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59418.log.gz/var/log/ceph/ceph-client.admin.66495.log: 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48709.log 2024-04-24T23:28:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59016.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66495.log.gz 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56074.log 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59016.log.gz 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69068.log 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56074.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48709.log.gz 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40029.log 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56074.log.gz 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61207.log 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40029.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40029.log.gz 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69068.log.gz 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60945.log 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61207.log: gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61207.log.gz 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53749.log 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48667.log 2024-04-24T23:28:50.826 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60945.log: /var/log/ceph/ceph-client.admin.53749.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60945.log.gz 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34825.log 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2%/var/log/ceph/ceph-client.admin.48667.log: -- replaced with /var/log/ceph/ceph-client.admin.53749.log.gz 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48667.log.gz 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47226.log 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34825.log: /var/log/ceph/ceph-client.admin.64467.log 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7%/var/log/ceph/ceph-client.admin.47226.log: -- replaced with /var/log/ceph/ceph-client.admin.34825.log.gz 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66663.log 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47226.log.gz 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64467.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.64467.log.gz 2024-04-24T23:28:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66663.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66663.log.gz 2024-04-24T23:28:50.829 INFO:teuthology.orchestra.run.smithi029.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-24T23:28:50.860 INFO:teuthology.orchestra.run.smithi029.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-24T23:28:50.863 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-24T23:28:50.867 INFO:teuthology.orchestra.run.smithi029.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-24T23:28:50.874 INFO:teuthology.orchestra.run.smithi029.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-24T23:28:50.876 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-04-24T23:28:50.876 INFO:teuthology.orchestra.run.smithi029.stderr:real 0m0.479s 2024-04-24T23:28:50.876 INFO:teuthology.orchestra.run.smithi029.stderr:user 0m2.739s 2024-04-24T23:28:50.876 INFO:teuthology.orchestra.run.smithi029.stderr:sys 0m0.100s 2024-04-24T23:28:50.897 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-24T23:28:50.919 INFO:teuthology.orchestra.run.smithi026.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-24T23:28:50.919 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-24T23:28:50.944 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-24T23:28:51.193 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-24T23:28:51.195 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-24T23:28:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m0.822s 2024-04-24T23:28:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:user 0m3.248s 2024-04-24T23:28:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m0.479s 2024-04-24T23:28:51.196 INFO:tasks.ceph:Archiving logs... 2024-04-24T23:28:51.196 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/remote/smithi026/log 2024-04-24T23:28:51.197 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:28:52.438 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/remote/smithi029/log 2024-04-24T23:28:52.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:28:52.844 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T23:28:52.853 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T23:28:52.854 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:28:52.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:28:52.954 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T23:28:52.964 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T23:28:52.964 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:28:52.967 DEBUG:teuthology.orchestra.run.smithi029:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:28:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:28:52.983 INFO:teuthology.orchestra.run.smithi029.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:28:52.985 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:28:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-04-24T23:28:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:^- hv01.front.sepia.ceph.com 2 6 377 59 +1064us[+1064us] +/- 566ms 2024-04-24T23:28:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 +101us[ +103us] +/- 68ms 2024-04-24T23:28:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -88us[ -88us] +/- 76ms 2024-04-24T23:28:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:28:52.990 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:28:52.990 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-04-24T23:28:52.990 INFO:teuthology.orchestra.run.smithi029.stdout:^- hv01.front.sepia.ceph.com 2 6 377 61 +1097us[+1112us] +/- 566ms 2024-04-24T23:28:52.990 INFO:teuthology.orchestra.run.smithi029.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 +114us[ +129us] +/- 68ms 2024-04-24T23:28:52.991 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -67us[ -67us] +/- 76ms 2024-04-24T23:28:52.991 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:28:52.991 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T23:28:53.059 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T23:28:53.060 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T23:28:53.070 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:28:53.158 DEBUG:teuthology.orchestra.run.smithi029:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:28:53.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:28:53.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:28:53.259 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:28:53.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:28:53.289 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T23:28:53.299 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T23:28:53.308 INFO:teuthology.task.internal:Duration was 864.404095 seconds 2024-04-24T23:28:53.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T23:28:53.317 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T23:28:53.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:28:53.320 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:28:53.358 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:28:53.370 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:28:53.648 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T23:28:53.648 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-04-24T23:28:53.648 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:28:53.711 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi029.front.sepia.ceph.com 2024-04-24T23:28:53.711 DEBUG:teuthology.orchestra.run.smithi029:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:28:53.734 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T23:28:53.734 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:28:53.754 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:28:53.801 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T23:28:53.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:28:53.821 DEBUG:teuthology.orchestra.run.smithi029:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:28:53.945 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T23:28:53.956 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T23:28:53.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:28:53.980 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:28:54.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T23:28:54.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:28:54.023 DEBUG:teuthology.orchestra.run.smithi029:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:28:54.043 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-04-24T23:28:54.067 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = core 2024-04-24T23:28:54.080 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:28:54.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:28:54.113 DEBUG:teuthology.orchestra.run.smithi029:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:28:54.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:28:54.135 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T23:28:54.146 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T23:28:54.147 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/remote/smithi026 2024-04-24T23:28:54.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:28:54.184 DEBUG:teuthology.misc:Transferring archived files from smithi029:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672190/remote/smithi029 2024-04-24T23:28:54.185 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:28:54.218 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T23:28:54.218 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:28:54.221 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:28:54.268 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T23:28:54.278 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T23:28:54.279 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T23:28:54.289 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T23:28:54.289 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:28:54.291 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:28:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:28 /home/ubuntu/cephtest 2024-04-24T23:28:54.322 INFO:teuthology.orchestra.run.smithi029.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:28 /home/ubuntu/cephtest 2024-04-24T23:28:54.323 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T23:28:54.335 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T23:28:54.363 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/multimds_misc} duration: 864.4040951728821 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-24T23:28:54.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:28:54.442 INFO:teuthology.run:pass